{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Introduction\nThis project will depend on traditional computer vision(CV) techniques for classification tasks. Traditional CV goes through the following steps:-<br>\n* Key points extraction using SIFT, SURF, ORB, BRIEF, FAST, or HOG\n* Calculating the descriptors for those key points\n* Clustering those descriptors into k-clusters\n* Create a histogram for each image with k-bins and then quantize those key points according to the clusters that they are closest to, we can use different similarity measures, like, L2-distance, L1-distance, or cosine and so on.\n* Then use those histogram vectors for the machine learning step, we can use SVM, logistic regression or trees for the classification step","metadata":{}},{"cell_type":"code","source":"#To ignore Sklearn warnings\ndef warn(*args, **kwargs):\n    pass\nimport warnings\nwarnings.warn = warn\n\nimport os\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport zipfile\nimport cv2\n\nfrom matplotlib.gridspec import GridSpec\nimport matplotlib.pyplot as plt\n\nfrom sklearn.linear_model import LogisticRegression, SGDClassifier\nfrom sklearn.metrics import classification_report, log_loss, confusion_matrix\nfrom sklearn.preprocessing import PolynomialFeatures\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-05-21T20:25:11.840861Z","iopub.execute_input":"2021-05-21T20:25:11.841401Z","iopub.status.idle":"2021-05-21T20:25:13.328854Z","shell.execute_reply.started":"2021-05-21T20:25:11.841219Z","shell.execute_reply":"2021-05-21T20:25:13.327791Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.rcParams[\"figure.figsize\"] = (10, 10)","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:25:13.330389Z","iopub.execute_input":"2021-05-21T20:25:13.330714Z","iopub.status.idle":"2021-05-21T20:25:13.335281Z","shell.execute_reply.started":"2021-05-21T20:25:13.330666Z","shell.execute_reply":"2021-05-21T20:25:13.334134Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Unzip The Training and Test Set","metadata":{}},{"cell_type":"code","source":"with zipfile.ZipFile(\"../input/dogs-vs-cats-redux-kernels-edition/\" + \"train.zip\",\"r\") as z:\n    z.extractall(\".\")\nwith zipfile.ZipFile(\"../input/dogs-vs-cats-redux-kernels-edition/\" + \"test.zip\",\"r\") as z:\n    z.extractall(\".\")\n","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:25:15.524006Z","iopub.execute_input":"2021-05-21T20:25:15.524372Z","iopub.status.idle":"2021-05-21T20:25:37.567096Z","shell.execute_reply.started":"2021-05-21T20:25:15.524342Z","shell.execute_reply":"2021-05-21T20:25:37.565921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Loading and Transforming Images\nI chose the SIFT extractor to get a descriptor vector of size 128, and I chose SIFT because it give better descriptors than ORB, SURF and BRIEF. But it isn't as good as the features that are generated by HOG. Also, I will utilize the bag of visual words concept (BoVW) to describe the features in which instead of depending on the 128 vector for classification, I will rely on the frequency of occurrence of each cluster in an image. Hence, I will generate k-clusters based on the k-means algorithm then will generate the vocabulary list that is going to be used by BOWImgDescriptorExtractor. But my implementation for this project necessitated that I can't rely on BOWImgDescriptorExtractor for descriptor extraction because I can't store the whole of the dataset on RAM, hence, I can't generate the ideal clusters. Hence, I implemented my own L2- norm distance function that calculate the distance from the descriptors to the clusters in which this will eventually participate in constructing my histogram for the number of occurrence of each cluster in an image. Hence, my feature vector that is used for classification is the histogram of the BoVW in which its dimension is equal to the number of clusters.","metadata":{"execution":{"iopub.status.busy":"2021-05-21T16:20:14.855163Z","iopub.execute_input":"2021-05-21T16:20:14.855644Z","iopub.status.idle":"2021-05-21T16:20:18.934832Z","shell.execute_reply.started":"2021-05-21T16:20:14.855607Z","shell.execute_reply":"2021-05-21T16:20:18.932217Z"}}},{"cell_type":"code","source":"class LoadingImages(object):\n\n\n    def __init__(self, trainingSize, numClusters=10, nfeatures=10):\n        self.counts = {\n            \"cat\": 0,\n            \"dog\": 0,\n            \"test\": 0\n        }\n        #Use SIFT descriptors\n        self.sift = cv2.SIFT_create(nfeatures, 3, 0.0, 10, 1.6, None)\n        #Use FLANN as your matcher between the BOW for each cluster in the 128 parameters space.\n        FLANN_INDEX_KDTREE = 1 \n        flann_params = {\"algorithm\": FLANN_INDEX_KDTREE, \"trees\": 5}\n        matcher = cv2.FlannBasedMatcher(flann_params, {})\n        #Compute image descriptor using bag of word method\n        self.bowExtractor = cv2.BOWImgDescriptorExtractor(self.sift, matcher)\n        self.bowKmeansClusters = cv2.BOWKMeansTrainer(numClusters)\n        self.nfeatures  = nfeatures\n        self.numClusters = numClusters\n        self.trainingSize = trainingSize\n        self.indeces = list(np.random.choice(self.trainingSize, self.trainingSize, replace=False))\n\n    def resetCounters(self):\n        self.counts = {\n            \"cat\": 0,\n            \"dog\": 0,\n            \"test\": 0\n        }\n\n    def loadData(self, numImages, windowSize, countType, p='', trainOrTest=\"train\", showMe=False, start=0):\n        images = []\n\n        for c in range(start, numImages):\n\n            try:\n                if countType == \"test\":\n                    ind = self.counts[\"test\"] + 1\n                else:\n                    ind = self.indeces[self.counts[countType]]\n                img = cv2.imread(f\"/kaggle/working/{trainOrTest}/\" + p + str(ind) + \".jpg\", cv2.IMREAD_GRAYSCALE)\n\n                #histogram equalization to remove skewness in the intensity\n                img_center = np.zeros((windowSize, windowSize), dtype=np.ubyte)\n                #print(img.shape)\n\n                # Using strides\n                # print(img.shape)\n                # print(img_center.shape)\n                numStrides = [int(np.ceil(img.shape[0]/windowSize)), int(np.ceil(img.shape[1]/windowSize ))]\n                numStrides[0] = 2 if numStrides[0] <= 1 else numStrides[0]\n                numStrides[1] = 2 if numStrides[1] <= 1 else numStrides[1]\n                #print(numStrides)\n                img = img[::numStrides[0], ::numStrides[1]]\n                img_center[:img.shape[0], :img.shape[1]] = img[:, :]\n\n                #Getting the middle part of the image\n                # center = (int(img.shape[0]/2), int(img.shape[1]/2))\n                # #Get a centered Verision of the image\n                # ptLeftX = center[0] - int(windowSize/2) if center[0] - int(windowSize/2) > 0 else 0\n                # ptLeftY = center[1] - int(windowSize/2) if center[1] - int(windowSize/2) > 0 else 0\n\n                # ptRightX = center[0] + int(windowSize/2) if center[0] + int(windowSize/2) < img.shape[0] else img.shape[0]\n                # ptRightY = center[1] + int(windowSize/2) if center[1] + int(windowSize/2) < img.shape[1] else img.shape[1]\n\n                # img_center = img[ptLeftX:ptRightX, ptLeftY:ptRightY]\n                \n                \n                img_center = cv2.equalizeHist(img_center)\n                img_center = np.asarray(img_center, np.float32)\n                means, stddev = cv2.meanStdDev(img_center)\n                img_center -= means\n                stddev = stddev if stddev != 0 else 1\n                img_center /= stddev\n\n                if showMe:\n                    cv2.imshow(p + str(ind), img_center)\n                    if cv2.waitKey(0) & 0xFF == ord('q'):\n                        cv2.destroyAllWindows()\n                        break\n                    cv2.destroyAllWindows()\n                #print(np.mean(img_center))\n                #print(np.std(img_center))\n                #print(img_center.shape)\n                images.append(img_center)\n                self.counts[countType] += 1\n\n\n            except Exception as e:\n                #Restart your image loading\n                print(e)\n                print(self.counts[countType])\n                self.counts[countType] = start\n                #images = []\n                break\n        return images\n\n    def getDescriptors(self, images, labels, visualize=False, kps=[]):\n        descriptors = []\n        counter = 0\n        for img in images:\n            img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX, cv2.CV_8UC1)\n            kp = self.sift.detect(img)\n            kps.append(kp)\n            des = np.zeros((self.nfeatures, 128))\n            if visualize:\n                channels = []\n                colored = None\n                channels.append(img)\n                channels.append(img)\n                channels.append(img)\n                colored = cv2.merge(channels)\n                colored = cv2.drawKeypoints(img, kp, colored, flags=cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)\n                cv2.imshow(str(counter), colored)\n                if cv2.waitKey(0) & 0xFF == ord('q'):\n                    cv2.destroyAllWindows()\n                    break\n                cv2.destroyAllWindows()\n            try:\n                #tmp = np.ravel(self.sift.compute(img, kp)[-1][0:self.nfeatures]).reshape(1, -1)\n                tmp = self.sift.compute(img, kp)[-1]\n                des[:len(tmp)] = tmp  \n            except:\n                pass\n\n            #Make feature vector 1x(128 * #kp)    \n            descriptors.append(des)\n            counter += 1\n        #Return #images x #features x 128, #imagesx x 1\n\n        return descriptors, labels\n\n    def getBOW(self, descriptors, labels, trainFlag=False):\n        #add our data to cluster the visual vocabularly on\n        #labels_extends = []\n        if trainFlag:\n            #nfeaturex128\n            for arr in descriptors:\n                #print(arr.shape)\n                self.bowKmeansClusters.add(arr)#This will keep previous data unless we clear the data structure\n            vocuabulary = self.bowKmeansClusters.cluster()\n            #print(vocuabulary)\n            #After finding vocabulary for each cluster, we need to set the BOW for those clusters\n            self.bowExtractor.setVocabulary(vocuabulary)\n        \n        #print(self.bowExtractor.getVocabulary())\n        vocuabulary = self.bowExtractor.getVocabulary()#numClusters x 128\n        hists = []\n        counter = 0\n        for arr in descriptors:\n            hist = np.zeros((self.numClusters, 1))#histogram for each image with respect to the clusters.\n            for row in arr:\n                #finding nearest cluster to construct visual word or codeblock\n                i = np.argmin(np.sqrt(np.sum(np.power(vocuabulary - row.reshape(1, -1), 2), axis=1)), axis=0)\n                hist[i, 0] += 1\n            hists.extend(hist.reshape(1, -1))\n            #labels_extends.extend([labels[counter]] * self.nfeatures)\n            counter += 1\n        return np.array(hists, np.float32), np.array(labels).reshape(-1, 1)\n\n\n    def loadDescriptorsAndLabels(self, dataSize, windowSize=200, visualize=False):\n        #Get 2 * dataSize of images\n        labels = [0] * dataSize\n        labels.extend([1] * dataSize)\n        images = self.loadData(dataSize, windowSize, \"cat\", \"cat.\")   \n        images.extend(self.loadData(dataSize, windowSize, \"dog\", \"dog.\"))\n\n        des, labels = self.getDescriptors(images, labels, visualize)\n        return np.array(des, np.float32) , labels\n\n    def loadDataRandomly(self, numImages, windowSize, showMe=False):\n        images = []\n        labels = []\n        indeces = np.random.choice(12500, numImages, replace=False)\n        typeAnimal = np.random.choice(2, numImages)\n        lab = {0: \"cat.\", 1:\"dog.\"}\n        for ind in range(0, len(indeces)):\n\n            try:\n                img = cv2.imread(f\"/kaggle/working/train/\" + lab[typeAnimal[ind]] + str(indeces[ind]) + \".jpg\", cv2.IMREAD_GRAYSCALE)\n\n                #histogram equalization to remove skewness in the intensity\n                img_center = np.zeros((windowSize, windowSize), dtype=np.ubyte)\n                numStrides = [int(np.ceil(img.shape[0]/windowSize)), int(np.ceil(img.shape[1]/windowSize ))]\n                numStrides[0] = 2 if numStrides[0] <= 1 else numStrides[0]\n                numStrides[1] = 2 if numStrides[1] <= 1 else numStrides[1]\n                #print(numStrides)\n                img = img[::numStrides[0], ::numStrides[1]]\n                img_center[:img.shape[0], :img.shape[1]] = img[:, :]\n                img_center = cv2.equalizeHist(img_center)\n                img_center = np.asarray(img_center, np.float32)\n                means, stddev = cv2.meanStdDev(img_center)\n                img_center -= means\n                stddev = stddev if stddev != 0 else 1\n                img_center /= stddev\n\n                images.append(img_center)\n                labels.append(typeAnimal[ind])\n\n            except Exception as e:\n                print(e)\n\n        return images, labels","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:25:45.936507Z","iopub.execute_input":"2021-05-21T20:25:45.937918Z","iopub.status.idle":"2021-05-21T20:25:45.993908Z","shell.execute_reply.started":"2021-05-21T20:25:45.937865Z","shell.execute_reply":"2021-05-21T20:25:45.992151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualizing the Images with the Features Descriptions\nNotice, that the images have a uniform shape in which I used the same shape for each image. But instead of taking the middle part of the image, I relied upon subsampling the image by striding within the image. Also, I utilized histogram equalization in order to minimize noise caused by non-uniform distribution of the intensity in the image.","metadata":{}},{"cell_type":"code","source":"obj = LoadingImages(10, 64, 256)\ncats = obj.loadData(10, 200, \"cat\", \"cat.\", \"train\", False)\n\ngs = GridSpec(2, 5)\nfor row in range(0, 2):\n    for col in range(0, 5):\n        axes = plt.subplot(gs[row, col])\n        axes.imshow(cats[row*2 + col], cmap=\"gray\")\n        axes.set_xticks([])\n        axes.set_yticks([])\n        axes.set_title(\"cat\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:25:50.224557Z","iopub.execute_input":"2021-05-21T20:25:50.224941Z","iopub.status.idle":"2021-05-21T20:25:51.088398Z","shell.execute_reply.started":"2021-05-21T20:25:50.224905Z","shell.execute_reply":"2021-05-21T20:25:51.086742Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"obj = LoadingImages(10, 64, 256)\ndogs = obj.loadData(10, 200, \"dog\", \"dog.\", \"train\", False)\n\ngs = GridSpec(2, 5)\nfor row in range(0, 2):\n    for col in range(0, 5):\n        axes = plt.subplot(gs[row, col])\n        axes.imshow(dogs[row*2 + col], cmap=\"gray\")\n        axes.set_xticks([])\n        axes.set_yticks([])\n        axes.set_title(\"cat\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:25:51.090762Z","iopub.execute_input":"2021-05-21T20:25:51.091072Z","iopub.status.idle":"2021-05-21T20:25:51.807324Z","shell.execute_reply.started":"2021-05-21T20:25:51.091043Z","shell.execute_reply":"2021-05-21T20:25:51.806058Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"kps = []\ndes, labels = obj.getDescriptors(cats, [0]*len(cats), False, kps)\ncolored_imgs = []\nfor img, kp in zip(cats, kps):\n    channels = []\n    colored = None\n    img = cv2.normalize(img, None, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8UC3)\n    colored = cv2.drawKeypoints(img, kp, img, flags=cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)\n    colored_imgs.append(cv2.cvtColor(colored, cv2.COLOR_BGR2RGB))\n    \ngs = GridSpec(2, 5)\nfor row in range(0, 2):\n    for col in range(0, 5):\n        axes = plt.subplot(gs[row, col])\n        axes.imshow(colored_imgs[row*2 + col], cmap=\"gray\")\n        axes.set_xticks([])\n        axes.set_yticks([])\n        axes.set_title(\"cat\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:25:51.809097Z","iopub.execute_input":"2021-05-21T20:25:51.809474Z","iopub.status.idle":"2021-05-21T20:25:52.988597Z","shell.execute_reply.started":"2021-05-21T20:25:51.809419Z","shell.execute_reply":"2021-05-21T20:25:52.987314Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"kps = []\ndes, labels = obj.getDescriptors(cats, [0]*len(dogs), False, kps)\ncolored_imgs = []\nfor img, kp in zip(dogs, kps):\n    channels = []\n    colored = None\n    img = cv2.normalize(img, None, alpha=0, beta=255, norm_type=cv2.NORM_MINMAX, dtype=cv2.CV_8UC3)\n    colored = cv2.drawKeypoints(img, kp, img, flags=cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)\n    colored_imgs.append(cv2.cvtColor(colored, cv2.COLOR_BGR2RGB))\n    \ngs = GridSpec(2, 5)\nfor row in range(0, 2):\n    for col in range(0, 5):\n        axes = plt.subplot(gs[row, col])\n        axes.imshow(colored_imgs[row*2 + col], cmap=\"gray\")\n        axes.set_xticks([])\n        axes.set_yticks([])\n        axes.set_title(\"dog\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:25:52.99063Z","iopub.execute_input":"2021-05-21T20:25:52.99126Z","iopub.status.idle":"2021-05-21T20:25:53.784992Z","shell.execute_reply.started":"2021-05-21T20:25:52.991196Z","shell.execute_reply":"2021-05-21T20:25:53.783765Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Utility methods For Training and Predicitions","metadata":{}},{"cell_type":"code","source":"def train(obj, training_errors, validation_errors, epochs, batch_size, valid_size, batchs_num, validation_size, train_size, model_type, windowSize, decisionValue, noiseUpdate=0.05, nu=0.9, gamma=0.005, kernel=\"rbf\", C=0.8):\n    # cat = 0, dog = 1\n    np.random.seed(42)\n    if model_type == \"svm\":\n        model = NuSVC(kernel=kernel, nu=nu, gamma=gamma)\n        modelTemp = NuSVC(kernel=kernel, nu=nu, gamma=gamma)\n    else:\n        model = LogisticRegression(solver='sag', penalty=\"l2\", C=C, warm_start=True, max_iter=400, n_jobs=2)#default parameters\n        modelTemp = LogisticRegression(solver='sag', penalty=\"l2\", C=C, warm_start=True, max_iter=400, n_jobs=2)#default parameters\n    counter = 0\n    prevError = np.finfo(np.float32).min\n\n    for epoch in range(0, epochs):\n        training_error_avg = []\n        #Clear clusters at each epoch.\n        obj.resetCounters()\n        #obj.bowKmeansClusters.clear()\n        #obj.bowExtractor.setVocabulary(np.array([]))\n        for batch in range(0, batchs_num):\n            #transform = PolynomialFeatures(degrees, interaction_only=True)\n            des, labels = obj.loadDescriptorsAndLabels(int(batch_size/2), windowSize) #batch_sizex128\n            des, labels = obj.getBOW(des, labels, False)\n            #des = transform.fit_transform(des)\n            indeces = np.arange(len(des))\n            np.random.shuffle(indeces)\n            if modelType == \"svm\":\n                modelTemp.fit(des[indeces], labels[indeces]);\n                y_pred = modelTemp.predict(des[indeces])\n            else:\n                modelTemp.fit(des[indeces], labels[indeces]); \n                #y_pred = model.predict(des)\n                y_pred = list(map(lambda v: 0 if v[0] > decisionValue else 1, modelTemp.predict_proba(des[indeces])))\n            try:\n                cnf = confusion_matrix(y_pred, labels[indeces])\n                training_error_avg.append(np.sum(np.diag(cnf))/np.sum(cnf))\n                #training_error_avg.append(log_loss(labels[indeces], y_pred))\n                print(f\"{epoch}-{batch} the accuracy is {np.sum(np.diag(cnf))/np.sum(cnf)}\")\n                print(cnf)\n\n            except Exception as e:\n                print(e)\n                pass\n\n            if training_error_avg[-1] + np.random.normal(0) * noiseUpdate > prevError :\n                print(\"Update model\")\n                model = modelTemp\n                prevError = training_error_avg[-1]\n            else:\n                print(\"Previous model is better\")\n                counter += 1\n\n            if counter > 15:\n                print(f\"No update for the parameter for {counter} passes\")\n                break\n\n        if batchs_num * batch_size < train_size - 1 and counter <= 15:\n        # transform = PolynomialFeatures(degrees, interaction_only=True)\n            des, labels = obj.loadDescriptorsAndLabels(int(train_size - batchs_num * batch_size), windowSize)\n            des, labels = obj.getBOW(des, labels, False)\n            indeces = np.arange(len(des))\n            np.random.shuffle(indeces)\n            if modelType == \"svm\":    \n                #des = transform.fit_transform(des)\n                modelTemp.fit(des[indeces], labels[indeces]);\n                y_pred = modelTemp.predict(des[indeces]) \n            else:\n                modelTemp.fit(des[indeces], labels[indeces]); \n                y_pred = list(map(lambda v: 0 if v[0] > decisionValue else 1, modelTemp.predict_proba(des[indeces])))\n                \n            try:\n                cnf = confusion_matrix(y_pred, labels[indeces])\n                training_error_avg.append(np.sum(np.diag(cnf))/np.sum(cnf))\n                print(f\"Last accuracy is {np.sum(np.diag(cnf))/np.sum(cnf)}\") \n                print(cnf)\n\n            except Exception as e:\n                    print(e) \n                    pass\n\n            if training_error_avg[-1] > prevError:\n                print(\"Update support Vectors\")\n                #model = modelTemp\n                prevError = training_error_avg[-1]\n        training_errors.append(np.mean(training_error_avg))\n\n        #For validation set\n        labels = []\n        y_pred = []\n        for valid_batch in range(0, int(validation_size/valid_size)):\n            des, labels_r = obj.loadDescriptorsAndLabels(valid_size, windowSize) #numb_key_ptx128\n            des, labels_r = obj.getBOW(des, labels_r, False)\n            #transform = PolynomialFeatures(degrees, interaction_only=True)\n            #des = transform.fit_transform(des[:])\n            labels.extend(labels_r)\n            if modelType == \"logistic\":\n                y_pred_r = list(map(lambda v: 0 if v[0] > decisionValue else 1, model.predict_proba(des)))\n            else:\n                y_pred_r = model.predict(des)\n\n            y_pred.extend(y_pred_r)\n        if len(y_pred) < validation_size:\n            des, labels_r = obj.loadDescriptorsAndLabels(valid_size, windowSize) #numb_key_ptx128\n            des, labels_r = obj.getBOW(des, labels_r, False)\n            #transform = PolynomialFeatures(degrees, interaction_only=True)\n            #des = transform.fit_transform(des[:])\n            labels.extend(labels_r)\n            if modelType == \"logistic\":\n                y_pred_r = list(map(lambda v: 0 if v[0] > decisionValue else 1, model.predict_proba(des)))\n            else:\n                y_pred_r = model.predict(des)\n            y_pred.extend(y_pred_r)\n        try:\n            cnf = confusion_matrix(y_pred, labels)\n            validation_errors.append(np.sum(np.diag(cnf))/np.sum(cnf))\n            print(cnf)\n            print(f\"{epoch} the validation accuracy is {np.sum(np.diag(cnf))/np.sum(cnf)}\") \n\n        except Exception as e:\n            print(e)\n            pass\n\n        if epoch%2 == 0:\n            print(f\"#epoch: {epoch} and the accuracy training is {training_errors[epoch]}\")    \n            print(f\"#epoch: {epoch} and the accuracy validation is {validation_errors[epoch]}\")\n\n\n    return model, training_errors, validation_errors\n\ndef prediction(obj, model, modelType, decisionValue, numImages, windowSize, datasetType, trainOrValid=True, visualize=False):\n    batch_size = 1024\n    numBatches = int(numImages/batch_size)\n    if datasetType == \"train\" and not trainOrValid:\n        obj.counts[\"cat\"] = train_size\n        obj.counts[\"dog\"] = train_size\n    else:\n        obj.resetCounters()\n\n    labels = []\n    y_pred = []\n    counter = 1 \n    print(obj.counts)\n    for batch in range(0, numBatches):\n        if datasetType == \"train\":\n            des, labels_r = obj.loadDescriptorsAndLabels(batch_size, windowSize) #numb_key_ptx128\n            des, labels_r = obj.getBOW(des, labels_r, False)\n        else:\n            images = obj.loadData(batch_size, windowSize, \"test\", \"\", \"test\", False, 0) #batch_sizex128\n            des, labels_r = obj.getDescriptors(images, [-1]*batch_size, False)\n            des, labels_r = obj.getBOW(des, [-1]*batch_size, False)\n        labels.extend(labels_r)\n        if modelType == \"logistic\":\n            y_pred_r = list(map(lambda v: 0 if v[0] > decisionValue else 1, model.predict_proba(des)))\n        else:\n            y_pred_r = model.predict(des)\n        y_pred.extend(y_pred_r)\n        if visualize:\n            for img in images:\n                img = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)\n                cv2.putText(img, 'cat' if y_pred_r[counter] == 0 else 'dog', (10, 180), cv2.FONT_HERSHEY_SIMPLEX, 2, (0, 0, 255))\n                cv2.imshow(\"Test_image\" + str(counter), img)\n                counter += 1\n                if cv2.waitKey(0) & 0xFF == ord('q'):\n                    cv2.destroyAllWindows()\n                    break\n        print(f\"Batch number {batch}, number of processed data {(batch + 1) * batch_size}\")\n    if len(y_pred) < numImages:\n        if datasetType == \"train\":\n            des, labels_r = obj.loadDescriptorsAndLabels(numImages - len(y_pred), windowSize) #numb_key_ptx128\n            des, labels_r = obj.getBOW(des, labels_r, False)\n        else:\n            images = obj.loadData(numImages - len(y_pred), windowSize, \"test\", \"\", \"test\", False, 0) #batch_sizex128\n            des, labels_r = obj.getDescriptors(images, [-1]*batch_size, False)\n            des, labels_r = obj.getBOW(des, [-1]*batch_size, False)\n        if modelType == \"logistic\":\n            y_pred_r = list(map(lambda v: 0 if v[0] > decisionValue else 1, model.predict_proba(des)))\n        else:\n            y_pred_r = model.predict(des)\n        y_pred.extend(y_pred_r)\n        \n    return labels, y_pred","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:26:30.450286Z","iopub.execute_input":"2021-05-21T20:26:30.45071Z","iopub.status.idle":"2021-05-21T20:26:30.498314Z","shell.execute_reply.started":"2021-05-21T20:26:30.450652Z","shell.execute_reply":"2021-05-21T20:26:30.497277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Setting up the unique Clusters","metadata":{}},{"cell_type":"code","source":"train_validation_ratio = 0.85\ntrainingSize = 12500\ntrain_size = int(trainingSize * train_validation_ratio)\nvalidation_size = int(trainingSize * (1 - train_validation_ratio))\nwindowSize = 200\nnfeatures = 256\nnumClusters = 64\n\nobj = LoadingImages(trainingSize, numClusters, nfeatures)#30 clusters within an image\n\n# Setting clusters early on\n#numObservations = numClusters * 100\nnumObservations = 2048\n\nimages, labels = obj.loadDataRandomly(numObservations, windowSize)\ndes, labels = obj.getDescriptors(images, labels)\ndes, labels = obj.getBOW(np.array(des, np.float32), labels, True)\n\nprint(f\"#clusters is {obj.bowExtractor.descriptorSize()}\")","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:26:31.662036Z","iopub.execute_input":"2021-05-21T20:26:31.662412Z","iopub.status.idle":"2021-05-21T20:31:16.338287Z","shell.execute_reply.started":"2021-05-21T20:26:31.66238Z","shell.execute_reply":"2021-05-21T20:31:16.337155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Setting up The Model","metadata":{}},{"cell_type":"code","source":"modelType = \"logistic\"\ntraining_errors = []\nvalidation_errors = []\nbatch_size = 2048#For svm\nbatch_size = 512\nbatchs_num = int(train_size/batch_size)\nvalid_size = 256\nepochs = 1\ndecisionValue = 0.52\n\nmodel, training_errors, validation_errors = train(obj, training_errors, validation_errors, epochs, batch_size, \n                                                  valid_size, batchs_num, validation_size, train_size, modelType,\n                                                  windowSize, decisionValue)","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:32:28.62365Z","iopub.execute_input":"2021-05-21T20:32:28.624094Z","iopub.status.idle":"2021-05-21T20:48:52.98966Z","shell.execute_reply.started":"2021-05-21T20:32:28.624052Z","shell.execute_reply":"2021-05-21T20:48:52.988273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Performance on the training set","metadata":{}},{"cell_type":"code","source":"labels, y_pred = prediction(obj, model, modelType, decisionValue, train_size, windowSize, \"train\", True)\n\ncnf = confusion_matrix(y_pred, labels)\nprint(cnf)\nprint(f\"Accuracy: {np.sum(np.diag(cnf))/np.sum(cnf)}\")","metadata":{"execution":{"iopub.status.busy":"2021-05-21T18:56:18.929668Z","iopub.execute_input":"2021-05-21T18:56:18.930343Z","iopub.status.idle":"2021-05-21T18:56:50.266752Z","shell.execute_reply.started":"2021-05-21T18:56:18.930285Z","shell.execute_reply":"2021-05-21T18:56:50.263875Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Performance on the validation set","metadata":{}},{"cell_type":"code","source":"labels, y_pred = prediction(obj, model, modelType, decisionValue, trainingSize - train_size, windowSize, \"train\", False)\n\ncnf = confusion_matrix(y_pred, labels)\nprint(cnf)\nprint(f\"Accuracy: {np.sum(np.diag(cnf))/np.sum(cnf)}\")","metadata":{"execution":{"iopub.status.busy":"2021-05-21T18:56:50.267828Z","iopub.status.idle":"2021-05-21T18:56:50.268281Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Performance on the test set","metadata":{}},{"cell_type":"code","source":"labels, y_pred = prediction(obj, model, modelType, decisionValue, 12500, windowSize, \"test\", True, False)","metadata":{"execution":{"iopub.status.busy":"2021-05-21T20:48:52.992251Z","iopub.execute_input":"2021-05-21T20:48:52.993004Z","iopub.status.idle":"2021-05-21T20:59:34.633467Z","shell.execute_reply.started":"2021-05-21T20:48:52.992937Z","shell.execute_reply":"2021-05-21T20:59:34.632266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results = pd.DataFrame(np.c_[list(range(1, len(y_pred) + 1)), y_pred], columns=[\"id\", \"label\"])\nresults.to_csv(\"submission.csv\", index=False)\nresults.head()","metadata":{"execution":{"iopub.status.busy":"2021-05-21T21:05:54.038953Z","iopub.execute_input":"2021-05-21T21:05:54.03932Z","iopub.status.idle":"2021-05-21T21:05:54.085079Z","shell.execute_reply.started":"2021-05-21T21:05:54.03929Z","shell.execute_reply":"2021-05-21T21:05:54.084033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# To download file, from https://www.kaggle.com/rtatman/download-a-csv-file-from-a-kernel\n# from IPython.display import HTML\n# import base64\n# html = '<a  href=\"{filename}\" target=\"_blank\">{title}</a>'\n# html = html.format(title=\"file\",filename=\"submission.csv\")\n# HTML(html)","metadata":{"execution":{"iopub.status.busy":"2021-05-21T21:09:33.707745Z","iopub.execute_input":"2021-05-21T21:09:33.708371Z","iopub.status.idle":"2021-05-21T21:09:33.716647Z","shell.execute_reply.started":"2021-05-21T21:09:33.708319Z","shell.execute_reply":"2021-05-21T21:09:33.715034Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Suggested Improvements\n* We can use HOG instead of SIFT\n* We can still rely on SIFT but instead of using BoVW, we can utilize the key descriptors themselves to construct the feature vector, but this will exponentially increase the number of parameters based on the number of features that you specify.\n* Choosing better regularization values, increase the number of epoch, and it is always recommended to use ensemble of models for the prediction step\n* As you can see from the accuracy for both validation and test set that deep learning is way better than traditional CV when you have a large dataset. But remember that this model had only 64 parameters(i.e the number of clusters), and this is significantly less than the millions of parameters that you see in CNN.","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}