{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\n","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"The competion asks us to classify the breed of dogs present in images. There are total 120 breeds and more than 10000 samples for training and testing each. Let's explore what are the various underlying distributions in the data. "},{"metadata":{"trusted":true,"_uuid":"6436f302c700e89c7a39126dcb469d01535124af","_kg_hide-input":true},"cell_type":"code","source":"df = pd.read_csv(\"../input/labels.csv\")\ndf.info()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"78a0fbb63d7d4d8957f305122addbeee025ae934"},"cell_type":"markdown","source":"# Number of breeds present "},{"metadata":{"trusted":true,"_uuid":"ecf7f70ad6d9d1fe25ea6d9a3ca893bf84ac549c","_kg_hide-output":true,"_kg_hide-input":true},"cell_type":"code","source":"# let's see how many breeds(classes) are there\nprint(\"Breeds Present in CSV: \",len(df.breed.unique()))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"75f6019e1d4f0d22f70ce8a5bb80a2596e4b57c3"},"cell_type":"markdown","source":"It means that the given CSV has all he formentioned breeds. \n\n# Check the frequency of the classes(Breeds)\n\nlets see how much images are present in distribution belonging to the top 5 classes"},{"metadata":{"trusted":true,"_uuid":"3c2e61ec6ee8c20108e767ca26d21b51746f29c6","_kg_hide-input":true},"cell_type":"code","source":"each_label = df.groupby(\"breed\").count()\neach_label = each_label.rename(columns = {\"id\" : \"count\"})\neach_label = each_label.sort_values(\"count\", ascending=False)\neach_label.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6b16060f5d756e099e67f920726e34a4c8b0e663"},"cell_type":"markdown","source":"Number of images of bottom 5 classes\n"},{"metadata":{"trusted":true,"_uuid":"c9b5372463bab75d437176b37fc15bf22552295f","_kg_hide-input":true},"cell_type":"code","source":"each_label.tail()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2706bc9a060b7a2fbff0e82ef329a6c0ee33ae4c"},"cell_type":"markdown","source":"That's a big difference. The most frequent classes have almost double images as compared to the least frequent classes. That raises a question that if the decrese in the number of classes is gradual(continous) or in a step-wise manner?"},{"metadata":{"trusted":true,"_uuid":"0aa7dc0c0735728ac495b5c34237f1f8c2dff184","scrolled":false,"_kg_hide-input":true},"cell_type":"code","source":"ax=pd.value_counts(df['breed'],ascending=True).plot(kind='barh',\n                                                       fontsize=\"40\",\n                                                       title=\"Class Distribution\",\n                                                       figsize=(50,100))\nax.set(xlabel=\"Images per class\", ylabel=\"Classes\")\nax.xaxis.label.set_size(20)\nax.yaxis.label.set_size(20)\nax.title.set_size(30)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9bc5081772bb2e1c9a5d96511a92ec2ecb43029e"},"cell_type":"markdown","source":"From above distribution graph, we can observe that the reduction in the quantity of samples per class is continous. "},{"metadata":{"trusted":true,"_uuid":"1e85c8b54a4b06b98d2a119d2afd8438edf25929"},"cell_type":"markdown","source":"# Resolution\n\nResolution is one of the most important property of an image. Most of the CNNs downsample data from a higher resolution to a lower resolution. However, if we are making our own CNN, the resolution of input image is a choice. Let's see what is the distribution of resolition of images"},{"metadata":{"trusted":true,"_uuid":"8ac93794c078b97b2957afd9da0890c2dd9075aa","_kg_hide-input":true},"cell_type":"code","source":"import cv2\ntraining_images = os.listdir(\"../input/train/\")\nls = []\nfor i in training_images:\n    img = cv2.imread(\"../input/train/\"+str(i))\n    h,w,_ = img.shape\n    ls.append([h,w])\nls = np.array((ls),np.int)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b13785e3c4a6c43db0424152ce7735dd2f60cf8b","_kg_hide-input":true},"cell_type":"code","source":"def reject_outliers(data, m = 2.5):\n    d = np.abs(data - np.median(data))\n    mdev = np.median(d)\n    s = d/(mdev if mdev else 1.)\n    return data[s<m]\nh = reject_outliers(ls[:,0])\nw = reject_outliers(ls[:,1])\n\nmax_ht = h.max()\nmean_ht = h.mean()\nmin_ht = h.min()\n\nmax_wd = w.max()\nmean_wd = w.mean()\nmin_wd = w.min()\n\nprint(\"width: Max {}, Mean {}, Min {}\".format(max_wd,mean_wd,min_wd))\nprint(\"hight: Max {}, Mean {}, Min {}\".format(max_ht,mean_ht,min_ht))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8e2b1dc14be7490601762b5a11ebc61dfc380a3c","_kg_hide-input":true},"cell_type":"code","source":"import seaborn as sns\nimport matplotlib.pyplot as plt\nh = h.astype(np.int).tolist()\nh.sort()\nplt.figure(figsize=(12,5))\nplt.subplot(121)\nplt.xlabel(\"height\")\nsns.distplot(h)\n\nh = w.tolist()\nh.sort()\nplt.subplot(122)\nplt.xlabel(\"width\")\nsns.distplot(h)\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a028e21fb62469139aa9422e81c0dca063fba535"},"cell_type":"markdown","source":"It looks like the** width of image is stable at around 500 pixels while the height has a range from 250 pixels to 500 pixels baring the outliers** that we haven't taken into account.\nOne main cause of the spread-out distribution of height is the orientation. From distribution, we can vaguely conclude that most of the image has an aspect ratio of 1:2, i.e, most of them are in Landscape/Portrait mode. \n\n\nTODO: Check how many, if any, image contains more than two classes.(Doesn't matter for purposes of training. We are one hot encoding the entire column anyway)\n\nTODO: Check if there are other object in images such as human or cats.\n\nTODO: Check for attributes such as hair/color that are unique to particular classes.\n\n"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}