{"cells":[{"metadata":{},"cell_type":"markdown","source":"This notebook is basically my attempt to introduce myself to the dataset a little bit by getting the full dog image dataset and the label (annotation) for every image. This in turn will allow me to analyze the class distributions for each dog breed, as well plot the dog images.\n\nFinally, since there are images with multiple dogs in them, I will attempt to separate each dog (with cropping) using the provided annotations that contain the coordinates of each object."},{"metadata":{},"cell_type":"markdown","source":"## References"},{"metadata":{},"cell_type":"markdown","source":"1. Xml parsing and cropping to specified bounding box - (https://www.kaggle.com/paulorzp/show-annotations-and-breeds)\n\n2. Also thanks to K.Amano for his cropping method with interpolation - (https://www.kaggle.com/amanooo/wgan-gp-keras)"},{"metadata":{},"cell_type":"markdown","source":"## Importing libraries"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport os\nimport glob\nimport math\nimport random\nimport time\nimport datetime\nfrom collections import defaultdict\nfrom tqdm import tqdm, tqdm_notebook\n\nimport xml.etree.ElementTree as ET \n\nimport cv2\n\nprint(os.listdir(\"../input\"))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Setting some constants according to the rules"},{"metadata":{"trusted":true},"cell_type":"code","source":"image_width = 64\nimage_height = 64\nimage_channels = 3\nimage_sample_size = 10000\nimage_output_dir = '../output_images/'\nimage_input_dir = '../input/all-dogs/all-dogs/'\nimage_ann_dir = \"../input/annotation/Annotation/\"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Storing the dog breed type in a dictionary by mapping it to its code in the annotations"},{"metadata":{},"cell_type":"markdown","source":"Here I've created the dictionary ```dog_breed_dict```, which will map each breed code to its original name."},{"metadata":{"trusted":true},"cell_type":"code","source":"dog_breed_dict = {}\nfor annotation in os.listdir(image_ann_dir):\n    annotations = annotation.split('-')\n    dog_breed_dict[annotations[0]] = annotations[1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(dog_breed_dict['n02097658'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Creating another dictionary (of lists), in order to map each input filename to a specific dog breed"},{"metadata":{},"cell_type":"markdown","source":"The following helper-function will allow to create another similar dictionary but for all of the input images. Since each dog breed can be found multiple times in the dataset, the value part here will be stored as a list."},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_input_image_dict(image_input_dir, labels_dict):\n    image_sample_dict = defaultdict(list)\n    for image in os.listdir(image_input_dir):\n        filename = image.split('.')\n        label_code = filename[0].split('_')[0]\n        breed_name = labels_dict[label_code]\n        #print('Code: {}, Breed: {}'.format(label_code, breed_name))\n        if image is not None:\n            image_sample_dict[breed_name].append(image)\n    \n    print('Created label dictionary for input images.')\n    return image_sample_dict","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"image_sample_dict = get_input_image_dict(image_input_dir, dog_breed_dict)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Plotting the class distributions of the input images by dog breed"},{"metadata":{},"cell_type":"markdown","source":"After we have this information, we can do some EDA. Firstly, we can print the total amount of dog breeds in the dataset and we can also count the total input images using the created ```image_sample_dict```. Secondly, we can plot the class distributions of each breed."},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_class_distributions(image_sample_dict, title=''):\n    class_lengths = []\n    labels = []\n    total_images = 0\n    \n    print('Total amount of dog breeds: ', len(image_sample_dict))\n    \n    for label, _ in image_sample_dict.items():\n        total_images += len(image_sample_dict[label])\n        class_lengths.append(len(image_sample_dict[label]))\n        labels.append(label)\n        \n    print('Total amount of input images: ', total_images)\n        \n    plt.figure(figsize = (10,30))\n    plt.barh(range(len(class_lengths)), class_lengths)\n    plt.yticks(range(len(labels)), labels)\n    plt.title(title)\n    plt.ylabel('Dog Breed')\n    plt.xlabel('Sample size')\n    plt.show()\n    \n    return total_images","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"total_images = plot_class_distributions(image_sample_dict)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Visualizing the images"},{"metadata":{},"cell_type":"markdown","source":"Using the <b>OpenCV</b> imaging library, we can load each image and create an example set of features."},{"metadata":{"trusted":true},"cell_type":"code","source":"def read_image(src):\n    img = cv2.imread(src)\n    if img is None:\n        raise FileNotFoundError\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    return img","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Using the previously created ```image_sample_dict``` we can plot a matrix of images to see what the dogs look like, along with their actual breed name."},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_images(directory, image_sample_dict, examples=25, disp_labels=True): \n  \n    if not math.sqrt(examples).is_integer():\n        print('Please select a valid number of examples.')\n        return\n    \n    imgs = []\n    classes = []\n    for i in range(examples):\n        rnd_class, _ = random.choice(list(image_sample_dict.items()))\n        #print(rnd_class)\n        rnd_idx = np.random.randint(0, len(image_sample_dict[rnd_class]))\n        filename = image_sample_dict[rnd_class][rnd_idx]\n        img = read_image(os.path.join(directory, filename))\n        imgs.append(img)\n        classes.append(rnd_class)\n    \n    \n    fig, axes = plt.subplots(round(math.sqrt(examples)), round(math.sqrt(examples)),figsize=(15,15),\n    subplot_kw = {'xticks':[], 'yticks':[]},\n    gridspec_kw = dict(hspace=0.3, wspace=0.1))\n    \n    for i, ax in enumerate(axes.flat):\n        if disp_labels == True:\n            ax.title.set_text(classes[i])\n        ax.imshow(imgs[i])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_images(image_input_dir, image_sample_dict)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can see quite a few images with more than one dog present in the samples."},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_images(image_input_dir, image_sample_dict, examples=36, disp_labels=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There are people present in a few samples as well."},{"metadata":{},"cell_type":"markdown","source":"## Cropping the images according to the annotations"},{"metadata":{},"cell_type":"markdown","source":"The following preprocessing function is used to create the actual features for the future model. The ```dog_images_np``` are first initialized with zeros with a fixed sample size of 25000 images. Each image from the dataset is read and information about the image objects is gathered from the <b>xml</b> file representing each image. \n\nAfter gathering the information on the coordinates, the input image is either cropped, shrunk or expanded, depending on the position of the object and finally, is stored to the array of features. I will also scale the input pixel values of the features to the range [-1, 1], due to the fact the future model will probably be using a <b> tanh </b>activation function."},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_cropped_images(dog_breed_dict=dog_breed_dict, image_ann_dir=image_ann_dir, sample_size=25000, \n                        image_width=image_width, image_height=image_height, image_channels=image_channels):\n    curIdx = 0\n    breeds = []\n    dog_images_np = np.zeros((sample_size,image_width,image_height,image_channels))\n    for breed_folder in os.listdir(image_ann_dir):\n        for dog_ann in tqdm(os.listdir(image_ann_dir + breed_folder)):\n            try:\n                img = read_image(os.path.join(image_input_dir, dog_ann + '.jpg'))\n            except FileNotFoundError:\n                continue\n                \n            tree = ET.parse(os.path.join(image_ann_dir + breed_folder, dog_ann))\n            root = tree.getroot()\n            \n            size = root.find('size')\n            width = int(size.find('width').text)\n            height = int(size.find('height').text)\n            objects = root.findall('object')\n            for o in objects:\n                bndbox = o.find('bndbox') \n                \n                xmin = int(bndbox.find('xmin').text)\n                ymin = int(bndbox.find('ymin').text)\n                xmax = int(bndbox.find('xmax').text)\n                ymax = int(bndbox.find('ymax').text)\n                \n                xmin = max(0, xmin - 4)        # 4 : margin\n                xmax = min(width, xmax + 4)\n                ymin = max(0, ymin - 4)\n                ymax = min(height, ymax + 4)\n\n                w = np.min((xmax - xmin, ymax - ymin))\n                w = min(w, width, height)                     # available w\n\n                if w > xmax - xmin:\n                    xmin = min(max(0, xmin - int((w - (xmax - xmin))/2)), width - w)\n                    xmax = xmin + w\n                if w > ymax - ymin:\n                    ymin = min(max(0, ymin - int((w - (ymax - ymin))/2)), height - w)\n                    ymax = ymin + w\n                \n                img_cropped = img[ymin:ymin+w, xmin:xmin+w, :]      # [h,w,c]\n                # Interpolation method\n                if xmax - xmin > image_width:\n                    interpolation = cv2.INTER_AREA          # shrink\n                else:\n                    interpolation = cv2.INTER_CUBIC         # expansion\n                    \n                img_cropped = cv2.resize(img_cropped, (image_width, image_height), \n                                         interpolation=interpolation)  # resize\n                    \n                dog_images_np[curIdx,:,:,:] = np.asarray(img_cropped)\n                dog_breed_name = dog_breed_dict[dog_ann.split('_')[0]]\n                breeds.append(dog_breed_name)\n                curIdx += 1\n                \n    dog_images_np = dog_images_np / 255.  # change the pixel range to [-1, 1]\n    return dog_images_np, breeds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"start_time = time.time()\ndog_images_np, breeds = load_cropped_images()\nest_time = round(time.time() - start_time)\nprint(\"Feature loading time: {}.\".format(str(datetime.timedelta(seconds=est_time))))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"After we have gathered the features and their labels, we can easily plot them once again to confirm that they have been cropped correctly and that there aren't multiple dogs in a single image."},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_features(features, labels, image_width=image_width, image_height=image_height, \n                image_channels=image_channels,\n                examples=25, disp_labels=True): \n  \n    if not math.sqrt(examples).is_integer():\n        print('Please select a valid number of examples.')\n        return\n    \n    imgs = []\n    classes = []\n    for i in range(examples):\n        rnd_idx = np.random.randint(0, len(labels))\n        imgs.append(features[rnd_idx, :, :, :])\n        classes.append(labels[rnd_idx])\n    \n    \n    fig, axes = plt.subplots(round(math.sqrt(examples)), round(math.sqrt(examples)),figsize=(15,15),\n    subplot_kw = {'xticks':[], 'yticks':[]},\n    gridspec_kw = dict(hspace=0.3, wspace=0.01))\n    \n    for i, ax in enumerate(axes.flat):\n        if disp_labels == True:\n            ax.title.set_text(classes[i])\n        ax.imshow(imgs[i])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Loaded features shape: ', dog_images_np.shape)\nprint('Loaded labels: ', len(breeds))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The total dog images seem to be 20579 with 22125 separately cropped dogs."},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Plotting cropped images by specified coordinates..')\nplot_features(dog_images_np, breeds, examples=16, disp_labels=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(dog_images_np[3])","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}