{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"Some really important links this notebook works on:\n\n1. YOLO basics: https://www.analyticsvidhya.com/blog/2018/12/practical-guide-object-detection-yolo-framewor-python/\n2. Implementing albumentations with tf.keras's Sequence: https://medium.com/the-artificial-impostor/custom-image-augmentation-with-keras-70595b01aeac\n3. Majority of content was inspired from this awesome Kernel: https://www.kaggle.com/mattbast/object-detection-tensorflow-end-to-end\n\n## Import the neccessary modules:","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\nimport albumentations as albu\nfrom tqdm import tqdm\nfrom PIL import Image, ImageDraw\n\nimport os\n\nplt.style.use(\"dark_background\")\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Load the neccessary files","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"! ls ../input/global-wheat-detection/","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"main_dir = \"../input/global-wheat-detection/\"\n\ntrain_files = tf.io.gfile.glob(main_dir + \"train/*\")\ntest_files = tf.io.gfile.glob(main_dir + \"test/*\")\n\nprint (\"No of Train files: {}\\nNo of Test files: {}\".format(len(train_files), len(test_files)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_sub = pd.read_csv(main_dir + \"sample_submission.csv\")\ntrain = pd.read_csv(main_dir + \"train.csv\")\n\ntrain.shape, sample_sub.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### EDA:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# how does the sub file look like?\nsample_sub.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It takes the following format: \n\n img_id | confidence x y w h\n\nA short desc on possible bounding box label representations. Note that our data is stored and expected to be of the **Coco Format**:\n\n
","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# how many unique values does each col contain?\nprint (\"Tot rows:\", train.shape[0])\ntrain.nunique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.width.unique(), train.height.unique()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Height and width have the same value and are redundant. Let's drop them.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train.drop(['width', 'height'], axis=1, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"temp = train.source.value_counts()\nax = temp.plot(kind='bar', figsize=(15, 5), title='Source: Count plots')\nfor i, value in enumerate(temp):\n ax.text(i-0.1, value, value);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"'''\nYou will use more than 3,000 images from Europe (France, UK, Switzerland) and North America (Canada). The test data includes about 1,000 images from Australia, Japan, and China.\n'''","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# An image can obviously come only from one source\n(train.groupby(\"image_id\")['source'].nunique() != 1).any()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# bbox is stored as strings, let process them to list values\ntrain['bbox'] = train['bbox'].transform(lambda x: list(map(float, x.strip(\"[]\").split(\", \"))))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"labels = train.groupby(\"image_id\")['bbox'].apply(np.stack)\nlen(labels)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# how many bbox on avg per image?\ntemp = labels.apply(len)\ntemp.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# avg bounding box based on source\ntemp = train.groupby([\"source\", \"image_id\"]).count().reset_index()\ntemp.groupby(\"source\")[\"bbox\"].agg([\"count\", \"min\", \"mean\", \"max\", \"std\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"f, ax = plt.subplots(nrows=7, figsize=(20, 15))\nfor i, source in enumerate(sorted(train.source.unique())):\n temp.query(f\"source == '{source}'\")['bbox'].plot(kind='hist', ax=ax[i], title=source)\n \nf.suptitle(\"Visualizing BBox Distributions for various sources\")\nf.tight_layout(rect=[0, 0.03, 1, 0.95]);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Let's plot Em images!\nLets write a simple utility function that when given an image_id & bbox, returns an image with bboxes drawn over it: ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def draw_bboxes(img_inp, bboxes, source='train', W=8, img_id_as_input=True, color='red'):\n '''Curtesy of : Mattbast\n From: https://www.kaggle.com/mattbast/object-detection-tensorflow-end-to-end'''\n \n def draw_bbox(draw, bbox):\n x, y, width, height = bbox\n draw.rectangle([x, y, x+width, y+height], width=W, outline=color)\n \n if img_id_as_input:\n image = Image.open(f\"{main_dir}{source}/{img_inp}.jpg\")\n else:\n image = Image.fromarray(img_inp)\n \n draw = ImageDraw.Draw(image)\n \n for bbox in bboxes:\n draw_bbox(draw, bbox)\n \n return np.array(image)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Plotting those images with very little bboxes and those with a lot of bboxes:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"f, ax = plt.subplots(ncols=5, nrows=3, figsize=(20, 10))\nax = ax.ravel()\n\nfor i, (_, img_id, count) in enumerate(temp[[\"image_id\", \"bbox\"]][temp.bbox < 3].sort_values(by='bbox').itertuples()):\n ax[i].imshow(draw_bboxes(img_id, labels[img_id]))\n ax[i].set_title(\"Bbox count: \" + str(count))\n ax[i].axis('off') \n \nf.suptitle(\"Tiny No. of bboxes!\");\n\nf, ax = plt.subplots(ncols=4, nrows=2, figsize=(20, 10))\nfor i, (_, img_id, count) in enumerate(temp[[\"image_id\", \"bbox\"]][temp.bbox > 100].sort_values(by='bbox').itertuples()):\n \n ax[0][i].imshow(plt.imread(f\"{main_dir}train/{img_id}.jpg\"))\n ax[0][i].axis('off')\n ax[0][i].set_title(f\"{img_id} without BBox\")\n \n ax[1][i].imshow(draw_bboxes(img_id, labels[img_id]))\n ax[1][i].set_title(\"Bbox Count: \" + str(count))\n ax[1][i].axis('off') \n \nf.suptitle(\"Large No. of bboxes!\");","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Plot the images not found in CSV. Does any exist?","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# those images in csv but not actually present. No such image exits\nlen(np.setdiff1d(labels.index.values, list(map(lambda x: x.split(\"/\")[-1].split(\".\")[0], train_files))))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# those images not in train.csv but in train images\nmissing_ids = np.setdiff1d(list(map(lambda x: x.split(\"/\")[-1].split(\".\")[0], train_files)), labels.index.values)\n\nf, ax = plt.subplots(ncols=5, nrows=5, figsize=(20, 20))\nax = ax.ravel()\nfor i, img_id in enumerate(np.random.choice(missing_ids, (25,))):\n ax[i].imshow(plt.imread(f\"{main_dir}train/{img_id}.jpg\"))\n ax[i].set_title(img_id)\n ax[i].axis('off')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Plotting the images from different categories:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# plotting without bboxes\nf, ax = plt.subplots(nrows=5, ncols=7, figsize=(20, 15))\nfor i, (source, img_ids) in enumerate(train.groupby(\"source\")['image_id'].apply(lambda x: list(x.sample(5))).iteritems()):\n for j, img_id in enumerate(img_ids):\n ax[j][i].imshow(plt.imread(f\"{main_dir}train/{img_id}.jpg\"))\n ax[j][i].set_title(source)\n ax[j][i].axis('off')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# with and Without Bounding boxes\nf, ax = plt.subplots(nrows=2, ncols=7, figsize=(20, 8))\ntemp = train.groupby(\"source\")['image_id'].apply(lambda x: x.sample(1)).reset_index(level=0)\n\nfor i, (_, source, img_id) in enumerate(temp.itertuples()):\n ax[0][i].imshow(plt.imread(f\"{main_dir}train/{img_id}.jpg\"))\n ax[0][i].set(title=source +\" Without Bboxes\")\n ax[0][i].axis('off')\n \n ax[1][i].imshow(draw_bboxes(img_id, labels[img_id]))\n ax[1][i].set(title=source +\" With Bboxes\")\n ax[1][i].axis('off')\n \nf.tight_layout();","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Plotting the test images:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# how does the test images look like?\nf, ax = plt.subplots(nrows=2, ncols=5, figsize=(20, 8))\nax = ax.ravel()\n\nfor i, img_loc in enumerate(test_files):\n ax[i].imshow(plt.imread(img_loc))\n ax[i].axis('off')\n ax[i].set_title(img_loc.split(\"/\")[-1].split(\".\")[0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's remove those bounding boxes which have very small height or width:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# How many such exists?\nremoved_bboxes = []\ncleaned_labels = {}\n\nfor img_id, bboxes in labels.iteritems():\n for bbox in bboxes:\n if bbox[2] < 10 or bbox[3] < 10:\n removed_bboxes.append((img_id, bbox))\n else:\n cleaned_labels[img_id] = cleaned_labels.get(img_id, []) + [bbox]\n \nlabels = pd.Series(map(np.stack, cleaned_labels.values()), index=cleaned_labels.keys()).rename_axis(\"image_id\")\ndel cleaned_labels\n\n\"No of bboxes removed: {}\".format(len(removed_bboxes))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Do a simple Train / Val split:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_img_ids = labels.iloc[:int(0.95 * len(labels))].index\nval_img_ids = np.setdiff1d(labels.index, train_img_ids)\n\nlen(train_img_ids), len(val_img_ids)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"How does the image look like with 32 grids?","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"GRID_SIZE = 32\n\nplt.figure(figsize=(15, 15))\ntemp = labels.map(len).argmax()\nplt.imshow(draw_bboxes(labels.index[temp], labels.iloc[temp]))\n\nbboxes = labels.iloc[temp].copy()\nbboxes[:, 0] = bboxes[:, 0] + (bboxes[:, 2] / 2)\nbboxes[:, 1] = bboxes[:, 1] + (bboxes[:, 3] / 2)\n\nplt.scatter(*list(zip(*bboxes[:, :2])), c='b')\n\nplt.vlines(range(0, 1024, 1024//GRID_SIZE), 0, 1024, color='y')\nplt.hlines(range(0, 1024, 1024//GRID_SIZE), 0, 1024, color='y')\nplt.title(\"Image with Max Bounding boxes\")\nplt.axis('off');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The image looks pretty clustered. 5 anchors will do us good with grid_size of 32!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"IMG_SHAPE = (256, 256)\nFACTOR = 1024 / IMG_SHAPE[0]\nGRID_SIZE = 32\nANCHORS = 5\nBATCH_SIZE = 16","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets perform some data augmentations to our dataset using the albumentation library:\n\nAgain from this Notebook: \n\n''' \n- random sized crop: The model needs to be able to detect a wheat head regardless of how close or far away the head is to the camera. To produce more zoom levels in the dataset the crop method will take a portion of the image and zoom in to create a new image with larger wheat heads.\n\n- flip amd rotate: The wheat heads can point in any direction. To create more examples of wheat heads pointing in different directions the image will randomly be flipped both horizontally and vertically or rotated.\n\n- hue saturation and brightness: these are various methods that will alter the lighting of the image which will help to create different lighting scenarios. This helps as the test pictures are from various countries each with their own lighting levels.\n\n- noise: Some wheat heads aren't quite in focus. Adding some noise to the images helps to catch these wheat heads while also forcing the model to learn more abstract wheat head shapes. This helps a lot with over-fitting.\n\n- cutout: randomly remove small squares of pixels in the image. This prevents the model simply memorizing certain wheat heads and instead forces it to learn the patterns that represent a wheat head.\n\n- clahe: this is a must have. In many images the wheat heads are a similar colour to the grass in the background making it tricky for the model to differentiate between them. CLAHE helps to exemplify the colour difference between the two.\n\n- grayscale: I found that there were a few images with a yellow/gold tint. My model was learning to detect wheat heads without a tint (as most images do not contain a tint) and was really struggling to detect anything on the yellow images. By converting all images to grey scale the model is forced to ignore these tints making it much more effective at identifying wheat heads regardless of tint.\n\n'''\n\nLet's create augmentation Pipeline for processing our images:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"TRAIN_AUG = albu.Compose([\n albu.RandomSizedCrop(\n min_max_height=[int(IMG_SHAPE[0] * 0.85)] * 2, \n height=IMG_SHAPE[0], \n width=IMG_SHAPE[1], \n p=0.8\n ),\n \n albu.OneOf([\n albu.Flip(),\n albu.RandomRotate90(),\n ], p=1),\n \n albu.OneOf([\n albu.HueSaturationValue(),\n albu.RandomBrightnessContrast()\n ], p=1),\n \n albu.OneOf([\n albu.GaussNoise(),\n albu.GlassBlur(),\n albu.ISONoise(),\n albu.MultiplicativeNoise(),\n ], p=0.5),\n \n albu.Cutout(\n num_holes=8, \n max_h_size=8, \n max_w_size=8, \n fill_value=0, \n p=0.5\n ),\n \n albu.CLAHE(p=1),\n albu.ToGray(p=1),\n ], \n bbox_params={'format': 'coco', 'label_fields': ['labels']}\n)\n\nVAL_AUG = albu.Compose([\n albu.CLAHE(p=1),\n albu.ToGray(p=1)\n])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Demo augmentation:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"org = plt.imread(f\"{main_dir}train/{img_id}.jpg\")\norg = tf.cast(tf.image.resize(org, IMG_SHAPE), tf.uint8).numpy()\norg_bboxes = labels[img_id] / FACTOR\naug = TRAIN_AUG(image=org, bboxes=org_bboxes, labels=np.ones(len(org_bboxes)))\n\naug.keys()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's visualize how are augmentation pipeline works:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"f, ax = plt.subplots(figsize=(20, 20), ncols=3, nrows=3)\nax = ax.ravel()\n\nax[0].imshow(draw_bboxes(org, org_bboxes, img_id_as_input=False, W=2))\nax[0].set_title(\"Original Image\")\nax[0].axis('off')\n\nfor i in range(8):\n aug = TRAIN_AUG(image=org, bboxes=org_bboxes, labels=np.ones(len(org_bboxes)))\n ax[i + 1].imshow(draw_bboxes(aug['image'], aug['bboxes'], img_id_as_input=False, W=2))\n ax[i + 1].axis(\"off\")\n ax[i + 1].set_title(f\"Train Aug {i + 2}\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"aug = VAL_AUG(image=org, bboxes=org_bboxes, labels=np.ones(len(org_bboxes)))\n\nf, ax, = plt.subplots(ncols=2, figsize=(16, 8))\nax[0].imshow(draw_bboxes(org, org_bboxes, img_id_as_input=False, W=2))\nax[0].set_title(\"Original Image\")\nax[0].axis('off')\nax[1].imshow(draw_bboxes(aug['image'], aug['bboxes'], img_id_as_input=False, W=2))\nax[1].axis(\"off\")\nax[1].set_title(\"Val Augmented\");","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"Now that we have our augmentation pipeline ready, let's get on with creating a custom sequence generator using: `keras.utils.Sequence`. Before doing this however let's load all the images to memory for faster processing:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_resize_image(image_loc):\n image = tf.io.read_file(image_loc)\n image = tf.image.resize(tf.image.decode_jpeg(image), IMG_SHAPE)\n image = tf.cast(image, tf.uint8).numpy()\n return image_loc[:-4].split(\"/\")[-1], image","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# load the images and resize them\ntrain_files = dict(map(lambda x: load_resize_image(x), tqdm(train_files)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# rain check! Working good?\ntemp = np.random.choice(train_img_ids)\nplt.figure(figsize=(10, 10))\nplt.imshow((draw_bboxes(train_files[temp], labels[temp]/FACTOR, W=2, img_id_as_input=False)))\nplt.axis('off');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's create our Dataset generator:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class GWD_Dataset(tf.keras.utils.Sequence):\n def __init__(self, image_ids, image_pixels, labels=None, grid_size=GRID_SIZE, batch_size=1, shuffle=False, augment=False):\n \n self.image_ids = image_ids\n self.image_pixels = image_pixels\n self.labels = labels\n self.grid_size = grid_size\n self.batch_size = batch_size\n self.shuffle = shuffle\n self.augment = augment\n self.on_epoch_end()\n \n self.train_augmentations = TRAIN_AUG\n self.val_augmentations = VAL_AUG\n \n self.image_grid = self.form_image_grid()\n \n \n def form_image_grid(self): \n '''We use this image grid later on for the creation of BBoxes which needs\n to have relative x,y,h,w coordinates. And for rescaling back from the predictions.\n \n ** We use a 16 x 16 grid (self.grid_size) for our model. **\n '''\n \n # Note the shape: [x, y, width, height] -> we create it in Coco format, later to YOLO\n image_grid = np.zeros((self.grid_size, self.grid_size, 4))\n\n # width=32, height=32 when grid_size=16 (size of each grid)\n hw_grid = IMG_SHAPE[0] / self.grid_size\n image_grid[..., 2:] = hw_grid\n\n # increase width as we move towards the right\n temp = np.arange(0, IMG_SHAPE[0], hw_grid)\n image_grid[..., :1] = temp.reshape(-1, 1)\n \n # increase height as we move down the grid (repeat n times before increasing)\n image_grid[..., 1:2] = np.repeat(temp, self.grid_size).reshape(self.grid_size, self.grid_size, 1)\n\n return image_grid\n \n def __len__(self):\n '''Overriding the function to obtain train/val dataset length needed for fit(). \n Returns number of batches in the Sequence.'''\n \n return len(self.image_ids) // self.batch_size\n\n\n def on_epoch_end(self):\n '''Executed automatically every end of each epoch.'''\n \n self.indexes = np.arange(len(self.image_ids))\n\n if self.shuffle == True:\n np.random.shuffle(self.indexes) # shuffle inplace\n \n \n def __getitem__(self, index):\n '''Gets batch AT a position index.'''\n \n indexes = self.indexes[index * self.batch_size:(index + 1) * self.batch_size]\n batch_ids = [self.image_ids[i] for i in indexes]\n\n X, y = self.__data_generation(batch_ids)\n\n return X, y\n\n\n def __data_generation(self, batch_ids):\n X, y = [], []\n\n # Generate data\n for image_id in batch_ids:\n pixels = self.image_pixels[image_id]\n bboxes = self.labels[image_id]\n\n if self.augment: \n pixels, bboxes = self.augment_image(pixels, bboxes)\n \n else:\n pixels = self.contrast_image(pixels)\n bboxes = self.form_label_grid(bboxes)\n\n X.append(pixels)\n y.append(bboxes)\n\n return np.array(X), np.array(y)\n\n def augment_image(self, pixels, bboxes):\n\n aug_result = self.train_augmentations(image=pixels, bboxes=bboxes, labels=np.ones(len(bboxes)))\n bboxes = self.form_label_grid(aug_result['bboxes'])\n\n return np.array(aug_result['image']) / 255, bboxes\n\n def contrast_image(self, pixels): \n \n aug_result = self.val_augmentations(image=pixels)\n return np.array(aug_result['image']) / 255\n \n def form_label_grid(self, Bboxes):\n 'Function only depends on the bboxes and it can be any image'\n\n label_grid = np.zeros((self.grid_size, self.grid_size, ANCHORS*5))\n\n if len(Bboxes): # if no bounding boxes, return 0\n # we convert all bboxes to yolo and return their centers \n # which can then be used to create the label grid\n bboxes, bx_centers, by_centers = self.yolo_shape(np.stack(Bboxes))\n\n else:\n return label_grid\n\n for i in range(0, self.grid_size):\n for j in range(0, self.grid_size):\n\n # intersect those bboxes whoes centers lie \n # inside cell we create n anchors\n label_grid[i,j] = self.rect_intersect(self.image_grid[i,j], bboxes, bx_centers, by_centers)\n\n return label_grid\n\n def rect_intersect(self, cell, bboxes, bbox_x_centers, bbox_y_centers): \n\n # if bbox lies with cell, True for that index\n cond = (\n (cell[0] <= bbox_x_centers) & \n (cell[1] <= bbox_y_centers) & \n (bbox_x_centers < cell[0] + cell[2]) & \n (bbox_y_centers < cell[1] + cell[3])\n )\n\n # pad with 10 (2 anchor boxes)\n temp_bboxes = bboxes[cond].ravel()\n temp_bboxes = np.pad(temp_bboxes, [0, 5*ANCHORS])[:5*ANCHORS]\n\n # output shape: (10,)\n return temp_bboxes \n\n def yolo_shape(self, bboxes):\n 'Convert the bboxes to yolo format and scale them'\n\n # scale factor\n scale_factor = (IMG_SHAPE[0] // self.grid_size)\n\n # calculate the box centers\n bboxes[:, 0] = bboxes[:, 0] + (bboxes[:, 2] / 2)\n bboxes[:, 1] = bboxes[:, 1] + (bboxes[:, 3] / 2)\n\n bbox_x_centers = bboxes[:, 0].copy()\n bbox_y_centers = bboxes[:, 1].copy()\n\n # we find the box lower limit given the grid size\n # for eg: 13 lies between (0, 16); (8, 16); (12, 16)\n # for grid sizes of 16, 32, 64..\n x_lowers = (bbox_x_centers // scale_factor) * scale_factor\n y_lowers = (bbox_y_centers // scale_factor) * scale_factor\n\n # Offset bbox x,y to cell x,y\n # bbox x,y,h,w will be calculated relative \n # to the grid position they would belong to\n bboxes[:, 0] = (bboxes[:, 0] - x_lowers) / scale_factor\n bboxes[:, 1] = (bboxes[:, 1] - y_lowers) / scale_factor\n bboxes[:, 2] = bboxes[:, 2] / IMG_SHAPE[0]\n bboxes[:, 3] = bboxes[:, 3] / IMG_SHAPE[1]\n\n return np.hstack([np.ones((len(bboxes), 1)), bboxes]), bbox_x_centers, bbox_y_centers","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"strategy = tf.distribute.MirroredStrategy()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n\n train_generator = GWD_Dataset(\n train_img_ids,\n train_files,\n labels/FACTOR, \n batch_size=BATCH_SIZE, \n shuffle=True,\n augment=True,\n )\n\n val_generator = GWD_Dataset(\n val_img_ids,\n train_files,\n labels/FACTOR, \n batch_size=BATCH_SIZE,\n shuffle=False,\n augment=False\n )\n\nimage_grid = train_generator.image_grid\n\nlen(train_generator), len(val_generator)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntemp = train_generator[0]\ntemp[0].shape, temp[1].shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntemp = val_generator[1]\ntemp[0].shape, temp[1].shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The input shape: (256, 256, 3) \nOutput shape: (32, 32, 10)\n\nThe basics we need to be aware of:\n\nIn YOLO, the coordinates assigned to all the grids are:\n\n\n\n\nx,y are the coordinates of midpoint of object with respect to the grid\n- height is the ratio of height of bounding box WRT height of grid cell\n- width is the ratio of width of bounding box WRT width of grid cell\n\n`Note:` x and y will always range between 0 and 1 as the midpoint will always lie within the grid. Whereas height and width can be more than 1 in case the dimensions of the bounding box are more than the dimension of the grid.\n\nThe Non-Max Suppression technique cleans up this up so that we get only a single detection per object. Steps are as follows:\n\n0. Discard all the boxes having probabilities less than or equal to a pre-defined threshold (say, 0.5)\n\n1. It first looks at the probabilities associated with each detection and takes the largest one. In the above image, 0.9 is the highest probability, so the box with 0.9 probability will be selected first:\n\n\n2. Now, it looks at all the other boxes in the image. The boxes which have high IoU with the current box are suppressed. So, the boxes with 0.6 and 0.7 probabilities will be suppressed in our example:\n\n\n3. After the boxes have been suppressed, it selects the next box from all the boxes with the highest probability, which is 0.8 in our case:\n\n\n4. Again it will look at the IoU of this box with the remaining boxes and compress the boxes with a high IoU:\n\n\n5. We repeat these steps until all the boxes have either been selected or compressed and we get the final bounding boxes:\n\n\n\nLet's now write some utility functions to get our predictions back to bboxes and implementing the Non max supression algorithm we had just discussed:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def prediction_to_bbox(Bboxes, image_grid): \n '''A function to convert predictions of Model from scaled YOLO back to COCO BBoxes'''\n \n bboxes = Bboxes.copy()\n \n # mutiplying with the number of grids\n im_width = (image_grid[:,:,2] * GRID_SIZE)\n im_height = (image_grid[:,:,3] * GRID_SIZE)\n \n # descale X, Y, HEIGHT, WIDTH\n for i in range(ANCHORS):\n bboxes[:,:, (5*i)+1] = (bboxes[:,:, (5*i)+1] * image_grid[:,:,2]) + image_grid[:,:,0]\n bboxes[:,:, (5*i)+2] = (bboxes[:,:, (5*i)+2] * image_grid[:,:,3]) + image_grid[:,:,1]\n bboxes[:,:, (5*i)+3] = (bboxes[:,:, (5*i)+3] * im_width)\n bboxes[:,:, (5*i)+4] = (bboxes[:,:, (5*i)+4] * im_height)\n \n \n # centre x,y to top left x,y\n for i in range(ANCHORS):\n bboxes[:,:, (5*i)+1] = bboxes[:,:, (5*i)+1] - (bboxes[:,:, (5*i)+3] / 2)\n bboxes[:,:, (5*i)+2] = bboxes[:,:, (5*i)+2] - (bboxes[:,:, (5*i)+4] / 2)\n\n \n # width,heigth to X_MAX,Y_MAX\n # this is important since we would be feeding\n # bboxes to tf.image.non_max_supression\n for i in range(ANCHORS):\n bboxes[:,:, (5*i)+3] = bboxes[:,:, (5*i)+1] + bboxes[:,:, (5*i)+3]\n bboxes[:,:, (5*i)+4] = bboxes[:,:, (5*i)+2] + bboxes[:,:, (5*i)+4]\n \n return bboxes\n\ndef non_max_suppression(predictions, nms_thresh, nms_score, top_n=10):\n \n probabilities = np.concatenate([predictions[:,:, 5*i].flatten() for i in range(ANCHORS)], axis=None)\n \n # having extracted probs, we extract just x,y,h,w from two anchors\n first_anchors = predictions[:, :, 1:5].reshape((GRID_SIZE * GRID_SIZE, 4))\n second_anchors = predictions[:, :, 6:10].reshape((GRID_SIZE * GRID_SIZE, 4))\n \n # we concatenate both anchor boxes together\n bboxes = np.concatenate(\n [predictions[:, :, (5*i)+1:(5*i)+5].reshape((GRID_SIZE * GRID_SIZE, 4)) for i in range(ANCHORS)],\n axis=0\n )\n \n # given the probablities of bounding boxes, we perform NMS and return \n # only bboxes above threshold IOU bboxes with high similarity are\n # removed and replaced with one with the highest conf\n # input is supplied as [y1, x1, y2, x2]\n top_indices = tf.image.non_max_suppression(\n boxes=bboxes[:, [1, 0, 3, 2]], \n scores=probabilities, \n max_output_size=top_n, \n iou_threshold=nms_thresh,\n score_threshold=nms_score,\n ).numpy()\n \n return np.hstack([probabilities[top_indices][..., np.newaxis], bboxes[top_indices]])\n\ndef process_predictions(Predictions, image_ids, image_grid, return_prob=False, nms_thresh=0.3, nms_score=0.3, rescale=False):\n '''Returns a dictionary. BBoxes as values & image_ids are keys.'''\n \n predictions = Predictions.copy()\n bboxes = {}\n \n for i, image_id in enumerate(image_ids):\n \n # convert YOLO to COCO\n predictions[i] = prediction_to_bbox(predictions[i], image_grid)\n # NMS, Keep only top 100 predictions\n bboxes[image_id] = non_max_suppression(predictions[i], top_n=100, nms_thresh=nms_thresh, nms_score=nms_score)\n \n # back to coco (x, y, X_MAX, y_MAX) -> (x, y, width, height)\n bboxes[image_id][:, 3:5] = bboxes[image_id][:, 3:5] - bboxes[image_id][:, 1:3]\n \n if not return_prob:\n bboxes[image_id] = bboxes[image_id][:, 1:]\n \n if rescale:\n bboxes[image_id][:, -4:] = bboxes[image_id][:, -4:] * FACTOR\n \n return bboxes","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets see if our process_predictions pipeline work as expected:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"img = []\nval_predictions = []\ntemp = val_generator[np.random.randint(0, len(val_generator))]\n\nfor i in tqdm(range(8)):\n \n img.append(temp[0][i])\n val_predictions.append(temp[1][i])\n \nval_predictions = process_predictions(val_predictions, range(8), image_grid)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"f, ax, = plt.subplots(nrows=4, ncols=2, figsize=(12, 24))\nax = ax.ravel()\n\nfor i in range(8):\n \n ax[i].imshow(draw_bboxes((img[i] * 255).astype(\"uint8\"), \n val_predictions[i], img_id_as_input=False, \n color='red', W=2))\n \n ax[i].axis('off')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Working good! Now let's create a tensorflow dataset for these generators to be able to cache it to memory:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n \n val_dataset = tf.data.Dataset.from_generator(\n lambda: GWD_Dataset(val_img_ids,\n train_files,\n labels/FACTOR, \n batch_size=BATCH_SIZE,\n shuffle=False,\n augment=False),\n output_types=(tf.float32, tf.float32),\n output_shapes=([BATCH_SIZE, *IMG_SHAPE, 3], [BATCH_SIZE, GRID_SIZE, GRID_SIZE, 5*ANCHORS])\n ).cache()\n\nval_dataset","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%time temp = next(iter(val_dataset))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's now define our model.\n\nWe are going to use the yolo_v3 model:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"tf.keras.backend.clear_session()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# with strategy.scope():\n \nx_input = tf.keras.Input(shape=(*IMG_SHAPE, 3))\n\nx = tf.keras.layers.Conv2D(32, (3, 3), strides=(1, 1), padding='same')(x_input)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n########## block 1 ##########\nx = tf.keras.layers.Conv2D(64, (3, 3), strides=(1, 1), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx_shortcut = x\n\nfor i in range(2):\n x = tf.keras.layers.Conv2D(32, (3, 3), strides=(1, 1), padding='same')(x)\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x = tf.keras.layers.Conv2D(64, (3, 3), strides=(1, 1), padding='same')(x)\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x = tf.keras.layers.Add()([x_shortcut, x])\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x_shortcut = x\n\n\n########## block 2 ##########\nx = tf.keras.layers.Conv2D(128, (3, 3), strides=(2, 2), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx_shortcut = x\n\nfor i in range(2):\n x = tf.keras.layers.Conv2D(64, (3, 3), strides=(1, 1), padding='same')(x)\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x = tf.keras.layers.Conv2D(128, (3, 3), strides=(1, 1), padding='same')(x)\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x = tf.keras.layers.Add()([x_shortcut, x])\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x_shortcut = x\n\n########## block 3 ##########\nx = tf.keras.layers.Conv2D(256, (3, 3), strides=(2, 2), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx_shortcut = x\n\nfor i in range(8):\n x = tf.keras.layers.Conv2D(128, (3, 3), strides=(1, 1), padding='same')(x)\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x = tf.keras.layers.Conv2D(256, (3, 3), strides=(1, 1), padding='same')(x)\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x = tf.keras.layers.Add()([x_shortcut, x])\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x_shortcut = x\n\n\n########## block 4 ##########\nx = tf.keras.layers.Conv2D(512, (3, 3), strides=(2, 2), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx_shortcut = x\n\nfor i in range(8):\n x = tf.keras.layers.Conv2D(256, (3, 3), strides=(1, 1), padding='same')(x)\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x = tf.keras.layers.Conv2D(512, (3, 3), strides=(1, 1), padding='same')(x)\n x = tf.keras.layers.BatchNormalization()(x)\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n x = tf.keras.layers.Add()([x_shortcut, x])\n x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n \n\n########## output layers ##########\nx = tf.keras.layers.Conv2D(512, (3, 3), strides=(1, 1), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx = tf.keras.layers.Conv2D(256, (3, 3), strides=(1, 1), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx = tf.keras.layers.Conv2D(128, (3, 3), strides=(1, 1), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\npredictions = tf.keras.layers.Conv2D(5*ANCHORS, (1, 1), strides=(1, 1), activation='sigmoid')(x)\n\nmodel = tf.keras.Model(inputs=x_input, outputs=predictions)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.output_shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"'''\n\n\nOne issue with yolo is that it is likely to contain more cells in its label grid that contain no objects than cells that do contain objects. It is easy then for the model to focus too much on learning to reduce no object cells to zero and not focus enough on getting the bounding boxes to the right shape. To overcome this the yolo paper suggests _weighting the cells containing bounding boxes five times higher and the cells with no bounding boxes by half._\n\nI have defined a custom loss function to do just this. I have also split the loss function into three parts. The first takes care of the confidence score that is trying to work out if a label grid cell contains a head of wheat or not. Binary cross entropy is used here as that is a binary classification task. The second part looks at the x,y position of the bounding boxes while the third looks at the width,height of the bounding boxes. MSE (mean squared loss) is used for the second and third parts as they are regression tasks.\n\n'''","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def custom_loss(y_true, y_pred):\n binary_crossentropy = tf.keras.losses.BinaryCrossentropy(\n reduction=tf.keras.losses.Reduction.SUM_OVER_BATCH_SIZE\n# reduction=tf.keras.losses.Reduction.NONE\n )\n \n # binary cross entropy for confidence scores\n prob_loss = binary_crossentropy(\n tf.concat([y_true[:,:,:,5*i] for i in range(ANCHORS)], axis=0), \n tf.concat([y_pred[:,:,:,5*i] for i in range(ANCHORS)], axis=0)\n )\n \n # MSE for x, y (TWO anchor boxes)\n xy_loss = tf.keras.losses.MSE(\n tf.concat([y_true[:,:,:,(5*i)+1:(5*i)+3] for i in range(ANCHORS)], axis=0),\n tf.concat([y_pred[:,:,:,(5*i)+1:(5*i)+3] for i in range(ANCHORS)], axis=0)\n )\n \n # MSE loss for Height and width (TWO anchor boxes)\n wh_loss = tf.keras.losses.MSE(\n tf.concat([y_true[:,:,:,(5*i)+3:(5*i)+5] for i in range(ANCHORS)], axis=0),\n tf.concat([y_pred[:,:,:,(5*i)+3:(5*i)+5] for i in range(ANCHORS)], axis=0)\n )\n \n # get mask for appropriate weighing of the losses\n bboxes_mask = get_mask(y_true)\n weighted_loss = (xy_loss + wh_loss) * bboxes_mask\n \n# return tf.reduce_mean(prob_loss) + weighted_loss\n return prob_loss + weighted_loss\n\n\ndef get_mask(y_true):\n '''If the grid cell contains a wheat head we need to weigh it \n 5 times higher and 0.5 times if it contains no wheat cell.'''\n \n bboxes_mask = tf.concat(\n [tf.where(y_true[:,:,:,5*i] == 0, 0.5, 5.0) for i in range(ANCHORS)],\n axis=0\n )\n \n return bboxes_mask","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# with strategy.scope():\n\nmodel.compile(\n optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001), \n loss=custom_loss\n)\n\ncallbacks = [\n tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', patience=2, factor=0.1),\n tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=4, restore_best_weights=True),\n tf.keras.callbacks.ModelCheckpoint(\"./model_save\", save_best_only=True),\n]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"history = model.fit(\n train_generator,\n validation_data=val_dataset,\n epochs=50,\n callbacks=callbacks\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.save(\"model.h5\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's visualize and evaluate our model performance:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1, 2, figsize=(20, 5))\n\nax[0].set_title('Training Loss')\nax[0].plot(history.history['loss'])\n\nax[1].set_title('Validation Loss')\nax[1].plot(history.history['val_loss'])\n\nprint('Epochs: ' + str(len(history.history['loss'])))\nprint('Final training loss: ' + str(history.history['loss'][-1]))\nprint('Final validation loss: ' + str(history.history['val_loss'][-1]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"val_predictions = model.predict(val_dataset)\nval_predictions = process_predictions(\n val_predictions, val_img_ids[:len(val_generator) * BATCH_SIZE], \n image_grid, nms_thresh=0.25, nms_score=0.35)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets visualize the predictions the model has made on the validation data:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"temp = np.random.choice(val_img_ids[:len(val_generator) * BATCH_SIZE], (8,))\n\n# plotting for validation images\nf, ax = plt.subplots(nrows=2, ncols=4, figsize=(20, 10))\nax = ax.ravel()\n\nfor i, img_id in enumerate(temp):\n org = draw_bboxes(img_id, labels[img_id], color='blue')\n ax[i].imshow(draw_bboxes(org, val_predictions[img_id]*FACTOR, img_id_as_input=False, color='red'))\n ax[i].axis('off')\n \nf.suptitle(\"Model Evaluation\", size=20)\nf.tight_layout(rect=[0, 0.03, 1, 0.95]);","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's make predictions on the test images:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"test_image_ids = os.listdir('/kaggle/input/global-wheat-detection/test/')\ntest_image_ids = [image_id[:-4] for image_id in test_image_ids]\n\ntest_predictions = []\n\nfor i, image_id in enumerate(test_image_ids):\n image = Image.open('/kaggle/input/global-wheat-detection/test/' + image_id + \".jpg\")\n image = image.resize(IMG_SHAPE) \n\n pixels = np.asarray(image)\n\n aug_result = VAL_AUG(image=pixels)\n pixels = np.array(aug_result['image']) / 255\n pixels = np.expand_dims(pixels, axis=0)\n\n bboxes = model.predict(pixels)\n\n test_predictions.append(bboxes)\n \ntest_predictions = np.concatenate(test_predictions)\ntest_predictions = process_predictions(\n test_predictions, test_image_ids, image_grid, \n return_prob=True, nms_thresh=0.2, nms_score=0.5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# plotting for test images\nf, ax = plt.subplots(nrows=3, ncols=3, figsize=(15, 15))\nax = ax.ravel()\n\nfor i, img_id in enumerate(test_image_ids[:9]):\n ax[i].imshow(draw_bboxes(img_id, test_predictions[img_id][:, 1:]*FACTOR, source='test'))\n ax[i].axis('off')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"op = pd.DataFrame(test_predictions.items())\nop[1] = op[1].map(lambda x: ' '.join(np.ravel(x).astype(str)))\nop.rename({0: \"image_id\", 1: \"PredictionString\"}, axis=1).to_csv(\"submission.csv\", index=False)\npd.read_csv(\"submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"How to improve on our model predictions?\n\nYOLO does not work pretty well for small objects. In order to improve its performance on smaller objects, you can try the following things:\n\n- Increase the number of anchor boxes\n- Decrease the threshold for IoU\n- Give wh_loss more weights than xy_loss since our model gets the height and width predictions wrong\n- Transfer learning: Use pretrained weights: https://github.com/jacksonxliu/YOLOv3-tiny-custom-object-detection","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}