{"cells":[{"metadata":{},"cell_type":"markdown","source":"I refered https://www.kaggle.com/mattbast/object-detection-tensorflow-end-to-end","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np \nimport pandas as pd \nimport tensorflow as tf\nimport os\nimport matplotlib.pyplot as plt\nfrom PIL import Image, ImageDraw, ImageEnhance\nimport albumentations as albu\nfrom tqdm.notebook import tqdm\n\nDIR_INPUT = '../input/global-wheat-detection'\nlabels = pd.read_csv(f'{DIR_INPUT}/train.csv')\ninput_size = (256,256)\nlabels.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To group bbox by image_id and convert bbox dtype from string into float","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def group_boxes(group):\n    \n    \"\"\"groups bbox by image_id and removes all without numbers\n    \n    Args:\n        group: Series of pandas\n    Returns:\n        Arrays grouped by the same image_id\n    \"\"\"\n    \n    boundaries = group['bbox'].str.split(',', expand=True)\n    \n    # To get rid of '[' , ']'.\n    boundaries[0] = boundaries[0].str.slice(start=1)\n    boundaries[3] = boundaries[3].str.slice(stop=-1)\n    \n    return boundaries.to_numpy().astype(float)\n\nlabels = labels.groupby('image_id').apply(group_boxes)\nlen(labels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Train data is not that a lot. So I separated only 10% of data to use as valid data. A model doesn't learn for valid data. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_image_ids = labels.index.to_numpy()[:-33]\nvalid_image_ids = labels.index.to_numpy()[-33:]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It needs to be organized by image's pixels data and bounding boxes data. And I will use a model that uses (256,256) of image size. So Let us resize it. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_image(image_id):\n    \"\"\"loads and resizes image to input size\n    Args:\n        image_id: An image id in train data\n    return:\n        resized image as array\n    \"\"\"\n    global input_size\n    image = Image.open(f'{DIR_INPUT}/train/{image_id}.jpg').resize(input_size)\n    return np.asarray(image)\n\ndef reorganize(image_ids):\n    \"\"\"separates image data to pixels and bboxes\n    Args:\n        image_ids: An iterator that contains ids of image\n    return:\n        resized image as array, bboxes\n    \"\"\"\n    \n    images = {}\n    bboxes = {}\n    for image_id in tqdm(image_ids):\n        #images[image_id] = np.expand_dims(load_image(image_id), axis = 0)\n        images[image_id] = load_image(image_id)\n        bboxes[image_id] = labels[image_id]\n        \n    return images, bboxes\n\ntrain_images, train_bboxes = reorganize(train_image_ids)\nvalid_images, valid_bboxes = reorganize(valid_image_ids)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's see what data looks like. In next cell, I make a function that draws the red line boxes using with bboxes data that is ground truth data. ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def draw_boxes_on_image(image, bboxes, color = 'red'):\n    \"\"\"draws lines on the picture where there are wheat\n    Args:\n        image: An image. Not array\n        bboxes: an iterator of box data. (x,y,w,h)\n        color : color of line\n    return:\n        image that lines are drawn on\n    \"\"\"\n    draw = ImageDraw.Draw(image)\n    for bbox in bboxes:\n        draw.rectangle([bbox[0], bbox[1], bbox[0]+bbox[2], bbox[1]+bbox[3]], \n                       width=4, outline=color)\n    return image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Take a look if the function works well.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"example_id = valid_image_ids[2]\nexample_image = Image.open(f'{DIR_INPUT}/train/{example_id}.jpg')\nexample_detection = draw_boxes_on_image(example_image, valid_bboxes[example_id])\nplt.figure(figsize=(8,8))\nplt.imshow(example_detection)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"DataGenerator is one of the most important parts of this notebook. I will use ***tf.keras.utils.Sequence*** as the data generator. DataGenerator will be inputted into ***model.fit*** as the argument x. Several types could be the argument x and one of them is ***Sequence***. ***Sequence*** has to have two functions in itself that are ***\\_\\_len__*** and ***\\_\\_getitem__***. If you inherit ***tf.keras.utils.Sequence*** but there are no that function, It is not going to work. The function ***\\_\\_len__*** returns a number that should be an information appears a length of something. In this case, ***\\_\\_len__*** returns ***image_ids/batch_size*** that represents how many times ***DataGenerator*** returns the input data in an epoch. e.g If I set 2 batch size and 10 of image data, then It should be running 5 times in an epoch. When you do ***a\\[i\\]***, it calls ***\\_\\_getitem__***. In this case, I will set ***\\_\\_getitem__*** returns the batch size of data from ***\\_\\_data_generation*** function.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class DataGenerator(tf.keras.utils.Sequence):\n    \"\"\"DataGenerator is input data going into model.fit and validation_data\n    Every Sequence must implement the __getitem__ and the __len__ methods. \n    The method __getitem__ should return a complete batch.\n    If you want to modify your dataset between epochs you may implement on_epoch_end. \n    \"\"\"\n    def __init__(self, image_ids, image_pixels, labels, \n                 batch_size=1, shuffle = False, augment = False):\n        self.image_ids = image_ids\n        self.image_pixels = image_pixels\n        self.labels = labels\n        self.batch_size = batch_size\n        self.shuffle = shuffle\n        self.augment = augment\n        self.image_grid = self.form_image_grid()\n        \n        self.on_epoch_end() \n        \n    def __len__(self):\n        \"\"\" is used to determine how many images there are in dataset.\n        Python len() function returns the length of the object.\n        This function internally calls __len__() function of the object. \n        So we can use len() function with any object that defines __len__() function. \n        \"\"\"\n        return int(np.floor(len(self.image_ids)/self.batch_size))\n          \n    def __getitem__(self, index):\n        \"\"\"When the batch corresponding to a given index is called, \n        the generator executes the __getitem__ method to generate it.\n        i.e To get batch at position 'index'\n        \"\"\"\n        \n        # Generate indices of the batch\n        indices = self.indices[index * self.batch_size : (index+1) * self.batch_size]\n        \n        # Find list of ids\n        batch_ids = [self.image_ids[k] for k in indices]\n        self.batch_ids = batch_ids\n        \n        # Generate data\n        X, y = self.__data_generation(batch_ids)\n        \n        return X, y\n        \n    def on_epoch_end(self):\n        \"\"\"If you want to modify your dataset between epochs you may implement on_epoch_end\"\"\"\n        \n        self.indices = np.arange(len(self.image_ids))\n        \n        if self.shuffle:\n            np.random.shuffle(self.indices)\n        \n    def __data_generation(self, batch_ids):\n        \"\"\"Produces batch-size of data \"\"\"\n        \n        X, y = [], []\n        \n        # Generate data\n        for image_id in batch_ids:\n            pixels = self.image_pixels[image_id]\n            bboxes = self.labels[image_id]\n            \n            if self.augment:\n                pixels, bboxes = self.augment_image(pixels, bboxes)\n                \n            else:\n                pixels = self.contrast_image(pixels)\n                bboxes = self.form_label_grid(bboxes)\n                \n            X.append(pixels)\n            y.append(bboxes)\n        \n        X = np.array(X)\n        y = np.array(y)\n        \n        return X, y\n    \n    def form_image_grid(self):    \n        \"\"\"creates image grid cells which indicate the information about the location where a cell is \"\"\"\n        \n        image_grid = np.zeros((16, 16, 4))\n        cell = [0, 0, 16,  16] \n\n        for i in range(0, 16):\n            for j in range(0, 16):\n                image_grid[i,j] = cell\n\n                cell[0] = cell[0] + cell[2]\n\n            cell[0] = 0\n            cell[1] = cell[1] + cell[3]\n\n        return image_grid\n    \n    \n    def augment_image(self, pixels, bboxes):\n        \"\"\"augments image\n        \n        Args:\n            pixels: a batch size of images as array\n            bboxes: a batch size of bboxes as array\n        retruns:\n            augmented images and bboxes scaled down 0 to 1,\n        \"\"\"\n        \n        # from 1024 to 256\n        downsized_bboxes = bboxes / 4\n        \n        bbox_labels = np.ones(len(bboxes))\n        aug_result = self.train_augmentations(image=pixels, bboxes=downsized_bboxes, labels=bbox_labels)\n        bboxes = self.form_label_grid(aug_result['bboxes'])\n        \n        return aug_result['image']/256, bboxes\n    \n    def contrast_image(self, pixels):\n        \"\"\"converts images into grayscale\"\"\"\n        \n        aug_result = self.val_augmentations(image=pixels)\n        return aug_result['image']/256\n        \n    def form_label_grid(self, bboxes):\n        \"\"\"returns Yolo shape of a label grid\"\"\"\n        \n        label_grid = np.zeros((16, 16, 5))\n        \n        for i in range(16):\n            for j in range(16):\n                cell = self.image_grid[i,j]\n                label_grid[i,j] = self.rect_intersect(cell, bboxes) \n        \n        return label_grid\n    \n    def rect_intersect(self, cell, bboxes):\n        \"\"\"puts all boundary boxes into appropriate cells in the grid.\"\"\"\n        \n        cell_x, cell_y, cell_width, cell_height = cell\n        cell_x_max = cell_x + cell_width\n        cell_y_max = cell_y + cell_height\n\n        anchor_one = np.zeros(5)\n        anchor_two = np.zeros(5)\n        \n        for bbox in bboxes:\n            if self.augment:\n                bbox_ = bbox\n            else :\n                bbox_ = bbox/4\n            box_x, box_y, box_width, box_height = bbox_\n            box_x_centre = box_x + box_width/2\n            box_y_centre = box_y + box_height/2\n            \n            # If the centre of box is in the cell, \n            if (box_x_centre >= cell_x and box_x_centre < cell_x_max\n               and box_y_centre >= cell_y and box_y_centre < cell_y_max):\n                \n                if anchor_one[0] == 0:\n                    anchor_one = self.yolo_shape(bbox_, cell)\n\n                else:\n                    break\n            \n        return anchor_one\n    \n    def yolo_shape(self, bbox, cell):\n        \"\"\"converts the shape of boundary boxes into the shape of Yolo \"\"\"\n    \n            \n        box_x, box_y, box_width, box_height = bbox \n        cell_x, cell_y, cell_width, cell_height = cell\n        \n        box_x_centre = box_x + box_width / 2\n        box_y_centre = box_y + box_height / 2\n        \n        resized_box_x = (box_x_centre - cell_x) / cell_width\n        resized_box_y = (box_y_centre - cell_y) / cell_height\n        resized_box_width = box_width / 256 \n        resized_box_height = box_height / 256\n        \n        return [1, resized_box_x, resized_box_y, resized_box_width, resized_box_height]\n\n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"A cell below is about agumentation.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"DataGenerator.train_augmentations = albu.Compose([\n    \n    albu.RandomSizedCrop(\n        min_max_height = (200,200),\n        height = 256,\n        width = 256,\n        p=0.8\n    ),\n    albu.OneOf([\n        albu.Flip(),\n        albu.RandomRotate90()\n    ], p=1),\n    albu.OneOf([\n        albu.HueSaturationValue(),\n        albu.RandomBrightnessContrast()\n    ], p=1),\n    albu.OneOf([\n        albu.GaussNoise(),\n        albu.GaussianBlur(),\n        albu.ISONoise(),\n        albu.MultiplicativeNoise()\n    ], p=1),\n    albu.Cutout(\n        num_holes = 8,\n        max_h_size = 16,\n        max_w_size = 16,\n        fill_value = 0,\n        p = 0.5\n    ),\n    albu.CLAHE(p=1),\n    albu.ToGray(p=1)\n    \n], bbox_params = {'format':'coco', 'label_fields': ['labels']})\n\nDataGenerator.val_augmentations = albu.Compose([\n    albu.CLAHE(p=1),\n    albu.ToGray(p=1)\n])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"'To generate data_generator'\n\n\ntrain_generator = DataGenerator(\n    train_image_ids,\n    train_images,\n    train_bboxes,\n    batch_size = 6,\n    shuffle = True,\n    augment = True\n)\n\nval_generator = DataGenerator(\n    valid_image_ids,\n    valid_images,\n    valid_bboxes,\n    batch_size = 1\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"x_input = tf.keras.Input(shape=(256,256,3))\n\nx = tf.keras.layers.Conv2D(32, (3, 3), strides=(1, 1), padding='same')(x_input)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n########## block 1 ##########\nx = tf.keras.layers.Conv2D(64, (3, 3), strides=(1, 1), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx_shortcut = x\n\nfor i in range(2):\n    x = tf.keras.layers.Conv2D(32, (3, 3), strides=(1, 1), padding='same')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x = tf.keras.layers.Conv2D(64, (3, 3), strides=(1, 1), padding='same')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x = tf.keras.layers.Add()([x_shortcut, x])\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x_shortcut = x\n\n\n########## block 2 ##########\nx = tf.keras.layers.Conv2D(128, (3, 3), strides=(1, 1), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx_shortcut = x\n\nfor i in range(2):\n    x = tf.keras.layers.Conv2D(64, (3, 3), strides=(1, 1), padding='same')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x = tf.keras.layers.Conv2D(128, (3, 3), strides=(1, 1), padding='same')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x = tf.keras.layers.Add()([x_shortcut, x])\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x_shortcut = x\n\n########## block 3 ##########\nx = tf.keras.layers.Conv2D(256, (3, 3), strides=(2, 2), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx_shortcut = x\n\nfor i in range(8):\n    x = tf.keras.layers.Conv2D(128, (3, 3), strides=(1, 1), padding='same')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x = tf.keras.layers.Conv2D(256, (3, 3), strides=(1, 1), padding='same')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x = tf.keras.layers.Add()([x_shortcut, x])\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x_shortcut = x\n\n    \n########## block 4 ##########\nx = tf.keras.layers.Conv2D(512, (3, 3), strides=(2, 2), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx_shortcut = x\n\nfor i in range(8):\n    x = tf.keras.layers.Conv2D(256, (3, 3), strides=(1, 1), padding='same')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x = tf.keras.layers.Conv2D(512, (3, 3), strides=(1, 1), padding='same')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x = tf.keras.layers.Add()([x_shortcut, x])\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x_shortcut = x\n\n########## block 5 ##########\nx = tf.keras.layers.Conv2D(1024, (3, 3), strides=(2, 2), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx_shortcut = x\n\nfor i in range(4):\n    x = tf.keras.layers.Conv2D(512, (3, 3), strides=(1, 1), padding='same')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x = tf.keras.layers.Conv2D(1024, (3, 3), strides=(1, 1), padding='same')(x)\n    x = tf.keras.layers.BatchNormalization()(x)\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x = tf.keras.layers.Add()([x_shortcut, x])\n    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\n    x_shortcut = x\n\n########## output layers ##########\nx = tf.keras.layers.Conv2D(512, (3, 3), strides=(1, 1), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx = tf.keras.layers.Conv2D(256, (3, 3), strides=(1, 1), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\nx = tf.keras.layers.Conv2D(128, (3, 3), strides=(1, 1), padding='same')(x)\nx = tf.keras.layers.BatchNormalization()(x)\nx = tf.keras.layers.LeakyReLU(alpha=0.1)(x)\n\npredictions = tf.keras.layers.Conv2D(5, (1, 1), strides=(2, 2), activation='sigmoid')(x)\n\nmodel = tf.keras.Model(inputs=x_input, outputs=predictions)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I refered https://arxiv.org/pdf/1506.02640.pdf to write the function ***loss\\_function***. It is Yolo v1 paper.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"'loss function'\n\ndef loss_function(y_true, y_pred):\n    \"\"\"\n    modified loss function that is refered to Yolo v1.\n    Some coefficents are adjusted for having a proper loss value by empirical try\n    \"\"\"\n\n    \n    OBJ_SCALE = 5\n    NO_OBJ_SCALE = 0.8\n    WH_SCALE = 1.5\n    \n    true_conf = y_true[...,0:1]\n    true_xy   = y_true[...,1:3]\n    true_wh   = y_true[...,3:] * WH_SCALE\n    \n    pred_conf = y_pred[...,0:1]\n    pred_xy   = y_pred[...,1:3]\n    pred_wh   = y_pred[...,3:] * WH_SCALE\n\n    obj_mask = tf.expand_dims(y_true[..., 0], axis = -1) * OBJ_SCALE\n    noobj_mask = (1 - obj_mask) * NO_OBJ_SCALE\n    \n    loss_xy    = tf.reduce_sum(tf.square((true_xy - pred_xy) * obj_mask))\n    loss_wh    = tf.reduce_sum(tf.square((true_wh - pred_wh) * obj_mask))\n    loss_obj   = tf.reduce_sum(tf.square((true_conf - pred_conf) * obj_mask))\n    loss_noobj = tf.reduce_sum(tf.square((true_conf - pred_conf) * noobj_mask))\n    \n    loss = loss_xy + loss_wh + loss_obj + loss_noobj\n    \n    tf.print('loss_xy', loss_xy)\n    tf.print('loss_wh', loss_wh)\n    tf.print('loss_obj', loss_obj)\n    tf.print('loss_noobj', loss_noobj)\n    tf.print('loss', loss)\n \n    return loss    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"optimizers = tf.keras.optimizers.Adam(learning_rate=0.00001)\n\nmodel.compile(\n    optimizer = optimizers,\n    loss = loss_function,\n    metrics = 'accuracy'\n)\n\n#history = model.fit(train_generator, validation_data = val_generator, epochs=5)\nmodel.load_weights('../input/first-model-yolov1/first model.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"A cell below is to comply with the output format. It reorganizes because all data is converted into 0 to 1 for training stability. But submission.csv is real size scaled by 1024. So It needs to scale up to 1024 size and filter grid cell with confidence score by threshold.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def prediction_to_submission(prediction, ids, threshold = 0.2):\n    \"\"\" The result of prediction doesn't have the same shape of submission.\n    So It scales up to 1024, \n    converts (centre_x, centre_y, width ,height) into (x, y, width, height), \n    groups boundary boxes of images by a relevant id.\n    \n    Args:\n        prediction: the result of prediction from the model\n        ids: Ids of the test images\n    returns:\n        prediction modified as the form of submission\n    \"\"\"\n    \n    grid_x = prediction.shape[1]\n    grid_y = prediction.shape[2]\n    \n    submission = {}\n    \n    for i, Id in enumerate(ids):\n        List = []\n        for j in range(grid_x):\n            for k in range(grid_y):\n                pred_ = prediction[i,j,k]\n                if pred_[0] > threshold:\n                    \n                    confidence = pred_[0]\n                    cell_x = 64 * k\n                    cell_y = 64 * j\n                    \n                    box_width = pred_[3] * 1024\n                    box_height = pred_[4] * 1024 \n                    \n                    box_x = cell_x + (pred_[1] * 64) - (box_width/2)\n                    box_y = cell_y + (pred_[2] * 64) - (box_height/2)\n                    \n                    List.append([confidence, box_x, box_y, box_width, box_height])\n                    \n        submission[Id] = List\n        \n    return submission\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To test if the model I traind works well, I evaluate the model to get prediction by evaluating with valid_data and try to visualize using ***prediction_to_submission***.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"'To get predictions. Because it takes long time, it needs to be separate for a cell below'\n\npredictions_val = model.predict(val_generator)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"'To try to visualize the val_predictions'\n\nid_in_valid = valid_image_ids[2]\nsubmission_val = prediction_to_submission(predictions_val, valid_image_ids, threshold=0.95)\nimage_1 = Image.open(f'{DIR_INPUT}/train/{id_in_valid}.jpg')\nimage_2 = image_1.copy()\n\nbbox_true = valid_bboxes[id_in_valid]\nimage_true = draw_boxes_on_image(image_1, bbox_true)\n\nbbox_pred = np.array(submission_val[id_in_valid])[:,1:]\nimage_pred = draw_boxes_on_image(image_2, bbox_pred)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1,2, figsize = (13,13))\n\nax[1].set_title('pred', fontsize = 17)\nax[1].set_xticks([])\nax[1].set_yticks([])\nax[1].imshow(image_pred)\n\nax[0].set_title('true', fontsize = 17)\nax[0].set_xticks([])\nax[0].set_yticks([])\nax[0].imshow(image_true)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In this picture, the model detects almost of wheats in the picture. But as you can see, It detects not wheat as wheats which seems my model has PF error quite a lot. And there are more than one red boxes on wheat. It needs to import non-max suppression.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"'submission'\n\ntest_albu = albu.Compose([\n    albu.CLAHE(p=1),\n    albu.ToGray(p=1)\n])\n\ntest_image_ids = os.listdir(f'{DIR_INPUT}/test/')\ntest_image_ids = [ Id[:-4] for Id in test_image_ids ]\ntest_images = []\n\nfor test_id in test_image_ids:\n    test_image = Image.open(f'{DIR_INPUT}/test/{test_id}.jpg').resize((256,256))\n    test_image = np.asarray(test_image)\n    test_augment = test_albu(image = test_image)\n    test_images.append(test_augment['image'])\n\ntest_images = np.asarray(test_images)/256\nprediction = model.predict(test_images)\nsubmission = prediction_to_submission(prediction, test_image_ids, threshold = 0.95)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In fact, The model doesn't learn for valid data in the training session. So the result of performance on test image would be similar to the result from valid data. Let's take a look.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"'To try to visualize the predictions'\n\nid_in_test = test_image_ids[0]\nbbox = np.array(submission[id_in_test])[:,1:]\nimage = Image.open(f'{DIR_INPUT}/test/{id_in_test}.jpg')\nimage = draw_boxes_on_image(image, bbox)\nplt.figure(figsize=(8,8))\nplt.imshow(image)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_list = []\nfor test_id in test_image_ids:\n    prediction_string = []\n    for pixel in submission[test_id]:\n        c,x,y,w,h = pixel\n        prediction_string.append(f'{c} {x} {y} {w} {h}')\n    prediction_string = ' '.join(prediction_string)\n    submission_list.append([test_id, prediction_string])\n\nfinal_submission = pd.DataFrame(submission_list , columns = ['image_id', 'PredictionString'])        \n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#final_submission.to_csv('submission.csv', index = False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Unfortunately, I missed due data of submission in this competition. So I don't know the final score. As compared with true and pred above, I can guess what I have to improve in the model. First, I should deal with positive false which means the model says it is wheat but not actually. There are a lot of boxes that indicate the wrong place. Of course, It is common that the model has pf. But the thing is the overall average value of pf is pretty high. It is not good to solve the problem. To overcome this problem, I should raise the coefficient value that is called no-object lambda. Second, this model doesn't take account of the color in the image. Because I converted all images as a grayscale in the data augmentation.  It would make inaccuracy. ","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}