{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"### Imagine accelerating research on almost all diseases, from lung cancer and heart disease to rare diseases. Data Science Bowl offers our most ambitious mission: to create an algorithm for automatic detection of cores. ","metadata":{}},{"cell_type":"markdown","source":"### This dataset contains a large number of segmented nuclei images. The images were acquired under a variety of conditions and vary in the cell type, magnification, and imaging modality (brightfield vs. fluorescence). The dataset is designed to challenge an algorithm's ability to generalize across these variations ","metadata":{}},{"cell_type":"markdown","source":" -------------------","metadata":{}},{"cell_type":"code","source":"import os\nimport zipfile\n\nimport numpy as np\nimport matplotlib.pyplot as plt\n\nfrom sklearn.model_selection import train_test_split\n\nimport cv2\n\nimport tensorflow as tf\nfrom keras import backend as K\n\nfrom tensorflow.keras.layers import Input, concatenate, Conv2D, MaxPooling2D, UpSampling2D, Dropout\nfrom tensorflow.keras import models\nfrom tensorflow.keras import optimizers\nfrom tensorflow.keras.preprocessing import image\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:44:41.388969Z","iopub.execute_input":"2022-02-22T23:44:41.389975Z","iopub.status.idle":"2022-02-22T23:44:47.300230Z","shell.execute_reply.started":"2022-02-22T23:44:41.389861Z","shell.execute_reply":"2022-02-22T23:44:47.299458Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with zipfile.ZipFile(\"../input/data-science-bowl-2018/stage1_train.zip\",'r') as z:\n z.extractall(\"stage1_train\")\n\nwith zipfile.ZipFile(\"../input/data-science-bowl-2018/stage2_test_final.zip\",'r') as z:\n z.extractall(\"stage2_test_final\")","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:44:47.301855Z","iopub.execute_input":"2022-02-22T23:44:47.302084Z","iopub.status.idle":"2022-02-22T23:44:58.721452Z","shell.execute_reply.started":"2022-02-22T23:44:47.302051Z","shell.execute_reply":"2022-02-22T23:44:58.720718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_path = '/kaggle/working/stage1_train/'\ntest_path = '/kaggle/working/stage2_test_final/'","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:44:58.724527Z","iopub.execute_input":"2022-02-22T23:44:58.724730Z","iopub.status.idle":"2022-02-22T23:44:58.730892Z","shell.execute_reply.started":"2022-02-22T23:44:58.724706Z","shell.execute_reply":"2022-02-22T23:44:58.730194Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_dir = os.listdir(train_path)\ntest_dir = os.listdir(test_path)","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:44:58.732818Z","iopub.execute_input":"2022-02-22T23:44:58.733585Z","iopub.status.idle":"2022-02-22T23:44:58.751512Z","shell.execute_reply.started":"2022-02-22T23:44:58.733549Z","shell.execute_reply":"2022-02-22T23:44:58.750822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n---------------------------","metadata":{}},{"cell_type":"markdown","source":"# Loading data\n\n","metadata":{}},{"cell_type":"markdown","source":"First, let's create empty tensors for future images","metadata":{}},{"cell_type":"code","source":"X_train = np.zeros((len(train_dir), 256, 256, 3), dtype=np.uint8)\nY_train = np.zeros((len(train_dir), 256, 256, 1), dtype=bool)\n\nX_test = np.zeros((len(test_dir), 256, 256, 3), dtype=np.uint8)","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:44:58.752788Z","iopub.execute_input":"2022-02-22T23:44:58.753157Z","iopub.status.idle":"2022-02-22T23:44:58.761039Z","shell.execute_reply.started":"2022-02-22T23:44:58.753117Z","shell.execute_reply":"2022-02-22T23:44:58.760345Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The directory structure looks like each **Spot Nuclei** has its own folder.\n\nThe folder with **id Spot Nuclei** contains two folders: **images** and **masks**\n\nimages contains the original **Spot Nuclei image**. We need to write it **into a pre-created tensor X**\n\nmasks contains several images with **different parts of the masks**. We need to **merge all the masks into one image** using **np.maximum**. Further, just as with the original image, write to the **pre-created tensor Y**\n\nWe will **compress all images** to a size of **256x256x3**, however, we will set the size of **mask images** to **256x256x1**, since it makes no sense for us to store them in **rgb format**","metadata":{}},{"cell_type":"code","source":"%%time\nfor i, name in enumerate(train_dir):\n path = train_path + name\n img_real = cv2.imread(path+'/images/'+ name +'.png')\n img_real = cv2.resize(img_real,(256,256))\n X_train[i] = img_real\n \n img_segment_full = np.zeros((256, 256 , 1), dtype=bool)\n segment_path = path+'/masks/'\n for name in os.listdir(segment_path):\n img_segment = cv2.imread(segment_path + name, 0)\n img_segment = cv2.resize(img_segment, (256, 256))\n img_segment = np.expand_dims(img_segment, axis=-1)\n img_segment_full = np.maximum(img_segment_full, img_segment)\n \n Y_train[i] = img_segment_full","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:44:58.763134Z","iopub.execute_input":"2022-02-22T23:44:58.763388Z","iopub.status.idle":"2022-02-22T23:45:23.640025Z","shell.execute_reply.started":"2022-02-22T23:44:58.763355Z","shell.execute_reply":"2022-02-22T23:45:23.639239Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nfor i, name in enumerate(test_dir):\n path = test_path + name\n img_real = cv2.imread(path+'/images/'+ name +'.png')\n img_real = cv2.resize(img_real, (256,256))\n X_test[i] = img_real","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:23.641438Z","iopub.execute_input":"2022-02-22T23:45:23.641861Z","iopub.status.idle":"2022-02-22T23:45:36.165467Z","shell.execute_reply.started":"2022-02-22T23:45:23.641822Z","shell.execute_reply":"2022-02-22T23:45:36.163965Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Ok, let's see the result.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(15,8))\nplt.subplot(121)\nplt.imshow(X_train[2])\nplt.title('Real image')\nplt.subplot(122)\nplt.imshow(Y_train[2])\nplt.title('Segmentation image');","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:36.166605Z","iopub.execute_input":"2022-02-22T23:45:36.167281Z","iopub.status.idle":"2022-02-22T23:45:36.596907Z","shell.execute_reply.started":"2022-02-22T23:45:36.167242Z","shell.execute_reply":"2022-02-22T23:45:36.596261Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Looks good, now let's work on augmenting our images\n\n\n--------------------------------","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":"# Data Preprocessing\n\n\n\n\n\n-------------------","metadata":{}},{"cell_type":"markdown","source":"**U-net** does not require a large amount of data, as for example for the classification task, however, there are only **670 data** in the train, I believe that **augmentation** here can improve the segmentation result","metadata":{}},{"cell_type":"code","source":"aug_gen_args = dict(shear_range = 0.2,\n zoom_range = 0.2,\n rotation_range=40,\n width_shift_range=0.2,\n height_shift_range=0.2,\n horizontal_flip=True,\n vertical_flip=True,\n fill_mode='reflect'\n )\n\nX_train_gen = ImageDataGenerator(**aug_gen_args)\ny_train_gen = ImageDataGenerator(**aug_gen_args)\nX_val_gen = ImageDataGenerator()\ny_val_gen = ImageDataGenerator()","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:36.597832Z","iopub.execute_input":"2022-02-22T23:45:36.598070Z","iopub.status.idle":"2022-02-22T23:45:36.605102Z","shell.execute_reply.started":"2022-02-22T23:45:36.598038Z","shell.execute_reply":"2022-02-22T23:45:36.604167Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To begin with, let's demonstrate on one image what augmentation looks like.\n\nIt is important to set the **seed** and **shuffle=False** in order for the **original augmented images** to match the **segmented augmented images**.","metadata":{}},{"cell_type":"code","source":"aug_image_real = X_train[5].reshape((1,)+X_train[1].shape)\naug_image_seg = Y_train[5].reshape((1,)+Y_train[1].shape)","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:36.609209Z","iopub.execute_input":"2022-02-22T23:45:36.609674Z","iopub.status.idle":"2022-02-22T23:45:36.614859Z","shell.execute_reply.started":"2022-02-22T23:45:36.609637Z","shell.execute_reply":"2022-02-22T23:45:36.614137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"aug_image_real_check = X_train_gen.flow(aug_image_real, batch_size=1, seed=17, shuffle=False)\naug_image_seg_check = y_train_gen.flow(aug_image_seg, batch_size=1, seed=17, shuffle=False)","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:36.616312Z","iopub.execute_input":"2022-02-22T23:45:36.616589Z","iopub.status.idle":"2022-02-22T23:45:36.624185Z","shell.execute_reply.started":"2022-02-22T23:45:36.616554Z","shell.execute_reply":"2022-02-22T23:45:36.623581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15,10))\nplt.subplot(141)\nplt.imshow(X_train[5])\nplt.title(\"original\")\ni=2\nfor batch in aug_image_real_check:\n plt.subplot(14*10+i)\n plt.imshow(image.array_to_img(batch[0]))\n plt.title(\"augmented\")\n i += 1\n if i % 5 == 0:\n break","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:36.625261Z","iopub.execute_input":"2022-02-22T23:45:36.625481Z","iopub.status.idle":"2022-02-22T23:45:37.559755Z","shell.execute_reply.started":"2022-02-22T23:45:36.625457Z","shell.execute_reply":"2022-02-22T23:45:37.559072Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15,10))\nplt.subplot(141)\nplt.imshow(Y_train[5])\nplt.title(\"original\")\ni=2\nfor batch in aug_image_seg_check:\n plt.subplot(14*10+i)\n plt.imshow(image.array_to_img(batch[0]))\n plt.title(\"augmented\")\n i += 1\n if i % 5 == 0:\n break","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:37.560800Z","iopub.execute_input":"2022-02-22T23:45:37.561176Z","iopub.status.idle":"2022-02-22T23:45:38.081918Z","shell.execute_reply.started":"2022-02-22T23:45:37.561140Z","shell.execute_reply":"2022-02-22T23:45:38.081226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Looks good, now let's split **X_train** into **2 parts**: **train** and **val**. For val of the selection, select **0.1 data size**\n\nNext, **apply the augmentation generators** to the train data; no augmentation will be performed for the val data\n\nSince we need to do augmentation for original images and segmented images, then we **need to combine the generators** into one using the **zip()**","metadata":{}},{"cell_type":"code","source":"train, val, y_train, y_val = train_test_split(X_train, Y_train, test_size=0.1, random_state=17)","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:38.083354Z","iopub.execute_input":"2022-02-22T23:45:38.083843Z","iopub.status.idle":"2022-02-22T23:45:38.142451Z","shell.execute_reply.started":"2022-02-22T23:45:38.083793Z","shell.execute_reply":"2022-02-22T23:45:38.141706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_gen.fit(train, augment=True, seed=17)\ny_train_gen.fit(y_train, augment=True, seed=17)\nX_val_gen.fit(val, seed=17)\ny_val_gen.fit(y_val, seed=17)\n\nX_train_generator = X_train_gen.flow(train, batch_size=16, seed=17, shuffle=False)\ny_train_generator = y_train_gen.flow(y_train, batch_size=16, seed=17, shuffle=False)\nX_val_generator = X_val_gen.flow(val, batch_size=16, seed=17, shuffle=False)\ny_val_generator = y_val_gen.flow(y_val, batch_size=16, seed=17, shuffle=False)\n\ntrain_generator = zip(X_train_generator, y_train_generator)\nval_generator = zip(X_val_generator, y_val_generator)","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:38.143955Z","iopub.execute_input":"2022-02-22T23:45:38.144220Z","iopub.status.idle":"2022-02-22T23:45:48.596491Z","shell.execute_reply.started":"2022-02-22T23:45:38.144184Z","shell.execute_reply":"2022-02-22T23:45:48.595763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Great, now let's create a custom metric that is required in this competition\n\n\n--------------------------","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":"# Creating custom metric and loss","metadata":{}},{"cell_type":"markdown","source":"### IoU metric","metadata":{}},{"cell_type":"markdown","source":"This competition is evaluated on the mean average precision at different intersection over union (IoU) thresholds.\n\nI have seen implementations in other kernels, they all use the built-in **tf.keras.metrics.MeanIoU** function to calculate **IoU**, and then average the results over **thresholds of 0.05**.\n\nI'm not sure that it works correctly and I tried to implement my own version of the **MeanIoU** metric for the competition.\n\nFirst, I wrote a function to calculate IoU:\n\n1) Calculate intersection\n2) Calculate union\n\n**P.S** In many IoU implementations on the Internet, I have not seen people subtract intersection from union, however, I think this is necessary, since we sum the intersection part 2 times instead of one. In this way, we stick with the set.union() implementation for sets.\n\n**!!!!CORRECT ME IF I AM WRONG!!!!**\n\n3) Calculate intersection/union\n\n","metadata":{}},{"cell_type":"markdown","source":"To calculate **mean_iou**, we calculate the **IoU for each threshold using a loop**, passing **t_y_pred** of type **float32**\n\n**P.S** I don’t know why, but the model reports an error if we pass any other data type to it, although we still get int values at the output\n\n**!!!!!CORRECT ME IF I AM WRONG!!!!!**\n\nafter counting the values of all threshold values t, we **average and return the result**","metadata":{}},{"cell_type":"code","source":"def iou(y_true, y_pred):\n intersection = K.sum(K.abs(y_true * y_pred), axis=[1,2,3])\n union = K.sum(y_true, axis=[1,2,3]) + K.sum(y_pred, axis=[1,2,3]) - intersection\n iou = K.mean((intersection + 1) / (union + 1), axis=0)\n return iou","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:48.597926Z","iopub.execute_input":"2022-02-22T23:45:48.598162Z","iopub.status.idle":"2022-02-22T23:45:48.603428Z","shell.execute_reply.started":"2022-02-22T23:45:48.598128Z","shell.execute_reply":"2022-02-22T23:45:48.602701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def mean_iou(y_true, y_pred):\n results = [] \n for t in np.arange(0.5, 1, 0.05):\n t_y_pred = tf.cast((y_pred > t), tf.float32)\n pred = iou(y_true, t_y_pred)\n results.append(pred)\n \n return K.mean(K.stack(results), axis=0)","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:48.604749Z","iopub.execute_input":"2022-02-22T23:45:48.605203Z","iopub.status.idle":"2022-02-22T23:45:48.613655Z","shell.execute_reply.started":"2022-02-22T23:45:48.605167Z","shell.execute_reply":"2022-02-22T23:45:48.612949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Dice loss","metadata":{}},{"cell_type":"markdown","source":"Also for U-net, **dice_loss** or **binary crossentropy** is most often used, in this work I tried to use **dise_loss**\n\nHowever, **IoU_loss** could also be used, but as I understand it, this metric **cannot be differentiated**\n\n**!!!!!CORRECT ME IF I AM WRONG!!!!!**","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"def dice_loss(y_true, y_pred):\n intersection = K.sum(K.abs(y_true * y_pred), axis=[1,2,3])\n union = K.sum(y_true, axis=[1,2,3]) + K.sum(y_pred, axis=[1,2,3])\n dice = K.mean((2. * intersection + 1) / (union + 1), axis=0)\n return 1. - dice","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:48.614671Z","iopub.execute_input":"2022-02-22T23:45:48.614910Z","iopub.status.idle":"2022-02-22T23:45:48.623560Z","shell.execute_reply.started":"2022-02-22T23:45:48.614879Z","shell.execute_reply":"2022-02-22T23:45:48.622868Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Great, now let's write the basic architecture of the model\n\n\n-------------------------","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":"# U-NET model\n\n","metadata":{}},{"cell_type":"markdown","source":"1. We feed the **Input() layer** with the size of the input images to the **U-NET input**, then using **Lambda** we **normalize** our image to a network-friendly range **(0:1)**\n\nU-net architecture can be divided into two parts: **encoder and decoder**\n\n2. **ENCODER**\n\n encoder is a typical CNN architecture. It consists of:\n\n* encoder consists of reapplying **Conv2D 3x3**;\n* In order not to lose border pixels during convolution, we use **padding**;\n* On each layer we apply **ReLU activation**;\n* Next comes **MaxPooling2D 2x2** for downsampling;\n* At each encoder step, the filter channels are doubled, we'll **start at 32 and end at 512**;\n\n3. **DECODER**\n\nEach step in the expanding path consists of a property map upsampling operation. It consists of:\n\n* **UpSampling2D 2×2**, which reduces the number of filter channels;\n* **concatenate** to link layers to the same property map;\n* Re-apply **Conv2D 3×3**;\n* In order not to lose border pixels during convolution, we use **padding**;\n* On each layer we apply **ReLU activation**;\n* At each step of the encoder, the filter channels are halved, we will **start at 512 and end at 32**;\n\n\n4. The last layer uses **Conv2D 1x1** to map each feature vector to the **desired class**.","metadata":{}},{"cell_type":"code","source":"inputs = Input((256, 256, 3))\ns = tf.keras.layers.Lambda(lambda x: x/255.0)(inputs)\n\nconv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(inputs)\nconv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(conv1)\npool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n\nconv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(pool1)\nconv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(conv2)\npool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n\nconv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(pool2)\nconv3 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv3)\npool3 = MaxPooling2D(pool_size=(2, 2))(conv3)\n\nconv4 = Conv2D(256, (3, 3), activation='relu', padding='same')(pool3)\nconv4 = Conv2D(256, (3, 3), activation='relu', padding='same')(conv4)\npool4 = MaxPooling2D(pool_size=(2, 2))(conv4)\n\nconv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(pool4)\nconv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(conv5)\nconv5 = Conv2D(512, (3, 3), activation='relu', padding='same')(conv5)\n\nup6 = UpSampling2D(size=(2,2))(conv5)\nup6 = concatenate([up6, conv4])\nconv6 = Conv2D(256, (3, 3), activation='relu', padding='same')(up6)\nconv6 = Conv2D(256, (3, 3), activation='relu', padding='same')(conv6)\n\nup7 = UpSampling2D(size=(2,2))(conv6)\nup7 = concatenate([up7, conv3])\nconv7 = Conv2D(128, (3, 3), activation='relu', padding='same')(up7)\nconv7 = Conv2D(128, (3, 3), activation='relu', padding='same')(conv7)\n\nup8 = UpSampling2D(size=(2,2))(conv7)\nup8 = concatenate([up8, conv2])\nconv8 = Conv2D(64, (3, 3), activation='relu', padding='same')(up8)\nconv8 = Conv2D(64, (3, 3), activation='relu', padding='same')(conv8)\n\nup9 = UpSampling2D(size=(2,2))(conv8)\nup9 = concatenate([up9, conv1])\nconv9 = Conv2D(32, (3, 3), activation='relu', padding='same')(up9)\nconv9 = Conv2D(32, (3, 3), activation='relu', padding='same')(conv9)\n\nconv10 = Conv2D(1, (1, 1), activation='sigmoid')(conv9)\n\nmodel = models.Model(inputs=[inputs], outputs=[conv10])\n\nmodel.compile(optimizer=optimizers.Adam(learning_rate=2e-4), loss=dice_loss, metrics=mean_iou)","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:48.625790Z","iopub.execute_input":"2022-02-22T23:45:48.626028Z","iopub.status.idle":"2022-02-22T23:45:51.216964Z","shell.execute_reply.started":"2022-02-22T23:45:48.626002Z","shell.execute_reply":"2022-02-22T23:45:51.216276Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"using **optimizer='Adam'** as the most basic and recommended, but for **better accuracy**, I **lowered the learning_rate**\n\n**loss='dice_loss'** we defined it with the **dice_loss** function earlier\n\n**metrics='mean_iou'** we defined it with the **mean_iou** function earlier","metadata":{}},{"cell_type":"code","source":"model.summary()","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:51.218467Z","iopub.execute_input":"2022-02-22T23:45:51.218711Z","iopub.status.idle":"2022-02-22T23:45:51.239728Z","shell.execute_reply.started":"2022-02-22T23:45:51.218678Z","shell.execute_reply":"2022-02-22T23:45:51.239064Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"------------------------------------------","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":"# U-net model training\n\n---------------------","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"Ok, now let's start training our model.\n\nPass **train and val generators**\n\nSet the **train/val epoch size** as **len(train/val)/bath_size**\n\nLet's set **epochs=25**, I think this is enough to get good results\n\nP.S. I won't use callbacks as a model with such a small amount of data and 25 epochs will learn quickly.","metadata":{}},{"cell_type":"code","source":"history = model.fit(train_generator,\n steps_per_epoch=len(train)/8,\n validation_data=val_generator,\n validation_steps=len(val)/8,\n epochs=25\n )","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:45:51.241335Z","iopub.execute_input":"2022-02-22T23:45:51.241582Z","iopub.status.idle":"2022-02-22T23:56:26.810877Z","shell.execute_reply.started":"2022-02-22T23:45:51.241548Z","shell.execute_reply":"2022-02-22T23:56:26.810128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss = history.history['mean_iou']\nval_loss = history.history['val_mean_iou']\n\nplt.figure(figsize=(15,10))\nplt.plot(loss, label='Train IOU')\nplt.plot(val_loss,'--', label='Val IOU')\nplt.title('Training and Validation mean IOU')\nplt.yticks(np.arange(0.5, 1, 0.05))\nplt.xticks(np.arange(0, 25))\nplt.grid()\nplt.legend();","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:56:26.812715Z","iopub.execute_input":"2022-02-22T23:56:26.812969Z","iopub.status.idle":"2022-02-22T23:56:27.151875Z","shell.execute_reply.started":"2022-02-22T23:56:26.812935Z","shell.execute_reply":"2022-02-22T23:56:27.151218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Great, we see that the average result for **Val_IoU is ~0.85** **Train_IoU ~0.8**\n","metadata":{}},{"cell_type":"markdown","source":"\nLet's take a look at the results in images\n\n------------------------------------","metadata":{}},{"cell_type":"markdown","source":"# Result on train/val data ","metadata":{}},{"cell_type":"code","source":"train_pred = model.predict(train, verbose = 1)\nval_pred = model.predict(val, verbose=1)","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:56:27.153222Z","iopub.execute_input":"2022-02-22T23:56:27.153708Z","iopub.status.idle":"2022-02-22T23:56:35.054540Z","shell.execute_reply.started":"2022-02-22T23:56:27.153669Z","shell.execute_reply":"2022-02-22T23:56:35.053730Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15,10))\nplt.subplot(131)\nplt.imshow(train[1])\nplt.title('Original image')\nplt.subplot(132)\nplt.imshow(np.squeeze(y_train[1]))\nplt.title('Segmented image')\nplt.subplot(133)\nplt.imshow(np.squeeze(train_pred[1]))\nplt.title('Predicted image');","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:56:35.059447Z","iopub.execute_input":"2022-02-22T23:56:35.061748Z","iopub.status.idle":"2022-02-22T23:56:35.622242Z","shell.execute_reply.started":"2022-02-22T23:56:35.061705Z","shell.execute_reply":"2022-02-22T23:56:35.621390Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15,10))\nplt.subplot(131)\nplt.imshow(val[3])\nplt.title('Original image')\nplt.subplot(132)\nplt.imshow(np.squeeze(y_val[3]))\nplt.title('Segmented image')\nplt.subplot(133)\nplt.imshow(np.squeeze(val_pred[3]))\nplt.title('Predicted image');","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:56:35.623882Z","iopub.execute_input":"2022-02-22T23:56:35.624132Z","iopub.status.idle":"2022-02-22T23:56:36.116855Z","shell.execute_reply.started":"2022-02-22T23:56:35.624099Z","shell.execute_reply":"2022-02-22T23:56:36.116198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Great, it looks like the neural network really works, now let's **predict** the result for **X_test data**.\n\nI won't be submitting on kaggle as the competition is over. The purpose of this notebook is to learn how to use **U-net for image segmentation**","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":" -------------------------","metadata":{}},{"cell_type":"markdown","source":"# Result on test data","metadata":{}},{"cell_type":"code","source":"test_pred = model.predict(X_test, verbose=1)","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:56:36.117919Z","iopub.execute_input":"2022-02-22T23:56:36.118270Z","iopub.status.idle":"2022-02-22T23:56:45.757510Z","shell.execute_reply.started":"2022-02-22T23:56:36.118238Z","shell.execute_reply":"2022-02-22T23:56:45.756744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(15, 32))\nfor i in range(421, 429):\n plt.subplot(i)\n if i % 2!=0:\n plt.imshow(X_test[i])\n plt.title('Original image')\n else:\n plt.imshow(np.squeeze(test_pred[i-1]))\n plt.title('Predicted image')","metadata":{"execution":{"iopub.status.busy":"2022-02-22T23:56:45.758835Z","iopub.execute_input":"2022-02-22T23:56:45.759086Z","iopub.status.idle":"2022-02-22T23:56:47.403777Z","shell.execute_reply.started":"2022-02-22T23:56:45.759049Z","shell.execute_reply":"2022-02-22T23:56:47.402334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" --------------------","metadata":{}},{"cell_type":"markdown","source":"## Thank you for watching this is my project, I will be grateful if you upvoted and give feedback about my work in the comments. I want to improve my skills, and if you find any mistakes in the work, please tell me about it. ","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}}]}