{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.9","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":8089,"databundleVersionId":44321,"sourceType":"competition"},{"sourceId":19989,"databundleVersionId":1160143,"sourceType":"competition"},{"sourceId":7081733,"sourceType":"datasetVersion","datasetId":4079653}],"dockerImageVersionId":30056,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
Image Segmentation
\n
Using U-Net with Keras
\n
\n
","metadata":{}},{"cell_type":"markdown","source":"**Objective**\n\n> This notebook is dedicated to implementing the U-Net algorithm, a specialized convolutional neural network architecture, to perform a task of image segmentation accurately and efficiently.","metadata":{}},{"cell_type":"markdown","source":"##
Table of contents
\n\n**1. Introduction**\n**
2. Load Data**\n**
3. Get Masks**\n**
4. Training Model**\n**
5. Predict and Visualize**","metadata":{}},{"cell_type":"markdown","source":"#

Introduction

","metadata":{}},{"cell_type":"markdown","source":"**What is a a U-net ?**\n\n> The U-Net is a type of convolutional neural network (CNN) that's specifically designed for swift and accurate segmentation of images.","metadata":{}},{"cell_type":"markdown","source":"\n\n![](https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/u-net-architecture.png)\n\n> The illustration depicts the U-Net architecture, using 32x32 pixels at the lowest resolution as an example. In this diagram, each blue box represents a multi-channel feature map. The quantity of channels for each feature map is indicated above the corresponding box, while the spatial dimensions (x-y size) are indicated at the lower left corner of the box. Boxes in white denote feature maps that have been copied from a previous layer. The arrows present in the diagram symbolize various operations carried out in the architecture.\n>\n> Please note that while this illustration serves as an example of the U-Net architecture, the actual dimensions or layout of the layers can vary, depending on specific use cases or modifications to the architecture.","metadata":{}},{"cell_type":"markdown","source":"

Setup

","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nimport zipfile\nimport numpy as np\nimport random\nimport os\nfrom tqdm import tqdm\nfrom skimage.io import imread, imshow\nfrom skimage.transform import resize\nimport matplotlib.pyplot as plt\n\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-29T10:39:11.900573Z","iopub.execute_input":"2023-11-29T10:39:11.900972Z","iopub.status.idle":"2023-11-29T10:39:17.637125Z","shell.execute_reply.started":"2023-11-29T10:39:11.900877Z","shell.execute_reply":"2023-11-29T10:39:17.636363Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"SEED = 42\nnp.random.seed = SEED\n\nUNZIP_PATH = '../input/data-science-bowl-2018/'\nTRAIN_PATH = './train/'\nTEST_PATH = './test/'\n\nIMG_WIDTH = 128\nIMG_HEIGHT = 128\nIMG_CHANNELS = 3","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-06-16T15:12:49.30224Z","iopub.execute_input":"2023-06-16T15:12:49.302647Z","iopub.status.idle":"2023-06-16T15:12:49.308795Z","shell.execute_reply.started":"2023-06-16T15:12:49.302607Z","shell.execute_reply":"2023-06-16T15:12:49.306861Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Import data

","metadata":{}},{"cell_type":"code","source":"# Unzip data\nwith zipfile.ZipFile(UNZIP_PATH + 'stage1_train.zip', 'r') as zip_ref:\n zip_ref.extractall('./train')\n \nwith zipfile.ZipFile(UNZIP_PATH + 'stage1_test.zip', 'r') as zip_ref:\n zip_ref.extractall('./test')","metadata":{"execution":{"iopub.status.busy":"2023-06-16T15:12:51.253364Z","iopub.execute_input":"2023-06-16T15:12:51.253736Z","iopub.status.idle":"2023-06-16T15:12:59.46112Z","shell.execute_reply.started":"2023-06-16T15:12:51.253699Z","shell.execute_reply":"2023-06-16T15:12:59.460103Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get list of all subfolders\ntrain_ids = next(os.walk(TRAIN_PATH))[1]\ntest_ids = next(os.walk(TEST_PATH))[1]","metadata":{"execution":{"iopub.status.busy":"2023-06-16T15:18:38.026812Z","iopub.execute_input":"2023-06-16T15:18:38.02728Z","iopub.status.idle":"2023-06-16T15:18:38.035564Z","shell.execute_reply.started":"2023-06-16T15:18:38.027242Z","shell.execute_reply":"2023-06-16T15:18:38.034491Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# define placeholders (also used to replace NaN in images to resize by 0)\nX_train = np.zeros((len(train_ids), IMG_HEIGHT, IMG_WIDTH, IMG_CHANNELS), dtype = np.uint8)\nY_train = np.zeros((len(train_ids), IMG_HEIGHT, IMG_WIDTH, 1), dtype = np.bool)","metadata":{"execution":{"iopub.status.busy":"2023-06-16T15:19:01.732444Z","iopub.execute_input":"2023-06-16T15:19:01.732855Z","iopub.status.idle":"2023-06-16T15:19:01.73953Z","shell.execute_reply.started":"2023-06-16T15:19:01.732806Z","shell.execute_reply":"2023-06-16T15:19:01.738359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#

Get groundtruths

","metadata":{}},{"cell_type":"code","source":"# Loop through all train_ids\nfor n, id_ in tqdm(enumerate(train_ids), total=len(train_ids)): \n \n # Get the path to the image and mask directories\n path = TRAIN_PATH + id_ \n \n # Read the image and keep the first IMG_CHANNELS channels\n img = imread(path + '/images/' + id_ + '.png')[:, :, :IMG_CHANNELS] \n \n # Resize the image to IMG_HEIGHT x IMG_WIDTH while keeping the pixel values (mode='constant')\n img = resize(img, (IMG_HEIGHT, IMG_WIDTH), mode='constant', preserve_range=True) \n \n # Store the processed image into the X_train array at index n\n X_train[n] = img \n \n # Initialize an empty mask of dimensions IMG_HEIGHT x IMG_WIDTH x 1 with boolean values\n mask = np.zeros((IMG_HEIGHT, IMG_WIDTH, 1), dtype=np.bool) \n \n # Loop through each file in the masks directory\n for mask_file in next(os.walk(path + '/masks/'))[2]: \n \n # Read the mask file\n mask_ = imread(path + '/masks/' + mask_file)\n \n # Resize the mask to IMG_HEIGHT x IMG_WIDTH while keeping the pixel values (mode='constant'), and add an extra dimension\n mask_ = np.expand_dims(resize(mask_, (IMG_HEIGHT, IMG_WIDTH), mode='constant', preserve_range=True), axis=-1)\n \n # Merge all the masks associated with the same image by taking the maximum value at each pixel location\n mask = np.maximum(mask, mask_)\n \n # Store the processed mask into the Y_train array at index n\n Y_train[n] = mask","metadata":{"execution":{"iopub.status.busy":"2023-06-16T15:19:17.344706Z","iopub.execute_input":"2023-06-16T15:19:17.345131Z","iopub.status.idle":"2023-06-16T15:19:25.630071Z","shell.execute_reply.started":"2023-06-16T15:19:17.345084Z","shell.execute_reply":"2023-06-16T15:19:25.628041Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create an empty array for test images with dimensions as (number of test images, height, width, channels)\n# The datatype for the array is uint8 which can hold values from 0 to 255\nX_test = np.zeros((len(test_ids), IMG_HEIGHT, IMG_WIDTH, IMG_CHANNELS), dtype=np.uint8)\n\n# Initialize an empty list to store the original sizes of the test images\nsizes_test = []\n\n# Loop through all the test_ids\nfor n, id_ in tqdm(enumerate(test_ids), total=len(test_ids)):\n \n # Get the path to the test image\n path = TEST_PATH + id_\n \n # Read the image and keep the first IMG_CHANNELS channels\n img = imread(path + '/images/' + id_ + '.png')[:, :, :IMG_CHANNELS]\n \n # Append the original size of the image to sizes_test list\n sizes_test.append([img.shape[0], img.shape[1]])\n \n # Resize the image to IMG_HEIGHT x IMG_WIDTH while keeping the pixel values\n img = resize(img, (IMG_HEIGHT, IMG_WIDTH), mode='constant', preserve_range=True)\n \n # Store the processed image into the X_test array at index n\n X_test[n] = img","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:54:35.235323Z","iopub.execute_input":"2021-10-03T19:54:35.23567Z","iopub.status.idle":"2021-10-03T19:54:36.389158Z","shell.execute_reply.started":"2021-10-03T19:54:35.235634Z","shell.execute_reply":"2021-10-03T19:54:36.388322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#

Training Unet

","metadata":{}},{"cell_type":"code","source":"# Create a function for a convolution block\ndef conv_block(inputs, num_filters):\n x = tf.keras.layers.Conv2D(num_filters, (3, 3), activation=\"relu\", \n kernel_initializer=\"he_normal\", padding=\"same\")(inputs)\n x = tf.keras.layers.Dropout(0.1)(x)\n x = tf.keras.layers.Conv2D(num_filters, (3, 3), activation=\"relu\", \n kernel_initializer=\"he_normal\", padding=\"same\")(x)\n return x\n\n# Create a function for the expanding path\ndef upsample_block(inputs, conv_prev, num_filters):\n up = tf.keras.layers.Conv2DTranspose(num_filters, (2, 2), strides=(2, 2), padding=\"same\")(inputs)\n concat = tf.keras.layers.concatenate([up, conv_prev])\n conv = conv_block(concat, num_filters)\n return conv","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Inputs\ninputs = tf.keras.layers.Input((IMG_WIDTH, IMG_HEIGHT, IMG_CHANNELS))\n\n# Normalization\ns = tf.keras.layers.Lambda(lambda x: x/255.0)(inputs) \n\n# Contraction path\nc1 = conv_block(s, 16)\np1 = tf.keras.layers.MaxPooling2D((2, 2))(c1)\n\nc2 = conv_block(p1, 32)\np2 = tf.keras.layers.MaxPooling2D((2, 2))(c2)\n\nc3 = conv_block(p2, 64)\np3 = tf.keras.layers.MaxPooling2D((2, 2))(c3)\n\nc4 = conv_block(p3, 128)\np4 = tf.keras.layers.MaxPooling2D((2, 2))(c4)\n\nc5 = conv_block(p4, 256)\n\n# Expansive path\nc6 = upsample_block(c5, c4, 128)\nc7 = upsample_block(c6, c3, 64)\nc8 = upsample_block(c7, c2, 32)\nc9 = upsample_block(c8, c1, 16)\n\n# Output layer\noutputs = tf.keras.layers.Conv2D(1, (1, 1), activation='sigmoid')(c9)","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:54:36.391636Z","iopub.execute_input":"2021-10-03T19:54:36.392002Z","iopub.status.idle":"2021-10-03T19:54:39.066794Z","shell.execute_reply.started":"2021-10-03T19:54:36.391965Z","shell.execute_reply":"2021-10-03T19:54:39.06594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Model\nmodel = tf.keras.Model(inputs=[inputs], outputs=[outputs])\n\n# Compilation\nmodel.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\n\n# Summary\nmodel.summary()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Callbacks\ncheckpoint = tf.keras.callbacks.ModelCheckpoint('model.h5', verbose = 1, save_best_only = True)\ncallbacks = [\n tf.keras.callbacks.EarlyStopping(patience = 5, monitor = 'val_loss'),\n tf.keras.callbacks.TensorBoard(log_dir = 'logs')\n ]","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:54:39.070577Z","iopub.execute_input":"2021-10-03T19:54:39.070819Z","iopub.status.idle":"2021-10-03T19:54:39.265392Z","shell.execute_reply.started":"2021-10-03T19:54:39.070793Z","shell.execute_reply":"2021-10-03T19:54:39.264077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results = model.fit(\n X_train, \n Y_train, \n validation_split = 0.1, \n batch_size = 16, \n epochs = 25, \n callbacks = callbacks)","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:54:39.267004Z","iopub.execute_input":"2021-10-03T19:54:39.267586Z","iopub.status.idle":"2021-10-03T19:55:09.654192Z","shell.execute_reply.started":"2021-10-03T19:54:39.267541Z","shell.execute_reply":"2021-10-03T19:55:09.653246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#

Inference

","metadata":{}},{"cell_type":"code","source":"# Predictions\nidx = random.randint(0, len(X_train))\n\npreds_train = model.predict(X_train[:int(X_train.shape[0]*0.9)], verbose = 1)\npreds_val = model.predict(X_train[int(X_train.shape[0]*0.9):], verbose = 1)\npreds_test = model.predict(X_test, verbose = 1)\n\npreds_train_t = (preds_train > 0.5).astype(np.uint8)\npreds_val_t = (preds_val > 0.5).astype(np.uint8)\npreds_test_t = (preds_test > 0.5).astype(np.uint8)","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:55:09.655926Z","iopub.execute_input":"2021-10-03T19:55:09.65632Z","iopub.status.idle":"2021-10-03T19:55:11.716367Z","shell.execute_reply.started":"2021-10-03T19:55:09.656281Z","shell.execute_reply":"2021-10-03T19:55:11.715596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Function to display images\ndef display_images(image):\n plt.imshow(image)\n plt.axis('off')\n plt.show()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Select a random index from training set\nix = random.randint(0, len(preds_train_t))\n\n# Display the input image\nprint(\"Input Image:\")\ndisplay_images(X_train[ix])\n\n# Display the ground truth mask\nprint(\"Ground Truth Mask:\")\ndisplay_images(np.squeeze(Y_train[ix]))\n\n# Display the predicted mask\nprint(\"Predicted Mask:\")\ndisplay_images(np.squeeze(preds_train_t[ix]))","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Select a random index from validation set\nix = random.randint(0, len(preds_val_t))\n\n# Calculate the index from where validation data starts\nval_data_start = int(X_train.shape[0] * 0.9)\n\n# Display the validation input image\nprint(\"Validation Input Image:\")\ndisplay_images(X_train[val_data_start:][ix])\n\n# Display the validation ground truth mask\nprint(\"Validation Ground Truth Mask:\")\ndisplay_images(np.squeeze(Y_train[val_data_start:][ix]))\n\n# Display the validation predicted mask\nprint(\"Validation Predicted Mask:\")\ndisplay_images(np.squeeze(preds_val_t[ix]))","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
\n
I appreciate your time and effort in reading this notebook. My aim is to address your questions and curiosities in a comprehensive and clear manner. I welcome any constructive feedback that will help me improve and motivate me to deliver higher quality content. My primary goal is to share knowledge and learn from others while fueling my passion for the subject. If you found this notebook valuable, please consider upvoting and sharing my work.
\n
\n
Thank you, and let passion be your guide.
","metadata":{}}]}