{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
Image Segmentation
\n
Using U-Net with Keras
\n
\n
","metadata":{}},{"cell_type":"markdown","source":"##
Table of contents
\n\n**1. Introduction**\n**
2. Load Data**\n**
3. Get Masks**\n**
4. Training Model**\n**
5. Predict and Visualize**","metadata":{}},{"cell_type":"markdown","source":"#
1. Introduction
","metadata":{}},{"cell_type":"markdown","source":"The u-net is convolutional network architecture for fast and precise segmentation of images. \n\n![](https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/u-net-architecture.png)\n\nU-net architecture (example for 32x32 pixels in the lowest resolution). Each blue box corresponds to a multi-channel feature map. The number of channels is denoted on top of the box. The x-y-size is provided at the lower left edge of the box. White boxes represent copied feature maps. The arrows denote the different operations.\n\nThis illustration is an example of Unet architecture but layers can have different size.","metadata":{}},{"cell_type":"markdown","source":"
","metadata":{}},{"cell_type":"markdown","source":"#
Load Data
","metadata":{}},{"cell_type":"markdown","source":"Libraries","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nimport zipfile\nimport numpy as np\nimport random\nimport os\nfrom tqdm import tqdm\nfrom skimage.io import imread, imshow\nfrom skimage.transform import resize\nimport matplotlib.pyplot as plt\n\nimport warnings\nwarnings.filterwarnings('ignore')\n\n# # Load the extension and start TensorBoard\n\n# %load_ext tensorboard\n# %tensorboard --logdir logs","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:50:25.405501Z","iopub.execute_input":"2021-10-03T19:50:25.405915Z","iopub.status.idle":"2021-10-03T19:50:30.617137Z","shell.execute_reply.started":"2021-10-03T19:50:25.405809Z","shell.execute_reply":"2021-10-03T19:50:30.616263Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Constants","metadata":{}},{"cell_type":"code","source":"SEED = 42\nnp.random.seed = SEED\n\nUNZIP_PATH = '../input/data-science-bowl-2018/'\nTRAIN_PATH = './train/'\nTEST_PATH = './test/'\n\nIMG_WIDTH = 128\nIMG_HEIGHT = 128\nIMG_CHANNELS = 3","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:50:30.619164Z","iopub.execute_input":"2021-10-03T19:50:30.619505Z","iopub.status.idle":"2021-10-03T19:50:30.623941Z","shell.execute_reply.started":"2021-10-03T19:50:30.619469Z","shell.execute_reply":"2021-10-03T19:50:30.622925Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Unzip data\nwith zipfile.ZipFile(UNZIP_PATH+'stage1_train.zip', 'r') as zip_ref:\n zip_ref.extractall('./train')\n \nwith zipfile.ZipFile(UNZIP_PATH+'stage1_test.zip', 'r') as zip_ref:\n zip_ref.extractall('./test')","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:50:30.636837Z","iopub.execute_input":"2021-10-03T19:50:30.637183Z","iopub.status.idle":"2021-10-03T19:50:36.675082Z","shell.execute_reply.started":"2021-10-03T19:50:30.637153Z","shell.execute_reply":"2021-10-03T19:50:36.674207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# get list of all subfolders\ntrain_ids = next(os.walk(TRAIN_PATH))[1]\ntest_ids = next(os.walk(TEST_PATH))[1]","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:50:36.697134Z","iopub.execute_input":"2021-10-03T19:50:36.697619Z","iopub.status.idle":"2021-10-03T19:50:36.706053Z","shell.execute_reply.started":"2021-10-03T19:50:36.697581Z","shell.execute_reply":"2021-10-03T19:50:36.705256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# define placeholders (also used to replace NaN in images to resize by 0)\nX_train = np.zeros((len(train_ids), IMG_HEIGHT, IMG_WIDTH, IMG_CHANNELS), dtype = np.uint8)\nY_train = np.zeros((len(train_ids), IMG_HEIGHT, IMG_WIDTH, 1), dtype = np.bool)","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:50:36.707561Z","iopub.execute_input":"2021-10-03T19:50:36.708015Z","iopub.status.idle":"2021-10-03T19:50:36.71528Z","shell.execute_reply.started":"2021-10-03T19:50:36.707975Z","shell.execute_reply":"2021-10-03T19:50:36.714425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#
Get Masks
","metadata":{}},{"cell_type":"code","source":"# Resize images and masks\nfor n, id_ in tqdm(enumerate(train_ids), total=len(train_ids)):\n \n path = TRAIN_PATH + id_\n img = imread(path + '/images/' + id_ + '.png')[:, :, :IMG_CHANNELS]\n img = resize(\n img, (IMG_HEIGHT, IMG_WIDTH), \n mode = 'constant', \n preserve_range = True\n )\n \n X_train[n] = img # fill empty X_train with values from img \n mask = np.zeros((IMG_HEIGHT, IMG_WIDTH, 1), dtype = np.bool)\n \n for mask_file in next(os.walk(path + '/masks/'))[2]:\n mask_ = imread(path + '/masks/' + mask_file)\n mask_ = np.expand_dims(\n resize(\n mask_, \n (IMG_HEIGHT, IMG_WIDTH), \n mode = 'constant', \n preserve_range = True\n ), \n axis = -1\n )\n mask = np.maximum(mask, mask_)\n \n Y_train[n] = mask","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:50:36.716447Z","iopub.execute_input":"2021-10-03T19:50:36.716721Z","iopub.status.idle":"2021-10-03T19:54:35.233643Z","shell.execute_reply.started":"2021-10-03T19:50:36.716679Z","shell.execute_reply":"2021-10-03T19:54:35.232771Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test images\nX_test = np.zeros(\n (\n len(test_ids), \n IMG_HEIGHT, \n IMG_WIDTH, \n IMG_CHANNELS\n ), \n dtype = np.uint8\n)\n\nsizes_test = []\nfor n, id_ in tqdm(enumerate(test_ids), total=len(test_ids)):\n path = TEST_PATH + id_\n img = imread(path + '/images/' + id_ + '.png')[:,:,:IMG_CHANNELS]\n sizes_test.append([img.shape[0], img.shape[1]])\n img = resize(\n img, \n (IMG_HEIGHT, IMG_WIDTH), \n mode = 'constant', \n preserve_range = True\n )\n X_test[n] = img","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:54:35.235323Z","iopub.execute_input":"2021-10-03T19:54:35.23567Z","iopub.status.idle":"2021-10-03T19:54:36.389158Z","shell.execute_reply.started":"2021-10-03T19:54:35.235634Z","shell.execute_reply":"2021-10-03T19:54:36.388322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#
4. Training Model
","metadata":{}},{"cell_type":"code","source":"inputs = tf.keras.layers.Input((IMG_WIDTH, IMG_HEIGHT, IMG_CHANNELS))\ns = tf.keras.layers.Lambda(lambda x: x/255.0)(inputs) # normalization\n\n# Contraction path\nc1 = tf.keras.layers.Conv2D(16, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(s) # start with normal distributed weights\nc1 = tf.keras.layers.Dropout(0.1)(c1)\nc1 = tf.keras.layers.Conv2D(16, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(c1)\np1 = tf.keras.layers.MaxPooling2D((2,2))(c1)\n\nc2 = tf.keras.layers.Conv2D(32, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(p1)\nc2 = tf.keras.layers.Dropout(0.1)(c2)\nc2 = tf.keras.layers.Conv2D(32, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(c2)\np2 = tf.keras.layers.MaxPooling2D((2,2))(c2)\n\nc3 = tf.keras.layers.Conv2D(64, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(p2)\nc3 = tf.keras.layers.Dropout(0.2)(c3)\nc3 = tf.keras.layers.Conv2D(64, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(c3)\np3 = tf.keras.layers.MaxPooling2D((2,2))(c3)\n\nc4 = tf.keras.layers.Conv2D(128, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(p3)\nc4 = tf.keras.layers.Dropout(0.2)(c4)\nc4 = tf.keras.layers.Conv2D(128, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(c4)\np4 = tf.keras.layers.MaxPooling2D((2,2))(c4)\n\nc5 = tf.keras.layers.Conv2D(256, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(p4)\nc5 = tf.keras.layers.Dropout(0.3)(c5)\nc5 = tf.keras.layers.Conv2D(256, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(c5)\n\n# Expansive path\nu6 = tf.keras.layers.Conv2DTranspose(128, (2,2), strides = (2,2), padding = \"same\")(c5)\nu6 = tf.keras.layers.concatenate([u6, c4])\nc6 = tf.keras.layers.Conv2D(128, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(u6)\nc6 = tf.keras.layers.Dropout(0.2)(c6)\nc6 = tf.keras.layers.Conv2D(128, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(c6)\n\nu7 = tf.keras.layers.Conv2DTranspose(64, (2,2), strides = (2,2), padding = \"same\")(c6)\nu7 = tf.keras.layers.concatenate([u7, c3])\nc7 = tf.keras.layers.Conv2D(64, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(u7)\nc7 = tf.keras.layers.Dropout(0.2)(c7)\nc7 = tf.keras.layers.Conv2D(64, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(c7)\n\nu8 = tf.keras.layers.Conv2DTranspose(32, (2,2), strides = (2,2), padding = \"same\")(c7)\nu8 = tf.keras.layers.concatenate([u8, c2])\nc8 = tf.keras.layers.Conv2D(32, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(u8)\nc8 = tf.keras.layers.Dropout(0.1)(c8)\nc8 = tf.keras.layers.Conv2D(32, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(c8)\n\nu9 = tf.keras.layers.Conv2DTranspose(16, (2,2), strides = (2,2), padding = \"same\")(c8)\nu9 = tf.keras.layers.concatenate([u9, c1])\nc9 = tf.keras.layers.Conv2D(16, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(u9)\nc9 = tf.keras.layers.Dropout(0.1)(c9)\nc9 = tf.keras.layers.Conv2D(16, (3,3), activation=\"relu\", kernel_initializer = \"he_normal\", padding = \"same\")(c9)\n\noutputs = tf.keras.layers.Conv2D(1, (1,1), activation='sigmoid')(c9)\n\nmodel = tf.keras.Model(inputs=[inputs], outputs=[outputs])\nmodel.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:54:36.391636Z","iopub.execute_input":"2021-10-03T19:54:36.392002Z","iopub.status.idle":"2021-10-03T19:54:39.066794Z","shell.execute_reply.started":"2021-10-03T19:54:36.391965Z","shell.execute_reply":"2021-10-03T19:54:39.06594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Callbacks\ncheckpoiter = tf.keras.callbacks.ModelCheckpoint('model.h5', verbose = 1, save_best_only = True)\ncallbacks = [\n tf.keras.callbacks.EarlyStopping(patience = 5, monitor = 'val_loss'),\n tf.keras.callbacks.TensorBoard(log_dir = 'logs')\n ]","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:54:39.070577Z","iopub.execute_input":"2021-10-03T19:54:39.070819Z","iopub.status.idle":"2021-10-03T19:54:39.265392Z","shell.execute_reply.started":"2021-10-03T19:54:39.070793Z","shell.execute_reply":"2021-10-03T19:54:39.264077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"results = model.fit(\n X_train, \n Y_train, \n validation_split = 0.1, \n batch_size = 16, \n epochs = 25, \n callbacks = callbacks)","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:54:39.267004Z","iopub.execute_input":"2021-10-03T19:54:39.267586Z","iopub.status.idle":"2021-10-03T19:55:09.654192Z","shell.execute_reply.started":"2021-10-03T19:54:39.267541Z","shell.execute_reply":"2021-10-03T19:55:09.653246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#
5. Predict and Visualize
","metadata":{}},{"cell_type":"code","source":"# Predictions\nidx = random.randint(0, len(X_train))\n\npreds_train = model.predict(X_train[:int(X_train.shape[0]*0.9)], verbose = 1)\npreds_val = model.predict(X_train[int(X_train.shape[0]*0.9):], verbose = 1)\npreds_test = model.predict(X_test, verbose = 1)\n\npreds_train_t = (preds_train > 0.5).astype(np.uint8)\npreds_val_t = (preds_val > 0.5).astype(np.uint8)\npreds_test_t = (preds_test > 0.5).astype(np.uint8)","metadata":{"execution":{"iopub.status.busy":"2021-10-03T19:55:09.655926Z","iopub.execute_input":"2021-10-03T19:55:09.65632Z","iopub.status.idle":"2021-10-03T19:55:11.716367Z","shell.execute_reply.started":"2021-10-03T19:55:09.656281Z","shell.execute_reply":"2021-10-03T19:55:11.715596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Sanity check on random training samples\nix = random.randint(0, len(preds_train_t))\nimshow(X_train[ix])\nplt.show()\nimshow(np.squeeze(Y_train[ix]))\nplt.show()\nimshow(np.squeeze(preds_train_t[ix]))\nplt.show()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Sanity check on random val samples\nix = random.randint(0, len(preds_val_t))\nimshow(X_train[int(X_train.shape[0]*0.9):][ix])\nplt.show()\nimshow(np.squeeze(Y_train[int(X_train.shape[0]*0.9):][ix]))\nplt.show()\nimshow(np.squeeze(preds_val_t[ix]))\nplt.show()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
\n
Thank you for taking the time to read this notebook. I hope that I was able to answer your questions or your curiosity and that it was quite understandable. any constructive comments are welcome. They help me to progress and motivate me to share better quality content. I am above all a passionate person who tries to advance my knowledge but also that of others. If you liked it, just let me know i'd appreciate.
\n
\n
Thank you and may passion guide you.
","metadata":{}}]}