{"cells":[{"metadata":{"_uuid":"49ba04ada4d11ea3f2a5431eb42491e0eda85440","_cell_guid":"950d37c3-18c0-4e5e-88fa-f278a4b38bb8"},"cell_type":"markdown","source":"## In this notebook we explain our morphological postprocessing step that improved our score from 0.421 to 0.429 on LB\n\nTo see the full end-to-end pipeline go to https://github.com/neptune-ml/data-science-bowl-2018"},{"metadata":{"collapsed":true,"_uuid":"0cc77e61a1aed78238be928a560e3bae9758fd8d","_cell_guid":"524d5b05-dfad-4b67-a8a8-411aa390d04a","trusted":true},"cell_type":"code","source":"%matplotlib inline\nimport os\n\nimport ipywidgets as ipy\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nfrom sklearn.externals import joblib","execution_count":1,"outputs":[]},{"metadata":{"collapsed":true,"_uuid":"0c5630af0f79d7296fa6426f1b2858a5a7eea727","_cell_guid":"cda728de-bb92-466b-8f32-8799fd493c96","trusted":true},"cell_type":"code","source":"masks, contours, ground_truth = joblib.load('../input/morphologicalpostprocessing/masks_contours_ground_truth_train.pkl')","execution_count":2,"outputs":[]},{"metadata":{"collapsed":true,"_uuid":"9ce98ba52aa60407210a616ffe211610ea30783e","_cell_guid":"76cdc98c-d933-4e56-923e-57ff317e2bad","trusted":true},"cell_type":"code","source":"def plot_list(images, labels):\n    n_img = len(images)\n    n_lab = len(labels)\n    n = n_lab+n_img\n    plt.figure(figsize=(12,8))\n    for i, image in enumerate(images):\n        plt.subplot(1,n,i+1)\n        plt.imshow(image)\n    for j, label in enumerate(labels):\n        plt.subplot(1,n,n_img+j+1)\n        plt.imshow(label, cmap='nipy_spectral')\n    plt.show()","execution_count":3,"outputs":[]},{"metadata":{"_uuid":"029953e69c092f5dc8cc708c2192869fef660075","_cell_guid":"6e159bc5-7a0f-4fdc-a56d-ac6f08acf4ff"},"cell_type":"markdown","source":"Lets look at our **inputs**:\n* maps\n* contours\n\nand **outputs**:\n* ground truth\n\nto get some intuition into what is possible"},{"metadata":{"_uuid":"967a05c0fc92d795457d0bf2ab64d3e1a4fcfae4","_cell_guid":"9ca5d7d4-3b8c-4014-a29f-89f594acbc6b","scrolled":false,"trusted":true},"cell_type":"code","source":"for idx in range(5):\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    gt = ground_truth[idx]\n    \n    plot_list(images = [mask,contour],labels = [gt])","execution_count":8,"outputs":[]},{"metadata":{"_uuid":"8fa8070adca3d036ab2fabdd3daaa006663186ce","_cell_guid":"16c93474-36ac-4da6-8b8f-8907925ab208"},"cell_type":"markdown","source":"Both masks and contours seem reasonably good and sometimes like in image 5 it is very beneficial to use both"},{"metadata":{"_uuid":"23222d9bdb8ea6ce0ce88fafcfbc1701cde5daaf","_cell_guid":"cba94bf6-2ab4-45b0-9033-6ea7bbbf2eeb"},"cell_type":"markdown","source":"# Plan:\nWe will use **marker based watershed** to detach nuclei and combine that with some **morphological** cleaning\n\nIf you don't know what it is check this link http://cmm.ensmp.fr/~beucher/wtshed.html\n\n# What we need:\n* Clean binary masks \n* Good nicely detached markers \n* Good distance "},{"metadata":{"_uuid":"ca9f7ba7ed9422bf4112702ca6857436c99c61ae","_cell_guid":"1fdf7174-a4cb-4fb6-9814-f5e8a687bc6c"},"cell_type":"markdown","source":"# Clean Masks"},{"metadata":{"_uuid":"c7ec91c0fc4f330772d7ceb494a9627d37604de2","_cell_guid":"432f791e-2ed3-4002-bd18-3de0ac076c9e"},"cell_type":"markdown","source":"## Problem 1 dirty masks"},{"metadata":{"_uuid":"759a9a4b0c56bf3188b7829ba6fd4a953ca69150","_cell_guid":"ec00c406-3928-4033-b271-f5c10965e73e","trusted":true},"cell_type":"code","source":"idx = 5\ndirty = masks[idx], contours[idx], ground_truth[idx]\n\nplot_list(images = [dirty[0],dirty[1]],\n          labels = [dirty[2]])","execution_count":9,"outputs":[]},{"metadata":{"_uuid":"9b893dfff45dd863e91ae3ab3414cbbc4750e49f","_cell_guid":"e686f31b-513e-4432-a40e-f8e7a4091f86"},"cell_type":"markdown","source":"## Problem 2 dirty at border"},{"metadata":{"_uuid":"28fc25d9df4427dcadffa739d60c73ddb47917b5","_cell_guid":"96507bb6-7498-4e79-b0e5-97b4a849165a","trusted":true},"cell_type":"code","source":"idx = 44\ndirty_at_border = masks[idx], contours[idx], ground_truth[idx]\n\nplot_list(images = [dirty_at_border[0],dirty_at_border[1]],\n          labels = [dirty_at_border[2]])","execution_count":11,"outputs":[]},{"metadata":{"_uuid":"a35f3fd3ef619c6e6b15f8afe39b68634057f0c7","_cell_guid":"f50ef071-f899-4e51-9e58-bf33e23bc787"},"cell_type":"markdown","source":"We can immediately see that some of our cells are are **not very smooth** and there are **holes in them**.\n\nLet's deal with that via morphological transformations. \nIf it is the first time you hear about them it's a good moment to do some reading. \nHere is an interesting link https://homepages.inf.ed.ac.uk/rbf/HIPR2/morops.htm but there is a ton of stuff online so google it if you feel like it."},{"metadata":{"_uuid":"2f51979dc2a5859405cb08e089660157a3e58c02","_cell_guid":"b2414c8d-6e6e-4d17-9eb6-ae7490da2bfe"},"cell_type":"markdown","source":"## Approach V1"},{"metadata":{"collapsed":true,"_uuid":"90326bb21d847a591e1e6128395562f085553020","_cell_guid":"968b5f1c-f925-407d-b040-accdc1ee9a20","trusted":true},"cell_type":"code","source":"import skimage.morphology as morph\nfrom skimage.filters import threshold_otsu\nimport scipy.ndimage as ndi\nfrom scipy.stats import itemfreq","execution_count":12,"outputs":[]},{"metadata":{"_uuid":"f775e88488caefe4808571285b82b0a9c90106a3","_cell_guid":"914ae379-f8d7-4123-aa67-1481a4594840"},"cell_type":"markdown","source":"Let's build a function that calculates the average size of the nuclei. We will need to to choose the structural element for our postprocessing"},{"metadata":{"collapsed":true,"_uuid":"dfcdbd87d8b420dc7d2247ac87739a4c50499ba2","_cell_guid":"1a4153ca-2e64-4a78-9739-374b7fafdab9","trusted":true},"cell_type":"code","source":"def mean_blob_size(mask):\n    labels, labels_nr = ndi.label(mask)\n    if labels_nr < 2:\n        mean_area = 1\n        mean_radius = 1\n    else:\n        mean_area = int(itemfreq(labels)[1:, 1].mean())\n        mean_radius = int(np.round(np.sqrt(mean_area) / np.pi))\n    return mean_area, mean_radius","execution_count":15,"outputs":[]},{"metadata":{"_uuid":"845a4e82413cb2ede30201cce7613487417279c4","_cell_guid":"edf44a53-22cb-4941-afb9-e035c6429900"},"cell_type":"markdown","source":"Let's work on image 5"},{"metadata":{"collapsed":true,"_uuid":"b088409fe63ef8287cda797e03ef2ae3fd700ba6","_cell_guid":"8033328a-ab94-4afa-b607-3cae00d0a527","trusted":true},"cell_type":"code","source":"m, c, t = dirty","execution_count":16,"outputs":[]},{"metadata":{"_uuid":"89234405e404b512ad7ba921c60028716c1843b4","_cell_guid":"ae7408b7-55d9-4a8d-bd38-69621d8c4bef"},"cell_type":"markdown","source":"Now let's proceed to cleaning.\n\nFirst we binarize both the mask and contours using global, otsu thresholding method:"},{"metadata":{"_uuid":"6db9454aace957f8841898dce532589607f528e3","_cell_guid":"a6fd4624-8e0a-4c46-bc9c-8bc6d194989f","trusted":true},"cell_type":"code","source":"m_b = m > threshold_otsu(m)\nc_b = c > threshold_otsu(c)\n\nplot_list(images=[m_b,c_b],labels=[])","execution_count":17,"outputs":[]},{"metadata":{"_uuid":"f2d1a8c64f4acf9e37196e23e28449f9c456734e","_cell_guid":"2b205952-a050-41d2-8c18-d2e968e7e500"},"cell_type":"markdown","source":"Then we go ahead and combine binarized masks and contours:"},{"metadata":{"_uuid":"30b7c548f0cdb7314c136c6605c9368d4159dd3c","_cell_guid":"047dad35-c494-47c4-bbd5-8c555bf0c914","trusted":true},"cell_type":"code","source":"m_ = np.where(m_b | c_b, 1, 0)\nplot_list(images=[m_],labels=[])","execution_count":19,"outputs":[]},{"metadata":{"_uuid":"456a7c42cd64a67e8e89cb11b8d716c90a2eac7a","_cell_guid":"230ebc3d-e1e8-4900-8cea-8d070a2de6f7"},"cell_type":"markdown","source":"and fill the holes that remained"},{"metadata":{"_uuid":"8d0a0f67c115db57b583cf17aba8ec3925b6232a","_cell_guid":"6c6427be-c827-4d41-93b3-abca5f10dd6e","trusted":true},"cell_type":"code","source":"m_ = ndi.binary_fill_holes(m_)\nplot_list(images=[m_],labels=[])","execution_count":20,"outputs":[]},{"metadata":{"_uuid":"fb064123c1ea34ae9ebfa9eb3b323c5edd4c1fed","_cell_guid":"53afaada-976f-465f-a71d-c99d181ec01b"},"cell_type":"markdown","source":"Now that we filled the holes in the cells we can detach them again because we have the contour information"},{"metadata":{"_uuid":"0a749f97f4ef7c553b804cc68df450f14aada1e4","_cell_guid":"80eb4587-a897-496c-a1f0-a4372f352397","trusted":true},"cell_type":"code","source":"m_ = np.where(c_b & (~m_b), 0, m_)\nplot_list(images=[m_],labels=[])","execution_count":23,"outputs":[]},{"metadata":{"_uuid":"3eef99c218624d7ef83a76ffea90b77291c01048","_cell_guid":"38170225-974c-4b18-b389-f05e303630a7"},"cell_type":"markdown","source":"There is a problem with this approach. We are left with artifacts. Let's use binary_openning to drop them."},{"metadata":{"_uuid":"47fbd327862c369499440ecfd1fc99a569ce112e","_cell_guid":"774ab24d-5aef-415f-9008-02c2660da245","trusted":true},"cell_type":"code","source":"area, radius = mean_blob_size(m_b)\nm_ = morph.binary_opening(m_, selem=morph.disk(0.25*radius))\nplot_list(images=[m_],labels=[])","execution_count":24,"outputs":[]},{"metadata":{"_uuid":"8ce3134810c3804e8a98a3462cbb0435932cf283","_cell_guid":"27b00035-9dd2-49b0-b391-eb71e9de0c5d"},"cell_type":"markdown","source":"Ok that looks fair.\nLet's put it all together in a function"},{"metadata":{"collapsed":true,"_uuid":"2ff69933d2ead844bc1929afc7d26891da5f97b1","_cell_guid":"70e52772-3d62-4e40-aaca-db401cd23710","trusted":true},"cell_type":"code","source":"def clean_mask_v1(m,c):\n    m_b = m > threshold_otsu(m)\n    c_b = c > threshold_otsu(c)\n\n    m_ = np.where(m_b | c_b, 1, 0)\n    m_ = ndi.binary_fill_holes(m_)\n    m_ = np.where(c_b & (~m_b), 0, m_)\n    \n    area, radius = mean_blob_size(m_b)\n    m_ = morph.binary_opening(m_, selem=morph.disk(0.25*radius))\n    return m_","execution_count":26,"outputs":[]},{"metadata":{"_uuid":"01a0c0615f6135cd682d520447609cf017782054","_cell_guid":"c81d0459-bbc9-48a8-976e-2a47e0223d28","trusted":true},"cell_type":"code","source":"m, c, t = dirty\n\nm_ = clean_mask_v1(m,c)\n\nplot_list(images = [m,c,m_],\n          labels = [t]\n         )","execution_count":27,"outputs":[]},{"metadata":{"_uuid":"9eca3ac8625ffc633258e53c3f8a9bf9d3477858","_cell_guid":"e8620df7-38d7-4787-9d37-991a6f3cc460"},"cell_type":"markdown","source":"It works to a certain extend but it removes things that where not connected and/or things around borders"},{"metadata":{"_uuid":"87c6437d18db787bf2c9127950005580514383b3","_cell_guid":"acf7890a-55e9-4edb-8f5f-b459c338cb8e","trusted":true},"cell_type":"code","source":"m,c,t = dirty_at_border\nm_ = clean_mask_v1(m,c)\n\nplot_list(images = [m,c,m_],\n          labels = [t]\n         )","execution_count":28,"outputs":[]},{"metadata":{"_uuid":"4301e4fdb91d7f944983d0dde9ca30a1875f15c4","_cell_guid":"06a47ef8-2405-42f5-b66a-5811b967dce0"},"cell_type":"markdown","source":"## Approach V2"},{"metadata":{"collapsed":true,"_uuid":"e57268a772cc2806330c02e6dc00bced5f490c67","_cell_guid":"255ea678-9179-4dd9-9b08-d88cbb8fa331","trusted":true},"cell_type":"code","source":"m,c,t = dirty_at_border","execution_count":31,"outputs":[]},{"metadata":{"_uuid":"c0acf2f9d15f7b0ef5c3814fc26767247e593526","_cell_guid":"28f5ca04-d5da-43c2-8a3e-3d96aff60f8d"},"cell_type":"markdown","source":"Let's start by binarizing and filling the holes again"},{"metadata":{"_uuid":"74a5232ffaefa923936c0a059b5b881e5fc11c4e","_cell_guid":"51da3b7a-f59b-49cd-9f03-004d8a6534f6","trusted":true},"cell_type":"code","source":"# threshold\nm_b = m > threshold_otsu(m)\nc_b = c > threshold_otsu(c)\n\n# combine contours and masks and fill the cells\nm_ = np.where(m_b | c_b, 1, 0)\nm_ = ndi.binary_fill_holes(m_)\nplot_list(images=[m_],labels=[])","execution_count":32,"outputs":[]},{"metadata":{"_uuid":"6b87a5a61103d036834991d0760155f873f44bab","_cell_guid":"87c6e7b1-f990-4f3d-9596-f42d7bb94959"},"cell_type":"markdown","source":"Now we will use `binary_closing` to fill what wasn't closed with fill holes. We will need two helper functions `pad_mask` and `crop_mask` to deal with problems around the edges "},{"metadata":{"collapsed":true,"_uuid":"8834d46d33f96c615f59c61b3dc1b016ce7eb95f","_cell_guid":"f689ea76-ac81-4f82-805a-0ea32eb6df81","trusted":true},"cell_type":"code","source":"def pad_mask(mask, pad):\n    if pad <= 1:\n        pad = 2\n    h, w = mask.shape\n    h_pad = h + 2 * pad\n    w_pad = w + 2 * pad\n    mask_padded = np.zeros((h_pad, w_pad))\n    mask_padded[pad:pad + h, pad:pad + w] = mask\n    mask_padded[pad - 1, :] = 1\n    mask_padded[pad + h + 1, :] = 1\n    mask_padded[:, pad - 1] = 1\n    mask_padded[:, pad + w + 1] = 1\n\n    return mask_padded\n\ndef crop_mask(mask, crop):\n    if crop <= 1:\n        crop = 2\n    h, w = mask.shape\n    mask_cropped = mask[crop:h - crop, crop:w - crop]\n    return mask_cropped","execution_count":34,"outputs":[]},{"metadata":{"_uuid":"a37bf3e2e2560a58ac2754aacce20581a9544d68","_cell_guid":"51deb55c-5ea8-4f2d-af24-4157886af0a7"},"cell_type":"markdown","source":"ok lets go for it"},{"metadata":{"_uuid":"25ed2d12a1d7c088b2f3a7d858edaf884bc7b28c","_cell_guid":"79944048-1eca-472e-ac56-1daa81cf81d0","trusted":true},"cell_type":"code","source":"# close what wasn't closed before \narea, radius = mean_blob_size(m_b)\nstruct_size = int(1.25*radius)\nstruct_el = morph.disk(struct_size)\nm_padded = pad_mask(m_, pad=struct_size)\nm_padded = morph.binary_closing(m_padded, selem=struct_el)\nm_ = crop_mask(m_padded, crop=struct_size)\nplot_list(images=[m_],labels=[])","execution_count":35,"outputs":[]},{"metadata":{"_uuid":"0d1bf4ca53915d5bb0f0163c2eb7c45d7431b2dd","_cell_guid":"fb68f6fe-af02-490e-9f70-2c11eabf0183"},"cell_type":"markdown","source":"Yeah we closed everything but it is way more than we wanted. Let's now cut it with our contours and see what we get"},{"metadata":{"_uuid":"3b75699360c3b5c1b5fb2c01f7d8192a34f3aa5a","_cell_guid":"63097608-4a91-4157-9a14-66fd2cb24adf","scrolled":false,"trusted":true},"cell_type":"code","source":"m_ = np.where(c_b & (~m_b), 0, m_)\nplot_list(images=[m_],labels=[])","execution_count":36,"outputs":[]},{"metadata":{"_uuid":"267cf8b0883b36dc7ff8444dfd5d68c2424add68","_cell_guid":"bc841c2f-f765-4990-b4ff-1b6b96c214b5"},"cell_type":"markdown","source":"We got a lot of artifacts but we are getting places. If we could clean up those small artifacts we would be almost golden. Luckily we can use `binary_openning` with a larger structural element. Let's try that"},{"metadata":{"_uuid":"5bbace6636817a5096a28c61a6d75c9da0857d79","_cell_guid":"bda4e430-46eb-4d8e-83ab-ba3c539ceb21","trusted":true},"cell_type":"code","source":"area, radius = mean_blob_size(m_b)\nstruct_size = int(0.75*radius)\nstruct_el = morph.disk(struct_size)\nm_padded = pad_mask(m_, pad=struct_size)\nm_padded = morph.binary_opening(m_padded, selem=struct_el)\nm_ = crop_mask(m_padded, crop=struct_size)\nplot_list(images=[m_,m],labels=[t])","execution_count":37,"outputs":[]},{"metadata":{"_uuid":"93cc5302e35cc6a2a8681cc1479868854226241a","_cell_guid":"34cf0939-d3cc-4935-a869-ef92cfdc2979"},"cell_type":"markdown","source":"Boom! It looks really nice. \n\nThere are two problems with this result. First we have slightly smaller cells (like the blue one on the right edge) and we created some new cells (like the one at the top edge)!. Let's deal with it one problem at a time starting with the two small cell problem. We can simply join the result with the original binarized mask."},{"metadata":{"_uuid":"49c08ab5c2be0ff36df3de8f5c4133b92eed6bea","_cell_guid":"d3dc5fe4-daf6-4d24-80b4-0f9728c6d538","trusted":true},"cell_type":"code","source":"# join the connected cells with what we had at the beginning\nm_ = np.where(m_b|m_,1,0)\nm_ = ndi.binary_fill_holes(m_)\nplot_list(images=[m_,m],labels=[t])","execution_count":38,"outputs":[]},{"metadata":{"_uuid":"2c88dac5a7c49b72c4d221d4ee60a8eddfa60804","_cell_guid":"128cd505-6b63-4ef6-abe7-89d0e1615b7f"},"cell_type":"markdown","source":"We'll deal with the second problem by going over all the resulting cells and calculating the coverage of the cell with the original binaraized image region. If nothing was there at the beginning nothing should be there now. Let's do that with the following `drop_artifacts` function"},{"metadata":{"collapsed":true,"_uuid":"c5ca22f5e0d813a2947445a68fe958dccd6bf2bf","_cell_guid":"ce74a794-6f7c-4611-9f20-4b679f88e1ef","trusted":true},"cell_type":"code","source":"def drop_artifacts(mask_after, mask_pre, min_coverage=0.5):\n    connected, nr_connected = ndi.label(mask_after)\n    mask = np.zeros_like(mask_after)\n    for i in range(1, nr_connected + 1):\n        conn_blob = np.where(connected == i, 1, 0)\n        initial_space = np.where(connected == i, mask_pre, 0)\n        blob_size = np.sum(conn_blob)\n        initial_blob_size = np.sum(initial_space)\n        coverage = float(initial_blob_size) / float(blob_size)\n        if coverage > min_coverage:\n            mask = mask + conn_blob\n        else:\n            mask = mask + initial_space\n    return mask","execution_count":39,"outputs":[]},{"metadata":{"_uuid":"5c59aafeae1573aa07d12ddba832807fd2e8da78","_cell_guid":"41a29ebc-af98-4b61-8d35-6b3375f3ec18","trusted":true},"cell_type":"code","source":"m_ = drop_artifacts(m_, m_b,min_coverage=0.25)\nplot_list(images=[m_,m,c],labels=[t])","execution_count":40,"outputs":[]},{"metadata":{"_uuid":"459dc053757ff5b38f6c094856ff97031eb69f13","_cell_guid":"c0d9f685-3130-4a72-b932-60c0b3d93926"},"cell_type":"markdown","source":"Not bad at all!\n\nLet's put it all together in one function"},{"metadata":{"collapsed":true,"_uuid":"4a01bb3e4afea8a4d29e315d3be84b34dac7658d","_cell_guid":"b32fa9f5-58b5-40bb-9ecb-1a5bf5eb3bc1","trusted":true},"cell_type":"code","source":"def clean_mask_v2(m,c):\n    # threshold\n    m_b = m > threshold_otsu(m)\n    c_b = c > threshold_otsu(c)\n\n    # combine contours and masks and fill the cells\n    m_ = np.where(m_b | c_b, 1, 0)\n    m_ = ndi.binary_fill_holes(m_)\n\n    # close what wasn't closed before \n    area, radius = mean_blob_size(m_b)\n    struct_size = int(1.25*radius)\n    struct_el = morph.disk(struct_size)\n    m_padded = pad_mask(m_, pad=struct_size)\n    m_padded = morph.binary_closing(m_padded, selem=struct_el)\n    m_ = crop_mask(m_padded, crop=struct_size)\n    \n    # open to cut the real cells from the artifacts\n    area, radius = mean_blob_size(m_b)\n    struct_size = int(0.75*radius)\n    struct_el = morph.disk(struct_size)\n    m_ = np.where(c_b & (~m_b), 0, m_)\n    m_padded = pad_mask(m_, pad=struct_size)\n    m_padded = morph.binary_opening(m_padded, selem=struct_el)\n    m_ = crop_mask(m_padded, crop=struct_size)\n\n    # join the connected cells with what we had at the beginning\n    m_ = np.where(m_b|m_,1,0)\n    m_ = ndi.binary_fill_holes(m_)\n    \n    # drop all the cells that weren't present at least in 25% of area in the initial mask \n    m_ = drop_artifacts(m_, m_b,min_coverage=0.25)\n    \n    return m_","execution_count":42,"outputs":[]},{"metadata":{"_uuid":"8697c0dee49b45aae8cf2510b5fc6553b2ce43b9","_cell_guid":"24e62c42-ad7b-474e-bd90-e7d94ee47e62","trusted":true},"cell_type":"code","source":"m,c,t = dirty_at_border\nm_ = clean_mask_v2(m,c)\n\nplot_list(images = [m,c,m_],\n          labels = [t]\n         )","execution_count":43,"outputs":[]},{"metadata":{"_uuid":"ea304aaad6503fac4df77493e7b4b7895ec471f3","_cell_guid":"aeb028fb-d4df-46d2-b426-c39417351b4f","trusted":true},"cell_type":"code","source":"for idx in range(5):\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    gt = ground_truth[idx]\n    \n    plot_list(images = [mask,contour, cleaned_mask],labels = [gt])","execution_count":44,"outputs":[]},{"metadata":{"_uuid":"c070e45ffc2f6d9606d582b16aec238598fd9ef3","_cell_guid":"4aee9cd8-218c-4f96-9ae3-1be150a0f418"},"cell_type":"markdown","source":"It is still not perfect for instance the following problems still arise:"},{"metadata":{"_uuid":"2c2d42b75f804e524863502e3f112b6fb49fc8f6","_cell_guid":"44761a46-977d-425c-a269-4cfca5742494"},"cell_type":"markdown","source":"# Problem 3 not everything gets filled"},{"metadata":{"_uuid":"3bf4b8906685bae1e520228e3f7e16a8b7acaf81","_cell_guid":"54e9fc3b-0ea7-488d-a54d-01a56994471b","trusted":true},"cell_type":"code","source":"for idx in [38]:\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    gt = ground_truth[idx]\n    \n    plot_list(images = [mask,contour, cleaned_mask],labels = [gt])","execution_count":45,"outputs":[]},{"metadata":{"_uuid":"2af1c2fc3ba2d447699f796150e9c25f52277d14","_cell_guid":"5b13693c-a8d5-4309-83ac-29ed2fb5529d"},"cell_type":"markdown","source":"Look at the blue cell on the yellow and blue cells on the left"},{"metadata":{"_uuid":"c7971ab9f1855a94e11aefa58c5f10a06c2c2bef","_cell_guid":"2bd05efa-9c39-4076-9f7e-72c5bcb716c8"},"cell_type":"markdown","source":"# Problem 4 some cells get connected"},{"metadata":{"_uuid":"21ccca5fe554fad6e38635d1146f1018e6bc5a57","_cell_guid":"ad91855a-e47a-4f85-88f9-8ec8e5b943a2","trusted":true},"cell_type":"code","source":"for idx in [0]:\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    gt = ground_truth[idx]\n    \n    plot_list(images = [mask,contour, cleaned_mask],labels = [gt])","execution_count":46,"outputs":[]},{"metadata":{"_uuid":"9ec5728f5a224774633b38f0612a8286f9457370","_cell_guid":"d1da62aa-e506-4030-90d1-777cf22606a4"},"cell_type":"markdown","source":"Look at this large blob at the top. Also there are artifacts that aren't supposed to be there"},{"metadata":{"_uuid":"87351e5d9aef012033a50911018ccd0a3623f8f5","_cell_guid":"a45d8215-8748-4c7b-b3df-f2e09e898abd"},"cell_type":"markdown","source":"# Ideas:\n* work more with dilation\n* do better around borders\n* drop some cells after watershed with drop_artifact function\n\nGo ahead and try to improve it. The floor is yours"},{"metadata":{"collapsed":true,"_uuid":"0c3632df84cd76da9599e0dcbdb73ce4e618b07d","_cell_guid":"b4c60081-2b3b-458e-88c8-4e936bdcb30d","trusted":true},"cell_type":"code","source":"def clean_mask_v3(m,c):\n    return","execution_count":48,"outputs":[]},{"metadata":{"_uuid":"dcaaaedd4d19b6ffe46a8113303d500c8b7b3c07","_cell_guid":"101b9bb3-df07-4942-9513-c2fa52464a78"},"cell_type":"markdown","source":"# Good Markers\n\nLet's now try end extract good markers for our watershed."},{"metadata":{"_uuid":"d69bb9260d62f694577ed4b083cef13853432ae6","_cell_guid":"a1de3cda-1ae4-45f2-a7f8-1202df685ceb"},"cell_type":"markdown","source":"# Approach v1\n\nIn this approach we will simply cut the masks with the contours and use that as markers. Simple but really effective."},{"metadata":{"collapsed":true,"_uuid":"17fd0b34034a402f091ee3a2c931c2e8cd6154f3","_cell_guid":"4d003a3c-8f64-4bcf-8296-143140055c09","trusted":true},"cell_type":"code","source":"def good_markers_v1(m,c):\n    # threshold\n    m_b = m > threshold_otsu(m)\n    c_b = c > threshold_otsu(c)\n    \n    mk_ = np.where(c_b,0,m)\n    return mk_","execution_count":49,"outputs":[]},{"metadata":{"_uuid":"8ed705230f7ccc8761be86c0881af41ac0f74ebd","_cell_guid":"2e9412a3-c246-4666-b0db-592d262b5efd","trusted":true},"cell_type":"code","source":"for idx in range(5):\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v1(mask,contour)\n    gt = ground_truth[idx]\n    \n    plot_list(images = [mask,contour, cleaned_mask, good_markers],labels = [gt])","execution_count":50,"outputs":[]},{"metadata":{"_uuid":"de1e6f6b2088981a7ac8286b0bfae14e143bb8fe","_cell_guid":"40f0377b-2eb3-451b-aade-7f51ab0d78d2"},"cell_type":"markdown","source":"## Problem 1 building markers on initial mask when we have better mask\n\nThere is no point in using initial masks when we worked so hard on making them better right?\n\nLet's use our results from the first step"},{"metadata":{"collapsed":true,"_uuid":"99d49f0596a2aee1eea986e79fac491988a6fb07","_cell_guid":"d772e5b5-e6c9-4d4d-a015-a4cfdd660740","trusted":true},"cell_type":"code","source":"def good_markers_v2(m_b,c):\n    # threshold\n    c_thresh = threshold_otsu(c)\n    c_b = c > c_thresh\n    \n    mk_ = np.where(c_b,0,m_b)\n    return mk_","execution_count":52,"outputs":[]},{"metadata":{"_uuid":"1059c22bc09a25b32e6c18a967d07a193edbfc5d","_cell_guid":"f6b1cdef-879c-45b7-bd3c-4e05c6f31c2d","trusted":true},"cell_type":"code","source":"for idx in range(5):\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v2(cleaned_mask,contour)\n    gt = ground_truth[idx]\n    \n    plot_list(images = [mask,contour, cleaned_mask, good_markers],labels = [gt])","execution_count":53,"outputs":[]},{"metadata":{"_uuid":"dc9e80405ec7a6faeedb54700e5b209e20e55593","_cell_guid":"10d212b8-0aeb-440d-9048-f13a3d042ca5"},"cell_type":"markdown","source":"## Problem 2 some markers are to large and connected\n\nUnfortunately it is not perfect. If we have to connected cells and we have one connected marker for those cells watershed will not detach it. We need to make them better, smaller and positioned more in the center of nuceli.\n\nIn order to get there we can use `binary_erosion` \nLet's first do what we know"},{"metadata":{"_uuid":"26399537eb635b45ded893ca1c54cf9d2ef207c1","_cell_guid":"b6ee259c-1e67-488e-a1a2-a84964bdd79f","trusted":true},"cell_type":"code","source":"m,c,t = dirty\n\ncleaned_mask = clean_mask_v2(m, c)\n\nc_b = c > threshold_otsu(c)\nmk_ = np.where(c_b,0,cleaned_mask)\nplot_list(images=[m,c,mk_],labels=[])","execution_count":54,"outputs":[]},{"metadata":{"_uuid":"479cefe167ec698190f0bb853ec55dbc92e2afba","_cell_guid":"8ae387ed-8776-4216-9b4c-f1991571d139"},"cell_type":"markdown","source":"For instance the two markers at the top left are still connected and will be treated as a single marker by the watershed. And nowe lets erode the markers"},{"metadata":{"_uuid":"2c9a63283d9e67457406887ff9a3352467a2cc87","_cell_guid":"315ad8dd-a81d-4e3e-a79c-bbcbe20714bd","trusted":true},"cell_type":"code","source":"area, radius = mean_blob_size(m_b)\nstruct_size = int(0.25*radius)\nstruct_el = morph.disk(struct_size)\nm_padded = pad_mask(mk_, pad=struct_size)\nm_padded = morph.erosion(m_padded, selem=struct_el)\nmk_ = crop_mask(m_padded, crop=struct_size)\nplot_list(images=[m,c,mk_],labels=[])","execution_count":55,"outputs":[]},{"metadata":{"_uuid":"7f9b82272f686da331cd4930e873d5fea6378a46","_cell_guid":"739f8f2d-ab92-4601-b89a-3b2234d88dc5"},"cell_type":"markdown","source":"ok thats not bad if we now compare those markers with the labels we get the following"},{"metadata":{"_uuid":"74c399b226e2027f01820761718d33a373a35754","_cell_guid":"8c29c9da-dbdf-408b-b049-f0c3344a63f9","trusted":true},"cell_type":"code","source":"mk_,_ = ndi.label(mk_) \nplot_list(images=[cleaned_mask],labels=[mk_, t])","execution_count":56,"outputs":[]},{"metadata":{"_uuid":"db00f647b91da1e1005a13107e2a7b215c5e7dbf","_cell_guid":"40adef21-9fa9-4221-bd0b-d742d344ac62"},"cell_type":"markdown","source":"So the markers and cleaned mask look really good!"},{"metadata":{"collapsed":true,"_uuid":"20944b83e6f10570cd8dc98d903b1d4c102f311d","_cell_guid":"25efc197-cb90-4c19-b9ef-0e74899f8837","trusted":true},"cell_type":"code","source":"def good_markers_v3(m_b,c):\n    # threshold\n    c_b = c > threshold_otsu(c)\n    \n    mk_ = np.where(c_b,0,m_b)\n    \n    area, radius = mean_blob_size(m_b)\n    struct_size = int(0.25*radius)\n    struct_el = morph.disk(struct_size)\n    m_padded = pad_mask(mk_, pad=struct_size)\n    m_padded = morph.erosion(m_padded, selem=struct_el)\n    mk_ = crop_mask(m_padded, crop=struct_size)\n    mk_,_ = ndi.label(mk_) \n    return mk_","execution_count":57,"outputs":[]},{"metadata":{"_uuid":"1d94fddd27cf6a56fe42812c7d4daaa293f8623d","_cell_guid":"bfd39165-8b6d-49ed-acaf-c1e14a59c658","trusted":true},"cell_type":"code","source":"for idx in range(5):\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v3(cleaned_mask,contour)\n    gt = ground_truth[idx]\n    \n    plot_list(images = [mask,contour, cleaned_mask],labels = [good_markers, gt])","execution_count":58,"outputs":[]},{"metadata":{"_uuid":"1d63a738d5ce9fa21cef5649c58e71159a1929db","_cell_guid":"783c55e9-0878-4333-a63a-17963708cf03"},"cell_type":"markdown","source":"## Problem 3 still some barely connected markers are left\n\nUnfortunately for some images the markers are not eroded enough and are left connected (look at the orange blob at the bottom right corner in the forth column).\nSome tweaking should improve it but beware that for other images it might decrease the score."},{"metadata":{"_uuid":"01b37fbded42deda0f54f8098bdb1b9669ccec69","_cell_guid":"ae2ca212-ba0e-46e4-a816-031010a972f3","trusted":true},"cell_type":"code","source":"for idx in [25,27]:\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v3(cleaned_mask,contour)\n    gt = ground_truth[idx]\n    \n    plot_list(images = [mask,contour, cleaned_mask],labels = [good_markers, gt])","execution_count":59,"outputs":[]},{"metadata":{"_uuid":"601b1d3a58a4c6a6d45384a4e390ebf71d37e806","_cell_guid":"b48be6fd-284a-41e3-af0f-3f18fef06003"},"cell_type":"markdown","source":"## Problem 4 we are dropping markers on many images with small cells"},{"metadata":{"_uuid":"98c120d3dfd88b2652aa30b3d784433dd17b8fe8","_cell_guid":"7e4d4f90-6a8f-4d9a-b6ed-c97b089a2147"},"cell_type":"markdown","source":"## Ideas\n* play with binary closing/opening\n* involve contours and/or centers in this\n* we will asume that lost markers are in facet small cells that don't need to be divided and we will get back all the cells that were dropped in watershed\n* use local maxima on distance transform"},{"metadata":{"collapsed":true,"_uuid":"bb50557a798b78fddb2fe71bf2cd9daeaabd9855","_cell_guid":"ccc963c0-33bb-41bc-a8f9-5743ba428e52","trusted":true},"cell_type":"code","source":"def good_markers_v4(m_b,c):\n    return","execution_count":60,"outputs":[]},{"metadata":{"_uuid":"9c90fe31a9268d7277fcf8adb438ffe4dc2580fe","_cell_guid":"6612f916-93ef-4823-8d98-3584a8f0527e"},"cell_type":"markdown","source":"# Good distance\n\nHere I have no better idea than to use the binary distance from the background.\nFeel free to improve on that!"},{"metadata":{"collapsed":true,"_uuid":"588448523a4c391e4607b5a2cc9153ef01fa0113","_cell_guid":"84e89f6a-d729-4ab7-8e0b-634f0132ea20","trusted":true},"cell_type":"code","source":"def good_distance_v1(m_b):\n    distance = ndi.distance_transform_edt(m_b)\n    return distance","execution_count":61,"outputs":[]},{"metadata":{"_uuid":"cb2b3774661d5d16d606d4d002c8f0600d592993","_cell_guid":"d9cbe4bf-78bd-40d9-b495-6f3c15ecfcec","trusted":true},"cell_type":"code","source":"for idx in range(5):\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v3(cleaned_mask,contour)\n    good_distance = good_distance_v1(cleaned_mask)\n    gt = ground_truth[idx]\n    \n    plot_list(images = [cleaned_mask, good_distance],labels = [good_markers, gt])","execution_count":62,"outputs":[]},{"metadata":{"_uuid":"defed4ff857996c6fb60533af937f41488ecc3f7","_cell_guid":"bdc0d197-7fc6-4c70-8b0d-c92cbf0d2e5a"},"cell_type":"markdown","source":"## Idea\n* investigate imposing some gradients on original image or good clean mask"},{"metadata":{"_uuid":"36838b98d6f2699226160367b764f3478159665e","_cell_guid":"93c0e83c-8466-4dad-9e17-a18620adbb08"},"cell_type":"markdown","source":"## Watershed\n\nNow that we have good masks, markersa and distance we can proceed to the watershed algorithm.\n\nLet's try the simplest option without any tweaks"},{"metadata":{"collapsed":true,"_uuid":"cfcdb473752f380b2cad9389ab6a780c023d6ada","_cell_guid":"bfe38b60-b178-4ad9-8c38-9c6cee6ccc3d","trusted":true},"cell_type":"code","source":"def watershed_v1(mask, contour):\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v3(cleaned_mask,contour)\n    good_distance = good_distance_v1(cleaned_mask)\n    \n    water = morph.watershed(-good_distance, good_markers, mask=cleaned_mask)\n    \n    return water","execution_count":63,"outputs":[]},{"metadata":{"_uuid":"fa7e2fd1760d2c887c97a8dab3c0deea012aa9c5","_cell_guid":"51c74b30-9e68-4a9c-9e2d-1efaa71edbf0","trusted":true},"cell_type":"code","source":"for idx in range(5):\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v3(cleaned_mask,contour)\n    good_distance = good_distance_v1(cleaned_mask)\n    \n    water = watershed_v1(mask, contour)\n    \n    gt = ground_truth[idx]\n    \n    plot_list(images = [cleaned_mask, good_distance],labels = [good_markers, water,gt])","execution_count":64,"outputs":[]},{"metadata":{"_uuid":"f857a9d54e57b3710d1407ff49081260a0bea81b","_cell_guid":"93021b4c-ec81-48ff-bae6-57973bf0933b"},"cell_type":"markdown","source":"## Problem 1 some cells are dumped\n## Problem 2 some artifacts from mask_cleaning remain\n\nUnfortunatelly some cells are dropped, some cells are oversegmented and some artifacts from the mask cleaning still remain.\n\nThe good thing is we can deal with some of those problems by using ideas we have already tried. \n\nBy the way do you know why the cells that are visible on the mask are sometimes dropped in the watershed?\n\nI will not answer it know but simply solve it by adding the dropped cells to the output."},{"metadata":{"collapsed":true,"_uuid":"a26acc74607baacb1a4ff85a91141e5495f44f71","_cell_guid":"d8f97d32-f1ee-4954-a6a0-b490f09ce766","trusted":true},"cell_type":"code","source":"def add_dropped_water_blobs(water, mask_cleaned):\n    water_mask = (water > 0).astype(np.uint8)\n    dropped = mask_cleaned - water_mask\n    dropped, _ = ndi.label(dropped)\n    dropped = np.where(dropped, dropped + water.max(), 0)\n    water = water + dropped\n    return water","execution_count":66,"outputs":[]},{"metadata":{"_uuid":"12810f6fdf37c3ab59ebf4587924b3ac08deefe1","_cell_guid":"d0cb555e-aa31-4897-8ad4-07d756baa29c"},"cell_type":"markdown","source":"When it comes to artifacts left we will recycle the idea of dropping cells that weren't visible on the initial image.\nOnly difference is that now we need to iterate over connected components and apply drop artifacts to each component. \n\nRemember that some cells that are touching are detached by watershed."},{"metadata":{"collapsed":true,"_uuid":"ae31ef2dd07d2e8e20e55c4363fdba2faa4ada31","_cell_guid":"431a675b-8a18-41ff-a4d7-a2f0358646c1","trusted":true},"cell_type":"code","source":"def drop_artifacts_per_label(labels, initial_mask):\n    labels_cleaned = np.zeros_like(labels)\n    for i in range(1, labels.max() + 1):\n        component = np.where(labels == i, 1, 0)\n        component_initial_mask = np.where(labels == i, initial_mask, 0)\n        component = drop_artifacts(component, component_initial_mask)\n        labels_cleaned = labels_cleaned + component * i\n    return labels_cleaned","execution_count":67,"outputs":[]},{"metadata":{"_uuid":"ef6c23a5f724847a71fa0d477f2a1d1aac2700b4","_cell_guid":"d2a6794d-1554-48e9-9bac-abaaccfe2baf"},"cell_type":"markdown","source":"Ok putting it all together gives us:"},{"metadata":{"collapsed":true,"_uuid":"1206686af7428124fd64b6d0721d7b5b28a1a3a4","_cell_guid":"c5fba24d-3f8e-4437-bcb5-c6fd69215fb2","trusted":true},"cell_type":"code","source":"def watershed_v2(mask, contour):\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v3(cleaned_mask,contour)\n    good_distance = good_distance_v1(cleaned_mask)\n    \n    water = morph.watershed(-good_distance, good_markers, mask=cleaned_mask)\n    \n    water = add_dropped_water_blobs(water, cleaned_mask)\n    \n    m_thresh = threshold_otsu(mask)\n    initial_mask_binary = (mask > m_thresh).astype(np.uint8)\n    water = drop_artifacts_per_label(water,initial_mask_binary)\n    return water","execution_count":68,"outputs":[]},{"metadata":{"_uuid":"0839fd6d92dce43d754ee09525286a3986282856","_cell_guid":"a17d8128-a18f-49bb-8c31-47b98d80506f","scrolled":false,"trusted":true},"cell_type":"code","source":"for idx in range(5):\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v3(cleaned_mask,contour)\n    good_distance = good_distance_v1(cleaned_mask)\n    \n    water = watershed_v2(mask, contour)\n    \n    gt = ground_truth[idx]\n    \n    plot_list(images = [cleaned_mask, good_distance],labels = [good_markers, water,gt])","execution_count":69,"outputs":[]},{"metadata":{"_uuid":"1338fdee70b1ad0b15e0590b26f68d730866e347","_cell_guid":"41103967-50e6-420e-8e0d-bbbee8dde3fb"},"cell_type":"markdown","source":"## Problem 3 some cells are oversemgmented and small cell chunks remain\n\nNow some small pieces of cells may remain at this point or the cells could get oversegmented.\nWe will deal with that by dropping to small to be a cell blobs. However when we drop them and those oversegmentations happen in the middle of the cell we might end up with a hole. That is why we will iterate over every connected component and apply `binary_fill_hole` to it to remove that problem."},{"metadata":{"collapsed":true,"_uuid":"d747811d4cb4af72b94b9e50bfd40086a9dda2af","_cell_guid":"5dae75f3-3cd2-41be-af75-f9941430ca30","trusted":true},"cell_type":"code","source":"from itertools import product\n\ndef relabel(img):\n    h, w = img.shape\n\n    relabel_dict = {}\n\n    for i, k in enumerate(np.unique(img)):\n        if k == 0:\n            relabel_dict[k] = 0\n        else:\n            relabel_dict[k] = i\n    for i, j in product(range(h), range(w)):\n        img[i, j] = relabel_dict[img[i, j]]\n    return img\n\ndef drop_small(img, min_size):\n    img = morph.remove_small_objects(img, min_size=min_size)\n    return relabel(img)\n\ndef fill_holes_per_blob(image):\n    image_cleaned = np.zeros_like(image)\n    for i in range(1, image.max() + 1):\n        mask = np.where(image == i, 1, 0)\n        mask = ndi.morphology.binary_fill_holes(mask)\n        image_cleaned = image_cleaned + mask * i\n    return image_cleaned","execution_count":70,"outputs":[]},{"metadata":{"collapsed":true,"_uuid":"41285d09c62a07678e1f4b5896cc3ea3a6574de5","_cell_guid":"798dc297-161e-40e8-8443-fc1fa79bde00","trusted":true},"cell_type":"code","source":"def watershed_v3(mask, contour):\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v3(cleaned_mask,contour)\n    good_distance = good_distance_v1(cleaned_mask)\n    \n    labels = morph.watershed(-good_distance, good_markers, mask=cleaned_mask)\n    \n    labels = add_dropped_water_blobs(labels, cleaned_mask)\n    \n    m_thresh = threshold_otsu(mask)\n    initial_mask_binary = (mask > m_thresh).astype(np.uint8)\n    labels = drop_artifacts_per_label(labels,initial_mask_binary)\n    \n    labels = drop_small(labels, min_size=20)\n    labels = fill_holes_per_blob(labels)\n        \n    return labels","execution_count":71,"outputs":[]},{"metadata":{"_uuid":"e645ba50b0f1febd30af03aa80d2c7ffcbea4dae","_cell_guid":"1a586267-bbfe-47c6-9c5a-f28fae3826ea","scrolled":false,"trusted":true},"cell_type":"code","source":"for idx in range(5):\n    print(idx)\n    mask = masks[idx]\n    contour = contours[idx]\n    cleaned_mask = clean_mask_v2(mask, contour)\n    good_markers = good_markers_v3(cleaned_mask,contour)\n    good_distance = good_distance_v1(cleaned_mask)\n    \n    water = watershed_v3(mask, contour)\n    \n    gt = ground_truth[idx]\n    \n    plot_list(images = [cleaned_mask, good_distance],labels = [good_markers, water,gt])","execution_count":72,"outputs":[]},{"metadata":{"_uuid":"90e0d296d084cd9d985ef6285ea07993d12ab43f","_cell_guid":"cd830bed-fb0e-44f5-a52a-2d551bf7e539"},"cell_type":"markdown","source":"Voila! \n\nThis solution works reasonably well. Of course there is still a lot of room for improvement.\n\nFor cells that get oversegmented we should implement a function that glues those oversegmented cells together rather than drop them. \n\nBut that I will leave for you to create!\n\nIf you liked the solution and would like to see how it works as a part of the end-to-end pipeline please go to:\n\nhttps://github.com/neptune-ml/data-science-bowl-2018\n\nTo stay up to date with new features and fully open sourced solution (with predictions) read our data science bowl journal thread https://www.kaggle.com/c/data-science-bowl-2018/discussion/47590 \n\nCheers and good luck!"},{"metadata":{"_uuid":"29d82be64a37fa176f465a3bbc4a45a6c2ad9d07","_cell_guid":"a1e02632-7096-4465-ad1d-b5e7fdeecc5e"},"cell_type":"markdown","source":"# LB Test set predictions"},{"metadata":{"collapsed":true,"_uuid":"66bf764b791554ce16d834f3a91696050798430f","_cell_guid":"4163191d-aee1-494d-a467-d71f4eda1c5c","trusted":true},"cell_type":"code","source":"test_masks = joblib.load('../input/test-predictions/test_masks.pkl')","execution_count":73,"outputs":[]},{"metadata":{"_uuid":"8166ecc382a143710d58e194819ec00078b08616","_cell_guid":"6c704d55-ab65-47a6-96da-cca560d60d19","trusted":true},"cell_type":"code","source":"from skimage.color import label2rgb\n\nfor mask in test_masks[:5]:\n    plt.imshow(label2rgb(mask))\n    plt.show()","execution_count":74,"outputs":[]},{"metadata":{"_uuid":"60e3954a40da714cc3e9c8b8513c72e2ca0e36b6","collapsed":true,"_cell_guid":"f27ca858-df08-4da1-916f-15d6691bcdb6","trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}