{"cells":[{"metadata":{"_cell_guid":"b8295924-dd43-46c9-bfed-10b2e12e8140","collapsed":true,"_uuid":"b7f8e8fa7e69dd90d6872f6becf01251a5100f2d"},"cell_type":"markdown","source":"# Exploratory Analysis\n\nDuring my undergraduate studies I had biology as one of my subjects and I enjoyed preparing the slides and looking at them using a microscope. So these images are interesting, and I get to learn a bit more about how Deep Learning impact the medical field.\n\nI liked the video overview given by [Dr. Anne Carpenter](https://www.kaggle.com/drannecarpenter) and I am sure I will be able to learn quite a few things by participating in this.\n\nThe images consist of different modalities and magnification as mentioned in this [thread](https://www.kaggle.com/c/data-science-bowl-2018/discussion/47640)\n\n> We want a single model that just works across all kinds of image modalities, no matter the size of the nuclei or the color scheme. Such a model could be built into software that biologists use with all kinds of microscopes and eliminate the need for them to train on their individual data or provide metadata about their cell type, microscope, resolution, etc.\n\n> <cite>[Dr. Anne Carpenter](https://www.kaggle.com/drannecarpenter)</cite>\n\n### Disclaimer\n\nMy knowledge of Computer Vision, Microbiology & Deep Learning is very limited. I have been trying to follow courses and learn these interesting techniques, but I have realized that this is a vast field and there are too many things to learn.\n\n* My approach may not be the right one, but I am still going to try\n* Most of the code here is borrowed from what others have been doing. I have just tweaked a few things here and there and expanded on the steps, so that I can see and understand what is happening behind the scenes.\n* Any suggestions to improve the code here is welcome."},{"metadata":{"_cell_guid":"0bda9050-f7a2-4541-abbd-fbd6206ca0f3","_uuid":"6484b6244ec765e8ab6ffc9780dd22c192cbeeb6"},"cell_type":"markdown","source":"## References\n\n* [Adrian Rosebrock's PyImageSearch](https://www.pyimagesearch.com) which covers great tutorials on OpenCV"},{"metadata":{"_cell_guid":"d53249e4-7a46-4224-bc47-a4d1527b47cd","_uuid":"5375ce9a2d7586c4c8a776f1a259ddc057aeccb1"},"cell_type":"markdown","source":"## Approach\n\nI am a newbie to Deep Learning and Computer Vision and this competition is my playground for learning. I will approach this competition with the limited knowledge that I have, and I will also attempt to incorporate methods that others are experimenting with, if I can grasp them and can implement on my own. \n\nThere are a lot of people who have built interesting kernels which take different approaches to reach the end goal."},{"metadata":{"_cell_guid":"6cba4335-dad1-4f35-b7ab-113798d15bab","_uuid":"f1e78da9446074c2b57a8b651a577787e556bc88"},"cell_type":"markdown","source":"## Questions\n\nHere are the questions I set out to answer in this notebook.\n\n* How many types of images are there?\n* Are the types of images in train and test similar or vastly different?\n* How does the image segmentation and object detection compare to the masks provided for the training set?"},{"metadata":{"_cell_guid":"46fca9ea-c8cd-4e89-bc9d-cc25364026de","collapsed":true,"_uuid":"7330350203589637a82dd3786bc5244f4e9d44a2","trusted":false},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport glob\nimport cv2\nimport math\nimport seaborn as sns\nimport json\n\nsns.set()\nsns.set_palette(\"husl\")","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"5593e351-c81e-475f-8e14-8ca7d8396564","collapsed":true,"_uuid":"794112eeaef6eabde34f954333ca9715696fdc01","trusted":false},"cell_type":"code","source":"TRAIN_PATH = '../input/stage1_train/'\nTEST_PATH = '../input/stage1_test/'\n\nRANDOM_SEED=75\n\nOUTPUT_PATH = './'\nCONTOUR_EXTRACT_MODE = cv2.RETR_TREE","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"6a76f6a0-8093-481b-949e-1bca7dd31e47","_uuid":"3dc985886dd51367ae8a90896eb462b832aa6a0c"},"cell_type":"markdown","source":"Ensuring that only the folders are picked up and any junk files in the same path are ignored"},{"metadata":{"_cell_guid":"d3953aac-9b0e-478f-b47a-f77cb9454520","collapsed":true,"_uuid":"626961b56004acba24c6fcdf412698e432ec7702","trusted":false},"cell_type":"code","source":"train_ids = [x for x in os.listdir(TRAIN_PATH) if os.path.isdir(TRAIN_PATH+x)]\ntest_ids = [x for x in os.listdir(TEST_PATH) if os.path.isdir(TEST_PATH+x)]","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"77c7c18b-ed35-4879-bcf1-88447cedcb73","_uuid":"be592e1f8054eebdd7d096b43f1884c3b278a8f8"},"cell_type":"markdown","source":"Create a pandas dataframe combining all images and marking them as train or test. This way we can do a comparison across all images."},{"metadata":{"_cell_guid":"36e2c08d-1659-4a1e-aff2-3ee25adebae6","_uuid":"a3f9ab1762996b3560be1f1a67fe5986b7704c0d","trusted":false,"collapsed":true},"cell_type":"code","source":"df = pd.DataFrame({'id':train_ids,'train_or_test':'train'})\ndf = df.append(pd.DataFrame({'id':test_ids,'train_or_test':'test'}))\n\ndf.groupby(['train_or_test']).count()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"1b0890bd-f6a8-46ca-af6e-c74a2452e2d7","_uuid":"bf3fab02383c39967f5c5678d500611754bcb899"},"cell_type":"markdown","source":"There are 670 training images and 65 test images making the test set less than 10% of the training set.\n\nBuild the paths for the individual images"},{"metadata":{"_cell_guid":"fb47ef5b-d578-4a24-894b-5686fb82871b","collapsed":true,"_uuid":"cf22cf191b4f27074fa0d1eca593e8e5000307e4","trusted":false},"cell_type":"code","source":"df['path'] = df.apply(lambda x:'../input/stage1_{}/{}/images/{}.png'.format(x[1],x[0],x[0]), axis=1)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"d1d9050a-8a4b-4ddc-9a00-54f42c274c15","_uuid":"186d1a0d7901026b42581eae004edb4af6c52b59"},"cell_type":"markdown","source":"### Observations\n\nI had initially glanced through some of the sample images and there were different types of images. This was before I read the thread on modalities and zooms. \n\nI saw that some were grayscale with black background and nuclei in grayscale intensities. Some of them were in color and some of them seemed to be black on white. Now, I have read that most of the CV tasks work best on grayscale images and it makes sense if all images follow a similar pattern of white on black. The code below helps us cluster the image into two groups. My assumption is that the group that forms the larger cluster is probably the background coolor and the other one is the foreground color. "},{"metadata":{"_cell_guid":"b4b97c81-e050-456f-bb61-52e3421a1bbe","_uuid":"e59439df3239d781bcb247699605870cda6dfd43"},"cell_type":"markdown","source":"Create a histogram and identify the centroid of the group as the color intensity whose variations are most similar to it. \n> <cite>[k-means-color-clustering](https://www.pyimagesearch.com/2014/05/26/opencv-python-k-means-color-clustering/)</cite>"},{"metadata":{"_cell_guid":"ceeb4434-6b52-460a-a5a6-03b001f87836","collapsed":true,"_uuid":"1dba8a74265f24c8c45f71efcaef334d959becf9","trusted":false},"cell_type":"code","source":"from sklearn.cluster import KMeans\n\ndef centroid_histogram(clt):\n    # grab the number of different clusters and create a histogram\n    # based on the number of pixels assigned to each cluster\n    numLabels = np.arange(0, len(np.unique(clt.labels_)) + 1)\n    (hist, _) = np.histogram(clt.labels_, bins = numLabels)\n\n    # normalize the histogram, such that it sums to one\n    hist = hist.astype(\"float\")\n    hist /= hist.sum()\n\n    # return the histogram\n    return hist","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"4f154de3-b0cb-478d-973e-19d2e862906f","_uuid":"3d98a7de8f174ffa2606e2b52710977dd1952f9e"},"cell_type":"markdown","source":"Given a path for an image, load the image and extract the shape attributes. Perform KMeans clustering on the RGB format. Using this extract the background and foreground colors. Since we have the color values for different channels it would be interesting to see how these values are spread across the images. I have calculated the grayscale value as an average of all three colors and this is marked as teh background and foreground color. \n\nIf the foreground color is darker than the background color mark the image to be inverted before processing. \n\nThere are different methods for converting a RGB to grayscale and I can revisit this later if the average value is not good enough."},{"metadata":{"_cell_guid":"de4a768d-494c-48e7-8d22-47b25b796baf","collapsed":true,"_uuid":"dc8e0e39304ba97a0aae3ea4497cf187eb7b2967","trusted":false},"cell_type":"code","source":"def get_image_info(path, clusters=2):\n    image = cv2.imread(path)\n    height,width,_ = image.shape\n    image = cv2.cvtColor(image, cv2.COLOR_BGRA2RGB)\n    image = image.reshape((image.shape[0] * image.shape[1], 3))\n    clt = KMeans(n_clusters = clusters)\n    clt.fit(image)\n    hist = centroid_histogram(clt)\n    \n    bg_idx, fg_idx = 0, clusters-1\n    if hist[bg_idx] < hist[fg_idx]:\n        bg_idx, fg_idx = clusters-1, 0\n    \n    bg_red, bg_green, bg_blue = clt.cluster_centers_[bg_idx]\n    fg_red, fg_green, fg_blue = clt.cluster_centers_[fg_idx]\n    \n    bg_color = sum(clt.cluster_centers_[bg_idx])/3\n    fg_color = sum(clt.cluster_centers_[fg_idx])/3\n    max_color_pct = hist[bg_idx]\n    min_color_pct = hist[fg_idx]\n    \n    return (pd.Series([height,width,\n                       bg_red, bg_green, bg_blue, bg_color,\n                       fg_red, fg_green, fg_blue, fg_color,\n                       hist[bg_idx],hist[fg_idx],\n                       fg_color < bg_color]))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"29bbe2f6-94ce-4b91-945b-09b36db92a3c","_uuid":"edfc84e6239e48104f15e276af0a7e72d3f42a49"},"cell_type":"markdown","source":"Collect information about the images and save it."},{"metadata":{"_cell_guid":"d0e20476-4b1e-487d-b843-145b4108eaa6","collapsed":true,"_uuid":"c79bc7f9604473c93bee9372b3f41c84a3e76a2e","trusted":false},"cell_type":"code","source":"image_info = os.path.join(OUTPUT_PATH,'images.json')\n\nif os.path.isfile(image_info):\n    with open(image_info, 'r') as datafile:\n        data = json.load(datafile)\n        df = pd.read_json(path_or_buf=data, orient='records')\n        data = None\nelse:\n    names = ['height','width',\n             'bg_red', 'bg_green', 'bg_blue','bg_color',\n             'fg_red', 'fg_green', 'fg_blue','fg_color',\n             'bg_color_pct','fg_color_pct','invert']\n\n    df[names] = df['path'].apply(lambda x: get_image_info(x))\n    df['shape'] = df[['height','width']].apply(lambda x: '{:04d}x{:04d}'.format(x[0], x[1]), axis=1)\n\n    with open(image_info, 'w') as outfile:\n        json.dump(df.to_json(orient='records'), outfile)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"e5ade616-1637-4cb0-a82f-04f103795bcb","_uuid":"5d106404de2c9b04a7262348541770c4dd99a369","trusted":false,"collapsed":true},"cell_type":"code","source":"len(df['shape'].unique()),len(df['width'].unique()), len(df['height'].unique())","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"b7b167b3-c545-4465-8917-cd929b8d0119","_uuid":"5bb89d6e28a5325beb986d16459d205b55d2d220"},"cell_type":"markdown","source":"## Distribition of images\n\nTime to play around with some graphs and see how the data is spread across all images."},{"metadata":{"_cell_guid":"87e43792-fd24-45b9-89cb-05fdde172d99","collapsed":true,"_uuid":"4063880db4a921cb95341dede71c7a46d8227fd3","trusted":false},"cell_type":"code","source":"%matplotlib inline\nfrom matplotlib import pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"fff3092c-8e3c-44f5-8f13-d3b4d752dd27","collapsed":true,"_uuid":"16c4445bcb5d927c220c7e32a3a4e16e0bb36c2f","trusted":false},"cell_type":"code","source":"agg = df[['shape','train_or_test','id']].groupby(['shape','train_or_test']).count().unstack()\nagg.columns = agg.columns.droplevel()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"7bf7c908-8c4c-43d7-9971-508c3299ede4","_uuid":"3a5a53b8aa237886320919363a2782dce4c9016e","trusted":false,"collapsed":true},"cell_type":"code","source":"agg.plot.barh(stacked=True,figsize=(16,4))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"f212ad3d-c100-4c67-a84b-7a6240f4302c","_uuid":"c7f31e241c7d65d51289860d892d32203326e831","trusted":false,"collapsed":true},"cell_type":"code","source":"agg[agg['train'].isnull()]","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"89c87fa0-0c2a-4b14-93da-4c5a14d42a10","_uuid":"bad53d3fed48d2256763b4fc11b929ef468bd72c"},"cell_type":"markdown","source":"The distribution is certainly skewed, which I guess is expected. As Anne mentioned there are some image sizes in test set that are not in the training set. These could be of different modality or magnification. There are 7 out of 16 different shapes that are in Test data set only and not in the train data set.\n\nWe can pick a random sample from each image size and have a look."},{"metadata":{"_cell_guid":"9d178f6b-1e90-490d-a486-f222ce3207d3","collapsed":true,"_uuid":"02c0ed85791f19cf138de3844f2fd6644b4b6cc9","trusted":false},"cell_type":"code","source":"def show_image(ax,title,image):\n    ax.grid(None)\n    ax.set_title(title)\n    ax.imshow(image)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"2a1034f7-cfad-4f63-9a36-312900016ee5","collapsed":true,"_uuid":"f73c96217a83adba9428a14e139cda28a1a10ed7","trusted":false},"cell_type":"code","source":"def n_of_each(df, n = 4):\n    shapes = df['shape'].unique()\n    sample = pd.DataFrame()\n    \n    for shape in shapes:\n        sample = sample.append(df[df['shape']==shape].sample(n, replace=True))\n    \n    return sample.sort_values(by=['shape']).reset_index()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"02ab707a-11e2-4dc3-9756-804f955c1716","collapsed":true,"_uuid":"bc923056adbeb9abf3f83583c7b1ce7080f40528","trusted":false},"cell_type":"code","source":"def show_row_col(sample,cols,path_col='path',image_col=None,label_col='title',mode='file'):\n    rows = math.ceil(len(sample)/cols)\n    \n    fig, ax = plt.subplots(rows,cols,figsize=(5*cols,5*rows))\n    \n    for index, data in sample.iterrows():\n    \n        title = data[label_col]\n        if mode=='image':\n            image = np.array(data[image_col],dtype=np.uint8)\n            #image = cv2.cvtColor(image,cv2.COLOR_GRAY2RGB)\n        else:\n            image = cv2.imread(data[path_col])\n            image = cv2.cvtColor(image,cv2.COLOR_BGRA2RGB)\n\n        row = index // cols\n        col = index % cols\n        show_image(ax[row,col],title,image)\n\n    plt.show()    ","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"cad20348-9ad9-42e2-8be9-fd4f6047574f","collapsed":true,"_uuid":"e4bd02ce1645241e0af44a691290ac2081c6540a","trusted":false},"cell_type":"code","source":"sample = n_of_each(df)","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"_cell_guid":"b219bded-0fa1-44ee-aa5b-52648257a535","_uuid":"93eedbf64c42f73dbadcfaf8071d715decc74500","trusted":false,"collapsed":true},"cell_type":"code","source":"sample['label'] = sample[['shape','train_or_test']].apply(lambda x: '{},{}'.format(x[0],x[1]), axis=1)\nshow_row_col(sample,4,path_col='path',label_col='label',mode='file')","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"3563cb8e-ae47-4837-8007-870b4a577c4a","_uuid":"4561979e46218f07fbac23503375399892058350"},"cell_type":"markdown","source":"## Masks\n\nExtract the masks metadata into a separate file. This should include \n\n* mask_id\n* a separate index number for all the mask objects\n* minimum bounding rectangle\n* angle of rotation (longer edge horizontal)\n* masked area\n* length & breadth\n\nBy using contour identification process individual mask objects can be extracted and rotated to a common rectangular shape and we can then get answers to more questions ."},{"metadata":{"_cell_guid":"dfc2ec24-3368-4c3e-aeed-133b80544355","_uuid":"e1df317cbbc7d530d618553abd6217e5a6c6c6ac"},"cell_type":"markdown","source":"Bounded rotation ensures that resulting rotated image is not clipped."},{"metadata":{"_cell_guid":"bfcebdf9-e106-4388-94f0-f3381a0b71f5","collapsed":true,"_uuid":"850f3a47cf0981d4200d189d15ca1cf9aa0cbcd2","trusted":false},"cell_type":"code","source":"def rotate_bound(image, cX, cY, angle, box = None):\n    # grab the dimensions of the image and then determine the\n    # center\n    (h, w) = image.shape[:2]\n    #(cX, cY) = (w // 2, h // 2)\n\n    # grab the rotation matrix (applying the negative of the\n    # angle to rotate clockwise), then grab the sine and cosine\n    # (i.e., the rotation components of the matrix)\n    M = cv2.getRotationMatrix2D((cX, cY), -angle, 1.0)\n    cos = np.abs(M[0, 0])\n    sin = np.abs(M[0, 1])\n\n    # compute the new bounding dimensions of the image\n    nW = int((h * sin) + (w * cos))\n    nH = int((h * cos) + (w * sin))\n\n    # adjust the rotation matrix to take into account translation\n    M[0, 2] += (nW / 2) - cX\n    M[1, 2] += (nH / 2) - cY\n\n    # perform the actual rotation and return the image\n    return cv2.warpAffine(image, M, (nW, nH)) ","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"b49db04f-ed56-48e2-ad6b-2543214cc1bb","collapsed":true,"_uuid":"656a4d55a683bda01d3613835d74a2cc4595a6a2","trusted":false},"cell_type":"code","source":"def rotate_points(points,center,origin,angle):\n    rbox = np.array(points.copy())\n    \n    rbox[:,0] -= center[0]\n    rbox[:,1] -= center[1]\n    \n    cos = math.cos(np.radians(angle))\n    sin = math.sin(np.radians(angle))\n    rmat = [[cos, -sin],\n            [sin, cos]]\n    \n    rbox = np.dot(np.matrix(rbox),rmat)\n    \n    rbox[:,0] += origin[0]  \n    rbox[:,1] += origin[1]\n    \n    return np.array(rbox)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"003ce7a6-6c7f-475b-bbe6-becf96d2e8d1","_uuid":"5b12579f1695ce0ec871043ea7ebe2e362ba91e7"},"cell_type":"markdown","source":"Given a mask and an image perform the following actions\n\n* Identify the contours in the mask\n* Apply mask on the image to leave only the identified objects\n* Identify contours in the mask\n* For each contour extract metadata about the mask\n* Extract the nuclei and rotate it to get a clipped horizontal rectangle"},{"metadata":{"_cell_guid":"2f0c6b1c-75fa-48ab-95a2-ab90d449fbdf","collapsed":true,"_uuid":"ed968e61b371c4d2b04d4f1d738287adfb9bc2e6","trusted":false},"cell_type":"code","source":"def extract(mask, image, mask_id, image_id, frame=3):\n    _, contours, _ = cv2.findContours(mask, CONTOUR_EXTRACT_MODE, cv2.CHAIN_APPROX_NONE)\n\n    data = []\n    \n    if len(image.shape) > 2:\n        all_nuclei = image.copy()\n        for i in range(image.shape[2]):\n            all_nuclei[:,:,i] = np.bitwise_and(all_nuclei[:,:,i],mask)\n    else:\n        all_nuclei = np.bitwise_and(image,mask)\n\n    for contour in contours:\n        ((cx, cy), r) =  cv2.minEnclosingCircle(contour)\n        rect = cv2.minAreaRect(contour)\n        box = cv2.boxPoints(rect)\n        angle = rect[2]\n    \n        x1 = max(0,math.floor(cx-r-frame))\n        y1 = max(0,math.floor(cy-r-frame))\n\n        x2 = min(mask.shape[1],math.ceil(cx+r+frame+1))\n        y2 = min(mask.shape[0],math.ceil(cy+r+frame+1))\n\n        sq_mask =  mask[y1:y2,x1:x2]\n        square =  all_nuclei[y1:y2,x1:x2,:]\n        \n        offset_cx = int(cx-x1)\n        offset_cy = int(cy-y1)\n        \n        # offset the points\n        box[:,0] -= x1\n        box[:,1] -= y1\n        # print(box)\n        \n        rotated = rotate_bound(sq_mask,offset_cx,offset_cy,-angle)\n        nuclei = rotate_bound(square,offset_cx,offset_cy,-angle)\n        rbox = rotate_points(box,(offset_cx,offset_cy),(nuclei.shape[1]/2,nuclei.shape[0]/2),angle)\n\n        xmin, xmax = max(0,math.floor(min(rbox[:,0]))), math.ceil(max(rbox[:,0]))\n        ymin, ymax = max(0,math.floor(min(rbox[:,1]))), math.ceil(max(rbox[:,1]))\n\n        #print(xmin,xmax,ymin,ymax)\n        nuclei = nuclei[ymin:ymax+1,xmin:xmax+1]\n        h = ymax - ymin + 1\n        w = xmax - xmin + 1\n        if h > w:\n            nuclei = rotate_bound(nuclei,int(w/2),int(h/2),90)\n            angle = angle -90\n            #h,w = nuclei.shape\n            \n        data.append({\n            'image_id':image_id,\n            'mask_id':mask_id,\n            'nuclei':nuclei.tolist(),\n            'width':nuclei.shape[1],\n            'height':nuclei.shape[0],\n            'cx':cx,\n            'cy':cy,\n            'square':square.tolist(),\n            'offset_cx':offset_cx,\n            'offset_cy':offset_cy,\n            'radius':r,\n            'box':box.tolist(),\n            'angle':angle\n        })\n        \n    return(data)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"3fd8c6c1-59fe-4c3d-b4a0-b3ed27c56dc4","collapsed":true,"_uuid":"67a46cf534cdc4c251a5c8aa43c198d7c80123e5"},"cell_type":"markdown","source":"Masks are in the masks directory and each mask is in a different file. \n"},{"metadata":{"_cell_guid":"cc29a3d0-92ed-4e4a-949c-0713011762dc","collapsed":true,"_uuid":"ac0cb52168e3ad189a93475bef66f8f679513500","trusted":false},"cell_type":"code","source":"# lambda function to flatten a list\n\nflatten = lambda l: [item for sublist in l for item in sublist]","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"c853517b-8546-4404-b891-687e833399f0","collapsed":true,"_uuid":"44fff108107d0d3ea68ea92873b3bd6c28f76b88","trusted":false},"cell_type":"code","source":"def get_masks(image_id, prefix, invert=False, enhance=True):\n    \n    image_file = '{}/{}/images/{}.png'.format(prefix,image_id,image_id)\n    image = cv2.imread(image_file) #, cv2.IMREAD_GRAYSCALE)\n    image = cv2.cvtColor(image,cv2.COLOR_BGRA2RGB)\n    \n    if enhance:\n        clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(3,3))\n        for channel in range(image.shape[2]):\n            image[:,:,channel] = clahe.apply(image[:,:,channel])\n\n    if invert:\n        image = np.invert(image)\n\n    data = []\n    mask_files = glob.glob(os.path.join('{}/{}/masks/'.format(prefix,image_id),'*.png'))\n    \n    index = 0\n    for mask_file in mask_files:\n        mask = cv2.imread(mask_file, cv2.IMREAD_GRAYSCALE)\n        mask_id = os.path.basename(mask_file).split('.')[0]\n        info = extract(mask, image, mask_id, image_id)\n        \n        data.append(info)\n        index +=1 \n    \n    return(flatten(data))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"fe893165-34a5-46ec-abff-6f91d7f34f03","collapsed":true,"_uuid":"aa0120da6cf5f9a43669cca3584fbf70f465fad9","trusted":false},"cell_type":"code","source":"train = df[df['train_or_test'] == 'train'][['id','invert']]","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"cf367ada-cccf-443a-957f-65fa89c7bff7","collapsed":true,"_uuid":"bcf34d1bed9b228648ccbb7c4b5638dde8f90a95","trusted":false},"cell_type":"code","source":"prefix = '../input/stage1_train'\ndata = train.apply(lambda x: get_masks(x[0],prefix,x[1]), axis = 1)\ndata = flatten(data)\nmasks = pd.DataFrame(data)\nmdf = masks.merge(df[['id','shape','invert']],how='inner', on=None, left_on = 'image_id', right_on='id')","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"f0c3c2eb-fa06-424a-98b3-6e2c49fe898f","_uuid":"10965432f406d28cc6da56856900cf3afcc094b9","trusted":false,"collapsed":true},"cell_type":"code","source":"print(len(masks),list(masks.columns))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"91b4c5ff-6229-4ba7-a2cf-b1e0670308a7","collapsed":true,"_uuid":"d0d74761be5fe977618c3a5bb625680ab0420c47","trusted":false},"cell_type":"code","source":"mdf = masks.merge(df[['id','shape','invert']],how='inner', on=None, left_on = 'image_id', right_on='id')","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"dc38fcce-75d5-4694-9912-e70f2cb73268","_uuid":"e2ee03d9161a9426d010d88b898e76384aa1e417"},"cell_type":"markdown","source":"# Processing Steps\n\nThe images below show the original, one of the masks, masked image and extracted square of the mask.\n"},{"metadata":{"_cell_guid":"ebb9e1d6-8ac1-4fe2-94ca-6d464cb10399","collapsed":true,"_uuid":"cd6272c102b86c6b3e174de4b0954960e52d36d4","trusted":false},"cell_type":"code","source":"def show_process(df):\n    \n    rows = len(df)\n    fix, ax = plt.subplots(rows,6,figsize=(5*6,5*rows))\n    for row,data in df.iterrows():\n\n        image_file = '{}/{}/images/{}.png'.format(prefix,data['image_id'],data['image_id'])\n        image = cv2.imread(image_file)\n        image = cv2.cvtColor(image,cv2.COLOR_BGRA2RGB)\n        show_image(ax[row,0],'original {}'.format(data['shape']),image)\n\n        clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(3,3))\n        enhanced = image.copy()\n        for channel in range(enhanced.shape[2]):\n            enhanced[:,:,channel] = clahe.apply(enhanced[:,:,channel])\n\n        circled = cv2.circle(enhanced, (int(data['cx']),int(data['cy'])), \n                             int(data['radius']+3), \n                             color=(255,255,0), thickness=3) \n        show_image(ax[row,1],'enhanced',circled)\n\n        mask_file = '{}/{}/masks/{}.png'.format(prefix,data['image_id'],data['mask_id'])\n        mask = cv2.imread(mask_file,cv2.IMREAD_GRAYSCALE)\n        \n        circled = cv2.cvtColor(mask,cv2.COLOR_GRAY2RGB)\n        circled = cv2.circle(circled, (int(data['cx']),int(data['cy'])), \n                             int(data['radius']+3), \n                             color=(255,255,0), thickness=3) \n\n        show_image(ax[row,2],'mask',circled)\n\n        masked_image = enhanced.copy()\n        if data['invert']:\n            masked_image = np.invert(masked_image)\n\n        for channel in range(masked_image.shape[2]):\n            masked_image[:,:,channel] = np.bitwise_and(masked_image[:,:,channel],mask)\n\n        circled = cv2.circle(masked_image, (int(data['cx']),int(data['cy'])), \n                             int(data['radius']+3), \n                             color=(0,255,255), thickness=3) \n        show_image(ax[row,3],'inverted & masked',cv2.cvtColor(circled,cv2.COLOR_BGRA2RGB))\n\n        square = np.array(data['square'], dtype=np.uint8)\n        circled = cv2.circle(square, (int(data['offset_cx']),int(data['offset_cy'])), \n                             int(data['radius']+3), \n                             color=(255,255,0), thickness=1) \n        \n        show_image(ax[row,4],'Cropped',circled)\n\n        nuclei = np.array(data['nuclei'], dtype=np.uint8)\n        show_image(ax[row,5],'rotated and clipped',nuclei)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"e7a75b2e-4ae6-415b-b594-7df31c852e96","_uuid":"1d8423f6f6cbc5984b790e0f95effb3df9195531"},"cell_type":"markdown","source":"The images below show the processing steps followed to extract the final nuclei. There is one example of each sample from the training set. I added a circle on the images to show the original location and to show that the nuclei on edges can be clipped based on which edge they are on.\n\n* Original image\n* Image after applying contrast stretch\n* A mask for the image\n* After converting dark on bright to bright on dark and applying the mask\n* Extracting a square. the circle shows the original center.\n* Rotating the image so that long edge is horizontal and clipping the boundaries."},{"metadata":{"_cell_guid":"f7efebf8-03be-4049-b7b6-beb75fc14299","_uuid":"94f07386882c1cbc0a03e5f0f4879b2ae4967203","trusted":false,"collapsed":true},"cell_type":"code","source":"detail = mdf[mdf['image_id'].isin(sample['id'])]\nsample_one = n_of_each(detail,n=1)\n\nshow_process(sample_one)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"24bb1475-45e0-48c7-a87c-db49ee04ba86","_uuid":"e32e3d049f4fcb6b43988aa28aa1630d79cd56a2"},"cell_type":"markdown","source":"I had assumed that each mask is one nucleus. I also assumed that the masked sections are completely solid and would not have holes. I wanted to confirm this assumption, but missed out. While looking at the image extraction progression above, I noticed that in some cases the masked section identified was a part of the mask and not the entire object. In these cases, the process I have used has made me detect more than one contour.\n\nI wonder if creating combining the contours together into a single set will give me a single composite image."},{"metadata":{"_cell_guid":"1a6b2e61-192f-44a4-875e-99308e30628a","_uuid":"ffa2c4f045578c8e87a9f53a43d87268ce286dbf"},"cell_type":"markdown","source":"### Masks with holes"},{"metadata":{"_cell_guid":"6fd51540-fd17-4a7e-a74d-bab51f40e727","_uuid":"a0736e2ac4c15c1b433430e3955e7a97dcc7228f","trusted":false,"collapsed":true},"cell_type":"code","source":"nuclei_by_mask = mdf[['mask_id','image_id']].groupby(['mask_id']).count().reset_index()\nnuclei_by_mask = nuclei_by_mask.rename(columns={\"image_id\": \"count\"})\nnuclei_by_mask = nuclei_by_mask[nuclei_by_mask['count']>1]\nnuclei_by_mask = nuclei_by_mask.sort_values(by=['count'], ascending=False).reset_index()\n\nnuclei_by_mask.head()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"53b044a1-a5ab-415e-8974-a863c8031501","_uuid":"403a47cd918ac1c6ce5a3eb51c02c5632b771ac3"},"cell_type":"markdown","source":"Let's have a look at the mask with 14 elements"},{"metadata":{"scrolled":false,"_cell_guid":"bd414afe-ae39-4b9d-be9b-c932d58ddadc","_uuid":"026e3ae6494426bec6e044daba62d8aacbde1aed","trusted":false,"collapsed":true},"cell_type":"code","source":"data = mdf[mdf['mask_id'] == nuclei_by_mask.loc[0]['mask_id']].copy().reset_index()\n\nshow_process(data)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"2c67f6c0-adfd-4146-aaf9-1bd7eefa1403","_uuid":"1c4eac2260f8640814991426d68a4d83be882079"},"cell_type":"markdown","source":"It looks like the mask has holes, and this may be intentional. CV2 find contours is identifying the parent contour and the holes as child contours. I think there is a way to identify whether the contour is a child or not. If yes I can exclude the child contours."},{"metadata":{"_cell_guid":"a61a6ffb-90f4-4680-b2b9-a5e080054e2f","_uuid":"5538ddab2424d0719591ab0eed2720f3314dd454"},"cell_type":"markdown","source":"Turns out that cv2 has a mode RETR_EXTERNAL which will exclude all child elements and this solves the problem above. "},{"metadata":{"_cell_guid":"491e9ff8-806c-4f6c-bc9e-524e14fb49e7","collapsed":true,"_uuid":"c2b83837c7ca8d6f62869eaec29193aa4bbfe425","trusted":false},"cell_type":"code","source":"holes_in_masks = mdf[mdf['mask_id'].isin(nuclei_by_mask['mask_id'].unique())].copy().reset_index()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"987f7dd9-3d5e-4e40-9d28-f330a67cb226","collapsed":true,"_uuid":"6c4546a365b1fee2232632f3fbf8a24054635b1f","trusted":false},"cell_type":"code","source":"CONTOUR_EXTRACT_MODE = cv2.RETR_EXTERNAL","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"aa770867-6833-469c-8b42-f3bafb10fefa","collapsed":true,"_uuid":"f68cc32bc0975f8015df268ea479d609d8ae1f04","trusted":false},"cell_type":"code","source":"masks_info = os.path.join(OUTPUT_PATH,'masks.json')\ndata = train.apply(lambda x: get_masks(x[0],prefix,x[1]), axis = 1)\ndata = flatten(data)\nmasks = pd.DataFrame(data) \nwith open(masks_info, 'w') as outfile:\n    json.dump(data, outfile)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"407b89af-4595-4f34-9376-1c52c4a973b2","_uuid":"859b252c927996c20cf54001a33f3bdaac3da5bb","trusted":false,"collapsed":true},"cell_type":"code","source":"mdf = masks.merge(df[['id','shape','invert']],how='inner', on=None, left_on = 'image_id', right_on='id')\nlen(mdf)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"685c872f-6f52-4f09-bb01-03fe209ce2f4","_uuid":"7bb65e7fd4774bffa43f8b9fcc2532b56dbb2e4b","trusted":false,"collapsed":true},"cell_type":"code","source":"count_by_mask = mdf[['mask_id','image_id']].groupby(['mask_id']).count().reset_index()\ncount_by_mask = count_by_mask.rename(columns={\"image_id\": \"count\"})\n\ncount_by_mask = count_by_mask[count_by_mask['count']>1].copy().reset_index()\ncount_by_mask","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"dfb9fd67-51bf-4886-ac5b-c3825f76383c","_uuid":"82dbc22dbf3b07e5a4b8bc642e75113e2461b495"},"cell_type":"markdown","source":"But there are still 11 images where the number of contours identified is 2. This means that these masks have a break between two sections and this is causing the"},{"metadata":{"_cell_guid":"de7f63a9-840b-4488-b646-b92ec739bf89","_uuid":"9144c8173c2f1b61852a2f473c0f75b421abd5f9"},"cell_type":"markdown","source":"## Masks with gaps"},{"metadata":{"scrolled":false,"_cell_guid":"fc308494-0c77-48c0-8788-8a3e1e287991","_uuid":"f0dd5b376d422c7125f828bae5abb2fa1351fdac","trusted":false,"collapsed":true},"cell_type":"code","source":"split_masks = mdf[mdf['mask_id'].isin(count_by_mask['mask_id'].unique())].copy().reset_index()\n\nshow_process(split_masks)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"fbd0782f-0a2b-42b1-8190-d4ab8dcfb74b","collapsed":true,"_uuid":"8e842620124326fdc6e3133ee9988d1ac1855c91","trusted":false},"cell_type":"code","source":"def tag_anomaly(label,df):\n    if type(df) == 'pandas.core.series.Series':\n        items = df[['image_id','mask_id']].copy()\n    else:\n        items = df[['image_id','mask_id']].copy().drop_duplicates()\n    items['issue'] = label\n    \n    return(items)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"ada42632-c0d9-4152-9ec0-409a38571d80","collapsed":true,"_uuid":"76127ca19dbf22ad674fd9643c60540568fe853a","trusted":false},"cell_type":"code","source":"anomalies = pd.DataFrame()\nanomalies = anomalies.append(tag_anomaly('Holes in mask',holes_in_masks))\nanomalies = anomalies.append(tag_anomaly('Split mask',split_masks))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"9f3e9af3-f3d3-4afe-84b9-ad5d91820fd4","_uuid":"ae468ec98000c5516592e17772db7ed45d8fed7c","trusted":false,"collapsed":true},"cell_type":"code","source":"agg = mdf[['shape','mask_id']].groupby(['shape']).count()\n#agg.columns = agg.columns.droplevel()\nagg.plot.barh(stacked=True,figsize=(16,4))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"e97de9cf-94ce-40d2-8835-fe50991f9ba6","_uuid":"c69bce4bfa4de1509f93dab2286d24ffaf2fef7e","trusted":false,"collapsed":true},"cell_type":"code","source":"len(mdf[mdf['shape']=='1040x1388'])","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"116606ff-df1b-4318-8625-4f25a04b5362","_uuid":"bb2f00dc7988c875f588e394f2b9ad20ec67a4e0"},"cell_type":"markdown","source":"The distribution of the number of nuclei identified by different shapes is also quite wide. There are four images sizes where the total number of nuclei identified is greater than 2000. "},{"metadata":{"_cell_guid":"9e8edefe-1dc1-40a7-bec1-945dee4ed30c","_uuid":"b067a891fd6ade26bdd6354341e1a287a81d16de","trusted":false,"collapsed":true},"cell_type":"code","source":"agg = mdf[['shape','mask_id','image_id']].groupby(['shape','image_id']).count().reset_index()\nagg.groupby(['shape'])['mask_id'].agg(['mean','min','max','std']).reset_index()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"1ba7668e-f1eb-4fed-811c-7d4213c477d2","collapsed":true,"_uuid":"81da81e54e7926c337f61b0ecb81b1c261425a16","trusted":false},"cell_type":"code","source":"nuclei_sample = n_of_each(mdf,8)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"e21b0d80-4611-4c25-8f0e-8a66c829b4af","_uuid":"b229f34752e63dca1752c3bba6f5768e809f393e","trusted":false,"collapsed":true},"cell_type":"code","source":"len(nuclei_sample)/8","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"_cell_guid":"ffd49246-ece0-441b-93ec-eb34fe35637e","_uuid":"989175ea5b80ae24e2e6775f1826161c1552d071","trusted":false,"collapsed":true},"cell_type":"code","source":"show_row_col(nuclei_sample,8,image_col='nuclei',label_col='shape',mode='image')","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"aabceb2e-e4b3-4818-84ae-19530d1496a4","_uuid":"b5b3191911c12fb192c40b9d2f64e6426e009355"},"cell_type":"markdown","source":"My first attempt at extracting the nuclei ended up with quite a lot of faded objects, so I added the contrast stretching to make the nuclei more visible. Since the sample itself is random, every run may not give the same results. I did see some very small objects which to my eye looked like black patches. "},{"metadata":{"_cell_guid":"145ffadb-054d-4a3e-b3c7-be1855dda634","_uuid":"aaf2b6ae69f1ffe6d5a5ff26f69ae7ff18d8ebe8"},"cell_type":"markdown","source":"## Very Small Nuclei\n\nFiltering the data set by the resulting height and width less than 10 pixels, I want to look at the smaller objects to verify that the code I wrote is not incorrect and these are actually as identified by Dr Anne & team."},{"metadata":{"_cell_guid":"d42938a4-780c-45b8-a2f2-3106736433a4","_uuid":"3c8ec75b4c61b56b33fe3ae9809a7778d536592e","trusted":false,"collapsed":true},"cell_type":"code","source":"len(mdf[mdf['height'] < 3])","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"05a075f2-7bfa-4926-a8a6-942c9471a645","collapsed":true,"_uuid":"cbbc011f5ced63bc52ec736e3c124e6ac0538368","trusted":false},"cell_type":"code","source":"def show_mask(mdf, mask_id, prefix):\n    image_id = mdf[mdf['mask_id']==mask_id]['image_id'].squeeze()\n    image_file = '{}/{}/images/{}.png'.format(prefix,image_id,image_id)\n    mask_file = '{}/{}/masks/{}.png'.format(prefix,image_id,mask_id) \n    \n    image = cv2.imread(image_file)\n    image = cv2.cvtColor(image,cv2.COLOR_BGRA2RGB)\n    \n    mask = cv2.imread(mask_file,cv2.IMREAD_GRAYSCALE)\n    mask = cv2.cvtColor(mask,cv2.COLOR_GRAY2RGB)\n    \n    fig,ax = plt.subplots(1,2,figsize = (20,20))\n    ax[0].grid(None)\n    ax[0].imshow(image)\n    ax[1].set_title('Image')\n    ax[1].grid(None)\n    ax[1].imshow(mask)\n    ax[1].set_title('Mask')","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"2525220b-5cc4-4164-a1e8-5cf2051c40a6","_uuid":"2f295d1644f1101b2bbd07de56b1dbbbed1cd5e7","trusted":false,"collapsed":true},"cell_type":"code","source":"mdf.plot.scatter('width','height')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"a7264ae5-fc92-49b3-abbb-8e668fb8910a","_uuid":"3566c91603fc14466b9632b35c943f6ecdb06e22","trusted":false,"collapsed":true},"cell_type":"code","source":"fig, axes = plt.subplots(2,1, figsize=(20,20))\nsns.violinplot('shape','height', data=mdf, ax = axes[0])\naxes[0].set_title('Height Distribution by Shape')\n\naxes[0].yaxis.grid(True)\naxes[0].set_xlabel('Shape')\naxes[0].set_ylabel('Height')\n\nsns.violinplot('shape','width', data=mdf, ax = axes[1])\naxes[1].set_title('Width Distribution by Shape')\n\naxes[1].yaxis.grid(True)\naxes[1].set_xlabel('Shape')\naxes[1].set_ylabel('Width')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"a4b818b8-d589-47e5-9380-ed2539b1ecf9","_uuid":"58872646cafb319afe182562da94a09e5738fad8"},"cell_type":"markdown","source":"Overall size distribution of the nuclei\n\n* The largest resolution 1040x1388 has larger nuclei width averaging around 110.\n* Most of the remaining shapes have average width between 10 & 30\n* One outlier is the 360x360 where there are groups which are generally bigger than 40, but there a re quite a few which are smaller than 20."},{"metadata":{"_cell_guid":"f04c8af0-30b0-4076-a93b-d1bcd87de600","_uuid":"f6974732e1205e5e97b5a6823304e8b6f0933882","trusted":false,"collapsed":true},"cell_type":"code","source":"small = mdf[mdf['width'] < 10]\n\nfig, axes = plt.subplots(2,1, figsize=(20,20))\nsns.violinplot('shape','height', data=small, ax = axes[0])\naxes[0].set_title('Height Distribution by Shape (Width < 10)')\n\naxes[0].yaxis.grid(True)\naxes[0].set_xlabel('Shape')\naxes[0].set_ylabel('Height')\n\nsns.violinplot('shape','width', data=small, ax = axes[1])\naxes[1].set_title('Width Distribution by Shape')\n\naxes[1].yaxis.grid(True)\naxes[1].set_xlabel('Shape')\naxes[1].set_ylabel('Width')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"33b3d1e1-a0bd-466d-a59b-41327ccd60de","_uuid":"d27d982f07be8dd9d33655bfb13f623af10aae22"},"cell_type":"markdown","source":"Interesting distribution of the nuclei image sizes. When width is less that 10 pixels, the height also is less than 10, however there are images where height is less than 10 but the width is greater than 10\n\n* Smaller sized (<5px) images are mostly in the image sizes 256x256, 256x360, and 360x360 \n* Rest of the images have larger overall images with few outliers in the smaller sizes"},{"metadata":{"_cell_guid":"84dc753a-643d-4ef5-b3e5-40d1deae94a8","_uuid":"e1ff7991529c4ebedb69a3f51d02395836d18267","trusted":false,"collapsed":true},"cell_type":"code","source":"small = mdf[mdf['height'] < 10]\nsmall = small[small['width']> 10]\n\nfig, axes = plt.subplots(2,1, figsize=(20,20))\nsns.violinplot('shape','height', data=small, ax = axes[0])\naxes[0].set_title('Height Distribution by Shape (Height < 10 & Width > 10)')\n\naxes[0].yaxis.grid(True)\naxes[0].set_xlabel('Shape')\naxes[0].set_ylabel('Height')\n\nsns.violinplot('shape','width', data=small, ax = axes[1])\naxes[1].set_title('Width Distribution by Shape (Height < 10 & Width > 10)')\n\naxes[1].yaxis.grid(True)\naxes[1].set_xlabel('Shape')\naxes[1].set_ylabel('Width')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"69357ff2-ac8f-4fcb-a9dc-85aa27d61089","_uuid":"7c8f8f6c323dc922c9d3efc6294574cc76f007ec"},"cell_type":"markdown","source":"There seem to be some extreme cases where height is less than 10pixels but the width is greater than 30"},{"metadata":{"_cell_guid":"a71d766e-bd90-4233-8194-0b7133574b80","_uuid":"212bad8b18b6a1ecc3d745c84a1ac6b2a72a742b","trusted":false,"collapsed":true},"cell_type":"code","source":"small = mdf[(mdf['height'] < 10) & (mdf['width']> 30)]\n\nfig, axes = plt.subplots(2,1, figsize=(20,20))\nsns.violinplot('shape','height', data=small, ax = axes[0])\naxes[0].set_title('Height Distribution by Shape (Height < 10 & Width > 30)')\n\naxes[0].yaxis.grid(True)\naxes[0].set_xlabel('Shape')\naxes[0].set_ylabel('Height')\n\nsns.violinplot('shape','width', data=small, ax = axes[1])\naxes[1].set_title('Width Distribution by Shape (Height < 10 & Width > 30)')\n\naxes[1].yaxis.grid(True)\naxes[1].set_xlabel('Shape')\naxes[1].set_ylabel('Width')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"03fe671c-134f-466a-982b-f8c00e4514ed","_uuid":"f79fda523d9b13c1e0ae4a93357529b770ac3f35","trusted":false,"collapsed":true},"cell_type":"code","source":"small = mdf[mdf['height'] < 10]\n\ng = sns.FacetGrid(small,col=\"shape\", col_wrap=4, size = 4)\ng = g.map(plt.scatter, \"height\", \"width\")","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"bef2d616-6989-4b54-8966-0a269017ef6d","_uuid":"d8c97371826a247492525cd4946e9c1634c28040","trusted":false,"collapsed":true},"cell_type":"code","source":"small = mdf[(mdf['height'] < 5) & (mdf['width'] > 20)]\n\ng = sns.FacetGrid(small,col=\"shape\", col_wrap=4, size = 4)\ng = g.map(plt.scatter, \"height\", \"width\")","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"30e4f012-36e1-4462-bd4e-961b19b8c15e","_uuid":"f901bc7d4bfb81dc633b85c8dc41a48867b6bcdd","trusted":false,"collapsed":true},"cell_type":"code","source":"len(small)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"9449a315-e373-414f-a43e-3bccafb9503c","_uuid":"83e6f954dd163f2764f7ae8bca6f358711e0176e"},"cell_type":"markdown","source":"Ditribution of the nuclei counts identified in images by shape of original image"},{"metadata":{"_cell_guid":"8b651c84-bf21-4a32-9c6c-d95f83472485","_uuid":"04d9aa9520ee1293f99040705ad43adec88f8aba","trusted":false,"collapsed":true},"cell_type":"code","source":"agg = mdf[['image_id','shape','mask_id']].groupby(['shape','image_id']).count().reset_index()\nagg = agg.rename(columns={'mask_id':'mask_count'})\n\ng = sns.FacetGrid(agg ,col=\"shape\", size = 4,  sharex=\"none\", col_wrap=3)\ng = g.map(plt.hist,'mask_count')","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"e47d769a-abc9-4078-8d71-50a84cfd212f","_uuid":"1a5114470bc2541bac3d297516d57454da4878ff"},"cell_type":"markdown","source":"Overall distribution of the number of nuclei annotated by image."},{"metadata":{"_cell_guid":"e5631bd5-dd07-4028-99f9-83a7757eb79c","_uuid":"dd960daa179908dfcc8c73f2d3a0ce06923f9ee2","trusted":false,"collapsed":true},"cell_type":"code","source":"agg = mdf[['image_id','mask_id']].groupby(['image_id']).count().reset_index()\nagg = agg.rename(columns={'mask_id':'mask_count'})\n\nfig, ax = plt.subplots(figsize=(20,5))\nsns.distplot(agg['mask_count'],ax=ax,rug=True)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"071cb3c3-1389-4633-8359-5931a9583d79","_uuid":"fc9333b5f0a33447c39eede2fdcee2c397be1c90","trusted":false,"collapsed":true},"cell_type":"code","source":"agg.loc[agg['mask_count']<10,'mask_count'].plot.hist()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"5fece7ab-7e34-42e6-a3c7-2b5385d103a6","_uuid":"33c025067870e9f3189f319b8e91da5f4eaae41f"},"cell_type":"markdown","source":"There are very few images with nuclei count less than 5"},{"metadata":{"_cell_guid":"868c0608-9478-452b-a3a6-987377673a7d","_uuid":"96c7202f75e6bc7351a9d507cb0b4edb1cab8bdb","trusted":false,"collapsed":true},"cell_type":"code","source":"agg[agg['mask_count']<4]","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"e07119bf-746f-4c1f-ae53-1976344ab225","_uuid":"8cedc7bef6e0ecc9d006f683b24a14f71597b223"},"cell_type":"markdown","source":"Processing details where only a single nuclei is annotated."},{"metadata":{"_cell_guid":"913ab719-e368-4dee-bbbe-bdf7fd5f12e5","_uuid":"fc1c39c06a60b8ad32f54e1111a07e291672d285","trusted":false,"collapsed":true},"cell_type":"code","source":"low_count = mdf[mdf['image_id'].isin(agg.loc[agg['mask_count']<2,'image_id'].unique())].copy().reset_index()\n\nshow_process(low_count)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"f9788c3f-1a21-4032-971e-a95157e7c709","collapsed":true,"_uuid":"673ac6fb1c115e94db7f3646b6d0f20e01ec57e9","trusted":false},"cell_type":"code","source":"anomalies = anomalies.append(tag_anomaly('Invalid Masks',low_count.loc[2]))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"203b1d69-3684-4251-b792-a8977315dc09","_uuid":"77ec797156d401731ce8ef36c567f9f0cbb8333e"},"cell_type":"markdown","source":"The third one in the above set has already been identified an incorrect train file. This is missing annotations. \n\nProcessing details where only two nuclei are annotated."},{"metadata":{"_cell_guid":"5d2391eb-3669-4e0e-aa49-6b3d0b2e5e46","_uuid":"19b5141e9828c3a869b3c389145ed0b32ecac474","trusted":false,"collapsed":true},"cell_type":"code","source":"low_count = mdf[mdf['image_id'].isin(agg.loc[agg['mask_count']==2,'image_id'].unique())].copy().reset_index()\n\nshow_process(low_count)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"d4e01c4e-899b-4c9b-a048-33d891264a13","_uuid":"7eb346b7be95264d0e20f65703032c5d15e402ce"},"cell_type":"markdown","source":"* Image 1: Is this supposed to be two nuclei?\n* Image 3: Is a middle one missing from the masks?"},{"metadata":{"_cell_guid":"923e8be9-d8da-4bdf-9661-dc3a95d5c735","_uuid":"ffb4860943fbcfdc6a7fc668ad63f6f97510ddaa","trusted":false,"collapsed":true},"cell_type":"code","source":"low_count = mdf[mdf['image_id'].isin(agg.loc[agg['mask_count']==3,'image_id'].unique())].copy().reset_index()\n\nshow_process(low_count)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"d6697cc6-09a4-4a04-9d4c-80c0fa2da491","_uuid":"9b71d13b6da7d7fc4828b0aff9c09bda93a78e95"},"cell_type":"markdown","source":"### Anomalies"},{"metadata":{"scrolled":false,"_cell_guid":"c96b5d2e-7dd0-4007-ac79-43e83280fd8a","_uuid":"3684e5fbe3c5aa70d95b77a48863dfc7da064807","trusted":false,"collapsed":true},"cell_type":"code","source":"abnormal = mdf[(mdf['height'] < 4) & \n               (mdf['width'] > 30)].copy().reset_index()\nshow_process(abnormal)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"9d3b072a-2c2a-4c58-865f-0a76cba81363","_uuid":"8f2003066989da6aa23467102e5d546f934321a9"},"cell_type":"markdown","source":"All of these look like incorrect annotation."},{"metadata":{"_cell_guid":"eec90dfb-91c5-46d3-95b2-9d62ae01aff4","collapsed":true,"_uuid":"bcf475cfa6bbc65b7f3d404b20cb5584c2199286","trusted":false},"cell_type":"code","source":"anomalies = anomalies.append(tag_anomaly('Look like lines',abnormal))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"620b7c5e-95af-46a8-8ce1-9480e7e47c72","_uuid":"1e468b433c9299b6e1450d192e57fdc22a45eaaa","trusted":false,"collapsed":true},"cell_type":"code","source":"abnormal = mdf[(mdf['height'] < 6) & \n               (mdf['height'] > 4) & \n               (mdf['width'] > 30)].copy().reset_index()\nlen(abnormal)\nshow_process(abnormal)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"6534b5e3-2082-4e79-b282-691d46f97c8e","collapsed":true,"_uuid":"46f4bc13e96059901efc6e2c72b59ec3ef943562","trusted":false},"cell_type":"code","source":"anomalies = anomalies.append(tag_anomaly('Look like lines',abnormal.loc[0]))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"aa84ef56-b1c0-4d4a-a025-0f055bb520d4","collapsed":true,"_uuid":"3080cd9ca149e7fdc8d1d425bb45ec57ae1faf88","trusted":false},"cell_type":"code","source":"with open(os.path.join(OUTPUT_PATH,'anomalies.json'), 'w') as outfile:\n    json.dump(anomalies.to_json(orient='records'), outfile)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"72c5e810-d45b-4f74-b426-20b7b0f75608","_uuid":"da7f2d8bba7f88f2966f04f772bd02255f7dda7f"},"cell_type":"markdown","source":"Maybe it is possible to detect if the extracted nuclei image dimensions are an outlier or not."},{"metadata":{"_cell_guid":"2668dd21-f1ff-44e5-8994-1714980d8a8d","_uuid":"413eaeaaf57d789da90e563f13fa441ebb3415c6","trusted":false,"collapsed":true},"cell_type":"code","source":"agg = anomalies[['issue','mask_id']].groupby('issue').count()\nagg = agg.rename(columns={'mask_id':'counts'})\nagg","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"7bb3069b-d2f8-466a-bc85-5795c935eb77","_uuid":"e60718138c17985fe17c538facdaee4a1d717e16"},"cell_type":"markdown","source":"## Conclusion\n\nSometimes mistakes help. Using RETR_TREE was a mistake, but it helped me identify masks that have holes. \n\nThere seem to be issues in the data. Some have already been identified, some are yet to be identified. \n\nThere are some missing annotations also as mentioned in this [Thread to post data quality issues](https://www.kaggle.com/c/data-science-bowl-2018/discussion/47770)\n\n> This might be a case where annotators disagree. For me, if I could see that something was a nucleus, even behind a piece of debris like this, I would annotate it. Thankfully, debris like this is not common.\n\n> I agree b1eb0123fe2d8c825694b193efb7b923d95effac9558ee4eaf3116374c2c94fe is missing two nuclei off to the right. I'm also disturbed that there are tiny black dots, indicating holes in the nuclei which should not be there. (I cannot swear to it, but in general nuclei as we have annotated them should NOT have any holes) \n\n> For 19f0653c33982a416feed56e5d1ce6849fd83314fd19dfa1c5b23c6b66e9868a I guess it depends how you generated the image you are showing on the right. You are right that in general they ought to be multi-colored whereas your result indicates the masks are improperly merged. Just not sure whether that is happening in the annotations or in your code that generates the multicolor visualization (I assume the former of course, in which case it's definitely an error). \n\n> For 9bb6e39d5f4415bc7554842ee5d1280403a602f2ba56122b87f453a62d37c06e indeed, that object is an error. I caught many of these before the training set went out but apparently not all! \n\n> For 1f0008060150b5b93084ae2e4dabd160ab80a95ce8071a321b80ec4e33b58aca: Those pairs of nuclei should indeed be split. Those are errors.\n> stage1_train/58c593bcb98386e7fd42a1d34e291db93477624b164e83ab2afa3caa90d1d921 I agree that nucleus ought to have been identified\n\n> I agree that image 12aeefb1b522b283819b12e4cfaf6b13c1264c0aadac3412b4edd2ace304cb40 was badly annotated.\n> 0a7d30b252359a10fd298b638b90cb9ada3acced4e0c0e5a3692013f432ee4e9. Single mask seems to cover two objects"},{"metadata":{"_cell_guid":"7a012d10-79d5-40ed-bcc8-982a8ed91128","collapsed":true,"_uuid":"96872394f7aa8235276a6136bd0907352426d64c","trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}