![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/316192/9211/Train.png\")
Here is an overview of the model that reached place #14 on stage2 with LB 0.578.\nBasically, I trained a few Mask-RCNN models on different stratified folds issued from images clustering. Then I performed ensembling on masks predicted from each model.
\n\nTraining\n
Additional external data (links available in related thread):
\n\nBalanced K-Folds: \nTraining images split in stratified folds with balanced modalities (see kernel published)
\n\nData cleaning: \"data quality issues\" thread was a great source.
\n\nNormalization:
\n\nArchitecture:\nMask-RCNN models with regular augmentations (rotations, flips, scale ...).\nResNet101 first as backbone but finally selected ResNet50 as it was enough for 2 classes only.\nSGD optimizer choosen even if convergence was slower.\n120 epochs minimum before saving the best model.
\n\nExecution\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/316192/9212/Execution.png\")
For each MRCNN model:\nSimple TTA (flip up/down and left/right only) followed by ensembling based on IoU.\nThe rule applied: If all models agree then perform union of masks. If one disagree then discard the mask. This rule provided really stable results during stage#1. Also, IoU criteria was a bit better than NMS on boxes (the other kernel attempt published).
\n\nFinal ensemble and cleaning:\nKeep predictions (union of masks) of each fold based on majority vote.\nFill small holes in predicted masks
\n\nResources:
\n\nResults:\nI think this model can score beyond 0.578 by including BBBC images. \nBTW: I've just started a quick run with only 100 additional images from BBBC6 and limited epochs and I'm already at 0.565.
\n\nEt voila, I had a lot of fun with this competition, I was totally novice in biology domain. It might be a reason why I did not overfit too much :-).\nI tried a few approaches such as UNet and LinkNet with post processing before moving to Mask-RCNN but 1st place solution used UNet so congratulations to topcoders team.\nThanks to Heng CherKeng too for all references , thoughts and code sharing that make the forum animated.
", "messages": [ { "id": "316192", "postDate": "04/18/2018 11:46:26", "content": "Here is an overview of the model that reached place #14 on stage2 with LB 0.578.\nBasically, I trained a few Mask-RCNN models on different stratified folds issued from images clustering. Then I performed ensembling on masks predicted from each model.
\n\nTraining\n
Additional external data (links available in related thread):
\n\nBalanced K-Folds: \nTraining images split in stratified folds with balanced modalities (see kernel published)
\n\nData cleaning: \"data quality issues\" thread was a great source.
\n\nNormalization:
\n\nArchitecture:\nMask-RCNN models with regular augmentations (rotations, flips, scale ...).\nResNet101 first as backbone but finally selected ResNet50 as it was enough for 2 classes only.\nSGD optimizer choosen even if convergence was slower.\n120 epochs minimum before saving the best model.
\n\nExecution\n
For each MRCNN model:\nSimple TTA (flip up/down and left/right only) followed by ensembling based on IoU.\nThe rule applied: If all models agree then perform union of masks. If one disagree then discard the mask. This rule provided really stable results during stage#1. Also, IoU criteria was a bit better than NMS on boxes (the other kernel attempt published).
\n\nFinal ensemble and cleaning:\nKeep predictions (union of masks) of each fold based on majority vote.\nFill small holes in predicted masks
\n\nResources:
\n\nResults:\nI think this model can score beyond 0.578 by including BBBC images. \nBTW: I've just started a quick run with only 100 additional images from BBBC6 and limited epochs and I'm already at 0.565.
\n\nEt voila, I had a lot of fun with this competition, I was totally novice in biology domain. It might be a reason why I did not overfit too much :-).\nI tried a few approaches such as UNet and LinkNet with post processing before moving to Mask-RCNN but 1st place solution used UNet so congratulations to topcoders team.\nThanks to Heng CherKeng too for all references , thoughts and code sharing that make the forum animated.
", "rawMarkdown": "Here is an overview of the model that reached place #14 on stage2 with LB 0.578.\nBasically, I trained a few Mask-RCNN models on different stratified folds issued from images clustering. Then I performed ensembling on masks predicted from each model. \n \n**Training**\n![enter image description here][1]\n\nAdditional external data (links available in [related][2] thread):\n\n - Dataset from TNBC (Breast Cancer) \n - Dataset from nuclei segmentation benchmark\n\nBalanced K-Folds: \nTraining images split in stratified folds with balanced modalities (see [kernel][3] published)\n\nData cleaning: \"data quality issues\" [thread][4] was a great source.\n\n - Bad masks and images excluded. \n - Images available in both training and testing dataset dropped as it may hurt training.\n - Fix small holes in masks with morphology.\n\nNormalization: \n\n- Contrast/luminosity enhancements with dark background. \n- Grayscale conversion: For generalization purpose, to avoid stain specifics in model.\n\nArchitecture:\nMask-RCNN models with regular augmentations (rotations, flips, scale ...).\nResNet101 first as backbone but finally selected ResNet50 as it was enough for 2 classes only.\nSGD optimizer choosen even if convergence was slower.\n120 epochs minimum before saving the best model.\n\n\n**Execution**\n![enter image description here][5]\n\nFor each MRCNN model:\nSimple TTA (flip up/down and left/right only) followed by ensembling based on IoU.\nThe rule applied: If all models agree then perform union of masks. If one disagree then discard the mask. This rule provided really stable results during stage#1. Also, IoU criteria was a bit better than NMS on boxes (the other [kernel][6] attempt published).\n\nFinal ensemble and cleaning:\nKeep predictions (union of masks) of each fold based on majority vote.\nFill small holes in predicted masks\n\nResources:\n\n - 1 to 3 GPU (Tesla K80 12GB)\n - Google VM: n1-standard-8 (8vCPU, 50GB RAM)\n\nResults:\nI think this model can score beyond 0.578 by including BBBC images. \nBTW: I've just started a quick run with only 100 additional images from BBBC6 and limited epochs and I'm already at 0.565.\n\nEt voila, I had a lot of fun with this competition, I was totally novice in biology domain. It might be a reason why I did not overfit too much :-).\nI tried a few approaches such as UNet and LinkNet with post processing before moving to Mask-RCNN but 1st place solution used UNet so congratulations to topcoders team.\nThanks to Heng CherKeng too for all references , thoughts and code sharing that make the forum animated.\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/316192/9211/Train.png\n [2]: https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572\n [3]: https://www.kaggle.com/mpware/stage1-eda-microscope-image-types-clustering\n [4]: https://www.kaggle.com/c/data-science-bowl-2018/discussion/47770\n [5]: https://kaggle2.blob.core.windows.net/forum-message-attachments/316192/9212/Execution.png\n [6]: https://www.kaggle.com/mpware/ensembling-on-instance-segmentation-lb-0-419", "votes": null }, { "id": "316526", "postDate": "04/19/2018 09:28:38", "content": "Thanks for sharing and congratulations on the result! What implementation of Mask-rcnn are you using? yours?
\n\nI was considering using IoU for matching masks coming form different models but did not had time to complete. Can you share your code for this?
", "rawMarkdown": "Thanks for sharing and congratulations on the result! What implementation of Mask-rcnn are you using? yours?\n\nI was considering using IoU for matching masks coming form different models but did not had time to complete. Can you share your code for this?", "votes": null }, { "id": "316548", "postDate": "04/19/2018 10:09:40", "content": "I used both MatterPort (based on Keras/TF) and the one provided by Heng (based on PyTorch) during stage#1. For stage#2 I just used the one from Heng because it was faster.
\n\nSee below for the code. It computes IoU for 3 set of masks:
\n\ndef compute_iou_3d(masks0, masks1, masks2):\n num_masks0 = len(np.unique(masks0))\n num_masks1 = len(np.unique(masks1))\n num_masks2 = len(np.unique(masks2))\n masks = np.array([masks0.flatten(), masks1.flatten(), masks2.flatten()])\n masks = np.moveaxis(masks, [0,1], [1,0])\n\n # Compute intersection between all objects\n intersection_all = np.histogramdd(masks, bins=(num_masks0, num_masks1, num_masks2))[0]\n\n intersection01 = np.histogram2d(masks0.flatten(), masks1.flatten(), bins=(num_masks0, num_masks1))[0]\n intersection02 = np.histogram2d(masks0.flatten(), masks2.flatten(), bins=(num_masks0, num_masks2))[0]\n intersection12 = np.histogram2d(masks1.flatten(), masks2.flatten(), bins=(num_masks1, num_masks2))[0]\n intersection01 = np.expand_dims(intersection01, -1)\n intersection02 = np.expand_dims(intersection02, 1)\n intersection12 = np.expand_dims(intersection12, 0)\n\n area_masks0 = np.histogram(masks0, bins = num_masks0)[0]\n area_masks1 = np.histogram(masks1, bins = num_masks1)[0]\n area_masks2 = np.histogram(masks2, bins = num_masks2)[0]\n\n area_masks0 = np.expand_dims(area_masks0, -1)\n area_masks0 = np.expand_dims(area_masks0, -1)\n area_masks1 = np.expand_dims(area_masks1, -1)\n area_masks1 = np.expand_dims(area_masks1, 0)\n area_masks2 = np.expand_dims(area_masks2, 0)\n area_masks2 = np.expand_dims(area_masks2, 0)\n\n # Compute union (https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle)\n union = area_masks0 + area_masks1 + area_masks2 - intersection01 - intersection02 - intersection12 + intersection_all\n\n # Exclude background\n intersection_all = intersection_all[1:,1:,1:]\n union = union[1:,1:,1:]\n union[union == 0] = 1e-9\n\n # Compute the intersection over union\n iou = intersection_all / union\n\n return iou\nThen you can filter on an IoU threshold and merge masks (here an union):
\n\n# Convert list of mask to index image (one different color per mask)\nmasks0 = masks_array_to_index_image(test_masks_array0)\nmasks1 = masks_array_to_index_image(test_masks_array1)\nmasks2 = masks_array_to_index_image(test_masks_array2)\n\niou = compute_iou_3d(masks0, masks1, masks2)\niou_groups = np.where(iou > IoU_THRESHOLD)\n\nresults = []\nfor group_index in range(0, len(iou_groups[0])):\n m0_group_index = iou_groups[0][group_index]\n m1_group_index = iou_groups[1][group_index]\n m2_group_index = iou_groups[2][group_index]\n iou_group_index = iou[m0_group_index][m1_group_index][m2_group_index]\n result = (group_index, [m0_group_index, m1_group_index, m2_group_index], iou_group_index)\n results.append(result)\n# Sort result by ascending IoU. Tuple is group_index + [matching mask index in each model] + IoU \nresults.sort(key=lambda tup: tup[2]) \n\n# Merge masks\nidx = 0\nresult = results[idx]\nresult_iou = result[2]\nmask0_group_index = test_masks_array0[result[1][0]]\nmask1_group_index = test_masks_array1[result[1][1]]\nmask2_group_index = test_masks_array2[result[1][2]]\nmerge_group_index = mask0_group_index + mask1_group_index + mask2_group_index\nThe compute_iou_3d can be generalized to N but I didn't have time to complete it.
", "rawMarkdown": "I used both MatterPort (based on Keras/TF) and the one provided by Heng (based on PyTorch) during stage#1. For stage#2 I just used the one from Heng because it was faster.\n\nSee below for the code. It computes IoU for 3 set of masks:\n\n\n def compute_iou_3d(masks0, masks1, masks2):\n num_masks0 = len(np.unique(masks0))\n num_masks1 = len(np.unique(masks1))\n num_masks2 = len(np.unique(masks2))\n masks = np.array([masks0.flatten(), masks1.flatten(), masks2.flatten()])\n masks = np.moveaxis(masks, [0,1], [1,0])\n \n # Compute intersection between all objects\n intersection_all = np.histogramdd(masks, bins=(num_masks0, num_masks1, num_masks2))[0]\n \n intersection01 = np.histogram2d(masks0.flatten(), masks1.flatten(), bins=(num_masks0, num_masks1))[0]\n intersection02 = np.histogram2d(masks0.flatten(), masks2.flatten(), bins=(num_masks0, num_masks2))[0]\n intersection12 = np.histogram2d(masks1.flatten(), masks2.flatten(), bins=(num_masks1, num_masks2))[0]\n intersection01 = np.expand_dims(intersection01, -1)\n intersection02 = np.expand_dims(intersection02, 1)\n intersection12 = np.expand_dims(intersection12, 0)\n \n area_masks0 = np.histogram(masks0, bins = num_masks0)[0]\n area_masks1 = np.histogram(masks1, bins = num_masks1)[0]\n area_masks2 = np.histogram(masks2, bins = num_masks2)[0]\n \n area_masks0 = np.expand_dims(area_masks0, -1)\n area_masks0 = np.expand_dims(area_masks0, -1)\n area_masks1 = np.expand_dims(area_masks1, -1)\n area_masks1 = np.expand_dims(area_masks1, 0)\n area_masks2 = np.expand_dims(area_masks2, 0)\n area_masks2 = np.expand_dims(area_masks2, 0)\n \n # Compute union (https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle)\n union = area_masks0 + area_masks1 + area_masks2 - intersection01 - intersection02 - intersection12 + intersection_all\n \n # Exclude background\n intersection_all = intersection_all[1:,1:,1:]\n union = union[1:,1:,1:]\n union[union == 0] = 1e-9\n \n # Compute the intersection over union\n iou = intersection_all / union\n \n return iou\n\nThen you can filter on an IoU threshold and merge masks (here an union):\n\n\n # Convert list of mask to index image (one different color per mask)\n masks0 = masks_array_to_index_image(test_masks_array0)\n masks1 = masks_array_to_index_image(test_masks_array1)\n masks2 = masks_array_to_index_image(test_masks_array2)\n \n iou = compute_iou_3d(masks0, masks1, masks2)\n iou_groups = np.where(iou > IoU_THRESHOLD)\n \n results = []\n for group_index in range(0, len(iou_groups[0])):\n m0_group_index = iou_groups[0][group_index]\n m1_group_index = iou_groups[1][group_index]\n m2_group_index = iou_groups[2][group_index]\n iou_group_index = iou[m0_group_index][m1_group_index][m2_group_index]\n result = (group_index, [m0_group_index, m1_group_index, m2_group_index], iou_group_index)\n results.append(result)\n # Sort result by ascending IoU. Tuple is group_index + [matching mask index in each model] + IoU \n results.sort(key=lambda tup: tup[2]) \n \n # Merge masks\n idx = 0\n result = results[idx]\n result_iou = result[2]\n mask0_group_index = test_masks_array0[result[1][0]]\n mask1_group_index = test_masks_array1[result[1][1]]\n mask2_group_index = test_masks_array2[result[1][2]]\n merge_group_index = mask0_group_index + mask1_group_index + mask2_group_index\n\n\nThe compute_iou_3d can be generalized to N but I didn't have time to complete it.", "votes": null }, { "id": "316584", "postDate": "04/19/2018 11:36:29", "content": "Very helpful, thanks.
", "rawMarkdown": "Very helpful, thanks.", "votes": null }, { "id": "317029", "postDate": "04/20/2018 14:36:19", "content": "@MPWARE Team - Congrats on your results, I looked into your clustering kernel in this competition as well, thanks for the contribution.
", "rawMarkdown": "MPWARE Team - Congrats on your results, I looked into your clustering kernel in this competition as well, thanks for the contribution.", "votes": null }, { "id": "317045", "postDate": "04/20/2018 15:07:26", "content": "Felicitations ! Well designed!
", "rawMarkdown": "Felicitations ! Well designed!", "votes": null }, { "id": "317054", "postDate": "04/20/2018 15:30:42", "content": "Thanks for sharing! Mask_RCNN results varied a lot between experiments; even for the same parameters. Creating an ensemble is a great idea.
\n\nI have been making a table with the matterport Mask_RCNN parameters used by various contestants in DSB-2018.\nThe main goal is to understand which parameters matter more. \nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54920
\n\nIf you have some time, could you send me your parameter selection ? I can add it to the table.
\n\nMehul
", "rawMarkdown": "Thanks for sharing! Mask_RCNN results varied a lot between experiments; even for the same parameters. Creating an ensemble is a great idea.\n\nI have been making a table with the matterport Mask_RCNN parameters used by various contestants in DSB-2018.\nThe main goal is to understand which parameters matter more. \nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54920\n\nIf you have some time, could you send me your parameter selection ? I can add it to the table.\n\nMehul", "votes": null }, { "id": "317060", "postDate": "04/20/2018 15:55:24", "content": "I will try to add it as a PR as soon as I get a chance. Maybe you should also add the \"seed\" used. Same parameters for different seed can lead to different results.
", "rawMarkdown": "I will try to add it as a PR as soon as I get a chance. Maybe you should also add the \"seed\" used. Same parameters for different seed can lead to different results.", "votes": null }, { "id": "320904", "postDate": "04/30/2018 06:41:55", "content": "Thanks for sharing and great job! You mentioned BBBC images. May I ask what BBBC images are and where to find them?
\n\nAndy
", "rawMarkdown": "Thanks for sharing and great job! You mentioned BBBC images. May I ask what BBBC images are and where to find them?\n\nAndy", "votes": null }, { "id": "320922", "postDate": "04/30/2018 07:52:12", "content": "Link is available in \"Official External Data thread\" at https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572
\n\nJust search for BBBC.
", "rawMarkdown": "Link is available in \"Official External Data thread\" at https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572\n\nJust search for BBBC.", "votes": null }, { "id": "320935", "postDate": "04/30/2018 08:41:46", "content": "Thank you so much for the prompt response!
", "rawMarkdown": "Thank you so much for the prompt response!", "votes": null }, { "id": "1604241", "postDate": "12/03/2021 07:35:29", "content": "@mpware
\n. Also, IoU criteria was a bit better than NMS on boxes (the other kernel attempt published).
\nin the kernel u use NMS or or IOU criteria. I feel its a NMS only. What is needed to use IOU criteria there
@mpware
\n. Also, IoU criteria was a bit better than NMS on boxes (the other kernel attempt published).
\nin the kernel u use NMS or or IOU criteria. I feel its a NMS only. What is needed to use IOU criteria there
Thanks for sharing and congratulations on the result! What implementation of Mask-rcnn are you using? yours?
\n\nI was considering using IoU for matching masks coming form different models but did not had time to complete. Can you share your code for this?
", "votes": null, "replies": [ { "id": 316548, "author_name": "mpware", "author_url": "", "post_date": "04/19/2018 10:09:40", "content": "I used both MatterPort (based on Keras/TF) and the one provided by Heng (based on PyTorch) during stage#1. For stage#2 I just used the one from Heng because it was faster.
\n\nSee below for the code. It computes IoU for 3 set of masks:
\n\ndef compute_iou_3d(masks0, masks1, masks2):\n num_masks0 = len(np.unique(masks0))\n num_masks1 = len(np.unique(masks1))\n num_masks2 = len(np.unique(masks2))\n masks = np.array([masks0.flatten(), masks1.flatten(), masks2.flatten()])\n masks = np.moveaxis(masks, [0,1], [1,0])\n\n # Compute intersection between all objects\n intersection_all = np.histogramdd(masks, bins=(num_masks0, num_masks1, num_masks2))[0]\n\n intersection01 = np.histogram2d(masks0.flatten(), masks1.flatten(), bins=(num_masks0, num_masks1))[0]\n intersection02 = np.histogram2d(masks0.flatten(), masks2.flatten(), bins=(num_masks0, num_masks2))[0]\n intersection12 = np.histogram2d(masks1.flatten(), masks2.flatten(), bins=(num_masks1, num_masks2))[0]\n intersection01 = np.expand_dims(intersection01, -1)\n intersection02 = np.expand_dims(intersection02, 1)\n intersection12 = np.expand_dims(intersection12, 0)\n\n area_masks0 = np.histogram(masks0, bins = num_masks0)[0]\n area_masks1 = np.histogram(masks1, bins = num_masks1)[0]\n area_masks2 = np.histogram(masks2, bins = num_masks2)[0]\n\n area_masks0 = np.expand_dims(area_masks0, -1)\n area_masks0 = np.expand_dims(area_masks0, -1)\n area_masks1 = np.expand_dims(area_masks1, -1)\n area_masks1 = np.expand_dims(area_masks1, 0)\n area_masks2 = np.expand_dims(area_masks2, 0)\n area_masks2 = np.expand_dims(area_masks2, 0)\n\n # Compute union (https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle)\n union = area_masks0 + area_masks1 + area_masks2 - intersection01 - intersection02 - intersection12 + intersection_all\n\n # Exclude background\n intersection_all = intersection_all[1:,1:,1:]\n union = union[1:,1:,1:]\n union[union == 0] = 1e-9\n\n # Compute the intersection over union\n iou = intersection_all / union\n\n return iou\nThen you can filter on an IoU threshold and merge masks (here an union):
\n\n# Convert list of mask to index image (one different color per mask)\nmasks0 = masks_array_to_index_image(test_masks_array0)\nmasks1 = masks_array_to_index_image(test_masks_array1)\nmasks2 = masks_array_to_index_image(test_masks_array2)\n\niou = compute_iou_3d(masks0, masks1, masks2)\niou_groups = np.where(iou > IoU_THRESHOLD)\n\nresults = []\nfor group_index in range(0, len(iou_groups[0])):\n m0_group_index = iou_groups[0][group_index]\n m1_group_index = iou_groups[1][group_index]\n m2_group_index = iou_groups[2][group_index]\n iou_group_index = iou[m0_group_index][m1_group_index][m2_group_index]\n result = (group_index, [m0_group_index, m1_group_index, m2_group_index], iou_group_index)\n results.append(result)\n# Sort result by ascending IoU. Tuple is group_index + [matching mask index in each model] + IoU \nresults.sort(key=lambda tup: tup[2]) \n\n# Merge masks\nidx = 0\nresult = results[idx]\nresult_iou = result[2]\nmask0_group_index = test_masks_array0[result[1][0]]\nmask1_group_index = test_masks_array1[result[1][1]]\nmask2_group_index = test_masks_array2[result[1][2]]\nmerge_group_index = mask0_group_index + mask1_group_index + mask2_group_index\nThe compute_iou_3d can be generalized to N but I didn't have time to complete it.
", "votes": null, "replies": [] }, { "id": 316584, "author_name": "cpmpml", "author_url": "", "post_date": "04/19/2018 11:36:29", "content": "Very helpful, thanks.
", "votes": null, "replies": [] } ] }, { "id": 317029, "author_name": "nicolefinnie", "author_url": "", "post_date": "04/20/2018 14:36:19", "content": "@MPWARE Team - Congrats on your results, I looked into your clustering kernel in this competition as well, thanks for the contribution.
", "votes": null, "replies": [] }, { "id": 317045, "author_name": "chabir", "author_url": "", "post_date": "04/20/2018 15:07:26", "content": "Felicitations ! Well designed!
", "votes": null, "replies": [] }, { "id": 317054, "author_name": "mpsampat", "author_url": "", "post_date": "04/20/2018 15:30:42", "content": "Thanks for sharing! Mask_RCNN results varied a lot between experiments; even for the same parameters. Creating an ensemble is a great idea.
\n\nI have been making a table with the matterport Mask_RCNN parameters used by various contestants in DSB-2018.\nThe main goal is to understand which parameters matter more. \nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54920
\n\nIf you have some time, could you send me your parameter selection ? I can add it to the table.
\n\nMehul
", "votes": null, "replies": [ { "id": 317060, "author_name": "mpware", "author_url": "", "post_date": "04/20/2018 15:55:24", "content": "I will try to add it as a PR as soon as I get a chance. Maybe you should also add the \"seed\" used. Same parameters for different seed can lead to different results.
", "votes": null, "replies": [] } ] }, { "id": 320904, "author_name": "andyzhang50", "author_url": "", "post_date": "04/30/2018 06:41:55", "content": "Thanks for sharing and great job! You mentioned BBBC images. May I ask what BBBC images are and where to find them?
\n\nAndy
", "votes": null, "replies": [ { "id": 320922, "author_name": "mpware", "author_url": "", "post_date": "04/30/2018 07:52:12", "content": "Link is available in \"Official External Data thread\" at https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572
\n\nJust search for BBBC.
", "votes": null, "replies": [] }, { "id": 320935, "author_name": "andyzhang50", "author_url": "", "post_date": "04/30/2018 08:41:46", "content": "Thank you so much for the prompt response!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "316192": "Here is an overview of the model that reached place #14 on stage2 with LB 0.578.\nBasically, I trained a few Mask-RCNN models on different stratified folds issued from images clustering. Then I performed ensembling on masks predicted from each model. \n \n**Training**\n![enter image description here][1]\n\nAdditional external data (links available in [related][2] thread):\n\n - Dataset from TNBC (Breast Cancer) \n - Dataset from nuclei segmentation benchmark\n\nBalanced K-Folds: \nTraining images split in stratified folds with balanced modalities (see [kernel][3] published)\n\nData cleaning: \"data quality issues\" [thread][4] was a great source.\n\n - Bad masks and images excluded. \n - Images available in both training and testing dataset dropped as it may hurt training.\n - Fix small holes in masks with morphology.\n\nNormalization: \n\n- Contrast/luminosity enhancements with dark background. \n- Grayscale conversion: For generalization purpose, to avoid stain specifics in model.\n\nArchitecture:\nMask-RCNN models with regular augmentations (rotations, flips, scale ...).\nResNet101 first as backbone but finally selected ResNet50 as it was enough for 2 classes only.\nSGD optimizer choosen even if convergence was slower.\n120 epochs minimum before saving the best model.\n\n\n**Execution**\n![enter image description here][5]\n\nFor each MRCNN model:\nSimple TTA (flip up/down and left/right only) followed by ensembling based on IoU.\nThe rule applied: If all models agree then perform union of masks. If one disagree then discard the mask. This rule provided really stable results during stage#1. Also, IoU criteria was a bit better than NMS on boxes (the other [kernel][6] attempt published).\n\nFinal ensemble and cleaning:\nKeep predictions (union of masks) of each fold based on majority vote.\nFill small holes in predicted masks\n\nResources:\n\n - 1 to 3 GPU (Tesla K80 12GB)\n - Google VM: n1-standard-8 (8vCPU, 50GB RAM)\n\nResults:\nI think this model can score beyond 0.578 by including BBBC images. \nBTW: I've just started a quick run with only 100 additional images from BBBC6 and limited epochs and I'm already at 0.565.\n\nEt voila, I had a lot of fun with this competition, I was totally novice in biology domain. It might be a reason why I did not overfit too much :-).\nI tried a few approaches such as UNet and LinkNet with post processing before moving to Mask-RCNN but 1st place solution used UNet so congratulations to topcoders team.\nThanks to Heng CherKeng too for all references , thoughts and code sharing that make the forum animated.\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/316192/9211/Train.png\n [2]: https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572\n [3]: https://www.kaggle.com/mpware/stage1-eda-microscope-image-types-clustering\n [4]: https://www.kaggle.com/c/data-science-bowl-2018/discussion/47770\n [5]: https://kaggle2.blob.core.windows.net/forum-message-attachments/316192/9212/Execution.png\n [6]: https://www.kaggle.com/mpware/ensembling-on-instance-segmentation-lb-0-419", "316526": "Thanks for sharing and congratulations on the result! What implementation of Mask-rcnn are you using? yours?\n\nI was considering using IoU for matching masks coming form different models but did not had time to complete. Can you share your code for this?", "316548": "I used both MatterPort (based on Keras/TF) and the one provided by Heng (based on PyTorch) during stage#1. For stage#2 I just used the one from Heng because it was faster.\n\nSee below for the code. It computes IoU for 3 set of masks:\n\n\n def compute_iou_3d(masks0, masks1, masks2):\n num_masks0 = len(np.unique(masks0))\n num_masks1 = len(np.unique(masks1))\n num_masks2 = len(np.unique(masks2))\n masks = np.array([masks0.flatten(), masks1.flatten(), masks2.flatten()])\n masks = np.moveaxis(masks, [0,1], [1,0])\n \n # Compute intersection between all objects\n intersection_all = np.histogramdd(masks, bins=(num_masks0, num_masks1, num_masks2))[0]\n \n intersection01 = np.histogram2d(masks0.flatten(), masks1.flatten(), bins=(num_masks0, num_masks1))[0]\n intersection02 = np.histogram2d(masks0.flatten(), masks2.flatten(), bins=(num_masks0, num_masks2))[0]\n intersection12 = np.histogram2d(masks1.flatten(), masks2.flatten(), bins=(num_masks1, num_masks2))[0]\n intersection01 = np.expand_dims(intersection01, -1)\n intersection02 = np.expand_dims(intersection02, 1)\n intersection12 = np.expand_dims(intersection12, 0)\n \n area_masks0 = np.histogram(masks0, bins = num_masks0)[0]\n area_masks1 = np.histogram(masks1, bins = num_masks1)[0]\n area_masks2 = np.histogram(masks2, bins = num_masks2)[0]\n \n area_masks0 = np.expand_dims(area_masks0, -1)\n area_masks0 = np.expand_dims(area_masks0, -1)\n area_masks1 = np.expand_dims(area_masks1, -1)\n area_masks1 = np.expand_dims(area_masks1, 0)\n area_masks2 = np.expand_dims(area_masks2, 0)\n area_masks2 = np.expand_dims(area_masks2, 0)\n \n # Compute union (https://en.wikipedia.org/wiki/Inclusion%E2%80%93exclusion_principle)\n union = area_masks0 + area_masks1 + area_masks2 - intersection01 - intersection02 - intersection12 + intersection_all\n \n # Exclude background\n intersection_all = intersection_all[1:,1:,1:]\n union = union[1:,1:,1:]\n union[union == 0] = 1e-9\n \n # Compute the intersection over union\n iou = intersection_all / union\n \n return iou\n\nThen you can filter on an IoU threshold and merge masks (here an union):\n\n\n # Convert list of mask to index image (one different color per mask)\n masks0 = masks_array_to_index_image(test_masks_array0)\n masks1 = masks_array_to_index_image(test_masks_array1)\n masks2 = masks_array_to_index_image(test_masks_array2)\n \n iou = compute_iou_3d(masks0, masks1, masks2)\n iou_groups = np.where(iou > IoU_THRESHOLD)\n \n results = []\n for group_index in range(0, len(iou_groups[0])):\n m0_group_index = iou_groups[0][group_index]\n m1_group_index = iou_groups[1][group_index]\n m2_group_index = iou_groups[2][group_index]\n iou_group_index = iou[m0_group_index][m1_group_index][m2_group_index]\n result = (group_index, [m0_group_index, m1_group_index, m2_group_index], iou_group_index)\n results.append(result)\n # Sort result by ascending IoU. Tuple is group_index + [matching mask index in each model] + IoU \n results.sort(key=lambda tup: tup[2]) \n \n # Merge masks\n idx = 0\n result = results[idx]\n result_iou = result[2]\n mask0_group_index = test_masks_array0[result[1][0]]\n mask1_group_index = test_masks_array1[result[1][1]]\n mask2_group_index = test_masks_array2[result[1][2]]\n merge_group_index = mask0_group_index + mask1_group_index + mask2_group_index\n\n\nThe compute_iou_3d can be generalized to N but I didn't have time to complete it.", "316584": "Very helpful, thanks.", "317029": "MPWARE Team - Congrats on your results, I looked into your clustering kernel in this competition as well, thanks for the contribution.", "317045": "Felicitations ! Well designed!", "317054": "Thanks for sharing! Mask_RCNN results varied a lot between experiments; even for the same parameters. Creating an ensemble is a great idea.\n\nI have been making a table with the matterport Mask_RCNN parameters used by various contestants in DSB-2018.\nThe main goal is to understand which parameters matter more. \nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54920\n\nIf you have some time, could you send me your parameter selection ? I can add it to the table.\n\nMehul", "317060": "I will try to add it as a PR as soon as I get a chance. Maybe you should also add the \"seed\" used. Same parameters for different seed can lead to different results.", "320904": "Thanks for sharing and great job! You mentioned BBBC images. May I ask what BBBC images are and where to find them?\n\nAndy", "320922": "Link is available in \"Official External Data thread\" at https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572\n\nJust search for BBBC.", "320935": "Thank you so much for the prompt response!", "1604241": "mpware \n`. Also, IoU criteria was a bit better than NMS on boxes (the other kernel attempt published).`\nin the kernel u use NMS or or IOU criteria. I feel its a NMS only. What is needed to use IOU criteria there" }, "source": "meta" }