{
  "id": 50795,
  "title": "Strategy for winning",
  "url": "/competitions/data-science-bowl-2018/discussion/50795",
  "author_name": "",
  "post_date": "2018-03-01T10:23:10.728823600Z",
  "votes": 99,
  "comment_count": 50,
  "views": 0,
  "content": "<p>In summary, this competition is about accuracy and generalization to unseen data. I will talk about \"generalization\" later. Here are some tips on building an accurate system, see each section below.</p>\n\n<p>(this thread will be constantly updated)</p>",
  "messages": [
    {
      "id": "289297",
      "postDate": "03/01/2018 10:23:10",
      "content": "<p>In summary, this competition is about accuracy and generalization to unseen data. I will talk about \"generalization\" later. Here are some tips on building an accurate system, see each section below.</p>\n\n<p>(this thread will be constantly updated)</p>",
      "rawMarkdown": "In summary, this competition is about accuracy and generalization to unseen data. I will talk about \"generalization\" later. Here are some tips on building an accurate system, see each section below.\n\n(this thread will be constantly updated)",
      "votes": null
    },
    {
      "id": "289298",
      "postDate": "03/01/2018 10:23:47",
      "content": "<p>overall system: how to make ensemble?</p>\n\n<ul>\n<li><p>it is easier to design small system and combine them together</p></li>\n<li><p>just need to ensemble them together. Results are cumulative\n(if you need a single system for practical use, you can always convert an ensemble system to single network using knowledge distill, teacher-student learning later) </p></li>\n<li><p>how about nuclei that are not detected by any of the CNN? You still have semantic segmentation like Unet to fall back on! (this is about 99% at pixel level!) </p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/289298/8656/system.png\" alt=\"enter image description here\"></p></li>\n</ul>",
      "rawMarkdown": "overall system: how to make ensemble?\n\n  -  it is easier to design small system and combine them together\n\n  - just need to ensemble them together. Results are cumulative\n    (if you need a single system for practical use, you can always convert an ensemble system to single network using knowledge distill, teacher-student learning later) \n\n  - how about nuclei that are not detected by any of the CNN? You still have semantic segmentation like Unet to fall back on! (this is about 99% at pixel level!) \n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/289298/8656/system.png",
      "votes": null
    },
    {
      "id": "289300",
      "postDate": "03/01/2018 10:35:28",
      "content": "<p>as a reference (COCO object detection at ilsvrc 2016 ):\n<a href=\"http://image-net.org/challenges/talks/2016/GRMI-COCO-slidedeck.pdf\">http://image-net.org/challenges/talks/2016/GRMI-COCO-slidedeck.pdf</a></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/289300/8657/ensmble.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "as a reference (COCO object detection at ilsvrc 2016 ):\nhttp://image-net.org/challenges/talks/2016/GRMI-COCO-slidedeck.pdf\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/289300/8657/ensmble.png",
      "votes": null
    },
    {
      "id": "289306",
      "postDate": "03/01/2018 10:55:05",
      "content": "<p>some baseline results  for reference:</p>\n\n<ul>\n<li><p>unet (single class, i.e. semantic segmentation) : 0.35</p></li>\n<li><p>enhance unet (multi-class with post processing) : 0.42</p></li>\n<li><p>mask-rcnn (instance segmentation) : 0.50</p></li>\n</ul>\n\n<hr>\n\n<p>The LB metric is based on image, not based on object. You should take care of test images with few nuclei first. They are the easiest, and accuracy contribution per nucleus object is higher in these images.</p>\n\n<hr>\n\n<p>An increase of 0.001 in LB score is about detecting 2 previously missed nuclei on one image (assuming that the image has about 50 nuclei object).</p>\n\n<hr>\n\n<p>Even if you have perfect precision 1.0 for bounding box (threshold at 0.5 IOU), the mask average precision (i.e. LB metric) is about 0.85 to 0.90 in my experiments. This is because there is many small nuclei, meaning that if you miss one pixel for the mask prediction, you will have an FP (false positive) at high IOU. You may want to think about if you want to include prediction for small nuclei in LB submission or not</p>\n\n<hr>\n\n<p>mask average precision is as follows:</p>\n\n<ul>\n<li><p>training (gray images only) : 0.69</p></li>\n<li><p>validation (gray images only, same kind of cells) : 0.69</p></li>\n<li><p>training (purple images only) : 0.55</p></li>\n<li><p>validation (purple images only, same kind of cells) : 0.41</p></li>\n</ul>",
      "rawMarkdown": "some baseline results  for reference:\n\n- unet (single class, i.e. semantic segmentation) : 0.35\n\n- enhance unet (multi-class with post processing) : 0.42\n\n- mask-rcnn (instance segmentation) : 0.50\n\n---\n\nThe LB metric is based on image, not based on object. You should take care of test images with few nuclei first. They are the easiest, and accuracy contribution per nucleus object is higher in these images.\n\n---\n\nAn increase of 0.001 in LB score is about detecting 2 previously missed nuclei on one image (assuming that the image has about 50 nuclei object).\n\n---\n\nEven if you have perfect precision 1.0 for bounding box (threshold at 0.5 IOU), the mask average precision (i.e. LB metric) is about 0.85 to 0.90 in my experiments. This is because there is many small nuclei, meaning that if you miss one pixel for the mask prediction, you will have an FP (false positive) at high IOU. You may want to think about if you want to include prediction for small nuclei in LB submission or not\n\n---\n\nmask average precision is as follows:\n\n  - training (gray images only) : 0.69\n\n  - validation (gray images only, same kind of cells) : 0.69\n\n  - training (purple images only) : 0.55\n\n  - validation (purple images only, same kind of cells) : 0.41",
      "votes": null
    },
    {
      "id": "289309",
      "postDate": "03/01/2018 11:05:02",
      "content": "<p>It is easier to perfect your boxing box detection first. i,e you still train a mask-rcnn with 3 loss, but pay more attention to the box label and regression loss. (you can also freeze the rpn and rcnn and finetune/add more parameters your mask head later). Try to train your mask rcnn to about &gt;0.90 box precision first.</p>\n\n<p>The trick to get accurate box detection is \"anchor box\". Have more anchor of the \"correct size\" and label them correctly for training. Note that there are thin objects and overlaping object, causing problems of background become FP and partial objects become FP respectively, see attached.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/289309/8659/FP-1.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "It is easier to perfect your boxing box detection first. i,e you still train a mask-rcnn with 3 loss, but pay more attention to the box label and regression loss. (you can also freeze the rpn and rcnn and finetune/add more parameters your mask head later). Try to train your mask rcnn to about &gt;0.90 box precision first.\n\nThe trick to get accurate box detection is \"anchor box\". Have more anchor of the \"correct size\" and label them correctly for training. Note that there are thin objects and overlaping object, causing problems of background become FP and partial objects become FP respectively, see attached.\n\n   ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/289309/8659/FP-1.png",
      "votes": null
    },
    {
      "id": "289340",
      "postDate": "03/01/2018 12:19:03",
      "content": "<p>This is what I have to wait for long time. I have one question about encoding RLE. After we ran masking, we obtained instance segmentation such as mask1 mask2, ..., maskN for object ID 1. When we encode RLE, we have two ways:</p>\n\n<p>First way: each row encode a mask so for object ID 1 we have</p>\n\n<h1>ImageId,EncodedPixels</h1>\n\n<h1>1,  9 2 ... encode for mask1</h1>\n\n<h1>1, 3 5 ...encode for mask2</h1>\n\n<h1>...</h1>\n\n<h1>1, 6 7 ...encode for maskN</h1>\n\n<p>Second way: combine all instance masks into one mask and then encode it</p>\n\n<h1>ImageId,EncodedPixels</h1>\n\n<h1>1,  9 2 3 5 6 7 encode for mask1 to maskN combination for ID 1</h1>\n\n<h1>2, 3 5  encode for mask1 to maskM combination for ID 2</h1>\n\n<p>The first way does not follow the rule of the challenge that said that each row is for a subject ID, but it shows that one id has multiple rows, each row represents its instance map encoding. The second way looks good but it cannot achieve the goal of the challenge that focusing on instance segmentation. Which way are you using?</p>",
      "rawMarkdown": "This is what I have to wait for long time. I have one question about encoding RLE. After we ran masking, we obtained instance segmentation such as mask1 mask2, ..., maskN for object ID 1. When we encode RLE, we have two ways:\n\nFirst way: each row encode a mask so for object ID 1 we have\n\n#ImageId,EncodedPixels\n#1,  9 2 ... encode for mask1\n#1, 3 5 ...encode for mask2\n#...\n#1, 6 7 ...encode for maskN\n\nSecond way: combine all instance masks into one mask and then encode it\n\n#ImageId,EncodedPixels\n#1,  9 2 3 5 6 7 encode for mask1 to maskN combination for ID 1\n#2, 3 5  encode for mask1 to maskM combination for ID 2\n\nThe first way does not follow the rule of the challenge that said that each row is for a subject ID, but it shows that one id has multiple rows, each row represents its instance map encoding. The second way looks good but it cannot achieve the goal of the challenge that focusing on instance segmentation. Which way are you using?",
      "votes": null
    },
    {
      "id": "289556",
      "postDate": "03/01/2018 21:21:59",
      "content": "<p>Heng, thanks for the interesting post!</p>\n\n<p>Two questions:</p>\n\n<pre><code>still train a mask-rcnn with 3 loss\n</code></pre>\n\n<p>what is 3 loss here?</p>\n\n<pre><code>Box overlap is used to label +ve  and -ve training samples\n</code></pre>\n\n<p>what is +ve and -ve here?</p>",
      "rawMarkdown": "Heng, thanks for the interesting post!\n\nTwo questions:\n\n    still train a mask-rcnn with 3 loss\nwhat is 3 loss here?\n\n\n    Box overlap is used to label +ve  and -ve training samples\nwhat is +ve and -ve here?",
      "votes": null
    },
    {
      "id": "289559",
      "postDate": "03/01/2018 21:24:48",
      "content": "<p>Extremely helpful! Thanks for sharing.</p>",
      "rawMarkdown": "Extremely helpful! Thanks for sharing.",
      "votes": null
    },
    {
      "id": "289645",
      "postDate": "03/02/2018 01:38:43",
      "content": "<p>if you need a learning based NMS, check these out!</p>\n\n<p>\"Learning non-maximum suppression\" - Jan Hosang, Rodrigo Benenson, Bernt Schiele, <a href=\"https://arxiv.org/abs/1705.02950\">https://arxiv.org/abs/1705.02950</a></p>\n\n<p>\"A CONVNET FOR NON-MAXIMUM SUPPRESSION\" - Jan Hosang, Rodrigo Benenson, and Bernt Schiele, <a href=\"https://arxiv.org/pdf/1511.06437.pdf\">https://arxiv.org/pdf/1511.06437.pdf</a></p>",
      "rawMarkdown": "if you need a learning based NMS, check these out!\n\n\"Learning non-maximum suppression\" - Jan Hosang, Rodrigo Benenson, Bernt Schiele, https://arxiv.org/abs/1705.02950\n\n\n\"A CONVNET FOR NON-MAXIMUM SUPPRESSION\" - Jan Hosang, Rodrigo Benenson, and Bernt Schiele, https://arxiv.org/pdf/1511.06437.pdf",
      "votes": null
    },
    {
      "id": "289744",
      "postDate": "03/02/2018 06:49:46",
      "content": "<p>you can merge giver train patches into full image and re-crop to create more training samples. an example is shown below. both train and test LB images can be merged. (for merging code you can refer to the Amazon Forest Kaggle competition)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/289744/8663/merge.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "you can merge giver train patches into full image and re-crop to create more training samples. an example is shown below. both train and test LB images can be merged. (for merging code you can refer to the Amazon Forest Kaggle competition)\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/289744/8663/merge.png",
      "votes": null
    },
    {
      "id": "289750",
      "postDate": "03/02/2018 07:03:56",
      "content": "<p>For cropping, if I train on cropped images such as 128x128 to obtain trained weight. For inference, I assume the input size is 512x512. Then I also want to test on 128x128 sub-images. How can I merge the final result to obtain final prediction of 512x512 from results of sub-images 128x128? Thanks</p>",
      "rawMarkdown": "For cropping, if I train on cropped images such as 128x128 to obtain trained weight. For inference, I assume the input size is 512x512. Then I also want to test on 128x128 sub-images. How can I merge the final result to obtain final prediction of 512x512 from results of sub-images 128x128? Thanks",
      "votes": null
    },
    {
      "id": "291469",
      "postDate": "03/06/2018 07:26:22",
      "content": "<p>in coco evaluation, there is box precision (detection) and mask instance  segmentation precision.</p>\n\n<p>Here is my results on mask-rcnn-resnet34 trained on 10 gray images and test on 43 gray images.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/291469/8695/box_and_mask1.png\" alt=\"enter image description here\"></p>\n\n<p>you can see that box precision is very high. this is the upper bound for mask LB metric.</p>\n\n<hr>\n\n<p>here is a good case.  Even so, there is a gap between box precision (evaluated at 0.50 only) and the mask LB metric (averaged over iou 0.50 to 1.00)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/291469/8697/box_and_mask2.png\" alt=\"enter image description here\"></p>\n\n<hr>\n\n<p>here is a bad case: some masks are missed (even though the box are detected) because of my threshold and wrong non-max suppression. Hence post-processing is important. you may want to spend time on this.</p>\n\n<p>Note that the red region(4th image labelled as iou) are missing prediction or predicted mask that have iou less than 0.5 which become FP at evaluation and bring down the LB metric score.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/291469/8696/box_and_mask3.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "in coco evaluation, there is box precision (detection) and mask instance  segmentation precision.\n\nHere is my results on mask-rcnn-resnet34 trained on 10 gray images and test on 43 gray images.\n\n  ![enter image description here][1]\n\n  you can see that box precision is very high. this is the upper bound for mask LB metric.\n\n---\n\nhere is a good case.  Even so, there is a gap between box precision (evaluated at 0.50 only) and the mask LB metric (averaged over iou 0.50 to 1.00)\n\n  ![enter image description here][2]\n\n\n---\n\nhere is a bad case: some masks are missed (even though the box are detected) because of my threshold and wrong non-max suppression. Hence post-processing is important. you may want to spend time on this.\n\nNote that the red region(4th image labelled as iou) are missing prediction or predicted mask that have iou less than 0.5 which become FP at evaluation and bring down the LB metric score.\n\n  ![enter image description here][3]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/291469/8695/box_and_mask1.png\n  [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/291469/8697/box_and_mask2.png\n  [3]: https://kaggle2.blob.core.windows.net/forum-message-attachments/291469/8696/box_and_mask3.png",
      "votes": null
    },
    {
      "id": "291567",
      "postDate": "03/06/2018 11:52:00",
      "content": "<p>see also:</p>\n\n<p><a href=\"https://www.kaggle.com/c/data-science-bowl-2018/discussion/51184\">https://www.kaggle.com/c/data-science-bowl-2018/discussion/51184</a></p>\n\n<p><a href=\"https://www.kaggle.com/bonlime/train-test-image-mosaic\">https://www.kaggle.com/bonlime/train-test-image-mosaic</a></p>\n\n<p>thanks to @Emil !!!</p>",
      "rawMarkdown": "see also:\n\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/51184\n\nhttps://www.kaggle.com/bonlime/train-test-image-mosaic\n\nthanks to @Emil !!!",
      "votes": null
    },
    {
      "id": "291579",
      "postDate": "03/06/2018 12:07:51",
      "content": "<p>@Heng CherKeng: When you experiment, does the cluster dataset helps improve performance comparison with no-clustering (using both color and gray for training)</p>",
      "rawMarkdown": "Heng CherKeng: When you experiment, does the cluster dataset helps improve performance comparison with no-clustering (using both color and gray for training)",
      "votes": null
    },
    {
      "id": "291653",
      "postDate": "03/06/2018 15:23:08",
      "content": "<p>yes, it improves results</p>",
      "rawMarkdown": "yes, it improves results",
      "votes": null
    },
    {
      "id": "291654",
      "postDate": "03/06/2018 15:25:28",
      "content": "<blockquote>\n  <blockquote>\n    <p>what is 3 loss here?</p>\n  </blockquote>\n</blockquote>\n\n<p>rpn loss, rcnn loss and mask loss</p>\n\n<blockquote>\n  <blockquote>\n    <p>what is +ve and -ve here?</p>\n  </blockquote>\n</blockquote>\n\n<p>positive and negative labels for training rpn and rcnn</p>",
      "rawMarkdown": "&gt;&gt; what is 3 loss here?\n\nrpn loss, rcnn loss and mask loss\n\n&gt;&gt; what is +ve and -ve here?\n\npositive and negative labels for training rpn and rcnn",
      "votes": null
    },
    {
      "id": "291667",
      "postDate": "03/06/2018 16:07:45",
      "content": "<p>example of learning a \"post processing network\" (e.g. for nms or ensembling)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/291667/8706/post%20processing%20netowrk.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "example of learning a \"post processing network\" (e.g. for nms or ensembling)\n\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/291667/8706/post%20processing%20netowrk.png",
      "votes": null
    },
    {
      "id": "291673",
      "postDate": "03/06/2018 16:19:48",
      "content": "<p>You can check my latest code at the pytorch mask rcnn thread. It has submission code to show how to predict instance mask for full image and encode them for submission csv</p>",
      "rawMarkdown": "You can check my latest code at the pytorch mask rcnn thread. It has submission code to show how to predict instance mask for full image and encode them for submission csv",
      "votes": null
    },
    {
      "id": "291816",
      "postDate": "03/06/2018 22:35:34",
      "content": "<p>another variant of the solution is to let the the output of the post processing net be probability(pixel belong to mask-i, mask-j or none). In this case you learn how to handle overlapping pixel of a pair of predicted mask.  </p>\n\n<p>Be consistent and naming i and j, e.g. i is always the mask nearest to top-left and j is nearest to bottom-right. </p>",
      "rawMarkdown": "another variant of the solution is to let the the output of the post processing net be probability(pixel belong to mask-i, mask-j or none). In this case you learn how to handle overlapping pixel of a pair of predicted mask.  \n\nBe consistent and naming i and j, e.g. i is always the mask nearest to top-left and j is nearest to bottom-right.",
      "votes": null
    },
    {
      "id": "292056",
      "postDate": "03/07/2018 10:06:39",
      "content": "<p>example of test time augmentation. Results of ensemble of two is mask metric = 0.41478.\nother possible test augment includes scale, flip, shift, color/illumination change  ...</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/292056/8707/tta.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "example of test time augmentation. Results of ensemble of two is mask metric = 0.41478.\nother possible test augment includes scale, flip, shift, color/illumination change  ...\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/292056/8707/tta.png",
      "votes": null
    },
    {
      "id": "292162",
      "postDate": "03/07/2018 15:05:42",
      "content": "<p>sample balancing in training:</p>\n\n<ul>\n<li><p>Train a deep network on train dataset with uniform sampling</p></li>\n<li><p>Use the trained network to label the difficulty of each image of the dataset (e.g. you can use LB metric)</p></li>\n<li><p>Modify your batch sampler so that the more difficult are sampled more frequently</p></li>\n<li><p>Train with new modified batch sampler</p></li>\n<li><p>Repeat the process</p></li>\n</ul>",
      "rawMarkdown": "sample balancing in training:\n\n-  Train a deep network on train dataset with uniform sampling\n\n-  Use the trained network to label the difficulty of each image of the dataset (e.g. you can use LB metric)\n\n- Modify your batch sampler so that the more difficult are sampled more frequently\n\n- Train with new modified batch sampler\n\n- Repeat the process",
      "votes": null
    },
    {
      "id": "292672",
      "postDate": "03/08/2018 12:03:09",
      "content": "<p>so what is the most important loss - mask loss??</p>",
      "rawMarkdown": "so what is the most important loss - mask loss??",
      "votes": null
    },
    {
      "id": "292706",
      "postDate": "03/08/2018 13:39:35",
      "content": "<p>The diagram shows a case where there is more correct detection results but worst LB metric. \nThis is because the mask prediction is not accurate enough.</p>\n\n<p>At high IOU metric computation it is penalized twice (one for being a FP and another for being miss)</p>",
      "rawMarkdown": "The diagram shows a case where there is more correct detection results but worst LB metric. \nThis is because the mask prediction is not accurate enough.\n\nAt high IOU metric computation it is penalized twice (one for being a FP and another for being miss)",
      "votes": null
    },
    {
      "id": "292900",
      "postDate": "03/08/2018 21:13:20",
      "content": "<p>I think you(Heng CherKeng)have just defined bootstrapping for CNN . may be out there. first time i saw it. very clever. you should make into general solution and publish.</p>",
      "rawMarkdown": "I think you(Heng CherKeng)have just defined bootstrapping for CNN . may be out there. first time i saw it. very clever. you should make into general solution and publish.",
      "votes": null
    },
    {
      "id": "292902",
      "postDate": "03/08/2018 21:14:42",
      "content": "<p>i just figured out how to reply. my apologies.</p>",
      "rawMarkdown": "i just figured out how to reply. my apologies.",
      "votes": null
    },
    {
      "id": "293101",
      "postDate": "03/09/2018 06:47:43",
      "content": "<p>Good job...Thank you.</p>",
      "rawMarkdown": "Good job...Thank you.",
      "votes": null
    },
    {
      "id": "293232",
      "postDate": "03/09/2018 13:23:53",
      "content": "<p>@Heng CherKeng: I am working in the same problem as you mentioned. Let see your example, the box 1 detects two masks: <code>1</code> and <code>2</code>. The box 2 detects 1 mask : <code>3</code>. The problem is that the mask 2nd and 3rd are in the same mask if we look at the whole image. How we know that and how we can combine them together in final result. Thanks for give me a clear example </p>",
      "rawMarkdown": "Heng CherKeng: I am working in the same problem as you mentioned. Let see your example, the box 1 detects two masks: `1` and `2`. The box 2 detects 1 mask : `3`. The problem is that the mask 2nd and 3rd are in the same mask if we look at the whole image. How we know that and how we can combine them together in final result. Thanks for give me a clear example",
      "votes": null
    },
    {
      "id": "293570",
      "postDate": "03/10/2018 06:32:36",
      "content": "<p>test on external images(randomly download from the web)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/113660/4d35e23daf1646f28901458155600bf9/external_test.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "test on external images(randomly download from the web)\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/inbox/113660/4d35e23daf1646f28901458155600bf9/external_test.png",
      "votes": null
    },
    {
      "id": "293848",
      "postDate": "03/10/2018 18:54:46",
      "content": "<p>I believe it is like Heng CherKeng metioned: </p>\n\n<blockquote>\n  <p>\"The LB metric is based on image, not based on object. You should take\n  care of test images with few nuclei first. They are the easiest, and\n  accuracy contribution per nucleus object is higher in these images.\"</p>\n</blockquote>\n\n<p>This happens because the competition chose to use the RLE encoding as the second form you mentioned. \nIf the first way were chosen, the LB would be computed based on instaces instead of images.</p>",
      "rawMarkdown": "I believe it is like Heng CherKeng metioned: \n\n&gt; \"The LB metric is based on image, not based on object. You should take\n&gt; care of test images with few nuclei first. They are the easiest, and\n&gt; accuracy contribution per nucleus object is higher in these images.\"\n\nThis happens because the competition chose to use the RLE encoding as the second form you mentioned. \nIf the first way were chosen, the LB would be computed based on instaces instead of images.",
      "votes": null
    },
    {
      "id": "294038",
      "postDate": "03/11/2018 05:56:24",
      "content": "<p>FYI: how to ensemble objector detector from resnet paper:</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/113660/ac6394a8a6ba4682a5edd37452649d98/ensemble.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "FYI: how to ensemble objector detector from resnet paper:\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/inbox/113660/ac6394a8a6ba4682a5edd37452649d98/ensemble.png",
      "votes": null
    },
    {
      "id": "294146",
      "postDate": "03/11/2018 11:58:46",
      "content": "<p>@Heng,</p>\n\n<blockquote>\n  <p>you can also freeze the rpn and rcnn and finetune/add more parameters your mask head later</p>\n</blockquote>\n\n<p>What do you refer with freeze the rcnn?  I understood the whole net except the mask fcn.</p>\n\n<p>Using matterport, then I should freeze all the net with the exception of: mrcnn_mask_loss, proposal_targets and mrcnn_mask (layer's name).  Isn't true?</p>\n\n<p>thanks a lot for sharing!</p>",
      "rawMarkdown": "Heng,\n\n&gt; you can also freeze the rpn and rcnn and finetune/add more parameters your mask head later\n\nWhat do you refer with freeze the rcnn?  I understood the whole net except the mask fcn.\n\nUsing matterport, then I should freeze all the net with the exception of: mrcnn_mask_loss, proposal_targets and mrcnn_mask (layer's name).  Isn't true?\n\nthanks a lot for sharing!",
      "votes": null
    },
    {
      "id": "294161",
      "postDate": "03/11/2018 12:24:45",
      "content": "<p>It is the first case. I have tested it. However, I did not understand what is the meaning of \"based on image not object\". If I understand correctly, it means if I select a good LB results (such as 53 gray images) in total of 65 images, then the LB only compute for these 53 images and I can got better peformance than using all 65 images. Am I right?</p>",
      "rawMarkdown": "It is the first case. I have tested it. However, I did not understand what is the meaning of \"based on image not object\". If I understand correctly, it means if I select a good LB results (such as 53 gray images) in total of 65 images, then the LB only compute for these 53 images and I can got better peformance than using all 65 images. Am I right?",
      "votes": null
    },
    {
      "id": "294167",
      "postDate": "03/11/2018 12:48:56",
      "content": "<p>Yes, I think you are right but I'm not sure if you can do this.. When you submitt your preditions you would be pennalized for the missing predictions anyway, I guess.. Although I've never tried</p>",
      "rawMarkdown": "Yes, I think you are right but I'm not sure if you can do this.. When you submitt your preditions you would be pennalized for the missing predictions anyway, I guess.. Although I've never tried",
      "votes": null
    },
    {
      "id": "294237",
      "postDate": "03/11/2018 16:04:45",
      "content": "<p>To detect label noise or abnormalities, check the train error.</p>\n\n<p>Then you can relabel the train images. I draw the miss and fp nuclei. This is useful:</p>\n\n<ul>\n<li><p>i can check if the train, validation and LB test error are similar or not</p></li>\n<li><p>i will know which train image are difficult (helps me to sample the images for making random batch for sgd)</p></li>\n<li><p>i can relabel for noises</p></li>\n<li><p>i make sure that my validation set does not have label error, so that i have fair comparison between different system</p></li>\n<li><p>i can estimate LB label noise (assuming it is same as training label noise). Hence I will know if two kaggler scored are statistically similar or not. It also let me estimate the highest possible LB score and whether i should trust LB score or valdiation score more.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/113660/63ceec30d4d742a8b426749480834153/error.png\" alt=\"enter image description here\"></p></li>\n</ul>",
      "rawMarkdown": "To detect label noise or abnormalities, check the train error.\n\nThen you can relabel the train images. I draw the miss and fp nuclei. This is useful:\n\n - i can check if the train, validation and LB test error are similar or not\n \n - i will know which train image are difficult (helps me to sample the images for making random batch for sgd)\n\n - i can relabel for noises\n\n - i make sure that my validation set does not have label error, so that i have fair comparison between different system\n\n - i can estimate LB label noise (assuming it is same as training label noise). Hence I will know if two kaggler scored are statistically similar or not. It also let me estimate the highest possible LB score and whether i should trust LB score or valdiation score more.\n\n  \n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/inbox/113660/63ceec30d4d742a8b426749480834153/error.png",
      "votes": null
    },
    {
      "id": "294598",
      "postDate": "03/12/2018 07:54:23",
      "content": "<p>how do you ensemble the two results:original and flip</p>",
      "rawMarkdown": "how do you ensemble the two results:original and flip",
      "votes": null
    },
    {
      "id": "294738",
      "postDate": "03/12/2018 14:50:57",
      "content": "<p>you can change your mask-head!</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/113660/b93cdf0622b74264903a7db58b8c14d9/m1.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/113660/4ac2df07c502403a9bd986f4504d6ba8/m2.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "you can change your mask-head!\n\n  ![enter image description here][1]\n\n   ![enter image description here][2]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/inbox/113660/b93cdf0622b74264903a7db58b8c14d9/m1.png\n  [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/inbox/113660/4ac2df07c502403a9bd986f4504d6ba8/m2.png",
      "votes": null
    },
    {
      "id": "294917",
      "postDate": "03/12/2018 20:32:36",
      "content": "<p>Hi Heng, did you add some color items in augmentation to achieve this result? Just modifying a purple test image to yellow (with any image editor) leads to bad prediction (totally different masks) for my Mask RCNN. </p>",
      "rawMarkdown": "Hi Heng, did you add some color items in augmentation to achieve this result? Just modifying a purple test image to yellow (with any image editor) leads to bad prediction (totally different masks) for my Mask RCNN.",
      "votes": null
    },
    {
      "id": "297605",
      "postDate": "03/17/2018 14:22:02",
      "content": "<p>@xiapidan: Have you find the way? I am using bitwise_or for the same position of cells but the performance reduces from 0.473 to 0.45 :(</p>",
      "rawMarkdown": "xiapidan: Have you find the way? I am using bitwise_or for the same position of cells but the performance reduces from 0.473 to 0.45 :(",
      "votes": null
    },
    {
      "id": "297645",
      "postDate": "03/17/2018 15:47:55",
      "content": "<p>I guess just average the probabilities, not bitwise_or for the masks</p>",
      "rawMarkdown": "I guess just average the probabilities, not bitwise_or for the masks",
      "votes": null
    },
    {
      "id": "298070",
      "postDate": "03/18/2018 17:38:48",
      "content": "<p>this is how instance segmentation results from mask rcnn  looks like. It is easy to see that you can further refine and improve the mask boundary using post processing. In fact classify many classic methods can be used because the texture is \"rather simple\". </p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/298070/8802/3582166ee20755856adf4882a8bfacb616fce4247911605a109c4862de421bcd.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "this is how instance segmentation results from mask rcnn  looks like. It is easy to see that you can further refine and improve the mask boundary using post processing. In fact classify many classic methods can be used because the texture is \"rather simple\". \n\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/298070/8802/3582166ee20755856adf4882a8bfacb616fce4247911605a109c4862de421bcd.png",
      "votes": null
    },
    {
      "id": "298091",
      "postDate": "03/18/2018 18:55:34",
      "content": "<p>Hi Heng, what kind of post processing do you prefer? I have try dilation, active contour, fill hole, remove small cell but they are hurt my performance. </p>",
      "rawMarkdown": "Hi Heng, what kind of post processing do you prefer? I have try dilation, active contour, fill hole, remove small cell but they are hurt my performance.",
      "votes": null
    },
    {
      "id": "298327",
      "postDate": "03/19/2018 11:21:01",
      "content": "<p>padding / mosaicing is very important (this is a common dark knowledge for people who do segmentation before).</p>\n\n<p>Deep network is the weakest at the boundary image and strongest are the center. Below shows mask rcnn results for the same network, but for padded image, unpadded image, difference of the two.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/298327/8811/pad-0.png\" alt=\"enter image description here\"></p>\n\n<p>i suggest kaggler to read the solutions of past kaggle segmentation challenges like carvana car and satellite images. There are many more tricks!</p>",
      "rawMarkdown": "padding / mosaicing is very important (this is a common dark knowledge for people who do segmentation before).\n\nDeep network is the weakest at the boundary image and strongest are the center. Below shows mask rcnn results for the same network, but for padded image, unpadded image, difference of the two.\n\n ![enter image description here][1]\n\ni suggest kaggler to read the solutions of past kaggle segmentation challenges like carvana car and satellite images. There are many more tricks!\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/298327/8811/pad-0.png",
      "votes": null
    },
    {
      "id": "298387",
      "postDate": "03/19/2018 13:53:22",
      "content": "<p>for each crop feature, do some augmentation (e.g. flip ). for each augmented feature, predict mask.\nhere is how it look like:</p>\n\n<p>green: average of all mask prediction.\nred: prediction for augmented crop features</p>\n\n<pre><code>            augment_image, augment_rcnn_proposal = augment_function(pad_image, rcnn_proposal)\n            input       = Variable(torch.from_numpy(augment_image.transpose((2,0,1))).float().div(255).unsqueeze(0)).cuda()\n            proposal   = Variable(torch.from_numpy(augment_rcnn_proposal)).cuda()\n\n            feature    = data_parallel(net.feature_net, input)\n            mask_crop  = net.mask_crop(feature, proposal)\n            mask_logit = data_parallel(net.mask_head, mask_crop)\n</code></pre>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/298387/8815/average1.png\" alt=\"enter image description here\"></p>",
      "rawMarkdown": "for each crop feature, do some augmentation (e.g. flip ). for each augmented feature, predict mask.\nhere is how it look like:\n\ngreen: average of all mask prediction.\nred: prediction for augmented crop features\n\n                augment_image, augment_rcnn_proposal = augment_function(pad_image, rcnn_proposal)\n                input       = Variable(torch.from_numpy(augment_image.transpose((2,0,1))).float().div(255).unsqueeze(0)).cuda()\n                proposal   = Variable(torch.from_numpy(augment_rcnn_proposal)).cuda()\n\n                feature    = data_parallel(net.feature_net, input)\n                mask_crop  = net.mask_crop(feature, proposal)\n                mask_logit = data_parallel(net.mask_head, mask_crop)\n\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/298387/8815/average1.png",
      "votes": null
    },
    {
      "id": "298388",
      "postDate": "03/19/2018 13:57:06",
      "content": "<p>border for the mask nuclei is too small. you can see some \"cropping effects\". try to change to larger crop (i.e. larger than 28x28) and more pixels for the \"border\"</p>",
      "rawMarkdown": "border for the mask nuclei is too small. you can see some \"cropping effects\". try to change to larger crop (i.e. larger than 28x28) and more pixels for the \"border\"",
      "votes": null
    },
    {
      "id": "298394",
      "postDate": "03/19/2018 14:03:18",
      "content": "<p>Thanks for your idea. Do you mean we will insert it in the below</p>\n\n<p>if len(self.detections)&gt;0:</p>\n\n<pre><code>          mask_crops = self.mask_crop(features, self.detections)\n\n          self.mask_logits = data_parallel(self.mask_head, mask_crops)\n\n          self.masks = mask_nms(cfg, mode, inputs, self.detections, self.mask_logits) #\n</code></pre>\n\n<p>Could you show your <code>augment_function</code>?</p>",
      "rawMarkdown": "Thanks for your idea. Do you mean we will insert it in the below\n\n\nif len(self.detections)&gt;0:\n\n              mask_crops = self.mask_crop(features, self.detections)\n\n              self.mask_logits = data_parallel(self.mask_head, mask_crops)\n\n              self.masks = mask_nms(cfg, mode, inputs, self.detections, self.mask_logits) #\n\nCould you show your `augment_function`?",
      "votes": null
    },
    {
      "id": "298399",
      "postDate": "03/19/2018 14:19:36",
      "content": "<p>Thanks for your suggestion. I will try. Actually, I have tried the matterport implementation in Keras that use crop size of 512x512 and frozen batch norm (batch size of 1) (due to using pretrain model), then the postprocessing help so much (+0.03 LB). Unfortunately, your code does not support pretrain model, so that we have use the batch size as much as possible, hence we cannot use crop size bigger than 256. I hope the next version can solve it by using pretrain model and frozen batch norm. Thanks for your great work, your current mask segmentation branch is quite good, so it is not good to apply dilation any more :)</p>",
      "rawMarkdown": "Thanks for your suggestion. I will try. Actually, I have tried the matterport implementation in Keras that use crop size of 512x512 and frozen batch norm (batch size of 1) (due to using pretrain model), then the postprocessing help so much (+0.03 LB). Unfortunately, your code does not support pretrain model, so that we have use the batch size as much as possible, hence we cannot use crop size bigger than 256. I hope the next version can solve it by using pretrain model and frozen batch norm. Thanks for your great work, your current mask segmentation branch is quite good, so it is not good to apply dilation any more :)",
      "votes": null
    },
    {
      "id": "298515",
      "postDate": "03/19/2018 16:49:33",
      "content": "<p>you can actually use pretrain model. just change the featureNet to the pretrain model network pytorch definition and see load_pretrain function</p>\n\n<p>alternatively, after training from scratch, freeze the model partially to increase batch size</p>",
      "rawMarkdown": "you can actually use pretrain model. just change the featureNet to the pretrain model network pytorch definition and see load_pretrain function\n\nalternatively, after training from scratch, freeze the model partially to increase batch size",
      "votes": null
    },
    {
      "id": "300908",
      "postDate": "03/22/2018 03:25:06",
      "content": "<p>@Heng, you mean increase <code>mask_crop_size</code> by modify the configuration,  or increase <code>mask_size</code> (which is now 28) by adding another deconv layer in mask head</p>",
      "rawMarkdown": "Heng, you mean increase `mask_crop_size` by modify the configuration,  or increase `mask_size` (which is now 28) by adding another deconv layer in mask head",
      "votes": null
    },
    {
      "id": "305436",
      "postDate": "03/28/2018 21:38:03",
      "content": "<p>Hi Heng, do you use nms to do TTA?</p>",
      "rawMarkdown": "Hi Heng, do you use nms to do TTA?",
      "votes": null
    },
    {
      "id": "309766",
      "postDate": "04/05/2018 23:50:48",
      "content": "<p>I know this might be a little outdated, but if we want to use random crops of mosaics for training, how can we merge the masks properly to create a single mask for the entire mosaic, this way the ground truth for the random crop can be extrapolated from the single mask?</p>",
      "rawMarkdown": "I know this might be a little outdated, but if we want to use random crops of mosaics for training, how can we merge the masks properly to create a single mask for the entire mosaic, this way the ground truth for the random crop can be extrapolated from the single mask?",
      "votes": null
    },
    {
      "id": "309783",
      "postDate": "04/06/2018 01:20:02",
      "content": "<p>This paper here <a href=\"https://www.nature.com/articles/s41598-018-22181-4\">https://www.nature.com/articles/s41598-018-22181-4</a>\ncovers it. It may be worth reading it!\nIt does not uses random crops but you can simply change their implementation.</p>",
      "rawMarkdown": "This paper here https://www.nature.com/articles/s41598-018-22181-4\ncovers it. It may be worth reading it!\nIt does not uses random crops but you can simply change their implementation.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 289298,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/01/2018 10:23:47",
      "content": "<p>overall system: how to make ensemble?</p>\n\n<ul>\n<li><p>it is easier to design small system and combine them together</p></li>\n<li><p>just need to ensemble them together. Results are cumulative\n(if you need a single system for practical use, you can always convert an ensemble system to single network using knowledge distill, teacher-student learning later) </p></li>\n<li><p>how about nuclei that are not detected by any of the CNN? You still have semantic segmentation like Unet to fall back on! (this is about 99% at pixel level!) </p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/289298/8656/system.png\" alt=\"enter image description here\"></p></li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 289645,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "03/02/2018 01:38:43",
          "content": "<p>if you need a learning based NMS, check these out!</p>\n\n<p>\"Learning non-maximum suppression\" - Jan Hosang, Rodrigo Benenson, Bernt Schiele, <a href=\"https://arxiv.org/abs/1705.02950\">https://arxiv.org/abs/1705.02950</a></p>\n\n<p>\"A CONVNET FOR NON-MAXIMUM SUPPRESSION\" - Jan Hosang, Rodrigo Benenson, and Bernt Schiele, <a href=\"https://arxiv.org/pdf/1511.06437.pdf\">https://arxiv.org/pdf/1511.06437.pdf</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 289300,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/01/2018 10:35:28",
      "content": "<p>as a reference (COCO object detection at ilsvrc 2016 ):\n<a href=\"http://image-net.org/challenges/talks/2016/GRMI-COCO-slidedeck.pdf\">http://image-net.org/challenges/talks/2016/GRMI-COCO-slidedeck.pdf</a></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/289300/8657/ensmble.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 289306,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/01/2018 10:55:05",
      "content": "<p>some baseline results  for reference:</p>\n\n<ul>\n<li><p>unet (single class, i.e. semantic segmentation) : 0.35</p></li>\n<li><p>enhance unet (multi-class with post processing) : 0.42</p></li>\n<li><p>mask-rcnn (instance segmentation) : 0.50</p></li>\n</ul>\n\n<hr>\n\n<p>The LB metric is based on image, not based on object. You should take care of test images with few nuclei first. They are the easiest, and accuracy contribution per nucleus object is higher in these images.</p>\n\n<hr>\n\n<p>An increase of 0.001 in LB score is about detecting 2 previously missed nuclei on one image (assuming that the image has about 50 nuclei object).</p>\n\n<hr>\n\n<p>Even if you have perfect precision 1.0 for bounding box (threshold at 0.5 IOU), the mask average precision (i.e. LB metric) is about 0.85 to 0.90 in my experiments. This is because there is many small nuclei, meaning that if you miss one pixel for the mask prediction, you will have an FP (false positive) at high IOU. You may want to think about if you want to include prediction for small nuclei in LB submission or not</p>\n\n<hr>\n\n<p>mask average precision is as follows:</p>\n\n<ul>\n<li><p>training (gray images only) : 0.69</p></li>\n<li><p>validation (gray images only, same kind of cells) : 0.69</p></li>\n<li><p>training (purple images only) : 0.55</p></li>\n<li><p>validation (purple images only, same kind of cells) : 0.41</p></li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 289340,
          "author_name": "john1231983",
          "author_url": "",
          "post_date": "03/01/2018 12:19:03",
          "content": "<p>This is what I have to wait for long time. I have one question about encoding RLE. After we ran masking, we obtained instance segmentation such as mask1 mask2, ..., maskN for object ID 1. When we encode RLE, we have two ways:</p>\n\n<p>First way: each row encode a mask so for object ID 1 we have</p>\n\n<h1>ImageId,EncodedPixels</h1>\n\n<h1>1,  9 2 ... encode for mask1</h1>\n\n<h1>1, 3 5 ...encode for mask2</h1>\n\n<h1>...</h1>\n\n<h1>1, 6 7 ...encode for maskN</h1>\n\n<p>Second way: combine all instance masks into one mask and then encode it</p>\n\n<h1>ImageId,EncodedPixels</h1>\n\n<h1>1,  9 2 3 5 6 7 encode for mask1 to maskN combination for ID 1</h1>\n\n<h1>2, 3 5  encode for mask1 to maskM combination for ID 2</h1>\n\n<p>The first way does not follow the rule of the challenge that said that each row is for a subject ID, but it shows that one id has multiple rows, each row represents its instance map encoding. The second way looks good but it cannot achieve the goal of the challenge that focusing on instance segmentation. Which way are you using?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 293848,
          "author_name": "arc144",
          "author_url": "",
          "post_date": "03/10/2018 18:54:46",
          "content": "<p>I believe it is like Heng CherKeng metioned: </p>\n\n<blockquote>\n  <p>\"The LB metric is based on image, not based on object. You should take\n  care of test images with few nuclei first. They are the easiest, and\n  accuracy contribution per nucleus object is higher in these images.\"</p>\n</blockquote>\n\n<p>This happens because the competition chose to use the RLE encoding as the second form you mentioned. \nIf the first way were chosen, the LB would be computed based on instaces instead of images.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 294161,
          "author_name": "john1231983",
          "author_url": "",
          "post_date": "03/11/2018 12:24:45",
          "content": "<p>It is the first case. I have tested it. However, I did not understand what is the meaning of \"based on image not object\". If I understand correctly, it means if I select a good LB results (such as 53 gray images) in total of 65 images, then the LB only compute for these 53 images and I can got better peformance than using all 65 images. Am I right?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 294167,
          "author_name": "arc144",
          "author_url": "",
          "post_date": "03/11/2018 12:48:56",
          "content": "<p>Yes, I think you are right but I'm not sure if you can do this.. When you submitt your preditions you would be pennalized for the missing predictions anyway, I guess.. Although I've never tried</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 289309,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/01/2018 11:05:02",
      "content": "<p>It is easier to perfect your boxing box detection first. i,e you still train a mask-rcnn with 3 loss, but pay more attention to the box label and regression loss. (you can also freeze the rpn and rcnn and finetune/add more parameters your mask head later). Try to train your mask rcnn to about &gt;0.90 box precision first.</p>\n\n<p>The trick to get accurate box detection is \"anchor box\". Have more anchor of the \"correct size\" and label them correctly for training. Note that there are thin objects and overlaping object, causing problems of background become FP and partial objects become FP respectively, see attached.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/289309/8659/FP-1.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": [
        {
          "id": 289556,
          "author_name": "fabsta",
          "author_url": "",
          "post_date": "03/01/2018 21:21:59",
          "content": "<p>Heng, thanks for the interesting post!</p>\n\n<p>Two questions:</p>\n\n<pre><code>still train a mask-rcnn with 3 loss\n</code></pre>\n\n<p>what is 3 loss here?</p>\n\n<pre><code>Box overlap is used to label +ve  and -ve training samples\n</code></pre>\n\n<p>what is +ve and -ve here?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 291654,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "03/06/2018 15:25:28",
          "content": "<blockquote>\n  <blockquote>\n    <p>what is 3 loss here?</p>\n  </blockquote>\n</blockquote>\n\n<p>rpn loss, rcnn loss and mask loss</p>\n\n<blockquote>\n  <blockquote>\n    <p>what is +ve and -ve here?</p>\n  </blockquote>\n</blockquote>\n\n<p>positive and negative labels for training rpn and rcnn</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 292672,
          "author_name": "aviwolfson",
          "author_url": "",
          "post_date": "03/08/2018 12:03:09",
          "content": "<p>so what is the most important loss - mask loss??</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 294146,
          "author_name": "virilo",
          "author_url": "",
          "post_date": "03/11/2018 11:58:46",
          "content": "<p>@Heng,</p>\n\n<blockquote>\n  <p>you can also freeze the rpn and rcnn and finetune/add more parameters your mask head later</p>\n</blockquote>\n\n<p>What do you refer with freeze the rcnn?  I understood the whole net except the mask fcn.</p>\n\n<p>Using matterport, then I should freeze all the net with the exception of: mrcnn_mask_loss, proposal_targets and mrcnn_mask (layer's name).  Isn't true?</p>\n\n<p>thanks a lot for sharing!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 289559,
      "author_name": "czhangcn",
      "author_url": "",
      "post_date": "03/01/2018 21:24:48",
      "content": "<p>Extremely helpful! Thanks for sharing.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 289744,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/02/2018 06:49:46",
      "content": "<p>you can merge giver train patches into full image and re-crop to create more training samples. an example is shown below. both train and test LB images can be merged. (for merging code you can refer to the Amazon Forest Kaggle competition)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/289744/8663/merge.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": [
        {
          "id": 289750,
          "author_name": "john1231983",
          "author_url": "",
          "post_date": "03/02/2018 07:03:56",
          "content": "<p>For cropping, if I train on cropped images such as 128x128 to obtain trained weight. For inference, I assume the input size is 512x512. Then I also want to test on 128x128 sub-images. How can I merge the final result to obtain final prediction of 512x512 from results of sub-images 128x128? Thanks</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 291567,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "03/06/2018 11:52:00",
          "content": "<p>see also:</p>\n\n<p><a href=\"https://www.kaggle.com/c/data-science-bowl-2018/discussion/51184\">https://www.kaggle.com/c/data-science-bowl-2018/discussion/51184</a></p>\n\n<p><a href=\"https://www.kaggle.com/bonlime/train-test-image-mosaic\">https://www.kaggle.com/bonlime/train-test-image-mosaic</a></p>\n\n<p>thanks to @Emil !!!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 291579,
          "author_name": "john1231983",
          "author_url": "",
          "post_date": "03/06/2018 12:07:51",
          "content": "<p>@Heng CherKeng: When you experiment, does the cluster dataset helps improve performance comparison with no-clustering (using both color and gray for training)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 291653,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "03/06/2018 15:23:08",
          "content": "<p>yes, it improves results</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 291673,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "03/06/2018 16:19:48",
          "content": "<p>You can check my latest code at the pytorch mask rcnn thread. It has submission code to show how to predict instance mask for full image and encode them for submission csv</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 309766,
          "author_name": "livingprogram",
          "author_url": "",
          "post_date": "04/05/2018 23:50:48",
          "content": "<p>I know this might be a little outdated, but if we want to use random crops of mosaics for training, how can we merge the masks properly to create a single mask for the entire mosaic, this way the ground truth for the random crop can be extrapolated from the single mask?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 309783,
          "author_name": "arc144",
          "author_url": "",
          "post_date": "04/06/2018 01:20:02",
          "content": "<p>This paper here <a href=\"https://www.nature.com/articles/s41598-018-22181-4\">https://www.nature.com/articles/s41598-018-22181-4</a>\ncovers it. It may be worth reading it!\nIt does not uses random crops but you can simply change their implementation.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 291469,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/06/2018 07:26:22",
      "content": "<p>in coco evaluation, there is box precision (detection) and mask instance  segmentation precision.</p>\n\n<p>Here is my results on mask-rcnn-resnet34 trained on 10 gray images and test on 43 gray images.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/291469/8695/box_and_mask1.png\" alt=\"enter image description here\"></p>\n\n<p>you can see that box precision is very high. this is the upper bound for mask LB metric.</p>\n\n<hr>\n\n<p>here is a good case.  Even so, there is a gap between box precision (evaluated at 0.50 only) and the mask LB metric (averaged over iou 0.50 to 1.00)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/291469/8697/box_and_mask2.png\" alt=\"enter image description here\"></p>\n\n<hr>\n\n<p>here is a bad case: some masks are missed (even though the box are detected) because of my threshold and wrong non-max suppression. Hence post-processing is important. you may want to spend time on this.</p>\n\n<p>Note that the red region(4th image labelled as iou) are missing prediction or predicted mask that have iou less than 0.5 which become FP at evaluation and bring down the LB metric score.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/291469/8696/box_and_mask3.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 291667,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/06/2018 16:07:45",
      "content": "<p>example of learning a \"post processing network\" (e.g. for nms or ensembling)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/291667/8706/post%20processing%20netowrk.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": [
        {
          "id": 291816,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "03/06/2018 22:35:34",
          "content": "<p>another variant of the solution is to let the the output of the post processing net be probability(pixel belong to mask-i, mask-j or none). In this case you learn how to handle overlapping pixel of a pair of predicted mask.  </p>\n\n<p>Be consistent and naming i and j, e.g. i is always the mask nearest to top-left and j is nearest to bottom-right. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 293232,
          "author_name": "john1231983",
          "author_url": "",
          "post_date": "03/09/2018 13:23:53",
          "content": "<p>@Heng CherKeng: I am working in the same problem as you mentioned. Let see your example, the box 1 detects two masks: <code>1</code> and <code>2</code>. The box 2 detects 1 mask : <code>3</code>. The problem is that the mask 2nd and 3rd are in the same mask if we look at the whole image. How we know that and how we can combine them together in final result. Thanks for give me a clear example </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 292056,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/07/2018 10:06:39",
      "content": "<p>example of test time augmentation. Results of ensemble of two is mask metric = 0.41478.\nother possible test augment includes scale, flip, shift, color/illumination change  ...</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/292056/8707/tta.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": [
        {
          "id": 294598,
          "author_name": "xhqlcwq",
          "author_url": "",
          "post_date": "03/12/2018 07:54:23",
          "content": "<p>how do you ensemble the two results:original and flip</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 297605,
          "author_name": "john1231983",
          "author_url": "",
          "post_date": "03/17/2018 14:22:02",
          "content": "<p>@xiapidan: Have you find the way? I am using bitwise_or for the same position of cells but the performance reduces from 0.473 to 0.45 :(</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 297645,
          "author_name": "wayne999",
          "author_url": "",
          "post_date": "03/17/2018 15:47:55",
          "content": "<p>I guess just average the probabilities, not bitwise_or for the masks</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 292162,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/07/2018 15:05:42",
      "content": "<p>sample balancing in training:</p>\n\n<ul>\n<li><p>Train a deep network on train dataset with uniform sampling</p></li>\n<li><p>Use the trained network to label the difficulty of each image of the dataset (e.g. you can use LB metric)</p></li>\n<li><p>Modify your batch sampler so that the more difficult are sampled more frequently</p></li>\n<li><p>Train with new modified batch sampler</p></li>\n<li><p>Repeat the process</p></li>\n</ul>",
      "votes": null,
      "replies": []
    },
    {
      "id": 292706,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/08/2018 13:39:35",
      "content": "<p>The diagram shows a case where there is more correct detection results but worst LB metric. \nThis is because the mask prediction is not accurate enough.</p>\n\n<p>At high IOU metric computation it is penalized twice (one for being a FP and another for being miss)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 292900,
      "author_name": "bcottman",
      "author_url": "",
      "post_date": "03/08/2018 21:13:20",
      "content": "<p>I think you(Heng CherKeng)have just defined bootstrapping for CNN . may be out there. first time i saw it. very clever. you should make into general solution and publish.</p>",
      "votes": null,
      "replies": [
        {
          "id": 292902,
          "author_name": "bcottman",
          "author_url": "",
          "post_date": "03/08/2018 21:14:42",
          "content": "<p>i just figured out how to reply. my apologies.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 293101,
      "author_name": "sahithivodnala",
      "author_url": "",
      "post_date": "03/09/2018 06:47:43",
      "content": "<p>Good job...Thank you.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 293570,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/10/2018 06:32:36",
      "content": "<p>test on external images(randomly download from the web)</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/113660/4d35e23daf1646f28901458155600bf9/external_test.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": [
        {
          "id": 294917,
          "author_name": "mpware",
          "author_url": "",
          "post_date": "03/12/2018 20:32:36",
          "content": "<p>Hi Heng, did you add some color items in augmentation to achieve this result? Just modifying a purple test image to yellow (with any image editor) leads to bad prediction (totally different masks) for my Mask RCNN. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 294038,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/11/2018 05:56:24",
      "content": "<p>FYI: how to ensemble objector detector from resnet paper:</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/113660/ac6394a8a6ba4682a5edd37452649d98/ensemble.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 294237,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/11/2018 16:04:45",
      "content": "<p>To detect label noise or abnormalities, check the train error.</p>\n\n<p>Then you can relabel the train images. I draw the miss and fp nuclei. This is useful:</p>\n\n<ul>\n<li><p>i can check if the train, validation and LB test error are similar or not</p></li>\n<li><p>i will know which train image are difficult (helps me to sample the images for making random batch for sgd)</p></li>\n<li><p>i can relabel for noises</p></li>\n<li><p>i make sure that my validation set does not have label error, so that i have fair comparison between different system</p></li>\n<li><p>i can estimate LB label noise (assuming it is same as training label noise). Hence I will know if two kaggler scored are statistically similar or not. It also let me estimate the highest possible LB score and whether i should trust LB score or valdiation score more.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/113660/63ceec30d4d742a8b426749480834153/error.png\" alt=\"enter image description here\"></p></li>\n</ul>",
      "votes": null,
      "replies": []
    },
    {
      "id": 294738,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/12/2018 14:50:57",
      "content": "<p>you can change your mask-head!</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/113660/b93cdf0622b74264903a7db58b8c14d9/m1.png\" alt=\"enter image description here\"></p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/inbox/113660/4ac2df07c502403a9bd986f4504d6ba8/m2.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 298070,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/18/2018 17:38:48",
      "content": "<p>this is how instance segmentation results from mask rcnn  looks like. It is easy to see that you can further refine and improve the mask boundary using post processing. In fact classify many classic methods can be used because the texture is \"rather simple\". </p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/298070/8802/3582166ee20755856adf4882a8bfacb616fce4247911605a109c4862de421bcd.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": [
        {
          "id": 298091,
          "author_name": "john1231983",
          "author_url": "",
          "post_date": "03/18/2018 18:55:34",
          "content": "<p>Hi Heng, what kind of post processing do you prefer? I have try dilation, active contour, fill hole, remove small cell but they are hurt my performance. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 298388,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "03/19/2018 13:57:06",
          "content": "<p>border for the mask nuclei is too small. you can see some \"cropping effects\". try to change to larger crop (i.e. larger than 28x28) and more pixels for the \"border\"</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 298399,
          "author_name": "john1231983",
          "author_url": "",
          "post_date": "03/19/2018 14:19:36",
          "content": "<p>Thanks for your suggestion. I will try. Actually, I have tried the matterport implementation in Keras that use crop size of 512x512 and frozen batch norm (batch size of 1) (due to using pretrain model), then the postprocessing help so much (+0.03 LB). Unfortunately, your code does not support pretrain model, so that we have use the batch size as much as possible, hence we cannot use crop size bigger than 256. I hope the next version can solve it by using pretrain model and frozen batch norm. Thanks for your great work, your current mask segmentation branch is quite good, so it is not good to apply dilation any more :)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 298515,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "03/19/2018 16:49:33",
          "content": "<p>you can actually use pretrain model. just change the featureNet to the pretrain model network pytorch definition and see load_pretrain function</p>\n\n<p>alternatively, after training from scratch, freeze the model partially to increase batch size</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 300908,
          "author_name": "wayne999",
          "author_url": "",
          "post_date": "03/22/2018 03:25:06",
          "content": "<p>@Heng, you mean increase <code>mask_crop_size</code> by modify the configuration,  or increase <code>mask_size</code> (which is now 28) by adding another deconv layer in mask head</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 298327,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/19/2018 11:21:01",
      "content": "<p>padding / mosaicing is very important (this is a common dark knowledge for people who do segmentation before).</p>\n\n<p>Deep network is the weakest at the boundary image and strongest are the center. Below shows mask rcnn results for the same network, but for padded image, unpadded image, difference of the two.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/298327/8811/pad-0.png\" alt=\"enter image description here\"></p>\n\n<p>i suggest kaggler to read the solutions of past kaggle segmentation challenges like carvana car and satellite images. There are many more tricks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 298387,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "03/19/2018 13:53:22",
      "content": "<p>for each crop feature, do some augmentation (e.g. flip ). for each augmented feature, predict mask.\nhere is how it look like:</p>\n\n<p>green: average of all mask prediction.\nred: prediction for augmented crop features</p>\n\n<pre><code>            augment_image, augment_rcnn_proposal = augment_function(pad_image, rcnn_proposal)\n            input       = Variable(torch.from_numpy(augment_image.transpose((2,0,1))).float().div(255).unsqueeze(0)).cuda()\n            proposal   = Variable(torch.from_numpy(augment_rcnn_proposal)).cuda()\n\n            feature    = data_parallel(net.feature_net, input)\n            mask_crop  = net.mask_crop(feature, proposal)\n            mask_logit = data_parallel(net.mask_head, mask_crop)\n</code></pre>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/298387/8815/average1.png\" alt=\"enter image description here\"></p>",
      "votes": null,
      "replies": [
        {
          "id": 298394,
          "author_name": "john1231983",
          "author_url": "",
          "post_date": "03/19/2018 14:03:18",
          "content": "<p>Thanks for your idea. Do you mean we will insert it in the below</p>\n\n<p>if len(self.detections)&gt;0:</p>\n\n<pre><code>          mask_crops = self.mask_crop(features, self.detections)\n\n          self.mask_logits = data_parallel(self.mask_head, mask_crops)\n\n          self.masks = mask_nms(cfg, mode, inputs, self.detections, self.mask_logits) #\n</code></pre>\n\n<p>Could you show your <code>augment_function</code>?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 305436,
          "author_name": "wayne999",
          "author_url": "",
          "post_date": "03/28/2018 21:38:03",
          "content": "<p>Hi Heng, do you use nms to do TTA?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "289297": "In summary, this competition is about accuracy and generalization to unseen data. I will talk about \"generalization\" later. Here are some tips on building an accurate system, see each section below.\n\n(this thread will be constantly updated)",
    "289298": "overall system: how to make ensemble?\n\n  -  it is easier to design small system and combine them together\n\n  - just need to ensemble them together. Results are cumulative\n    (if you need a single system for practical use, you can always convert an ensemble system to single network using knowledge distill, teacher-student learning later) \n\n  - how about nuclei that are not detected by any of the CNN? You still have semantic segmentation like Unet to fall back on! (this is about 99% at pixel level!) \n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/289298/8656/system.png",
    "289300": "as a reference (COCO object detection at ilsvrc 2016 ):\nhttp://image-net.org/challenges/talks/2016/GRMI-COCO-slidedeck.pdf\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/289300/8657/ensmble.png",
    "289306": "some baseline results  for reference:\n\n- unet (single class, i.e. semantic segmentation) : 0.35\n\n- enhance unet (multi-class with post processing) : 0.42\n\n- mask-rcnn (instance segmentation) : 0.50\n\n---\n\nThe LB metric is based on image, not based on object. You should take care of test images with few nuclei first. They are the easiest, and accuracy contribution per nucleus object is higher in these images.\n\n---\n\nAn increase of 0.001 in LB score is about detecting 2 previously missed nuclei on one image (assuming that the image has about 50 nuclei object).\n\n---\n\nEven if you have perfect precision 1.0 for bounding box (threshold at 0.5 IOU), the mask average precision (i.e. LB metric) is about 0.85 to 0.90 in my experiments. This is because there is many small nuclei, meaning that if you miss one pixel for the mask prediction, you will have an FP (false positive) at high IOU. You may want to think about if you want to include prediction for small nuclei in LB submission or not\n\n---\n\nmask average precision is as follows:\n\n  - training (gray images only) : 0.69\n\n  - validation (gray images only, same kind of cells) : 0.69\n\n  - training (purple images only) : 0.55\n\n  - validation (purple images only, same kind of cells) : 0.41",
    "289309": "It is easier to perfect your boxing box detection first. i,e you still train a mask-rcnn with 3 loss, but pay more attention to the box label and regression loss. (you can also freeze the rpn and rcnn and finetune/add more parameters your mask head later). Try to train your mask rcnn to about &gt;0.90 box precision first.\n\nThe trick to get accurate box detection is \"anchor box\". Have more anchor of the \"correct size\" and label them correctly for training. Note that there are thin objects and overlaping object, causing problems of background become FP and partial objects become FP respectively, see attached.\n\n   ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/289309/8659/FP-1.png",
    "289340": "This is what I have to wait for long time. I have one question about encoding RLE. After we ran masking, we obtained instance segmentation such as mask1 mask2, ..., maskN for object ID 1. When we encode RLE, we have two ways:\n\nFirst way: each row encode a mask so for object ID 1 we have\n\n#ImageId,EncodedPixels\n#1,  9 2 ... encode for mask1\n#1, 3 5 ...encode for mask2\n#...\n#1, 6 7 ...encode for maskN\n\nSecond way: combine all instance masks into one mask and then encode it\n\n#ImageId,EncodedPixels\n#1,  9 2 3 5 6 7 encode for mask1 to maskN combination for ID 1\n#2, 3 5  encode for mask1 to maskM combination for ID 2\n\nThe first way does not follow the rule of the challenge that said that each row is for a subject ID, but it shows that one id has multiple rows, each row represents its instance map encoding. The second way looks good but it cannot achieve the goal of the challenge that focusing on instance segmentation. Which way are you using?",
    "289556": "Heng, thanks for the interesting post!\n\nTwo questions:\n\n    still train a mask-rcnn with 3 loss\nwhat is 3 loss here?\n\n\n    Box overlap is used to label +ve  and -ve training samples\nwhat is +ve and -ve here?",
    "289559": "Extremely helpful! Thanks for sharing.",
    "289645": "if you need a learning based NMS, check these out!\n\n\"Learning non-maximum suppression\" - Jan Hosang, Rodrigo Benenson, Bernt Schiele, https://arxiv.org/abs/1705.02950\n\n\n\"A CONVNET FOR NON-MAXIMUM SUPPRESSION\" - Jan Hosang, Rodrigo Benenson, and Bernt Schiele, https://arxiv.org/pdf/1511.06437.pdf",
    "289744": "you can merge giver train patches into full image and re-crop to create more training samples. an example is shown below. both train and test LB images can be merged. (for merging code you can refer to the Amazon Forest Kaggle competition)\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/289744/8663/merge.png",
    "289750": "For cropping, if I train on cropped images such as 128x128 to obtain trained weight. For inference, I assume the input size is 512x512. Then I also want to test on 128x128 sub-images. How can I merge the final result to obtain final prediction of 512x512 from results of sub-images 128x128? Thanks",
    "291469": "in coco evaluation, there is box precision (detection) and mask instance  segmentation precision.\n\nHere is my results on mask-rcnn-resnet34 trained on 10 gray images and test on 43 gray images.\n\n  ![enter image description here][1]\n\n  you can see that box precision is very high. this is the upper bound for mask LB metric.\n\n---\n\nhere is a good case.  Even so, there is a gap between box precision (evaluated at 0.50 only) and the mask LB metric (averaged over iou 0.50 to 1.00)\n\n  ![enter image description here][2]\n\n\n---\n\nhere is a bad case: some masks are missed (even though the box are detected) because of my threshold and wrong non-max suppression. Hence post-processing is important. you may want to spend time on this.\n\nNote that the red region(4th image labelled as iou) are missing prediction or predicted mask that have iou less than 0.5 which become FP at evaluation and bring down the LB metric score.\n\n  ![enter image description here][3]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/291469/8695/box_and_mask1.png\n  [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/291469/8697/box_and_mask2.png\n  [3]: https://kaggle2.blob.core.windows.net/forum-message-attachments/291469/8696/box_and_mask3.png",
    "291567": "see also:\n\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/51184\n\nhttps://www.kaggle.com/bonlime/train-test-image-mosaic\n\nthanks to @Emil !!!",
    "291579": "Heng CherKeng: When you experiment, does the cluster dataset helps improve performance comparison with no-clustering (using both color and gray for training)",
    "291653": "yes, it improves results",
    "291654": "&gt;&gt; what is 3 loss here?\n\nrpn loss, rcnn loss and mask loss\n\n&gt;&gt; what is +ve and -ve here?\n\npositive and negative labels for training rpn and rcnn",
    "291667": "example of learning a \"post processing network\" (e.g. for nms or ensembling)\n\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/291667/8706/post%20processing%20netowrk.png",
    "291673": "You can check my latest code at the pytorch mask rcnn thread. It has submission code to show how to predict instance mask for full image and encode them for submission csv",
    "291816": "another variant of the solution is to let the the output of the post processing net be probability(pixel belong to mask-i, mask-j or none). In this case you learn how to handle overlapping pixel of a pair of predicted mask.  \n\nBe consistent and naming i and j, e.g. i is always the mask nearest to top-left and j is nearest to bottom-right.",
    "292056": "example of test time augmentation. Results of ensemble of two is mask metric = 0.41478.\nother possible test augment includes scale, flip, shift, color/illumination change  ...\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/292056/8707/tta.png",
    "292162": "sample balancing in training:\n\n-  Train a deep network on train dataset with uniform sampling\n\n-  Use the trained network to label the difficulty of each image of the dataset (e.g. you can use LB metric)\n\n- Modify your batch sampler so that the more difficult are sampled more frequently\n\n- Train with new modified batch sampler\n\n- Repeat the process",
    "292672": "so what is the most important loss - mask loss??",
    "292706": "The diagram shows a case where there is more correct detection results but worst LB metric. \nThis is because the mask prediction is not accurate enough.\n\nAt high IOU metric computation it is penalized twice (one for being a FP and another for being miss)",
    "292900": "I think you(Heng CherKeng)have just defined bootstrapping for CNN . may be out there. first time i saw it. very clever. you should make into general solution and publish.",
    "292902": "i just figured out how to reply. my apologies.",
    "293101": "Good job...Thank you.",
    "293232": "Heng CherKeng: I am working in the same problem as you mentioned. Let see your example, the box 1 detects two masks: `1` and `2`. The box 2 detects 1 mask : `3`. The problem is that the mask 2nd and 3rd are in the same mask if we look at the whole image. How we know that and how we can combine them together in final result. Thanks for give me a clear example",
    "293570": "test on external images(randomly download from the web)\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/inbox/113660/4d35e23daf1646f28901458155600bf9/external_test.png",
    "293848": "I believe it is like Heng CherKeng metioned: \n\n&gt; \"The LB metric is based on image, not based on object. You should take\n&gt; care of test images with few nuclei first. They are the easiest, and\n&gt; accuracy contribution per nucleus object is higher in these images.\"\n\nThis happens because the competition chose to use the RLE encoding as the second form you mentioned. \nIf the first way were chosen, the LB would be computed based on instaces instead of images.",
    "294038": "FYI: how to ensemble objector detector from resnet paper:\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/inbox/113660/ac6394a8a6ba4682a5edd37452649d98/ensemble.png",
    "294146": "Heng,\n\n&gt; you can also freeze the rpn and rcnn and finetune/add more parameters your mask head later\n\nWhat do you refer with freeze the rcnn?  I understood the whole net except the mask fcn.\n\nUsing matterport, then I should freeze all the net with the exception of: mrcnn_mask_loss, proposal_targets and mrcnn_mask (layer's name).  Isn't true?\n\nthanks a lot for sharing!",
    "294161": "It is the first case. I have tested it. However, I did not understand what is the meaning of \"based on image not object\". If I understand correctly, it means if I select a good LB results (such as 53 gray images) in total of 65 images, then the LB only compute for these 53 images and I can got better peformance than using all 65 images. Am I right?",
    "294167": "Yes, I think you are right but I'm not sure if you can do this.. When you submitt your preditions you would be pennalized for the missing predictions anyway, I guess.. Although I've never tried",
    "294237": "To detect label noise or abnormalities, check the train error.\n\nThen you can relabel the train images. I draw the miss and fp nuclei. This is useful:\n\n - i can check if the train, validation and LB test error are similar or not\n \n - i will know which train image are difficult (helps me to sample the images for making random batch for sgd)\n\n - i can relabel for noises\n\n - i make sure that my validation set does not have label error, so that i have fair comparison between different system\n\n - i can estimate LB label noise (assuming it is same as training label noise). Hence I will know if two kaggler scored are statistically similar or not. It also let me estimate the highest possible LB score and whether i should trust LB score or valdiation score more.\n\n  \n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/inbox/113660/63ceec30d4d742a8b426749480834153/error.png",
    "294598": "how do you ensemble the two results:original and flip",
    "294738": "you can change your mask-head!\n\n  ![enter image description here][1]\n\n   ![enter image description here][2]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/inbox/113660/b93cdf0622b74264903a7db58b8c14d9/m1.png\n  [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/inbox/113660/4ac2df07c502403a9bd986f4504d6ba8/m2.png",
    "294917": "Hi Heng, did you add some color items in augmentation to achieve this result? Just modifying a purple test image to yellow (with any image editor) leads to bad prediction (totally different masks) for my Mask RCNN.",
    "297605": "xiapidan: Have you find the way? I am using bitwise_or for the same position of cells but the performance reduces from 0.473 to 0.45 :(",
    "297645": "I guess just average the probabilities, not bitwise_or for the masks",
    "298070": "this is how instance segmentation results from mask rcnn  looks like. It is easy to see that you can further refine and improve the mask boundary using post processing. In fact classify many classic methods can be used because the texture is \"rather simple\". \n\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/298070/8802/3582166ee20755856adf4882a8bfacb616fce4247911605a109c4862de421bcd.png",
    "298091": "Hi Heng, what kind of post processing do you prefer? I have try dilation, active contour, fill hole, remove small cell but they are hurt my performance.",
    "298327": "padding / mosaicing is very important (this is a common dark knowledge for people who do segmentation before).\n\nDeep network is the weakest at the boundary image and strongest are the center. Below shows mask rcnn results for the same network, but for padded image, unpadded image, difference of the two.\n\n ![enter image description here][1]\n\ni suggest kaggler to read the solutions of past kaggle segmentation challenges like carvana car and satellite images. There are many more tricks!\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/298327/8811/pad-0.png",
    "298387": "for each crop feature, do some augmentation (e.g. flip ). for each augmented feature, predict mask.\nhere is how it look like:\n\ngreen: average of all mask prediction.\nred: prediction for augmented crop features\n\n                augment_image, augment_rcnn_proposal = augment_function(pad_image, rcnn_proposal)\n                input       = Variable(torch.from_numpy(augment_image.transpose((2,0,1))).float().div(255).unsqueeze(0)).cuda()\n                proposal   = Variable(torch.from_numpy(augment_rcnn_proposal)).cuda()\n\n                feature    = data_parallel(net.feature_net, input)\n                mask_crop  = net.mask_crop(feature, proposal)\n                mask_logit = data_parallel(net.mask_head, mask_crop)\n\n\n  ![enter image description here][1]\n\n\n  [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/298387/8815/average1.png",
    "298388": "border for the mask nuclei is too small. you can see some \"cropping effects\". try to change to larger crop (i.e. larger than 28x28) and more pixels for the \"border\"",
    "298394": "Thanks for your idea. Do you mean we will insert it in the below\n\n\nif len(self.detections)&gt;0:\n\n              mask_crops = self.mask_crop(features, self.detections)\n\n              self.mask_logits = data_parallel(self.mask_head, mask_crops)\n\n              self.masks = mask_nms(cfg, mode, inputs, self.detections, self.mask_logits) #\n\nCould you show your `augment_function`?",
    "298399": "Thanks for your suggestion. I will try. Actually, I have tried the matterport implementation in Keras that use crop size of 512x512 and frozen batch norm (batch size of 1) (due to using pretrain model), then the postprocessing help so much (+0.03 LB). Unfortunately, your code does not support pretrain model, so that we have use the batch size as much as possible, hence we cannot use crop size bigger than 256. I hope the next version can solve it by using pretrain model and frozen batch norm. Thanks for your great work, your current mask segmentation branch is quite good, so it is not good to apply dilation any more :)",
    "298515": "you can actually use pretrain model. just change the featureNet to the pretrain model network pytorch definition and see load_pretrain function\n\nalternatively, after training from scratch, freeze the model partially to increase batch size",
    "300908": "Heng, you mean increase `mask_crop_size` by modify the configuration,  or increase `mask_size` (which is now 28) by adding another deconv layer in mask head",
    "305436": "Hi Heng, do you use nms to do TTA?",
    "309766": "I know this might be a little outdated, but if we want to use random crops of mosaics for training, how can we merge the masks properly to create a single mask for the entire mosaic, this way the ground truth for the random crop can be extrapolated from the single mask?",
    "309783": "This paper here https://www.nature.com/articles/s41598-018-22181-4\ncovers it. It may be worth reading it!\nIt does not uses random crops but you can simply change their implementation."
  },
  "source": "meta"
}