（currently 8th place while leaderboard is being updated）
\n\nHi all. It is my first time join this competition. All my methods are learned from open discussion. As return, I decide to share my solution to the community. Thanks a lot to those who have made many contributions to the community.
\n\nFirst of all, I thank Allen, John1231983, Keven Wang, Mehul Sampat, Heng CherKeng, CPMP, bsp2020, kamil, YaGana Sheriff-Hussaini, Yan Wang, Konstantin Maksimov, xiapidan, xuan, Waleed for your very impressive discussions. I learned a lot from those discussions. @Mehul Sampat hope we can team up together next time :) thanks again.
\n\nMy score: Stage1 LB 0.489(87th), Stage2 LB 0.590(11th). I am using matterport mask-rcnn.
\n\nAugmentations
\n\nSince there are hundreds of training images, we must find some useful augmentations to prevent our models from overfitting and make them generalizable. Here are some methods I have tried but it didn't work for me:
\n\nI only use flip up&down&left&right
\n\nAdditional data
\n\nNo. I have tried to add h&e dataset to training data, but it didn't improve my performance. I am using https://github.com/lopuhin/kaggle-dsbowl-2018-dataset-fixes
\n\nEnsembling
\n\nNo. I spend almost one week trying this method but it didn't work so well. I devided the training data into two categories: color & grey. I have seen someone get a high score (0.5+) in stage1 by using this method. I am very interested in this so I really hope that someone can share some solutions about this.
\n\nParameters
\n\ntrain:
\n\ninti_with= coco
\n\nRESNET_ARCHITECTURE = \"resnet101\"
\n\nMEAN_PIXEL = np.array([0., 0., 0.])
\n\nRPN_NMS_THRESHOLD = 0.7
\n\nDETECTION_MIN_CONFIDENCE = 0.7
\n\nDETECTION_NMS_THRESHOLD = 0.3
\n\nTRAIN_ROIS_PER_IMAGE = 600
\n\nRPN_TRAIN_ANCHORS_PER_IMAGE = 320
\n\nLEARNING_RATE = 1e-3
\n\ninference:
\n\nRPN_NMS_THRESHOLD = 0.6
\n\nDETECTION_NMS_THRESHOLD = 0.1
\n\nTraining
\n\n1e-3 all 20epochs. Choose the best point according to the display of tensorboard, then 1e-4 or 1e-5 train all 20epochs. (if val loss stops decreasing, stops training) \noptimizer=Adam.
\n\nPost processing \nUse binary_dilation.
\n\nOther method\ngeneral standardization.\nimg = img-mean(img)/std(img).
\n\nSome methods might be useful
\n\n1.Filter
\n\n> We use single mask rcnn, but with special post process to filter noises (i.e. small FPs) based on clustering and outlier detection of mask sizes, which boosted us a lot (kill lots of noises). Less FPs is the key to get high mAP as Heng said. I should thank him a lot
\n\n2.fill_holes
\n\n\n\n3.wateshed
\n\nSome interesting discussions&methods links
\n\nLast
\n\nI have tried a lot of experiments. Some of them i just don't remeber.....sorry :( So here is my email mdlszhengli@@gmail.com. I am very happy to discuss via email or wechat(removed).
", "messages": [ { "id": 316229, "postDate": "2018-04-18T13:40:09.943Z", "content": "（currently 8th place while leaderboard is being updated）
\n\nHi all. It is my first time join this competition. All my methods are learned from open discussion. As return, I decide to share my solution to the community. Thanks a lot to those who have made many contributions to the community.
\n\nFirst of all, I thank Allen, John1231983, Keven Wang, Mehul Sampat, Heng CherKeng, CPMP, bsp2020, kamil, YaGana Sheriff-Hussaini, Yan Wang, Konstantin Maksimov, xiapidan, xuan, Waleed for your very impressive discussions. I learned a lot from those discussions. @Mehul Sampat hope we can team up together next time :) thanks again.
\n\nMy score: Stage1 LB 0.489(87th), Stage2 LB 0.590(11th). I am using matterport mask-rcnn.
\n\nAugmentations
\n\nSince there are hundreds of training images, we must find some useful augmentations to prevent our models from overfitting and make them generalizable. Here are some methods I have tried but it didn't work for me:
\n\nI only use flip up&down&left&right
\n\nAdditional data
\n\nNo. I have tried to add h&e dataset to training data, but it didn't improve my performance. I am using https://github.com/lopuhin/kaggle-dsbowl-2018-dataset-fixes
\n\nEnsembling
\n\nNo. I spend almost one week trying this method but it didn't work so well. I devided the training data into two categories: color & grey. I have seen someone get a high score (0.5+) in stage1 by using this method. I am very interested in this so I really hope that someone can share some solutions about this.
\n\nParameters
\n\ntrain:
\n\ninti_with= coco
\n\nRESNET_ARCHITECTURE = \"resnet101\"
\n\nMEAN_PIXEL = np.array([0., 0., 0.])
\n\nRPN_NMS_THRESHOLD = 0.7
\n\nDETECTION_MIN_CONFIDENCE = 0.7
\n\nDETECTION_NMS_THRESHOLD = 0.3
\n\nTRAIN_ROIS_PER_IMAGE = 600
\n\nRPN_TRAIN_ANCHORS_PER_IMAGE = 320
\n\nLEARNING_RATE = 1e-3
\n\ninference:
\n\nRPN_NMS_THRESHOLD = 0.6
\n\nDETECTION_NMS_THRESHOLD = 0.1
\n\nTraining
\n\n1e-3 all 20epochs. Choose the best point according to the display of tensorboard, then 1e-4 or 1e-5 train all 20epochs. (if val loss stops decreasing, stops training) \noptimizer=Adam.
\n\nPost processing \nUse binary_dilation.
\n\nOther method\ngeneral standardization.\nimg = img-mean(img)/std(img).
\n\nSome methods might be useful
\n\n1.Filter
\n\n> We use single mask rcnn, but with special post process to filter noises (i.e. small FPs) based on clustering and outlier detection of mask sizes, which boosted us a lot (kill lots of noises). Less FPs is the key to get high mAP as Heng said. I should thank him a lot
\n\n2.fill_holes
\n\n\n\n3.wateshed
\n\nSome interesting discussions&methods links
\n\nLast
\n\nI have tried a lot of experiments. Some of them i just don't remeber.....sorry :( So here is my email mdlszhengli@@gmail.com. I am very happy to discuss via email or wechat(removed).
", "rawMarkdown": "（currently 8th place while leaderboard is being updated）\n\nHi all. It is my first time join this competition. All my methods are learned from open discussion. As return, I decide to share my solution to the community. Thanks a lot to those who have made many contributions to the community. \n\nFirst of all, I thank Allen, John1231983, Keven Wang, Mehul Sampat, Heng CherKeng, CPMP, bsp2020, kamil, YaGana Sheriff-Hussaini, Yan Wang, Konstantin Maksimov, xiapidan, xuan, Waleed for your very impressive discussions. I learned a lot from those discussions. @Mehul Sampat hope we can team up together next time :) thanks again.\n\nMy score: Stage1 LB 0.489(87th), Stage2 LB 0.590(11th). I am using matterport mask-rcnn.\n\n**Augmentations**\n\nSince there are hundreds of training images, we must find some useful augmentations to prevent our models from overfitting and make them generalizable. Here are some methods I have tried but it didn't work for me:\n \n - add gaussian noise\n - color to gray\n - contrast and brightness\n - random crop 512x512 if image size is bigger than 512 otherwise resize the image to 512x512\n - mosacis\n - mosacis+random crop\n - mosacis+random crop+ h&e\n - rotate 90 degrees \n - radnom rotate 90,180,270 degrees\n - rotate +-5 degrees on top of flip & 90 degree rotation\n - elastic transform\n\nI only use flip up&down&left&right\n\n**Additional data**\n\nNo. I have tried to add h&e dataset to training data, but it didn't improve my performance. I am using https://github.com/lopuhin/kaggle-dsbowl-2018-dataset-fixes\n\n**Ensembling**\n\nNo. I spend almost one week trying this method but it didn't work so well. I devided the training data into two categories: color & grey. I have seen someone get a high score (0.5+) in stage1 by using this method. I am very interested in this so I really hope that someone can share some solutions about this.\n\n**Parameters**\n\ntrain:\n\ninti_with= coco\n\nRESNET_ARCHITECTURE = \"resnet101\"\n\nMEAN_PIXEL = np.array([0., 0., 0.])\n\nRPN_NMS_THRESHOLD = 0.7\n\nDETECTION_MIN_CONFIDENCE = 0.7\n\nDETECTION_NMS_THRESHOLD = 0.3\n\nTRAIN_ROIS_PER_IMAGE = 600\n\nRPN_TRAIN_ANCHORS_PER_IMAGE = 320\n\nLEARNING_RATE = 1e-3\n\ninference:\n\nRPN_NMS_THRESHOLD = 0.6\n\nDETECTION_NMS_THRESHOLD = 0.1\n\n**Training**\n\n1e-3 all 20epochs. Choose the best point according to the display of tensorboard, then 1e-4 or 1e-5 train all 20epochs. (if val loss stops decreasing, stops training) \noptimizer=Adam.\n\n**Post processing** \nUse binary_dilation.\n\n**Other method**\ngeneral standardization.\nimg = img-mean(img)/std(img).\n\n**Some methods might be useful**\n\n1.Filter\n\n> We use single mask rcnn, but with special post process to filter noises (i.e. small FPs) based on clustering and outlier detection of mask sizes, which boosted us a lot (kill lots of noises). Less FPs is the key to get high mAP as Heng said. I should thank him a lot\n\n2.fill_holes\n\nhttps://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.ndimage.morphology.binary_fill_holes.html#scipy-ndimage-morphology-binary-fill-holes\n\n3.wateshed\n\n**Some interesting discussions&methods links**\n\n 1. https://github.com/matterport/Mask_RCNN/issues/230\n 2. https://github.com/matterport/Mask_RCNN/issues/281\n 3. https://github.com/killthekitten/kaggle-ds-bowl-2018-baseline/issues/5\n 4. https://www.kaggle.com/bostjanm/overlapping-objects-separation-method/notebook\n 5. https://www.kaggle.com/c/data-science-bowl-2018/discussion/52989#30758\n\n**Last**\n\nI have tried a lot of experiments. Some of them i just don't remeber.....sorry :( So here is my email mdlszhengli@@gmail.com. I am very happy to discuss via email or wechat(removed).", "votes": 45 }, { "id": 318739, "postDate": "2018-04-24T11:32:06.060Z", "content": "@Zheng Li , whew, I'm glad you'll be reinstated, make sure to upload your model in your next 2-stage competition, since you were very close to the prize this time. You're the \"outlier\" of the samples I'd say ;) your model is very well generalized and totally beat our LB at the 2nd stage even our LB was higher at the 1st stage. Good job.
", "rawMarkdown": "@Zheng Li , whew, I'm glad you'll be reinstated, make sure to upload your model in your next 2-stage competition, since you were very close to the prize this time. You're the \"outlier\" of the samples I'd say ;) your model is very well generalized and totally beat our LB at the 2nd stage even our LB was higher at the 1st stage. Good job. ", "votes": 1, "replies": [ { "id": 318746, "postDate": "2018-04-24T11:55:17.420Z", "content": "Hi Nicole. Thank you. See you in the next competition. ;)
", "rawMarkdown": "Hi Nicole. Thank you. See you in the next competition. ;)", "votes": 1 }, { "id": 1621965, "postDate": "2021-12-18T08:43:12.260Z", "content": "@mdlszhengli opening very old discussion.
\ncould you explain about the fitering method you used in the post processing
Nice job, Congratulations, Zheng!
", "rawMarkdown": "Nice job, Congratulations, Zheng!", "votes": 1 }, { "id": 316510, "postDate": "2018-04-19T08:36:55.587Z", "content": "Thanks for sharing! The score is very impressive.
\n\nSeveral questions I had:\n(1) What is the purpose for binary dilation as post-processing, compared to other methods such as opening? Does dilation make the masks larger? \n(1) Did learning rate decay help the convergence? I found decreasing LR to 1e-4 or below causes overfitting, but I was using SGD with momentum. Did you see an improvement using Adam vs. SGD? \n(2) Since you're using 512x512, do you use a batch size (IMAGES_PER_GPU) > 1? \nThanks!
", "rawMarkdown": "Thanks for sharing! The score is very impressive. \n\nSeveral questions I had:\n(1) What is the purpose for binary dilation as post-processing, compared to other methods such as opening? Does dilation make the masks larger? \n(1) Did learning rate decay help the convergence? I found decreasing LR to 1e-4 or below causes overfitting, but I was using SGD with momentum. Did you see an improvement using Adam vs. SGD? \n(2) Since you're using 512x512, do you use a batch size (IMAGES_PER_GPU) > 1? \nThanks! ", "votes": 1, "replies": [ { "id": 316528, "postDate": "2018-04-19T09:33:15.253Z", "content": "Hi, Yueqi. Dilation is generally used to extend edges and fill some small holes. As Heng mentioned before \"Deep network is the weakest at the boundary image and strongest are the center.\". In this competition, it is very necessary to do some works on boundary detection. It actually improve my result. (2) I have not used SGD, so it is hard to tell which one is more suitable in this competition. As far as i know, Keven Wang and Panpan Zhou is using SGD, you can discuss with them. (3) batch_size=1. Thanks Zheng
", "rawMarkdown": "Hi, Yueqi. Dilation is generally used to extend edges and fill some small holes. As Heng mentioned before \"Deep network is the weakest at the boundary image and strongest are the center.\". In this competition, it is very necessary to do some works on boundary detection. It actually improve my result. (2) I have not used SGD, so it is hard to tell which one is more suitable in this competition. As far as i know, Keven Wang and Panpan Zhou is using SGD, you can discuss with them. (3) batch_size=1. Thanks Zheng" }, { "id": 316751, "postDate": "2018-04-19T19:59:17.183Z", "content": "Thanks for the explanation!
", "rawMarkdown": "Thanks for the explanation! " }, { "id": 316754, "postDate": "2018-04-19T20:14:13.453Z", "content": "Hi Yueqi,
\n\nI have collected the parameters from various contestants here: \nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54920\n(this includes, Zheng, Waleed, Avi and me).
\n\nI will be happy to add your parameters to the table here:\nhttps://github.com/mpsampat/DSB_2018_Parameter_Selection_For_Matterport_Mask_RCNN/blob/master/comparison_of_Matterport_Mask_RCNN_parameters.md
\n\nThanks\nMehul
", "rawMarkdown": "Hi Yueqi, \n\nI have collected the parameters from various contestants here: \nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54920\n(this includes, Zheng, Waleed, Avi and me). \n\nI will be happy to add your parameters to the table here:\nhttps://github.com/mpsampat/DSB_2018_Parameter_Selection_For_Matterport_Mask_RCNN/blob/master/comparison_of_Matterport_Mask_RCNN_parameters.md\n\nThanks\nMehul", "votes": 1 }, { "id": 316757, "postDate": "2018-04-19T20:29:59.380Z", "content": "Hi Mehul,
\n\nThanks! It's a great idea to distill what works best for MaskRCNN. I'll follow up on your new thread.
", "rawMarkdown": "Hi Mehul,\n\nThanks! It's a great idea to distill what works best for MaskRCNN. I'll follow up on your new thread. ", "votes": 1 }, { "id": 317997, "postDate": "2018-04-23T02:38:11.727Z", "content": "Hi Mehul,
\n\nI've checked your summary for this model, it's great!
\n\nI also used this model for this competition and got similar results. And I found the threshold in unmold_mask function also has some influence on the final results. I wonder if you guys tried that.
\n\nAlso I've used the tiramisu model as well, and it's interesting that tiramisu needs a higher threshold to perform better while mask rcnn needs lower.
", "rawMarkdown": "Hi Mehul,\n\nI've checked your summary for this model, it's great!\n\nI also used this model for this competition and got similar results. And I found the threshold in unmold_mask function also has some influence on the final results. I wonder if you guys tried that.\n\nAlso I've used the tiramisu model as well, and it's interesting that tiramisu needs a higher threshold to perform better while mask rcnn needs lower." } ] }, { "id": 316320, "postDate": "2018-04-18T17:53:05.743Z", "content": "thanks for sharing, nice work and congrats
\n\nbtw. it just show how important it's do do CV resp. some early validation - also was using matterport but guess I overfit quite badly (though still don't understand how ... maybe I actually just \"overaugmented\", was doing a lot of color augmentation, blur, random noise, brigthness, rotations, all with 50% chance), next time I need to be more carefull of what I do :)
", "rawMarkdown": "thanks for sharing, nice work and congrats\n\nbtw. it just show how important it's do do CV resp. some early validation - also was using matterport but guess I overfit quite badly (though still don't understand how ... maybe I actually just \"overaugmented\", was doing a lot of color augmentation, blur, random noise, brigthness, rotations, all with 50% chance), next time I need to be more carefull of what I do :)", "votes": 1, "replies": [ { "id": 316469, "postDate": "2018-04-19T06:26:26.540Z", "content": "Hi steelrose. When I use general normalization, I guess this method doesn't work well with other methods, so I just use flip. 1st place solution use many augmentation methods and it actually works very well. So always choose the best methods for your model. Good luck. Thanks Zheng
", "rawMarkdown": "Hi steelrose. When I use general normalization, I guess this method doesn't work well with other methods, so I just use flip. 1st place solution use many augmentation methods and it actually works very well. So always choose the best methods for your model. Good luck. Thanks Zheng" } ] }, { "id": 316315, "postDate": "2018-04-18T17:39:51.600Z", "content": "Great job
", "rawMarkdown": "Great job", "votes": 1 }, { "id": 316272, "postDate": "2018-04-18T15:38:17.430Z", "content": "Awesome!
", "rawMarkdown": "Awesome!", "votes": 1 }, { "id": 316252, "postDate": "2018-04-18T14:38:31.560Z", "content": "Cool! Your model is generalized very well since it outperformed at the 2nd stage. Quite impressive. Very curious to know how your model can bump up so much. We have a very similar approach with a U-net, but our positions at both stages were pretty similar (at around top 2% at both stages).
\n\nCool @Emil will be happy to know you applied mosaic as well :) I just posted your post on his kernel. \nhttps://www.kaggle.com/bonlime/train-test-image-mosaic
", "rawMarkdown": "Cool! Your model is generalized very well since it outperformed at the 2nd stage. Quite impressive. Very curious to know how your model can bump up so much. We have a very similar approach with a U-net, but our positions at both stages were pretty similar (at around top 2% at both stages). \n\n - A question - If you rotated your images by 5 degree, how did you fill the part outside of the image, just left it 0? \n - Your model is very impressive, bumped up by LB=0.1 on the unseen data, it's worth a big praise. \n\nCool @Emil will be happy to know you applied mosaic as well :) I just posted your post on his kernel. \nhttps://www.kaggle.com/bonlime/train-test-image-mosaic\n", "votes": 1, "replies": [ { "id": 316260, "postDate": "2018-04-18T14:59:38.830Z", "content": "HI Nicole!
\n\nI think ZhengLi said none of the augmentations worked for him and he said he only used \"\"use flip up&down&left&right\"; For me too H&E data did not help and rotations by 0-15 did not help.
(We were discussing it within the team and we thought it might be due to interpolations)
Also @ZhengLi you forked Mask-RCNN from here https://github.com/YanWang2014/Mask_RCNN right ?
@Waleed made a lot of improvements to Mask-RCNN after that fork. He is the original author of Matterport Mask_RCNN; @Waleed got 0.476 on stage1 and we got 0.472; but we both did not go above 0.5 in stage2. I could only train for 25 epochs;
But there are atleast 2 other groups that went above 0.59 with Mask-RCNN.
Zheng used dilation for post-processing whereas we did no post-processing.
It is very interesting that differences in parameters made such a large difference in performance. It will be interesting to learn what is the root cause of difference in performance. Maybe how training/validation data is split ?
@Nicole Finnie\nHi Nicole. Thank you very much. :) Just left it 0. Here is my code.
\n\nfrom imgaug import augmenters as iaa\n\nAUGMENTATION_AFFINE = {\n \"order\": 0,\n \"rotate\": (-5, 5)}\n\nif config.AUGMENTATION_AFFINE:\n augmentations.append(iaa.Affine(**config.AUGMENTATION_AFFINE))\n\nseq=iaa.Sequential(augmentations)\nseq_det=seq.to_deterministic()\nimage, mask =seq_det.augment_image(image), seq_det.augment_image(mask)\n@Mehul Sampat Hi Mehul. Thanks for your explanation. You are right. I only use flip.\nMask rcnn code is from here: https://github.com/killthekitten/kaggle-ds-bowl-2018-baseline\nI am also very interested in other groups' solutions. I think training/validation data split is very important and it can tell us where is the best point( local optimum). Thanks Zheng
", "rawMarkdown": "@Mehul Sampat Hi Mehul. Thanks for your explanation. You are right. I only use flip.\nMask rcnn code is from here: https://github.com/killthekitten/kaggle-ds-bowl-2018-baseline\nI am also very interested in other groups' solutions. I think training/validation data split is very important and it can tell us where is the best point( local optimum). Thanks Zheng" }, { "id": 317567, "postDate": "2018-04-21T21:45:05.967Z", "content": "Hi Zheng. Thanks for sharing your solution and I am really impressed that you managed to achieve such great result with the minimal tweak to matterport's solution.
\n\nMay I ask how did you split the images between training and validation sets?
\n\nMany thanks in advance!
", "rawMarkdown": "Hi Zheng. Thanks for sharing your solution and I am really impressed that you managed to achieve such great result with the minimal tweak to matterport's solution.\n\nMay I ask how did you split the images between training and validation sets? \n\nMany thanks in advance!", "votes": 1 }, { "id": 317732, "postDate": "2018-04-22T12:08:30.063Z", "rawMarkdown": "", "votes": -6, "isDeleted": true }, { "id": 317746, "postDate": "2018-04-22T12:56:49.907Z", "content": "I agree. I gave them up vote because I thought the post gave some clarification. but I also think kaggle didn't handle this issue properly and fairly. I am with you on this one.
", "rawMarkdown": "I agree. I gave them up vote because I thought the post gave some clarification. but I also think kaggle didn't handle this issue properly and fairly. I am with you on this one. " }, { "id": 317763, "postDate": "2018-04-22T13:52:50.627Z", "content": "Hi YL. Thank you very much.
", "rawMarkdown": "Hi YL. Thank you very much.", "votes": -1 } ] }, { "id": 316251, "postDate": "2018-04-18T14:37:04.567Z", "content": "@ZhengLi, Congratulations! and thanks for sharing your approach.
", "rawMarkdown": "@ZhengLi, Congratulations! and thanks for sharing your approach.", "votes": 1 }, { "id": 316238, "postDate": "2018-04-18T14:12:13.027Z", "content": "Thanks for your sharing and congrats to you ！
", "rawMarkdown": "Thanks for your sharing and congrats to you ！", "votes": 1 }, { "id": 317769, "postDate": "2018-04-22T14:09:09.117Z", "content": "hi Zheng, Congratulations! You say \"Less FPs is the key to get high mAP as Heng said.\" So what the meaning of Fps here? thanks.
", "rawMarkdown": "hi Zheng, Congratulations! You say \"Less FPs is the key to get high mAP as Heng said.\" So what the meaning of Fps here? thanks.", "votes": 2, "replies": [ { "id": 317848, "postDate": "2018-04-22T17:17:40.510Z", "content": "FP=False Positive. Detected masks that are actually not there.
", "rawMarkdown": "FP=False Positive. Detected masks that are actually not there.", "votes": 2 } ] }, { "id": 316481, "postDate": "2018-04-19T07:13:25.027Z", "content": "Congrats!\nI have 3 questions to consult:\n1. I am also using matterport mask-rcnn with image-size 1024 * 1024. I found it gets better score than small sizes. What size are you using?\n2. you are using a very low DETECTION_NMS_THRESHOLD, will this cause problems when the GT is few? \n3. you are using zero MEAN_PIXEL, what is the consideration for this?\nThank you.
", "rawMarkdown": "Congrats!\nI have 3 questions to consult:\n1. I am also using matterport mask-rcnn with image-size 1024 * 1024. I found it gets better score than small sizes. What size are you using?\n2. you are using a very low DETECTION_NMS_THRESHOLD, will this cause problems when the GT is few? \n3. you are using zero MEAN_PIXEL, what is the consideration for this?\nThank you.", "votes": 2, "replies": [ { "id": 316497, "postDate": "2018-04-19T07:54:51.677Z", "content": "Hi YUNFEI. 1. I am using 512*512. \n2. Sorry there are some parameters I didn't mentioned above. When I was training, I set DETECTION_NMS_THRESHOLD = 0.3, DETECTION_MIN_CONFIDENCE = 0.7. \nPOST_NMS_ROIS_TRAINING = 2000, POST_NMS_ROIS_INFERENCE = 2000\n3. general standardization can speed up the convergence of your algorithm.
", "rawMarkdown": "Hi YUNFEI. 1. I am using 512*512. \n2. Sorry there are some parameters I didn't mentioned above. When I was training, I set DETECTION_NMS_THRESHOLD = 0.3, DETECTION_MIN_CONFIDENCE = 0.7. \nPOST_NMS_ROIS_TRAINING = 2000, POST_NMS_ROIS_INFERENCE = 2000\n3. general standardization can speed up the convergence of your algorithm.", "votes": 1 }, { "id": 316505, "postDate": "2018-04-19T08:22:03.387Z", "content": "Thank you for your explain, I will have a try.
", "rawMarkdown": "Thank you for your explain, I will have a try." }, { "id": 316513, "postDate": "2018-04-19T08:39:01.370Z", "content": "I also found that 1024x1024 got me better score than smaller sizes. So I was surprised to see 512x512 without cropping to work so well.
", "rawMarkdown": "I also found that 1024x1024 got me better score than smaller sizes. So I was surprised to see 512x512 without cropping to work so well. " } ] }, { "id": 316271, "postDate": "2018-04-18T15:35:39.830Z", "content": "Nice work!
", "rawMarkdown": " Nice work!", "votes": 2 }, { "id": 316242, "postDate": "2018-04-18T14:24:19.513Z", "content": "666
", "rawMarkdown": "666", "votes": -4 }, { "id": 316314, "postDate": "2018-04-18T17:32:58.763Z", "content": "Thanks for sharing, I was wondering if compared the model with pretrained coco weights and without? And whether or not it made a significant difference?
", "rawMarkdown": "Thanks for sharing, I was wondering if compared the model with pretrained coco weights and without? And whether or not it made a significant difference?", "replies": [ { "id": 316467, "postDate": "2018-04-19T06:16:54.393Z", "content": "Hi. I only compared the model with pretrained coco and imagenet and i found that coco is better than imagenet in this competition.
", "rawMarkdown": "Hi. I only compared the model with pretrained coco and imagenet and i found that coco is better than imagenet in this competition.", "votes": 1 }, { "id": 316582, "postDate": "2018-04-19T11:29:36.140Z", "content": "Thanks
", "rawMarkdown": "Thanks" } ] }, { "id": 316305, "postDate": "2018-04-18T17:09:28.917Z", "content": "Great job! Thanks!
", "rawMarkdown": "Great job! Thanks!", "votes": 1 }, { "id": 316286, "postDate": "2018-04-18T16:16:54.793Z", "content": "Nice job! Thanks for sharing!
", "rawMarkdown": "Nice job! Thanks for sharing!", "votes": 1 }, { "id": 318087, "postDate": "2018-04-23T07:08:33.203Z", "content": "Well done ! Thanks for sharing :)
", "rawMarkdown": "Well done ! Thanks for sharing :) " } ], "comments": [ { "id": 318739, "author_name": "Nicole Finnie", "author_url": "", "post_date": "2018-04-24T11:32:06.060000", "content": "@Zheng Li , whew, I'm glad you'll be reinstated, make sure to upload your model in your next 2-stage competition, since you were very close to the prize this time. You're the \"outlier\" of the samples I'd say ;) your model is very well generalized and totally beat our LB at the 2nd stage even our LB was higher at the 1st stage. Good job.
", "votes": 1, "replies": [ { "id": 318746, "author_name": "Zheng Li", "author_url": "", "post_date": "2018-04-24T11:55:17.420000", "content": "Hi Nicole. Thank you. See you in the next competition. ;)
", "votes": 1, "replies": [] }, { "id": 1621965, "author_name": "Jaideep", "author_url": "", "post_date": "2021-12-18T08:43:12.260000", "content": "@mdlszhengli opening very old discussion.
\ncould you explain about the fitering method you used in the post processing
Nice job, Congratulations, Zheng!
", "votes": 1, "replies": [] }, { "id": 316510, "author_name": "yueqi", "author_url": "", "post_date": "2018-04-19T08:36:55.587000", "content": "Thanks for sharing! The score is very impressive.
\n\nSeveral questions I had:\n(1) What is the purpose for binary dilation as post-processing, compared to other methods such as opening? Does dilation make the masks larger? \n(1) Did learning rate decay help the convergence? I found decreasing LR to 1e-4 or below causes overfitting, but I was using SGD with momentum. Did you see an improvement using Adam vs. SGD? \n(2) Since you're using 512x512, do you use a batch size (IMAGES_PER_GPU) > 1? \nThanks!
", "votes": 1, "replies": [ { "id": 316528, "author_name": "Zheng Li", "author_url": "", "post_date": "2018-04-19T09:33:15.253000", "content": "Hi, Yueqi. Dilation is generally used to extend edges and fill some small holes. As Heng mentioned before \"Deep network is the weakest at the boundary image and strongest are the center.\". In this competition, it is very necessary to do some works on boundary detection. It actually improve my result. (2) I have not used SGD, so it is hard to tell which one is more suitable in this competition. As far as i know, Keven Wang and Panpan Zhou is using SGD, you can discuss with them. (3) batch_size=1. Thanks Zheng
", "votes": 0, "replies": [] }, { "id": 316751, "author_name": "yueqi", "author_url": "", "post_date": "2018-04-19T19:59:17.183000", "content": "Thanks for the explanation!
", "votes": 0, "replies": [] }, { "id": 316754, "author_name": "Mehul Sampat", "author_url": "", "post_date": "2018-04-19T20:14:13.453000", "content": "Hi Yueqi,
\n\nI have collected the parameters from various contestants here: \nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/54920\n(this includes, Zheng, Waleed, Avi and me).
\n\nI will be happy to add your parameters to the table here:\nhttps://github.com/mpsampat/DSB_2018_Parameter_Selection_For_Matterport_Mask_RCNN/blob/master/comparison_of_Matterport_Mask_RCNN_parameters.md
\n\nThanks\nMehul
", "votes": 1, "replies": [] }, { "id": 316757, "author_name": "yueqi", "author_url": "", "post_date": "2018-04-19T20:29:59.380000", "content": "Hi Mehul,
\n\nThanks! It's a great idea to distill what works best for MaskRCNN. I'll follow up on your new thread.
", "votes": 1, "replies": [] }, { "id": 317997, "author_name": "ryan", "author_url": "", "post_date": "2018-04-23T02:38:11.727000", "content": "Hi Mehul,
\n\nI've checked your summary for this model, it's great!
\n\nI also used this model for this competition and got similar results. And I found the threshold in unmold_mask function also has some influence on the final results. I wonder if you guys tried that.
\n\nAlso I've used the tiramisu model as well, and it's interesting that tiramisu needs a higher threshold to perform better while mask rcnn needs lower.
", "votes": 0, "replies": [] } ] }, { "id": 316320, "author_name": "steelrose", "author_url": "", "post_date": "2018-04-18T17:53:05.743000", "content": "thanks for sharing, nice work and congrats
\n\nbtw. it just show how important it's do do CV resp. some early validation - also was using matterport but guess I overfit quite badly (though still don't understand how ... maybe I actually just \"overaugmented\", was doing a lot of color augmentation, blur, random noise, brigthness, rotations, all with 50% chance), next time I need to be more carefull of what I do :)
", "votes": 1, "replies": [ { "id": 316469, "author_name": "Zheng Li", "author_url": "", "post_date": "2018-04-19T06:26:26.540000", "content": "Hi steelrose. When I use general normalization, I guess this method doesn't work well with other methods, so I just use flip. 1st place solution use many augmentation methods and it actually works very well. So always choose the best methods for your model. Good luck. Thanks Zheng
", "votes": 0, "replies": [] } ] }, { "id": 316315, "author_name": "William Green", "author_url": "", "post_date": "2018-04-18T17:39:51.600000", "content": "Great job
", "votes": 1, "replies": [] }, { "id": 316272, "author_name": "CPMP", "author_url": "", "post_date": "2018-04-18T15:38:17.430000", "content": "Awesome!
", "votes": 1, "replies": [] }, { "id": 316252, "author_name": "Nicole Finnie", "author_url": "", "post_date": "2018-04-18T14:38:31.560000", "content": "Cool! Your model is generalized very well since it outperformed at the 2nd stage. Quite impressive. Very curious to know how your model can bump up so much. We have a very similar approach with a U-net, but our positions at both stages were pretty similar (at around top 2% at both stages).
\n\nCool @Emil will be happy to know you applied mosaic as well :) I just posted your post on his kernel. \nhttps://www.kaggle.com/bonlime/train-test-image-mosaic
", "votes": 1, "replies": [ { "id": 316260, "author_name": "Mehul Sampat", "author_url": "", "post_date": "2018-04-18T14:59:38.830000", "content": "HI Nicole!
\n\nI think ZhengLi said none of the augmentations worked for him and he said he only used \"\"use flip up&down&left&right\"; For me too H&E data did not help and rotations by 0-15 did not help.
(We were discussing it within the team and we thought it might be due to interpolations)
Also @ZhengLi you forked Mask-RCNN from here https://github.com/YanWang2014/Mask_RCNN right ?
@Waleed made a lot of improvements to Mask-RCNN after that fork. He is the original author of Matterport Mask_RCNN; @Waleed got 0.476 on stage1 and we got 0.472; but we both did not go above 0.5 in stage2. I could only train for 25 epochs;
But there are atleast 2 other groups that went above 0.59 with Mask-RCNN.
Zheng used dilation for post-processing whereas we did no post-processing.
It is very interesting that differences in parameters made such a large difference in performance. It will be interesting to learn what is the root cause of difference in performance. Maybe how training/validation data is split ?
@Nicole Finnie\nHi Nicole. Thank you very much. :) Just left it 0. Here is my code.
\n\nfrom imgaug import augmenters as iaa\n\nAUGMENTATION_AFFINE = {\n \"order\": 0,\n \"rotate\": (-5, 5)}\n\nif config.AUGMENTATION_AFFINE:\n augmentations.append(iaa.Affine(**config.AUGMENTATION_AFFINE))\n\nseq=iaa.Sequential(augmentations)\nseq_det=seq.to_deterministic()\nimage, mask =seq_det.augment_image(image), seq_det.augment_image(mask)\n@Mehul Sampat Hi Mehul. Thanks for your explanation. You are right. I only use flip.\nMask rcnn code is from here: https://github.com/killthekitten/kaggle-ds-bowl-2018-baseline\nI am also very interested in other groups' solutions. I think training/validation data split is very important and it can tell us where is the best point( local optimum). Thanks Zheng
", "votes": 0, "replies": [] }, { "id": 317567, "author_name": "YL", "author_url": "", "post_date": "2018-04-21T21:45:05.967000", "content": "Hi Zheng. Thanks for sharing your solution and I am really impressed that you managed to achieve such great result with the minimal tweak to matterport's solution.
\n\nMay I ask how did you split the images between training and validation sets?
\n\nMany thanks in advance!
", "votes": 1, "replies": [] }, { "id": 317732, "author_name": "", "author_url": "", "post_date": "2018-04-22T12:08:30.063000", "content": "", "votes": -6, "replies": [] }, { "id": 317746, "author_name": "YL", "author_url": "", "post_date": "2018-04-22T12:56:49.907000", "content": "I agree. I gave them up vote because I thought the post gave some clarification. but I also think kaggle didn't handle this issue properly and fairly. I am with you on this one.
", "votes": 0, "replies": [] }, { "id": 317763, "author_name": "Zheng Li", "author_url": "", "post_date": "2018-04-22T13:52:50.627000", "content": "Hi YL. Thank you very much.
", "votes": -1, "replies": [] } ] }, { "id": 316251, "author_name": "Mehul Sampat", "author_url": "", "post_date": "2018-04-18T14:37:04.567000", "content": "@ZhengLi, Congratulations! and thanks for sharing your approach.
", "votes": 1, "replies": [] }, { "id": 316238, "author_name": "Zhiran Ye", "author_url": "", "post_date": "2018-04-18T14:12:13.027000", "content": "Thanks for your sharing and congrats to you ！
", "votes": 1, "replies": [] }, { "id": 317769, "author_name": "chen w", "author_url": "", "post_date": "2018-04-22T14:09:09.117000", "content": "hi Zheng, Congratulations! You say \"Less FPs is the key to get high mAP as Heng said.\" So what the meaning of Fps here? thanks.
", "votes": 2, "replies": [ { "id": 317848, "author_name": "Oscar Takeshita", "author_url": "", "post_date": "2018-04-22T17:17:40.510000", "content": "FP=False Positive. Detected masks that are actually not there.
", "votes": 2, "replies": [] } ] }, { "id": 316481, "author_name": "YUNFEI DUAN", "author_url": "", "post_date": "2018-04-19T07:13:25.027000", "content": "Congrats!\nI have 3 questions to consult:\n1. I am also using matterport mask-rcnn with image-size 1024 * 1024. I found it gets better score than small sizes. What size are you using?\n2. you are using a very low DETECTION_NMS_THRESHOLD, will this cause problems when the GT is few? \n3. you are using zero MEAN_PIXEL, what is the consideration for this?\nThank you.
", "votes": 2, "replies": [ { "id": 316497, "author_name": "Zheng Li", "author_url": "", "post_date": "2018-04-19T07:54:51.677000", "content": "Hi YUNFEI. 1. I am using 512*512. \n2. Sorry there are some parameters I didn't mentioned above. When I was training, I set DETECTION_NMS_THRESHOLD = 0.3, DETECTION_MIN_CONFIDENCE = 0.7. \nPOST_NMS_ROIS_TRAINING = 2000, POST_NMS_ROIS_INFERENCE = 2000\n3. general standardization can speed up the convergence of your algorithm.
", "votes": 1, "replies": [] }, { "id": 316505, "author_name": "YUNFEI DUAN", "author_url": "", "post_date": "2018-04-19T08:22:03.387000", "content": "Thank you for your explain, I will have a try.
", "votes": 0, "replies": [] }, { "id": 316513, "author_name": "yueqi", "author_url": "", "post_date": "2018-04-19T08:39:01.370000", "content": "I also found that 1024x1024 got me better score than smaller sizes. So I was surprised to see 512x512 without cropping to work so well.
", "votes": 0, "replies": [] } ] }, { "id": 316271, "author_name": "hengck23", "author_url": "", "post_date": "2018-04-18T15:35:39.830000", "content": "Nice work!
", "votes": 2, "replies": [] }, { "id": 316242, "author_name": "NullPoint", "author_url": "", "post_date": "2018-04-18T14:24:19.513000", "content": "666
", "votes": -4, "replies": [] }, { "id": 316314, "author_name": "LivingProgram", "author_url": "", "post_date": "2018-04-18T17:32:58.763000", "content": "Thanks for sharing, I was wondering if compared the model with pretrained coco weights and without? And whether or not it made a significant difference?
", "votes": 0, "replies": [ { "id": 316467, "author_name": "Zheng Li", "author_url": "", "post_date": "2018-04-19T06:16:54.393000", "content": "Hi. I only compared the model with pretrained coco and imagenet and i found that coco is better than imagenet in this competition.
", "votes": 1, "replies": [] }, { "id": 316582, "author_name": "LivingProgram", "author_url": "", "post_date": "2018-04-19T11:29:36.140000", "content": "Thanks
", "votes": 0, "replies": [] } ] }, { "id": 316305, "author_name": "Brian Luo", "author_url": "", "post_date": "2018-04-18T17:09:28.917000", "content": "Great job! Thanks!
", "votes": 1, "replies": [] }, { "id": 316286, "author_name": "Panpan Zhou", "author_url": "", "post_date": "2018-04-18T16:16:54.793000", "content": "Nice job! Thanks for sharing!
", "votes": 1, "replies": [] }, { "id": 318087, "author_name": "Nathan Lauga", "author_url": "", "post_date": "2018-04-23T07:08:33.203000", "content": "Well done ! Thanks for sharing :)
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "316229": "（currently 8th place while leaderboard is being updated）\n\nHi all. It is my first time join this competition. All my methods are learned from open discussion. As return, I decide to share my solution to the community. Thanks a lot to those who have made many contributions to the community. \n\nFirst of all, I thank Allen, John1231983, Keven Wang, Mehul Sampat, Heng CherKeng, CPMP, bsp2020, kamil, YaGana Sheriff-Hussaini, Yan Wang, Konstantin Maksimov, xiapidan, xuan, Waleed for your very impressive discussions. I learned a lot from those discussions. @Mehul Sampat hope we can team up together next time :) thanks again.\n\nMy score: Stage1 LB 0.489(87th), Stage2 LB 0.590(11th). I am using matterport mask-rcnn.\n\n**Augmentations**\n\nSince there are hundreds of training images, we must find some useful augmentations to prevent our models from overfitting and make them generalizable. Here are some methods I have tried but it didn't work for me:\n \n - add gaussian noise\n - color to gray\n - contrast and brightness\n - random crop 512x512 if image size is bigger than 512 otherwise resize the image to 512x512\n - mosacis\n - mosacis+random crop\n - mosacis+random crop+ h&e\n - rotate 90 degrees \n - radnom rotate 90,180,270 degrees\n - rotate +-5 degrees on top of flip & 90 degree rotation\n - elastic transform\n\nI only use flip up&down&left&right\n\n**Additional data**\n\nNo. I have tried to add h&e dataset to training data, but it didn't improve my performance. I am using https://github.com/lopuhin/kaggle-dsbowl-2018-dataset-fixes\n\n**Ensembling**\n\nNo. I spend almost one week trying this method but it didn't work so well. I devided the training data into two categories: color & grey. I have seen someone get a high score (0.5+) in stage1 by using this method. I am very interested in this so I really hope that someone can share some solutions about this.\n\n**Parameters**\n\ntrain:\n\ninti_with= coco\n\nRESNET_ARCHITECTURE = \"resnet101\"\n\nMEAN_PIXEL = np.array([0., 0., 0.])\n\nRPN_NMS_THRESHOLD = 0.7\n\nDETECTION_MIN_CONFIDENCE = 0.7\n\nDETECTION_NMS_THRESHOLD = 0.3\n\nTRAIN_ROIS_PER_IMAGE = 600\n\nRPN_TRAIN_ANCHORS_PER_IMAGE = 320\n\nLEARNING_RATE = 1e-3\n\ninference:\n\nRPN_NMS_THRESHOLD = 0.6\n\nDETECTION_NMS_THRESHOLD = 0.1\n\n**Training**\n\n1e-3 all 20epochs. Choose the best point according to the display of tensorboard, then 1e-4 or 1e-5 train all 20epochs. (if val loss stops decreasing, stops training) \noptimizer=Adam.\n\n**Post processing** \nUse binary_dilation.\n\n**Other method**\ngeneral standardization.\nimg = img-mean(img)/std(img).\n\n**Some methods might be useful**\n\n1.Filter\n\n> We use single mask rcnn, but with special post process to filter noises (i.e. small FPs) based on clustering and outlier detection of mask sizes, which boosted us a lot (kill lots of noises). Less FPs is the key to get high mAP as Heng said. I should thank him a lot\n\n2.fill_holes\n\nhttps://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.ndimage.morphology.binary_fill_holes.html#scipy-ndimage-morphology-binary-fill-holes\n\n3.wateshed\n\n**Some interesting discussions&methods links**\n\n 1. https://github.com/matterport/Mask_RCNN/issues/230\n 2. https://github.com/matterport/Mask_RCNN/issues/281\n 3. https://github.com/killthekitten/kaggle-ds-bowl-2018-baseline/issues/5\n 4. https://www.kaggle.com/bostjanm/overlapping-objects-separation-method/notebook\n 5. https://www.kaggle.com/c/data-science-bowl-2018/discussion/52989#30758\n\n**Last**\n\nI have tried a lot of experiments. Some of them i just don't remeber.....sorry :( So here is my email mdlszhengli@@gmail.com. I am very happy to discuss via email or wechat(removed).", "318739": "@Zheng Li , whew, I'm glad you'll be reinstated, make sure to upload your model in your next 2-stage competition, since you were very close to the prize this time. You're the \"outlier\" of the samples I'd say ;) your model is very well generalized and totally beat our LB at the 2nd stage even our LB was higher at the 1st stage. Good job. ", "316775": "Nice job, Congratulations, Zheng!", "316510": "Thanks for sharing! The score is very impressive. \n\nSeveral questions I had:\n(1) What is the purpose for binary dilation as post-processing, compared to other methods such as opening? Does dilation make the masks larger? \n(1) Did learning rate decay help the convergence? I found decreasing LR to 1e-4 or below causes overfitting, but I was using SGD with momentum. Did you see an improvement using Adam vs. SGD? \n(2) Since you're using 512x512, do you use a batch size (IMAGES_PER_GPU) > 1? \nThanks! ", "316320": "thanks for sharing, nice work and congrats\n\nbtw. it just show how important it's do do CV resp. some early validation - also was using matterport but guess I overfit quite badly (though still don't understand how ... maybe I actually just \"overaugmented\", was doing a lot of color augmentation, blur, random noise, brigthness, rotations, all with 50% chance), next time I need to be more carefull of what I do :)", "316315": "Great job", "316272": "Awesome!", "316252": "Cool! Your model is generalized very well since it outperformed at the 2nd stage. Quite impressive. Very curious to know how your model can bump up so much. We have a very similar approach with a U-net, but our positions at both stages were pretty similar (at around top 2% at both stages). \n\n - A question - If you rotated your images by 5 degree, how did you fill the part outside of the image, just left it 0? \n - Your model is very impressive, bumped up by LB=0.1 on the unseen data, it's worth a big praise. \n\nCool @Emil will be happy to know you applied mosaic as well :) I just posted your post on his kernel. \nhttps://www.kaggle.com/bonlime/train-test-image-mosaic\n", "316251": "@ZhengLi, Congratulations! and thanks for sharing your approach.", "316238": "Thanks for your sharing and congrats to you ！", "317769": "hi Zheng, Congratulations! You say \"Less FPs is the key to get high mAP as Heng said.\" So what the meaning of Fps here? thanks.", "316481": "Congrats!\nI have 3 questions to consult:\n1. I am also using matterport mask-rcnn with image-size 1024 * 1024. I found it gets better score than small sizes. What size are you using?\n2. you are using a very low DETECTION_NMS_THRESHOLD, will this cause problems when the GT is few? \n3. you are using zero MEAN_PIXEL, what is the consideration for this?\nThank you.", "316271": " Nice work!", "316242": "666", "316314": "Thanks for sharing, I was wondering if compared the model with pretrained coco weights and without? And whether or not it made a significant difference?", "316305": "Great job! Thanks!", "316286": "Nice job! Thanks for sharing!", "318087": "Well done ! Thanks for sharing :) " }, "complete": true, "source": "existing_discussion", "competition_id": 8089, "competition_slug": "data-science-bowl-2018", "team_id": 1437044, "solution_url": "/competitions/data-science-bowl-2018/writeups/zheng-li-my-solution-11th-place-lb-0-590" }