Hi, I don't know either U-Net or mask R-CNN models. But i see most of the contestants are using one of these. So could someone help me to choose one of the two?\nAlso, should i be using a pre-trained model, or should i be building the model from scratch?\nThanks a lot.
", "messages": [ { "id": "293677", "postDate": "03/10/2018 11:04:56", "content": "Hi, I don't know either U-Net or mask R-CNN models. But i see most of the contestants are using one of these. So could someone help me to choose one of the two?\nAlso, should i be using a pre-trained model, or should i be building the model from scratch?\nThanks a lot.
", "rawMarkdown": "Hi, I don't know either U-Net or mask R-CNN models. But i see most of the contestants are using one of these. So could someone help me to choose one of the two?\nAlso, should i be using a pre-trained model, or should i be building the model from scratch?\nThanks a lot.", "votes": null }, { "id": "293697", "postDate": "03/10/2018 11:42:19", "content": "Mask-rcnn solves the problem direclty, i.e. it produces masks for each recognized object. Unet can produce only one mask, hence it is used to predict the union of all masks, followed by a post processing to split the predicted mask into one mask per object.
\n\nUnet is way simpler than mask rcnn, but it requires way more complex post processing. There is no silver bullet here I'm afraid.
", "rawMarkdown": "Mask-rcnn solves the problem direclty, i.e. it produces masks for each recognized object. Unet can produce only one mask, hence it is used to predict the union of all masks, followed by a post processing to split the predicted mask into one mask per object.\n\nUnet is way simpler than mask rcnn, but it requires way more complex post processing. There is no silver bullet here I'm afraid.", "votes": null }, { "id": "293951", "postDate": "03/10/2018 23:20:07", "content": "", "rawMarkdown": "", "votes": null }, { "id": "294199", "postDate": "03/11/2018 14:55:21", "content": "Hi CPMP
\n\nYour team name is \"U-net Nuke\" (it is a very interesting name:-) ) So I guess you use U-net too? If it is true I must say that your job is amazing , 0.461 Is a super high score for pure U-net model
\n\nI use U-net too and I use watershed algorithm to split the connected mask, The highest score I got in LB by my U-net model is 0.440 And I cant make any progress I try some methods but sadly the score in LB is lower and lower.... (only around 0.42).
\n\nEspecially the \"white - purple\" images like this one (17b9bf4356db24967c4677b8376ac38f826de73a88b93a8d73a8b452e399cdff.png\n) are disaster for U-net
May I ask what is your \"Nuke\" ?
", "rawMarkdown": "Hi CPMP\n\nYour team name is \"U-net Nuke\" (it is a very interesting name:-) ) So I guess you use U-net too? If it is true I must say that your job is amazing , 0.461 Is a super high score for pure U-net model\n\nI use U-net too and I use watershed algorithm to split the connected mask, The highest score I got in LB by my U-net model is 0.440 And I cant make any progress I try some methods but sadly the score in LB is lower and lower.... (only around 0.42).\n\nEspecially the \"white - purple\" images like this one (`17b9bf4356db24967c4677b8376ac38f826de73a88b93a8d73a8b452e399cdff.png\n`) are disaster for U-net\n\nMay I ask what is your \"Nuke\" ?", "votes": null }, { "id": "294226", "postDate": "03/11/2018 15:41:01", "content": "Yes, I use only Unet. And I am not using mosaic, or cropping or any data data augmentation beside image flipping. I hope this leaves some room for improvement ;) But that will have to wait until Toxic competition ends as I am busy there (each time I come back here I see my rank going down...)
\n\nYes, I am using some watershed too, so you're on the right track to beat my score! And yes, the white purple images are the toughest for me too. I guess this is where mask crnn makes adifference because in a way it looks for ellipses whereas unet can predict any mask.
\n\nNuke refers to nuclei, no secret weapon on my side ;)
", "rawMarkdown": "Yes, I use only Unet. And I am not using mosaic, or cropping or any data data augmentation beside image flipping. I hope this leaves some room for improvement ;) But that will have to wait until Toxic competition ends as I am busy there (each time I come back here I see my rank going down...)\n\nYes, I am using some watershed too, so you're on the right track to beat my score! And yes, the white purple images are the toughest for me too. I guess this is where mask crnn makes adifference because in a way it looks for ellipses whereas unet can predict any mask.\n\nNuke refers to nuclei, no secret weapon on my side ;)", "votes": null }, { "id": "294245", "postDate": "03/11/2018 16:19:57", "content": "@CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.
\n\nI suspect it should better than mask-rcnn for some type of cells (e.g. small or very elongated cells?)
", "rawMarkdown": "CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.\n\nI suspect it should better than mask-rcnn for some type of cells (e.g. small or very elongated cells?)", "votes": null }, { "id": "294332", "postDate": "03/11/2018 18:48:05", "content": "Me too. My score goes down every day :(. I guess someone just run the shared code from Heng and submit it. As the mentioned in his post, it can achieved about 0.51 LB when you run this code without any modification. Am I right about score, Heng?
", "rawMarkdown": "Me too. My score goes down every day :(. I guess someone just run the shared code from Heng and submit it. As the mentioned in his post, it can achieved about 0.51 LB when you run this code without any modification. Am I right about score, Heng?", "votes": null }, { "id": "294385", "postDate": "03/11/2018 21:24:41", "content": "@heng, very interesting. You are right, marker seeds are the key.
", "rawMarkdown": "heng, very interesting. You are right, marker seeds are the key.", "votes": null }, { "id": "294424", "postDate": "03/11/2018 23:50:50", "content": "@John, interesting, which code you are using now?
", "rawMarkdown": "John, interesting, which code you are using now?", "votes": null }, { "id": "294432", "postDate": "03/12/2018 00:20:14", "content": "@john1231983 no, you need modification to my code (like pre, post processing)
", "rawMarkdown": "john1231983 no, you need modification to my code (like pre, post processing)", "votes": null }, { "id": "294443", "postDate": "03/12/2018 01:00:39", "content": "@Yan: I used matterport code in Keras. I did not family with pytorch so I choose it. The baseline of this code is 0.41~0.44 LB with coco and same learning schedule with coco: first training in the heads then train all. @Heng: Could you tell me how many LBs we can achieve if we use your published code? I guess pytorch often better than Keras if we use same idea
i think my baseline performance is about the same at 0.45. But i think the software package is not important. It is better to stick to one that you are familiar, becuase you need many changes to get to the range of LB 0.50 and above.
", "rawMarkdown": "i think my baseline performance is about the same at 0.45. But i think the software package is not important. It is better to stick to one that you are familiar, becuase you need many changes to get to the range of LB 0.50 and above.", "votes": null }, { "id": "294630", "postDate": "03/12/2018 09:52:29", "content": "@john1231983, I rather think it is the mosaic things, and various pre/post processing ideas that have been shared directly, or via shared published papers.
", "rawMarkdown": "john1231983, I rather think it is the mosaic things, and various pre/post processing ideas that have been shared directly, or via shared published papers.", "votes": null }, { "id": "294741", "postDate": "03/12/2018 14:59:29", "content": "@CPMP Currently, mask-rcnn only predict foreground pixels for each detected nucleus. But we can modify mask-rcnn to output foreground pixel, center and boundary per detected nucleus. These information can be used for further post processing.
\n\nauxiliary information may be useful. e.g. given a cropped patch from rpn/rcnn proposals, we now predict only the bounding box and class label. We can add other predicted attributes like, nucleus size, which train image it comes from (i.e. train image id), density (nucleus per area), background/foreground color, etc ...
\n\nSuch attributes are also useful for error correction and post processing or making a level-two classifier for stacking
", "rawMarkdown": "CPMP Currently, mask-rcnn only predict foreground pixels for each detected nucleus. But we can modify mask-rcnn to output foreground pixel, center and boundary per detected nucleus. These information can be used for further post processing.\n\nauxiliary information may be useful. e.g. given a cropped patch from rpn/rcnn proposals, we now predict only the bounding box and class label. We can add other predicted attributes like, nucleus size, which train image it comes from (i.e. train image id), density (nucleus per area), background/foreground color, etc ...\n\nSuch attributes are also useful for error correction and post processing or making a level-two classifier for stacking", "votes": null }, { "id": "294750", "postDate": "03/12/2018 15:10:41", "content": "\n\n\n@CPMP Currently, mask-rcnn only predict foreground pixels for each detected nucleus. But we can modify mask-rcnn to output foreground pixel, center and boundary per detected nucleus. These information can be used for further post processing.
\n
Agreed, same can be done for Unet, for the same purpose ;)
", "rawMarkdown": "> @CPMP Currently, mask-rcnn only predict foreground pixels for each detected nucleus. But we can modify mask-rcnn to output foreground pixel, center and boundary per detected nucleus. These information can be used for further post processing.\n\nAgreed, same can be done for Unet, for the same purpose ;)", "votes": null }, { "id": "295863", "postDate": "03/14/2018 10:29:22", "content": "thanks for your reply ! \nMy u-net model predict the foreground pixel, center and boundary too , but I didn't use the center
\nas markers for watershed function but use the longest distance(from one pixel to the background) , It looks like the difference between the two is very small. \nYou mean I should use the boundary and center I predicted as the markers ?
Mask R-CNN has much more hyperparams so it can end up as a big pain for you if you start the competition late. I tried both approaches and decided on U-Net eventually. You can compare for yourself rather quickly using one of the ready-to-submit solutions:
\n\nThat maybe a good idea. I have not (yet) tried to predict mask center with unet.
", "rawMarkdown": "That maybe a good idea. I have not (yet) tried to predict mask center with unet.", "votes": null }, { "id": "300312", "postDate": "03/21/2018 14:04:32", "content": "\n\n\n@CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.
\n
@Heng I'll be back soon and will try it. However, I need a little time to digest the losss of our gold medal in toxic after our team was removed.
", "rawMarkdown": "> @CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.\n\n@Heng I'll be back soon and will try it. However, I need a little time to digest the losss of our gold medal in toxic after our team was removed.", "votes": null }, { "id": "304961", "postDate": "03/28/2018 09:38:07", "content": "@Heng CherKeng, I am ready to try it now. Which (public) output do you suggest I start with?
", "rawMarkdown": "Heng CherKeng, I am ready to try it now. Which (public) output do you suggest I start with?", "votes": null }, { "id": "304964", "postDate": "03/28/2018 09:44:40", "content": "https://drive.google.com/drive/folders/1EtBdlPK3BVahHBZtB103QQxNUdWaJ6di
\n\nunder 20180314, there are results for LB 0.43 for the gray 53 images.\nthis is based on mask-rcnn with se-resnext backbone.\nthe csv file is also there. You can start from there and see if post-processing make results better
", "rawMarkdown": "https://drive.google.com/drive/folders/1EtBdlPK3BVahHBZtB103QQxNUdWaJ6di\n\nunder 20180314, there are results for LB 0.43 for the gray 53 images.\nthis is based on mask-rcnn with se-resnext backbone.\nthe csv file is also there. You can start from there and see if post-processing make results better", "votes": null }, { "id": "304983", "postDate": "03/28/2018 10:16:24", "content": "Thanks. Just to be sure: the png shows the union of the masks your model predicted, and I need to parse the submission to retrieve the individual masks?
\n\nAssuming I get it right, let me work on the latter, unless somone has shared code to turn RLE into 2D masks already.
", "rawMarkdown": "Thanks. Just to be sure: the png shows the union of the masks your model predicted, and I need to parse the submission to retrieve the individual masks?\n\nAssuming I get it right, let me work on the latter, unless somone has shared code to turn RLE into 2D masks already.", "votes": null }, { "id": "304986", "postDate": "03/28/2018 10:18:28", "content": "this is the code to reverse RLE to 2d mask
\n\ndef run_length_decode(rle, H, W, fill_value=255):\n\nmask = np.zeros((H * W), np.uint8)\nrle = np.array([int(s) for s in rle.split(' ')]).reshape(-1, 2)\nfor r in rle:\n start = r[0]-1\n end = start + r[1]\n mask[start : end] = fill_value\nmask = mask.reshape(W, H).T # H, W need to swap as transposing.\nreturn mask\nAwesome, I'll work on it today, and submit asap! I'll use my predicted mask for the remaining images.
", "rawMarkdown": "Awesome, I'll work on it today, and submit asap! I'll use my predicted mask for the remaining images.", "votes": null }, { "id": "304989", "postDate": "03/28/2018 10:28:39", "content": "For visualization you can use https://github.com/killthekitten/kaggle-ds-bowl-2018-baseline/blob/master/test_submission_for_errors.ipynb
", "rawMarkdown": "For visualization you can use https://github.com/killthekitten/kaggle-ds-bowl-2018-baseline/blob/master/test_submission_for_errors.ipynb", "votes": null }, { "id": "305747", "postDate": "03/29/2018 10:39:47", "content": "hi Heng,\nI did post-processing on your results, but by mistake, i picked the same file for making LB submission. Can i post this link on external data thread, so that Kaggle could track the publicly available solutions.
", "rawMarkdown": "hi Heng,\nI did post-processing on your results, but by mistake, i picked the same file for making LB submission. Can i post this link on external data thread, so that Kaggle could track the publicly available solutions.", "votes": null }, { "id": "305786", "postDate": "03/29/2018 12:04:59", "content": "@Heng CherKeng, using your masks as markers improved my LB a bit to 0.465 from 0.461. I'm now exploring smarter ways to use your masks, will submit few more subs today.
\n\nBut this shows that combining both is better than each of them. And this is only for the gray images.
", "rawMarkdown": "Heng CherKeng, using your masks as markers improved my LB a bit to 0.465 from 0.461. I'm now exploring smarter ways to use your masks, will submit few more subs today. \n\nBut this shows that combining both is better than each of them. And this is only for the gray images.", "votes": null }, { "id": "305788", "postDate": "03/29/2018 12:06:49", "content": "Yes u can. Results on google drive is meant to be public
", "rawMarkdown": "Yes u can. Results on google drive is meant to be public", "votes": null }, { "id": "305791", "postDate": "03/29/2018 12:12:25", "content": "This is good news. Color image score is about 0.05 from mask rcnn without fanciful post processing. Can you briefly describe how your post processing work? I will upload the code for making the lb 0.430 csv model so that you can use it in your submission. Please wait
", "rawMarkdown": "This is good news. Color image score is about 0.05 from mask rcnn without fanciful post processing. Can you briefly describe how your post processing work? I will upload the code for making the lb 0.430 csv model so that you can use it in your submission. Please wait", "votes": null }, { "id": "305797", "postDate": "03/29/2018 12:26:37", "content": "I just tried to use the center of mass of your masks as markers, it is way worse.
\n\nMy post processing is based on this: https://www.kaggle.com/c/data-science-bowl-2018/discussion/48825#289906
\n\nI use the predictions of which pixels are part of mask to create markers, and I use the prediction of common boundaries + prediction of background to stop watershed growth. In the experiment I just did I replaced my marker by your masks. I will now see if I can combine both markers.
", "rawMarkdown": "I just tried to use the center of mass of your masks as markers, it is way worse.\n\nMy post processing is based on this: https://www.kaggle.com/c/data-science-bowl-2018/discussion/48825#289906\n\nI use the predictions of which pixels are part of mask to create markers, and I use the prediction of common boundaries + prediction of background to stop watershed growth. In the experiment I just did I replaced my marker by your masks. I will now see if I can combine both markers.", "votes": null }, { "id": "305805", "postDate": "03/29/2018 12:43:19", "content": "I use only U-Net. I believe that some top teams do so too. \nYou know, many competitors use Mask R-CNN because Allen do. But now we are becoming aware that Allen's amazing score is not because of Mask R-CNN, but preferable preprocessing and postprocessing.\nMain advantage of 2 stage approach (e.g. mask R-CNN) derives from splitting detection and classification. But in this competition, there is only one class. So we should choose 1 stage approach (e.g. U-Net, FCN) which is more simple than 2 stage approach.
", "rawMarkdown": "I use only U-Net. I believe that some top teams do so too. \nYou know, many competitors use Mask R-CNN because Allen do. But now we are becoming aware that Allen's amazing score is not because of Mask R-CNN, but preferable preprocessing and postprocessing.\nMain advantage of 2 stage approach (e.g. mask R-CNN) derives from splitting detection and classification. But in this competition, there is only one class. So we should choose 1 stage approach (e.g. U-Net, FCN) which is more simple than 2 stage approach.", "votes": null }, { "id": "305814", "postDate": "03/29/2018 12:53:56", "content": "We are in wrong direction from beginning. :)). It teached me one thing: do not follow something if you are not expert in it
", "rawMarkdown": "We are in wrong direction from beginning. :)). It teached me one thing: do not follow something if you are not expert in it", "votes": null }, { "id": "305821", "postDate": "03/29/2018 12:59:25", "content": "\n\n\nin this competition, there is only one class.
\n
That's why I started with unet indeed.
", "rawMarkdown": "> in this competition, there is only one class.\n\nThat's why I started with unet indeed.", "votes": null }, { "id": "305836", "postDate": "03/29/2018 13:43:51", "content": "@Heng CherKeng, Common boundary prediction is no longer useful when using your masks. It gets to 0.469 LB, up from my 0.461. Thanks for asking that question:
\n\n\n\n\n@CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.
\n
I'll now work on catching up with all the good ideas shared during last 3 weeks, starting with mosaic, cropping, and rescaling.
", "rawMarkdown": "Heng CherKeng, Common boundary prediction is no longer useful when using your masks. It gets to 0.469 LB, up from my 0.461. Thanks for asking that question:\n\n> @CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.\n\n I'll now work on catching up with all the good ideas shared during last 3 weeks, starting with mosaic, cropping, and rescaling.", "votes": null }, { "id": "305839", "postDate": "03/29/2018 13:49:18", "content": "Gald to know that it helps. So a good solution is treat the mask rcnn results as a mini crop unet results and do further post processing
", "rawMarkdown": "Gald to know that it helps. So a good solution is treat the mask rcnn results as a mini crop unet results and do further post processing", "votes": null }, { "id": "305923", "postDate": "03/29/2018 16:16:42", "content": "For those how are using U-Net, may I ask you guys a question?\nI've started implementing my U-Net like the original paper but then I've tried to change it to detect (background, inner cell, cell boundaries) and use the boundaries in the post processing. However, I'm not being able to train the new model by any means, it never converges and I've checked everything. I would like to know if you guys use the 3 channel U-net and if so, do you train it from the start or start using the pre-trained weights from the regular U-net?\nThanks
", "rawMarkdown": "For those how are using U-Net, may I ask you guys a question?\nI've started implementing my U-Net like the original paper but then I've tried to change it to detect (background, inner cell, cell boundaries) and use the boundaries in the post processing. However, I'm not being able to train the new model by any means, it never converges and I've checked everything. I would like to know if you guys use the 3 channel U-net and if so, do you train it from the start or start using the pre-trained weights from the regular U-net?\nThanks", "votes": null }, { "id": "306032", "postDate": "03/29/2018 19:08:25", "content": "To be honest, I'm not using original U-Net. As same as some mentioned above, I am using watershed approach ( https://arxiv.org/abs/1611.08303 ). My model is a implement of DeepLabV3+ ( https://arxiv.org/abs/1802.02611 ) and using pre-trained weights of Xception. I'm also using some other technics but I can't say yet.
", "rawMarkdown": "To be honest, I'm not using original U-Net. As same as some mentioned above, I am using watershed approach ( https://arxiv.org/abs/1611.08303 ). My model is a implement of DeepLabV3+ ( https://arxiv.org/abs/1802.02611 ) and using pre-trained weights of Xception. I'm also using some other technics but I can't say yet.", "votes": null }, { "id": "306096", "postDate": "03/29/2018 21:09:02", "content": "Using pretrained weights is on my todo list.
", "rawMarkdown": "Using pretrained weights is on my todo list.", "votes": null }, { "id": "306138", "postDate": "03/29/2018 22:54:49", "content": "You may want to use weighted cross entropy as your loss function if you are predicting (background, inner cell, cell boundaries) to deal with imbalance, other than that if you set a right learning rate and use batchnorm it should do the trick. In general I would first overfit only to a single image to see if there is a bug or not. It's similar to something I've tried where I was trying to predict not all cell boundaries but only touching walls as the 3rd class. Though I haven't start experimenting with pre and post processing I believe it's a huge boost to get a good score on LB.
", "rawMarkdown": "You may want to use weighted cross entropy as your loss function if you are predicting (background, inner cell, cell boundaries) to deal with imbalance, other than that if you set a right learning rate and use batchnorm it should do the trick. In general I would first overfit only to a single image to see if there is a bug or not. It's similar to something I've tried where I was trying to predict not all cell boundaries but only touching walls as the 3rd class. Though I haven't start experimenting with pre and post processing I believe it's a huge boost to get a good score on LB.", "votes": null }, { "id": "306196", "postDate": "03/30/2018 02:48:12", "content": "I am now trying unet and mask rcnn combined network. The semantic loss of unet is additional task in mask rcnn. This is work in progress and I haven’t submit to lb yet. More information is in my attention network thread
", "rawMarkdown": "I am now trying unet and mask rcnn combined network. The semantic loss of unet is additional task in mask rcnn. This is work in progress and I haven’t submit to lb yet. More information is in my attention network thread", "votes": null }, { "id": "306524", "postDate": "03/30/2018 15:20:24", "content": "Thanks for your help. But what about an U-net which receives cropped patches of the images? This would have a severe imbalance. Do you think that it could be successfully trained only by using a weighted cross entropy?
", "rawMarkdown": "Thanks for your help. But what about an U-net which receives cropped patches of the images? This would have a severe imbalance. Do you think that it could be successfully trained only by using a weighted cross entropy?", "votes": null }, { "id": "306791", "postDate": "03/31/2018 02:55:54", "content": "I haven't experimented on patches but you can use a dynamic loss function such is mentioned here: https://arxiv.org/pdf/1708.02002.pdf. It would be interesting to compare it against a traditional weighted cross entropy for this task.
", "rawMarkdown": "I haven't experimented on patches but you can use a dynamic loss function such is mentioned here: https://arxiv.org/pdf/1708.02002.pdf. It would be interesting to compare it against a traditional weighted cross entropy for this task.", "votes": null }, { "id": "307376", "postDate": "04/01/2018 13:31:01", "content": "Hi OsciiArt, could you tell us what is base score of using deeplabv3+ and watershed? It is interesting if it is higher than maskrcnn baseline ( 0.49)
", "rawMarkdown": "Hi OsciiArt, could you tell us what is base score of using deeplabv3+ and watershed? It is interesting if it is higher than maskrcnn baseline ( 0.49)", "votes": null }, { "id": "307445", "postDate": "04/01/2018 16:14:09", "content": "I'm not sure because I don't know the condition of mask R-CNN baseline. Score varies by processing condition (especially by scaling) but score of DeepLabV3+ and watershed is about 0.46
", "rawMarkdown": "I'm not sure because I don't know the condition of mask R-CNN baseline. Score varies by processing condition (especially by scaling) but score of DeepLabV3+ and watershed is about 0.46", "votes": null }, { "id": "307641", "postDate": "04/02/2018 06:48:59", "content": "@OsciiArt How did you learn about the deep watershed trainsform network you are using, and how did you create GT mask?
", "rawMarkdown": "OsciiArt How did you learn about the deep watershed trainsform network you are using, and how did you create GT mask?", "votes": null }, { "id": "308173", "postDate": "04/03/2018 04:21:40", "content": "Sorry, I am not sure what you mean \"how did you learn\". I use usual cross entropy and Adam. In original watershed transform, segment net, angle net and depth net are connected in series, but I connected all in parallel. Depth is calculated as distance from edge (as same as original paper). This is for simplification. I don't know it is better in score. In original paper, depth is binned depending on absolute distance. In my implementation, depth is normalized to (0,1) in each instance because this make easy to apply scaling augmentation.
", "rawMarkdown": "Sorry, I am not sure what you mean \"how did you learn\". I use usual cross entropy and Adam. In original watershed transform, segment net, angle net and depth net are connected in series, but I connected all in parallel. Depth is calculated as distance from edge (as same as original paper). This is for simplification. I don't know it is better in score. In original paper, depth is binned depending on absolute distance. In my implementation, depth is normalized to (0,1) in each instance because this make easy to apply scaling augmentation.", "votes": null }, { "id": "308951", "postDate": "04/04/2018 12:24:43", "content": "@CPMP if you post process results from mask rcnn ensemble the results would even be better. See
\n\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/53720
", "rawMarkdown": "CPMP if you post process results from mask rcnn ensemble the results would even be better. See\n\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/53720", "votes": null }, { "id": "310253", "postDate": "04/06/2018 23:20:34", "content": "Hi @OsciiArt, thanks for your help. So in the paper they generate the gt for the direction net (the first net, I think it is what you call angle net) using the normalized gradient of the distance transform. But you, on the other hand, used just a distance from edge like a scipy.ndimage.morphology.distance_transform_edt ?
Wouldn't the result of this be a single channel (distance) instead of the 2 channels gradient vector of the paper?
\n\nSorry, if it is a dumb question. I'm really new to all this.\nThanks
", "rawMarkdown": "Hi @OsciiArt, thanks for your help. So in the paper they generate the gt for the direction net (the first net, I think it is what you call angle net) using the normalized gradient of the distance transform. But you, on the other hand, used just a distance from edge like a `scipy.ndimage.morphology.distance_transform_edt` ? \n\nWouldn't the result of this be a single channel (distance) instead of the 2 channels gradient vector of the paper?\n\nSorry, if it is a dumb question. I'm really new to all this.\nThanks", "votes": null }, { "id": "310270", "postDate": "04/07/2018 01:18:26", "content": "Yes, I meant direction net. Sorry, I'm wrong. It looks like I confused you. I use depth and direction in same definition of the original paper. I use cv2.distanceTransform for depth and cv2.Sobel for direction. BTW I'm not sure direction is needed for this task. Maybe It can be removed.
", "rawMarkdown": "Yes, I meant direction net. Sorry, I'm wrong. It looks like I confused you. I use depth and direction in same definition of the original paper. I use cv2.distanceTransform for depth and cv2.Sobel for direction. BTW I'm not sure direction is needed for this task. Maybe It can be removed.", "votes": null }, { "id": "310274", "postDate": "04/07/2018 01:31:57", "content": "No worries, you helping a lot. I'm very thankful! So, you get the segmentation mask, let's say 128x128x1, and them apply a distance transform to get another 128x128x1 vector. Finally, you apply Sobel twice, one for x and another for y, to get the final 128x128x2 ground truth?\nThanks!
", "rawMarkdown": "No worries, you helping a lot. I'm very thankful! So, you get the segmentation mask, let's say 128x128x1, and them apply a distance transform to get another 128x128x1 vector. Finally, you apply Sobel twice, one for x and another for y, to get the final 128x128x2 ground truth?\nThanks!", "votes": null }, { "id": "310286", "postDate": "04/07/2018 02:27:38", "content": "Hi @OsciiArt
\n\nDo you think that the supervisory signal from depth and direction is better than just label?
\n\nI am rhinking that if my mask head of mask rcnn should learn those depth and durection as well.
", "rawMarkdown": "Hi @OsciiArt \n\n\nDo you think that the supervisory signal from depth and direction is better than just label?\n\nI am rhinking that if my mask head of mask rcnn should learn those depth and durection as well.", "votes": null }, { "id": "310335", "postDate": "04/07/2018 07:05:57", "content": "Thanks, will have a look.
", "rawMarkdown": "Thanks, will have a look.", "votes": null }, { "id": "310803", "postDate": "04/08/2018 16:37:35", "content": "Hi @Heng CherKeng\nWithout direction, there is no significant change in loss of segmentation.\nI didn't try to remove depth because depth is demand for watershed transform. \nSo, I'm not sure learning of depth refines segmentation.
", "rawMarkdown": "Hi @Heng CherKeng\nWithout direction, there is no significant change in loss of segmentation.\nI didn't try to remove depth because depth is demand for watershed transform. \nSo, I'm not sure learning of depth refines segmentation.", "votes": null }, { "id": "311393", "postDate": "04/10/2018 01:15:56", "content": "@OsciiArt are you using the VGG pretrained weights for the direction network similar to the paper?
", "rawMarkdown": "OsciiArt are you using the VGG pretrained weights for the direction network similar to the paper?", "votes": null }, { "id": "311541", "postDate": "04/10/2018 08:15:39", "content": "@Edu :) I use Xception's pretrained weights but there is no difference between pretrained and from scratch.
", "rawMarkdown": "Edu :) I use Xception's pretrained weights but there is no difference between pretrained and from scratch.", "votes": null }, { "id": "311543", "postDate": "04/10/2018 08:20:33", "content": "As the mentioned above, OsciiArt used a supper post processing (boost from 0.46 to 0.557). Too amazing his postprocessing . Do you use any external dataset like BBBC006 dataset?
", "rawMarkdown": "As the mentioned above, OsciiArt used a supper post processing (boost from 0.46 to 0.557). Too amazing his postprocessing . Do you use any external dataset like BBBC006 dataset?", "votes": null }, { "id": "311546", "postDate": "04/10/2018 08:28:30", "content": "@john1231983 No I've not try extra dataset yet. Is BBBC006 useful? As mentioned in other Topic, rescaling and intensity normalizing is crucial.
", "rawMarkdown": "john1231983 No I've not try extra dataset yet. Is BBBC006 useful? As mentioned in other Topic, rescaling and intensity normalizing is crucial.", "votes": null }, { "id": "311553", "postDate": "04/10/2018 08:36:30", "content": "Yes. It helps me improve from 0.45 to 0.47 on gray. Actually 0.47 is lucky point. I cannot reproduce it when I retrain :D. I just normalized by div by 255. Do you normalize in range [0,1], (div by 255) or (zero mean and unit variance)?
", "rawMarkdown": "Yes. It helps me improve from 0.45 to 0.47 on gray. Actually 0.47 is lucky point. I cannot reproduce it when I retrain :D. I just normalized by div by 255. Do you normalize in range [0,1], (div by 255) or (zero mean and unit variance)?", "votes": null }, { "id": "311569", "postDate": "04/10/2018 09:01:07", "content": "for train, I use this kernel's normalize function (type=1) (https://www.kaggle.com/zavodrobotov/best-public-notebook-kernel ).\nfor prediction I use modified minmax scaling. code is like this.
\n\nMIN = np.minimum(np.min(img), 255*0.25)
\nMAX = np.maximum(np.max(img), 255*0.75)
\nimg = (img - MIN) / (MAX - MIN)\n
Mask-rcnn solves the problem direclty, i.e. it produces masks for each recognized object. Unet can produce only one mask, hence it is used to predict the union of all masks, followed by a post processing to split the predicted mask into one mask per object.
\n\nUnet is way simpler than mask rcnn, but it requires way more complex post processing. There is no silver bullet here I'm afraid.
", "votes": null, "replies": [ { "id": 294199, "author_name": "ermiao", "author_url": "", "post_date": "03/11/2018 14:55:21", "content": "Hi CPMP
\n\nYour team name is \"U-net Nuke\" (it is a very interesting name:-) ) So I guess you use U-net too? If it is true I must say that your job is amazing , 0.461 Is a super high score for pure U-net model
\n\nI use U-net too and I use watershed algorithm to split the connected mask, The highest score I got in LB by my U-net model is 0.440 And I cant make any progress I try some methods but sadly the score in LB is lower and lower.... (only around 0.42).
\n\nEspecially the \"white - purple\" images like this one (17b9bf4356db24967c4677b8376ac38f826de73a88b93a8d73a8b452e399cdff.png\n) are disaster for U-net
May I ask what is your \"Nuke\" ?
", "votes": null, "replies": [] }, { "id": 294226, "author_name": "cpmpml", "author_url": "", "post_date": "03/11/2018 15:41:01", "content": "Yes, I use only Unet. And I am not using mosaic, or cropping or any data data augmentation beside image flipping. I hope this leaves some room for improvement ;) But that will have to wait until Toxic competition ends as I am busy there (each time I come back here I see my rank going down...)
\n\nYes, I am using some watershed too, so you're on the right track to beat my score! And yes, the white purple images are the toughest for me too. I guess this is where mask crnn makes adifference because in a way it looks for ellipses whereas unet can predict any mask.
\n\nNuke refers to nuclei, no secret weapon on my side ;)
", "votes": null, "replies": [] }, { "id": 294245, "author_name": "hengck23", "author_url": "", "post_date": "03/11/2018 16:19:57", "content": "@CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.
\n\nI suspect it should better than mask-rcnn for some type of cells (e.g. small or very elongated cells?)
", "votes": null, "replies": [] }, { "id": 294332, "author_name": "john1231983", "author_url": "", "post_date": "03/11/2018 18:48:05", "content": "Me too. My score goes down every day :(. I guess someone just run the shared code from Heng and submit it. As the mentioned in his post, it can achieved about 0.51 LB when you run this code without any modification. Am I right about score, Heng?
", "votes": null, "replies": [] }, { "id": 294385, "author_name": "cpmpml", "author_url": "", "post_date": "03/11/2018 21:24:41", "content": "@heng, very interesting. You are right, marker seeds are the key.
", "votes": null, "replies": [] }, { "id": 294424, "author_name": "wayne999", "author_url": "", "post_date": "03/11/2018 23:50:50", "content": "@John, interesting, which code you are using now?
", "votes": null, "replies": [] }, { "id": 294432, "author_name": "hengck23", "author_url": "", "post_date": "03/12/2018 00:20:14", "content": "@john1231983 no, you need modification to my code (like pre, post processing)
", "votes": null, "replies": [] }, { "id": 294443, "author_name": "john1231983", "author_url": "", "post_date": "03/12/2018 01:00:39", "content": "@Yan: I used matterport code in Keras. I did not family with pytorch so I choose it. The baseline of this code is 0.41~0.44 LB with coco and same learning schedule with coco: first training in the heads then train all. @Heng: Could you tell me how many LBs we can achieve if we use your published code? I guess pytorch often better than Keras if we use same idea
i think my baseline performance is about the same at 0.45. But i think the software package is not important. It is better to stick to one that you are familiar, becuase you need many changes to get to the range of LB 0.50 and above.
", "votes": null, "replies": [] }, { "id": 294630, "author_name": "cpmpml", "author_url": "", "post_date": "03/12/2018 09:52:29", "content": "@john1231983, I rather think it is the mosaic things, and various pre/post processing ideas that have been shared directly, or via shared published papers.
", "votes": null, "replies": [] }, { "id": 294741, "author_name": "hengck23", "author_url": "", "post_date": "03/12/2018 14:59:29", "content": "@CPMP Currently, mask-rcnn only predict foreground pixels for each detected nucleus. But we can modify mask-rcnn to output foreground pixel, center and boundary per detected nucleus. These information can be used for further post processing.
\n\nauxiliary information may be useful. e.g. given a cropped patch from rpn/rcnn proposals, we now predict only the bounding box and class label. We can add other predicted attributes like, nucleus size, which train image it comes from (i.e. train image id), density (nucleus per area), background/foreground color, etc ...
\n\nSuch attributes are also useful for error correction and post processing or making a level-two classifier for stacking
", "votes": null, "replies": [] }, { "id": 294750, "author_name": "cpmpml", "author_url": "", "post_date": "03/12/2018 15:10:41", "content": "\n\n\n@CPMP Currently, mask-rcnn only predict foreground pixels for each detected nucleus. But we can modify mask-rcnn to output foreground pixel, center and boundary per detected nucleus. These information can be used for further post processing.
\n
Agreed, same can be done for Unet, for the same purpose ;)
", "votes": null, "replies": [] }, { "id": 295863, "author_name": "ermiao", "author_url": "", "post_date": "03/14/2018 10:29:22", "content": "thanks for your reply ! \nMy u-net model predict the foreground pixel, center and boundary too , but I didn't use the center
\nas markers for watershed function but use the longest distance(from one pixel to the background) , It looks like the difference between the two is very small. \nYou mean I should use the boundary and center I predicted as the markers ?
That maybe a good idea. I have not (yet) tried to predict mask center with unet.
", "votes": null, "replies": [] }, { "id": 300312, "author_name": "cpmpml", "author_url": "", "post_date": "03/21/2018 14:04:32", "content": "\n\n\n@CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.
\n
@Heng I'll be back soon and will try it. However, I need a little time to digest the losss of our gold medal in toxic after our team was removed.
", "votes": null, "replies": [] }, { "id": 304961, "author_name": "cpmpml", "author_url": "", "post_date": "03/28/2018 09:38:07", "content": "@Heng CherKeng, I am ready to try it now. Which (public) output do you suggest I start with?
", "votes": null, "replies": [] }, { "id": 304964, "author_name": "hengck23", "author_url": "", "post_date": "03/28/2018 09:44:40", "content": "https://drive.google.com/drive/folders/1EtBdlPK3BVahHBZtB103QQxNUdWaJ6di
\n\nunder 20180314, there are results for LB 0.43 for the gray 53 images.\nthis is based on mask-rcnn with se-resnext backbone.\nthe csv file is also there. You can start from there and see if post-processing make results better
", "votes": null, "replies": [] }, { "id": 304983, "author_name": "cpmpml", "author_url": "", "post_date": "03/28/2018 10:16:24", "content": "Thanks. Just to be sure: the png shows the union of the masks your model predicted, and I need to parse the submission to retrieve the individual masks?
\n\nAssuming I get it right, let me work on the latter, unless somone has shared code to turn RLE into 2D masks already.
", "votes": null, "replies": [] }, { "id": 304986, "author_name": "hengck23", "author_url": "", "post_date": "03/28/2018 10:18:28", "content": "this is the code to reverse RLE to 2d mask
\n\ndef run_length_decode(rle, H, W, fill_value=255):\n\nmask = np.zeros((H * W), np.uint8)\nrle = np.array([int(s) for s in rle.split(' ')]).reshape(-1, 2)\nfor r in rle:\n start = r[0]-1\n end = start + r[1]\n mask[start : end] = fill_value\nmask = mask.reshape(W, H).T # H, W need to swap as transposing.\nreturn mask\nAwesome, I'll work on it today, and submit asap! I'll use my predicted mask for the remaining images.
", "votes": null, "replies": [] }, { "id": 304989, "author_name": "john1231983", "author_url": "", "post_date": "03/28/2018 10:28:39", "content": "For visualization you can use https://github.com/killthekitten/kaggle-ds-bowl-2018-baseline/blob/master/test_submission_for_errors.ipynb
", "votes": null, "replies": [] }, { "id": 305747, "author_name": "vikas15", "author_url": "", "post_date": "03/29/2018 10:39:47", "content": "hi Heng,\nI did post-processing on your results, but by mistake, i picked the same file for making LB submission. Can i post this link on external data thread, so that Kaggle could track the publicly available solutions.
", "votes": null, "replies": [] }, { "id": 305786, "author_name": "cpmpml", "author_url": "", "post_date": "03/29/2018 12:04:59", "content": "@Heng CherKeng, using your masks as markers improved my LB a bit to 0.465 from 0.461. I'm now exploring smarter ways to use your masks, will submit few more subs today.
\n\nBut this shows that combining both is better than each of them. And this is only for the gray images.
", "votes": null, "replies": [] }, { "id": 305788, "author_name": "hengck23", "author_url": "", "post_date": "03/29/2018 12:06:49", "content": "Yes u can. Results on google drive is meant to be public
", "votes": null, "replies": [] }, { "id": 305791, "author_name": "hengck23", "author_url": "", "post_date": "03/29/2018 12:12:25", "content": "This is good news. Color image score is about 0.05 from mask rcnn without fanciful post processing. Can you briefly describe how your post processing work? I will upload the code for making the lb 0.430 csv model so that you can use it in your submission. Please wait
", "votes": null, "replies": [] }, { "id": 305797, "author_name": "cpmpml", "author_url": "", "post_date": "03/29/2018 12:26:37", "content": "I just tried to use the center of mass of your masks as markers, it is way worse.
\n\nMy post processing is based on this: https://www.kaggle.com/c/data-science-bowl-2018/discussion/48825#289906
\n\nI use the predictions of which pixels are part of mask to create markers, and I use the prediction of common boundaries + prediction of background to stop watershed growth. In the experiment I just did I replaced my marker by your masks. I will now see if I can combine both markers.
", "votes": null, "replies": [] }, { "id": 305836, "author_name": "cpmpml", "author_url": "", "post_date": "03/29/2018 13:43:51", "content": "@Heng CherKeng, Common boundary prediction is no longer useful when using your masks. It gets to 0.469 LB, up from my 0.461. Thanks for asking that question:
\n\n\n\n\n@CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.
\n
I'll now work on catching up with all the good ideas shared during last 3 weeks, starting with mosaic, cropping, and rescaling.
", "votes": null, "replies": [] }, { "id": 305839, "author_name": "hengck23", "author_url": "", "post_date": "03/29/2018 13:49:18", "content": "Gald to know that it helps. So a good solution is treat the mask rcnn results as a mini crop unet results and do further post processing
", "votes": null, "replies": [] }, { "id": 308951, "author_name": "hengck23", "author_url": "", "post_date": "04/04/2018 12:24:43", "content": "@CPMP if you post process results from mask rcnn ensemble the results would even be better. See
\n\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/53720
", "votes": null, "replies": [] }, { "id": 310335, "author_name": "cpmpml", "author_url": "", "post_date": "04/07/2018 07:05:57", "content": "Thanks, will have a look.
", "votes": null, "replies": [] } ] }, { "id": 293951, "author_name": "yosoufe", "author_url": "", "post_date": "03/10/2018 23:20:07", "content": "", "votes": null, "replies": [] }, { "id": 295878, "author_name": "killthekitten", "author_url": "", "post_date": "03/14/2018 10:58:36", "content": "Mask R-CNN has much more hyperparams so it can end up as a big pain for you if you start the competition late. I tried both approaches and decided on U-Net eventually. You can compare for yourself rather quickly using one of the ready-to-submit solutions:
\n\nI use only U-Net. I believe that some top teams do so too. \nYou know, many competitors use Mask R-CNN because Allen do. But now we are becoming aware that Allen's amazing score is not because of Mask R-CNN, but preferable preprocessing and postprocessing.\nMain advantage of 2 stage approach (e.g. mask R-CNN) derives from splitting detection and classification. But in this competition, there is only one class. So we should choose 1 stage approach (e.g. U-Net, FCN) which is more simple than 2 stage approach.
", "votes": null, "replies": [ { "id": 305814, "author_name": "john1231983", "author_url": "", "post_date": "03/29/2018 12:53:56", "content": "We are in wrong direction from beginning. :)). It teached me one thing: do not follow something if you are not expert in it
", "votes": null, "replies": [] }, { "id": 305821, "author_name": "cpmpml", "author_url": "", "post_date": "03/29/2018 12:59:25", "content": "\n\n\nin this competition, there is only one class.
\n
That's why I started with unet indeed.
", "votes": null, "replies": [] }, { "id": 305923, "author_name": "arc144", "author_url": "", "post_date": "03/29/2018 16:16:42", "content": "For those how are using U-Net, may I ask you guys a question?\nI've started implementing my U-Net like the original paper but then I've tried to change it to detect (background, inner cell, cell boundaries) and use the boundaries in the post processing. However, I'm not being able to train the new model by any means, it never converges and I've checked everything. I would like to know if you guys use the 3 channel U-net and if so, do you train it from the start or start using the pre-trained weights from the regular U-net?\nThanks
", "votes": null, "replies": [] }, { "id": 306032, "author_name": "osciiart", "author_url": "", "post_date": "03/29/2018 19:08:25", "content": "To be honest, I'm not using original U-Net. As same as some mentioned above, I am using watershed approach ( https://arxiv.org/abs/1611.08303 ). My model is a implement of DeepLabV3+ ( https://arxiv.org/abs/1802.02611 ) and using pre-trained weights of Xception. I'm also using some other technics but I can't say yet.
", "votes": null, "replies": [] }, { "id": 306096, "author_name": "cpmpml", "author_url": "", "post_date": "03/29/2018 21:09:02", "content": "Using pretrained weights is on my todo list.
", "votes": null, "replies": [] }, { "id": 306138, "author_name": "keremt", "author_url": "", "post_date": "03/29/2018 22:54:49", "content": "You may want to use weighted cross entropy as your loss function if you are predicting (background, inner cell, cell boundaries) to deal with imbalance, other than that if you set a right learning rate and use batchnorm it should do the trick. In general I would first overfit only to a single image to see if there is a bug or not. It's similar to something I've tried where I was trying to predict not all cell boundaries but only touching walls as the 3rd class. Though I haven't start experimenting with pre and post processing I believe it's a huge boost to get a good score on LB.
", "votes": null, "replies": [] }, { "id": 306524, "author_name": "arc144", "author_url": "", "post_date": "03/30/2018 15:20:24", "content": "Thanks for your help. But what about an U-net which receives cropped patches of the images? This would have a severe imbalance. Do you think that it could be successfully trained only by using a weighted cross entropy?
", "votes": null, "replies": [] }, { "id": 306791, "author_name": "keremt", "author_url": "", "post_date": "03/31/2018 02:55:54", "content": "I haven't experimented on patches but you can use a dynamic loss function such is mentioned here: https://arxiv.org/pdf/1708.02002.pdf. It would be interesting to compare it against a traditional weighted cross entropy for this task.
", "votes": null, "replies": [] }, { "id": 307376, "author_name": "john1231983", "author_url": "", "post_date": "04/01/2018 13:31:01", "content": "Hi OsciiArt, could you tell us what is base score of using deeplabv3+ and watershed? It is interesting if it is higher than maskrcnn baseline ( 0.49)
", "votes": null, "replies": [] }, { "id": 307445, "author_name": "osciiart", "author_url": "", "post_date": "04/01/2018 16:14:09", "content": "I'm not sure because I don't know the condition of mask R-CNN baseline. Score varies by processing condition (especially by scaling) but score of DeepLabV3+ and watershed is about 0.46
", "votes": null, "replies": [] }, { "id": 307641, "author_name": "ykamikawa", "author_url": "", "post_date": "04/02/2018 06:48:59", "content": "@OsciiArt How did you learn about the deep watershed trainsform network you are using, and how did you create GT mask?
", "votes": null, "replies": [] }, { "id": 308173, "author_name": "osciiart", "author_url": "", "post_date": "04/03/2018 04:21:40", "content": "Sorry, I am not sure what you mean \"how did you learn\". I use usual cross entropy and Adam. In original watershed transform, segment net, angle net and depth net are connected in series, but I connected all in parallel. Depth is calculated as distance from edge (as same as original paper). This is for simplification. I don't know it is better in score. In original paper, depth is binned depending on absolute distance. In my implementation, depth is normalized to (0,1) in each instance because this make easy to apply scaling augmentation.
", "votes": null, "replies": [] }, { "id": 310253, "author_name": "arc144", "author_url": "", "post_date": "04/06/2018 23:20:34", "content": "Hi @OsciiArt, thanks for your help. So in the paper they generate the gt for the direction net (the first net, I think it is what you call angle net) using the normalized gradient of the distance transform. But you, on the other hand, used just a distance from edge like a scipy.ndimage.morphology.distance_transform_edt ?
Wouldn't the result of this be a single channel (distance) instead of the 2 channels gradient vector of the paper?
\n\nSorry, if it is a dumb question. I'm really new to all this.\nThanks
", "votes": null, "replies": [] }, { "id": 310270, "author_name": "osciiart", "author_url": "", "post_date": "04/07/2018 01:18:26", "content": "Yes, I meant direction net. Sorry, I'm wrong. It looks like I confused you. I use depth and direction in same definition of the original paper. I use cv2.distanceTransform for depth and cv2.Sobel for direction. BTW I'm not sure direction is needed for this task. Maybe It can be removed.
", "votes": null, "replies": [] }, { "id": 310274, "author_name": "arc144", "author_url": "", "post_date": "04/07/2018 01:31:57", "content": "No worries, you helping a lot. I'm very thankful! So, you get the segmentation mask, let's say 128x128x1, and them apply a distance transform to get another 128x128x1 vector. Finally, you apply Sobel twice, one for x and another for y, to get the final 128x128x2 ground truth?\nThanks!
", "votes": null, "replies": [] }, { "id": 310286, "author_name": "hengck23", "author_url": "", "post_date": "04/07/2018 02:27:38", "content": "Hi @OsciiArt
\n\nDo you think that the supervisory signal from depth and direction is better than just label?
\n\nI am rhinking that if my mask head of mask rcnn should learn those depth and durection as well.
", "votes": null, "replies": [] }, { "id": 310803, "author_name": "osciiart", "author_url": "", "post_date": "04/08/2018 16:37:35", "content": "Hi @Heng CherKeng\nWithout direction, there is no significant change in loss of segmentation.\nI didn't try to remove depth because depth is demand for watershed transform. \nSo, I'm not sure learning of depth refines segmentation.
", "votes": null, "replies": [] }, { "id": 311393, "author_name": "arc144", "author_url": "", "post_date": "04/10/2018 01:15:56", "content": "@OsciiArt are you using the VGG pretrained weights for the direction network similar to the paper?
", "votes": null, "replies": [] }, { "id": 311541, "author_name": "osciiart", "author_url": "", "post_date": "04/10/2018 08:15:39", "content": "@Edu :) I use Xception's pretrained weights but there is no difference between pretrained and from scratch.
", "votes": null, "replies": [] }, { "id": 311543, "author_name": "john1231983", "author_url": "", "post_date": "04/10/2018 08:20:33", "content": "As the mentioned above, OsciiArt used a supper post processing (boost from 0.46 to 0.557). Too amazing his postprocessing . Do you use any external dataset like BBBC006 dataset?
", "votes": null, "replies": [] }, { "id": 311546, "author_name": "osciiart", "author_url": "", "post_date": "04/10/2018 08:28:30", "content": "@john1231983 No I've not try extra dataset yet. Is BBBC006 useful? As mentioned in other Topic, rescaling and intensity normalizing is crucial.
", "votes": null, "replies": [] }, { "id": 311553, "author_name": "john1231983", "author_url": "", "post_date": "04/10/2018 08:36:30", "content": "Yes. It helps me improve from 0.45 to 0.47 on gray. Actually 0.47 is lucky point. I cannot reproduce it when I retrain :D. I just normalized by div by 255. Do you normalize in range [0,1], (div by 255) or (zero mean and unit variance)?
", "votes": null, "replies": [] }, { "id": 311569, "author_name": "osciiart", "author_url": "", "post_date": "04/10/2018 09:01:07", "content": "for train, I use this kernel's normalize function (type=1) (https://www.kaggle.com/zavodrobotov/best-public-notebook-kernel ).\nfor prediction I use modified minmax scaling. code is like this.
\n\nMIN = np.minimum(np.min(img), 255*0.25)
\nMAX = np.maximum(np.max(img), 255*0.75)
\nimg = (img - MIN) / (MAX - MIN)\n
I am now trying unet and mask rcnn combined network. The semantic loss of unet is additional task in mask rcnn. This is work in progress and I haven’t submit to lb yet. More information is in my attention network thread
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "293677": "Hi, I don't know either U-Net or mask R-CNN models. But i see most of the contestants are using one of these. So could someone help me to choose one of the two?\nAlso, should i be using a pre-trained model, or should i be building the model from scratch?\nThanks a lot.", "293697": "Mask-rcnn solves the problem direclty, i.e. it produces masks for each recognized object. Unet can produce only one mask, hence it is used to predict the union of all masks, followed by a post processing to split the predicted mask into one mask per object.\n\nUnet is way simpler than mask rcnn, but it requires way more complex post processing. There is no silver bullet here I'm afraid.", "293951": "", "294199": "Hi CPMP\n\nYour team name is \"U-net Nuke\" (it is a very interesting name:-) ) So I guess you use U-net too? If it is true I must say that your job is amazing , 0.461 Is a super high score for pure U-net model\n\nI use U-net too and I use watershed algorithm to split the connected mask, The highest score I got in LB by my U-net model is 0.440 And I cant make any progress I try some methods but sadly the score in LB is lower and lower.... (only around 0.42).\n\nEspecially the \"white - purple\" images like this one (`17b9bf4356db24967c4677b8376ac38f826de73a88b93a8d73a8b452e399cdff.png\n`) are disaster for U-net\n\nMay I ask what is your \"Nuke\" ?", "294226": "Yes, I use only Unet. And I am not using mosaic, or cropping or any data data augmentation beside image flipping. I hope this leaves some room for improvement ;) But that will have to wait until Toxic competition ends as I am busy there (each time I come back here I see my rank going down...)\n\nYes, I am using some watershed too, so you're on the right track to beat my score! And yes, the white purple images are the toughest for me too. I guess this is where mask crnn makes adifference because in a way it looks for ellipses whereas unet can predict any mask.\n\nNuke refers to nuclei, no secret weapon on my side ;)", "294245": "CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.\n\nI suspect it should better than mask-rcnn for some type of cells (e.g. small or very elongated cells?)", "294332": "Me too. My score goes down every day :(. I guess someone just run the shared code from Heng and submit it. As the mentioned in his post, it can achieved about 0.51 LB when you run this code without any modification. Am I right about score, Heng?", "294385": "heng, very interesting. You are right, marker seeds are the key.", "294424": "John, interesting, which code you are using now?", "294432": "john1231983 no, you need modification to my code (like pre, post processing)", "294443": "Yan: I used matterport code in Keras. I did not family with pytorch so I choose it. The baseline of this code is 0.41~0.44 LB with coco and same learning schedule with coco: first training in the `heads` then train `all`. @Heng: Could you tell me how many LBs we can achieve if we use your published code? I guess pytorch often better than Keras if we use same idea", "294459": "i think my baseline performance is about the same at 0.45. But i think the software package is not important. It is better to stick to one that you are familiar, becuase you need many changes to get to the range of LB 0.50 and above.", "294630": "john1231983, I rather think it is the mosaic things, and various pre/post processing ideas that have been shared directly, or via shared published papers.", "294741": "CPMP Currently, mask-rcnn only predict foreground pixels for each detected nucleus. But we can modify mask-rcnn to output foreground pixel, center and boundary per detected nucleus. These information can be used for further post processing.\n\nauxiliary information may be useful. e.g. given a cropped patch from rpn/rcnn proposals, we now predict only the bounding box and class label. We can add other predicted attributes like, nucleus size, which train image it comes from (i.e. train image id), density (nucleus per area), background/foreground color, etc ...\n\nSuch attributes are also useful for error correction and post processing or making a level-two classifier for stacking", "294750": "> @CPMP Currently, mask-rcnn only predict foreground pixels for each detected nucleus. But we can modify mask-rcnn to output foreground pixel, center and boundary per detected nucleus. These information can be used for further post processing.\n\nAgreed, same can be done for Unet, for the same purpose ;)", "295863": "thanks for your reply ! \nMy u-net model predict the foreground pixel, center and boundary too , but I didn't use the center \nas markers for watershed function but use the longest distance(from one pixel to the background) , It looks like the difference between the two is very small. \nYou mean I should use the boundary and center I predicted as the markers ?", "295878": "Mask R-CNN has much more hyperparams so it can end up as a big pain for you if you start the competition late. I tried both approaches and decided on U-Net eventually. You can compare for yourself rather quickly using one of the ready-to-submit solutions:\n\n 1. https://www.kaggle.com/c/data-science-bowl-2018/discussion/50903 TF matterport Mask R-CNN fork from me\n 2. https://www.kaggle.com/c/data-science-bowl-2018/discussion/49692 pytorch Mask R-CNN from Heng (might be harder to setup) \n 3. https://www.kaggle.com/keegil/keras-u-net-starter-lb-0-277 U-Net starter from Kjetil (might require some extra tuning)\n 4. https://www.kaggle.com/jakubczakon/morphological-postprocessing-on-unet-lb-0-428 morphology for the U-Net model from Jakub", "295898": "That maybe a good idea. I have not (yet) tried to predict mask center with unet.", "300312": "> @CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.\n\n@Heng I'll be back soon and will try it. However, I need a little time to digest the losss of our gold medal in toxic after our team was removed.", "304961": "Heng CherKeng, I am ready to try it now. Which (public) output do you suggest I start with?", "304964": "https://drive.google.com/drive/folders/1EtBdlPK3BVahHBZtB103QQxNUdWaJ6di\n\nunder 20180314, there are results for LB 0.43 for the gray 53 images.\nthis is based on mask-rcnn with se-resnext backbone.\nthe csv file is also there. You can start from there and see if post-processing make results better", "304983": "Thanks. Just to be sure: the png shows the union of the masks your model predicted, and I need to parse the submission to retrieve the individual masks?\n\nAssuming I get it right, let me work on the latter, unless somone has shared code to turn RLE into 2D masks already.", "304986": "this is the code to reverse RLE to 2d mask\n\n \n def run_length_decode(rle, H, W, fill_value=255):\n\n mask = np.zeros((H * W), np.uint8)\n rle = np.array([int(s) for s in rle.split(' ')]).reshape(-1, 2)\n for r in rle:\n start = r[0]-1\n end = start + r[1]\n mask[start : end] = fill_value\n mask = mask.reshape(W, H).T # H, W need to swap as transposing.\n return mask", "304987": "Awesome, I'll work on it today, and submit asap! I'll use my predicted mask for the remaining images.", "304989": "For visualization you can use https://github.com/killthekitten/kaggle-ds-bowl-2018-baseline/blob/master/test_submission_for_errors.ipynb", "305747": "hi Heng,\nI did post-processing on your results, but by mistake, i picked the same file for making LB submission. Can i post this link on external data thread, so that Kaggle could track the publicly available solutions.", "305786": "Heng CherKeng, using your masks as markers improved my LB a bit to 0.465 from 0.461. I'm now exploring smarter ways to use your masks, will submit few more subs today. \n\nBut this shows that combining both is better than each of them. And this is only for the gray images.", "305788": "Yes u can. Results on google drive is meant to be public", "305791": "This is good news. Color image score is about 0.05 from mask rcnn without fanciful post processing. Can you briefly describe how your post processing work? I will upload the code for making the lb 0.430 csv model so that you can use it in your submission. Please wait", "305797": "I just tried to use the center of mass of your masks as markers, it is way worse.\n\nMy post processing is based on this: https://www.kaggle.com/c/data-science-bowl-2018/discussion/48825#289906\n\nI use the predictions of which pixels are part of mask to create markers, and I use the prediction of common boundaries + prediction of background to stop watershed growth. In the experiment I just did I replaced my marker by your masks. I will now see if I can combine both markers.", "305805": "I use only U-Net. I believe that some top teams do so too. \nYou know, many competitors use Mask R-CNN because Allen do. But now we are becoming aware that Allen's amazing score is not because of Mask R-CNN, but preferable preprocessing and postprocessing.\nMain advantage of 2 stage approach (e.g. mask R-CNN) derives from splitting detection and classification. But in this competition, there is only one class. So we should choose 1 stage approach (e.g. U-Net, FCN) which is more simple than 2 stage approach.", "305814": "We are in wrong direction from beginning. :)). It teached me one thing: do not follow something if you are not expert in it", "305821": "> in this competition, there is only one class.\n\nThat's why I started with unet indeed.", "305836": "Heng CherKeng, Common boundary prediction is no longer useful when using your masks. It gets to 0.469 LB, up from my 0.461. Thanks for asking that question:\n\n> @CPMP let me know when you are back from Toxic competition. I wonder if you are interested to try my mask-rcnn results (via the forum) as markers (seed) for watershed, etc, and how well it will perform.\n\n I'll now work on catching up with all the good ideas shared during last 3 weeks, starting with mosaic, cropping, and rescaling.", "305839": "Gald to know that it helps. So a good solution is treat the mask rcnn results as a mini crop unet results and do further post processing", "305923": "For those how are using U-Net, may I ask you guys a question?\nI've started implementing my U-Net like the original paper but then I've tried to change it to detect (background, inner cell, cell boundaries) and use the boundaries in the post processing. However, I'm not being able to train the new model by any means, it never converges and I've checked everything. I would like to know if you guys use the 3 channel U-net and if so, do you train it from the start or start using the pre-trained weights from the regular U-net?\nThanks", "306032": "To be honest, I'm not using original U-Net. As same as some mentioned above, I am using watershed approach ( https://arxiv.org/abs/1611.08303 ). My model is a implement of DeepLabV3+ ( https://arxiv.org/abs/1802.02611 ) and using pre-trained weights of Xception. I'm also using some other technics but I can't say yet.", "306096": "Using pretrained weights is on my todo list.", "306138": "You may want to use weighted cross entropy as your loss function if you are predicting (background, inner cell, cell boundaries) to deal with imbalance, other than that if you set a right learning rate and use batchnorm it should do the trick. In general I would first overfit only to a single image to see if there is a bug or not. It's similar to something I've tried where I was trying to predict not all cell boundaries but only touching walls as the 3rd class. Though I haven't start experimenting with pre and post processing I believe it's a huge boost to get a good score on LB.", "306196": "I am now trying unet and mask rcnn combined network. The semantic loss of unet is additional task in mask rcnn. This is work in progress and I haven’t submit to lb yet. More information is in my attention network thread", "306524": "Thanks for your help. But what about an U-net which receives cropped patches of the images? This would have a severe imbalance. Do you think that it could be successfully trained only by using a weighted cross entropy?", "306791": "I haven't experimented on patches but you can use a dynamic loss function such is mentioned here: https://arxiv.org/pdf/1708.02002.pdf. It would be interesting to compare it against a traditional weighted cross entropy for this task.", "307376": "Hi OsciiArt, could you tell us what is base score of using deeplabv3+ and watershed? It is interesting if it is higher than maskrcnn baseline ( 0.49)", "307445": "I'm not sure because I don't know the condition of mask R-CNN baseline. Score varies by processing condition (especially by scaling) but score of DeepLabV3+ and watershed is about 0.46", "307641": "OsciiArt How did you learn about the deep watershed trainsform network you are using, and how did you create GT mask?", "308173": "Sorry, I am not sure what you mean \"how did you learn\". I use usual cross entropy and Adam. In original watershed transform, segment net, angle net and depth net are connected in series, but I connected all in parallel. Depth is calculated as distance from edge (as same as original paper). This is for simplification. I don't know it is better in score. In original paper, depth is binned depending on absolute distance. In my implementation, depth is normalized to (0,1) in each instance because this make easy to apply scaling augmentation.", "308951": "CPMP if you post process results from mask rcnn ensemble the results would even be better. See\n\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/53720", "310253": "Hi @OsciiArt, thanks for your help. So in the paper they generate the gt for the direction net (the first net, I think it is what you call angle net) using the normalized gradient of the distance transform. But you, on the other hand, used just a distance from edge like a `scipy.ndimage.morphology.distance_transform_edt` ? \n\nWouldn't the result of this be a single channel (distance) instead of the 2 channels gradient vector of the paper?\n\nSorry, if it is a dumb question. I'm really new to all this.\nThanks", "310270": "Yes, I meant direction net. Sorry, I'm wrong. It looks like I confused you. I use depth and direction in same definition of the original paper. I use cv2.distanceTransform for depth and cv2.Sobel for direction. BTW I'm not sure direction is needed for this task. Maybe It can be removed.", "310274": "No worries, you helping a lot. I'm very thankful! So, you get the segmentation mask, let's say 128x128x1, and them apply a distance transform to get another 128x128x1 vector. Finally, you apply Sobel twice, one for x and another for y, to get the final 128x128x2 ground truth?\nThanks!", "310286": "Hi @OsciiArt \n\n\nDo you think that the supervisory signal from depth and direction is better than just label?\n\nI am rhinking that if my mask head of mask rcnn should learn those depth and durection as well.", "310335": "Thanks, will have a look.", "310803": "Hi @Heng CherKeng\nWithout direction, there is no significant change in loss of segmentation.\nI didn't try to remove depth because depth is demand for watershed transform. \nSo, I'm not sure learning of depth refines segmentation.", "311393": "OsciiArt are you using the VGG pretrained weights for the direction network similar to the paper?", "311541": "Edu :) I use Xception's pretrained weights but there is no difference between pretrained and from scratch.", "311543": "As the mentioned above, OsciiArt used a supper post processing (boost from 0.46 to 0.557). Too amazing his postprocessing . Do you use any external dataset like BBBC006 dataset?", "311546": "john1231983 No I've not try extra dataset yet. Is BBBC006 useful? As mentioned in other Topic, rescaling and intensity normalizing is crucial.", "311553": "Yes. It helps me improve from 0.45 to 0.47 on gray. Actually 0.47 is lucky point. I cannot reproduce it when I retrain :D. I just normalized by div by 255. Do you normalize in range [0,1], (div by 255) or (zero mean and unit variance)?", "311569": "for train, I use this kernel's normalize function (type=1) (https://www.kaggle.com/zavodrobotov/best-public-notebook-kernel ).\nfor prediction I use modified minmax scaling. code is like this. \n```\nMIN = np.minimum(np.min(img), 255*0.25) \nMAX = np.maximum(np.max(img), 255*0.75) \nimg = (img - MIN) / (MAX - MIN)\n```" }, "source": "meta" }