{ "id": 50038, "title": "Segmenting Body/Edge Results", "url": "/competitions/data-science-bowl-2018/discussion/50038", "author_name": "", "post_date": "2018-02-18T20:12:41.722772200Z", "votes": 17, "comment_count": 8, "views": 0, "content": "

I've had a couple of people as me questions about how I segmented the results after predicting the body and edge regions of the nuclei, so I thought I'd put up a quick outline of my approach here. Basically, you:

\n\n\n\n

\"First

\n\n\n\n

\nfinal_labels = morphology.watershed(-ndi.distance_transform_edt(bodies), labels, mask=bodies) \n

\n\n\n\n

\"Final

\n\n

Note that I predict bodies and edges using two separate 2-class models, not a single \n3-class model.

\n\n

And alternative to the first three steps, which tends to work a bit better, but is harder to explain, \nto use `raw_bodies - k * raw_edges > threshold* for the seeds, where k is around 2.

", "messages": [ { "id": "284920", "postDate": "02/18/2018 20:12:41", "content": "

I've had a couple of people as me questions about how I segmented the results after predicting the body and edge regions of the nuclei, so I thought I'd put up a quick outline of my approach here. Basically, you:

\n\n\n\n

\"First

\n\n\n\n

\nfinal_labels = morphology.watershed(-ndi.distance_transform_edt(bodies), labels, mask=bodies) \n

\n\n\n\n

\"Final

\n\n

Note that I predict bodies and edges using two separate 2-class models, not a single \n3-class model.

\n\n

And alternative to the first three steps, which tends to work a bit better, but is harder to explain, \nto use `raw_bodies - k * raw_edges > threshold* for the seeds, where k is around 2.

", "rawMarkdown": "I've had a couple of people as me questions about how I segmented the results after predicting the body and edge regions of the nuclei, so I thought I'd put up a quick outline of my approach here. Basically, you:\n\n* Threshold the raw body results to get body regions.\n* Threshold the raw edge results to get edge regions.\n* Remove the edge regions from the body regions to get 'seed' regions.\n* label the seed regions.\n\n![First 3 labeling steps][1]\n\n* Use `skimage.morphology.waterfall` or equivalent to extend the seed regions out to fill\n the body regions. The specific incantation I use for this is: \n\n```\nfinal_labels = morphology.watershed(-ndi.distance_transform_edt(bodies), labels, mask=bodies) \n```\n\n* Discard very small regions.\n\n![Final labels][2]\n\nNote that I predict bodies and edges using two separate 2-class models, not a single \n3-class model.\n\nAnd alternative to the first three steps, which tends to work a bit better, but is harder to explain, \nto use `raw_bodies - k * raw_edges > threshold* for the seeds, where k is around 2. \n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/284920/8596/Labelling_1.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/284920/8597/Labelling_2.png", "votes": null }, { "id": "285370", "postDate": "02/20/2018 04:19:40", "content": "

Hi Tim, thanks for the great write up. I'm pursuing a similar solution and was following the DCAN paper where the bodies and edges are trained in the same model. It wasn't producing very good results for the edges. I was contemplating switching to separate models; it's good to see you had some success with it.

\n\n

One question, where do the images labeled 'raw' come from? Are they the output from your classifiers?

", "rawMarkdown": "Hi Tim, thanks for the great write up. I'm pursuing a similar solution and was following the DCAN paper where the bodies and edges are trained in the same model. It wasn't producing very good results for the edges. I was contemplating switching to separate models; it's good to see you had some success with it.\n\nOne question, where do the images labeled 'raw' come from? Are they the output from your classifiers?", "votes": null }, { "id": "285494", "postDate": "02/20/2018 12:51:05", "content": "

This is a good approach, I was using similar architecture with one multi-class model. It has given good results. Right now I'm trying something different instead of watershed algorithm (you can check my kernel). I'm planning to use features from my algorithm as inputs of smaller unet in hope for handling only nuclei separation edges.

", "rawMarkdown": "This is a good approach, I was using similar architecture with one multi-class model. It has given good results. Right now I'm trying something different instead of watershed algorithm (you can check my kernel). I'm planning to use features from my algorithm as inputs of smaller unet in hope for handling only nuclei separation edges.", "votes": null }, { "id": "285498", "postDate": "02/20/2018 12:55:11", "content": "

@emergent complexity: that's right, those are the raw (sigmoid) net outputs. I have two output channels, one for bodies and one for edges.

\n\n

Since I'm using two, 2-state outputs instead of one 3-state output, these could be generated from different nets entirely, but I'm using a single net.

", "rawMarkdown": "emergent complexity: that's right, those are the raw (sigmoid) net outputs. I have two output channels, one for bodies and one for edges. \n\nSince I'm using two, 2-state outputs instead of one 3-state output, these could be generated from different nets entirely, but I'm using a single net.", "votes": null }, { "id": "285607", "postDate": "02/20/2018 17:36:03", "content": "

Re: the raw images, that makes sense.

\n\n

I misunderstood about the two nets. So you are using one net and outputting bodies and edges similar to dcan (and what I've attempted). If you don't mind me asking, what does your loss function look like? Is it some weighted sum of the loss from both outputs? I used a flat sum, but weighted each pixel label for bodies and edges separately based on the global ratio in the respective ground truths. I haven't really had a chance to deep dive into my output, but at first blush the bodies looked reasonable but the edges were not so nice. Certainly not as nice as the raw edge image you have displayed above.

", "rawMarkdown": "Re: the raw images, that makes sense.\n\nI misunderstood about the two nets. So you are using one net and outputting bodies and edges similar to dcan (and what I've attempted). If you don't mind me asking, what does your loss function look like? Is it some weighted sum of the loss from both outputs? I used a flat sum, but weighted each pixel label for bodies and edges separately based on the global ratio in the respective ground truths. I haven't really had a chance to deep dive into my output, but at first blush the bodies looked reasonable but the edges were not so nice. Certainly not as nice as the raw edge image you have displayed above.", "votes": null }, { "id": "285611", "postDate": "02/20/2018 17:55:01", "content": "

@emergent complexity: I thought the dcan paper used a three state output and I'm using 2, 2-state outputs, so it could easily be broken into two nets. Although I may be misremembering. In any case, I tried a bunch of fancy weighting, but in the end I didn't get much out of it, so I just sum the loss functions.

\n\n

I'm using \"hard\" bootstrapped cross entropy from https://arxiv.org/pdf/1412.6596.pdf, but I don't think that actually matters much. So that wouldn't be the first, or even the third thing I'd try ;-) As long as you're already using cross entropy as the loss.

", "rawMarkdown": "emergent complexity: I thought the dcan paper used a three state output and I'm using 2, 2-state outputs, so it could easily be broken into two nets. Although I may be misremembering. In any case, I tried a bunch of fancy weighting, but in the end I didn't get much out of it, so I just sum the loss functions. \n\nI'm using \"hard\" bootstrapped cross entropy from https://arxiv.org/pdf/1412.6596.pdf, but I don't think that actually matters much. So that wouldn't be the first, or even the third thing I'd try ;-) As long as you're already using cross entropy as the loss.", "votes": null }, { "id": "285654", "postDate": "02/20/2018 19:47:23", "content": "

Yes, the dcan method from the paper pulls results from three separate points from the downsampling convnet for both objects and contours, sums the 3 results from each 'path' and then upsamples with a transposed convolution to get two sets of logits. The loss function is then summed based on each set of ground truths. At least that's how I'm reading it - the details are a little vague. The paper talks of 'discount weights' but I haven't figured out what that is yet.

\n\n

I am using cross entropy. It's helpful to know that you are getting good results using something similar. I have a couple ideas I want to try and knowing that I have the basics down allows me to jump into the more interesting ideas.

", "rawMarkdown": "Yes, the dcan method from the paper pulls results from three separate points from the downsampling convnet for both objects and contours, sums the 3 results from each 'path' and then upsamples with a transposed convolution to get two sets of logits. The loss function is then summed based on each set of ground truths. At least that's how I'm reading it - the details are a little vague. The paper talks of 'discount weights' but I haven't figured out what that is yet.\n\nI am using cross entropy. It's helpful to know that you are getting good results using something similar. I have a couple ideas I want to try and knowing that I have the basics down allows me to jump into the more interesting ideas.", "votes": null }, { "id": "293182", "postDate": "03/09/2018 10:53:07", "content": "

@Tim Hochberg\nI tried something similar but with a lot of additional morphological cleaning operations.\nYou can read about them here https://www.kaggle.com/jakubczakon/morphological-postprocessing-on-unet-lb-0-428

\n\n

I am really interested to see if they improve your results as well !

", "rawMarkdown": "Tim Hochberg\nI tried something similar but with a lot of additional morphological cleaning operations.\nYou can read about them here https://www.kaggle.com/jakubczakon/morphological-postprocessing-on-unet-lb-0-428\n\nI am really interested to see if they improve your results as well !", "votes": null }, { "id": "293793", "postDate": "03/10/2018 16:08:36", "content": "

@jakub, Thanks for the pointer. That's an impressive variety of cleaning! I can't try any of it at the moment though; I'm trying to get a different approach to work right now

", "rawMarkdown": "jakub, Thanks for the pointer. That's an impressive variety of cleaning! I can't try any of it at the moment though; I'm trying to get a different approach to work right now", "votes": null } ], "comments": [ { "id": 285370, "author_name": "jfaath", "author_url": "", "post_date": "02/20/2018 04:19:40", "content": "

Hi Tim, thanks for the great write up. I'm pursuing a similar solution and was following the DCAN paper where the bodies and edges are trained in the same model. It wasn't producing very good results for the edges. I was contemplating switching to separate models; it's good to see you had some success with it.

\n\n

One question, where do the images labeled 'raw' come from? Are they the output from your classifiers?

", "votes": null, "replies": [ { "id": 285498, "author_name": "bitsofbits", "author_url": "", "post_date": "02/20/2018 12:55:11", "content": "

@emergent complexity: that's right, those are the raw (sigmoid) net outputs. I have two output channels, one for bodies and one for edges.

\n\n

Since I'm using two, 2-state outputs instead of one 3-state output, these could be generated from different nets entirely, but I'm using a single net.

", "votes": null, "replies": [] }, { "id": 285607, "author_name": "jfaath", "author_url": "", "post_date": "02/20/2018 17:36:03", "content": "

Re: the raw images, that makes sense.

\n\n

I misunderstood about the two nets. So you are using one net and outputting bodies and edges similar to dcan (and what I've attempted). If you don't mind me asking, what does your loss function look like? Is it some weighted sum of the loss from both outputs? I used a flat sum, but weighted each pixel label for bodies and edges separately based on the global ratio in the respective ground truths. I haven't really had a chance to deep dive into my output, but at first blush the bodies looked reasonable but the edges were not so nice. Certainly not as nice as the raw edge image you have displayed above.

", "votes": null, "replies": [] }, { "id": 285611, "author_name": "bitsofbits", "author_url": "", "post_date": "02/20/2018 17:55:01", "content": "

@emergent complexity: I thought the dcan paper used a three state output and I'm using 2, 2-state outputs, so it could easily be broken into two nets. Although I may be misremembering. In any case, I tried a bunch of fancy weighting, but in the end I didn't get much out of it, so I just sum the loss functions.

\n\n

I'm using \"hard\" bootstrapped cross entropy from https://arxiv.org/pdf/1412.6596.pdf, but I don't think that actually matters much. So that wouldn't be the first, or even the third thing I'd try ;-) As long as you're already using cross entropy as the loss.

", "votes": null, "replies": [] }, { "id": 285654, "author_name": "jfaath", "author_url": "", "post_date": "02/20/2018 19:47:23", "content": "

Yes, the dcan method from the paper pulls results from three separate points from the downsampling convnet for both objects and contours, sums the 3 results from each 'path' and then upsamples with a transposed convolution to get two sets of logits. The loss function is then summed based on each set of ground truths. At least that's how I'm reading it - the details are a little vague. The paper talks of 'discount weights' but I haven't figured out what that is yet.

\n\n

I am using cross entropy. It's helpful to know that you are getting good results using something similar. I have a couple ideas I want to try and knowing that I have the basics down allows me to jump into the more interesting ideas.

", "votes": null, "replies": [] } ] }, { "id": 285494, "author_name": "bostjanm", "author_url": "", "post_date": "02/20/2018 12:51:05", "content": "

This is a good approach, I was using similar architecture with one multi-class model. It has given good results. Right now I'm trying something different instead of watershed algorithm (you can check my kernel). I'm planning to use features from my algorithm as inputs of smaller unet in hope for handling only nuclei separation edges.

", "votes": null, "replies": [] }, { "id": 293182, "author_name": "jakubczakon", "author_url": "", "post_date": "03/09/2018 10:53:07", "content": "

@Tim Hochberg\nI tried something similar but with a lot of additional morphological cleaning operations.\nYou can read about them here https://www.kaggle.com/jakubczakon/morphological-postprocessing-on-unet-lb-0-428

\n\n

I am really interested to see if they improve your results as well !

", "votes": null, "replies": [ { "id": 293793, "author_name": "bitsofbits", "author_url": "", "post_date": "03/10/2018 16:08:36", "content": "

@jakub, Thanks for the pointer. That's an impressive variety of cleaning! I can't try any of it at the moment though; I'm trying to get a different approach to work right now

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "284920": "I've had a couple of people as me questions about how I segmented the results after predicting the body and edge regions of the nuclei, so I thought I'd put up a quick outline of my approach here. Basically, you:\n\n* Threshold the raw body results to get body regions.\n* Threshold the raw edge results to get edge regions.\n* Remove the edge regions from the body regions to get 'seed' regions.\n* label the seed regions.\n\n![First 3 labeling steps][1]\n\n* Use `skimage.morphology.waterfall` or equivalent to extend the seed regions out to fill\n the body regions. The specific incantation I use for this is: \n\n```\nfinal_labels = morphology.watershed(-ndi.distance_transform_edt(bodies), labels, mask=bodies) \n```\n\n* Discard very small regions.\n\n![Final labels][2]\n\nNote that I predict bodies and edges using two separate 2-class models, not a single \n3-class model.\n\nAnd alternative to the first three steps, which tends to work a bit better, but is harder to explain, \nto use `raw_bodies - k * raw_edges > threshold* for the seeds, where k is around 2. \n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/284920/8596/Labelling_1.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/284920/8597/Labelling_2.png", "285370": "Hi Tim, thanks for the great write up. I'm pursuing a similar solution and was following the DCAN paper where the bodies and edges are trained in the same model. It wasn't producing very good results for the edges. I was contemplating switching to separate models; it's good to see you had some success with it.\n\nOne question, where do the images labeled 'raw' come from? Are they the output from your classifiers?", "285494": "This is a good approach, I was using similar architecture with one multi-class model. It has given good results. Right now I'm trying something different instead of watershed algorithm (you can check my kernel). I'm planning to use features from my algorithm as inputs of smaller unet in hope for handling only nuclei separation edges.", "285498": "emergent complexity: that's right, those are the raw (sigmoid) net outputs. I have two output channels, one for bodies and one for edges. \n\nSince I'm using two, 2-state outputs instead of one 3-state output, these could be generated from different nets entirely, but I'm using a single net.", "285607": "Re: the raw images, that makes sense.\n\nI misunderstood about the two nets. So you are using one net and outputting bodies and edges similar to dcan (and what I've attempted). If you don't mind me asking, what does your loss function look like? Is it some weighted sum of the loss from both outputs? I used a flat sum, but weighted each pixel label for bodies and edges separately based on the global ratio in the respective ground truths. I haven't really had a chance to deep dive into my output, but at first blush the bodies looked reasonable but the edges were not so nice. Certainly not as nice as the raw edge image you have displayed above.", "285611": "emergent complexity: I thought the dcan paper used a three state output and I'm using 2, 2-state outputs, so it could easily be broken into two nets. Although I may be misremembering. In any case, I tried a bunch of fancy weighting, but in the end I didn't get much out of it, so I just sum the loss functions. \n\nI'm using \"hard\" bootstrapped cross entropy from https://arxiv.org/pdf/1412.6596.pdf, but I don't think that actually matters much. So that wouldn't be the first, or even the third thing I'd try ;-) As long as you're already using cross entropy as the loss.", "285654": "Yes, the dcan method from the paper pulls results from three separate points from the downsampling convnet for both objects and contours, sums the 3 results from each 'path' and then upsamples with a transposed convolution to get two sets of logits. The loss function is then summed based on each set of ground truths. At least that's how I'm reading it - the details are a little vague. The paper talks of 'discount weights' but I haven't figured out what that is yet.\n\nI am using cross entropy. It's helpful to know that you are getting good results using something similar. I have a couple ideas I want to try and knowing that I have the basics down allows me to jump into the more interesting ideas.", "293182": "Tim Hochberg\nI tried something similar but with a lot of additional morphological cleaning operations.\nYou can read about them here https://www.kaggle.com/jakubczakon/morphological-postprocessing-on-unet-lb-0-428\n\nI am really interested to see if they improve your results as well !", "293793": "jakub, Thanks for the pointer. That's an impressive variety of cleaning! I can't try any of it at the moment though; I'm trying to get a different approach to work right now" }, "source": "meta" }