{ "id": 54353, "title": "post your stage2 test results here", "url": "/competitions/data-science-bowl-2018/discussion/54353", "author_name": "", "post_date": "2018-04-12T12:40:49.818080100Z", "votes": 6, "comment_count": 33, "views": 0, "content": "

I did not finetune with the stage1 test labels ... though i think it might help ..

\n\n

I use mask-rcnn . I would like to see how unet perform.

\n\n

Here are some few stage2 images that work

\n\n

\"enter

\n\n

\"enter

", "messages": [ { "id": "312836", "postDate": "04/12/2018 12:40:49", "content": "

I did not finetune with the stage1 test labels ... though i think it might help ..

\n\n

I use mask-rcnn . I would like to see how unet perform.

\n\n

Here are some few stage2 images that work

\n\n

\"enter

\n\n

\"enter

", "rawMarkdown": "I did not finetune with the stage1 test labels ... though i think it might help ..\n\nI use mask-rcnn . I would like to see how unet perform.\n\nHere are some few stage2 images that work\n\n ![enter image description here][1]\n\n ![enter image description here][2]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312836/9116/09d5c0a305af734e600a4ef547580c713cede403c6ae79424c12e57739417bf9.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312836/9123/36dac2ace4f3c79b683d0bad0c2de03b9b01f762f7f4df0e7f009d9fcc599972.png", "votes": null }, { "id": "312837", "postDate": "04/12/2018 12:41:02", "content": "

here are those that \"does not work at all\"

\n\n

\"enter

\n\n

\"enter

", "rawMarkdown": "here are those that \"does not work at all\"\n\n\n ![enter image description here][1]\n\n ![enter image description here][2]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312837/9117/9fca513009e13f413c1dc322325f9f21c9b5b926df50ef348db7ad2e02e13aca.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312837/9122/22f0fc6fcff08d806138e400c3320ad8b4ff3ddb12c4adacab5b83fcd46d32a0.png", "votes": null }, { "id": "312838", "postDate": "04/12/2018 12:41:21", "content": "

here are those that \"i don't know work or not\"

\n\n

\"enter

", "rawMarkdown": "here are those that \"i don't know work or not\"\n\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312838/9121/21e6d64a92ae99477f21b9b512c35dd1d3c390d0867dbb287f406f91aedfe0dd.png", "votes": null }, { "id": "312843", "postDate": "04/12/2018 12:49:09", "content": "

here are some that \"work a little\"

\n\n

\"enter

\n\n

\"enter

", "rawMarkdown": "here are some that \"work a little\"\n\n ![enter image description here][1]\n\n\n ![enter image description here][2]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312843/9111/9b03511ba5dbeaba579a12621ec7a953e8eb5283c4c531a71bd09a5fdf55fd1c.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312843/9119/20d0d2ac8ecc95216d056b46163466a18b7f74c921e9f2b5e46e538a58de65e6.png", "votes": null }, { "id": "312849", "postDate": "04/12/2018 12:58:35", "content": "

if we assume that the private LB will choose equal images per cell type, I estimate my private score could be in the range of 0.3 or less

", "rawMarkdown": "if we assume that the private LB will choose equal images per cell type, I estimate my private score could be in the range of 0.3 or less", "votes": null }, { "id": "312885", "postDate": "04/12/2018 13:55:32", "content": "

Heng,\nDo you think detecting nuclei for 9ac881c9dc02cb901f16f287a0a514e733c751710e9c797f750b9aa1d82057a0 is the correct answer? Not sure whether the image was prepared with valid method that biologist use to get useful information or was generated to trigger false positive. There is also an image that is hand-drawn. I'm not sure a model that detects a nuclei on it is better than a model that does not detect any nucleus.

\n\n

Though it is interesting to see neural net model detecting nuclei in these images, it seems to me that a model that can filter them out as not useful seem more useful to biologists.

", "rawMarkdown": "Heng,\nDo you think detecting nuclei for 9ac881c9dc02cb901f16f287a0a514e733c751710e9c797f750b9aa1d82057a0 is the correct answer? Not sure whether the image was prepared with valid method that biologist use to get useful information or was generated to trigger false positive. There is also an image that is hand-drawn. I'm not sure a model that detects a nuclei on it is better than a model that does not detect any nucleus.\n\nThough it is interesting to see neural net model detecting nuclei in these images, it seems to me that a model that can filter them out as not useful seem more useful to biologists.", "votes": null }, { "id": "312893", "postDate": "04/12/2018 14:04:31", "content": "

i think this:\nhttp://mcb.asm.org/content/21/1/281/F3.expansion.html

\n\n

\"enter

\n\n

\"enter

", "rawMarkdown": "i think this:\nhttp://mcb.asm.org/content/21/1/281/F3.expansion.html\n\n ![enter image description here][1]\n\n ![enter image description here][2]\n\n\n [1]: http://www.mdpi.com/cancers/cancers-10-00025/article_deploy/html/images/cancers-10-00025-g002.png\n [2]: http://mcb.asm.org/content/21/1/281/F3.large.jpg", "votes": null }, { "id": "312908", "postDate": "04/12/2018 14:19:50", "content": "

After looking as your results, I think it’ll do better. Have you submitted?

", "rawMarkdown": "After looking as your results, I think it’ll do better. Have you submitted?", "votes": null }, { "id": "312912", "postDate": "04/12/2018 14:23:17", "content": "

not yet. 3000 images x 12 ensembles will take maybe 2 days

", "rawMarkdown": "not yet. 3000 images x 12 ensembles will take maybe 2 days", "votes": null }, { "id": "312957", "postDate": "04/12/2018 15:22:01", "content": "

Yeah, it is a nucleus. Same as 21da1704bd63963b3c6bc47480786f7d4c1735ae4bbfbb63a22ce376e036f403

", "rawMarkdown": "Yeah, it is a nucleus. Same as 21da1704bd63963b3c6bc47480786f7d4c1735ae4bbfbb63a22ce376e036f403", "votes": null }, { "id": "312960", "postDate": "04/12/2018 15:25:39", "content": "

https://youtu.be/oKxsW8DKJQQ?t=1m14s

\n\n

Somewhere, after seeing the phase two images, @Heng mutters to himself “I’m going to need a bigger ensemble.” 😂

", "rawMarkdown": "https://youtu.be/oKxsW8DKJQQ?t=1m14s\n\nSomewhere, after seeing the phase two images, @Heng mutters to himself “I’m going to need a bigger ensemble.” 😂", "votes": null }, { "id": "312963", "postDate": "04/12/2018 15:28:09", "content": "

@Allen \"Heng mutters to himself “I’m going to need a bigger ensemble\" , you predicted it when i am preparing for my next post.

\n\n

\"enter

", "rawMarkdown": "Allen \"Heng mutters to himself “I’m going to need a bigger ensemble\" , you predicted it when i am preparing for my next post.\n\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312963/9127/draft.png", "votes": null }, { "id": "312994", "postDate": "04/12/2018 16:37:27", "content": "

Ok.

\n\n

I'd still argue, though, that everything else being equal, a model that does not detect any nuclei in hand drawn images are actually better.

", "rawMarkdown": "Ok.\n\nI'd still argue, though, that everything else being equal, a model that does not detect any nuclei in hand drawn images are actually better.", "votes": null }, { "id": "313001", "postDate": "04/12/2018 16:45:46", "content": "

I guess you could argue that if you were analysing a pathologists notes for sketches of irregular nuclei (that could be directly related back to a linked patient's sample), detecting them might be useful. However, you might as well just take a snap under the scope, as the image the drawing is based on has obviously undergone H&E staining for microscopy.

", "rawMarkdown": "I guess you could argue that if you were analysing a pathologists notes for sketches of irregular nuclei (that could be directly related back to a linked patient's sample), detecting them might be useful. However, you might as well just take a snap under the scope, as the image the drawing is based on has obviously undergone H&E staining for microscopy.", "votes": null }, { "id": "313064", "postDate": "04/12/2018 18:29:45", "content": "

Heng, your bottom left image here is a strange case. So, you are actually detecting entire cells and not nuclei, but the \"rounded up\" cells are appearing this way. From a biological perspective I have NO IDEA how these were annotated. If I were to try to manually segment this data I would say that I am only able to segment cells, and no nuclei can be detected. Your algorithm may actually be working perfectly as I might decide that the rounded up cells are mostly nuclei and only circle those :)

", "rawMarkdown": "Heng, your bottom left image here is a strange case. So, you are actually detecting entire cells and not nuclei, but the \"rounded up\" cells are appearing this way. From a biological perspective I have NO IDEA how these were annotated. If I were to try to manually segment this data I would say that I am only able to segment cells, and no nuclei can be detected. Your algorithm may actually be working perfectly as I might decide that the rounded up cells are mostly nuclei and only circle those :)", "votes": null }, { "id": "313149", "postDate": "04/12/2018 20:50:57", "content": "

interesting results, I actually have no idea what i am doing when look at some seg results :)

", "rawMarkdown": "interesting results, I actually have no idea what i am doing when look at some seg results :)", "votes": null }, { "id": "313188", "postDate": "04/12/2018 21:53:44", "content": "

I think you should have put a blur before your nucleus finder (probably as a parallel pipeline to the un-blurred one, ensembled).

", "rawMarkdown": "I think you should have put a blur before your nucleus finder (probably as a parallel pipeline to the un-blurred one, ensembled).", "votes": null }, { "id": "313254", "postDate": "04/13/2018 02:14:09", "content": "

hi Heng, for 9c1ad, 9c7eb, 09d5c, do you think the smaller brighter dots wrapped in the ellipse are nucleus, and current masks are on the cells ?

", "rawMarkdown": "hi Heng, for 9c1ad, 9c7eb, 09d5c, do you think the smaller brighter dots wrapped in the ellipse are nucleus, and current masks are on the cells ?", "votes": null }, { "id": "313283", "postDate": "04/13/2018 03:41:17", "content": "

@Heng, as requested here are some results. These are results for the same images you tested that are produced by a model that is neither a U-net nor R-CNN exactly. It's topology is more or less that of a U-net, but it predicts (foreground, relative-center-x, relative-center-y, area) at each pixel. Then it clusters the points classified as foreground on predicted center- x, center-y and area using DBSCAN.

\n\n

It completely falls down on the speckled images, but I think that's a data augmentation failure on my part. It fails on a couple others pretty badly, but mostly does OK.

", "rawMarkdown": "Heng, as requested here are some results. These are results for the same images you tested that are produced by a model that is neither a U-net nor R-CNN exactly. It's topology is more or less that of a U-net, but it predicts (foreground, relative-center-x, relative-center-y, area) at each pixel. Then it clusters the points classified as foreground on predicted center- x, center-y and area using DBSCAN.\n\nIt completely falls down on the speckled images, but I think that's a data augmentation failure on my part. It fails on a couple others pretty badly, but mostly does OK.", "votes": null }, { "id": "313298", "postDate": "04/13/2018 04:25:33", "content": "

Hi, this is my results on stage2 testing set. Note that, no postprocessing, no ensembling are applied\n\"enter\n\"enter\n\"enter\n\"enter\n\"enter\n\"enter\n\"enter

", "rawMarkdown": "Hi, this is my results on stage2 testing set. Note that, no postprocessing, no ensembling are applied\n![enter image description here][1]\n![enter image description here][2]\n![enter image description here][3]\n![enter image description here][4]\n![enter image description here][5]\n![enter image description here][6]\n![enter image description here][7]\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9149/9ac881c9dc02cb901f16f287a0a514e733c751710e9c797f750b9aa1d82057a0.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9150/36dac2ace4f3c79b683d0bad0c2de03b9b01f762f7f4df0e7f009d9fcc599972.png\n [3]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9151/9b889d736e06f0182622661f54f72e5c90e3480049940523262745a41c89bbc5.png\n [4]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9152/09d5c0a305af734e600a4ef547580c713cede403c6ae79424c12e57739417bf9.png\n [5]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9153/9c1ad3fcb45d89072864a18608a0778468e7580b2b8207ae1cf5d096d380368a.png\n [6]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9154/9c7ebccedebb7e33a8ab8cef3c031d7845deb7c86064f81710691c7e64611b81.png\n [7]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9165/21e6d64a92ae99477f21b9b512c35dd1d3c390d0867dbb287f406f91aedfe0dd.png", "votes": null }, { "id": "313314", "postDate": "04/13/2018 05:12:21", "content": "

i show the intermediate results of ensemble of 4 mask rcnns. the final submission should have 12 mask rcnns, but inference of the other 8 are still on going.

\n\n

Ensemble is clustering and removing false detection. Final results is average of clustered predictions. From left to right

\n\n\n\n

final results = cluster - post filtering (too small mask and those at the border of images are removed)

\n\n

\"enter

\n\n

\"enter

", "rawMarkdown": "i show the intermediate results of ensemble of 4 mask rcnns. the final submission should have 12 mask rcnns, but inference of the other 8 are still on going.\n\nEnsemble is clustering and removing false detection. Final results is average of clustered predictions. From left to right\n\n- input image\n\n- final results (color overlay)\n\n- final results (contour overlay)\n\n- cluster(contour overlay)\n\n- cluster(color overlay)\n\n- rejected instances (from ensemble of mask rcnn predictions , flip scale)\n\n- sum of all instances (accepted and rejected)\n\nfinal results = cluster - post filtering (too small mask and those at the border of images are removed)\n\n ![enter image description here][1]\n\n ![enter image description here][2]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313314/9155/b083e597b2cd3f7fe8f8ae24b0e38f569314b36a4c7c10442a2e5225d757e1af.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313314/9160/dd58e56930c29ad238a73471303c1644c592e086d0d3a24fc7f56164b5e24b02.png", "votes": null }, { "id": "313350", "postDate": "04/13/2018 06:19:08", "content": "

Hi All, I just used the models from stage 1 without using test1 data to predict. I only got outputs from 2600-2900 images out of the total 3019 images. Does this sound right to you?

", "rawMarkdown": "Hi All, I just used the models from stage 1 without using test1 data to predict. I only got outputs from 2600-2900 images out of the total 3019 images. Does this sound right to you?", "votes": null }, { "id": "313641", "postDate": "04/13/2018 16:05:00", "content": "

I cannot WAIT to see his LB results post!

", "rawMarkdown": "I cannot WAIT to see his LB results post!", "votes": null }, { "id": "313721", "postDate": "04/13/2018 18:49:18", "content": "

Here are some of mine. Some good, some unsure, some epically bad. On the now infamous textbook scan (aka Allen's art class progeny), my model did better than I expected

", "rawMarkdown": "Here are some of mine. Some good, some unsure, some epically bad. On the now infamous textbook scan (aka Allen's art class progeny), my model did better than I expected", "votes": null }, { "id": "313726", "postDate": "04/13/2018 19:01:06", "content": "

there are some images without detection. i haven't count it in detail. for gray stage2 test images, i have 2557 with detections out of 2800 input images for the normal model. other test augmented images varies in this number.

", "rawMarkdown": "there are some images without detection. i haven't count it in detail. for gray stage2 test images, i have 2557 with detections out of 2800 input images for the normal model. other test augmented images varies in this number.", "votes": null }, { "id": "313824", "postDate": "04/13/2018 22:14:10", "content": "

I'm not Heng, but I think his masks are on the nuclei, and the stuff outside is the rest of the cells, neurons specifically.

", "rawMarkdown": "I'm not Heng, but I think his masks are on the nuclei, and the stuff outside is the rest of the cells, neurons specifically.", "votes": null }, { "id": "313884", "postDate": "04/14/2018 02:02:24", "content": "

hi Ian, thanks for your comment! I meant the stuff inside the masks, thought they were nucleus yesterday. But after I asked someone in hospital, he think the mask is right and the smaller white things maybe DNA or something.

", "rawMarkdown": "hi Ian, thanks for your comment! I meant the stuff inside the masks, thought they were nucleus yesterday. But after I asked someone in hospital, he think the mask is right and the smaller white things maybe DNA or something.", "votes": null }, { "id": "313893", "postDate": "04/14/2018 02:52:29", "content": "

interesting stage2 test images:\nbeee9cf93c88f1c4f892460101b6d64931ec64c2a72d91aad8c8c1306115bbbc

\n\n

not sure if this will be used in the private LB score ...

", "rawMarkdown": "interesting stage2 test images:\nbeee9cf93c88f1c4f892460101b6d64931ec64c2a72d91aad8c8c1306115bbbc\n\nnot sure if this will be used in the private LB score ...", "votes": null }, { "id": "314393", "postDate": "04/15/2018 13:34:35", "content": "

Hi, Heng, how could you get your stage 2 score, I submit my csv on public leaderboard but the display results are LB 0.

", "rawMarkdown": "Hi, Heng, how could you get your stage 2 score, I submit my csv on public leaderboard but the display results are LB 0.", "votes": null }, { "id": "315206", "postDate": "04/16/2018 21:37:28", "content": "

Here is one of the cases, the algorithm fails on: for some cases it seems to learn the black background as cell.

", "rawMarkdown": "Here is one of the cases, the algorithm fails on: for some cases it seems to learn the black background as cell.", "votes": null }, { "id": "315261", "postDate": "04/17/2018 00:12:43", "content": "

our stage2 submission (private LB 0.574) is here. you can see results for all the images

\n\n

https://drive.google.com/open?id=1QeMFRfiW95rudGLwKYNy-weLpP-5L6-S

", "rawMarkdown": "our stage2 submission (private LB 0.574) is here. you can see results for all the images\n\nhttps://drive.google.com/open?id=1QeMFRfiW95rudGLwKYNy-weLpP-5L6-S", "votes": null }, { "id": "315263", "postDate": "04/17/2018 00:13:31", "content": "

Nice work!

", "rawMarkdown": "Nice work!", "votes": null }, { "id": "315270", "postDate": "04/17/2018 00:33:05", "content": "

@Allen thanks! It seems that the in-house baseline results of 0.4x on stage2 has been surpassed? Or are they evaluated on different dataset?

", "rawMarkdown": "Allen thanks! It seems that the in-house baseline results of 0.4x on stage2 has been surpassed? Or are they evaluated on different dataset?", "votes": null }, { "id": "315289", "postDate": "04/17/2018 00:48:43", "content": "

They were evaluated different and I had a few models working. Nevertheless, my best performing model on phase two, 0.582, was easily surpassed! I’m thrilled. We still want to test a few other models we regularly use in the lab (one from a forthcoming fully convolutional paper and another from CellProfiler).

", "rawMarkdown": "They were evaluated different and I had a few models working. Nevertheless, my best performing model on phase two, 0.582, was easily surpassed! I’m thrilled. We still want to test a few other models we regularly use in the lab (one from a forthcoming fully convolutional paper and another from CellProfiler).", "votes": null } ], "comments": [ { "id": 312837, "author_name": "hengck23", "author_url": "", "post_date": "04/12/2018 12:41:02", "content": "

here are those that \"does not work at all\"

\n\n

\"enter

\n\n

\"enter

", "votes": null, "replies": [] }, { "id": 312838, "author_name": "hengck23", "author_url": "", "post_date": "04/12/2018 12:41:21", "content": "

here are those that \"i don't know work or not\"

\n\n

\"enter

", "votes": null, "replies": [] }, { "id": 312843, "author_name": "hengck23", "author_url": "", "post_date": "04/12/2018 12:49:09", "content": "

here are some that \"work a little\"

\n\n

\"enter

\n\n

\"enter

", "votes": null, "replies": [ { "id": 313064, "author_name": "bleuknight", "author_url": "", "post_date": "04/12/2018 18:29:45", "content": "

Heng, your bottom left image here is a strange case. So, you are actually detecting entire cells and not nuclei, but the \"rounded up\" cells are appearing this way. From a biological perspective I have NO IDEA how these were annotated. If I were to try to manually segment this data I would say that I am only able to segment cells, and no nuclei can be detected. Your algorithm may actually be working perfectly as I might decide that the rounded up cells are mostly nuclei and only circle those :)

", "votes": null, "replies": [] } ] }, { "id": 312849, "author_name": "hengck23", "author_url": "", "post_date": "04/12/2018 12:58:35", "content": "

if we assume that the private LB will choose equal images per cell type, I estimate my private score could be in the range of 0.3 or less

", "votes": null, "replies": [ { "id": 312908, "author_name": "agoodman", "author_url": "", "post_date": "04/12/2018 14:19:50", "content": "

After looking as your results, I think it’ll do better. Have you submitted?

", "votes": null, "replies": [] }, { "id": 312912, "author_name": "hengck23", "author_url": "", "post_date": "04/12/2018 14:23:17", "content": "

not yet. 3000 images x 12 ensembles will take maybe 2 days

", "votes": null, "replies": [] }, { "id": 312960, "author_name": "agoodman", "author_url": "", "post_date": "04/12/2018 15:25:39", "content": "

https://youtu.be/oKxsW8DKJQQ?t=1m14s

\n\n

Somewhere, after seeing the phase two images, @Heng mutters to himself “I’m going to need a bigger ensemble.” 😂

", "votes": null, "replies": [] }, { "id": 313641, "author_name": "drannecarpenter", "author_url": "", "post_date": "04/13/2018 16:05:00", "content": "

I cannot WAIT to see his LB results post!

", "votes": null, "replies": [] } ] }, { "id": 312885, "author_name": "bsp2020", "author_url": "", "post_date": "04/12/2018 13:55:32", "content": "

Heng,\nDo you think detecting nuclei for 9ac881c9dc02cb901f16f287a0a514e733c751710e9c797f750b9aa1d82057a0 is the correct answer? Not sure whether the image was prepared with valid method that biologist use to get useful information or was generated to trigger false positive. There is also an image that is hand-drawn. I'm not sure a model that detects a nuclei on it is better than a model that does not detect any nucleus.

\n\n

Though it is interesting to see neural net model detecting nuclei in these images, it seems to me that a model that can filter them out as not useful seem more useful to biologists.

", "votes": null, "replies": [ { "id": 312893, "author_name": "hengck23", "author_url": "", "post_date": "04/12/2018 14:04:31", "content": "

i think this:\nhttp://mcb.asm.org/content/21/1/281/F3.expansion.html

\n\n

\"enter

\n\n

\"enter

", "votes": null, "replies": [] }, { "id": 312957, "author_name": "dstjhb", "author_url": "", "post_date": "04/12/2018 15:22:01", "content": "

Yeah, it is a nucleus. Same as 21da1704bd63963b3c6bc47480786f7d4c1735ae4bbfbb63a22ce376e036f403

", "votes": null, "replies": [] }, { "id": 312994, "author_name": "bsp2020", "author_url": "", "post_date": "04/12/2018 16:37:27", "content": "

Ok.

\n\n

I'd still argue, though, that everything else being equal, a model that does not detect any nuclei in hand drawn images are actually better.

", "votes": null, "replies": [] }, { "id": 313001, "author_name": "dstjhb", "author_url": "", "post_date": "04/12/2018 16:45:46", "content": "

I guess you could argue that if you were analysing a pathologists notes for sketches of irregular nuclei (that could be directly related back to a linked patient's sample), detecting them might be useful. However, you might as well just take a snap under the scope, as the image the drawing is based on has obviously undergone H&E staining for microscopy.

", "votes": null, "replies": [] } ] }, { "id": 312963, "author_name": "hengck23", "author_url": "", "post_date": "04/12/2018 15:28:09", "content": "

@Allen \"Heng mutters to himself “I’m going to need a bigger ensemble\" , you predicted it when i am preparing for my next post.

\n\n

\"enter

", "votes": null, "replies": [] }, { "id": 313149, "author_name": "wayne999", "author_url": "", "post_date": "04/12/2018 20:50:57", "content": "

interesting results, I actually have no idea what i am doing when look at some seg results :)

", "votes": null, "replies": [ { "id": 313188, "author_name": "pointyointment", "author_url": "", "post_date": "04/12/2018 21:53:44", "content": "

I think you should have put a blur before your nucleus finder (probably as a parallel pipeline to the un-blurred one, ensembled).

", "votes": null, "replies": [] } ] }, { "id": 313254, "author_name": "cranecooker", "author_url": "", "post_date": "04/13/2018 02:14:09", "content": "

hi Heng, for 9c1ad, 9c7eb, 09d5c, do you think the smaller brighter dots wrapped in the ellipse are nucleus, and current masks are on the cells ?

", "votes": null, "replies": [ { "id": 313824, "author_name": "pointyointment", "author_url": "", "post_date": "04/13/2018 22:14:10", "content": "

I'm not Heng, but I think his masks are on the nuclei, and the stuff outside is the rest of the cells, neurons specifically.

", "votes": null, "replies": [] }, { "id": 313884, "author_name": "cranecooker", "author_url": "", "post_date": "04/14/2018 02:02:24", "content": "

hi Ian, thanks for your comment! I meant the stuff inside the masks, thought they were nucleus yesterday. But after I asked someone in hospital, he think the mask is right and the smaller white things maybe DNA or something.

", "votes": null, "replies": [] } ] }, { "id": 313283, "author_name": "bitsofbits", "author_url": "", "post_date": "04/13/2018 03:41:17", "content": "

@Heng, as requested here are some results. These are results for the same images you tested that are produced by a model that is neither a U-net nor R-CNN exactly. It's topology is more or less that of a U-net, but it predicts (foreground, relative-center-x, relative-center-y, area) at each pixel. Then it clusters the points classified as foreground on predicted center- x, center-y and area using DBSCAN.

\n\n

It completely falls down on the speckled images, but I think that's a data augmentation failure on my part. It fails on a couple others pretty badly, but mostly does OK.

", "votes": null, "replies": [] }, { "id": 313298, "author_name": "msl23518", "author_url": "", "post_date": "04/13/2018 04:25:33", "content": "

Hi, this is my results on stage2 testing set. Note that, no postprocessing, no ensembling are applied\n\"enter\n\"enter\n\"enter\n\"enter\n\"enter\n\"enter\n\"enter

", "votes": null, "replies": [] }, { "id": 313314, "author_name": "hengck23", "author_url": "", "post_date": "04/13/2018 05:12:21", "content": "

i show the intermediate results of ensemble of 4 mask rcnns. the final submission should have 12 mask rcnns, but inference of the other 8 are still on going.

\n\n

Ensemble is clustering and removing false detection. Final results is average of clustered predictions. From left to right

\n\n\n\n

final results = cluster - post filtering (too small mask and those at the border of images are removed)

\n\n

\"enter

\n\n

\"enter

", "votes": null, "replies": [] }, { "id": 313350, "author_name": "ybwu01", "author_url": "", "post_date": "04/13/2018 06:19:08", "content": "

Hi All, I just used the models from stage 1 without using test1 data to predict. I only got outputs from 2600-2900 images out of the total 3019 images. Does this sound right to you?

", "votes": null, "replies": [ { "id": 313726, "author_name": "hengck23", "author_url": "", "post_date": "04/13/2018 19:01:06", "content": "

there are some images without detection. i haven't count it in detail. for gray stage2 test images, i have 2557 with detections out of 2800 input images for the normal model. other test augmented images varies in this number.

", "votes": null, "replies": [] } ] }, { "id": 313721, "author_name": "jbfarrar", "author_url": "", "post_date": "04/13/2018 18:49:18", "content": "

Here are some of mine. Some good, some unsure, some epically bad. On the now infamous textbook scan (aka Allen's art class progeny), my model did better than I expected

", "votes": null, "replies": [] }, { "id": 313893, "author_name": "hengck23", "author_url": "", "post_date": "04/14/2018 02:52:29", "content": "

interesting stage2 test images:\nbeee9cf93c88f1c4f892460101b6d64931ec64c2a72d91aad8c8c1306115bbbc

\n\n

not sure if this will be used in the private LB score ...

", "votes": null, "replies": [] }, { "id": 314393, "author_name": "paopao2018", "author_url": "", "post_date": "04/15/2018 13:34:35", "content": "

Hi, Heng, how could you get your stage 2 score, I submit my csv on public leaderboard but the display results are LB 0.

", "votes": null, "replies": [] }, { "id": 315206, "author_name": "mpsampat", "author_url": "", "post_date": "04/16/2018 21:37:28", "content": "

Here is one of the cases, the algorithm fails on: for some cases it seems to learn the black background as cell.

", "votes": null, "replies": [] }, { "id": 315261, "author_name": "hengck23", "author_url": "", "post_date": "04/17/2018 00:12:43", "content": "

our stage2 submission (private LB 0.574) is here. you can see results for all the images

\n\n

https://drive.google.com/open?id=1QeMFRfiW95rudGLwKYNy-weLpP-5L6-S

", "votes": null, "replies": [ { "id": 315263, "author_name": "agoodman", "author_url": "", "post_date": "04/17/2018 00:13:31", "content": "

Nice work!

", "votes": null, "replies": [] }, { "id": 315270, "author_name": "hengck23", "author_url": "", "post_date": "04/17/2018 00:33:05", "content": "

@Allen thanks! It seems that the in-house baseline results of 0.4x on stage2 has been surpassed? Or are they evaluated on different dataset?

", "votes": null, "replies": [] }, { "id": 315289, "author_name": "agoodman", "author_url": "", "post_date": "04/17/2018 00:48:43", "content": "

They were evaluated different and I had a few models working. Nevertheless, my best performing model on phase two, 0.582, was easily surpassed! I’m thrilled. We still want to test a few other models we regularly use in the lab (one from a forthcoming fully convolutional paper and another from CellProfiler).

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "312836": "I did not finetune with the stage1 test labels ... though i think it might help ..\n\nI use mask-rcnn . I would like to see how unet perform.\n\nHere are some few stage2 images that work\n\n ![enter image description here][1]\n\n ![enter image description here][2]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312836/9116/09d5c0a305af734e600a4ef547580c713cede403c6ae79424c12e57739417bf9.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312836/9123/36dac2ace4f3c79b683d0bad0c2de03b9b01f762f7f4df0e7f009d9fcc599972.png", "312837": "here are those that \"does not work at all\"\n\n\n ![enter image description here][1]\n\n ![enter image description here][2]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312837/9117/9fca513009e13f413c1dc322325f9f21c9b5b926df50ef348db7ad2e02e13aca.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312837/9122/22f0fc6fcff08d806138e400c3320ad8b4ff3ddb12c4adacab5b83fcd46d32a0.png", "312838": "here are those that \"i don't know work or not\"\n\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312838/9121/21e6d64a92ae99477f21b9b512c35dd1d3c390d0867dbb287f406f91aedfe0dd.png", "312843": "here are some that \"work a little\"\n\n ![enter image description here][1]\n\n\n ![enter image description here][2]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312843/9111/9b03511ba5dbeaba579a12621ec7a953e8eb5283c4c531a71bd09a5fdf55fd1c.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312843/9119/20d0d2ac8ecc95216d056b46163466a18b7f74c921e9f2b5e46e538a58de65e6.png", "312849": "if we assume that the private LB will choose equal images per cell type, I estimate my private score could be in the range of 0.3 or less", "312885": "Heng,\nDo you think detecting nuclei for 9ac881c9dc02cb901f16f287a0a514e733c751710e9c797f750b9aa1d82057a0 is the correct answer? Not sure whether the image was prepared with valid method that biologist use to get useful information or was generated to trigger false positive. There is also an image that is hand-drawn. I'm not sure a model that detects a nuclei on it is better than a model that does not detect any nucleus.\n\nThough it is interesting to see neural net model detecting nuclei in these images, it seems to me that a model that can filter them out as not useful seem more useful to biologists.", "312893": "i think this:\nhttp://mcb.asm.org/content/21/1/281/F3.expansion.html\n\n ![enter image description here][1]\n\n ![enter image description here][2]\n\n\n [1]: http://www.mdpi.com/cancers/cancers-10-00025/article_deploy/html/images/cancers-10-00025-g002.png\n [2]: http://mcb.asm.org/content/21/1/281/F3.large.jpg", "312908": "After looking as your results, I think it’ll do better. Have you submitted?", "312912": "not yet. 3000 images x 12 ensembles will take maybe 2 days", "312957": "Yeah, it is a nucleus. Same as 21da1704bd63963b3c6bc47480786f7d4c1735ae4bbfbb63a22ce376e036f403", "312960": "https://youtu.be/oKxsW8DKJQQ?t=1m14s\n\nSomewhere, after seeing the phase two images, @Heng mutters to himself “I’m going to need a bigger ensemble.” 😂", "312963": "Allen \"Heng mutters to himself “I’m going to need a bigger ensemble\" , you predicted it when i am preparing for my next post.\n\n ![enter image description here][1]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/312963/9127/draft.png", "312994": "Ok.\n\nI'd still argue, though, that everything else being equal, a model that does not detect any nuclei in hand drawn images are actually better.", "313001": "I guess you could argue that if you were analysing a pathologists notes for sketches of irregular nuclei (that could be directly related back to a linked patient's sample), detecting them might be useful. However, you might as well just take a snap under the scope, as the image the drawing is based on has obviously undergone H&E staining for microscopy.", "313064": "Heng, your bottom left image here is a strange case. So, you are actually detecting entire cells and not nuclei, but the \"rounded up\" cells are appearing this way. From a biological perspective I have NO IDEA how these were annotated. If I were to try to manually segment this data I would say that I am only able to segment cells, and no nuclei can be detected. Your algorithm may actually be working perfectly as I might decide that the rounded up cells are mostly nuclei and only circle those :)", "313149": "interesting results, I actually have no idea what i am doing when look at some seg results :)", "313188": "I think you should have put a blur before your nucleus finder (probably as a parallel pipeline to the un-blurred one, ensembled).", "313254": "hi Heng, for 9c1ad, 9c7eb, 09d5c, do you think the smaller brighter dots wrapped in the ellipse are nucleus, and current masks are on the cells ?", "313283": "Heng, as requested here are some results. These are results for the same images you tested that are produced by a model that is neither a U-net nor R-CNN exactly. It's topology is more or less that of a U-net, but it predicts (foreground, relative-center-x, relative-center-y, area) at each pixel. Then it clusters the points classified as foreground on predicted center- x, center-y and area using DBSCAN.\n\nIt completely falls down on the speckled images, but I think that's a data augmentation failure on my part. It fails on a couple others pretty badly, but mostly does OK.", "313298": "Hi, this is my results on stage2 testing set. Note that, no postprocessing, no ensembling are applied\n![enter image description here][1]\n![enter image description here][2]\n![enter image description here][3]\n![enter image description here][4]\n![enter image description here][5]\n![enter image description here][6]\n![enter image description here][7]\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9149/9ac881c9dc02cb901f16f287a0a514e733c751710e9c797f750b9aa1d82057a0.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9150/36dac2ace4f3c79b683d0bad0c2de03b9b01f762f7f4df0e7f009d9fcc599972.png\n [3]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9151/9b889d736e06f0182622661f54f72e5c90e3480049940523262745a41c89bbc5.png\n [4]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9152/09d5c0a305af734e600a4ef547580c713cede403c6ae79424c12e57739417bf9.png\n [5]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9153/9c1ad3fcb45d89072864a18608a0778468e7580b2b8207ae1cf5d096d380368a.png\n [6]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9154/9c7ebccedebb7e33a8ab8cef3c031d7845deb7c86064f81710691c7e64611b81.png\n [7]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313298/9165/21e6d64a92ae99477f21b9b512c35dd1d3c390d0867dbb287f406f91aedfe0dd.png", "313314": "i show the intermediate results of ensemble of 4 mask rcnns. the final submission should have 12 mask rcnns, but inference of the other 8 are still on going.\n\nEnsemble is clustering and removing false detection. Final results is average of clustered predictions. From left to right\n\n- input image\n\n- final results (color overlay)\n\n- final results (contour overlay)\n\n- cluster(contour overlay)\n\n- cluster(color overlay)\n\n- rejected instances (from ensemble of mask rcnn predictions , flip scale)\n\n- sum of all instances (accepted and rejected)\n\nfinal results = cluster - post filtering (too small mask and those at the border of images are removed)\n\n ![enter image description here][1]\n\n ![enter image description here][2]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313314/9155/b083e597b2cd3f7fe8f8ae24b0e38f569314b36a4c7c10442a2e5225d757e1af.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/313314/9160/dd58e56930c29ad238a73471303c1644c592e086d0d3a24fc7f56164b5e24b02.png", "313350": "Hi All, I just used the models from stage 1 without using test1 data to predict. I only got outputs from 2600-2900 images out of the total 3019 images. Does this sound right to you?", "313641": "I cannot WAIT to see his LB results post!", "313721": "Here are some of mine. Some good, some unsure, some epically bad. On the now infamous textbook scan (aka Allen's art class progeny), my model did better than I expected", "313726": "there are some images without detection. i haven't count it in detail. for gray stage2 test images, i have 2557 with detections out of 2800 input images for the normal model. other test augmented images varies in this number.", "313824": "I'm not Heng, but I think his masks are on the nuclei, and the stuff outside is the rest of the cells, neurons specifically.", "313884": "hi Ian, thanks for your comment! I meant the stuff inside the masks, thought they were nucleus yesterday. But after I asked someone in hospital, he think the mask is right and the smaller white things maybe DNA or something.", "313893": "interesting stage2 test images:\nbeee9cf93c88f1c4f892460101b6d64931ec64c2a72d91aad8c8c1306115bbbc\n\nnot sure if this will be used in the private LB score ...", "314393": "Hi, Heng, how could you get your stage 2 score, I submit my csv on public leaderboard but the display results are LB 0.", "315206": "Here is one of the cases, the algorithm fails on: for some cases it seems to learn the black background as cell.", "315261": "our stage2 submission (private LB 0.574) is here. you can see results for all the images\n\nhttps://drive.google.com/open?id=1QeMFRfiW95rudGLwKYNy-weLpP-5L6-S", "315263": "Nice work!", "315270": "Allen thanks! It seems that the in-house baseline results of 0.4x on stage2 has been surpassed? Or are they evaluated on different dataset?", "315289": "They were evaluated different and I had a few models working. Nevertheless, my best performing model on phase two, 0.582, was easily surpassed! I’m thrilled. We still want to test a few other models we regularly use in the lab (one from a forthcoming fully convolutional paper and another from CellProfiler)." }, "source": "meta" }