My name’s Allen. I helped organize the competition.
\n\nI’m a member of the Broad Institute’s Imaging Platform and the majority of professional life is spent thinking about and solving problems that are similar to the problem presented in the competition. I’m using this thread as a public lab notebook to discuss some of the challenges I encounter while working on the problem alongside other competitors.
\n\nHowever, before I begin, I want to stress:
\n\nI’ll submit scores, but I’m ineligible for a prize.
\n\nI’ll refrain from submitting during the second phase to avoid cluttering the leaderboard.
\n\nWhile I know some things other participants don’t, I intentionally don’t know anything about either the first or second phase test sets (e.g. number of images or experimental conditions).
\n\nIn addition to this thread, I’ll continue monitoring and help answer questions on the forum. You can also follow me on Twitter where I write about computer vision issues.
", "messages": [ { "id": 272845, "postDate": "2018-01-23T20:32:21.620Z", "content": "My name’s Allen. I helped organize the competition.
\n\nI’m a member of the Broad Institute’s Imaging Platform and the majority of professional life is spent thinking about and solving problems that are similar to the problem presented in the competition. I’m using this thread as a public lab notebook to discuss some of the challenges I encounter while working on the problem alongside other competitors.
\n\nHowever, before I begin, I want to stress:
\n\nI’ll submit scores, but I’m ineligible for a prize.
\n\nI’ll refrain from submitting during the second phase to avoid cluttering the leaderboard.
\n\nWhile I know some things other participants don’t, I intentionally don’t know anything about either the first or second phase test sets (e.g. number of images or experimental conditions).
\n\nIn addition to this thread, I’ll continue monitoring and help answer questions on the forum. You can also follow me on Twitter where I write about computer vision issues.
", "rawMarkdown": "My name’s [Allen][1]. I helped organize the competition. \n\nI’m a member of the Broad Institute’s [Imaging Platform][2] and the majority of professional life is spent thinking about and solving problems that are similar to the problem presented in the competition. I’m using this thread as a public lab notebook to discuss some of the challenges I encounter while working on the problem alongside other competitors. \n\nHowever, before I begin, I want to stress:\n\n**I’ll submit scores, but I’m ineligible for a prize.**\n\n**I’ll refrain from submitting during the second phase to avoid cluttering the leaderboard.**\n\nWhile I know some things other participants don’t, I intentionally don’t know anything about either the first or second phase test sets (e.g. number of images or experimental conditions).\n\nIn addition to this thread, I’ll continue monitoring and help answer questions on the forum. You can also [follow me on Twitter][3] where I write about computer vision issues.\n\n [1]: https://personal.broadinstitute.org/agoodman/\n [2]: https://www.broadinstitute.org/imaging\n [3]: https://twitter.com/0x00B1", "votes": 193 }, { "id": 272854, "postDate": "2018-01-23T21:01:07.490Z", "content": "My immediate observations about the competition metric:
\n\nMy immediate observations about the training set:
\n\nYou'd mentioned \"It’s also unclear if the test set will include entirely new conditions or the same conditions that are present in the training set.\"
\n\nI just made that public:\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/48136#272845
", "rawMarkdown": "You'd mentioned \"It’s also unclear if the test set will include entirely new conditions or the same conditions that are present in the training set.\" \n\nI just made that public:\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/48136#272845", "votes": 5 }, { "id": 273431, "postDate": "2018-01-24T15:56:38.840Z", "content": "Awesome! Thanks, Anne!
", "rawMarkdown": "Awesome! Thanks, Anne!" }, { "id": 274079, "postDate": "2018-01-25T21:24:56.213Z", "content": "What About Tiramisu .. ! I am trying my hand on that... what Do You Say...
", "rawMarkdown": "What About Tiramisu .. ! I am trying my hand on that... what Do You Say..." }, { "id": 274969, "postDate": "2018-01-27T23:14:52.253Z", "content": "I think fully convolutional neural networks (FCNNs) like SegNet, Tiramisu, and U-Net are certainly viable but only when paired with pre-or-post-processing steps that help disconnect object instances.
", "rawMarkdown": "I think fully convolutional neural networks (FCNNs) like SegNet, Tiramisu, and U-Net are certainly viable but only when paired with pre-or-post-processing steps that help disconnect object instances.", "votes": 1 }, { "id": 279287, "postDate": "2018-02-07T18:07:50.163Z", "content": "Is this still an instance segmentation challenge if all the objects in the image are of the same class? I would think that if two nuclei in the image are touching then the RLE would be the same regardless if instance or semantic segmentation is used.
", "rawMarkdown": "Is this still an instance segmentation challenge if all the objects in the image are of the same class? I would think that if two nuclei in the image are touching then the RLE would be the same regardless if instance or semantic segmentation is used. ", "votes": 2 }, { "id": 279600, "postDate": "2018-02-08T10:24:28.243Z", "content": "@jackvial: That was my confusion too, but you submit a different row for each instance. So if you detect 30 different nuclei for image X, then image X has 30 rows in the submission CSV file.
", "rawMarkdown": "@jackvial: That was my confusion too, but you submit a different row for each instance. So if you detect 30 different nuclei for image X, then image X has 30 rows in the submission CSV file.", "votes": 3 }, { "id": 279664, "postDate": "2018-02-08T13:48:41.777Z", "content": "Could anyone please explain the idea behind splitting images into 5-or-more classes? Say, I did it, then what? Train the special network for every class alone? Or feed the predicted class as some additional input to my general model?
\n\nI mean, say, I can predict the class on a test image. How I should use this class then?
", "rawMarkdown": "Could anyone please explain the idea behind splitting images into 5-or-more classes? Say, I did it, then what? Train the special network for every class alone? Or feed the predicted class as some additional input to my general model? \n\nI mean, say, I can predict the class on a test image. How I should use this class then?", "votes": 2 }, { "id": 280028, "postDate": "2018-02-09T05:59:21.610Z", "content": "Just balancing.
\n\nLets pretend you have 1000 grey-scale images and 100 colorful. You splitting in into 2 parts randomly and it would be a distribution like 950/150 (in validation part lets say 148 grey-scale and ONLY 2 colorful), so you never know how is good you model in case of prediction colorful images. Maybe good or may be terrible? This is why it`s the point to balance validation set. Moreover in test set in could be 800 grey-scale and 300 colorful (your model works bad here and you notice it only after the private leaderboard will be revealed)
", "rawMarkdown": "Just balancing.\n\nLets pretend you have 1000 grey-scale images and 100 colorful. You splitting in into 2 parts randomly and it would be a distribution like 950/150 (in validation part lets say 148 grey-scale and ONLY 2 colorful), so you never know how is good you model in case of prediction colorful images. Maybe good or may be terrible? This is why it`s the point to balance validation set. Moreover in test set in could be 800 grey-scale and 300 colorful (your model works bad here and you notice it only after the private leaderboard will be revealed)", "votes": 3 }, { "id": 280038, "postDate": "2018-02-09T06:28:43.443Z", "content": "So how can I fix this even if I know? Can I just augment and resample the 5 different group and make a new more balanced training set?
", "rawMarkdown": "So how can I fix this even if I know? Can I just augment and resample the 5 different group and make a new more balanced training set?", "votes": 1 } ] }, { "id": 273531, "postDate": "2018-01-24T18:20:45.887Z", "content": "I continued thinking about my validation strategy.
\n\nI accomplished the first task of stratifying the training images into five “modalities” (or modality-like classes). It was a straightforward process. First, a labmate (Hi, Jane!) stratified the images into three classes by the sum of their intensities. The classes were images with white backgrounds, images with black backgrounds, and images with purple foreground and/or purple background. I further stratified the “purple” class into three new classes: images with purple foreground and purple background, images with purple foreground and white background, and images with purple foreground and yellow background.
\n\nHere’re the total number of images per class:
\n\nI’ve attached a spreadsheet (classes.csv) with the filenames for each class. Enjoy! 😎
\n\nA few additional observations:
\n\nI think it’s entirely possible to split the black foreground and white background class into a few additional classes (e.g. fuzzy and non-fuzzy foreground objects) to further ensure generalizability around non-intensity related variables like shape and size (and simplify training by balancing the classes), but the aforementioned five classes seem sufficient to start the validation process. Nevertheless, I plan on revisiting this soon.
\n\nI found a number of images that will be very problematic from an object detection perspective. A few examples of images that will give me nightmares:
\n\nHowever, in my opinion, the most worrying image is this monster:
\n\nThis is exactly the type of image that you use to penalize overfitting.
\n\nIn addition to the modality subsets that I described I’m also going to put together a subset with the aforementioned nightmare images to further punish myself. I’ll probably add a few from the “problems” thread too.
\n\nI really want to ask the biologist in my lab about a few of the modalities since my biology background is limited to college biology and my microscopy knowledge is non-existent. However, I’ve managed to be good. I think they are entertained by my newfound interest in their fields of expertise but disappointed I’m still using phrases like “purple foreground and purple background” to describe something that’s probably significant.
", "rawMarkdown": "I continued thinking about my validation strategy. \n\nI accomplished the first task of stratifying the training images into five “modalities” (or modality-like classes). It was a straightforward process. First, a labmate (Hi, Jane!) stratified the images into three classes by the sum of their intensities. The classes were images with white backgrounds, images with black backgrounds, and images with purple foreground and/or purple background. I further stratified the “purple” class into three new classes: images with purple foreground and purple background, images with purple foreground and white background, and images with purple foreground and yellow background.\n\nHere’re the total number of images per class:\n\n* black foreground and white background (16)\n* purple background and purple foreground (71)\n* purple foreground and white background (41)\n* purple foreground and yellow background (8)\n* white foreground and black background (599)\n\nI’ve attached a spreadsheet (classes.csv) with the filenames for each class. Enjoy! 😎\n\nA few additional observations:\n\nI think it’s entirely possible to split the black foreground and white background class into a few additional classes (e.g. fuzzy and non-fuzzy foreground objects) to further ensure generalizability around non-intensity related variables like shape and size (and simplify training by balancing the classes), but the aforementioned five classes seem sufficient to start the validation process. Nevertheless, I plan on revisiting this soon.\n\nI found a number of images that will be very problematic from an object detection perspective. A few examples of images that will give me nightmares:\n\n* 259b35151d4a7a5ffdd7ab7f171b142db8cfe40beeee67277fac6adca4d042c4.png\n* f4b7c24baf69b8752c49d0eb5db4b7b5e1524945d48e54925bff401d5658045d.png\n* 8f27ebc74164eddfe989a98a754dcf5a9c85ef599a1321de24bcf097df1814ca.png\n\nHowever, in my opinion, the most worrying image is this monster:\n\n* af576e8ec3a8d0b57eb6a311299e9e4fd2047970d3dd9d6f52e54ea6a91109da.png\n\nThis is exactly the type of image that you use to penalize overfitting.\n\nIn addition to the modality subsets that I described I’m also going to put together a subset with the aforementioned nightmare images to further punish myself. I’ll probably add a few from the “problems” thread too.\n\nI really want to ask the biologist in my lab about a few of the modalities since my biology background is limited to college biology and my microscopy knowledge is non-existent. However, I’ve managed to be good. I think they are entertained by my newfound interest in their fields of expertise but disappointed I’m still using phrases like “purple foreground and purple background” to describe something that’s probably significant.", "votes": 18, "replies": [ { "id": 273751, "postDate": "2018-01-25T04:30:46.797Z", "content": "Speaking of af576e8ec3a8d0b57eb6a311299e9e4fd2047970d3dd9d6f52e54ea6a91109da, are you sure that two objects marked with a red line on attached picture are not nuclea outside of focal range?
", "rawMarkdown": "Speaking of af576e8ec3a8d0b57eb6a311299e9e4fd2047970d3dd9d6f52e54ea6a91109da, are you sure that two objects marked with a red line on [attached picture](https://imgur.com/a/gXoLY) are not nuclea outside of focal range? \n\n", "votes": 1 }, { "id": 273968, "postDate": "2018-01-25T15:42:16.107Z", "content": "Honestly, I don’t have an opinion. I’m mostly focused on finding a model that best models the perception of the annotator (or annotators). Essentially I’m assuming the annotations are correct (i.e. uniform).
", "rawMarkdown": "Honestly, I don’t have an opinion. I’m mostly focused on finding a model that best models the perception of the annotator (or annotators). Essentially I’m assuming the annotations are correct (i.e. uniform).\n" }, { "id": 274783, "postDate": "2018-01-27T09:53:13.533Z", "content": "Thank you Allen. I went over the classes.csv file and found 735 different entries (for my understanding, this mean 735 different images). But when looking at stage1_train I found only 670 different folders. What is the source of this difference?
", "rawMarkdown": "Thank you Allen. I went over the classes.csv file and found 735 different entries (for my understanding, this mean 735 different images). But when looking at stage1_train I found only 670 different folders. What is the source of this difference?", "votes": 1 }, { "id": 274784, "postDate": "2018-01-27T10:06:00.420Z", "content": "Ok, I just figure out that the file contain the both train and test images.
", "rawMarkdown": "Ok, I just figure out that the file contain the both train and test images.", "votes": 1 }, { "id": 274968, "postDate": "2018-01-27T23:13:24.270Z", "content": "Yep. 😎
\n\nI’ll add an additional column to classify them into phase one training and phase one test.
", "rawMarkdown": "Yep. 😎 \n\nI’ll add an additional column to classify them into phase one training and phase one test.", "votes": 1 }, { "id": 276998, "postDate": "2018-02-02T00:37:16.267Z", "content": "What's the purpose to divide the dataset to aforementioned subclasses? Is it to have balanced batch (i.e. equally includes images from different modalities) for training and validation?
", "rawMarkdown": "What's the purpose to divide the dataset to aforementioned subclasses? Is it to have balanced batch (i.e. equally includes images from different modalities) for training and validation?", "votes": 1 }, { "id": 277029, "postDate": "2018-02-02T03:00:46.890Z", "content": "Yeah. I want to validate whether my model can generalize to new experimental conditions.
", "rawMarkdown": "Yeah. I want to validate whether my model can generalize to new experimental conditions.\n", "votes": 2 }, { "id": 289380, "postDate": "2018-03-01T14:11:53.307Z", "content": "Hi,\nDoes your separation mean that you trained five different models (each trained on a subset of data that you explained)?
", "rawMarkdown": "Hi,\nDoes your separation mean that you trained five different models (each trained on a subset of data that you explained)?", "votes": 2 } ] }, { "id": 275389, "postDate": "2018-01-29T02:21:30.613Z", "content": "Whew. A very busy weekend. I posted a new score. I’m still overfitting, but I’ll deal with that in the future. What changed? A lot. Here’s a few of the most noteworthy changes:
\n\nI’m using a Feature Pyramid Network (FPN) before both the region proposal network (RPN) and the region-based convolutional neural network (RCNN). I think most of the improvement can be attributed to this.
I added a basic augmentation loop (random crops and some focus shifts).
I normalized the images (by mean subtraction).
There’s a few other things, but I’m too tired to remember! I’ll post more Monday or Tuesday.
", "rawMarkdown": "Whew. A very busy weekend. I posted a new score. I’m still overfitting, but I’ll deal with that in the future. What changed? A lot. Here’s a few of the most noteworthy changes:\n\n* I’m using a Feature Pyramid Network (FPN) before both the region proposal network (RPN) and the region-based convolutional neural network (RCNN). I think most of the improvement can be attributed to this.\n\n* I added a basic augmentation loop (random crops and some focus shifts).\n\n* I normalized the images (by mean subtraction).\n\nThere’s a few other things, but I’m too tired to remember! I’ll post more Monday or Tuesday.", "votes": 19 }, { "id": 278694, "postDate": "2018-02-06T18:52:57.083Z", "content": "Thanks for sharing, Allen! I have read your Mask Rcnn code. I found that you only use the last three outputs of ResNet50, so the largest size of feature map is 32*32 if the input size is 256*256. It means that the smallest size of anchor grids is 8*8, but many nuclei in the samples are smaller than 8*8 pixels, so the model can not figure out two different nuclei if both of them are in one grid. How do you deal with this situation?
", "rawMarkdown": "Thanks for sharing, Allen! I have read your Mask Rcnn code. I found that you only use the last three outputs of ResNet50, so the largest size of feature map is 32*32 if the input size is 256*256. It means that the smallest size of anchor grids is 8*8, but many nuclei in the samples are smaller than 8*8 pixels, so the model can not figure out two different nuclei if both of them are in one grid. How do you deal with this situation?", "votes": 7, "replies": [ { "id": 282978, "postDate": "2018-02-14T20:30:11.467Z", "content": "The first strategy works well on most problems (e.g. see CIFAR research). However, we want to work on making the second strategy more attractive (i.e. easier to implement).
", "rawMarkdown": "The first strategy works well on most problems (e.g. see CIFAR research). However, we want to work on making the second strategy more attractive (i.e. easier to implement).", "votes": 1 } ] }, { "id": 273516, "postDate": "2018-01-24T17:58:16.023Z", "content": "Allen,
\n\nIts great to have an expert providing a continuous benchmark:) This is a great addition and great motivator for other competitors.
", "rawMarkdown": "Allen, \n\nIts great to have an expert providing a continuous benchmark:) This is a great addition and great motivator for other competitors. ", "votes": 6, "replies": [ { "id": 273540, "postDate": "2018-01-24T18:34:58.630Z", "content": "Thanks, dude. I think the next few weeks will be really fun!
", "rawMarkdown": "Thanks, dude. I think the next few weeks will be really fun!", "votes": 4 } ] }, { "id": 275986, "postDate": "2018-01-30T12:18:48.300Z", "content": "Hi Allen,
\n\n\n\n\nThe classes were images with white backgrounds, images with black backgrounds, and images with purple foreground and/or purple background. I further stratified the “purple” class into three new classes: images with purple foreground and purple background, images with purple foreground and white background, and images with purple foreground and yellow background.
\n
Any guesses on what's the difference between those within the purple class which u further segmented into
\n\nWould it be some sort of artifact introduced such as the imaging lab or batch?
", "rawMarkdown": "Hi Allen, \n\n> The classes were images with white backgrounds, images with black backgrounds, and images with purple foreground and/or purple background. I further stratified the “purple” class into three new classes: images with purple foreground and purple background, images with purple foreground and white background, and images with purple foreground and yellow background.\n\nAny guesses on what's the difference between those within the purple class which u further segmented into \n\n 1. Purple foreground purple background\n 2. purple foreground and white background\n 3. purple foreground and yellow background\n\nWould it be some sort of artifact introduced such as the imaging lab or batch?", "votes": 3, "replies": [ { "id": 277269, "postDate": "2018-02-02T17:16:11.680Z", "content": "The images come from lots of different labs and staining procedures; differences in color balance like this stem from that.
", "rawMarkdown": "The images come from lots of different labs and staining procedures; differences in color balance like this stem from that.", "votes": 1 }, { "id": 278521, "postDate": "2018-02-06T11:33:11.303Z", "content": "There are some image in test set with \"yellow background/purple foreground\". But I can't find this kind of data in train set. My model works badly on these data. Where can I find this kind of data?
", "rawMarkdown": "There are some image in test set with \"yellow background/purple foreground\". But I can't find this kind of data in train set. My model works badly on these data. Where can I find this kind of data?", "votes": 1 }, { "id": 278528, "postDate": "2018-02-06T11:58:39.260Z", "content": "I think the goal here is to generalize well. So having these yellow background images in the test set is a good thing. If your model does not work well on them, then you won't get a high score in the second stage.
", "rawMarkdown": "I think the goal here is to generalize well. So having these yellow background images in the test set is a good thing. If your model does not work well on them, then you won't get a high score in the second stage.", "votes": 2 } ] }, { "id": 294218, "postDate": "2018-03-11T15:31:56.230Z", "content": "Hi Allen,
\n\nThanks for sharing.
\n\nI encountered a problem that when I use mask-rcnn in Detectron to detect nuclei, It seems marked the same cell multiple times, This leads me to still cannot submit the result. How can I do to make my result be able to submit?
\n\nAnything will be helpful!
", "rawMarkdown": "Hi Allen,\n\nThanks for sharing.\n\nI encountered a problem that when I use mask-rcnn in Detectron to detect nuclei, It seems marked the same cell multiple times, This leads me to still cannot submit the result. How can I do to make my result be able to submit?\n\nAnything will be helpful!", "votes": 1, "replies": [ { "id": 294233, "postDate": "2018-03-11T15:55:46.687Z", "content": "@Brain: Very interesting if someone uses Detectron for this task. I also look at the repo but it requires converting the dataset to coco format. Could you please give me the way to convert it? How much does it takes to train your model? I guess it will be very fast
", "rawMarkdown": "@Brain: Very interesting if someone uses Detectron for this task. I also look at the repo but it requires converting the dataset to coco format. Could you please give me the way to convert it? How much does it takes to train your model? I guess it will be very fast", "votes": 1 }, { "id": 294431, "postDate": "2018-03-12T00:15:06.887Z", "content": "I write a simple script to find polygons and convert it to json format. (About this code I planned to open a new topic, but considering that I haven't submitted results yet, I don't think this code is reliable.)
\n\nIt's fast, I didn't train from scratch and it's only take my about 40 minutes on single 1080Ti. And the results seems accurate and impressive, but as I said, it predict a single nuclei multiple times, which leads me still can't make my submission.
\n\nI will open a topic about how to use Detectron in this game as soon as I solve this problem if it will be helpful to you guys.
", "rawMarkdown": "I write a simple script to find polygons and convert it to json format. (About this code I planned to open a new topic, but considering that I haven't submitted results yet, I don't think this code is reliable.)\n\nIt's fast, I didn't train from scratch and it's only take my about 40 minutes on single 1080Ti. And the results seems accurate and impressive, but as I said, it predict a single nuclei multiple times, which leads me still can't make my submission.\n\nI will open a topic about how to use Detectron in this game as soon as I solve this problem if it will be helpful to you guys.", "votes": 1 }, { "id": 294441, "postDate": "2018-03-12T00:54:02.937Z", "content": "It really helps us. I am very interested in Detectron. Fo your problem, I guess your results in some overlapping mask that one pixel belongs to 2 cells but the challenge only allows that one pixels belong 1 cell. Thus, you may use this function to remove it. Try it if it work https://github.com/YanWang2014/Mask_RCNN/blob/master/Spot-Nuclei2/functions.py#L48
", "rawMarkdown": "It really helps us. I am very interested in Detectron. Fo your problem, I guess your results in some overlapping mask that one pixel belongs to 2 cells but the challenge only allows that one pixels belong 1 cell. Thus, you may use this function to remove it. Try it if it work https://github.com/YanWang2014/Mask_RCNN/blob/master/Spot-Nuclei2/functions.py#L48", "votes": 2 }, { "id": 294449, "postDate": "2018-03-12T01:07:06.107Z", "content": "We actually tried detectron, so far not good on val set.
", "rawMarkdown": "We actually tried detectron, so far not good on val set.", "votes": 1 }, { "id": 294456, "postDate": "2018-03-12T01:30:09.513Z", "content": "Thanks and I'll try it out.
", "rawMarkdown": "Thanks and I'll try it out." }, { "id": 294458, "postDate": "2018-03-12T01:32:08.250Z", "content": "Do you know any reason about that?
", "rawMarkdown": "Do you know any reason about that?" }, { "id": 294466, "postDate": "2018-03-12T01:53:36.803Z", "content": "Hi Brain, can you post your validation results using detectron, which is given by coco api? We got pretty good results for bbox, but not so good for seg.
", "rawMarkdown": "Hi Brain, can you post your validation results using detectron, which is given by coco api? We got pretty good results for bbox, but not so good for seg." }, { "id": 294482, "postDate": "2018-03-12T02:23:11.763Z", "content": "You mean predicted mask from 'stage1-test' folder?
", "rawMarkdown": "You mean predicted mask from 'stage1-test' folder?" }, { "id": 294483, "postDate": "2018-03-12T02:27:49.107Z", "content": "No. He means validation set that you can separate from stage1-train set with proportion likes 80/20. Then you can evaluate in your machine, without submitting it to the kagge website
", "rawMarkdown": "No. He means validation set that you can separate from stage1-train set with proportion likes 80/20. Then you can evaluate in your machine, without submitting it to the kagge website" }, { "id": 294485, "postDate": "2018-03-12T02:28:08.077Z", "content": "something like this:
\n\nRunning per image evaluation...
\nDONE (t=0.46s).
\nAccumulating evaluation results...
\nDONE (t=0.38s).
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.505
\n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.697
\n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.573
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.586
\n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.519
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.501
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.387
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.594
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.595
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.640
\n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.566
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.564
Actually I don't have it. Because I assume this a first try, so I use all the image in train set to train.
", "rawMarkdown": "Actually I don't have it. Because I assume this a first try, so I use all the image in train set to train." }, { "id": 294499, "postDate": "2018-03-12T02:39:28.317Z", "content": "You can still evaluate on some images, either they are in the training set or not. We find even in the training set, the seg is not so good as bbox.
", "rawMarkdown": "You can still evaluate on some images, either they are in the training set or not. We find even in the training set, the seg is not so good as bbox." }, { "id": 294502, "postDate": "2018-03-12T02:46:58.940Z", "content": "I have putted mask to some pictures in train set yesterday and I found that the seg in gray scale image fitted very well, in color images not very well. And still have problem that mark same cell multiple times.
\n\nI will use cocoapi to evaluate some images in train set this afternoon.
", "rawMarkdown": "I have putted mask to some pictures in train set yesterday and I found that the seg in gray scale image fitted very well, in color images not very well. And still have problem that mark same cell multiple times.\n\nI will use cocoapi to evaluate some images in train set this afternoon." }, { "id": 295063, "postDate": "2018-03-13T02:23:30.187Z", "content": "According to my observation, You can't just visually check your detection result cause it's hard to tell the difference under high iou metric.
", "rawMarkdown": "According to my observation, You can't just visually check your detection result cause it's hard to tell the difference under high iou metric. " } ] }, { "id": 288283, "postDate": "2018-02-27T05:19:41.620Z", "content": "Can you give more details about how to make supplement data? I can't understand why it's useful. Thanks for your work!
", "rawMarkdown": "Can you give more details about how to make supplement data? I can't understand why it's useful. Thanks for your work!", "votes": 1, "replies": [ { "id": 288452, "postDate": "2018-02-27T14:27:34.410Z", "content": "This thread might help about how to make more annotations: https://www.kaggle.com/c/data-science-bowl-2018/discussion/48347
\n\nAs I understand it, making more annotated images is not useful in the sense of giving you very much of an advantage over other teams, because external data must be shared in the external data thread here: https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572
\n\nBut, extra data will certainly make a better result!
", "rawMarkdown": "This thread might help about how to make more annotations: https://www.kaggle.com/c/data-science-bowl-2018/discussion/48347\n\nAs I understand it, making more annotated images is not useful in the sense of giving you very much of an advantage over other teams, because external data must be shared in the external data thread here: https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572\n\nBut, extra data will certainly make a better result!", "votes": 2 }, { "id": 288465, "postDate": "2018-02-27T14:59:15.033Z", "content": "hi Anne,\ncan you please confirm that you (as the organizer) consider manual changes to the data as \"external data\"? Would also the proces which Allen is using (creating new masks by watershed etc.) considered as external data?\nIf yes - what is the way to disclose the information? Do we need to upload it (since it must be publicly available)?
\n\nWhy I ask is, that normally any changes to the training data itself (i.e. cleanup, manual re-annotations etc.) are not considered as external data and thus are allowed (even in competitions where no external data is permitted !)
", "rawMarkdown": "hi Anne,\ncan you please confirm that you (as the organizer) consider manual changes to the data as \"external data\"? Would also the proces which Allen is using (creating new masks by watershed etc.) considered as external data?\nIf yes - what is the way to disclose the information? Do we need to upload it (since it must be publicly available)?\n\nWhy I ask is, that normally any changes to the training data itself (i.e. cleanup, manual re-annotations etc.) are not considered as external data and thus are allowed (even in competitions where no external data is permitted !)", "votes": 2 }, { "id": 288472, "postDate": "2018-02-27T15:38:31.300Z", "content": "Making modifications to the train data has never been considered as external data.
", "rawMarkdown": "Making modifications to the train data has never been considered as external data.", "votes": 3 }, { "id": 289880, "postDate": "2018-03-02T14:29:03.990Z", "content": "I'm not the rule-maker, so will defer to @wendykan on this one.
", "rawMarkdown": "I'm not the rule-maker, so will defer to @wendykan on this one.", "votes": 2 }, { "id": 289897, "postDate": "2018-03-02T15:12:22.137Z", "content": "Anne, are you saying that annotations to external data need to be shared?
", "rawMarkdown": "Anne, are you saying that annotations to external data need to be shared?" }, { "id": 289898, "postDate": "2018-03-02T15:13:17.107Z", "content": "I'm saying \"I'm not the rule-maker, so will defer to @wendykan on this one.\"
", "rawMarkdown": "I'm saying \"I'm not the rule-maker, so will defer to @wendykan on this one.\"", "votes": 3 }, { "id": 289900, "postDate": "2018-03-02T15:19:55.223Z", "content": "https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572#284672
\n\n\n", "rawMarkdown": "https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572#284672\n\n> Wendy Kan \nYou don't need to post them as that is considered part of your methodology, not external data.", "votes": 3 } ] }, { "id": 280902, "postDate": "2018-02-11T11:41:40.723Z", "content": "Wendy Kan
\n
\n You don't need to post them as that is considered part of your methodology, not external data.
hi Allen, thank you very much for sharing
\n\nHope you don't mind a few questions/clarifications about using the \"mask rcnn\":\n- I downloaded the \"features/preprocessing-objectdetection\" branch\n- and created a new model based on the RCNN, but adding the keras_rcnn.layers.MaskRCNN() at the end
\n\nWhat I wasn't sure how to do:\n1.) how to initialize the output_masks? currently I have:\noutput_masks = keras.layers.TimeDistributed(keras.layers.Dense(mask_r * mask_c, kernel_initializer=\"zero\"))(output_features)\noutput_masks = Reshape(target_shape=(-1, mask_r, mask_c,1))(output_masks)\noutput_masks = keras_rcnn.layers.MaskRCNN()([target_categories, target_masks, output_masks])
\n\n2.) how to properly train the model?\nWith the above model I get a runtime error:
\n\nInvalidArgumentError (see above for traceback): Incompatible shapes: [1,32,28,28] vs. [1,7,28,28]\n[[Node: mask_rcnn_1/logistic_loss/mul = Mul[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:GPU:0\"](mask_rcnn_1/Log, _arg_target_images_0_4/_81)]]
\n\nthe [1,7,28,28] is the target mask (tried with a generator which returns just one file, and has 7 masks)\nbut where does the [1,32,28,28] come from? Does it mean the model predicted 32 masks?
", "rawMarkdown": "hi Allen, thank you very much for sharing\n\nHope you don't mind a few questions/clarifications about using the \"mask rcnn\":\n- I downloaded the \"features/preprocessing-objectdetection\" branch\n- and created a new model based on the RCNN, but adding the keras_rcnn.layers.MaskRCNN() at the end\n\nWhat I wasn't sure how to do:\n1.) how to initialize the output_masks? currently I have:\noutput_masks = keras.layers.TimeDistributed(keras.layers.Dense(mask_r * mask_c, kernel_initializer=\"zero\"))(output_features)\noutput_masks = Reshape(target_shape=(-1, mask_r, mask_c,1))(output_masks)\noutput_masks = keras_rcnn.layers.MaskRCNN()([target_categories, target_masks, output_masks])\n\n2.) how to properly train the model?\nWith the above model I get a runtime error:\n\nInvalidArgumentError (see above for traceback): Incompatible shapes: [1,32,28,28] vs. [1,7,28,28]\n[[Node: mask_rcnn_1/logistic_loss/mul = Mul[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:GPU:0\"](mask_rcnn_1/Log, _arg_target_images_0_4/_81)]]\n\nthe [1,7,28,28] is the target mask (tried with a generator which returns just one file, and has 7 masks)\nbut where does the [1,32,28,28] come from? Does it mean the model predicted 32 masks? ", "votes": 1, "replies": [ { "id": 282964, "postDate": "2018-02-14T20:16:09.627Z", "content": "No problem.
\n\nA basic mask branch will look like this:
\n\noutput_masks = keras.layers.TimeDistributed(\n keras.layers.Conv2D(\n filters=256, \n kernel_size=(3, 3), \n activation=\"relu\", \n padding=\"same\"\n )\n)(output_pooled_features)\n\noutput_masks = keras.layers.TimeDistributed(\n keras.layers.Conv2DTranspose(\n activation=\"relu\",\n filters=256,\n kernel_size=(2, 2),\n strides=2\n )\n)(output_masks)\n\noutput_masks = keras.layers.TimeDistributed(\n keras.layers.Conv2D(\n activation=\"sigmoid\",\n filters=n_categories, \n kernel_size=(1, 1), \n strides=1\n )\n)(output_masks)\nOops. I missed your second question. The output should look like this:
\n\n\n(images, objects, region_r, region_c, 2)\n
For example:
\n\n\n(1, 32, 28, 28, 2)\n
You use the (non-maximum suppressed) predicted bounding boxes (or categories) to score and drop your masks (i.e. the number of objects will be the same for the predicted bounding boxes, categories, and masks).
", "rawMarkdown": "Oops. I missed your second question. The output should look like this:\n\n```\n(images, objects, region_r, region_c, 2)\n```\n\nFor example:\n\n```\n(1, 32, 28, 28, 2)\n```\n\nYou use the (non-maximum suppressed) predicted bounding boxes (or categories) to score and drop your masks (i.e. the number of objects will be the same for the predicted bounding boxes, categories, and masks).\n" }, { "id": 282981, "postDate": "2018-02-14T20:31:42.883Z", "content": "thank you very much
", "rawMarkdown": "thank you very much" }, { "id": 282983, "postDate": "2018-02-14T20:33:01.590Z", "content": "No problem!
", "rawMarkdown": "No problem! ", "votes": 1 } ] }, { "id": 280562, "postDate": "2018-02-10T09:10:16.173Z", "content": "I've got a question about creating the submission. How are you dealing with overlapping masks?\nFirst I just created one mask image from all the predictions but I think that's not optimal and the score is lower because of that. But when I create single masks per nuclei than there are overlapping masks and I get an Exception when submitting the solution.
", "rawMarkdown": "I've got a question about creating the submission. How are you dealing with overlapping masks?\nFirst I just created one mask image from all the predictions but I think that's not optimal and the score is lower because of that. But when I create single masks per nuclei than there are overlapping masks and I get an Exception when submitting the solution.", "votes": 1, "replies": [ { "id": 280961, "postDate": "2018-02-11T15:00:40.013Z", "content": "I just sorted predicted masks from smaller to larger and removed duplicate pixels from the larger image. Not sure whether that's the best way to do it (or a good way to do it.) but it lets me create a solution I can submit.
", "rawMarkdown": "I just sorted predicted masks from smaller to larger and removed duplicate pixels from the larger image. Not sure whether that's the best way to do it (or a good way to do it.) but it lets me create a solution I can submit." }, { "id": 282973, "postDate": "2018-02-14T20:26:43.227Z", "content": "I sort the masks by their corresponding score (i.e. each pixel uses the pixel from the mask with the best score). I’ve mentioned this elsewhere, but I believe post-processing (e.g. non-maximum suppression) is the most difficult part of every image segmentation and object detection competition. It’s sadly under-researched. 😔
", "rawMarkdown": "I sort the masks by their corresponding score (i.e. each pixel uses the pixel from the mask with the best score). I’ve mentioned this elsewhere, but I believe post-processing (e.g. non-maximum suppression) is the most difficult part of every image segmentation and object detection competition. It’s sadly under-researched. 😔", "votes": 2 }, { "id": 291783, "postDate": "2018-03-06T20:52:04.663Z", "content": "So i am working on this part of the problem right now. I am somewhat confused by: \"each pixel uses the pixel from the mask with the best score\".
\n\nCase 1 - Partial overlap: Two masks share some but not all pixels. Do you keep the pixels from the higher scored mask and leave the residual from the second mask, or do you discard the second mask? I am currently keeping the residual as a second mask, but I can see that this is correct sometimes and wrong others.
\n\nCase 2 - Total overlap A: Higher scored mask is a subset of the lower scored mask. Do you keep the higher scored mask and discard the lower scored \"donut\" residual mask? I am currently planning to discard.
\n\nCase 3: Total overlap B: Higher scored mask is a superset of the lower scored mask. Do you discard the the smaller lower scored mask? Seems the only logical option.
", "rawMarkdown": "So i am working on this part of the problem right now. I am somewhat confused by: \"each pixel uses the pixel from the mask with the best score\". \n\nCase 1 - Partial overlap: Two masks share some but not all pixels. Do you keep the pixels from the higher scored mask and leave the residual from the second mask, or do you discard the second mask? I am currently keeping the residual as a second mask, but I can see that this is correct sometimes and wrong others.\n\nCase 2 - Total overlap A: Higher scored mask is a subset of the lower scored mask. Do you keep the higher scored mask and discard the lower scored \"donut\" residual mask? I am currently planning to discard. \n\nCase 3: Total overlap B: Higher scored mask is a superset of the lower scored mask. Do you discard the the smaller lower scored mask? Seems the only logical option.\n\n", "votes": 1 } ] }, { "id": 279030, "postDate": "2018-02-07T08:07:06.993Z", "content": "Hi Allen, do you have any idea about the issue posted here. It seems that the training process is locked or killed somehow by using keras_rcnn on both python3.5 and python3.6.
", "rawMarkdown": "Hi Allen, do you have any idea about the issue posted [here]. It seems that the training process is locked or killed somehow by using keras_rcnn on both python3.5 and python3.6.\n\n [here]: https://www.kaggle.com/kmader/keras-rcnn-based-overview-wip/comments#279024", "votes": 1, "replies": [ { "id": 282974, "postDate": "2018-02-14T20:27:20.973Z", "content": "Hi, this has been fixed. You should use this pull request:
\n\nhttps://github.com/broadinstitute/keras-rcnn/pull/164
\n\nor wait until next week when we release the new version (0.0.3).
", "rawMarkdown": "Hi, this has been fixed. You should use this pull request:\n\nhttps://github.com/broadinstitute/keras-rcnn/pull/164\n\nor wait until next week when we release the new version (0.0.3).", "votes": 1 } ] }, { "id": 277825, "postDate": "2018-02-04T13:13:46.863Z", "content": "Thanks Allen, I have a question regarding image size. Does your model allows to use different input sizes or all the images need to have the same size? If that is the case which size are you using?
\nThanks
Yes. It has two options:
\n\nWhen you use a batch_size of 1 you can use a (None, None) input_shape (i.e. the shape is inferred and can vary between batches); and
You can increase the batch_size and specify a strict target_size that’s either rescaled (proportionally) or cropped.
In my opinion, the mask-rcnn does not support different sizes input for it needs image size to generate anchors and predict the bounding boxes. Could you explain more on how to use various input size in mask rcnn? I found there were some high resolution images which have very tiny instances that if I resized it to 256*256, those objects would missing.
", "rawMarkdown": "In my opinion, the mask-rcnn does not support different sizes input for it needs image size to generate anchors and predict the bounding boxes. Could you explain more on how to use various input size in mask rcnn? I found there were some high resolution images which have very tiny instances that if I resized it to 256*256, those objects would missing.", "votes": 1 }, { "id": 280385, "postDate": "2018-02-09T21:27:58.187Z", "content": "Yeah. Eventually your objects disappear. There’s a few strategies that can complement (or replace) resizing. I think randomly cropping (especially since the images are rotationally invariant) is probably the easiest to use. You could also think carefully about anchor scales, convolutional features (e.g. feature pyramids), and upscaling particularly small objects.
", "rawMarkdown": "Yeah. Eventually your objects disappear. There’s a few strategies that can complement (or replace) resizing. I think randomly cropping (especially since the images are rotationally invariant) is probably the easiest to use. You could also think carefully about anchor scales, convolutional features (e.g. feature pyramids), and upscaling particularly small objects.\n", "votes": 1 } ] }, { "id": 275482, "postDate": "2018-01-29T09:12:57.177Z", "content": "Gread job, Allen.\nI try to use the maskRCNN model to train and segment the images with keras_rcnn.\nBut the loss is not decay．I think RCNNMaskLoss is necessary．\nWould you tell me the usage of RCNNMaskLoss.\nMany thanks!
", "rawMarkdown": "Gread job, Allen.\nI try to use the maskRCNN model to train and segment the images with keras_rcnn.\nBut the loss is not decay．I think RCNNMaskLoss is necessary．\nWould you tell me the usage of RCNNMaskLoss.\nMany thanks!\n", "votes": 1, "replies": [ { "id": 275575, "postDate": "2018-01-29T14:13:40.187Z", "content": "Yep. Your understanding is correct. You’ll need to attach both the RCNN and MaskRCNN layers, e.g.
\n\noutput_deltas, output_scores = keras_rcnn.layers.RCNN()([target_proposals, target_proposal_labels, output_deltas, output_scores])
output_masks = keras_rcnn.layers.MaskRCNN()[target_proposals, output_deltas, target_masks, output_masks]
output_bounding_boxes, output_labels = keras_rcnn.layers.ObjectDetection()([output_proposals, output_deltas, output_scores, target_metadata])
output = [output_bounding_boxes, output_labels, output_masks]
Thanks for your help.There is not MaskRCNN model in keras_rcnn/models/ yet.I think I have to create _maskrcnn.py first.
", "rawMarkdown": "Thanks for your help.There is not MaskRCNN model in keras_rcnn/models/ yet.I think I have to create _maskrcnn.py first.", "votes": 2 } ] }, { "id": 273098, "postDate": "2018-01-24T05:58:54.760Z", "content": "Great! you have thought out of the box. Almost team are following the UNet, I think that UNet is for semantic segmentation while the challenge somehow differences as you mentioned. I think you applied Mask-RCNN (first detect the cell region and then segment the object from individual boxes). Am I right? I am selecting the good framework for this challenge: Tensorflow, caffe2. The benefit of caffe2 is that it released the Mask-RCNN, while Tensorflow is flexible for training, such as GAN model. Which model should I select for this challenge?
", "rawMarkdown": "Great! you have thought out of the box. Almost team are following the UNet, I think that UNet is for semantic segmentation while the challenge somehow differences as you mentioned. I think you applied Mask-RCNN (first detect the cell region and then segment the object from individual boxes). Am I right? I am selecting the good framework for this challenge: Tensorflow, caffe2. The benefit of caffe2 is that it released the Mask-RCNN, while Tensorflow is flexible for training, such as GAN model. Which model should I select for this challenge?", "votes": 1, "replies": [ { "id": 273428, "postDate": "2018-01-24T15:55:06.117Z", "content": "Yes. Exactly. However, I should note that Google provides a fantastic implementation for TensorFlow and I also contribute to Keras-RCNN that provides this functionality. I’m planning on writing more about the underlying model architecture today or tomorrow.
", "rawMarkdown": "Yes. Exactly. However, I should note that Google provides a fantastic implementation for TensorFlow and I also contribute to Keras-RCNN that provides this functionality. I’m planning on writing more about the underlying model architecture today or tomorrow.\n", "votes": 4 }, { "id": 273470, "postDate": "2018-01-24T16:18:32.563Z", "content": "\n\n\nAllen Goodman wrote
\n \n\n\nYes. Exactly. However, I should note that Google provides a fantastic implementation for TensorFlow and I also contribute to Keras-RCNN that provides this functionality. I’m planning on writing more about the underlying model architecture today or tomorrow.
\n
Great, looking forward to your approach. By the way, can you give a link to Google's implementation in TensorFlow? If I understand correctly, the Mask-RCNN in TensorFlow Object Dectection API is on the way, but not released yet
", "rawMarkdown": "> **Allen Goodman wrote**\n> \n> > Yes. Exactly. However, I should note that Google provides a fantastic implementation for TensorFlow and I also contribute to Keras-RCNN that provides this functionality. I’m planning on writing more about the underlying model architecture today or tomorrow.\n> \n\nGreat, looking forward to your approach. By the way, can you give a link to Google's implementation in TensorFlow? If I understand correctly, the Mask-RCNN in TensorFlow Object Dectection API is on the way, but not released yet", "votes": 1 }, { "id": 273474, "postDate": "2018-01-24T16:23:54.227Z", "content": "https://github.com/tensorflow/models/tree/master/research/object_detection
\n\nYou could also use Keras-RCNN which has the needed layers implemented (e.g. the updated ROI pooling layer and the segmentation loss).
", "rawMarkdown": "https://github.com/tensorflow/models/tree/master/research/object_detection\n\nYou could also use Keras-RCNN which has the needed layers implemented (e.g. the updated ROI pooling layer and the segmentation loss).", "votes": 4 }, { "id": 273502, "postDate": "2018-01-24T17:04:19.967Z", "content": "Facebook recently released this: https://github.com/facebookresearch/Detectron
", "rawMarkdown": "Facebook recently released this: https://github.com/facebookresearch/Detectron", "votes": 4 }, { "id": 273547, "postDate": "2018-01-24T18:48:31.767Z", "content": "I haven’t used this, but the authors are fantastic people and fantastic researchers. I’d look into it (or use the alternative I co-wrote: Keras-RCNN 😎).
", "rawMarkdown": "I haven’t used this, but the authors are fantastic people and fantastic researchers. I’d look into it (or use the alternative I co-wrote: [Keras-RCNN][1] 😎).\n\n[1]: https://github.com/broadinstitute/keras-rcnn", "votes": 1 }, { "id": 273817, "postDate": "2018-01-25T08:16:45.147Z", "content": "Allen, which backbone network you have used for this task?
", "rawMarkdown": "Allen, which backbone network you have used for this task?", "votes": 2 }, { "id": 273964, "postDate": "2018-01-25T15:39:48.343Z", "content": "I’m using ResNet-50 (using the “basic” residual block) provided by Keras-ResNet alongside a Feature Pyramid Network (FPN). However, I plan on using something most sophisticated in the future (e.g. an ensemble of residual networks). It’s something I want to write about in the next day or two.
", "rawMarkdown": "I’m using ResNet-50 (using the “basic” residual block) provided by Keras-ResNet alongside a Feature Pyramid Network (FPN). However, I plan on using something most sophisticated in the future (e.g. an ensemble of residual networks). It’s something I want to write about in the next day or two.", "votes": 1 }, { "id": 274493, "postDate": "2018-01-26T17:53:07.263Z", "content": "Thanks Allen, so fisrt you cluster the dataset into five groups then design 5 networks (same architecture but different weights due to training on different input types). In the testing, you will select the network corresponding to the image types and perform inferenxe. Am I right? If I am correct, then the problem is that the number of training set in each group will small (because you have splitted from original dataset), that makes difficult for training with deep learning
", "rawMarkdown": "Thanks Allen, so fisrt you cluster the dataset into five groups then design 5 networks (same architecture but different weights due to training on different input types). In the testing, you will select the network corresponding to the image types and perform inferenxe. Am I right? If I am correct, then the problem is that the number of training set in each group will small (because you have splitted from original dataset), that makes difficult for training with deep learning", "votes": 1 }, { "id": 274974, "postDate": "2018-01-27T23:24:53.820Z", "content": "I’m still thinking about this.
\n\nInitially, I used k-fold cross-validation to evaluate whether my model was robust to new experimental conditions by holding out entire experiments in my validation and test sets. However, I’ve started thinking about an ensemble learning method to ensemble k models into one model (e.g. by averaging their weights). I have considered the solution you outlined and I think it’s certainly viable. Hell, I think it’d be extremely competitive (if not the best method). I didn’t use it because it’d take substantial time to implement and it felt antithetical to the spirt of the competition (we envisioned one model could generalize across seen and unseen experimental variables).
\n\nI’ll add that I recently worked on a paper that did something similar.
\n\nThe architecture looked like this:
\n\nCNN -> object detection network -> CNN -> image classifier
\n\nAccuracy (we used the same metric as the competition) skyrocketed when we piped the output of the object detection network (the bounding boxes and class scores) into the second CNN and image classifier. However, it did require us to restructure the problem so that the object detection network predicted fewer classes. Unfortunately, we’re still unsatisfied with the solution. It was a monster to properly train without under-or-overfitting and it was difficult (if not impossible) to justify the second convolutional neural network (CNN) since the feature space should have been captured by the first.
", "rawMarkdown": "I’m still thinking about this.\n\nInitially, I used *k*-fold cross-validation to evaluate whether my model was robust to new experimental conditions by holding out entire experiments in my validation and test sets. However, I’ve started thinking about an ensemble learning method to ensemble *k* models into one model (e.g. by averaging their weights). I have considered the solution you outlined and I think it’s certainly viable. Hell, I think it’d be extremely competitive (if not the best method). I didn’t use it because it’d take substantial time to implement and it felt antithetical to the spirt of the competition (we envisioned one model could generalize across seen and unseen experimental variables).\n\nI’ll add that I recently worked on a paper that did something similar.\n\nThe architecture looked like this:\n\nCNN -> object detection network -> CNN -> image classifier\n\nAccuracy (we used the same metric as the competition) skyrocketed when we piped the output of the object detection network (the bounding boxes and class scores) into the second CNN and image classifier. However, it did require us to restructure the problem so that the object detection network predicted fewer classes. Unfortunately, we’re still unsatisfied with the solution. It was a monster to properly train without under-or-overfitting and it was difficult (if not impossible) to justify the second convolutional neural network (CNN) since the feature space should have been captured by the first. ", "votes": 1 }, { "id": 275864, "postDate": "2018-01-30T05:59:43.323Z", "content": "Hello Allen,\nI'm want to try out Keras-RCNN but I'm having issues. I tried building the dock the generated file does not look correct. I tried installing it by
\n\npython setup.py build \npython setup.py install\ninstall output here\nand it seems to install. But I can't get the datasets...\nI get
\n\n\n\n\n$ ipython
\n \n
\n Python 3.6.3 |Anaconda custom (64-bit)| (default, Oct 13 2017, 12:02:49)
\n Type 'copyright', 'credits' or 'license' for more information
\n IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help.
\n In 1: import keras_rcnn
\n In [2]: training, validation, test = keras_rcnn.datasets.malaria_phenotypes.load
\n ...: _data()...:
\n \nAttributeError Traceback (most recent call last)
\n \n
\n in ()
\n ----> 1 training, validation, test = keras_rcnn.datasets.malaria_phenotypes.load_data()AttributeError: module 'keras_rcnn' has no attribute 'datasets'
\n
Could you please help?
", "rawMarkdown": "Hello Allen,\nI'm want to try out Keras-RCNN but I'm having issues. I tried building the dock the generated file does not look correct. I tried installing it by\n\n python setup.py build \n python setup.py install\n[install output here][1]\nand it seems to install. But I can't get the datasets...\nI get \n\n> $ ipython \nPython 3.6.3 |Anaconda custom (64-bit)| (default, Oct 13 2017, 12:02:49) \nType 'copyright', 'credits' or 'license' for more information \nIPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help. \nIn [1]: import keras_rcnn \nIn [2]: training, validation, test = keras_rcnn.datasets.malaria_phenotypes.load \n ...: _data() \n ...: \n--------------------------------------------------------------------------- \n> AttributeError Traceback (most recent call last) \n", "votes": 1 }, { "id": 275911, "postDate": "2018-01-30T08:11:07.290Z", "content": "I solved the problem this way
\n\n\n", "rawMarkdown": "I solved the problem this way\n\n> from keras_rcnn.datasets.malaria_phenotypes import load_data \n\n> from keras_rcnn.preprocessing import ObjectDetectionGenerator\n\n> training, validation, test = load_data()\n" }, { "id": 276027, "postDate": "2018-01-30T14:11:18.577Z", "content": "from keras_rcnn.datasets.malaria_phenotypes import load_data
\n \nfrom keras_rcnn.preprocessing import ObjectDetectionGenerator
\n \ntraining, validation, test = load_data()
\n
Thank you!!!!!!
", "rawMarkdown": "Thank you!!!!!! " }, { "id": 276036, "postDate": "2018-01-30T14:45:16.760Z", "content": "For training, do you use coco pretrain model? What is learning rate and dropout probability do you use ? I am using pretrained coco model with learing rate of 1e-4. I think it is better to train with Batch normalization (with moving mean and variance), but I frozen them now. Thanks
", "rawMarkdown": "For training, do you use coco pretrain model? What is learning rate and dropout probability do you use ? I am using pretrained coco model with learing rate of 1e-4. I think it is better to train with Batch normalization (with moving mean and variance), but I frozen them now. Thanks", "votes": 2 }, { "id": 276047, "postDate": "2018-01-30T15:23:07.463Z", "content": "I don’t pre-train with COCO for a variety of reasons. I did, however, use pre-trained weights for convolutional layers inside the ResNet. I also don’t use dropout.
\n\nI used the following hardcoded learning rate schedule with standard SGD:
\n\nAllen, when I heard that you will be averaging the weights, I was confused.
\n\n\n\n\nHowever, I’ve started thinking about an ensemble learning method to ensemble k models into one model (e.g. by averaging their weights).
\n
You cannot average the weights of multiple models together. That does not work. If a filter learned the same representations, it could be in a different order, and that would be horrible for the model. Do you mean averaging the outputs together? If you are concerned for making sure you segment the nuclei properly into multiple instances, could you use intersection instead of the union or average all the masks? Also, I am very interested segmenting the nuclei into multiple pieces to make sure they are separate. I am having a problem with that because my predictions from U-Net are merged nuclei. Could you elaborate on how you will do that, or is there somewhere I can gain some knowledge on this? Thanks
", "rawMarkdown": "Allen, when I heard that you will be averaging the weights, I was confused.\n\n> However, I’ve started thinking about an ensemble learning method to ensemble k models into one model (e.g. by averaging their weights).\n\nYou cannot average the weights of multiple models together. That does not work. If a filter learned the same representations, it could be in a different order, and that would be horrible for the model. Do you mean averaging the outputs together? If you are concerned for making sure you segment the nuclei properly into multiple instances, could you use intersection instead of the union or average all the masks? Also, I am very interested segmenting the nuclei into multiple pieces to make sure they are separate. I am having a problem with that because my predictions from U-Net are merged nuclei. Could you elaborate on how you will do that, or is there somewhere I can gain some knowledge on this? Thanks", "votes": 4 }, { "id": 276829, "postDate": "2018-02-01T16:00:40.810Z", "content": "I misspoke. Indeed, I meant averaging their outputs. ☺️
\n\nIf you’re open to using a supplementary model, you could do one of many things. For example:
\n\nYou could train an additional FCNN to predict a boundary and subtract the boundary from the two-class prediction predicted by your existing FCNN.
You could train a three-class FCNN to predict a “foreground,” “background,” and “boundary” class.
You could use a classical method like distance transforming your predicted mask and using watershed to isolate the instances.
I also wanted to reiterate that I’m not using an FCNN so I’m not encountering these problems. I’m predicting multiple object masks rather than one image mask.
", "rawMarkdown": "I misspoke. Indeed, I meant averaging their outputs. ☺️\n\nIf you’re open to using a supplementary model, you could do one of many things. For example:\n\n* You could train an additional FCNN to predict a boundary and subtract the boundary from the two-class prediction predicted by your existing FCNN.\n\n* You could train a three-class FCNN to predict a “foreground,” “background,” and “boundary” class. \n\n* You could use a classical method like distance transforming your predicted mask and using watershed to isolate the instances. \n\nI also wanted to reiterate that I’m not using an FCNN so I’m not encountering these problems. I’m predicting multiple object masks rather than one image mask. ", "votes": 2 }, { "id": 277581, "postDate": "2018-02-03T16:15:44.917Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 277671, "postDate": "2018-02-03T22:07:15.577Z", "content": "Hi, excuse my brevity. I’m writing from my phone. I’ll write a more detailed response when I’m home.
\n\nUnfortuantly, the NaN issue is a “known issue.” I (and others) have been investigating this problem for a few weeks without success. Hopefully it’ll be resolved soon. Until it’s fixed you can use the Keras NaN callback to stops and restart training.
\n\nYou can make a prediction just like any other Keras model. If you use predict_generator remember to make the shuffle parameter False. I should add that I already converted the DSB data. You can find that in the DSB2018 PR. You should also take a look at the generator PR I opened last week. It has a number of helpful augmentation steps.
", "rawMarkdown": "Hi, excuse my brevity. I’m writing from my phone. I’ll write a more detailed response when I’m home. \n\nUnfortuantly, the NaN issue is a “known issue.” I (and others) have been investigating this problem for a few weeks without success. Hopefully it’ll be resolved soon. Until it’s fixed you can use the Keras NaN callback to stops and restart training. \n\nYou can make a prediction just like any other Keras model. If you use predict_generator remember to make the shuffle parameter False. I should add that I already converted the DSB data. You can find that in the DSB2018 PR. You should also take a look at the generator PR I opened last week. It has a number of helpful augmentation steps. ", "votes": 2 }, { "id": 279080, "postDate": "2018-02-07T10:10:26.173Z", "content": "Hi Allen, I want to ask you the question about parameter setting:\n1. Do you train with batch normalization? If yes, what is your batch size? I am using batch size of 4 and random cropping image size of 128x128\n2. What is your ANCHOR SCALES? My case is (8, 16, 32, 64,128)\n3. If you crop the image into 128x128. In the inference you also need to crop the input image into 128x128 sub-images, because MaskRCNN used the fully connected layer in detection, it does not allow to arbitrary size as FCN. If I right, how you create final segmented image from segmented of sub-images\nThanks!
", "rawMarkdown": "Hi Allen, I want to ask you the question about parameter setting:\n1. Do you train with batch normalization? If yes, what is your batch size? I am using batch size of 4 and random cropping image size of 128x128\n2. What is your ANCHOR SCALES? My case is (8, 16, 32, 64,128)\n3. If you crop the image into 128x128. In the inference you also need to crop the input image into 128x128 sub-images, because MaskRCNN used the fully connected layer in detection, it does not allow to arbitrary size as FCN. If I right, how you create final segmented image from segmented of sub-images\nThanks!", "votes": 1 }, { "id": 280275, "postDate": "2018-02-09T17:15:11.723Z", "content": "Hi Allen, you said we can find augmentation steps in the DSB2018 PR. However, I cannot find the this post. Would you please post a link for the data augmentation? Thank you very much.
", "rawMarkdown": "Hi Allen, you said we can find augmentation steps in the DSB2018 PR. However, I cannot find the this post. Would you please post a link for the data augmentation? Thank you very much.", "votes": 1 }, { "id": 280383, "postDate": "2018-02-09T21:23:54.930Z", "content": "Hi, you should track the following PR: https://github.com/broadinstitute/keras-rcnn/pull/164/files
", "rawMarkdown": "Hi, you should track the following PR: https://github.com/broadinstitute/keras-rcnn/pull/164/files", "votes": 1 }, { "id": 280600, "postDate": "2018-02-10T12:12:29Z", "content": "Hi allen. What is your ANCHOR SCALES? I am using (8, 16, 32, 64,128) . And how you achieve instance segmentation in testing set if the input is 128x128, while the testing image size is 1024x1024? Thanks!
", "rawMarkdown": "Hi allen. What is your ANCHOR SCALES? I am using (8, 16, 32, 64,128) . And how you achieve instance segmentation in testing set if the input is 128x128, while the testing image size is 1024x1024? Thanks!" } ] }, { "id": 283778, "postDate": "2018-02-16T00:25:49.080Z", "content": "Hi allen,
\n\nThanks for sharing.
\n\nCould you also sharing the score with different method? for example pure unet:0.2, with data augumentation 0.3, with resnxt50 0.35, pure mask rcnn 0.5, with data augumentation 0.6
\n\nHere i did not mean unet will be worse than maskrcnn. Just from the public kernel, most people start with unet get score below 0.3.\nThe architecture of unet is very simple compared with faster rcnn(maybe I should use the word beautiful). This maybe the reason lots of people choose it to start.
", "rawMarkdown": "Hi allen,\n\nThanks for sharing.\n\nCould you also sharing the score with different method? for example pure unet:0.2, with data augumentation 0.3, with resnxt50 0.35, pure mask rcnn 0.5, with data augumentation 0.6\n\nHere i did not mean unet will be worse than maskrcnn. Just from the public kernel, most people start with unet get score below 0.3.\nThe architecture of unet is very simple compared with faster rcnn(maybe I should use the word beautiful). This maybe the reason lots of people choose it to start.", "votes": 2, "replies": [ { "id": 289235, "postDate": "2018-03-01T07:55:27.567Z", "content": "hello, see you again.
", "rawMarkdown": "hello, see you again." }, { "id": 295732, "postDate": "2018-03-14T04:57:59.593Z", "content": "Because there is a kernel has already inferenced the U-Net, but not Mask-RCNN. hhhh
", "rawMarkdown": "Because there is a kernel has already inferenced the U-Net, but not Mask-RCNN. hhhh" } ] }, { "id": 282856, "postDate": "2018-02-14T17:50:43.060Z", "content": "Hi Allen,
\nHave you cleaned the data before training your model?
Today I have been visualizing the annotated masks and the quality is so terrible that your current score seems like magic.
", "rawMarkdown": "Hi Allen, \nHave you cleaned the data before training your model?\n\nToday I have been visualizing the annotated masks and the quality is so terrible that your current score seems like magic.", "votes": 2, "replies": [ { "id": 282959, "postDate": "2018-02-14T20:09:58.787Z", "content": "Yeah, I did a few things.
\n\nI’ll briefly summarize the process, but let me know if you want to know more about a particular topic.
\n\nMy cleanup process was painful but straightforward:
\n\nI removed the alpha channel from the annotations and masks.
I fixed the problems that were outlined in the “data quality issues” thread (I did this a few weeks ago and there’s been a few more problems that popped up)
I closed small holes in objects. I believe I used a (1, 1) structuring element. I also spot checked every object to ensure it was a mistake (e.g. or it seemed like a mistake to my eye).
I also supplemented the training set in an attempt to standardize annotation disagreements:
\n\nI computed a distance transform on each of the masks (not the flattened label matrix) and found the maximum peak. I computed a watershed segmentation for each masked image (that was dilated with a (1, 1) structuring element) using the peaks as markers. I added this new set of masks to the existing set.
I took the centroid of each mask and used the random walker algorithm to compute a third set of masks using the aforementioned masked images. I then used the same watershed process from before.
Finally, I created a fourth and fifth set of annotations by using a geodesic active contour with two different edge preserved responses (from a median filter and an inverse Gaussian filter). I then used the same watershed process from before.
I used the supplemental data like this:
\n\nI took the initial set of training data and made a 80%/20% split. I repeated this process five times to create five training sets and five validation sets.
Each epoch I would randomly swap each annotation for one of the five buckets (the real training data, the watershed data, the random walker and watershed data, and the two buckets of geodesic and watershed data) so each epoch would only see one image once.
This is an area that I believe can be improved further with more time.
\n\nI augmented the data like this:
\n\nrandom scaled down version of the annotation and image (with nearest neighbor and bilinear respectively)
random crop
random horizontal or vertical flip
semi-random color perturbation (it’d randomly shift colors to fall into one of three categories of color features, “blue,” “blue and yellow,” and “grayscale”).
random blurriness (or focus)
random (shot or Poisson) noise
I also did some mild post-processing on the masks. I’d like to start experimenting with a refinement step (e.g. ideas from the Sharp Mask paper), but I don’t think it’ll improve much because of the aforementioned annotation disagreement.
\n\nI didn’t do anything to the bounding box predictions besides non-maximum suppression (which I should write a lengthy post about since it’s probably the most important step in the entire process).
", "rawMarkdown": "Yeah, I did a few things. \n\nI’ll briefly summarize the process, but let me know if you want to know more about a particular topic.\n\nMy cleanup process was painful but straightforward:\n\n* I removed the alpha channel from the annotations and masks.\n\n* I fixed the problems that were outlined in the “data quality issues” thread (I did this a few weeks ago and there’s been a few more problems that popped up)\n\n* I closed small holes in objects. I believe I used a (1, 1) structuring element. I also spot checked every object to ensure it was a mistake (e.g. or it seemed like a mistake to my eye).\n\nI also supplemented the training set in an attempt to standardize annotation disagreements:\n\n* I computed a distance transform on each of the masks (not the flattened label matrix) and found the maximum peak. I computed a watershed segmentation for each masked image (that was dilated with a (1, 1) structuring element) using the peaks as markers. I added this new set of masks to the existing set.\n\n* I took the centroid of each mask and used the random walker algorithm to compute a third set of masks using the aforementioned masked images. I then used the same watershed process from before. \n\n* Finally, I created a fourth and fifth set of annotations by using a geodesic active contour with two different edge preserved responses (from a median filter and an inverse Gaussian filter). I then used the same watershed process from before.\n\nI used the supplemental data like this:\n\n* I took the initial set of training data and made a 80%/20% split. I repeated this process five times to create five training sets and five validation sets. \n\n* Each epoch I would randomly swap each annotation for one of the five buckets (the real training data, the watershed data, the random walker and watershed data, and the two buckets of geodesic and watershed data) so each epoch would only see one image once. \n\nThis is an area that I believe can be improved further with more time.\n\nI augmented the data like this:\n\n* random scaled down version of the annotation and image (with nearest neighbor and bilinear respectively)\n\n* random crop\n\n* random horizontal or vertical flip\n\n* semi-random color perturbation (it’d randomly shift colors to fall into one of three categories of color features, “blue,” “blue and yellow,” and “grayscale”).\n\n* random blurriness (or focus)\n\n* random (shot or Poisson) noise\n\nI also did some mild post-processing on the masks. I’d like to start experimenting with a refinement step (e.g. ideas from the Sharp Mask paper), but I don’t think it’ll improve much because of the aforementioned annotation disagreement.\n\nI didn’t do anything to the bounding box predictions besides non-maximum suppression (which I should write a lengthy post about since it’s probably the most important step in the entire process).", "votes": 26 }, { "id": 282963, "postDate": "2018-02-14T20:15:06.427Z", "content": "Thanks a lot for your detailed answer. That's a very interesting approach.
", "rawMarkdown": "Thanks a lot for your detailed answer. That's a very interesting approach.", "votes": 1 }, { "id": 282980, "postDate": "2018-02-14T20:31:25.790Z", "content": "thank you very much, certainly an easier approach than redoing the masks manually :) (what I did for some of the obviously \"wrong\" ones, i.e. mostly the oversized masks in some images etc.)
", "rawMarkdown": "thank you very much, certainly an easier approach than redoing the masks manually :) (what I did for some of the obviously \"wrong\" ones, i.e. mostly the oversized masks in some images etc.)", "votes": 1 }, { "id": 282982, "postDate": "2018-02-14T20:31:46.640Z", "content": "No problem. I hope you don’t hesitate to keep reaching out. I’m here to help (or even spitball). 😎
", "rawMarkdown": "No problem. I hope you don’t hesitate to keep reaching out. I’m here to help (or even spitball). 😎 \n", "votes": 2 }, { "id": 283276, "postDate": "2018-02-15T06:22:42.370Z", "content": "Wow! Thank you, Allen! I learned a lot from your information.
", "rawMarkdown": "Wow! Thank you, Allen! I learned a lot from your information." }, { "id": 283325, "postDate": "2018-02-15T09:33:32.307Z", "content": "Hi, Allen!
\n\nAre 5 different models from cross-validation just averaging to make final prediction?
", "rawMarkdown": "Hi, Allen!\n\nAre 5 different models from cross-validation just averaging to make final prediction?", "votes": 1 }, { "id": 287555, "postDate": "2018-02-25T06:54:02.917Z", "content": "Hi Allen,\nI am trying to supplement the dataset like you did with creating additional annotations. I have one doubt about how did you create the masked image. Here is what I understood
\n\nFor a single image:
\n\nyou take the distance transform of each mask and find the peak using local maxima, and you create a marker with these peaks, now you merge all the masks and create a single mask( is this what you mean by masked image?) and create a distance transform of this merged mask and using the marker and the distance transform you do watershed segmentation.
", "rawMarkdown": "Hi Allen,\nI am trying to supplement the dataset like you did with creating additional annotations. I have one doubt about how did you create the masked image. Here is what I understood \n\nFor a single image:\n\nyou take the distance transform of each mask and find the peak using local maxima, and you create a marker with these peaks, now you merge all the masks and create a single mask( is this what you mean by masked image?) and create a distance transform of this merged mask and using the marker and the distance transform you do watershed segmentation. ", "votes": 2 }, { "id": 292430, "postDate": "2018-03-08T00:15:56.593Z", "content": "Hi Allen
\n\n\n\n\n\n \nI closed small holes in objects. I believe I used a (1, 1) structuring element. I also spot checked every object to ensure it was\na mistake (e.g. or it seemed like a mistake to my eye).
\n
I did dilation with (1,1) structuring element and it has no effect on the mask, is this due to implementation details of different libraries or am I missing something. Thanks in advance, you notebook has been very helpful.
", "rawMarkdown": "Hi Allen \n\n> I closed small holes in objects. I believe I used a (1, 1) structuring element. I also spot checked every object to ensure it was\n> a mistake (e.g. or it seemed like a mistake to my eye).\n\nI did dilation with (1,1) structuring element and it has no effect on the mask, is this due to implementation details of different libraries or am I missing something. Thanks in advance, you notebook has been very helpful.", "votes": 1 }, { "id": 298208, "postDate": "2018-03-19T03:29:33.243Z", "content": "Dear Allen:\n I try out to utilize morphological_geodesic_active_contour and inverse_gaussian_gradient API in python to achieve the external annotations you mentioned. However, I found it difficult to achieve a good result for these two algorithm. Let me describe the processing:\n1. load the each of sub_mask for one image, each sub_mask represent one cell.\n2. sub_image=dilation(sub_image,square(3))\n3. sub_image = inverse_gaussian_gradient(sub_image)\n4. sub_mask = morphological_geodesic_active_contour(sub_image, 50, init,\n smoothing=3, balloon=-1,\n threshold=0.5)\nThe final masks are so bad, I missed a lot of sub_mask. So what's your hyperparameters you used. By the way, I think your external annotation's aim is standardize the boundary. So your external annotation is a little overestimated for mask compared with original mask. Is it right?
", "rawMarkdown": "Dear Allen:\n I try out to utilize morphological_geodesic_active_contour and inverse_gaussian_gradient API in python to achieve the external annotations you mentioned. However, I found it difficult to achieve a good result for these two algorithm. Let me describe the processing:\n1. load the each of sub_mask for one image, each sub_mask represent one cell.\n2. sub_image=dilation(sub_image,square(3))\n3. sub_image = inverse_gaussian_gradient(sub_image)\n4. sub_mask = morphological_geodesic_active_contour(sub_image, 50, init,\n smoothing=3, balloon=-1,\n threshold=0.5)\nThe final masks are so bad, I missed a lot of sub_mask. So what's your hyperparameters you used. By the way, I think your external annotation's aim is standardize the boundary. So your external annotation is a little overestimated for mask compared with original mask. Is it right? ", "votes": 1 }, { "id": 298653, "postDate": "2018-03-19T21:01:13.297Z", "content": "I had pretty good results following this general outline Alan provided using cv2 and skimage functions.
\n\nMy steps were:
\n\nwatershed
\n\nrandom_walk
\n\nactive_contours
\n\nDepending on parameter choices masks can be slightly larger or smaller than ground truth masks, with slightly different shapes. I am just beginning to substitute into my training loop, so remains to be seen whether LB improvement is to come.
", "rawMarkdown": "I had pretty good results following this general outline Alan provided using cv2 and skimage functions. \n\nMy steps were:\n\n**watershed**\n\n - dilate with 2x2 kernel\n - calculate distance transform and threshold\n - create markers with cv2.connectedComponents\n - compute the mask with watershed\n\n**random_walk**\n\n- dilate with 2x2 kernel\n- calculate distance transform\n- calculate centroid of distance transform\n- generate markers relative to the centroid pixel value\n- generate mask candidate using markers with random_walker\n- apply watershed from above\n\n**active_contours**\n\n- find centroid of mask\n- gaussian blur (2 pixels) or median blur (2x2 kernel)\n- draw candidate bounding circle around mask\n- apply active contours\n- convert returned contour with grid_points_in_poly to a mask\n- apply watershed from above\n\nDepending on parameter choices masks can be slightly larger or smaller than ground truth masks, with slightly different shapes. I am just beginning to substitute into my training loop, so remains to be seen whether LB improvement is to come.\n", "votes": 4 }, { "id": 298659, "postDate": "2018-03-19T21:14:57.053Z", "content": "Appreciate for your detailed reply. I think standard external dataset must have benefit for LB. By the way, your mask images are a single mask which foreground are only 1 or the multi-mask which foreground start from 1? If you have good results, please tell me :)
", "rawMarkdown": "Appreciate for your detailed reply. I think standard external dataset must have benefit for LB. By the way, your mask images are a single mask which foreground are only 1 or the multi-mask which foreground start from 1? If you have good results, please tell me :) " }, { "id": 311790, "postDate": "2018-04-10T17:06:34.347Z", "content": "if augmentation is used to obtain translational invariance in a CNN then what you doing is creating mask method invariance. Should i assume the \"ground truth masks were obtained with a labeled tool? and/or they were manual generated. (probably answered somewhere); the latter is preferable as we are trying to beat human \"expert\" performance. what you are doing now, IMHO, is how i would approach it in the real world, as this would allow to generalize/ improve the CNN or add whatever comes along with improvements in image segmentation and then test them. now that i think about it, ... oh never mind .. i digress...
", "rawMarkdown": "if augmentation is used to obtain translational invariance in a CNN then what you doing is creating mask method invariance. Should i assume the \"ground truth masks were obtained with a labeled tool? and/or they were manual generated. (probably answered somewhere); the latter is preferable as we are trying to beat human \"expert\" performance. what you are doing now, IMHO, is how i would approach it in the real world, as this would allow to generalize/ improve the CNN or add whatever comes along with improvements in image segmentation and then test them. now that i think about it, ... oh never mind .. i digress...", "votes": 1 } ] }, { "id": 278033, "postDate": "2018-02-05T07:10:17.750Z", "content": "Hi Allen,\nthanks for sharing!!!\nA doubt about overfitting: do you just experience a gap between training and validation or also between your validation and submission scores? I ask you because I'm mimicking the competition score function but my validation score is always 0.1 higher than the related submission score (which is A LOT), although almost coherent with it.\nThanks in advance!!!
", "rawMarkdown": "Hi Allen,\nthanks for sharing!!!\nA doubt about overfitting: do you just experience a gap between training and validation or also between your validation and submission scores? I ask you because I'm mimicking the competition score function but my validation score is always 0.1 higher than the related submission score (which is A LOT), although almost coherent with it.\nThanks in advance!!!", "votes": 2, "replies": [ { "id": 278195, "postDate": "2018-02-05T16:00:06.837Z", "content": "Interesting. 0.1 is more than I’ve seen. The differences I’ve seen range between 0.049 and 0.074. How are you evaluating? SRS of the training data?
", "rawMarkdown": "Interesting. 0.1 is more than I’ve seen. The differences I’ve seen range between 0.049 and 0.074. How are you evaluating? SRS of the training data?", "votes": 1 }, { "id": 278251, "postDate": "2018-02-05T18:10:40.553Z", "content": "Actually the difference is between 0.07 and 0.1, always positive (validation score > submission score). This is strange because the valid data is not used at all during the training (if not to select a good epoch to use, but this cannot explain the big difference in score). I'll search my score function for a bias (or a bug :-)).
", "rawMarkdown": "Actually the difference is between 0.07 and 0.1, always positive (validation score > submission score). This is strange because the valid data is not used at all during the training (if not to select a good epoch to use, but this cannot explain the big difference in score). I'll search my score function for a bias (or a bug :-)).", "votes": 1 }, { "id": 294230, "postDate": "2018-03-11T15:49:41.400Z", "content": "@Andrea Rapuzzi did you find the reason? I have the same problem, I thought because of my simple way to split train/valid (random split 0.9-0.1)
", "rawMarkdown": "@Andrea Rapuzzi did you find the reason? I have the same problem, I thought because of my simple way to split train/valid (random split 0.9-0.1)", "votes": 1 }, { "id": 294236, "postDate": "2018-03-11T15:58:55.520Z", "content": "the correct way to do validation is:
\n\ndivide training set into cell types. e.g. set A,B,C,D, ...E However, we do not have the meta data. The closest guess is to cluster the train images based on size, nuclei size, k-means, etc
then train on e.g. A,B,C and validate on D,E
the test LB set may be of different cell types from the train
", "rawMarkdown": "the correct way to do validation is: \n\n- divide training set into cell types. e.g. set A,B,C,D, ...E However, we do not have the meta data. The closest guess is to cluster the train images based on size, nuclei size, k-means, etc\n\n- then train on e.g. A,B,C and validate on D,E\n\nthe test LB set may be of different cell types from the train\n", "votes": 2 }, { "id": 294248, "postDate": "2018-03-11T16:23:21.437Z", "content": "Thanks for your advice
", "rawMarkdown": "Thanks for your advice" }, { "id": 294590, "postDate": "2018-03-12T07:36:56.467Z", "content": "Hi @QuangTeo, since then I've changed the train-validation split (with a StratifiedShuffle based on the classes provided by Allen) and I've applied some refactoring to my code (perhaps eliminating some bias, but I'm not sure). Now my gap between validation and test score is around 0.5-0.7 which seems fine given the differences in the sets.
", "rawMarkdown": "Hi @QuangTeo, since then I've changed the train-validation split (with a StratifiedShuffle based on the classes provided by Allen) and I've applied some refactoring to my code (perhaps eliminating some bias, but I'm not sure). Now my gap between validation and test score is around 0.5-0.7 which seems fine given the differences in the sets.", "votes": 2 }, { "id": 294754, "postDate": "2018-03-12T15:26:23.243Z", "content": "Thanks, I think train-validation split method can be important in this competition, It reminds me of the Fish competition since the test data may contain many different types of images.
", "rawMarkdown": "Thanks, I think train-validation split method can be important in this competition, It reminds me of the Fish competition since the test data may contain many different types of images.", "votes": 1 } ] }, { "id": 277195, "postDate": "2018-02-02T13:10:10.703Z", "content": "Allen, how much did it take to train your current score? I mean in terms of learning time of rcnn / epochs / steps per epoch ? Just want to have some time benchmark. Thank you!
", "rawMarkdown": "Allen, how much did it take to train your current score? I mean in terms of learning time of rcnn / epochs / steps per epoch ? Just want to have some time benchmark. Thank you!\n", "votes": 2, "replies": [ { "id": 277305, "postDate": "2018-02-02T18:53:15.790Z", "content": "Hi, Maksimov: It took a “weekend.” It’s still not trivial to train so there was a lot of starting, stopping, and even restarting. If it was a perfect process, it would’ve taken ~16 hours using my Mask RCNN Implementation (that’s presently two-to-three times slower than what FAIR has reported about their implementation). I used the training schedule from the paper but I decreased the learning rates by -10^2.
", "rawMarkdown": "Hi, Maksimov: It took a “weekend.” It’s still not trivial to train so there was a lot of starting, stopping, and even restarting. If it was a perfect process, it would’ve taken ~16 hours using my Mask RCNN Implementation (that’s presently two-to-three times slower than what FAIR has reported about their implementation). I used the training schedule from the paper but I decreased the learning rates by -10^2.\n\n", "votes": 6 }, { "id": 277805, "postDate": "2018-02-04T11:51:48.237Z", "content": "Thanks Allen! I also decide to learn from scratch without pre trained model on COCO dataset, I launched the training on Friday and it should take about 25 hours of training using single 1070 card. Byt the way, which hardware do you use?
", "rawMarkdown": "Thanks Allen! I also decide to learn from scratch without pre trained model on COCO dataset, I launched the training on Friday and it should take about 25 hours of training using single 1070 card. Byt the way, which hardware do you use?", "votes": 3 }, { "id": 277941, "postDate": "2018-02-04T21:34:50.703Z", "content": "I have two 1080s and two Titan Xs in my machine. However, I typically only use one of the cards for training.
", "rawMarkdown": "I have two 1080s and two Titan Xs in my machine. However, I typically only use one of the cards for training.", "votes": 2 } ] }, { "id": 273250, "postDate": "2018-01-24T11:03:32.553Z", "content": "Can you pls share your approach for train set preparation? what technique helped you the most (load as grey / resize/ slicing / etc)?
", "rawMarkdown": "Can you pls share your approach for train set preparation? what technique helped you the most (load as grey / resize/ slicing / etc)?", "votes": 2, "replies": [ { "id": 273430, "postDate": "2018-01-24T15:56:25.437Z", "content": "Yeah! I plan on releasing and discussing everything. However, I haven’t done any data preparation outside of restructuring the training data. I plan on working on this over the weekend or next week sometime.
", "rawMarkdown": "Yeah! I plan on releasing and discussing everything. However, I haven’t done any data preparation outside of restructuring the training data. I plan on working on this over the weekend or next week sometime.\n", "votes": 4 }, { "id": 274225, "postDate": "2018-01-26T04:30:50.193Z", "content": "Awesome^_^
", "rawMarkdown": "Awesome^_^" } ] }, { "id": 273054, "postDate": "2018-01-24T04:10:15.813Z", "content": "@Allen Goodman
\n\nyou current LB score 0.516 is impressive. Any hints on the methods you used?
", "rawMarkdown": "@Allen Goodman\n\nyou current LB score 0.516 is impressive. Any hints on the methods you used?\n", "votes": 2, "replies": [ { "id": 273426, "postDate": "2018-01-24T15:52:29.750Z", "content": "> you current LB score 0.516 is impressive. Any hints on the methods you\n> used?
\n\nThanks! I’ll write more about this today or tomorrow, but, briefly, I’m using a Mask RCNN-like architecture available from Keras-RCNN. You’re more than welcome to use my code! I should note that this is without any pre-processing (or augmentation), modifications to the architecture (e.g. an ensemble CNN), sophisticated sampling, etc. There’s a lot of room for improvement!
", "rawMarkdown": "> you current LB score 0.516 is impressive. Any hints on the methods you\n> used?\n\nThanks! I’ll write more about this today or tomorrow, but, briefly, I’m using a Mask RCNN-like architecture available from [Keras-RCNN][1]. You’re more than welcome to use my code! I should note that this is without any pre-processing (or augmentation), modifications to the architecture (e.g. an ensemble CNN), sophisticated sampling, etc. There’s a lot of room for improvement!\n\n [1]: https://github.com/broadinstitute/keras-rcnn", "votes": 15 }, { "id": 273490, "postDate": "2018-01-24T16:43:44.903Z", "content": "Thanks! Seems that we are thinking about the same thing, but i am missing some details.
\n\nCurrently, I am using pytorch framework. I use 2 stage network (not end-to-end)
\n\nThis step will guarantee the TP, FP and precision accuracy of my solution.
\n\nbased on estimated nuclei radius, i can resize the whole image to a \"standard size\".
Then i have another fully convolution network read the \"standard size\" image and the nuclei center candidates and predict if each pixel in the 32x32 region centering a \"nuclei center\" is part of the nuclei or not (aka. instance segmentation here). I want to avoid the cropping by using a output channels of 32x32=1024.
This step will ensure i have the max iou for threshold 0.5 to 0.9.
\n\nLooking forward your solution to fill in my missing details! Thanks again!
", "rawMarkdown": "Thanks! Seems that we are thinking about the same thing, but i am missing some details.\n\nCurrently, I am using pytorch framework. I use 2 stage network (not end-to-end)\n\n# 1. detect nuclei center #\n\n- construct 256x256 input unet with 128x128 output. For each pixel output, i estimate if it is nuclei center or not and the estimated nuclei radius.\n\nThis step will guarantee the TP, FP and precision accuracy of my solution.\n\n# 2. segment nuclei #\n\n- based on estimated nuclei radius, i can resize the whole image to a \"standard size\".\n\n- Then i have another fully convolution network read the \"standard size\" image and the nuclei center candidates and predict if each pixel in the 32x32 region centering a \"nuclei center\" is part of the nuclei or not (aka. instance segmentation here). I want to avoid the cropping by using a output channels of 32x32=1024.\n\nThis step will ensure i have the max iou for threshold 0.5 to 0.9.\n\n\nLooking forward your solution to fill in my missing details! Thanks again!", "votes": 6 }, { "id": 273514, "postDate": "2018-01-24T17:55:51.307Z", "rawMarkdown": "", "votes": 1 }, { "id": 273545, "postDate": "2018-01-24T18:46:55.677Z", "content": "The architecture I’m using is less focused on segmentation. Here’s a brief outline:
\n\nI use a convolutional neural network (CNN) to extract image features.
I use a region proposal network (RPN) to extract region proposals. The RPN has two outputs: bounding boxes and an “objectness” score that estimates whether the bounding box encloses an object.
I use a region-based convolutional neural network (RCNN) to pool the aforementioned region proposals and pass the pooled region proposals and the image features estimated by the CNN into three sub-networks: a regression network that regresses an object’s bounding box coordinates, a classification network that estimates the likelihood the object’s bounding box contained a particular class (I use two classes “nucleus” and “background”), and, finally, a second classification network that classifies each pixel into the classes predicted by the first classification network.
My solution, however, requires a few additional steps like non-maximum suppression and mask refinement (presently that’s just a morphological dilation). I should note that this post-processing is critical to the success of the architecture and these two steps (non-maximum suppression and mask refinement) will probably require the most work from an architectural perspective. I made a mistake with the dilation that was responsible for a difference of 0.4 and 0.5 in my leaderboard score!
", "rawMarkdown": "The architecture I’m using is less focused on segmentation. Here’s a brief outline:\n\n* I use a convolutional neural network (CNN) to extract image features.\n\n* I use a region proposal network (RPN) to extract region proposals. The RPN has two outputs: bounding boxes and an “objectness” score that estimates whether the bounding box encloses an object.\n\n* I use a region-based convolutional neural network (RCNN) to pool the aforementioned region proposals and pass the pooled region proposals and the image features estimated by the CNN into three sub-networks: a regression network that regresses an object’s bounding box coordinates, a classification network that estimates the likelihood the object’s bounding box contained a particular class (I use two classes “nucleus” and “background”), and, finally, a second classification network that classifies each pixel into the classes predicted by the first classification network. \n\nMy solution, however, requires a few additional steps like non-maximum suppression and mask refinement (presently that’s just a morphological dilation). I should note that this post-processing is critical to the success of the architecture and these two steps (non-maximum suppression and mask refinement) will probably require the most work from an architectural perspective. I made a mistake with the dilation that was responsible for a difference of 0.4 and 0.5 in my leaderboard score!", "votes": 18 }, { "id": 273570, "postDate": "2018-01-24T20:23:14.743Z", "content": "For anyone like me new to R-CNN, here is a good blog post describing the architecture in general: https://tryolabs.com/blog/2018/01/18/faster-r-cnn-down-the-rabbit-hole-of-modern-object-detection/
", "rawMarkdown": "For anyone like me new to R-CNN, here is a good blog post describing the architecture in general: https://tryolabs.com/blog/2018/01/18/faster-r-cnn-down-the-rabbit-hole-of-modern-object-detection/", "votes": 19 }, { "id": 273615, "postDate": "2018-01-24T21:50:30.513Z", "content": "Thank you
", "rawMarkdown": "Thank you" }, { "id": 273764, "postDate": "2018-01-25T05:21:16.927Z", "content": "High, I am also trying to use mask R-CNN. The pretrained models available are mostly based on resnet101 and fpn. Is it too large for this task?
", "rawMarkdown": "High, I am also trying to use mask R-CNN. The pretrained models available are mostly based on resnet101 and fpn. Is it too large for this task?", "votes": 2 }, { "id": 273837, "postDate": "2018-01-25T09:08:07.087Z", "content": "Are you use any external data, or only augmentation?
", "rawMarkdown": "Are you use any external data, or only augmentation?", "votes": 1 }, { "id": 273969, "postDate": "2018-01-25T15:45:21.303Z", "content": "No. I don’t plan on using external data (with mask annotations) since it’d take way too much time. I’m considering annotating additional bounding boxes but so far (and without the better evaluation scheme I outlined) my model is localizing objects well.
\n\nI will, however, spend a lot of time experimenting with data augmentation.
", "rawMarkdown": "No. I don’t plan on using external data (with mask annotations) since it’d take way too much time. I’m considering annotating additional bounding boxes but so far (and without the better evaluation scheme I outlined) my model is localizing objects well.\n\nI will, however, spend a lot of time experimenting with data augmentation.", "votes": 2 }, { "id": 275484, "postDate": "2018-01-29T09:23:00.363Z", "content": "@Allen Goodman, do you use pre-trained version of R-CNN? do you train it only on the competition data? other external data?
", "rawMarkdown": "@Allen Goodman, do you use pre-trained version of R-CNN? do you train it only on the competition data? other external data? ", "votes": 1 }, { "id": 281122, "postDate": "2018-02-12T03:27:03.630Z", "content": "Why mask refinement like morphological dilation is necessary?
", "rawMarkdown": "Why mask refinement like morphological dilation is necessary? ", "votes": 2 }, { "id": 281542, "postDate": "2018-02-12T20:46:33.400Z", "rawMarkdown": "", "votes": 2, "isDeleted": true } ] }, { "id": 306604, "postDate": "2018-03-30T18:23:17.027Z", "content": "By the way, your github profile picture is fantastic!!
", "rawMarkdown": "By the way, your github profile picture is fantastic!!" }, { "id": 280749, "postDate": "2018-02-10T21:55:18.260Z", "content": "Hi, I've been attempting the rcnn getting started tutorial from the readme of rcnn github but have had issues are follows:
\n\nIssues I have managed to resolve:
\n\nThere were no imports provided so I added those required.
I seemed to be missing malaria_phenotypes.py from the pip install but just added to the appropriate place manually.
Issues I have not yet resolved:
\n\n-In python 2 it fails at the following line (complaining about the number of arguments):
\n\n\n\n\nmodel = keras_rcnn.models.RCNN(image, classes=len(classes) + 1)
\n
-In python 3.5 and 3.6 on different versions of tensorflow it crashes on the line:
\n\n\n\n\nfrom keras_rcnn.preprocessing import ObjectDetectionGenerator
\n
Has anyone got the tutorial working? If so are you able to share any tips or share a full .py file with a list of versions you are using?
\n\nReally grateful for any help anyone can offer!
", "rawMarkdown": "Hi, I've been attempting the [rcnn getting started tutorial][1] from the readme of rcnn github but have had issues are follows: \n\nIssues I have managed to resolve:\n\n- There were no imports provided so I added those required.\n\n- I seemed to be missing malaria_phenotypes.py from the pip install but just added to the appropriate place manually.\n\nIssues I have not yet resolved:\n\n-In python 2 it fails at the following line (complaining about the number of arguments):\n\n> model = keras_rcnn.models.RCNN(image, classes=len(classes) + 1)\n\n-In python 3.5 and 3.6 on different versions of tensorflow it crashes on the line:\n\n> from keras_rcnn.preprocessing import ObjectDetectionGenerator\n\nHas anyone got the tutorial working? If so are you able to share any tips or share a full .py file with a list of versions you are using? \n\nReally grateful for any help anyone can offer!\n\n [1]: https://github.com/broadinstitute/keras-rcnn", "replies": [ { "id": 282972, "postDate": "2018-02-14T20:24:00.537Z", "content": "Keras-RCNN is still a work-in-progress. I’d recommend using the following pull request:
\n\nhttps://github.com/broadinstitute/keras-rcnn/pull/164
\n\nor waiting until next week when the next version (0.0.3) is released.
", "rawMarkdown": "Keras-RCNN is still a work-in-progress. I’d recommend using the following pull request:\n\nhttps://github.com/broadinstitute/keras-rcnn/pull/164\n\nor waiting until next week when the next version (0.0.3) is released.\n", "votes": 2 }, { "id": 283147, "postDate": "2018-02-15T00:57:19.377Z", "content": "Brilliant, will do. Thanks.
", "rawMarkdown": "Brilliant, will do. Thanks. " }, { "id": 312204, "postDate": "2018-04-11T11:25:54.573Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 278616, "postDate": "2018-02-06T15:42:43.267Z", "content": "Hi, Allen! Could you share your experience on how to prevent overfitting when training mask rcnn with resnet50 + fpn backbone. The model overfits soon after running about 40-50 epochs, and the validation loss is far more higher than the training loss.
", "rawMarkdown": "Hi, Allen! Could you share your experience on how to prevent overfitting when training mask rcnn with resnet50 + fpn backbone. The model overfits soon after running about 40-50 epochs, and the validation loss is far more higher than the training loss.", "replies": [ { "id": 278631, "postDate": "2018-02-06T16:10:53.790Z", "content": "A few questions:
\n\nIs your validation set a simple random sample of the training images?
Are you training end-to-end (i.e. are you training the CNN, RPN, and RCNN together)?
Which losses are overfitting? All of them?
Thanks for your reply!\n1. Yes, I sampled it randomly from training images.\n2. Yes, I have tried to trained it respectively but seems perform worse than directly end-to-end training.\n3. Almost all. \nMy currently solution is adding dropout after last four stages of resnet, but it seems you didn't encounter such case even without dropout. So I think maybe I missed something.
", "rawMarkdown": "Thanks for your reply!\n1. Yes, I sampled it randomly from training images.\n2. Yes, I have tried to trained it respectively but seems perform worse than directly end-to-end training.\n3. Almost all. \nMy currently solution is adding dropout after last four stages of resnet, but it seems you didn't encounter such case even without dropout. So I think maybe I missed something." }, { "id": 278912, "postDate": "2018-02-07T03:27:00.577Z", "content": "Besides, I found a problem. There are samples that have more than 300 instances. Do you include all the ground truth instances during training, or randomly subsampled them?
", "rawMarkdown": "Besides, I found a problem. There are samples that have more than 300 instances. Do you include all the ground truth instances during training, or randomly subsampled them?", "votes": 1 }, { "id": 282987, "postDate": "2018-02-14T20:38:58.830Z", "content": "Great question. I didn’t intentionally do anything so I think I didn’t encounter this problem because I was randomly cropping (i.e. no image had more than 300 objects). However, you can also increase the maximum number of proposals (I believe it's an ObjectDetection parameter).
", "rawMarkdown": "Great question. I didn’t intentionally do anything so I think I didn’t encounter this problem because I was randomly cropping (i.e. no image had more than 300 objects). However, you can also increase the maximum number of proposals (I believe it's an ObjectDetection parameter)." } ] }, { "id": 274356, "postDate": "2018-01-26T12:20:02.357Z", "content": "hello Allen ,i want to know whether the stage1_sample_submission.csv a perfect mask for train data? i make a csv file by the mask pictuers of train data and use RLE,but the csv file is different from stage1_sample_submission.csv
", "rawMarkdown": "hello Allen ,i want to know whether the stage1_sample_submission.csv a perfect mask for train data? i make a csv file by the mask pictuers of train data and use RLE,but the csv file is different from stage1_sample_submission.csv ", "replies": [ { "id": 274360, "postDate": "2018-01-26T12:30:16.953Z", "content": "stage1_sample_submission.csv contains only one connected component (first pixel) for each test images. It was provided just for better intuition about data format.
", "rawMarkdown": "stage1_sample_submission.csv contains only one connected component (first pixel) for each test images. It was provided just for better intuition about data format.", "votes": 1 }, { "id": 274365, "postDate": "2018-01-26T12:57:45.720Z", "content": "it's very nice of you ,thank you very much ! can you help with another question : if a picture's pixel is [[0 0 0 ],[ 0 255 255],[ 0 0 255]] ,whether it's RLE : 5 2 9 1 ?
", "rawMarkdown": "it's very nice of you ,thank you very much ! can you help with another question : if a picture's pixel is [[0 0 0 ],[ 0 255 255],[ 0 0 255]] ,whether it's RLE : 5 2 9 1 ?" }, { "id": 274684, "postDate": "2018-01-27T03:15:31.737Z", "content": "@Justin Assuming that your matrix rows represent horizontal image lines, top to bottom, then RLE is 5 1 8 2
", "rawMarkdown": "@Justin Assuming that your matrix rows represent horizontal image lines, top to bottom, then RLE is 5 1 8 2", "votes": 3 }, { "id": 274698, "postDate": "2018-01-27T03:23:51.313Z", "content": "Paul ,thanks very much ! i get it from your intuitive explanation .
", "rawMarkdown": "Paul ,thanks very much ! i get it from your intuitive explanation .", "votes": 1 } ] }, { "id": 273002, "postDate": "2018-01-24T02:23:13.157Z", "content": "Thank you for your post, it provided some clues for me to improve my score.
", "rawMarkdown": "Thank you for your post, it provided some clues for me to improve my score." } ], "comments": [ { "id": 272854, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-23T21:01:07.490000", "content": "My immediate observations about the competition metric:
\n\nMy immediate observations about the training set:
\n\nYou'd mentioned \"It’s also unclear if the test set will include entirely new conditions or the same conditions that are present in the training set.\"
\n\nI just made that public:\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/48136#272845
", "votes": 5, "replies": [] }, { "id": 273431, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-24T15:56:38.840000", "content": "Awesome! Thanks, Anne!
", "votes": 0, "replies": [] }, { "id": 274079, "author_name": "ADL", "author_url": "", "post_date": "2018-01-25T21:24:56.213000", "content": "What About Tiramisu .. ! I am trying my hand on that... what Do You Say...
", "votes": 0, "replies": [] }, { "id": 274969, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-27T23:14:52.253000", "content": "I think fully convolutional neural networks (FCNNs) like SegNet, Tiramisu, and U-Net are certainly viable but only when paired with pre-or-post-processing steps that help disconnect object instances.
", "votes": 1, "replies": [] }, { "id": 279287, "author_name": "Jack Vial", "author_url": "", "post_date": "2018-02-07T18:07:50.163000", "content": "Is this still an instance segmentation challenge if all the objects in the image are of the same class? I would think that if two nuclei in the image are touching then the RLE would be the same regardless if instance or semantic segmentation is used.
", "votes": 2, "replies": [] }, { "id": 279600, "author_name": "Human Analog", "author_url": "", "post_date": "2018-02-08T10:24:28.243000", "content": "@jackvial: That was my confusion too, but you submit a different row for each instance. So if you detect 30 different nuclei for image X, then image X has 30 rows in the submission CSV file.
", "votes": 3, "replies": [] }, { "id": 279664, "author_name": "nikita.astronaut", "author_url": "", "post_date": "2018-02-08T13:48:41.777000", "content": "Could anyone please explain the idea behind splitting images into 5-or-more classes? Say, I did it, then what? Train the special network for every class alone? Or feed the predicted class as some additional input to my general model?
\n\nI mean, say, I can predict the class on a test image. How I should use this class then?
", "votes": 2, "replies": [] }, { "id": 280028, "author_name": "DmitryKustikov", "author_url": "", "post_date": "2018-02-09T05:59:21.610000", "content": "Just balancing.
\n\nLets pretend you have 1000 grey-scale images and 100 colorful. You splitting in into 2 parts randomly and it would be a distribution like 950/150 (in validation part lets say 148 grey-scale and ONLY 2 colorful), so you never know how is good you model in case of prediction colorful images. Maybe good or may be terrible? This is why it`s the point to balance validation set. Moreover in test set in could be 800 grey-scale and 300 colorful (your model works bad here and you notice it only after the private leaderboard will be revealed)
", "votes": 3, "replies": [] }, { "id": 280038, "author_name": "delta_K", "author_url": "", "post_date": "2018-02-09T06:28:43.443000", "content": "So how can I fix this even if I know? Can I just augment and resample the 5 different group and make a new more balanced training set?
", "votes": 1, "replies": [] } ] }, { "id": 273531, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-24T18:20:45.887000", "content": "I continued thinking about my validation strategy.
\n\nI accomplished the first task of stratifying the training images into five “modalities” (or modality-like classes). It was a straightforward process. First, a labmate (Hi, Jane!) stratified the images into three classes by the sum of their intensities. The classes were images with white backgrounds, images with black backgrounds, and images with purple foreground and/or purple background. I further stratified the “purple” class into three new classes: images with purple foreground and purple background, images with purple foreground and white background, and images with purple foreground and yellow background.
\n\nHere’re the total number of images per class:
\n\nI’ve attached a spreadsheet (classes.csv) with the filenames for each class. Enjoy! 😎
\n\nA few additional observations:
\n\nI think it’s entirely possible to split the black foreground and white background class into a few additional classes (e.g. fuzzy and non-fuzzy foreground objects) to further ensure generalizability around non-intensity related variables like shape and size (and simplify training by balancing the classes), but the aforementioned five classes seem sufficient to start the validation process. Nevertheless, I plan on revisiting this soon.
\n\nI found a number of images that will be very problematic from an object detection perspective. A few examples of images that will give me nightmares:
\n\nHowever, in my opinion, the most worrying image is this monster:
\n\nThis is exactly the type of image that you use to penalize overfitting.
\n\nIn addition to the modality subsets that I described I’m also going to put together a subset with the aforementioned nightmare images to further punish myself. I’ll probably add a few from the “problems” thread too.
\n\nI really want to ask the biologist in my lab about a few of the modalities since my biology background is limited to college biology and my microscopy knowledge is non-existent. However, I’ve managed to be good. I think they are entertained by my newfound interest in their fields of expertise but disappointed I’m still using phrases like “purple foreground and purple background” to describe something that’s probably significant.
", "votes": 18, "replies": [ { "id": 273751, "author_name": "Paul Jurczak", "author_url": "", "post_date": "2018-01-25T04:30:46.797000", "content": "Speaking of af576e8ec3a8d0b57eb6a311299e9e4fd2047970d3dd9d6f52e54ea6a91109da, are you sure that two objects marked with a red line on attached picture are not nuclea outside of focal range?
", "votes": 1, "replies": [] }, { "id": 273968, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-25T15:42:16.107000", "content": "Honestly, I don’t have an opinion. I’m mostly focused on finding a model that best models the perception of the annotator (or annotators). Essentially I’m assuming the annotations are correct (i.e. uniform).
", "votes": 0, "replies": [] }, { "id": 274783, "author_name": "Ori Tal", "author_url": "", "post_date": "2018-01-27T09:53:13.533000", "content": "Thank you Allen. I went over the classes.csv file and found 735 different entries (for my understanding, this mean 735 different images). But when looking at stage1_train I found only 670 different folders. What is the source of this difference?
", "votes": 1, "replies": [] }, { "id": 274784, "author_name": "Ori Tal", "author_url": "", "post_date": "2018-01-27T10:06:00.420000", "content": "Ok, I just figure out that the file contain the both train and test images.
", "votes": 1, "replies": [] }, { "id": 274968, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-27T23:13:24.270000", "content": "Yep. 😎
\n\nI’ll add an additional column to classify them into phase one training and phase one test.
", "votes": 1, "replies": [] }, { "id": 276998, "author_name": "Oleksandr Bailo", "author_url": "", "post_date": "2018-02-02T00:37:16.267000", "content": "What's the purpose to divide the dataset to aforementioned subclasses? Is it to have balanced batch (i.e. equally includes images from different modalities) for training and validation?
", "votes": 1, "replies": [] }, { "id": 277029, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-02T03:00:46.890000", "content": "Yeah. I want to validate whether my model can generalize to new experimental conditions.
", "votes": 2, "replies": [] }, { "id": 289380, "author_name": "learner", "author_url": "", "post_date": "2018-03-01T14:11:53.307000", "content": "Hi,\nDoes your separation mean that you trained five different models (each trained on a subset of data that you explained)?
", "votes": 2, "replies": [] } ] }, { "id": 275389, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-29T02:21:30.613000", "content": "Whew. A very busy weekend. I posted a new score. I’m still overfitting, but I’ll deal with that in the future. What changed? A lot. Here’s a few of the most noteworthy changes:
\n\nI’m using a Feature Pyramid Network (FPN) before both the region proposal network (RPN) and the region-based convolutional neural network (RCNN). I think most of the improvement can be attributed to this.
I added a basic augmentation loop (random crops and some focus shifts).
I normalized the images (by mean subtraction).
There’s a few other things, but I’m too tired to remember! I’ll post more Monday or Tuesday.
", "votes": 19, "replies": [] }, { "id": 278694, "author_name": "Junsheng Zhou", "author_url": "", "post_date": "2018-02-06T18:52:57.083000", "content": "Thanks for sharing, Allen! I have read your Mask Rcnn code. I found that you only use the last three outputs of ResNet50, so the largest size of feature map is 32*32 if the input size is 256*256. It means that the smallest size of anchor grids is 8*8, but many nuclei in the samples are smaller than 8*8 pixels, so the model can not figure out two different nuclei if both of them are in one grid. How do you deal with this situation?
", "votes": 7, "replies": [ { "id": 282978, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-14T20:30:11.467000", "content": "The first strategy works well on most problems (e.g. see CIFAR research). However, we want to work on making the second strategy more attractive (i.e. easier to implement).
", "votes": 1, "replies": [] } ] }, { "id": 273516, "author_name": "Devin Anzelmo", "author_url": "", "post_date": "2018-01-24T17:58:16.023000", "content": "Allen,
\n\nIts great to have an expert providing a continuous benchmark:) This is a great addition and great motivator for other competitors.
", "votes": 6, "replies": [ { "id": 273540, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-24T18:34:58.630000", "content": "Thanks, dude. I think the next few weeks will be really fun!
", "votes": 4, "replies": [] } ] }, { "id": 275986, "author_name": "Wesley Goi", "author_url": "", "post_date": "2018-01-30T12:18:48.300000", "content": "Hi Allen,
\n\n\n\n\nThe classes were images with white backgrounds, images with black backgrounds, and images with purple foreground and/or purple background. I further stratified the “purple” class into three new classes: images with purple foreground and purple background, images with purple foreground and white background, and images with purple foreground and yellow background.
\n
Any guesses on what's the difference between those within the purple class which u further segmented into
\n\nWould it be some sort of artifact introduced such as the imaging lab or batch?
", "votes": 3, "replies": [ { "id": 277269, "author_name": "Anne Carpenter", "author_url": "", "post_date": "2018-02-02T17:16:11.680000", "content": "The images come from lots of different labs and staining procedures; differences in color balance like this stem from that.
", "votes": 1, "replies": [] }, { "id": 278521, "author_name": "zmingen", "author_url": "", "post_date": "2018-02-06T11:33:11.303000", "content": "There are some image in test set with \"yellow background/purple foreground\". But I can't find this kind of data in train set. My model works badly on these data. Where can I find this kind of data?
", "votes": 1, "replies": [] }, { "id": 278528, "author_name": "Morteza Milani", "author_url": "", "post_date": "2018-02-06T11:58:39.260000", "content": "I think the goal here is to generalize well. So having these yellow background images in the test set is a good thing. If your model does not work well on them, then you won't get a high score in the second stage.
", "votes": 2, "replies": [] } ] }, { "id": 294218, "author_name": "Brian", "author_url": "", "post_date": "2018-03-11T15:31:56.230000", "content": "Hi Allen,
\n\nThanks for sharing.
\n\nI encountered a problem that when I use mask-rcnn in Detectron to detect nuclei, It seems marked the same cell multiple times, This leads me to still cannot submit the result. How can I do to make my result be able to submit?
\n\nAnything will be helpful!
", "votes": 1, "replies": [ { "id": 294233, "author_name": "john1231983", "author_url": "", "post_date": "2018-03-11T15:55:46.687000", "content": "@Brain: Very interesting if someone uses Detectron for this task. I also look at the repo but it requires converting the dataset to coco format. Could you please give me the way to convert it? How much does it takes to train your model? I guess it will be very fast
", "votes": 1, "replies": [] }, { "id": 294431, "author_name": "Brian", "author_url": "", "post_date": "2018-03-12T00:15:06.887000", "content": "I write a simple script to find polygons and convert it to json format. (About this code I planned to open a new topic, but considering that I haven't submitted results yet, I don't think this code is reliable.)
\n\nIt's fast, I didn't train from scratch and it's only take my about 40 minutes on single 1080Ti. And the results seems accurate and impressive, but as I said, it predict a single nuclei multiple times, which leads me still can't make my submission.
\n\nI will open a topic about how to use Detectron in this game as soon as I solve this problem if it will be helpful to you guys.
", "votes": 1, "replies": [] }, { "id": 294441, "author_name": "john1231983", "author_url": "", "post_date": "2018-03-12T00:54:02.937000", "content": "It really helps us. I am very interested in Detectron. Fo your problem, I guess your results in some overlapping mask that one pixel belongs to 2 cells but the challenge only allows that one pixels belong 1 cell. Thus, you may use this function to remove it. Try it if it work https://github.com/YanWang2014/Mask_RCNN/blob/master/Spot-Nuclei2/functions.py#L48
", "votes": 2, "replies": [] }, { "id": 294449, "author_name": "Yan Wang", "author_url": "", "post_date": "2018-03-12T01:07:06.107000", "content": "We actually tried detectron, so far not good on val set.
", "votes": 1, "replies": [] }, { "id": 294456, "author_name": "Brian", "author_url": "", "post_date": "2018-03-12T01:30:09.513000", "content": "Thanks and I'll try it out.
", "votes": 0, "replies": [] }, { "id": 294458, "author_name": "Brian", "author_url": "", "post_date": "2018-03-12T01:32:08.250000", "content": "Do you know any reason about that?
", "votes": 0, "replies": [] }, { "id": 294466, "author_name": "Yan Wang", "author_url": "", "post_date": "2018-03-12T01:53:36.803000", "content": "Hi Brain, can you post your validation results using detectron, which is given by coco api? We got pretty good results for bbox, but not so good for seg.
", "votes": 0, "replies": [] }, { "id": 294482, "author_name": "Brian", "author_url": "", "post_date": "2018-03-12T02:23:11.763000", "content": "You mean predicted mask from 'stage1-test' folder?
", "votes": 0, "replies": [] }, { "id": 294483, "author_name": "john1231983", "author_url": "", "post_date": "2018-03-12T02:27:49.107000", "content": "No. He means validation set that you can separate from stage1-train set with proportion likes 80/20. Then you can evaluate in your machine, without submitting it to the kagge website
", "votes": 0, "replies": [] }, { "id": 294485, "author_name": "Yan Wang", "author_url": "", "post_date": "2018-03-12T02:28:08.077000", "content": "something like this:
\n\nRunning per image evaluation...
\nDONE (t=0.46s).
\nAccumulating evaluation results...
\nDONE (t=0.38s).
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.505
\n Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.697
\n Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.573
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.586
\n Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.519
\n Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.501
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.387
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.594
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.595
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.640
\n Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.566
\n Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.564
Actually I don't have it. Because I assume this a first try, so I use all the image in train set to train.
", "votes": 0, "replies": [] }, { "id": 294499, "author_name": "Yan Wang", "author_url": "", "post_date": "2018-03-12T02:39:28.317000", "content": "You can still evaluate on some images, either they are in the training set or not. We find even in the training set, the seg is not so good as bbox.
", "votes": 0, "replies": [] }, { "id": 294502, "author_name": "Brian", "author_url": "", "post_date": "2018-03-12T02:46:58.940000", "content": "I have putted mask to some pictures in train set yesterday and I found that the seg in gray scale image fitted very well, in color images not very well. And still have problem that mark same cell multiple times.
\n\nI will use cocoapi to evaluate some images in train set this afternoon.
", "votes": 0, "replies": [] }, { "id": 295063, "author_name": "delta_K", "author_url": "", "post_date": "2018-03-13T02:23:30.187000", "content": "According to my observation, You can't just visually check your detection result cause it's hard to tell the difference under high iou metric.
", "votes": 0, "replies": [] } ] }, { "id": 288283, "author_name": "xuhuaren", "author_url": "", "post_date": "2018-02-27T05:19:41.620000", "content": "Can you give more details about how to make supplement data? I can't understand why it's useful. Thanks for your work!
", "votes": 1, "replies": [ { "id": 288452, "author_name": "Anne Carpenter", "author_url": "", "post_date": "2018-02-27T14:27:34.410000", "content": "This thread might help about how to make more annotations: https://www.kaggle.com/c/data-science-bowl-2018/discussion/48347
\n\nAs I understand it, making more annotated images is not useful in the sense of giving you very much of an advantage over other teams, because external data must be shared in the external data thread here: https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572
\n\nBut, extra data will certainly make a better result!
", "votes": 2, "replies": [] }, { "id": 288465, "author_name": "steelrose", "author_url": "", "post_date": "2018-02-27T14:59:15.033000", "content": "hi Anne,\ncan you please confirm that you (as the organizer) consider manual changes to the data as \"external data\"? Would also the proces which Allen is using (creating new masks by watershed etc.) considered as external data?\nIf yes - what is the way to disclose the information? Do we need to upload it (since it must be publicly available)?
\n\nWhy I ask is, that normally any changes to the training data itself (i.e. cleanup, manual re-annotations etc.) are not considered as external data and thus are allowed (even in competitions where no external data is permitted !)
", "votes": 2, "replies": [] }, { "id": 288472, "author_name": "Guillermo Barbadillo", "author_url": "", "post_date": "2018-02-27T15:38:31.300000", "content": "Making modifications to the train data has never been considered as external data.
", "votes": 3, "replies": [] }, { "id": 289880, "author_name": "Anne Carpenter", "author_url": "", "post_date": "2018-03-02T14:29:03.990000", "content": "I'm not the rule-maker, so will defer to @wendykan on this one.
", "votes": 2, "replies": [] }, { "id": 289897, "author_name": "CPMP", "author_url": "", "post_date": "2018-03-02T15:12:22.137000", "content": "Anne, are you saying that annotations to external data need to be shared?
", "votes": 0, "replies": [] }, { "id": 289898, "author_name": "Anne Carpenter", "author_url": "", "post_date": "2018-03-02T15:13:17.107000", "content": "I'm saying \"I'm not the rule-maker, so will defer to @wendykan on this one.\"
", "votes": 3, "replies": [] }, { "id": 289900, "author_name": "bsp2020", "author_url": "", "post_date": "2018-03-02T15:19:55.223000", "content": "https://www.kaggle.com/c/data-science-bowl-2018/discussion/47572#284672
\n\n\n", "votes": 3, "replies": [] } ] }, { "id": 280902, "author_name": "steelrose", "author_url": "", "post_date": "2018-02-11T11:41:40.723000", "content": "Wendy Kan
\n
\n You don't need to post them as that is considered part of your methodology, not external data.
hi Allen, thank you very much for sharing
\n\nHope you don't mind a few questions/clarifications about using the \"mask rcnn\":\n- I downloaded the \"features/preprocessing-objectdetection\" branch\n- and created a new model based on the RCNN, but adding the keras_rcnn.layers.MaskRCNN() at the end
\n\nWhat I wasn't sure how to do:\n1.) how to initialize the output_masks? currently I have:\noutput_masks = keras.layers.TimeDistributed(keras.layers.Dense(mask_r * mask_c, kernel_initializer=\"zero\"))(output_features)\noutput_masks = Reshape(target_shape=(-1, mask_r, mask_c,1))(output_masks)\noutput_masks = keras_rcnn.layers.MaskRCNN()([target_categories, target_masks, output_masks])
\n\n2.) how to properly train the model?\nWith the above model I get a runtime error:
\n\nInvalidArgumentError (see above for traceback): Incompatible shapes: [1,32,28,28] vs. [1,7,28,28]\n[[Node: mask_rcnn_1/logistic_loss/mul = Mul[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:GPU:0\"](mask_rcnn_1/Log, _arg_target_images_0_4/_81)]]
\n\nthe [1,7,28,28] is the target mask (tried with a generator which returns just one file, and has 7 masks)\nbut where does the [1,32,28,28] come from? Does it mean the model predicted 32 masks?
", "votes": 1, "replies": [ { "id": 282964, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-14T20:16:09.627000", "content": "No problem.
\n\nA basic mask branch will look like this:
\n\noutput_masks = keras.layers.TimeDistributed(\n keras.layers.Conv2D(\n filters=256, \n kernel_size=(3, 3), \n activation=\"relu\", \n padding=\"same\"\n )\n)(output_pooled_features)\n\noutput_masks = keras.layers.TimeDistributed(\n keras.layers.Conv2DTranspose(\n activation=\"relu\",\n filters=256,\n kernel_size=(2, 2),\n strides=2\n )\n)(output_masks)\n\noutput_masks = keras.layers.TimeDistributed(\n keras.layers.Conv2D(\n activation=\"sigmoid\",\n filters=n_categories, \n kernel_size=(1, 1), \n strides=1\n )\n)(output_masks)\nOops. I missed your second question. The output should look like this:
\n\n\n(images, objects, region_r, region_c, 2)\n
For example:
\n\n\n(1, 32, 28, 28, 2)\n
You use the (non-maximum suppressed) predicted bounding boxes (or categories) to score and drop your masks (i.e. the number of objects will be the same for the predicted bounding boxes, categories, and masks).
", "votes": 0, "replies": [] }, { "id": 282981, "author_name": "steelrose", "author_url": "", "post_date": "2018-02-14T20:31:42.883000", "content": "thank you very much
", "votes": 0, "replies": [] }, { "id": 282983, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-14T20:33:01.590000", "content": "No problem!
", "votes": 1, "replies": [] } ] }, { "id": 280562, "author_name": "Gábor Vecsei", "author_url": "", "post_date": "2018-02-10T09:10:16.173000", "content": "I've got a question about creating the submission. How are you dealing with overlapping masks?\nFirst I just created one mask image from all the predictions but I think that's not optimal and the score is lower because of that. But when I create single masks per nuclei than there are overlapping masks and I get an Exception when submitting the solution.
", "votes": 1, "replies": [ { "id": 280961, "author_name": "bsp2020", "author_url": "", "post_date": "2018-02-11T15:00:40.013000", "content": "I just sorted predicted masks from smaller to larger and removed duplicate pixels from the larger image. Not sure whether that's the best way to do it (or a good way to do it.) but it lets me create a solution I can submit.
", "votes": 0, "replies": [] }, { "id": 282973, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-14T20:26:43.227000", "content": "I sort the masks by their corresponding score (i.e. each pixel uses the pixel from the mask with the best score). I’ve mentioned this elsewhere, but I believe post-processing (e.g. non-maximum suppression) is the most difficult part of every image segmentation and object detection competition. It’s sadly under-researched. 😔
", "votes": 2, "replies": [] }, { "id": 291783, "author_name": "Brian Farrar", "author_url": "", "post_date": "2018-03-06T20:52:04.663000", "content": "So i am working on this part of the problem right now. I am somewhat confused by: \"each pixel uses the pixel from the mask with the best score\".
\n\nCase 1 - Partial overlap: Two masks share some but not all pixels. Do you keep the pixels from the higher scored mask and leave the residual from the second mask, or do you discard the second mask? I am currently keeping the residual as a second mask, but I can see that this is correct sometimes and wrong others.
\n\nCase 2 - Total overlap A: Higher scored mask is a subset of the lower scored mask. Do you keep the higher scored mask and discard the lower scored \"donut\" residual mask? I am currently planning to discard.
\n\nCase 3: Total overlap B: Higher scored mask is a superset of the lower scored mask. Do you discard the the smaller lower scored mask? Seems the only logical option.
", "votes": 1, "replies": [] } ] }, { "id": 279030, "author_name": "Paul Chen", "author_url": "", "post_date": "2018-02-07T08:07:06.993000", "content": "Hi Allen, do you have any idea about the issue posted here. It seems that the training process is locked or killed somehow by using keras_rcnn on both python3.5 and python3.6.
", "votes": 1, "replies": [ { "id": 282974, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-14T20:27:20.973000", "content": "Hi, this has been fixed. You should use this pull request:
\n\nhttps://github.com/broadinstitute/keras-rcnn/pull/164
\n\nor wait until next week when we release the new version (0.0.3).
", "votes": 1, "replies": [] } ] }, { "id": 277825, "author_name": "Guillermo Barbadillo", "author_url": "", "post_date": "2018-02-04T13:13:46.863000", "content": "Thanks Allen, I have a question regarding image size. Does your model allows to use different input sizes or all the images need to have the same size? If that is the case which size are you using?
\nThanks
Yes. It has two options:
\n\nWhen you use a batch_size of 1 you can use a (None, None) input_shape (i.e. the shape is inferred and can vary between batches); and
You can increase the batch_size and specify a strict target_size that’s either rescaled (proportionally) or cropped.
In my opinion, the mask-rcnn does not support different sizes input for it needs image size to generate anchors and predict the bounding boxes. Could you explain more on how to use various input size in mask rcnn? I found there were some high resolution images which have very tiny instances that if I resized it to 256*256, those objects would missing.
", "votes": 1, "replies": [] }, { "id": 280385, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-09T21:27:58.187000", "content": "Yeah. Eventually your objects disappear. There’s a few strategies that can complement (or replace) resizing. I think randomly cropping (especially since the images are rotationally invariant) is probably the easiest to use. You could also think carefully about anchor scales, convolutional features (e.g. feature pyramids), and upscaling particularly small objects.
", "votes": 1, "replies": [] } ] }, { "id": 275482, "author_name": "zhousheng", "author_url": "", "post_date": "2018-01-29T09:12:57.177000", "content": "Gread job, Allen.\nI try to use the maskRCNN model to train and segment the images with keras_rcnn.\nBut the loss is not decay．I think RCNNMaskLoss is necessary．\nWould you tell me the usage of RCNNMaskLoss.\nMany thanks!
", "votes": 1, "replies": [ { "id": 275575, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-29T14:13:40.187000", "content": "Yep. Your understanding is correct. You’ll need to attach both the RCNN and MaskRCNN layers, e.g.
\n\noutput_deltas, output_scores = keras_rcnn.layers.RCNN()([target_proposals, target_proposal_labels, output_deltas, output_scores])
output_masks = keras_rcnn.layers.MaskRCNN()[target_proposals, output_deltas, target_masks, output_masks]
output_bounding_boxes, output_labels = keras_rcnn.layers.ObjectDetection()([output_proposals, output_deltas, output_scores, target_metadata])
output = [output_bounding_boxes, output_labels, output_masks]
Thanks for your help.There is not MaskRCNN model in keras_rcnn/models/ yet.I think I have to create _maskrcnn.py first.
", "votes": 2, "replies": [] } ] }, { "id": 273098, "author_name": "john1231983", "author_url": "", "post_date": "2018-01-24T05:58:54.760000", "content": "Great! you have thought out of the box. Almost team are following the UNet, I think that UNet is for semantic segmentation while the challenge somehow differences as you mentioned. I think you applied Mask-RCNN (first detect the cell region and then segment the object from individual boxes). Am I right? I am selecting the good framework for this challenge: Tensorflow, caffe2. The benefit of caffe2 is that it released the Mask-RCNN, while Tensorflow is flexible for training, such as GAN model. Which model should I select for this challenge?
", "votes": 1, "replies": [ { "id": 273428, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-24T15:55:06.117000", "content": "Yes. Exactly. However, I should note that Google provides a fantastic implementation for TensorFlow and I also contribute to Keras-RCNN that provides this functionality. I’m planning on writing more about the underlying model architecture today or tomorrow.
", "votes": 4, "replies": [] }, { "id": 273470, "author_name": "Yan Wang", "author_url": "", "post_date": "2018-01-24T16:18:32.563000", "content": "\n\n\nAllen Goodman wrote
\n \n\n\nYes. Exactly. However, I should note that Google provides a fantastic implementation for TensorFlow and I also contribute to Keras-RCNN that provides this functionality. I’m planning on writing more about the underlying model architecture today or tomorrow.
\n
Great, looking forward to your approach. By the way, can you give a link to Google's implementation in TensorFlow? If I understand correctly, the Mask-RCNN in TensorFlow Object Dectection API is on the way, but not released yet
", "votes": 1, "replies": [] }, { "id": 273474, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-24T16:23:54.227000", "content": "https://github.com/tensorflow/models/tree/master/research/object_detection
\n\nYou could also use Keras-RCNN which has the needed layers implemented (e.g. the updated ROI pooling layer and the segmentation loss).
", "votes": 4, "replies": [] }, { "id": 273502, "author_name": "Vijay Jadhav", "author_url": "", "post_date": "2018-01-24T17:04:19.967000", "content": "Facebook recently released this: https://github.com/facebookresearch/Detectron
", "votes": 4, "replies": [] }, { "id": 273547, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-24T18:48:31.767000", "content": "I haven’t used this, but the authors are fantastic people and fantastic researchers. I’d look into it (or use the alternative I co-wrote: Keras-RCNN 😎).
", "votes": 1, "replies": [] }, { "id": 273817, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2018-01-25T08:16:45.147000", "content": "Allen, which backbone network you have used for this task?
", "votes": 2, "replies": [] }, { "id": 273964, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-25T15:39:48.343000", "content": "I’m using ResNet-50 (using the “basic” residual block) provided by Keras-ResNet alongside a Feature Pyramid Network (FPN). However, I plan on using something most sophisticated in the future (e.g. an ensemble of residual networks). It’s something I want to write about in the next day or two.
", "votes": 1, "replies": [] }, { "id": 274493, "author_name": "john1231983", "author_url": "", "post_date": "2018-01-26T17:53:07.263000", "content": "Thanks Allen, so fisrt you cluster the dataset into five groups then design 5 networks (same architecture but different weights due to training on different input types). In the testing, you will select the network corresponding to the image types and perform inferenxe. Am I right? If I am correct, then the problem is that the number of training set in each group will small (because you have splitted from original dataset), that makes difficult for training with deep learning
", "votes": 1, "replies": [] }, { "id": 274974, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-27T23:24:53.820000", "content": "I’m still thinking about this.
\n\nInitially, I used k-fold cross-validation to evaluate whether my model was robust to new experimental conditions by holding out entire experiments in my validation and test sets. However, I’ve started thinking about an ensemble learning method to ensemble k models into one model (e.g. by averaging their weights). I have considered the solution you outlined and I think it’s certainly viable. Hell, I think it’d be extremely competitive (if not the best method). I didn’t use it because it’d take substantial time to implement and it felt antithetical to the spirt of the competition (we envisioned one model could generalize across seen and unseen experimental variables).
\n\nI’ll add that I recently worked on a paper that did something similar.
\n\nThe architecture looked like this:
\n\nCNN -> object detection network -> CNN -> image classifier
\n\nAccuracy (we used the same metric as the competition) skyrocketed when we piped the output of the object detection network (the bounding boxes and class scores) into the second CNN and image classifier. However, it did require us to restructure the problem so that the object detection network predicted fewer classes. Unfortunately, we’re still unsatisfied with the solution. It was a monster to properly train without under-or-overfitting and it was difficult (if not impossible) to justify the second convolutional neural network (CNN) since the feature space should have been captured by the first.
", "votes": 1, "replies": [] }, { "id": 275864, "author_name": "bsp2020", "author_url": "", "post_date": "2018-01-30T05:59:43.323000", "content": "Hello Allen,\nI'm want to try out Keras-RCNN but I'm having issues. I tried building the dock the generated file does not look correct. I tried installing it by
\n\npython setup.py build \npython setup.py install\ninstall output here\nand it seems to install. But I can't get the datasets...\nI get
\n\n\n\n\n$ ipython
\n \n
\n Python 3.6.3 |Anaconda custom (64-bit)| (default, Oct 13 2017, 12:02:49)
\n Type 'copyright', 'credits' or 'license' for more information
\n IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help.
\n In 1: import keras_rcnn
\n In [2]: training, validation, test = keras_rcnn.datasets.malaria_phenotypes.load
\n ...: _data()...:
\n \nAttributeError Traceback (most recent call last)
\n \n
\n in ()
\n ----> 1 training, validation, test = keras_rcnn.datasets.malaria_phenotypes.load_data()AttributeError: module 'keras_rcnn' has no attribute 'datasets'
\n
Could you please help?
", "votes": 1, "replies": [] }, { "id": 275911, "author_name": "Serhii", "author_url": "", "post_date": "2018-01-30T08:11:07.290000", "content": "I solved the problem this way
\n\n\n", "votes": 0, "replies": [] }, { "id": 276027, "author_name": "William Green", "author_url": "", "post_date": "2018-01-30T14:11:18.577000", "content": "from keras_rcnn.datasets.malaria_phenotypes import load_data
\n \nfrom keras_rcnn.preprocessing import ObjectDetectionGenerator
\n \ntraining, validation, test = load_data()
\n
Thank you!!!!!!
", "votes": 0, "replies": [] }, { "id": 276036, "author_name": "john1231983", "author_url": "", "post_date": "2018-01-30T14:45:16.760000", "content": "For training, do you use coco pretrain model? What is learning rate and dropout probability do you use ? I am using pretrained coco model with learing rate of 1e-4. I think it is better to train with Batch normalization (with moving mean and variance), but I frozen them now. Thanks
", "votes": 2, "replies": [] }, { "id": 276047, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-30T15:23:07.463000", "content": "I don’t pre-train with COCO for a variety of reasons. I did, however, use pre-trained weights for convolutional layers inside the ResNet. I also don’t use dropout.
\n\nI used the following hardcoded learning rate schedule with standard SGD:
\n\nAllen, when I heard that you will be averaging the weights, I was confused.
\n\n\n\n\nHowever, I’ve started thinking about an ensemble learning method to ensemble k models into one model (e.g. by averaging their weights).
\n
You cannot average the weights of multiple models together. That does not work. If a filter learned the same representations, it could be in a different order, and that would be horrible for the model. Do you mean averaging the outputs together? If you are concerned for making sure you segment the nuclei properly into multiple instances, could you use intersection instead of the union or average all the masks? Also, I am very interested segmenting the nuclei into multiple pieces to make sure they are separate. I am having a problem with that because my predictions from U-Net are merged nuclei. Could you elaborate on how you will do that, or is there somewhere I can gain some knowledge on this? Thanks
", "votes": 4, "replies": [] }, { "id": 276829, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-01T16:00:40.810000", "content": "I misspoke. Indeed, I meant averaging their outputs. ☺️
\n\nIf you’re open to using a supplementary model, you could do one of many things. For example:
\n\nYou could train an additional FCNN to predict a boundary and subtract the boundary from the two-class prediction predicted by your existing FCNN.
You could train a three-class FCNN to predict a “foreground,” “background,” and “boundary” class.
You could use a classical method like distance transforming your predicted mask and using watershed to isolate the instances.
I also wanted to reiterate that I’m not using an FCNN so I’m not encountering these problems. I’m predicting multiple object masks rather than one image mask.
", "votes": 2, "replies": [] }, { "id": 277581, "author_name": "", "author_url": "", "post_date": "2018-02-03T16:15:44.917000", "content": "", "votes": 1, "replies": [] }, { "id": 277671, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-03T22:07:15.577000", "content": "Hi, excuse my brevity. I’m writing from my phone. I’ll write a more detailed response when I’m home.
\n\nUnfortuantly, the NaN issue is a “known issue.” I (and others) have been investigating this problem for a few weeks without success. Hopefully it’ll be resolved soon. Until it’s fixed you can use the Keras NaN callback to stops and restart training.
\n\nYou can make a prediction just like any other Keras model. If you use predict_generator remember to make the shuffle parameter False. I should add that I already converted the DSB data. You can find that in the DSB2018 PR. You should also take a look at the generator PR I opened last week. It has a number of helpful augmentation steps.
", "votes": 2, "replies": [] }, { "id": 279080, "author_name": "john1231983", "author_url": "", "post_date": "2018-02-07T10:10:26.173000", "content": "Hi Allen, I want to ask you the question about parameter setting:\n1. Do you train with batch normalization? If yes, what is your batch size? I am using batch size of 4 and random cropping image size of 128x128\n2. What is your ANCHOR SCALES? My case is (8, 16, 32, 64,128)\n3. If you crop the image into 128x128. In the inference you also need to crop the input image into 128x128 sub-images, because MaskRCNN used the fully connected layer in detection, it does not allow to arbitrary size as FCN. If I right, how you create final segmented image from segmented of sub-images\nThanks!
", "votes": 1, "replies": [] }, { "id": 280275, "author_name": "Zhen Chen", "author_url": "", "post_date": "2018-02-09T17:15:11.723000", "content": "Hi Allen, you said we can find augmentation steps in the DSB2018 PR. However, I cannot find the this post. Would you please post a link for the data augmentation? Thank you very much.
", "votes": 1, "replies": [] }, { "id": 280383, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-09T21:23:54.930000", "content": "Hi, you should track the following PR: https://github.com/broadinstitute/keras-rcnn/pull/164/files
", "votes": 1, "replies": [] }, { "id": 280600, "author_name": "john1231983", "author_url": "", "post_date": "2018-02-10T12:12:29", "content": "Hi allen. What is your ANCHOR SCALES? I am using (8, 16, 32, 64,128) . And how you achieve instance segmentation in testing set if the input is 128x128, while the testing image size is 1024x1024? Thanks!
", "votes": 0, "replies": [] } ] }, { "id": 283778, "author_name": "大王叫我来寻山", "author_url": "", "post_date": "2018-02-16T00:25:49.080000", "content": "Hi allen,
\n\nThanks for sharing.
\n\nCould you also sharing the score with different method? for example pure unet:0.2, with data augumentation 0.3, with resnxt50 0.35, pure mask rcnn 0.5, with data augumentation 0.6
\n\nHere i did not mean unet will be worse than maskrcnn. Just from the public kernel, most people start with unet get score below 0.3.\nThe architecture of unet is very simple compared with faster rcnn(maybe I should use the word beautiful). This maybe the reason lots of people choose it to start.
", "votes": 2, "replies": [ { "id": 289235, "author_name": "Nomoreaday", "author_url": "", "post_date": "2018-03-01T07:55:27.567000", "content": "hello, see you again.
", "votes": 0, "replies": [] }, { "id": 295732, "author_name": "Hatuw", "author_url": "", "post_date": "2018-03-14T04:57:59.593000", "content": "Because there is a kernel has already inferenced the U-Net, but not Mask-RCNN. hhhh
", "votes": 0, "replies": [] } ] }, { "id": 282856, "author_name": "Guillermo Barbadillo", "author_url": "", "post_date": "2018-02-14T17:50:43.060000", "content": "Hi Allen,
\nHave you cleaned the data before training your model?
Today I have been visualizing the annotated masks and the quality is so terrible that your current score seems like magic.
", "votes": 2, "replies": [ { "id": 282959, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-14T20:09:58.787000", "content": "Yeah, I did a few things.
\n\nI’ll briefly summarize the process, but let me know if you want to know more about a particular topic.
\n\nMy cleanup process was painful but straightforward:
\n\nI removed the alpha channel from the annotations and masks.
I fixed the problems that were outlined in the “data quality issues” thread (I did this a few weeks ago and there’s been a few more problems that popped up)
I closed small holes in objects. I believe I used a (1, 1) structuring element. I also spot checked every object to ensure it was a mistake (e.g. or it seemed like a mistake to my eye).
I also supplemented the training set in an attempt to standardize annotation disagreements:
\n\nI computed a distance transform on each of the masks (not the flattened label matrix) and found the maximum peak. I computed a watershed segmentation for each masked image (that was dilated with a (1, 1) structuring element) using the peaks as markers. I added this new set of masks to the existing set.
I took the centroid of each mask and used the random walker algorithm to compute a third set of masks using the aforementioned masked images. I then used the same watershed process from before.
Finally, I created a fourth and fifth set of annotations by using a geodesic active contour with two different edge preserved responses (from a median filter and an inverse Gaussian filter). I then used the same watershed process from before.
I used the supplemental data like this:
\n\nI took the initial set of training data and made a 80%/20% split. I repeated this process five times to create five training sets and five validation sets.
Each epoch I would randomly swap each annotation for one of the five buckets (the real training data, the watershed data, the random walker and watershed data, and the two buckets of geodesic and watershed data) so each epoch would only see one image once.
This is an area that I believe can be improved further with more time.
\n\nI augmented the data like this:
\n\nrandom scaled down version of the annotation and image (with nearest neighbor and bilinear respectively)
random crop
random horizontal or vertical flip
semi-random color perturbation (it’d randomly shift colors to fall into one of three categories of color features, “blue,” “blue and yellow,” and “grayscale”).
random blurriness (or focus)
random (shot or Poisson) noise
I also did some mild post-processing on the masks. I’d like to start experimenting with a refinement step (e.g. ideas from the Sharp Mask paper), but I don’t think it’ll improve much because of the aforementioned annotation disagreement.
\n\nI didn’t do anything to the bounding box predictions besides non-maximum suppression (which I should write a lengthy post about since it’s probably the most important step in the entire process).
", "votes": 26, "replies": [] }, { "id": 282963, "author_name": "Guillermo Barbadillo", "author_url": "", "post_date": "2018-02-14T20:15:06.427000", "content": "Thanks a lot for your detailed answer. That's a very interesting approach.
", "votes": 1, "replies": [] }, { "id": 282980, "author_name": "steelrose", "author_url": "", "post_date": "2018-02-14T20:31:25.790000", "content": "thank you very much, certainly an easier approach than redoing the masks manually :) (what I did for some of the obviously \"wrong\" ones, i.e. mostly the oversized masks in some images etc.)
", "votes": 1, "replies": [] }, { "id": 282982, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-14T20:31:46.640000", "content": "No problem. I hope you don’t hesitate to keep reaching out. I’m here to help (or even spitball). 😎
", "votes": 2, "replies": [] }, { "id": 283276, "author_name": "Cheng, Hua", "author_url": "", "post_date": "2018-02-15T06:22:42.370000", "content": "Wow! Thank you, Allen! I learned a lot from your information.
", "votes": 0, "replies": [] }, { "id": 283325, "author_name": "super504", "author_url": "", "post_date": "2018-02-15T09:33:32.307000", "content": "Hi, Allen!
\n\nAre 5 different models from cross-validation just averaging to make final prediction?
", "votes": 1, "replies": [] }, { "id": 287555, "author_name": "Rahul Kumar", "author_url": "", "post_date": "2018-02-25T06:54:02.917000", "content": "Hi Allen,\nI am trying to supplement the dataset like you did with creating additional annotations. I have one doubt about how did you create the masked image. Here is what I understood
\n\nFor a single image:
\n\nyou take the distance transform of each mask and find the peak using local maxima, and you create a marker with these peaks, now you merge all the masks and create a single mask( is this what you mean by masked image?) and create a distance transform of this merged mask and using the marker and the distance transform you do watershed segmentation.
", "votes": 2, "replies": [] }, { "id": 292430, "author_name": "Gangadhar Payyavula", "author_url": "", "post_date": "2018-03-08T00:15:56.593000", "content": "Hi Allen
\n\n\n\n\n\n \nI closed small holes in objects. I believe I used a (1, 1) structuring element. I also spot checked every object to ensure it was\na mistake (e.g. or it seemed like a mistake to my eye).
\n
I did dilation with (1,1) structuring element and it has no effect on the mask, is this due to implementation details of different libraries or am I missing something. Thanks in advance, you notebook has been very helpful.
", "votes": 1, "replies": [] }, { "id": 298208, "author_name": "renxuhua_sjtu", "author_url": "", "post_date": "2018-03-19T03:29:33.243000", "content": "Dear Allen:\n I try out to utilize morphological_geodesic_active_contour and inverse_gaussian_gradient API in python to achieve the external annotations you mentioned. However, I found it difficult to achieve a good result for these two algorithm. Let me describe the processing:\n1. load the each of sub_mask for one image, each sub_mask represent one cell.\n2. sub_image=dilation(sub_image,square(3))\n3. sub_image = inverse_gaussian_gradient(sub_image)\n4. sub_mask = morphological_geodesic_active_contour(sub_image, 50, init,\n smoothing=3, balloon=-1,\n threshold=0.5)\nThe final masks are so bad, I missed a lot of sub_mask. So what's your hyperparameters you used. By the way, I think your external annotation's aim is standardize the boundary. So your external annotation is a little overestimated for mask compared with original mask. Is it right?
", "votes": 1, "replies": [] }, { "id": 298653, "author_name": "Brian Farrar", "author_url": "", "post_date": "2018-03-19T21:01:13.297000", "content": "I had pretty good results following this general outline Alan provided using cv2 and skimage functions.
\n\nMy steps were:
\n\nwatershed
\n\nrandom_walk
\n\nactive_contours
\n\nDepending on parameter choices masks can be slightly larger or smaller than ground truth masks, with slightly different shapes. I am just beginning to substitute into my training loop, so remains to be seen whether LB improvement is to come.
", "votes": 4, "replies": [] }, { "id": 298659, "author_name": "renxuhua_sjtu", "author_url": "", "post_date": "2018-03-19T21:14:57.053000", "content": "Appreciate for your detailed reply. I think standard external dataset must have benefit for LB. By the way, your mask images are a single mask which foreground are only 1 or the multi-mask which foreground start from 1? If you have good results, please tell me :)
", "votes": 0, "replies": [] }, { "id": 311790, "author_name": "Bcottman", "author_url": "", "post_date": "2018-04-10T17:06:34.347000", "content": "if augmentation is used to obtain translational invariance in a CNN then what you doing is creating mask method invariance. Should i assume the \"ground truth masks were obtained with a labeled tool? and/or they were manual generated. (probably answered somewhere); the latter is preferable as we are trying to beat human \"expert\" performance. what you are doing now, IMHO, is how i would approach it in the real world, as this would allow to generalize/ improve the CNN or add whatever comes along with improvements in image segmentation and then test them. now that i think about it, ... oh never mind .. i digress...
", "votes": 1, "replies": [] } ] }, { "id": 278033, "author_name": "Andrea Rapuzzi", "author_url": "", "post_date": "2018-02-05T07:10:17.750000", "content": "Hi Allen,\nthanks for sharing!!!\nA doubt about overfitting: do you just experience a gap between training and validation or also between your validation and submission scores? I ask you because I'm mimicking the competition score function but my validation score is always 0.1 higher than the related submission score (which is A LOT), although almost coherent with it.\nThanks in advance!!!
", "votes": 2, "replies": [ { "id": 278195, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-05T16:00:06.837000", "content": "Interesting. 0.1 is more than I’ve seen. The differences I’ve seen range between 0.049 and 0.074. How are you evaluating? SRS of the training data?
", "votes": 1, "replies": [] }, { "id": 278251, "author_name": "Andrea Rapuzzi", "author_url": "", "post_date": "2018-02-05T18:10:40.553000", "content": "Actually the difference is between 0.07 and 0.1, always positive (validation score > submission score). This is strange because the valid data is not used at all during the training (if not to select a good epoch to use, but this cannot explain the big difference in score). I'll search my score function for a bias (or a bug :-)).
", "votes": 1, "replies": [] }, { "id": 294230, "author_name": "QuangTeo", "author_url": "", "post_date": "2018-03-11T15:49:41.400000", "content": "@Andrea Rapuzzi did you find the reason? I have the same problem, I thought because of my simple way to split train/valid (random split 0.9-0.1)
", "votes": 1, "replies": [] }, { "id": 294236, "author_name": "hengck23", "author_url": "", "post_date": "2018-03-11T15:58:55.520000", "content": "the correct way to do validation is:
\n\ndivide training set into cell types. e.g. set A,B,C,D, ...E However, we do not have the meta data. The closest guess is to cluster the train images based on size, nuclei size, k-means, etc
then train on e.g. A,B,C and validate on D,E
the test LB set may be of different cell types from the train
", "votes": 2, "replies": [] }, { "id": 294248, "author_name": "QuangTeo", "author_url": "", "post_date": "2018-03-11T16:23:21.437000", "content": "Thanks for your advice
", "votes": 0, "replies": [] }, { "id": 294590, "author_name": "Andrea Rapuzzi", "author_url": "", "post_date": "2018-03-12T07:36:56.467000", "content": "Hi @QuangTeo, since then I've changed the train-validation split (with a StratifiedShuffle based on the classes provided by Allen) and I've applied some refactoring to my code (perhaps eliminating some bias, but I'm not sure). Now my gap between validation and test score is around 0.5-0.7 which seems fine given the differences in the sets.
", "votes": 2, "replies": [] }, { "id": 294754, "author_name": "QuangTeo", "author_url": "", "post_date": "2018-03-12T15:26:23.243000", "content": "Thanks, I think train-validation split method can be important in this competition, It reminds me of the Fish competition since the test data may contain many different types of images.
", "votes": 1, "replies": [] } ] }, { "id": 277195, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2018-02-02T13:10:10.703000", "content": "Allen, how much did it take to train your current score? I mean in terms of learning time of rcnn / epochs / steps per epoch ? Just want to have some time benchmark. Thank you!
", "votes": 2, "replies": [ { "id": 277305, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-02T18:53:15.790000", "content": "Hi, Maksimov: It took a “weekend.” It’s still not trivial to train so there was a lot of starting, stopping, and even restarting. If it was a perfect process, it would’ve taken ~16 hours using my Mask RCNN Implementation (that’s presently two-to-three times slower than what FAIR has reported about their implementation). I used the training schedule from the paper but I decreased the learning rates by -10^2.
", "votes": 6, "replies": [] }, { "id": 277805, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2018-02-04T11:51:48.237000", "content": "Thanks Allen! I also decide to learn from scratch without pre trained model on COCO dataset, I launched the training on Friday and it should take about 25 hours of training using single 1070 card. Byt the way, which hardware do you use?
", "votes": 3, "replies": [] }, { "id": 277941, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-04T21:34:50.703000", "content": "I have two 1080s and two Titan Xs in my machine. However, I typically only use one of the cards for training.
", "votes": 2, "replies": [] } ] }, { "id": 273250, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2018-01-24T11:03:32.553000", "content": "Can you pls share your approach for train set preparation? what technique helped you the most (load as grey / resize/ slicing / etc)?
", "votes": 2, "replies": [ { "id": 273430, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-24T15:56:25.437000", "content": "Yeah! I plan on releasing and discussing everything. However, I haven’t done any data preparation outside of restructuring the training data. I plan on working on this over the weekend or next week sometime.
", "votes": 4, "replies": [] }, { "id": 274225, "author_name": "yyll008", "author_url": "", "post_date": "2018-01-26T04:30:50.193000", "content": "Awesome^_^
", "votes": 0, "replies": [] } ] }, { "id": 273054, "author_name": "hengck23", "author_url": "", "post_date": "2018-01-24T04:10:15.813000", "content": "@Allen Goodman
\n\nyou current LB score 0.516 is impressive. Any hints on the methods you used?
", "votes": 2, "replies": [ { "id": 273426, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-24T15:52:29.750000", "content": "> you current LB score 0.516 is impressive. Any hints on the methods you\n> used?
\n\nThanks! I’ll write more about this today or tomorrow, but, briefly, I’m using a Mask RCNN-like architecture available from Keras-RCNN. You’re more than welcome to use my code! I should note that this is without any pre-processing (or augmentation), modifications to the architecture (e.g. an ensemble CNN), sophisticated sampling, etc. There’s a lot of room for improvement!
", "votes": 15, "replies": [] }, { "id": 273490, "author_name": "hengck23", "author_url": "", "post_date": "2018-01-24T16:43:44.903000", "content": "Thanks! Seems that we are thinking about the same thing, but i am missing some details.
\n\nCurrently, I am using pytorch framework. I use 2 stage network (not end-to-end)
\n\nThis step will guarantee the TP, FP and precision accuracy of my solution.
\n\nbased on estimated nuclei radius, i can resize the whole image to a \"standard size\".
Then i have another fully convolution network read the \"standard size\" image and the nuclei center candidates and predict if each pixel in the 32x32 region centering a \"nuclei center\" is part of the nuclei or not (aka. instance segmentation here). I want to avoid the cropping by using a output channels of 32x32=1024.
This step will ensure i have the max iou for threshold 0.5 to 0.9.
\n\nLooking forward your solution to fill in my missing details! Thanks again!
", "votes": 6, "replies": [] }, { "id": 273514, "author_name": "Thomas Yokota", "author_url": "", "post_date": "2018-01-24T17:55:51.307000", "content": "", "votes": 1, "replies": [] }, { "id": 273545, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-24T18:46:55.677000", "content": "The architecture I’m using is less focused on segmentation. Here’s a brief outline:
\n\nI use a convolutional neural network (CNN) to extract image features.
I use a region proposal network (RPN) to extract region proposals. The RPN has two outputs: bounding boxes and an “objectness” score that estimates whether the bounding box encloses an object.
I use a region-based convolutional neural network (RCNN) to pool the aforementioned region proposals and pass the pooled region proposals and the image features estimated by the CNN into three sub-networks: a regression network that regresses an object’s bounding box coordinates, a classification network that estimates the likelihood the object’s bounding box contained a particular class (I use two classes “nucleus” and “background”), and, finally, a second classification network that classifies each pixel into the classes predicted by the first classification network.
My solution, however, requires a few additional steps like non-maximum suppression and mask refinement (presently that’s just a morphological dilation). I should note that this post-processing is critical to the success of the architecture and these two steps (non-maximum suppression and mask refinement) will probably require the most work from an architectural perspective. I made a mistake with the dilation that was responsible for a difference of 0.4 and 0.5 in my leaderboard score!
", "votes": 18, "replies": [] }, { "id": 273570, "author_name": "Morteza Milani", "author_url": "", "post_date": "2018-01-24T20:23:14.743000", "content": "For anyone like me new to R-CNN, here is a good blog post describing the architecture in general: https://tryolabs.com/blog/2018/01/18/faster-r-cnn-down-the-rabbit-hole-of-modern-object-detection/
", "votes": 19, "replies": [] }, { "id": 273615, "author_name": "William Green", "author_url": "", "post_date": "2018-01-24T21:50:30.513000", "content": "Thank you
", "votes": 0, "replies": [] }, { "id": 273764, "author_name": "delta_K", "author_url": "", "post_date": "2018-01-25T05:21:16.927000", "content": "High, I am also trying to use mask R-CNN. The pretrained models available are mostly based on resnet101 and fpn. Is it too large for this task?
", "votes": 2, "replies": [] }, { "id": 273837, "author_name": "Anton Pimenov", "author_url": "", "post_date": "2018-01-25T09:08:07.087000", "content": "Are you use any external data, or only augmentation?
", "votes": 1, "replies": [] }, { "id": 273969, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-01-25T15:45:21.303000", "content": "No. I don’t plan on using external data (with mask annotations) since it’d take way too much time. I’m considering annotating additional bounding boxes but so far (and without the better evaluation scheme I outlined) my model is localizing objects well.
\n\nI will, however, spend a lot of time experimenting with data augmentation.
", "votes": 2, "replies": [] }, { "id": 275484, "author_name": "Ori Tal", "author_url": "", "post_date": "2018-01-29T09:23:00.363000", "content": "@Allen Goodman, do you use pre-trained version of R-CNN? do you train it only on the competition data? other external data?
", "votes": 1, "replies": [] }, { "id": 281122, "author_name": "Yan Wang", "author_url": "", "post_date": "2018-02-12T03:27:03.630000", "content": "Why mask refinement like morphological dilation is necessary?
", "votes": 2, "replies": [] }, { "id": 281542, "author_name": "", "author_url": "", "post_date": "2018-02-12T20:46:33.400000", "content": "", "votes": 2, "replies": [] } ] }, { "id": 306604, "author_name": "Andrea Rapuzzi", "author_url": "", "post_date": "2018-03-30T18:23:17.027000", "content": "By the way, your github profile picture is fantastic!!
", "votes": 0, "replies": [] }, { "id": 280749, "author_name": "Stanwar", "author_url": "", "post_date": "2018-02-10T21:55:18.260000", "content": "Hi, I've been attempting the rcnn getting started tutorial from the readme of rcnn github but have had issues are follows:
\n\nIssues I have managed to resolve:
\n\nThere were no imports provided so I added those required.
I seemed to be missing malaria_phenotypes.py from the pip install but just added to the appropriate place manually.
Issues I have not yet resolved:
\n\n-In python 2 it fails at the following line (complaining about the number of arguments):
\n\n\n\n\nmodel = keras_rcnn.models.RCNN(image, classes=len(classes) + 1)
\n
-In python 3.5 and 3.6 on different versions of tensorflow it crashes on the line:
\n\n\n\n\nfrom keras_rcnn.preprocessing import ObjectDetectionGenerator
\n
Has anyone got the tutorial working? If so are you able to share any tips or share a full .py file with a list of versions you are using?
\n\nReally grateful for any help anyone can offer!
", "votes": 0, "replies": [ { "id": 282972, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-14T20:24:00.537000", "content": "Keras-RCNN is still a work-in-progress. I’d recommend using the following pull request:
\n\nhttps://github.com/broadinstitute/keras-rcnn/pull/164
\n\nor waiting until next week when the next version (0.0.3) is released.
", "votes": 2, "replies": [] }, { "id": 283147, "author_name": "Stanwar", "author_url": "", "post_date": "2018-02-15T00:57:19.377000", "content": "Brilliant, will do. Thanks.
", "votes": 0, "replies": [] }, { "id": 312204, "author_name": "", "author_url": "", "post_date": "2018-04-11T11:25:54.573000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 278616, "author_name": "delta_K", "author_url": "", "post_date": "2018-02-06T15:42:43.267000", "content": "Hi, Allen! Could you share your experience on how to prevent overfitting when training mask rcnn with resnet50 + fpn backbone. The model overfits soon after running about 40-50 epochs, and the validation loss is far more higher than the training loss.
", "votes": 0, "replies": [ { "id": 278631, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-06T16:10:53.790000", "content": "A few questions:
\n\nIs your validation set a simple random sample of the training images?
Are you training end-to-end (i.e. are you training the CNN, RPN, and RCNN together)?
Which losses are overfitting? All of them?
Thanks for your reply!\n1. Yes, I sampled it randomly from training images.\n2. Yes, I have tried to trained it respectively but seems perform worse than directly end-to-end training.\n3. Almost all. \nMy currently solution is adding dropout after last four stages of resnet, but it seems you didn't encounter such case even without dropout. So I think maybe I missed something.
", "votes": 0, "replies": [] }, { "id": 278912, "author_name": "delta_K", "author_url": "", "post_date": "2018-02-07T03:27:00.577000", "content": "Besides, I found a problem. There are samples that have more than 300 instances. Do you include all the ground truth instances during training, or randomly subsampled them?
", "votes": 1, "replies": [] }, { "id": 282987, "author_name": "Allen Goodman", "author_url": "", "post_date": "2018-02-14T20:38:58.830000", "content": "Great question. I didn’t intentionally do anything so I think I didn’t encounter this problem because I was randomly cropping (i.e. no image had more than 300 objects). However, you can also increase the maximum number of proposals (I believe it's an ObjectDetection parameter).
", "votes": 0, "replies": [] } ] }, { "id": 274356, "author_name": "HaoChen", "author_url": "", "post_date": "2018-01-26T12:20:02.357000", "content": "hello Allen ,i want to know whether the stage1_sample_submission.csv a perfect mask for train data? i make a csv file by the mask pictuers of train data and use RLE,but the csv file is different from stage1_sample_submission.csv
", "votes": 0, "replies": [ { "id": 274360, "author_name": "MykolaSharhan", "author_url": "", "post_date": "2018-01-26T12:30:16.953000", "content": "stage1_sample_submission.csv contains only one connected component (first pixel) for each test images. It was provided just for better intuition about data format.
", "votes": 1, "replies": [] }, { "id": 274365, "author_name": "HaoChen", "author_url": "", "post_date": "2018-01-26T12:57:45.720000", "content": "it's very nice of you ,thank you very much ! can you help with another question : if a picture's pixel is [[0 0 0 ],[ 0 255 255],[ 0 0 255]] ,whether it's RLE : 5 2 9 1 ?
", "votes": 0, "replies": [] }, { "id": 274684, "author_name": "Paul Jurczak", "author_url": "", "post_date": "2018-01-27T03:15:31.737000", "content": "@Justin Assuming that your matrix rows represent horizontal image lines, top to bottom, then RLE is 5 1 8 2
", "votes": 3, "replies": [] }, { "id": 274698, "author_name": "HaoChen", "author_url": "", "post_date": "2018-01-27T03:23:51.313000", "content": "Paul ,thanks very much ! i get it from your intuitive explanation .
", "votes": 1, "replies": [] } ] }, { "id": 273002, "author_name": "panthole", "author_url": "", "post_date": "2018-01-24T02:23:13.157000", "content": "Thank you for your post, it provided some clues for me to improve my score.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "272845": "My name’s [Allen][1]. I helped organize the competition. \n\nI’m a member of the Broad Institute’s [Imaging Platform][2] and the majority of professional life is spent thinking about and solving problems that are similar to the problem presented in the competition. I’m using this thread as a public lab notebook to discuss some of the challenges I encounter while working on the problem alongside other competitors. \n\nHowever, before I begin, I want to stress:\n\n**I’ll submit scores, but I’m ineligible for a prize.**\n\n**I’ll refrain from submitting during the second phase to avoid cluttering the leaderboard.**\n\nWhile I know some things other participants don’t, I intentionally don’t know anything about either the first or second phase test sets (e.g. number of images or experimental conditions).\n\nIn addition to this thread, I’ll continue monitoring and help answer questions on the forum. You can also [follow me on Twitter][3] where I write about computer vision issues.\n\n [1]: https://personal.broadinstitute.org/agoodman/\n [2]: https://www.broadinstitute.org/imaging\n [3]: https://twitter.com/0x00B1", "272854": "My immediate observations about the competition metric:\n\n* The evaluation metric, mean average precision (mAP), is the de facto standard for the major object detection and image segmentation competitions in computer vision (e.g. [Cityscapes][1], [PASCAL Visual Object Classes][2], and [Common Objects in Context][3]). Their respective leaderboards should provide some clues about model selection.\n* It’s probably wise to consider this an object detection rather than image segmentation (and post-processing) challenge (i.e. categorical accuracy is not a reliable predictor for mAP). I should prioritize computing the correct number of objects rather than morphological accuracy since I can always improve morphological accuracy with a refinement step or additional post-processing.\n* Likewise, this is an [instance segmentation rather semantic segmentation][4] challenge (i.e. objects must be disconnected) so I should probably think about this problem differently than previous Kaggle competitions and consider alternatives to fully convolutional networks like U-Net.\n\nMy immediate observations about the training set:\n\n* It’s clear from the training set that I need to focus on generalizability rather than accuracy. I’ll do far better on the leaderboard with an average score across multiple experimental conditions than good scores on one or two. Unfortunately, it’s unclear how many groups of experimental conditions are present. I (and a few others on the forum) have identified three, but I’m not a biologist, so there might be more that I’m missing. It’s also unclear if the test set will include entirely new conditions or the same conditions that are present in the training set.\n* Since generalizability is my primary focus I need to think carefully about model validation. A major problem with deep learning models is accidently overfitting. I’ll also need to be careful around optimization and ensure that I use an appropriate loss (or an ensemble of losses).\n* I think [k-fold cross-validation][5] where *k* is the number of experimental conditions will be fundamental to combating overfitting issues. Unfortunately, we’re not provided metadata to split the training images into their experiment so I’ll need to do this by eye or clustering. I’ll also need to be considerate of training time since it’ll be *k* × *t*.\n\n [1]: https://www.cityscapes-dataset.com\n [2]: http://host.robots.ox.ac.uk/pascal/VOC/\n [3]: http://cocodataset.org/#home\n [4]: https://stackoverflow.com/questions/33947823/what-is-semantic-segmentation-compared-to-segmentation-and-scene-labeling\n [5]: https://en.wikipedia.org/wiki/Cross-validation_(statistics)#k-fold_cross-validation", "273531": "I continued thinking about my validation strategy. \n\nI accomplished the first task of stratifying the training images into five “modalities” (or modality-like classes). It was a straightforward process. First, a labmate (Hi, Jane!) stratified the images into three classes by the sum of their intensities. The classes were images with white backgrounds, images with black backgrounds, and images with purple foreground and/or purple background. I further stratified the “purple” class into three new classes: images with purple foreground and purple background, images with purple foreground and white background, and images with purple foreground and yellow background.\n\nHere’re the total number of images per class:\n\n* black foreground and white background (16)\n* purple background and purple foreground (71)\n* purple foreground and white background (41)\n* purple foreground and yellow background (8)\n* white foreground and black background (599)\n\nI’ve attached a spreadsheet (classes.csv) with the filenames for each class. Enjoy! 😎\n\nA few additional observations:\n\nI think it’s entirely possible to split the black foreground and white background class into a few additional classes (e.g. fuzzy and non-fuzzy foreground objects) to further ensure generalizability around non-intensity related variables like shape and size (and simplify training by balancing the classes), but the aforementioned five classes seem sufficient to start the validation process. Nevertheless, I plan on revisiting this soon.\n\nI found a number of images that will be very problematic from an object detection perspective. A few examples of images that will give me nightmares:\n\n* 259b35151d4a7a5ffdd7ab7f171b142db8cfe40beeee67277fac6adca4d042c4.png\n* f4b7c24baf69b8752c49d0eb5db4b7b5e1524945d48e54925bff401d5658045d.png\n* 8f27ebc74164eddfe989a98a754dcf5a9c85ef599a1321de24bcf097df1814ca.png\n\nHowever, in my opinion, the most worrying image is this monster:\n\n* af576e8ec3a8d0b57eb6a311299e9e4fd2047970d3dd9d6f52e54ea6a91109da.png\n\nThis is exactly the type of image that you use to penalize overfitting.\n\nIn addition to the modality subsets that I described I’m also going to put together a subset with the aforementioned nightmare images to further punish myself. I’ll probably add a few from the “problems” thread too.\n\nI really want to ask the biologist in my lab about a few of the modalities since my biology background is limited to college biology and my microscopy knowledge is non-existent. However, I’ve managed to be good. I think they are entertained by my newfound interest in their fields of expertise but disappointed I’m still using phrases like “purple foreground and purple background” to describe something that’s probably significant.", "275389": "Whew. A very busy weekend. I posted a new score. I’m still overfitting, but I’ll deal with that in the future. What changed? A lot. Here’s a few of the most noteworthy changes:\n\n* I’m using a Feature Pyramid Network (FPN) before both the region proposal network (RPN) and the region-based convolutional neural network (RCNN). I think most of the improvement can be attributed to this.\n\n* I added a basic augmentation loop (random crops and some focus shifts).\n\n* I normalized the images (by mean subtraction).\n\nThere’s a few other things, but I’m too tired to remember! I’ll post more Monday or Tuesday.", "278694": "Thanks for sharing, Allen! I have read your Mask Rcnn code. I found that you only use the last three outputs of ResNet50, so the largest size of feature map is 32*32 if the input size is 256*256. It means that the smallest size of anchor grids is 8*8, but many nuclei in the samples are smaller than 8*8 pixels, so the model can not figure out two different nuclei if both of them are in one grid. How do you deal with this situation?", "273516": "Allen, \n\nIts great to have an expert providing a continuous benchmark:) This is a great addition and great motivator for other competitors. ", "275986": "Hi Allen, \n\n> The classes were images with white backgrounds, images with black backgrounds, and images with purple foreground and/or purple background. I further stratified the “purple” class into three new classes: images with purple foreground and purple background, images with purple foreground and white background, and images with purple foreground and yellow background.\n\nAny guesses on what's the difference between those within the purple class which u further segmented into \n\n 1. Purple foreground purple background\n 2. purple foreground and white background\n 3. purple foreground and yellow background\n\nWould it be some sort of artifact introduced such as the imaging lab or batch?", "294218": "Hi Allen,\n\nThanks for sharing.\n\nI encountered a problem that when I use mask-rcnn in Detectron to detect nuclei, It seems marked the same cell multiple times, This leads me to still cannot submit the result. How can I do to make my result be able to submit?\n\nAnything will be helpful!", "288283": "Can you give more details about how to make supplement data? I can't understand why it's useful. Thanks for your work!", "280902": "hi Allen, thank you very much for sharing\n\nHope you don't mind a few questions/clarifications about using the \"mask rcnn\":\n- I downloaded the \"features/preprocessing-objectdetection\" branch\n- and created a new model based on the RCNN, but adding the keras_rcnn.layers.MaskRCNN() at the end\n\nWhat I wasn't sure how to do:\n1.) how to initialize the output_masks? currently I have:\noutput_masks = keras.layers.TimeDistributed(keras.layers.Dense(mask_r * mask_c, kernel_initializer=\"zero\"))(output_features)\noutput_masks = Reshape(target_shape=(-1, mask_r, mask_c,1))(output_masks)\noutput_masks = keras_rcnn.layers.MaskRCNN()([target_categories, target_masks, output_masks])\n\n2.) how to properly train the model?\nWith the above model I get a runtime error:\n\nInvalidArgumentError (see above for traceback): Incompatible shapes: [1,32,28,28] vs. [1,7,28,28]\n[[Node: mask_rcnn_1/logistic_loss/mul = Mul[T=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:GPU:0\"](mask_rcnn_1/Log, _arg_target_images_0_4/_81)]]\n\nthe [1,7,28,28] is the target mask (tried with a generator which returns just one file, and has 7 masks)\nbut where does the [1,32,28,28] come from? Does it mean the model predicted 32 masks? ", "280562": "I've got a question about creating the submission. How are you dealing with overlapping masks?\nFirst I just created one mask image from all the predictions but I think that's not optimal and the score is lower because of that. But when I create single masks per nuclei than there are overlapping masks and I get an Exception when submitting the solution.", "279030": "Hi Allen, do you have any idea about the issue posted [here]. It seems that the training process is locked or killed somehow by using keras_rcnn on both python3.5 and python3.6.\n\n [here]: https://www.kaggle.com/kmader/keras-rcnn-based-overview-wip/comments#279024", "277825": "Thanks Allen, I have a question regarding image size. Does your model allows to use different input sizes or all the images need to have the same size? If that is the case which size are you using? \nThanks", "275482": "Gread job, Allen.\nI try to use the maskRCNN model to train and segment the images with keras_rcnn.\nBut the loss is not decay．I think RCNNMaskLoss is necessary．\nWould you tell me the usage of RCNNMaskLoss.\nMany thanks!\n", "273098": "Great! you have thought out of the box. Almost team are following the UNet, I think that UNet is for semantic segmentation while the challenge somehow differences as you mentioned. I think you applied Mask-RCNN (first detect the cell region and then segment the object from individual boxes). Am I right? I am selecting the good framework for this challenge: Tensorflow, caffe2. The benefit of caffe2 is that it released the Mask-RCNN, while Tensorflow is flexible for training, such as GAN model. Which model should I select for this challenge?", "283778": "Hi allen,\n\nThanks for sharing.\n\nCould you also sharing the score with different method? for example pure unet:0.2, with data augumentation 0.3, with resnxt50 0.35, pure mask rcnn 0.5, with data augumentation 0.6\n\nHere i did not mean unet will be worse than maskrcnn. Just from the public kernel, most people start with unet get score below 0.3.\nThe architecture of unet is very simple compared with faster rcnn(maybe I should use the word beautiful). This maybe the reason lots of people choose it to start.", "282856": "Hi Allen, \nHave you cleaned the data before training your model?\n\nToday I have been visualizing the annotated masks and the quality is so terrible that your current score seems like magic.", "278033": "Hi Allen,\nthanks for sharing!!!\nA doubt about overfitting: do you just experience a gap between training and validation or also between your validation and submission scores? I ask you because I'm mimicking the competition score function but my validation score is always 0.1 higher than the related submission score (which is A LOT), although almost coherent with it.\nThanks in advance!!!", "277195": "Allen, how much did it take to train your current score? I mean in terms of learning time of rcnn / epochs / steps per epoch ? Just want to have some time benchmark. Thank you!\n", "273250": "Can you pls share your approach for train set preparation? what technique helped you the most (load as grey / resize/ slicing / etc)?", "273054": "@Allen Goodman\n\nyou current LB score 0.516 is impressive. Any hints on the methods you used?\n", "306604": "By the way, your github profile picture is fantastic!!", "280749": "Hi, I've been attempting the [rcnn getting started tutorial][1] from the readme of rcnn github but have had issues are follows: \n\nIssues I have managed to resolve:\n\n- There were no imports provided so I added those required.\n\n- I seemed to be missing malaria_phenotypes.py from the pip install but just added to the appropriate place manually.\n\nIssues I have not yet resolved:\n\n-In python 2 it fails at the following line (complaining about the number of arguments):\n\n> model = keras_rcnn.models.RCNN(image, classes=len(classes) + 1)\n\n-In python 3.5 and 3.6 on different versions of tensorflow it crashes on the line:\n\n> from keras_rcnn.preprocessing import ObjectDetectionGenerator\n\nHas anyone got the tutorial working? If so are you able to share any tips or share a full .py file with a list of versions you are using? \n\nReally grateful for any help anyone can offer!\n\n [1]: https://github.com/broadinstitute/keras-rcnn", "278616": "Hi, Allen! Could you share your experience on how to prevent overfitting when training mask rcnn with resnet50 + fpn backbone. The model overfits soon after running about 40-50 epochs, and the validation loss is far more higher than the training loss.", "274356": "hello Allen ,i want to know whether the stage1_sample_submission.csv a perfect mask for train data? i make a csv file by the mask pictuers of train data and use RLE,but the csv file is different from stage1_sample_submission.csv ", "273002": "Thank you for your post, it provided some clues for me to improve my score." } }