I am using mask r-cnn to do instance segmentation, the results looks good, but the score is only 0.19. Any suggestions?
\n\nfor n, id_ in enumerate(test_ids):\nimage = cv2.imread(os.path.join(test_path, id_, 'images', id_) + '.png', 1)\nresized_image, window, scale, padding = utils.resize_image(\n image,\n min_dim=config.IMAGE_MIN_DIM,\n max_dim=config.IMAGE_MAX_DIM,\n padding=config.IMAGE_PADDING)\n\nresults = model.detect([resized_image], verbose=1)\nr = results[0]\n\nfor index in range(r['masks'].shape[-1]):\n mask_origin_size = cv2.resize(r['masks'][:, :, index],\n (image.shape[1], image.shape[0]), interpolation=cv2.INTER_NEAREST)\n rle = list(prob_to_rles(mask_origin_size))\n rles.extend(rle)\n new_test_ids.extend([id_] * len(rle))\n\nsub = pd.DataFrame()\nsub['ImageId'] = new_test_ids\nsub['EncodedPixels'] = pd.Series(rles).apply(lambda x: ' '.join(str(y) for y in x))\nsub.to_csv('sub.csv', index=False)\nI am using mask r-cnn to do instance segmentation, the results looks good, but the score is only 0.19. Any suggestions?
\n\nfor n, id_ in enumerate(test_ids):\nimage = cv2.imread(os.path.join(test_path, id_, 'images', id_) + '.png', 1)\nresized_image, window, scale, padding = utils.resize_image(\n image,\n min_dim=config.IMAGE_MIN_DIM,\n max_dim=config.IMAGE_MAX_DIM,\n padding=config.IMAGE_PADDING)\n\nresults = model.detect([resized_image], verbose=1)\nr = results[0]\n\nfor index in range(r['masks'].shape[-1]):\n mask_origin_size = cv2.resize(r['masks'][:, :, index],\n (image.shape[1], image.shape[0]), interpolation=cv2.INTER_NEAREST)\n rle = list(prob_to_rles(mask_origin_size))\n rles.extend(rle)\n new_test_ids.extend([id_] * len(rle))\n\nsub = pd.DataFrame()\nsub['ImageId'] = new_test_ids\nsub['EncodedPixels'] = pd.Series(rles).apply(lambda x: ' '.join(str(y) for y in x))\nsub.to_csv('sub.csv', index=False)\nDo you mind sharing you code?
", "rawMarkdown": "Do you mind sharing you code?", "votes": null }, { "id": "276258", "postDate": "01/31/2018 03:18:16", "content": "I modified \"train_shapes.ipynb\" file in this code repository: https://github.com/matterport/Mask_RCNN. And then trained on the nuclei data set.
", "rawMarkdown": "I modified \"train_shapes.ipynb\" file in this code repository: https://github.com/matterport/Mask_RCNN. And then trained on the nuclei data set.", "votes": null }, { "id": "276267", "postDate": "01/31/2018 03:30:19", "content": "I also observed that the results visually getting better and the score is not improving.
", "rawMarkdown": "I also observed that the results visually getting better and the score is not improving.", "votes": null }, { "id": "276285", "postDate": "01/31/2018 05:09:28", "content": "I am did the same thing.\nI set params:\nUSE_MINI_MASK = False,\nIMAGE_PADDING = False,\nNUM_CLASSES = 2\nIMAGE_MIN_DIM = 128\nIMAGE_MAX_DIM = 128\nand train on the nuclei data set.
", "rawMarkdown": "I am did the same thing.\nI set params:\nUSE_MINI_MASK = False,\nIMAGE_PADDING = False,\nNUM_CLASSES = 2\nIMAGE_MIN_DIM = 128\nIMAGE_MAX_DIM = 128\nand train on the nuclei data set.", "votes": null }, { "id": "276287", "postDate": "01/31/2018 05:19:36", "content": "How many epochs did you use? How about the score?
", "rawMarkdown": "How many epochs did you use? How about the score?", "votes": null }, { "id": "276312", "postDate": "01/31/2018 06:45:43", "content": "Training is not over yet.Training the maskrcnn model is slowly.my epoch is 120.
", "rawMarkdown": "Training is not over yet.Training the maskrcnn model is slowly.my epoch is 120.", "votes": null }, { "id": "276432", "postDate": "01/31/2018 13:19:39", "content": "how do you process the mask dataset? convert it to coco dataset?
", "rawMarkdown": "how do you process the mask dataset? convert it to coco dataset?", "votes": null }, { "id": "276474", "postDate": "01/31/2018 15:28:31", "content": "Is the mask_origin_size = cv2.resize necessary? Seems that in the mode.detect, the masks are resized back to original size
", "rawMarkdown": "Is the mask_origin_size = cv2.resize necessary? Seems that in the mode.detect, the masks are resized back to original size", "votes": null }, { "id": "276482", "postDate": "01/31/2018 15:43:46", "content": "BTW, can you share your config file? I am using the same code base, but sometimes I meet NaN loss --
", "rawMarkdown": "BTW, can you share your config file? I am using the same code base, but sometimes I meet NaN loss --", "votes": null }, { "id": "276631", "postDate": "02/01/2018 01:07:53", "content": "I submissioned my results,but the following exceptions occurred e.g.\none or more submitted values exceeded the image dimensions
", "rawMarkdown": "I submissioned my results,but the following exceptions occurred e.g.\none or more submitted values exceeded the image dimensions", "votes": null }, { "id": "276637", "postDate": "02/01/2018 01:25:47", "content": "how do you process the mask dataset?
", "rawMarkdown": "how do you process the mask dataset?", "votes": null }, { "id": "277002", "postDate": "02/02/2018 01:07:40", "content": "have you solve the problem, I met the same problem with you.
", "rawMarkdown": "have you solve the problem, I met the same problem with you.", "votes": null }, { "id": "277011", "postDate": "02/02/2018 01:45:44", "content": "The num of results is less the U-net results.but I can't solve it:(.
", "rawMarkdown": "The num of results is less the U-net results.but I can't solve it:(.", "votes": null }, { "id": "277013", "postDate": "02/02/2018 01:50:55", "content": "@iFighting you'll need to sub-class like:\nDSB2018Dataset(utils.Dataset):provides a consistent way to training\nDSB2018Config(Config):modify the attributes you need to change.
", "rawMarkdown": "iFighting you'll need to sub-class like:\nDSB2018Dataset(utils.Dataset):provides a consistent way to training\nDSB2018Config(Config):modify the attributes you need to change.", "votes": null }, { "id": "277209", "postDate": "02/02/2018 14:33:33", "content": "Can you explain how can we modify the train_shapes.ipynb. I am facing problems with it.
", "rawMarkdown": "Can you explain how can we modify the train_shapes.ipynb. I am facing problems with it.", "votes": null }, { "id": "277395", "postDate": "02/03/2018 01:08:26", "content": "I can suggest to you something: 1. You have to split dataset into 3 or 5 image groups. You may use knn to do it. 2. Use coco pretrain model. 3. Dropout or early stop to handle with overfitting. Have you consider all of them?
", "rawMarkdown": "I can suggest to you something: 1. You have to split dataset into 3 or 5 image groups. You may use knn to do it. 2. Use coco pretrain model. 3. Dropout or early stop to handle with overfitting. Have you consider all of them?", "votes": null }, { "id": "278200", "postDate": "02/05/2018 16:16:06", "content": "If I understand correctly, the 0.19 is the LB score. The results looks relative good to me, what about other images? I know that model (U-Net) model is doing good on this kind of images but on other images it fail.
", "rawMarkdown": "If I understand correctly, the 0.19 is the LB score. The results looks relative good to me, what about other images? I know that model (U-Net) model is doing good on this kind of images but on other images it fail.", "votes": null }, { "id": "279341", "postDate": "02/07/2018 21:08:47", "content": "I used a basic setting, and got LB 0.29. Will tune it further
", "rawMarkdown": "I used a basic setting, and got LB 0.29. Will tune it further", "votes": null }, { "id": "279626", "postDate": "02/08/2018 11:45:22", "content": "Do you have a working config file that you could share? I'm falling over on the bug below but don't know whether it's something to do with my config, or something else...
\n\nslice index 7 of dimension 0 out of bounds.
\n\n [[Node: rpn_bbox_loss/strided_slice_14 = StridedSlice[Index=DT_INT32, T=DT_INT32, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](rpn_bbox_loss/Sum, rpn_bbox_loss/strided_slice_12/stack_1, rpn_bbox_loss/strided_slice_14/stack_1, rpn_bbox_loss/strided_slice/stack_1)]]\nEDIT: found what was causing this error. I had IMAGES_PER_GPU > batch size, which leads to an error in batch_pack_graph(). Now to dig into the next reason why it's falling over........
", "rawMarkdown": "Do you have a working config file that you could share? I'm falling over on the bug below but don't know whether it's something to do with my config, or something else...\n\nslice index 7 of dimension 0 out of bounds.\n\n\t [[Node: rpn_bbox_loss/strided_slice_14 = StridedSlice[Index=DT_INT32, T=DT_INT32, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](rpn_bbox_loss/Sum, rpn_bbox_loss/strided_slice_12/stack_1, rpn_bbox_loss/strided_slice_14/stack_1, rpn_bbox_loss/strided_slice/stack_1)]]\n\nEDIT: found what was causing this error. I had IMAGES_PER_GPU > batch size, which leads to an error in batch_pack_graph(). Now to dig into the next reason why it's falling over........", "votes": null }, { "id": "279696", "postDate": "02/08/2018 14:54:42", "content": "This should work:\n # Give the configuration a recognizable name\n NAME = \"nuclei\"\n train_data_root = 'data/stage1_train/' \n val_data_root = 'data/stage1_train/'\n test_data_root = 'data/stage1_test/'
\n\nMODEL_DIR = 'checkpoints'\nCOCO_MODEL_PATH = 'mask_rcnn_coco.h5'\n# imagenet, coco, or last\ninit_with = \"last\" \n\nLEARNING_RATE = 0.001\n\n# If enabled, resizes instance masks to a smaller size to reduce\n# memory load. Recommended when using high-resolution images.\nUSE_MINI_MASK = False\nMINI_MASK_SHAPE = (56, 56) # (height, width) of the mini-mask\n\n# Train on 1 GPU and 8 images per GPU. Batch size is GPUs * images/GPU.\nGPU_COUNT = 2\nIMAGES_PER_GPU = 8\nbs = GPU_COUNT * IMAGES_PER_GPU\n# Total number of steps (batches of samples) to yield from generator before declaring one epoch finished and starting the next epoch.\n# typically be equal to the number of samples of your dataset divided by the batch size\nSTEPS_PER_EPOCH = 620//bs\nVALIDATION_STEPS = 70//bs\n\n# Number of classes (including background)\nNUM_CLASSES = 1 + 1 # background + nucleis\n\n# Input image resing\n# Images are resized such that the smallest side is >= IMAGE_MIN_DIM and\n# the longest side is <= IMAGE_MAX_DIM. In case both conditions can't\n# be satisfied together the IMAGE_MAX_DIM is enforced.\nIMAGE_MIN_DIM = 256\nIMAGE_MAX_DIM = 256\n# If True, pad images with zeros such that they're (max_dim by max_dim)\n IMAGE_PADDING = True # currently, the False option is not supported
\n\n# Use smaller anchors because our image and objects are small\nRPN_ANCHOR_SCALES = (8, 16, 32, 64, 128) # anchor side in pixels\n\n# How many anchors per image to use for RPN training\nRPN_TRAIN_ANCHORS_PER_IMAGE = 256\n\n# Number of ROIs per image to feed to classifier/mask heads\n# The Mask RCNN paper uses 512 but often the RPN doesn't generate\n# enough positive proposals to fill this and keep a positive:negative\n# ratio of 1:3. You can increase the number of proposals by adjusting\n# the RPN NMS threshold.\n# Reduce training ROIs per image because the images are small and have\n# few objects. Aim to allow ROI sampling to pick 33% positive ROIs.\nTRAIN_ROIS_PER_IMAGE = 128\n\n# Maximum number of ground truth instances to use in one image\nMAX_GT_INSTANCES = 120\n\n# Max number of final detections\nDETECTION_MAX_INSTANCES = 120\nCheers
", "rawMarkdown": "Cheers", "votes": null }, { "id": "279732", "postDate": "02/08/2018 16:06:09", "content": "Hi Yan Wang,
\n\nYou mentioned in your first post \"I am using the same code base, but sometimes I meet NaN loss\"
\n\nNow that I have the code base working I'm also encountering this. Any useful findings for avoiding it? The usual learning-rate decreases / batch size decreases don't seem to be making a difference.
", "rawMarkdown": "Hi Yan Wang,\n\nYou mentioned in your first post \"I am using the same code base, but sometimes I meet NaN loss\"\n\nNow that I have the code base working I'm also encountering this. Any useful findings for avoiding it? The usual learning-rate decreases / batch size decreases don't seem to be making a difference.", "votes": null }, { "id": "279832", "postDate": "02/08/2018 19:49:22", "content": "Seems that this two parameter should not be too small like 128, otherwise some mask may be lost during the resize operation, causing the NaN problem.
\n\nIMAGE_MIN_DIM = 256\nIMAGE_MAX_DIM = 256\nAh, ok, thanks. I had been doing a test run with 128, 128. That was probably what was causing it.
", "rawMarkdown": "Ah, ok, thanks. I had been doing a test run with 128, 128. That was probably what was causing it.", "votes": null }, { "id": "279889", "postDate": "02/08/2018 22:16:16", "content": "Hi Yan. You are in right direction. I am also did same as you but I did not submit result yet. Could you tell me your testing score using your configuration? Thanks
", "rawMarkdown": "Hi Yan. You are in right direction. I am also did same as you but I did not submit result yet. Could you tell me your testing score using your configuration? Thanks", "votes": null }, { "id": "279917", "postDate": "02/09/2018 00:06:53", "content": "Hi John, I got 0.307 using a similar configuration. I am running more computation demanding ones.
", "rawMarkdown": "Hi John, I got 0.307 using a similar configuration. I am running more computation demanding ones.", "votes": null }, { "id": "279923", "postDate": "02/09/2018 00:25:51", "content": "Great. I want to suggest you to split data into 3 clusters or 5 clusters as many guys did and then normalize or enhancement method as preprocessing. It will increase your performance. I have one more question that is how you can achieve instance segmented image in testing phase? Bacause you cropped image into 256, but testing images are various dimension. AFAIK, the maskrcnn only allows fixed size because it used fully connected in detection part.
", "rawMarkdown": "Great. I want to suggest you to split data into 3 clusters or 5 clusters as many guys did and then normalize or enhancement method as preprocessing. It will increase your performance. I have one more question that is how you can achieve instance segmented image in testing phase? Bacause you cropped image into 256, but testing images are various dimension. AFAIK, the maskrcnn only allows fixed size because it used fully connected in detection part.", "votes": null }, { "id": "279929", "postDate": "02/09/2018 00:43:03", "content": "Fixed size is required because we want to put several images in one mini batch. The code remember the original image size, and then resize the obtained masks back to the original size.
", "rawMarkdown": "Fixed size is required because we want to put several images in one mini batch. The code remember the original image size, and then resize the obtained masks back to the original size.", "votes": null }, { "id": "280022", "postDate": "02/09/2018 05:24:25", "content": "I found directly e2e training performs better. I used the resnet50-fpn backbone and have experienced severe overfitting. The validation loss goes extremely unstable also. Does anyone have some ideas to fix this?
", "rawMarkdown": "I found directly e2e training performs better. I used the resnet50-fpn backbone and have experienced severe overfitting. The validation loss goes extremely unstable also. Does anyone have some ideas to fix this?", "votes": null }, { "id": "280024", "postDate": "02/09/2018 05:29:47", "content": "I would suggest you to use higher resolution image size, or just for some specific images. 256 * 256 is not large enough that you may still get some zero size bbox. You can set use_mini_mask = True to check if for it has a exception check that tells you which image gets this exception.
", "rawMarkdown": "I would suggest you to use higher resolution image size, or just for some specific images. 256 * 256 is not large enough that you may still get some zero size bbox. You can set use_mini_mask = True to check if for it has a exception check that tells you which image gets this exception.", "votes": null }, { "id": "280146", "postDate": "02/09/2018 13:23:41", "content": "Thank you. I will try larger images. Unfortunately, this implementation is a bit too slow, I am going to work with some faster implementations.
", "rawMarkdown": "Thank you. I will try larger images. Unfortunately, this implementation is a bit too slow, I am going to work with some faster implementations.", "votes": null }, { "id": "280168", "postDate": "02/09/2018 13:57:46", "content": "Hi Yan, when you encode the result for submission, which kind of result (instance segmentation or semantic segmentation) do you use for input RLE? For example, semantic segmentation likes UNet result, while instance segmentation like maskrcnn. Because we use RLE to encode the masks, so I think semantic and instance have same performance (if they are made from same ground-truth). Am I right?
", "rawMarkdown": "Hi Yan, when you encode the result for submission, which kind of result (instance segmentation or semantic segmentation) do you use for input RLE? For example, semantic segmentation likes UNet result, while instance segmentation like maskrcnn. Because we use RLE to encode the masks, so I think semantic and instance have same performance (if they are made from same ground-truth). Am I right?", "votes": null }, { "id": "280175", "postDate": "02/09/2018 14:11:46", "content": "For Mask R-CNN, I encode each instance independently.
", "rawMarkdown": "For Mask R-CNN, I encode each instance independently.", "votes": null }, { "id": "280180", "postDate": "02/09/2018 14:19:35", "content": "So you first resize the testing image into 256x256. After obtain the instance segmentation result, you resize the instance results to the original testing image. Am I right? If yes, then you may lost so much information in resize step
", "rawMarkdown": "So you first resize the testing image into 256x256. After obtain the instance segmentation result, you resize the instance results to the original testing image. Am I right? If yes, then you may lost so much information in resize step", "votes": null }, { "id": "280207", "postDate": "02/09/2018 15:05:24", "content": "This implementation seems the best I've found based on tensorflow, although it has some differences between the original paper's implementation, for example the ROIAlign layer. Or you could try to implement your own maskrcnn based on tensorflow's official faster-rcnn model. FAIR's detectron is better for it is the official implementation and support all tricks mentioned in the paper to improve its performance, but it does not support training on your own dataset.
", "rawMarkdown": "This implementation seems the best I've found based on tensorflow, although it has some differences between the original paper's implementation, for example the ROIAlign layer. Or you could try to implement your own maskrcnn based on tensorflow's official faster-rcnn model. FAIR's detectron is better for it is the official implementation and support all tricks mentioned in the paper to improve its performance, but it does not support training on your own dataset.", "votes": null }, { "id": "280224", "postDate": "02/09/2018 15:26:29", "content": "Yes, perfect implementation is very limited now. Seems it's possible to use detectron as shown here. Unfortunately I have no root access to install caffe2. BTW, I am also from THU. Wechat: wizard_123
", "rawMarkdown": "Yes, perfect implementation is very limited now. Seems it's possible to use detectron [as shown here][1]. Unfortunately I have no root access to install caffe2. BTW, I am also from THU. Wechat: wizard_123\n\n\n [1]: https://github.com/facebookresearch/Detectron/issues/98", "votes": null }, { "id": "280280", "postDate": "02/09/2018 17:18:16", "content": "By e2e do you mean that you load the resnet50 fpn backbone but train the weights from scratch rather than using imagenet weights? Or use a different backbone completely?
", "rawMarkdown": "By e2e do you mean that you load the resnet50 fpn backbone but train the weights from scratch rather than using imagenet weights? Or use a different backbone completely?", "votes": null }, { "id": "280297", "postDate": "02/09/2018 17:30:07", "content": "Can I check something with anyone who is using Mask_RCNN:
\n\nI have
\n\nactive_class_ids = np.array([0, 1], dtype=np.int32) # Background, Nucleus
\n\nand then each mask needs to be assigned a class_id, but in our case that is 1 for each mask, so class_id is just an array of 1's? I think I'm missing something somewhere...
", "rawMarkdown": "Can I check something with anyone who is using Mask_RCNN:\n\nI have \n\nactive_class_ids = np.array([0, 1], dtype=np.int32) # Background, Nucleus\n\nand then each mask needs to be assigned a class_id, but in our case that is 1 for each mask, so class_id is just an array of 1's? I think I'm missing something somewhere...", "votes": null }, { "id": "280791", "postDate": "02/11/2018 02:43:30", "content": "You're correct. The resize operation should be removed
", "rawMarkdown": "You're correct. The resize operation should be removed", "votes": null }, { "id": "280794", "postDate": "02/11/2018 02:53:20", "content": "Some updates:\n1. Remove the cv2.resize code\n2. Use larger image size;\n3. Split the nuclei to multiple cluster\nThese steps can help to improve the performance.
", "rawMarkdown": "Some updates:\n1. Remove the cv2.resize code\n2. Use larger image size;\n3. Split the nuclei to multiple cluster\nThese steps can help to improve the performance.", "votes": null }, { "id": "280796", "postDate": "02/11/2018 03:06:27", "content": "class_id is 1 for each mask. But it's better to split the mask into different kinds.
", "rawMarkdown": "class_id is 1 for each mask. But it's better to split the mask into different kinds.", "votes": null }, { "id": "280799", "postDate": "02/11/2018 03:48:01", "content": "I don't think it is a good idea to cluster into 3 groups, because you don't know how many groups are there in stage2 data. If there are lots of different groups this method I think would fail.
", "rawMarkdown": "I don't think it is a good idea to cluster into 3 groups, because you don't know how many groups are there in stage2 data. If there are lots of different groups this method I think would fail.", "votes": null }, { "id": "280886", "postDate": "02/11/2018 10:11:47", "content": "i saw that here are images with 350 nuclies so i think i should change the DETECTION_MAX_INSTANCES to more then 100\nshould i change the RPN_TRAIN_ANCHORS_PER_IMAGE too?
", "rawMarkdown": "i saw that here are images with 350 nuclies so i think i should change the DETECTION_MAX_INSTANCES to more then 100\nshould i change the RPN_TRAIN_ANCHORS_PER_IMAGE too?", "votes": null } ], "comments": [ { "id": 276251, "author_name": "dskswu", "author_url": "", "post_date": "01/31/2018 03:09:42", "content": "Do you mind sharing you code?
", "votes": null, "replies": [ { "id": 276258, "author_name": "zmingen", "author_url": "", "post_date": "01/31/2018 03:18:16", "content": "I modified \"train_shapes.ipynb\" file in this code repository: https://github.com/matterport/Mask_RCNN. And then trained on the nuclei data set.
", "votes": null, "replies": [] }, { "id": 277209, "author_name": "nikhilroxtomar", "author_url": "", "post_date": "02/02/2018 14:33:33", "content": "Can you explain how can we modify the train_shapes.ipynb. I am facing problems with it.
", "votes": null, "replies": [] } ] }, { "id": 276267, "author_name": "vijaybj", "author_url": "", "post_date": "01/31/2018 03:30:19", "content": "I also observed that the results visually getting better and the score is not improving.
", "votes": null, "replies": [] }, { "id": 276285, "author_name": "zhouzhiguang", "author_url": "", "post_date": "01/31/2018 05:09:28", "content": "I am did the same thing.\nI set params:\nUSE_MINI_MASK = False,\nIMAGE_PADDING = False,\nNUM_CLASSES = 2\nIMAGE_MIN_DIM = 128\nIMAGE_MAX_DIM = 128\nand train on the nuclei data set.
", "votes": null, "replies": [ { "id": 276287, "author_name": "zmingen", "author_url": "", "post_date": "01/31/2018 05:19:36", "content": "How many epochs did you use? How about the score?
", "votes": null, "replies": [] }, { "id": 276312, "author_name": "zhouzhiguang", "author_url": "", "post_date": "01/31/2018 06:45:43", "content": "Training is not over yet.Training the maskrcnn model is slowly.my epoch is 120.
", "votes": null, "replies": [] }, { "id": 276631, "author_name": "zhouzhiguang", "author_url": "", "post_date": "02/01/2018 01:07:53", "content": "I submissioned my results,but the following exceptions occurred e.g.\none or more submitted values exceeded the image dimensions
", "votes": null, "replies": [] }, { "id": 276637, "author_name": "ifighting", "author_url": "", "post_date": "02/01/2018 01:25:47", "content": "how do you process the mask dataset?
", "votes": null, "replies": [] }, { "id": 277002, "author_name": "chuanzhixu", "author_url": "", "post_date": "02/02/2018 01:07:40", "content": "have you solve the problem, I met the same problem with you.
", "votes": null, "replies": [] }, { "id": 277011, "author_name": "zhouzhiguang", "author_url": "", "post_date": "02/02/2018 01:45:44", "content": "The num of results is less the U-net results.but I can't solve it:(.
", "votes": null, "replies": [] }, { "id": 277013, "author_name": "zhouzhiguang", "author_url": "", "post_date": "02/02/2018 01:50:55", "content": "@iFighting you'll need to sub-class like:\nDSB2018Dataset(utils.Dataset):provides a consistent way to training\nDSB2018Config(Config):modify the attributes you need to change.
", "votes": null, "replies": [] } ] }, { "id": 276432, "author_name": "ifighting", "author_url": "", "post_date": "01/31/2018 13:19:39", "content": "how do you process the mask dataset? convert it to coco dataset?
", "votes": null, "replies": [] }, { "id": 276474, "author_name": "wayne999", "author_url": "", "post_date": "01/31/2018 15:28:31", "content": "Is the mask_origin_size = cv2.resize necessary? Seems that in the mode.detect, the masks are resized back to original size
", "votes": null, "replies": [ { "id": 280791, "author_name": "zmingen", "author_url": "", "post_date": "02/11/2018 02:43:30", "content": "You're correct. The resize operation should be removed
", "votes": null, "replies": [] } ] }, { "id": 276482, "author_name": "wayne999", "author_url": "", "post_date": "01/31/2018 15:43:46", "content": "BTW, can you share your config file? I am using the same code base, but sometimes I meet NaN loss --
", "votes": null, "replies": [ { "id": 279341, "author_name": "wayne999", "author_url": "", "post_date": "02/07/2018 21:08:47", "content": "I used a basic setting, and got LB 0.29. Will tune it further
", "votes": null, "replies": [] }, { "id": 279626, "author_name": "fergusoci", "author_url": "", "post_date": "02/08/2018 11:45:22", "content": "Do you have a working config file that you could share? I'm falling over on the bug below but don't know whether it's something to do with my config, or something else...
\n\nslice index 7 of dimension 0 out of bounds.
\n\n [[Node: rpn_bbox_loss/strided_slice_14 = StridedSlice[Index=DT_INT32, T=DT_INT32, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](rpn_bbox_loss/Sum, rpn_bbox_loss/strided_slice_12/stack_1, rpn_bbox_loss/strided_slice_14/stack_1, rpn_bbox_loss/strided_slice/stack_1)]]\nEDIT: found what was causing this error. I had IMAGES_PER_GPU > batch size, which leads to an error in batch_pack_graph(). Now to dig into the next reason why it's falling over........
", "votes": null, "replies": [] }, { "id": 279696, "author_name": "wayne999", "author_url": "", "post_date": "02/08/2018 14:54:42", "content": "This should work:\n # Give the configuration a recognizable name\n NAME = \"nuclei\"\n train_data_root = 'data/stage1_train/' \n val_data_root = 'data/stage1_train/'\n test_data_root = 'data/stage1_test/'
\n\nMODEL_DIR = 'checkpoints'\nCOCO_MODEL_PATH = 'mask_rcnn_coco.h5'\n# imagenet, coco, or last\ninit_with = \"last\" \n\nLEARNING_RATE = 0.001\n\n# If enabled, resizes instance masks to a smaller size to reduce\n# memory load. Recommended when using high-resolution images.\nUSE_MINI_MASK = False\nMINI_MASK_SHAPE = (56, 56) # (height, width) of the mini-mask\n\n# Train on 1 GPU and 8 images per GPU. Batch size is GPUs * images/GPU.\nGPU_COUNT = 2\nIMAGES_PER_GPU = 8\nbs = GPU_COUNT * IMAGES_PER_GPU\n# Total number of steps (batches of samples) to yield from generator before declaring one epoch finished and starting the next epoch.\n# typically be equal to the number of samples of your dataset divided by the batch size\nSTEPS_PER_EPOCH = 620//bs\nVALIDATION_STEPS = 70//bs\n\n# Number of classes (including background)\nNUM_CLASSES = 1 + 1 # background + nucleis\n\n# Input image resing\n# Images are resized such that the smallest side is >= IMAGE_MIN_DIM and\n# the longest side is <= IMAGE_MAX_DIM. In case both conditions can't\n# be satisfied together the IMAGE_MAX_DIM is enforced.\nIMAGE_MIN_DIM = 256\nIMAGE_MAX_DIM = 256\n# If True, pad images with zeros such that they're (max_dim by max_dim)\n IMAGE_PADDING = True # currently, the False option is not supported
\n\n# Use smaller anchors because our image and objects are small\nRPN_ANCHOR_SCALES = (8, 16, 32, 64, 128) # anchor side in pixels\n\n# How many anchors per image to use for RPN training\nRPN_TRAIN_ANCHORS_PER_IMAGE = 256\n\n# Number of ROIs per image to feed to classifier/mask heads\n# The Mask RCNN paper uses 512 but often the RPN doesn't generate\n# enough positive proposals to fill this and keep a positive:negative\n# ratio of 1:3. You can increase the number of proposals by adjusting\n# the RPN NMS threshold.\n# Reduce training ROIs per image because the images are small and have\n# few objects. Aim to allow ROI sampling to pick 33% positive ROIs.\nTRAIN_ROIS_PER_IMAGE = 128\n\n# Maximum number of ground truth instances to use in one image\nMAX_GT_INSTANCES = 120\n\n# Max number of final detections\nDETECTION_MAX_INSTANCES = 120\nCheers
", "votes": null, "replies": [] }, { "id": 279732, "author_name": "fergusoci", "author_url": "", "post_date": "02/08/2018 16:06:09", "content": "Hi Yan Wang,
\n\nYou mentioned in your first post \"I am using the same code base, but sometimes I meet NaN loss\"
\n\nNow that I have the code base working I'm also encountering this. Any useful findings for avoiding it? The usual learning-rate decreases / batch size decreases don't seem to be making a difference.
", "votes": null, "replies": [] }, { "id": 279832, "author_name": "wayne999", "author_url": "", "post_date": "02/08/2018 19:49:22", "content": "Seems that this two parameter should not be too small like 128, otherwise some mask may be lost during the resize operation, causing the NaN problem.
\n\nIMAGE_MIN_DIM = 256\nIMAGE_MAX_DIM = 256\nAh, ok, thanks. I had been doing a test run with 128, 128. That was probably what was causing it.
", "votes": null, "replies": [] }, { "id": 279889, "author_name": "john1231983", "author_url": "", "post_date": "02/08/2018 22:16:16", "content": "Hi Yan. You are in right direction. I am also did same as you but I did not submit result yet. Could you tell me your testing score using your configuration? Thanks
", "votes": null, "replies": [] }, { "id": 279917, "author_name": "wayne999", "author_url": "", "post_date": "02/09/2018 00:06:53", "content": "Hi John, I got 0.307 using a similar configuration. I am running more computation demanding ones.
", "votes": null, "replies": [] }, { "id": 279923, "author_name": "john1231983", "author_url": "", "post_date": "02/09/2018 00:25:51", "content": "Great. I want to suggest you to split data into 3 clusters or 5 clusters as many guys did and then normalize or enhancement method as preprocessing. It will increase your performance. I have one more question that is how you can achieve instance segmented image in testing phase? Bacause you cropped image into 256, but testing images are various dimension. AFAIK, the maskrcnn only allows fixed size because it used fully connected in detection part.
", "votes": null, "replies": [] }, { "id": 279929, "author_name": "wayne999", "author_url": "", "post_date": "02/09/2018 00:43:03", "content": "Fixed size is required because we want to put several images in one mini batch. The code remember the original image size, and then resize the obtained masks back to the original size.
", "votes": null, "replies": [] }, { "id": 280024, "author_name": "dingkun", "author_url": "", "post_date": "02/09/2018 05:29:47", "content": "I would suggest you to use higher resolution image size, or just for some specific images. 256 * 256 is not large enough that you may still get some zero size bbox. You can set use_mini_mask = True to check if for it has a exception check that tells you which image gets this exception.
", "votes": null, "replies": [] }, { "id": 280146, "author_name": "wayne999", "author_url": "", "post_date": "02/09/2018 13:23:41", "content": "Thank you. I will try larger images. Unfortunately, this implementation is a bit too slow, I am going to work with some faster implementations.
", "votes": null, "replies": [] }, { "id": 280168, "author_name": "john1231983", "author_url": "", "post_date": "02/09/2018 13:57:46", "content": "Hi Yan, when you encode the result for submission, which kind of result (instance segmentation or semantic segmentation) do you use for input RLE? For example, semantic segmentation likes UNet result, while instance segmentation like maskrcnn. Because we use RLE to encode the masks, so I think semantic and instance have same performance (if they are made from same ground-truth). Am I right?
", "votes": null, "replies": [] }, { "id": 280175, "author_name": "wayne999", "author_url": "", "post_date": "02/09/2018 14:11:46", "content": "For Mask R-CNN, I encode each instance independently.
", "votes": null, "replies": [] }, { "id": 280180, "author_name": "john1231983", "author_url": "", "post_date": "02/09/2018 14:19:35", "content": "So you first resize the testing image into 256x256. After obtain the instance segmentation result, you resize the instance results to the original testing image. Am I right? If yes, then you may lost so much information in resize step
", "votes": null, "replies": [] }, { "id": 280207, "author_name": "dingkun", "author_url": "", "post_date": "02/09/2018 15:05:24", "content": "This implementation seems the best I've found based on tensorflow, although it has some differences between the original paper's implementation, for example the ROIAlign layer. Or you could try to implement your own maskrcnn based on tensorflow's official faster-rcnn model. FAIR's detectron is better for it is the official implementation and support all tricks mentioned in the paper to improve its performance, but it does not support training on your own dataset.
", "votes": null, "replies": [] }, { "id": 280224, "author_name": "wayne999", "author_url": "", "post_date": "02/09/2018 15:26:29", "content": "Yes, perfect implementation is very limited now. Seems it's possible to use detectron as shown here. Unfortunately I have no root access to install caffe2. BTW, I am also from THU. Wechat: wizard_123
", "votes": null, "replies": [] }, { "id": 280297, "author_name": "fergusoci", "author_url": "", "post_date": "02/09/2018 17:30:07", "content": "Can I check something with anyone who is using Mask_RCNN:
\n\nI have
\n\nactive_class_ids = np.array([0, 1], dtype=np.int32) # Background, Nucleus
\n\nand then each mask needs to be assigned a class_id, but in our case that is 1 for each mask, so class_id is just an array of 1's? I think I'm missing something somewhere...
", "votes": null, "replies": [] }, { "id": 280796, "author_name": "zmingen", "author_url": "", "post_date": "02/11/2018 03:06:27", "content": "class_id is 1 for each mask. But it's better to split the mask into different kinds.
", "votes": null, "replies": [] }, { "id": 280799, "author_name": "dingkun", "author_url": "", "post_date": "02/11/2018 03:48:01", "content": "I don't think it is a good idea to cluster into 3 groups, because you don't know how many groups are there in stage2 data. If there are lots of different groups this method I think would fail.
", "votes": null, "replies": [] } ] }, { "id": 277395, "author_name": "john1231983", "author_url": "", "post_date": "02/03/2018 01:08:26", "content": "I can suggest to you something: 1. You have to split dataset into 3 or 5 image groups. You may use knn to do it. 2. Use coco pretrain model. 3. Dropout or early stop to handle with overfitting. Have you consider all of them?
", "votes": null, "replies": [] }, { "id": 278200, "author_name": "ori226", "author_url": "", "post_date": "02/05/2018 16:16:06", "content": "If I understand correctly, the 0.19 is the LB score. The results looks relative good to me, what about other images? I know that model (U-Net) model is doing good on this kind of images but on other images it fail.
", "votes": null, "replies": [] }, { "id": 280022, "author_name": "dingkun", "author_url": "", "post_date": "02/09/2018 05:24:25", "content": "I found directly e2e training performs better. I used the resnet50-fpn backbone and have experienced severe overfitting. The validation loss goes extremely unstable also. Does anyone have some ideas to fix this?
", "votes": null, "replies": [ { "id": 280280, "author_name": "fergusoci", "author_url": "", "post_date": "02/09/2018 17:18:16", "content": "By e2e do you mean that you load the resnet50 fpn backbone but train the weights from scratch rather than using imagenet weights? Or use a different backbone completely?
", "votes": null, "replies": [] } ] }, { "id": 280794, "author_name": "zmingen", "author_url": "", "post_date": "02/11/2018 02:53:20", "content": "Some updates:\n1. Remove the cv2.resize code\n2. Use larger image size;\n3. Split the nuclei to multiple cluster\nThese steps can help to improve the performance.
", "votes": null, "replies": [] }, { "id": 280886, "author_name": "aviwolfson", "author_url": "", "post_date": "02/11/2018 10:11:47", "content": "i saw that here are images with 350 nuclies so i think i should change the DETECTION_MAX_INSTANCES to more then 100\nshould i change the RPN_TRAIN_ANCHORS_PER_IMAGE too?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "276230": "I am using mask r-cnn to do instance segmentation, the results looks good, but the score is only 0.19. Any suggestions?\n\n for n, id_ in enumerate(test_ids):\n image = cv2.imread(os.path.join(test_path, id_, 'images', id_) + '.png', 1)\n resized_image, window, scale, padding = utils.resize_image(\n image,\n min_dim=config.IMAGE_MIN_DIM,\n max_dim=config.IMAGE_MAX_DIM,\n padding=config.IMAGE_PADDING)\n\n results = model.detect([resized_image], verbose=1)\n r = results[0]\n\n for index in range(r['masks'].shape[-1]):\n mask_origin_size = cv2.resize(r['masks'][:, :, index],\n (image.shape[1], image.shape[0]), interpolation=cv2.INTER_NEAREST)\n rle = list(prob_to_rles(mask_origin_size))\n rles.extend(rle)\n new_test_ids.extend([id_] * len(rle))\n\n sub = pd.DataFrame()\n sub['ImageId'] = new_test_ids\n sub['EncodedPixels'] = pd.Series(rles).apply(lambda x: ' '.join(str(y) for y in x))\n sub.to_csv('sub.csv', index=False)", "276251": "Do you mind sharing you code?", "276258": "I modified \"train_shapes.ipynb\" file in this code repository: https://github.com/matterport/Mask_RCNN. And then trained on the nuclei data set.", "276267": "I also observed that the results visually getting better and the score is not improving.", "276285": "I am did the same thing.\nI set params:\nUSE_MINI_MASK = False,\nIMAGE_PADDING = False,\nNUM_CLASSES = 2\nIMAGE_MIN_DIM = 128\nIMAGE_MAX_DIM = 128\nand train on the nuclei data set.", "276287": "How many epochs did you use? How about the score?", "276312": "Training is not over yet.Training the maskrcnn model is slowly.my epoch is 120.", "276432": "how do you process the mask dataset? convert it to coco dataset?", "276474": "Is the mask_origin_size = cv2.resize necessary? Seems that in the mode.detect, the masks are resized back to original size", "276482": "BTW, can you share your config file? I am using the same code base, but sometimes I meet NaN loss --", "276631": "I submissioned my results,but the following exceptions occurred e.g.\none or more submitted values exceeded the image dimensions", "276637": "how do you process the mask dataset?", "277002": "have you solve the problem, I met the same problem with you.", "277011": "The num of results is less the U-net results.but I can't solve it:(.", "277013": "iFighting you'll need to sub-class like:\nDSB2018Dataset(utils.Dataset):provides a consistent way to training\nDSB2018Config(Config):modify the attributes you need to change.", "277209": "Can you explain how can we modify the train_shapes.ipynb. I am facing problems with it.", "277395": "I can suggest to you something: 1. You have to split dataset into 3 or 5 image groups. You may use knn to do it. 2. Use coco pretrain model. 3. Dropout or early stop to handle with overfitting. Have you consider all of them?", "278200": "If I understand correctly, the 0.19 is the LB score. The results looks relative good to me, what about other images? I know that model (U-Net) model is doing good on this kind of images but on other images it fail.", "279341": "I used a basic setting, and got LB 0.29. Will tune it further", "279626": "Do you have a working config file that you could share? I'm falling over on the bug below but don't know whether it's something to do with my config, or something else...\n\nslice index 7 of dimension 0 out of bounds.\n\n\t [[Node: rpn_bbox_loss/strided_slice_14 = StridedSlice[Index=DT_INT32, T=DT_INT32, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](rpn_bbox_loss/Sum, rpn_bbox_loss/strided_slice_12/stack_1, rpn_bbox_loss/strided_slice_14/stack_1, rpn_bbox_loss/strided_slice/stack_1)]]\n\nEDIT: found what was causing this error. I had IMAGES_PER_GPU > batch size, which leads to an error in batch_pack_graph(). Now to dig into the next reason why it's falling over........", "279696": "This should work:\n # Give the configuration a recognizable name\n NAME = \"nuclei\"\n train_data_root = 'data/stage1_train/' \n val_data_root = 'data/stage1_train/'\n test_data_root = 'data/stage1_test/'\n \n MODEL_DIR = 'checkpoints'\n COCO_MODEL_PATH = 'mask_rcnn_coco.h5'\n # imagenet, coco, or last\n init_with = \"last\" \n \n LEARNING_RATE = 0.001\n \n # If enabled, resizes instance masks to a smaller size to reduce\n # memory load. Recommended when using high-resolution images.\n USE_MINI_MASK = False\n MINI_MASK_SHAPE = (56, 56) # (height, width) of the mini-mask\n \n # Train on 1 GPU and 8 images per GPU. Batch size is GPUs * images/GPU.\n GPU_COUNT = 2\n IMAGES_PER_GPU = 8\n bs = GPU_COUNT * IMAGES_PER_GPU\n # Total number of steps (batches of samples) to yield from generator before declaring one epoch finished and starting the next epoch.\n # typically be equal to the number of samples of your dataset divided by the batch size\n STEPS_PER_EPOCH = 620//bs\n VALIDATION_STEPS = 70//bs\n\n # Number of classes (including background)\n NUM_CLASSES = 1 + 1 # background + nucleis\n\n # Input image resing\n # Images are resized such that the smallest side is >= IMAGE_MIN_DIM and\n # the longest side is <= IMAGE_MAX_DIM. In case both conditions can't\n # be satisfied together the IMAGE_MAX_DIM is enforced.\n IMAGE_MIN_DIM = 256\n IMAGE_MAX_DIM = 256\n # If True, pad images with zeros such that they're (max_dim by max_dim)\n IMAGE_PADDING = True # currently, the False option is not supported\n\n # Use smaller anchors because our image and objects are small\n RPN_ANCHOR_SCALES = (8, 16, 32, 64, 128) # anchor side in pixels\n \n # How many anchors per image to use for RPN training\n RPN_TRAIN_ANCHORS_PER_IMAGE = 256\n \n # Number of ROIs per image to feed to classifier/mask heads\n # The Mask RCNN paper uses 512 but often the RPN doesn't generate\n # enough positive proposals to fill this and keep a positive:negative\n # ratio of 1:3. You can increase the number of proposals by adjusting\n # the RPN NMS threshold.\n # Reduce training ROIs per image because the images are small and have\n # few objects. Aim to allow ROI sampling to pick 33% positive ROIs.\n TRAIN_ROIS_PER_IMAGE = 128\n \n # Maximum number of ground truth instances to use in one image\n MAX_GT_INSTANCES = 120\n \n # Max number of final detections\n DETECTION_MAX_INSTANCES = 120", "279697": "Cheers", "279732": "Hi Yan Wang,\n\nYou mentioned in your first post \"I am using the same code base, but sometimes I meet NaN loss\"\n\nNow that I have the code base working I'm also encountering this. Any useful findings for avoiding it? The usual learning-rate decreases / batch size decreases don't seem to be making a difference.", "279832": "Seems that this two parameter should not be too small like 128, otherwise some mask may be lost during the resize operation, causing the NaN problem.\n\n IMAGE_MIN_DIM = 256\n IMAGE_MAX_DIM = 256", "279852": "Ah, ok, thanks. I had been doing a test run with 128, 128. That was probably what was causing it.", "279889": "Hi Yan. You are in right direction. I am also did same as you but I did not submit result yet. Could you tell me your testing score using your configuration? Thanks", "279917": "Hi John, I got 0.307 using a similar configuration. I am running more computation demanding ones.", "279923": "Great. I want to suggest you to split data into 3 clusters or 5 clusters as many guys did and then normalize or enhancement method as preprocessing. It will increase your performance. I have one more question that is how you can achieve instance segmented image in testing phase? Bacause you cropped image into 256, but testing images are various dimension. AFAIK, the maskrcnn only allows fixed size because it used fully connected in detection part.", "279929": "Fixed size is required because we want to put several images in one mini batch. The code remember the original image size, and then resize the obtained masks back to the original size.", "280022": "I found directly e2e training performs better. I used the resnet50-fpn backbone and have experienced severe overfitting. The validation loss goes extremely unstable also. Does anyone have some ideas to fix this?", "280024": "I would suggest you to use higher resolution image size, or just for some specific images. 256 * 256 is not large enough that you may still get some zero size bbox. You can set use_mini_mask = True to check if for it has a exception check that tells you which image gets this exception.", "280146": "Thank you. I will try larger images. Unfortunately, this implementation is a bit too slow, I am going to work with some faster implementations.", "280168": "Hi Yan, when you encode the result for submission, which kind of result (instance segmentation or semantic segmentation) do you use for input RLE? For example, semantic segmentation likes UNet result, while instance segmentation like maskrcnn. Because we use RLE to encode the masks, so I think semantic and instance have same performance (if they are made from same ground-truth). Am I right?", "280175": "For Mask R-CNN, I encode each instance independently.", "280180": "So you first resize the testing image into 256x256. After obtain the instance segmentation result, you resize the instance results to the original testing image. Am I right? If yes, then you may lost so much information in resize step", "280207": "This implementation seems the best I've found based on tensorflow, although it has some differences between the original paper's implementation, for example the ROIAlign layer. Or you could try to implement your own maskrcnn based on tensorflow's official faster-rcnn model. FAIR's detectron is better for it is the official implementation and support all tricks mentioned in the paper to improve its performance, but it does not support training on your own dataset.", "280224": "Yes, perfect implementation is very limited now. Seems it's possible to use detectron [as shown here][1]. Unfortunately I have no root access to install caffe2. BTW, I am also from THU. Wechat: wizard_123\n\n\n [1]: https://github.com/facebookresearch/Detectron/issues/98", "280280": "By e2e do you mean that you load the resnet50 fpn backbone but train the weights from scratch rather than using imagenet weights? Or use a different backbone completely?", "280297": "Can I check something with anyone who is using Mask_RCNN:\n\nI have \n\nactive_class_ids = np.array([0, 1], dtype=np.int32) # Background, Nucleus\n\nand then each mask needs to be assigned a class_id, but in our case that is 1 for each mask, so class_id is just an array of 1's? I think I'm missing something somewhere...", "280791": "You're correct. The resize operation should be removed", "280794": "Some updates:\n1. Remove the cv2.resize code\n2. Use larger image size;\n3. Split the nuclei to multiple cluster\nThese steps can help to improve the performance.", "280796": "class_id is 1 for each mask. But it's better to split the mask into different kinds.", "280799": "I don't think it is a good idea to cluster into 3 groups, because you don't know how many groups are there in stage2 data. If there are lots of different groups this method I think would fail.", "280886": "i saw that here are images with 350 nuclies so i think i should change the DETECTION_MAX_INSTANCES to more then 100\nshould i change the RPN_TRAIN_ANCHORS_PER_IMAGE too?" }, "source": "meta" }