Hi guys, I want to share my solution in this competition. It's another version of pytorch mask rcnn based on multimodallearning's pytorch-mask-rcnn. You can get the code at https://github.com/louis-she/dsb. I got my current place only by training 10 epoch and with a little tweak on the configuration, because of the GPU resource limitation.
", "messages": [ { "id": "307580", "postDate": "04/02/2018 04:02:56", "content": "Hi guys, I want to share my solution in this competition. It's another version of pytorch mask rcnn based on multimodallearning's pytorch-mask-rcnn. You can get the code at https://github.com/louis-she/dsb. I got my current place only by training 10 epoch and with a little tweak on the configuration, because of the GPU resource limitation.
", "rawMarkdown": "Hi guys, I want to share my solution in this competition. It's another version of pytorch mask rcnn based on multimodallearning's [pytorch-mask-rcnn][1]. You can get the code at https://github.com/louis-she/dsb. I got my current place only by training 10 epoch and with a little tweak on the configuration, because of the GPU resource limitation.\n\n\n [1]: https://github.com/multimodallearning/pytorch-mask-rcnn\n [2]: http://git@github.com:louis-she/dsb.git", "votes": null }, { "id": "307586", "postDate": "04/02/2018 04:38:05", "content": "Great. How about training speed between matterport version and your version? For materport, it spends about 5 min/epoch with size of 512x512. And the matterport has updated some new features: augmentation (fliplr, rotation...) and changed something in newest version today, Do you consider it? Thanks
", "rawMarkdown": "Great. How about training speed between matterport version and your version? For materport, it spends about 5 min/epoch with size of 512x512. And the matterport has updated some new features: augmentation (fliplr, rotation...) and changed something in newest version today, Do you consider it? Thanks", "votes": null }, { "id": "307594", "postDate": "04/02/2018 05:08:20", "content": "In my forked version, I have added image augmentation in the model.py. (Random crop, rotate and flip). The matterport's version when I forked only have horizontal flip : )
", "rawMarkdown": "In my forked version, I have added image augmentation in the model.py. (Random crop, rotate and flip). The matterport's version when I forked only have horizontal flip : )", "votes": null }, { "id": "307595", "postDate": "04/02/2018 05:12:31", "content": "Good job. How about the PR? https://github.com/matterport/Mask_RCNN/commit/e6290877a0da712ba078d3a8454d90d24e9744f4
\n\nThe purpose is here https://github.com/matterport/Mask_RCNN/issues/359
", "rawMarkdown": "Good job. How about the PR? https://github.com/matterport/Mask_RCNN/commit/e6290877a0da712ba078d3a8454d90d24e9744f4\n\nThe purpose is here https://github.com/matterport/Mask_RCNN/issues/359", "votes": null }, { "id": "307600", "postDate": "04/02/2018 05:19:44", "content": "Cool! I will look into this and try to get the new features working in my solution. Thanks bro.
", "rawMarkdown": "Cool! I will look into this and try to get the new features working in my solution. Thanks bro.", "votes": null }, { "id": "307833", "postDate": "04/02/2018 15:18:00", "content": "Hello, 璐. Could i make a friend with you?
", "rawMarkdown": "Hello, 璐. Could i make a friend with you?", "votes": null }, { "id": "307974", "postDate": "04/02/2018 18:35:54", "content": "Thank you! When run python dsb_train.py, I get the following error. Is it because the pretrained weights are .h5 file versus .pt file?
\n\nTraceback (most recent call last):\n File \"dsb_train.py\", line 41, in <module>\n state_dict = torch.load(dsb_config.STATE_DICT_PATH)\n File \"/usr/local/lib/python3.6/site-packages/torch/serialization.py\", line 261, in load\n return _load(f, map_location, pickle_module)\n File \"/usr/local/lib/python3.6/site-packages/torch/serialization.py\", line 399, in _load\n magic_number = pickle_module.load(f)\n_pickle.UnpicklingError: invalid load key, 'H'.\n@snaker I would also be very interested in the questions that @john1231983 raised. For instance @Heng CherKeng code which is a pytorch implementation of mask-rcnn has a lot of fluctuations during training. The gpu memory is is fluctuating its allocation from 45% to 95% and the volatility is a low as 0%, sometimes even for a minute and then spikes up back again at 100%. How is your implementation in those regards. Have you noticed those things?
", "rawMarkdown": "snaker I would also be very interested in the questions that @john1231983 raised. For instance @Heng CherKeng code which is a pytorch implementation of mask-rcnn has a lot of fluctuations during training. The gpu memory is is fluctuating its allocation from 45% to 95% and the volatility is a low as 0%, sometimes even for a minute and then spikes up back again at 100%. How is your implementation in those regards. Have you noticed those things?", "votes": null }, { "id": "308499", "postDate": "04/03/2018 15:31:33", "content": "Of course, 彬. My wechat: dl_snaker mail: chenglu.she@gmail.com
", "rawMarkdown": "Of course, 彬. My wechat: dl_snaker mail: chenglu.she@gmail.com", "votes": null }, { "id": "308501", "postDate": "04/03/2018 15:33:56", "content": "Torch's load function is relied on pickle, which is not compatible between python2 and python3. And you must have been using python2 to run the code. Switch to python3 the error will be gone.
I believe you first need to transform keras-based weights (mask_rcnn_coco.h5) to pytorch model weights (mask_rcnn_coco.pth). You can use the convert_from_keras.py (included in the distribution) to do that.
", "rawMarkdown": "I believe you first need to transform keras-based weights (mask_rcnn_coco.h5) to pytorch model weights (mask_rcnn_coco.pth). You can use the convert_from_keras.py (included in the distribution) to do that.", "votes": null }, { "id": "308519", "postDate": "04/03/2018 16:21:53", "content": "@snaker I am using python 3.
\n\n@Costas Thanks. I will test it out today.
", "rawMarkdown": "snaker I am using python 3. \n\n@Costas Thanks. I will test it out today.", "votes": null }, { "id": "308532", "postDate": "04/03/2018 16:48:16", "content": "I just did the conversion and it seems to work. However, I had strange memory issues and training stopped when I was loading the weights. When I started training from scratch everything worked fine. Let me now if you experience the same behaviour.
", "rawMarkdown": "I just did the conversion and it seems to work. However, I had strange memory issues and training stopped when I was loading the weights. When I started training from scratch everything worked fine. Let me now if you experience the same behaviour.", "votes": null }, { "id": "308544", "postDate": "04/03/2018 17:08:54", "content": "@Costas Voglis & @璐 I get the following error when I try to run the model:\n\nTraceback (most recent call last):\n File \"dsb_train.py\", line 68, in <module>\n layers='heads'\n File \"./pytorch-mask-rcnn/model.py\", line 1859, in train_model\n loss = self.train_epoch(train_generator, optimizer)\n File \"./pytorch-mask-rcnn/model.py\", line 1916, in train_epoch\n self.predict([images, image_metas, gt_class_ids, gt_boxes, gt_masks], mode='training')\n File \"./pytorch-mask-rcnn/model.py\", line 1768, in predict\n detection_target_layer(rpn_rois, gt_class_ids, gt_boxes, gt_masks, self.config)\n File \"./pytorch-mask-rcnn/model.py\", line 580, in detection_target_layer\n positive_indices = torch.nonzero(positive_roi_bool)[:, 0]\nIndexError: too many indices for tensor of dimension 1\n
nice sharing~
", "rawMarkdown": "nice sharing~", "votes": null }, { "id": "308848", "postDate": "04/04/2018 07:33:08", "content": "Could you paste your dsb_config.py here? I didn't run into this error.
@Costas Voglis Would you help me out how to transform keras-based weights (mask_rcnn_coco.h5) to pytorch model weights (mask_rcnn_coco.pth) using convert_from_keras.py? I came across the same problem and cann't sovle it. Great thanks!
", "rawMarkdown": "Costas Voglis Would you help me out how to transform keras-based weights (mask_rcnn_coco.h5) to pytorch model weights (mask_rcnn_coco.pth) using convert_from_keras.py? I came across the same problem and cann't sovle it. Great thanks!", "votes": null }, { "id": "308870", "postDate": "04/04/2018 08:25:39", "content": "The mask_rcnn_coo.pth is already a pytorch weights. Check the md5sum of it , if it's 20d64236204a4806c7051721bee61ad1, then it't definately a pytorch weights since I'm using it.
I'm very sorry I put a wrong link in the README.md, which is the weights for Keras version. I've already fix it.
", "rawMarkdown": "I'm very sorry I put a wrong link in the README.md, which is the weights for Keras version. I've already fix it.", "votes": null }, { "id": "308876", "postDate": "04/04/2018 08:37:03", "content": "@城璐 The md5sum of the mask_rcnn_coo.pth in my machine is e98aaff6f99e307b5e2a8a3ff741a518.
", "rawMarkdown": "城璐 The md5sum of the mask_rcnn_coo.pth in my machine is e98aaff6f99e307b5e2a8a3ff741a518.", "votes": null }, { "id": "308886", "postDate": "04/04/2018 08:47:01", "content": "@城璐 Thanks for the correction!
", "rawMarkdown": "城璐 Thanks for the correction!", "votes": null }, { "id": "308898", "postDate": "04/04/2018 09:23:05", "content": "Sure:
\n\nfrom os import getcwd\nfrom os.path import join, expanduser\n\nimport numpy as np\nfrom config import Config\n\n\nclass DsbConfig(Config):\n\n # DSB data root directory\n ROOT_DIR = getcwd()\n HOME_DIR = expanduser('~')\n DSB_DATA_DIR = join(HOME_DIR, 'projects/dsbshe/data/')\n\n # Weights path\n STATE_DICT_PATH = join(ROOT_DIR, \"mask_rcnn_coco.pth\")\n\n # If the state dict file is mask_rcnn_coco, set this to True\n IS_COCO_STATE = True\n\n # Give the configuration a recognizable name\n NAME = \"dsb\"\n\n # Validation dataset proportion\n VALIDATION_PROPORTION = 0.1\n\n # Backbone architecture, valid options are resnet50, resnet101\n BACKBONE_ARCH = 'resnet101'\n\n # Print progress every n steps\n SHOW_PROGRESS_STEPS = 2\n\n # If enabled, resizes instance masks to a smaller size to reduce\n # memory load. Recommended when using high-resolution image\n # Tweak this\n USE_MINI_MASK = True\n\n # If the previous is False, then this is not used\n MINI_MASK_SHAPE = (56, 56) # (height, width) of the mini-mask\n\n # Train on 1 GPU and 8 images per GPU. Batch size is GPUs * images/GPU.\n GPU_COUNT = 1\n IMAGES_PER_GPU = 1\n\n # Total number of steps (batches of samples) to yield from\n # generator before declaring one epoch finished and starting the\n # next epoch. typically be equal to the number of samples of your\n # dataset divided by the batch size\n STEPS_PER_EPOCH = 612\n VALIDATION_STEPS = 58\n\n # Number of classes (including background)\n NUM_CLASSES = 2\n\n IMAGE_MIN_DIM = 512\n IMAGE_MAX_DIM = 512\n\n IMAGE_PADDING = True # currently, the False option is not supported\n\n RPN_ANCHOR_SCALES = (8, 16, 32, 64, 128) # anchor side in pixels, maybe add a 256?\n # The strides of each layer of the FPN Pyramid. These values\n # are based on a Resnet101 backbone.\n BACKBONE_STRIDES = [4, 8, 16, 32, 64]\n # How many anchors per image to use for RPN training\n RPN_TRAIN_ANCHORS_PER_IMAGE = 320 # 300\n\n # ROIs kept after non-maximum supression (training and inference)\n POST_NMS_ROIS_TRAINING = 2000\n POST_NMS_ROIS_INFERENCE = 2000\n POOL_SIZE = 7\n MASK_POOL_SIZE = 14\n MASK_SHAPE = [28, 28]\n TRAIN_ROIS_PER_IMAGE = 512\n RPN_NMS_THRESHOLD = 0.7\n MAX_GT_INSTANCES = 256\n DETECTION_MAX_INSTANCES = 500\n # Minimum probability value to accept a detected instance\n # ROIs below this threshold are skipped\n DETECTION_MIN_CONFIDENCE = 0.7 # may be smaller?\n # Non-maximum suppression threshold for detection\n DETECTION_NMS_THRESHOLD = 0.3 # 0.3\n\n MEAN_PIXEL = np.array([0., 0., 0.])\n\n # Weight decay regularization\n WEIGHT_DECAY = 0.0001\n\n\nclass InferenceConfig(DsbConfig):\n GPU_COUNT = 1\n IMAGES_PER_GPU = 1\n # MEAN_PIXEL = np.array([56.02288505, 54.02376286, 54.26675248])\nNothing weird except the indent. I just pushed a new branch multimodallearning-mask-rcnn, which is more stable than master. You can switch to that branch by
git fetch\ngit checkout multimodallearning-mask-rcnn\ngit submodule update\nand then try python dsb_train.py to see if there're any errors.
Same error:
\n\n\n\n\n\n \nStarting at epoch 1. LR=0.001\n\nCheckpoint Path: /home/user/projects/dsb/./logs/dsb20180404T1557/mask_rcnn_dsb_{:04d}.pth\nEpoch 1/20.\n/home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\nUserWarning: From scipy 0.13.0, the output shape of zoom() is\n calculated with round() instead of int() - for these inputs the size\n of the returned array has changed.\n \"the returned array has changed.\", UserWarning)\n /home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\n UserWarning: From scipy 0.13.0, the output shape of zoom() is\n calculated with round() instead of int() - for these inputs the size\n of the returned array has changed.\n \"the returned array has changed.\", UserWarning)\n /home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\n UserWarning: From scipy 0.13.0, the output shape of zoom() is\n calculated with round() instead of int() - for these inputs the size\n of the returned array has changed.\n \"the returned array has changed.\", UserWarning)\n /home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\n UserWarning: From scipy 0.13.0, the output shape of zoom() is\n calculated with round() instead of int() - for these inputs the size\n of the returned array has changed.\n \"the returned array has changed.\", UserWarning)\n Traceback (most recent call last):\n File \"dsb_train.py\", line 64, in \n layers='heads')\n File \"./pytorch-mask-rcnn/model.py\", line 1794, in train_model\n loss, loss_rpn_class, loss_rpn_bbox, loss_mrcnn_class, loss_mrcnn_bbox, loss_mrcnn_mask = self.train_epoch(train_generator,\n optimizer, self.config.STEPS_PER_EPOCH)\n File \"./pytorch-mask-rcnn/model.py\", line 1856, in train_epoch\n self.predict([images, image_metas, gt_class_ids, gt_boxes, gt_masks], mode='training')\n File \"./pytorch-mask-rcnn/model.py\", line 1715, in predict\n detection_target_layer(rpn_rois, gt_class_ids, gt_boxes, gt_masks, self.config)\n File \"./pytorch-mask-rcnn/model.py\", line 569, in detection_target_layer\n crowd_ix = torch.nonzero(gt_class_ids < 0)[:, 0]\n IndexError: too many indices for tensor of dimension 1
\n
I also believe that you have some commands in the readme that are necessary:
\n\n\n\n\ncp -R _ext dsb/pytorch-mask-rcnn/nms/_ext cp -R _ext\n dsb/pytorch-mask-rcnn/roialign/roi_align/_ext
\n
the _ext folder is already in place after python build.py.
I forgot to mention that I am using pytorch 0.4 but I don't think that's the problem here. Also in my above comments the necessary I meant unnecessary but it seems that in the kaggle realm of commenting I'm unable to change that when I edit my previous message it shows me only the first part of the blockquote, not possible to get to the second one. Kaggle folks should look into that as well.
My issue was resolved by downloading the file from github. I downloaded the wrong file from a different site. Thanks Costas and Snaker
", "rawMarkdown": "My issue was resolved by downloading the file from github. I downloaded the wrong file from a different site. Thanks Costas and Snaker", "votes": null }, { "id": "309205", "postDate": "04/04/2018 19:48:53", "content": "Is there supposed to be a log folder?
\n\nTraceback (most recent call last):\n File \"dsb_train.py\", line 65, in <module>\nlayers='heads')\nFile \"./pytorch-mask-rcnn/model.py\", line 1867, in train_model\nvisualize.plot_loss(self.loss_history, self.val_loss_history, save=True, log_dir=self.log_dir)\n File \"./pytorch-mask-rcnn/visualize.py\", line 414, in plot_loss\nplt.savefig(save_path)\n File \"/usr/local/lib/python3.6/site-packages/matplotlib/pyplot.py\", line 701, in savefig\nres = fig.savefig(*args, **kwargs)\nFile \"/usr/local/lib/python3.6/site-packages/matplotlib/figure.py\", line 1834, in savefig\nself.canvas.print_figure(fname, **kwargs)\nFile \"/usr/local/lib/python3.6/site-packages/matplotlib/backend_bases.py\", line 2267, in print_figure\n**kwargs)\nFile \"/usr/local/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py\", line 512, in print_png\nfilename_or_obj = open(filename_or_obj, 'wb')\n **FileNotFoundError: [Errno 2] No such file or directory: '.logs/loss.png'**\nThere's no dsb_utils in the file.
", "rawMarkdown": "There's no dsb_utils in the file.", "votes": null }, { "id": "309333", "postDate": "04/05/2018 04:12:15", "content": "I came across with the same issue, still don't know how to solve...
", "rawMarkdown": "I came across with the same issue, still don't know how to solve...", "votes": null }, { "id": "309385", "postDate": "04/05/2018 07:37:50", "content": "I solved this problem just by changing the model_dir in dsb_train.py to your desired path.
", "rawMarkdown": "I solved this problem just by changing the model_dir in dsb_train.py to your desired path.", "votes": null }, { "id": "309479", "postDate": "04/05/2018 12:35:33", "content": "What did you change the model_dir in dsb_train.py to? It's the same setup in predict.py. I was able to run dsb_train, but unable to run predict.
", "rawMarkdown": "What did you change the model_dir in dsb_train.py to? It's the same setup in predict.py. I was able to run dsb_train, but unable to run predict.", "votes": null }, { "id": "309898", "postDate": "04/06/2018 06:14:56", "content": "good job
", "rawMarkdown": "good job", "votes": null }, { "id": "309944", "postDate": "04/06/2018 08:42:18", "content": "Is this the same as https://github.com/neptune-ml/open-solution-data-science-bowl-2018/tree/mask_rcnn_notebook ? And how do the two compare? @kamilkk
", "rawMarkdown": "Is this the same as https://github.com/neptune-ml/open-solution-data-science-bowl-2018/tree/mask_rcnn_notebook ? And how do the two compare? @kamilkk", "votes": null }, { "id": "310352", "postDate": "04/07/2018 08:53:32", "content": "Already fixed.
", "rawMarkdown": "Already fixed.", "votes": null }, { "id": "310354", "postDate": "04/07/2018 08:54:43", "content": "The link you give is based on Mask RCNN on Keras, this is another version which is on Pytorch.
", "rawMarkdown": "The link you give is based on Mask RCNN on Keras, this is another version which is on Pytorch.", "votes": null }, { "id": "310427", "postDate": "04/07/2018 13:39:19", "content": "I still cann't find the dsb_utils in the updated file...
", "rawMarkdown": "I still cann't find the dsb_utils in the updated file...", "votes": null }, { "id": "336598", "postDate": "06/01/2018 02:25:33", "content": "Hi have you solved the problem somehow? Facing the same error currently. best
", "rawMarkdown": "Hi have you solved the problem somehow? Facing the same error currently. best", "votes": null }, { "id": "596384", "postDate": "08/10/2019 15:11:36", "content": "Can I train mask r-cnn, if I have annotation data with only segmentation values and no bounding boxes? If yes, how?
", "rawMarkdown": "Can I train mask r-cnn, if I have annotation data with only segmentation values and no bounding boxes? If yes, how?", "votes": null } ], "comments": [ { "id": 307586, "author_name": "john1231983", "author_url": "", "post_date": "04/02/2018 04:38:05", "content": "Great. How about training speed between matterport version and your version? For materport, it spends about 5 min/epoch with size of 512x512. And the matterport has updated some new features: augmentation (fliplr, rotation...) and changed something in newest version today, Do you consider it? Thanks
", "votes": null, "replies": [ { "id": 307594, "author_name": "snaker", "author_url": "", "post_date": "04/02/2018 05:08:20", "content": "In my forked version, I have added image augmentation in the model.py. (Random crop, rotate and flip). The matterport's version when I forked only have horizontal flip : )
", "votes": null, "replies": [] }, { "id": 307595, "author_name": "john1231983", "author_url": "", "post_date": "04/02/2018 05:12:31", "content": "Good job. How about the PR? https://github.com/matterport/Mask_RCNN/commit/e6290877a0da712ba078d3a8454d90d24e9744f4
\n\nThe purpose is here https://github.com/matterport/Mask_RCNN/issues/359
", "votes": null, "replies": [] }, { "id": 307600, "author_name": "snaker", "author_url": "", "post_date": "04/02/2018 05:19:44", "content": "Cool! I will look into this and try to get the new features working in my solution. Thanks bro.
", "votes": null, "replies": [] } ] }, { "id": 307833, "author_name": "lianandrew", "author_url": "", "post_date": "04/02/2018 15:18:00", "content": "Hello, 璐. Could i make a friend with you?
", "votes": null, "replies": [ { "id": 308499, "author_name": "snaker", "author_url": "", "post_date": "04/03/2018 15:31:33", "content": "Of course, 彬. My wechat: dl_snaker mail: chenglu.she@gmail.com
", "votes": null, "replies": [] } ] }, { "id": 307974, "author_name": "dskswu", "author_url": "", "post_date": "04/02/2018 18:35:54", "content": "Thank you! When run python dsb_train.py, I get the following error. Is it because the pretrained weights are .h5 file versus .pt file?
\n\nTraceback (most recent call last):\n File \"dsb_train.py\", line 41, in <module>\n state_dict = torch.load(dsb_config.STATE_DICT_PATH)\n File \"/usr/local/lib/python3.6/site-packages/torch/serialization.py\", line 261, in load\n return _load(f, map_location, pickle_module)\n File \"/usr/local/lib/python3.6/site-packages/torch/serialization.py\", line 399, in _load\n magic_number = pickle_module.load(f)\n_pickle.UnpicklingError: invalid load key, 'H'.\nTorch's load function is relied on pickle, which is not compatible between python2 and python3. And you must have been using python2 to run the code. Switch to python3 the error will be gone.
I believe you first need to transform keras-based weights (mask_rcnn_coco.h5) to pytorch model weights (mask_rcnn_coco.pth). You can use the convert_from_keras.py (included in the distribution) to do that.
", "votes": null, "replies": [] }, { "id": 308519, "author_name": "dskswu", "author_url": "", "post_date": "04/03/2018 16:21:53", "content": "@snaker I am using python 3.
\n\n@Costas Thanks. I will test it out today.
", "votes": null, "replies": [] }, { "id": 308532, "author_name": "voglinio", "author_url": "", "post_date": "04/03/2018 16:48:16", "content": "I just did the conversion and it seems to work. However, I had strange memory issues and training stopped when I was loading the weights. When I started training from scratch everything worked fine. Let me now if you experience the same behaviour.
", "votes": null, "replies": [] }, { "id": 308867, "author_name": "chenxaionan", "author_url": "", "post_date": "04/04/2018 08:19:28", "content": "@Costas Voglis Would you help me out how to transform keras-based weights (mask_rcnn_coco.h5) to pytorch model weights (mask_rcnn_coco.pth) using convert_from_keras.py? I came across the same problem and cann't sovle it. Great thanks!
", "votes": null, "replies": [] }, { "id": 308870, "author_name": "snaker", "author_url": "", "post_date": "04/04/2018 08:25:39", "content": "The mask_rcnn_coo.pth is already a pytorch weights. Check the md5sum of it , if it's 20d64236204a4806c7051721bee61ad1, then it't definately a pytorch weights since I'm using it.
I'm very sorry I put a wrong link in the README.md, which is the weights for Keras version. I've already fix it.
", "votes": null, "replies": [] }, { "id": 308876, "author_name": "chenxaionan", "author_url": "", "post_date": "04/04/2018 08:37:03", "content": "@城璐 The md5sum of the mask_rcnn_coo.pth in my machine is e98aaff6f99e307b5e2a8a3ff741a518.
", "votes": null, "replies": [] }, { "id": 308886, "author_name": "chenxaionan", "author_url": "", "post_date": "04/04/2018 08:47:01", "content": "@城璐 Thanks for the correction!
", "votes": null, "replies": [] }, { "id": 309184, "author_name": "dskswu", "author_url": "", "post_date": "04/04/2018 19:16:34", "content": "My issue was resolved by downloading the file from github. I downloaded the wrong file from a different site. Thanks Costas and Snaker
", "votes": null, "replies": [] } ] }, { "id": 308404, "author_name": "kirk86", "author_url": "", "post_date": "04/03/2018 13:10:12", "content": "@snaker I would also be very interested in the questions that @john1231983 raised. For instance @Heng CherKeng code which is a pytorch implementation of mask-rcnn has a lot of fluctuations during training. The gpu memory is is fluctuating its allocation from 45% to 95% and the volatility is a low as 0%, sometimes even for a minute and then spikes up back again at 100%. How is your implementation in those regards. Have you noticed those things?
", "votes": null, "replies": [] }, { "id": 308544, "author_name": "kirk86", "author_url": "", "post_date": "04/03/2018 17:08:54", "content": "@Costas Voglis & @璐 I get the following error when I try to run the model:\n\nTraceback (most recent call last):\n File \"dsb_train.py\", line 68, in <module>\n layers='heads'\n File \"./pytorch-mask-rcnn/model.py\", line 1859, in train_model\n loss = self.train_epoch(train_generator, optimizer)\n File \"./pytorch-mask-rcnn/model.py\", line 1916, in train_epoch\n self.predict([images, image_metas, gt_class_ids, gt_boxes, gt_masks], mode='training')\n File \"./pytorch-mask-rcnn/model.py\", line 1768, in predict\n detection_target_layer(rpn_rois, gt_class_ids, gt_boxes, gt_masks, self.config)\n File \"./pytorch-mask-rcnn/model.py\", line 580, in detection_target_layer\n positive_indices = torch.nonzero(positive_roi_bool)[:, 0]\nIndexError: too many indices for tensor of dimension 1\n
Could you paste your dsb_config.py here? I didn't run into this error.
Sure:
\n\nfrom os import getcwd\nfrom os.path import join, expanduser\n\nimport numpy as np\nfrom config import Config\n\n\nclass DsbConfig(Config):\n\n # DSB data root directory\n ROOT_DIR = getcwd()\n HOME_DIR = expanduser('~')\n DSB_DATA_DIR = join(HOME_DIR, 'projects/dsbshe/data/')\n\n # Weights path\n STATE_DICT_PATH = join(ROOT_DIR, \"mask_rcnn_coco.pth\")\n\n # If the state dict file is mask_rcnn_coco, set this to True\n IS_COCO_STATE = True\n\n # Give the configuration a recognizable name\n NAME = \"dsb\"\n\n # Validation dataset proportion\n VALIDATION_PROPORTION = 0.1\n\n # Backbone architecture, valid options are resnet50, resnet101\n BACKBONE_ARCH = 'resnet101'\n\n # Print progress every n steps\n SHOW_PROGRESS_STEPS = 2\n\n # If enabled, resizes instance masks to a smaller size to reduce\n # memory load. Recommended when using high-resolution image\n # Tweak this\n USE_MINI_MASK = True\n\n # If the previous is False, then this is not used\n MINI_MASK_SHAPE = (56, 56) # (height, width) of the mini-mask\n\n # Train on 1 GPU and 8 images per GPU. Batch size is GPUs * images/GPU.\n GPU_COUNT = 1\n IMAGES_PER_GPU = 1\n\n # Total number of steps (batches of samples) to yield from\n # generator before declaring one epoch finished and starting the\n # next epoch. typically be equal to the number of samples of your\n # dataset divided by the batch size\n STEPS_PER_EPOCH = 612\n VALIDATION_STEPS = 58\n\n # Number of classes (including background)\n NUM_CLASSES = 2\n\n IMAGE_MIN_DIM = 512\n IMAGE_MAX_DIM = 512\n\n IMAGE_PADDING = True # currently, the False option is not supported\n\n RPN_ANCHOR_SCALES = (8, 16, 32, 64, 128) # anchor side in pixels, maybe add a 256?\n # The strides of each layer of the FPN Pyramid. These values\n # are based on a Resnet101 backbone.\n BACKBONE_STRIDES = [4, 8, 16, 32, 64]\n # How many anchors per image to use for RPN training\n RPN_TRAIN_ANCHORS_PER_IMAGE = 320 # 300\n\n # ROIs kept after non-maximum supression (training and inference)\n POST_NMS_ROIS_TRAINING = 2000\n POST_NMS_ROIS_INFERENCE = 2000\n POOL_SIZE = 7\n MASK_POOL_SIZE = 14\n MASK_SHAPE = [28, 28]\n TRAIN_ROIS_PER_IMAGE = 512\n RPN_NMS_THRESHOLD = 0.7\n MAX_GT_INSTANCES = 256\n DETECTION_MAX_INSTANCES = 500\n # Minimum probability value to accept a detected instance\n # ROIs below this threshold are skipped\n DETECTION_MIN_CONFIDENCE = 0.7 # may be smaller?\n # Non-maximum suppression threshold for detection\n DETECTION_NMS_THRESHOLD = 0.3 # 0.3\n\n MEAN_PIXEL = np.array([0., 0., 0.])\n\n # Weight decay regularization\n WEIGHT_DECAY = 0.0001\n\n\nclass InferenceConfig(DsbConfig):\n GPU_COUNT = 1\n IMAGES_PER_GPU = 1\n # MEAN_PIXEL = np.array([56.02288505, 54.02376286, 54.26675248])\nNothing weird except the indent. I just pushed a new branch multimodallearning-mask-rcnn, which is more stable than master. You can switch to that branch by
git fetch\ngit checkout multimodallearning-mask-rcnn\ngit submodule update\nand then try python dsb_train.py to see if there're any errors.
Same error:
\n\n\n\n\n\n \nStarting at epoch 1. LR=0.001\n\nCheckpoint Path: /home/user/projects/dsb/./logs/dsb20180404T1557/mask_rcnn_dsb_{:04d}.pth\nEpoch 1/20.\n/home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\nUserWarning: From scipy 0.13.0, the output shape of zoom() is\n calculated with round() instead of int() - for these inputs the size\n of the returned array has changed.\n \"the returned array has changed.\", UserWarning)\n /home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\n UserWarning: From scipy 0.13.0, the output shape of zoom() is\n calculated with round() instead of int() - for these inputs the size\n of the returned array has changed.\n \"the returned array has changed.\", UserWarning)\n /home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\n UserWarning: From scipy 0.13.0, the output shape of zoom() is\n calculated with round() instead of int() - for these inputs the size\n of the returned array has changed.\n \"the returned array has changed.\", UserWarning)\n /home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\n UserWarning: From scipy 0.13.0, the output shape of zoom() is\n calculated with round() instead of int() - for these inputs the size\n of the returned array has changed.\n \"the returned array has changed.\", UserWarning)\n Traceback (most recent call last):\n File \"dsb_train.py\", line 64, in \n layers='heads')\n File \"./pytorch-mask-rcnn/model.py\", line 1794, in train_model\n loss, loss_rpn_class, loss_rpn_bbox, loss_mrcnn_class, loss_mrcnn_bbox, loss_mrcnn_mask = self.train_epoch(train_generator,\n optimizer, self.config.STEPS_PER_EPOCH)\n File \"./pytorch-mask-rcnn/model.py\", line 1856, in train_epoch\n self.predict([images, image_metas, gt_class_ids, gt_boxes, gt_masks], mode='training')\n File \"./pytorch-mask-rcnn/model.py\", line 1715, in predict\n detection_target_layer(rpn_rois, gt_class_ids, gt_boxes, gt_masks, self.config)\n File \"./pytorch-mask-rcnn/model.py\", line 569, in detection_target_layer\n crowd_ix = torch.nonzero(gt_class_ids < 0)[:, 0]\n IndexError: too many indices for tensor of dimension 1
\n
I also believe that you have some commands in the readme that are necessary:
\n\n\n\n\ncp -R _ext dsb/pytorch-mask-rcnn/nms/_ext cp -R _ext\n dsb/pytorch-mask-rcnn/roialign/roi_align/_ext
\n
the _ext folder is already in place after python build.py.
I forgot to mention that I am using pytorch 0.4 but I don't think that's the problem here. Also in my above comments the necessary I meant unnecessary but it seems that in the kaggle realm of commenting I'm unable to change that when I edit my previous message it shows me only the first part of the blockquote, not possible to get to the second one. Kaggle folks should look into that as well.
Hi have you solved the problem somehow? Facing the same error currently. best
", "votes": null, "replies": [] } ] }, { "id": 308629, "author_name": "czhangcn", "author_url": "", "post_date": "04/03/2018 20:19:05", "content": "nice sharing~
", "votes": null, "replies": [] }, { "id": 309205, "author_name": "dskswu", "author_url": "", "post_date": "04/04/2018 19:48:53", "content": "Is there supposed to be a log folder?
\n\nTraceback (most recent call last):\n File \"dsb_train.py\", line 65, in <module>\nlayers='heads')\nFile \"./pytorch-mask-rcnn/model.py\", line 1867, in train_model\nvisualize.plot_loss(self.loss_history, self.val_loss_history, save=True, log_dir=self.log_dir)\n File \"./pytorch-mask-rcnn/visualize.py\", line 414, in plot_loss\nplt.savefig(save_path)\n File \"/usr/local/lib/python3.6/site-packages/matplotlib/pyplot.py\", line 701, in savefig\nres = fig.savefig(*args, **kwargs)\nFile \"/usr/local/lib/python3.6/site-packages/matplotlib/figure.py\", line 1834, in savefig\nself.canvas.print_figure(fname, **kwargs)\nFile \"/usr/local/lib/python3.6/site-packages/matplotlib/backend_bases.py\", line 2267, in print_figure\n**kwargs)\nFile \"/usr/local/lib/python3.6/site-packages/matplotlib/backends/backend_agg.py\", line 512, in print_png\nfilename_or_obj = open(filename_or_obj, 'wb')\n **FileNotFoundError: [Errno 2] No such file or directory: '.logs/loss.png'**\nI came across with the same issue, still don't know how to solve...
", "votes": null, "replies": [] }, { "id": 309385, "author_name": "chenxaionan", "author_url": "", "post_date": "04/05/2018 07:37:50", "content": "I solved this problem just by changing the model_dir in dsb_train.py to your desired path.
", "votes": null, "replies": [] }, { "id": 309479, "author_name": "dskswu", "author_url": "", "post_date": "04/05/2018 12:35:33", "content": "What did you change the model_dir in dsb_train.py to? It's the same setup in predict.py. I was able to run dsb_train, but unable to run predict.
", "votes": null, "replies": [] } ] }, { "id": 309301, "author_name": "dskswu", "author_url": "", "post_date": "04/05/2018 02:02:14", "content": "There's no dsb_utils in the file.
", "votes": null, "replies": [ { "id": 310352, "author_name": "snaker", "author_url": "", "post_date": "04/07/2018 08:53:32", "content": "Already fixed.
", "votes": null, "replies": [] }, { "id": 310427, "author_name": "chenxaionan", "author_url": "", "post_date": "04/07/2018 13:39:19", "content": "I still cann't find the dsb_utils in the updated file...
", "votes": null, "replies": [] } ] }, { "id": 309898, "author_name": "kunwang", "author_url": "", "post_date": "04/06/2018 06:14:56", "content": "good job
", "votes": null, "replies": [] }, { "id": 309944, "author_name": "yipcma", "author_url": "", "post_date": "04/06/2018 08:42:18", "content": "Is this the same as https://github.com/neptune-ml/open-solution-data-science-bowl-2018/tree/mask_rcnn_notebook ? And how do the two compare? @kamilkk
", "votes": null, "replies": [ { "id": 310354, "author_name": "snaker", "author_url": "", "post_date": "04/07/2018 08:54:43", "content": "The link you give is based on Mask RCNN on Keras, this is another version which is on Pytorch.
", "votes": null, "replies": [] } ] }, { "id": 596384, "author_name": "harsh93", "author_url": "", "post_date": "08/10/2019 15:11:36", "content": "Can I train mask r-cnn, if I have annotation data with only segmentation values and no bounding boxes? If yes, how?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "307580": "Hi guys, I want to share my solution in this competition. It's another version of pytorch mask rcnn based on multimodallearning's [pytorch-mask-rcnn][1]. You can get the code at https://github.com/louis-she/dsb. I got my current place only by training 10 epoch and with a little tweak on the configuration, because of the GPU resource limitation.\n\n\n [1]: https://github.com/multimodallearning/pytorch-mask-rcnn\n [2]: http://git@github.com:louis-she/dsb.git", "307586": "Great. How about training speed between matterport version and your version? For materport, it spends about 5 min/epoch with size of 512x512. And the matterport has updated some new features: augmentation (fliplr, rotation...) and changed something in newest version today, Do you consider it? Thanks", "307594": "In my forked version, I have added image augmentation in the model.py. (Random crop, rotate and flip). The matterport's version when I forked only have horizontal flip : )", "307595": "Good job. How about the PR? https://github.com/matterport/Mask_RCNN/commit/e6290877a0da712ba078d3a8454d90d24e9744f4\n\nThe purpose is here https://github.com/matterport/Mask_RCNN/issues/359", "307600": "Cool! I will look into this and try to get the new features working in my solution. Thanks bro.", "307833": "Hello, 璐. Could i make a friend with you?", "307974": "Thank you! When run python dsb_train.py, I get the following error. Is it because the pretrained weights are .h5 file versus .pt file? \n\n Traceback (most recent call last):\n File \"dsb_train.py\", line 41, in", "308404": "snaker I would also be very interested in the questions that @john1231983 raised. For instance @Heng CherKeng code which is a pytorch implementation of mask-rcnn has a lot of fluctuations during training. The gpu memory is is fluctuating its allocation from 45% to 95% and the volatility is a low as 0%, sometimes even for a minute and then spikes up back again at 100%. How is your implementation in those regards. Have you noticed those things?", "308499": "Of course, 彬. My wechat: dl_snaker mail: chenglu.she@gmail.com", "308501": "Torch's `load` function is relied on `pickle`, which is not compatible between python2 and python3. And you must have been using python2 to run the code. Switch to python3 the error will be gone.", "308508": "I believe you first need to transform keras-based weights (mask_rcnn_coco.h5) to pytorch model weights (mask_rcnn_coco.pth). You can use the convert_from_keras.py (included in the distribution) to do that.", "308519": "snaker I am using python 3. \n\n@Costas Thanks. I will test it out today.", "308532": "I just did the conversion and it seems to work. However, I had strange memory issues and training stopped when I was loading the weights. When I started training from scratch everything worked fine. Let me now if you experience the same behaviour.", "308544": "Costas Voglis & @璐 I get the following error when I try to run the model:\n```\nTraceback (most recent call last):\n File \"dsb_train.py\", line 68, in", "308629": "nice sharing~", "308848": "Could you paste your `dsb_config.py` here? I didn't run into this error.", "308867": "Costas Voglis Would you help me out how to transform keras-based weights (mask_rcnn_coco.h5) to pytorch model weights (mask_rcnn_coco.pth) using convert_from_keras.py? I came across the same problem and cann't sovle it. Great thanks!", "308870": "The `mask_rcnn_coo.pth` is already a pytorch weights. Check the md5sum of it , if it's `20d64236204a4806c7051721bee61ad1`, then it't definately a pytorch weights since I'm using it.", "308872": "I'm very sorry I put a wrong link in the README.md, which is the weights for Keras version. I've already fix it.", "308876": "城璐 The md5sum of the mask_rcnn_coo.pth in my machine is e98aaff6f99e307b5e2a8a3ff741a518.", "308886": "城璐 Thanks for the correction!", "308898": "Sure:\n\n from os import getcwd\n from os.path import join, expanduser\n\n import numpy as np\n from config import Config\n\n\n class DsbConfig(Config):\n\n # DSB data root directory\n ROOT_DIR = getcwd()\n HOME_DIR = expanduser('~')\n DSB_DATA_DIR = join(HOME_DIR, 'projects/dsbshe/data/')\n\n # Weights path\n STATE_DICT_PATH = join(ROOT_DIR, \"mask_rcnn_coco.pth\")\n\n # If the state dict file is mask_rcnn_coco, set this to True\n IS_COCO_STATE = True\n\n # Give the configuration a recognizable name\n NAME = \"dsb\"\n\n # Validation dataset proportion\n VALIDATION_PROPORTION = 0.1\n\n # Backbone architecture, valid options are resnet50, resnet101\n BACKBONE_ARCH = 'resnet101'\n\n # Print progress every n steps\n SHOW_PROGRESS_STEPS = 2\n\n # If enabled, resizes instance masks to a smaller size to reduce\n # memory load. Recommended when using high-resolution image\n # Tweak this\n USE_MINI_MASK = True\n\n # If the previous is False, then this is not used\n MINI_MASK_SHAPE = (56, 56) # (height, width) of the mini-mask\n\n # Train on 1 GPU and 8 images per GPU. Batch size is GPUs * images/GPU.\n GPU_COUNT = 1\n IMAGES_PER_GPU = 1\n\n # Total number of steps (batches of samples) to yield from\n # generator before declaring one epoch finished and starting the\n # next epoch. typically be equal to the number of samples of your\n # dataset divided by the batch size\n STEPS_PER_EPOCH = 612\n VALIDATION_STEPS = 58\n\n # Number of classes (including background)\n NUM_CLASSES = 2\n\n IMAGE_MIN_DIM = 512\n IMAGE_MAX_DIM = 512\n\n IMAGE_PADDING = True # currently, the False option is not supported\n\n RPN_ANCHOR_SCALES = (8, 16, 32, 64, 128) # anchor side in pixels, maybe add a 256?\n # The strides of each layer of the FPN Pyramid. These values\n # are based on a Resnet101 backbone.\n BACKBONE_STRIDES = [4, 8, 16, 32, 64]\n # How many anchors per image to use for RPN training\n RPN_TRAIN_ANCHORS_PER_IMAGE = 320 # 300\n\n # ROIs kept after non-maximum supression (training and inference)\n POST_NMS_ROIS_TRAINING = 2000\n POST_NMS_ROIS_INFERENCE = 2000\n POOL_SIZE = 7\n MASK_POOL_SIZE = 14\n MASK_SHAPE = [28, 28]\n TRAIN_ROIS_PER_IMAGE = 512\n RPN_NMS_THRESHOLD = 0.7\n MAX_GT_INSTANCES = 256\n DETECTION_MAX_INSTANCES = 500\n # Minimum probability value to accept a detected instance\n # ROIs below this threshold are skipped\n DETECTION_MIN_CONFIDENCE = 0.7 # may be smaller?\n # Non-maximum suppression threshold for detection\n DETECTION_NMS_THRESHOLD = 0.3 # 0.3\n\n MEAN_PIXEL = np.array([0., 0., 0.])\n\n # Weight decay regularization\n WEIGHT_DECAY = 0.0001\n\n\n class InferenceConfig(DsbConfig):\n GPU_COUNT = 1\n IMAGES_PER_GPU = 1\n # MEAN_PIXEL = np.array([56.02288505, 54.02376286, 54.26675248])", "309013": "Nothing weird except the indent. I just pushed a new branch `multimodallearning-mask-rcnn`, which is more stable than master. You can switch to that branch by \n\n git fetch\n git checkout multimodallearning-mask-rcnn\n git submodule update\n\nand then try `python dsb_train.py` to see if there're any errors.", "309050": "Same error:\n\n\n> Starting at epoch 1. LR=0.001\n> \n> Checkpoint Path: /home/user/projects/dsb/./logs/dsb20180404T1557/mask_rcnn_dsb_{:04d}.pth\n> Epoch 1/20.\n> /home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\n> UserWarning: From scipy 0.13.0, the output shape of zoom() is\n> calculated with round() instead of int() - for these inputs the size\n> of the returned array has changed.\n> \"the returned array has changed.\", UserWarning)\n> /home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\n> UserWarning: From scipy 0.13.0, the output shape of zoom() is\n> calculated with round() instead of int() - for these inputs the size\n> of the returned array has changed.\n> \"the returned array has changed.\", UserWarning)\n> /home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\n> UserWarning: From scipy 0.13.0, the output shape of zoom() is\n> calculated with round() instead of int() - for these inputs the size\n> of the returned array has changed.\n> \"the returned array has changed.\", UserWarning)\n> /home/user/miniconda3/envs/torch/lib/python3.6/site-packages/scipy/ndimage/interpolation.py:616:\n> UserWarning: From scipy 0.13.0, the output shape of zoom() is\n> calculated with round() instead of int() - for these inputs the size\n> of the returned array has changed.\n> \"the returned array has changed.\", UserWarning)\n> Traceback (most recent call last):\n> File \"dsb_train.py\", line 64, in", "309054": "I forgot to mention that I am using pytorch 0.4 but I don't think that's the problem here. Also in my above comments the `necessary` I meant `unnecessary` but it seems that in the kaggle realm of commenting I'm unable to change that when I edit my previous message it shows me only the first part of the blockquote, not possible to get to the second one. Kaggle folks should look into that as well.", "309184": "My issue was resolved by downloading the file from github. I downloaded the wrong file from a different site. Thanks Costas and Snaker", "309205": "Is there supposed to be a log folder? \n\n Traceback (most recent call last):\n File \"dsb_train.py\", line 65, in", "309301": "There's no dsb_utils in the file.", "309333": "I came across with the same issue, still don't know how to solve...", "309385": "I solved this problem just by changing the model_dir in dsb_train.py to your desired path.", "309479": "What did you change the model_dir in dsb_train.py to? It's the same setup in predict.py. I was able to run dsb_train, but unable to run predict.", "309898": "good job", "309944": "Is this the same as https://github.com/neptune-ml/open-solution-data-science-bowl-2018/tree/mask_rcnn_notebook ? And how do the two compare? @kamilkk", "310352": "Already fixed.", "310354": "The link you give is based on Mask RCNN on Keras, this is another version which is on Pytorch.", "310427": "I still cann't find the dsb_utils in the updated file...", "336598": "Hi have you solved the problem somehow? Facing the same error currently. best", "596384": "Can I train mask r-cnn, if I have annotation data with only segmentation values and no bounding boxes? If yes, how?" }, "source": "meta" }