Has anyone tried fine tuning the backbone used in faster RCNN model for improving the performance? Default backbone in pytorch's faster RCNN is resnet-50 pretrained on imagenet dataset. I think that this backbone remains unchanged during the model's training.\nPlease correct me if I am wrong and feel free to add any ideas.
", "messages": [ { "id": "940127", "postDate": "07/22/2020 17:26:58", "content": "Has anyone tried fine tuning the backbone used in faster RCNN model for improving the performance? Default backbone in pytorch's faster RCNN is resnet-50 pretrained on imagenet dataset. I think that this backbone remains unchanged during the model's training.\nPlease correct me if I am wrong and feel free to add any ideas.
", "rawMarkdown": "Has anyone tried fine tuning the backbone used in faster RCNN model for improving the performance? Default backbone in pytorch's faster RCNN is resnet-50 pretrained on imagenet dataset. I think that this backbone remains unchanged during the model's training.\nPlease correct me if I am wrong and feel free to add any ideas.", "votes": null }, { "id": "940142", "postDate": "07/22/2020 17:36:53", "content": "The backbone does change during fine tuning unless you freeze certain layers.
", "rawMarkdown": "The backbone does change during fine tuning unless you freeze certain layers.", "votes": null }, { "id": "941837", "postDate": "07/23/2020 12:46:47", "content": "you can change the backbone with resnet_fpn_backbone('resnet101', pretrained=True)`. I tried fine tuning it, it helped a bit.
Hi @vguptakota , \nYou can use many backbones... \nSome of the good ones are resnet101 , resneXt101_32x8d, wide_resnet101_2 etc... \nTo know which all models you can use check this link\nHope this helps...
To add to this, the torchvision FasterRCNN model does freeze layers 1-4 by default.
", "rawMarkdown": "To add to this, the torchvision FasterRCNN model does freeze layers 1-4 by default.", "votes": null }, { "id": "945925", "postDate": "07/26/2020 08:12:58", "content": "In the latest version, if you use the pre-trained weights, only layer1 and conv1 will be frozen. Layers 2-4 will remain unfrozen.[1] (\"Layer\" isn't maybe the best word for these, they're more like \"blocks\". I think @matthewmasters and I are agreeing, block1 == layers 1-4)
\n\nI've tried unfreezing layer1 and conv1 with the pre-trained weights and it does seem to make a small improvement. The downside is that it slows down training a bit.
\n\nI tried to train only block 4(layer 4) but it didn't turned out well, score was horrible. Maybe it has to be 2-4 blocks to train.
", "rawMarkdown": "I tried to train only block 4(layer 4) but it didn't turned out well, score was horrible. Maybe it has to be 2-4 blocks to train.", "votes": null } ], "comments": [ { "id": 940142, "author_name": "jonykarki", "author_url": "", "post_date": "07/22/2020 17:36:53", "content": "The backbone does change during fine tuning unless you freeze certain layers.
", "votes": null, "replies": [ { "id": 942846, "author_name": "matthewmasters", "author_url": "", "post_date": "07/24/2020 03:22:30", "content": "To add to this, the torchvision FasterRCNN model does freeze layers 1-4 by default.
", "votes": null, "replies": [] } ] }, { "id": 941837, "author_name": "anku5hk", "author_url": "", "post_date": "07/23/2020 12:46:47", "content": "you can change the backbone with resnet_fpn_backbone('resnet101', pretrained=True)`. I tried fine tuning it, it helped a bit.
Hi @vguptakota , \nYou can use many backbones... \nSome of the good ones are resnet101 , resneXt101_32x8d, wide_resnet101_2 etc... \nTo know which all models you can use check this link\nHope this helps...
In the latest version, if you use the pre-trained weights, only layer1 and conv1 will be frozen. Layers 2-4 will remain unfrozen.[1] (\"Layer\" isn't maybe the best word for these, they're more like \"blocks\". I think @matthewmasters and I are agreeing, block1 == layers 1-4)
\n\nI've tried unfreezing layer1 and conv1 with the pre-trained weights and it does seem to make a small improvement. The downside is that it slows down training a bit.
\n\nI tried to train only block 4(layer 4) but it didn't turned out well, score was horrible. Maybe it has to be 2-4 blocks to train.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "940127": "Has anyone tried fine tuning the backbone used in faster RCNN model for improving the performance? Default backbone in pytorch's faster RCNN is resnet-50 pretrained on imagenet dataset. I think that this backbone remains unchanged during the model's training.\nPlease correct me if I am wrong and feel free to add any ideas.", "940142": "The backbone does change during fine tuning unless you freeze certain layers.", "941837": "you can change the backbone with `resnet_fpn_backbone('resnet101',` pretrained=True)`. I tried fine tuning it, it helped a bit.", "942219": "Hi @vguptakota , \nYou can use many backbones... \nSome of the good ones are `resnet101` , `resneXt101_32x8d`, `wide_resnet101_2` etc... \nTo know which all models you can use check this [link](https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py)\nHope this helps...", "942846": "To add to this, the torchvision FasterRCNN model does freeze layers 1-4 by default.", "945925": "In the latest version, if you use the pre-trained weights, only layer1 and conv1 will be frozen. Layers 2-4 will remain unfrozen.[1] (\"Layer\" isn't maybe the best word for these, they're more like \"blocks\". I think @matthewmasters and I are agreeing, block1 == layers 1-4) \n\nI've tried unfreezing layer1 and conv1 with the pre-trained weights and it does seem to make a small improvement. The downside is that it slows down training a bit. \n\n1. [source](https://github.com/pytorch/vision/blob/348dd5a732f929ff3f0e404ceed0b2feff88ee07/torchvision/models/detection/backbone_utils.py#L44-L94)", "946307": "I tried to train only block 4(layer 4) but it didn't turned out well, score was horrible. Maybe it has to be 2-4 blocks to train." }, "source": "meta" }