{ "id": 164602, "title": "Object detection zoology", "url": "/competitions/global-wheat-detection/discussion/164602", "author_name": "", "post_date": "2020-07-06T20:31:33.827138Z", "votes": 16, "comment_count": 6, "views": 0, "content": "

Since I am new to this whole object detection business, I thought \"what a better way to learn then listing some of the most popular models?\". So here is my list so far (will update along the way and add more details):

\n\n

To dig deeper, here are some more details:

\n\n

That was a lot of new concepts to grok, so as a reward, here is an approximate timeline: in red the one step models, and green the two steps ones. Notice that FPN doesn't have a category since it is used in both:

\n

\"\"

", "messages": [ { "id": "917944", "postDate": "07/06/2020 20:31:33", "content": "

Since I am new to this whole object detection business, I thought \"what a better way to learn then listing some of the most popular models?\". So here is my list so far (will update along the way and add more details):

\n\n

To dig deeper, here are some more details:

\n\n

That was a lot of new concepts to grok, so as a reward, here is an approximate timeline: in red the one step models, and green the two steps ones. Notice that FPN doesn't have a category since it is used in both:

\n

\"\"

", "rawMarkdown": "Since I am new to this whole object detection business, I thought \"what a better way to learn then listing some of the most popular models?\". So here is my list so far (will update along the way and add more details): \n\n\n* **SSD**: [paper](https://arxiv.org/pdf/1512.02325.pdf) and pytorch [implementation]. (https://pytorch.org/hub/nvidia_deeplearningexamples_ssd/). Notice that SSD stands for \"Single Shot MultiBox Detector\".\n\n* **FPN**: [paper](https://openaccess.thecvf.com/content_cvpr_2017/papers/Lin_Feature_Pyramid_Networks_CVPR_2017_paper.pdf) and pytorch [implementation](https://github.com/jwyang/fpn.pytorch). Notice that FPN stands for \"Feature Pyramid Networks\". \n\n* **RetinaNET**: [paper](https://arxiv.org/pdf/1708.02002.pdf) and pytorch [implementation](https://github.com/yhenon/pytorch-retinanet). \n\n* **Mask RCNN**: [paper](https://arxiv.org/pdf/1703.06870.pdf) and pytorch [tutorial](https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html) on how to fine-tune it. Notice that this model is a generalization of [Faster RCNN](https://arxiv.org/abs/1506.01497) that adds instance segmentation on top of object detection. \n\n* **CenterNet**: [paper](https://arxiv.org/pdf/1904.08189.pdf) and pytorch [implementation](https://github.com/xingyizhou/CenterNet).\n\n* **YOLO**: [website](https://pjreddie.com/darknet/yolo/) and v3 [paper](https://pjreddie.com/media/files/papers/YOLOv3.pdf). As the website claims, it is 100 times faster than Mask RCNN. There is even a v5 [now](https://github.com/ultralytics/yolov5). Notice that YOLO stands for \"You Only Look Once\". \n\n* **EfficientDet**: [paper](https://arxiv.org/pdf/1911.09070.pdf) and pytorch [implementation](https://github.com/rwightman/efficientdet-pytorch). \n\n* **DETR**: model detection using the transformer architecture. [paper](https://arxiv.org/abs/2005.12872) and pytorch [implementation](https://github.com/facebookresearch/detr). \n\n\n\nTo dig deeper, here are some more details:\n\n\n\n \n* **SSD**\n\n * SSD is short for single shot detector\n * As it is clear from its name, it belonges to the one step family\n * Paper accepted around 2016 (end of)\n \n \n \n \n* **FPN**\n\n * FPN is short for feature pyarmic network\n * This isn't an object detection network per say but rather a part\n * It is used in many architecture as the features extraction part\n * Paper accepted in 2017\n\n* **Mask R-CNN**\n\n * From the region proposals series of models, this is the latest one\n * It is an extension of faster r-cnn for instance segmentation\n * It seems that it gives better results even for object detection\n * To learn more, read [this](https://medium.com/@jonathan_hui/what-do-we-learn-from-region-based-object-detectors-faster-r-cnn-r-fcn-fpn-7e354377a7c9) blog post and [this](https://paperswithcode.com/method/mask-r-cnn) paperswithcode page.\n\n\n* **RetinaNet**\n\n * Uses focal loss\n \n \n \n * Uses FPN as its features extraction layer \n \n \n\n* **CenterNet**\n\n * Paper accepted in 2019\n * Estimates the center of the bounding box in addition to the corners\n * Belongs to the one stage family of models \n * Claims to have the best performance for the one stage family and not very far from the best in two stages as well\n * Has two networks: one for finding the center and one for finding the corners\n * As usual, to learn more check the paperswithcode [page](https://paperswithcode.com/method/centernet)\n \n \n\n* **YOLO**\n\n * Uses a different format for the outputted bounding box: the center + width and height instead of top-left and bottom-right corners. \n * Among the one stage family of moddels\n * Fast predictions with great mAP\n * Latest versions use FPN as a features extracting layer\n * Has been through many iterations (latest being v4). If you want a very complete blog post, check this [one](https://medium.com/@jonathan_hui/yolov4-c9901eaa8e61) and the [paperswithcode](https://paperswithcode.com/method/yolov4) page. \n * V4 paper accepted in 2020\n \n \n\n\n\n* **EfficientDet**\n\n * Originally developed by the google brain team using architecture search. \n * The original accompanying code could be found [here](https://github.com/google/automl/tree/master/efficientdet) \n * A great [notebook](https://www.kaggle.com/shonenkov/training-efficientdet/notebook) to get started training an EfficientDet model on this competition dataset. Thanks to [Alex Shonenkov](https://www.kaggle.com/shonenkov). \n * To get the backbon weights, go to the [README](https://github.com/rwightman/efficientdet-pytorch/blob/master/README.md#models) page, models section, where you can find this table and click on the weights you want\n\n \n \n * COCO AP (average precision) vs FLOPS (number of operations per second) graph:\n\n \n\n\n * SOTA according to [paperwithcode](https://paperswithcode.com/paper/efficientdet-scalable-and-efficient-object). Here is graph that shows that EfficientDet-D7x (largest model) achieves it on the COCO object detection benchmark:\n \n ![coco_object_detection_score.png](attachment:coco_object_detection_score.png)\n \n\n* **DETR** \n\n * DETR is short for detection with transformer\n * Paper accepted in 2020\n * Uses a transformer layer after the feature extraction one\n * Developed by FAIR\n * To learn more, check the paperwithcodes [page](https://paperswithcode.com/method/detr)\n \n \n\n\nThat was a lot of new concepts to grok, so as a reward, here is an approximate timeline: in red the one step models, and green the two steps ones. Notice that FPN doesn't have a category since it is used in both:\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F43b5252a12b471baa12e846bca60737d%2Fobject_detection_recent_timeline.png?generation=1600704962462342&alt=media)", "votes": null }, { "id": "922233", "postDate": "07/09/2020 23:57:56", "content": "

Nice find ! U forgot FB's Detr :\nhttps://github.com/facebookresearch/detr

\n\n

cheers\nsid

", "rawMarkdown": "Nice find ! U forgot FB's Detr :\nhttps://github.com/facebookresearch/detr\n\ncheers\nsid", "votes": null }, { "id": "923454", "postDate": "07/10/2020 21:35:45", "content": "

Indeed. Will add it, thanks. 👌

", "rawMarkdown": "Indeed. Will add it, thanks. 👌", "votes": null }, { "id": "924738", "postDate": "07/11/2020 15:51:08", "content": "

Added! 👌

", "rawMarkdown": "Added! 👌", "votes": null }, { "id": "932168", "postDate": "07/16/2020 19:18:45", "content": "

[EDIT 16-7-2020] Started adding some details about each model, starting with EfficientDet.

", "rawMarkdown": "[EDIT 16-7-2020] Started adding some details about each model, starting with EfficientDet.", "votes": null }, { "id": "934196", "postDate": "07/18/2020 09:55:19", "content": "

[EDIT 18-7-2020] More details for EfficientDet, CenterNet, and DETR.

", "rawMarkdown": "[EDIT 18-7-2020] More details for EfficientDet, CenterNet, and DETR.", "votes": null }, { "id": "1021104", "postDate": "09/21/2020 16:16:56", "content": "

[EDIT 21-9-2020] Much more details for various models and a nice timeline. :)

", "rawMarkdown": "[EDIT 21-9-2020] Much more details for various models and a nice timeline. :)", "votes": null } ], "comments": [ { "id": 1021104, "author_name": "yassinealouini", "author_url": "", "post_date": "09/21/2020 16:16:56", "content": "

[EDIT 21-9-2020] Much more details for various models and a nice timeline. :)

", "votes": null, "replies": [] }, { "id": 922233, "author_name": "sidneyng", "author_url": "", "post_date": "07/09/2020 23:57:56", "content": "

Nice find ! U forgot FB's Detr :\nhttps://github.com/facebookresearch/detr

\n\n

cheers\nsid

", "votes": null, "replies": [ { "id": 923454, "author_name": "yassinealouini", "author_url": "", "post_date": "07/10/2020 21:35:45", "content": "

Indeed. Will add it, thanks. 👌

", "votes": null, "replies": [] }, { "id": 924738, "author_name": "yassinealouini", "author_url": "", "post_date": "07/11/2020 15:51:08", "content": "

Added! 👌

", "votes": null, "replies": [] } ] }, { "id": 932168, "author_name": "yassinealouini", "author_url": "", "post_date": "07/16/2020 19:18:45", "content": "

[EDIT 16-7-2020] Started adding some details about each model, starting with EfficientDet.

", "votes": null, "replies": [] }, { "id": 934196, "author_name": "yassinealouini", "author_url": "", "post_date": "07/18/2020 09:55:19", "content": "

[EDIT 18-7-2020] More details for EfficientDet, CenterNet, and DETR.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "917944": "Since I am new to this whole object detection business, I thought \"what a better way to learn then listing some of the most popular models?\". So here is my list so far (will update along the way and add more details): \n\n\n* **SSD**: [paper](https://arxiv.org/pdf/1512.02325.pdf) and pytorch [implementation]. (https://pytorch.org/hub/nvidia_deeplearningexamples_ssd/). Notice that SSD stands for \"Single Shot MultiBox Detector\".\n\n* **FPN**: [paper](https://openaccess.thecvf.com/content_cvpr_2017/papers/Lin_Feature_Pyramid_Networks_CVPR_2017_paper.pdf) and pytorch [implementation](https://github.com/jwyang/fpn.pytorch). Notice that FPN stands for \"Feature Pyramid Networks\". \n\n* **RetinaNET**: [paper](https://arxiv.org/pdf/1708.02002.pdf) and pytorch [implementation](https://github.com/yhenon/pytorch-retinanet). \n\n* **Mask RCNN**: [paper](https://arxiv.org/pdf/1703.06870.pdf) and pytorch [tutorial](https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html) on how to fine-tune it. Notice that this model is a generalization of [Faster RCNN](https://arxiv.org/abs/1506.01497) that adds instance segmentation on top of object detection. \n\n* **CenterNet**: [paper](https://arxiv.org/pdf/1904.08189.pdf) and pytorch [implementation](https://github.com/xingyizhou/CenterNet).\n\n* **YOLO**: [website](https://pjreddie.com/darknet/yolo/) and v3 [paper](https://pjreddie.com/media/files/papers/YOLOv3.pdf). As the website claims, it is 100 times faster than Mask RCNN. There is even a v5 [now](https://github.com/ultralytics/yolov5). Notice that YOLO stands for \"You Only Look Once\". \n\n* **EfficientDet**: [paper](https://arxiv.org/pdf/1911.09070.pdf) and pytorch [implementation](https://github.com/rwightman/efficientdet-pytorch). \n\n* **DETR**: model detection using the transformer architecture. [paper](https://arxiv.org/abs/2005.12872) and pytorch [implementation](https://github.com/facebookresearch/detr). \n\n\n\nTo dig deeper, here are some more details:\n\n\n\n \n* **SSD**\n\n * SSD is short for single shot detector\n * As it is clear from its name, it belonges to the one step family\n * Paper accepted around 2016 (end of)\n \n \n \n \n* **FPN**\n\n * FPN is short for feature pyarmic network\n * This isn't an object detection network per say but rather a part\n * It is used in many architecture as the features extraction part\n * Paper accepted in 2017\n\n* **Mask R-CNN**\n\n * From the region proposals series of models, this is the latest one\n * It is an extension of faster r-cnn for instance segmentation\n * It seems that it gives better results even for object detection\n * To learn more, read [this](https://medium.com/@jonathan_hui/what-do-we-learn-from-region-based-object-detectors-faster-r-cnn-r-fcn-fpn-7e354377a7c9) blog post and [this](https://paperswithcode.com/method/mask-r-cnn) paperswithcode page.\n\n\n* **RetinaNet**\n\n * Uses focal loss\n \n \n \n * Uses FPN as its features extraction layer \n \n \n\n* **CenterNet**\n\n * Paper accepted in 2019\n * Estimates the center of the bounding box in addition to the corners\n * Belongs to the one stage family of models \n * Claims to have the best performance for the one stage family and not very far from the best in two stages as well\n * Has two networks: one for finding the center and one for finding the corners\n * As usual, to learn more check the paperswithcode [page](https://paperswithcode.com/method/centernet)\n \n \n\n* **YOLO**\n\n * Uses a different format for the outputted bounding box: the center + width and height instead of top-left and bottom-right corners. \n * Among the one stage family of moddels\n * Fast predictions with great mAP\n * Latest versions use FPN as a features extracting layer\n * Has been through many iterations (latest being v4). If you want a very complete blog post, check this [one](https://medium.com/@jonathan_hui/yolov4-c9901eaa8e61) and the [paperswithcode](https://paperswithcode.com/method/yolov4) page. \n * V4 paper accepted in 2020\n \n \n\n\n\n* **EfficientDet**\n\n * Originally developed by the google brain team using architecture search. \n * The original accompanying code could be found [here](https://github.com/google/automl/tree/master/efficientdet) \n * A great [notebook](https://www.kaggle.com/shonenkov/training-efficientdet/notebook) to get started training an EfficientDet model on this competition dataset. Thanks to [Alex Shonenkov](https://www.kaggle.com/shonenkov). \n * To get the backbon weights, go to the [README](https://github.com/rwightman/efficientdet-pytorch/blob/master/README.md#models) page, models section, where you can find this table and click on the weights you want\n\n \n \n * COCO AP (average precision) vs FLOPS (number of operations per second) graph:\n\n \n\n\n * SOTA according to [paperwithcode](https://paperswithcode.com/paper/efficientdet-scalable-and-efficient-object). Here is graph that shows that EfficientDet-D7x (largest model) achieves it on the COCO object detection benchmark:\n \n ![coco_object_detection_score.png](attachment:coco_object_detection_score.png)\n \n\n* **DETR** \n\n * DETR is short for detection with transformer\n * Paper accepted in 2020\n * Uses a transformer layer after the feature extraction one\n * Developed by FAIR\n * To learn more, check the paperwithcodes [page](https://paperswithcode.com/method/detr)\n \n \n\n\nThat was a lot of new concepts to grok, so as a reward, here is an approximate timeline: in red the one step models, and green the two steps ones. Notice that FPN doesn't have a category since it is used in both:\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F172860%2F43b5252a12b471baa12e846bca60737d%2Fobject_detection_recent_timeline.png?generation=1600704962462342&alt=media)", "922233": "Nice find ! U forgot FB's Detr :\nhttps://github.com/facebookresearch/detr\n\ncheers\nsid", "923454": "Indeed. Will add it, thanks. 👌", "924738": "Added! 👌", "932168": "[EDIT 16-7-2020] Started adding some details about each model, starting with EfficientDet.", "934196": "[EDIT 18-7-2020] More details for EfficientDet, CenterNet, and DETR.", "1021104": "[EDIT 21-9-2020] Much more details for various models and a nice timeline. :)" }, "source": "meta" }