Congratulations on the winners for a challenge that wasn't without some hiccups.
\n\nAs some of you are aware, I'm the author of one of the EfficientDet impl out there (https://github.com/rwightman/efficientdet-pytorch) that was used by some in this challenge.
\n\nMy impl was based on the official Google automl (https://github.com/google/automl/tree/master/efficientdet) impl. Both he Google impl and mine have only been around for about 4 months now. They are still a little rough around the edges and lacking some of the bells and whistles available in YoloV5 (building on Glenn's previous work on Yolov3), and mmdetection, etc that are building on ~2 years of contributions.
\n\nSo, my question to all, based on your experience in this challenge, what should I add that would improve the performance of the models? Keep in mind I want to keep the repo streamlined, keep it focused one-stage FPN based methods. I am aware of the usability/dataset issues in my code so far, it was focused on reproducing COCO validation and training results so that is an area I will improve. There are also numerous places to improve error logging, robustness to unexpected input shapes/etc.
\n\nIn terms of loss fn, augmentations, etc I feel there could be some big improvements. I'm currently evaluating an IoU loss PR that I have to sort out some issues with. I'd like to add my own flavour of mosaic aug and hopefully some auto-aug/rand-aug type augmentations to the pipeline.
\n\nCurious for input on the priorities of the above, or possibly other additions that you'd see as being useful based on experience with this challenge (or others).
\n\nAlso, I've recently updated my timm model collection to be able to use any of the models as a backbone for my EfficientDet impl. I did one experiment with a ResNet50 backbone that hit .42 mAP with the basic scale jitter aug. I've trained some CSPResNeXT/DarkNet53 models in timm recently to > 80% top-1, so running a CSPDarkDet or CspResDeXt experiment should be doable :)
Congratulations on the winners for a challenge that wasn't without some hiccups.
\n\nAs some of you are aware, I'm the author of one of the EfficientDet impl out there (https://github.com/rwightman/efficientdet-pytorch) that was used by some in this challenge.
\n\nMy impl was based on the official Google automl (https://github.com/google/automl/tree/master/efficientdet) impl. Both he Google impl and mine have only been around for about 4 months now. They are still a little rough around the edges and lacking some of the bells and whistles available in YoloV5 (building on Glenn's previous work on Yolov3), and mmdetection, etc that are building on ~2 years of contributions.
\n\nSo, my question to all, based on your experience in this challenge, what should I add that would improve the performance of the models? Keep in mind I want to keep the repo streamlined, keep it focused one-stage FPN based methods. I am aware of the usability/dataset issues in my code so far, it was focused on reproducing COCO validation and training results so that is an area I will improve. There are also numerous places to improve error logging, robustness to unexpected input shapes/etc.
\n\nIn terms of loss fn, augmentations, etc I feel there could be some big improvements. I'm currently evaluating an IoU loss PR that I have to sort out some issues with. I'd like to add my own flavour of mosaic aug and hopefully some auto-aug/rand-aug type augmentations to the pipeline.
\n\nCurious for input on the priorities of the above, or possibly other additions that you'd see as being useful based on experience with this challenge (or others).
\n\nAlso, I've recently updated my timm model collection to be able to use any of the models as a backbone for my EfficientDet impl. I did one experiment with a ResNet50 backbone that hit .42 mAP with the basic scale jitter aug. I've trained some CSPResNeXT/DarkNet53 models in timm recently to > 80% top-1, so running a CSPDarkDet or CspResDeXt experiment should be doable :)
Hi Ross,
\nthanks for your efforts!
\nI played a bit with your code and tried a few changes:
I believe, CIoU loss gives some improvements over huber loss. I haven't had enough time to train my models until the full convergence (I did about 20-30 epochs per each run, starting from pretrained on COCO weights). The results for the underfitted models were quite promising :) I saw improvements both on the local validation and PL.
I tried cluster NMS from the same authors and had some improvements, but it might be due to a better threshold value. Didn't study this one thoroughly.
Can't say much about mosaic. I think, I've messed this one :D
Thanks for the great repo @rwightman
\nYOLOv5 was handy for detection noobies because the entire COCO training pipeline was wrapped within the repo.
\nMaybe if you include the Effdet train pipeline within the repo too, people can train easily on custom datasets!
Thank you very much for your work. I have found it very useful and convenient.
\n\nIn addition to what you mention, I think a very easy to use fine tuning script would make your repo more popular. Also maybe with additional options for up-down flips and random transpositions (or rotations) during training, which can be very useful for custom datasets with these symmetries. Like this one.
\n\nHere is more that I’ve done with your code:\n https://www.kaggle.com/riblidezso/gwd-efficientdet-clean-training-and-inference-code
", "rawMarkdown": "Thank you very much for your work. I have found it very useful and convenient. \n\nIn addition to what you mention, I think a very easy to use fine tuning script would make your repo more popular. Also maybe with additional options for up-down flips and random transpositions (or rotations) during training, which can be very useful for custom datasets with these symmetries. Like this one.\n\nHere is more that I’ve done with your code:\n https://www.kaggle.com/riblidezso/gwd-efficientdet-clean-training-and-inference-code", "votes": null }, { "id": "964414", "postDate": "08/09/2020 20:44:47", "content": "Definitely something to check out in the future, thanks for info and of course all your work :)
", "rawMarkdown": "Definitely something to check out in the future, thanks for info and of course all your work :)", "votes": null }, { "id": "964494", "postDate": "08/10/2020 00:10:28", "content": "I'll put cluster NMS on the short-list of things to look at, thx
", "rawMarkdown": "I'll put cluster NMS on the short-list of things to look at, thx", "votes": null }, { "id": "964500", "postDate": "08/10/2020 00:19:54", "content": "@riblidezso @kyoshioka47 thanks, there is a full COCO training pipeline with datasets and augs there, but it's VERY COCO specific right now with the main goal to replicate the original impl' COCO results, which I did on the smaller models (takes a long time). The plan is to add some fine-tuning support and add some more variety of available augmentations out of the box, and cover at least the major benchmark dataset 'types' (COCO, VOC, OIV5).
\n\nIt's not practical to cover all variations on augmentations, they often need to get dataset specific to be relevant.
\n\nI'd also like to point out an issue with the way Kaggle is currently operating for the 'code competitions'. A lot of people used various versions of my EfficientDet by Kaggle datasets to work with constraints of these challenges. One cannot simply do a pip/github based install of the latest. Since I was actively developing, some of them may have included code prior to a weight init fix, etc. Always a good idea to check the original source. Kaggle REALLY needs to find a way to allow proper, natural use of libraries via pip, github etc in a manner that firewalls the private data.
", "rawMarkdown": "riblidezso @kyoshioka47 thanks, there is a full COCO training pipeline with datasets and augs there, but it's VERY COCO specific right now with the main goal to replicate the original impl' COCO results, which I did on the smaller models (takes a long time). The plan is to add some fine-tuning support and add some more variety of available augmentations out of the box, and cover at least the major benchmark dataset 'types' (COCO, VOC, OIV5). \n\nIt's not practical to cover all variations on augmentations, they often need to get dataset specific to be relevant. \n\nI'd also like to point out an issue with the way Kaggle is currently operating for the 'code competitions'. A lot of people used various versions of my EfficientDet by Kaggle datasets to work with constraints of these challenges. One cannot simply do a pip/github based install of the latest. Since I was actively developing, some of them may have included code prior to a weight init fix, etc. Always a good idea to check the original source. Kaggle REALLY needs to find a way to allow proper, natural use of libraries via pip, github etc in a manner that firewalls the private data.", "votes": null }, { "id": "964758", "postDate": "08/10/2020 06:33:59", "content": "\n\n\nI'd also like to point out an issue with the way Kaggle is currently operating for the 'code competitions'. A lot of people used various versions of my EfficientDet by Kaggle datasets to work with constraints of these challenges. One cannot simply do a pip/github based install of the latest. Since I was actively developing, some of them may have included code prior to a weight init fix, etc. Always a good idea to check the original source. Kaggle REALLY needs to find a way to allow proper, natural use of libraries via pip, github etc in a manner that firewalls the private data.
\n
I dunno, I don't see a big issue with how it works now.
\n\nI find it nice that notebooks are always linked to an offline version which they are guaranteed to work with. So you never have to start with debugging errors due to changes in the library. E.g.: Alex's notebook does not work with your recent code, some new changes broke it.
\n\nCompetitions are quite short and I think it is easier to work with a single version during a competition, unless you really want some new functionality. (e.g.: update to pytorch 1.6 broke my training code on Colab 3 days before the deadline, it was an easy fix, but it could be worse.).
\n\nAlso It is not a big effort if you want to update a library. I went to github, downloaded your latest version, uploaded as my own dataset and modified the notebook to work with that. And repeated this when I wanted to use D7x.
", "rawMarkdown": "> I'd also like to point out an issue with the way Kaggle is currently operating for the 'code competitions'. A lot of people used various versions of my EfficientDet by Kaggle datasets to work with constraints of these challenges. One cannot simply do a pip/github based install of the latest. Since I was actively developing, some of them may have included code prior to a weight init fix, etc. Always a good idea to check the original source. Kaggle REALLY needs to find a way to allow proper, natural use of libraries via pip, github etc in a manner that firewalls the private data.\n\nI dunno, I don't see a big issue with how it works now.\n\nI find it nice that notebooks are always linked to an offline version which they are guaranteed to work with. So you never have to start with debugging errors due to changes in the library. E.g.: Alex's notebook does not work with your recent code, some new changes broke it.\n\nCompetitions are quite short and I think it is easier to work with a single version during a competition, unless you really want some new functionality. (e.g.: update to pytorch 1.6 broke my training code on Colab 3 days before the deadline, it was an easy fix, but it could be worse.). \n\nAlso It is not a big effort if you want to update a library. I went to github, downloaded your latest version, uploaded as my own dataset and modified the notebook to work with that. And repeated this when I wanted to use D7x.", "votes": null }, { "id": "964759", "postDate": "08/10/2020 06:34:51", "content": "I couldn't agree with you more. The mmdet's dataset style is very unfriendly.
", "rawMarkdown": "I couldn't agree with you more. The mmdet's dataset style is very unfriendly.", "votes": null }, { "id": "964771", "postDate": "08/10/2020 06:46:28", "content": "Maybe TPU support?
", "rawMarkdown": "Maybe TPU support?", "votes": null }, { "id": "965348", "postDate": "08/10/2020 15:12:29", "content": "After training your pytorch implementation of Effdet D3 to D7 for over 100s of hours in this competition alone and experimenting on everything except architecture of the model, all I wish to add apart from the already mentioned changes here are:\n(Note that I've used Alex's notebook as starter which has custom changes, so I hope they extend in general scenario too outside the scope of this challenge)
\n\nNeed for a custom best checkpoint saving metric, I found that saving on the basis of least Validation loss alone doesn't give the best general model (even in non-overfitting cases).\nMaybe add a combination of mAP, Recall, IoU (Better if G/C/DIoU) etc in your complete pipeline.
I don't know much about the architecture side but as in section 2.3 of https://arxiv.org/pdf/2004.10934.pdf they suggest boosts on fixing the image size in some ratio fitting their stride size of pooling layers. \nYoloV5 also does something similar, it has a fixed input size of each image (640*640 iirc), or any suitable multiple of 32 if input is less than it using the 'letterbox' named function.\nMaybe you could build upon this idea further in your implementation of Mosaic in the complete pipeline.
In the same paper as above they suggest the changes in section 4.5 that need to be done to make the model computationally inexpensive, would be friendlier to starters if it can be implemented in Effdet too. \nOr atleast integrated mixed precision support using Nvidia Apex or default pytorch 1.6 implementation.
These were just from my basic experiments and understanding. Please ignore if any change doesn't fall under your domain or are already implemented in some or the other way.
", "rawMarkdown": "After training your pytorch implementation of Effdet D3 to D7 for over 100s of hours in this competition alone and experimenting on everything except architecture of the model, all I wish to add apart from the already mentioned changes here are:\n(Note that I've used Alex's notebook as starter which has custom changes, so I hope they extend in general scenario too outside the scope of this challenge)\n\n1. Need for a custom best checkpoint saving metric, I found that saving on the basis of least Validation loss alone doesn't give the best general model (even in non-overfitting cases).\nMaybe add a combination of mAP, Recall, IoU (Better if G/C/DIoU) etc in your complete pipeline.\n\n2. I don't know much about the architecture side but as in section 2.3 of https://arxiv.org/pdf/2004.10934.pdf they suggest boosts on fixing the image size in some ratio fitting their stride size of pooling layers. \nYoloV5 also does something similar, it has a fixed input size of each image (640*640 iirc), or any suitable multiple of 32 if input is less than it using the 'letterbox' named function.\nMaybe you could build upon this idea further in your implementation of Mosaic in the complete pipeline.\n\n3. In the same paper as above they suggest the changes in section 4.5 that need to be done to make the model computationally inexpensive, would be friendlier to starters if it can be implemented in Effdet too. \nOr atleast integrated mixed precision support using Nvidia Apex or default pytorch 1.6 implementation.\n\nThese were just from my basic experiments and understanding. Please ignore if any change doesn't fall under your domain or are already implemented in some or the other way.", "votes": null }, { "id": "965553", "postDate": "08/10/2020 17:52:30", "content": "\n\n\nI find it nice that notebooks are always linked to an offline version which they are guaranteed to work with. So you never have to start with debugging errors due to changes in the library. E.g.: Alex's notebook does not work with your recent code, some new changes broke it.
\n
I'd argue that's the whole point of git, source control in general, and it's done far more elegantly than dumping a bunch of code into a Kaggle dataset and calling it a day. A fork of my code and/or pip git+ install w/ a specific branch/tag/commit would have served the same purpose, and then it would have been far easier to actually compare, merge, cherry pick fixes and improvements that were made (or adapt to/ignore breaking changes).
", "rawMarkdown": "> I find it nice that notebooks are always linked to an offline version which they are guaranteed to work with. So you never have to start with debugging errors due to changes in the library. E.g.: Alex's notebook does not work with your recent code, some new changes broke it.\n\nI'd argue that's the whole point of git, source control in general, and it's done far more elegantly than dumping a bunch of code into a Kaggle dataset and calling it a day. A fork of my code and/or pip git+ install w/ a specific branch/tag/commit would have served the same purpose, and then it would have been far easier to actually compare, merge, cherry pick fixes and improvements that were made (or adapt to/ignore breaking changes).", "votes": null }, { "id": "965608", "postDate": "08/10/2020 18:22:04", "content": "Sure it would be nice. I just don't think it's such a big issue.
\n\nI think most development is done outside of internet free kaggle instances, so people can use all the source control tools they want. The only thing is that, at the end of the day they have to dump the source code they used as kaggle datasets, and use those in their internet free submission run.
\n\nI guess the main point is that this way of doing things is not making the competitors life too hard to make kaggle attempt to try to solve installing packages with pip/git while making sure they can maintain the integrity of the private data.\nI am no security expert, but I imagine that there are very resourceful malevolent individuals, who could try to hack the private data and use it for cheating if kaggle makes even the slightest mistake with their firewall.
", "rawMarkdown": "Sure it would be nice. I just don't think it's such a big issue.\n\nI think most development is done outside of internet free kaggle instances, so people can use all the source control tools they want. The only thing is that, at the end of the day they have to dump the source code they used as kaggle datasets, and use those in their internet free submission run.\n\nI guess the main point is that this way of doing things is not making the competitors life too hard to make kaggle attempt to try to solve installing packages with pip/git while making sure they can maintain the integrity of the private data.\nI am no security expert, but I imagine that there are very resourceful malevolent individuals, who could try to hack the private data and use it for cheating if kaggle makes even the slightest mistake with their firewall.", "votes": null }, { "id": "968116", "postDate": "08/12/2020 17:55:45", "content": "I'd like to have TPU working, but there are some issues with the NMS / anchor part of the code.
\nIn general, have you found it worth running PyTorch models on a TPU? My last experiments with PyTorch XLA comparing EfficientNet classification models vs GPU weren't very promising, but that was end of 2019 and I imagine things have improved… ?
", "rawMarkdown": "I'd like to have TPU working, but there are some issues with the NMS / anchor part of the code. \n\nIn general, have you found it worth running PyTorch models on a TPU? My last experiments with PyTorch XLA comparing EfficientNet classification models vs GPU weren't very promising, but that was end of 2019 and I imagine things have improved... ?", "votes": null }, { "id": "968133", "postDate": "08/12/2020 18:04:24", "content": "Thanks for the comments.
\nDefinitely, not great to use just the loss. I was kinda hoping someone would have come up with a nice simple mAP evaluator that works with my code during this challenge. I use the COCO evaluator for checkpoint selection when training w/ COCO style datasets, but haven't written a simpler stand alone AP/AR routine yet.
Supporting different image anchor layouts (non-square) is on my TODO list. Allowing dynamic layout selection would work, it would incure a penalty each time it decides it needs a different anchor config, but not a huge deal, some impl do that every single batch/image anyways. One gotcha with the BiFPN though is that with feature stride up to 128, I believe it's constrained to img_size % 128 == 0, so you'd be fitting multiples of 128 instead of 32 and would thus still have a reasonable number of letterboxing situations
I've got APEX AMP support hard coded to on in the main repository (falls back to FP32 if AMP not installed properly), it's brutal without it. Not sure if that was lost when it was brough over to Kaggle. There is also a trained EfficientDet-Lite0 model that someone helped me train as an experiment, and I've created other definitions for lighter weight backbones based on other timm models that have yet ot be trained.
Hi Ross,
\nthanks for your efforts!
\nI played a bit with your code and tried a few changes:
I believe, CIoU loss gives some improvements over huber loss. I haven't had enough time to train my models until the full convergence (I did about 20-30 epochs per each run, starting from pretrained on COCO weights). The results for the underfitted models were quite promising :) I saw improvements both on the local validation and PL.
I tried cluster NMS from the same authors and had some improvements, but it might be due to a better threshold value. Didn't study this one thoroughly.
Can't say much about mosaic. I think, I've messed this one :D
I'll put cluster NMS on the short-list of things to look at, thx
", "votes": null, "replies": [] } ] }, { "id": 963387, "author_name": "kyoshioka47", "author_url": "", "post_date": "08/09/2020 01:38:36", "content": "Thanks for the great repo @rwightman
\nYOLOv5 was handy for detection noobies because the entire COCO training pipeline was wrapped within the repo.
\nMaybe if you include the Effdet train pipeline within the repo too, people can train easily on custom datasets!
I couldn't agree with you more. The mmdet's dataset style is very unfriendly.
", "votes": null, "replies": [] } ] }, { "id": 964414, "author_name": "jacekpoplawski", "author_url": "", "post_date": "08/09/2020 20:44:47", "content": "Definitely something to check out in the future, thanks for info and of course all your work :)
", "votes": null, "replies": [] }, { "id": 963750, "author_name": "riblidezso", "author_url": "", "post_date": "08/09/2020 09:01:09", "content": "Thank you very much for your work. I have found it very useful and convenient.
\n\nIn addition to what you mention, I think a very easy to use fine tuning script would make your repo more popular. Also maybe with additional options for up-down flips and random transpositions (or rotations) during training, which can be very useful for custom datasets with these symmetries. Like this one.
\n\nHere is more that I’ve done with your code:\n https://www.kaggle.com/riblidezso/gwd-efficientdet-clean-training-and-inference-code
", "votes": null, "replies": [] }, { "id": 964500, "author_name": "rwightman", "author_url": "", "post_date": "08/10/2020 00:19:54", "content": "@riblidezso @kyoshioka47 thanks, there is a full COCO training pipeline with datasets and augs there, but it's VERY COCO specific right now with the main goal to replicate the original impl' COCO results, which I did on the smaller models (takes a long time). The plan is to add some fine-tuning support and add some more variety of available augmentations out of the box, and cover at least the major benchmark dataset 'types' (COCO, VOC, OIV5).
\n\nIt's not practical to cover all variations on augmentations, they often need to get dataset specific to be relevant.
\n\nI'd also like to point out an issue with the way Kaggle is currently operating for the 'code competitions'. A lot of people used various versions of my EfficientDet by Kaggle datasets to work with constraints of these challenges. One cannot simply do a pip/github based install of the latest. Since I was actively developing, some of them may have included code prior to a weight init fix, etc. Always a good idea to check the original source. Kaggle REALLY needs to find a way to allow proper, natural use of libraries via pip, github etc in a manner that firewalls the private data.
", "votes": null, "replies": [ { "id": 964758, "author_name": "riblidezso", "author_url": "", "post_date": "08/10/2020 06:33:59", "content": "\n\n\nI'd also like to point out an issue with the way Kaggle is currently operating for the 'code competitions'. A lot of people used various versions of my EfficientDet by Kaggle datasets to work with constraints of these challenges. One cannot simply do a pip/github based install of the latest. Since I was actively developing, some of them may have included code prior to a weight init fix, etc. Always a good idea to check the original source. Kaggle REALLY needs to find a way to allow proper, natural use of libraries via pip, github etc in a manner that firewalls the private data.
\n
I dunno, I don't see a big issue with how it works now.
\n\nI find it nice that notebooks are always linked to an offline version which they are guaranteed to work with. So you never have to start with debugging errors due to changes in the library. E.g.: Alex's notebook does not work with your recent code, some new changes broke it.
\n\nCompetitions are quite short and I think it is easier to work with a single version during a competition, unless you really want some new functionality. (e.g.: update to pytorch 1.6 broke my training code on Colab 3 days before the deadline, it was an easy fix, but it could be worse.).
\n\nAlso It is not a big effort if you want to update a library. I went to github, downloaded your latest version, uploaded as my own dataset and modified the notebook to work with that. And repeated this when I wanted to use D7x.
", "votes": null, "replies": [] }, { "id": 965553, "author_name": "rwightman", "author_url": "", "post_date": "08/10/2020 17:52:30", "content": "\n\n\nI find it nice that notebooks are always linked to an offline version which they are guaranteed to work with. So you never have to start with debugging errors due to changes in the library. E.g.: Alex's notebook does not work with your recent code, some new changes broke it.
\n
I'd argue that's the whole point of git, source control in general, and it's done far more elegantly than dumping a bunch of code into a Kaggle dataset and calling it a day. A fork of my code and/or pip git+ install w/ a specific branch/tag/commit would have served the same purpose, and then it would have been far easier to actually compare, merge, cherry pick fixes and improvements that were made (or adapt to/ignore breaking changes).
", "votes": null, "replies": [ { "id": 965608, "author_name": "riblidezso", "author_url": "", "post_date": "08/10/2020 18:22:04", "content": "Sure it would be nice. I just don't think it's such a big issue.
\n\nI think most development is done outside of internet free kaggle instances, so people can use all the source control tools they want. The only thing is that, at the end of the day they have to dump the source code they used as kaggle datasets, and use those in their internet free submission run.
\n\nI guess the main point is that this way of doing things is not making the competitors life too hard to make kaggle attempt to try to solve installing packages with pip/git while making sure they can maintain the integrity of the private data.\nI am no security expert, but I imagine that there are very resourceful malevolent individuals, who could try to hack the private data and use it for cheating if kaggle makes even the slightest mistake with their firewall.
", "votes": null, "replies": [] } ] } ] }, { "id": 964771, "author_name": "riblidezso", "author_url": "", "post_date": "08/10/2020 06:46:28", "content": "Maybe TPU support?
", "votes": null, "replies": [ { "id": 968116, "author_name": "rwightman", "author_url": "", "post_date": "08/12/2020 17:55:45", "content": "I'd like to have TPU working, but there are some issues with the NMS / anchor part of the code.
\nIn general, have you found it worth running PyTorch models on a TPU? My last experiments with PyTorch XLA comparing EfficientNet classification models vs GPU weren't very promising, but that was end of 2019 and I imagine things have improved… ?
", "votes": null, "replies": [] } ] }, { "id": 965348, "author_name": "ii5m0k3ii", "author_url": "", "post_date": "08/10/2020 15:12:29", "content": "After training your pytorch implementation of Effdet D3 to D7 for over 100s of hours in this competition alone and experimenting on everything except architecture of the model, all I wish to add apart from the already mentioned changes here are:\n(Note that I've used Alex's notebook as starter which has custom changes, so I hope they extend in general scenario too outside the scope of this challenge)
\n\nNeed for a custom best checkpoint saving metric, I found that saving on the basis of least Validation loss alone doesn't give the best general model (even in non-overfitting cases).\nMaybe add a combination of mAP, Recall, IoU (Better if G/C/DIoU) etc in your complete pipeline.
I don't know much about the architecture side but as in section 2.3 of https://arxiv.org/pdf/2004.10934.pdf they suggest boosts on fixing the image size in some ratio fitting their stride size of pooling layers. \nYoloV5 also does something similar, it has a fixed input size of each image (640*640 iirc), or any suitable multiple of 32 if input is less than it using the 'letterbox' named function.\nMaybe you could build upon this idea further in your implementation of Mosaic in the complete pipeline.
In the same paper as above they suggest the changes in section 4.5 that need to be done to make the model computationally inexpensive, would be friendlier to starters if it can be implemented in Effdet too. \nOr atleast integrated mixed precision support using Nvidia Apex or default pytorch 1.6 implementation.
These were just from my basic experiments and understanding. Please ignore if any change doesn't fall under your domain or are already implemented in some or the other way.
", "votes": null, "replies": [ { "id": 968133, "author_name": "rwightman", "author_url": "", "post_date": "08/12/2020 18:04:24", "content": "Thanks for the comments.
\nDefinitely, not great to use just the loss. I was kinda hoping someone would have come up with a nice simple mAP evaluator that works with my code during this challenge. I use the COCO evaluator for checkpoint selection when training w/ COCO style datasets, but haven't written a simpler stand alone AP/AR routine yet.
Supporting different image anchor layouts (non-square) is on my TODO list. Allowing dynamic layout selection would work, it would incure a penalty each time it decides it needs a different anchor config, but not a huge deal, some impl do that every single batch/image anyways. One gotcha with the BiFPN though is that with feature stride up to 128, I believe it's constrained to img_size % 128 == 0, so you'd be fitting multiples of 128 instead of 32 and would thus still have a reasonable number of letterboxing situations
I've got APEX AMP support hard coded to on in the main repository (falls back to FP32 if AMP not installed properly), it's brutal without it. Not sure if that was lost when it was brough over to Kaggle. There is also a trained EfficientDet-Lite0 model that someone helped me train as an experiment, and I've created other definitions for lighter weight backbones based on other timm models that have yet ot be trained.