{ "id": 152368, "title": "EfficientDet Baseline", "url": "/competitions/global-wheat-detection/discussion/152368", "author_name": "Alex Shonenkov", "post_date": "2020-05-19T15:01:24.843000", "votes": 25, "comment_count": 22, "views": 0, "content": "

Hi everyone!

\n\n

I have published 2 new kernels about EfficientDet approach for this competition ;)

\n\n

PyTorch

\n\n\n\n

You are welcome!

\n\n

P.S. Without Pseudo-Labeling ;)

\n\n

https://miro.medium.com/max/2400/0*ApAKUWtseHcvRV2U.png' width=400>

", "messages": [ { "id": 853879, "postDate": "2020-05-19T15:01:24.843Z", "content": "

Hi everyone!

\n\n

I have published 2 new kernels about EfficientDet approach for this competition ;)

\n\n

PyTorch

\n\n\n\n

You are welcome!

\n\n

P.S. Without Pseudo-Labeling ;)

\n\n

https://miro.medium.com/max/2400/0*ApAKUWtseHcvRV2U.png' width=400>

", "rawMarkdown": "Hi everyone!\n\nI have published 2 new kernels about [EfficientDet approach](https://arxiv.org/pdf/1911.09070.pdf) for this competition ;)\n\nPyTorch\n\n- [[Training] EfficientDet](https://www.kaggle.com/shonenkov/training-efficientdet)\n- [[Inference] EfficientDet](https://www.kaggle.com/shonenkov/inference-efficientdet)\n\nYou are welcome!\n\nP.S. Without Pseudo-Labeling ;)\n\n\n ", "votes": 25 }, { "id": 887768, "postDate": "2020-06-15T21:15:20.103Z", "content": "

@shonenkov thanks a lot for your great kernel! Below you said that it is a good idea to use RAM for DatasetRetriever. I used Colab free and I have only 12gb RAM. I tried to load all images into RAM, but 12 gb is not enough for me. Maybe I do something wrong. How many RAM do you use?

", "rawMarkdown": "@shonenkov thanks a lot for your great kernel! Below you said that it is a good idea to use RAM for DatasetRetriever. I used Colab free and I have only 12gb RAM. I tried to load all images into RAM, but 12 gb is not enough for me. Maybe I do something wrong. How many RAM do you use?", "votes": 1, "replies": [ { "id": 888087, "postDate": "2020-06-16T05:45:43.560Z", "content": "

@smolsnastya thank you! I used HIGH RAM mode (colab pro), it costs 10$/month. There are 36 GB RAM

", "rawMarkdown": "@smolsnastya thank you! I used HIGH RAM mode (colab pro), it costs 10$/month. There are 36 GB RAM" } ] }, { "id": 854333, "postDate": "2020-05-20T00:55:16.823Z", "content": "

really nice sharing, which is huge improvement of public kernel score. many thanks.

", "rawMarkdown": "really nice sharing, which is huge improvement of public kernel score. many thanks.", "votes": 1 }, { "id": 854138, "postDate": "2020-05-19T19:36:23.730Z", "content": "

Wow, looks very promising. Is the model used in your inference kernel submission (0.7176) the exact model that comes from your training kernel? Or is that with optimized parameters + other tricks?

\n\n

Also, I see that you are using an ensemble in this approach. Do you know what kind of LB score a standalone EfficientDet model would obtain? It would be helpful to know in order to compare against the different models.

", "rawMarkdown": "Wow, looks very promising. Is the model used in your inference kernel submission (0.7176) the exact model that comes from your training kernel? Or is that with optimized parameters + other tricks?\n\nAlso, I see that you are using an ensemble in this approach. Do you know what kind of LB score a standalone EfficientDet model would obtain? It would be helpful to know in order to compare against the different models.", "votes": 1, "replies": [ { "id": 854159, "postDate": "2020-05-19T19:56:17.987Z", "content": "

Single model, no ensemble for these kernels. No any other tricks :)

", "rawMarkdown": "Single model, no ensemble for these kernels. No any other tricks :)", "votes": 2 }, { "id": 854167, "postDate": "2020-05-19T20:10:30.367Z", "content": "

Oh wow, even more impressive! I suppose the augmentation you have built up improves it's performance a lot. Other efficientdet implementations didn't break 0.6

\n\n

Thanks again for your contribution

", "rawMarkdown": "Oh wow, even more impressive! I suppose the augmentation you have built up improves it's performance a lot. [Other efficientdet implementations didn't break 0.6](https://www.kaggle.com/c/global-wheat-detection/discussion/150780)\n\nThanks again for your contribution", "votes": 1 }, { "id": 854175, "postDate": "2020-05-19T20:17:41.303Z", "content": "

Using only flip as augmentations for effdet I have got 0.6812 for single model with image size 512x512. So, yes - augmentation is very important. But model architecture gives boost from fasterrcnn 0.6415 --> 0.6812 effdet with equal setup of opt/sched/augs.

", "rawMarkdown": "Using only flip as augmentations for effdet I have got 0.6812 for single model with image size 512x512. So, yes - augmentation is very important. But model architecture gives boost from fasterrcnn [0.6415](https://www.kaggle.com/shonenkov/wbf-approach-for-ensemble#847104) --> 0.6812 effdet with equal setup of opt/sched/augs.", "votes": 2 } ] }, { "id": 1136092, "postDate": "2021-01-02T18:11:21.507Z", "content": "

For multiclass classification what can I do? Can anyone please help!

", "rawMarkdown": "For multiclass classification what can I do? Can anyone please help!" }, { "id": 854434, "postDate": "2020-05-20T02:56:21.853Z", "content": "

thanks a lot for your nice kernel. I would try Efficient-Det.

\n\n

could you tell me gpu memory usage, training time and machine settings when you trained?\nI'm afraid that my 2070 super does not meet requirements for D5.

", "rawMarkdown": "thanks a lot for your nice kernel. I would try Efficient-Det.\n\ncould you tell me gpu memory usage, training time and machine settings when you trained?\nI'm afraid that my 2070 super does not meet requirements for D5.", "replies": [ { "id": 854669, "postDate": "2020-05-20T07:52:53.437Z", "content": "

@lisosia For training I have used colab pro with the same GPU as Kaggle (P100) 👍 You can see all information in kernel. Welcome!

", "rawMarkdown": "@lisosia For training I have used colab pro with the same GPU as Kaggle (P100) 👍 You can see all information in kernel. Welcome!", "votes": 2 }, { "id": 857097, "postDate": "2020-05-22T09:49:44.757Z", "content": "

I run the train baseline locally on my 2070 GPU with batchsize=2.

", "rawMarkdown": "I run the train baseline locally on my 2070 GPU with batchsize=2.", "votes": 2 }, { "id": 857593, "postDate": "2020-05-22T18:26:14.690Z", "content": "

@vectoraaa I am doing the same thing. any luck in increasing batch_size to 4

", "rawMarkdown": "@vectoraaa I am doing the same thing. any luck in increasing batch_size to 4 \n" }, { "id": 858295, "postDate": "2020-05-23T11:01:57.960Z", "content": "

Also training on a 2070. It's definitely a bit slower than Faster RCNN but hopefully it is worth it! Unfortunately the RAM isn't big enough for a 1024^2 image model :(

", "rawMarkdown": "Also training on a 2070. It's definitely a bit slower than Faster RCNN but hopefully it is worth it! Unfortunately the RAM isn't big enough for a 1024^2 image model :(" }, { "id": 858301, "postDate": "2020-05-23T11:07:44.223Z", "content": "

@matthewmasters try augmentation and make the image_size to 512...

", "rawMarkdown": "@matthewmasters try augmentation and make the image_size to 512..." }, { "id": 858307, "postDate": "2020-05-23T11:09:36.067Z", "content": "

Yeah, I'm currently training with 512. Just remarking that it's not possible with 1024 on a 2070.

", "rawMarkdown": "Yeah, I'm currently training with 512. Just remarking that it's not possible with 1024 on a 2070." }, { "id": 881628, "postDate": "2020-06-11T08:06:24.583Z", "content": "

On colab pro (P100)\nEfficientDet5: 25min/epoch (same training kernel as @shonenkov)\nfasterRCNN: 10min/epoch\nDid I miss something or efficientDet is much slower ? On @shonenkov kernel one epoch is about 10min so I'm a bit confusing. Anyway thanks again for your amazing kernels

", "rawMarkdown": "On colab pro (P100)\nEfficientDet5: 25min/epoch (same training kernel as @shonenkov)\nfasterRCNN: 10min/epoch\nDid I miss something or efficientDet is much slower ? On @shonenkov kernel one epoch is about 10min so I'm a bit confusing. Anyway thanks again for your amazing kernels" }, { "id": 881731, "postDate": "2020-06-11T09:59:59.100Z", "content": "

@alexj21 I did about 7.5-9.0 mins. per epoch on d5, depending on the fold #.

", "rawMarkdown": "@alexj21 I did about 7.5-9.0 mins. per epoch on d5, depending on the fold #." }, { "id": 881733, "postDate": "2020-06-11T10:01:49.040Z", "rawMarkdown": "", "isDeleted": true }, { "id": 881734, "postDate": "2020-06-11T10:02:21.307Z", "content": "

@alexj21 you should use RAM for DatasetRetriever. On Colab free disk is very slow. Hope it helps you!

", "rawMarkdown": "@alexj21 you should use RAM for DatasetRetriever. On Colab free disk is very slow. Hope it helps you!" }, { "id": 882964, "postDate": "2020-06-12T09:03:47.547Z", "content": "

@shonenkov what do you mean \"RAM for DatasetRetriever\" ? I'm missing something. I've enabled the high ram option on colab pro but it does not increase training speed\nthanks

", "rawMarkdown": "@shonenkov what do you mean \"RAM for DatasetRetriever\" ? I'm missing something. I've enabled the high ram option on colab pro but it does not increase training speed\nthanks" }, { "id": 883713, "postDate": "2020-06-12T21:11:42.577Z", "content": "

I believe he means that you should load all the images into RAM at the start and then retrieve from there using your dataloader. This will be much faster than loading from disk every time

", "rawMarkdown": "I believe he means that you should load all the images into RAM at the start and then retrieve from there using your dataloader. This will be much faster than loading from disk every time", "votes": 1 }, { "id": 884167, "postDate": "2020-06-13T08:13:59.410Z", "content": "

Oh OK I've got it. I'll give it a try thanks 👍

", "rawMarkdown": "Oh OK I've got it. I'll give it a try thanks 👍 " } ] } ], "comments": [ { "id": 887768, "author_name": "Anastasia Smolskaya", "author_url": "", "post_date": "2020-06-15T21:15:20.103000", "content": "

@shonenkov thanks a lot for your great kernel! Below you said that it is a good idea to use RAM for DatasetRetriever. I used Colab free and I have only 12gb RAM. I tried to load all images into RAM, but 12 gb is not enough for me. Maybe I do something wrong. How many RAM do you use?

", "votes": 1, "replies": [ { "id": 888087, "author_name": "Alex Shonenkov", "author_url": "", "post_date": "2020-06-16T05:45:43.560000", "content": "

@smolsnastya thank you! I used HIGH RAM mode (colab pro), it costs 10$/month. There are 36 GB RAM

", "votes": 0, "replies": [] } ] }, { "id": 854333, "author_name": "Tian", "author_url": "", "post_date": "2020-05-20T00:55:16.823000", "content": "

really nice sharing, which is huge improvement of public kernel score. many thanks.

", "votes": 1, "replies": [] }, { "id": 854138, "author_name": "Matt", "author_url": "", "post_date": "2020-05-19T19:36:23.730000", "content": "

Wow, looks very promising. Is the model used in your inference kernel submission (0.7176) the exact model that comes from your training kernel? Or is that with optimized parameters + other tricks?

\n\n

Also, I see that you are using an ensemble in this approach. Do you know what kind of LB score a standalone EfficientDet model would obtain? It would be helpful to know in order to compare against the different models.

", "votes": 1, "replies": [ { "id": 854159, "author_name": "Alex Shonenkov", "author_url": "", "post_date": "2020-05-19T19:56:17.987000", "content": "

Single model, no ensemble for these kernels. No any other tricks :)

", "votes": 2, "replies": [] }, { "id": 854167, "author_name": "Matt", "author_url": "", "post_date": "2020-05-19T20:10:30.367000", "content": "

Oh wow, even more impressive! I suppose the augmentation you have built up improves it's performance a lot. Other efficientdet implementations didn't break 0.6

\n\n

Thanks again for your contribution

", "votes": 1, "replies": [] }, { "id": 854175, "author_name": "Alex Shonenkov", "author_url": "", "post_date": "2020-05-19T20:17:41.303000", "content": "

Using only flip as augmentations for effdet I have got 0.6812 for single model with image size 512x512. So, yes - augmentation is very important. But model architecture gives boost from fasterrcnn 0.6415 --> 0.6812 effdet with equal setup of opt/sched/augs.

", "votes": 2, "replies": [] } ] }, { "id": 1136092, "author_name": "AIFahim", "author_url": "", "post_date": "2021-01-02T18:11:21.507000", "content": "

For multiclass classification what can I do? Can anyone please help!

", "votes": 0, "replies": [] }, { "id": 854434, "author_name": "yama", "author_url": "", "post_date": "2020-05-20T02:56:21.853000", "content": "

thanks a lot for your nice kernel. I would try Efficient-Det.

\n\n

could you tell me gpu memory usage, training time and machine settings when you trained?\nI'm afraid that my 2070 super does not meet requirements for D5.

", "votes": 0, "replies": [ { "id": 854669, "author_name": "Alex Shonenkov", "author_url": "", "post_date": "2020-05-20T07:52:53.437000", "content": "

@lisosia For training I have used colab pro with the same GPU as Kaggle (P100) 👍 You can see all information in kernel. Welcome!

", "votes": 2, "replies": [] }, { "id": 857097, "author_name": "Ning Zhang", "author_url": "", "post_date": "2020-05-22T09:49:44.757000", "content": "

I run the train baseline locally on my 2070 GPU with batchsize=2.

", "votes": 2, "replies": [] }, { "id": 857593, "author_name": "Nitin Datta", "author_url": "", "post_date": "2020-05-22T18:26:14.690000", "content": "

@vectoraaa I am doing the same thing. any luck in increasing batch_size to 4

", "votes": 0, "replies": [] }, { "id": 858295, "author_name": "Matt", "author_url": "", "post_date": "2020-05-23T11:01:57.960000", "content": "

Also training on a 2070. It's definitely a bit slower than Faster RCNN but hopefully it is worth it! Unfortunately the RAM isn't big enough for a 1024^2 image model :(

", "votes": 0, "replies": [] }, { "id": 858301, "author_name": "Nitin Datta", "author_url": "", "post_date": "2020-05-23T11:07:44.223000", "content": "

@matthewmasters try augmentation and make the image_size to 512...

", "votes": 0, "replies": [] }, { "id": 858307, "author_name": "Matt", "author_url": "", "post_date": "2020-05-23T11:09:36.067000", "content": "

Yeah, I'm currently training with 512. Just remarking that it's not possible with 1024 on a 2070.

", "votes": 0, "replies": [] }, { "id": 881628, "author_name": "Alex", "author_url": "", "post_date": "2020-06-11T08:06:24.583000", "content": "

On colab pro (P100)\nEfficientDet5: 25min/epoch (same training kernel as @shonenkov)\nfasterRCNN: 10min/epoch\nDid I miss something or efficientDet is much slower ? On @shonenkov kernel one epoch is about 10min so I'm a bit confusing. Anyway thanks again for your amazing kernels

", "votes": 0, "replies": [] }, { "id": 881731, "author_name": "M2KeyOfZ", "author_url": "", "post_date": "2020-06-11T09:59:59.100000", "content": "

@alexj21 I did about 7.5-9.0 mins. per epoch on d5, depending on the fold #.

", "votes": 0, "replies": [] }, { "id": 881733, "author_name": "", "author_url": "", "post_date": "2020-06-11T10:01:49.040000", "content": "", "votes": 0, "replies": [] }, { "id": 881734, "author_name": "Alex Shonenkov", "author_url": "", "post_date": "2020-06-11T10:02:21.307000", "content": "

@alexj21 you should use RAM for DatasetRetriever. On Colab free disk is very slow. Hope it helps you!

", "votes": 0, "replies": [] }, { "id": 882964, "author_name": "Alex", "author_url": "", "post_date": "2020-06-12T09:03:47.547000", "content": "

@shonenkov what do you mean \"RAM for DatasetRetriever\" ? I'm missing something. I've enabled the high ram option on colab pro but it does not increase training speed\nthanks

", "votes": 0, "replies": [] }, { "id": 883713, "author_name": "Matt", "author_url": "", "post_date": "2020-06-12T21:11:42.577000", "content": "

I believe he means that you should load all the images into RAM at the start and then retrieve from there using your dataloader. This will be much faster than loading from disk every time

", "votes": 1, "replies": [] }, { "id": 884167, "author_name": "Alex", "author_url": "", "post_date": "2020-06-13T08:13:59.410000", "content": "

Oh OK I've got it. I'll give it a try thanks 👍

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "853879": "Hi everyone!\n\nI have published 2 new kernels about [EfficientDet approach](https://arxiv.org/pdf/1911.09070.pdf) for this competition ;)\n\nPyTorch\n\n- [[Training] EfficientDet](https://www.kaggle.com/shonenkov/training-efficientdet)\n- [[Inference] EfficientDet](https://www.kaggle.com/shonenkov/inference-efficientdet)\n\nYou are welcome!\n\nP.S. Without Pseudo-Labeling ;)\n\n\n ", "887768": "@shonenkov thanks a lot for your great kernel! Below you said that it is a good idea to use RAM for DatasetRetriever. I used Colab free and I have only 12gb RAM. I tried to load all images into RAM, but 12 gb is not enough for me. Maybe I do something wrong. How many RAM do you use?", "854333": "really nice sharing, which is huge improvement of public kernel score. many thanks.", "854138": "Wow, looks very promising. Is the model used in your inference kernel submission (0.7176) the exact model that comes from your training kernel? Or is that with optimized parameters + other tricks?\n\nAlso, I see that you are using an ensemble in this approach. Do you know what kind of LB score a standalone EfficientDet model would obtain? It would be helpful to know in order to compare against the different models.", "1136092": "For multiclass classification what can I do? Can anyone please help!", "854434": "thanks a lot for your nice kernel. I would try Efficient-Det.\n\ncould you tell me gpu memory usage, training time and machine settings when you trained?\nI'm afraid that my 2070 super does not meet requirements for D5." } }