{ "id": 169117, "title": "Train EfficientDet like YOLO V5(with code)", "url": "/competitions/global-wheat-detection/discussion/169117", "author_name": "", "post_date": "2020-07-23T01:30:02.128033400Z", "votes": 27, "comment_count": 51, "views": 0, "content": "

Hi,as we are known that Yolo v5 is out of the competition.But V5 did work.As the discussion mention:

\n\n

First --> It is doing gradient accumulation by default to a batch size of 64\nsecond --> It is also giving mixed precision training(fp16) by default if we have apex installed

\n\n

So let change it to EfficientDet.May be work :)) Hope the notebook will be helpful for someone. please feel free to upvote

\n\n

code available: Here

", "messages": [ { "id": "940490", "postDate": "07/23/2020 01:30:02", "content": "

Hi,as we are known that Yolo v5 is out of the competition.But V5 did work.As the discussion mention:

\n\n

First --> It is doing gradient accumulation by default to a batch size of 64\nsecond --> It is also giving mixed precision training(fp16) by default if we have apex installed

\n\n

So let change it to EfficientDet.May be work :)) Hope the notebook will be helpful for someone. please feel free to upvote

\n\n

code available: Here

", "rawMarkdown": "Hi,as we are known that Yolo v5 is out of the competition.But V5 did work.As the [discussion](https://www.kaggle.com/c/global-wheat-detection/discussion/167554) mention:\n\nFirst --> It is doing gradient accumulation by default to a batch size of 64\nsecond --> It is also giving mixed precision training(fp16) by default if we have apex installed\n\n\nSo let change it to EfficientDet.May be work :)) Hope the notebook will be helpful for someone. please feel free to upvote\n \ncode available: [Here](https://www.kaggle.com/raininbox/train-efficientdet-like-yolo-v5)", "votes": null }, { "id": "940501", "postDate": "07/23/2020 01:45:40", "content": "

Could u tell me the score it gets?

", "rawMarkdown": "Could u tell me the score it gets?", "votes": null }, { "id": "940523", "postDate": "07/23/2020 02:13:09", "content": "

😄 I can't tell you till now.Because the model still under train.If there any boost, I will public the result here

", "rawMarkdown": "😄 I can't tell you till now.Because the model still under train.If there any boost, I will public the result here", "votes": null }, { "id": "940529", "postDate": "07/23/2020 02:16:05", "content": "

Thanks for sharing!

\n\n

May I ask that loss = outputs['loss'] / self.accumulation_step is necessary or not?

\n\n

p.s. I saw that this is not in your kernel.

", "rawMarkdown": "Thanks for sharing!\n\nMay I ask that `loss = outputs['loss'] / self.accumulation_step` is necessary or not?\n\np.s. I saw that this is not in your kernel.", "votes": null }, { "id": "940533", "postDate": "07/23/2020 02:20:03", "content": "

I am trying some tricks ,too.😃 .May I ask what is your notebook's different with Alex's Notebook?

", "rawMarkdown": "I am trying some tricks ,too.😃 .May I ask what is your notebook's different with Alex's Notebook?", "votes": null }, { "id": "940549", "postDate": "07/23/2020 02:38:51", "content": "

:D I check the yolo v5's code without the \"/accumulate_step\".My code is copy from the V5.The size of the loss doesn't matter I guess.

", "rawMarkdown": ":D I check the yolo v5's code without the \"/accumulate_step\".My code is copy from the V5.The size of the loss doesn't matter I guess.", "votes": null }, { "id": "940552", "postDate": "07/23/2020 02:39:46", "content": "

I highlight it wiht some comment to show how to use it :D apex and gradient accumulate

", "rawMarkdown": "I highlight it wiht some comment to show how to use it :D apex and gradient accumulate", "votes": null }, { "id": "940555", "postDate": "07/23/2020 02:43:09", "content": "

OH,Thanks

", "rawMarkdown": "OH,Thanks", "votes": null }, { "id": "941293", "postDate": "07/23/2020 06:53:36", "content": "

@piantic If you are using Ross Wightman code, dont do that. because loss is getting averaged out by the below line.

\n\n

' loss, _, _ = self.model(images, boxes, labels)

", "rawMarkdown": "piantic If you are using Ross Wightman code, dont do that. because loss is getting averaged out by the below line.\n\n' loss, _, _ = self.model(images, boxes, labels)", "votes": null }, { "id": "941301", "postDate": "07/23/2020 06:55:14", "content": "

@raininbox IMHO it does matter as the amount of gradient in the backward pass will get lowered if we average it out only in Ross Wightman code

", "rawMarkdown": "raininbox IMHO it does matter as the amount of gradient in the backward pass will get lowered if we average it out only in Ross Wightman code", "votes": null }, { "id": "941391", "postDate": "07/23/2020 07:46:17", "content": "

@raininbox thanks..\nWhat is your train-validation strategy. does your cv score complies with lb ?

", "rawMarkdown": "raininbox thanks..\nWhat is your train-validation strategy. does your cv score complies with lb ?", "votes": null }, { "id": "941461", "postDate": "07/23/2020 08:21:15", "content": "

Thanks! @kalyanpichuka

", "rawMarkdown": "Thanks! @kalyanpichuka", "votes": null }, { "id": "941653", "postDate": "07/23/2020 10:39:28", "content": "

That's true.Got it

", "rawMarkdown": "That's true.Got it", "votes": null }, { "id": "941659", "postDate": "07/23/2020 10:40:19", "content": "

I haven't fininshed the train and submited it

", "rawMarkdown": "I haven't fininshed the train and submited it", "votes": null }, { "id": "941691", "postDate": "07/23/2020 11:00:22", "content": "

ok... do you perform bbox cleaning, is there any notebook that has published such dataset

", "rawMarkdown": "ok... do you perform bbox cleaning, is there any notebook that has published such dataset", "votes": null }, { "id": "942198", "postDate": "07/23/2020 16:25:14", "content": "

@kalyanpichuka if there any articles about this issue?

", "rawMarkdown": "kalyanpichuka if there any articles about this issue?", "votes": null }, { "id": "943144", "postDate": "07/24/2020 07:00:16", "content": "

I tried your approach, my score decreased(

", "rawMarkdown": "I tried your approach, my score decreased(", "votes": null }, { "id": "943559", "postDate": "07/24/2020 12:36:53", "content": "

I used your training kernal for batch size of 4, got LB score of 0.7407\nReally great! a definite upvote

", "rawMarkdown": "I used your training kernal for batch size of 4, got LB score of 0.7407\nReally great! a definite upvote", "votes": null }, { "id": "943570", "postDate": "07/24/2020 12:52:46", "content": "

May i ask, how many epochs did you run for?

", "rawMarkdown": "May i ask, how many epochs did you run for?", "votes": null }, { "id": "943650", "postDate": "07/24/2020 13:51:06", "content": "

Hello,did you use Single Model?

", "rawMarkdown": "Hello,did you use Single Model?", "votes": null }, { "id": "943728", "postDate": "07/24/2020 14:38:20", "content": "

Yes single model socre is 0.7407. I trained the model for 80 epochs.

", "rawMarkdown": "Yes single model socre is 0.7407. I trained the model for 80 epochs.", "votes": null }, { "id": "943831", "postDate": "07/24/2020 15:56:26", "content": "

@ahmmadrashid Is it with TTA or without ? \nAnd do you think changing scheduler will help ?

", "rawMarkdown": "ahmmadrashid Is it with TTA or without ? \nAnd do you think changing scheduler will help ?", "votes": null }, { "id": "944028", "postDate": "07/24/2020 18:53:31", "content": "

I ddint change the scheduler. Inference is with TTA.\nRemove some augmentation from the training pipeline such as togray its not needed.\nAlso you can try running with SGD optimization instead of AdamW.

", "rawMarkdown": "I ddint change the scheduler. Inference is with TTA.\nRemove some augmentation from the training pipeline such as togray its not needed.\nAlso you can try running with SGD optimization instead of AdamW.", "votes": null }, { "id": "944286", "postDate": "07/25/2020 01:59:08", "content": "

That's useful for me, thanks for the experiment!

", "rawMarkdown": "That's useful for me, thanks for the experiment!", "votes": null }, { "id": "944336", "postDate": "07/25/2020 03:05:29", "content": "

@ahmmadrashid did u use grad accumulation also ?

", "rawMarkdown": "ahmmadrashid did u use grad accumulation also ?", "votes": null }, { "id": "945015", "postDate": "07/25/2020 14:06:17", "content": "

From my experience, gradient accumulation are only LR optimization and fp16 will not improve Accuracy.\nI think yolov5 simply works because wheat labels are labeled by YOLO as well..

", "rawMarkdown": "From my experience, gradient accumulation are only LR optimization and fp16 will not improve Accuracy.\nI think yolov5 simply works because wheat labels are labeled by YOLO as well..", "votes": null }, { "id": "945231", "postDate": "07/25/2020 17:01:49", "content": "

They only do this for the “unlabeled sub datasets” though. Meaning only the UTokyo_2 images in the test dataset are affected (<10%). Maybe this is one reason why YoloV5 works well but I don't think it's the whole story.

", "rawMarkdown": "They only do this for the “unlabeled sub datasets” though. Meaning only the `UTokyo_2` images in the test dataset are affected (<10%). Maybe this is one reason why YoloV5 works well but I don't think it's the whole story.", "votes": null }, { "id": "945885", "postDate": "07/26/2020 07:42:10", "content": "

@raininbox you said you'd post an update here with the results but I haven't seen anything. How did it go?

", "rawMarkdown": "raininbox you said you'd post an update here with the results but I haven't seen anything. How did it go?", "votes": null }, { "id": "945887", "postDate": "07/26/2020 07:42:49", "content": "

@ahmmadrashid for the WBF, you used the Alex's default parameters (iou threshold and skip box threshold) or you made LB probing to select more appropriate ones?

", "rawMarkdown": "ahmmadrashid for the WBF, you used the Alex's default parameters (iou threshold and skip box threshold) or you made LB probing to select more appropriate ones?", "votes": null }, { "id": "945949", "postDate": "07/26/2020 08:29:32", "content": "

:D nice job.The max performance may get higher

", "rawMarkdown": ":D nice job.The max performance may get higher", "votes": null }, { "id": "945952", "postDate": "07/26/2020 08:30:57", "content": "

I am still adjusting the hyperparameter 😄

", "rawMarkdown": "I am still adjusting the hyperparameter 😄", "votes": null }, { "id": "945955", "postDate": "07/26/2020 08:31:49", "content": "

https://www.kaggle.com/raininbox/check-clean-big-small-bboxes

", "rawMarkdown": "https://www.kaggle.com/raininbox/check-clean-big-small-bboxes", "votes": null }, { "id": "946283", "postDate": "07/26/2020 13:43:47", "content": "

@chihaoui I used OOF prediction to set the IoU threshold and skip box threshold. Also I used the clean bbox dataset for training. But with pseudo labellin the LB score boosts about 1.5%.

", "rawMarkdown": "chihaoui I used OOF prediction to set the IoU threshold and skip box threshold. Also I used the clean bbox dataset for training. But with pseudo labellin the LB score boosts about 1.5%.", "votes": null }, { "id": "946284", "postDate": "07/26/2020 13:44:38", "content": "

@jaideepvalani Yes I used gradient accumulation.

", "rawMarkdown": "jaideepvalani Yes I used gradient accumulation.", "votes": null }, { "id": "946314", "postDate": "07/26/2020 14:01:30", "content": "

@ahmmadrashid Thanks for the reply. but what you do mean by \"clean bbox dataset for training\" exactly? you mean that you manually remove the noisy bboxes (too large, too small..) ?

", "rawMarkdown": "ahmmadrashid Thanks for the reply. but what you do mean by \"clean bbox dataset for training\" exactly? you mean that you manually remove the noisy bboxes (too large, too small..) ?", "votes": null }, { "id": "946331", "postDate": "07/26/2020 14:12:50", "content": "

I used this kernel\nhttps://www.kaggle.com/raininbox/check-clean-big-small-bboxes

", "rawMarkdown": "I used this kernel\nhttps://www.kaggle.com/raininbox/check-clean-big-small-bboxes", "votes": null }, { "id": "946340", "postDate": "07/26/2020 14:15:22", "content": "

Thanks

", "rawMarkdown": "Thanks", "votes": null }, { "id": "947074", "postDate": "07/27/2020 04:11:56", "content": "

Sorry to hear that, modify the hyperparameter may be work :D In my experiment it did work

", "rawMarkdown": "Sorry to hear that, modify the hyperparameter may be work :D In my experiment it did work", "votes": null }, { "id": "948101", "postDate": "07/27/2020 17:03:30", "content": "

I`ll try) Now it is always overfits

", "rawMarkdown": "I`ll try) Now it is always overfits", "votes": null }, { "id": "949447", "postDate": "07/28/2020 16:32:10", "content": "

Can you share some observations?

", "rawMarkdown": "Can you share some observations?", "votes": null }, { "id": "949509", "postDate": "07/28/2020 17:18:42", "content": "

Did you use SGD with or without momentum?

", "rawMarkdown": "Did you use SGD with or without momentum?", "votes": null }, { "id": "953594", "postDate": "07/31/2020 22:48:02", "content": "

:D after a great number of experiments, i sure that gradient accumulation did work. and clean data is also important.What's more, my 2x2080ti can only train the 1024x1024 images with apex&EfficientDet

", "rawMarkdown": ":D after a great number of experiments, i sure that gradient accumulation did work. and clean data is also important.What's more, my 2x2080ti can only train the 1024x1024 images with apex&EfficientDet", "votes": null }, { "id": "953728", "postDate": "08/01/2020 03:52:03", "content": "

Could you please share your scores \nThank you

", "rawMarkdown": "Could you please share your scores \nThank you", "votes": null }, { "id": "954063", "postDate": "08/01/2020 10:51:04", "content": "

@ahmmadrashid thanks.. what is best val loss did u obtain

", "rawMarkdown": "ahmmadrashid thanks.. what is best val loss did u obtain", "votes": null }, { "id": "954292", "postDate": "08/01/2020 15:51:12", "content": "

Singe model without any post trick(like pseudo label/TTA/OOF) can get LB 725😄

", "rawMarkdown": "Singe model without any post trick(like pseudo label/TTA/OOF) can get LB 725😄", "votes": null }, { "id": "954302", "postDate": "08/01/2020 15:59:31", "content": "

@raininbox ,\nThank you

", "rawMarkdown": "raininbox ,\nThank you", "votes": null }, { "id": "954647", "postDate": "08/02/2020 00:30:05", "content": "

@raininbox Hi, which kind of kernel do you use for Inference?\nI am confused about whether I should change the image size in inference phase and how.\nI try on this, but I encounter Error of 'Notebook Exceeded Allowed Compute'.

", "rawMarkdown": "raininbox Hi, which kind of kernel do you use for Inference?\nI am confused about whether I should change the image size in inference phase and how.\nI try on [this](https://www.kaggle.com/gofarther/efficientdet-framework-tta-pl-etc), but I encounter Error of 'Notebook Exceeded Allowed Compute'.", "votes": null }, { "id": "954650", "postDate": "08/02/2020 00:42:23", "content": "

Mee,too

", "rawMarkdown": "Mee,too", "votes": null }, { "id": "954765", "postDate": "08/02/2020 04:38:37", "content": "

嘤嘤嘤

", "rawMarkdown": "嘤嘤嘤", "votes": null }, { "id": "957433", "postDate": "08/04/2020 09:48:10", "content": "

Commenting on the plt code in inference phase may be helpful

", "rawMarkdown": "Commenting on the plt code in inference phase may be helpful", "votes": null }, { "id": "1016302", "postDate": "09/18/2020 20:00:26", "content": "

Hi, @raininbox.
\nCould you please say.
\nWhat optimization techniques did YOLO v5 use by default?

", "rawMarkdown": "Hi, @raininbox.\nCould you please say.\nWhat optimization techniques did YOLO v5 use by default?", "votes": null }, { "id": "1057928", "postDate": "10/23/2020 06:52:27", "content": "

Is EfficientDet as fast as YOLO? If not then what FPS can EfficientDet provide?

", "rawMarkdown": "Is EfficientDet as fast as YOLO? If not then what FPS can EfficientDet provide?", "votes": null } ], "comments": [ { "id": 1016302, "author_name": "yuliyakoshevaya", "author_url": "", "post_date": "09/18/2020 20:00:26", "content": "

Hi, @raininbox.
\nCould you please say.
\nWhat optimization techniques did YOLO v5 use by default?

", "votes": null, "replies": [] }, { "id": 1057928, "author_name": "haashaatif", "author_url": "", "post_date": "10/23/2020 06:52:27", "content": "

Is EfficientDet as fast as YOLO? If not then what FPS can EfficientDet provide?

", "votes": null, "replies": [] }, { "id": 940501, "author_name": "zekun98", "author_url": "", "post_date": "07/23/2020 01:45:40", "content": "

Could u tell me the score it gets?

", "votes": null, "replies": [ { "id": 940523, "author_name": "raininbox", "author_url": "", "post_date": "07/23/2020 02:13:09", "content": "

😄 I can't tell you till now.Because the model still under train.If there any boost, I will public the result here

", "votes": null, "replies": [] }, { "id": 940533, "author_name": "zekun98", "author_url": "", "post_date": "07/23/2020 02:20:03", "content": "

I am trying some tricks ,too.😃 .May I ask what is your notebook's different with Alex's Notebook?

", "votes": null, "replies": [] }, { "id": 940552, "author_name": "raininbox", "author_url": "", "post_date": "07/23/2020 02:39:46", "content": "

I highlight it wiht some comment to show how to use it :D apex and gradient accumulate

", "votes": null, "replies": [] }, { "id": 940555, "author_name": "zekun98", "author_url": "", "post_date": "07/23/2020 02:43:09", "content": "

OH,Thanks

", "votes": null, "replies": [] } ] }, { "id": 940529, "author_name": "piantic", "author_url": "", "post_date": "07/23/2020 02:16:05", "content": "

Thanks for sharing!

\n\n

May I ask that loss = outputs['loss'] / self.accumulation_step is necessary or not?

\n\n

p.s. I saw that this is not in your kernel.

", "votes": null, "replies": [ { "id": 940549, "author_name": "raininbox", "author_url": "", "post_date": "07/23/2020 02:38:51", "content": "

:D I check the yolo v5's code without the \"/accumulate_step\".My code is copy from the V5.The size of the loss doesn't matter I guess.

", "votes": null, "replies": [] }, { "id": 941293, "author_name": "kalyanpichuka", "author_url": "", "post_date": "07/23/2020 06:53:36", "content": "

@piantic If you are using Ross Wightman code, dont do that. because loss is getting averaged out by the below line.

\n\n

' loss, _, _ = self.model(images, boxes, labels)

", "votes": null, "replies": [] }, { "id": 941301, "author_name": "kalyanpichuka", "author_url": "", "post_date": "07/23/2020 06:55:14", "content": "

@raininbox IMHO it does matter as the amount of gradient in the backward pass will get lowered if we average it out only in Ross Wightman code

", "votes": null, "replies": [] }, { "id": 941461, "author_name": "piantic", "author_url": "", "post_date": "07/23/2020 08:21:15", "content": "

Thanks! @kalyanpichuka

", "votes": null, "replies": [] }, { "id": 941653, "author_name": "raininbox", "author_url": "", "post_date": "07/23/2020 10:39:28", "content": "

That's true.Got it

", "votes": null, "replies": [] }, { "id": 942198, "author_name": "kupchanski", "author_url": "", "post_date": "07/23/2020 16:25:14", "content": "

@kalyanpichuka if there any articles about this issue?

", "votes": null, "replies": [] } ] }, { "id": 941391, "author_name": "jaideepvalani", "author_url": "", "post_date": "07/23/2020 07:46:17", "content": "

@raininbox thanks..\nWhat is your train-validation strategy. does your cv score complies with lb ?

", "votes": null, "replies": [ { "id": 941659, "author_name": "raininbox", "author_url": "", "post_date": "07/23/2020 10:40:19", "content": "

I haven't fininshed the train and submited it

", "votes": null, "replies": [] }, { "id": 941691, "author_name": "jaideepvalani", "author_url": "", "post_date": "07/23/2020 11:00:22", "content": "

ok... do you perform bbox cleaning, is there any notebook that has published such dataset

", "votes": null, "replies": [] }, { "id": 945955, "author_name": "raininbox", "author_url": "", "post_date": "07/26/2020 08:31:49", "content": "

https://www.kaggle.com/raininbox/check-clean-big-small-bboxes

", "votes": null, "replies": [] } ] }, { "id": 943144, "author_name": "kupchanski", "author_url": "", "post_date": "07/24/2020 07:00:16", "content": "

I tried your approach, my score decreased(

", "votes": null, "replies": [ { "id": 947074, "author_name": "raininbox", "author_url": "", "post_date": "07/27/2020 04:11:56", "content": "

Sorry to hear that, modify the hyperparameter may be work :D In my experiment it did work

", "votes": null, "replies": [] }, { "id": 948101, "author_name": "kupchanski", "author_url": "", "post_date": "07/27/2020 17:03:30", "content": "

I`ll try) Now it is always overfits

", "votes": null, "replies": [] } ] }, { "id": 943559, "author_name": "ahmmadrashid", "author_url": "", "post_date": "07/24/2020 12:36:53", "content": "

I used your training kernal for batch size of 4, got LB score of 0.7407\nReally great! a definite upvote

", "votes": null, "replies": [ { "id": 943570, "author_name": "mohammadzunaed", "author_url": "", "post_date": "07/24/2020 12:52:46", "content": "

May i ask, how many epochs did you run for?

", "votes": null, "replies": [] }, { "id": 943650, "author_name": "zekunn", "author_url": "", "post_date": "07/24/2020 13:51:06", "content": "

Hello,did you use Single Model?

", "votes": null, "replies": [] }, { "id": 943728, "author_name": "ahmmadrashid", "author_url": "", "post_date": "07/24/2020 14:38:20", "content": "

Yes single model socre is 0.7407. I trained the model for 80 epochs.

", "votes": null, "replies": [] }, { "id": 943831, "author_name": "nacirbouazizi", "author_url": "", "post_date": "07/24/2020 15:56:26", "content": "

@ahmmadrashid Is it with TTA or without ? \nAnd do you think changing scheduler will help ?

", "votes": null, "replies": [] }, { "id": 944028, "author_name": "ahmmadrashid", "author_url": "", "post_date": "07/24/2020 18:53:31", "content": "

I ddint change the scheduler. Inference is with TTA.\nRemove some augmentation from the training pipeline such as togray its not needed.\nAlso you can try running with SGD optimization instead of AdamW.

", "votes": null, "replies": [] }, { "id": 944286, "author_name": "charliezhao", "author_url": "", "post_date": "07/25/2020 01:59:08", "content": "

That's useful for me, thanks for the experiment!

", "votes": null, "replies": [] }, { "id": 944336, "author_name": "jaideepvalani", "author_url": "", "post_date": "07/25/2020 03:05:29", "content": "

@ahmmadrashid did u use grad accumulation also ?

", "votes": null, "replies": [] }, { "id": 945887, "author_name": "chihaoui", "author_url": "", "post_date": "07/26/2020 07:42:49", "content": "

@ahmmadrashid for the WBF, you used the Alex's default parameters (iou threshold and skip box threshold) or you made LB probing to select more appropriate ones?

", "votes": null, "replies": [] }, { "id": 945949, "author_name": "raininbox", "author_url": "", "post_date": "07/26/2020 08:29:32", "content": "

:D nice job.The max performance may get higher

", "votes": null, "replies": [] }, { "id": 946283, "author_name": "ahmmadrashid", "author_url": "", "post_date": "07/26/2020 13:43:47", "content": "

@chihaoui I used OOF prediction to set the IoU threshold and skip box threshold. Also I used the clean bbox dataset for training. But with pseudo labellin the LB score boosts about 1.5%.

", "votes": null, "replies": [] }, { "id": 946284, "author_name": "ahmmadrashid", "author_url": "", "post_date": "07/26/2020 13:44:38", "content": "

@jaideepvalani Yes I used gradient accumulation.

", "votes": null, "replies": [] }, { "id": 946314, "author_name": "chihaoui", "author_url": "", "post_date": "07/26/2020 14:01:30", "content": "

@ahmmadrashid Thanks for the reply. but what you do mean by \"clean bbox dataset for training\" exactly? you mean that you manually remove the noisy bboxes (too large, too small..) ?

", "votes": null, "replies": [] }, { "id": 946331, "author_name": "ahmmadrashid", "author_url": "", "post_date": "07/26/2020 14:12:50", "content": "

I used this kernel\nhttps://www.kaggle.com/raininbox/check-clean-big-small-bboxes

", "votes": null, "replies": [] }, { "id": 946340, "author_name": "chihaoui", "author_url": "", "post_date": "07/26/2020 14:15:22", "content": "

Thanks

", "votes": null, "replies": [] }, { "id": 949509, "author_name": "zacchaeus", "author_url": "", "post_date": "07/28/2020 17:18:42", "content": "

Did you use SGD with or without momentum?

", "votes": null, "replies": [] }, { "id": 954063, "author_name": "jaideepvalani", "author_url": "", "post_date": "08/01/2020 10:51:04", "content": "

@ahmmadrashid thanks.. what is best val loss did u obtain

", "votes": null, "replies": [] } ] }, { "id": 945015, "author_name": "kyoshioka47", "author_url": "", "post_date": "07/25/2020 14:06:17", "content": "

From my experience, gradient accumulation are only LR optimization and fp16 will not improve Accuracy.\nI think yolov5 simply works because wheat labels are labeled by YOLO as well..

", "votes": null, "replies": [ { "id": 945231, "author_name": "matthewmasters", "author_url": "", "post_date": "07/25/2020 17:01:49", "content": "

They only do this for the “unlabeled sub datasets” though. Meaning only the UTokyo_2 images in the test dataset are affected (<10%). Maybe this is one reason why YoloV5 works well but I don't think it's the whole story.

", "votes": null, "replies": [] } ] }, { "id": 945885, "author_name": "matthewmasters", "author_url": "", "post_date": "07/26/2020 07:42:10", "content": "

@raininbox you said you'd post an update here with the results but I haven't seen anything. How did it go?

", "votes": null, "replies": [ { "id": 945952, "author_name": "raininbox", "author_url": "", "post_date": "07/26/2020 08:30:57", "content": "

I am still adjusting the hyperparameter 😄

", "votes": null, "replies": [] }, { "id": 949447, "author_name": "zacchaeus", "author_url": "", "post_date": "07/28/2020 16:32:10", "content": "

Can you share some observations?

", "votes": null, "replies": [] } ] }, { "id": 953594, "author_name": "raininbox", "author_url": "", "post_date": "07/31/2020 22:48:02", "content": "

:D after a great number of experiments, i sure that gradient accumulation did work. and clean data is also important.What's more, my 2x2080ti can only train the 1024x1024 images with apex&EfficientDet

", "votes": null, "replies": [ { "id": 953728, "author_name": "nitindatta", "author_url": "", "post_date": "08/01/2020 03:52:03", "content": "

Could you please share your scores \nThank you

", "votes": null, "replies": [] }, { "id": 954292, "author_name": "raininbox", "author_url": "", "post_date": "08/01/2020 15:51:12", "content": "

Singe model without any post trick(like pseudo label/TTA/OOF) can get LB 725😄

", "votes": null, "replies": [] }, { "id": 954302, "author_name": "nitindatta", "author_url": "", "post_date": "08/01/2020 15:59:31", "content": "

@raininbox ,\nThank you

", "votes": null, "replies": [] } ] }, { "id": 954647, "author_name": "chrisstan", "author_url": "", "post_date": "08/02/2020 00:30:05", "content": "

@raininbox Hi, which kind of kernel do you use for Inference?\nI am confused about whether I should change the image size in inference phase and how.\nI try on this, but I encounter Error of 'Notebook Exceeded Allowed Compute'.

", "votes": null, "replies": [ { "id": 954650, "author_name": "zekunn", "author_url": "", "post_date": "08/02/2020 00:42:23", "content": "

Mee,too

", "votes": null, "replies": [] }, { "id": 954765, "author_name": "chrisstan", "author_url": "", "post_date": "08/02/2020 04:38:37", "content": "

嘤嘤嘤

", "votes": null, "replies": [] }, { "id": 957433, "author_name": "wjfearth", "author_url": "", "post_date": "08/04/2020 09:48:10", "content": "

Commenting on the plt code in inference phase may be helpful

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "940490": "Hi,as we are known that Yolo v5 is out of the competition.But V5 did work.As the [discussion](https://www.kaggle.com/c/global-wheat-detection/discussion/167554) mention:\n\nFirst --> It is doing gradient accumulation by default to a batch size of 64\nsecond --> It is also giving mixed precision training(fp16) by default if we have apex installed\n\n\nSo let change it to EfficientDet.May be work :)) Hope the notebook will be helpful for someone. please feel free to upvote\n \ncode available: [Here](https://www.kaggle.com/raininbox/train-efficientdet-like-yolo-v5)", "940501": "Could u tell me the score it gets?", "940523": "😄 I can't tell you till now.Because the model still under train.If there any boost, I will public the result here", "940529": "Thanks for sharing!\n\nMay I ask that `loss = outputs['loss'] / self.accumulation_step` is necessary or not?\n\np.s. I saw that this is not in your kernel.", "940533": "I am trying some tricks ,too.😃 .May I ask what is your notebook's different with Alex's Notebook?", "940549": ":D I check the yolo v5's code without the \"/accumulate_step\".My code is copy from the V5.The size of the loss doesn't matter I guess.", "940552": "I highlight it wiht some comment to show how to use it :D apex and gradient accumulate", "940555": "OH,Thanks", "941293": "piantic If you are using Ross Wightman code, dont do that. because loss is getting averaged out by the below line.\n\n' loss, _, _ = self.model(images, boxes, labels)", "941301": "raininbox IMHO it does matter as the amount of gradient in the backward pass will get lowered if we average it out only in Ross Wightman code", "941391": "raininbox thanks..\nWhat is your train-validation strategy. does your cv score complies with lb ?", "941461": "Thanks! @kalyanpichuka", "941653": "That's true.Got it", "941659": "I haven't fininshed the train and submited it", "941691": "ok... do you perform bbox cleaning, is there any notebook that has published such dataset", "942198": "kalyanpichuka if there any articles about this issue?", "943144": "I tried your approach, my score decreased(", "943559": "I used your training kernal for batch size of 4, got LB score of 0.7407\nReally great! a definite upvote", "943570": "May i ask, how many epochs did you run for?", "943650": "Hello,did you use Single Model?", "943728": "Yes single model socre is 0.7407. I trained the model for 80 epochs.", "943831": "ahmmadrashid Is it with TTA or without ? \nAnd do you think changing scheduler will help ?", "944028": "I ddint change the scheduler. Inference is with TTA.\nRemove some augmentation from the training pipeline such as togray its not needed.\nAlso you can try running with SGD optimization instead of AdamW.", "944286": "That's useful for me, thanks for the experiment!", "944336": "ahmmadrashid did u use grad accumulation also ?", "945015": "From my experience, gradient accumulation are only LR optimization and fp16 will not improve Accuracy.\nI think yolov5 simply works because wheat labels are labeled by YOLO as well..", "945231": "They only do this for the “unlabeled sub datasets” though. Meaning only the `UTokyo_2` images in the test dataset are affected (<10%). Maybe this is one reason why YoloV5 works well but I don't think it's the whole story.", "945885": "raininbox you said you'd post an update here with the results but I haven't seen anything. How did it go?", "945887": "ahmmadrashid for the WBF, you used the Alex's default parameters (iou threshold and skip box threshold) or you made LB probing to select more appropriate ones?", "945949": ":D nice job.The max performance may get higher", "945952": "I am still adjusting the hyperparameter 😄", "945955": "https://www.kaggle.com/raininbox/check-clean-big-small-bboxes", "946283": "chihaoui I used OOF prediction to set the IoU threshold and skip box threshold. Also I used the clean bbox dataset for training. But with pseudo labellin the LB score boosts about 1.5%.", "946284": "jaideepvalani Yes I used gradient accumulation.", "946314": "ahmmadrashid Thanks for the reply. but what you do mean by \"clean bbox dataset for training\" exactly? you mean that you manually remove the noisy bboxes (too large, too small..) ?", "946331": "I used this kernel\nhttps://www.kaggle.com/raininbox/check-clean-big-small-bboxes", "946340": "Thanks", "947074": "Sorry to hear that, modify the hyperparameter may be work :D In my experiment it did work", "948101": "I`ll try) Now it is always overfits", "949447": "Can you share some observations?", "949509": "Did you use SGD with or without momentum?", "953594": ":D after a great number of experiments, i sure that gradient accumulation did work. and clean data is also important.What's more, my 2x2080ti can only train the 1024x1024 images with apex&EfficientDet", "953728": "Could you please share your scores \nThank you", "954063": "ahmmadrashid thanks.. what is best val loss did u obtain", "954292": "Singe model without any post trick(like pseudo label/TTA/OOF) can get LB 725😄", "954302": "raininbox ,\nThank you", "954647": "raininbox Hi, which kind of kernel do you use for Inference?\nI am confused about whether I should change the image size in inference phase and how.\nI try on [this](https://www.kaggle.com/gofarther/efficientdet-framework-tta-pl-etc), but I encounter Error of 'Notebook Exceeded Allowed Compute'.", "954650": "Mee,too", "954765": "嘤嘤嘤", "957433": "Commenting on the plt code in inference phase may be helpful", "1016302": "Hi, @raininbox.\nCould you please say.\nWhat optimization techniques did YOLO v5 use by default?", "1057928": "Is EfficientDet as fast as YOLO? If not then what FPS can EfficientDet provide?" }, "source": "meta" }