![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F5007869%2F41252aa351f1a4b9bd6120cf0b6a39f9%2Fdefault.png?generation=1596704580723748&alt=media\")
Yolov5 was forbidden in this competition. However, I and my team have spent a lot of time working on Yolov5. We've chosen EffDet for the final submission but the fact that we spent a lot of time on fine-tuning Yolov5 remains a fact. To make some use of it I share our results with you. I hope at least someone of you will find it useful just cause the Yolov5 itself is a very strong Network. So here are the results of our fine-tuning.
\n\nAt the screenshot below you can see the default hyperparameters:\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F5007869%2F3c921fd85db3d157cb08a428471737d2%2FvGcSQlt6aws.jpg?generation=1596706903479908&alt=media\")
\nWe changed weight decay as well according to the formula but it didn't help. \nBy the way, we were also displaying the learning rate after each epoch, and looking at this we've detected that the learning rate is changing differently with different batch sizes (with batch size = 8 it was changing much slower). However, there is no batch size in the scheduler's formula:\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F5007869%2F606ae57d923c08b8cdc81b3bee07b533%2Flx36236pdx0.jpg?generation=1596707275509623&alt=media\")
\nIf you know what's the reason for this, please share it in the comments. I'll update this topic if I see the answer.![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F5007869%2Fe678e3118b6f61b7b11746ecab1e4cf6%2F1.jpg?generation=1596707775165156&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F5007869%2F5ede18bb8cc1310dae3b7e65e118da07%2F3.jpg?generation=1596707775391485&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F5007869%2Faeb42b132bbc4bdc27dd22c4df1e0167%2F2.jpg?generation=1596707775959159&alt=media\")
\nIf you know good parameters for augmentations, feel free to share it in the comments. I'll update this topic and will mention to you if you tell something useful in the comments about that.That's all. Never trust default parameters, always try to find out the better ones.
", "messages": [ { "id": 960316, "postDate": "2020-08-06T09:58:43.623Z", "content": "Yolov5 was forbidden in this competition. However, I and my team have spent a lot of time working on Yolov5. We've chosen EffDet for the final submission but the fact that we spent a lot of time on fine-tuning Yolov5 remains a fact. To make some use of it I share our results with you. I hope at least someone of you will find it useful just cause the Yolov5 itself is a very strong Network. So here are the results of our fine-tuning.
\n\nAt the screenshot below you can see the default hyperparameters:\n
\nWe changed weight decay as well according to the formula but it didn't help. \nBy the way, we were also displaying the learning rate after each epoch, and looking at this we've detected that the learning rate is changing differently with different batch sizes (with batch size = 8 it was changing much slower). However, there is no batch size in the scheduler's formula:\n\nIf you know what's the reason for this, please share it in the comments. I'll update this topic if I see the answer.\n\n\nIf you know good parameters for augmentations, feel free to share it in the comments. I'll update this topic and will mention to you if you tell something useful in the comments about that.That's all. Never trust default parameters, always try to find out the better ones.
", "rawMarkdown": "Yolov5 was forbidden in this competition. However, I and my team have spent a lot of time working on Yolov5. We've chosen EffDet for the final submission but the fact that we spent a lot of time on fine-tuning Yolov5 remains a fact. To make some use of it I share our results with you. I hope at least someone of you will find it useful just cause the Yolov5 itself is a very strong Network. So here are the results of our fine-tuning.\n\nAt the screenshot below you can see the default hyperparameters:\n\n\n* LR and Optimizer \nI think we should consider LR and Optimizer together.\nWe have tried LR = 0.001, 0.003, 0.005, 0.01 (Default) with SGD (Default) and Adam.\nAdam always was worse than SGD. 0.001 and 0.005 learning rates were pretty weak. Default set up with SGD and lr=0.01 was a strong one. But during fine-tuning, we found out the better set up which is SGD with lr=0.003. This one boosted our score a little bit.\n* Weight decay\nIt seemed small to us at once so we decided to increase it up to 0.001. It was a really good decision. It boosted our score on 0.02 approximately. We tried other weight decays as well but didn't get a better score. So in the future, we were always using only this one.\n* Loss and its parameters\nWe tried to find a better loss function for this task. Expect of BCELoss we used Focal Loss but it didn't show us better results. So we decided to fine-tune BCELoss, we changed smooth-param to 0.01 but it didn't work.\n* Batch size\nWe all get used to thinking that a higher batch size usually equals better results. However, it seems like Yolov5 is an exception. Batch size 2 was much better than batch size 8size 8. We tried to find out the cause of that. First, we found out that weight decay has been changed according to batch size:\n\nWe changed weight decay as well according to the formula but it didn't help. \nBy the way, we were also displaying the learning rate after each epoch, and looking at this we've detected that the learning rate is changing differently with different batch sizes (with batch size = 8 it was changing much slower). However, there is no batch size in the scheduler's formula:\n\nIf you know what's the reason for this, please share it in the comments. I'll update this topic if I see the answer.\n* Schedulers\nWe've tried different schedulers as well: Cosine, Step, Cyclic. We were fine-tuning their parameters but finally, we've realized that the default scheduler (LamdaLR, the screenshot above) is the best in this task.\n* Augmentations\nWe had big hopes about Augmentations profit, but we got a worse result using them. I don't believe that augmentations are useless here, likely, we have just chosen wrong parameters for them. Here are some of them:\n\n\n\nIf you know good parameters for augmentations, feel free to share it in the comments. I'll update this topic and will mention to you if you tell something useful in the comments about that.\n\nThat's all. Never trust default parameters, always try to find out the better ones. ", "votes": 10 }, { "id": 1280009, "postDate": "2021-04-21T13:18:53.433Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1280009, "author_name": "", "author_url": "", "post_date": "2021-04-21T13:18:53.433000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "960316": "Yolov5 was forbidden in this competition. However, I and my team have spent a lot of time working on Yolov5. We've chosen EffDet for the final submission but the fact that we spent a lot of time on fine-tuning Yolov5 remains a fact. To make some use of it I share our results with you. I hope at least someone of you will find it useful just cause the Yolov5 itself is a very strong Network. So here are the results of our fine-tuning.\n\nAt the screenshot below you can see the default hyperparameters:\n\n\n* LR and Optimizer \nI think we should consider LR and Optimizer together.\nWe have tried LR = 0.001, 0.003, 0.005, 0.01 (Default) with SGD (Default) and Adam.\nAdam always was worse than SGD. 0.001 and 0.005 learning rates were pretty weak. Default set up with SGD and lr=0.01 was a strong one. But during fine-tuning, we found out the better set up which is SGD with lr=0.003. This one boosted our score a little bit.\n* Weight decay\nIt seemed small to us at once so we decided to increase it up to 0.001. It was a really good decision. It boosted our score on 0.02 approximately. We tried other weight decays as well but didn't get a better score. So in the future, we were always using only this one.\n* Loss and its parameters\nWe tried to find a better loss function for this task. Expect of BCELoss we used Focal Loss but it didn't show us better results. So we decided to fine-tune BCELoss, we changed smooth-param to 0.01 but it didn't work.\n* Batch size\nWe all get used to thinking that a higher batch size usually equals better results. However, it seems like Yolov5 is an exception. Batch size 2 was much better than batch size 8size 8. We tried to find out the cause of that. First, we found out that weight decay has been changed according to batch size:\n\nWe changed weight decay as well according to the formula but it didn't help. \nBy the way, we were also displaying the learning rate after each epoch, and looking at this we've detected that the learning rate is changing differently with different batch sizes (with batch size = 8 it was changing much slower). However, there is no batch size in the scheduler's formula:\n\nIf you know what's the reason for this, please share it in the comments. I'll update this topic if I see the answer.\n* Schedulers\nWe've tried different schedulers as well: Cosine, Step, Cyclic. We were fine-tuning their parameters but finally, we've realized that the default scheduler (LamdaLR, the screenshot above) is the best in this task.\n* Augmentations\nWe had big hopes about Augmentations profit, but we got a worse result using them. I don't believe that augmentations are useless here, likely, we have just chosen wrong parameters for them. Here are some of them:\n\n\n\nIf you know good parameters for augmentations, feel free to share it in the comments. I'll update this topic and will mention to you if you tell something useful in the comments about that.\n\nThat's all. Never trust default parameters, always try to find out the better ones. ", "1280009": "" } }