As the current first place of the public leaderboard, we hereby confirm that we will be using compliant solutions(LB < .76). Though not impressive, we think that being rules-abiding is the number one priority in this community. We also encourage you to use compliant solutions as there's absolutely no point to risk getting removed from the leaderboard and I am sure you won't be able to get away with any clandestine misconduct.
\n\nWe used this notebook for inference if you need it(for effdet).
\n\nRegarding the observations, we noticed the following:\n1. pseudo labelling with effdet(exact same configurations) gives you LB +- 0.003\n2. good training augmentations usually give you better LB(went from 0.73 to 0.75)\n3. fake indexing without oof evaluation(finding the optimal threshold) can give you a boost\n4. using Yolo v5 version 2.0 gives really great training loss(better than version 1.0) but poor LB(0.75 to 0.76), we think that this is because this dataset is labeled with version 1.0\n5. we can expect a shakeup but private LB is still not a good reflection as many teams will still use non-compliant solutions
\n\nWe are sorry for being repetitive. Thanks. Happy kaggling!
", "messages": [ { "id": 958182, "postDate": "2020-08-04T20:25:57.890Z", "content": "As the current first place of the public leaderboard, we hereby confirm that we will be using compliant solutions(LB < .76). Though not impressive, we think that being rules-abiding is the number one priority in this community. We also encourage you to use compliant solutions as there's absolutely no point to risk getting removed from the leaderboard and I am sure you won't be able to get away with any clandestine misconduct.
\n\nWe used this notebook for inference if you need it(for effdet).
\n\nRegarding the observations, we noticed the following:\n1. pseudo labelling with effdet(exact same configurations) gives you LB +- 0.003\n2. good training augmentations usually give you better LB(went from 0.73 to 0.75)\n3. fake indexing without oof evaluation(finding the optimal threshold) can give you a boost\n4. using Yolo v5 version 2.0 gives really great training loss(better than version 1.0) but poor LB(0.75 to 0.76), we think that this is because this dataset is labeled with version 1.0\n5. we can expect a shakeup but private LB is still not a good reflection as many teams will still use non-compliant solutions
\n\nWe are sorry for being repetitive. Thanks. Happy kaggling!
", "rawMarkdown": "As the current first place of the public leaderboard, we hereby confirm that we will be using compliant solutions(LB < .76). Though not impressive, we think that being rules-abiding is the number one priority in this community. We also encourage you to use compliant solutions as there's absolutely no point to risk getting removed from the leaderboard and I am sure you won't be able to get away with any clandestine misconduct. \n\nWe used this [notebook](https://www.kaggle.com/programzq/fork-of-normal-pl-eff-tta-wbf) for inference if you need it(for effdet).\n\nRegarding the observations, we noticed the following:\n1. pseudo labelling with effdet(exact same configurations) gives you LB +- 0.003\n2. good training augmentations usually give you better LB(went from 0.73 to 0.75)\n3. fake indexing without oof evaluation(finding the optimal threshold) can give you a boost\n4. using Yolo v5 version 2.0 gives really great training loss(better than version 1.0) but poor LB(0.75 to 0.76), we think that this is because this dataset is labeled with version 1.0\n5. we can expect a shakeup but private LB is still not a good reflection as many teams will still use non-compliant solutions\n\nWe are sorry for being repetitive. Thanks. Happy kaggling!", "votes": 14 }, { "id": 958300, "postDate": "2020-08-04T23:27:15.997Z", "content": "Great to see that the top team is also going to make their final submission compliant! My team shares many of the same effdet observations, and OOF gives us a bit of further score. I guess it's unfortunate that yolov5 version 1 was used to create the dataset, but it does explain a lot of the discrepancies we have seen; yolov4 and effdetd6 are supposed to perform better, but even putting many hours in, neither were even close. Best of luck to you and your team.
", "rawMarkdown": "Great to see that the top team is also going to make their final submission compliant! My team shares many of the same effdet observations, and OOF gives us a bit of further score. I guess it's unfortunate that yolov5 version 1 was used to create the dataset, but it does explain a lot of the discrepancies we have seen; yolov4 and effdetd6 are supposed to perform better, but even putting many hours in, neither were even close. Best of luck to you and your team.", "votes": 1, "replies": [ { "id": 958302, "postDate": "2020-08-04T23:28:55.437Z", "content": "Thanks. Good luck to you all.
", "rawMarkdown": "Thanks. Good luck to you all." }, { "id": 958315, "postDate": "2020-08-05T00:05:52.717Z", "rawMarkdown": "", "isDeleted": true }, { "id": 958322, "postDate": "2020-08-05T00:09:06.757Z", "content": "hahahah what a shakeup
", "rawMarkdown": "hahahah what a shakeup" } ] }, { "id": 958255, "postDate": "2020-08-04T21:57:59.110Z", "content": "Can you give brief idea about good training augmentation. Is adding more augmentations to @shonenkov 's kernel particularly helpful(it was not really helpful for me). \nCan you also give some information about number of epochs you trained the model
", "rawMarkdown": "Can you give brief idea about good training augmentation. Is adding more augmentations to @shonenkov 's kernel particularly helpful(it was not really helpful for me). \nCan you also give some information about number of epochs you trained the model", "replies": [ { "id": 958258, "postDate": "2020-08-04T22:00:39.683Z", "content": "Hi Sameer, we have found that using similar augmentations as yolo generate good results. We trained 120 epochs for D5.
", "rawMarkdown": "Hi Sameer, we have found that using similar augmentations as yolo generate good results. We trained 120 epochs for D5.", "votes": 2 }, { "id": 958271, "postDate": "2020-08-04T22:23:27.693Z", "content": "Do you use local environment for training? Training is especially slower on colab not to mention the timeouts
", "rawMarkdown": "Do you use local environment for training? Training is especially slower on colab not to mention the timeouts" }, { "id": 958273, "postDate": "2020-08-04T22:24:46.040Z", "content": "Yes we used local machines.
", "rawMarkdown": "Yes we used local machines.", "votes": 1 }, { "id": 958276, "postDate": "2020-08-04T22:29:33.687Z", "content": "Thank you for your insights in this competition!
", "rawMarkdown": "Thank you for your insights in this competition!" }, { "id": 958293, "postDate": "2020-08-04T23:07:35.363Z", "content": "> Hi Sameer, we have found that using similar augmentations as yolo generate good results. We trained 120 epochs for D5.
\n\nyes, I have the same finding
", "rawMarkdown": "> Hi Sameer, we have found that using similar augmentations as yolo generate good results. We trained 120 epochs for D5.\n\nyes, I have the same finding" } ] } ], "comments": [ { "id": 958300, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-04T23:27:15.997000", "content": "Great to see that the top team is also going to make their final submission compliant! My team shares many of the same effdet observations, and OOF gives us a bit of further score. I guess it's unfortunate that yolov5 version 1 was used to create the dataset, but it does explain a lot of the discrepancies we have seen; yolov4 and effdetd6 are supposed to perform better, but even putting many hours in, neither were even close. Best of luck to you and your team.
", "votes": 1, "replies": [ { "id": 958302, "author_name": "sin", "author_url": "", "post_date": "2020-08-04T23:28:55.437000", "content": "Thanks. Good luck to you all.
", "votes": 0, "replies": [] }, { "id": 958315, "author_name": "", "author_url": "", "post_date": "2020-08-05T00:05:52.717000", "content": "", "votes": 0, "replies": [] }, { "id": 958322, "author_name": "sin", "author_url": "", "post_date": "2020-08-05T00:09:06.757000", "content": "hahahah what a shakeup
", "votes": 0, "replies": [] } ] }, { "id": 958255, "author_name": "Sameer M", "author_url": "", "post_date": "2020-08-04T21:57:59.110000", "content": "Can you give brief idea about good training augmentation. Is adding more augmentations to @shonenkov 's kernel particularly helpful(it was not really helpful for me). \nCan you also give some information about number of epochs you trained the model
", "votes": 0, "replies": [ { "id": 958258, "author_name": "sin", "author_url": "", "post_date": "2020-08-04T22:00:39.683000", "content": "Hi Sameer, we have found that using similar augmentations as yolo generate good results. We trained 120 epochs for D5.
", "votes": 2, "replies": [] }, { "id": 958271, "author_name": "Sameer M", "author_url": "", "post_date": "2020-08-04T22:23:27.693000", "content": "Do you use local environment for training? Training is especially slower on colab not to mention the timeouts
", "votes": 0, "replies": [] }, { "id": 958273, "author_name": "sin", "author_url": "", "post_date": "2020-08-04T22:24:46.040000", "content": "Yes we used local machines.
", "votes": 1, "replies": [] }, { "id": 958276, "author_name": "Sameer M", "author_url": "", "post_date": "2020-08-04T22:29:33.687000", "content": "Thank you for your insights in this competition!
", "votes": 0, "replies": [] }, { "id": 958293, "author_name": "Jacob", "author_url": "", "post_date": "2020-08-04T23:07:35.363000", "content": "> Hi Sameer, we have found that using similar augmentations as yolo generate good results. We trained 120 epochs for D5.
\n\nyes, I have the same finding
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "958182": "As the current first place of the public leaderboard, we hereby confirm that we will be using compliant solutions(LB < .76). Though not impressive, we think that being rules-abiding is the number one priority in this community. We also encourage you to use compliant solutions as there's absolutely no point to risk getting removed from the leaderboard and I am sure you won't be able to get away with any clandestine misconduct. \n\nWe used this [notebook](https://www.kaggle.com/programzq/fork-of-normal-pl-eff-tta-wbf) for inference if you need it(for effdet).\n\nRegarding the observations, we noticed the following:\n1. pseudo labelling with effdet(exact same configurations) gives you LB +- 0.003\n2. good training augmentations usually give you better LB(went from 0.73 to 0.75)\n3. fake indexing without oof evaluation(finding the optimal threshold) can give you a boost\n4. using Yolo v5 version 2.0 gives really great training loss(better than version 1.0) but poor LB(0.75 to 0.76), we think that this is because this dataset is labeled with version 1.0\n5. we can expect a shakeup but private LB is still not a good reflection as many teams will still use non-compliant solutions\n\nWe are sorry for being repetitive. Thanks. Happy kaggling!", "958300": "Great to see that the top team is also going to make their final submission compliant! My team shares many of the same effdet observations, and OOF gives us a bit of further score. I guess it's unfortunate that yolov5 version 1 was used to create the dataset, but it does explain a lot of the discrepancies we have seen; yolov4 and effdetd6 are supposed to perform better, but even putting many hours in, neither were even close. Best of luck to you and your team.", "958255": "Can you give brief idea about good training augmentation. Is adding more augmentations to @shonenkov 's kernel particularly helpful(it was not really helpful for me). \nCan you also give some information about number of epochs you trained the model" } }