![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2030621%2F9ba1ff9cbd68b3ece995190c7e63a149%2FScreen%20Shot%202020-08-10%20at%2023.30.41.png?generation=1597097222510934&alt=media\")
Hi all! Thank you for all of your great work on this competition.
\nAs Ian noted, we are in the process of updating the leaderboard using the updated private test set. Your selected submissions are being re-run against the updated set; private scores will change as this takes place. We are also performing our standard leaderboard finalization. This will take some time and we will notify you when the process is complete.
\nPlease let us know if you have questions!
", "messages": [ { "id": 964578, "postDate": "2020-08-10T02:58:35.463Z", "content": "Hi, @philculliton.\nI'm happy to hear that private lb will be updated again.
\n\nMy question is two:\n1) May I ask that will yolov5(or other package of license issues) be removed in re-run private lb?
\n\n2) How long will it take approximately to complete all process?
\n\nI would be very happy if you respond and also other competitors in this competition would be happy.\nThanks!
", "rawMarkdown": "Hi, @philculliton.\nI'm happy to hear that private lb will be updated again.\n\nMy question is two:\n1) May I ask that will yolov5(or other package of license issues) be removed in re-run private lb?\n\n2) How long will it take approximately to complete all process?\n\nI would be very happy if you respond and also other competitors in this competition would be happy.\nThanks!\n", "votes": 23 }, { "id": 976556, "postDate": "2020-08-19T00:22:14.903Z", "content": "This is by far the worst managed competition I have ever had the chance to participate in. Nothing, nothing gets even close to this. Promises being repeatedly broken and the unsatisfactory explanation as to why this is even happening !!!
", "rawMarkdown": "This is by far the worst managed competition I have ever had the chance to participate in. Nothing, nothing gets even close to this. Promises being repeatedly broken and the unsatisfactory explanation as to why this is even happening !!! ", "votes": 24, "replies": [ { "id": 976567, "postDate": "2020-08-19T00:30:50.260Z", "content": "The worst part is that they don't seem to care about our comments at all.
", "rawMarkdown": "The worst part is that they don't seem to care about our comments at all. ", "votes": 8 }, { "id": 977410, "postDate": "2020-08-19T13:08:50.007Z", "content": "Or Just may'be they do care .. but they need all there time and attention to finalize the LB.
\nI dont like the waiting either but looking at the LB changes it is worth it.
", "rawMarkdown": "Or Just may'be they do care .. but they need all there time and attention to finalize the LB.\n\nI dont like the waiting either but looking at the LB changes it is worth it.\n", "votes": 5 }, { "id": 977501, "postDate": "2020-08-19T14:18:13.203Z", "content": "@rsmits, my sentiments exactly.
", "rawMarkdown": "@rsmits, my sentiments exactly.", "votes": 1 } ] }, { "id": 974322, "postDate": "2020-08-17T22:12:23.720Z", "content": "Hi all.
\nWe had some additional checks we needed to do today, and ran out of staff time to do the final database update (which is not a small or risk-less thing). We will do so tomorrow and finalize the LB. I'm sorry for how long this is taking. The number of moving parts are considerably higher than a normal competition close, and we're trying to do the best job we can.
\nRe: questions - yes, we will be opening late submissions following the finalization of the LB.
\nThanks again, all. We really appreciate your patience.
", "rawMarkdown": "Hi all.\n\nWe had some additional checks we needed to do today, and ran out of staff time to do the final database update (which is not a small or risk-less thing). We will do so tomorrow and finalize the LB. I'm sorry for how long this is taking. The number of moving parts are considerably higher than a normal competition close, and we're trying to do the best job we can.\n\nRe: questions - yes, we will be opening late submissions following the finalization of the LB.\n\nThanks again, all. We really appreciate your patience.", "votes": 25, "replies": [ { "id": 975547, "postDate": "2020-08-18T10:51:05.323Z", "content": "And please release the complete dataset.
", "rawMarkdown": "And please release the complete dataset. ", "votes": -4 }, { "id": 977920, "postDate": "2020-08-19T19:48:20.420Z", "content": "You said late submission will be enabled, but the competition ended it's still not on.
", "rawMarkdown": "You said late submission will be enabled, but the competition ended it's still not on." } ] }, { "id": 964463, "postDate": "2020-08-09T22:54:40.613Z", "content": "Hi all! Thank you for all of your great work on this competition.
\nAs Ian noted, we are in the process of updating the leaderboard using the updated private test set. Your selected submissions are being re-run against the updated set; private scores will change as this takes place. We are also performing our standard leaderboard finalization. This will take some time and we will notify you when the process is complete.
\nPlease let us know if you have questions!
", "rawMarkdown": "Hi all! Thank you for all of your great work on this competition.\n\nAs Ian noted, we are in the process of updating the leaderboard using the updated private test set. Your selected submissions are being re-run against the updated set; private scores *will* change as this takes place. We are also performing our standard leaderboard finalization. This will take some time and we will notify you when the process is complete.\n\nPlease let us know if you have questions!", "votes": 30 }, { "id": 977892, "postDate": "2020-08-19T19:29:44.780Z", "content": "Hi all!
\nAs some of you have noted, the leaderboard has changed significantly. We have done our best to remove submissions that contained YOLOv5. We have also performed our normal post-competition process that has removed some other teams.
\nIf you believe your submission (or team) was incorrectly removed, please contact kaggle.com/compliance - this is our normal process for resolving disputes about team removals, submission invalidations, etc. I appreciate the Kagglers who have reached out to me directly, but if you follow the official route we can ensure your case is looked at properly and in a timely fashion.
\nI am, once again, sorry that this took so long. It was detailed work that required a great deal of time from several of us. We appreciate your patience and the support you've shown. Finally, congratulations to the winners - I know the hosts are excited to examine your solutions!
", "rawMarkdown": "Hi all!\n\nAs some of you have noted, the leaderboard has changed significantly. We have done our best to remove submissions that contained YOLOv5. We have also performed our normal post-competition process that has removed some other teams.\n\nIf you believe your submission (or team) was incorrectly removed, please contact [kaggle.com/compliance](https://www.kaggle.com/compliance) - this is our normal process for resolving disputes about team removals, submission invalidations, etc. I appreciate the Kagglers who have reached out to me directly, but if you follow the official route we can ensure your case is looked at properly and in a timely fashion.\n\nI am, once again, sorry that this took so long. It was detailed work that required a great deal of time from several of us. We appreciate your patience and the support you've shown. Finally, congratulations to the winners - I know the hosts are excited to examine your solutions!", "votes": 17, "replies": [ { "id": 977899, "postDate": "2020-08-19T19:34:33.510Z", "content": "Thanks, great job! Where draws the line for medals now? Is it calculated from the number of teams before the removal of the teams, like all other competitions?
", "rawMarkdown": "Thanks, great job! Where draws the line for medals now? Is it calculated from the number of teams before the removal of the teams, like all other competitions?", "votes": 1 }, { "id": 977905, "postDate": "2020-08-19T19:37:59.653Z", "content": "Hi Phil, thank you for taking the time to review the leaderboard submissions and make your best effort at removing non-compliant submissions. However, I am very disappointed to see that some YoloV5 submissions are still on the leaderboard… (for example: https://www.kaggle.com/naimur978/wheat-head-detection-yolov5-pseudo-labelling-oof)
\nI hope that the compliance team will continue to remove these submissions over the coming weeks.
", "rawMarkdown": "Hi Phil, thank you for taking the time to review the leaderboard submissions and make your best effort at removing non-compliant submissions. However, I am very disappointed to see that some YoloV5 submissions are still on the leaderboard... (for example: https://www.kaggle.com/naimur978/wheat-head-detection-yolov5-pseudo-labelling-oof)\n\nI hope that the compliance team will continue to remove these submissions over the coming weeks.", "votes": 1 }, { "id": 977934, "postDate": "2020-08-19T19:58:06.740Z", "content": "Thanks @matthewmasters - that was one of a small handful (~20) that was re-run last night and it slipped back into the general pool. It's been removed.
\nWe will continue to remove teams that broke the rules if they become apparent - the host will be largely dealing with that task now.
", "rawMarkdown": "Thanks @matthewmasters - that was one of a small handful (~20) that was re-run last night and it slipped back into the general pool. It's been removed.\n\nWe will continue to remove teams that broke the rules if they become apparent - the host will be largely dealing with that task now.", "votes": 3 }, { "id": 977935, "postDate": "2020-08-19T19:58:34.343Z", "content": "@kirderf That's correct! We will be calculating medals based on the original number of teams.
", "rawMarkdown": "@kirderf That's correct! We will be calculating medals based on the original number of teams.", "votes": 2 }, { "id": 977938, "postDate": "2020-08-19T20:03:31.920Z", "rawMarkdown": "", "isDeleted": true }, { "id": 977943, "postDate": "2020-08-19T20:06:25.027Z", "content": "@philculliton sounds good. Is it possible to tell right now what the lines for silver and bronze are.
", "rawMarkdown": "@philculliton sounds good. Is it possible to tell right now what the lines for silver and bronze are.", "votes": 1 }, { "id": 977953, "postDate": "2020-08-19T20:14:01.910Z", "rawMarkdown": "", "isDeleted": true }, { "id": 977960, "postDate": "2020-08-19T20:19:30.667Z", "content": "Thanks @philculliton for results. You guys made it a fair competition. I sincerely appreciate Kaggle Team efforts in doing so. I must say this…KAGGLE TEAM ARE MEN OF ACTION…you guys spoke less..but at the end you guys did what has to be done..much appreciate
", "rawMarkdown": "Thanks @philculliton for results. You guys made it a fair competition. I sincerely appreciate Kaggle Team efforts in doing so. I must say this...KAGGLE TEAM ARE MEN OF ACTION...you guys spoke less..but at the end you guys did what has to be done..much appreciate ", "votes": -1 }, { "id": 977964, "postDate": "2020-08-19T20:25:07.320Z", "content": "@stanleyjzheng OK so there were 2,245 teams before the removal?
", "rawMarkdown": "@stanleyjzheng OK so there were 2,245 teams before the removal?", "votes": 1 }, { "id": 977967, "postDate": "2020-08-19T20:27:20.583Z", "rawMarkdown": "", "isDeleted": true }, { "id": 977990, "postDate": "2020-08-19T20:48:26.137Z", "content": "Then I lost a medal on the finish line again within 10 places to the level, crazy bad luck, is the fifth competition in a short time where I lose within 10 places to the medal level.
\nLucky that one gets some points for the work in any case, rank 169 in 10 months always something.
\nJust to fight on and get better until next time!
Thank you all for your contributions, Kaggle for the tremendous work with this competition and congratulations to all the winners!
", "rawMarkdown": "Then I lost a medal on the finish line again within 10 places to the level, crazy bad luck, is the fifth competition in a short time where I lose within 10 places to the medal level.\nLucky that one gets some points for the work in any case, rank 169 in 10 months always something.\nJust to fight on and get better until next time!\n\nThank you all for your contributions, Kaggle for the tremendous work with this competition and congratulations to all the winners!", "votes": 1 }, { "id": 977992, "postDate": "2020-08-19T20:50:43.227Z", "content": "I'm so sorry Kirderf, that's some crazy bad luck; Top 200 worldwide is still something to be extremely proud of. Keep trying, your luck is bound to turn around sometime! The leaderboard is still fluid, so it's possible a few more submissions will be removed to push you into the medal positions.
", "rawMarkdown": "I'm so sorry Kirderf, that's some crazy bad luck; Top 200 worldwide is still something to be extremely proud of. Keep trying, your luck is bound to turn around sometime! The leaderboard is still fluid, so it's possible a few more submissions will be removed to push you into the medal positions.", "votes": 1 }, { "id": 977993, "postDate": "2020-08-19T20:51:34.447Z", "content": "We have - thanks to an extremely helpful report - identified another small set of kernels to invalidate for non-Yolo reasons - basically the last of the bookkeeping from the long process. We will be doing so shortly.
\nThanks to everyone for your feedback. Please let us know if you spot anomalies.
", "rawMarkdown": "We have - thanks to an extremely helpful report - identified another small set of kernels to invalidate for non-Yolo reasons - basically the last of the bookkeeping from the long process. We will be doing so shortly.\n\nThanks to everyone for your feedback. Please let us know if you spot anomalies.", "votes": 2 }, { "id": 977997, "postDate": "2020-08-19T20:57:25.197Z", "content": "Thanks @philculliton for the update and a job well done on removing cheaters. Are the points awarded yet?
", "rawMarkdown": "Thanks @philculliton for the update and a job well done on removing cheaters. Are the points awarded yet?" }, { "id": 978040, "postDate": "2020-08-19T22:01:53.723Z", "content": "A miracle, signed @philculliton, I'm now on the right side of the medal zone! hallelujah 👍
", "rawMarkdown": "A miracle, signed @philculliton, I'm now on the right side of the medal zone! hallelujah 👍", "votes": 4 } ] }, { "id": 970878, "postDate": "2020-08-14T23:40:59.777Z", "content": "Hi all!
\nWe're still updating the leaderboard. As you might imagine, sorting this all out has been an involved process. We appreciate your patience as we continue to finalize things. We are shooting to have everything done and finalized on Monday.
", "rawMarkdown": "Hi all!\n\nWe're still updating the leaderboard. As you might imagine, sorting this all out has been an involved process. We appreciate your patience as we continue to finalize things. We are shooting to have everything done and finalized on Monday.", "votes": 16, "replies": [ { "id": 970884, "postDate": "2020-08-15T00:16:11.103Z", "content": "Thanks a ton for the status update. I'm excited to see the final leaderboard!
", "rawMarkdown": "Thanks a ton for the status update. I'm excited to see the final leaderboard!", "votes": 3 }, { "id": 971400, "postDate": "2020-08-15T13:33:40.590Z", "rawMarkdown": "", "votes": -1, "isDeleted": true }, { "id": 971416, "postDate": "2020-08-15T14:01:44.387Z", "content": "@philculliton till now i think we have not got an ans to whether YOlov5 sub be cancelled during this re run (IRRESPECTIVE OF WHETHER THOSE SUB ARE IN MONEY OR NOT) ,if people have selected it for final sub
", "rawMarkdown": "@philculliton till now i think we have not got an ans to whether YOlov5 sub be cancelled during this re run (**IRRESPECTIVE OF WHETHER THOSE SUB ARE IN MONEY OR NOT**) ,if people have selected it for final sub", "votes": 1 }, { "id": 971446, "postDate": "2020-08-15T14:30:04.997Z", "content": "@kshitijpatil09 I am fairly certain there will be late submissions enabled following the finalized leaderboard. Some people were doing papers, and late submission was turned on for a short period following the first end of the competition.
", "rawMarkdown": "@kshitijpatil09 I am fairly certain there will be late submissions enabled following the finalized leaderboard. Some people were doing papers, and late submission was turned on for a short period following the first end of the competition.", "votes": 1 }, { "id": 972049, "postDate": "2020-08-16T07:35:06.577Z", "content": "Thanks for the update! Just a quick suggestion that if all the kernels are rerun, it is more reasonable to let us re-select our submission entries. This is because there are random factors inside pseudo-label training, and we usually select the final entries based on our public scores, right?
", "rawMarkdown": "Thanks for the update! Just a quick suggestion that if all the kernels are rerun, it is more reasonable to let us re-select our submission entries. This is because there are random factors inside pseudo-label training, and we usually select the final entries based on our public scores, right?" } ] }, { "id": 965825, "postDate": "2020-08-10T22:09:28.223Z", "content": "i am confused ^^
\n
Kaggle knows how to make me smile…
", "rawMarkdown": "i am confused ^^\n\n\n\n*Kaggle knows how to make me smile...*", "votes": 11, "replies": [ { "id": 965885, "postDate": "2020-08-10T23:58:40.367Z", "content": "Another one to add to the collection! Great editing, gave me a good laugh.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F5221270%2F77c1be80605f0ed5d9c14bd855e65c15%2FScreenshot%20from%202020-08-10%2017-57-54.png?generation=1597103902218311&alt=media\")
This competition is one of the worst organised competitions on kaggle.
", "rawMarkdown": "This competition is one of the worst organised competitions on kaggle.\n", "votes": 13, "replies": [ { "id": 975556, "postDate": "2020-08-18T10:56:33.583Z", "content": "Also it's quite unfair to kaggler who spends months to optimize his/her solution despite so messy dataset. Now, at the end of the competition organizers decided to completely change the test dataset which will mess up a lot of people's solutions.
", "rawMarkdown": "Also it's quite unfair to kaggler who spends months to optimize his/her solution despite so messy dataset. Now, at the end of the competition organizers decided to completely change the test dataset which will mess up a lot of people's solutions. ", "votes": 9 }, { "id": 975724, "postDate": "2020-08-18T12:47:39.050Z", "content": "One can't disagree with you. I like the Kaggle platform, but the organization of this competition is not up to par.
", "rawMarkdown": "One can't disagree with you. I like the Kaggle platform, but the organization of this competition is not up to par.", "votes": 2 } ] }, { "id": 976548, "postDate": "2020-08-19T00:08:23.587Z", "content": "\n\n@tanulsingh077, I think the right answer is yes all teams with Yolov5 submission chosen as one of their final 2 subs will be removed from the LB. I do not understand why people are confused. It is not hard to write a simple script that goes through 4000+ submissions to check for Yolov5 models hence I believe Kaggle will do it. Heck if this were posted as Kaggle challenge, many teams would score 99.99 accuracy within the 1st week. Just my 2 cents :-)
\n
I posted the above 24 days ago in the Leaderboard of Compliant Submissions where people were still asking about YOLOV5 submission. The thread I responded to was asking specific questions one of which was
\n\n\nIf a team uses yolo will it be removed from the lb?
\n
This is after the question have been answered by the Kaggle team be it indirectly for example by @philculliton stating
\n\n\nyou cannot win prizes if you use YOLOv5.
\n
I have seen the question answered by others as well but people kept asking over and over again in the discussion section.
\nSo kudos to the Kaggle team for doing what I thought they would because it simply makes perfect sense. I know LB is not finished updating but it is now down to 1815 teams from 2270. Thanks @philculliton and team.
", "rawMarkdown": "> @tanulsingh077, I think the right answer is yes all teams with Yolov5 submission chosen as one of their final 2 subs will be removed from the LB. I do not understand why people are confused. It is not hard to write a simple script that goes through 4000+ submissions to check for Yolov5 models hence I believe Kaggle will do it. Heck if this were posted as Kaggle challenge, many teams would score 99.99 accuracy within the 1st week. Just my 2 cents :-)\n\nI posted the above 24 days ago in the [Leaderboard of Compliant Submissions](https://www.kaggle.com/c/global-wheat-detection/discussion/166793) where people were still asking about YOLOV5 submission. The thread I responded to was asking specific questions one of which was\n> If a team uses yolo will it be removed from the lb?\n\nThis is after the question have been answered by the Kaggle team be it indirectly for example by @philculliton stating \n\n> you cannot win prizes if you use YOLOv5. \n\nI have seen the question answered by others as well but people kept asking over and over again in the discussion section.\n\nSo kudos to the Kaggle team for doing what I thought they would because it simply makes perfect sense. I know LB is not finished updating but it is now down to 1815 teams from 2270. Thanks @philculliton and team.", "votes": 10 }, { "id": 970659, "postDate": "2020-08-14T16:40:10.540Z", "content": "Only 7 hours to go! Let's hope all the Yolo submissions are removed!
", "rawMarkdown": "Only 7 hours to go! Let's hope all the Yolo submissions are removed!", "votes": 6, "replies": [ { "id": 970747, "postDate": "2020-08-14T18:19:32.910Z", "content": "I put my hope in that direction
", "rawMarkdown": "I put my hope in that direction" }, { "id": 970814, "postDate": "2020-08-14T20:37:30.557Z", "content": "One more day added!
", "rawMarkdown": "One more day added!", "votes": 2 }, { "id": 970824, "postDate": "2020-08-14T20:59:01.960Z", "content": "Its ridiculous.
", "rawMarkdown": "Its ridiculous. ", "votes": 5 }, { "id": 970917, "postDate": "2020-08-15T02:08:25.373Z", "content": "@kaushal2896 you wanted 7 but got 72, more than you wished for! 😃
", "rawMarkdown": "@kaushal2896 you wanted 7 but got 72, more than you wished for! 😃", "votes": 1 }, { "id": 971398, "postDate": "2020-08-15T13:32:46.133Z", "content": "I guess there will be another one
\n![\"haha\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F724759%2Fd4029696bf2bb2812d96df0fc26c4c00%2FScreen%20Shot%202020-08-15%20at%209.31.52%20AM.png?generation=1597498356669873&alt=media\")
@creatrol you were right lol
", "rawMarkdown": "@creatrol you were right lol" } ] }, { "id": 978562, "postDate": "2020-08-20T08:58:43.550Z", "content": "Hey @philculliton,
\nMe and my teammate appreciate the effort put into removing the Yolov5 submissions.
Having said that, we cannot find ourselves on the leaderboard. Both of our submissions seem to have been invalidated with a failed final private rerun error message. We have used the MIT-compliant EfficientDet for both submissions with different ensembling methods. As of yesterday (19/08/2020), we could find our submissions on the leaderboard but they have been removed since then. We are also sure that the runtime of our models is <6h because we have made multiple submissions (80+) before our final submission.
We'd appreciate your help in the matter because like many other participants here, we have put in multiple months of effort and worked only on compliant solutions.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F863665%2Fc6010747711e2146b1017e1200f08e3a%2FScreenshot%202020-08-20%20at%2010.49.25.png?generation=1597913997098285&alt=media\")
Hi, did you make any modification to your dataset or notebook after the original deadline?
", "rawMarkdown": "Hi, did you make any modification to your dataset or notebook after the original deadline?" }, { "id": 979243, "postDate": "2020-08-20T17:56:28.733Z", "content": "Best thing to do is contact the compliance team here: https://www.kaggle.com/compliance
", "rawMarkdown": "Best thing to do is contact the compliance team here: https://www.kaggle.com/compliance" }, { "id": 986186, "postDate": "2020-08-26T09:20:33.483Z", "content": "@qingyuanwang no modifications were made
\n@matthewmasters we already contacted them on 20/08/2020 with no response so far
I have the similar situation to yours and I got a very confusing response. see here
", "rawMarkdown": "I have the similar situation to yours and I got a very confusing response. see [here](https://www.kaggle.com/c/global-wheat-detection/discussion/176066)" } ] }, { "id": 976495, "postDate": "2020-08-18T22:29:59.667Z", "content": "A day to go, again and again!
", "rawMarkdown": "A day to go, again and again!", "votes": 5, "replies": [ { "id": 976498, "postDate": "2020-08-18T22:37:13.120Z", "content": "As they are actively removing the yolo submissions(already removed majority of them), I think it is okay if they need one or two days more. Huge props to them for making the leaderboard compliant.
", "rawMarkdown": "As they are actively removing the yolo submissions(already removed majority of them), I think it is okay if they need one or two days more. Huge props to them for making the leaderboard compliant.", "votes": 8 }, { "id": 976500, "postDate": "2020-08-18T22:41:54.387Z", "content": "I really appreciate their hardwork to find & delete non-compliant submission. We should be all grateful!
", "rawMarkdown": "I really appreciate their hardwork to find & delete non-compliant submission. We should be all grateful! ", "votes": 6 }, { "id": 976503, "postDate": "2020-08-18T22:48:56.827Z", "content": "Yes they do an honorable and important job, and thoroughly as it seems, many teams have been removed. It must take its time, rules are rules, and will be a standard to bring before the next competition, a good signal value.
\nImpatience is the only thing one need to work with while they do thier work, in contrast it should be easier to deal with impatience, so be it.
Feel like Okarin from Stein's Gate, looking for the best PB score..
\nBut amazed by the kaggle teams hard work on keeping the competition fair. Thanks.
The host is great work for keeping fairness.(remove uncompliant solution).
\nI'm looking forward to finalizing Private LB.
Another one bites the dust, and now @tereka is #1 :)
", "rawMarkdown": "Another one bites the dust, and now @tereka is #1 :)", "votes": 2 }, { "id": 978060, "postDate": "2020-08-19T22:43:38.550Z", "content": "@alexanderliao Congrat getting solo gold and become grand master!
", "rawMarkdown": "@alexanderliao Congrat getting solo gold and become grand master!", "votes": 1 } ] }, { "id": 965315, "postDate": "2020-08-10T14:39:53.167Z", "content": "Rerun process has some problem because after rerun both public and private score of my submission changed. If only private test dataset is updated, it shouldn't do anything to public score.
\n\nSo I guess that the host simply reran the whole kernel so that kernel with PL will create a brand new checkpoint and result in a different public score. Another shakeup is coming.
", "rawMarkdown": "Rerun process has some problem because after rerun both public and private score of my submission changed. If only private test dataset is updated, it shouldn't do anything to public score.\n\nSo I guess that the host simply reran the whole kernel so that kernel with PL will create a brand new checkpoint and result in a different public score. Another shakeup is coming.", "votes": 5 }, { "id": 965234, "postDate": "2020-08-10T13:39:43.903Z", "content": "I can't understand what is going on
", "rawMarkdown": "I can't understand what is going on", "votes": 5, "replies": [ { "id": 965247, "postDate": "2020-08-10T13:48:44.273Z", "content": "How I see it: They are making a re-run with a new testset on your 2 submissions, starting with oldest, to many subs/team to start together. After the re-runs, when we have the final scores, the review will start.
", "rawMarkdown": "How I see it: They are making a re-run with a new testset on your 2 submissions, starting with oldest, to many subs/team to start together. After the re-runs, when we have the final scores, the review will start. ", "votes": 6 } ] }, { "id": 976496, "postDate": "2020-08-18T22:33:33.800Z", "content": "A day to go again? And why can't one post images anymore in the comment field, need to post that \"crazy cow\"-gif animation right now, thats how I feel.
", "rawMarkdown": "A day to go again? And why can't one post images anymore in the comment field, need to post that \"crazy cow\"-gif animation right now, thats how I feel.", "votes": 6 }, { "id": 974002, "postDate": "2020-08-17T16:51:52.133Z", "content": "Again, only 7 hours to go! Let's hope all the Yolo submissions are removed!
", "rawMarkdown": "Again, only 7 hours to go! Let's hope all the Yolo submissions are removed!", "votes": 5 }, { "id": 978124, "postDate": "2020-08-20T01:04:51.253Z", "content": "I am honored to be able to participate in this competition. I am very grateful to the staff for this competition, because this is my first time participating in this competition. I am very confused about this result. My result has been deleted. The notebook I refer to is open source. , And I modified it, I used eff_d5, and 10 epoch pesudo-lebaling, img_size=512, I want to know why my results are invalid。 @philculliton ![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4084996%2F84e64193f3995d724de8f82470367484%2F1.png?generation=1597885483970051&alt=media\")
Best thing to do is contact the compliance team here: https://www.kaggle.com/compliance
", "rawMarkdown": "Best thing to do is contact the compliance team here: https://www.kaggle.com/compliance" } ] }, { "id": 965990, "postDate": "2020-08-11T03:20:02.460Z", "content": "if rerun all the kernels, we should also select the submissions again. otherwise it's totally unfair! The selections are chosen based on the LB feedback.
", "rawMarkdown": "if rerun all the kernels, we should also select the submissions again. otherwise it's totally unfair! The selections are chosen based on the LB feedback.", "votes": 5 }, { "id": 965978, "postDate": "2020-08-11T02:58:46.697Z", "content": "I noticed that our public testset score changed after rerunning the whole kernel. Since we choose the submission kernel according to public LB score, the competition holder should rerun all the kernel and let us re-choose the submission again.
", "rawMarkdown": "I noticed that our public testset score changed after rerunning the whole kernel. Since we choose the submission kernel according to public LB score, the competition holder should rerun all the kernel and let us re-choose the submission again.", "votes": 4 }, { "id": 978184, "postDate": "2020-08-20T02:36:18.677Z", "content": "Hi Phil, thanks for such huge workload!
\nIs the current private leaderboard the final one? Will there be any changes？
I see movemement in the list, OK folks time to hold on to something, here we go.
", "rawMarkdown": "I see movemement in the list, OK folks time to hold on to something, here we go.", "votes": 3 }, { "id": 965463, "postDate": "2020-08-10T16:50:28.527Z", "content": "Anyone got a green clock next to Succeeded? Maybe it's worth knowing.\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3924325%2F1ff18093a93e35c4a6b1ee6686f02d28%2Fclock.JPG?generation=1597078054629118&alt=media\")
Yeah, I am having the same red tick @kirderf :|
", "rawMarkdown": "Yeah, I am having the same red tick @kirderf :|" } ] }, { "id": 976212, "postDate": "2020-08-18T18:06:16.343Z", "content": "I guess they are removing non compliant submissions as I see myself ranking up xD
", "rawMarkdown": "I guess they are removing non compliant submissions as I see myself ranking up xD ", "votes": 2, "replies": [ { "id": 976222, "postDate": "2020-08-18T18:17:40.233Z", "content": "Yes! I've moved up nearly 500 places! haha
", "rawMarkdown": "Yes! I've moved up nearly 500 places! haha", "votes": 2, "replies": [ { "id": 976224, "postDate": "2020-08-18T18:21:29.657Z", "content": "Cant wait to see :p how much I will drop in private LB :p hehehe. But good that Kaggle actually maintain a proper standard and removed all non-complaint solution
", "rawMarkdown": "Cant wait to see :p how much I will drop in private LB :p hehehe. But good that Kaggle actually maintain a proper standard and removed all non-complaint solution" }, { "id": 976279, "postDate": "2020-08-18T19:02:06.867Z", "content": "\n\nremoved all non-complaint solution
\n
At this moment, public lb is just what you picked as the final subs. Lots of kagglers decided to follow the competition rule and didn't pick the yolov5 solution. That seems the main reason for the lb change right now. So can't really tell. But I hope eventually all these non-compliant subs will be removed.
", "rawMarkdown": ">removed all non-complaint solution\n\nAt this moment, public lb is just what you picked as the final subs. Lots of kagglers decided to follow the competition rule and didn't pick the yolov5 solution. That seems the main reason for the lb change right now. So can't really tell. But I hope eventually all these non-compliant subs will be removed.", "votes": 2 }, { "id": 976326, "postDate": "2020-08-18T19:38:53.200Z", "content": "Huzzah! #55. It'll be exciting to see the shakeup, maybe the people who were 500+ in the first private LB may even get a medal.
", "rawMarkdown": "Huzzah! #55. It'll be exciting to see the shakeup, maybe the people who were 500+ in the first private LB may even get a medal. ", "votes": 2 } ] }, { "id": 976245, "postDate": "2020-08-18T18:34:34.073Z", "rawMarkdown": "", "votes": -1, "isDeleted": true }, { "id": 976350, "postDate": "2020-08-18T19:58:54.073Z", "content": "Oh wow, I went from 544 —> 85 on the public lb. Huge thanks to Kaggle & Hosts for making the effort to remove non compliant submissions.
", "rawMarkdown": "Oh wow, I went from 544 —> 85 on the public lb. Huge thanks to Kaggle & Hosts for making the effort to remove non compliant submissions.", "votes": 3 }, { "id": 976679, "postDate": "2020-08-19T03:13:43.873Z", "content": "From 489 -> 36. It's funny that I thought this was Private LB because of the deadline. Welp! let's just wait and see for another shakeup with the new test dataset.
", "rawMarkdown": "From 489 -> 36. It's funny that I thought this was Private LB because of the deadline. Welp! let's just wait and see for another shakeup with the new test dataset.", "votes": 1 } ] }, { "id": 964816, "postDate": "2020-08-10T07:35:45.023Z", "content": "\n\nLeaderboard Update Is Underway
\n
\n Please let us know if you have questions!
Excellent, thank you, understand that it is a lot of work and takes time, and thank you for the quality of each competition, including this one.
\nAs for the questions:
\nI think there are many questions but many fewer than it seems because there are many shades of the same wonder.
\nIt would be great if we gathered all the questions and answers in this topic, so we make this more efficient for everyone.
@piantic two questions sums up a couple of hundred posts and comments, I add one to his list.
\n3) How many of the teams will be reviewed , all or where is the line drawn?
\nI would also recommend to save and post the questions with your answers at the top of this topic, will be more accessible and people can first try to find their answer there, like a Q&A bulletin board.
", "rawMarkdown": "> Leaderboard Update Is Underway\n> Please let us know if you have questions!\n\nExcellent, thank you, understand that it is a lot of work and takes time, and thank you for the quality of each competition, including this one.\n\nAs for the questions:\nI think there are many questions but many fewer than it seems because there are many shades of the same wonder.\nIt would be great if we gathered all the questions and answers in this topic, so we make this more efficient for everyone.\n\n@piantic two questions sums up a couple of hundred posts and comments, I add one to his list.\n\n3) How many of the teams will be reviewed , all or where is the line drawn?\n\nI would also recommend to save and post the questions with your answers at the top of this topic, will be more accessible and people can first try to find their answer there, like a Q&A bulletin board.", "votes": 2 }, { "id": 965761, "postDate": "2020-08-10T20:38:12.897Z", "content": "Hi all. Thanks for your questions and comments. I am in the process of working through the re-run. Regarding the timeline, it will likely take until tomorrow to re-run all selected submissions. This is a process that requires some care and does take time. The error messages that many of you noted regarding submissions being processed after the deadline are due to a bug on our end, and will have no bearing on your final scores or submissions.
\nRe: questions about the leaderboard and scores, etc. The leaderboard will definitely continue changing until the process is complete.
\nThanks all for your patience!
", "rawMarkdown": "Hi all. Thanks for your questions and comments. I am in the process of working through the re-run. Regarding the timeline, it will likely take until tomorrow to re-run all selected submissions. This is a process that requires some care and does take time. The error messages that many of you noted regarding submissions being processed after the deadline are due to a bug on our end, and will have no bearing on your final scores or submissions.\n\nRe: questions about the leaderboard and scores, etc. The leaderboard will definitely continue changing until the process is complete.\n\nThanks all for your patience!", "votes": 1, "replies": [ { "id": 965762, "postDate": "2020-08-10T20:40:12.590Z", "content": "@philculliton okay..what about the removal of YoloV5 submissions? I could see many people asking the same question about this. However I see no reply from the Kaggle Team. Kindly respond to avoid confusion.
", "rawMarkdown": "@philculliton okay..what about the removal of YoloV5 submissions? I could see many people asking the same question about this. However I see no reply from the Kaggle Team. Kindly respond to avoid confusion.", "votes": 8 }, { "id": 966239, "postDate": "2020-08-11T09:05:38.750Z", "content": "@kalyanpichuka , they wrote this:
\n\n\nWe are also performing our standard leaderboard finalization.
\n
So if they use the same standard as previous similar competitions they will review all teams in the leaderboard. I think we can rely and stick to that.
\nThen if they do it in parallel or after publishing new scores, I would suggest that they publish a final reviewed leaderboard, since it is removed now.
@philculliton when are you guys removing YoloV5 submissions from the leader board? Please ensure that those who are receiving medals have compliant submissions. Make it a fair competition.
", "rawMarkdown": "@philculliton when are you guys removing YoloV5 submissions from the leader board? Please ensure that those who are receiving medals have compliant submissions. Make it a fair competition.", "votes": 1, "replies": [ { "id": 965706, "postDate": "2020-08-10T19:32:35.437Z", "content": "I think they started removing those. Today there are only 2215 teams on the leaderboard compared to 2270 teams when I last checked.
", "rawMarkdown": "I think they started removing those. Today there are only 2215 teams on the leaderboard compared to 2270 teams when I last checked.", "votes": 2 }, { "id": 965715, "postDate": "2020-08-10T19:43:12.423Z", "content": "I think they have some issues or is it me.
\nI don't even see the private leader board.
Haha no, its 2270 again, I guess there are some hiccups still :D
", "rawMarkdown": "Haha no, its 2270 again, I guess there are some hiccups still :D" } ] }, { "id": 965755, "postDate": "2020-08-10T20:27:40.270Z", "content": "same - all I can see now is public leaderboard like when the competition hasn't finished and the private leaderboard has not yet been revealed.
", "rawMarkdown": "same - all I can see now is public leaderboard like when the competition hasn't finished and the private leaderboard has not yet been revealed.", "votes": -1 } ] }, { "id": 965192, "postDate": "2020-08-10T13:01:38.647Z", "content": "After this leader board update, are you going to remove YOLOv5 solutions?
", "rawMarkdown": "After this leader board update, are you going to remove YOLOv5 solutions?", "votes": 1 }, { "id": 976557, "postDate": "2020-08-19T00:24:03.153Z", "content": "Why is one of them an error when you chose two that are both fine for licensing?
\nIs it still a work in progress?
@mkawakami if your submission took over 6 hours?
", "rawMarkdown": "@mkawakami if your submission took over 6 hours? ", "votes": 1 } ] }, { "id": 969400, "postDate": "2020-08-13T16:51:11.420Z", "content": "A silent wonder, is the idea that the final LB should be presented when the countdown has passed the new deadline, or is it nothing to keep track of.
", "rawMarkdown": "A silent wonder, is the idea that the final LB should be presented when the countdown has passed the new deadline, or is it nothing to keep track of.", "votes": 1 }, { "id": 965999, "postDate": "2020-08-11T03:36:51.017Z", "content": "My guess is that Kaggle changed some data's image size to 1024x1024, which was their commitment.
\nTo fully mitigate that effect, kaggle decided to rerun the notebook to see fair evaluation results.
\nThis makes sense and should not bring large shakes.
If that is the case, I hope they don’t forget to rescale the ground truth bounding boxes to 1024x1024 also, as many of us rescale the bounding boxes according to image sizes, not 1024.
", "rawMarkdown": "If that is the case, I hope they don’t forget to rescale the ground truth bounding boxes to 1024x1024 also, as many of us rescale the bounding boxes according to image sizes, not 1024.", "votes": 1 } ] }, { "id": 964860, "postDate": "2020-08-10T08:18:01.053Z", "content": "Thank you for your efforts to ensure the fairness of the compete！👍
\nI have two questions.
Thanks for your reply, but I find an interesting phenomena. A score of one of selected submissions on My Submissions changed today, while the other didn‘t. And the changed score is different from the score on the Private Leaderboard now.
", "rawMarkdown": "Thanks for your reply, but I find an interesting phenomena. A score of one of selected submissions on My Submissions changed today, while the other didn‘t. And the changed score is different from the score on the Private Leaderboard now." }, { "id": 964956, "postDate": "2020-08-10T09:43:56.433Z", "content": "I think the algo is evaluating it from Oldest to Newest Submissions.\nMy older submission got evaluated (and is showing SUBMITTED AFTER DEADLINE - although it was submitted before deadline.)
\n\nMy guess is your OLDER submission has got evaluated and there is a RED tick there.
", "rawMarkdown": "I think the algo is evaluating it from Oldest to Newest Submissions.\nMy older submission got evaluated (and is showing SUBMITTED AFTER DEADLINE - although it was submitted before deadline.)\n\nMy guess is your OLDER submission has got evaluated and there is a RED tick there.", "votes": 3 }, { "id": 964975, "postDate": "2020-08-10T09:55:34.663Z", "content": "You're right, there is a RED tick😂
", "rawMarkdown": "You're right, there is a RED tick😂" }, { "id": 964978, "postDate": "2020-08-10T09:57:32.233Z", "content": "@watzisname yes. And I am victim to that. Both my submissions got red ticks now.
", "rawMarkdown": "@watzisname yes. And I am victim to that. Both my submissions got red ticks now. " } ] }, { "id": 970463, "postDate": "2020-08-14T13:32:14.737Z", "content": "What are the modified things?
\nI think…
1.image size modified 1024px.(with bbox?)
\n2.modify bounding box(training and testing are differences)
What's happen?
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F282250%2F8dcbd07f2abc7b891ca5ae80481cd9e0%2F2020-08-11%200.45.49.png?generation=1597074393836176&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F282250%2F60d4e7dcfc13f87bb40e99b719b62475%2F2020-08-11%200.45.56.png?generation=1597074394321862&alt=media\")
I think they are still re-running all the submissions with a completely new dataset. So wait a bit :/
", "rawMarkdown": "I think they are still re-running all the submissions with a completely new dataset. So wait a bit :/\n", "votes": 1 }, { "id": 965411, "postDate": "2020-08-10T15:53:48.720Z", "content": "This announce is re-run. but it's not equal Private LB & my selection(two).
\nI guess that Private LB in now is not exactly real Leaderboard.\n(I understand host doing a re-run at now)
We have the same situation
", "rawMarkdown": "We have the same situation ", "votes": 8 }, { "id": 965424, "postDate": "2020-08-10T16:01:52Z", "content": "Sure. I'm waiting for finishing the finalized Leaderboard.
", "rawMarkdown": "Sure. I'm waiting for finishing the finalized Leaderboard." }, { "id": 965425, "postDate": "2020-08-10T16:05:10.400Z", "content": "Your score in private lb is just selected that have highest public lb score.\nIt will be changed after cleaning up private lb.\n@tereka
", "rawMarkdown": "Your score in private lb is just selected that have highest public lb score.\nIt will be changed after cleaning up private lb.\n@tereka\n", "votes": 4 }, { "id": 965573, "postDate": "2020-08-10T18:02:03.557Z", "content": "I can not fathom what was the reasoning behind first **publishing the winners ( or so we thought ...) and then running a new procedure which **shuffles the rankings. Why publish the scores of the first phase anyway letting contenders (me and my team) falsely believe we have won?
", "rawMarkdown": "I can not fathom what was the reasoning behind **first **publishing the winners ( or so we thought ...) and then running a new procedure which **shuffles the rankings**. Why publish the scores of the **first phase anyway** letting contenders (me and my team) falsely believe we have won? ", "votes": 12 }, { "id": 965645, "postDate": "2020-08-10T18:33:55.800Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 966249, "postDate": "2020-08-11T09:23:22.527Z", "content": "May I ask if non-compliant submissions will be removed?
", "rawMarkdown": "May I ask if non-compliant submissions will be removed?", "votes": 1 }, { "id": 977086, "postDate": "2020-08-19T09:44:04.497Z", "content": "I don't know which library I used is prohibited, and these are my two submissions yolov4 + effdd7, hope someone can help me.
", "rawMarkdown": "I don't know which library I used is prohibited, and these are my two submissions [yolov4 + effdd7](https://www.kaggle.com/marcowu/ap2-0effd-yolov4-pseudo-labeling-oof-evalu/notebook?select=train.py), hope someone can help me.", "replies": [ { "id": 977139, "postDate": "2020-08-19T10:12:29.043Z", "content": "same…….
", "rawMarkdown": "same.......\n", "votes": 2 }, { "id": 977166, "postDate": "2020-08-19T10:31:05.967Z", "content": "@chanhu Could you tell me which library I used is prohibited, I would appreciate it!
", "rawMarkdown": "@chanhu Could you tell me which library I used is prohibited, I would appreciate it!" }, { "id": 977433, "postDate": "2020-08-19T13:24:27.150Z", "content": "I have no idea. 🤒
", "rawMarkdown": "I have no idea. 🤒", "votes": 1 }, { "id": 977607, "postDate": "2020-08-19T15:39:07.783Z", "content": "@philculliton Why this notebook even could get score,but I can not?
\n", "rawMarkdown": "@philculliton Why this notebook even could get score,but I can not?\n\nhttps://www.kaggle.com/naimur978/wheat-head-detection-yolov5-pseudo-labelling-oof?scriptVersionId=38595668", "votes": 9 }, { "id": 978055, "postDate": "2020-08-19T22:39:17.580Z", "content": "https://www.kaggle.com/c/global-wheat-detection/discussion/164845#923524 please read @rwightman post from about month ago about efficientDet
", "rawMarkdown": "https://www.kaggle.com/c/global-wheat-detection/discussion/164845#923524 please read @rwightman post from about month ago about efficientDet" }, { "id": 978172, "postDate": "2020-08-20T02:16:13.760Z", "content": "😣The same ,effd5+faster. It make me so upset…
", "rawMarkdown": "😣The same ,effd5+faster. It make me so upset...", "votes": 2 } ] }, { "id": 970122, "postDate": "2020-08-14T08:12:52.053Z", "content": "Hey you all, I need help on this project. Actually I have choose it as my project in my college and want to work on it. I need guidance of someone step by step like an instructor. Is here anyone who can help me? Please
", "rawMarkdown": "Hey you all, I need help on this project. Actually I have choose it as my project in my college and want to work on it. I need guidance of someone step by step like an instructor. Is here anyone who can help me? Please", "votes": -5, "replies": [ { "id": 970289, "postDate": "2020-08-14T10:16:17.167Z", "content": "Read the room man…
", "rawMarkdown": "Read the room man...", "votes": 1 }, { "id": 970290, "postDate": "2020-08-14T10:19:03.043Z", "content": "Legit though, there are plenty of great topics in the discussion forums and notebooks:
\nWhy update scor after publishing winners? makes no sesne.
", "rawMarkdown": "Why update scor after publishing winners? makes no sesne. ", "votes": -8 }, { "id": 978761, "postDate": "2020-08-20T11:14:49.840Z", "content": "Hi Phil,
\nAll the submission model are trained using my computer and google colab, and I only shared my trained models to my teammembers for testing and anaylsis.
\nThe submitted results is using efficientd5, and it is based on a open notebook of this competition. one of the notebook, I refered is https://www.kaggle.com/shonenkov/training-efficientdet.I did huge study on data augmentation, trainning strategy, and you can see it from my submission history.
\nI quite confused why there are so many error in my submission result, except two of them are acceptable.
Thanks a lot for your time on investigation in advance,
", "rawMarkdown": "Hi Phil,\n\nAll the submission model are trained using my computer and google colab, and I only shared my trained models to my teammembers for testing and anaylsis.\n\nThe submitted results is using efficientd5, and it is based on a open notebook of this competition. one of the notebook, I refered is https://www.kaggle.com/shonenkov/training-efficientdet.I did huge study on data augmentation, trainning strategy, and you can see it from my submission history.\nI quite confused why there are so many error in my submission result, except two of them are acceptable.\n\nThanks a lot for your time on investigation in advance,", "votes": -4, "replies": [ { "id": 979241, "postDate": "2020-08-20T17:55:38.150Z", "content": "The rest of the scores no longer show up because of the re-run they did on the updated private set. If you wish to submit again, I'm sure they will open up late submissions in a few days.
", "rawMarkdown": "The rest of the scores no longer show up because of the re-run they did on the updated private set. If you wish to submit again, I'm sure they will open up late submissions in a few days.", "votes": 1 }, { "id": 979509, "postDate": "2020-08-20T22:48:51.247Z", "content": "We have enabled late submissions.
", "rawMarkdown": "We have enabled late submissions. ", "votes": 1 } ] }, { "id": 978740, "postDate": "2020-08-20T11:04:00.987Z", "content": "All the submission model are trained using my computer and google colab, and I only shared my trained models to my teammembers for testing and anaylsis.
\nThe submitted results is using efficientd5, and it is based on a open notebook of this competition. one of the notebook, I refered is Alex Shonenkov. I did huge study on data augmentation, trainning strategy, and you can see it from my submission history.
\nI quite confused why there are so many error in my submission result, except two of them are acceptable.
Thanks a lot for your time on investigation in advance,
\nJay
", "rawMarkdown": "All the submission model are trained using my computer and google colab, and I only shared my trained models to my teammembers for testing and anaylsis.\n\nThe submitted results is using efficientd5, and it is based on a open notebook of this competition. one of the notebook, I refered is [Alex Shonenkov](https://www.kaggle.com/shonenkov/training-efficientdet). I did huge study on data augmentation, trainning strategy, and you can see it from my submission history.\nI quite confused why there are so many error in my submission result, except two of them are acceptable.\n\nThanks a lot for your time on investigation in advance,\n\n\nJay", "votes": -2 }, { "id": 972045, "postDate": "2020-08-16T07:31:15.150Z", "content": "\nHi there! Just a quick suggestion that if all the kernels are rerun, it is more reasonable to let us re-select our submission entries. This is because there are random factors inside pseudo-label training, and we usually select the final entries based on our public scores, right?
", "rawMarkdown": "@philculliton \n\nHi there! Just a quick suggestion that if all the kernels are rerun, it is more reasonable to let us re-select our submission entries. This is because there are random factors inside pseudo-label training, and we usually select the final entries based on our public scores, right?", "votes": -1 }, { "id": 965867, "postDate": "2020-08-10T23:09:46.180Z", "content": "Hi，why I cannot see the private leaderboard?
", "rawMarkdown": "Hi，why I cannot see the private leaderboard?", "votes": -7 }, { "id": 1755999, "postDate": "2022-04-15T06:23:37.940Z", "content": "Great job!
", "rawMarkdown": "Great job!" }, { "id": 981867, "postDate": "2020-08-22T19:03:27.377Z", "content": "Well done everybody!
", "rawMarkdown": "Well done everybody!" }, { "id": 978681, "postDate": "2020-08-20T10:16:54.547Z", "content": "The last night i have droped 400, cause i overfitted .
", "rawMarkdown": "The last night i have droped 400, cause i overfitted . " }, { "id": 978281, "postDate": "2020-08-20T04:38:49.890Z", "content": "Thanks @philculliton!
\nAnd it is really great job.
\nWe are so happy that all of this is back to good.
Thanks again Kaggle Team!
", "rawMarkdown": "Thanks @philculliton!\n\nAnd it is really great job. \nWe are so happy that all of this is back to good.\n\nThanks again Kaggle Team!" }, { "id": 977060, "postDate": "2020-08-19T09:25:41.340Z", "content": "Appreciate Kaggle team. Even though they didn't respond to many questions which were posted, they ensured that they will remove all non compliant submissions. 15 hours to go.. I am sure it would be a compliant leaderboard..
", "rawMarkdown": "Appreciate Kaggle team. Even though they didn't respond to many questions which were posted, they ensured that they will remove all non compliant submissions. 15 hours to go.. I am sure it would be a compliant leaderboard.. " }, { "id": 977049, "postDate": "2020-08-19T09:04:47.550Z", "content": "I see a big drop in the number of teams today..what happened? :0
", "rawMarkdown": "I see a big drop in the number of teams today..what happened? :0", "replies": [ { "id": 977062, "postDate": "2020-08-19T09:26:04.210Z", "content": "My guess is Non compliant submission teams would have been removed..
", "rawMarkdown": "My guess is Non compliant submission teams would have been removed..", "votes": 2 } ] }, { "id": 976818, "postDate": "2020-08-19T05:40:23.883Z", "content": "The chosen one is not the best one, could you tell me why? thanks very much for your help.
", "rawMarkdown": "The chosen one is not the best one, could you tell me why? thanks very much for your help." }, { "id": 976347, "postDate": "2020-08-18T19:57:09.547Z", "content": "Woah what is happening guys??
", "rawMarkdown": "Woah what is happening guys??", "replies": [ { "id": 976419, "postDate": "2020-08-18T20:48:04.753Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 972147, "postDate": "2020-08-16T09:11:58.380Z", "content": "waiting to see the result
", "rawMarkdown": "waiting to see the result " }, { "id": 965886, "postDate": "2020-08-11T00:03:28.410Z", "content": "Thanks for the update. Why is my public score different now? I understand that the private LB is still being processed and adjusted for but why would that change my score for the other dataset?
", "rawMarkdown": "Thanks for the update. Why is my public score different now? I understand that the private LB is still being processed and adjusted for but why would that change my score for the other dataset?", "replies": [ { "id": 965934, "postDate": "2020-08-11T01:51:57.820Z", "content": "Instead update private test ground-truth and rerun evaluation script, they rerun whole notebook. That destroys all tuning parameters (pseudo lr, nms threshold, box threshold ) based on public lb. If you pseudo on hidden testset(public+private), you will get another public result. I hate this competition and I wish I didn't join it
", "rawMarkdown": "Instead update private test ground-truth and rerun evaluation script, they rerun whole notebook. That destroys all tuning parameters (pseudo lr, nms threshold, box threshold ) based on public lb. If you pseudo on hidden testset(public+private), you will get another public result. I hate this competition and I wish I didn't join it", "votes": 24 }, { "id": 965947, "postDate": "2020-08-11T02:08:13.623Z", "content": "@nguyenbadung I'm sorry you feel bad for joining this competition. You should definitely be proud of your accomplishment though, you made a great solo submission! I hope the final LB turns out well for you
", "rawMarkdown": "@nguyenbadung I'm sorry you feel bad for joining this competition. You should definitely be proud of your accomplishment though, you made a great solo submission! I hope the final LB turns out well for you", "votes": 7 }, { "id": 966012, "postDate": "2020-08-11T03:49:33.533Z", "content": "@nguyenbadung You had done great work in this competition. (I hate this competition too)
", "rawMarkdown": "@nguyenbadung You had done great work in this competition. (I hate this competition too)", "votes": 1 }, { "id": 966095, "postDate": "2020-08-11T06:05:39.393Z", "content": "Honestly this competition has been really frustrating for me as well. I'm pretty sure I've spent more than 200 hours on this, but the labeling quality is so poor that traditional and novel techniques in object detection just don't work for most of the time. I hope that everyone can get the results they deserve.
", "rawMarkdown": "Honestly this competition has been really frustrating for me as well. I'm pretty sure I've spent more than 200 hours on this, but the labeling quality is so poor that traditional and novel techniques in object detection just don't work for most of the time. I hope that everyone can get the results they deserve.", "votes": 7 }, { "id": 966118, "postDate": "2020-08-11T06:42:17.107Z", "content": "And the yolov5
", "rawMarkdown": "And the yolov5" }, { "id": 966121, "postDate": "2020-08-11T06:45:07.463Z", "content": "This competition problem setting is very impressive for me.
\nAlso, the host discusses about many problem topics. It's great.\nBut host could not handle such problems(re-run, data changing, license) as well. \nI'm a disappointment and very frustrating too.
Hey, can anyone tell me why are my submissions showing \"submission after deadline\"? I had both my submissions done a long time before the deadline
", "rawMarkdown": "Hey, can anyone tell me why are my submissions showing \"submission after deadline\"? I had both my submissions done a long time before the deadline", "replies": [ { "id": 964953, "postDate": "2020-08-10T09:43:00.493Z", "content": "Did they finished before the deadline? Otherwise that's the reason
", "rawMarkdown": "Did they finished before the deadline? Otherwise that's the reason", "votes": -1 }, { "id": 964961, "postDate": "2020-08-10T09:47:59.173Z", "content": "Yes, I was 649th yesterday and now it's 1739th. I am sure my submissions are not discarded as they were not YOLOv5 and complied with all the rules of the competition. Even if they are for some reason, I should be notified right? But my submissions are showing \"submitted after deadline\". Both my submissions were finished before the deadline.
", "rawMarkdown": "Yes, I was 649th yesterday and now it's 1739th. I am sure my submissions are not discarded as they were not YOLOv5 and complied with all the rules of the competition. Even if they are for some reason, I should be notified right? But my submissions are showing \"submitted after deadline\". Both my submissions were finished before the deadline." }, { "id": 964973, "postDate": "2020-08-10T09:55:00.363Z", "content": "OK that was a more tricky one to answer, one for the Kaggle team, if it's not just something that shows during the process and will disappear when all is reviewed. Hard to tell.
", "rawMarkdown": "OK that was a more tricky one to answer, one for the Kaggle team, if it's not just something that shows during the process and will disappear when all is reviewed. Hard to tell.", "votes": 1 }, { "id": 964984, "postDate": "2020-08-10T09:59:19.887Z", "content": "I hope it's temporary. I dropped from 600th position, and this was my first kaggle competition. I will lose all my motivation if that's the case :(
", "rawMarkdown": "I hope it's temporary. I dropped from 600th position, and this was my first kaggle competition. I will lose all my motivation if that's the case :(", "votes": 2 }, { "id": 964997, "postDate": "2020-08-10T10:10:08.217Z", "content": "Submissions which have been evaluated are showing up with \"SUBMITTED AFTER DEADLINE\" in my view.
\n\nMy older selected submission got evaluated and is showing with a RED timeline mark. \nThe other submission is not yet showing up with RED mark and guess it has not been re-evaluated.
\n\nAre both your selected submissions showing with the red timeline mark ?
", "rawMarkdown": "Submissions which have been evaluated are showing up with \"SUBMITTED AFTER DEADLINE\" in my view.\n\nMy older selected submission got evaluated and is showing with a RED timeline mark. \nThe other submission is not yet showing up with RED mark and guess it has not been re-evaluated.\n\nAre both your selected submissions showing with the red timeline mark ?", "votes": 1 }, { "id": 965038, "postDate": "2020-08-10T10:45:42.377Z", "content": "@watzisname : Even i am getting the same. I had submitted my submissions long back. Got submitted after competition deadline. Hopefully, its temporary. And my score has been reduced tremendously as well..
", "rawMarkdown": "@watzisname : Even i am getting the same. I had submitted my submissions long back. Got submitted after competition deadline. Hopefully, its temporary. And my score has been reduced tremendously as well.. " }, { "id": 965054, "postDate": "2020-08-10T11:01:31.363Z", "content": "Yes I see the same with my subs now, make sense that they are tagged SUBMITTED AFTER DEADLINE, since they are submitted again(re-run) after deadline by Kaggle.The score went down though, but I prepare for worst and hope for the best with my second sub
", "rawMarkdown": "Yes I see the same with my subs now, make sense that they are tagged SUBMITTED AFTER DEADLINE, since they are submitted again(re-run) after deadline by Kaggle.The score went down though, but I prepare for worst and hope for the best with my second sub", "votes": 1 }, { "id": 965065, "postDate": "2020-08-10T11:10:31.980Z", "rawMarkdown": "", "isDeleted": true }, { "id": 965068, "postDate": "2020-08-10T11:13:38.617Z", "content": "@kiderf @manoj
\n\nMy score on LB is not equal to my re-evaluated submission score. \nIs that the case for you too ?
", "rawMarkdown": "@kiderf @manoj\n\nMy score on LB is not equal to my re-evaluated submission score. \nIs that the case for you too ?" }, { "id": 965072, "postDate": "2020-08-10T11:17:33.763Z", "content": "Yeah @watzisname
", "rawMarkdown": "Yeah @watzisname " }, { "id": 965073, "postDate": "2020-08-10T11:18:06.253Z", "content": "@watzisname
\nMe too, I guess we need to wait for an update
My score reduced as well. I have no clue what's happening :/
", "rawMarkdown": "My score reduced as well. I have no clue what's happening :/\n" }, { "id": 965076, "postDate": "2020-08-10T11:20:10.317Z", "content": "yes for sure I see it, and we can call it not equal, it sounds less dramatic than my first thought on the score diff. But 1 sub left to review, anything can happen, I lay my hope in that final sub review.
", "rawMarkdown": "yes for sure I see it, and we can call it not equal, it sounds less dramatic than my first thought on the score diff. But 1 sub left to review, anything can happen, I lay my hope in that final sub review.", "votes": 1 }, { "id": 965093, "postDate": "2020-08-10T11:30:31.537Z", "content": "All the best to you mate @kirderf . Fingers crossed. Anyways, I expected some small + or - 0.01 would be there. But -0.04 on both the subs, didn't expect it though. Hopefully, you get a better score..
", "rawMarkdown": "All the best to you mate @kirderf . Fingers crossed. Anyways, I expected some small + or - 0.01 would be there. But -0.04 on both the subs, didn't expect it though. Hopefully, you get a better score.. ", "votes": 1 } ] }, { "id": 964522, "postDate": "2020-08-10T01:12:15.753Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 964578, "author_name": "Heroseo", "author_url": "", "post_date": "2020-08-10T02:58:35.463000", "content": "Hi, @philculliton.\nI'm happy to hear that private lb will be updated again.
\n\nMy question is two:\n1) May I ask that will yolov5(or other package of license issues) be removed in re-run private lb?
\n\n2) How long will it take approximately to complete all process?
\n\nI would be very happy if you respond and also other competitors in this competition would be happy.\nThanks!
", "votes": 23, "replies": [] }, { "id": 976556, "author_name": "QuantScientist", "author_url": "", "post_date": "2020-08-19T00:22:14.903000", "content": "This is by far the worst managed competition I have ever had the chance to participate in. Nothing, nothing gets even close to this. Promises being repeatedly broken and the unsatisfactory explanation as to why this is even happening !!!
", "votes": 24, "replies": [ { "id": 976567, "author_name": "962853", "author_url": "", "post_date": "2020-08-19T00:30:50.260000", "content": "The worst part is that they don't seem to care about our comments at all.
", "votes": 8, "replies": [] }, { "id": 977410, "author_name": "Robin Smits", "author_url": "", "post_date": "2020-08-19T13:08:50.007000", "content": "Or Just may'be they do care .. but they need all there time and attention to finalize the LB.
\nI dont like the waiting either but looking at the LB changes it is worth it.
", "votes": 5, "replies": [] }, { "id": 977501, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2020-08-19T14:18:13.203000", "content": "@rsmits, my sentiments exactly.
", "votes": 1, "replies": [] } ] }, { "id": 974322, "author_name": "Phil Culliton", "author_url": "", "post_date": "2020-08-17T22:12:23.720000", "content": "Hi all.
\nWe had some additional checks we needed to do today, and ran out of staff time to do the final database update (which is not a small or risk-less thing). We will do so tomorrow and finalize the LB. I'm sorry for how long this is taking. The number of moving parts are considerably higher than a normal competition close, and we're trying to do the best job we can.
\nRe: questions - yes, we will be opening late submissions following the finalization of the LB.
\nThanks again, all. We really appreciate your patience.
", "votes": 25, "replies": [ { "id": 975547, "author_name": "", "author_url": "", "post_date": "2020-08-18T10:51:05.323000", "content": "And please release the complete dataset.
", "votes": -4, "replies": [] }, { "id": 977920, "author_name": "", "author_url": "", "post_date": "2020-08-19T19:48:20.420000", "content": "You said late submission will be enabled, but the competition ended it's still not on.
", "votes": 0, "replies": [] } ] }, { "id": 977892, "author_name": "Phil Culliton", "author_url": "", "post_date": "2020-08-19T19:29:44.780000", "content": "Hi all!
\nAs some of you have noted, the leaderboard has changed significantly. We have done our best to remove submissions that contained YOLOv5. We have also performed our normal post-competition process that has removed some other teams.
\nIf you believe your submission (or team) was incorrectly removed, please contact kaggle.com/compliance - this is our normal process for resolving disputes about team removals, submission invalidations, etc. I appreciate the Kagglers who have reached out to me directly, but if you follow the official route we can ensure your case is looked at properly and in a timely fashion.
\nI am, once again, sorry that this took so long. It was detailed work that required a great deal of time from several of us. We appreciate your patience and the support you've shown. Finally, congratulations to the winners - I know the hosts are excited to examine your solutions!
", "votes": 17, "replies": [ { "id": 977899, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-19T19:34:33.510000", "content": "Thanks, great job! Where draws the line for medals now? Is it calculated from the number of teams before the removal of the teams, like all other competitions?
", "votes": 1, "replies": [] }, { "id": 977905, "author_name": "Matt", "author_url": "", "post_date": "2020-08-19T19:37:59.653000", "content": "Hi Phil, thank you for taking the time to review the leaderboard submissions and make your best effort at removing non-compliant submissions. However, I am very disappointed to see that some YoloV5 submissions are still on the leaderboard… (for example: https://www.kaggle.com/naimur978/wheat-head-detection-yolov5-pseudo-labelling-oof)
\nI hope that the compliance team will continue to remove these submissions over the coming weeks.
", "votes": 1, "replies": [] }, { "id": 977934, "author_name": "Phil Culliton", "author_url": "", "post_date": "2020-08-19T19:58:06.740000", "content": "Thanks @matthewmasters - that was one of a small handful (~20) that was re-run last night and it slipped back into the general pool. It's been removed.
\nWe will continue to remove teams that broke the rules if they become apparent - the host will be largely dealing with that task now.
", "votes": 3, "replies": [] }, { "id": 977935, "author_name": "Phil Culliton", "author_url": "", "post_date": "2020-08-19T19:58:34.343000", "content": "@kirderf That's correct! We will be calculating medals based on the original number of teams.
", "votes": 2, "replies": [] }, { "id": 977938, "author_name": "", "author_url": "", "post_date": "2020-08-19T20:03:31.920000", "content": "", "votes": 0, "replies": [] }, { "id": 977943, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-19T20:06:25.027000", "content": "@philculliton sounds good. Is it possible to tell right now what the lines for silver and bronze are.
", "votes": 1, "replies": [] }, { "id": 977953, "author_name": "", "author_url": "", "post_date": "2020-08-19T20:14:01.910000", "content": "", "votes": 0, "replies": [] }, { "id": 977960, "author_name": "Kalyan Kumar Pichuka", "author_url": "", "post_date": "2020-08-19T20:19:30.667000", "content": "Thanks @philculliton for results. You guys made it a fair competition. I sincerely appreciate Kaggle Team efforts in doing so. I must say this…KAGGLE TEAM ARE MEN OF ACTION…you guys spoke less..but at the end you guys did what has to be done..much appreciate
", "votes": -1, "replies": [] }, { "id": 977964, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-19T20:25:07.320000", "content": "@stanleyjzheng OK so there were 2,245 teams before the removal?
", "votes": 1, "replies": [] }, { "id": 977967, "author_name": "", "author_url": "", "post_date": "2020-08-19T20:27:20.583000", "content": "", "votes": 0, "replies": [] }, { "id": 977990, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-19T20:48:26.137000", "content": "Then I lost a medal on the finish line again within 10 places to the level, crazy bad luck, is the fifth competition in a short time where I lose within 10 places to the medal level.
\nLucky that one gets some points for the work in any case, rank 169 in 10 months always something.
\nJust to fight on and get better until next time!
Thank you all for your contributions, Kaggle for the tremendous work with this competition and congratulations to all the winners!
", "votes": 1, "replies": [] }, { "id": 977992, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-19T20:50:43.227000", "content": "I'm so sorry Kirderf, that's some crazy bad luck; Top 200 worldwide is still something to be extremely proud of. Keep trying, your luck is bound to turn around sometime! The leaderboard is still fluid, so it's possible a few more submissions will be removed to push you into the medal positions.
", "votes": 1, "replies": [] }, { "id": 977993, "author_name": "Phil Culliton", "author_url": "", "post_date": "2020-08-19T20:51:34.447000", "content": "We have - thanks to an extremely helpful report - identified another small set of kernels to invalidate for non-Yolo reasons - basically the last of the bookkeeping from the long process. We will be doing so shortly.
\nThanks to everyone for your feedback. Please let us know if you spot anomalies.
", "votes": 2, "replies": [] }, { "id": 977997, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2020-08-19T20:57:25.197000", "content": "Thanks @philculliton for the update and a job well done on removing cheaters. Are the points awarded yet?
", "votes": 0, "replies": [] }, { "id": 978040, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-19T22:01:53.723000", "content": "A miracle, signed @philculliton, I'm now on the right side of the medal zone! hallelujah 👍
", "votes": 4, "replies": [] } ] }, { "id": 970878, "author_name": "Phil Culliton", "author_url": "", "post_date": "2020-08-14T23:40:59.777000", "content": "Hi all!
\nWe're still updating the leaderboard. As you might imagine, sorting this all out has been an involved process. We appreciate your patience as we continue to finalize things. We are shooting to have everything done and finalized on Monday.
", "votes": 16, "replies": [ { "id": 970884, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-15T00:16:11.103000", "content": "Thanks a ton for the status update. I'm excited to see the final leaderboard!
", "votes": 3, "replies": [] }, { "id": 971400, "author_name": "", "author_url": "", "post_date": "2020-08-15T13:33:40.590000", "content": "", "votes": -1, "replies": [] }, { "id": 971416, "author_name": "Jaideep", "author_url": "", "post_date": "2020-08-15T14:01:44.387000", "content": "@philculliton till now i think we have not got an ans to whether YOlov5 sub be cancelled during this re run (IRRESPECTIVE OF WHETHER THOSE SUB ARE IN MONEY OR NOT) ,if people have selected it for final sub
", "votes": 1, "replies": [] }, { "id": 971446, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-15T14:30:04.997000", "content": "@kshitijpatil09 I am fairly certain there will be late submissions enabled following the finalized leaderboard. Some people were doing papers, and late submission was turned on for a short period following the first end of the competition.
", "votes": 1, "replies": [] }, { "id": 972049, "author_name": "zclzc", "author_url": "", "post_date": "2020-08-16T07:35:06.577000", "content": "Thanks for the update! Just a quick suggestion that if all the kernels are rerun, it is more reasonable to let us re-select our submission entries. This is because there are random factors inside pseudo-label training, and we usually select the final entries based on our public scores, right?
", "votes": 0, "replies": [] } ] }, { "id": 965825, "author_name": "Maciej A. Czyzewski", "author_url": "", "post_date": "2020-08-10T22:09:28.223000", "content": "i am confused ^^
\n
Kaggle knows how to make me smile…
", "votes": 11, "replies": [ { "id": 965885, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-10T23:58:40.367000", "content": "Another one to add to the collection! Great editing, gave me a good laugh.
\n\n
This competition is one of the worst organised competitions on kaggle.
", "votes": 13, "replies": [ { "id": 975556, "author_name": "Rifat Rashid", "author_url": "", "post_date": "2020-08-18T10:56:33.583000", "content": "Also it's quite unfair to kaggler who spends months to optimize his/her solution despite so messy dataset. Now, at the end of the competition organizers decided to completely change the test dataset which will mess up a lot of people's solutions.
", "votes": 9, "replies": [] }, { "id": 975724, "author_name": "Karmanov Alexey", "author_url": "", "post_date": "2020-08-18T12:47:39.050000", "content": "One can't disagree with you. I like the Kaggle platform, but the organization of this competition is not up to par.
", "votes": 2, "replies": [] } ] }, { "id": 976548, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2020-08-19T00:08:23.587000", "content": "\n\n@tanulsingh077, I think the right answer is yes all teams with Yolov5 submission chosen as one of their final 2 subs will be removed from the LB. I do not understand why people are confused. It is not hard to write a simple script that goes through 4000+ submissions to check for Yolov5 models hence I believe Kaggle will do it. Heck if this were posted as Kaggle challenge, many teams would score 99.99 accuracy within the 1st week. Just my 2 cents :-)
\n
I posted the above 24 days ago in the Leaderboard of Compliant Submissions where people were still asking about YOLOV5 submission. The thread I responded to was asking specific questions one of which was
\n\n\nIf a team uses yolo will it be removed from the lb?
\n
This is after the question have been answered by the Kaggle team be it indirectly for example by @philculliton stating
\n\n\nyou cannot win prizes if you use YOLOv5.
\n
I have seen the question answered by others as well but people kept asking over and over again in the discussion section.
\nSo kudos to the Kaggle team for doing what I thought they would because it simply makes perfect sense. I know LB is not finished updating but it is now down to 1815 teams from 2270. Thanks @philculliton and team.
", "votes": 10, "replies": [] }, { "id": 970659, "author_name": "Kaushal Shah", "author_url": "", "post_date": "2020-08-14T16:40:10.540000", "content": "Only 7 hours to go! Let's hope all the Yolo submissions are removed!
", "votes": 6, "replies": [ { "id": 970747, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-14T18:19:32.910000", "content": "I put my hope in that direction
", "votes": 0, "replies": [] }, { "id": 970814, "author_name": "PrasunMishra", "author_url": "", "post_date": "2020-08-14T20:37:30.557000", "content": "One more day added!
", "votes": 2, "replies": [] }, { "id": 970824, "author_name": "QuantScientist", "author_url": "", "post_date": "2020-08-14T20:59:01.960000", "content": "Its ridiculous.
", "votes": 5, "replies": [] }, { "id": 970917, "author_name": "PrasunMishra", "author_url": "", "post_date": "2020-08-15T02:08:25.373000", "content": "@kaushal2896 you wanted 7 but got 72, more than you wished for! 😃
", "votes": 1, "replies": [] }, { "id": 971398, "author_name": "Jacob", "author_url": "", "post_date": "2020-08-15T13:32:46.133000", "content": "I guess there will be another one
\n
@creatrol you were right lol
", "votes": 0, "replies": [] } ] }, { "id": 978562, "author_name": "Vaibhav Mehrotra", "author_url": "", "post_date": "2020-08-20T08:58:43.550000", "content": "Hey @philculliton,
\nMe and my teammate appreciate the effort put into removing the Yolov5 submissions.
Having said that, we cannot find ourselves on the leaderboard. Both of our submissions seem to have been invalidated with a failed final private rerun error message. We have used the MIT-compliant EfficientDet for both submissions with different ensembling methods. As of yesterday (19/08/2020), we could find our submissions on the leaderboard but they have been removed since then. We are also sure that the runtime of our models is <6h because we have made multiple submissions (80+) before our final submission.
We'd appreciate your help in the matter because like many other participants here, we have put in multiple months of effort and worked only on compliant solutions.
\n
Hi, did you make any modification to your dataset or notebook after the original deadline?
", "votes": 0, "replies": [] }, { "id": 979243, "author_name": "Matt", "author_url": "", "post_date": "2020-08-20T17:56:28.733000", "content": "Best thing to do is contact the compliance team here: https://www.kaggle.com/compliance
", "votes": 0, "replies": [] }, { "id": 986186, "author_name": "Vaibhav Mehrotra", "author_url": "", "post_date": "2020-08-26T09:20:33.483000", "content": "@qingyuanwang no modifications were made
\n@matthewmasters we already contacted them on 20/08/2020 with no response so far
I have the similar situation to yours and I got a very confusing response. see here
", "votes": 0, "replies": [] } ] }, { "id": 976495, "author_name": "Fares FOURATI", "author_url": "", "post_date": "2020-08-18T22:29:59.667000", "content": "A day to go, again and again!
", "votes": 5, "replies": [ { "id": 976498, "author_name": "Mohammad Zunaed", "author_url": "", "post_date": "2020-08-18T22:37:13.120000", "content": "As they are actively removing the yolo submissions(already removed majority of them), I think it is okay if they need one or two days more. Huge props to them for making the leaderboard compliant.
", "votes": 8, "replies": [] }, { "id": 976500, "author_name": "Fares FOURATI", "author_url": "", "post_date": "2020-08-18T22:41:54.387000", "content": "I really appreciate their hardwork to find & delete non-compliant submission. We should be all grateful!
", "votes": 6, "replies": [] }, { "id": 976503, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-18T22:48:56.827000", "content": "Yes they do an honorable and important job, and thoroughly as it seems, many teams have been removed. It must take its time, rules are rules, and will be a standard to bring before the next competition, a good signal value.
\nImpatience is the only thing one need to work with while they do thier work, in contrast it should be easier to deal with impatience, so be it.
Feel like Okarin from Stein's Gate, looking for the best PB score..
\nBut amazed by the kaggle teams hard work on keeping the competition fair. Thanks.
The host is great work for keeping fairness.(remove uncompliant solution).
\nI'm looking forward to finalizing Private LB.
Another one bites the dust, and now @tereka is #1 :)
", "votes": 2, "replies": [] }, { "id": 978060, "author_name": "tereka", "author_url": "", "post_date": "2020-08-19T22:43:38.550000", "content": "@alexanderliao Congrat getting solo gold and become grand master!
", "votes": 1, "replies": [] } ] }, { "id": 965315, "author_name": "wasupandceacar", "author_url": "", "post_date": "2020-08-10T14:39:53.167000", "content": "Rerun process has some problem because after rerun both public and private score of my submission changed. If only private test dataset is updated, it shouldn't do anything to public score.
\n\nSo I guess that the host simply reran the whole kernel so that kernel with PL will create a brand new checkpoint and result in a different public score. Another shakeup is coming.
", "votes": 5, "replies": [] }, { "id": 965234, "author_name": "Benben", "author_url": "", "post_date": "2020-08-10T13:39:43.903000", "content": "I can't understand what is going on
", "votes": 5, "replies": [ { "id": 965247, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-10T13:48:44.273000", "content": "How I see it: They are making a re-run with a new testset on your 2 submissions, starting with oldest, to many subs/team to start together. After the re-runs, when we have the final scores, the review will start.
", "votes": 6, "replies": [] } ] }, { "id": 976496, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-18T22:33:33.800000", "content": "A day to go again? And why can't one post images anymore in the comment field, need to post that \"crazy cow\"-gif animation right now, thats how I feel.
", "votes": 6, "replies": [] }, { "id": 974002, "author_name": "Kaushal Shah", "author_url": "", "post_date": "2020-08-17T16:51:52.133000", "content": "Again, only 7 hours to go! Let's hope all the Yolo submissions are removed!
", "votes": 5, "replies": [] }, { "id": 978124, "author_name": "raozhan", "author_url": "", "post_date": "2020-08-20T01:04:51.253000", "content": "I am honored to be able to participate in this competition. I am very grateful to the staff for this competition, because this is my first time participating in this competition. I am very confused about this result. My result has been deleted. The notebook I refer to is open source. , And I modified it, I used eff_d5, and 10 epoch pesudo-lebaling, img_size=512, I want to know why my results are invalid。 @philculliton
Best thing to do is contact the compliance team here: https://www.kaggle.com/compliance
", "votes": 0, "replies": [] } ] }, { "id": 965990, "author_name": "CoooolerData", "author_url": "", "post_date": "2020-08-11T03:20:02.460000", "content": "if rerun all the kernels, we should also select the submissions again. otherwise it's totally unfair! The selections are chosen based on the LB feedback.
", "votes": 5, "replies": [] }, { "id": 965978, "author_name": "sm047", "author_url": "", "post_date": "2020-08-11T02:58:46.697000", "content": "I noticed that our public testset score changed after rerunning the whole kernel. Since we choose the submission kernel according to public LB score, the competition holder should rerun all the kernel and let us re-choose the submission again.
", "votes": 4, "replies": [] }, { "id": 978184, "author_name": "StoneWST98", "author_url": "", "post_date": "2020-08-20T02:36:18.677000", "content": "Hi Phil, thanks for such huge workload!
\nIs the current private leaderboard the final one? Will there be any changes？
I see movemement in the list, OK folks time to hold on to something, here we go.
", "votes": 3, "replies": [] }, { "id": 965463, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-10T16:50:28.527000", "content": "Anyone got a green clock next to Succeeded? Maybe it's worth knowing.\n
Yeah, I am having the same red tick @kirderf :|
", "votes": 0, "replies": [] } ] }, { "id": 976212, "author_name": "SADMAN ARAF", "author_url": "", "post_date": "2020-08-18T18:06:16.343000", "content": "I guess they are removing non compliant submissions as I see myself ranking up xD
", "votes": 2, "replies": [ { "id": 976222, "author_name": "Alexander Soare", "author_url": "", "post_date": "2020-08-18T18:17:40.233000", "content": "Yes! I've moved up nearly 500 places! haha
", "votes": 2, "replies": [ { "id": 976224, "author_name": "Rifat Rashid", "author_url": "", "post_date": "2020-08-18T18:21:29.657000", "content": "Cant wait to see :p how much I will drop in private LB :p hehehe. But good that Kaggle actually maintain a proper standard and removed all non-complaint solution
", "votes": 0, "replies": [] }, { "id": 976279, "author_name": "cooleel", "author_url": "", "post_date": "2020-08-18T19:02:06.867000", "content": "\n\nremoved all non-complaint solution
\n
At this moment, public lb is just what you picked as the final subs. Lots of kagglers decided to follow the competition rule and didn't pick the yolov5 solution. That seems the main reason for the lb change right now. So can't really tell. But I hope eventually all these non-compliant subs will be removed.
", "votes": 2, "replies": [] }, { "id": 976326, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-18T19:38:53.200000", "content": "Huzzah! #55. It'll be exciting to see the shakeup, maybe the people who were 500+ in the first private LB may even get a medal.
", "votes": 2, "replies": [] } ] }, { "id": 976245, "author_name": "", "author_url": "", "post_date": "2020-08-18T18:34:34.073000", "content": "", "votes": -1, "replies": [] }, { "id": 976350, "author_name": "Ryan Epp", "author_url": "", "post_date": "2020-08-18T19:58:54.073000", "content": "Oh wow, I went from 544 —> 85 on the public lb. Huge thanks to Kaggle & Hosts for making the effort to remove non compliant submissions.
", "votes": 3, "replies": [] }, { "id": 976679, "author_name": "bendang", "author_url": "", "post_date": "2020-08-19T03:13:43.873000", "content": "From 489 -> 36. It's funny that I thought this was Private LB because of the deadline. Welp! let's just wait and see for another shakeup with the new test dataset.
", "votes": 1, "replies": [] } ] }, { "id": 964816, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-10T07:35:45.023000", "content": "\n\nLeaderboard Update Is Underway
\n
\n Please let us know if you have questions!
Excellent, thank you, understand that it is a lot of work and takes time, and thank you for the quality of each competition, including this one.
\nAs for the questions:
\nI think there are many questions but many fewer than it seems because there are many shades of the same wonder.
\nIt would be great if we gathered all the questions and answers in this topic, so we make this more efficient for everyone.
@piantic two questions sums up a couple of hundred posts and comments, I add one to his list.
\n3) How many of the teams will be reviewed , all or where is the line drawn?
\nI would also recommend to save and post the questions with your answers at the top of this topic, will be more accessible and people can first try to find their answer there, like a Q&A bulletin board.
", "votes": 2, "replies": [] }, { "id": 965761, "author_name": "Phil Culliton", "author_url": "", "post_date": "2020-08-10T20:38:12.897000", "content": "Hi all. Thanks for your questions and comments. I am in the process of working through the re-run. Regarding the timeline, it will likely take until tomorrow to re-run all selected submissions. This is a process that requires some care and does take time. The error messages that many of you noted regarding submissions being processed after the deadline are due to a bug on our end, and will have no bearing on your final scores or submissions.
\nRe: questions about the leaderboard and scores, etc. The leaderboard will definitely continue changing until the process is complete.
\nThanks all for your patience!
", "votes": 1, "replies": [ { "id": 965762, "author_name": "Kalyan Kumar Pichuka", "author_url": "", "post_date": "2020-08-10T20:40:12.590000", "content": "@philculliton okay..what about the removal of YoloV5 submissions? I could see many people asking the same question about this. However I see no reply from the Kaggle Team. Kindly respond to avoid confusion.
", "votes": 8, "replies": [] }, { "id": 966239, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-11T09:05:38.750000", "content": "@kalyanpichuka , they wrote this:
\n\n\nWe are also performing our standard leaderboard finalization.
\n
So if they use the same standard as previous similar competitions they will review all teams in the leaderboard. I think we can rely and stick to that.
\nThen if they do it in parallel or after publishing new scores, I would suggest that they publish a final reviewed leaderboard, since it is removed now.
@philculliton when are you guys removing YoloV5 submissions from the leader board? Please ensure that those who are receiving medals have compliant submissions. Make it a fair competition.
", "votes": 1, "replies": [ { "id": 965706, "author_name": "Dezső Ribli", "author_url": "", "post_date": "2020-08-10T19:32:35.437000", "content": "I think they started removing those. Today there are only 2215 teams on the leaderboard compared to 2270 teams when I last checked.
", "votes": 2, "replies": [] }, { "id": 965715, "author_name": "Rifat Rashid", "author_url": "", "post_date": "2020-08-10T19:43:12.423000", "content": "I think they have some issues or is it me.
\nI don't even see the private leader board.
Haha no, its 2270 again, I guess there are some hiccups still :D
", "votes": 0, "replies": [] } ] }, { "id": 965755, "author_name": "MichaelP", "author_url": "", "post_date": "2020-08-10T20:27:40.270000", "content": "same - all I can see now is public leaderboard like when the competition hasn't finished and the private leaderboard has not yet been revealed.
", "votes": -1, "replies": [] } ] }, { "id": 965192, "author_name": "Rifat Rashid", "author_url": "", "post_date": "2020-08-10T13:01:38.647000", "content": "After this leader board update, are you going to remove YOLOv5 solutions?
", "votes": 1, "replies": [] }, { "id": 976557, "author_name": "m.kawakami", "author_url": "", "post_date": "2020-08-19T00:24:03.153000", "content": "Why is one of them an error when you chose two that are both fine for licensing?
\nIs it still a work in progress?
@mkawakami if your submission took over 6 hours?
", "votes": 1, "replies": [] } ] }, { "id": 969400, "author_name": "Kirderf", "author_url": "", "post_date": "2020-08-13T16:51:11.420000", "content": "A silent wonder, is the idea that the final LB should be presented when the countdown has passed the new deadline, or is it nothing to keep track of.
", "votes": 1, "replies": [] }, { "id": 965999, "author_name": "arutema47", "author_url": "", "post_date": "2020-08-11T03:36:51.017000", "content": "My guess is that Kaggle changed some data's image size to 1024x1024, which was their commitment.
\nTo fully mitigate that effect, kaggle decided to rerun the notebook to see fair evaluation results.
\nThis makes sense and should not bring large shakes.
If that is the case, I hope they don’t forget to rescale the ground truth bounding boxes to 1024x1024 also, as many of us rescale the bounding boxes according to image sizes, not 1024.
", "votes": 1, "replies": [] } ] }, { "id": 964860, "author_name": "", "author_url": "", "post_date": "2020-08-10T08:18:01.053000", "content": "", "votes": 1, "replies": [ { "id": 964920, "author_name": "", "author_url": "", "post_date": "2020-08-10T09:09:50.250000", "content": "", "votes": 0, "replies": [] }, { "id": 964949, "author_name": "", "author_url": "", "post_date": "2020-08-10T09:40:25.300000", "content": "", "votes": 0, "replies": [] }, { "id": 964956, "author_name": "", "author_url": "", "post_date": "2020-08-10T09:43:56.433000", "content": "", "votes": 3, "replies": [] }, { "id": 964975, "author_name": "", "author_url": "", "post_date": "2020-08-10T09:55:34.663000", "content": "", "votes": 0, "replies": [] }, { "id": 964978, "author_name": "", "author_url": "", "post_date": "2020-08-10T09:57:32.233000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 970463, "author_name": "", "author_url": "", "post_date": "2020-08-14T13:32:14.737000", "content": "", "votes": 2, "replies": [] }, { "id": 965398, "author_name": "", "author_url": "", "post_date": "2020-08-10T15:46:48.403000", "content": "", "votes": 2, "replies": [ { "id": 965404, "author_name": "", "author_url": "", "post_date": "2020-08-10T15:50:26.850000", "content": "", "votes": 1, "replies": [] }, { "id": 965411, "author_name": "", "author_url": "", "post_date": "2020-08-10T15:53:48.720000", "content": "", "votes": 1, "replies": [] }, { "id": 965421, "author_name": "", "author_url": "", "post_date": "2020-08-10T15:59:37.843000", "content": "", "votes": 8, "replies": [] }, { "id": 965424, "author_name": "", "author_url": "", "post_date": "2020-08-10T16:01:52", "content": "", "votes": 0, "replies": [] }, { "id": 965425, "author_name": "", "author_url": "", "post_date": "2020-08-10T16:05:10.400000", "content": "", "votes": 4, "replies": [] }, { "id": 965573, "author_name": "", "author_url": "", "post_date": "2020-08-10T18:02:03.557000", "content": "", "votes": 12, "replies": [] }, { "id": 965645, "author_name": "", "author_url": "", "post_date": "2020-08-10T18:33:55.800000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 966249, "author_name": "", "author_url": "", "post_date": "2020-08-11T09:23:22.527000", "content": "", "votes": 1, "replies": [] }, { "id": 977086, "author_name": "", "author_url": "", "post_date": "2020-08-19T09:44:04.497000", "content": "", "votes": 0, "replies": [ { "id": 977139, "author_name": "", "author_url": "", "post_date": "2020-08-19T10:12:29.043000", "content": "", "votes": 2, "replies": [] }, { "id": 977166, "author_name": "", "author_url": "", "post_date": "2020-08-19T10:31:05.967000", "content": "", "votes": 0, "replies": [] }, { "id": 977433, "author_name": "", "author_url": "", "post_date": "2020-08-19T13:24:27.150000", "content": "", "votes": 1, "replies": [] }, { "id": 977607, "author_name": "", "author_url": "", "post_date": "2020-08-19T15:39:07.783000", "content": "", "votes": 9, "replies": [] }, { "id": 978055, "author_name": "", "author_url": "", "post_date": "2020-08-19T22:39:17.580000", "content": "", "votes": 0, "replies": [] }, { "id": 978172, "author_name": "", "author_url": "", "post_date": "2020-08-20T02:16:13.760000", "content": "", "votes": 2, "replies": [] } ] }, { "id": 970122, "author_name": "", "author_url": "", "post_date": "2020-08-14T08:12:52.053000", "content": "", "votes": -5, "replies": [ { "id": 970289, "author_name": "", "author_url": "", "post_date": "2020-08-14T10:16:17.167000", "content": "", "votes": 1, "replies": [] }, { "id": 970290, "author_name": "", "author_url": "", "post_date": "2020-08-14T10:19:03.043000", "content": "", "votes": 5, "replies": [] } ] }, { "id": 965037, "author_name": "", "author_url": "", "post_date": "2020-08-10T10:43:15.320000", "content": "", "votes": -8, "replies": [] }, { "id": 978761, "author_name": "", "author_url": "", "post_date": "2020-08-20T11:14:49.840000", "content": "", "votes": -4, "replies": [ { "id": 979241, "author_name": "", "author_url": "", "post_date": "2020-08-20T17:55:38.150000", "content": "", "votes": 1, "replies": [] }, { "id": 979509, "author_name": "", "author_url": "", "post_date": "2020-08-20T22:48:51.247000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 978740, "author_name": "", "author_url": "", "post_date": "2020-08-20T11:04:00.987000", "content": "", "votes": -2, "replies": [] }, { "id": 972045, "author_name": "", "author_url": "", "post_date": "2020-08-16T07:31:15.150000", "content": "", "votes": -1, "replies": [] }, { "id": 965867, "author_name": "", "author_url": "", "post_date": "2020-08-10T23:09:46.180000", "content": "", "votes": -7, "replies": [] }, { "id": 1755999, "author_name": "", "author_url": "", "post_date": "2022-04-15T06:23:37.940000", "content": "", "votes": 0, "replies": [] }, { "id": 981867, "author_name": "", "author_url": "", "post_date": "2020-08-22T19:03:27.377000", "content": "", "votes": 0, "replies": [] }, { "id": 978681, "author_name": "", "author_url": "", "post_date": "2020-08-20T10:16:54.547000", "content": "", "votes": 0, "replies": [] }, { "id": 978281, "author_name": "", "author_url": "", "post_date": "2020-08-20T04:38:49.890000", "content": "", "votes": 0, "replies": [] }, { "id": 977060, "author_name": "", "author_url": "", "post_date": "2020-08-19T09:25:41.340000", "content": "", "votes": 0, "replies": [] }, { "id": 977049, "author_name": "", "author_url": "", "post_date": "2020-08-19T09:04:47.550000", "content": "", "votes": 0, "replies": [ { "id": 977062, "author_name": "", "author_url": "", "post_date": "2020-08-19T09:26:04.210000", "content": "", "votes": 2, "replies": [] } ] }, { "id": 976818, "author_name": "", "author_url": "", "post_date": "2020-08-19T05:40:23.883000", "content": "", "votes": 0, "replies": [] }, { "id": 976347, "author_name": "", "author_url": "", "post_date": "2020-08-18T19:57:09.547000", "content": "", "votes": 0, "replies": [ { "id": 976419, "author_name": "", "author_url": "", "post_date": "2020-08-18T20:48:04.753000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 972147, "author_name": "", "author_url": "", "post_date": "2020-08-16T09:11:58.380000", "content": "", "votes": 0, "replies": [] }, { "id": 965886, "author_name": "", "author_url": "", "post_date": "2020-08-11T00:03:28.410000", "content": "", "votes": 0, "replies": [ { "id": 965934, "author_name": "", "author_url": "", "post_date": "2020-08-11T01:51:57.820000", "content": "", "votes": 24, "replies": [] }, { "id": 965947, "author_name": "", "author_url": "", "post_date": "2020-08-11T02:08:13.623000", "content": "", "votes": 7, "replies": [] }, { "id": 966012, "author_name": "", "author_url": "", "post_date": "2020-08-11T03:49:33.533000", "content": "", "votes": 1, "replies": [] }, { "id": 966095, "author_name": "", "author_url": "", "post_date": "2020-08-11T06:05:39.393000", "content": "", "votes": 7, "replies": [] }, { "id": 966118, "author_name": "", "author_url": "", "post_date": "2020-08-11T06:42:17.107000", "content": "", "votes": 0, "replies": [] }, { "id": 966121, "author_name": "", "author_url": "", "post_date": "2020-08-11T06:45:07.463000", "content": "", "votes": 7, "replies": [] } ] }, { "id": 964947, "author_name": "", "author_url": "", "post_date": "2020-08-10T09:33:30.023000", "content": "", "votes": 0, "replies": [ { "id": 964953, "author_name": "", "author_url": "", "post_date": "2020-08-10T09:43:00.493000", "content": "", "votes": -1, "replies": [] }, { "id": 964961, "author_name": "", "author_url": "", "post_date": "2020-08-10T09:47:59.173000", "content": "", "votes": 0, "replies": [] }, { "id": 964973, "author_name": "", "author_url": "", "post_date": "2020-08-10T09:55:00.363000", "content": "", "votes": 1, "replies": [] }, { "id": 964984, "author_name": "", "author_url": "", "post_date": "2020-08-10T09:59:19.887000", "content": "", "votes": 2, "replies": [] }, { "id": 964997, "author_name": "", "author_url": "", "post_date": "2020-08-10T10:10:08.217000", "content": "", "votes": 1, "replies": [] }, { "id": 965038, "author_name": "", "author_url": "", "post_date": "2020-08-10T10:45:42.377000", "content": "", "votes": 0, "replies": [] }, { "id": 965054, "author_name": "", "author_url": "", "post_date": "2020-08-10T11:01:31.363000", "content": "", "votes": 1, "replies": [] }, { "id": 965065, "author_name": "", "author_url": "", "post_date": "2020-08-10T11:10:31.980000", "content": "", "votes": 0, "replies": [] }, { "id": 965068, "author_name": "", "author_url": "", "post_date": "2020-08-10T11:13:38.617000", "content": "", "votes": 0, "replies": [] }, { "id": 965072, "author_name": "", "author_url": "", "post_date": "2020-08-10T11:17:33.763000", "content": "", "votes": 0, "replies": [] }, { "id": 965073, "author_name": "", "author_url": "", "post_date": "2020-08-10T11:18:06.253000", "content": "", "votes": 0, "replies": [] }, { "id": 965074, "author_name": "", "author_url": "", "post_date": "2020-08-10T11:18:51.540000", "content": "", "votes": 0, "replies": [] }, { "id": 965076, "author_name": "", "author_url": "", "post_date": "2020-08-10T11:20:10.317000", "content": "", "votes": 1, "replies": [] }, { "id": 965093, "author_name": "", "author_url": "", "post_date": "2020-08-10T11:30:31.537000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 964522, "author_name": "", "author_url": "", "post_date": "2020-08-10T01:12:15.753000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "964578": "Hi, @philculliton.\nI'm happy to hear that private lb will be updated again.\n\nMy question is two:\n1) May I ask that will yolov5(or other package of license issues) be removed in re-run private lb?\n\n2) How long will it take approximately to complete all process?\n\nI would be very happy if you respond and also other competitors in this competition would be happy.\nThanks!\n", "976556": "This is by far the worst managed competition I have ever had the chance to participate in. Nothing, nothing gets even close to this. Promises being repeatedly broken and the unsatisfactory explanation as to why this is even happening !!! ", "974322": "Hi all.\n\nWe had some additional checks we needed to do today, and ran out of staff time to do the final database update (which is not a small or risk-less thing). We will do so tomorrow and finalize the LB. I'm sorry for how long this is taking. The number of moving parts are considerably higher than a normal competition close, and we're trying to do the best job we can.\n\nRe: questions - yes, we will be opening late submissions following the finalization of the LB.\n\nThanks again, all. We really appreciate your patience.", "964463": "Hi all! Thank you for all of your great work on this competition.\n\nAs Ian noted, we are in the process of updating the leaderboard using the updated private test set. Your selected submissions are being re-run against the updated set; private scores *will* change as this takes place. We are also performing our standard leaderboard finalization. This will take some time and we will notify you when the process is complete.\n\nPlease let us know if you have questions!", "977892": "Hi all!\n\nAs some of you have noted, the leaderboard has changed significantly. We have done our best to remove submissions that contained YOLOv5. We have also performed our normal post-competition process that has removed some other teams.\n\nIf you believe your submission (or team) was incorrectly removed, please contact [kaggle.com/compliance](https://www.kaggle.com/compliance) - this is our normal process for resolving disputes about team removals, submission invalidations, etc. I appreciate the Kagglers who have reached out to me directly, but if you follow the official route we can ensure your case is looked at properly and in a timely fashion.\n\nI am, once again, sorry that this took so long. It was detailed work that required a great deal of time from several of us. We appreciate your patience and the support you've shown. Finally, congratulations to the winners - I know the hosts are excited to examine your solutions!", "970878": "Hi all!\n\nWe're still updating the leaderboard. As you might imagine, sorting this all out has been an involved process. We appreciate your patience as we continue to finalize things. We are shooting to have everything done and finalized on Monday.", "965825": "i am confused ^^\n\n\n\n*Kaggle knows how to make me smile...*", "975543": "This competition is one of the worst organised competitions on kaggle.\n", "976548": "> @tanulsingh077, I think the right answer is yes all teams with Yolov5 submission chosen as one of their final 2 subs will be removed from the LB. I do not understand why people are confused. It is not hard to write a simple script that goes through 4000+ submissions to check for Yolov5 models hence I believe Kaggle will do it. Heck if this were posted as Kaggle challenge, many teams would score 99.99 accuracy within the 1st week. Just my 2 cents :-)\n\nI posted the above 24 days ago in the [Leaderboard of Compliant Submissions](https://www.kaggle.com/c/global-wheat-detection/discussion/166793) where people were still asking about YOLOV5 submission. The thread I responded to was asking specific questions one of which was\n> If a team uses yolo will it be removed from the lb?\n\nThis is after the question have been answered by the Kaggle team be it indirectly for example by @philculliton stating \n\n> you cannot win prizes if you use YOLOv5. \n\nI have seen the question answered by others as well but people kept asking over and over again in the discussion section.\n\nSo kudos to the Kaggle team for doing what I thought they would because it simply makes perfect sense. I know LB is not finished updating but it is now down to 1815 teams from 2270. Thanks @philculliton and team.", "970659": "Only 7 hours to go! Let's hope all the Yolo submissions are removed!", "978562": "Hey @philculliton,\nMe and my teammate appreciate the effort put into removing the Yolov5 submissions. \n\nHaving said that, we cannot find ourselves on the leaderboard. Both of our submissions seem to have been invalidated with a `failed final private rerun` error message. We have used the MIT-compliant EfficientDet for both submissions with different ensembling methods. As of yesterday (19/08/2020), we could find our submissions on the leaderboard but they have been removed since then. We are also sure that the runtime of our models is <6h because we have made multiple submissions (80+) before our final submission. \n\nWe'd appreciate your help in the matter because like many other participants here, we have put in multiple months of effort and worked only on compliant solutions. \n\n", "976495": "A day to go, again and again!", "965315": "Rerun process has some problem because after rerun both public and private score of my submission changed. If only private test dataset is updated, it shouldn't do anything to public score.\n\nSo I guess that the host simply reran the whole kernel so that kernel with PL will create a brand new checkpoint and result in a different public score. Another shakeup is coming.", "965234": "I can't understand what is going on", "976496": "A day to go again? And why can't one post images anymore in the comment field, need to post that \"crazy cow\"-gif animation right now, thats how I feel.", "974002": "Again, only 7 hours to go! Let's hope all the Yolo submissions are removed!", "978124": "I am honored to be able to participate in this competition. I am very grateful to the staff for this competition, because this is my first time participating in this competition. I am very confused about this result. My result has been deleted. The notebook I refer to is open source. , And I modified it, I used eff_d5, and 10 epoch pesudo-lebaling, img_size=512, I want to know why my results are invalid。 @philculliton ", "965990": "if rerun all the kernels, we should also select the submissions again. otherwise it's totally unfair! The selections are chosen based on the LB feedback.", "965978": "I noticed that our public testset score changed after rerunning the whole kernel. Since we choose the submission kernel according to public LB score, the competition holder should rerun all the kernel and let us re-choose the submission again.", "978184": "Hi Phil, thanks for such huge workload!\nIs the current private leaderboard the final one? Will there be any changes？", "976293": "I see movemement in the list, OK folks time to hold on to something, here we go.", "965463": "Anyone got a green clock next to Succeeded? Maybe it's worth knowing.\n\n", "976212": "I guess they are removing non compliant submissions as I see myself ranking up xD ", "964816": "> Leaderboard Update Is Underway\n> Please let us know if you have questions!\n\nExcellent, thank you, understand that it is a lot of work and takes time, and thank you for the quality of each competition, including this one.\n\nAs for the questions:\nI think there are many questions but many fewer than it seems because there are many shades of the same wonder.\nIt would be great if we gathered all the questions and answers in this topic, so we make this more efficient for everyone.\n\n@piantic two questions sums up a couple of hundred posts and comments, I add one to his list.\n\n3) How many of the teams will be reviewed , all or where is the line drawn?\n\nI would also recommend to save and post the questions with your answers at the top of this topic, will be more accessible and people can first try to find their answer there, like a Q&A bulletin board.", "965761": "Hi all. Thanks for your questions and comments. I am in the process of working through the re-run. Regarding the timeline, it will likely take until tomorrow to re-run all selected submissions. This is a process that requires some care and does take time. The error messages that many of you noted regarding submissions being processed after the deadline are due to a bug on our end, and will have no bearing on your final scores or submissions.\n\nRe: questions about the leaderboard and scores, etc. The leaderboard will definitely continue changing until the process is complete.\n\nThanks all for your patience!", "965594": "@philculliton when are you guys removing YoloV5 submissions from the leader board? Please ensure that those who are receiving medals have compliant submissions. Make it a fair competition.", "965192": "After this leader board update, are you going to remove YOLOv5 solutions?", "976557": "Why is one of them an error when you chose two that are both fine for licensing?\nIs it still a work in progress?", "969400": "A silent wonder, is the idea that the final LB should be presented when the countdown has passed the new deadline, or is it nothing to keep track of.", "965999": "My guess is that Kaggle changed some data's image size to 1024x1024, which was their commitment.\nTo fully mitigate that effect, kaggle decided to rerun the notebook to see fair evaluation results. \nThis makes sense and should not bring large shakes.", "964860": "Thank you for your efforts to ensure the fairness of the compete！👍\nI have two questions. \n1. Are both selected submissions rerun or one of them?\n2. Why does the score on the Private Leaderboard not match any score on My Submissions?", "970463": "What are the modified things? \nI think...\n\n1.image size modified 1024px.(with bbox?)\n2.modify bounding box(training and testing are differences)", "965398": "What's happen?\n\n\n\n", "966249": "May I ask if non-compliant submissions will be removed?", "977086": "I don't know which library I used is prohibited, and these are my two submissions [yolov4 + effdd7](https://www.kaggle.com/marcowu/ap2-0effd-yolov4-pseudo-labeling-oof-evalu/notebook?select=train.py), hope someone can help me.", "970122": "Hey you all, I need help on this project. Actually I have choose it as my project in my college and want to work on it. I need guidance of someone step by step like an instructor. Is here anyone who can help me? Please", "965037": "Why update scor after publishing winners? makes no sesne. ", "978761": "Hi Phil,\n\nAll the submission model are trained using my computer and google colab, and I only shared my trained models to my teammembers for testing and anaylsis.\n\nThe submitted results is using efficientd5, and it is based on a open notebook of this competition. one of the notebook, I refered is https://www.kaggle.com/shonenkov/training-efficientdet.I did huge study on data augmentation, trainning strategy, and you can see it from my submission history.\nI quite confused why there are so many error in my submission result, except two of them are acceptable.\n\nThanks a lot for your time on investigation in advance,", "978740": "All the submission model are trained using my computer and google colab, and I only shared my trained models to my teammembers for testing and anaylsis.\n\nThe submitted results is using efficientd5, and it is based on a open notebook of this competition. one of the notebook, I refered is [Alex Shonenkov](https://www.kaggle.com/shonenkov/training-efficientdet). I did huge study on data augmentation, trainning strategy, and you can see it from my submission history.\nI quite confused why there are so many error in my submission result, except two of them are acceptable.\n\nThanks a lot for your time on investigation in advance,\n\n\nJay", "972045": "@philculliton \n\nHi there! Just a quick suggestion that if all the kernels are rerun, it is more reasonable to let us re-select our submission entries. This is because there are random factors inside pseudo-label training, and we usually select the final entries based on our public scores, right?", "965867": "Hi，why I cannot see the private leaderboard?", "1755999": "Great job!", "981867": "Well done everybody!", "978681": "The last night i have droped 400, cause i overfitted . ", "978281": "Thanks @philculliton!\n\nAnd it is really great job. \nWe are so happy that all of this is back to good.\n\nThanks again Kaggle Team!", "977060": "Appreciate Kaggle team. Even though they didn't respond to many questions which were posted, they ensured that they will remove all non compliant submissions. 15 hours to go.. I am sure it would be a compliant leaderboard.. ", "977049": "I see a big drop in the number of teams today..what happened? :0", "976818": "The chosen one is not the best one, could you tell me why? thanks very much for your help.", "976347": "Woah what is happening guys??", "972147": "waiting to see the result ", "965886": "Thanks for the update. Why is my public score different now? I understand that the private LB is still being processed and adjusted for but why would that change my score for the other dataset?", "964947": "Hey, can anyone tell me why are my submissions showing \"submission after deadline\"? I had both my submissions done a long time before the deadline", "964522": "" } }