\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5221270%2Fe6ccdaff4c6910c7707488806a118eb7%2FScreenshot%20from%202020-08-10%2017-57-54.png?generation=1597245294094956&alt=media\")
Hi everyone,\nI was just going through the discussions and found that many people are facing the issue of 'submission after deadline'. I just have some questions if you could answer.\n- Whoever faced this issue can you guys please share whether your score increased/decreased.\n- Is the public score changing too?\n- Is the change taking place only for the 2 selected submissions or all the subs?
\n\nAlso, if you guys can add approximately how much time it takes to get the updated score it could be helpful..
\n\nThank you.
", "messages": [ { "id": 965300, "postDate": "2020-08-10T14:30:44.930Z", "content": "our score dropped from 0.7031 to 0.6576 which is impossible with the minor change in the private labels. I hope this is because of \"SUBMITTED AFTER DEADLINE\" issue
", "rawMarkdown": "our score dropped from 0.7031 to 0.6576 which is impossible with the minor change in the private labels. I hope this is because of \"SUBMITTED AFTER DEADLINE\" issue", "votes": 6, "replies": [ { "id": 965309, "postDate": "2020-08-10T14:36:05.800Z", "content": "đŸ˜® that is a huge drop...\nWish you luck for the second one...
", "rawMarkdown": "đŸ˜® that is a huge drop...\nWish you luck for the second one...", "votes": 1 }, { "id": 965349, "postDate": "2020-08-10T15:12:54Z", "content": "Mine went from 0.66 to 0.63 :(
", "rawMarkdown": "Mine went from 0.66 to 0.63 :(" }, { "id": 965449, "postDate": "2020-08-10T16:39:23.920Z", "content": "Hi @nvnnghia , \nIf I am not wrong your 2nd sub evaluation just finished some time ago and the sub which scored highest on your public LB its corresponding private score is reflecting on the private LB
", "rawMarkdown": "Hi @nvnnghia , \nIf I am not wrong your 2nd sub evaluation just finished some time ago and the sub which scored highest on your public LB its corresponding private score is reflecting on the private LB" }, { "id": 965903, "postDate": "2020-08-11T00:40:50.213Z", "content": "@nvnnghia
\nWe saw the same in our team (0.05 drop). Let's hope that this is a bug.
\nIf we have cleaned the labels, the data should be easier to infer and PB should improve (at least for half people).
\nSince we see only score drop reported, I think the organizer implemented a major bug and rerunning because of it.
Hi everyone,\nI was just going through the discussions and found that many people are facing the issue of 'submission after deadline'. I just have some questions if you could answer.\n- Whoever faced this issue can you guys please share whether your score increased/decreased.\n- Is the public score changing too?\n- Is the change taking place only for the 2 selected submissions or all the subs?
\n\nAlso, if you guys can add approximately how much time it takes to get the updated score it could be helpful..
\n\nThank you.
", "rawMarkdown": "Hi everyone,\nI was just going through the discussions and found that many people are facing the issue of 'submission after deadline'. I just have some questions if you could answer.\n- Whoever faced this issue can you guys please share whether your score increased/decreased.\n- Is the public score changing too?\n- Is the change taking place only for the 2 selected submissions or all the subs?\n\nAlso, if you guys can add approximately how much time it takes to get the updated score it could be helpful..\n\nThank you.", "votes": 4 }, { "id": 967566, "postDate": "2020-08-12T10:43:29.683Z", "content": "As a fun note, my submission marked as submitted after deadline no longer is ! Is it the same for you guys ?
", "rawMarkdown": "As a fun note, my submission marked as submitted after deadline no longer is ! Is it the same for you guys ?", "votes": 1, "replies": [ { "id": 967587, "postDate": "2020-08-12T11:00:52.770Z", "content": "Yes, because they reactivate the competition timeline to deal with the deadline issue.
", "rawMarkdown": "Yes, because they reactivate the competition timeline to deal with the deadline issue." }, { "id": 967625, "postDate": "2020-08-12T11:42:17.123Z", "content": "I think they were still marked as such despite having the competition reopened so I was genuinely confused ! But I may be wrong !
", "rawMarkdown": "I think they were still marked as such despite having the competition reopened so I was genuinely confused ! But I may be wrong !" }, { "id": 967902, "postDate": "2020-08-12T15:15:24.280Z", "content": "Classic Kaggle, submission after deadline while the competition is still running! /s
\n
Only one of my kernel is marked as such which was a YoloV5 submission !
\nHuge score decrease as well of around 0.05 on the private test set and of 0.03 on the public test set.
\nIt was my second highest kernel and was probably selected automatically for submission as all my notebooks were YoloV5, due to lack of time for developing compliant solutions…
", "rawMarkdown": "Only one of my kernel is marked as such which was a YoloV5 submission ! \n\nHuge score decrease as well of around 0.05 on the private test set and of 0.03 on the public test set. \n\nIt was my second highest kernel and was probably selected automatically for submission as all my notebooks were YoloV5, due to lack of time for developing compliant solutions... ", "votes": 1, "replies": [ { "id": 965365, "postDate": "2020-08-10T15:24:07.207Z", "content": "Thanks @dimartinot, \nI am not understanding why is there a change in public LB score if they updated the private data set.
", "rawMarkdown": "Thanks @dimartinot, \nI am not understanding why is there a change in public LB score if they updated the private data set. \n", "votes": 1 }, { "id": 965371, "postDate": "2020-08-10T15:28:29.573Z", "content": "@nitindatta , yes my thought exactly. Private score may increase or decrease, but why is the public score changing as only private dataset has been updated? i wonder if there is any problem associated with the rerun process.
", "rawMarkdown": "@nitindatta , yes my thought exactly. Private score may increase or decrease, but why is the public score changing as only private dataset has been updated? i wonder if there is any problem associated with the rerun process. ", "votes": 1 }, { "id": 965377, "postDate": "2020-08-10T15:31:59.777Z", "content": "We have to wait for the hosts and kaggle team's response
", "rawMarkdown": "We have to wait for the hosts and kaggle team's response", "votes": 1 } ] }, { "id": 965381, "postDate": "2020-08-10T15:36:29.497Z", "content": "Our two submissions just got updated. Was 0.6886, now 0.6560. Quite the big drop in private LB right? Nope, after our private LB dropped, we went up from 500th to 200th. I'm so confused.
", "rawMarkdown": "Our two submissions just got updated. Was 0.6886, now 0.6560. Quite the big drop in private LB right? Nope, after our private LB dropped, we went up from 500th to 200th. I'm so confused. ", "votes": 2, "replies": [ { "id": 965385, "postDate": "2020-08-10T15:39:12.003Z", "content": "I think I finally figured out what's happening with the private LB (this is just speculation). After you are re-evaluated, your submissions become \"past the deadline\" and are no longer counted towards your private LB. Then, your highest private LB submission is chosen instead?
", "rawMarkdown": "I think I finally figured out what's happening with the private LB (this is just speculation). After you are re-evaluated, your submissions become \"past the deadline\" and are no longer counted towards your private LB. Then, your highest private LB submission is chosen instead? ", "votes": 3 }, { "id": 965394, "postDate": "2020-08-10T15:44:21.210Z", "content": "Whaaaat?\nI am so confused...
", "rawMarkdown": "Whaaaat?\nI am so confused... ", "votes": 1 }, { "id": 965396, "postDate": "2020-08-10T15:46:35.390Z", "content": "@stanleyjzheng if you say after re-running your highest private LB is chosen then dont you think there should be drop in the rank (as you have a drop in both your scores)
", "rawMarkdown": "@stanleyjzheng if you say after re-running your highest private LB is chosen then dont you think there should be drop in the rank (as you have a drop in both your scores)", "votes": 1 }, { "id": 965433, "postDate": "2020-08-10T16:16:11.660Z", "rawMarkdown": "", "isDeleted": true }, { "id": 965435, "postDate": "2020-08-10T16:18:05.140Z", "content": "I think it's not the highest private lb score, it's the one with highest pub lb score. but it's just for now. Can't image what the final lb looks like....
", "rawMarkdown": "I think it's not the highest private lb score, it's the one with highest pub lb score. but it's just for now. Can't image what the final lb looks like....", "votes": 3 }, { "id": 965438, "postDate": "2020-08-10T16:20:40.443Z", "content": "Wow... You are completely right. Since our 2 final submissions were \"after deadline\", our 2 highest public LB scores were chosen, and #2 is our private LB score since its private LB is higher. This is so odd, the scores on the leaderboard right now are completely worthless, and I expect to see another massive shakeup.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F5221270%2F7ac1abdeabf0fde75f5cbf92a90505ac%2FScreenshot%20from%202020-08-10%2010-19-05.png?generation=1597076411827012&alt=media\")
For some reason this competition reactivated for me, saying it's \"4 days to go\". Private LB is hidden for me now.
", "rawMarkdown": "For some reason this competition reactivated for me, saying it's \"4 days to go\". Private LB is hidden for me now.", "votes": 1, "replies": [ { "id": 965744, "postDate": "2020-08-10T20:09:56.880Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 965391, "postDate": "2020-08-10T15:41:40.003Z", "content": "The two of my submissions have marked Submission After Deadline. In both case, the score has decreased badly. But, the LB score that showing right now is 0.6955 which is my YOLOv5 submission and I didn't select this. How can this be possible ?
", "rawMarkdown": "The two of my submissions have marked **Submission After Deadline**. In both case, the score has decreased badly. But, the LB score that showing right now is 0.6955 which is my YOLOv5 submission and I **didn't select** this. How can this be possible ? ", "votes": 1, "replies": [ { "id": 965455, "postDate": "2020-08-10T16:46:02.580Z", "content": "Since your two final submissions are \"Submission After Deadline\", your two highest public LB models were chosen, which is most likely yolov5. Just ignore it for now, I'm sure they'll fix this mess later.
", "rawMarkdown": "Since your two final submissions are \"Submission After Deadline\", your two highest public LB models were chosen, which is most likely yolov5. Just ignore it for now, I'm sure they'll fix this mess later. ", "votes": 2 }, { "id": 965462, "postDate": "2020-08-10T16:50:03.643Z", "content": "Yes, that seems the most likely answer. The modified Private LB showing right now (while the re-evaluation is going on) is of NO relevance.
", "rawMarkdown": "Yes, that seems the most likely answer. The modified Private LB showing right now (while the re-evaluation is going on) is of NO relevance.", "votes": 1 } ] }, { "id": 965437, "postDate": "2020-08-10T16:19:00.077Z", "content": "The public score still changed until finishing the verification step:\n\"This competition has completed. This leaderboard reflects preliminary final standings. The result will become final after the competition organizers verify the results.\"
", "rawMarkdown": "The public score still changed until finishing the verification step:\n\"This competition has completed. This leaderboard reflects preliminary final standings. The result will become final after the competition organizers verify the results.\" " } ], "comments": [ { "id": 965300, "author_name": "nvnn", "author_url": "", "post_date": "2020-08-10T14:30:44.930000", "content": "our score dropped from 0.7031 to 0.6576 which is impossible with the minor change in the private labels. I hope this is because of \"SUBMITTED AFTER DEADLINE\" issue
", "votes": 6, "replies": [ { "id": 965309, "author_name": "Nitin Datta", "author_url": "", "post_date": "2020-08-10T14:36:05.800000", "content": "đŸ˜® that is a huge drop...\nWish you luck for the second one...
", "votes": 1, "replies": [] }, { "id": 965349, "author_name": "mjsML", "author_url": "", "post_date": "2020-08-10T15:12:54", "content": "Mine went from 0.66 to 0.63 :(
", "votes": 0, "replies": [] }, { "id": 965449, "author_name": "Nitin Datta", "author_url": "", "post_date": "2020-08-10T16:39:23.920000", "content": "Hi @nvnnghia , \nIf I am not wrong your 2nd sub evaluation just finished some time ago and the sub which scored highest on your public LB its corresponding private score is reflecting on the private LB
", "votes": 0, "replies": [] }, { "id": 965903, "author_name": "arutema47", "author_url": "", "post_date": "2020-08-11T00:40:50.213000", "content": "@nvnnghia
\nWe saw the same in our team (0.05 drop). Let's hope that this is a bug.
\nIf we have cleaned the labels, the data should be easier to infer and PB should improve (at least for half people).
\nSince we see only score drop reported, I think the organizer implemented a major bug and rerunning because of it.
As a fun note, my submission marked as submitted after deadline no longer is ! Is it the same for you guys ?
", "votes": 1, "replies": [ { "id": 967587, "author_name": "Mohammad Zunaed", "author_url": "", "post_date": "2020-08-12T11:00:52.770000", "content": "Yes, because they reactivate the competition timeline to deal with the deadline issue.
", "votes": 0, "replies": [] }, { "id": 967625, "author_name": "Thomas Di Martino", "author_url": "", "post_date": "2020-08-12T11:42:17.123000", "content": "I think they were still marked as such despite having the competition reopened so I was genuinely confused ! But I may be wrong !
", "votes": 0, "replies": [] }, { "id": 967902, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-12T15:15:24.280000", "content": "Classic Kaggle, submission after deadline while the competition is still running! /s
\n
Only one of my kernel is marked as such which was a YoloV5 submission !
\nHuge score decrease as well of around 0.05 on the private test set and of 0.03 on the public test set.
\nIt was my second highest kernel and was probably selected automatically for submission as all my notebooks were YoloV5, due to lack of time for developing compliant solutions…
", "votes": 1, "replies": [ { "id": 965365, "author_name": "Nitin Datta", "author_url": "", "post_date": "2020-08-10T15:24:07.207000", "content": "Thanks @dimartinot, \nI am not understanding why is there a change in public LB score if they updated the private data set.
", "votes": 1, "replies": [] }, { "id": 965371, "author_name": "Mohammad Zunaed", "author_url": "", "post_date": "2020-08-10T15:28:29.573000", "content": "@nitindatta , yes my thought exactly. Private score may increase or decrease, but why is the public score changing as only private dataset has been updated? i wonder if there is any problem associated with the rerun process.
", "votes": 1, "replies": [] }, { "id": 965377, "author_name": "Nitin Datta", "author_url": "", "post_date": "2020-08-10T15:31:59.777000", "content": "We have to wait for the hosts and kaggle team's response
", "votes": 1, "replies": [] } ] }, { "id": 965381, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-10T15:36:29.497000", "content": "Our two submissions just got updated. Was 0.6886, now 0.6560. Quite the big drop in private LB right? Nope, after our private LB dropped, we went up from 500th to 200th. I'm so confused.
", "votes": 2, "replies": [ { "id": 965385, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-10T15:39:12.003000", "content": "I think I finally figured out what's happening with the private LB (this is just speculation). After you are re-evaluated, your submissions become \"past the deadline\" and are no longer counted towards your private LB. Then, your highest private LB submission is chosen instead?
", "votes": 3, "replies": [] }, { "id": 965394, "author_name": "Nitin Datta", "author_url": "", "post_date": "2020-08-10T15:44:21.210000", "content": "Whaaaat?\nI am so confused...
", "votes": 1, "replies": [] }, { "id": 965396, "author_name": "Nitin Datta", "author_url": "", "post_date": "2020-08-10T15:46:35.390000", "content": "@stanleyjzheng if you say after re-running your highest private LB is chosen then dont you think there should be drop in the rank (as you have a drop in both your scores)
", "votes": 1, "replies": [] }, { "id": 965433, "author_name": "", "author_url": "", "post_date": "2020-08-10T16:16:11.660000", "content": "", "votes": 0, "replies": [] }, { "id": 965435, "author_name": "cooleel", "author_url": "", "post_date": "2020-08-10T16:18:05.140000", "content": "I think it's not the highest private lb score, it's the one with highest pub lb score. but it's just for now. Can't image what the final lb looks like....
", "votes": 3, "replies": [] }, { "id": 965438, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-10T16:20:40.443000", "content": "Wow... You are completely right. Since our 2 final submissions were \"after deadline\", our 2 highest public LB scores were chosen, and #2 is our private LB score since its private LB is higher. This is so odd, the scores on the leaderboard right now are completely worthless, and I expect to see another massive shakeup.
\n\n
For some reason this competition reactivated for me, saying it's \"4 days to go\". Private LB is hidden for me now.
", "votes": 1, "replies": [ { "id": 965744, "author_name": "", "author_url": "", "post_date": "2020-08-10T20:09:56.880000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 965391, "author_name": "Md Fahim", "author_url": "", "post_date": "2020-08-10T15:41:40.003000", "content": "The two of my submissions have marked Submission After Deadline. In both case, the score has decreased badly. But, the LB score that showing right now is 0.6955 which is my YOLOv5 submission and I didn't select this. How can this be possible ?
", "votes": 1, "replies": [ { "id": 965455, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-08-10T16:46:02.580000", "content": "Since your two final submissions are \"Submission After Deadline\", your two highest public LB models were chosen, which is most likely yolov5. Just ignore it for now, I'm sure they'll fix this mess later.
", "votes": 2, "replies": [] }, { "id": 965462, "author_name": "Vee", "author_url": "", "post_date": "2020-08-10T16:50:03.643000", "content": "Yes, that seems the most likely answer. The modified Private LB showing right now (while the re-evaluation is going on) is of NO relevance.
", "votes": 1, "replies": [] } ] }, { "id": 965437, "author_name": "Haithem Hermassi", "author_url": "", "post_date": "2020-08-10T16:19:00.077000", "content": "The public score still changed until finishing the verification step:\n\"This competition has completed. This leaderboard reflects preliminary final standings. The result will become final after the competition organizers verify the results.\"
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "965300": "our score dropped from 0.7031 to 0.6576 which is impossible with the minor change in the private labels. I hope this is because of \"SUBMITTED AFTER DEADLINE\" issue", "965293": "Hi everyone,\nI was just going through the discussions and found that many people are facing the issue of 'submission after deadline'. I just have some questions if you could answer.\n- Whoever faced this issue can you guys please share whether your score increased/decreased.\n- Is the public score changing too?\n- Is the change taking place only for the 2 selected submissions or all the subs?\n\nAlso, if you guys can add approximately how much time it takes to get the updated score it could be helpful..\n\nThank you.", "967566": "As a fun note, my submission marked as submitted after deadline no longer is ! Is it the same for you guys ?", "965333": "Only one of my kernel is marked as such which was a YoloV5 submission ! \n\nHuge score decrease as well of around 0.05 on the private test set and of 0.03 on the public test set. \n\nIt was my second highest kernel and was probably selected automatically for submission as all my notebooks were YoloV5, due to lack of time for developing compliant solutions... ", "965381": "Our two submissions just got updated. Was 0.6886, now 0.6560. Quite the big drop in private LB right? Nope, after our private LB dropped, we went up from 500th to 200th. I'm so confused. ", "965718": "For some reason this competition reactivated for me, saying it's \"4 days to go\". Private LB is hidden for me now.", "965391": "The two of my submissions have marked **Submission After Deadline**. In both case, the score has decreased badly. But, the LB score that showing right now is 0.6955 which is my YOLOv5 submission and I **didn't select** this. How can this be possible ? ", "965437": "The public score still changed until finishing the verification step:\n\"This competition has completed. This leaderboard reflects preliminary final standings. The result will become final after the competition organizers verify the results.\" " } }