{ "id": 352846, "title": "Second Leaderboard Update", "url": "/competitions/AI4Code/discussion/352846", "author_name": "", "post_date": "2022-09-15T22:13:18.814566200Z", "votes": 19, "comment_count": 23, "views": 0, "content": "

\n

\n

2022-09-19: The second is published now. Please let us know if you suspect your submissions weren't scored correctly.

", "messages": [ { "id": "1941266", "postDate": "09/15/2022 22:13:18", "content": "

\n

\n

2022-09-19: The second is published now. Please let us know if you suspect your submissions weren't scored correctly.

", "rawMarkdown": "~~2022-09-16: It looks like our DB table for the submission rescores had gotten corrupted causing the selected submissions to not be rescored in some cases. We're going to pull the submission ids from the first update and run another rescore. I'll post the results after the weekend but wipe the leaderboard in the meantime so as not to leave it in an untrue state.~~\n\n~~Apologies for the confusion everyone and thank you for your patience while we get this sorted out. We will make sure everyone ends up with their correct score.~~\n\n2022-09-19: The second is published now. Please let us know if you suspect your submissions weren't scored correctly.", "votes": null }, { "id": "1942610", "postDate": "09/16/2022 19:15:45", "content": "

Is this the final status of the second rerun?
\nMy Submissions shows two second reruns but they are based on the same single original submission, and my new second reruns got a bit worse results than my first rerun but the current leaderboard for mine still shows the best result of the first rerun.

\n

Edit:
\nIt seems the submissions for the second rerun were automatically selected from two best submissions based on the Public LB score. I submitted the same model twice for debugging so my results of the second rerun show the same score.

", "rawMarkdown": "Is this the final status of the second rerun?\nMy Submissions shows two second reruns but they are based on the same single original submission, and my new second reruns got a bit worse results than my first rerun but the current leaderboard for mine still shows the best result of the first rerun.\n\nEdit:\nIt seems the submissions for the second rerun were automatically selected from two best submissions based on the Public LB score. I submitted the same model twice for debugging so my results of the second rerun show the same score.", "votes": null }, { "id": "1942654", "postDate": "09/16/2022 20:11:18", "content": "

Thanks for the heads up. The first rerun scores should have gotten wiped out. I'll leave these up for now and update shortly.

", "rawMarkdown": "Thanks for the heads up. The first rerun scores should have gotten wiped out. I'll leave these up for now and update shortly.", "votes": null }, { "id": "1942682", "postDate": "09/16/2022 21:05:03", "content": "

Is it finalized? only 917 teams are left?

", "rawMarkdown": "Is it finalized? only 917 teams are left?", "votes": null }, { "id": "1942685", "postDate": "09/16/2022 21:18:11", "content": "

Yes, these are the results of the second rerun.

\n

Reviewing the error log, it looks like the failures this time are almost all due to missing files from datasets.

", "rawMarkdown": "Yes, these are the results of the second rerun.\n\nReviewing the error log, it looks like the failures this time are almost all due to missing files from datasets.", "votes": null }, { "id": "1942686", "postDate": "09/16/2022 21:23:15", "content": "

same here，seems not rerun of the 2 my last choosen subs

", "rawMarkdown": "same here，seems not rerun of the 2 my last choosen subs", "votes": null }, { "id": "1942691", "postDate": "09/16/2022 21:31:28", "content": "

same here, the rerun is not the two selected subs.

", "rawMarkdown": "same here, the rerun is not the two selected subs.", "votes": null }, { "id": "1942694", "postDate": "09/16/2022 21:40:21", "content": "

I will take a look

", "rawMarkdown": "I will take a look", "votes": null }, { "id": "1942710", "postDate": "09/16/2022 22:20:45", "content": "

Thanks. In our case, the rerun is one of our selected sub and another sub. The other selected sub is marked as error. Unless mistaken of course.

", "rawMarkdown": "Thanks. In our case, the rerun is one of our selected sub and another sub. The other selected sub is marked as error. Unless mistaken of course.", "votes": null }, { "id": "1943346", "postDate": "09/17/2022 13:18:08", "content": "

@ryanholbrook Could you take a look to my submissions? They passed the 1st run but not sure why both failed in the second one (should have no missing files in the datasets I used, just checked again). Thanks.

", "rawMarkdown": "ryanholbrook Could you take a look to my submissions? They passed the 1st run but not sure why both failed in the second one (should have no missing files in the datasets I used, just checked again). Thanks.", "votes": null }, { "id": "1944387", "postDate": "09/18/2022 09:24:59", "content": "

this is good achivement

", "rawMarkdown": "this is good achivement", "votes": null }, { "id": "1944905", "postDate": "09/18/2022 16:55:54", "content": "

Same here.

", "rawMarkdown": "Same here.", "votes": null }, { "id": "1945046", "postDate": "09/18/2022 19:28:46", "content": "

Same here.

", "rawMarkdown": "Same here.", "votes": null }, { "id": "1945478", "postDate": "09/19/2022 07:01:39", "content": "

same here.

", "rawMarkdown": "same here.", "votes": null }, { "id": "1945591", "postDate": "09/19/2022 08:47:45", "content": "

Here too, passed the first following error this run: Clearing corrupted data and rescoring.

", "rawMarkdown": "Here too, passed the first following error this run: `Clearing corrupted data and rescoring.`", "votes": null }, { "id": "1945810", "postDate": "09/19/2022 12:19:53", "content": "

same here.

", "rawMarkdown": "same here.", "votes": null }, { "id": "1945844", "postDate": "09/19/2022 12:48:30", "content": "

@ollibolli please wait a few more hours for the new update

", "rawMarkdown": "ollibolli please wait a few more hours for the new update", "votes": null }, { "id": "1945912", "postDate": "09/19/2022 13:36:51", "content": "

I get back to the LB in the 2nd rerun, but they are not the two submissions I have selected. Is there something wrong or it's just not finished yet?

", "rawMarkdown": "I get back to the LB in the 2nd rerun, but they are not the two submissions I have selected. Is there something wrong or it's just not finished yet?", "votes": null }, { "id": "1945921", "postDate": "09/19/2022 13:44:58", "content": "

My bad, it is the 2 submissions I selected.

", "rawMarkdown": "My bad, it is the 2 submissions I selected.", "votes": null }, { "id": "1945935", "postDate": "09/19/2022 13:58:09", "content": "

My submissions just back to life again! Thanks for the rerun @ryanholbrook.

", "rawMarkdown": "My submissions just back to life again! Thanks for the rerun @ryanholbrook.", "votes": null }, { "id": "1946615", "postDate": "09/20/2022 00:47:37", "content": "

@hydantess You being #1 on the LB after this rerun is a great opportunity to share some details about your solution! 😂

", "rawMarkdown": "hydantess You being #1 on the LB after this rerun is a great opportunity to share some details about your solution! 😂", "votes": null }, { "id": "1946699", "postDate": "09/20/2022 04:05:07", "content": "

I will share my solution after the 5th update or last update.😄

", "rawMarkdown": "I will share my solution after the 5th update or last update.😄", "votes": null }, { "id": "1946938", "postDate": "09/20/2022 07:15:08", "content": "

Thanks. Now it seems to be fine.

", "rawMarkdown": "Thanks. Now it seems to be fine.", "votes": null }, { "id": "1948182", "postDate": "09/20/2022 22:24:26", "content": "

Congrats on the #1 place on the second re-run! Very interested to read your solution.

", "rawMarkdown": "Congrats on the #1 place on the second re-run! Very interested to read your solution.", "votes": null } ], "comments": [ { "id": 1942610, "author_name": "yufuin", "author_url": "", "post_date": "09/16/2022 19:15:45", "content": "

Is this the final status of the second rerun?
\nMy Submissions shows two second reruns but they are based on the same single original submission, and my new second reruns got a bit worse results than my first rerun but the current leaderboard for mine still shows the best result of the first rerun.

\n

Edit:
\nIt seems the submissions for the second rerun were automatically selected from two best submissions based on the Public LB score. I submitted the same model twice for debugging so my results of the second rerun show the same score.

", "votes": null, "replies": [ { "id": 1942654, "author_name": "ryanholbrook", "author_url": "", "post_date": "09/16/2022 20:11:18", "content": "

Thanks for the heads up. The first rerun scores should have gotten wiped out. I'll leave these up for now and update shortly.

", "votes": null, "replies": [] }, { "id": 1942686, "author_name": "goldenlock", "author_url": "", "post_date": "09/16/2022 21:23:15", "content": "

same here，seems not rerun of the 2 my last choosen subs

", "votes": null, "replies": [] }, { "id": 1942691, "author_name": "cpmpml", "author_url": "", "post_date": "09/16/2022 21:31:28", "content": "

same here, the rerun is not the two selected subs.

", "votes": null, "replies": [] }, { "id": 1942694, "author_name": "ryanholbrook", "author_url": "", "post_date": "09/16/2022 21:40:21", "content": "

I will take a look

", "votes": null, "replies": [] }, { "id": 1942710, "author_name": "cpmpml", "author_url": "", "post_date": "09/16/2022 22:20:45", "content": "

Thanks. In our case, the rerun is one of our selected sub and another sub. The other selected sub is marked as error. Unless mistaken of course.

", "votes": null, "replies": [] }, { "id": 1946938, "author_name": "yufuin", "author_url": "", "post_date": "09/20/2022 07:15:08", "content": "

Thanks. Now it seems to be fine.

", "votes": null, "replies": [] } ] }, { "id": 1942682, "author_name": "css919", "author_url": "", "post_date": "09/16/2022 21:05:03", "content": "

Is it finalized? only 917 teams are left?

", "votes": null, "replies": [ { "id": 1942685, "author_name": "ryanholbrook", "author_url": "", "post_date": "09/16/2022 21:18:11", "content": "

Yes, these are the results of the second rerun.

\n

Reviewing the error log, it looks like the failures this time are almost all due to missing files from datasets.

", "votes": null, "replies": [] } ] }, { "id": 1943346, "author_name": "markpeng", "author_url": "", "post_date": "09/17/2022 13:18:08", "content": "

@ryanholbrook Could you take a look to my submissions? They passed the 1st run but not sure why both failed in the second one (should have no missing files in the datasets I used, just checked again). Thanks.

", "votes": null, "replies": [ { "id": 1944905, "author_name": "cabbage972", "author_url": "", "post_date": "09/18/2022 16:55:54", "content": "

Same here.

", "votes": null, "replies": [] }, { "id": 1945046, "author_name": "sanghuynh1501", "author_url": "", "post_date": "09/18/2022 19:28:46", "content": "

Same here.

", "votes": null, "replies": [] }, { "id": 1945478, "author_name": "julesvanligtenberg", "author_url": "", "post_date": "09/19/2022 07:01:39", "content": "

same here.

", "votes": null, "replies": [] }, { "id": 1945591, "author_name": "ollibolli", "author_url": "", "post_date": "09/19/2022 08:47:45", "content": "

Here too, passed the first following error this run: Clearing corrupted data and rescoring.

", "votes": null, "replies": [] }, { "id": 1945810, "author_name": "yuanzhezhou", "author_url": "", "post_date": "09/19/2022 12:19:53", "content": "

same here.

", "votes": null, "replies": [] }, { "id": 1945844, "author_name": "goldenlock", "author_url": "", "post_date": "09/19/2022 12:48:30", "content": "

@ollibolli please wait a few more hours for the new update

", "votes": null, "replies": [] }, { "id": 1945935, "author_name": "markpeng", "author_url": "", "post_date": "09/19/2022 13:58:09", "content": "

My submissions just back to life again! Thanks for the rerun @ryanholbrook.

", "votes": null, "replies": [] } ] }, { "id": 1944387, "author_name": "muhammadyasirsaleem", "author_url": "", "post_date": "09/18/2022 09:24:59", "content": "

this is good achivement

", "votes": null, "replies": [] }, { "id": 1945912, "author_name": "snaker", "author_url": "", "post_date": "09/19/2022 13:36:51", "content": "

I get back to the LB in the 2nd rerun, but they are not the two submissions I have selected. Is there something wrong or it's just not finished yet?

", "votes": null, "replies": [ { "id": 1945921, "author_name": "snaker", "author_url": "", "post_date": "09/19/2022 13:44:58", "content": "

My bad, it is the 2 submissions I selected.

", "votes": null, "replies": [] } ] }, { "id": 1946615, "author_name": "cabbage972", "author_url": "", "post_date": "09/20/2022 00:47:37", "content": "

@hydantess You being #1 on the LB after this rerun is a great opportunity to share some details about your solution! 😂

", "votes": null, "replies": [ { "id": 1946699, "author_name": "hydantess", "author_url": "", "post_date": "09/20/2022 04:05:07", "content": "

I will share my solution after the 5th update or last update.😄

", "votes": null, "replies": [] }, { "id": 1948182, "author_name": "dmytropoplavskiy", "author_url": "", "post_date": "09/20/2022 22:24:26", "content": "

Congrats on the #1 place on the second re-run! Very interested to read your solution.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1941266": "~~2022-09-16: It looks like our DB table for the submission rescores had gotten corrupted causing the selected submissions to not be rescored in some cases. We're going to pull the submission ids from the first update and run another rescore. I'll post the results after the weekend but wipe the leaderboard in the meantime so as not to leave it in an untrue state.~~\n\n~~Apologies for the confusion everyone and thank you for your patience while we get this sorted out. We will make sure everyone ends up with their correct score.~~\n\n2022-09-19: The second is published now. Please let us know if you suspect your submissions weren't scored correctly.", "1942610": "Is this the final status of the second rerun?\nMy Submissions shows two second reruns but they are based on the same single original submission, and my new second reruns got a bit worse results than my first rerun but the current leaderboard for mine still shows the best result of the first rerun.\n\nEdit:\nIt seems the submissions for the second rerun were automatically selected from two best submissions based on the Public LB score. I submitted the same model twice for debugging so my results of the second rerun show the same score.", "1942654": "Thanks for the heads up. The first rerun scores should have gotten wiped out. I'll leave these up for now and update shortly.", "1942682": "Is it finalized? only 917 teams are left?", "1942685": "Yes, these are the results of the second rerun.\n\nReviewing the error log, it looks like the failures this time are almost all due to missing files from datasets.", "1942686": "same here，seems not rerun of the 2 my last choosen subs", "1942691": "same here, the rerun is not the two selected subs.", "1942694": "I will take a look", "1942710": "Thanks. In our case, the rerun is one of our selected sub and another sub. The other selected sub is marked as error. Unless mistaken of course.", "1943346": "ryanholbrook Could you take a look to my submissions? They passed the 1st run but not sure why both failed in the second one (should have no missing files in the datasets I used, just checked again). Thanks.", "1944387": "this is good achivement", "1944905": "Same here.", "1945046": "Same here.", "1945478": "same here.", "1945591": "Here too, passed the first following error this run: `Clearing corrupted data and rescoring.`", "1945810": "same here.", "1945844": "ollibolli please wait a few more hours for the new update", "1945912": "I get back to the LB in the 2nd rerun, but they are not the two submissions I have selected. Is there something wrong or it's just not finished yet?", "1945921": "My bad, it is the 2 submissions I selected.", "1945935": "My submissions just back to life again! Thanks for the rerun @ryanholbrook.", "1946615": "hydantess You being #1 on the LB after this rerun is a great opportunity to share some details about your solution! 😂", "1946699": "I will share my solution after the 5th update or last update.😄", "1946938": "Thanks. Now it seems to be fine.", "1948182": "Congrats on the #1 place on the second re-run! Very interested to read your solution." }, "source": "meta" }