![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1098915%2Fb75041f121ca6bf06d3e488f7f027e40%2FScreenshot%20from%202019-12-28%2010-19-50.png?generation=1577499615370584&alt=media\")
With less than 24 hours to go, and still a lot of chatter on \"data leaks\" (we can get into the semantics of the terminology at a later date ๐ ) we noted that we'll ensure none of the leaked data will appear in the test set.
\n\nTo prevent any massive, unexpected shakeups after the competition has closed, we will be performing a few final checks on the leaderboard and will reveal the private leaderboard when we finalize it (i.e. at the same time we have performed our anti-cheating removals).
\n\nWe expect to finalize the leaderboard approximately 24-48 business hours after the competition has closed, per our usual timeline.
\n\nThanks for your patience, we want to maintain the integrity of \"the grand reveal\" as best as possible. Feel free to post solutions and write-ups in the meantime.
\n\nAddison
", "messages": [ { "id": 698380, "postDate": "2019-12-19T06:40:29.253Z", "content": "With less than 24 hours to go, and still a lot of chatter on \"data leaks\" (we can get into the semantics of the terminology at a later date ๐ ) we noted that we'll ensure none of the leaked data will appear in the test set.
\n\nTo prevent any massive, unexpected shakeups after the competition has closed, we will be performing a few final checks on the leaderboard and will reveal the private leaderboard when we finalize it (i.e. at the same time we have performed our anti-cheating removals).
\n\nWe expect to finalize the leaderboard approximately 24-48 business hours after the competition has closed, per our usual timeline.
\n\nThanks for your patience, we want to maintain the integrity of \"the grand reveal\" as best as possible. Feel free to post solutions and write-ups in the meantime.
\n\nAddison
", "rawMarkdown": "With less than 24 hours to go, and still a lot of chatter on \"data leaks\" (we can get into the semantics of the terminology at a later date ๐ ) we noted that we'll ensure none of the leaked data will appear in the test set. \n\nTo prevent any massive, unexpected shakeups *after* the competition has closed, we will be performing a few final checks on the leaderboard and will reveal the private leaderboard when we finalize it (i.e. at the same time we have performed our anti-cheating removals). \n\nWe expect to finalize the leaderboard approximately 24-48 business hours after the competition has closed, per our usual timeline.\n\nThanks for your patience, we want to maintain the integrity of \"the grand reveal\" as best as possible. Feel free to post solutions and write-ups in the meantime.\n\nAddison", "votes": 62 }, { "id": 699019, "postDate": "2019-12-20T01:18:31.567Z", "content": "I've kicked off the process of rescoring submissions to remove leaked data from the private set. The public leaderboard will look odd while this is underway, but will return to its previous state. I didn't want to kick this off earlier in the week to avoid disrupting submissions that were already underway.
\n\nThe final private set includes roughly 2.4 million data points. I will most likely be unable to respond to questions about the process until it is done as I need to focus on getting it and the cheat review screen completed.
", "rawMarkdown": "I've kicked off the process of rescoring submissions to remove leaked data from the private set. The public leaderboard will look odd while this is underway, but will return to its previous state. I didn't want to kick this off earlier in the week to avoid disrupting submissions that were already underway.\n\nThe final private set includes roughly 2.4 million data points. I will most likely be unable to respond to questions about the process until it is done as I need to focus on getting it and the cheat review screen completed.\n\n", "votes": 23, "replies": [ { "id": 699070, "postDate": "2019-12-20T02:47:27.727Z", "content": "Can we re-select submission? I want to choose my last submission as one final result, but it wasn't scored before end of competition. If that breaks the rule, please ignore it.
\n\n--update\nI think kaggle should allow us to choose it as the final result before the end of competition even though the submission is in scoring.
\n\n--update \nMy private score of last submission is 1.2413097349 which can get a gold. So sad.\n
Is \"business hours\" mean 8h each day and then 24-48 hours = 3-6 days?
\n\n.jpeg?generation=1576981632330663&alt=media)
@muhakabartay I've forgottent about Christmas holidays...
", "rawMarkdown": "@muhakabartay I've forgottent about Christmas holidays...", "votes": 7 } ] }, { "id": 699435, "postDate": "2019-12-20T13:07:14.013Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F573171%2F91df3bd9ba02d3f18db8233d7f9692c8%2Finbox_2034058_4cfd34eafa7d06e8ba2016fd9e0aa0eb_me-waiting.jpg?generation=1576847211795718&alt=media\")
most of the teams now ๐
", "rawMarkdown": "\n\nmost of the teams now ๐ ", "votes": 16 }, { "id": 700405, "postDate": "2019-12-22T00:11:20.127Z", "content": "So, 48h left, thus we need new updates.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1788308%2F8080b84cb18761ff7b3857a8644cc4e8%2F3k5anu.jpg?generation=1576973473229625&alt=media\")
Hold the LeaderBoard until December 24 at night and it is Santa Claus who discovers the Private Leaderboard, don't you think?
", "rawMarkdown": "Hold the LeaderBoard until December 24 at night and it is Santa Claus who discovers the Private Leaderboard, don't you think?\n\n\n", "votes": 10, "replies": [ { "id": 698740, "postDate": "2019-12-19T16:44:57.567Z", "content": "You think Santa has time for any other competition than his own right now?
", "rawMarkdown": "You think Santa has time for any other competition than [his own](https://www.kaggle.com/c/santa-workshop-tour-2019) right now? ", "votes": 21 } ] }, { "id": 699887, "postDate": "2019-12-21T04:03:33.963Z", "content": "Scoring update: our original rescoring job was taking longer to run than expected so I've moved the rescoring to a separate system. It's going much faster and is currently on track to finish over the weekend. Once that's done I will need to review and debug some edge cases. The main downside is that the rescore progress will no longer update in real time; there will likely not be any progress displayed until Monday regardless of how far we actually get.
\n\nThank you all for your patience! Rest assured that we want to see leaderboard finalized before the holidays as much as you do :)
", "rawMarkdown": "Scoring update: our original rescoring job was taking longer to run than expected so I've moved the rescoring to a separate system. It's going much faster and is currently on track to finish over the weekend. Once that's done I will need to review and debug some edge cases. The main downside is that the rescore progress will no longer update in real time; there will likely not be any progress displayed until Monday regardless of how far we actually get.\n\nThank you all for your patience! Rest assured that we want to see leaderboard finalized before the holidays as much as you do :)", "votes": 7, "replies": [ { "id": 699889, "postDate": "2019-12-21T04:07:20.397Z", "content": "@sohier why not rescoring only the selected submissions?
", "rawMarkdown": "@sohier why not rescoring only the selected submissions?", "votes": 5 }, { "id": 699896, "postDate": "2019-12-21T04:30:10.287Z", "content": "I'd guess that not everyone will have manually selected 2 submissions, so they still have to rescore all of them.
\n\nI don't know what percentage of the teams preselected two, so some sort of hybrid system that does the teams with selected submissions and then grinds through all the submissions of those teams who didn't preselect may or may not really help much.
", "rawMarkdown": "I'd guess that not everyone will have manually selected 2 submissions, so they still have to rescore all of them. \n\nI don't know what percentage of the teams preselected two, so some sort of hybrid system that does the teams with selected submissions and then grinds through all the submissions of those teams who didn't preselect may or may not really help much.", "votes": 3 }, { "id": 699918, "postDate": "2019-12-21T05:20:31.357Z", "content": "for the teams who didn't select 2 submissions,the private lb may be simply rescore with the best two before competitions end,instead of rescoring all and choose best two,so what you worry about may be not a problem at all
", "rawMarkdown": "for the teams who didn't select 2 submissions,the private lb may be simply rescore with the best two before competitions end,instead of rescoring all and choose best two,so what you worry about may be not a problem at all", "votes": 3 }, { "id": 699964, "postDate": "2019-12-21T07:36:26.833Z", "content": "\n\nSelecting 2 submissions is a test I did once and I missed a silver medal... I will not play with that anymore.
", "rawMarkdown": "@mutantspore \n\nSelecting 2 submissions is a test I did once and I missed a silver medal... I will not play with that anymore.", "votes": 2 }, { "id": 700209, "postDate": "2019-12-21T16:18:24.577Z", "content": "I think kaggle rescore every submission to detect the cheaters too. New accounts can just upload the submissions alone and if nobody checks how would kaggle know they are clones in case they use a different pc and ip address? This is my hypothesis only, but I have some sorts of evidence.
", "rawMarkdown": "I think kaggle rescore every submission to detect the cheaters too. New accounts can just upload the submissions alone and if nobody checks how would kaggle know they are clones in case they use a different pc and ip address? This is my hypothesis only, but I have some sorts of evidence.", "votes": 1 }, { "id": 700217, "postDate": "2019-12-21T16:31:13.563Z", "content": "@pfichou \nwhat do you mean? Is it a bad practice to select submission before the competition ends?
", "rawMarkdown": "@pfichou \nwhat do you mean? Is it a bad practice to select submission before the competition ends?\n", "votes": 1 }, { "id": 700580, "postDate": "2019-12-22T08:44:07.577Z", "content": "No, not a ยซ bad ยป practice, but there is a risk of missing a good performance in which you did not believe.
\n\nScience, technique, computation... they donโt need hope but proofs.
", "rawMarkdown": "No, not a ยซ bad ยป practice, but there is a risk of missing a good performance in which you did not believe. \n\nScience, technique, computation... they donโt need hope but proofs." }, { "id": 701313, "postDate": "2019-12-23T10:55:59.933Z", "content": "@sohier \nIs there a chance that we'll see the progress displayed today?
", "rawMarkdown": "@sohier \nIs there a chance that we'll see the progress displayed today?", "votes": 7 }, { "id": 701679, "postDate": "2019-12-23T19:00:35.670Z", "content": "Hopefully it'll be just in time for Christmas :)
", "rawMarkdown": "Hopefully it'll be just in time for Christmas :)", "votes": 1 } ] }, { "id": 699818, "postDate": "2019-12-21T01:56:38.657Z", "content": "@addisonhoward \nThank you for rescoring. But, I think rescoring all submits will take a while.\nWhy not calculate in this order?\nprivate -> public(final sub) -> public(other)
", "rawMarkdown": "@addisonhoward \nThank you for rescoring. But, I think rescoring all submits will take a while.\nWhy not calculate in this order?\nprivate -> public(final sub) -> public(other)", "votes": 5 }, { "id": 699770, "postDate": "2019-12-20T21:55:16.757Z", "content": "It doesn't look like it is progressing.
", "rawMarkdown": "It doesn't look like it is progressing.", "votes": 3, "replies": [ { "id": 699773, "postDate": "2019-12-20T22:07:45.923Z", "content": "I might be mistaken, but I think its rescoring all submissions for everyone, not just the selected ones. that will take a while
", "rawMarkdown": "I might be mistaken, but I think its rescoring all submissions for everyone, not just the selected ones. that will take a while" }, { "id": 700013, "postDate": "2019-12-21T09:19:17.990Z", "content": "exactly, over all the submissions we made as a team, only one (first one) has been calculated.
", "rawMarkdown": "exactly, over all the submissions we made as a team, only one (first one) has been calculated.", "votes": 1 } ] }, { "id": 698700, "postDate": "2019-12-19T15:40:41.830Z", "content": "So, does you mean we can't see the private leaderboard today?
", "rawMarkdown": "So, does you mean we can't see the private leaderboard today?", "votes": 3 }, { "id": 699135, "postDate": "2019-12-20T04:49:51.490Z", "content": "private 22%(until yesterday) -> 11%(now)\nwhy?????
", "rawMarkdown": "private 22%(until yesterday) -> 11%(now)\nwhy?????", "votes": 4, "replies": [ { "id": 699143, "postDate": "2019-12-20T05:02:58.807Z", "content": "because they removed the leaked data from Pvt test set.
", "rawMarkdown": "because they removed the leaked data from Pvt test set.", "votes": 2 }, { "id": 699147, "postDate": "2019-12-20T05:08:08.697Z", "content": "Is it officially announced?\n( I do not think it โs good to change the rules later๐ฃ )
", "rawMarkdown": "Is it officially announced?\n( I do not think it โs good to change the rules later๐ฃ )", "votes": 2 }, { "id": 699153, "postDate": "2019-12-20T05:19:25.363Z", "content": "Organizers had confirmed (on discussion posts) when the leaks came out that they will remove the leaked data from the Pvt. Test set
", "rawMarkdown": "Organizers had confirmed (on discussion posts) when the leaks came out that they will remove the leaked data from the Pvt. Test set", "votes": 2 }, { "id": 699162, "postDate": "2019-12-20T05:41:14.200Z", "content": "I only used a 2016 training set and did not include leaked data during model training that's why I struggle to improve my score. Anyone who used leaked data as part of model training had a great advantage of improving their score in Private LB. Good luck to all of us. Anyway, the Public (from 78% to 89%) and Private LB dataset distribution (from 22% to 11%) was not officially announced.
", "rawMarkdown": "I only used a 2016 training set and did not include leaked data during model training that's why I struggle to improve my score. Anyone who used leaked data as part of model training had a great advantage of improving their score in Private LB. Good luck to all of us. Anyway, the Public (from 78% to 89%) and Private LB dataset distribution (from 22% to 11%) was not officially announced." }, { "id": 699172, "postDate": "2019-12-20T05:51:51.320Z", "content": "I think the admin should have removed the leak data and restarted the competition.
", "rawMarkdown": "I think the admin should have removed the leak data and restarted the competition.", "votes": 5 }, { "id": 699333, "postDate": "2019-12-20T10:17:12.193Z", "content": "\n\n\nPrivate LB dataset distribution (from 22% to 11%) was not officially announced
\n
It was.
", "rawMarkdown": "> Private LB dataset distribution (from 22% to 11%) was not officially announced\n\nIt was." } ] }, { "id": 699328, "postDate": "2019-12-20T10:12:07.013Z", "content": "There were 3000+ teams, but now they are only 1800+. what happened?
", "rawMarkdown": "There were 3000+ teams, but now they are only 1800+. what happened?", "votes": 1, "replies": [ { "id": 699539, "postDate": "2019-12-20T15:06:23.310Z", "content": "banhammer (seems like a lot of participants used LB probing with fake accounts)
", "rawMarkdown": "banhammer (seems like a lot of participants used LB probing with fake accounts)", "votes": 1 } ] }, { "id": 700462, "postDate": "2019-12-22T04:01:31.860Z", "content": "Very much helpful
", "rawMarkdown": "Very much helpful ", "votes": 2 }, { "id": 699681, "postDate": "2019-12-20T18:50:27.707Z", "content": "\"our anti-cheating removals\" - what do you understand by term \"cheating\"? Private code sharing or something else?
", "rawMarkdown": "\"our anti-cheating removals\" - what do you understand by term \"cheating\"? Private code sharing or something else?", "replies": [ { "id": 701424, "postDate": "2019-12-23T13:28:57.943Z", "content": "Based on @detkov comment it seems they can detect fake accounts that were made for the purpose of LB probing. Quite shocking and interesting that more than half of the participants were actually just some fake accounts. But not sure what other sort of cheatings could be detected!?
", "rawMarkdown": "Based on @detkov comment it seems they can detect fake accounts that were made for the purpose of LB probing. Quite shocking and interesting that more than half of the participants were actually just some fake accounts. But not sure what other sort of cheatings could be detected!?" }, { "id": 701549, "postDate": "2019-12-23T15:46:07.813Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 699149, "postDate": "2019-12-20T05:12:30.653Z", "content": "Why am I removed from the public Leaderboard?
", "rawMarkdown": "Why am I removed from the public Leaderboard?", "replies": [ { "id": 699156, "postDate": "2019-12-20T05:24:01.360Z", "content": "You are not removed mate, kaggle is evaluating submissions and team who have been evaluated with 1 or both submissions are placed at the bottom as of now.\nScroll down you will see your team ๐
", "rawMarkdown": "You are not removed mate, kaggle is evaluating submissions and team who have been evaluated with 1 or both submissions are placed at the bottom as of now.\nScroll down you will see your team ๐", "votes": 7 }, { "id": 699159, "postDate": "2019-12-20T05:31:26.777Z", "content": "waw, thank you . it's shocked me/๐
", "rawMarkdown": "waw, thank you . it's shocked me/๐ " } ] }, { "id": 699908, "postDate": "2019-12-21T04:56:56.860Z", "content": "I wonder how the final submissions will be selected for teams who didn't chose them manually. \nIf the automatic selection will be based on public LB score after re-scoring, then some people may be uncomfortable with this. It will choose different submits than we expected at the moment of competition end.\nIf the automatic selection will be based on public LB before re-scoring, then private LB can be calculated without waiting for re-scoring of (much bigger) public one.
", "rawMarkdown": "I wonder how the final submissions will be selected for teams who didn't chose them manually. \nIf the automatic selection will be based on public LB score after re-scoring, then some people may be uncomfortable with this. It will choose different submits than we expected at the moment of competition end.\nIf the automatic selection will be based on public LB before re-scoring, then private LB can be calculated without waiting for re-scoring of (much bigger) public one. ", "replies": [ { "id": 699915, "postDate": "2019-12-21T05:12:17.847Z", "content": "It must be chosen from the best public LB submissions before rescoring (which is already known). I'm not sure why the public LB data / rows should change at all.
\n\nQuoting @sohier \n> The public leaderboard will look odd while this is underway, but will return to its previous state.
\n\nSo probably the original public LB will be back which is best.
\n\nIMO, just rescoring all submissions to the new private LB is all that's required.
", "rawMarkdown": "It **must** be chosen from the best public LB submissions before rescoring (which is already known). I'm not sure why the public LB data / rows should change at all.\n\nQuoting @sohier \n> The public leaderboard will look odd while this is underway, but will return to its previous state.\n\nSo probably the original public LB will be back which is best.\n\nIMO, just rescoring all submissions to the new private LB is all that's required.", "votes": 1 } ] }, { "id": 699369, "postDate": "2019-12-20T11:07:13.393Z", "content": "Does Private leaderboard include only 11% data or Combined (89% public + 11 % private data)
", "rawMarkdown": "Does Private leaderboard include only 11% data or Combined (89% public + 11 % private data)", "replies": [ { "id": 699371, "postDate": "2019-12-20T11:08:54.390Z", "content": "Just the 11%
", "rawMarkdown": "Just the 11%" } ] }, { "id": 698777, "postDate": "2019-12-19T17:22:31.270Z", "content": "Does it mean that the Pvt LB won't be avilable before Tuesday next week? or will you be putting up a preliminary Pvt LB tomorrow and the final version will come after a few days?
", "rawMarkdown": "Does it mean that the Pvt LB won't be avilable before Tuesday next week? or will you be putting up a preliminary Pvt LB tomorrow and the final version will come after a few days?", "replies": [ { "id": 698788, "postDate": "2019-12-19T17:31:50.390Z", "content": "it doesn't make sense to put a preliminary one
", "rawMarkdown": "it doesn't make sense to put a preliminary one ", "votes": 2 }, { "id": 698791, "postDate": "2019-12-19T17:34:31.513Z", "content": "@jielu0728 it has happened in past where a preliminary LB was released at first (e.g. last Santander Competition) and the medals were awarded after 2 weeks once the LB was finalized.
", "rawMarkdown": "@jielu0728 it has happened in past where a preliminary LB was released at first (e.g. last Santander Competition) and the medals were awarded after 2 weeks once the LB was finalized.", "votes": 2 }, { "id": 699467, "postDate": "2019-12-20T13:42:08.473Z", "content": "This is all I would say. You got to have patience man.
", "rawMarkdown": "This is all I would say. [You got to have patience man](https://www.youtube.com/watch?v=hvQqDfTn6s0)." }, { "id": 699621, "postDate": "2019-12-20T16:42:50.403Z", "content": "Only the final private LB will be released, no preliminary.
", "rawMarkdown": "Only the final private LB will be released, no preliminary.", "votes": 4 } ] } ], "comments": [ { "id": 699019, "author_name": "Sohier Dane", "author_url": "", "post_date": "2019-12-20T01:18:31.567000", "content": "I've kicked off the process of rescoring submissions to remove leaked data from the private set. The public leaderboard will look odd while this is underway, but will return to its previous state. I didn't want to kick this off earlier in the week to avoid disrupting submissions that were already underway.
\n\nThe final private set includes roughly 2.4 million data points. I will most likely be unable to respond to questions about the process until it is done as I need to focus on getting it and the cheat review screen completed.
", "votes": 23, "replies": [ { "id": 699070, "author_name": "Correlation", "author_url": "", "post_date": "2019-12-20T02:47:27.727000", "content": "Can we re-select submission? I want to choose my last submission as one final result, but it wasn't scored before end of competition. If that breaks the rule, please ignore it.
\n\n--update\nI think kaggle should allow us to choose it as the final result before the end of competition even though the submission is in scoring.
\n\n--update \nMy private score of last submission is 1.2413097349 which can get a gold. So sad.\n
Is \"business hours\" mean 8h each day and then 24-48 hours = 3-6 days?
\n\n.jpeg?generation=1576981632330663&alt=media)
@muhakabartay I've forgottent about Christmas holidays...
", "votes": 7, "replies": [] } ] }, { "id": 699435, "author_name": "sandy1112", "author_url": "", "post_date": "2019-12-20T13:07:14.013000", "content": "
most of the teams now ๐
", "votes": 16, "replies": [] }, { "id": 700405, "author_name": "Mukharbek Organokov", "author_url": "", "post_date": "2019-12-22T00:11:20.127000", "content": "So, 48h left, thus we need new updates.
\n\n
Hold the LeaderBoard until December 24 at night and it is Santa Claus who discovers the Private Leaderboard, don't you think?
", "votes": 10, "replies": [ { "id": 698740, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-12-19T16:44:57.567000", "content": "You think Santa has time for any other competition than his own right now?
", "votes": 21, "replies": [] } ] }, { "id": 699887, "author_name": "Sohier Dane", "author_url": "", "post_date": "2019-12-21T04:03:33.963000", "content": "Scoring update: our original rescoring job was taking longer to run than expected so I've moved the rescoring to a separate system. It's going much faster and is currently on track to finish over the weekend. Once that's done I will need to review and debug some edge cases. The main downside is that the rescore progress will no longer update in real time; there will likely not be any progress displayed until Monday regardless of how far we actually get.
\n\nThank you all for your patience! Rest assured that we want to see leaderboard finalized before the holidays as much as you do :)
", "votes": 7, "replies": [ { "id": 699889, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2019-12-21T04:07:20.397000", "content": "@sohier why not rescoring only the selected submissions?
", "votes": 5, "replies": [] }, { "id": 699896, "author_name": "Bruce Young", "author_url": "", "post_date": "2019-12-21T04:30:10.287000", "content": "I'd guess that not everyone will have manually selected 2 submissions, so they still have to rescore all of them.
\n\nI don't know what percentage of the teams preselected two, so some sort of hybrid system that does the teams with selected submissions and then grinds through all the submissions of those teams who didn't preselect may or may not really help much.
", "votes": 3, "replies": [] }, { "id": 699918, "author_name": "Wu_Yiqun", "author_url": "", "post_date": "2019-12-21T05:20:31.357000", "content": "for the teams who didn't select 2 submissions,the private lb may be simply rescore with the best two before competitions end,instead of rescoring all and choose best two,so what you worry about may be not a problem at all
", "votes": 3, "replies": [] }, { "id": 699964, "author_name": "mezoganet", "author_url": "", "post_date": "2019-12-21T07:36:26.833000", "content": "\n\nSelecting 2 submissions is a test I did once and I missed a silver medal... I will not play with that anymore.
", "votes": 2, "replies": [] }, { "id": 700209, "author_name": "Phi", "author_url": "", "post_date": "2019-12-21T16:18:24.577000", "content": "I think kaggle rescore every submission to detect the cheaters too. New accounts can just upload the submissions alone and if nobody checks how would kaggle know they are clones in case they use a different pc and ip address? This is my hypothesis only, but I have some sorts of evidence.
", "votes": 1, "replies": [] }, { "id": 700217, "author_name": "John Oliver Taco Lรณpez", "author_url": "", "post_date": "2019-12-21T16:31:13.563000", "content": "@pfichou \nwhat do you mean? Is it a bad practice to select submission before the competition ends?
", "votes": 1, "replies": [] }, { "id": 700580, "author_name": "mezoganet", "author_url": "", "post_date": "2019-12-22T08:44:07.577000", "content": "No, not a ยซ bad ยป practice, but there is a risk of missing a good performance in which you did not believe.
\n\nScience, technique, computation... they donโt need hope but proofs.
", "votes": 0, "replies": [] }, { "id": 701313, "author_name": "Stanislav Blinov", "author_url": "", "post_date": "2019-12-23T10:55:59.933000", "content": "@sohier \nIs there a chance that we'll see the progress displayed today?
", "votes": 7, "replies": [] }, { "id": 701679, "author_name": "Wei Hao Khoong", "author_url": "", "post_date": "2019-12-23T19:00:35.670000", "content": "Hopefully it'll be just in time for Christmas :)
", "votes": 1, "replies": [] } ] }, { "id": 699818, "author_name": "Yust", "author_url": "", "post_date": "2019-12-21T01:56:38.657000", "content": "@addisonhoward \nThank you for rescoring. But, I think rescoring all submits will take a while.\nWhy not calculate in this order?\nprivate -> public(final sub) -> public(other)
", "votes": 5, "replies": [] }, { "id": 699770, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2019-12-20T21:55:16.757000", "content": "It doesn't look like it is progressing.
", "votes": 3, "replies": [ { "id": 699773, "author_name": "Tim Yee", "author_url": "", "post_date": "2019-12-20T22:07:45.923000", "content": "I might be mistaken, but I think its rescoring all submissions for everyone, not just the selected ones. that will take a while
", "votes": 0, "replies": [] }, { "id": 700013, "author_name": "Firat Gonen", "author_url": "", "post_date": "2019-12-21T09:19:17.990000", "content": "exactly, over all the submissions we made as a team, only one (first one) has been calculated.
", "votes": 1, "replies": [] } ] }, { "id": 698700, "author_name": "Yoshi", "author_url": "", "post_date": "2019-12-19T15:40:41.830000", "content": "So, does you mean we can't see the private leaderboard today?
", "votes": 3, "replies": [] }, { "id": 699135, "author_name": "Yust", "author_url": "", "post_date": "2019-12-20T04:49:51.490000", "content": "private 22%(until yesterday) -> 11%(now)\nwhy?????
", "votes": 4, "replies": [ { "id": 699143, "author_name": "sandy1112", "author_url": "", "post_date": "2019-12-20T05:02:58.807000", "content": "because they removed the leaked data from Pvt test set.
", "votes": 2, "replies": [] }, { "id": 699147, "author_name": "Yoshi", "author_url": "", "post_date": "2019-12-20T05:08:08.697000", "content": "Is it officially announced?\n( I do not think it โs good to change the rules later๐ฃ )
", "votes": 2, "replies": [] }, { "id": 699153, "author_name": "sandy1112", "author_url": "", "post_date": "2019-12-20T05:19:25.363000", "content": "Organizers had confirmed (on discussion posts) when the leaks came out that they will remove the leaked data from the Pvt. Test set
", "votes": 2, "replies": [] }, { "id": 699162, "author_name": "FGPC", "author_url": "", "post_date": "2019-12-20T05:41:14.200000", "content": "I only used a 2016 training set and did not include leaked data during model training that's why I struggle to improve my score. Anyone who used leaked data as part of model training had a great advantage of improving their score in Private LB. Good luck to all of us. Anyway, the Public (from 78% to 89%) and Private LB dataset distribution (from 22% to 11%) was not officially announced.
", "votes": 0, "replies": [] }, { "id": 699172, "author_name": "Yoshi", "author_url": "", "post_date": "2019-12-20T05:51:51.320000", "content": "I think the admin should have removed the leak data and restarted the competition.
", "votes": 5, "replies": [] }, { "id": 699333, "author_name": "Stanislav Blinov", "author_url": "", "post_date": "2019-12-20T10:17:12.193000", "content": "\n\n\nPrivate LB dataset distribution (from 22% to 11%) was not officially announced
\n
It was.
", "votes": 0, "replies": [] } ] }, { "id": 699328, "author_name": "Navid Hakimi", "author_url": "", "post_date": "2019-12-20T10:12:07.013000", "content": "There were 3000+ teams, but now they are only 1800+. what happened?
", "votes": 1, "replies": [ { "id": 699539, "author_name": "Nikita Detkov", "author_url": "", "post_date": "2019-12-20T15:06:23.310000", "content": "banhammer (seems like a lot of participants used LB probing with fake accounts)
", "votes": 1, "replies": [] } ] }, { "id": 700462, "author_name": "Saikot Roy", "author_url": "", "post_date": "2019-12-22T04:01:31.860000", "content": "Very much helpful
", "votes": 2, "replies": [] }, { "id": 699681, "author_name": "Roman Trotsyuk", "author_url": "", "post_date": "2019-12-20T18:50:27.707000", "content": "\"our anti-cheating removals\" - what do you understand by term \"cheating\"? Private code sharing or something else?
", "votes": 0, "replies": [ { "id": 701424, "author_name": "Novin Shahroudi", "author_url": "", "post_date": "2019-12-23T13:28:57.943000", "content": "Based on @detkov comment it seems they can detect fake accounts that were made for the purpose of LB probing. Quite shocking and interesting that more than half of the participants were actually just some fake accounts. But not sure what other sort of cheatings could be detected!?
", "votes": 0, "replies": [] }, { "id": 701549, "author_name": "", "author_url": "", "post_date": "2019-12-23T15:46:07.813000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 699149, "author_name": "Zhang Yunfei", "author_url": "", "post_date": "2019-12-20T05:12:30.653000", "content": "Why am I removed from the public Leaderboard?
", "votes": 0, "replies": [ { "id": 699156, "author_name": "HarshitMehta", "author_url": "", "post_date": "2019-12-20T05:24:01.360000", "content": "You are not removed mate, kaggle is evaluating submissions and team who have been evaluated with 1 or both submissions are placed at the bottom as of now.\nScroll down you will see your team ๐
", "votes": 7, "replies": [] }, { "id": 699159, "author_name": "Zhang Yunfei", "author_url": "", "post_date": "2019-12-20T05:31:26.777000", "content": "waw, thank you . it's shocked me/๐
", "votes": 0, "replies": [] } ] }, { "id": 699908, "author_name": "Josef Slavicek", "author_url": "", "post_date": "2019-12-21T04:56:56.860000", "content": "I wonder how the final submissions will be selected for teams who didn't chose them manually. \nIf the automatic selection will be based on public LB score after re-scoring, then some people may be uncomfortable with this. It will choose different submits than we expected at the moment of competition end.\nIf the automatic selection will be based on public LB before re-scoring, then private LB can be calculated without waiting for re-scoring of (much bigger) public one.
", "votes": 0, "replies": [ { "id": 699915, "author_name": "Vopani", "author_url": "", "post_date": "2019-12-21T05:12:17.847000", "content": "It must be chosen from the best public LB submissions before rescoring (which is already known). I'm not sure why the public LB data / rows should change at all.
\n\nQuoting @sohier \n> The public leaderboard will look odd while this is underway, but will return to its previous state.
\n\nSo probably the original public LB will be back which is best.
\n\nIMO, just rescoring all submissions to the new private LB is all that's required.
", "votes": 1, "replies": [] } ] }, { "id": 699369, "author_name": "Utkarsh Tripathi", "author_url": "", "post_date": "2019-12-20T11:07:13.393000", "content": "Does Private leaderboard include only 11% data or Combined (89% public + 11 % private data)
", "votes": 0, "replies": [ { "id": 699371, "author_name": "sandy1112", "author_url": "", "post_date": "2019-12-20T11:08:54.390000", "content": "Just the 11%
", "votes": 0, "replies": [] } ] }, { "id": 698777, "author_name": "sandy1112", "author_url": "", "post_date": "2019-12-19T17:22:31.270000", "content": "Does it mean that the Pvt LB won't be avilable before Tuesday next week? or will you be putting up a preliminary Pvt LB tomorrow and the final version will come after a few days?
", "votes": 0, "replies": [ { "id": 698788, "author_name": "Jie Lu", "author_url": "", "post_date": "2019-12-19T17:31:50.390000", "content": "it doesn't make sense to put a preliminary one
", "votes": 2, "replies": [] }, { "id": 698791, "author_name": "sandy1112", "author_url": "", "post_date": "2019-12-19T17:34:31.513000", "content": "@jielu0728 it has happened in past where a preliminary LB was released at first (e.g. last Santander Competition) and the medals were awarded after 2 weeks once the LB was finalized.
", "votes": 2, "replies": [] }, { "id": 699467, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-12-20T13:42:08.473000", "content": "This is all I would say. You got to have patience man.
", "votes": 0, "replies": [] }, { "id": 699621, "author_name": "Sohier Dane", "author_url": "", "post_date": "2019-12-20T16:42:50.403000", "content": "Only the final private LB will be released, no preliminary.
", "votes": 4, "replies": [] } ] } ], "raw_markdown_by_id": { "698380": "With less than 24 hours to go, and still a lot of chatter on \"data leaks\" (we can get into the semantics of the terminology at a later date ๐ ) we noted that we'll ensure none of the leaked data will appear in the test set. \n\nTo prevent any massive, unexpected shakeups *after* the competition has closed, we will be performing a few final checks on the leaderboard and will reveal the private leaderboard when we finalize it (i.e. at the same time we have performed our anti-cheating removals). \n\nWe expect to finalize the leaderboard approximately 24-48 business hours after the competition has closed, per our usual timeline.\n\nThanks for your patience, we want to maintain the integrity of \"the grand reveal\" as best as possible. Feel free to post solutions and write-ups in the meantime.\n\nAddison", "699019": "I've kicked off the process of rescoring submissions to remove leaked data from the private set. The public leaderboard will look odd while this is underway, but will return to its previous state. I didn't want to kick this off earlier in the week to avoid disrupting submissions that were already underway.\n\nThe final private set includes roughly 2.4 million data points. I will most likely be unable to respond to questions about the process until it is done as I need to focus on getting it and the cheat review screen completed.\n\n", "700439": "Is \"business hours\" mean 8h each day and then 24-48 hours = 3-6 days?\n\n.jpeg?generation=1576981632330663&alt=media)\n", "699435": "\n\nmost of the teams now ๐ ", "700405": "So, 48h left, thus we need new updates. \n\n\n", "698589": "Hold the LeaderBoard until December 24 at night and it is Santa Claus who discovers the Private Leaderboard, don't you think?\n\n\n", "699887": "Scoring update: our original rescoring job was taking longer to run than expected so I've moved the rescoring to a separate system. It's going much faster and is currently on track to finish over the weekend. Once that's done I will need to review and debug some edge cases. The main downside is that the rescore progress will no longer update in real time; there will likely not be any progress displayed until Monday regardless of how far we actually get.\n\nThank you all for your patience! Rest assured that we want to see leaderboard finalized before the holidays as much as you do :)", "699818": "@addisonhoward \nThank you for rescoring. But, I think rescoring all submits will take a while.\nWhy not calculate in this order?\nprivate -> public(final sub) -> public(other)", "699770": "It doesn't look like it is progressing.", "698700": "So, does you mean we can't see the private leaderboard today?", "699135": "private 22%(until yesterday) -> 11%(now)\nwhy?????", "699328": "There were 3000+ teams, but now they are only 1800+. what happened?", "700462": "Very much helpful ", "699681": "\"our anti-cheating removals\" - what do you understand by term \"cheating\"? Private code sharing or something else?", "699149": "Why am I removed from the public Leaderboard?", "699908": "I wonder how the final submissions will be selected for teams who didn't chose them manually. \nIf the automatic selection will be based on public LB score after re-scoring, then some people may be uncomfortable with this. It will choose different submits than we expected at the moment of competition end.\nIf the automatic selection will be based on public LB before re-scoring, then private LB can be calculated without waiting for re-scoring of (much bigger) public one. ", "699369": "Does Private leaderboard include only 11% data or Combined (89% public + 11 % private data)", "698777": "Does it mean that the Pvt LB won't be avilable before Tuesday next week? or will you be putting up a preliminary Pvt LB tomorrow and the final version will come after a few days?" } }