![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2225116%2F9ba67f2a9732e08b23164b6c06b3d17e%2Fwaiting.jpeg?generation=1578286889672679&alt=media\")
I have good news and bad news. The good news is that we have finished calculating everyone's scores and our cheat review. The bad news is that we seem to have angered Santa; our oncall engineer's workstation broke. Since we're short staffed over the holidays this means we won't be able merge the scores into the official leaderboard until next week. However, I have posted csvs of the leaderboard and all scores here: https://www.kaggle.com/sohier/ashrae-interim-scores.
\n\nI will personally be on vacation and offline most of next week and unable to respond to questions until I'm back, but Addison and other members of the team will be online besides new years eve and day.
\n\nThank you all again for your patience! We will be conducting an internal review to head off delays like this in the future.
", "messages": [ { "id": 704754, "postDate": "2019-12-28T00:58:39.210Z", "content": "I have good news and bad news. The good news is that we have finished calculating everyone's scores and our cheat review. The bad news is that we seem to have angered Santa; our oncall engineer's workstation broke. Since we're short staffed over the holidays this means we won't be able merge the scores into the official leaderboard until next week. However, I have posted csvs of the leaderboard and all scores here: https://www.kaggle.com/sohier/ashrae-interim-scores.
\n\nI will personally be on vacation and offline most of next week and unable to respond to questions until I'm back, but Addison and other members of the team will be online besides new years eve and day.
\n\nThank you all again for your patience! We will be conducting an internal review to head off delays like this in the future.
", "rawMarkdown": "I have good news and bad news. The good news is that we have finished calculating everyone's scores and our cheat review. The bad news is that we seem to have angered Santa; our oncall engineer's workstation broke. Since we're short staffed over the holidays this means we won't be able merge the scores into the official leaderboard until next week. However, I have posted csvs of the leaderboard and all scores here: [https://www.kaggle.com/sohier/ashrae-interim-scores](https://www.kaggle.com/sohier/ashrae-interim-scores).\n\nI will personally be on vacation and offline most of next week and unable to respond to questions until I'm back, but Addison and other members of the team will be online besides new years eve and day.\n\nThank you all again for your patience! We will be conducting an internal review to head off delays like this in the future.", "votes": 71 }, { "id": 704777, "postDate": "2019-12-28T02:01:06.690Z", "content": "Congrats everyone! Here is the final lb in a kernel in case you don't want to deal with csvs\nhttps://www.kaggle.com/robikscube/ashrae-leaderboard
", "rawMarkdown": "Congrats everyone! Here is the final lb in a kernel in case you don't want to deal with csvs\nhttps://www.kaggle.com/robikscube/ashrae-leaderboard", "votes": 24, "replies": [ { "id": 704812, "postDate": "2019-12-28T03:16:16.720Z", "content": "Hi @robikscube thanks a ton for taking the time out and making this. Got a medal and will now be a kaggle competitions expert! Thanks!
", "rawMarkdown": "Hi @robikscube thanks a ton for taking the time out and making this. Got a medal and will now be a kaggle competitions expert! Thanks!", "votes": 4 }, { "id": 704815, "postDate": "2019-12-28T03:19:35.470Z", "content": "Thats awesome @sri3128018 - Congrats! ✊
", "rawMarkdown": "Thats awesome @sri3128018 - Congrats! ✊" }, { "id": 705315, "postDate": "2019-12-28T19:04:21.113Z", "content": "@robikscube, thanks for sharing. For those interested, check my kernel for the medal ranges based on the original 3675 teams public LB before rescoring.
", "rawMarkdown": "@robikscube, thanks for sharing. For those interested, check [my kernel](https://www.kaggle.com/sheriytm/ashrae-public-lb-before-rescoring) for the medal ranges based on the original 3675 teams [public LB before rescoring](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/123497).", "votes": 1 }, { "id": 705353, "postDate": "2019-12-28T20:09:53.650Z", "content": "Nice work @sheriytm - I added a little medal emoji column in my kernel based on your medal range calculation!
", "rawMarkdown": "Nice work @sheriytm - I added a little medal emoji column in my kernel based on your medal range calculation!", "votes": 3 }, { "id": 705370, "postDate": "2019-12-28T20:42:59.933Z", "content": "Nice @robikscube. Please note that the medals range would not be accurate for the LB csv file list because the number of teams is decreased from the original 3675 to 3594.
\n\nBTW an upvote of my kernel would be nice :-)
", "rawMarkdown": "Nice @robikscube. Please note that the medals range would not be accurate for the LB csv file list because the number of teams is decreased from the original 3675 to 3594.\n\nBTW an upvote of my kernel would be nice :-)", "votes": 1 }, { "id": 705612, "postDate": "2019-12-29T06:34:22.320Z", "content": "In case of 3594 teams, teams till 179 or 180 will get silver and teams till 359 or 360 will get bronze!
", "rawMarkdown": "In case of 3594 teams, teams till 179 or 180 will get silver and teams till 359 or 360 will get bronze!", "votes": 1 }, { "id": 706695, "postDate": "2019-12-30T17:41:31.973Z", "content": "Hi @robikscube , really appreciate your efforts to convert the CSV into a nice kernel...!! Keep it up...!!
", "rawMarkdown": "Hi @robikscube , really appreciate your efforts to convert the CSV into a nice kernel...!! Keep it up...!!" }, { "id": 709678, "postDate": "2020-01-03T20:13:37.490Z", "content": "based on your kernel, we're getting a bronze (UnderDogs) , fingers crossed, thanks @robikscube
", "rawMarkdown": "based on your kernel, we're getting a bronze (UnderDogs) , fingers crossed, thanks @robikscube " } ] }, { "id": 706943, "postDate": "2019-12-31T03:45:14.383Z", "content": "Leaderboard Update 5.5
\n\nI have marginal news - we likely won't be able to get the formalized leaderboard updated before the New Year. We were able to make some progress on manually processing each submission to display on the leaderboard, but not the full amount. (Again, please note this is an edge case for our competition format, and we don't intend to encounter something like this again)
\n\nA number of you have messaged me directly about missing submissions or questions, and I've responded to each. You're welcome to continue messaging me with questions (or just replying in thread here). The formal compliance appeal process will begin when the leaderboard is published.
\n\nTo provide a bit of a teaser in the meantime, however, we're also working on an exciting new competition format that we think will add a lot of value to your machine learning education and skill refinement a we enter the 2020s!
\n\nHappy New Year to all and again, thanks for your patience!
", "rawMarkdown": "**Leaderboard Update 5.5**\n\nI have marginal news - we likely won't be able to get the formalized leaderboard updated before the New Year. We were able to make some progress on manually processing each submission to display on the leaderboard, but not the full amount. (Again, please note this is an edge case for our competition format, and we don't intend to encounter something like this again)\n\nA number of you have messaged me directly about missing submissions or questions, and I've responded to each. You're welcome to continue messaging me with questions (or just replying in thread here). The formal compliance appeal process will begin when the leaderboard is published.\n\nTo provide a bit of a teaser in the meantime, however, we're also working on an exciting new competition format that we think will add a lot of value to your machine learning education and skill refinement a we enter the 2020s!\n\nHappy New Year to all and again, thanks for your patience!\n\n", "votes": 13, "replies": [ { "id": 706944, "postDate": "2019-12-31T03:48:46.337Z", "content": "Can we know which sites are removed from the private LB?
", "rawMarkdown": "Can we know which sites are removed from the private LB?", "votes": 3 }, { "id": 707126, "postDate": "2019-12-31T09:27:38.460Z", "content": "Thanks for the update @addisonhoward. Looking forward to the new format!
", "rawMarkdown": "Thanks for the update @addisonhoward. Looking forward to the new format!" }, { "id": 708535, "postDate": "2020-01-02T11:54:42.213Z", "content": "@addisonhoward - any chance of seeing the LB before Thursday EOD?
", "rawMarkdown": "@addisonhoward - any chance of seeing the LB before Thursday EOD?", "votes": 1 }, { "id": 711699, "postDate": "2020-01-06T12:43:52.020Z", "content": "Hi Addison, \nIf one or some teams have been scored all their submissions, this rule would be generalized? \n@addisonhoward: if it is the case, we would like that you review all ours submissions which gives more relevant evaluation score to ours work in this competition. \nThank you in advance,\n Electrium Z team: @lucamassaron @steubk @pietromarinelli @serigne
", "rawMarkdown": "Hi Addison, \nIf one or some teams have been scored all their submissions, this rule would be generalized? \n@addisonhoward: if it is the case, we would like that you review all ours submissions which gives more relevant evaluation score to ours work in this competition. \nThank you in advance,\n Electrium Z team: @lucamassaron @steubk @pietromarinelli @serigne " }, { "id": 711969, "postDate": "2020-01-06T17:59:55.423Z", "content": "@aminepy, if you have not selected your best submission, I do not think rescoring all your submissions would help change your position as shown on the leaderboard because your best score on private LB i.e. 1.266217 is not on top of your public LB score.
\n\nI feel your pain. Mine is not my best either. My best score on the private LB would put me at #27 or so. I selected one submission and the 2nd was chosen automatically based on my best public LB score☹️
", "rawMarkdown": "@aminepy, if you have not selected your best submission, I do not think rescoring all your submissions would help change your position as shown on the leaderboard because your best score on private LB i.e. 1.266217 is not on top of your public LB score.\n\nI feel your pain. Mine is not my best either. My best score on the private LB would put me at #27 or so. I selected one submission and the 2nd was chosen automatically based on my best public LB score☹️ ", "votes": 1 }, { "id": 712411, "postDate": "2020-01-07T07:47:49.717Z", "content": "@sheriytm: Kaggle choose two best if none is chosen\nIn this competition: If you chose the two best before removing the leak, this would be the worst case... \nOn the other hand, as far as mentioned, organizers have removed the leak before processing the LB\nthat means it might have gave advantage to those who didn't choose (or chose only one). This is why we demand for review on all submissions for all teams.
", "rawMarkdown": "@sheriytm: Kaggle choose two best if none is chosen\nIn this competition: If you chose the two best before removing the leak, this would be the worst case... \nOn the other hand, as far as mentioned, organizers have removed the leak before processing the LB\nthat means it might have gave advantage to those who didn't choose (or chose only one). This is why we demand for review on all submissions for all teams. " }, { "id": 712416, "postDate": "2020-01-07T08:08:35.353Z", "content": "@aminepy, I see. It makes sense that you are asking for all teams. I look forward to @addisonhoward's response.
\n\nMine looks like submissions are ordered with the leaks.
", "rawMarkdown": "@aminepy, I see. It makes sense that you are asking for all teams. I look forward to @addisonhoward's response.\n\nMine looks like submissions are ordered with the leaks.", "votes": 1 }, { "id": 712418, "postDate": "2020-01-07T08:13:01.173Z", "content": "@sheriytm: yes the whole review submissions should be done for all teams :)
", "rawMarkdown": "@sheriytm: yes the whole review submissions should be done for all teams :) " }, { "id": 712430, "postDate": "2020-01-07T08:31:39.690Z", "content": "@sheriytm @aminepy - what are your thoughts on teams who carefully selected the two submissions and got the desired results? I selected one with the leaks and another one which didn't use the leaks. The one with the leak had my best Public LB score but a bad pvt Lb score. However, if I didn't bother selecting my submission then the second best would also be a score that had used the leak and fares poorly on the private set. For my personal submissions, I want to use the submissions that I selected knowing very well that the Pvt LB would be calculated by removing the Pvt test set - the competition organizers had confirmed that before the submissions were closed.
\n\nRescoring using all submissions for all the teams is a bit of a gray area where many would want to do it, where some teams (like me) would not want to do it. Not sure what is feasible solution.
", "rawMarkdown": "@sheriytm @aminepy - what are your thoughts on teams who carefully selected the two submissions and got the desired results? I selected one with the leaks and another one which didn't use the leaks. The one with the leak had my best Public LB score but a bad pvt Lb score. However, if I didn't bother selecting my submission then the second best would also be a score that had used the leak and fares poorly on the private set. For my personal submissions, I want to use the submissions that I selected knowing very well that the Pvt LB would be calculated by removing the Pvt test set - the competition organizers had confirmed that before the submissions were closed.\n\nRescoring using all submissions for all the teams is a bit of a gray area where many would want to do it, where some teams (like me) would not want to do it. Not sure what is feasible solution." }, { "id": 712480, "postDate": "2020-01-07T09:34:19.613Z", "content": "@sandy1112: In general, we chose the 2 best scores submissions before the end. However, in this competitions most of teams have a good public LB with leak data. Thus, moving out the leak data and scoring would change the 2 best chosen scores. And we don't know yet how the organizers process to score teams who have chosen 1 submissions or none one? Since that we just suggest to generalize the review of all submissions for all teams which is appropriate in my opinion. \nBy the way, reviewing all submissions and chose the best score is in the interest of all teams.
", "rawMarkdown": "@sandy1112: In general, we chose the 2 best scores submissions before the end. However, in this competitions most of teams have a good public LB with leak data. Thus, moving out the leak data and scoring would change the 2 best chosen scores. And we don't know yet how the organizers process to score teams who have chosen 1 submissions or none one? Since that we just suggest to generalize the review of all submissions for all teams which is appropriate in my opinion. \nBy the way, reviewing all submissions and chose the best score is in the interest of all teams." }, { "id": 712485, "postDate": "2020-01-07T09:42:13.927Z", "content": "@Sandy1112, I think what they have done is used our submissions on the public LB before re-scoring for the auto selection. Which I believe is fair since we all know that the leaks will be removed from the private LB.
\n\nThe reason I chose only one submission was because my last submission did not finish scoring for more than 30 minutes until closing. So I let that to be auto-selected as I believed it to be my best.
\n\nMy best private LB score I mentioned earlier also had the leak but was not the best on the public LB.
", "rawMarkdown": "@Sandy1112, I think what they have done is used our submissions on the public LB before re-scoring for the auto selection. Which I believe is fair since we all know that the leaks will be removed from the private LB. \n\nThe reason I chose only one submission was because my last submission did not finish scoring for more than 30 minutes until closing. So I let that to be auto-selected as I believed it to be my best. \n\nMy best private LB score I mentioned earlier also had the leak but was not the best on the public LB." }, { "id": 712501, "postDate": "2020-01-07T10:01:39.673Z", "content": "@aminepy when you say 'reviewing all submissions and chose the best score', do you mean the best Public score or the best Private score? If it's the best Private score, then I agree - its in the interest of all teams. Not sure if organizers would do that. let's hear more on this from Addison and team
", "rawMarkdown": "@aminepy when you say 'reviewing all submissions and chose the best score', do you mean the best Public score or the best Private score? If it's the best Private score, then I agree - its in the interest of all teams. Not sure if organizers would do that. let's hear more on this from Addison and team " }, { "id": 712523, "postDate": "2020-01-07T10:33:42.797Z", "content": "\n\n\nIf it's the best Private score, then I agree - its in the interest of all teams.
\n
It's not. It will heavily favour teams with high number of submissions and be completely unfair. Process for selecting final submissions is already discussed and confirmed by admins in another thread: https://www.kaggle.com/c/ashrae-energy-prediction/discussion/122987#703901
", "rawMarkdown": ">If it's the best Private score, then I agree - its in the interest of all teams.\n\nIt's not. It will heavily favour teams with high number of submissions and be completely unfair. Process for selecting final submissions is already discussed and confirmed by admins in another thread: https://www.kaggle.com/c/ashrae-energy-prediction/discussion/122987#703901" }, { "id": 712553, "postDate": "2020-01-07T11:17:03.603Z", "content": "@rohanrao 💯 agree with you. I think the current method of considering all subs for teams who didn't select any final submission and using the two selected ones for the teams who did make a choice, has some amount of right balance. Although it's pretty difficult to get unified vote in favour of any method that have been discussed in the forum. Every method has its pros and cons.
\n\nI just want the final LB to be published and let's get over with it. Its almost 3 weeks of waiting. Never saw such a long wait for any competitions I have done till date (which is a small duration compared to Kaggle veterans like you)
", "rawMarkdown": "@rohanrao 💯 agree with you. I think the current method of considering all subs for teams who didn't select any final submission and using the two selected ones for the teams who did make a choice, has some amount of right balance. Although it's pretty difficult to get unified vote in favour of any method that have been discussed in the forum. Every method has its pros and cons.\n\nI just want the final LB to be published and let's get over with it. Its almost 3 weeks of waiting. Never saw such a long wait for any competitions I have done till date (which is a small duration compared to Kaggle veterans like you)" }, { "id": 712614, "postDate": "2020-01-07T12:56:27.547Z", "content": "@rohanrao: I'm agree with you and i have seen that discussion. However, in case they have scored teams after check all their submissions, this rule should be generalized for all teams. Sorry but doesn't make any sens to have a good public LB with leak and then move it out form private LB score!! also the private test set has been reduce to 11% So mathematically, your chosen 2 submissions are no longer the best ones. Well, let's see the organizes @addisonhoward opinion about that issues... \n@sandy1112: Hope they can take care for ours comments before publish the private LB! this why, it makes sens to mention right now!! so if you are convinced please do it!
", "rawMarkdown": "@rohanrao: I'm agree with you and i have seen that discussion. However, in case they have scored teams after check all their submissions, this rule should be generalized for all teams. Sorry but doesn't make any sens to have a good public LB with leak and then move it out form private LB score!! also the private test set has been reduce to 11% So mathematically, your chosen 2 submissions are no longer the best ones. Well, let's see the organizes @addisonhoward opinion about that issues... \n@sandy1112: Hope they can take care for ours comments before publish the private LB! this why, it makes sens to mention right now!! so if you are convinced please do it! " }, { "id": 712837, "postDate": "2020-01-07T16:24:10.447Z", "content": "I think there are a lot of possible solutions, but now there is a private leaderboard. So people who are in a good position would not agree with changing the results. All possible solutions had to be discussed before the publication of the private leaderboard. Now, it's too late.
", "rawMarkdown": "I think there are a lot of possible solutions, but now there is a private leaderboard. So people who are in a good position would not agree with changing the results. All possible solutions had to be discussed before the publication of the private leaderboard. Now, it's too late. " } ] }, { "id": 712176, "postDate": "2020-01-06T22:50:48.213Z", "content": "Leaderboard Update 5.6 - more marginal news. The whole team was back in the office today, including the owner of a system we ended up needing to consider. We believe we have a new path for executing the score finalization. Hope to have more news for you tomorrow.
", "rawMarkdown": "Leaderboard Update 5.6 - more marginal news. The whole team was back in the office today, including the owner of a system we ended up needing to consider. We believe we have a new path for executing the score finalization. Hope to have more news for you tomorrow.", "votes": 11, "replies": [ { "id": 712180, "postDate": "2020-01-06T23:01:16.707Z", "content": "@sohier Thanks for the updating.
", "rawMarkdown": "@sohier Thanks for the updating.", "votes": 2 }, { "id": 712206, "postDate": "2020-01-07T00:05:29.350Z", "content": "Hi, one question. \nThe number of teams will be 1877 or 3500?\nthanks :D
", "rawMarkdown": "Hi, one question. \nThe number of teams will be 1877 or 3500?\nthanks :D", "votes": 1 }, { "id": 712267, "postDate": "2020-01-07T02:32:44.667Z", "content": "This update was more like a 5.5.1 rather than 5.6. thanks for the update anyway
", "rawMarkdown": "This update was more like a 5.5.1 rather than 5.6. thanks for the update anyway", "votes": 2 }, { "id": 712378, "postDate": "2020-01-07T06:55:30.407Z", "content": "Score finalization??
\n\nDid the final scores leader board csv shared final or not?
there is also an update 5.75 :)
", "rawMarkdown": "there is also an update 5.75 :)" }, { "id": 712438, "postDate": "2020-01-07T08:35:58.123Z", "content": "@frtgnn just noticed that and 5.75 is older than 5.6 ... lol 😂
", "rawMarkdown": "@frtgnn just noticed that and 5.75 is older than 5.6 ... lol 😂 ", "votes": 2 }, { "id": 712473, "postDate": "2020-01-07T09:29:03.410Z", "content": "exactly @sandy1112 :)
", "rawMarkdown": "exactly @sandy1112 :)" } ] }, { "id": 711433, "postDate": "2020-01-06T05:01:34.720Z", "content": "
Any chance to see the leaderboard today?
", "rawMarkdown": "Any chance to see the leaderboard today?", "votes": 8 }, { "id": 709384, "postDate": "2020-01-03T12:43:01.523Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F573171%2F4bc9befcaf3f978d02edb21346c9d818%2Finbox_413189_d429d1c35f04b63288e80f05aaa20f9c_and-still-waiting--meme-50106.jpg?generation=1578055356218548&alt=media\")
15 days passed...
", "rawMarkdown": "\n\n15 days passed...", "votes": 8 }, { "id": 709828, "postDate": "2020-01-04T01:08:30.470Z", "content": "The competition is now listed under completed competitions instead of active competitions, hoping to hear the good news soon! :)
", "rawMarkdown": "The competition is now listed under completed competitions instead of active competitions, hoping to hear the good news soon! :)", "votes": 6, "replies": [ { "id": 709978, "postDate": "2020-01-04T06:53:59.340Z", "content": "I saw that too. Just want to ask Kaggle @addisonhoward , will the leaderboard be update or not?
", "rawMarkdown": "I saw that too. Just want to ask Kaggle @addisonhoward , will the leaderboard be update or not? ", "votes": 1 }, { "id": 709993, "postDate": "2020-01-04T07:23:47.903Z", "content": "Hey again. Update 5.75.
\n\nWe thought we had a fix to merge the scores into the leaderboard on Thursday but it failed. We need to wait for some of our engineers who are on vacation to return, but we have a meeting scheduled for Monday to work on the resolution.
", "rawMarkdown": "Hey again. Update 5.75.\n\nWe thought we had a fix to merge the scores into the leaderboard on Thursday but it failed. We need to wait for some of our engineers who are on vacation to return, but we have a meeting scheduled for Monday to work on the resolution. ", "votes": 4 }, { "id": 710033, "postDate": "2020-01-04T08:40:10.247Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F573171%2Faf686a59c57ff59b669e1d5c16a4afd9%2F20200104_140309.jpg?generation=1578127207878956&alt=media\")
When you hear another 5.X LB update but still no final LB
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F573171%2F23f547840ded1aa90496df0b7c192344%2F78bb1ad13ad0b8952a8e860bf4791ddc.jpeg?generation=1578396619993828&alt=media\")
On a serious note, I do appreciate all effort Kaggle team is putting in to sort this out!!\nCheers!
", "rawMarkdown": "When you hear another 5.X LB update but still no final LB\n\n\n\n\nOn a serious note, I do appreciate all effort Kaggle team is putting in to sort this out!!\nCheers!", "votes": 3, "replies": [ { "id": 771356, "postDate": "2020-03-14T04:36:15.360Z", "content": "👍
", "rawMarkdown": "👍 " } ] }, { "id": 704813, "postDate": "2019-12-28T03:18:51.677Z", "content": "Hi @sohier and Adison! Please have a great time during the new year! Thanks for all the work you guys put in!
", "rawMarkdown": "Hi @sohier and Adison! Please have a great time during the new year! Thanks for all the work you guys put in!", "votes": 3 }, { "id": 712537, "postDate": "2020-01-07T10:57:25.910Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2190082%2Fc4b157c8777a9f73e210db18d5a959c1%2F7ece7d2230498b44afdfb2bfc424928c.jpg?generation=1578394643669609&alt=media\")
Just out of curiosity, what is the validation process you are performing to produce the final leaderboard? other than computing RMSLE on non-leaked data?
", "rawMarkdown": "Just out of curiosity, what is the validation process you are performing to produce the final leaderboard? other than computing RMSLE on non-leaked data?", "votes": 1, "replies": [ { "id": 712840, "postDate": "2020-01-07T16:25:01.660Z", "content": "A process involving a whole lot of tribbles.
", "rawMarkdown": "A process involving a whole lot of tribbles." } ] }, { "id": 709665, "postDate": "2020-01-03T19:41:15.473Z", "content": "I don’t think so...
\n\n15 days to complete the leaderboard, I’m afraid.
\n\nKinda longest competition in time of deliver a final leaderboard, not the usual time.
\n\nWhat can we do ?
", "rawMarkdown": "I don’t think so...\n\n15 days to complete the leaderboard, I’m afraid.\n\nKinda longest competition in time of deliver a final leaderboard, not the usual time.\n\nWhat can we do ?", "votes": 1 }, { "id": 709523, "postDate": "2020-01-03T16:04:01.720Z", "content": "I put in my resume that I got a bronze medal xd
", "rawMarkdown": "I put in my resume that I got a bronze medal xd", "votes": 1 }, { "id": 706623, "postDate": "2019-12-30T16:18:34.197Z", "content": "Can Addison or someone from the team confirm if the results that were shared via CSV a few days back will be integrated with the official LB today? Or are we looking at releasing final LB after 1st Jan? Thanks in advance.
", "rawMarkdown": "Can Addison or someone from the team confirm if the results that were shared via CSV a few days back will be integrated with the official LB today? Or are we looking at releasing final LB after 1st Jan? Thanks in advance.", "votes": 1, "replies": [ { "id": 706659, "postDate": "2019-12-30T17:11:58.930Z", "content": "We're hoping to have it out today! There are a few pressing items for our engineering team, but fingers crossed! If not, it will be after 1 Jan unfortunately.
", "rawMarkdown": "We're hoping to have it out today! There are a few pressing items for our engineering team, but fingers crossed! If not, it will be after 1 Jan unfortunately.", "votes": 6 }, { "id": 706689, "postDate": "2019-12-30T17:34:41.050Z", "rawMarkdown": "", "isDeleted": true }, { "id": 706694, "postDate": "2019-12-30T17:41:02.860Z", "content": "Hi Masoud,
\n\nI have replied to your email with the reasoning for your removal.
", "rawMarkdown": "Hi Masoud,\n\nI have replied to your email with the reasoning for your removal." }, { "id": 706723, "postDate": "2019-12-30T18:26:20.123Z", "content": "Thanks a lot @addisonhoward and the team, for working through these days! Have a great year ahead!
", "rawMarkdown": "Thanks a lot @addisonhoward and the team, for working through these days! Have a great year ahead!" }, { "id": 706755, "postDate": "2019-12-30T19:30:03.967Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 704789, "postDate": "2019-12-28T02:18:33.623Z", "content": "Does the private score shows the selected submission or the best score in all submissions? could anybody tell!!
", "rawMarkdown": "Does the private score shows the selected submission or the best score in all submissions? could anybody tell!!", "votes": 1, "replies": [ { "id": 704794, "postDate": "2019-12-28T02:24:58.987Z", "content": "For me it looks like the best of selected submissions.
", "rawMarkdown": "For me it looks like the best of selected submissions. " }, { "id": 704799, "postDate": "2019-12-28T02:29:37.110Z", "content": "For me it is the best of selected submissions.because my best one of the final_all_scores.csv didn't appear in final_leaderboard_order.csv.
", "rawMarkdown": "For me it is the best of selected submissions.because my best one of the final_all_scores.csv didn't appear in final_leaderboard_order.csv.\n\n" }, { "id": 704840, "postDate": "2019-12-28T04:23:26.440Z", "content": "I miss the all score file... Thanks, mine is also the best of selected submission
", "rawMarkdown": "I miss the all score file... Thanks, mine is also the best of selected submission" } ] }, { "id": 710745, "postDate": "2020-01-05T06:49:53.003Z", "content": "I hope the leaderboard gets updated before the WW3 starts. Waiting so eagerly to see the competition expert title in my profile 😭.
", "rawMarkdown": "I hope the leaderboard gets updated before the WW3 starts. Waiting so eagerly to see the competition expert title in my profile 😭.", "votes": 2, "replies": [ { "id": 710968, "postDate": "2020-01-05T13:44:19.567Z", "content": "Haha.. Same here mate! I'm eagerly waiting for that competitions expert title on my profile. Congratulations man!
", "rawMarkdown": "Haha.. Same here mate! I'm eagerly waiting for that competitions expert title on my profile. Congratulations man!", "votes": 2 }, { "id": 711037, "postDate": "2020-01-05T15:28:38.920Z", "content": "Thanks ^^
", "rawMarkdown": "Thanks ^^", "votes": 1 }, { "id": 711073, "postDate": "2020-01-05T16:49:51.087Z", "content": "Even i am waiting for same
", "rawMarkdown": "Even i am waiting for same", "votes": 1 } ] }, { "id": 704952, "postDate": "2019-12-28T07:56:48.413Z", "content": "Yay, top 20% ish for my first competition.\nAlso, What's up with the insane amount of tie for both public and private lb?
", "rawMarkdown": "Yay, top 20% ish for my first competition.\nAlso, What's up with the insane amount of tie for both public and private lb?", "votes": 2, "replies": [ { "id": 706693, "postDate": "2019-12-30T17:38:36.190Z", "content": "Those are submissions from public kernels.
", "rawMarkdown": "Those are submissions from public kernels.", "votes": 2 } ] }, { "id": 704849, "postDate": "2019-12-28T04:43:24.083Z", "content": "1877 or 3594 teams? What is the final number of teams?
", "rawMarkdown": "1877 or 3594 teams? What is the final number of teams? ", "votes": 2, "replies": [ { "id": 704925, "postDate": "2019-12-28T06:58:41.960Z", "content": "same to you...
\nis \"1877\" didn't update nor \"3594\" didn't filter?
deleted
", "rawMarkdown": "deleted", "votes": -2 }, { "id": 709506, "postDate": "2020-01-03T15:42:21.360Z", "content": "Thanks for sharing that was just awesome
", "rawMarkdown": "Thanks for sharing that was just awesome", "votes": -6 }, { "id": 706554, "postDate": "2019-12-30T14:56:04.963Z", "content": "nice !!! lovely
", "rawMarkdown": "nice !!! lovely ", "votes": -1 }, { "id": 704800, "postDate": "2019-12-28T02:34:33.457Z", "content": "Which sites are used for the final LB?
", "rawMarkdown": "Which sites are used for the final LB?", "votes": -1 }, { "id": 704774, "postDate": "2019-12-28T01:59:12.903Z", "content": "Someone should make a filtered leaderboard. I need to go to sleep for now
", "rawMarkdown": "Someone should make a filtered leaderboard. I need to go to sleep for now", "votes": -1 }, { "id": 711441, "postDate": "2020-01-06T05:22:19.013Z", "content": "so today is the day ?
", "rawMarkdown": "so today is the day ?", "replies": [ { "id": 711443, "postDate": "2020-01-06T05:26:32.840Z", "content": "Maybe unlikely because Addison mentioned that they will have a meeting today to find a resolution to the issues. My guess would be that best case scenario would be sometime on 7th Jan
", "rawMarkdown": "Maybe unlikely because Addison mentioned that they will have a meeting today to find a resolution to the issues. My guess would be that best case scenario would be sometime on 7th Jan", "votes": 2 } ] }, { "id": 710701, "postDate": "2020-01-05T05:03:29.850Z", "content": "Awesome competition
", "rawMarkdown": "Awesome competition" }, { "id": 706912, "postDate": "2019-12-31T01:57:15.843Z", "content": "Seems we can't see the final privateLB in 2019...
", "rawMarkdown": "Seems we can't see the final privateLB in 2019...", "replies": [ { "id": 709525, "postDate": "2020-01-03T16:04:43.980Z", "content": "neither 2020 xd
", "rawMarkdown": "neither 2020 xd" } ] }, { "id": 706845, "postDate": "2019-12-30T23:03:29.803Z", "content": "\n\nDid I have been removed too.?
\n\nCan’t see my name here !
\n\nNever been a cheater though.
\n\nHave a good night and best New Year’s Eve anyway...
\n\nSee you in 2020, next year.
", "rawMarkdown": "@sohier \n\nDid I have been removed too.? \n\nCan’t see my name here !\n\nNever been a cheater though.\n\nHave a good night and best New Year’s Eve anyway...\n\nSee you in 2020, next year." }, { "id": 706842, "postDate": "2019-12-30T22:57:07.130Z", "content": "I did say January 13th, some days ago, will be my birthday...
\n\nThanks to confirm !!!
", "rawMarkdown": "I did say January 13th, some days ago, will be my birthday... \n\nThanks to confirm !!!" }, { "id": 705027, "postDate": "2019-12-28T10:10:09.547Z", "content": "Dude are you sure you used the best score of the private leader board for ranking? Our teams 'toronto_ds' best score for private leader board was 1.281930 but you used 1.29508 for ranking us
", "rawMarkdown": "Dude are you sure you used the best score of the private leader board for ranking? Our teams 'toronto_ds' best score for private leader board was 1.281930 but you used 1.29508 for ranking us", "replies": [ { "id": 705028, "postDate": "2019-12-28T10:12:31.050Z", "content": "they dont use your best private score for ranking, they chose your best public score (assuming no submissions were selected) and rank you based on that
", "rawMarkdown": "they dont use your best private score for ranking, they chose your best public score (assuming no submissions were selected) and rank you based on that", "votes": 1 }, { "id": 705031, "postDate": "2019-12-28T10:15:01.960Z", "content": "One more thing is, we did not use leaks for improving our score until the very last few days. we kept improving our public scores without leaks, but know I see that the private score of our intial submissions is better than all the improvements we got based on public leader board. \nAre you sure you used an unbiased split between public and private data? the public leader board should be a tool to guide people, not to confuse them!\nobviously I am not talking about LB hacking. But generally, when I see a significant improvement in the public leader board without any leaks, I think I am going in the right direction.\nshouldn't that be the case?! What went wrong?
", "rawMarkdown": "One more thing is, we did not use leaks for improving our score until the very last few days. we kept improving our public scores without leaks, but know I see that the private score of our intial submissions is better than all the improvements we got based on public leader board. \nAre you sure you used an unbiased split between public and private data? the public leader board should be a tool to guide people, not to confuse them!\nobviously I am not talking about LB hacking. But generally, when I see a significant improvement in the public leader board without any leaks, I think I am going in the right direction.\nshouldn't that be the case?! What went wrong?" }, { "id": 705033, "postDate": "2019-12-28T10:16:51.297Z", "content": "why did they score all of the submissions then?
", "rawMarkdown": "why did they score all of the submissions then?" }, { "id": 705037, "postDate": "2019-12-28T10:22:38.537Z", "content": "i assume because some people did not select 2 submissions
", "rawMarkdown": "i assume because some people did not select 2 submissions" } ] }, { "id": 704870, "postDate": "2019-12-28T05:40:30.780Z", "content": "@sohier, thanks for the update and have a wonderful holidays.
", "rawMarkdown": "@sohier, thanks for the update and have a wonderful holidays." }, { "id": 704857, "postDate": "2019-12-28T05:04:05.710Z", "content": "I am satisfied with my private score. It was my best :-)
", "rawMarkdown": "I am satisfied with my private score. It was my best :-) " }, { "id": 704804, "postDate": "2019-12-28T02:48:21.853Z", "content": "Why my best score of the final-all-score.csv is not in the finalleaderboardorder.csv.
", "rawMarkdown": "Why my best score of the final-all-score.csv is not in the finalleaderboardorder.csv.", "replies": [ { "id": 704809, "postDate": "2019-12-28T03:09:53.953Z", "content": "because you didn't select it as your final two submission,I Encounter the same situation as you
", "rawMarkdown": "because you didn't select it as your final two submission,I Encounter the same situation as you" } ] }, { "id": 704802, "postDate": "2019-12-28T02:36:23.740Z", "content": "There are instant data check kernel using pixiedust!\nPlease copy and edit !\nhttps://www.kaggle.com/snakayama/ashrae-lb-191228
", "rawMarkdown": "There are instant data check kernel using pixiedust!\nPlease copy and edit !\nhttps://www.kaggle.com/snakayama/ashrae-lb-191228" }, { "id": 704764, "postDate": "2019-12-28T01:40:59.890Z", "content": "I've been waited so long.\nThank you!\nHave a nice vacation!
", "rawMarkdown": "I've been waited so long.\nThank you!\nHave a nice vacation!" }, { "id": 706499, "postDate": "2019-12-30T13:25:41.177Z", "rawMarkdown": "", "isDeleted": true }, { "id": 706241, "postDate": "2019-12-30T05:38:11.017Z", "rawMarkdown": "", "isDeleted": true }, { "id": 705321, "postDate": "2019-12-28T19:18:34.670Z", "rawMarkdown": "", "isDeleted": true }, { "id": 704807, "postDate": "2019-12-28T03:02:42.710Z", "rawMarkdown": "", "votes": 9, "isDeleted": true, "replies": [ { "id": 704826, "postDate": "2019-12-28T03:44:17.907Z", "content": "I cannot find our team's name too, what's happen?
", "rawMarkdown": "I cannot find our team's name too, what's happen?", "votes": 2 }, { "id": 704869, "postDate": "2019-12-28T05:39:43.187Z", "content": "@smasoudn, @arvissu, send the Kaggle team an email because I don't think they are done with cheaters removal. The number of teams decreased only from 3675 to 3594 in the final_leaderboard_order.csv file
", "rawMarkdown": "@smasoudn, @arvissu, send the Kaggle team an email because I don't think they are done with cheaters removal. The number of teams decreased only from 3675 to 3594 in the final_leaderboard_order.csv file", "votes": 2 }, { "id": 704872, "postDate": "2019-12-28T05:43:09.090Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 704878, "postDate": "2019-12-28T05:51:43.923Z", "content": "@smasoudn, they will probably get back to you next week. I just read @sohier's post again and he mentioned that they have done the cheats review. I hope they will take a closer look at that before posting the final LB.
", "rawMarkdown": "@smasoudn, they will probably get back to you next week. I just read @sohier's post again and he mentioned that they have done the cheats review. I hope they will take a closer look at that before posting the final LB.", "votes": 2 }, { "id": 704947, "postDate": "2019-12-28T07:41:27.253Z", "content": "Can't find my team name too, I really don't know what is going on.Would you please tell me how to contact the organizers?
", "rawMarkdown": "Can't find my team name too, I really don't know what is going on.Would you please tell me how to contact the organizers?", "votes": 1 }, { "id": 704949, "postDate": "2019-12-28T07:47:07.783Z", "content": "Can't find my team name too. what is your email ? we can discuss it
", "rawMarkdown": "Can't find my team name too. what is your email ? we can discuss it" }, { "id": 704966, "postDate": "2019-12-28T08:15:23.543Z", "content": "i just send an email to you
", "rawMarkdown": "i just send an email to you" }, { "id": 705009, "postDate": "2019-12-28T09:48:35.780Z", "content": "Please let me know if there is any reply. I cant find it too
", "rawMarkdown": "Please let me know if there is any reply. I cant find it too" }, { "id": 705012, "postDate": "2019-12-28T09:50:57.443Z", "content": "Even my name is missing in the leaderboard. Is this lb final?? I cant even see my submission scores.
", "rawMarkdown": "Even my name is missing in the leaderboard. Is this lb final?? I cant even see my submission scores.", "votes": 1 } ] }, { "id": 704772, "postDate": "2019-12-28T01:55:50.800Z", "rawMarkdown": "", "isDeleted": true }, { "id": 705561, "postDate": "2019-12-29T04:00:13.693Z", "content": "Thanks
", "rawMarkdown": "Thanks", "votes": 1 }, { "id": 771355, "postDate": "2020-03-14T04:35:56.263Z", "content": "Thanks for sharing
", "rawMarkdown": "Thanks for sharing " }, { "id": 707755, "postDate": "2020-01-01T12:26:59.577Z", "content": "thank you ^^ happy new year
", "rawMarkdown": "thank you ^^ happy new year" }, { "id": 704775, "postDate": "2019-12-28T01:59:34.507Z", "content": "Thank you for your good work !!
", "rawMarkdown": "Thank you for your good work !!" } ], "comments": [ { "id": 704777, "author_name": "Rob Mulla", "author_url": "", "post_date": "2019-12-28T02:01:06.690000", "content": "Congrats everyone! Here is the final lb in a kernel in case you don't want to deal with csvs\nhttps://www.kaggle.com/robikscube/ashrae-leaderboard
", "votes": 24, "replies": [ { "id": 704812, "author_name": "Sri Yogesh", "author_url": "", "post_date": "2019-12-28T03:16:16.720000", "content": "Hi @robikscube thanks a ton for taking the time out and making this. Got a medal and will now be a kaggle competitions expert! Thanks!
", "votes": 4, "replies": [] }, { "id": 704815, "author_name": "Rob Mulla", "author_url": "", "post_date": "2019-12-28T03:19:35.470000", "content": "Thats awesome @sri3128018 - Congrats! ✊
", "votes": 0, "replies": [] }, { "id": 705315, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-12-28T19:04:21.113000", "content": "@robikscube, thanks for sharing. For those interested, check my kernel for the medal ranges based on the original 3675 teams public LB before rescoring.
", "votes": 1, "replies": [] }, { "id": 705353, "author_name": "Rob Mulla", "author_url": "", "post_date": "2019-12-28T20:09:53.650000", "content": "Nice work @sheriytm - I added a little medal emoji column in my kernel based on your medal range calculation!
", "votes": 3, "replies": [] }, { "id": 705370, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-12-28T20:42:59.933000", "content": "Nice @robikscube. Please note that the medals range would not be accurate for the LB csv file list because the number of teams is decreased from the original 3675 to 3594.
\n\nBTW an upvote of my kernel would be nice :-)
", "votes": 1, "replies": [] }, { "id": 705612, "author_name": "Mohammad Farzanullah", "author_url": "", "post_date": "2019-12-29T06:34:22.320000", "content": "In case of 3594 teams, teams till 179 or 180 will get silver and teams till 359 or 360 will get bronze!
", "votes": 1, "replies": [] }, { "id": 706695, "author_name": "Barun Kumar", "author_url": "", "post_date": "2019-12-30T17:41:31.973000", "content": "Hi @robikscube , really appreciate your efforts to convert the CSV into a nice kernel...!! Keep it up...!!
", "votes": 0, "replies": [] }, { "id": 709678, "author_name": "Firat Gonen", "author_url": "", "post_date": "2020-01-03T20:13:37.490000", "content": "based on your kernel, we're getting a bronze (UnderDogs) , fingers crossed, thanks @robikscube
", "votes": 0, "replies": [] } ] }, { "id": 706943, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-12-31T03:45:14.383000", "content": "Leaderboard Update 5.5
\n\nI have marginal news - we likely won't be able to get the formalized leaderboard updated before the New Year. We were able to make some progress on manually processing each submission to display on the leaderboard, but not the full amount. (Again, please note this is an edge case for our competition format, and we don't intend to encounter something like this again)
\n\nA number of you have messaged me directly about missing submissions or questions, and I've responded to each. You're welcome to continue messaging me with questions (or just replying in thread here). The formal compliance appeal process will begin when the leaderboard is published.
\n\nTo provide a bit of a teaser in the meantime, however, we're also working on an exciting new competition format that we think will add a lot of value to your machine learning education and skill refinement a we enter the 2020s!
\n\nHappy New Year to all and again, thanks for your patience!
", "votes": 13, "replies": [ { "id": 706944, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2019-12-31T03:48:46.337000", "content": "Can we know which sites are removed from the private LB?
", "votes": 3, "replies": [] }, { "id": 707126, "author_name": "Alexander Schmidt", "author_url": "", "post_date": "2019-12-31T09:27:38.460000", "content": "Thanks for the update @addisonhoward. Looking forward to the new format!
", "votes": 0, "replies": [] }, { "id": 708535, "author_name": "sandy1112", "author_url": "", "post_date": "2020-01-02T11:54:42.213000", "content": "@addisonhoward - any chance of seeing the LB before Thursday EOD?
", "votes": 1, "replies": [] }, { "id": 711699, "author_name": "Amine MARZOUKI", "author_url": "", "post_date": "2020-01-06T12:43:52.020000", "content": "Hi Addison, \nIf one or some teams have been scored all their submissions, this rule would be generalized? \n@addisonhoward: if it is the case, we would like that you review all ours submissions which gives more relevant evaluation score to ours work in this competition. \nThank you in advance,\n Electrium Z team: @lucamassaron @steubk @pietromarinelli @serigne
", "votes": 0, "replies": [] }, { "id": 711969, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2020-01-06T17:59:55.423000", "content": "@aminepy, if you have not selected your best submission, I do not think rescoring all your submissions would help change your position as shown on the leaderboard because your best score on private LB i.e. 1.266217 is not on top of your public LB score.
\n\nI feel your pain. Mine is not my best either. My best score on the private LB would put me at #27 or so. I selected one submission and the 2nd was chosen automatically based on my best public LB score☹️
", "votes": 1, "replies": [] }, { "id": 712411, "author_name": "Amine MARZOUKI", "author_url": "", "post_date": "2020-01-07T07:47:49.717000", "content": "@sheriytm: Kaggle choose two best if none is chosen\nIn this competition: If you chose the two best before removing the leak, this would be the worst case... \nOn the other hand, as far as mentioned, organizers have removed the leak before processing the LB\nthat means it might have gave advantage to those who didn't choose (or chose only one). This is why we demand for review on all submissions for all teams.
", "votes": 0, "replies": [] }, { "id": 712416, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2020-01-07T08:08:35.353000", "content": "@aminepy, I see. It makes sense that you are asking for all teams. I look forward to @addisonhoward's response.
\n\nMine looks like submissions are ordered with the leaks.
", "votes": 1, "replies": [] }, { "id": 712418, "author_name": "Amine MARZOUKI", "author_url": "", "post_date": "2020-01-07T08:13:01.173000", "content": "@sheriytm: yes the whole review submissions should be done for all teams :)
", "votes": 0, "replies": [] }, { "id": 712430, "author_name": "sandy1112", "author_url": "", "post_date": "2020-01-07T08:31:39.690000", "content": "@sheriytm @aminepy - what are your thoughts on teams who carefully selected the two submissions and got the desired results? I selected one with the leaks and another one which didn't use the leaks. The one with the leak had my best Public LB score but a bad pvt Lb score. However, if I didn't bother selecting my submission then the second best would also be a score that had used the leak and fares poorly on the private set. For my personal submissions, I want to use the submissions that I selected knowing very well that the Pvt LB would be calculated by removing the Pvt test set - the competition organizers had confirmed that before the submissions were closed.
\n\nRescoring using all submissions for all the teams is a bit of a gray area where many would want to do it, where some teams (like me) would not want to do it. Not sure what is feasible solution.
", "votes": 0, "replies": [] }, { "id": 712480, "author_name": "Amine MARZOUKI", "author_url": "", "post_date": "2020-01-07T09:34:19.613000", "content": "@sandy1112: In general, we chose the 2 best scores submissions before the end. However, in this competitions most of teams have a good public LB with leak data. Thus, moving out the leak data and scoring would change the 2 best chosen scores. And we don't know yet how the organizers process to score teams who have chosen 1 submissions or none one? Since that we just suggest to generalize the review of all submissions for all teams which is appropriate in my opinion. \nBy the way, reviewing all submissions and chose the best score is in the interest of all teams.
", "votes": 0, "replies": [] }, { "id": 712485, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2020-01-07T09:42:13.927000", "content": "@Sandy1112, I think what they have done is used our submissions on the public LB before re-scoring for the auto selection. Which I believe is fair since we all know that the leaks will be removed from the private LB.
\n\nThe reason I chose only one submission was because my last submission did not finish scoring for more than 30 minutes until closing. So I let that to be auto-selected as I believed it to be my best.
\n\nMy best private LB score I mentioned earlier also had the leak but was not the best on the public LB.
", "votes": 0, "replies": [] }, { "id": 712501, "author_name": "sandy1112", "author_url": "", "post_date": "2020-01-07T10:01:39.673000", "content": "@aminepy when you say 'reviewing all submissions and chose the best score', do you mean the best Public score or the best Private score? If it's the best Private score, then I agree - its in the interest of all teams. Not sure if organizers would do that. let's hear more on this from Addison and team
", "votes": 0, "replies": [] }, { "id": 712523, "author_name": "Vopani", "author_url": "", "post_date": "2020-01-07T10:33:42.797000", "content": "\n\n\nIf it's the best Private score, then I agree - its in the interest of all teams.
\n
It's not. It will heavily favour teams with high number of submissions and be completely unfair. Process for selecting final submissions is already discussed and confirmed by admins in another thread: https://www.kaggle.com/c/ashrae-energy-prediction/discussion/122987#703901
", "votes": 0, "replies": [] }, { "id": 712553, "author_name": "sandy1112", "author_url": "", "post_date": "2020-01-07T11:17:03.603000", "content": "@rohanrao 💯 agree with you. I think the current method of considering all subs for teams who didn't select any final submission and using the two selected ones for the teams who did make a choice, has some amount of right balance. Although it's pretty difficult to get unified vote in favour of any method that have been discussed in the forum. Every method has its pros and cons.
\n\nI just want the final LB to be published and let's get over with it. Its almost 3 weeks of waiting. Never saw such a long wait for any competitions I have done till date (which is a small duration compared to Kaggle veterans like you)
", "votes": 0, "replies": [] }, { "id": 712614, "author_name": "Amine MARZOUKI", "author_url": "", "post_date": "2020-01-07T12:56:27.547000", "content": "@rohanrao: I'm agree with you and i have seen that discussion. However, in case they have scored teams after check all their submissions, this rule should be generalized for all teams. Sorry but doesn't make any sens to have a good public LB with leak and then move it out form private LB score!! also the private test set has been reduce to 11% So mathematically, your chosen 2 submissions are no longer the best ones. Well, let's see the organizes @addisonhoward opinion about that issues... \n@sandy1112: Hope they can take care for ours comments before publish the private LB! this why, it makes sens to mention right now!! so if you are convinced please do it!
", "votes": 0, "replies": [] }, { "id": 712837, "author_name": "John Oliver Taco López", "author_url": "", "post_date": "2020-01-07T16:24:10.447000", "content": "I think there are a lot of possible solutions, but now there is a private leaderboard. So people who are in a good position would not agree with changing the results. All possible solutions had to be discussed before the publication of the private leaderboard. Now, it's too late.
", "votes": 0, "replies": [] } ] }, { "id": 712176, "author_name": "Sohier Dane", "author_url": "", "post_date": "2020-01-06T22:50:48.213000", "content": "Leaderboard Update 5.6 - more marginal news. The whole team was back in the office today, including the owner of a system we ended up needing to consider. We believe we have a new path for executing the score finalization. Hope to have more news for you tomorrow.
", "votes": 11, "replies": [ { "id": 712180, "author_name": "Jie Wu", "author_url": "", "post_date": "2020-01-06T23:01:16.707000", "content": "@sohier Thanks for the updating.
", "votes": 2, "replies": [] }, { "id": 712206, "author_name": "John Oliver Taco López", "author_url": "", "post_date": "2020-01-07T00:05:29.350000", "content": "Hi, one question. \nThe number of teams will be 1877 or 3500?\nthanks :D
", "votes": 1, "replies": [] }, { "id": 712267, "author_name": "sandy1112", "author_url": "", "post_date": "2020-01-07T02:32:44.667000", "content": "This update was more like a 5.5.1 rather than 5.6. thanks for the update anyway
", "votes": 2, "replies": [] }, { "id": 712378, "author_name": "Utkarsh Tripathi", "author_url": "", "post_date": "2020-01-07T06:55:30.407000", "content": "Score finalization??
\n\nDid the final scores leader board csv shared final or not?
there is also an update 5.75 :)
", "votes": 0, "replies": [] }, { "id": 712438, "author_name": "sandy1112", "author_url": "", "post_date": "2020-01-07T08:35:58.123000", "content": "@frtgnn just noticed that and 5.75 is older than 5.6 ... lol 😂
", "votes": 2, "replies": [] }, { "id": 712473, "author_name": "Firat Gonen", "author_url": "", "post_date": "2020-01-07T09:29:03.410000", "content": "exactly @sandy1112 :)
", "votes": 0, "replies": [] } ] }, { "id": 711433, "author_name": "Mohammad Farzanullah", "author_url": "", "post_date": "2020-01-06T05:01:34.720000", "content": "
Any chance to see the leaderboard today?
", "votes": 8, "replies": [] }, { "id": 709384, "author_name": "sandy1112", "author_url": "", "post_date": "2020-01-03T12:43:01.523000", "content": "
15 days passed...
", "votes": 8, "replies": [] }, { "id": 709828, "author_name": "Sri Yogesh", "author_url": "", "post_date": "2020-01-04T01:08:30.470000", "content": "The competition is now listed under completed competitions instead of active competitions, hoping to hear the good news soon! :)
", "votes": 6, "replies": [ { "id": 709978, "author_name": "Jie Wu", "author_url": "", "post_date": "2020-01-04T06:53:59.340000", "content": "I saw that too. Just want to ask Kaggle @addisonhoward , will the leaderboard be update or not?
", "votes": 1, "replies": [] }, { "id": 709993, "author_name": "Addison Howard", "author_url": "", "post_date": "2020-01-04T07:23:47.903000", "content": "Hey again. Update 5.75.
\n\nWe thought we had a fix to merge the scores into the leaderboard on Thursday but it failed. We need to wait for some of our engineers who are on vacation to return, but we have a meeting scheduled for Monday to work on the resolution.
", "votes": 4, "replies": [] }, { "id": 710033, "author_name": "sandy1112", "author_url": "", "post_date": "2020-01-04T08:40:10.247000", "content": "
When you hear another 5.X LB update but still no final LB
\n\n
On a serious note, I do appreciate all effort Kaggle team is putting in to sort this out!!\nCheers!
", "votes": 3, "replies": [ { "id": 771356, "author_name": "", "author_url": "", "post_date": "2020-03-14T04:36:15.360000", "content": "👍
", "votes": 0, "replies": [] } ] }, { "id": 704813, "author_name": "Sri Yogesh", "author_url": "", "post_date": "2019-12-28T03:18:51.677000", "content": "Hi @sohier and Adison! Please have a great time during the new year! Thanks for all the work you guys put in!
", "votes": 3, "replies": [] }, { "id": 712537, "author_name": "c-weed", "author_url": "", "post_date": "2020-01-07T10:57:25.910000", "content": "
Just out of curiosity, what is the validation process you are performing to produce the final leaderboard? other than computing RMSLE on non-leaked data?
", "votes": 1, "replies": [ { "id": 712840, "author_name": "عثمان", "author_url": "", "post_date": "2020-01-07T16:25:01.660000", "content": "A process involving a whole lot of tribbles.
", "votes": 0, "replies": [] } ] }, { "id": 709665, "author_name": "mezoganet", "author_url": "", "post_date": "2020-01-03T19:41:15.473000", "content": "I don’t think so...
\n\n15 days to complete the leaderboard, I’m afraid.
\n\nKinda longest competition in time of deliver a final leaderboard, not the usual time.
\n\nWhat can we do ?
", "votes": 1, "replies": [] }, { "id": 709523, "author_name": "John Oliver Taco López", "author_url": "", "post_date": "2020-01-03T16:04:01.720000", "content": "I put in my resume that I got a bronze medal xd
", "votes": 1, "replies": [] }, { "id": 706623, "author_name": "sandy1112", "author_url": "", "post_date": "2019-12-30T16:18:34.197000", "content": "Can Addison or someone from the team confirm if the results that were shared via CSV a few days back will be integrated with the official LB today? Or are we looking at releasing final LB after 1st Jan? Thanks in advance.
", "votes": 1, "replies": [ { "id": 706659, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-12-30T17:11:58.930000", "content": "We're hoping to have it out today! There are a few pressing items for our engineering team, but fingers crossed! If not, it will be after 1 Jan unfortunately.
", "votes": 6, "replies": [] }, { "id": 706689, "author_name": "", "author_url": "", "post_date": "2019-12-30T17:34:41.050000", "content": "", "votes": 0, "replies": [] }, { "id": 706694, "author_name": "Addison Howard", "author_url": "", "post_date": "2019-12-30T17:41:02.860000", "content": "Hi Masoud,
\n\nI have replied to your email with the reasoning for your removal.
", "votes": 0, "replies": [] }, { "id": 706723, "author_name": "Sri Yogesh", "author_url": "", "post_date": "2019-12-30T18:26:20.123000", "content": "Thanks a lot @addisonhoward and the team, for working through these days! Have a great year ahead!
", "votes": 0, "replies": [] }, { "id": 706755, "author_name": "", "author_url": "", "post_date": "2019-12-30T19:30:03.967000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 704789, "author_name": "Clancey_Lee", "author_url": "", "post_date": "2019-12-28T02:18:33.623000", "content": "Does the private score shows the selected submission or the best score in all submissions? could anybody tell!!
", "votes": 1, "replies": [ { "id": 704794, "author_name": "Halvard", "author_url": "", "post_date": "2019-12-28T02:24:58.987000", "content": "For me it looks like the best of selected submissions.
", "votes": 0, "replies": [] }, { "id": 704799, "author_name": "Wu_Yiqun", "author_url": "", "post_date": "2019-12-28T02:29:37.110000", "content": "For me it is the best of selected submissions.because my best one of the final_all_scores.csv didn't appear in final_leaderboard_order.csv.
", "votes": 0, "replies": [] }, { "id": 704840, "author_name": "Clancey_Lee", "author_url": "", "post_date": "2019-12-28T04:23:26.440000", "content": "I miss the all score file... Thanks, mine is also the best of selected submission
", "votes": 0, "replies": [] } ] }, { "id": 710745, "author_name": "Md Khairul Islam", "author_url": "", "post_date": "2020-01-05T06:49:53.003000", "content": "I hope the leaderboard gets updated before the WW3 starts. Waiting so eagerly to see the competition expert title in my profile 😭.
", "votes": 2, "replies": [ { "id": 710968, "author_name": "Sri Yogesh", "author_url": "", "post_date": "2020-01-05T13:44:19.567000", "content": "Haha.. Same here mate! I'm eagerly waiting for that competitions expert title on my profile. Congratulations man!
", "votes": 2, "replies": [] }, { "id": 711037, "author_name": "Md Khairul Islam", "author_url": "", "post_date": "2020-01-05T15:28:38.920000", "content": "Thanks ^^
", "votes": 1, "replies": [] }, { "id": 711073, "author_name": "Utkarsh Tripathi", "author_url": "", "post_date": "2020-01-05T16:49:51.087000", "content": "Even i am waiting for same
", "votes": 1, "replies": [] } ] }, { "id": 704952, "author_name": "brachester", "author_url": "", "post_date": "2019-12-28T07:56:48.413000", "content": "Yay, top 20% ish for my first competition.\nAlso, What's up with the insane amount of tie for both public and private lb?
", "votes": 2, "replies": [ { "id": 706693, "author_name": "Vopani", "author_url": "", "post_date": "2019-12-30T17:38:36.190000", "content": "Those are submissions from public kernels.
", "votes": 2, "replies": [] } ] }, { "id": 704849, "author_name": "John Oliver Taco López", "author_url": "", "post_date": "2019-12-28T04:43:24.083000", "content": "1877 or 3594 teams? What is the final number of teams?
", "votes": 2, "replies": [ { "id": 704925, "author_name": "Shiyuan Zeng", "author_url": "", "post_date": "2019-12-28T06:58:41.960000", "content": "same to you...
\nis \"1877\" didn't update nor \"3594\" didn't filter?
deleted
", "votes": -2, "replies": [] }, { "id": 709506, "author_name": "ask9", "author_url": "", "post_date": "2020-01-03T15:42:21.360000", "content": "Thanks for sharing that was just awesome
", "votes": -6, "replies": [] }, { "id": 706554, "author_name": "Ömer Faruk Başpınar", "author_url": "", "post_date": "2019-12-30T14:56:04.963000", "content": "nice !!! lovely
", "votes": -1, "replies": [] }, { "id": 704800, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2019-12-28T02:34:33.457000", "content": "Which sites are used for the final LB?
", "votes": -1, "replies": [] }, { "id": 704774, "author_name": "c-weed", "author_url": "", "post_date": "2019-12-28T01:59:12.903000", "content": "Someone should make a filtered leaderboard. I need to go to sleep for now
", "votes": -1, "replies": [] }, { "id": 711441, "author_name": "Firat Gonen", "author_url": "", "post_date": "2020-01-06T05:22:19.013000", "content": "so today is the day ?
", "votes": 0, "replies": [ { "id": 711443, "author_name": "sandy1112", "author_url": "", "post_date": "2020-01-06T05:26:32.840000", "content": "Maybe unlikely because Addison mentioned that they will have a meeting today to find a resolution to the issues. My guess would be that best case scenario would be sometime on 7th Jan
", "votes": 2, "replies": [] } ] }, { "id": 710701, "author_name": "LeonCom", "author_url": "", "post_date": "2020-01-05T05:03:29.850000", "content": "Awesome competition
", "votes": 0, "replies": [] }, { "id": 706912, "author_name": "Wu_Yiqun", "author_url": "", "post_date": "2019-12-31T01:57:15.843000", "content": "Seems we can't see the final privateLB in 2019...
", "votes": 0, "replies": [ { "id": 709525, "author_name": "John Oliver Taco López", "author_url": "", "post_date": "2020-01-03T16:04:43.980000", "content": "neither 2020 xd
", "votes": 0, "replies": [] } ] }, { "id": 706845, "author_name": "mezoganet", "author_url": "", "post_date": "2019-12-30T23:03:29.803000", "content": "\n\nDid I have been removed too.?
\n\nCan’t see my name here !
\n\nNever been a cheater though.
\n\nHave a good night and best New Year’s Eve anyway...
\n\nSee you in 2020, next year.
", "votes": 0, "replies": [] }, { "id": 706842, "author_name": "mezoganet", "author_url": "", "post_date": "2019-12-30T22:57:07.130000", "content": "I did say January 13th, some days ago, will be my birthday...
\n\nThanks to confirm !!!
", "votes": 0, "replies": [] }, { "id": 705027, "author_name": "Navid Hakimi", "author_url": "", "post_date": "2019-12-28T10:10:09.547000", "content": "Dude are you sure you used the best score of the private leader board for ranking? Our teams 'toronto_ds' best score for private leader board was 1.281930 but you used 1.29508 for ranking us
", "votes": 0, "replies": [ { "id": 705028, "author_name": "Proletheus", "author_url": "", "post_date": "2019-12-28T10:12:31.050000", "content": "they dont use your best private score for ranking, they chose your best public score (assuming no submissions were selected) and rank you based on that
", "votes": 1, "replies": [] }, { "id": 705031, "author_name": "Navid Hakimi", "author_url": "", "post_date": "2019-12-28T10:15:01.960000", "content": "One more thing is, we did not use leaks for improving our score until the very last few days. we kept improving our public scores without leaks, but know I see that the private score of our intial submissions is better than all the improvements we got based on public leader board. \nAre you sure you used an unbiased split between public and private data? the public leader board should be a tool to guide people, not to confuse them!\nobviously I am not talking about LB hacking. But generally, when I see a significant improvement in the public leader board without any leaks, I think I am going in the right direction.\nshouldn't that be the case?! What went wrong?
", "votes": 0, "replies": [] }, { "id": 705033, "author_name": "Navid Hakimi", "author_url": "", "post_date": "2019-12-28T10:16:51.297000", "content": "why did they score all of the submissions then?
", "votes": 0, "replies": [] }, { "id": 705037, "author_name": "Proletheus", "author_url": "", "post_date": "2019-12-28T10:22:38.537000", "content": "i assume because some people did not select 2 submissions
", "votes": 0, "replies": [] } ] }, { "id": 704870, "author_name": "", "author_url": "", "post_date": "2019-12-28T05:40:30.780000", "content": "", "votes": 0, "replies": [] }, { "id": 704857, "author_name": "", "author_url": "", "post_date": "2019-12-28T05:04:05.710000", "content": "", "votes": 0, "replies": [] }, { "id": 704804, "author_name": "", "author_url": "", "post_date": "2019-12-28T02:48:21.853000", "content": "", "votes": 0, "replies": [ { "id": 704809, "author_name": "", "author_url": "", "post_date": "2019-12-28T03:09:53.953000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 704802, "author_name": "", "author_url": "", "post_date": "2019-12-28T02:36:23.740000", "content": "", "votes": 0, "replies": [] }, { "id": 704764, "author_name": "", "author_url": "", "post_date": "2019-12-28T01:40:59.890000", "content": "", "votes": 0, "replies": [] }, { "id": 706499, "author_name": "", "author_url": "", "post_date": "2019-12-30T13:25:41.177000", "content": "", "votes": 0, "replies": [] }, { "id": 706241, "author_name": "", "author_url": "", "post_date": "2019-12-30T05:38:11.017000", "content": "", "votes": 0, "replies": [] }, { "id": 705321, "author_name": "", "author_url": "", "post_date": "2019-12-28T19:18:34.670000", "content": "", "votes": 0, "replies": [] }, { "id": 704807, "author_name": "", "author_url": "", "post_date": "2019-12-28T03:02:42.710000", "content": "", "votes": 9, "replies": [ { "id": 704826, "author_name": "", "author_url": "", "post_date": "2019-12-28T03:44:17.907000", "content": "", "votes": 2, "replies": [] }, { "id": 704869, "author_name": "", "author_url": "", "post_date": "2019-12-28T05:39:43.187000", "content": "", "votes": 2, "replies": [] }, { "id": 704872, "author_name": "", "author_url": "", "post_date": "2019-12-28T05:43:09.090000", "content": "", "votes": 1, "replies": [] }, { "id": 704878, "author_name": "", "author_url": "", "post_date": "2019-12-28T05:51:43.923000", "content": "", "votes": 2, "replies": [] }, { "id": 704947, "author_name": "", "author_url": "", "post_date": "2019-12-28T07:41:27.253000", "content": "", "votes": 1, "replies": [] }, { "id": 704949, "author_name": "", "author_url": "", "post_date": "2019-12-28T07:47:07.783000", "content": "", "votes": 0, "replies": [] }, { "id": 704966, "author_name": "", "author_url": "", "post_date": "2019-12-28T08:15:23.543000", "content": "", "votes": 0, "replies": [] }, { "id": 705009, "author_name": "", "author_url": "", "post_date": "2019-12-28T09:48:35.780000", "content": "", "votes": 0, "replies": [] }, { "id": 705012, "author_name": "", "author_url": "", "post_date": "2019-12-28T09:50:57.443000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 704772, "author_name": "", "author_url": "", "post_date": "2019-12-28T01:55:50.800000", "content": "", "votes": 0, "replies": [] }, { "id": 705561, "author_name": "", "author_url": "", "post_date": "2019-12-29T04:00:13.693000", "content": "", "votes": 1, "replies": [] }, { "id": 771355, "author_name": "", "author_url": "", "post_date": "2020-03-14T04:35:56.263000", "content": "", "votes": 0, "replies": [] }, { "id": 707755, "author_name": "", "author_url": "", "post_date": "2020-01-01T12:26:59.577000", "content": "", "votes": 0, "replies": [] }, { "id": 704775, "author_name": "", "author_url": "", "post_date": "2019-12-28T01:59:34.507000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "704754": "I have good news and bad news. The good news is that we have finished calculating everyone's scores and our cheat review. The bad news is that we seem to have angered Santa; our oncall engineer's workstation broke. Since we're short staffed over the holidays this means we won't be able merge the scores into the official leaderboard until next week. However, I have posted csvs of the leaderboard and all scores here: [https://www.kaggle.com/sohier/ashrae-interim-scores](https://www.kaggle.com/sohier/ashrae-interim-scores).\n\nI will personally be on vacation and offline most of next week and unable to respond to questions until I'm back, but Addison and other members of the team will be online besides new years eve and day.\n\nThank you all again for your patience! We will be conducting an internal review to head off delays like this in the future.", "704777": "Congrats everyone! Here is the final lb in a kernel in case you don't want to deal with csvs\nhttps://www.kaggle.com/robikscube/ashrae-leaderboard", "706943": "**Leaderboard Update 5.5**\n\nI have marginal news - we likely won't be able to get the formalized leaderboard updated before the New Year. We were able to make some progress on manually processing each submission to display on the leaderboard, but not the full amount. (Again, please note this is an edge case for our competition format, and we don't intend to encounter something like this again)\n\nA number of you have messaged me directly about missing submissions or questions, and I've responded to each. You're welcome to continue messaging me with questions (or just replying in thread here). The formal compliance appeal process will begin when the leaderboard is published.\n\nTo provide a bit of a teaser in the meantime, however, we're also working on an exciting new competition format that we think will add a lot of value to your machine learning education and skill refinement a we enter the 2020s!\n\nHappy New Year to all and again, thanks for your patience!\n\n", "712176": "Leaderboard Update 5.6 - more marginal news. The whole team was back in the office today, including the owner of a system we ended up needing to consider. We believe we have a new path for executing the score finalization. Hope to have more news for you tomorrow.", "711433": "\n", "709553": "Any chance to see the leaderboard today?", "709384": "\n\n15 days passed...", "709828": "The competition is now listed under completed competitions instead of active competitions, hoping to hear the good news soon! :)", "712562": "When you hear another 5.X LB update but still no final LB\n\n\n\n\nOn a serious note, I do appreciate all effort Kaggle team is putting in to sort this out!!\nCheers!", "704813": "Hi @sohier and Adison! Please have a great time during the new year! Thanks for all the work you guys put in!", "712537": "\n", "712471": "Just out of curiosity, what is the validation process you are performing to produce the final leaderboard? other than computing RMSLE on non-leaked data?", "709665": "I don’t think so...\n\n15 days to complete the leaderboard, I’m afraid.\n\nKinda longest competition in time of deliver a final leaderboard, not the usual time.\n\nWhat can we do ?", "709523": "I put in my resume that I got a bronze medal xd", "706623": "Can Addison or someone from the team confirm if the results that were shared via CSV a few days back will be integrated with the official LB today? Or are we looking at releasing final LB after 1st Jan? Thanks in advance.", "704789": "Does the private score shows the selected submission or the best score in all submissions? could anybody tell!!", "710745": "I hope the leaderboard gets updated before the WW3 starts. Waiting so eagerly to see the competition expert title in my profile 😭.", "704952": "Yay, top 20% ish for my first competition.\nAlso, What's up with the insane amount of tie for both public and private lb?", "704849": "1877 or 3594 teams? What is the final number of teams? ", "706838": "deleted", "709506": "Thanks for sharing that was just awesome", "706554": "nice !!! lovely ", "704800": "Which sites are used for the final LB?", "704774": "Someone should make a filtered leaderboard. I need to go to sleep for now", "711441": "so today is the day ?", "710701": "Awesome competition", "706912": "Seems we can't see the final privateLB in 2019...", "706845": "@sohier \n\nDid I have been removed too.? \n\nCan’t see my name here !\n\nNever been a cheater though.\n\nHave a good night and best New Year’s Eve anyway...\n\nSee you in 2020, next year.", "706842": "I did say January 13th, some days ago, will be my birthday... \n\nThanks to confirm !!!", "705027": "Dude are you sure you used the best score of the private leader board for ranking? Our teams 'toronto_ds' best score for private leader board was 1.281930 but you used 1.29508 for ranking us", "704870": "@sohier, thanks for the update and have a wonderful holidays.", "704857": "I am satisfied with my private score. It was my best :-) ", "704804": "Why my best score of the final-all-score.csv is not in the finalleaderboardorder.csv.", "704802": "There are instant data check kernel using pixiedust!\nPlease copy and edit !\nhttps://www.kaggle.com/snakayama/ashrae-lb-191228", "704764": "I've been waited so long.\nThank you!\nHave a nice vacation!", "706499": "", "706241": "", "705321": "", "704807": "", "704772": "", "705561": "Thanks", "771355": "Thanks for sharing ", "707755": "thank you ^^ happy new year", "704775": "Thank you for your good work !!" } }