Some competitors have merged all available ground truth (GT) test labels onto submission file. Some have partial GT merged. Others have no GT in their submission files. The LB is currently scattered. There is still quite a bit of shuffling around at the top, which I expect to continue until the last day. Anyone in the top 1000 may have competitive models and not show their hand by merging all or any of the GT labels.
", "messages": [ { "id": "694473", "postDate": "12/13/2019 16:56:22", "content": "Some competitors have merged all available ground truth (GT) test labels onto submission file. Some have partial GT merged. Others have no GT in their submission files. The LB is currently scattered. There is still quite a bit of shuffling around at the top, which I expect to continue until the last day. Anyone in the top 1000 may have competitive models and not show their hand by merging all or any of the GT labels.
", "rawMarkdown": "Some competitors have merged all available ground truth (GT) test labels onto submission file. Some have partial GT merged. Others have no GT in their submission files. The LB is currently scattered. There is still quite a bit of shuffling around at the top, which I expect to continue until the last day. Anyone in the top 1000 may have competitive models and not show their hand by merging all or any of the GT labels.", "votes": null }, { "id": "694508", "postDate": "12/13/2019 18:10:01", "content": "I also expect it surely.
", "rawMarkdown": "I also expect it surely.", "votes": null }, { "id": "694530", "postDate": "12/13/2019 18:40:30", "content": "When leaderboard will be rescored without leak data?\nIs it safe to use as final submission one without leaked prediction?
", "rawMarkdown": "When leaderboard will be rescored without leak data?\nIs it safe to use as final submission one without leaked prediction?", "votes": null }, { "id": "694607", "postDate": "12/13/2019 20:42:50", "content": "I guess when the competition ends.
", "rawMarkdown": "I guess when the competition ends.", "votes": null }, { "id": "695091", "postDate": "12/14/2019 15:10:01", "content": "If I understand your post correctly, some people are including in their submission file the real meter reading from leak data instead of the predicted meter reading from their machine learning models? if that is the case, what is the point of doing this (apart from getting a better superficial LB score) ? I am a bit confused ...
", "rawMarkdown": "If I understand your post correctly, some people are including in their submission file the real meter reading from leak data instead of the predicted meter reading from their machine learning models? if that is the case, what is the point of doing this (apart from getting a better superficial LB score) ? I am a bit confused ...", "votes": null }, { "id": "695137", "postDate": "12/14/2019 16:41:15", "content": "I believe there will be shakeup due to leaks as you said, but not overfitting.
", "rawMarkdown": "I believe there will be shakeup due to leaks as you said, but not overfitting.", "votes": null }, { "id": "695179", "postDate": "12/14/2019 17:53:40", "content": "I didn't even both mentioning overfitting. Its still possible to overfit using blends. I saw a few notebooks showing how to blend similar to this.
", "rawMarkdown": "I didn't even both mentioning overfitting. Its still possible to overfit using blends. I saw a few notebooks showing how to blend similar to [this](https://www.kaggle.com/yamsam/ashrae-leak-validation-and-more).", "votes": null }, { "id": "695214", "postDate": "12/14/2019 19:17:27", "content": "If you want a more accurate response to public lb and private lb, you should use all leaks. For example, for a 1.078 model, a score of 0.979 after using leakage and 0.973 after using leakage for a 1.065 model, indicating that the 1.065 model is not much better than the 1.078 model in the non leakage part
", "rawMarkdown": "If you want a more accurate response to public lb and private lb, you should use all leaks. For example, for a 1.078 model, a score of 0.979 after using leakage and 0.973 after using leakage for a 1.065 model, indicating that the 1.065 model is not much better than the 1.078 model in the non leakage part", "votes": null }, { "id": "695378", "postDate": "12/15/2019 03:23:39", "content": "What is the leak data?
", "rawMarkdown": "What is the leak data?", "votes": null }, { "id": "695455", "postDate": "12/15/2019 07:10:57", "content": "Because a model trained on and overfit to the leak data would have a better LB score than a model that generalizes well if you didn't replace both of their \"predictions\" with the real meter readings from the leak data. And if everyone included the actual readings in their submission, the public LB would be more useful
", "rawMarkdown": "Because a model trained on and overfit to the leak data would have a better LB score than a model that generalizes well if you didn't replace both of their \"predictions\" with the real meter readings from the leak data. And if *everyone* included the actual readings in their submission, the public LB would be more useful", "votes": null }, { "id": "696807", "postDate": "12/17/2019 04:50:18", "content": "Seems there should be shakeup. Indeed, this competition makes me feel that LGBM stands not for LightGBM but for LeakGBM 😄
", "rawMarkdown": "Seems there should be shakeup. Indeed, this competition makes me feel that LGBM stands not for LightGBM but for LeakGBM 😄", "votes": null } ], "comments": [ { "id": 694508, "author_name": "hanjoonchoe", "author_url": "", "post_date": "12/13/2019 18:10:01", "content": "I also expect it surely.
", "votes": null, "replies": [] }, { "id": 694530, "author_name": "stenford23", "author_url": "", "post_date": "12/13/2019 18:40:30", "content": "When leaderboard will be rescored without leak data?\nIs it safe to use as final submission one without leaked prediction?
", "votes": null, "replies": [ { "id": 694607, "author_name": "prschdt", "author_url": "", "post_date": "12/13/2019 20:42:50", "content": "I guess when the competition ends.
", "votes": null, "replies": [] } ] }, { "id": 695091, "author_name": "nanowa", "author_url": "", "post_date": "12/14/2019 15:10:01", "content": "If I understand your post correctly, some people are including in their submission file the real meter reading from leak data instead of the predicted meter reading from their machine learning models? if that is the case, what is the point of doing this (apart from getting a better superficial LB score) ? I am a bit confused ...
", "votes": null, "replies": [ { "id": 695455, "author_name": "rahulsd91", "author_url": "", "post_date": "12/15/2019 07:10:57", "content": "Because a model trained on and overfit to the leak data would have a better LB score than a model that generalizes well if you didn't replace both of their \"predictions\" with the real meter readings from the leak data. And if everyone included the actual readings in their submission, the public LB would be more useful
", "votes": null, "replies": [] } ] }, { "id": 695137, "author_name": "gunesevitan", "author_url": "", "post_date": "12/14/2019 16:41:15", "content": "I believe there will be shakeup due to leaks as you said, but not overfitting.
", "votes": null, "replies": [ { "id": 695179, "author_name": "teeyee314", "author_url": "", "post_date": "12/14/2019 17:53:40", "content": "I didn't even both mentioning overfitting. Its still possible to overfit using blends. I saw a few notebooks showing how to blend similar to this.
", "votes": null, "replies": [] } ] }, { "id": 695214, "author_name": "firstbloody", "author_url": "", "post_date": "12/14/2019 19:17:27", "content": "If you want a more accurate response to public lb and private lb, you should use all leaks. For example, for a 1.078 model, a score of 0.979 after using leakage and 0.973 after using leakage for a 1.065 model, indicating that the 1.065 model is not much better than the 1.078 model in the non leakage part
", "votes": null, "replies": [] }, { "id": 695378, "author_name": "yadvik", "author_url": "", "post_date": "12/15/2019 03:23:39", "content": "What is the leak data?
", "votes": null, "replies": [] }, { "id": 696807, "author_name": "starl1ght", "author_url": "", "post_date": "12/17/2019 04:50:18", "content": "Seems there should be shakeup. Indeed, this competition makes me feel that LGBM stands not for LightGBM but for LeakGBM 😄
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "694473": "Some competitors have merged all available ground truth (GT) test labels onto submission file. Some have partial GT merged. Others have no GT in their submission files. The LB is currently scattered. There is still quite a bit of shuffling around at the top, which I expect to continue until the last day. Anyone in the top 1000 may have competitive models and not show their hand by merging all or any of the GT labels.", "694508": "I also expect it surely.", "694530": "When leaderboard will be rescored without leak data?\nIs it safe to use as final submission one without leaked prediction?", "694607": "I guess when the competition ends.", "695091": "If I understand your post correctly, some people are including in their submission file the real meter reading from leak data instead of the predicted meter reading from their machine learning models? if that is the case, what is the point of doing this (apart from getting a better superficial LB score) ? I am a bit confused ...", "695137": "I believe there will be shakeup due to leaks as you said, but not overfitting.", "695179": "I didn't even both mentioning overfitting. Its still possible to overfit using blends. I saw a few notebooks showing how to blend similar to [this](https://www.kaggle.com/yamsam/ashrae-leak-validation-and-more).", "695214": "If you want a more accurate response to public lb and private lb, you should use all leaks. For example, for a 1.078 model, a score of 0.979 after using leakage and 0.973 after using leakage for a 1.065 model, indicating that the 1.065 model is not much better than the 1.078 model in the non leakage part", "695378": "What is the leak data?", "695455": "Because a model trained on and overfit to the leak data would have a better LB score than a model that generalizes well if you didn't replace both of their \"predictions\" with the real meter readings from the leak data. And if *everyone* included the actual readings in their submission, the public LB would be more useful", "696807": "Seems there should be shakeup. Indeed, this competition makes me feel that LGBM stands not for LightGBM but for LeakGBM 😄" }, "source": "meta" }