![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2405813%2F5ad2a34b172706566c7772ce05bb395c%2F2019-11-11%20%2012.44.49.png?generation=1573476311656027&alt=media\")
I decided to share this kernel since this is not a secret anymore. The output of the kernel is training and test set combined with only site 0 meter readings, so you can just add the data to your own kernel, and merge it your own dataframes on ['timestamp', 'building_id', 'meter']. You don't need to fork it.
https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels
\n\nEdit: There is bug on Kaggle right now. Kernels can't be switched to public. I'll make it public as soon as it is fixed.
", "messages": [ { "id": "670305", "postDate": "11/11/2019 09:42:20", "content": "I decided to share this kernel since this is not a secret anymore. The output of the kernel is training and test set combined with only site 0 meter readings, so you can just add the data to your own kernel, and merge it your own dataframes on ['timestamp', 'building_id', 'meter']. You don't need to fork it.
https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels
\n\nEdit: There is bug on Kaggle right now. Kernels can't be switched to public. I'll make it public as soon as it is fixed.
", "rawMarkdown": "I decided to share this kernel since this is not a secret anymore. The output of the kernel is training and test set combined with only site 0 meter readings, so you can just add the data to your own kernel, and merge it your own dataframes on `['timestamp', 'building_id', 'meter']`. You don't need to fork it.\n\nhttps://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\n\nEdit: There is bug on Kaggle right now. Kernels can't be switched to public. I'll make it public as soon as it is fixed.", "votes": null }, { "id": "670327", "postDate": "11/11/2019 10:29:18", "content": "I wonder that whether using test labels is allowed?😂
", "rawMarkdown": "I wonder that whether using test labels is allowed?😂", "votes": null }, { "id": "670362", "postDate": "11/11/2019 11:20:01", "content": "Competition organizers and Kaggle Team should disclose \"using\" in this case. It shouldn't be used in training or while making predictions, but it should be okay for EDA in my opinion.
", "rawMarkdown": "Competition organizers and Kaggle Team should disclose \"using\" in this case. It shouldn't be used in training or while making predictions, but it should be okay for EDA in my opinion.", "votes": null }, { "id": "670366", "postDate": "11/11/2019 11:26:03", "content": "@gunesevitan, thanks for sharing.
\n\nI could not see \"meter reading\" column in site0 files or am I looking in the wrong place?
\n\nUPDATE:\nNever mind seen it.
", "rawMarkdown": "gunesevitan, thanks for sharing. \n\nI could not see \"meter reading\" column in site0 files or am I looking in the wrong place?\n\nUPDATE:\nNever mind seen it.", "votes": null }, { "id": "670387", "postDate": "11/11/2019 12:11:30", "content": "Thanks a lot for sharing this. I think we need some clarity on how this will be handled.
\n\n\"Do not use for training or scoring\" is not a straightforward solution as it can be exploited in other ways. Post-processing can be done to overfit the values to closely match the actuals. Certain insights can be used hidden in the code to tune parameters or outputs using the actual labels.
\nWhether or not this data was used then becomes very subjective.
Organizers should acknowledge publicly and allow the use of this data in any way to make it fair. Ignoring Site 0 rows for scoring in private LB is a good option and easy to do since they already mention about not scoring some rows.
", "rawMarkdown": "Thanks a lot for sharing this. I think we need some clarity on how this will be handled.\n\n\"Do not use for training or scoring\" is not a straightforward solution as it can be exploited in other ways. Post-processing can be done to overfit the values to closely match the actuals. Certain insights can be used hidden in the code to tune parameters or outputs using the actual labels. \nWhether or not this data was used then becomes very subjective.\n\nOrganizers should acknowledge publicly and allow the use of this data in any way to make it fair. Ignoring Site 0 rows for scoring in private LB is a good option and easy to do since they already mention about not scoring some rows.", "votes": null }, { "id": "670413", "postDate": "11/11/2019 12:45:19", "content": "
https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698#665239
", "rawMarkdown": "\n\nhttps://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698#665239", "votes": null }, { "id": "670610", "postDate": "11/11/2019 16:43:41", "content": "I bet he was talking about way different things 7 days ago.
", "rawMarkdown": "I bet he was talking about way different things 7 days ago.", "votes": null }, { "id": "670617", "postDate": "11/11/2019 16:50:56", "content": "He's not clearly saying that we can use this data directly in the submission.\nAnd he told nothing about the consequences of using such data leak either in the model or in the submission.\nI just hate that after two months' hard work at last being disqualified.\nNeed some official crew to clarify this.
", "rawMarkdown": "He's not clearly saying that we can use this data directly in the submission.\nAnd he told nothing about the consequences of using such data leak either in the model or in the submission.\nI just hate that after two months' hard work at last being disqualified.\nNeed some official crew to clarify this.", "votes": null }, { "id": "670636", "postDate": "11/11/2019 17:09:59", "content": "\n\n\nMy concern is UCF - as meter readings are publicity available.
\n \nPredicting that at least public LB will be ruined soon if this data won't be excluded.
\n
answer is
\n\n\n\n\nWe're aware, not to worry. Just remember to keep the external data thread up to date as needed.
\n
links were posted\nhttps://www.kaggle.com/c/ashrae-energy-prediction/discussion/112841#665274
\n\nI don't see any reason for disqualification.
", "rawMarkdown": "> My concern is UCF - as meter readings are publicity available.\n\n> Predicting that at least public LB will be ruined soon if this data won't be excluded.\n\nanswer is \n\n> We're aware, not to worry. Just remember to keep the external data thread up to date as needed.\n\nlinks were posted\nhttps://www.kaggle.com/c/ashrae-energy-prediction/discussion/112841#665274\n\nI don't see any reason for disqualification.", "votes": null }, { "id": "670719", "postDate": "11/11/2019 19:25:49", "content": "Thanks for sharing it! This also explains why public/private split is 78/22. I have never seen larger public than private before. So they put all the leaky ones in public probably.
", "rawMarkdown": "Thanks for sharing it! This also explains why public/private split is 78/22. I have never seen larger public than private before. So they put all the leaky ones in public probably.", "votes": null } ], "comments": [ { "id": 670327, "author_name": "daishu", "author_url": "", "post_date": "11/11/2019 10:29:18", "content": "I wonder that whether using test labels is allowed?😂
", "votes": null, "replies": [ { "id": 670362, "author_name": "gunesevitan", "author_url": "", "post_date": "11/11/2019 11:20:01", "content": "Competition organizers and Kaggle Team should disclose \"using\" in this case. It shouldn't be used in training or while making predictions, but it should be okay for EDA in my opinion.
", "votes": null, "replies": [] } ] }, { "id": 670366, "author_name": "sheriytm", "author_url": "", "post_date": "11/11/2019 11:26:03", "content": "@gunesevitan, thanks for sharing.
\n\nI could not see \"meter reading\" column in site0 files or am I looking in the wrong place?
\n\nUPDATE:\nNever mind seen it.
", "votes": null, "replies": [] }, { "id": 670387, "author_name": "rohanrao", "author_url": "", "post_date": "11/11/2019 12:11:30", "content": "Thanks a lot for sharing this. I think we need some clarity on how this will be handled.
\n\n\"Do not use for training or scoring\" is not a straightforward solution as it can be exploited in other ways. Post-processing can be done to overfit the values to closely match the actuals. Certain insights can be used hidden in the code to tune parameters or outputs using the actual labels.
\nWhether or not this data was used then becomes very subjective.
Organizers should acknowledge publicly and allow the use of this data in any way to make it fair. Ignoring Site 0 rows for scoring in private LB is a good option and easy to do since they already mention about not scoring some rows.
", "votes": null, "replies": [ { "id": 670413, "author_name": "kyakovlev", "author_url": "", "post_date": "11/11/2019 12:45:19", "content": "
https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698#665239
", "votes": null, "replies": [] }, { "id": 670610, "author_name": "nroman", "author_url": "", "post_date": "11/11/2019 16:43:41", "content": "I bet he was talking about way different things 7 days ago.
", "votes": null, "replies": [] }, { "id": 670617, "author_name": "jielu0728", "author_url": "", "post_date": "11/11/2019 16:50:56", "content": "He's not clearly saying that we can use this data directly in the submission.\nAnd he told nothing about the consequences of using such data leak either in the model or in the submission.\nI just hate that after two months' hard work at last being disqualified.\nNeed some official crew to clarify this.
", "votes": null, "replies": [] }, { "id": 670636, "author_name": "kyakovlev", "author_url": "", "post_date": "11/11/2019 17:09:59", "content": "\n\n\nMy concern is UCF - as meter readings are publicity available.
\n \nPredicting that at least public LB will be ruined soon if this data won't be excluded.
\n
answer is
\n\n\n\n\nWe're aware, not to worry. Just remember to keep the external data thread up to date as needed.
\n
links were posted\nhttps://www.kaggle.com/c/ashrae-energy-prediction/discussion/112841#665274
\n\nI don't see any reason for disqualification.
", "votes": null, "replies": [] } ] }, { "id": 670719, "author_name": "aerdem4", "author_url": "", "post_date": "11/11/2019 19:25:49", "content": "Thanks for sharing it! This also explains why public/private split is 78/22. I have never seen larger public than private before. So they put all the leaky ones in public probably.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "670305": "I decided to share this kernel since this is not a secret anymore. The output of the kernel is training and test set combined with only site 0 meter readings, so you can just add the data to your own kernel, and merge it your own dataframes on `['timestamp', 'building_id', 'meter']`. You don't need to fork it.\n\nhttps://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\n\nEdit: There is bug on Kaggle right now. Kernels can't be switched to public. I'll make it public as soon as it is fixed.", "670327": "I wonder that whether using test labels is allowed?😂", "670362": "Competition organizers and Kaggle Team should disclose \"using\" in this case. It shouldn't be used in training or while making predictions, but it should be okay for EDA in my opinion.", "670366": "gunesevitan, thanks for sharing. \n\nI could not see \"meter reading\" column in site0 files or am I looking in the wrong place?\n\nUPDATE:\nNever mind seen it.", "670387": "Thanks a lot for sharing this. I think we need some clarity on how this will be handled.\n\n\"Do not use for training or scoring\" is not a straightforward solution as it can be exploited in other ways. Post-processing can be done to overfit the values to closely match the actuals. Certain insights can be used hidden in the code to tune parameters or outputs using the actual labels. \nWhether or not this data was used then becomes very subjective.\n\nOrganizers should acknowledge publicly and allow the use of this data in any way to make it fair. Ignoring Site 0 rows for scoring in private LB is a good option and easy to do since they already mention about not scoring some rows.", "670413": "\n\nhttps://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698#665239", "670610": "I bet he was talking about way different things 7 days ago.", "670617": "He's not clearly saying that we can use this data directly in the submission.\nAnd he told nothing about the consequences of using such data leak either in the model or in the submission.\nI just hate that after two months' hard work at last being disqualified.\nNeed some official crew to clarify this.", "670636": "> My concern is UCF - as meter readings are publicity available.\n\n> Predicting that at least public LB will be ruined soon if this data won't be excluded.\n\nanswer is \n\n> We're aware, not to worry. Just remember to keep the external data thread up to date as needed.\n\nlinks were posted\nhttps://www.kaggle.com/c/ashrae-energy-prediction/discussion/112841#665274\n\nI don't see any reason for disqualification.", "670719": "Thanks for sharing it! This also explains why public/private split is 78/22. I have never seen larger public than private before. So they put all the leaky ones in public probably." }, "source": "meta" }