![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3562473%2F031dcb6b9a26c607f8d3b96c34cfa08a%2F520120B5-D2E3-4727-8F57-41FE3A83E965.png?generation=1574160364041990&alt=media\")
\njust like this data.https://engagementdashboard.com/ucb/ucb/building/270\nthis is UC Berkeley energy web. Maybe it's can be useful.\nhttps://facilities.umn.edu/fm-teams/energy-management\nthis is University of Minnesota energy web. Maybe it's can be useful.
\n\nand I have to test the Southampton data ,but it's not in this competitions.\nhttps://data.southampton.ac.uk/time-series?\nyou can use like [https://data.southampton.ac.uk/time-series?action=fetch&series=elec/b1/ekw&type=average&format=json&resolution=3600&startTime=1451602800000.0&endTime=1546304400000.0] to get json data.
\n\nmore building data you can used this link
\n\nhttps://data.southampton.ac.uk/time-series?action=fetch&series=elec/b2/ekw&type=average&format=html&resolution=3600&startTime=1573553038000.0
", "messages": [ { "id": "676568", "postDate": "11/19/2019 10:06:29", "content": "https://engagementdashboard.com/ucb/ucb/building/270\nthis is UC Berkeley energy web. Maybe it's can be useful.\nhttps://facilities.umn.edu/fm-teams/energy-management\nthis is University of Minnesota energy web. Maybe it's can be useful.
\n\nand I have to test the Southampton data ,but it's not in this competitions.\nhttps://data.southampton.ac.uk/time-series?\nyou can use like [https://data.southampton.ac.uk/time-series?action=fetch&series=elec/b1/ekw&type=average&format=json&resolution=3600&startTime=1451602800000.0&endTime=1546304400000.0] to get json data.
\n\nmore building data you can used this link
\n\nhttps://data.southampton.ac.uk/time-series?action=fetch&series=elec/b2/ekw&type=average&format=html&resolution=3600&startTime=1573553038000.0
", "rawMarkdown": "https://engagementdashboard.com/ucb/ucb/building/270\nthis is UC Berkeley energy web. Maybe it's can be useful.\nhttps://facilities.umn.edu/fm-teams/energy-management\nthis is University of Minnesota energy web. Maybe it's can be useful.\n\nand I have to test the Southampton data ,but it's not in this competitions.\nhttps://data.southampton.ac.uk/time-series?\nyou can use like [https://data.southampton.ac.uk/time-series?action=fetch&series=elec/b1/ekw&type=average&format=json&resolution=3600&startTime=1451602800000.0&endTime=1546304400000.0] to get json data.\n\nmore building data you can used this link\n\n[https://data.southampton.ac.uk/time-series?action=fetch&series=elec/b2/ekw&type=average&format=html&resolution=3600&startTime=1573553038000.0]()", "votes": null }, { "id": "676571", "postDate": "11/19/2019 10:10:12", "content": "I finded a lot real data in BuildingOS . But I can't get the data.
", "rawMarkdown": "I finded a lot real data in BuildingOS . But I can't get the data.", "votes": null }, { "id": "676580", "postDate": "11/19/2019 10:22:59", "content": "Hi @pp2file Congratulations , do you use these data to replace the values in the submissions directly or you are using them to fine tune the model ? , If you don't mind can i ask you about your CV strategy ?
", "rawMarkdown": "Hi @pp2file Congratulations , do you use these data to replace the values in the submissions directly or you are using them to fine tune the model ? , If you don't mind can i ask you about your CV strategy ?", "votes": null }, { "id": "676588", "postDate": "11/19/2019 10:30:01", "content": "Thank you for sharing.\nI’m afraid that all the test labels will be leaked before the deadline and this really interesting competition turns into completely meaningless.
", "rawMarkdown": "Thank you for sharing.\nI’m afraid that all the test labels will be leaked before the deadline and this really interesting competition turns into completely meaningless.", "votes": null }, { "id": "676593", "postDate": "11/19/2019 10:32:35", "content": "I don't think you have to worry about this problem. because it can not be happen. i have try to find a lot of data. but some site can not be finded.
", "rawMarkdown": "I don't think you have to worry about this problem. because it can not be happen. i have try to find a lot of data. but some site can not be finded.", "votes": null }, { "id": "676597", "postDate": "11/19/2019 10:36:52", "content": "in the public score , i just used these data to replace. but i think the real useful skill is to finding dirty points in training data. I found this data is very very easily to overfitting. so i don't used shuffle in Kfold. don't used to many splits(just use 2 folds).
", "rawMarkdown": "in the public score , i just used these data to replace. but i think the real useful skill is to finding dirty points in training data. I found this data is very very easily to overfitting. so i don't used shuffle in Kfold. don't used to many splits(just use 2 folds).", "votes": null }, { "id": "676598", "postDate": "11/19/2019 10:37:05", "content": "Thank you! I’m glad to hear that.
", "rawMarkdown": "Thank you! I’m glad to hear that.", "votes": null }, { "id": "676602", "postDate": "11/19/2019 10:42:59", "content": "Thanks for the response , I agree with the face that the data is too prone to overfitting , what do you mean by dirty points in the training data?
", "rawMarkdown": "Thanks for the response , I agree with the face that the data is too prone to overfitting , what do you mean by dirty points in the training data?", "votes": null }, { "id": "676605", "postDate": "11/19/2019 10:46:16", "content": "\njust like this data.
We have reason to believe that enough of the sites are unavailable online that there will be a meaningful final leaderboard.
", "rawMarkdown": "We have reason to believe that enough of the sites are unavailable online that there will be a meaningful final leaderboard.", "votes": null }, { "id": "676981", "postDate": "11/19/2019 17:18:32", "content": "Yeah but we can't get any meaningful feedback no more from public LB and that's painful 😨 0.98 on the public LB now can finally get about the same score of 1.06 on the public LB (who doesn't use leaked labels) when switching to private.
", "rawMarkdown": "Yeah but we can't get any meaningful feedback no more from public LB and that's painful 😨 0.98 on the public LB now can finally get about the same score of 1.06 on the public LB (who doesn't use leaked labels) when switching to private.", "votes": null }, { "id": "677056", "postDate": "11/19/2019 18:41:04", "content": "@sohier why not to say openly which sites will not be used in the final score estimations? if I do not read the forum carefully or someone does not share a link with another site from the internet, you insist on people investing their time into wasteful effort?
", "rawMarkdown": "sohier why not to say openly which sites will not be used in the final score estimations? if I do not read the forum carefully or someone does not share a link with another site from the internet, you insist on people investing their time into wasteful effort?", "votes": null }, { "id": "677155", "postDate": "11/19/2019 21:29:14", "content": "I would even go one step further: completely remove the leaked data from both the public and private test sets and get a clean LB.
", "rawMarkdown": "I would even go one step further: completely remove the leaked data from both the public and private test sets and get a clean LB.", "votes": null }, { "id": "677231", "postDate": "11/20/2019 00:05:40", "content": "this weekend i will open all leakage data i have.\nand there have two news to say.\n1. bad news. maybe one site i have finded, it's in Private LB. it's realy realy is a bad news. so i want to double check this thing.\n2. good news. i think it's a very important thing. must let everybody to know that,let the compation fair.
", "rawMarkdown": "this weekend i will open all leakage data i have.\nand there have two news to say.\n1. bad news. maybe one site i have finded, it's in Private LB. it's realy realy is a bad news. so i want to double check this thing.\n2. good news. i think it's a very important thing. must let everybody to know that,let the compation fair.", "votes": null }, { "id": "677232", "postDate": "11/20/2019 00:07:37", "content": "Tanks @pp2file, and I look forward to your update.
", "rawMarkdown": "Tanks @pp2file, and I look forward to your update.", "votes": null }, { "id": "677233", "postDate": "11/20/2019 00:08:05", "content": "but now i think some things happened.
", "rawMarkdown": "but now i think some things happened.", "votes": null }, { "id": "677454", "postDate": "11/20/2019 07:07:55", "content": "How do you know it will / won't be scored in private LB?\nFrom what the Kaggle admins responded here, it seems they can decide which rows not to score from test data at a later stage after all public data has been shared / released.
\n\nSo I would assume any public data that is shared on the forums would eventually be removed from private LB scoring. Or at least it should, to be fair, clean and actually useful as a competition.
", "rawMarkdown": "How do you know it will / won't be scored in private LB?\nFrom what the Kaggle admins responded [here](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/117357), it seems they can decide which rows not to score from test data at a later stage after all public data has been shared / released.\n\nSo I would assume any public data that is shared on the forums would eventually be removed from private LB scoring. Or at least it should, to be fair, clean and actually useful as a competition.", "votes": null }, { "id": "677508", "postDate": "11/20/2019 08:58:21", "content": "I retrieved the full data of UC Berkeley from its Engagement Dashboard. Anyone can fork the following kernel: https://www.kaggle.com/serengil/ucb-data-leakage-site-4
", "rawMarkdown": "I retrieved the full data of UC Berkeley from its Engagement Dashboard. Anyone can fork the following kernel: https://www.kaggle.com/serengil/ucb-data-leakage-site-4", "votes": null }, { "id": "680251", "postDate": "11/24/2019 10:32:20", "content": "https://www.kaggle.com/pp2file/ashrae-site15-cornell\nall sixes leakage has describe in there.
", "rawMarkdown": "https://www.kaggle.com/pp2file/ashrae-site15-cornell\nall sixes leakage has describe in there.", "votes": null } ], "comments": [ { "id": 676571, "author_name": "pp2file", "author_url": "", "post_date": "11/19/2019 10:10:12", "content": "I finded a lot real data in BuildingOS . But I can't get the data.
", "votes": null, "replies": [ { "id": 676580, "author_name": "avikrams", "author_url": "", "post_date": "11/19/2019 10:22:59", "content": "Hi @pp2file Congratulations , do you use these data to replace the values in the submissions directly or you are using them to fine tune the model ? , If you don't mind can i ask you about your CV strategy ?
", "votes": null, "replies": [] }, { "id": 676597, "author_name": "pp2file", "author_url": "", "post_date": "11/19/2019 10:36:52", "content": "in the public score , i just used these data to replace. but i think the real useful skill is to finding dirty points in training data. I found this data is very very easily to overfitting. so i don't used shuffle in Kfold. don't used to many splits(just use 2 folds).
", "votes": null, "replies": [] }, { "id": 676602, "author_name": "avikrams", "author_url": "", "post_date": "11/19/2019 10:42:59", "content": "Thanks for the response , I agree with the face that the data is too prone to overfitting , what do you mean by dirty points in the training data?
", "votes": null, "replies": [] }, { "id": 676605, "author_name": "pp2file", "author_url": "", "post_date": "11/19/2019 10:46:16", "content": "\njust like this data.
Thank you for sharing.\nI’m afraid that all the test labels will be leaked before the deadline and this really interesting competition turns into completely meaningless.
", "votes": null, "replies": [ { "id": 676593, "author_name": "pp2file", "author_url": "", "post_date": "11/19/2019 10:32:35", "content": "I don't think you have to worry about this problem. because it can not be happen. i have try to find a lot of data. but some site can not be finded.
", "votes": null, "replies": [] }, { "id": 676598, "author_name": "hiroyukiex", "author_url": "", "post_date": "11/19/2019 10:37:05", "content": "Thank you! I’m glad to hear that.
", "votes": null, "replies": [] }, { "id": 676966, "author_name": "sohier", "author_url": "", "post_date": "11/19/2019 16:57:47", "content": "We have reason to believe that enough of the sites are unavailable online that there will be a meaningful final leaderboard.
", "votes": null, "replies": [] }, { "id": 676981, "author_name": "jielu0728", "author_url": "", "post_date": "11/19/2019 17:18:32", "content": "Yeah but we can't get any meaningful feedback no more from public LB and that's painful 😨 0.98 on the public LB now can finally get about the same score of 1.06 on the public LB (who doesn't use leaked labels) when switching to private.
", "votes": null, "replies": [] }, { "id": 677056, "author_name": "snovik1975", "author_url": "", "post_date": "11/19/2019 18:41:04", "content": "@sohier why not to say openly which sites will not be used in the final score estimations? if I do not read the forum carefully or someone does not share a link with another site from the internet, you insist on people investing their time into wasteful effort?
", "votes": null, "replies": [] }, { "id": 677155, "author_name": "panosc", "author_url": "", "post_date": "11/19/2019 21:29:14", "content": "I would even go one step further: completely remove the leaked data from both the public and private test sets and get a clean LB.
", "votes": null, "replies": [] }, { "id": 677233, "author_name": "pp2file", "author_url": "", "post_date": "11/20/2019 00:08:05", "content": "but now i think some things happened.
", "votes": null, "replies": [] } ] }, { "id": 677231, "author_name": "pp2file", "author_url": "", "post_date": "11/20/2019 00:05:40", "content": "this weekend i will open all leakage data i have.\nand there have two news to say.\n1. bad news. maybe one site i have finded, it's in Private LB. it's realy realy is a bad news. so i want to double check this thing.\n2. good news. i think it's a very important thing. must let everybody to know that,let the compation fair.
", "votes": null, "replies": [ { "id": 677232, "author_name": "sheriytm", "author_url": "", "post_date": "11/20/2019 00:07:37", "content": "Tanks @pp2file, and I look forward to your update.
", "votes": null, "replies": [] }, { "id": 677454, "author_name": "rohanrao", "author_url": "", "post_date": "11/20/2019 07:07:55", "content": "How do you know it will / won't be scored in private LB?\nFrom what the Kaggle admins responded here, it seems they can decide which rows not to score from test data at a later stage after all public data has been shared / released.
\n\nSo I would assume any public data that is shared on the forums would eventually be removed from private LB scoring. Or at least it should, to be fair, clean and actually useful as a competition.
", "votes": null, "replies": [] }, { "id": 677508, "author_name": "pp2file", "author_url": "", "post_date": "11/20/2019 08:58:21", "content": "https://www.kaggle.com/pp2file/ashrae-site15-cornell\nall sixes leakage has describe in there.
", "votes": null, "replies": [] } ] }, { "id": 677932, "author_name": "serengil", "author_url": "", "post_date": "11/20/2019 19:39:40", "content": "I retrieved the full data of UC Berkeley from its Engagement Dashboard. Anyone can fork the following kernel: https://www.kaggle.com/serengil/ucb-data-leakage-site-4
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "676568": "https://engagementdashboard.com/ucb/ucb/building/270\nthis is UC Berkeley energy web. Maybe it's can be useful.\nhttps://facilities.umn.edu/fm-teams/energy-management\nthis is University of Minnesota energy web. Maybe it's can be useful.\n\nand I have to test the Southampton data ,but it's not in this competitions.\nhttps://data.southampton.ac.uk/time-series?\nyou can use like [https://data.southampton.ac.uk/time-series?action=fetch&series=elec/b1/ekw&type=average&format=json&resolution=3600&startTime=1451602800000.0&endTime=1546304400000.0] to get json data.\n\nmore building data you can used this link\n\n[https://data.southampton.ac.uk/time-series?action=fetch&series=elec/b2/ekw&type=average&format=html&resolution=3600&startTime=1573553038000.0]()", "676571": "I finded a lot real data in BuildingOS . But I can't get the data.", "676580": "Hi @pp2file Congratulations , do you use these data to replace the values in the submissions directly or you are using them to fine tune the model ? , If you don't mind can i ask you about your CV strategy ?", "676588": "Thank you for sharing.\nI’m afraid that all the test labels will be leaked before the deadline and this really interesting competition turns into completely meaningless.", "676593": "I don't think you have to worry about this problem. because it can not be happen. i have try to find a lot of data. but some site can not be finded.", "676597": "in the public score , i just used these data to replace. but i think the real useful skill is to finding dirty points in training data. I found this data is very very easily to overfitting. so i don't used shuffle in Kfold. don't used to many splits(just use 2 folds).", "676598": "Thank you! I’m glad to hear that.", "676602": "Thanks for the response , I agree with the face that the data is too prone to overfitting , what do you mean by dirty points in the training data?", "676605": "\njust like this data.", "676966": "We have reason to believe that enough of the sites are unavailable online that there will be a meaningful final leaderboard.", "676981": "Yeah but we can't get any meaningful feedback no more from public LB and that's painful 😨 0.98 on the public LB now can finally get about the same score of 1.06 on the public LB (who doesn't use leaked labels) when switching to private.", "677056": "sohier why not to say openly which sites will not be used in the final score estimations? if I do not read the forum carefully or someone does not share a link with another site from the internet, you insist on people investing their time into wasteful effort?", "677155": "I would even go one step further: completely remove the leaked data from both the public and private test sets and get a clean LB.", "677231": "this weekend i will open all leakage data i have.\nand there have two news to say.\n1. bad news. maybe one site i have finded, it's in Private LB. it's realy realy is a bad news. so i want to double check this thing.\n2. good news. i think it's a very important thing. must let everybody to know that,let the compation fair.", "677232": "Tanks @pp2file, and I look forward to your update.", "677233": "but now i think some things happened.", "677454": "How do you know it will / won't be scored in private LB?\nFrom what the Kaggle admins responded [here](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/117357), it seems they can decide which rows not to score from test data at a later stage after all public data has been shared / released.\n\nSo I would assume any public data that is shared on the forums would eventually be removed from private LB scoring. Or at least it should, to be fair, clean and actually useful as a competition.", "677508": "1. I used a site data(this weekend I will be open kernel) to submit, but the public LB did not improve.\n2. because, I have find some site data ,not yet be shared.", "677932": "I retrieved the full data of UC Berkeley from its Engagement Dashboard. Anyone can fork the following kernel: https://www.kaggle.com/serengil/ucb-data-leakage-site-4", "680251": "https://www.kaggle.com/pp2file/ashrae-site15-cornell\nall sixes leakage has describe in there." }, "source": "meta" }