![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1963347%2F44ce500d49a59212fcffb0141e55d3d3%2FAvg%20daily%20temperature.png?generation=1572379065265626&alt=media\")
TLDR:\nI hypothesize that:\nSite 12 = Dublin\nSite 1 = London\nSite 5 = Loughborough
\n\nAlso, buildings in site 12 seem to turn things off 1 day a month (maintenance?)
\n\nIn a previous post I detailed my theory that siteIDs 1, 5, 12 were in Europe, mainly due to their hypothesized timezone, but also because the temperatures seemed to have been collected directly in Celsius (rather than converted from F).
\n\nAnother confirmation came from observing low usage on dates that are UK holidays that I reported in site 1. I will provide more info about holidays in this post.
\n\nIf we graph the daily average temperatures for sites1, 5, 12, they are a bit similar, but with:
\n\n** T(siteID 1)> T(siteID 5) > T(siteID 12)** (my general impression)\nNote that if I had plotted any other site on that same graph, the differences would have been much more pronounced.
\n\n
If you look at the ASHRAE sites in Europe, you have 3 in UK+Ireland: London, Loughborough, Dublin.
\n\nOther Europe sites (Madrid,Lisbon,Athens, Romania) sound like they would have a very different temperature profile.
\n\nLet's see if the pattern of holidays work with our theory.
\n\nSome holidays in Ireland (2016):
\n\nMar 25-28\nMay 2\nJune 6\n*Aug 1*\nOct 31
\n\nSome holidays in the UK (2016):
\n\nMar 25-28\nMay 2\nMay 30\n*Aug 29*
\n\nI bolded the ones that are different
\n\nI filtered meter reading time series data to have only week days and education buildings.
\n\nLet's look at the troughs on the following 3 plots.
\n\nSite 1 (London?) has:\nMay 2\nMay 30\nAug 29
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1963347%2F307965de1e26789ce04bad7a88513b5e%2FMeter%20Reading%20Time%20Series%20Site%201.png?generation=1572380124860698&alt=media\")
Site 5 (Loughborough?) \nMar 25-28\nMay 2\nMay 19 (???)\nMay 30\nJuly 7 (???)\nAug 29
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1963347%2Fc66204f20d15b33cb632e1dd6c16375e%2FMeter%20Reading%20Time%20Series%20site%205.png?generation=1572380158964048&alt=media\")
Site 12 (Dublin?) has:\nMarch 25-28\nMay 3rd\nJune 6 (and not May 30 - UK)\n(not Aug 1 - Ireland)\n(not Aug 29 - UK)\nOctober 31
\n\nBut also a bunch of other troughs!
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1963347%2F8631ec2571c08f07183e68414e9dd831%2FMeter%20Reading%20Time%20Series%20site%2012.png?generation=1572380239497256&alt=media\")
It's not quite perfect, but it looks like Site 12 has the Ireland-only holidays while sites 1 and 5 have the UK only holidays.
\n\nAn unrelated but important observation is that buildings in site 12 seem to turn off their systems on a monthly basis often around the 6th of the month. Could this be some sort of regular maintenance?
", "messages": [ { "id": 660970, "postDate": "2019-10-29T20:21:01.170Z", "content": "TLDR:\nI hypothesize that:\nSite 12 = Dublin\nSite 1 = London\nSite 5 = Loughborough
\n\nAlso, buildings in site 12 seem to turn things off 1 day a month (maintenance?)
\n\nIn a previous post I detailed my theory that siteIDs 1, 5, 12 were in Europe, mainly due to their hypothesized timezone, but also because the temperatures seemed to have been collected directly in Celsius (rather than converted from F).
\n\nAnother confirmation came from observing low usage on dates that are UK holidays that I reported in site 1. I will provide more info about holidays in this post.
\n\nIf we graph the daily average temperatures for sites1, 5, 12, they are a bit similar, but with:
\n\n** T(siteID 1)> T(siteID 5) > T(siteID 12)** (my general impression)\nNote that if I had plotted any other site on that same graph, the differences would have been much more pronounced.
\n\n
If you look at the ASHRAE sites in Europe, you have 3 in UK+Ireland: London, Loughborough, Dublin.
\n\nOther Europe sites (Madrid,Lisbon,Athens, Romania) sound like they would have a very different temperature profile.
\n\nLet's see if the pattern of holidays work with our theory.
\n\nSome holidays in Ireland (2016):
\n\nMar 25-28\nMay 2\nJune 6\n*Aug 1*\nOct 31
\n\nSome holidays in the UK (2016):
\n\nMar 25-28\nMay 2\nMay 30\n*Aug 29*
\n\nI bolded the ones that are different
\n\nI filtered meter reading time series data to have only week days and education buildings.
\n\nLet's look at the troughs on the following 3 plots.
\n\nSite 1 (London?) has:\nMay 2\nMay 30\nAug 29
\n\n
Site 5 (Loughborough?) \nMar 25-28\nMay 2\nMay 19 (???)\nMay 30\nJuly 7 (???)\nAug 29
\n\n
Site 12 (Dublin?) has:\nMarch 25-28\nMay 3rd\nJune 6 (and not May 30 - UK)\n(not Aug 1 - Ireland)\n(not Aug 29 - UK)\nOctober 31
\n\nBut also a bunch of other troughs!
\n\n
It's not quite perfect, but it looks like Site 12 has the Ireland-only holidays while sites 1 and 5 have the UK only holidays.
\n\nAn unrelated but important observation is that buildings in site 12 seem to turn off their systems on a monthly basis often around the 6th of the month. Could this be some sort of regular maintenance?
", "rawMarkdown": "TLDR:\nI hypothesize that:\nSite 12 = Dublin\nSite 1 = London\nSite 5 = Loughborough\n\n\nAlso, buildings in site 12 seem to turn things off 1 day a month (maintenance?)\n\n-------------------------------------------------------------------------------------------------------\n\n\nIn a[ previous post](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114483) I detailed my theory that siteIDs 1, 5, 12 were in Europe, mainly due to their hypothesized timezone, but also because the temperatures seemed to have been collected directly in Celsius (rather than converted from F).\n\nAnother confirmation came from observing low usage on dates that are UK holidays that I reported in site 1. I will provide more info about holidays in this post.\n\n------------------------------------------------------------------------------------------------------------\nIf we graph the daily average temperatures for sites1, 5, 12, they are a bit similar, but with:\n\n** T(siteID 1)> T(siteID 5) > T(siteID 12)** (my general impression)\nNote that if I had plotted any other site on that same graph, the differences would have been much more pronounced.\n\n\n\n\n-------------------------------------------------------------------------------------------------------------\n\nIf you look at the [ASHRAE sites in Europe,](https://www.ashrae.org/File%20Library/Communities/Regions/ASHRAE_08072019.pdf) you have 3 in UK+Ireland: London, Loughborough, Dublin.\n\nOther Europe sites (Madrid,Lisbon,Athens, Romania) sound like they would have a very different temperature profile.\n\n\nCould site 12 be Dublin because it is colder? And maybe site 5 Loughborough, site 1 = London (London is more south)?\n----------------------------------------------------------------------------------------------------------------\n\nLet's see if the pattern of holidays work with our theory.\n\nSome holidays in Ireland (2016):\n\nMar 25-28\nMay 2\n**June 6**\n**Aug 1**\n**Oct 31**\n\n\nSome holidays in the UK (2016):\n\nMar 25-28\nMay 2\n**May 30**\n**Aug 29**\n\nI bolded the ones that are different \n\n\nI filtered meter reading time series data to have only week days and education buildings. \n\nLet's look at the troughs on the following 3 plots.\n\nSite 1 (London?) has:\nMay 2\nMay 30\nAug 29\n\n\n\n\nSite 5 (Loughborough?) \nMar 25-28\nMay 2\nMay 19 (???)\nMay 30\nJuly 7 (???)\nAug 29\n\n\n\n\n\n\nSite 12 (Dublin?) has:\nMarch 25-28\nMay 3rd\nJune 6 (and not May 30 - UK)\n(not Aug 1 - Ireland)\n(not Aug 29 - UK)\nOctober 31\n\nBut also a bunch of other troughs!\n\n\n\n\n\n\n\n\n\nIt's not quite perfect, but it looks like Site 12 has the Ireland-only holidays while sites 1 and 5 have the UK only holidays.\n\n\nAn unrelated but important observation is that buildings in site 12 seem to turn off their systems on a monthly basis often around the 6th of the month. Could this be some sort of regular maintenance?\n\n\n", "votes": 65 }, { "id": 662042, "postDate": "2019-10-31T02:57:16.813Z", "content": "Knowing the right location would help to use the holiday feature effectively. I earlier saw some discussion about using US holiday as feature. But the problem is, as we don't know the location we can't say for sure about the holiday list. I guess that's why my holiday feature failed earlier. Also maintenance day, natural disaster. festival day so many features to think about if the location is known .
", "rawMarkdown": "Knowing the right location would help to use the holiday feature effectively. I earlier saw some discussion about using US holiday as feature. But the problem is, as we don't know the location we can't say for sure about the holiday list. I guess that's why my holiday feature failed earlier. Also maintenance day, natural disaster. festival day so many features to think about if the location is known .", "votes": 1 }, { "id": 660976, "postDate": "2019-10-29T20:28:21.060Z", "content": "Thanks for posting this, as someone who has yet to incorporate anything about holidays into my modeling, it should be helpful
", "rawMarkdown": "Thanks for posting this, as someone who has yet to incorporate anything about holidays into my modeling, it should be helpful", "votes": 1 }, { "id": 665810, "postDate": "2019-11-05T12:29:15.543Z", "content": "I got a better corr with Birmingham than loughborough. Check it out here: https://www.kaggle.com/datadugong/locate-better-cities-by-weather-temp-fill-nans
", "rawMarkdown": "I got a better corr with Birmingham than loughborough. Check it out here: https://www.kaggle.com/datadugong/locate-better-cities-by-weather-temp-fill-nans", "replies": [ { "id": 665982, "postDate": "2019-11-05T15:55:40.913Z", "content": "Could be. Loughborough is just based on the official Ashrae sites. \nBut I couldn't find the Loughborough library building in our dataset so it could be wrong.\nI see that on the Ashrae pdf you posted, that chapter is called Midlands UK and not Loughborough, so Birmingham could fit.
", "rawMarkdown": "Could be. Loughborough is just based on the official Ashrae sites. \nBut I couldn't find the Loughborough library building in our dataset so it could be wrong.\nI see that on the Ashrae pdf you posted, that chapter is called Midlands UK and not Loughborough, so Birmingham could fit." }, { "id": 666016, "postDate": "2019-11-05T16:35:17.197Z", "content": "Yer - it might well not be Birmingham as the corr isn't perfect. Birmingham Airport came up as a weather station in an ASHRAE report when referencing the \"Central England\" region in the PDFs, hence why I tried it.
", "rawMarkdown": "Yer - it might well not be Birmingham as the corr isn't perfect. Birmingham Airport came up as a weather station in an ASHRAE report when referencing the \"Central England\" region in the PDFs, hence why I tried it. " } ] }, { "id": 661736, "postDate": "2019-10-30T16:45:47.983Z", "content": "Damn this is so good. Thanks for sharing.
", "rawMarkdown": "Damn this is so good. Thanks for sharing." }, { "id": 661645, "postDate": "2019-10-30T14:46:55.363Z", "content": "striking way of finding hidden insights from data. Would be helpful to many for sure, who are yet to incorporate holidays in their features.
", "rawMarkdown": "striking way of finding hidden insights from data. Would be helpful to many for sure, who are yet to incorporate holidays in their features." }, { "id": 661021, "postDate": "2019-10-29T21:51:54.960Z", "content": "I don't know if if we'll help, but it's really impressive...
", "rawMarkdown": "I don't know if if we'll help, but it's really impressive...", "replies": [ { "id": 661039, "postDate": "2019-10-29T22:33:11.493Z", "content": "Identifying variation that happens systematically should definitely help assuming it continues into the future
", "rawMarkdown": "Identifying variation that happens systematically should definitely help assuming it continues into the future" } ] }, { "id": 662135, "postDate": "2019-10-31T06:00:29.053Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 662636, "postDate": "2019-10-31T19:35:59.683Z", "content": "You could use external data based on this knowledge as long as you post the source in the external data disclosure thread. As specified in the rules:
\n\n\n", "rawMarkdown": "You could use external data based on this knowledge as long as you post the source in the [external data disclosure thread](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/112841#latest-662218). As specified in the rules:\n\n> C. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will (i) ensure the External Data is available to use by all participants of the competition for purposes of the competition at no cost to the other participants and (ii) post such access to the External Data for the participants to the official competition forum prior to the Entry Deadline." } ] }, { "id": 662820, "postDate": "2019-11-01T03:19:09.377Z", "content": "C. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will (i) ensure the External Data is available to use by all participants of the competition for purposes of the competition at no cost to the other participants and (ii) post such access to the External Data for the participants to the official competition forum prior to the Entry Deadline.
\n
Thank you for sharing. Thank you
", "rawMarkdown": "Thank you for sharing. Thank you" }, { "id": 661722, "postDate": "2019-10-30T16:30:22.227Z", "content": "Terrific analysis! Thanks for sharing!
", "rawMarkdown": "Terrific analysis! Thanks for sharing!" } ], "comments": [ { "id": 662042, "author_name": "Md Khairul Islam", "author_url": "", "post_date": "2019-10-31T02:57:16.813000", "content": "Knowing the right location would help to use the holiday feature effectively. I earlier saw some discussion about using US holiday as feature. But the problem is, as we don't know the location we can't say for sure about the holiday list. I guess that's why my holiday feature failed earlier. Also maintenance day, natural disaster. festival day so many features to think about if the location is known .
", "votes": 1, "replies": [] }, { "id": 660976, "author_name": "SteveKane", "author_url": "", "post_date": "2019-10-29T20:28:21.060000", "content": "Thanks for posting this, as someone who has yet to incorporate anything about holidays into my modeling, it should be helpful
", "votes": 1, "replies": [] }, { "id": 665810, "author_name": "Silverback", "author_url": "", "post_date": "2019-11-05T12:29:15.543000", "content": "I got a better corr with Birmingham than loughborough. Check it out here: https://www.kaggle.com/datadugong/locate-better-cities-by-weather-temp-fill-nans
", "votes": 0, "replies": [ { "id": 665982, "author_name": "S D", "author_url": "", "post_date": "2019-11-05T15:55:40.913000", "content": "Could be. Loughborough is just based on the official Ashrae sites. \nBut I couldn't find the Loughborough library building in our dataset so it could be wrong.\nI see that on the Ashrae pdf you posted, that chapter is called Midlands UK and not Loughborough, so Birmingham could fit.
", "votes": 0, "replies": [] }, { "id": 666016, "author_name": "Silverback", "author_url": "", "post_date": "2019-11-05T16:35:17.197000", "content": "Yer - it might well not be Birmingham as the corr isn't perfect. Birmingham Airport came up as a weather station in an ASHRAE report when referencing the \"Central England\" region in the PDFs, hence why I tried it.
", "votes": 0, "replies": [] } ] }, { "id": 661736, "author_name": "Manraj Singh", "author_url": "", "post_date": "2019-10-30T16:45:47.983000", "content": "Damn this is so good. Thanks for sharing.
", "votes": 0, "replies": [] }, { "id": 661645, "author_name": "Atul Anand {Jha}", "author_url": "", "post_date": "2019-10-30T14:46:55.363000", "content": "striking way of finding hidden insights from data. Would be helpful to many for sure, who are yet to incorporate holidays in their features.
", "votes": 0, "replies": [] }, { "id": 661021, "author_name": "mezoganet", "author_url": "", "post_date": "2019-10-29T21:51:54.960000", "content": "I don't know if if we'll help, but it's really impressive...
", "votes": 0, "replies": [ { "id": 661039, "author_name": "SteveKane", "author_url": "", "post_date": "2019-10-29T22:33:11.493000", "content": "Identifying variation that happens systematically should definitely help assuming it continues into the future
", "votes": 0, "replies": [] } ] }, { "id": 662135, "author_name": "", "author_url": "", "post_date": "2019-10-31T06:00:29.053000", "content": "", "votes": 0, "replies": [ { "id": 662636, "author_name": "tahaum", "author_url": "", "post_date": "2019-10-31T19:35:59.683000", "content": "You could use external data based on this knowledge as long as you post the source in the external data disclosure thread. As specified in the rules:
\n\n\n", "votes": 0, "replies": [] } ] }, { "id": 662820, "author_name": "jiruideng6", "author_url": "", "post_date": "2019-11-01T03:19:09.377000", "content": "C. External Data. You may use data other than the Competition Data (“External Data”) to develop and test your models and Submissions. However, you will (i) ensure the External Data is available to use by all participants of the competition for purposes of the competition at no cost to the other participants and (ii) post such access to the External Data for the participants to the official competition forum prior to the Entry Deadline.
\n
Thank you for sharing. Thank you
", "votes": 0, "replies": [] }, { "id": 661722, "author_name": "bluetrain", "author_url": "", "post_date": "2019-10-30T16:30:22.227000", "content": "Terrific analysis! Thanks for sharing!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "660970": "TLDR:\nI hypothesize that:\nSite 12 = Dublin\nSite 1 = London\nSite 5 = Loughborough\n\n\nAlso, buildings in site 12 seem to turn things off 1 day a month (maintenance?)\n\n-------------------------------------------------------------------------------------------------------\n\n\nIn a[ previous post](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114483) I detailed my theory that siteIDs 1, 5, 12 were in Europe, mainly due to their hypothesized timezone, but also because the temperatures seemed to have been collected directly in Celsius (rather than converted from F).\n\nAnother confirmation came from observing low usage on dates that are UK holidays that I reported in site 1. I will provide more info about holidays in this post.\n\n------------------------------------------------------------------------------------------------------------\nIf we graph the daily average temperatures for sites1, 5, 12, they are a bit similar, but with:\n\n** T(siteID 1)> T(siteID 5) > T(siteID 12)** (my general impression)\nNote that if I had plotted any other site on that same graph, the differences would have been much more pronounced.\n\n\n\n\n-------------------------------------------------------------------------------------------------------------\n\nIf you look at the [ASHRAE sites in Europe,](https://www.ashrae.org/File%20Library/Communities/Regions/ASHRAE_08072019.pdf) you have 3 in UK+Ireland: London, Loughborough, Dublin.\n\nOther Europe sites (Madrid,Lisbon,Athens, Romania) sound like they would have a very different temperature profile.\n\n\nCould site 12 be Dublin because it is colder? And maybe site 5 Loughborough, site 1 = London (London is more south)?\n----------------------------------------------------------------------------------------------------------------\n\nLet's see if the pattern of holidays work with our theory.\n\nSome holidays in Ireland (2016):\n\nMar 25-28\nMay 2\n**June 6**\n**Aug 1**\n**Oct 31**\n\n\nSome holidays in the UK (2016):\n\nMar 25-28\nMay 2\n**May 30**\n**Aug 29**\n\nI bolded the ones that are different \n\n\nI filtered meter reading time series data to have only week days and education buildings. \n\nLet's look at the troughs on the following 3 plots.\n\nSite 1 (London?) has:\nMay 2\nMay 30\nAug 29\n\n\n\n\nSite 5 (Loughborough?) \nMar 25-28\nMay 2\nMay 19 (???)\nMay 30\nJuly 7 (???)\nAug 29\n\n\n\n\n\n\nSite 12 (Dublin?) has:\nMarch 25-28\nMay 3rd\nJune 6 (and not May 30 - UK)\n(not Aug 1 - Ireland)\n(not Aug 29 - UK)\nOctober 31\n\nBut also a bunch of other troughs!\n\n\n\n\n\n\n\n\n\nIt's not quite perfect, but it looks like Site 12 has the Ireland-only holidays while sites 1 and 5 have the UK only holidays.\n\n\nAn unrelated but important observation is that buildings in site 12 seem to turn off their systems on a monthly basis often around the 6th of the month. Could this be some sort of regular maintenance?\n\n\n", "662042": "Knowing the right location would help to use the holiday feature effectively. I earlier saw some discussion about using US holiday as feature. But the problem is, as we don't know the location we can't say for sure about the holiday list. I guess that's why my holiday feature failed earlier. Also maintenance day, natural disaster. festival day so many features to think about if the location is known .", "660976": "Thanks for posting this, as someone who has yet to incorporate anything about holidays into my modeling, it should be helpful", "665810": "I got a better corr with Birmingham than loughborough. Check it out here: https://www.kaggle.com/datadugong/locate-better-cities-by-weather-temp-fill-nans", "661736": "Damn this is so good. Thanks for sharing.", "661645": "striking way of finding hidden insights from data. Would be helpful to many for sure, who are yet to incorporate holidays in their features.", "661021": "I don't know if if we'll help, but it's really impressive...", "662135": "", "662820": "Thank you for sharing. Thank you", "661722": "Terrific analysis! Thanks for sharing!" } }