![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3021244%2Fef43ab9315d530dcdfdd505041883a32%2Fplot.png?generation=1571726661540409&alt=media\")
Hi,\nI created this kernel to visualize the yearly air temperate groupby month.
\n\nMy Kernel -> https://www.kaggle.com/ccjoshua/ashrae-airtemperature
\n\n
Most of the temperature peak around mid of the year. It has to be at northern hemisphere.
\n\nIf we compare with Wikipedia average monthly temperature across the cities around the world, can we roughly guess where are those stations located ?
\n\nhttps://en.wikipedia.org/wiki/List_of_cities_by_average_temperature
\n\nWhat do you think ?
", "messages": [ { "id": "654627", "postDate": "10/22/2019 05:35:47", "content": "Hi,\nI created this kernel to visualize the yearly air temperate groupby month.
\n\nMy Kernel -> https://www.kaggle.com/ccjoshua/ashrae-airtemperature
\n\n
Most of the temperature peak around mid of the year. It has to be at northern hemisphere.
\n\nIf we compare with Wikipedia average monthly temperature across the cities around the world, can we roughly guess where are those stations located ?
\n\nhttps://en.wikipedia.org/wiki/List_of_cities_by_average_temperature
\n\nWhat do you think ?
", "rawMarkdown": "Hi,\nI created this kernel to visualize the yearly air temperate groupby month. \n\n[My Kernel -> https://www.kaggle.com/ccjoshua/ashrae-airtemperature](https://www.kaggle.com/ccjoshua/ashrae-airtemperature)\n\n\n\nMost of the temperature peak around mid of the year. It has to be at **northern hemisphere**.\n\nIf we compare with Wikipedia average monthly temperature across the cities around the world, can we roughly guess where are those stations located ?\n\n\n[https://en.wikipedia.org/wiki/List_of_cities_by_average_temperature](https://en.wikipedia.org/wiki/List_of_cities_by_average_temperature)\n\n\nWhat do you think ?", "votes": null }, { "id": "654676", "postDate": "10/22/2019 07:03:47", "content": "So what are the cities? or are you trying to crowd source it? ;)
\nYou should scrape a site with the averages per year, though. Or at least, average all 3 years in this dataset, otherwise you'll be comparing apples to oranges...
Inside my kernel I have a 3 years plot, it repeated pattern is obvious, average 3 years out, I'm expecting similar pattern.
\n\nI will presume all station are in United State for now :D . \nSite_0 air temperature look similar to Tampa city
\n\nPerhaps can combine with other metrics to further narrow down.
", "rawMarkdown": "Inside my kernel I have a 3 years plot, it repeated pattern is obvious, average 3 years out, I'm expecting similar pattern.\n\nI will presume all station are in United State for now :D . \nSite_0 air temperature look similar to Tampa city\n \nPerhaps can combine with other metrics to further narrow down.", "votes": null }, { "id": "654770", "postDate": "10/22/2019 09:42:48", "content": "I think sites (0, 8) and (7, 11) are the same
", "rawMarkdown": "I think sites (0, 8) and (7, 11) are the same", "votes": null }, { "id": "654791", "postDate": "10/22/2019 10:20:13", "content": "Wow!! nice intuition. I'm not aware of that.
\n\nMay be they are very close to each other.
", "rawMarkdown": "Wow!! nice intuition. I'm not aware of that.\n\nMay be they are very close to each other.", "votes": null }, { "id": "655213", "postDate": "10/22/2019 20:11:29", "content": "Definitely all places in the northern hemisphere
", "rawMarkdown": "Definitely all places in the northern hemisphere", "votes": null }, { "id": "655224", "postDate": "10/22/2019 20:36:59", "content": "Cool\nYou can find the time zone by looking at what time the meter 0/electricity goes up (early morning)\nAs someone mentioned, it looks like the timestamp is in UTC...
\n\nEdit: Sorry, looking at the hours I can't see any difference on starting times, so everything has to be local time.
", "rawMarkdown": "Cool\nYou can find the time zone by looking at what time the meter 0/electricity goes up (early morning)\nAs someone mentioned, it looks like the timestamp is in UTC...\n\nEdit: Sorry, looking at the hours I can't see any difference on starting times, so everything has to be local time.", "votes": null }, { "id": "655286", "postDate": "10/22/2019 21:54:13", "content": "if you PCA the air temperatures grouped by site and timestamp you get sort of a map:\n![\"Weather](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F451025%2Fb8b8f436b3735aa372e0e82b9c54131f%2Fweather_pca.png?generation=1571780461852902&alt=media\")
\nIf you rotate that map, you should get somehow the map of USA? (after some non-linear distortion lol)\nLeft is cold and right is warm.\nUp and down maybe the east and west? not sure...
Sites (0, 8) and (7, 11) are indeed the same (in the plot they're on top of each other) All the values match to the hour, abs diff == 0.0
", "rawMarkdown": "if you PCA the air temperatures grouped by site and timestamp you get sort of a map:\n\nIf you rotate that map, you should get somehow the map of USA? (after some non-linear distortion lol)\nLeft is cold and right is warm.\nUp and down maybe the east and west? not sure...\n\nSites (0, 8) and (7, 11) are indeed the same (in the plot they're on top of each other) All the values match to the hour, abs diff == 0.0", "votes": null }, { "id": "655353", "postDate": "10/23/2019 00:58:13", "content": "Never though of to use PCA to 'visualize' the location ! 👍 👍 \nVery interesting approach !!!
", "rawMarkdown": "Never though of to use PCA to 'visualize' the location ! 👍 👍 \nVery interesting approach !!!", "votes": null }, { "id": "655373", "postDate": "10/23/2019 01:42:19", "content": "Definitely planet Earth. Only Earth people would use something other than Kelvin.
", "rawMarkdown": "Definitely planet Earth. Only Earth people would use something other than Kelvin.", "votes": null }, { "id": "655720", "postDate": "10/23/2019 12:38:01", "content": "Definitely those are Mars temperatures. 👀😄
", "rawMarkdown": "Definitely those are Mars temperatures. 👀😄", "votes": null }, { "id": "656909", "postDate": "10/24/2019 18:37:41", "content": "(UPD) Another confirmation, site's (0-8) and (7-11) weather chunks are indeed the same, they have the same missing values in full 2016-2018 weather dataset\n
Full info you can view in this kernel\n(however, PCA approach of @hmendonca is a much more elegant way to spot it 👍 )\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F724138%2F04d8cebedc851e6577af4eae89905d0c%2FScreenshot_weather_data.png?generation=1571943341532204&alt=media\")
I think 7 and 11 are in Canada, and 1,5,12 in Europe. Your PCA seems to capture those clusters.\nAlso I think 2,4,10 are more west than the rest, but your \"map\" doesn't capture that.\nsource
", "rawMarkdown": "I think 7 and 11 are in Canada, and 1,5,12 in Europe. Your PCA seems to capture those clusters.\nAlso I think 2,4,10 are more west than the rest, but your \"map\" doesn't capture that.\n[source](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/114483)", "votes": null }, { "id": "659048", "postDate": "10/27/2019 01:38:56", "content": "Cool idea
", "rawMarkdown": "Cool idea", "votes": null }, { "id": "659060", "postDate": "10/27/2019 02:22:56", "content": "how did you do the PCA? I kind of don't understand how PCA could be used to show location of sites, thanks^ ^ @hmendonca
", "rawMarkdown": "how did you do the PCA? I kind of don't understand how PCA could be used to show location of sites, thanks^ ^ @hmendonca", "votes": null }, { "id": "659112", "postDate": "10/27/2019 05:23:31", "content": "@hmendonca Are you a wizard?
", "rawMarkdown": "hmendonca Are you a wizard?", "votes": null }, { "id": "659930", "postDate": "10/28/2019 13:13:59", "content": "In nutshell, PCA will 'group' similar/relevant data closer to each other. \nIf sites are close to each other, the temperature reading has be relatively similar.
", "rawMarkdown": "In nutshell, PCA will 'group' similar/relevant data closer to each other. \nIf sites are close to each other, the temperature reading has be relatively similar.", "votes": null }, { "id": "659981", "postDate": "10/28/2019 14:18:56", "content": "@chenyamei you have to create a matrix of site_id by hourly temperatures, and then apply [sklearn] PCA on it to reduce to the 2 dimensions in the plot above. I'll try to post the code if I find some time (it's only a couple of lines though).
", "rawMarkdown": "chenyamei you have to create a matrix of site_id by hourly temperatures, and then apply [sklearn] PCA on it to reduce to the 2 dimensions in the plot above. I'll try to post the code if I find some time (it's only a couple of lines though).", "votes": null }, { "id": "661207", "postDate": "10/30/2019 02:42:00", "content": "I just located most of sites according to external dataset from kaggle.\n(Already post in \"External Data Disclosure Thread\")
\n\nHere's the notebook:\nhttps://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature
\n\nHope it helps!
", "rawMarkdown": "I just located most of sites according to external dataset from kaggle.\n(Already post in \"External Data Disclosure Thread\")\n\nHere's the notebook:\nhttps://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature\n\nHope it helps!", "votes": null }, { "id": "666941", "postDate": "11/06/2019 16:34:09", "content": "Sorry, with much delay, I finally posted the code for the PCA in this kernel: https://www.kaggle.com/hmendonca/clean-weather-data-eda\nIt's good that we now know the location of the buildings and can confirm our theories hehe\ncheers
", "rawMarkdown": "Sorry, with much delay, I finally posted the code for the PCA in this kernel: https://www.kaggle.com/hmendonca/clean-weather-data-eda\nIt's good that we now know the location of the buildings and can confirm our theories hehe\ncheers", "votes": null } ], "comments": [ { "id": 654676, "author_name": "hmendonca", "author_url": "", "post_date": "10/22/2019 07:03:47", "content": "So what are the cities? or are you trying to crowd source it? ;)
\nYou should scrape a site with the averages per year, though. Or at least, average all 3 years in this dataset, otherwise you'll be comparing apples to oranges...
Inside my kernel I have a 3 years plot, it repeated pattern is obvious, average 3 years out, I'm expecting similar pattern.
\n\nI will presume all station are in United State for now :D . \nSite_0 air temperature look similar to Tampa city
\n\nPerhaps can combine with other metrics to further narrow down.
", "votes": null, "replies": [] }, { "id": 655224, "author_name": "hmendonca", "author_url": "", "post_date": "10/22/2019 20:36:59", "content": "Cool\nYou can find the time zone by looking at what time the meter 0/electricity goes up (early morning)\nAs someone mentioned, it looks like the timestamp is in UTC...
\n\nEdit: Sorry, looking at the hours I can't see any difference on starting times, so everything has to be local time.
", "votes": null, "replies": [] }, { "id": 659060, "author_name": "chenyamei", "author_url": "", "post_date": "10/27/2019 02:22:56", "content": "how did you do the PCA? I kind of don't understand how PCA could be used to show location of sites, thanks^ ^ @hmendonca
", "votes": null, "replies": [] }, { "id": 659930, "author_name": "ccjoshua", "author_url": "", "post_date": "10/28/2019 13:13:59", "content": "In nutshell, PCA will 'group' similar/relevant data closer to each other. \nIf sites are close to each other, the temperature reading has be relatively similar.
", "votes": null, "replies": [] }, { "id": 659981, "author_name": "hmendonca", "author_url": "", "post_date": "10/28/2019 14:18:56", "content": "@chenyamei you have to create a matrix of site_id by hourly temperatures, and then apply [sklearn] PCA on it to reduce to the 2 dimensions in the plot above. I'll try to post the code if I find some time (it's only a couple of lines though).
", "votes": null, "replies": [] } ] }, { "id": 654770, "author_name": "sergeylebedev", "author_url": "", "post_date": "10/22/2019 09:42:48", "content": "I think sites (0, 8) and (7, 11) are the same
", "votes": null, "replies": [ { "id": 654791, "author_name": "ccjoshua", "author_url": "", "post_date": "10/22/2019 10:20:13", "content": "Wow!! nice intuition. I'm not aware of that.
\n\nMay be they are very close to each other.
", "votes": null, "replies": [] }, { "id": 655286, "author_name": "hmendonca", "author_url": "", "post_date": "10/22/2019 21:54:13", "content": "if you PCA the air temperatures grouped by site and timestamp you get sort of a map:\n\nIf you rotate that map, you should get somehow the map of USA? (after some non-linear distortion lol)\nLeft is cold and right is warm.\nUp and down maybe the east and west? not sure...
Sites (0, 8) and (7, 11) are indeed the same (in the plot they're on top of each other) All the values match to the hour, abs diff == 0.0
", "votes": null, "replies": [] }, { "id": 655353, "author_name": "ccjoshua", "author_url": "", "post_date": "10/23/2019 00:58:13", "content": "Never though of to use PCA to 'visualize' the location ! 👍 👍 \nVery interesting approach !!!
", "votes": null, "replies": [] }, { "id": 658881, "author_name": "sdoria", "author_url": "", "post_date": "10/26/2019 17:16:05", "content": "I think 7 and 11 are in Canada, and 1,5,12 in Europe. Your PCA seems to capture those clusters.\nAlso I think 2,4,10 are more west than the rest, but your \"map\" doesn't capture that.\nsource
", "votes": null, "replies": [] }, { "id": 659112, "author_name": "gunesevitan", "author_url": "", "post_date": "10/27/2019 05:23:31", "content": "@hmendonca Are you a wizard?
", "votes": null, "replies": [] }, { "id": 666941, "author_name": "hmendonca", "author_url": "", "post_date": "11/06/2019 16:34:09", "content": "Sorry, with much delay, I finally posted the code for the PCA in this kernel: https://www.kaggle.com/hmendonca/clean-weather-data-eda\nIt's good that we now know the location of the buildings and can confirm our theories hehe\ncheers
", "votes": null, "replies": [] } ] }, { "id": 655213, "author_name": "sergeylebedev", "author_url": "", "post_date": "10/22/2019 20:11:29", "content": "Definitely all places in the northern hemisphere
", "votes": null, "replies": [ { "id": 655373, "author_name": "abednadir", "author_url": "", "post_date": "10/23/2019 01:42:19", "content": "Definitely planet Earth. Only Earth people would use something other than Kelvin.
", "votes": null, "replies": [] }, { "id": 655720, "author_name": "drexpz", "author_url": "", "post_date": "10/23/2019 12:38:01", "content": "Definitely those are Mars temperatures. 👀😄
", "votes": null, "replies": [] } ] }, { "id": 656909, "author_name": "frednavruzov", "author_url": "", "post_date": "10/24/2019 18:37:41", "content": "(UPD) Another confirmation, site's (0-8) and (7-11) weather chunks are indeed the same, they have the same missing values in full 2016-2018 weather dataset\n
Full info you can view in this kernel\n(however, PCA approach of @hmendonca is a much more elegant way to spot it 👍 )\n
Cool idea
", "votes": null, "replies": [] }, { "id": 661207, "author_name": "patrick0302", "author_url": "", "post_date": "10/30/2019 02:42:00", "content": "I just located most of sites according to external dataset from kaggle.\n(Already post in \"External Data Disclosure Thread\")
\n\nHere's the notebook:\nhttps://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature
\n\nHope it helps!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "654627": "Hi,\nI created this kernel to visualize the yearly air temperate groupby month. \n\n[My Kernel -> https://www.kaggle.com/ccjoshua/ashrae-airtemperature](https://www.kaggle.com/ccjoshua/ashrae-airtemperature)\n\n\n\nMost of the temperature peak around mid of the year. It has to be at **northern hemisphere**.\n\nIf we compare with Wikipedia average monthly temperature across the cities around the world, can we roughly guess where are those stations located ?\n\n\n[https://en.wikipedia.org/wiki/List_of_cities_by_average_temperature](https://en.wikipedia.org/wiki/List_of_cities_by_average_temperature)\n\n\nWhat do you think ?", "654676": "So what are the cities? or are you trying to crowd source it? ;) \nYou should scrape a site with the averages per year, though. Or at least, average all 3 years in this dataset, otherwise you'll be comparing apples to oranges...", "654721": "Inside my kernel I have a 3 years plot, it repeated pattern is obvious, average 3 years out, I'm expecting similar pattern.\n\nI will presume all station are in United State for now :D . \nSite_0 air temperature look similar to Tampa city\n \nPerhaps can combine with other metrics to further narrow down.", "654770": "I think sites (0, 8) and (7, 11) are the same", "654791": "Wow!! nice intuition. I'm not aware of that.\n\nMay be they are very close to each other.", "655213": "Definitely all places in the northern hemisphere", "655224": "Cool\nYou can find the time zone by looking at what time the meter 0/electricity goes up (early morning)\nAs someone mentioned, it looks like the timestamp is in UTC...\n\nEdit: Sorry, looking at the hours I can't see any difference on starting times, so everything has to be local time.", "655286": "if you PCA the air temperatures grouped by site and timestamp you get sort of a map:\n\nIf you rotate that map, you should get somehow the map of USA? (after some non-linear distortion lol)\nLeft is cold and right is warm.\nUp and down maybe the east and west? not sure...\n\nSites (0, 8) and (7, 11) are indeed the same (in the plot they're on top of each other) All the values match to the hour, abs diff == 0.0", "655353": "Never though of to use PCA to 'visualize' the location ! 👍 👍 \nVery interesting approach !!!", "655373": "Definitely planet Earth. Only Earth people would use something other than Kelvin.", "655720": "Definitely those are Mars temperatures. 👀😄", "656909": "**(UPD)** Another confirmation, site's (0-8) and (7-11) weather chunks are indeed the same, they have the same missing values in full 2016-2018 weather dataset\n