{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\n#import os\n#for dirname, _, filenames in os.walk('/kaggle/input'):\n#    for filename in filenames:\n#        print(os.path.join(dirname, filename))\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**HYPOTHESIS**\n* **The weather data is not in local time**\n\n**POTENTIAL CONSEQUENCES**\n* **We will count with very uninformative weather features for our models**\n* **Straightforward features such as DAY, HOUROFDAY, etc. will have little meaning if they do not match the weather conditions and the measured data localtime timestamp**"},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_data = pd.read_csv(\"/kaggle/input/ashrae-energy-prediction/weather_train.csv\")\nweather_data.set_index('timestamp', inplace=True)\nweather_data.index = pd.to_datetime(weather_data.index)\nweather_data['hourofday'] = weather_data.index.hour\nweather_data['dayofyear'] = weather_data.index.dayofyear\nweather_data.tail()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Lets plot some days in site 0 during winter**"},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_data[weather_data['site_id']== 0]['air_temperature'][:24].plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Lets plot the some days in site 0 during Summer**"},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_data[weather_data['site_id']== 0]['air_temperature'][5496:5520].plot()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_data[weather_data['site_id']== 0]['air_temperature'][24:48].plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Now Let's look at a random site, let's say site 10 for winter**"},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_data[weather_data['site_id']== 10]['air_temperature'][:24].plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Now for Summer**"},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_data[weather_data['site_id']== 10]['air_temperature'][5496:5520].plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**PRELIMINAR CONCLUSIONS**\n\n* **There seems to be something off. High temperatures during the day at 8:00pm or even at mid-night?**\n* **We need to do some extra checks: E.g., Estimate the the frequency of peak temperatures per hour of the day for each site**"},{"metadata":{"trusted":true},"cell_type":"code","source":"site_ids = list(set(weather_data['site_id']))\nresult = pd.DataFrame()\nfor site in site_ids:\n    weather_data_site = weather_data[weather_data['site_id']== site]\n    t_air_max_every_day = weather_data_site.groupby('dayofyear')['air_temperature'].max()\n    weather_data_site['hit_t_air_max'] = weather_data_site.apply(lambda x: 1 if x['air_temperature'] == t_air_max_every_day[x['dayofyear']] else 0,axis=1)\n    result['site'+str(site)] = weather_data_site.groupby('hourofday')['hit_t_air_max'].sum().values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(20, 10))\nsns.heatmap(result, annot=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"** FINAL CONCLUSIONS**\n* **The peak temperature of outdoor air takes place most of the time between 7:00pm and 9:00pm (sites 0, 3,6,7,8,9, 11, 14, 15)**\n* **Some locations even register the peak daily temperature close to 1:00am (sites 2, 4, 10, and 13) **\n* **This does not seem to be a realistic behavior**\n* **I believe the data is not in a local time zone which makes hard to extract correct representations of features such as DAY, HOUROFDAY etc.**\n* **Should this behavior be different than that of the measurements, we might be having a mismatch of energy consumption and weather conditions, that would lead us to very uninformative weather features**\n* **Some locations could be ok, from what we could expect a typical peak of temperature to occur (between 12:00pm and 5:00pm).\n\n** NEXT STEPS**\n\n* **Perhaps the organizers have some ideas if this is the case?**\n* **Perhaps the 'error' is systematic? i.e., all data's timestamp is shifted equally?**"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}