{"cells":[{"metadata":{"toc":true},"cell_type":"markdown","source":"<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#0.-Dataset\" data-toc-modified-id=\"0.-Dataset-1\">0. Dataset</a></span></li><li><span><a href=\"#1.-Preprocessing\" data-toc-modified-id=\"1.-Preprocessing-2\">1. Preprocessing</a></span><ul class=\"toc-item\"><li><span><a href=\"#1.1-Data-Import\" data-toc-modified-id=\"1.1-Data-Import-2.1\">1.1 Data Import</a></span></li><li><span><a href=\"#1.2-Data-Preprocessing\" data-toc-modified-id=\"1.2-Data-Preprocessing-2.2\">1.2 Data Preprocessing</a></span><ul class=\"toc-item\"><li><span><a href=\"#1.2.1-Building-Metadata\" data-toc-modified-id=\"1.2.1-Building-Metadata-2.2.1\">1.2.1 Building Metadata</a></span></li><li><span><a href=\"#1.2.2-weather-data\" data-toc-modified-id=\"1.2.2-weather-data-2.2.2\">1.2.2 weather data</a></span></li><li><span><a href=\"#1.2.3-Train/Test-Dataset\" data-toc-modified-id=\"1.2.3-Train/Test-Dataset-2.2.3\">1.2.3 Train/Test Dataset</a></span></li></ul></li></ul></li><li><span><a href=\"#2.-Baseline-Model\" data-toc-modified-id=\"2.-Baseline-Model-3\">2. Baseline Model</a></span><ul class=\"toc-item\"><li><span><a href=\"#2.-Feature-Engineering\" data-toc-modified-id=\"2.-Feature-Engineering-3.1\">2. Feature Engineering</a></span><ul class=\"toc-item\"><li><span><a href=\"#2.1-ID-features\" data-toc-modified-id=\"2.1-ID-features-3.1.1\">2.1 ID features</a></span></li></ul></li></ul></li></ul></div>"},{"metadata":{},"cell_type":"markdown","source":"https://www.kaggle.com/c/ashrae-energy-prediction/overview\n\nQ: How much does it cost to cool a skyscraper in the summer?\nA: A lot! And not just in dollars, but in environmental impact.\n\nThankfully, significant investments are being made to improve building efficiencies to reduce costs and emissions. The question is, are the improvements working? That’s where you come in. Under pay-for-performance financing, the building owner makes payments based on the difference between their real energy consumption and what they would have used without any retrofits. The latter values have to come from a model. Current methods of estimation are fragmented and do not scale well. Some assume a specific meter type or don’t work with different building types.\n\nIn this competition, you’ll develop accurate models of metered building energy usage in the following areas: chilled water, electric, hot water, and steam meters. The data comes from over 1,000 buildings over a three-year timeframe. With better estimates of these energy-saving investments, large scale investors and financial institutions will be more inclined to invest in this area to enable progress in building efficiencies.\n"},{"metadata":{},"cell_type":"markdown","source":"## 0. Dataset"},{"metadata":{},"cell_type":"markdown","source":"**1.train.csv** <br>\n>**building_id** - Foreign key for the building metadata.<br>\n>**meter** - The meter id code. Read as {0: electricity, 1: chilledwater, 2: steam, 3: hotwater}. Not every building has all meter types. <br>\n>**timestamp** - When the measurement was taken <br>\n>**meter_reading** - The target variable. Energy consumption in kWh (or equivalent). Note that this is real data with measurement error, which we expect will impose a baseline level of modeling error. **UPDATE: as discussed here, the site 0 electric meter readings are in kBTU.**\n\n**2.building_meta.csv** <br>\n>**site_id** - Foreign key for the weather files. <br>\n>**building_id** - Foreign key for training.csv <br>\n>**primary_use** - Indicator of the primary category of activities for the building based on EnergyStar property type definitions <br>\n>**square_feet** - Gross floor area of the building <br>\n>**year_built** - Year building was opened <br>\n>**loor_count** - Number of floors of the building\n\n**3.weather_(train/test).csv** <br>\nWeather data from a meteorological station as close as possible to the site. <br>\n>**site_id**<br>\n>**air_temperature** - Degrees Celsius <br>\n>**cloud_coverage** - Portion of the sky covered in clouds, in oktas <br>\n>**dew_temperature** - Degrees Celsius <br>\n>**precip_depth_1_hr** - Millimeters <br>\n>**sea_level_pressure** - Millibar/hectopascals <br>\n>**wind_direction** - Compass direction (0-360) <br>\n>**wind_speed** - Meters per second\n\n**4.test.csv**\nThe submission files use row numbers for ID codes in order to save space on the file uploads. test.csv has no feature data; it exists so you can get your predictions into the correct order.\n\n>**row_id** - Row id for your submission file <br>\n>**building_id** - Building id code <br>\n>**meter** - The meter id code <br>\n>**timestamp** - Timestamps for the test data period <br>\n\n**5.sample_submission.csv**\nA valid sample submission.\n\nAll floats in the solution file were truncated to **four decimal places**; we recommend you do the same to save space on your file upload. There are gaps in some of the meter readings for both the train and test sets. Gaps in the test set are not revealed or scored."},{"metadata":{"trusted":true},"cell_type":"code","source":"import os, sys, datetime, gc\n\nimport numpy as np, pandas as pd\nfrom pandas import Series, DataFrame\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n%matplotlib inline\nsns.set()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 1. Preprocessing"},{"metadata":{},"cell_type":"markdown","source":"### 1.1 Data Import"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"../input/ashrae-energy-prediction/train.csv\")\ntrain.head(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test = pd.read_csv(\"../input/ashrae-energy-prediction/test.csv\")\ntest.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"building_meta = pd.read_csv(\"../input/ashrae-energy-prediction/building_metadata.csv\")\nbuilding_meta.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"building_meta.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_train = pd.read_csv(\"../input/ashrae-energy-prediction/weather_train.csv\")\nweather_train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_test = pd.read_csv(\"../input/ashrae-energy-prediction/weather_test.csv\")\nweather_test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_test.info()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 1.2 Data Preprocessing"},{"metadata":{},"cell_type":"markdown","source":"#### 1.2.1 Building Metadata"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Building meta data contains null values.\n\nprint('Missing Value Percentage:\\n',\n     building_meta.isnull().sum() / building_meta.shape[0]) \nsns.heatmap(building_meta.isnull(), cbar = False, cmap = 'bwr')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# shrink the size of DataFrame\n\ndef df_col_size(df):\n    print('Total Size (Mb): {:.3f}'.format(df.memory_usage().sum() * 1e-6 ))\n    print(df.memory_usage() * 1e-6)\n    for col in df.columns:\n        print(f\"{col}: Type: {df[col].dtypes},\\\n        Min: {df[col].min()},\\\n        Max: {df[col].max()}\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_col_size(building_meta)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\n\nuse_le = LabelEncoder()\n\nuse_le.fit(building_meta.primary_use)\nbuilding_meta['primary_use'] = use_le.transform(building_meta.primary_use)\n\nuse_le.classes_","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"use_le.classes_","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"building_meta['site_id'] = building_meta['site_id'].astype(np.int8)\nbuilding_meta['building_id'] = building_meta['building_id'].astype(np.int16)\nbuilding_meta['square_feet'] = building_meta['square_feet'].astype(np.int32)\nbuilding_meta['primary_use'] = building_meta['primary_use'].astype(np.int8)\nbuilding_meta['year_built'] = building_meta['year_built'].astype(np.float16)\nbuilding_meta['floor_count'] = building_meta['floor_count'].astype(np.float16)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_col_size(building_meta)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# How many unique site ID?\n\nbuilding_meta['site_id'].value_counts().sort_index().plot.bar(figsize = (8,6))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# What's the missing rate per each site ID\nbuilding_meta.groupby('site_id').apply(lambda x: x.isnull().sum() / (x.count() + x.isnull().sum()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Cannot impute missing values only based on site_id. Join the train data and impute all together.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### 1.2.2 weather data"},{"metadata":{"trusted":true},"cell_type":"code","source":"# weather is split into train and test data. We combine them together and then analyze.\n\nprint('size of train:', weather_train.shape,'\\n',\n     'size of test', weather_test.shape)\n\nweather_all = pd.concat([weather_train, weather_test], axis = 0, keys = ('train', 'test'))\n\nprint('size of all', weather_all.shape)\n\ndel weather_train\ndel weather_test","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#weather data contains null values.\n\nprint('Missing Value Percentage:\\n',\n     weather_all.isnull().sum() / weather_all.shape[0])\nplt.figure(figsize = (10,10))\nsns.heatmap(weather_all.isnull(), cbar = False, cmap = 'bwr')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# input data before shrinking the size\nweather_all['timestamp'] = pd.to_datetime(weather_all.timestamp)\n\nweather_all['year'] = weather_all['timestamp'].dt.year\nweather_all['month'] = weather_all['timestamp'].dt.month\nweather_all['day'] = weather_all['timestamp'].dt.day\nweather_all['week'] = weather_all['timestamp'].dt.week\nweather_all['weekday'] = weather_all['timestamp'].dt.weekday\nweather_all['hour'] = weather_all['timestamp'].dt.hour\nweather_all['season'] = np.where(weather_all['timestamp'].dt.month <=2, 4, \n         np.where(weather_all['timestamp'].dt.month <= 5, 1,\n                 np.where(weather_all['timestamp'].dt.month <= 8, 2,\n                         np.where(weather_all['timestamp'].dt.month <= 11, 3, 4))))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# air_temp\n\nprint('NaN before impute: {}'.format(weather_all.air_temperature.isnull().sum()))\n\nair_temp_filler = weather_all.groupby(['site_id', 'year','month','hour'])['air_temperature'].median()\n\nfor site_id, year, month, hour in air_temp_filler.index:\n    weather_all.loc[(weather_all['site_id'] == site_id) & \n                    (weather_all['year'] == year) & \n                    (weather_all['month'] == month) & \n                    (weather_all['hour'] == hour) &\n                    (weather_all.air_temperature.isnull()), 'air_temperature']\\\n    = air_temp_filler[(site_id, year, month, hour)]\n\nprint('NaN after impute: {}'.format(weather_all.air_temperature.isnull().sum()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_all.groupby(['site_id', 'year','month'])['air_temperature'].describe().head(12)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# cloud_average\n\nprint(weather_all.cloud_coverage.isnull().sum(), \n      'Pecentage: {:.2f}'.format(weather_all.cloud_coverage.isnull().sum() / weather_all.shape[0]))","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"# study the cloud coverage (take a small portion of data and study its pattern)\n\nfig, ax = plt.subplots(len(set(weather_all['site_id'].values)),1, figsize = (20, 60))\nidx = 0\nfor site_id in set(weather_all['site_id'].values):\n    ax[idx].plot(weather_all[(weather_all['site_id'] == site_id) &\n                             (weather_all['year'] == 2016)][['timestamp', 'cloud_coverage']]\\\n    .set_index('timestamp'))\n    ax[idx].set_title('Site ID: {}'.format(site_id), fontsize = 10)\n    idx += 1\n    \n    ","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"# site 7 and 11 did not have cloud_coverage data at all. Hence inpute the rest NaN's with -1.\nweather_all.groupby(['site_id','year','day'])['cloud_coverage'].mean()\\\n[weather_all.groupby(['site_id','year','day'])['cloud_coverage'].mean().isnull()].index.tolist()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# from the analysis, the cloud coverage should be similar for the same day of each month.\n\ncloud_coverage_fillter = weather_all.groupby(['site_id','year','day'])['cloud_coverage'].median()\n\nprint(\"NaN before impute: {}\".format(weather_all.cloud_coverage.isnull().sum()))\n\nfor site_id, year, day in cloud_coverage_fillter.index:\n    weather_all.loc[(weather_all['site_id'] == site_id) & \n               (weather_all['year'] == year) & \n               (weather_all['day'] == day) & \n                    (weather_all.cloud_coverage.isnull()), 'cloud_coverage'] = cloud_coverage_fillter[(site_id, year, day)]\n\nweather_all['cloud_coverage'].fillna(-999, inplace = True)\nprint(\"NaN after impute: {}\".format(weather_all.cloud_coverage.isnull().sum()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_all.groupby(['site_id', 'year','day'])['cloud_coverage'].describe().head(12)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Temperaure should be the same for the same hour of each day within a specific month. \n# Therefore we impute the missing values this way.\n    \ndew_temperature_filler = weather_all.groupby(['site_id', 'year', 'month', 'hour'])['dew_temperature'].median()\n\nprint('NaN before impute: {}'.format(weather_all.dew_temperature.isnull().sum()))\n\nfor site_id, year, month, hour in dew_temperature_filler.index:\n    weather_all.loc[(weather_all['site_id'] == site_id) & \n                    (weather_all['year'] == year) & \n                    (weather_all['month'] == month) & \n                    (weather_all['hour'] == hour) &\n                    (weather_all.dew_temperature.isnull()), 'dew_temperature']\\\n    = dew_temperature_filler[(site_id, year, month, hour)]\n\nprint('NaN after impute: {}'.format(weather_all.dew_temperature.isnull().sum()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_all.groupby(['site_id', 'year','month'])['dew_temperature'].describe().head(12)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# precip_depth_1_hr\n\nprint('Nan count:', weather_all.precip_depth_1_hr.isnull().sum(), 'Percentage: {:.2f}'.format(weather_all.precip_depth_1_hr.isnull().sum() / weather_all.shape[0]))\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# site_id 5, 12, 1 did not have any percipitation records. Therefore impute -999.\nweather_all.groupby(['site_id','year','month']).precip_depth_1_hr\\\n.apply(lambda x: x.isnull().sum() / (x.count() + x.isnull().sum())).sort_values(ascending = False)[:108].index.tolist()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('NaN before impute: {}'.format(weather_all.precip_depth_1_hr.isnull().sum()))\n\nprecipt_depth_filler = weather_all.groupby(['site_id','year','month']).precip_depth_1_hr.median()\n\nfor site_id, year, month in precipt_depth_filler.index:\n    weather_all.loc[(weather_all['site_id'] == site_id) & \n                    (weather_all['year'] == year) & \n                    (weather_all['month'] == month) & \n                    (weather_all.precip_depth_1_hr.isnull()), 'precip_depth_1_hr']\\\n    = precipt_depth_filler[(site_id, year, month)]\n\nweather_all.precip_depth_1_hr.fillna(-999, inplace = True)\n\nprint('NaN after impute: {}'.format(weather_all.precip_depth_1_hr.isnull().sum()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# sea_level_pressure\n\nprint('Nan count:', weather_all.sea_level_pressure.isnull().sum(), \n      'Percentage: {:.2f}'.format(weather_all.sea_level_pressure.isnull().sum() / weather_all.shape[0]))\n","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"# site 5 did not have sea level pressure data. therefore impute with -1\nweather_all.groupby(['site_id', 'year','month']).sea_level_pressure.median()\\\n[weather_all.groupby(['site_id', 'year','month']).sea_level_pressure.median().isnull()]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('NaN before impute: {}'.format(weather_all.sea_level_pressure.isnull().sum()))\n\nsea_level_filler = weather_all.groupby(['site_id','year', 'month']).sea_level_pressure.median()\n\nfor site_id, year, month in sea_level_filler.index:\n    weather_all.loc[(weather_all['site_id'] == site_id) & \n                    (weather_all['year'] == year) & \n                    (weather_all['month'] == month) & \n                    (weather_all.sea_level_pressure.isnull()), 'sea_level_pressure']\\\n    = sea_level_filler[(site_id, year, month)]\n\nweather_all.sea_level_pressure.fillna(-999, inplace = True)\n\nprint('NaN after impute: {}'.format(weather_all.sea_level_pressure.isnull().sum()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# wind_direction\n\nprint('Nan count:', weather_all.wind_direction.isnull().sum(), \n      'Percentage: {:.2f}'.format(weather_all.wind_direction.isnull().sum() / weather_all.shape[0]))\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('NaN before impute: {}'.format(weather_all.wind_direction.isnull().sum()))\n\nwind_direction_filler = weather_all.groupby(['site_id','year','month','hour']).wind_direction.median()\n\nfor site_id, year, month, hour in wind_direction_filler.index:\n    weather_all.loc[(weather_all['site_id'] == site_id) & \n                    (weather_all['year'] == year) & \n                    (weather_all['month'] == month) &\n                    (weather_all['hour'] == hour) &\n                    (weather_all.wind_direction.isnull()), 'wind_direction']\\\n    = wind_direction_filler[(site_id, year, month, hour)]\n\nprint('NaN after impute: {}'.format(weather_all.wind_direction.isnull().sum()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# wind_speed\n\nprint('NaN before impute: {}'.format(weather_all.wind_speed.isnull().sum()))\n\nwind_speed_filler = weather_all.groupby(['site_id','year','month','hour']).wind_speed.median()\n\nfor site_id, year, month, hour in wind_speed_filler.index:\n    weather_all.loc[(weather_all['site_id'] == site_id) & \n                    (weather_all['year'] == year) & \n                    (weather_all['month'] == month) &\n                    (weather_all['hour'] == hour) &\n                    (weather_all.wind_speed.isnull()), 'wind_speed']\\\n    = wind_speed_filler[(site_id, year, month, hour)]\n\nprint('NaN after impute: {}'.format(weather_all.wind_speed.isnull().sum()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_all.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_col_size(weather_all)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_all['site_id'] = weather_all['site_id'].astype(np.int8)\nweather_all['air_temperature'] = weather_all['air_temperature'].astype(np.float16)\nweather_all['cloud_coverage'] = weather_all['cloud_coverage'].astype(np.float16)\nweather_all['dew_temperature'] = weather_all['dew_temperature'].astype(np.float16)\nweather_all['precip_depth_1_hr'] = weather_all['precip_depth_1_hr'].astype(np.float16)\nweather_all['sea_level_pressure'] = weather_all['sea_level_pressure'].astype(np.float16)\nweather_all['wind_direction'] = weather_all['wind_direction'].astype(np.float16)\nweather_all['wind_speed'] = weather_all['wind_speed'].astype(np.float16)\nweather_all['year'] = weather_all['year'].astype(np.float16)\nweather_all['month'] = weather_all['month'].astype(np.float16)\nweather_all['day'] = weather_all['day'].astype(np.float16)\nweather_all['week'] = weather_all['week'].astype(np.float16)\nweather_all['weekday'] = weather_all['weekday'].astype(np.float16)\nweather_all['hour'] = weather_all['hour'].astype(np.float16)\nweather_all['season'] = weather_all['season'].astype(np.float16)\n\ndf_col_size(weather_all)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### 1.2.3 Train/Test Dataset\n\nconvert data type to prepare data merge."},{"metadata":{"trusted":true},"cell_type":"code","source":"df_col_size(train)\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train['building_id'] = train['building_id'].astype(np.int16)\ntrain['meter'] = train['meter'].astype(np.int8)\ntrain['timestamp'] = pd.to_datetime(train['timestamp'])\n\ndf_col_size(train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_col_size(test)\ntest.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test['building_id'] = test['building_id'].astype(np.int16)\ntest['meter'] = test['meter'].astype(np.int8)\ntest['timestamp'] = pd.to_datetime(test['timestamp'])\n\ndf_col_size(test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Merge dataframes to prepare training dataset\n\nprint(train.building_id.nunique(), building_meta.building_id.nunique())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = train.merge(building_meta, how = 'inner', on = 'building_id' )\n\ndf_col_size(train_df)\ntrain_df.head()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = train_df.merge(weather_all, how = 'left', on = ['site_id', 'timestamp'])\n\ndf_col_size(train_df)\ntrain_df.head()\n\ndel train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.to_csv('../working/train_df.csv', index = False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"train_df.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"id_col = ['site_id','building_id','meter']\ntrain_col = ['primary_use','square_feet','year_built','floor_count',\n            'air_temperature','cloud_coverage','dew_temperature',\n            'precip_depth_1_hr','sea_level_pressure','wind_direction',\n            'wind_speed','month','day','week','weekday','hour','season']\ntarget_col = ['meter_reading']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"train_df = pd.read_csv('', compression = 'zip')"},{"metadata":{},"cell_type":"markdown","source":"## 2. Baseline Model"},{"metadata":{},"cell_type":"markdown","source":"#### 2.1 GradiantBoostingRegressor"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.ensemble import GradiantBoostingRegressor\n\ntrain_set = train_df[train_df.timestamp]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"from catboost import Pool, CatBoostRegressor\nimport math\n\nclass RMSLE():\n    \n    def get_final_error(self, error, weight):\n        return error / (weight + 1e-38)\n    \n    def is_max_optimal(self):\n        return False\n    \n    def evaluate(self, approxes, target, weight):\n        '''\n            approxes is list of indexed containers\n            (containers with only __len__ and __getitem__ defined), one container\n            per approx dimension. Each container contains floats.\n            weight is one dimensional indexed container.\n            target is float.\n            weight parameter can be None.\n            Returns pair (error, weights sum)\n            \n        '''\n        \n        assert len(approxes) == 1\n        assert len(target) == len(approxes[0])\n        \n        approx = approxes[0]\n        \n        error_sum = 0.0\n        weight_sum = 0.0\n        \n        for i in range(len(approx)):\n            w = 1.0 if weight is None else weight[i]\n            weight_sum += w\n            error_sum += ((math.log(approx[i] + 1) - math.log(target[i] + 1))**2) * w\n\n        return math.sqrt(error_sum / weight_sum), weight_sum        ","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"demo = train_df[(train_df.site_id == 1) &\n        (train_df.building_id == 119) &\n        (train_df.meter == 0)]\n\ntrain_pool = Pool(data = demo.loc[demo.timestamp.dt.month <= 9, train_col],\n                 label = demo.loc[demo.timestamp.dt.month <= 9, target_col],\n                 cat_features = ['primary_use'])\n\nvalidation_pool = Pool(data = demo.loc[demo.timestamp.dt.month > 9, train_col],\n                 label = demo.loc[demo.timestamp.dt.month > 9, target_col],\n                 cat_features = ['primary_use'])","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"trusted":false},"cell_type":"code","source":"model = CatBoostRegressor(iterations = 1000, learning_rate = 0.1, depth = 6,\n                         loss_function = 'RMSE',\n                         eval_metric = RMSLE(),\n                         random_seed = 1234,\n                         use_best_model = True)\n\nmodel.fit(train_pool, eval_set = validation_pool, silent = True, early_stopping_rounds = 50)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"DataFrame({'names': model.feature_names_,\n           'importance': model.feature_importances_\n          }).sort_values(by = 'importance', ascending = False)\n           \n       ","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"model.drop_unused_features()\nmodel.feature_names_","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"","_uuid":"","trusted":true},"cell_type":"code","source":"import pandas as pd\nbuilding_metadata = pd.read_csv(\"../input/ashrae-energy-prediction/building_metadata.csv\")\nsample_submission = pd.read_csv(\"../input/ashrae-energy-prediction/sample_submission.csv\")\ntest = pd.read_csv(\"../input/ashrae-energy-prediction/test.csv\")\ntrain = pd.read_csv(\"../input/ashrae-energy-prediction/train.csv\")\nweather_test = pd.read_csv(\"../input/ashrae-energy-prediction/weather_test.csv\")\nweather_train = pd.read_csv(\"../input/ashrae-energy-prediction/weather_train.csv\")","execution_count":0,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.1"},"toc":{"base_numbering":1,"nav_menu":{},"number_sections":false,"sideBar":true,"skip_h1_title":true,"title_cell":"Table of Contents","title_sidebar":"Contents","toc_cell":true,"toc_position":{},"toc_section_display":true,"toc_window_display":false}},"nbformat":4,"nbformat_minor":1}