{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction\n\nIn this notebook a model is generated for predicting building energy peformance as set out in the ASHRAE - Great Energy Predictor III competition. The analysis proceeds as follows:\n\n<a href='#1'>1. Loading Data</a>\n\n<a href='#2'>2. Missing Values</a>\n\n<a href='#3'>3. Combining Datasets</a>\n\n   <a href='#3.1'>3.1. Additional Missing Data</a>\n    \n<a href='#4'>4. Reducing Memory Requirements</a>\n\n<a href='#5'>5. Adding Features</a>\n\n<a href='#6'>6. Outlier Treatment</a>\n\n<a href='#7'>7. Model Training</a>\n\n<a href='#8'>8. Model Predictions</a>"},{"metadata":{},"cell_type":"markdown","source":"# <a id='1'>1. Loading Data</a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"#import libraries\nimport pandas as pd\nimport numpy as np\nimport missingno as msno \nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport pickle","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#load data\ntrain = pd.read_csv('/kaggle/input/ashrae-energy-prediction/train.csv')\ntrain['timestamp'] = pd.to_datetime(train['timestamp']) #the train dataset contains a 'timestamp' column we convert to a datetime object for ease of use\ntest = pd.read_csv('/kaggle/input/ashrae-energy-prediction/test.csv')\ntest['timestamp'] = pd.to_datetime(test['timestamp'])\nweather_train = pd.read_csv('/kaggle/input/ashrae-energy-prediction/weather_train.csv')\nweather_test = pd.read_csv('/kaggle/input/ashrae-energy-prediction/weather_test.csv')\nbuild_meta = pd.read_csv('/kaggle/input/ashrae-energy-prediction/building_metadata.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# <a id='2'>2. Missing Values</a>\n\nThe train and test data contains missing values and is spread out over several dataframes (meter readings, building meta-data, and weather data). Here we fill the missing values and combine the datsets into overall train/test sets for model training and prediction."},{"metadata":{"trusted":true},"cell_type":"code","source":"#Define a function to check if there are is any missing information in the datasets\ndef get_missing_info(df):\n    num_entries = df.shape[0]*df.shape[1]\n    null_entries = df.isnull().sum().sum()\n    percent_empty = null_entries/num_entries*100\n    num_missing = df.isna().sum()\n    percent_missing = num_missing/len(df)*100\n    col_modes = df.mode().loc[0]\n    percent_mode = [df[x].isin([df[x].mode()[0]]).sum()/len(df)*100 for x in df]\n    missing_value_df = pd.DataFrame({'num_missing': num_missing,\n                                     'percent_missing': percent_missing, \n                                     'mode': col_modes,\n                                     'percent_mode':percent_mode})\n    print('total empty percent:', percent_empty, '%')\n    print('columns that are more than 97% mode:', missing_value_df.loc[missing_value_df['percent_mode']>97].index.values)\n    return(missing_value_df)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"get_missing_info(train)\n# get_missing_info(test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There are no missing entries in the train or test dataframes."},{"metadata":{"trusted":true},"cell_type":"code","source":"get_missing_info(build_meta)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Filling in missing values in the building meta-data column\n#Make a copy so we are not changing the initial data\nbuild_meta_f = build_meta.copy()\n#fill all the missing floor counts by the mode (1) and the missing year built by the mean. Nothing else is missing\nbuild_meta_f.fillna({'floor_count':1, 'year_built':int(build_meta['year_built'].mean())}, inplace=True) \n#this is the only categorical column. Convert so it can be handled later by lgbm during fitting\nbuild_meta_f['primary_use'] = build_meta_f['primary_use'].astype('category') ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"get_missing_info(weather_train)\n# get_missing_info(weather_test)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"When filling the missing weather data, using previous or future measurements can be good provided they are not too far in the future or past. We will start with forward/back filling weather data up to 24 hours for each site."},{"metadata":{"trusted":true},"cell_type":"code","source":"#Forward filling missing data in the weather dataset +-24 hours\n#Train weather\nweather_train_f = weather_train.copy() #make a copy so we aren't changing our oridinal data\nweather_train_f['timestamp'] = pd.to_datetime(weather_train_f['timestamp']) #turn the timestamp column into a datetime object\nweather_train_f = weather_train_f.sort_values(by=['site_id', 'timestamp']) #short values by site id then timestamp\n\n#test weather\nweather_test_f = weather_test.copy() #make a copy so we aren't changing our oridinal data\nweather_test_f['timestamp'] = pd.to_datetime(weather_test_f['timestamp']) #turn the timestamp column into a datetime object\nweather_test_f = weather_test_f.sort_values(by=['site_id', 'timestamp']) #short values by site id then timestamp\n\nweather_train_f.fillna(method = 'ffill', inplace=True, limit = 24)#forward fill the missing data up to 12 hours\nweather_train_f.fillna(method = 'bfill', inplace=True, limit = 24)#backfill up to 12 hours\n\nweather_test_f.fillna(method = 'ffill', inplace=True, limit = 24)#forward fill the missing data up to 12 hours\nweather_test_f.fillna(method = 'bfill', inplace=True, limit = 24)#forward fill the missing data up to 12 hours","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"msno.matrix(weather_train_f)\n# msno.matrix(weather_test_f)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As seen above, there are still blocks of data missing over long periods of time. For these we will fill with the mean values for the given site."},{"metadata":{"trusted":true},"cell_type":"code","source":"#Train data\nmissing_cols = [col for col in weather_train_f.columns if weather_train_f[col].isna().any()] \nfill_lib = weather_train_f.groupby('site_id')[missing_cols].transform('mean')#stores the mean of each feature for each site id\nweather_train_f.fillna(fill_lib, inplace=True) #for each feature with missing values, fill the missing entry with the mean for that site\n\n#Test data\nmissing_cols = [col for col in weather_test_f.columns if weather_test_f[col].isna().any()]\nfill_lib = weather_test_f.groupby('site_id')[missing_cols].transform('mean')\nweather_test_f.fillna(fill_lib, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# <a id='3'>3. Combining Datasets</a>\nNow that we have filled in all the missing data, we will merge everything into train and test dataframes."},{"metadata":{"trusted":true},"cell_type":"code","source":"#merge the building meta data and weather data into the train data\ntrain_m = train.merge(build_meta_f, how='left', on = ['building_id'], validate='many_to_one') #merge the building meta data into the train data\ntrain_m = train_m.merge(weather_train_f, how='left', on = ['site_id', 'timestamp'], validate='many_to_one')#add weather data to each time entry for each site ID\n\n#merge the building meta data and weather data into the test data\ntest_m = test.merge(build_meta_f, how='left', on = ['building_id'], validate='many_to_one') #merge the building meta data into the train data\ntest_m = test_m.merge(weather_test_f, how='left', on = ['site_id', 'timestamp'], validate='many_to_one')#add weather data to each time entry for each site ID\n\n#we now delete some of the dataframes we no longer need to free up memory\ndel train, test, weather_train, weather_test, weather_train_f, weather_test_f, build_meta\nimport gc\ngc.collect()\n\ntrain_m.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id='3.1'>3.1. Additional Missing Data</a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"get_missing_info(train_m)\n#get_missing_info(test_m)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can now see some additional missing data in the merged datasets from timestamps present in the train/test sets that\nwere not in the weather data. We will forward fill this data for each building."},{"metadata":{"trusted":true},"cell_type":"code","source":"#train_data\ntrain_m = train_m.sort_values(by=['building_id', 'timestamp'])\ntrain_m.fillna(method = 'ffill', inplace=True)\n\n#we need to temporarily remove the train_m datset from memory here so we do not run out of memory\npickle.dump( train_m, open( \"train_m.p\", \"wb\" ) )\ndel train_m\n\n#test data\ntest_m = test_m.sort_values(by=['building_id', 'timestamp'])\ntest_m.fillna(method = 'ffill', inplace=True)\n\ntrain_m = pickle.load( open('train_m.p', 'rb')) #load back in the train data","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# <a id='4'>4. Reducing Memory Requirements</a>"},{"metadata":{},"cell_type":"markdown","source":"We can reduce the size of our data by changing the datatype based on the precision required by the numbers. Below we use a custom function to achieve this. Source: https://www.kaggle.com/caesarlupum/ashrae-start-here-a-gentle-introduction"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"def reduce_mem_usage(df, verbose=True):\n    numerics = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']\n    start_mem = df.memory_usage().sum() / 1024**2    \n    for col in df.columns:\n        col_type = df[col].dtypes\n        if col_type in numerics:\n            c_min = df[col].min()\n            c_max = df[col].max()\n            if str(col_type)[:3] == 'int':\n                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                    df[col] = df[col].astype(np.int8)\n                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                    df[col] = df[col].astype(np.int16)\n                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                    df[col] = df[col].astype(np.int32)\n                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                    df[col] = df[col].astype(np.int64)  \n            else:\n                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n                    df[col] = df[col].astype(np.float16)\n                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n                    df[col] = df[col].astype(np.float32)\n                else:\n                    df[col] = df[col].astype(np.float64)    \n    end_mem = df.memory_usage().sum() / 1024**2\n    if verbose: print('Mem. usage decreased to {:5.2f} Mb ({:.1f}% reduction)'.format(end_mem, 100 * (start_mem - end_mem) / start_mem))\n    return df\n\ntrain_m = reduce_mem_usage(train_m)\ntest_m = reduce_mem_usage(test_m)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# <a id='5'>5. Adding Features</a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"#Add hour, time of year, and weekend columns\n#Train\ntrain_m['hour'] = train_m['timestamp'].dt.hour\ntrain_m['day_of_year'] = (train_m['timestamp'] - pd.Timestamp('2016-01-01')).dt.days%365\ntrain_m['is_weekend'] = train_m['timestamp'].dt.weekday.isin([5,6]).astype(int)\n\n#Test\ntest_m['hour'] = test_m['timestamp'].dt.hour\ntest_m['day_of_year'] = (test_m['timestamp'] - pd.Timestamp('2016-01-01')).dt.days%365\ntest_m['is_weekend'] = test_m['timestamp'].dt.weekday.isin([5,6]).astype(int)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# <a id='6'>6. Outlier Treatment</a>"},{"metadata":{},"cell_type":"markdown","source":"Below we look at the buildings which tend to have the highest meter readings for each meter type. As we can see, relatively few buildings tend to dominate in terms of meter reading. As an example, building 803 has an average meter reading 0 roughly 20x the average."},{"metadata":{"trusted":true},"cell_type":"code","source":"for m in range(4):\n    idxm = train_m[train_m['meter']==m].groupby('timestamp')['meter_reading'].idxmax()#index of max meter reading for the timestamp\n    print('meter {}'.format(m))\n    #print the number of hours the building was the highest consumer for the top 5 buildings\n    print(train_m.loc[idxm, 'building_id'].value_counts().iloc[:5]) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Mean meter 0 reading: outlier building #803')\nprint(train_m[(train_m['building_id']==803) & (train_m['meter']==0)]['meter_reading'].mean())\nprint('Mean meter 0  reading: overall')\nprint(train_m[(train_m['meter']==0)]['meter_reading'].mean())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Outliers can produce outsized error terms in a model that fits the rest of the data well. To address this, we will rescale the target variable (meter reading) for each building and meter type. \n\nThe overall average meter reading is about 2000 so we will rescale to the range (0,2000). Note that the scale we choose has an effect on model training because the evaluation metric (root mean squared log error) is sensitive to the scale of the numbers."},{"metadata":{"trusted":true},"cell_type":"code","source":"#We would like to rescale the meter reading column for each building and meter reading to prevent outliers from skewing the reults.\n#This is a class to achieve that for any chosen groups. It is a modified version of code by Szymon Maszke: \n#https://stackoverflow.com/questions/55601928/apply-multiple-standardscalers-to-individual-groups\nfrom sklearn.base import clone\nclass GroupTargetTransform:\n    def __init__(self, transformation):\n        self.transformation = transformation\n        self._group_transforms = {} #this library will hold the group transforms\n\n    def _call_with_function(self, X, y, function: str):\n        yhat = pd.Series(dtype = 'float32')#this will hold the rescaled target data\n        X['target'] = pd.Series(y, index=X.index)\n        for gr in X.groupby(self.features):\n            n = gr[0] #this is a tuple id for the group\n            g_X = gr[1] #this is the group dataframe\n            g_yhat = getattr(self._group_transforms[n], function)(g_X['target'].values.reshape(-1,1))#scale the target variable\n            g_yhat = pd.Series(g_yhat.flatten(), index = g_X.index)\n            yhat = yhat.append(g_yhat)\n        X.drop('target', axis=1, inplace = True)\n        return yhat.sort_index()\n    \n    def fit(self, X, y, features):\n        self.features = features\n        X['target'] = pd.Series(y, index=X.index) \n        for gr in X.groupby(self.features):\n            n = gr[0] #this is a tuple id for the group\n            g_X = gr[1] #this is the group dataframe\n            sc = clone(self.transformation) #create a new instance of the transform\n            self._group_transforms[n] = sc.fit(g_X['target'].values.reshape(-1,1))\n        X.drop('target', axis=1, inplace=True)\n        return self\n\n    def transform(self, X, y):\n        return self._call_with_function(X, y, \"transform\")\n\n    def fit_transform(self, X, y, features):\n        self.fit(X, y, features)\n        return self.transform(X, y)\n\n    def inverse_transform(self, X, y):\n        return self._call_with_function(X, y, \"inverse_transform\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#rescale the target variable for each building and meter type.\nfrom sklearn.preprocessing import MinMaxScaler\n\nscaler = GroupTargetTransform(MinMaxScaler(feature_range = (0,2000))) #2000 is roughly the average meter reading for all the train data\ntrain_m['meter_reading_rescaled'] = scaler.fit_transform(train_m, train_m['meter_reading'], ['building_id', 'meter'])\n#convert to log(y+1) so the RMSE evaluation metric is actually giving the RMSLE (teh evaluation metric for the competition)\ntrain_m['meter_reading_rescaled'] = np.log1p(train_m['meter_reading_rescaled']) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# <a id='7'>7. Model Training</a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Save test and train data onto the HDD so we can free up some memory\nimport pickle\npickle.dump( test_m, open( \"test_m.p\", \"wb\" ) )\npickle.dump( train_m, open( \"train_m.p\", \"wb\" ) )\ndel test_m","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Below we will tune the model parameters and get an idea of how well the model can perform on unseen data. We have done this by:\n\n* Holding out the last 2.5 months of data for validation\n* Holding out 10% of the buildings for validation\n* Cross-validating time-series wise (for parameter tuning)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nfrom sklearn.model_selection import train_test_split, GridSearchCV, TimeSeriesSplit\nfrom lightgbm import LGBMRegressor\nfrom sklearn.metrics import make_scorer, mean_squared_error, mean_absolute_error, mean_squared_log_error\n\n#defining a couple of functions for later use\ndef clip(x):\n    return np.clip(x, a_min=0, a_max=None)\ndef rmse(y, y_pred):\n    out = np.sqrt(mean_squared_error(clip(y), clip(y_pred)))\n    return out\n\n#prepare training data\nX = train_m.dropna(subset=['meter_reading']) #drop all rows where the meter reading is not included\nX = X.sort_values(by=['timestamp'], axis=0) #ensure X is sorted by timstamp for later timeseries cross-validation\n\nbuilds = X['building_id'].unique()#array of building ids in the dataset\nbuild_train, build_val = train_test_split(builds, test_size = 0.1, random_state=0)#hold out 10% of the buildings for validation\n\ntrain = X.loc[(X['timestamp']<'2016-10-15') \n          & (X['building_id'].isin(build_train))] #we will train on only the first 80% of the year and 90% buildings\nval_t = X.loc[(X['timestamp']>='2016-10-15') & (X['building_id'].isin(build_train))] #rest of the year and same buildings as above\nval_b = X.loc[(X['building_id'].isin(build_val))] #full year and the rest of the buildings\n\ny_train, y_val_t, y_val_b = train['meter_reading_rescaled'], val_t['meter_reading'], val_b['meter_reading'] #extracting the meter reading as our target variable\nX_train, X_val_t, X_val_b = train.drop(['meter_reading', 'meter_reading_rescaled', 'timestamp'], axis=1), val_t.drop(['meter_reading', 'meter_reading_rescaled','timestamp'], axis=1), val_b.drop(['meter_reading','meter_reading_rescaled','timestamp'], axis=1)\n\ndel X, train, val_t, val_b #no longer needed - free up memory\n\n# lgbm model\nmodel = LGBMRegressor(\nnum_leaves = 600,\nmin_data_in_leaf = 50,\nrandom_state = 0\n)\n\n#cross-validation for paramter tuning\n# params = {\n#     'num_leaves': [600],#add values to these lists to run a parmaeter optimization. These were found to be optimum.\n#          }\n\n# #define a rmse scorer for gridsearchcv\n# rmse_scorer = make_scorer(rmse, greater_is_better=False)\n# #split training data time series-wise for cross-validation\n# tscv = TimeSeriesSplit(n_splits=3)\n# #grid search\n# #Note that the scores given are based on the rescaled meter readings, so are not a direct representation of model performance\n# for model_name, grid in params.items():\n#     searchCV = GridSearchCV(model, scoring=rmse_scorer, cv=tscv, param_grid=params)\n#     print('GridSearchCV fitting...')\n#     searchCV.fit(X_train, y_train)\n#     scores = -1*searchCV.cv_results_['mean_test_score']\n#     params = searchCV.cv_results_['params']\n#     for i in range(0, len(scores)):\n#       print(params[i], '->', scores[i])\n\n#Evaluate combined model on the ramining validation data\nprint('Fitting...')\nmodel.fit(X_train, y_train)\n\nprint('Time predictions...')\npreds = clip(model.predict(X_val_t)) #make time predictions\npreds_inv = scaler.inverse_transform(X_val_t, np.expm1(preds)) #convert back to original scale, remembering to invert the log transform\ny_val_t = y_val_t.sort_index()\nscore = mean_absolute_error(preds_inv, y_val_t)\nprint('Mean absolute error - time prediction:', score)\nRMSLE = np.sqrt(mean_squared_log_error(preds_inv, y_val_t))\nprint('RMSLE - time prediction:', RMSLE)\n\nprint('Building predictions...')\npreds = clip(model.predict(X_val_b))\npreds_inv = scaler.inverse_transform(X_val_b, np.expm1(preds))\ny_val_b = y_val_b.sort_index()\nscore = mean_absolute_error(preds_inv, y_val_b)\nprint('Mean absolute error - new buildings:', score)\nRMSLE = np.sqrt(mean_squared_log_error(preds_inv, y_val_b))\nprint('RMSLE - new buildings:', RMSLE)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now that we have tuned the model parameters and have an idea of model performance. We will fit on the entire training dataset so we have as much information as possible for the final test set prediction."},{"metadata":{"trusted":true},"cell_type":"code","source":"#prepare training data\ntrain = train_m.dropna(subset=['meter_reading']) #drop all rows where the meter reading is not included\n\ny_train = train['meter_reading_rescaled'] #extracting the meter reading as our target variable\nX_train = train.drop(['meter_reading', 'meter_reading_rescaled', 'timestamp'], axis=1)\n\ndel train, train_m, X_val_t #no longer needed - free up memory\ngc.collect()\n\n#Fitting on all training data\nprint('Final Fitting...')\nmodel.fit(X_train, y_train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# <a id='8'>8. Model Predictions</a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"#free up memory then load in the test data\ndel X_train, y_train\ngc.collect()\nX_test = pickle.load( open( \"test_m.p\", \"rb\" ) )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#output the final predictions on the test data to a csv file\npreds = np.empty(len(X_test))#we will predict in three steps to save memory\nprint('A')\npreds[:int(len(X_test)/3)] = model.predict(X_test.drop(['row_id', 'timestamp'], axis=1).iloc[:int(len(X_test)/3)])\npreds[int(len(X_test)/3):int(len(X_test)*2/3)] = model.predict(X_test.drop(['row_id', 'timestamp'], axis=1).iloc[int(len(X_test)/3):int(len(X_test)*2/3)])\npreds[int(len(X_test)*2/3):] = model.predict(X_test.drop(['row_id', 'timestamp'], axis=1).iloc[int(len(X_test)*2/3):])\nfinal_predictions = scaler.inverse_transform(X_test, np.expm1(preds))\nX_test = X_test.sort_index()\noutput = pd.DataFrame({'row_id': X_test['row_id'], 'meter_reading': clip(final_predictions)})\noutput['meter_reading'] = output['meter_reading'].round(decimals=4)#to save space\noutput.to_csv('sub.csv', index_label = 'row_id', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As a final check, we can plot the model predictions with the existing data for a specific building and meter type. Everything during 2016 is real data while the rest is our final predictions."},{"metadata":{"trusted":true},"cell_type":"code","source":"b_id = 400 #building id\nm_id = 0 #meter id\n\ntrain_m = pickle.load( open('train_m.p', 'rb'))\nbuilding_current = train_m.loc[(train_m['building_id']==b_id) & (train_m['meter']==m_id)]\nbuilding_forecast = X_test.loc[(X_test['building_id']==b_id) & (X_test['meter']==m_id)].merge(output, how='left', on = ['row_id'], validate='one_to_one')\nbuilding = pd.concat([building_current, building_forecast])\n\nX_o = building.drop(['meter_reading', 'row_id', 'timestamp', 'site_id'], axis=1)\ny_o = building['meter_reading']\n\nmod_plot = pd.DataFrame(data={#'meter_reading (predicted)':building_forecast['meter_reading'],\n                                    'meter_reading (actual and predicted)':y_o.values},\n                                    index=building['timestamp'])\nstart_time = '2016-01-01'\nend_time = '2019-01-01'\nmod_plot = mod_plot.loc[(start_time<mod_plot.index)&(mod_plot.index<end_time)].resample('D').mean()\nmod_plot.plot(rot=45)#plot each model vs target","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}