{"cells":[{"metadata":{"trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# plotting libraries\nimport matplotlib\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nimport gc # garbage collector\n\n# stats models\nimport statsmodels.api as sm\n\n\n# deal with date in x-axis of plots\nfrom pandas.plotting import register_matplotlib_converters\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('../input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Introduction\nI am still a novice when it comes to time series so I want to start with a simple baseline model that only considers the time series feature. Let's jump in..."},{"metadata":{},"cell_type":"markdown","source":"# Load Data\nimport the training dataframe"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('../input/ashrae-energy-prediction/train.csv')\ntest_df = pd.read_csv('../input/ashrae-energy-prediction/test.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"train_df.columns","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n## Set Column Datatypes"},{"metadata":{"trusted":false},"cell_type":"code","source":"# Saving some memory\nd_types = {'building_id': np.int16,\n          'meter': np.int8}\n\nfor feature in d_types:\n    train_df[feature] = train_df[feature].astype(d_types[feature])\n    \n    \ntrain_df[\"timestamp\"] = pd.to_datetime(train_df[\"timestamp\"], infer_datetime_format=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Add Columns to Training Data\nFirst let's describe the distribution of the meter reading for each meter type."},{"metadata":{"trusted":false},"cell_type":"code","source":"meter_mapping = {0: 'electricity', 1: 'chilledwater', 2: 'steam', 3: 'hotwater'}\ntrain_df['meter_type'] = train_df['meter'].map(meter_mapping)\ntest_df['meter_type'] = test_df['meter'].map(meter_mapping)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Add the log of each meter reading. I am not sure if either of these extra columns will be useful for prediction, but I like to have them."},{"metadata":{"trusted":false},"cell_type":"code","source":"train_df[\"log_meter_reading\"]=np.log(train_df[\"meter_reading\"]+.00001)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Prediction Time\nI will go through the most naive approaches and code them manually. This first function is used for evaluating the models."},{"metadata":{"trusted":false},"cell_type":"code","source":"def rmsle(pred_series,true_series):\n    sum_series = (np.log(pred_series+1) - \\\n        np.log(true_series+1))**2\n    return np.sqrt(np.sum(sum_series))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We need to split the original training data into a training and validation set. I decided to splits the training set into the first 9 months and the validation set into the last 3 months."},{"metadata":{"trusted":false},"cell_type":"code","source":"train = train_df.loc[train_df[\"timestamp\"]<'2016-10-01',:]\nvalid = train_df.loc[train_df[\"timestamp\"]>='2016-10-01',:]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Naive Approach 1: Just take the value of the previous timepoint\nFor this approach I use the previous hour to predict the current hour. Here are my steps:\n1. Get the meter reading for the previous hour\n2. Predict the current meter reading by setting it to the 1 hour prior value"},{"metadata":{"trusted":false},"cell_type":"code","source":"# create new data frame for this model\nvalid_1hrPrior_df = valid.copy()\n# rename timestamp to signify the current meter reading time\nvalid_1hrPrior_df = valid_1hrPrior_df.rename(\n    columns={\"timestamp\": \"now\", \n             \"meter_reading\": \"cur_meter_reading\",\n            \"log_meter_reading\":\"cur_log_meter_reading\"})\n\n# get previous hour before validation set\nprev_hour = pd.to_datetime(\"2016-10-01 00:00:00\") - \\\n        pd.Timedelta(hours=1)\n\n# This model splits the data based on \n# building ID and model type\nfor b_id in list(valid[\"building_id\"].unique()):\n    for meter_t in list(\n        valid_1hrPrior_df.loc[valid_1hrPrior_df[\"building_id\"]==b_id,\"meter\"].unique()):\n        if(not ((b_id in train[\"building_id\"]) and\n           (meter_t in train.loc[train[\"building_id\"]==b_id,\"meter\"].values) and\n           ((train.loc[(\n               (train[\"building_id\"]==b_id) & \n               (train[\"meter\"]==meter_t)),\"timestamp\"] == prev_hour).any()))):\n            print(\"missing!\")\n            print(b_id)\n            print(meter_t)\n            # if there is no meter reading for a specific\n            # building ID in the previous hour\n            # then I'll just set the reading\n            # to 0\n            valid_1hrPrior_df.loc[((valid_1hrPrior_df[\"building_id\"]==b_id) &\n                valid[\"meter\"]==meter_t),\"pred_meter_reading\"] = 0.0\n        else:\n            valid_1hrPrior_df.loc[((valid_1hrPrior_df[\"building_id\"]==b_id) &\n                valid_1hrPrior_df[\"meter\"]==meter_t),\"pred_meter_reading\"] = \\\n                train.loc[(\n                (train[\"building_id\"]==b_id) &\n                (train[\"meter\"]==meter_t) &\n                (train[\"timestamp\"]==prev_hour)),\"meter_reading\"].values[0]\n\n        \n","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"print(\"Naive Approach - RMSLE value:\")\nprint(rmsle(valid_1hrPrior_df[\"pred_meter_reading\"],\n           valid_1hrPrior_df[\"cur_meter_reading\"]))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This is a really high value. Hopefully the next approach does a bit better for comparison."},{"metadata":{},"cell_type":"markdown","source":"## Naive Approach 2: Just take the average values from the training data\nFor this approach I use the average of the training data to predict the validation data."},{"metadata":{"trusted":false},"cell_type":"code","source":"# create new data frame for this model\nvalid_avgVal_df = valid.copy()\n# rename timestamp to signify the current meter reading time\nvalid_avgVal_df = valid_avgVal_df.rename(\n    columns={\"timestamp\": \"now\", \n             \"meter_reading\": \"cur_meter_reading\",\n            \"log_meter_reading\":\"cur_log_meter_reading\"})\n\n# This model splits the data based on \n# building ID and model type\nfor b_id in list(valid[\"building_id\"].unique()):\n    for meter_t in list(\n        valid_avgVal_df.loc[valid_avgVal_df[\"building_id\"]==b_id,\"meter\"].unique()):\n        if(not ((b_id in train[\"building_id\"]) and\n           (meter_t in train.loc[train[\"building_id\"]==b_id,\"meter\"].values))):\n            print(\"missing!\")\n            print(b_id)\n            print(meter_t)\n            # if there is no meter reading for a specific\n            # building ID then I'll just set the reading\n            # to the average value of that meter given\n            # all of the building IDs.\n            valid.loc[((valid_avgVal_df[\"building_id\"]==b_id) &\n                valid_avgVal_df[\"meter\"]==meter_t),\"pred_meter_reading\"] = \\\n                train.loc[train[\"meter\"]==meter_t,\"meter_reading\"].mean()\n        else:\n            # calculate the average meter_reading values\n            # for each meter given the building id\n            valid_avgVal_df.loc[((valid_avgVal_df[\"building_id\"]==b_id) &\n                valid_avgVal_df[\"meter\"]==meter_t),\"pred_meter_reading\"] = \\\n                train.loc[(\n                (train[\"building_id\"]==b_id) &\n                (train[\"meter\"]==meter_t)),\"meter_reading\"].mean()","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"print(\"Naive Approach - RMSLE value:\")\nprint(rmsle(valid_avgVal_df[\"pred_meter_reading\"],\n           valid_avgVal_df[\"cur_meter_reading\"]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"markdown","source":"This value is still high but may be a better baseline. Since each meter seems to have its own patter I am also interested in the RMSLE value for each meter."},{"metadata":{"trusted":true},"cell_type":"code","source":"avgVal_rmsle_list=[]\nfor meter_t in list(valid_avgVal_df[\"meter\"].unique()):\n        sub_valid_avgVal_df = valid_avgVal_df.loc[(\n            valid_avgVal_df[\"meter\"]==meter_t),:].copy()\n        sub_rmsle = rmsle(sub_valid_avgVal_df[\"pred_meter_reading\"],\n           sub_valid_avgVal_df[\"cur_meter_reading\"])\n        sub_rmsle_df = pd.DataFrame({\"meter\":[meter_t],\n                                   \"rmsle\":[sub_rmsle]})\n        avgVal_rmsle_list.append(sub_rmsle_df)\navgVal_rmsle_df = pd.concat(avgVal_rmsle_list)\navgVal_rmsle_df  ","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.1"}},"nbformat":4,"nbformat_minor":1}