{"cells":[{"metadata":{},"cell_type":"markdown","source":"# ASHRAE – Great Energy Predictor III\nBy Mehul Haria, Raymond Huang, Naureen Pethani, Shelina Khoja, Kinnari Patel\n\n### Abstract:\nWe are using a dataset related to ASHRAE – Great Energy Predictor III (How much energy will a building consume?). The goal is to develop models from ASHRAE’s 2016 data in order to better understand metered building energy usage in the following areas: chilled water, electric, hot water, and steam meters. The data comes from over 1,000 buildings over a one-year timeframe. The method chosen to solve the problem is Linear Regression.\n    \n\n### Introduction:\nAs the impact of climate change is being felt more increasingly, organizations are looking for ways to lower their energy footprint. Our organization (Greentech Inc) has developed technology to provide highly efficient heating and cooling systems that operate at much lower energy consumption.\n\nWe have partnered with governments across the world to retrofit the highest energy users with significant subsidies from our government partners. Highest energy users are defined as those with the highest energy consumption as determined by their energy meter readings.\n\nBy starting with the highest energy consumption users, we can prioritize our resources to provide the greatest impact, ultimately resulting in reduced energy consumption and a more environmentally friendly solution.\n\n    \n    \n### Background:\nThe four dataset presented in (Kaggle ASHRAE Energy Predictor III dataset) are related to ASHRAE -  Energy  Predictor III.\n\nAs a specific purpose of lab assignment, we are looking at Linear Regression problem using ASHRAE -Energy Predictor III dataset. Full library of the datasets and their description are located here: (Kaggle ASHRAE Energy Predictor III dataset).\n\n### Objective\n\nThe objective of this article is to provide a reliable and feasible recommendation algorithm to predict\nHow much energy will a building consume?  The train dataset has our target variable called “meter reading” with datatype float, hence the task could be solved by Linear Regression methods. The following methodology is used: \n\nLinear Regression tasks will be applied to the problem:\n• By putting all relevant variables in the model\n• Leave the irrelevant variables out\n• Check linearity\n• Check regression assumptions:\n– Residuals have a mean of zero\n– Normality of errors\n– Linearity of variables\n\n### Outline\n1.Data Understanding\n\n2.Data Preparation\n\n2.1 Mergin tables\n\n2.2 Droping columns and filling null value for column: 'air_temperature', 'wind_speed', 'precip_depth_1_hr', 'cloud_coverage'\n\n3.Data Modeling\n\n3.1 Linear Modeling and optimization\n\n3.2 Decision Tree modeling\n\n3.3 Random Forest modeling\n    \n    \n    "},{"metadata":{"trusted":true},"cell_type":"code","source":"%matplotlib inline\nimport pandas as pd\nimport seaborn as sns\nimport statsmodels.formula.api as smf\nfrom sklearn.linear_model import LinearRegression\nfrom sklearn import metrics\nfrom sklearn.model_selection  import train_test_split\nimport numpy as np\nfrom scipy.stats import norm # for scientific Computing\nfrom scipy import stats, integrate\nimport matplotlib.pyplot as plt\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 1.Data Understanding\n\nReading the datasets"},{"metadata":{},"cell_type":"markdown","source":"The dataset presented in (Kaggle ASHRAE Energy Predictor III dataset) of Energy Predictor has Five datasets. \nThe five primary files to be used are described below, with the variable names also included:\ntrain.csv (202116100, 3)\n\nbuilding_id - Foreign key for the building metadata.\nmeter - The meter id code. Read as {0: electricity, 1: chilledwater, 2: steam, 3: hotwater}. Not every building has all meter types.\ntimestamp - When the measurement was taken\nmeter_reading - The target variable. Energy consumption in kWh (or equivalent). Note that this is real data with measurement error, which we expect will impose a baseline level of modeling error. \n\nbuilding_meta.csv with (1449,6) data1449, 6) (1449, 6)(\n\nsite_id - Foreign key for the weather files.\nbuilding_id - Foreign key for training.csv\nprimary_use - Indicator of the primary category of activities for the building based on EnergyStar property type definitions\nsquare_feet - Gross floor area of the building\nyear_built - Year building was opened\nfloor_count - Number of floors of the building\n\nweather_[train/test].csv with (139773, 8) / (277243, 8) data\n\nWeather data from a meteorological station as close as possible to the site.\nsite_id\nair_temperature - Degrees Celsius\ncloud_coverage - Portion of the sky covered in clouds, in oktas\ndew_temperature - Degrees Celsius\nprecip_depth_1_hr - Millimeters\nsea_level_pressure - Millibar/hectopascals\nwind_direction - Compass direction (0-360)\nwind_speed - Meters per second\n\ntest.csv with (41697600, 3) data\n\nrow_id - Row id for your submission file\nbuilding_id - Building id code\nmeter - The meter id code\ntimestamp - Timestamps for the test data period\n\nsample_submission.csv with (41697600, 2) data\n\nA valid sample submission.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\n# Any results you write to the current directory are saved as output.\nASHRAE_train =  pd.read_csv('/kaggle/input/ashrae-energy-prediction/train.csv')\nASHRAE_test=pd.read_csv('/kaggle/input/ashrae-energy-prediction/weather_test.csv')\nweather_train=pd.read_csv('/kaggle/input/ashrae-energy-prediction/weather_train.csv')\nweather_test=pd.read_csv('/kaggle/input/ashrae-energy-prediction/weather_test.csv')\nbuilding_meta=pd.read_csv('/kaggle/input/ashrae-energy-prediction/building_metadata.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To perform the analysis, certain Python libraries were used. The code was used to load and initialize the libraries. We have 41 million rows to predict with the built model.\nWe combined three datasets train.csv, building_metadata, weather train with foreign keys buildeing_id and timestamp respectively. So, we are dealing with big datasets here (20 and 40 million rows).\nTo save some space from the memory, we are going use a function built as part of this popular notebook to reduce the memory size use of the datasets. \nAfter memory reduction, the original datatype was changed from int_64 to int_16 for building_id. Memory usage was reduced greatly by half in order to improve speed and performance.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"ASHRAE_train.info()\n","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"weather_train.info()\n","execution_count":null,"outputs":[]},{"metadata":{"scrolled":true},"cell_type":"markdown","source":"So let us reduce the data type and reduce memory usage using define function."},{"metadata":{"trusted":true},"cell_type":"code","source":"## Function to reduce the DF size\ndef reduce_memory_usage(df, verbose=True):\n    numerics = ['int16', 'int32', 'int64', 'float16', 'float32', 'float64']\n    start_mem = df.memory_usage().sum() / 1024**2    \n    for col in df.columns:\n        col_type = df[col].dtypes\n        if col_type in numerics:\n            c_min = df[col].min()\n            c_max = df[col].max()\n            if str(col_type)[:3] == 'int':\n                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                    df[col] = df[col].astype(np.int8)\n                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                    df[col] = df[col].astype(np.int16)\n                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                    df[col] = df[col].astype(np.int32)\n                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                    df[col] = df[col].astype(np.int64)  \n            else:\n                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n                    df[col] = df[col].astype(np.float16)\n                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n                    df[col] = df[col].astype(np.float32)\n                else:\n                    df[col] = df[col].astype(np.float64)    \n    end_mem = df.memory_usage().sum() / 1024**2\n    if verbose: print('Mem. usage decreased to {:5.2f} Mb ({:.1f}% reduction)'.format(end_mem, 100 * (start_mem - end_mem) / start_mem))\n    return df\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"reduce_memory_usage(building_meta)\nreduce_memory_usage(weather_train)\nreduce_memory_usage(ASHRAE_train)\n\nreduce_memory_usage(weather_test)\nreduce_memory_usage(ASHRAE_test)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Data attributes summary\n\nQuick view of the data attributes statistics presented in the Table 2. For each attribute in the dataset\nthis table shows min, max, mean and normal distribution 1st and 3rd quartiles values.\n\nyear_built, floor_count, cloud_coverage, precip_depth_1_hr, sea_level_pressure and wind_direction are all missing significant information based on the counts. \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"ASHRAE_train.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"After memory reduction, the original type was changed from int 64 to int 16 for building_id. Memory usage was reduced greatly by half in order to improve speed and performance."},{"metadata":{},"cell_type":"markdown","source":"Preview of ASHRAE_train data\n\nQuick view of the data attributes statistics of ASHRAE_train and building_meta presented below. For each attribute in the dataset this table shows min, max, mean and normal distribution 1st and 3rd quartiles values."},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Size of the building dataset is', building_meta.shape)\nprint('Size of the weather_train dataset is', weather_train.shape)\nprint('Size of the train dataset is', ASHRAE_train.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ASHRAE_train.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"building_meta.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Checking unique elements in primary use column within building_meta table."},{"metadata":{"trusted":true},"cell_type":"code","source":"primary_use_numbersOfUniqueValue = building_meta['primary_use'].nunique()\n \nprint('Number of unique values in column \"primary_use\" of the building_meta : ')\nprint(primary_use_numbersOfUniqueValue)\nprimary_use_element = building_meta['primary_use'].unique()\n \nprint('Unique element in column \"primary_use\" of the building_meta : ')\nprint(primary_use_element)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Columns of the building dataset is', building_meta.columns)\nprint('Columns of the weather_train dataset is', weather_train.columns)\nprint('Columns of the train dataset is', ASHRAE_train.columns)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The connection between building_meta and train dataset is building_id while building_meta dataset can merge with weather_train dataset on site_id.\n\nSo basically building_id was the primary key and site_id was the foreign key for the building_meta table\n\nbuilding_id was primary key for train dataset while site_id was primary key for weather_train dataset."},{"metadata":{},"cell_type":"markdown","source":"Here, Heatmap was used to check missing value in building_meta. And we found out that year_built and floor_count are missing a lot of data, thus may not be useful for this analysis."},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(15,7))\nsns.heatmap(building_meta.isnull(), yticklabels=False,cmap='viridis')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\nprint(\"Percentage of missing values in the building_meta dataset\")\nbuilding_meta.isna().sum()/len(building_meta)*100","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\nprint(\"Percentage of missing values in the train dataset\")\nASHRAE_train.isna().sum()/len(ASHRAE_train)*100","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\nprint(\"Percentage of missing values in the weather_train dataset\")\nweather_train.isna().sum()/len(weather_train)*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2.0 Data Praparation\n\n### 2.1 Merging tables for analysis"},{"metadata":{"trusted":true},"cell_type":"code","source":"#pd.merge(df1, df2, on='employee')\nBuildingTrainMerge=building_meta.merge(ASHRAE_train,left_on='building_id',right_on='building_id',how='left')\nBuildingTrainMerge.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train=BuildingTrainMerge.merge(weather_train,left_on=['site_id','timestamp'],right_on=['site_id','timestamp'],how='left')\nBTW_train.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Percentage of missing values in the BTW_train dataset\")\nBTW_train.isna().sum()/len(BTW_train)*100","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.hist('sea_level_pressure')\nBTW_train[['sea_level_pressure']].describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.hist('cloud_coverage')\nBTW_train[['cloud_coverage']].describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.hist('precip_depth_1_hr')\nBTW_train[['precip_depth_1_hr']].describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.hist('wind_speed')\nBTW_train[['wind_speed']].describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.hist(column='air_temperature')\nBTW_train[['air_temperature']].describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We wanted to take a bit more of a look at \"sea level pressure\", \"cloud coverage\", \"air temperature\", \"wind speed\" and \"precip_depth_1hr\" to get a better idea of how the values are spread out. The precip_depth_1hr is heavily skewed, while \"cloud coverage\" and \"sea level pressure\" have a relatively small range between their max and min. Air temperature and wind speed have fairly normal distribution. For now, the decision was made to keep the data."},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.boxplot(x = 'meter', y = 'meter_reading', data = BTW_train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Define outlier"},{"metadata":{"trusted":true},"cell_type":"code","source":"def outlier_function(df, col_name):\n    ''' this function detects first and third quartile and interquartile range for a given column of a dataframe\n    then calculates upper and lower limits to determine outliers conservatively\n    returns the number of lower and uper limit and number of outliers respectively\n    '''\n    first_quartile = np.percentile(\n        np.array(df[col_name].tolist()), 25)\n    third_quartile = np.percentile(\n        np.array(df[col_name].tolist()), 75)\n    IQR = third_quartile - first_quartile\n                      \n    upper_limit = third_quartile+(3*IQR)\n    lower_limit = first_quartile-(3*IQR)\n    outlier_count = 0\n                      \n    for value in df[col_name].tolist():\n        if (value < lower_limit) | (value > upper_limit):\n            outlier_count +=1\n    return lower_limit, upper_limit, outlier_count","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"{} percent of {} are outliers.\"\n      .format((\n              (100 * outlier_function(BTW_train, 'meter_reading')[2])\n               / len(BTW_train['meter_reading'])),\n              'meter_reading'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Distribution of the meter reading in meters without zeros\nplt.figure(figsize=(12,10))\n\n#list of different meters\nmeters = sorted(BTW_train['meter'].unique().tolist())\n\n# plot meter_reading distribution for each meter\nfor meter_type in meters:\n    subset = BTW_train[BTW_train['meter'] == meter_type]\n    sns.kdeplot(np.log1p(subset[\"meter_reading\"]), \n                label=meter_type, linewidth=2)\n\n# set title, legends and labels\nplt.ylabel(\"Density\")\nplt.xlabel(\"Meter_reading\")\nplt.legend(['electricity', 'chilled water', 'steam', 'hot water'])\nplt.title(\"Density of Logartihm(Meter Reading + 1) Among Different Meters\", size=14)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"corrmat=BTW_train.corr()\nfig,ax=plt.subplots(figsize=(12,10))\nsns.heatmap(corrmat,annot=True,annot_kws={'size': 12})\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 2.2 Dropping columns and filling null value\n\n\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train = BTW_train.drop(columns=['year_built', 'floor_count', 'wind_direction', 'dew_temperature'])\nBTW_train ['timestamp'] =  pd.to_datetime(BTW_train['timestamp'])\nBTW_train['Month']=pd.DatetimeIndex(BTW_train['timestamp']).month\nBTW_train['Day']=pd.DatetimeIndex(BTW_train['timestamp']).day\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train= BTW_train.groupby(['meter',BTW_train['building_id'],'primary_use',BTW_train['Month'], BTW_train['Day']]).agg({'meter_reading':'sum', 'air_temperature': 'mean', 'wind_speed': 'mean', 'precip_depth_1_hr': 'mean', 'cloud_coverage': 'mean', 'square_feet': 'mean'})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train = BTW_train.reset_index()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Change data type to float 32 for filling NA value before transforming them into int for smooth modeling processing"},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train['wind_speed'] = BTW_train['wind_speed'].astype('float32')\nBTW_train['air_temperature'] = BTW_train['air_temperature'].astype('float32')\nBTW_train['precip_depth_1_hr'] = BTW_train['precip_depth_1_hr'].astype('float32')\nBTW_train['cloud_coverage'] = BTW_train['cloud_coverage'].astype('float32')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train['precip_depth_1_hr'].fillna(method='ffill', inplace = True)\nBTW_train['cloud_coverage'].fillna(method='bfill', inplace = True)\n\nBTW_train['wind_speed'].fillna(BTW_train['wind_speed'].mean(), inplace=True)\nBTW_train['air_temperature'].fillna(BTW_train['air_temperature'].mean(), inplace=True)\nBTW_train.isnull().sum()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.dtypes","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_train.columns","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data Modeling  845701 records for modeling\n\n### 3.1 Linear Regression"},{"metadata":{},"cell_type":"markdown","source":"\nHere column 'primaty_use' was treated by get_dummies function"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nBTW_linearR = pd.get_dummies(BTW_train, columns=['primary_use'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BTW_linearR.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X =BTW_linearR[['building_id', 'meter', 'air_temperature', 'wind_speed', 'precip_depth_1_hr', 'cloud_coverage',\n       'square_feet', 'primary_use_Education', 'primary_use_Entertainment/public assembly',\n       'primary_use_Food sales and service', 'primary_use_Healthcare',\n       'primary_use_Lodging/residential',\n       'primary_use_Manufacturing/industrial', 'primary_use_Office',\n       'primary_use_Other', 'primary_use_Parking',\n       'primary_use_Public services', 'primary_use_Religious worship',\n       'primary_use_Retail', 'primary_use_Services',\n       'primary_use_Technology/science', 'primary_use_Utility',\n       'primary_use_Warehouse/storage', 'Month', 'Day']]\n\n# Create target variable\ny = BTW_linearR['meter_reading']\n\n# Train, test, split\nfrom sklearn.model_selection import train_test_split\nX_train, X_test, y_train, y_test = train_test_split(X,y, test_size = .20, random_state= 0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Fit\n# Import model\nfrom sklearn.linear_model import LinearRegression\n\n# Create linear regression object\nregressor = LinearRegression()\n\n# Fit model to training data\nregressor.fit(X_train,y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Predicting test set results\ny_pred = regressor.predict(X_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Accuracy %d', regressor.score(X_test, y_test))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Calculate R Sqaured\nprint('R^2 =',metrics.explained_variance_score(y_test,y_pred))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cdf = pd.DataFrame(data = regressor.coef_, index = X.columns, columns = ['Coefficients'])\ncdf","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cdf.Coefficients.nlargest(10).plot(kind='barh')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import statsmodels.api as sm\nfrom scipy import stats\nX =BTW_linearR[['building_id', 'meter', 'air_temperature', 'wind_speed', 'precip_depth_1_hr', 'cloud_coverage',\n       'square_feet', 'primary_use_Education', 'primary_use_Entertainment/public assembly',\n       'primary_use_Food sales and service', 'primary_use_Healthcare',\n       'primary_use_Lodging/residential',\n       'primary_use_Manufacturing/industrial', 'primary_use_Office',\n       'primary_use_Other', 'primary_use_Parking',\n       'primary_use_Public services', 'primary_use_Religious worship',\n       'primary_use_Retail', 'primary_use_Services',\n       'primary_use_Technology/science', 'primary_use_Utility',\n       'primary_use_Warehouse/storage', 'Month', 'Day']]\n\n# Create target variable\ny = BTW_linearR['meter_reading']\n \n \n \nX2 = sm.add_constant(X)\nest = sm.OLS(y, X2)\nest2 = est.fit()\nprint(est2.summary())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"P-Values of those criteria suggest that they are acceptable variables to use in modelling as they likely reject the null hypothesis.\nHowever, looking at the split at the first node, 95% of the data was under represented for falling under the square feet criteria. The R2 value was then only representative of 5% of the data, skewed towards the higher square feet values. The weight given to “meter’ is understandable, as it was shown in a box plot earlier that the “meter” type influenced the ‘meter readings’, specifically for higher values.\n\nBecause of the high skew towards larger ‘Square feet’ values, we don’t believe that the Decision tree model is an accurate model to use, despite the higher R2 value.\n"},{"metadata":{},"cell_type":"markdown","source":"The linear modeling suggest that the primary use have great impact on the meter reading modeling. But it comes with a low accuracy score.\n\n\n### 3.1 Linear Modeling only with few important features"},{"metadata":{"trusted":true},"cell_type":"code","source":"K =BTW_linearR[['meter','wind_speed', 'cloud_coverage',\n                'primary_use_Education','primary_use_Entertainment/public assembly', 'primary_use_Healthcare',\n       'primary_use_Manufacturing/industrial', 'primary_use_Office',\n       'primary_use_Other', 'primary_use_Parking','primary_use_Religious worship',\n       'primary_use_Retail','primary_use_Technology/science', 'primary_use_Utility', 'Month']]\n\n# Create target variable\ny = BTW_linearR['meter_reading']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lm = LinearRegression()\n\n# Fit model to training data\nlm.fit(K,y)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Train, test, split\nfrom sklearn.model_selection import train_test_split\nK_train, K_test, y_train, y_test = train_test_split(K,y, test_size = .20, random_state= 0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Accuracy %d', lm.score(K_test, y_test))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"y_pred = lm.predict(K_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('R^2 =',metrics.explained_variance_score(y_test,y_pred))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"lm.score(K,y)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"regressor.score(X_train,y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cdf1 = pd.DataFrame(data = lm.coef_, index = K.columns, columns = ['Coefficients'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cdf1 .Coefficients.nlargest(10).plot(kind='barh')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Optimizing this linear modeling did not change any big difference in improving score."},{"metadata":{},"cell_type":"markdown","source":"### Model 3.2 Decision Tree"},{"metadata":{"trusted":true},"cell_type":"code","source":"XD =BTW_linearR[['building_id', 'meter', 'air_temperature', 'wind_speed', 'precip_depth_1_hr', 'cloud_coverage',\n       'square_feet', 'primary_use_Education', 'primary_use_Entertainment/public assembly',\n       'primary_use_Food sales and service', 'primary_use_Healthcare',\n       'primary_use_Lodging/residential',\n       'primary_use_Manufacturing/industrial', 'primary_use_Office',\n       'primary_use_Other', 'primary_use_Parking',\n       'primary_use_Public services', 'primary_use_Religious worship',\n       'primary_use_Retail', 'primary_use_Services',\n       'primary_use_Technology/science', 'primary_use_Utility',\n       'primary_use_Warehouse/storage', 'Month', 'Day']]\n\n# Create target variable\nYD = BTW_linearR['meter_reading']\n\n# Train, test, split\nfrom sklearn.model_selection import train_test_split\nXD_train,XD_test, YD_train, YD_test = train_test_split(XD,YD, test_size = .20, random_state= 0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.tree import DecisionTreeRegressor\nregr_depth2 = DecisionTreeRegressor(max_depth=2)\nregr_depth5 = DecisionTreeRegressor(max_depth=5)\nregr_depth2.fit(XD_train, YD_train)\nregr_depth5.fit(XD_train, YD_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"y_1 = regr_depth2.predict(XD_test)\ny_2 = regr_depth5.predict(XD_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df=pd.DataFrame({'Actual':y_test, 'Predicted':y_1})\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Mean Absolute Error:', metrics.mean_absolute_error(y_test, y_1))\nprint('Mean Squared Error:', metrics.mean_squared_error(y_test, y_1))\nprint('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(y_test, y_1)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Calculate R Sqaured\nprint('R^2 =',metrics.explained_variance_score(y_test,y_1))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For depth 2 desicion tree modeling, R2 was obtained at 0.147"},{"metadata":{"trusted":true},"cell_type":"code","source":"df=pd.DataFrame({'Actual':y_test, 'Predicted':y_2})\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Mean Absolute Error:', metrics.mean_absolute_error(y_test, y_2))\nprint('Mean Squared Error:', metrics.mean_squared_error(y_test, y_2))\nprint('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(y_test, y_2)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('R^2 =',metrics.explained_variance_score(y_test,y_2))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For depth 5 desicion tree modeling, R2 was obtained at 0.723"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.plot(XD_test, y_1, color=\"blue\",label=\"max_depth=2\", linewidth=2)\nplt.plot(XD_test, y_2, color=\"green\", label=\"max_depth=5\", linewidth=2)\nplt.xlabel(\"data\")\nplt.ylabel(\"target\")\nplt.title(\"Decision Tree Regression\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Accuracy %d', regr_depth2.score(XD_train, YD_train))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Accuracy %d', regr_depth5.score(XD_train, YD_train))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"yd_pred = regr_depth5.predict(XD_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"yd_pred","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('XD 19',XD.columns[19], 'XD 6',XD.columns[6],'X0',XD.columns[0],'X1',XD.columns[1],'X23',XD.columns[23],'X19',XD.columns[19],'X2',XD.columns[2])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"From desicion tree, square feet under 333282.5, meter is electricity, month from January to June, air temperature under 4.488 result in 81 samples point showing 276450653 and 63 samples showing 68649752 for meter reading."},{"metadata":{"trusted":true},"cell_type":"code","source":"YD.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"XD.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"feat_importancesDT = pd.Series(regr_depth5.feature_importances_, index=XD.columns)\nfeat_importancesDT.nlargest(10).plot(kind='barh')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In general, applying decision tree modeling, accuracy score was increased from 0.0012-0.0014 in linear modleing to 0.147 or 0.723 from decision tree depending the depth of a tree. Decision tree modeling also suggest month, air temperature, meter type, and square feet plays a important role in its modeling."},{"metadata":{},"cell_type":"markdown","source":"### Model 3.2 Randomforest model"},{"metadata":{"trusted":true},"cell_type":"code","source":"import sklearn.ensemble as ske\nimport matplotlib.pyplot as plt\nRFR = ske.RandomForestRegressor()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"RFR.fit(XD,YD)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"RFR.score(XD,YD)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Randomforest gave the highest score for this modeling at 0.97 on accuracy score"},{"metadata":{"trusted":true},"cell_type":"code","source":"YR_pred = RFR.predict(XD_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"feat_importancesRFR = pd.Series(RFR.feature_importances_, index=XD.columns)\nfeat_importancesRFR.nlargest(10).plot(kind='barh')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#pip install pydot","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Random forest visualization"},{"metadata":{},"cell_type":"markdown","source":"## 4.0 Conclusion\nASHRAE energy dataset was explored using a different methods of Regression. For each method,\nan algorithm was developped to predict the meter readings, representing energy output, using characteristics of the building, use, location, and weather.\nTo begin with, a linear regression model was applied and achieved a low accuracy score of less than 0.01%, and R2 score of less than 0.01%. This suggested that a linear regression model was an extremely poor model for predicting energy usage.\n\nThen a decision tree was applied at different depths. At a depth of 2, the R2 value increased to almost 15%, but at a depth of 5, the R2 jumped to over 82%. At the mean time, the accuracy score was changed from 0.147 to 0.723 as the treee depth increased from 2 to 5. This suggested a strong fit for our “Decision Tree Regression” model. However, a closer look at the data showed that the results were skewed towards high square feet, which only represented a small percentage of overall data.\n\nOn the basis of results from decision tree, a Random Forest regression was applied, which gave an accuracy score of 96%, which was higher than previous models. This can be explained by the theory that random forest was serveal combinations of decision tree. In Random forest tree model, there were more trees and depth for modeling and splitting.\n\nUltimately a model was implemented that had a high accuracy rate of predicting the energy use using a Random Forest Regression model. The more important variables accounted for the seasonality (month), site location (air_temperature), and characteristics of the building itself (square feet and meter).\n\nOur conclusion is that a random forest model is a strong model to predict energy readings, and more exploration needs to be done on the variables that were considered more important in this model.\n\nIf anyone interested in visualization of decision tree, let me know as the code did not work here due to separate package needed to be installed.\n\n"},{"metadata":{},"cell_type":"markdown","source":"## 5.0 Future Work\n\n1.Developing a way to dealing with memory issue and visualization of random forest modeling. Batch processing may be a good way in this regard.\n\n2.Applying confusion metrices to further analysis relationship between accuracy score and precision.\n\n3.Different ways of replacing null value will be investigated and so dose the consequence."},{"metadata":{"scrolled":true,"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.3"}},"nbformat":4,"nbformat_minor":1}