{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# GREAT ENERGY PREDICTOR - ML REGRESSION\n","metadata":{}},{"cell_type":"markdown","source":"# Introduction to Competition\nThe data used in this project comes from a [kaggle comptition](https://www.kaggle.com/competitions/ashrae-energy-prediction) that attempts to predict hourly electricity meter readings for different buildings situated at different sites using different electricity meters.
\nData for 2016 has been provided as Training Dataset and Test Dataset comprises of data from 2017 and 2018 for 1000+ buldings.
**Now the organisation wants to make some improvements to the buildings so that buildings become energy efficient.**\n
\n> ![](\"https://i.imgur.com/6R0px08.png\")
","metadata":{}},{"cell_type":"markdown","source":"## Objective:\nThe **GOAL** of this competition is to create best models that can accurately **predict meter readings for coming years if there were no improvements/changes made.** So that best comparisons can be made with actual meter readings obtained after applying the improvements. That is why the test set is doublt the size of train set.\n
\n**This would help organisation** to show case their products and their performances to **attract large scale investors and financial institutions to invest in this area** to enable progress in building efficiencies.","metadata":{}},{"cell_type":"markdown","source":"## All Data\nThis data consists of various buildingmore than 1000 buildings having different types of meters installed. There are 3 files containing building features,types of meters, timestamps and meter readings and weather parameters for the site respectively.","metadata":{}},{"cell_type":"markdown","source":"The dataset includes three years of hourly meter readings from over one thousand buildings at several different sites around the world.
\n**ALL FILES:-**\n> **train.csv**\n> (21 million rows of data)\n- **building_id** - Foreign key for the building metadata.\n- **meter** - The meter id code. Read as {0: electricity, 1: chilledwater, 2: steam, 3: hotwater}. Not every building has all meter types.\n- **timestamp** - When the measurement was taken\n- **meter_reading** - The target variable. Energy consumption in kWh (or equivalent). This is real data with measurement error, which can impose a baseline level of modeling error.\n> **building_meta.csv**\n- **site_id** - Foreign key for the weather files.\n- **building_id** - Foreign key for training.csv\n- **primary_use** - Indicator of the primary category of activities for the building based on EnergyStar property type definitions\n- **square_feet** - Gross floor area of the building\n- **year_built** - Year building was opened\n- **floor_count** - Number of floors of the building\n> **weather_[train/test].csv**\nWeather data from a meteorological station as close as possible to the site.\n\n- **site_id**\n- air_temperature - Degrees Celsius\n- cloud_coverage - Portion of the sky covered in clouds, in oktas\n- dew_temperature - Degrees Celsius\n- precip_depth_1_hr - Millimeters\n- sea_level_pressure - Millibar/hectopascals\n- wind_direction - Compass direction (0-360)\n- wind_speed - Meters per second\n> **test.csv**(42 Million rows of data)\n
\n- **row_id** - Row id for your submission file\n- **building_id** - Building id code\n- **meter** - The meter id code\n- **timestamp** - Timestamps for the test data period\n
\nThe submission files use row numbers for ID codes in order to save space on the file uploads. test.csv has no feature data.","metadata":{}},{"cell_type":"markdown","source":"## Evaluation","metadata":{}},{"cell_type":"markdown","source":"Submissions are to done in CSV format. For each row id in the test set, Meter readings need to be predicted. The Loss function recommended by the competition host is Logarithmic Mean Squared Error (RMLSE) \n\nEvaluation is also done by finding RMLSE between the predicted values and the observed values. Here is the screenshot of Evaluation metric:\n\n
\n**More information about RMLSE:**\n- RMSLE metric considers the relative error between and the Predicted and the actual value and the scale of the error is not significant. On the other hand, RMSE value Increases in magnitude if the scale of error increases.\n- RMSLE incurs a larger penalty for the underestimation of the Actual variable than the Overestimation.\n\nHere is a [blog post](https://medium.com/analytics-vidhya/root-mean-square-log-error-rmse-vs-rmlse-935c6cc1802a) that compares RMLSE with RMSE very nicely and explains in depth about RMLSE\n","metadata":{}},{"cell_type":"markdown","source":"## Attributes of this Project\nCompetition makes it a very challenging and interesting Machine Learning Regression Project as it:\n1. Contains huge amont of data (More than 60 Million Rows).Handling which is itself a challenging task\n2. Requires lots of EDA \n3. Involves processing time series data\n4. Lots of Missing Data\n5. Requires feature engineering\n6. As it is a featured competition, It has quite complicated data, so handling this requires lots of analytical and rational sense. \n7. All Regression Models can be tested and applied for better results.\n
\nThe techniques necessary to work with this data are applicable to many other types of Machine Learning problems. ","metadata":{}},{"cell_type":"markdown","source":"# Importing Required Libraries\n","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport plotly.express as px\nimport math\nimport sklearn\nfrom sklearn.preprocessing import MinMaxScaler\nfrom sklearn.preprocessing import OneHotEncoder\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn.ensemble import RandomForestRegressor\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import accuracy_score\nfrom sklearn.ensemble import RandomForestClassifier\nimport missingno as msg\nfrom xgboost import XGBRegressor\nimport lightgbm as lgb\nimport gc\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"execution":{"iopub.status.busy":"2023-01-27T23:04:45.715780Z","iopub.execute_input":"2023-01-27T23:04:45.716187Z","iopub.status.idle":"2023-01-27T23:04:49.007769Z","shell.execute_reply.started":"2023-01-27T23:04:45.716159Z","shell.execute_reply":"2023-01-27T23:04:49.006450Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Exploratory Data Analysis\n
\nIt is always a good practise to explore all the data that you want to use to find predictions using ML models. So that you can select best models to train your data. Also you can clean the data by finding and removing erratic data and outliers and impute missing values with best strategy. Finding relations between columns can also help you alot figuring out different paramets you need to provide to your models.","metadata":{}},{"cell_type":"markdown","source":"## Getting a Highlevel information about data","metadata":{}},{"cell_type":"code","source":"data_dir=\"/kaggle/input/ashrae-energy-prediction\"\n! ls -lh {data_dir}","metadata":{"execution":{"iopub.status.busy":"2023-01-27T23:05:12.123425Z","iopub.execute_input":"2023-01-27T23:05:12.124402Z","iopub.status.idle":"2023-01-27T23:05:12.408331Z","shell.execute_reply.started":"2023-01-27T23:05:12.124350Z","shell.execute_reply":"2023-01-27T23:05:12.406806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!wc -l {data_dir}/train.csv","metadata":{"execution":{"iopub.status.busy":"2023-01-24T03:06:24.928862Z","iopub.execute_input":"2023-01-24T03:06:24.929288Z","iopub.status.idle":"2023-01-24T03:06:33.153121Z","shell.execute_reply.started":"2023-01-24T03:06:24.929253Z","shell.execute_reply":"2023-01-24T03:06:33.151801Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!wc -l {data_dir}/test.csv","metadata":{"execution":{"iopub.status.busy":"2023-01-24T03:13:38.877377Z","iopub.execute_input":"2023-01-24T03:13:38.878003Z","iopub.status.idle":"2023-01-24T03:13:55.132282Z","shell.execute_reply.started":"2023-01-24T03:13:38.877948Z","shell.execute_reply":"2023-01-24T03:13:55.130604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!head {data_dir}/train.csv","metadata":{"execution":{"iopub.status.busy":"2023-01-24T03:09:46.311537Z","iopub.execute_input":"2023-01-24T03:09:46.312082Z","iopub.status.idle":"2023-01-24T03:09:47.366383Z","shell.execute_reply.started":"2023-01-24T03:09:46.312032Z","shell.execute_reply":"2023-01-24T03:09:47.364865Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!head {data_dir}/test.csv","metadata":{"execution":{"iopub.status.busy":"2023-01-24T03:10:15.233118Z","iopub.execute_input":"2023-01-24T03:10:15.233565Z","iopub.status.idle":"2023-01-24T03:10:16.276745Z","shell.execute_reply.started":"2023-01-24T03:10:15.233524Z","shell.execute_reply":"2023-01-24T03:10:16.275280Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!head {data_dir}/building_metadata.csv","metadata":{"execution":{"iopub.status.busy":"2023-01-24T19:09:27.150028Z","iopub.execute_input":"2023-01-24T19:09:27.150538Z","iopub.status.idle":"2023-01-24T19:09:28.262518Z","shell.execute_reply.started":"2023-01-24T19:09:27.150496Z","shell.execute_reply":"2023-01-24T19:09:28.260854Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## EDA on Building Metadataset","metadata":{}},{"cell_type":"markdown","source":"Reading the file by giving suitable dtypes to reduce memory occupation without loss of any data","metadata":{}},{"cell_type":"code","source":"building_dtypes={'site_id':'int16','building_id':'int16','square_feet':'float32'}\nbuilding_df=pd.read_csv(f\"{data_dir}/building_metadata.csv\",dtype=building_dtypes)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T23:05:16.387159Z","iopub.execute_input":"2023-01-27T23:05:16.387873Z","iopub.status.idle":"2023-01-27T23:05:16.416099Z","shell.execute_reply.started":"2023-01-27T23:05:16.387839Z","shell.execute_reply":"2023-01-27T23:05:16.415273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"building_df.info()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T03:18:02.736692Z","iopub.execute_input":"2023-01-24T03:18:02.737173Z","iopub.status.idle":"2023-01-24T03:18:02.765679Z","shell.execute_reply.started":"2023-01-24T03:18:02.737124Z","shell.execute_reply":"2023-01-24T03:18:02.764767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"building_df.describe()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T03:18:56.713538Z","iopub.execute_input":"2023-01-24T03:18:56.714068Z","iopub.status.idle":"2023-01-24T03:18:56.752164Z","shell.execute_reply.started":"2023-01-24T03:18:56.714020Z","shell.execute_reply":"2023-01-24T03:18:56.751370Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Seems like we have lots of missing values for floor count and year built columns","metadata":{}},{"cell_type":"code","source":"building_df.isna().sum()*100/len(building_df)","metadata":{"execution":{"iopub.status.busy":"2023-01-24T20:28:37.576426Z","iopub.execute_input":"2023-01-24T20:28:37.576963Z","iopub.status.idle":"2023-01-24T20:28:37.591101Z","shell.execute_reply.started":"2023-01-24T20:28:37.576923Z","shell.execute_reply":"2023-01-24T20:28:37.589681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So let's check relations between floor count and year built with other columns to check if the values can be filled with a nice strategy. However , if we don't get any strong and reliable strategy to fill these columns we will have to remove them.","metadata":{}},{"cell_type":"code","source":"building_df.corr()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T03:19:27.354628Z","iopub.execute_input":"2023-01-24T03:19:27.355057Z","iopub.status.idle":"2023-01-24T03:19:27.377008Z","shell.execute_reply.started":"2023-01-24T03:19:27.355021Z","shell.execute_reply":"2023-01-24T03:19:27.376159Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There seems perfect linear relation between building_id and site_id. May be the building ids are sorted based on site ids. floor count seems to have some relation with building_id. Let's check it's relation with different columns as it can have some influence on meter reading (target column). Based on cognition Year the building was built does not seem to have musch effect on meter reading still we will check.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10,3))\nplt.plot(building_df.site_id,building_df.building_id);\nplt.title(\"Site_id vs Building_id\")\nplt.xlabel('Building_id')\nplt.ylabel('Site_id')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T04:08:19.631727Z","iopub.execute_input":"2023-01-24T04:08:19.632256Z","iopub.status.idle":"2023-01-24T04:08:19.831878Z","shell.execute_reply.started":"2023-01-24T04:08:19.632209Z","shell.execute_reply":"2023-01-24T04:08:19.830968Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"yes building are provided ids based on sites. Site 3 has highest number of buildings","metadata":{}},{"cell_type":"code","source":"#Checking how many unique buildings and sites we have\nprint('unique buidings: ',building_df.building_id.nunique(),\" Unique Sites : \",building_df.site_id.nunique())\n","metadata":{"execution":{"iopub.status.busy":"2023-01-24T04:09:55.583757Z","iopub.execute_input":"2023-01-24T04:09:55.584287Z","iopub.status.idle":"2023-01-24T04:09:55.593231Z","shell.execute_reply.started":"2023-01-24T04:09:55.584238Z","shell.execute_reply":"2023-01-24T04:09:55.591779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#checking Count of buildings based on their usage'\nfig=px.histogram(building_df,'primary_use',histfunc='count',title='Count of Buildings based on primary Usage')\nfig.update_yaxes(title_text=\"Count of Buildings\",);\nfig.update_xaxes(title_text=\"Primary Use\",);\nfig.show();","metadata":{"execution":{"iopub.status.busy":"2023-01-27T23:05:22.270795Z","iopub.execute_input":"2023-01-27T23:05:22.271486Z","iopub.status.idle":"2023-01-27T23:05:23.934870Z","shell.execute_reply.started":"2023-01-27T23:05:22.271447Z","shell.execute_reply":"2023-01-27T23:05:23.933332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plotting Count of Buildings based on Floor Count \nfig=px.histogram(building_df,'floor_count',title='Count of Buildings based on Floor Count')\nfig.update_layout(bargap=0.2)\nfig.update_yaxes(title_text=\"Count of Buildings\",);\n\nfig.update_xaxes(title_text=\"Number of Floors\",)\n","metadata":{"execution":{"iopub.status.busy":"2023-01-24T19:18:56.006847Z","iopub.execute_input":"2023-01-24T19:18:56.007533Z","iopub.status.idle":"2023-01-24T19:18:56.084416Z","shell.execute_reply.started":"2023-01-24T19:18:56.007482Z","shell.execute_reply":"2023-01-24T19:18:56.082985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(14,6))\nfig=px.box(building_df,x='primary_use',y='floor_count',title='Floor count vs Primary Use');\nfig.update_yaxes(title_text=\"Floor Count\",);\nfig.update_xaxes(title_text=\"Building Usage\");\nfig.show();","metadata":{"execution":{"iopub.status.busy":"2023-01-25T21:06:28.874336Z","iopub.execute_input":"2023-01-25T21:06:28.874874Z","iopub.status.idle":"2023-01-25T21:06:28.951666Z","shell.execute_reply.started":"2023-01-25T21:06:28.874827Z","shell.execute_reply":"2023-01-25T21:06:28.950226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There is huge variation of floor count for almost all usages. Ones it seemed to be tempting to fill floor count based on mean of floor counts we have buildings for same usage. But after this visualisation that seem to make sense.","metadata":{}},{"cell_type":"code","source":"fig=px.histogram(building_df,'year_built',color='primary_use',title='Count of new Buildings built for the year')\nfig.update_yaxes(title_text=\"Count of Buildings\",);\nfig.update_xaxes(title_text=\"Year Built\")","metadata":{"execution":{"iopub.status.busy":"2023-01-24T19:24:36.867395Z","iopub.execute_input":"2023-01-24T19:24:36.868090Z","iopub.status.idle":"2023-01-24T19:24:37.035311Z","shell.execute_reply.started":"2023-01-24T19:24:36.868034Z","shell.execute_reply":"2023-01-24T19:24:37.033813Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig=px.treemap((building_df.groupby(\"primary_use\").median()).reset_index(),\n path=[px.Constant(\"Building_Types\"),'primary_use'],\n values=f'floor_count',color=f'square_feet',\n color_continuous_scale='RdBu'\n )\nfig.update_layout(title=\"Comparing Primary use , Floor Count and Area (square feet)\") # adding title for graph\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T19:26:47.863660Z","iopub.execute_input":"2023-01-24T19:26:47.864183Z","iopub.status.idle":"2023-01-24T19:26:48.018391Z","shell.execute_reply.started":"2023-01-24T19:26:47.864144Z","shell.execute_reply":"2023-01-24T19:26:48.017172Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There is no much evidence that floor count and square feet have any linear relation. So dropping floor count and year built would be a good decision.","metadata":{}},{"cell_type":"markdown","source":"## Exploring Train CSV","metadata":{}},{"cell_type":"code","source":"# Setting training_dtypes based on very first section when we checked files with terminal commands\ntraining_dtypes={\"building_id\":\"int16\",\"meter\":\"int16\",\"meter_reading\":\"float32\"}\ntrain_df=pd.read_csv(\"/kaggle/input/ashrae-energy-prediction/train.csv\",dtype=training_dtypes)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T00:04:14.615191Z","iopub.execute_input":"2023-01-27T00:04:14.615586Z","iopub.status.idle":"2023-01-27T00:04:32.440643Z","shell.execute_reply.started":"2023-01-27T00:04:14.615556Z","shell.execute_reply":"2023-01-27T00:04:32.439725Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.info()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T20:25:22.071239Z","iopub.execute_input":"2023-01-24T20:25:22.071703Z","iopub.status.idle":"2023-01-24T20:25:22.087602Z","shell.execute_reply.started":"2023-01-24T20:25:22.071669Z","shell.execute_reply":"2023-01-24T20:25:22.086438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.describe()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T20:25:22.089410Z","iopub.execute_input":"2023-01-24T20:25:22.090438Z","iopub.status.idle":"2023-01-24T20:25:23.883673Z","shell.execute_reply.started":"2023-01-24T20:25:22.090393Z","shell.execute_reply":"2023-01-24T20:25:23.882283Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Total unique buildings in Train.csv: \",train_df.building_id.nunique())\nprint(\"Unique meters in Train.csv: \",train_df.meter.unique())","metadata":{"execution":{"iopub.status.busy":"2023-01-24T20:32:03.604318Z","iopub.execute_input":"2023-01-24T20:32:03.604792Z","iopub.status.idle":"2023-01-24T20:32:03.899569Z","shell.execute_reply.started":"2023-01-24T20:32:03.604734Z","shell.execute_reply":"2023-01-24T20:32:03.897989Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T20:27:30.943260Z","iopub.execute_input":"2023-01-24T20:27:30.943808Z","iopub.status.idle":"2023-01-24T20:27:31.878925Z","shell.execute_reply.started":"2023-01-24T20:27:30.943764Z","shell.execute_reply":"2023-01-24T20:27:31.877601Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Wow we donot have any missing value in this file.
\nNow, meter_reading is our target column. So we should check if we have any outliers for this meter.\n
before that we should also check timestamps","metadata":{}},{"cell_type":"code","source":"print(train_df.timestamp.min())\nprint(train_df.timestamp.max())\nprint(train_df.timestamp.nunique())","metadata":{"execution":{"iopub.status.busy":"2023-01-25T21:30:43.083847Z","iopub.execute_input":"2023-01-25T21:30:43.084877Z","iopub.status.idle":"2023-01-25T21:30:47.183499Z","shell.execute_reply.started":"2023-01-25T21:30:43.084833Z","shell.execute_reply":"2023-01-25T21:30:47.182150Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So we have data for all hours in train file from start to end of 2016.
\nLet's analyse meter columns","metadata":{}},{"cell_type":"code","source":"x1=train_df.groupby('meter')\ndisplay(x1.describe())\ndel x1\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T20:38:16.025219Z","iopub.execute_input":"2023-01-24T20:38:16.025704Z","iopub.status.idle":"2023-01-24T20:38:19.885500Z","shell.execute_reply.started":"2023-01-24T20:38:16.025669Z","shell.execute_reply":"2023-01-24T20:38:19.884138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So from above data we can sense that their are very huge outliers for every meter. It would be better to train different models for different meters. May be to save some time we can club meter 1 and meter 3 data ( which seem to have intermediate values). But there should be separate models for meters 1 and 2.\n\nAlso count of rows decreases as the meter number increases","metadata":{}},{"cell_type":"code","source":"x1=len(train_df[train_df.meter_reading==0])*100/len(train_df)\nprint(\"percent rows with reading 0 : \",x1)\ndel x1\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T20:41:06.817250Z","iopub.execute_input":"2023-01-24T20:41:06.817722Z","iopub.status.idle":"2023-01-24T20:41:07.034598Z","shell.execute_reply.started":"2023-01-24T20:41:06.817687Z","shell.execute_reply":"2023-01-24T20:41:07.032725Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It is mentioned in the competition that the meter readings 0 can be erratic though reason behind it is not well known. So, we will remove these values as well as they are just 9% of the total dataset.","metadata":{}},{"cell_type":"code","source":"train_df.drop(train_df.loc[train_df['meter_reading']==0].index, inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-01-24T21:05:53.065649Z","iopub.execute_input":"2023-01-24T21:05:53.066817Z","iopub.status.idle":"2023-01-24T21:05:55.127929Z","shell.execute_reply.started":"2023-01-24T21:05:53.066723Z","shell.execute_reply":"2023-01-24T21:05:55.126417Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we should consider removing outliers for all meters as well. But before that let's visualize a sample of this training file.","metadata":{}},{"cell_type":"code","source":"for _ in range(4):\n print(f\"For meter {_} unique building count\",train_df[train_df.meter==_].building_id.nunique())#*100/t_df.building_id.nunique()","metadata":{"execution":{"iopub.status.busy":"2023-01-25T21:37:26.584996Z","iopub.execute_input":"2023-01-25T21:37:26.585473Z","iopub.status.idle":"2023-01-25T21:37:27.777660Z","shell.execute_reply.started":"2023-01-25T21:37:26.585435Z","shell.execute_reply":"2023-01-25T21:37:27.776308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Removing top five percent of values\ndef see_v_counts(meterno,r1,r2,r3,high=True):\n m0_df=train_df[train_df.meter==meterno]\n if high:\n for _ in range(r1,r2,r3):\n y=m0_df[m0_df.meter_reading>_]\n x=len(y)*100/len(m0_df)\n print(f\"Above {_} %values\",x,\" unique buildings - \",y.building_id.nunique(),'rows: ',len(y))\n else:\n for _ in range(r1,r2,r3):\n y=m0_df[m0_df.meter_reading<_]\n x=len(y)*100/len(m0_df)\n print(f\"Below {_} %values\",x,\" unique buildings - \",y.building_id.nunique(),'rows: ',len(y))\n if x >5:\n break\n if y.building_id.nunique()<7:\n print('building_ids - ',y.building_id.unique())\ndef spot_outliers(meterno,value,high=True):\n df_0=train_df[train_df.meter==meterno]\n \n if high:\n y=df_0[df_0.meter_reading>value]\n print(f\"Total buildings for meter {meterno} - \",df_0.building_id.nunique(),\n f\"Unique buildings above {value} - \",y.building_id.nunique(), \n f\"Total rows above {value} - \",len(y))\n n=df_0[df_0['building_id'].isin(list(y.building_id.unique()))]\n x=n[n['meter_reading']>value*.7]\n print(f\"Total rows for these building_ids above {value*.7} - \",len(x))\n else: \n y=df_0[df_0.meter_reading19000].building_id.value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T21:31:11.980976Z","iopub.execute_input":"2023-01-24T21:31:11.981458Z","iopub.status.idle":"2023-01-24T21:31:12.364218Z","shell.execute_reply.started":"2023-01-24T21:31:11.981419Z","shell.execute_reply":"2023-01-24T21:31:12.363059Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df1[df1.building_id.isin( [778,1088,1284])].groupby('building_id').meter_reading.describe()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T21:29:55.125155Z","iopub.execute_input":"2023-01-24T21:29:55.125624Z","iopub.status.idle":"2023-01-24T21:29:55.166128Z","shell.execute_reply.started":"2023-01-24T21:29:55.125588Z","shell.execute_reply":"2023-01-24T21:29:55.165156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Remove values above 32000","metadata":{}},{"cell_type":"code","source":"see_v_counts(1,0,10,1,False)","metadata":{"execution":{"iopub.status.busy":"2023-01-24T21:33:53.644332Z","iopub.execute_input":"2023-01-24T21:33:53.645027Z","iopub.status.idle":"2023-01-24T21:33:53.962340Z","shell.execute_reply.started":"2023-01-24T21:33:53.644986Z","shell.execute_reply":"2023-01-24T21:33:53.961076Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"spot_outliers(1,.0005,False)","metadata":{"execution":{"iopub.status.busy":"2023-01-24T21:37:51.685788Z","iopub.execute_input":"2023-01-24T21:37:51.688126Z","iopub.status.idle":"2023-01-24T21:37:51.979439Z","shell.execute_reply.started":"2023-01-24T21:37:51.688066Z","shell.execute_reply":"2023-01-24T21:37:51.978227Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df1[df1.meter_reading<0.0003].building_id.value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T21:42:32.651341Z","iopub.execute_input":"2023-01-24T21:42:32.652241Z","iopub.status.idle":"2023-01-24T21:42:32.670096Z","shell.execute_reply.started":"2023-01-24T21:42:32.652190Z","shell.execute_reply":"2023-01-24T21:42:32.668241Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del df1\ndel t_df\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:02:33.577634Z","iopub.execute_input":"2023-01-24T22:02:33.578721Z","iopub.status.idle":"2023-01-24T22:02:34.391698Z","shell.execute_reply.started":"2023-01-24T22:02:33.578676Z","shell.execute_reply":"2023-01-24T22:02:34.390642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Similarly very low lying and very high lying values were found for all meters.
\nFor meter 1 - remove values below 0.0003 and above 33000. Exceptional building ids - 1088,778,1284,1013(low)
\nFor meter 2 - remove values below 0.02 and above 37000. Exceptional building ids - 799,740(low),304(low)
\nFor meter 3 - remove values below 0.05 and above 21000. Exceptional building ids - 1099,1168,1197,1159,1148
\nFor meter 0 - remove values below 0.005 and above 9000. Exceptional building ids - 1021,1258,1331,794,1241(low),1253(low)
","metadata":{}},{"cell_type":"markdown","source":"The score matrix we have been provided in the competition is logarithmic mse. So lower prdictions will be punished lesser than higher predictions. That is why we should be very conservative while removing any low lying values.","metadata":{}},{"cell_type":"markdown","source":"## Analysing Building and Training files merged together","metadata":{}},{"cell_type":"code","source":"bt_df=pd.merge(train_df,building_df,on='building_id',how='left')","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:00:40.999916Z","iopub.execute_input":"2023-01-24T22:00:41.000806Z","iopub.status.idle":"2023-01-24T22:00:46.236672Z","shell.execute_reply.started":"2023-01-24T22:00:41.000731Z","shell.execute_reply":"2023-01-24T22:00:46.235464Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"bts_df=bt_df.sample(frac=0.002)","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:03:36.754666Z","iopub.execute_input":"2023-01-24T22:03:36.755386Z","iopub.status.idle":"2023-01-24T22:03:39.995459Z","shell.execute_reply.started":"2023-01-24T22:03:36.755349Z","shell.execute_reply":"2023-01-24T22:03:39.994365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(14,4))\nfig = sns.barplot(\n x=\"meter_reading\",\n y=\"site_id\",\n data=bts_df[bts_df.meter_reading<500000],\n orient=\"h\"\n)\n\nplt.title(\"meter_reading vs site_id\");\n","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:14:53.207188Z","iopub.execute_input":"2023-01-24T22:14:53.207669Z","iopub.status.idle":"2023-01-24T22:14:54.460139Z","shell.execute_reply.started":"2023-01-24T22:14:53.207631Z","shell.execute_reply":"2023-01-24T22:14:54.458691Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Surely we should use site id as categorical column","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(14,5))\nsns.lineplot(x=bts_df['site_id'],y=bts_df['square_feet']);","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:16:43.653286Z","iopub.execute_input":"2023-01-24T22:16:43.654141Z","iopub.status.idle":"2023-01-24T22:16:44.701140Z","shell.execute_reply.started":"2023-01-24T22:16:43.654086Z","shell.execute_reply":"2023-01-24T22:16:44.699789Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There Seems strong relation between square_feet and meter reading","metadata":{}},{"cell_type":"code","source":"bts_df.corr()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:21:20.731272Z","iopub.execute_input":"2023-01-24T22:21:20.731783Z","iopub.status.idle":"2023-01-24T22:21:20.760556Z","shell.execute_reply.started":"2023-01-24T22:21:20.731729Z","shell.execute_reply":"2023-01-24T22:21:20.759106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There's least strong correlation between the square feet and meter reading, may be above visualization appeared that way because there is huge variation of values.","metadata":{}},{"cell_type":"code","source":"px.histogram(bts_df[bts_df.meter_reading<500000],x='floor_count',y='meter_reading',histfunc='avg')","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:23:01.901609Z","iopub.execute_input":"2023-01-24T22:23:01.902204Z","iopub.status.idle":"2023-01-24T22:23:01.987343Z","shell.execute_reply.started":"2023-01-24T22:23:01.902162Z","shell.execute_reply":"2023-01-24T22:23:01.985904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Appears good relation between floor count and meter reading. We can imput floor count based on meter readings. But there's huge variation in meter_readings and we have only 25% of values for floor count. So idea to impute floor count was rejected. ","metadata":{}},{"cell_type":"markdown","source":"## Exploring Weather files","metadata":{}},{"cell_type":"code","source":"!head {data_dir}/weather_train.csv","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:29:03.638108Z","iopub.execute_input":"2023-01-24T22:29:03.638650Z","iopub.status.idle":"2023-01-24T22:29:04.807884Z","shell.execute_reply.started":"2023-01-24T22:29:03.638614Z","shell.execute_reply":"2023-01-24T22:29:04.805944Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nweather_train_df = pd.read_csv('/kaggle/input/ashrae-energy-prediction/weather_train.csv',dtype={'site_id':'int16'})\nweather_train_df.info()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T23:09:33.995725Z","iopub.execute_input":"2023-01-24T23:09:33.996190Z","iopub.status.idle":"2023-01-24T23:09:34.293183Z","shell.execute_reply.started":"2023-01-24T23:09:33.996149Z","shell.execute_reply":"2023-01-24T23:09:34.291900Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"weather_train_df.describe()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:30:03.157396Z","iopub.execute_input":"2023-01-24T22:30:03.157935Z","iopub.status.idle":"2023-01-24T22:30:03.252453Z","shell.execute_reply.started":"2023-01-24T22:30:03.157875Z","shell.execute_reply":"2023-01-24T22:30:03.251155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"For Weather \\nMax time Reading - \",weather_train_df.timestamp.max(),\n \"\\nMin time Reading - \",weather_train_df.timestamp.min())\nprint(\"For Train \\nMax time Reading - \",train_df.timestamp.max(),\n \"\\nMin time Reading - \",train_df.timestamp.min())","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:31:38.745892Z","iopub.execute_input":"2023-01-24T22:31:38.746819Z","iopub.status.idle":"2023-01-24T22:31:41.033780Z","shell.execute_reply.started":"2023-01-24T22:31:38.746739Z","shell.execute_reply":"2023-01-24T22:31:41.032804Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(train_df.timestamp.nunique())\nweather_train_df.timestamp.nunique()","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:32:28.515254Z","iopub.execute_input":"2023-01-24T22:32:28.516683Z","iopub.status.idle":"2023-01-24T22:32:30.091446Z","shell.execute_reply.started":"2023-01-24T22:32:28.516625Z","shell.execute_reply":"2023-01-24T22:32:30.090152Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Seems like we have all timestamps there in weather files as well","metadata":{}},{"cell_type":"code","source":"weather_train_df.isna().sum()/len(weather_train_df)","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:32:42.904015Z","iopub.execute_input":"2023-01-24T22:32:42.904470Z","iopub.status.idle":"2023-01-24T22:32:42.927316Z","shell.execute_reply.started":"2023-01-24T22:32:42.904436Z","shell.execute_reply":"2023-01-24T22:32:42.925876Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"weather_train_df.sort_values(by=['site_id','timestamp'],inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:37:33.849532Z","iopub.execute_input":"2023-01-24T22:37:33.850582Z","iopub.status.idle":"2023-01-24T22:37:33.909661Z","shell.execute_reply.started":"2023-01-24T22:37:33.850533Z","shell.execute_reply":"2023-01-24T22:37:33.908433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#import missingno as msg\nmsg.matrix(weather_train_df);","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:37:35.967829Z","iopub.execute_input":"2023-01-24T22:37:35.969016Z","iopub.status.idle":"2023-01-24T22:37:36.939481Z","shell.execute_reply.started":"2023-01-24T22:37:35.968963Z","shell.execute_reply":"2023-01-24T22:37:36.938508Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Cloud coverage, precipitation, sea level pressure seem to have lot of effect on energy consumption of that day. But there are huge chuncks of missing values and impute them could be challenging. Interpolating will only work well if the missing value gap is low. Now we would have to go by generalized imputing like with mean of day or the hour for whole month if we donot have any value for some days. ","metadata":{}},{"cell_type":"code","source":"print(\"Number of Hours for each site id in weather train file: \",len(weather_train_df)/weather_train_df.site_id.nunique())","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:45:21.376243Z","iopub.execute_input":"2023-01-24T22:45:21.376698Z","iopub.status.idle":"2023-01-24T22:45:21.384277Z","shell.execute_reply.started":"2023-01-24T22:45:21.376661Z","shell.execute_reply":"2023-01-24T22:45:21.382844Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Here's the catch**
\nThere are missing timestamps for sites. The avegare was expected to be 8784 but it's lesser. Let's explore this more","metadata":{}},{"cell_type":"code","source":"should_be=8784\nfor i in range(16):\n x=weather_train_df[weather_train_df.site_id==i].timestamp.nunique()\n print(f'For site {i} unique timestamps are: ',x,\" number of missing hours :\",should_be-x)","metadata":{"execution":{"iopub.status.busy":"2023-01-24T22:46:33.790423Z","iopub.execute_input":"2023-01-24T22:46:33.791024Z","iopub.status.idle":"2023-01-24T22:46:33.864825Z","shell.execute_reply.started":"2023-01-24T22:46:33.790978Z","shell.execute_reply":"2023-01-24T22:46:33.863510Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's find missing hours for all sites and add them into our dataset so that when we mearge it with training data to make our final dataframe we donot have any missing values there.","metadata":{}},{"cell_type":"code","source":"import datetime\ntime_format = \"%Y-%m-%d %H:%M:%S\"\nfirst_date = datetime.datetime.strptime(weather_train_df['timestamp'].min(),time_format)\ntotal_hours=24*366\nall_hours=[]\nfor _ in range(total_hours):\n all_hours.append((first_date + datetime.timedelta(hours=_)).strftime(time_format))\nlen(all_hours)\n","metadata":{"execution":{"iopub.status.busy":"2023-01-24T23:09:53.233686Z","iopub.execute_input":"2023-01-24T23:09:53.234049Z","iopub.status.idle":"2023-01-24T23:09:53.283861Z","shell.execute_reply.started":"2023-01-24T23:09:53.234019Z","shell.execute_reply":"2023-01-24T23:09:53.283137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for site_id in range(16):\n site_hours = weather_train_df[weather_train_df['site_id'] == site_id]['timestamp']\n missing_hours_df= pd.DataFrame(np.setdiff1d(all_hours,site_hours),columns=['timestamp'])\n missing_hours_df['site_id'] = site_id\n weather_train_df = pd.concat([weather_train_df,missing_hours_df])\nweather_train_df = weather_train_df.reset_index(drop=True) \ndel site_hours,missing_hours_df","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(16):\n x=weather_train_df[weather_train_df.site_id==i].timestamp.nunique()\n print(f'For site {i} unique timestamps are ',x)","metadata":{"execution":{"iopub.status.busy":"2023-01-24T23:12:00.018279Z","iopub.execute_input":"2023-01-24T23:12:00.018842Z","iopub.status.idle":"2023-01-24T23:12:00.081996Z","shell.execute_reply.started":"2023-01-24T23:12:00.018799Z","shell.execute_reply":"2023-01-24T23:12:00.080426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now every thing is fine. Let start filling missing values. Filling by mean of day seems good strategy. But we have a number of missing values so it may be possible we donot have any value for some days. But let's give it a try.","metadata":{}},{"cell_type":"code","source":"weather_train_df['hour']=weather_train_df.timestamp.dt.hour\nweather_train_df['day']=weather_train_df.timestamp.dt.day\nweather_train_df['month']=weather_train_df.timestamp.dt.month\nweather_train_df.set_index(['site_id','month','day'],inplace=True\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"weather_features=['cloud_coverage','air_temperature','dew_temperature','sea_level_pressure','precip_depth_1_hr']\nfor i in (weather_features):\n filler = weather_train_df.groupby(['site_id','month','day'])[f'{i}'].mean()\n filler = pd.DataFrame(filler.fillna(method='ffill'),columns=[f\"{i}\"])\n weather_train_df.update(filler,overwrite=False)\nweather_train_df=weather_train_df[weather_features] \nweather_train_df.reset_index(inplace=True)\nweather_train_df.isna().sum()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Fortunately there is no missing value now. So with this we have figured out how to clean and prepare all our training data ","metadata":{}},{"cell_type":"markdown","source":"# Preparing Data for Training\n**DATA CLEANING , IMPUTING MISSING VALUES, FEATURE ENGINEERING, DATA PREPROCESSING**
\nBased on above EDA and proposed strategies to clean, fill, add features, encode and scale the values. Let's define some basic functions to get our data prepared, so that writing code again and again can be avoided.","metadata":{}},{"cell_type":"code","source":"w_df=pd.read_csv('/kaggle/input/ashrae-energy-prediction/weather_train.csv')","metadata":{"execution":{"iopub.status.busy":"2023-01-27T00:28:45.049660Z","iopub.execute_input":"2023-01-27T00:28:45.050086Z","iopub.status.idle":"2023-01-27T00:28:45.298386Z","shell.execute_reply.started":"2023-01-27T00:28:45.050054Z","shell.execute_reply":"2023-01-27T00:28:45.297372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import datetime\nformat_ = \"%Y-%m-%d %H:%M:%S\"\nfirst_date = datetime.datetime.strptime(w_df['timestamp'].min(),format_) \nall_hours=[]\nfor _ in range(366*24):\n all_hours.append((first_date + datetime.timedelta(hours=_)).strftime(format_))","metadata":{"execution":{"iopub.status.busy":"2023-01-27T00:30:20.025210Z","iopub.execute_input":"2023-01-27T00:30:20.026001Z","iopub.status.idle":"2023-01-27T00:30:20.081483Z","shell.execute_reply.started":"2023-01-27T00:30:20.025965Z","shell.execute_reply":"2023-01-27T00:30:20.080515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def prepare_weather(w_df):\n \n # Finding missing timestamps and adding them to weather data\n \n all_hours=list(w_df.timestamp.unique()) # getting unique timestamps\n for site_id in range(16):\n # getting hours/timestamps for the site\n site_hours = weather_train_df[weather_train_df['site_id'] == site_id]['timestamp']\n # Finding missing timestamps using setdiff1d\n missing_hours_df= pd.DataFrame(np.setdiff1d(all_hours,site_hours),columns=['timestamp'])\n missing_hours_df['site_id'] = site_id\n # concating missing timestaamps with weather data\n w_df = pd.concat([w_df,missing_hours_df])\n w_df = w_df.reset_index(drop=True) \n # deleting unrequired features to reduce memory\n del site_hours,missing_hours_df \n \n # converting timestamp to datetime and adding features so that we can impute missing values\n w_df['timestamp1']=pd.to_datetime(weather_train_df.timestamp)\n w_df['hour']=w_df.timestamp1.dt.hour\n w_df['day']=w_df.timestamp1.dt.day\n w_df['month']=w_df.timestamp1.dt.month\n # changing index so that grouping operation can be don efficiently\n w_df.set_index(['site_id','month','day'],inplace=True)\n # Listing all weather features we want to give to our training data\n w_features=['cloud_coverage','air_temperature','dew_temperature','sea_level_pressure','precip_depth_1_hr']\n # Filling missing values based on mean for that day \n for i in (w_features):\n filler = w_df.groupby(['site_id','month','day'])[f'{i}'].mean()\n filler = pd.DataFrame(filler.fillna(method='ffill'),columns=[f\"{i}\"])\n w_df.update(filler,overwrite=False)\n w_features+=['site_id','timestamp'] \n w_df=w_df[w_features] # keeping only useful features \n w_df.reset_index(inplace=True) #resetting index again\n return w_df","metadata":{"execution":{"iopub.status.busy":"2023-01-27T07:34:03.148699Z","iopub.execute_input":"2023-01-27T07:34:03.149118Z","iopub.status.idle":"2023-01-27T07:34:03.164318Z","shell.execute_reply.started":"2023-01-27T07:34:03.149087Z","shell.execute_reply":"2023-01-27T07:34:03.162538Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Function to prepare Building Dataframe. As discusssed in EDA section, we will be removing Floor count, year built columns and scale the values for square_feet columns as there is huge variation. However, we will still be scaling numerical columns while preparing main dataset.","metadata":{}},{"cell_type":"code","source":"def prepare_building(b_df):\n b_df.drop(['floor_count','year_built'],axis=1,inplace=True)\n b_df['square_feet'] = np.log1p(b_df['square_feet'])\n return b_df","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def prepare_main_df(df,w_df,b_df,train=True):\n if train:\n df = df.query('meter_reading!=0')\n df['meter_reading']=np.log1p(df['meter_reading'])\n \n df['timestamp']=pd.to_datetime(df['timestamp'],format=\"%Y-%m-%d %H:%M:%S\")\n df[\"hour\"] = df[\"timestamp\"].dt.hour\n df[\"month\"] = df[\"timestamp\"].dt.month\n df['weekday']=df['timestamp'].dt.dayofweek\n \n return df\n ","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_sub_dfs(train_df,train=True):\n train0_df=train_df[train_df['meter']==0].copy()\n train0_df_1=train0_df.query(\"building_id == [693,112,799]\").copy()\n train0_df=train0_df.query(\"building_id != [799,740,112,693,304]\")\n if train:\n train0_df=train0_df.query(\"meter_reading < 9000 & meter_reading > 0.005\")\n \n train1_df=train_df[train_df['meter']==1].copy()\n train1_df_1=train1_df.query(\"building_id == [778,1088,1013,1284]\")\n train1_df=train1_df.query(\"building_id != [778,1088,1013,1284]\")\n if train:\n train1_df=train1_df.query(\"meter_reading < 19000 & meter_reading > 0.001\")\n \n train2_df=train_df[train_df['meter']==2].copy()\n train2_df_1=train2_df.query(\"building_id == [1168,1197,1159,1148,1099]\").copy()\n train2_df=train2_df.query(\"building_id != [1168,1197,1159,1148,1099]\")\n if train:\n train2_df_1.drop(train2_df_1.loc[train2_df_1['meter_reading']<100].index, inplace=True)\n train2_df=train2_df.query(\"meter_reading < 37000 & meter_reading > 0.04\")\n train3_df=train_df[train_df['meter']==3].copy()\n train3_df_1=train3_df.query(\"building_id == [1241,1253,1258,1331,794,1021]\")\n train3_df=train3_df.query(\"building_id != [1241,1253,1258,1331,794,1021]\")\n if train:\n train3_df=train3_df.query(\"meter_reading < 9000 & meter_reading > 0.05\")\n return train0_df,train0_df_1,train1_df,train1_df_1,train2_df,train2_df_1,train3_df,train3_df_1\n ","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"MinMax scaler and One hot encoding will be used to scale and encode numeric and categorical features","metadata":{}},{"cell_type":"code","source":"scaler = MinMaxScaler()\nencoder = OneHotEncoder(sparse=False, handle_unknown='ignore')","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:39:34.399476Z","iopub.execute_input":"2023-01-16T12:39:34.400647Z","iopub.status.idle":"2023-01-16T12:39:34.405891Z","shell.execute_reply.started":"2023-01-16T12:39:34.400620Z","shell.execute_reply":"2023-01-16T12:39:34.404577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def rmse(predictions, targets):\n return np.sqrt(((predictions - targets) ** 2).mean())","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:39:34.407459Z","iopub.execute_input":"2023-01-16T12:39:34.407832Z","iopub.status.idle":"2023-01-16T12:39:34.418725Z","shell.execute_reply.started":"2023-01-16T12:39:34.407800Z","shell.execute_reply":"2023-01-16T12:39:34.415865Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Defining a function to merge required datafames and get a sample to train the model. This is being done beacuse we have huge data size training the whole data will take lots and lots of time. So we will just be training our models by giving it a sample of whole dataframe.","metadata":{}},{"cell_type":"code","source":"def dataset_sample_meter(training_df,sample_fraction,num_cols,cat_cols,scaler,encoder,test=False):\n # merging with building_df\n # merging with weather data\n bt_df=pd.merge(training_df,building_df,on='building_id',how='left')\n if test:\n wbt_df=pd.merge(bt_df,weather_test_df,how='left',on=['site_id','timestamp'])\n wbt_df.sort_values(by=['site_id','timestamp'],inplace=True)\n return wbt_df\n\n wbt_df=pd.merge(bt_df,weather_train_df,on=['site_id','timestamp'],how='left')\n wbt_df.sort_values(by=['site_id','timestamp'],inplace=True)\n # fitting scaler and encoder\n scaler.fit(wbt_df[num_cols])\n encoder.fit(wbt_df[cat_cols])\n # getting a sample\n s_df=wbt_df.sample(frac=sample_fraction,random_state=5)\n return wbt_df,s_df","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:39:38.909451Z","iopub.execute_input":"2023-01-16T12:39:38.909942Z","iopub.status.idle":"2023-01-16T12:39:38.922378Z","shell.execute_reply.started":"2023-01-16T12:39:38.909905Z","shell.execute_reply":"2023-01-16T12:39:38.920454Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we need to preprocess the data by scaling, encoding and spliting dataframe into training and validation sets and return the inputs and target values that we need to gve to our models.","metadata":{}},{"cell_type":"code","source":"def preprocess_data(df0,num_cols,cat_cols,target_col,scaler,encoder):\n df0[num_cols]=scaler.transform(df0[num_cols])\n encoded_cols = list(encoder.get_feature_names_out(cat_cols))\n df0[encoded_cols] = encoder.transform(df0[cat_cols])\n df0=df0[num_cols+encoded_cols+[target_col]]\n t_df0, v_df0 = train_test_split(df0, test_size=0.3, random_state=42)\n t_targets=t_df0[target_col]\n v_targets=v_df0[target_col]\n t_inputs=t_df0[num_cols+encoded_cols]\n v_inputs=v_df0[num_cols+encoded_cols]\n return t_inputs,v_inputs,t_targets,v_targets","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:39:41.184801Z","iopub.execute_input":"2023-01-16T12:39:41.185304Z","iopub.status.idle":"2023-01-16T12:39:41.197237Z","shell.execute_reply.started":"2023-01-16T12:39:41.185265Z","shell.execute_reply":"2023-01-16T12:39:41.195023Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Defining another function for test data\ndef preprocess_test_data(df0,num_cols,cat_cols,scaler,encoder):\n df0[num_cols]=scaler.transform(df0[num_cols])\n encoded_cols = list(encoder.get_feature_names_out(cat_cols))\n df0[encoded_cols] = encoder.transform(df0[cat_cols])\n test_inputs=df0[num_cols+encoded_cols]\n return test_inputs","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:39:42.285285Z","iopub.execute_input":"2023-01-16T12:39:42.286578Z","iopub.status.idle":"2023-01-16T12:39:42.294203Z","shell.execute_reply.started":"2023-01-16T12:39:42.286497Z","shell.execute_reply":"2023-01-16T12:39:42.293028Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training Different Models","metadata":{}},{"cell_type":"code","source":"%%time\ntraining_dtypes={\"building_id\":\"int16\",\"meter\":\"int16\",\"meter_reading\":\"float32\"}\ntrain_df=pd.read_csv(\"/kaggle/input/ashrae-energy-prediction/train.csv\",dtype=training_dtypes)\nbuilding_dtypes={'site_id':'int16','building_id':'int16','square_feet':'float32'}\nbuilding_df=pd.read_csv(\"/kaggle/input/ashrae-energy-prediction/building_metadata.csv\",dtype=building_dtypes)\nweather_train_df = pd.read_csv('/kaggle/input/ashrae-energy-prediction/weather_train.csv',dtype={'site_id':'int16'})\nweather_test_df=pd.read_csv('/kaggle/input/ashrae-energy-prediction/weather_test.csv',dtype={'site_id':'int16'})\ntest_df=pd.read_csv(\"/kaggle/input/ashrae-energy-prediction/test.csv\",dtype=training_dtypes)","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:38:17.673281Z","iopub.execute_input":"2023-01-16T12:38:17.674032Z","iopub.status.idle":"2023-01-16T12:39:05.051276Z","shell.execute_reply.started":"2023-01-16T12:38:17.673998Z","shell.execute_reply":"2023-01-16T12:39:05.049502Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df=prepare_main_df(train_df)\ntest_df=prepare_main_df(test_df)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train0_df,train0_df_1,train1_df,train1_df_1,train2_df,train2_df_1,train3_df,train3_df_1=get_sub_dfs(train_df)\ntest0_df,test0_df_1,test1_df,test1_df_1,test2_df,test2_df_1,test3_df,test3_df_1=get_sub_dfs(test_df,train=False)\ndel train_df,test_df\ngc.collect()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x=(len(test0_df)/4)\nx=math.floor(x)\ntest01_df=test0_df[x:x*2].copy()\ntest02_df=test0_df[x*2:x*3].copy()\ntest03_df=test0_df[x*3:].copy()\ntest0_df=test0_df[:x]\ndel x","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:39:22.437107Z","iopub.execute_input":"2023-01-16T12:39:22.438061Z","iopub.status.idle":"2023-01-16T12:39:24.901958Z","shell.execute_reply.started":"2023-01-16T12:39:22.438016Z","shell.execute_reply":"2023-01-16T12:39:24.899296Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Defining a function to check if tune hyperparameters of our models","metadata":{}},{"cell_type":"code","source":"def train_model(Model,params,param_name,r,t_inputs,v_inputs,t_targets,v_targets):\n param_values,train_losses,val_losses=[],[],[]\n for _ in r:\n if isinstance(_,float):\n _=False\n break\n # As range function does not work with decimal values , so adding this piece of code.\n if _==False:\n _=r[0]\n while _ < r[1]:\n params[param_name]=_\n model = Model(n_jobs=-1, random_state=42,**params)\n model.fit(t_inputs, t_targets)\n t_preds=model.predict(t_inputs)\n t_loss=rmse(t_preds,t_targets)\n v_preds=model.predict(v_inputs)\n v_loss=rmse(v_preds,v_targets)\n print(f'For {_}: ',t_loss,v_loss)\n train_losses.append(t_loss)\n val_losses.append(v_loss)\n param_values.append(_)\n _+=r[2]\n else:\n for _ in range(r[0],r[1],r[2]):\n params[param_name]=_\n model = Model(n_jobs=-1, random_state=42,**params)\n model.fit(t_inputs, t_targets)\n t_preds=model.predict(t_inputs)\n t_loss=rmse(t_preds,t_targets)\n v_preds=model.predict(v_inputs)\n v_loss=rmse(v_preds,v_targets)\n print(f'For {_}: ',t_loss,v_loss)\n train_losses.append(t_loss)\n val_losses.append(v_loss)\n param_values.append(_)\n plt.figure(figsize=(10,6))\n plt.title('Overfitting curve: ' + param_name)\n plt.plot(param_values, train_losses, 'b-o')\n plt.plot(param_values, val_losses, 'r-o')\n plt.xlabel(param_name)\n plt.ylabel('RMSE')\n plt.legend(['Training', 'Validation'])\n return param_values,train_losses, val_losses","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:39:43.940337Z","iopub.execute_input":"2023-01-16T12:39:43.941080Z","iopub.status.idle":"2023-01-16T12:39:43.953966Z","shell.execute_reply.started":"2023-01-16T12:39:43.941038Z","shell.execute_reply":"2023-01-16T12:39:43.951857Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_cols=['square_feet', 'air_temperature','dew_temperature',\n 'cloud_coverage', 'precip_depth_1_hr',\n 'sea_level_pressure', 'wind_speed']\ncat_cols=['primary_use','month','weekday','hour','building_id']\ntarget_col='meter_reading'","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Random Forest\n\nRandom forests take a long time to train. So we will only be training on sample of around 200000 - 300000 rows.
\nAlso we will only be playing with a few of the most common hyperparameters of Random forest to check for overfitting and improving model performance.
\n> **Common Hyperparameters for Random Forest Regressor**\n> - tree depth\n> - number of variables sampled at each split.\n> - no.of trees\n> - sample size\n> - no.of features for each tree\n> - no.of leaf nodes","metadata":{}},{"cell_type":"markdown","source":"### Training For Meter 1","metadata":{}},{"cell_type":"code","source":"2e5/len(train1_df)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s1_df=dataset_sample_meter(train1_df,0.06)\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\ndel wbt_df\ndel s1_df","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params={'n_estimators':75,'max_depth':0}\nlosrf_md_m1=train_model(RandomForestRegressor,params,'max_depth',[4,41,4],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T09:36:16.940077Z","iopub.execute_input":"2023-01-10T09:36:16.940472Z","iopub.status.idle":"2023-01-10T09:40:47.021354Z","shell.execute_reply.started":"2023-01-10T09:36:16.940439Z","shell.execute_reply":"2023-01-10T09:40:47.019782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':75,'max_leaf_nodes':0,'max_depth':32}\nlosrf_mln_m1=train_model(RandomForestRegressor,params,'max_leaf_nodes',[300,3001,300],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T09:58:42.836724Z","iopub.execute_input":"2023-01-10T09:58:42.837118Z","iopub.status.idle":"2023-01-10T10:03:26.177069Z","shell.execute_reply.started":"2023-01-10T09:58:42.837086Z","shell.execute_reply":"2023-01-10T10:03:26.175806Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':75,'max_leaf_nodes':0,'max_depth':32}\nlosrf_mln_m1=train_model(RandomForestRegressor,params,'max_leaf_nodes',[3000,6001,1000],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T10:52:58.824978Z","iopub.execute_input":"2023-01-10T10:52:58.825533Z","iopub.status.idle":"2023-01-10T10:57:04.697923Z","shell.execute_reply.started":"2023-01-10T10:52:58.825488Z","shell.execute_reply":"2023-01-10T10:57:04.696311Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':75,'max_leaf_nodes':0,'max_depth':32}\nlosrf_mln_m1=train_model(RandomForestRegressor,params,'max_leaf_nodes',[6000,10001,1000],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T10:59:00.149009Z","iopub.execute_input":"2023-01-10T10:59:00.149574Z","iopub.status.idle":"2023-01-10T11:04:27.019226Z","shell.execute_reply.started":"2023-01-10T10:59:00.149529Z","shell.execute_reply":"2023-01-10T11:04:27.017960Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params={'n_estimators':50,'max_features':'sqrt','max_leaf_nodes':5000,'max_depth':32}\ntrain_model(RandomForestRegressor,params,'max_leaf_nodes',[1,552,50],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-08T17:49:20.221451Z","iopub.execute_input":"2023-01-08T17:49:20.221966Z","iopub.status.idle":"2023-01-08T17:49:57.389732Z","shell.execute_reply.started":"2023-01-08T17:49:20.221926Z","shell.execute_reply":"2023-01-08T17:49:57.388908Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So the best parameters for this model are n_estimators = 75 max_depth= 32. We will be creating model with these parameters, fitting the training data and then predicting meter readings for test data","metadata":{}},{"cell_type":"code","source":"%%time\nwbt_df,s1_df=dataset_sample_meter(train1_df,0.06)\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=RandomForestRegressor(n_jobs=-1,n_estimators=75,max_depth=32,random_state=42)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df\ndel s1_df","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:35:07.190931Z","iopub.execute_input":"2023-01-12T09:35:07.191272Z","iopub.status.idle":"2023-01-12T09:35:55.999703Z","shell.execute_reply.started":"2023-01-12T09:35:07.191248Z","shell.execute_reply":"2023-01-12T09:35:55.998512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test1_df,.06,test=True)\ntest1_df=test1_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest1_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test1_preds\n\ntest1_df=pd.merge(test1_df,wbt_df,on='row_id')\ndel wbt_df\ndel test1_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:35:56.001623Z","iopub.execute_input":"2023-01-12T09:35:56.001877Z","iopub.status.idle":"2023-01-12T09:37:09.062574Z","shell.execute_reply.started":"2023-01-12T09:35:56.001845Z","shell.execute_reply":"2023-01-12T09:37:09.061899Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training Model for Meter 2","metadata":{}},{"cell_type":"code","source":"2e5/len(train2_df)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s2_df=dataset_sample_meter(train2_df,0.1)\ndf0=s2_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\ndel wbt_df\ndel s2_df","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':0}\nlosrf_ne_m2=train_model(RandomForestRegressor,params,'n_estimators',[50,151,25],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-11T11:45:31.992493Z","iopub.execute_input":"2023-01-11T11:45:31.992886Z","iopub.status.idle":"2023-01-11T11:51:45.691128Z","shell.execute_reply.started":"2023-01-11T11:45:31.992852Z","shell.execute_reply":"2023-01-11T11:51:45.689704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> n estimators 100 is fine","metadata":{}},{"cell_type":"code","source":"%%time\nparams={'max_depth':0,'n_estimators':100}\nlos_md=train_model(RandomForestRegressor,params,'max_depth',[3,31,3],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T09:06:06.111947Z","iopub.execute_input":"2023-01-10T09:06:06.113035Z","iopub.status.idle":"2023-01-10T09:08:57.125803Z","shell.execute_reply.started":"2023-01-10T09:06:06.112982Z","shell.execute_reply":"2023-01-10T09:08:57.124834Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> max depth of 25 is fine","metadata":{}},{"cell_type":"code","source":"%%time\nparams={'n_estimators':100,'max_leaf_nodes':0,'max_depth':25}\nlosrf_mln_m2=train_model(RandomForestRegressor,params,'max_leaf_nodes',[100,1551,100],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T09:24:26.669324Z","iopub.execute_input":"2023-01-10T09:24:26.669713Z","iopub.status.idle":"2023-01-10T09:28:48.128649Z","shell.execute_reply.started":"2023-01-10T09:24:26.669682Z","shell.execute_reply":"2023-01-10T09:28:48.127598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s2_df=dataset_sample_meter(train2_df,0.1)\ndf0=s2_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=RandomForestRegressor(n_jobs=-1,n_estimators=100,max_depth=25,random_state=42)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df\ndel s2_df","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:37:09.063776Z","iopub.execute_input":"2023-01-12T09:37:09.064628Z","iopub.status.idle":"2023-01-12T09:38:23.269174Z","shell.execute_reply.started":"2023-01-12T09:37:09.064599Z","shell.execute_reply":"2023-01-12T09:38:23.267816Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test2_df,.06,test=True)\ntest2_df=test2_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest2_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test2_preds\ntest2_df=pd.merge(test2_df,wbt_df,on='row_id')\ndel wbt_df\ndel test2_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:38:23.270623Z","iopub.execute_input":"2023-01-12T09:38:23.270936Z","iopub.status.idle":"2023-01-12T09:39:20.684526Z","shell.execute_reply.started":"2023-01-12T09:38:23.270912Z","shell.execute_reply":"2023-01-12T09:39:20.683133Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training Model for Meter 3","metadata":{}},{"cell_type":"code","source":"2e5/len(train3_df)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s3_df=dataset_sample_meter(train3_df,0.3)\ndf0=s3_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\ndel wbt_df,s3_df,df0\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-11T11:51:46.132130Z","iopub.execute_input":"2023-01-11T11:51:46.132566Z","iopub.status.idle":"2023-01-11T11:51:46.489327Z","shell.execute_reply.started":"2023-01-11T11:51:46.132524Z","shell.execute_reply":"2023-01-11T11:51:46.488244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':0}\nlosrf_ne_m3=train_model(RandomForestRegressor,params,'n_estimators',[50,176,25],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-11T11:51:46.490891Z","iopub.execute_input":"2023-01-11T11:51:46.491511Z","iopub.status.idle":"2023-01-11T11:54:19.032461Z","shell.execute_reply.started":"2023-01-11T11:51:46.491466Z","shell.execute_reply":"2023-01-11T11:54:19.031140Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"n_estimators 125","metadata":{}},{"cell_type":"code","source":"%%time\nparams={'max_depth':0,'n_estimators':75}\nlosrf_md_m3=train_model(RandomForestRegressor,params,'max_depth',[4,41,4],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T09:28:48.266949Z","iopub.execute_input":"2023-01-10T09:28:48.267224Z","iopub.status.idle":"2023-01-10T09:30:29.511031Z","shell.execute_reply.started":"2023-01-10T09:28:48.267197Z","shell.execute_reply":"2023-01-10T09:30:29.510009Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s3_df=dataset_sample_meter(train3_df,0.3)\ndf0=s3_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=RandomForestRegressor(n_jobs=-1,n_estimators=125,random_state=42)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df\ndel s2_df","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:39:20.686716Z","iopub.execute_input":"2023-01-12T09:39:20.687291Z","iopub.status.idle":"2023-01-12T09:40:38.892338Z","shell.execute_reply.started":"2023-01-12T09:39:20.687258Z","shell.execute_reply":"2023-01-12T09:40:38.891415Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test3_df,.06,test=True)\ntest3_df=test3_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest1_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test1_preds\ntest3_df=pd.merge(test3_df,wbt_df,on='row_id')\ndel wbt_df\ndel test1_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:40:38.893585Z","iopub.execute_input":"2023-01-12T09:40:38.894750Z","iopub.status.idle":"2023-01-12T09:41:02.938140Z","shell.execute_reply.started":"2023-01-12T09:40:38.894712Z","shell.execute_reply":"2023-01-12T09:41:02.936670Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training Model for Meter 0","metadata":{}},{"cell_type":"code","source":"2e5/len(train0_df)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T08:54:12.920684Z","iopub.execute_input":"2023-01-10T08:54:12.921060Z","iopub.status.idle":"2023-01-10T08:54:12.928084Z","shell.execute_reply.started":"2023-01-10T08:54:12.921031Z","shell.execute_reply":"2023-01-10T08:54:12.927166Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s0_df=dataset_sample_meter(train0_df,0.02)\ndf0=s0_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\ndel wbt_df,s0_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-11T11:54:25.546101Z","iopub.execute_input":"2023-01-11T11:54:25.546564Z","iopub.status.idle":"2023-01-11T11:54:28.349020Z","shell.execute_reply.started":"2023-01-11T11:54:25.546524Z","shell.execute_reply":"2023-01-11T11:54:28.347869Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':0}\nlosrf_ne_m0=train_model(RandomForestRegressor,params,'n_estimators',[50,126,25],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-11T11:54:28.351475Z","iopub.execute_input":"2023-01-11T11:54:28.352390Z","iopub.status.idle":"2023-01-11T11:59:24.812793Z","shell.execute_reply.started":"2023-01-11T11:54:28.352353Z","shell.execute_reply":"2023-01-11T11:59:24.811592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"ne=75","metadata":{}},{"cell_type":"code","source":"%%time\nparams={'max_depth':0,'n_estimators':75}\nlosrf_md_m0=train_model(RandomForestRegressor,params,'max_depth',[28,53,4],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T15:57:48.591830Z","iopub.execute_input":"2023-01-10T15:57:48.592729Z","iopub.status.idle":"2023-01-10T16:03:02.035227Z","shell.execute_reply.started":"2023-01-10T15:57:48.592673Z","shell.execute_reply":"2023-01-10T16:03:02.034000Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s1_df=dataset_sample_meter(train0_df,0.02)\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=RandomForestRegressor(n_jobs=-1,n_estimators=75,max_depth=40,random_state=42)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s1_df","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:29:30.737464Z","iopub.execute_input":"2023-01-12T09:29:30.737846Z","iopub.status.idle":"2023-01-12T09:30:34.589140Z","shell.execute_reply.started":"2023-01-12T09:29:30.737819Z","shell.execute_reply":"2023-01-12T09:30:34.587557Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test0_df,.06,test=True)\ntest0_df=test0_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest1_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test1_preds\ntest0_df=pd.merge(test0_df,wbt_df,on='row_id')\ndel wbt_df\ndel test1_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:30:34.590819Z","iopub.execute_input":"2023-01-12T09:30:34.591116Z","iopub.status.idle":"2023-01-12T09:31:28.983750Z","shell.execute_reply.started":"2023-01-12T09:30:34.591089Z","shell.execute_reply":"2023-01-12T09:31:28.982273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test01_df,.06,test=True)\ntest01_df=test01_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest01_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test01_preds\ntest01_df=pd.merge(test01_df,wbt_df,on='row_id')\ndel wbt_df\ndel test01_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:31:28.985074Z","iopub.execute_input":"2023-01-12T09:31:28.985353Z","iopub.status.idle":"2023-01-12T09:32:31.538542Z","shell.execute_reply.started":"2023-01-12T09:31:28.985329Z","shell.execute_reply":"2023-01-12T09:32:31.536550Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test02_df,.06,test=True)\ntest02_df=test02_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest1_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test1_preds\ntest02_df=pd.merge(test02_df,wbt_df,on='row_id')\ndel wbt_df\ndel test1_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:32:31.542601Z","iopub.execute_input":"2023-01-12T09:32:31.542980Z","iopub.status.idle":"2023-01-12T09:33:31.384967Z","shell.execute_reply.started":"2023-01-12T09:32:31.542946Z","shell.execute_reply":"2023-01-12T09:33:31.383523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test03_df,.06,test=True)\ntest03_df=test03_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest1_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test1_preds\ntest03_df=pd.merge(test03_df,wbt_df,on='row_id')\ndel wbt_df\ndel test1_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:33:31.387282Z","iopub.execute_input":"2023-01-12T09:33:31.387699Z","iopub.status.idle":"2023-01-12T09:34:31.597714Z","shell.execute_reply.started":"2023-01-12T09:33:31.387669Z","shell.execute_reply":"2023-01-12T09:34:31.596753Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del test_inputs","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:34:31.599258Z","iopub.execute_input":"2023-01-12T09:34:31.599889Z","iopub.status.idle":"2023-01-12T09:34:31.613527Z","shell.execute_reply.started":"2023-01-12T09:34:31.599850Z","shell.execute_reply":"2023-01-12T09:34:31.612010Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Other Models**","metadata":{}},{"cell_type":"code","source":"cat_cols.append('building_id')","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:41:37.349797Z","iopub.execute_input":"2023-01-12T09:41:37.350172Z","iopub.status.idle":"2023-01-12T09:41:37.355842Z","shell.execute_reply.started":"2023-01-12T09:41:37.350146Z","shell.execute_reply":"2023-01-12T09:41:37.354304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s01_df=dataset_sample_meter(train0_df_1,1)\ndf0=s01_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\n#model=RandomForestRegressor(n_jobs=-1,n_estimators=75,random_state=42)\n#model.fit(t_inputs,t_targets)\ndel wbt_df,s01_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:43:09.203982Z","iopub.execute_input":"2023-01-12T09:43:09.204499Z","iopub.status.idle":"2023-01-12T09:43:09.323537Z","shell.execute_reply.started":"2023-01-12T09:43:09.204452Z","shell.execute_reply":"2023-01-12T09:43:09.321820Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':0}\nlosrf_ne_m0=train_model(RandomForestRegressor,params,'n_estimators',[10,51,5],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:43:35.719128Z","iopub.execute_input":"2023-01-12T09:43:35.719522Z","iopub.status.idle":"2023-01-12T09:43:45.957243Z","shell.execute_reply.started":"2023-01-12T09:43:35.719476Z","shell.execute_reply":"2023-01-12T09:43:45.956051Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s2_df=dataset_sample_meter(train0_df_1,1)\ndf0=s2_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=RandomForestRegressor(n_jobs=-1,n_estimators=34,max_depth=25,random_state=42)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s2_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:49:57.445326Z","iopub.execute_input":"2023-01-12T09:49:57.445760Z","iopub.status.idle":"2023-01-12T09:49:58.466847Z","shell.execute_reply.started":"2023-01-12T09:49:57.445730Z","shell.execute_reply":"2023-01-12T09:49:58.465082Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test0_df_1,.06,test=True)\ntest0_df_1=test0_df_1[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest1_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test1_preds\ntest0_df_1=pd.merge(test0_df_1,wbt_df,on='row_id')\ndel wbt_df\ndel test1_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:51:17.636124Z","iopub.execute_input":"2023-01-12T09:51:17.636526Z","iopub.status.idle":"2023-01-12T09:51:17.901631Z","shell.execute_reply.started":"2023-01-12T09:51:17.636498Z","shell.execute_reply":"2023-01-12T09:51:17.900414Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s11_df=dataset_sample_meter(train1_df_1,1)\ndf0=s11_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\ndel wbt_df,df0,s11_df","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:53:11.038402Z","iopub.execute_input":"2023-01-12T09:53:11.038805Z","iopub.status.idle":"2023-01-12T09:53:11.142095Z","shell.execute_reply.started":"2023-01-12T09:53:11.038778Z","shell.execute_reply":"2023-01-12T09:53:11.140634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':0}\nlosrf_ne_m11=train_model(RandomForestRegressor,params,'n_estimators',[10,51,5],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:54:51.701179Z","iopub.execute_input":"2023-01-12T09:54:51.701561Z","iopub.status.idle":"2023-01-12T09:55:02.056234Z","shell.execute_reply.started":"2023-01-12T09:54:51.701534Z","shell.execute_reply":"2023-01-12T09:55:02.054857Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s11_df=dataset_sample_meter(train1_df_1,1)\ndf0=s11_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=RandomForestRegressor(n_jobs=-1,n_estimators=43,random_state=42)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s11_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:57:01.011404Z","iopub.execute_input":"2023-01-12T09:57:01.011807Z","iopub.status.idle":"2023-01-12T09:57:02.332375Z","shell.execute_reply.started":"2023-01-12T09:57:01.011779Z","shell.execute_reply":"2023-01-12T09:57:02.331639Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test1_df_1,.06,test=True)\ntest1_df_1=test1_df_1[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest1_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test1_preds\ntest1_df_1=pd.merge(test1_df_1,wbt_df,on='row_id')\ndel wbt_df\ndel test1_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:57:52.764547Z","iopub.execute_input":"2023-01-12T09:57:52.764905Z","iopub.status.idle":"2023-01-12T09:57:53.159377Z","shell.execute_reply.started":"2023-01-12T09:57:52.764879Z","shell.execute_reply":"2023-01-12T09:57:53.157956Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s2_df=dataset_sample_meter(train2_df_1,1)\ndf0=s2_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\ndel wbt_df,s2_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:59:14.379690Z","iopub.execute_input":"2023-01-12T09:59:14.380079Z","iopub.status.idle":"2023-01-12T09:59:14.540607Z","shell.execute_reply.started":"2023-01-12T09:59:14.380051Z","shell.execute_reply":"2023-01-12T09:59:14.539556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':0}\nlosrf_ne_m21=train_model(RandomForestRegressor,params,'n_estimators',[10,51,5],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T09:59:35.895627Z","iopub.execute_input":"2023-01-12T09:59:35.895963Z","iopub.status.idle":"2023-01-12T09:59:57.420866Z","shell.execute_reply.started":"2023-01-12T09:59:35.895937Z","shell.execute_reply":"2023-01-12T09:59:57.419531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':30}\nlosrf_ne_m21=train_model(RandomForestRegressor,params,'max_depth',[10,31,2],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T10:01:07.786632Z","iopub.execute_input":"2023-01-12T10:01:07.786973Z","iopub.status.idle":"2023-01-12T10:01:29.353554Z","shell.execute_reply.started":"2023-01-12T10:01:07.786949Z","shell.execute_reply":"2023-01-12T10:01:29.352451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s2_df=dataset_sample_meter(train2_df_1,1)\ndf0=s2_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=RandomForestRegressor(n_jobs=-1,n_estimators=27,max_depth=22,random_state=42)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s2_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-12T10:22:23.103035Z","iopub.execute_input":"2023-01-12T10:22:23.103395Z","iopub.status.idle":"2023-01-12T10:22:25.083049Z","shell.execute_reply.started":"2023-01-12T10:22:23.103368Z","shell.execute_reply":"2023-01-12T10:22:25.081053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test2_df_1,.06,test=True)\ntest2_df_1=test2_df_1[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest2_1preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test2_1preds\ntest2_df_1=pd.merge(test2_df_1,wbt_df,on='row_id')\ndel wbt_df\ndel test2_1preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T10:22:32.652284Z","iopub.execute_input":"2023-01-12T10:22:32.652651Z","iopub.status.idle":"2023-01-12T10:22:33.075206Z","shell.execute_reply.started":"2023-01-12T10:22:32.652622Z","shell.execute_reply":"2023-01-12T10:22:33.073906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s31_df=dataset_sample_meter(train3_df_1,1)\ndf0=s31_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T10:07:10.652814Z","iopub.execute_input":"2023-01-12T10:07:10.653178Z","iopub.status.idle":"2023-01-12T10:07:10.811982Z","shell.execute_reply.started":"2023-01-12T10:07:10.653153Z","shell.execute_reply":"2023-01-12T10:07:10.810821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':0}\nlosrf_ne_m31=train_model(RandomForestRegressor,params,'n_estimators',[10,71,5],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T10:10:05.582081Z","iopub.execute_input":"2023-01-12T10:10:05.582469Z","iopub.status.idle":"2023-01-12T10:10:47.100591Z","shell.execute_reply.started":"2023-01-12T10:10:05.582441Z","shell.execute_reply":"2023-01-12T10:10:47.099308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nparams={'n_estimators':45}\nlosrf_ne_m31=train_model(RandomForestRegressor,params,'max_depth',[30,41,2],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T10:12:13.225986Z","iopub.execute_input":"2023-01-12T10:12:13.227546Z","iopub.status.idle":"2023-01-12T10:12:34.181731Z","shell.execute_reply.started":"2023-01-12T10:12:13.227459Z","shell.execute_reply":"2023-01-12T10:12:34.180326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s2_df=dataset_sample_meter(train3_df_1,1)\ndf0=s2_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=RandomForestRegressor(n_jobs=-1,n_estimators=43,max_depth=33,random_state=42)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s2_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-12T10:13:38.788284Z","iopub.execute_input":"2023-01-12T10:13:38.788696Z","iopub.status.idle":"2023-01-12T10:13:41.869850Z","shell.execute_reply.started":"2023-01-12T10:13:38.788669Z","shell.execute_reply":"2023-01-12T10:13:41.867786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test3_df_1,.06,test=True)\ntest3_df_1=test3_df_1[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest1_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test1_preds\ntest3_df_1=pd.merge(test3_df_1,wbt_df,on='row_id')\ndel wbt_df\ndel test1_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-12T10:16:26.457373Z","iopub.execute_input":"2023-01-12T10:16:26.457765Z","iopub.status.idle":"2023-01-12T10:16:27.042047Z","shell.execute_reply.started":"2023-01-12T10:16:26.457738Z","shell.execute_reply":"2023-01-12T10:16:27.040533Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s22_df=dataset_sample_meter(train2_df_2,1)\ndf0=s22_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0}\nlosrf_ne_m22=train_model(RandomForestRegressor,params,'n_estimators',[50,131,10],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T16:50:27.181086Z","iopub.execute_input":"2023-01-10T16:50:27.181520Z","iopub.status.idle":"2023-01-10T16:50:55.707944Z","shell.execute_reply.started":"2023-01-10T16:50:27.181484Z","shell.execute_reply":"2023-01-10T16:50:55.706256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s11_df=dataset_sample_meter(train1_df_1,1)\ndf0=s11_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0}\nlosrf_ne_m31=train_model(RandomForestRegressor,params,'n_estimators',[20,121,10],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T16:54:31.300920Z","iopub.execute_input":"2023-01-10T16:54:31.301341Z","iopub.status.idle":"2023-01-10T16:54:41.547095Z","shell.execute_reply.started":"2023-01-10T16:54:31.301309Z","shell.execute_reply":"2023-01-10T16:54:41.545887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s32_df=dataset_sample_meter(train3_df_2,1)\ndf0=s32_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0}\nlosrf_ne_m32=train_model(RandomForestRegressor,params,'n_estimators',[20,121,10],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-10T16:55:22.717007Z","iopub.execute_input":"2023-01-10T16:55:22.717406Z","iopub.status.idle":"2023-01-10T16:55:34.332464Z","shell.execute_reply.started":"2023-01-10T16:55:22.717375Z","shell.execute_reply":"2023-01-10T16:55:34.331179Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df=pd.read_csv('/kaggle/input/ashrae-energy-prediction/test.csv',dtype=training_dtypes)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T11:09:55.503531Z","iopub.execute_input":"2023-01-12T11:09:55.503871Z","iopub.status.idle":"2023-01-12T11:10:07.022227Z","shell.execute_reply.started":"2023-01-12T11:09:55.503845Z","shell.execute_reply":"2023-01-12T11:10:07.020587Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df=test_df[['row_id']]\ntest_df['meter_reading']=0","metadata":{"execution":{"iopub.status.busy":"2023-01-12T11:10:53.255949Z","iopub.execute_input":"2023-01-12T11:10:53.256343Z","iopub.status.idle":"2023-01-12T11:10:53.636868Z","shell.execute_reply.started":"2023-01-12T11:10:53.256315Z","shell.execute_reply":"2023-01-12T11:10:53.635128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df.set_index('row_id',inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T11:11:01.925419Z","iopub.execute_input":"2023-01-12T11:11:01.925996Z","iopub.status.idle":"2023-01-12T11:11:01.931017Z","shell.execute_reply.started":"2023-01-12T11:11:01.925966Z","shell.execute_reply":"2023-01-12T11:11:01.929935Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#test0_df.set_index('row_id',inplace=True)\ntest0_df_1.set_index('row_id',inplace=True)\ntest01_df.set_index('row_id',inplace=True)\ntest02_df.set_index('row_id',inplace=True)\ntest03_df.set_index('row_id',inplace=True)\n\ntest1_df.set_index('row_id',inplace=True)\ntest1_df_1.set_index('row_id',inplace=True)\n\ntest2_df.set_index('row_id',inplace=True)\ntest2_df_1.set_index('row_id',inplace=True)\n\ntest3_df.set_index('row_id',inplace=True)\ntest3_df_1.set_index('row_id',inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T11:14:13.978435Z","iopub.execute_input":"2023-01-12T11:14:13.978794Z","iopub.status.idle":"2023-01-12T11:14:13.993389Z","shell.execute_reply.started":"2023-01-12T11:14:13.978768Z","shell.execute_reply":"2023-01-12T11:14:13.991998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"allparts=[test0_df,test01_df,test02_df,test03_df,test0_df_1,test0_df_2,\n test1_df,test1_df_1,\n test2_df,test2_df_1,\n test3_df,test3_df_1,]\nsubmission_df=pd.concat(allparts)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.sort_values(by='row_id',inplace=True)\nsubmission_df.reset_index(drop=True)\nsubmission_df.meter_reading=np.expm1(submission_df.meter_reading)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T11:19:41.215580Z","iopub.execute_input":"2023-01-12T11:19:41.215936Z","iopub.status.idle":"2023-01-12T11:19:41.561622Z","shell.execute_reply.started":"2023-01-12T11:19:41.215909Z","shell.execute_reply":"2023-01-12T11:19:41.560516Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(submission_df)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T11:29:12.805116Z","iopub.execute_input":"2023-01-12T11:29:12.805588Z","iopub.status.idle":"2023-01-12T11:29:12.817418Z","shell.execute_reply.started":"2023-01-12T11:29:12.805560Z","shell.execute_reply":"2023-01-12T11:29:12.815928Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nsubmission_df.to_csv('rf_meter_wise_submission1.csv',index=None)","metadata":{"execution":{"iopub.status.busy":"2023-01-12T12:06:56.233455Z","iopub.execute_input":"2023-01-12T12:06:56.233833Z","iopub.status.idle":"2023-01-12T12:08:21.503773Z","shell.execute_reply.started":"2023-01-12T12:06:56.233807Z","shell.execute_reply":"2023-01-12T12:08:21.502568Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Submission Score:-
\n","metadata":{}},{"cell_type":"markdown","source":"## XGBoost Regressor","metadata":{}},{"cell_type":"code","source":"num_cols=['square_feet', 'air_temperature','dew_temperature',\n 'cloud_coverage', 'precip_depth_1_hr',\n 'sea_level_pressure', 'wind_speed']\ncat_cols=['primary_use','month','weekday','hour']\ntarget_col='meter_reading'","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:57:15.821349Z","iopub.execute_input":"2023-01-13T10:57:15.821852Z","iopub.status.idle":"2023-01-13T10:57:15.829060Z","shell.execute_reply.started":"2023-01-13T10:57:15.821816Z","shell.execute_reply":"2023-01-13T10:57:15.827461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s1_df=dataset_sample_meter(train1_df,.07)\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0,'objective':'reg:squarederror'}\nlosrf_ne_m1=train_model(XGBRegressor,params,'n_estimators',[100,501,100],t_inputs,v_inputs,t_targets,v_targets)\ndel s1_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-13T10:49:32.073595Z","iopub.execute_input":"2023-01-13T10:49:32.074551Z","iopub.status.idle":"2023-01-13T10:55:51.096523Z","shell.execute_reply.started":"2023-01-13T10:49:32.074486Z","shell.execute_reply":"2023-01-13T10:55:51.095591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There negligible difference in losses. Let's try adding site_id in cat_cols and building_id in num cols as one hot encoding building id will take lots of space and may get us out of RAM.","metadata":{}},{"cell_type":"code","source":"num_cols=['square_feet', 'air_temperature','dew_temperature',\n 'cloud_coverage', 'precip_depth_1_hr',\n 'sea_level_pressure', 'wind_speed','building_id']\ncat_cols=['primary_use','month','weekday','hour','site_id']\ntarget_col='meter_reading'","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:46:37.934880Z","iopub.execute_input":"2023-01-16T12:46:37.935375Z","iopub.status.idle":"2023-01-16T12:46:37.942972Z","shell.execute_reply.started":"2023-01-16T12:46:37.935328Z","shell.execute_reply":"2023-01-16T12:46:37.941019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s1_df=dataset_sample_meter(train1_df,.07)\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0,'objective':'reg:squarederror'}\nlosxb_ne_m1=train_model(XGBRegressor,params,'n_estimators',[200,901,100],t_inputs,v_inputs,t_targets,v_targets)\ndel s1_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-13T11:15:44.169259Z","iopub.execute_input":"2023-01-13T11:15:44.169826Z","iopub.status.idle":"2023-01-13T11:38:17.603389Z","shell.execute_reply.started":"2023-01-13T11:15:44.169782Z","shell.execute_reply":"2023-01-13T11:38:17.602042Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We got best results with latest modifications.\nNo overfitting is happening till 900. We will not go ahead due to yime constraints","metadata":{}},{"cell_type":"code","source":"%%time\nparams={'n_estimators':900,'objective':'reg:squarederror','learning_rate':0}\nlosxgb_lr_m1=train_model(XGBRegressor,params,'learning_rate',[.05,0.16,.05],t_inputs,v_inputs,t_targets,v_targets)\n#del s1_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-13T12:00:02.592791Z","iopub.execute_input":"2023-01-13T12:00:02.593280Z","iopub.status.idle":"2023-01-13T12:14:11.062217Z","shell.execute_reply.started":"2023-01-13T12:00:02.593246Z","shell.execute_reply":"2023-01-13T12:14:11.058718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s1_df=dataset_sample_meter(train1_df,.07)\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':900,'objective':'reg:squarederror','learning_rate':0}\nlosxgb_lr_m1=train_model(XGBRegressor,params,'learning_rate',[.25,0.36,.05],t_inputs,v_inputs,t_targets,v_targets)\ndel s1_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-14T19:49:49.716271Z","iopub.execute_input":"2023-01-14T19:49:49.717703Z","iopub.status.idle":"2023-01-14T20:04:09.706980Z","shell.execute_reply.started":"2023-01-14T19:49:49.717638Z","shell.execute_reply":"2023-01-14T20:04:09.705387Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params={'n_estimators':900,'objective':'reg:squarederror','learning_rate':0.1,'max_depth':0}\nlosxgb_md_m1=train_model(XGBRegressor,params,'max_depth',[5,10,2],t_inputs,v_inputs,t_targets,v_targets)","metadata":{"execution":{"iopub.status.busy":"2023-01-13T12:14:11.076026Z","iopub.execute_input":"2023-01-13T12:14:11.076430Z","iopub.status.idle":"2023-01-13T12:29:32.700912Z","shell.execute_reply.started":"2023-01-13T12:14:11.076397Z","shell.execute_reply":"2023-01-13T12:29:32.699610Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s1_df=dataset_sample_meter(train1_df,.07)\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':900,'objective':'reg:squarederror','learning_rate':0.3,'max_depth':0}\nlosxgb_md_m1=train_model(XGBRegressor,params,'max_depth',[5,14,2],t_inputs,v_inputs,t_targets,v_targets)\ndel s1_df,df0,wbt_df\n","metadata":{"execution":{"iopub.status.busy":"2023-01-14T20:31:49.487780Z","iopub.execute_input":"2023-01-14T20:31:49.488220Z","iopub.status.idle":"2023-01-14T21:10:14.892860Z","shell.execute_reply.started":"2023-01-14T20:31:49.488186Z","shell.execute_reply":"2023-01-14T21:10:14.891221Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params={'n_estimators':900,'objective':'reg:squarederror','learning_rate':.15,'max_depth':11}","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:46:46.988305Z","iopub.execute_input":"2023-01-16T12:46:46.988797Z","iopub.status.idle":"2023-01-16T12:46:46.994742Z","shell.execute_reply.started":"2023-01-16T12:46:46.988754Z","shell.execute_reply":"2023-01-16T12:46:46.992858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s1_df=dataset_sample_meter(train1_df,.07)\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=XGBRegressor(n_jobs=-1,random_state=42,**params)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s1_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:46:54.548161Z","iopub.execute_input":"2023-01-16T12:46:54.548662Z","iopub.status.idle":"2023-01-16T12:54:24.137775Z","shell.execute_reply.started":"2023-01-16T12:46:54.548626Z","shell.execute_reply":"2023-01-16T12:54:24.136680Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test1_df,.06,test=True)\ntest1_df=test1_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest1_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test1_preds\n\ntest1_df=pd.merge(test1_df,wbt_df,on='row_id')\ndel wbt_df\ndel test1_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:54:24.139833Z","iopub.execute_input":"2023-01-16T12:54:24.140932Z","iopub.status.idle":"2023-01-16T12:58:10.754910Z","shell.execute_reply.started":"2023-01-16T12:54:24.140892Z","shell.execute_reply":"2023-01-16T12:58:10.752891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training for Meter 2","metadata":{}},{"cell_type":"code","source":"wbt_df,s1_df=dataset_sample_meter(train2_df,.1)\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0,'objective':'reg:squarederror'}\n#losxgb_md_m1=train_model(XGBRegressor,params,'max_depth',[5,11,2],t_inputs,v_inputs,t_targets,v_targets)\nlosxgb_ne_m2=train_model(XGBRegressor,params,'n_estimators',[200,901,100],t_inputs,v_inputs,t_targets,v_targets)\ndel s1_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-13T12:29:32.713086Z","iopub.execute_input":"2023-01-13T12:29:32.714049Z","iopub.status.idle":"2023-01-13T12:48:31.327654Z","shell.execute_reply.started":"2023-01-13T12:29:32.714001Z","shell.execute_reply":"2023-01-13T12:48:31.326275Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s1_df=dataset_sample_meter(train2_df,.1)\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':800,'objective':'reg:squarederror','learning_rate':0}\nlosxgb_lr_m2=train_model(XGBRegressor,params,'learning_rate',[.1,.26,.05],t_inputs,v_inputs,t_targets,v_targets)\ndel s1_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-13T17:32:23.415905Z","iopub.execute_input":"2023-01-13T17:32:23.416879Z","iopub.status.idle":"2023-01-13T17:44:04.247011Z","shell.execute_reply.started":"2023-01-13T17:32:23.416817Z","shell.execute_reply":"2023-01-13T17:44:04.246381Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s2_df=dataset_sample_meter(train2_df,.1)\ndf0=s2_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':900,'objective':'reg:squarederror','learning_rate':0}\nlosxgb_lr_m2=train_model(XGBRegressor,params,'learning_rate',[.15,.26,.05],t_inputs,v_inputs,t_targets,v_targets)\ndel s2_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-14T21:10:14.894941Z","iopub.execute_input":"2023-01-14T21:10:14.895616Z","iopub.status.idle":"2023-01-14T21:22:52.098408Z","shell.execute_reply.started":"2023-01-14T21:10:14.895571Z","shell.execute_reply":"2023-01-14T21:22:52.097366Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s2_df=dataset_sample_meter(train2_df,.1)\ndf0=s2_df.copy()","metadata":{"execution":{"iopub.status.busy":"2023-01-15T20:57:36.297332Z","iopub.execute_input":"2023-01-15T20:57:36.297806Z","iopub.status.idle":"2023-01-15T20:57:38.823183Z","shell.execute_reply.started":"2023-01-15T20:57:36.297774Z","shell.execute_reply":"2023-01-15T20:57:38.822090Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params={'n_estimators':900,'objective':'reg:squarederror','learning_rate':.1,'max_depth':10}","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:58:10.756975Z","iopub.execute_input":"2023-01-16T12:58:10.758394Z","iopub.status.idle":"2023-01-16T12:58:10.764990Z","shell.execute_reply.started":"2023-01-16T12:58:10.758345Z","shell.execute_reply":"2023-01-16T12:58:10.763618Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s2_df=dataset_sample_meter(train2_df,.1)\ndf0=s2_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=XGBRegressor(n_jobs=-1,random_state=42,**params)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s2_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:58:10.768118Z","iopub.execute_input":"2023-01-16T12:58:10.768614Z","iopub.status.idle":"2023-01-16T13:04:22.233372Z","shell.execute_reply.started":"2023-01-16T12:58:10.768581Z","shell.execute_reply":"2023-01-16T13:04:22.231396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test2_df,.06,test=True)\ntest2_df=test2_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest2_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test2_preds\n\ntest2_df=pd.merge(test2_df,wbt_df,on='row_id')\ndel wbt_df\ndel test2_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:04:22.235072Z","iopub.execute_input":"2023-01-16T13:04:22.235455Z","iopub.status.idle":"2023-01-16T13:06:27.554946Z","shell.execute_reply.started":"2023-01-16T13:04:22.235418Z","shell.execute_reply":"2023-01-16T13:06:27.552845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training for Meter 3","metadata":{}},{"cell_type":"code","source":"wbt_df,s3_df=dataset_sample_meter(train3_df,.35)\ndf0=s3_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0,'objective':'reg:squarederror'}\n#losxgb_md_m1=train_model(XGBRegressor,params,'max_depth',[5,11,2],t_inputs,v_inputs,t_targets,v_targets)\nlosxgb_ne_m3=train_model(XGBRegressor,params,'n_estimators',[200,901,100],t_inputs,v_inputs,t_targets,v_targets)\ndel s3_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-13T12:48:31.329784Z","iopub.execute_input":"2023-01-13T12:48:31.330598Z","iopub.status.idle":"2023-01-13T13:13:00.954691Z","shell.execute_reply.started":"2023-01-13T12:48:31.330546Z","shell.execute_reply":"2023-01-13T13:13:00.953125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s3_df=dataset_sample_meter(train3_df,.35)\ndf0=s3_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':900,'objective':'reg:squarederror'}\n#losxgb_md_m1=train_model(XGBRegressor,params,'max_depth',[5,11,2],t_inputs,v_inputs,t_targets,v_targets)\nlosxgb_lr_m3=train_model(XGBRegressor,params,'learning_rate',[.1,.26,.05],t_inputs,v_inputs,t_targets,v_targets)\ndel s3_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-13T17:44:04.270985Z","iopub.execute_input":"2023-01-13T17:44:04.271795Z","iopub.status.idle":"2023-01-13T18:00:34.266916Z","shell.execute_reply.started":"2023-01-13T17:44:04.271760Z","shell.execute_reply":"2023-01-13T18:00:34.265604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s3_df=dataset_sample_meter(train3_df,.35)\ndf0=s3_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':900,'objective':'reg:squarederror'}\n#losxgb_md_m1=train_model(XGBRegressor,params,'max_depth',[5,11,2],t_inputs,v_inputs,t_targets,v_targets)\nlosxgb_lr_m3=train_model(XGBRegressor,params,'learning_rate',[.3,.36,.05],t_inputs,v_inputs,t_targets,v_targets)\ndel s3_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-14T20:04:11.782966Z","iopub.execute_input":"2023-01-14T20:04:11.783306Z","iopub.status.idle":"2023-01-14T20:15:03.022096Z","shell.execute_reply.started":"2023-01-14T20:04:11.783275Z","shell.execute_reply":"2023-01-14T20:15:03.020858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s3_df=dataset_sample_meter(train3_df,.35)\ndf0=s3_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':900,'objective':'reg:squarederror','max_depth':0}\nlosxgb_md_m3=train_model(XGBRegressor,params,'max_depth',[5,14,2],t_inputs,v_inputs,t_targets,v_targets)\n#losxgb_lr_m3=train_model(XGBRegressor,params,'learning_rate',[.1,.26,.05],t_inputs,v_inputs,t_targets,v_targets)\ndel s3_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-13T18:22:34.350456Z","iopub.execute_input":"2023-01-13T18:22:34.350707Z","iopub.status.idle":"2023-01-13T18:56:36.557430Z","shell.execute_reply.started":"2023-01-13T18:22:34.350683Z","shell.execute_reply":"2023-01-13T18:56:36.555993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s3_df=dataset_sample_meter(train3_df,.35)\ndf0=s3_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':900,'objective':'reg:squarederror','max_depth':0,'learning_rate':.15}\nlosxgb_md_m3=train_model(XGBRegressor,params,'max_depth',[9,12,2],t_inputs,v_inputs,t_targets,v_targets)\n#losxgb_lr_m3=train_model(XGBRegressor,params,'learning_rate',[.1,.26,.05],t_inputs,v_inputs,t_targets,v_targets)\ndel s3_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-14T21:49:25.177385Z","iopub.execute_input":"2023-01-14T21:49:25.178287Z","iopub.status.idle":"2023-01-14T22:09:38.356685Z","shell.execute_reply.started":"2023-01-14T21:49:25.178241Z","shell.execute_reply":"2023-01-14T22:09:38.355142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params={'n_estimators':900,'objective':'reg:squarederror','learning_rate':.15,'max_depth':11}","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:06:27.558731Z","iopub.execute_input":"2023-01-16T13:06:27.559454Z","iopub.status.idle":"2023-01-16T13:06:27.568852Z","shell.execute_reply.started":"2023-01-16T13:06:27.559401Z","shell.execute_reply":"2023-01-16T13:06:27.566601Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s3_df=dataset_sample_meter(train3_df,.35)\ndf0=s3_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=XGBRegressor(n_jobs=-1,random_state=42,**params)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s3_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:06:27.570916Z","iopub.execute_input":"2023-01-16T13:06:27.571329Z","iopub.status.idle":"2023-01-16T13:14:59.033366Z","shell.execute_reply.started":"2023-01-16T13:06:27.571295Z","shell.execute_reply":"2023-01-16T13:14:59.031209Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test3_df,.06,test=True)\ntest3_df=test3_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest3_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test3_preds\n\ntest3_df=pd.merge(test3_df,wbt_df,on='row_id')\ndel wbt_df\ndel test3_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:14:59.036619Z","iopub.execute_input":"2023-01-16T13:14:59.037926Z","iopub.status.idle":"2023-01-16T13:16:04.822003Z","shell.execute_reply.started":"2023-01-16T13:14:59.037855Z","shell.execute_reply":"2023-01-16T13:16:04.820751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training for Meter 0","metadata":{}},{"cell_type":"code","source":"%%time\nwbt_df,s0_df=dataset_sample_meter(train0_df,.03)\ndf0=s0_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0,'objective':'reg:squarederror'}\nlosxgb_ne_m2=train_model(XGBRegressor,params,'n_estimators',[200,901,100],t_inputs,v_inputs,t_targets,v_targets)\ndel s0_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-13T13:13:00.956778Z","iopub.execute_input":"2023-01-13T13:13:00.957490Z","iopub.status.idle":"2023-01-13T13:46:13.404101Z","shell.execute_reply.started":"2023-01-13T13:13:00.957451Z","shell.execute_reply":"2023-01-13T13:46:13.402943Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s0_df=dataset_sample_meter(train0_df,.03)\ndf0=s0_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0,'objective':'reg:squarederror'}\nlosxgb_ne_m0=train_model(XGBRegressor,params,'n_estimators',[1000,1101,100],t_inputs,v_inputs,t_targets,v_targets)\ndel s0_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-14T20:15:03.024893Z","iopub.execute_input":"2023-01-14T20:15:03.025263Z","iopub.status.idle":"2023-01-14T20:31:49.483323Z","shell.execute_reply.started":"2023-01-14T20:15:03.025229Z","shell.execute_reply":"2023-01-14T20:31:49.482081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s0_df=dataset_sample_meter(train0_df,.03)\ndf0=s0_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':1100,'objective':'reg:squarederror','learning_rate':0}\nlosxgb_lr_m0=train_model(XGBRegressor,params,'learning_rate',[.2,.35,.065],t_inputs,v_inputs,t_targets,v_targets)\ndel s0_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-14T21:22:52.101375Z","iopub.execute_input":"2023-01-14T21:22:52.101718Z","iopub.status.idle":"2023-01-14T21:49:25.175243Z","shell.execute_reply.started":"2023-01-14T21:22:52.101685Z","shell.execute_reply":"2023-01-14T21:49:25.173995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s0_df=dataset_sample_meter(train0_df,.03)\ndf0=s0_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':1100,'objective':'reg:squarederror','learning_rate':.17}\nlosxgb_md_m0=train_model(XGBRegressor,params,'max_depth',[8,13,2],t_inputs,v_inputs,t_targets,v_targets)\ndel s0_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-14T22:33:15.715258Z","iopub.execute_input":"2023-01-14T22:33:15.716253Z","iopub.status.idle":"2023-01-14T23:20:49.596592Z","shell.execute_reply.started":"2023-01-14T22:33:15.716206Z","shell.execute_reply":"2023-01-14T23:20:49.593228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params={'n_estimators':1100,'objective':'reg:squarederror','learning_rate':.17,'max_depth':11}","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:16:04.824888Z","iopub.execute_input":"2023-01-16T13:16:04.825892Z","iopub.status.idle":"2023-01-16T13:16:04.833024Z","shell.execute_reply.started":"2023-01-16T13:16:04.825856Z","shell.execute_reply":"2023-01-16T13:16:04.830623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s0_df=dataset_sample_meter(train0_df,.03)\ndf0=s0_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=XGBRegressor(n_jobs=-1,random_state=42,**params)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s0_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:16:04.839636Z","iopub.execute_input":"2023-01-16T13:16:04.840147Z","iopub.status.idle":"2023-01-16T13:29:35.103096Z","shell.execute_reply.started":"2023-01-16T13:16:04.840113Z","shell.execute_reply":"2023-01-16T13:29:35.101424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test0_df,.06,test=True)\ntest0_df=test0_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest0_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test0_preds\n\ntest0_df=pd.merge(test0_df,wbt_df,on='row_id')\ndel wbt_df\ndel test0_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:29:35.104558Z","iopub.execute_input":"2023-01-16T13:29:35.104879Z","iopub.status.idle":"2023-01-16T13:32:58.955323Z","shell.execute_reply.started":"2023-01-16T13:29:35.104852Z","shell.execute_reply":"2023-01-16T13:32:58.953458Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test01_df,.06,test=True)\ntest01_df=test01_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest01_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test01_preds\n\ntest01_df=pd.merge(test01_df,wbt_df,on='row_id')\ndel wbt_df\ndel test01_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:32:58.957339Z","iopub.execute_input":"2023-01-16T13:32:58.958047Z","iopub.status.idle":"2023-01-16T13:36:26.352868Z","shell.execute_reply.started":"2023-01-16T13:32:58.958006Z","shell.execute_reply":"2023-01-16T13:36:26.351373Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test02_df,.06,test=True)\ntest02_df=test02_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest02_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test02_preds\n\ntest02_df=pd.merge(test02_df,wbt_df,on='row_id')\ndel wbt_df\ndel test02_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:36:26.355306Z","iopub.execute_input":"2023-01-16T13:36:26.359037Z","iopub.status.idle":"2023-01-16T13:39:51.134940Z","shell.execute_reply.started":"2023-01-16T13:36:26.358979Z","shell.execute_reply":"2023-01-16T13:39:51.132386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test03_df,.06,test=True)\ntest03_df=test03_df[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest03_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test03_preds\n\ntest03_df=pd.merge(test03_df,wbt_df,on='row_id')\ndel wbt_df\ndel test03_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:39:51.137488Z","iopub.execute_input":"2023-01-16T13:39:51.138128Z","iopub.status.idle":"2023-01-16T13:43:17.604410Z","shell.execute_reply.started":"2023-01-16T13:39:51.138080Z","shell.execute_reply":"2023-01-16T13:43:17.602555Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training sub models","metadata":{}},{"cell_type":"code","source":"num_cols=['square_feet', 'air_temperature','dew_temperature',\n 'cloud_coverage', 'precip_depth_1_hr',\n 'sea_level_pressure', 'wind_speed']\ncat_cols=['primary_use','month','weekday','hour','building_id','site_id']\ntarget_col='meter_reading'","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:40:28.010597Z","iopub.execute_input":"2023-01-16T12:40:28.011023Z","iopub.status.idle":"2023-01-16T12:40:28.017934Z","shell.execute_reply.started":"2023-01-16T12:40:28.010994Z","shell.execute_reply":"2023-01-16T12:40:28.016562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s11_df=dataset_sample_meter(train1_df_1,1)\ndf0=s11_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0}\nlosrf_ne_m1=train_model(XGBRegressor,params,'n_estimators',[100,501,100],t_inputs,v_inputs,t_targets,v_targets)\ndel s11_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T08:45:05.856323Z","iopub.execute_input":"2023-01-15T08:45:05.856851Z","iopub.status.idle":"2023-01-15T08:45:38.275160Z","shell.execute_reply.started":"2023-01-15T08:45:05.856810Z","shell.execute_reply":"2023-01-15T08:45:38.273857Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s11_df=dataset_sample_meter(train1_df_1,1)\ndf0=s11_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':400,'learning_rate':.1}\nlosxgb_lr_m11=train_model(XGBRegressor,params,'learning_rate',[.05,.26,.05],t_inputs,v_inputs,t_targets,v_targets)\ndel s11_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T08:54:27.592737Z","iopub.execute_input":"2023-01-15T08:54:27.593164Z","iopub.status.idle":"2023-01-15T08:55:11.390200Z","shell.execute_reply.started":"2023-01-15T08:54:27.593126Z","shell.execute_reply":"2023-01-15T08:55:11.388903Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s11_df=dataset_sample_meter(train1_df_1,1)\ndf0=s11_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':400,'learning_rate':.1,'max_depth':0}\nlosrf_md_m11=train_model(XGBRegressor,params,'max_depth',[6,13,2],t_inputs,v_inputs,t_targets,v_targets)\n#both site id and building id in num_cols and site_id only in cat_cols\ndel s11_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T09:03:11.742613Z","iopub.execute_input":"2023-01-15T09:03:11.743319Z","iopub.status.idle":"2023-01-15T09:04:05.957488Z","shell.execute_reply.started":"2023-01-15T09:03:11.743281Z","shell.execute_reply":"2023-01-15T09:04:05.956208Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s11_df=dataset_sample_meter(train1_df_1,1)\ndf0=s11_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=XGBRegressor(n_jobs=-1,n_estimators=400,max_depth=10,random_state=42,learning_rate=.1,objective='reg:squarederror')\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s11_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:43:11.956962Z","iopub.execute_input":"2023-01-16T12:43:11.957491Z","iopub.status.idle":"2023-01-16T12:43:24.440656Z","shell.execute_reply.started":"2023-01-16T12:43:11.957448Z","shell.execute_reply":"2023-01-16T12:43:24.438511Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test1_df_1,.06,test=True)\ntest1_df_1=test1_df_1[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest1_1preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test1_1preds\n\ntest1_df_1=pd.merge(test1_df_1,wbt_df,on='row_id')\ndel wbt_df\ndel test1_1preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:43:24.443018Z","iopub.execute_input":"2023-01-16T12:43:24.443432Z","iopub.status.idle":"2023-01-16T12:43:25.183384Z","shell.execute_reply.started":"2023-01-16T12:43:24.443397Z","shell.execute_reply":"2023-01-16T12:43:25.182408Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s21_df=dataset_sample_meter(train2_df_1,1)\ndf0=s21_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0}\nlosrf_ne_m1=train_model(XGBRegressor,params,'n_estimators',[100,501,100],t_inputs,v_inputs,t_targets,v_targets)\ndel s21_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T09:42:47.191443Z","iopub.execute_input":"2023-01-15T09:42:47.191905Z","iopub.status.idle":"2023-01-15T09:43:49.182646Z","shell.execute_reply.started":"2023-01-15T09:42:47.191869Z","shell.execute_reply":"2023-01-15T09:43:49.181287Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s21_df=dataset_sample_meter(train2_df_1,1)\ndf0=s21_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':400,'learning_rate':0}\nlosrf_lr_m21=train_model(XGBRegressor,params,'learning_rate',[.05,.26,0.05],t_inputs,v_inputs,t_targets,v_targets)\ndel s21_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T09:45:48.456601Z","iopub.execute_input":"2023-01-15T09:45:48.459134Z","iopub.status.idle":"2023-01-15T09:47:09.946290Z","shell.execute_reply.started":"2023-01-15T09:45:48.459066Z","shell.execute_reply":"2023-01-15T09:47:09.944908Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s21_df=dataset_sample_meter(train2_df_1,1)\ndf0=s21_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':400,'learning_rate':0.1}\nlosxg_md_m21=train_model(XGBRegressor,params,'max_depth',[6,13,2],t_inputs,v_inputs,t_targets,v_targets)\ndel s21_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T09:48:48.933158Z","iopub.execute_input":"2023-01-15T09:48:48.933625Z","iopub.status.idle":"2023-01-15T09:50:37.896756Z","shell.execute_reply.started":"2023-01-15T09:48:48.933586Z","shell.execute_reply":"2023-01-15T09:50:37.895529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params={'n_estimators':400,'learning_rate':0.1,'max_depth':10}","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:44:06.337775Z","iopub.execute_input":"2023-01-16T12:44:06.338230Z","iopub.status.idle":"2023-01-16T12:44:06.343883Z","shell.execute_reply.started":"2023-01-16T12:44:06.338201Z","shell.execute_reply":"2023-01-16T12:44:06.342787Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s21_df=dataset_sample_meter(train2_df_1,1)\ndf0=s21_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=XGBRegressor(n_jobs=-1,objective='reg:squarederror',random_state=42)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s21_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:44:10.378444Z","iopub.execute_input":"2023-01-16T12:44:10.379015Z","iopub.status.idle":"2023-01-16T12:44:14.358996Z","shell.execute_reply.started":"2023-01-16T12:44:10.378972Z","shell.execute_reply":"2023-01-16T12:44:14.357947Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test2_df_1,.06,test=True)\ntest2_df_1=test2_df_1[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest21_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test21_preds\n\ntest2_df_1=pd.merge(test2_df_1,wbt_df,on='row_id')\ndel wbt_df\ndel test21_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:44:14.362966Z","iopub.execute_input":"2023-01-16T12:44:14.363353Z","iopub.status.idle":"2023-01-16T12:44:14.681960Z","shell.execute_reply.started":"2023-01-16T12:44:14.363324Z","shell.execute_reply":"2023-01-16T12:44:14.680243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train3_df_1=train3_df_1.query('meter_reading < 110000')","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:44:39.390238Z","iopub.execute_input":"2023-01-16T12:44:39.390674Z","iopub.status.idle":"2023-01-16T12:44:39.404064Z","shell.execute_reply.started":"2023-01-16T12:44:39.390646Z","shell.execute_reply":"2023-01-16T12:44:39.401739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s31_df=dataset_sample_meter(train3_df_1,1)\ndf0=s31_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0}\nlosxgb_ne_m1=train_model(XGBRegressor,params,'n_estimators',[50,151,20],t_inputs,v_inputs,t_targets,v_targets)\ndel s31_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T11:33:35.491238Z","iopub.execute_input":"2023-01-15T11:33:35.491692Z","iopub.status.idle":"2023-01-15T11:34:03.452422Z","shell.execute_reply.started":"2023-01-15T11:33:35.491654Z","shell.execute_reply":"2023-01-15T11:34:03.450752Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s31_df=dataset_sample_meter(train3_df_1,1)\ndf0=s31_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':110}\nlosxgb_ne_m1=train_model(XGBRegressor,params,'learning_rate',[.05,.26,.05],t_inputs,v_inputs,t_targets,v_targets)\ndel s31_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T13:28:04.068839Z","iopub.execute_input":"2023-01-15T13:28:04.069673Z","iopub.status.idle":"2023-01-15T13:28:27.884424Z","shell.execute_reply.started":"2023-01-15T13:28:04.069623Z","shell.execute_reply":"2023-01-15T13:28:27.883226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s31_df=dataset_sample_meter(train3_df_1,1)\ndf0=s31_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':110,'learning_rate':.13}\nlosxgb_ne_m1=train_model(XGBRegressor,params,'max_depth',[6,11,2],t_inputs,v_inputs,t_targets,v_targets)\ndel s31_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T13:30:21.668712Z","iopub.execute_input":"2023-01-15T13:30:21.669151Z","iopub.status.idle":"2023-01-15T13:30:41.080584Z","shell.execute_reply.started":"2023-01-15T13:30:21.669112Z","shell.execute_reply":"2023-01-15T13:30:41.079661Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s31_df=dataset_sample_meter(train3_df_1,1)\ndf0=s31_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=XGBRegressor(n_jobs=-1,n_estimators=110,learning_rate=.13,max_depth=8,random_state=42,objective='reg:squarederror')\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s31_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:44:57.495662Z","iopub.execute_input":"2023-01-16T12:44:57.496216Z","iopub.status.idle":"2023-01-16T12:45:03.764476Z","shell.execute_reply.started":"2023-01-16T12:44:57.496173Z","shell.execute_reply":"2023-01-16T12:45:03.763676Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test3_df_1,.06,test=True)\ntest3_df_1=test3_df_1[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest31_preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test31_preds\n\ntest3_df_1=pd.merge(test3_df_1,wbt_df,on='row_id')\ndel wbt_df\ndel test31_preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:45:03.766606Z","iopub.execute_input":"2023-01-16T12:45:03.767183Z","iopub.status.idle":"2023-01-16T12:45:04.225202Z","shell.execute_reply.started":"2023-01-16T12:45:03.767148Z","shell.execute_reply":"2023-01-16T12:45:04.224225Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wbt_df,s1_df=dataset_sample_meter(train0_df_1,1)\n\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0}\nlosrf_ne_m1=train_model(XGBRegressor,params,'n_estimators',[100,501,100],t_inputs,v_inputs,t_targets,v_targets)\ndel s1_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T11:36:24.505904Z","iopub.execute_input":"2023-01-15T11:36:24.506458Z","iopub.status.idle":"2023-01-15T11:37:02.470437Z","shell.execute_reply.started":"2023-01-15T11:36:24.506411Z","shell.execute_reply":"2023-01-15T11:37:02.469459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s1_df=dataset_sample_meter(train0_df_1,1)\n\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':0,'objective':'reg:squarederror'}\nlosrf_ne_m1=train_model(XGBRegressor,params,'n_estimators',[100,501,100],t_inputs,v_inputs,t_targets,v_targets)\ndel s1_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T12:40:34.711176Z","iopub.execute_input":"2023-01-15T12:40:34.711998Z","iopub.status.idle":"2023-01-15T12:41:08.748971Z","shell.execute_reply.started":"2023-01-15T12:40:34.711956Z","shell.execute_reply":"2023-01-15T12:41:08.747862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s1_df=dataset_sample_meter(train0_df_1,1)\n\ndf0=s1_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nparams={'n_estimators':500,'objective':'reg:squarederror','max_depth':0,'learning_rate':.15}\nlosrf_ne_m1=train_model(XGBRegressor,params,'max_depth',[8,13,2],t_inputs,v_inputs,t_targets,v_targets)\ndel s1_df,df0,wbt_df","metadata":{"execution":{"iopub.status.busy":"2023-01-15T12:43:18.996656Z","iopub.execute_input":"2023-01-15T12:43:18.997181Z","iopub.status.idle":"2023-01-15T12:44:19.144234Z","shell.execute_reply.started":"2023-01-15T12:43:18.997137Z","shell.execute_reply":"2023-01-15T12:44:19.143120Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"params={'n_estimators':500,'objective':'reg:squarederror','learning_rate':0.15,'max_depth':12}","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:45:04.227014Z","iopub.execute_input":"2023-01-16T12:45:04.227755Z","iopub.status.idle":"2023-01-16T12:45:04.234496Z","shell.execute_reply.started":"2023-01-16T12:45:04.227715Z","shell.execute_reply":"2023-01-16T12:45:04.232472Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df,s01_df=dataset_sample_meter(train0_df_1,1)\ndf0=s01_df.copy()\nt_inputs,v_inputs,t_targets,v_targets=preprocess_data(df0,num_cols,cat_cols,target_col)\nmodel=XGBRegressor(n_jobs=-1,random_state=42,**params)\nmodel.fit(t_inputs,t_targets)\ndel wbt_df,s01_df,df0","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:45:09.664149Z","iopub.execute_input":"2023-01-16T12:45:09.664733Z","iopub.status.idle":"2023-01-16T12:45:31.201564Z","shell.execute_reply.started":"2023-01-16T12:45:09.664685Z","shell.execute_reply":"2023-01-16T12:45:31.199074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nwbt_df=dataset_sample_meter(test0_df_1,.06,test=True)\ntest0_df_1=test0_df_1[['row_id']]\ntest_inputs=preprocess_test_data(wbt_df,num_cols,cat_cols)\nwbt_df=wbt_df[['row_id']]\ntest0_1preds=model.predict(test_inputs)\nwbt_df['meter_reading']=test0_1preds\n\ntest0_df_1=pd.merge(test0_df_1,wbt_df,on='row_id')\ndel wbt_df\ndel test0_1preds","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:45:31.204901Z","iopub.execute_input":"2023-01-16T12:45:31.205473Z","iopub.status.idle":"2023-01-16T12:45:32.200549Z","shell.execute_reply.started":"2023-01-16T12:45:31.205424Z","shell.execute_reply":"2023-01-16T12:45:32.199410Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test0_df_2['meter_reading']=0\ntest0_df_2.loc[test0_df_2.building_id==304,'meter_reading']=.08\ntest0_df_2.loc[test0_df_2.building_id==740,'meter_reading']=.0004\ntest0_df_2=test0_df_2[['row_id','meter_reading']]","metadata":{"execution":{"iopub.status.busy":"2023-01-16T12:45:32.202086Z","iopub.execute_input":"2023-01-16T12:45:32.202438Z","iopub.status.idle":"2023-01-16T12:45:32.218170Z","shell.execute_reply.started":"2023-01-16T12:45:32.202410Z","shell.execute_reply":"2023-01-16T12:45:32.216129Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"allparts=[test0_df,test01_df,test02_df,test03_df,test0_df_1,test0_df_2,\n test1_df,test1_df_1,\n test2_df,test2_df_1,\n test3_df,test3_df_1,]\nsubmission_df=pd.concat(allparts)","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:51:33.771635Z","iopub.execute_input":"2023-01-16T13:51:33.772599Z","iopub.status.idle":"2023-01-16T13:51:34.363791Z","shell.execute_reply.started":"2023-01-16T13:51:33.772560Z","shell.execute_reply":"2023-01-16T13:51:34.361397Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.sort_values(by='row_id',inplace=True)\nsubmission_df.reset_index(drop=True)\nsubmission_df.meter_reading=np.expm1(submission_df.meter_reading)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.row_id.nunique()","metadata":{"execution":{"iopub.status.busy":"2023-01-16T13:51:45.274314Z","iopub.execute_input":"2023-01-16T13:51:45.274835Z","iopub.status.idle":"2023-01-16T13:51:48.897151Z","shell.execute_reply.started":"2023-01-16T13:51:45.274794Z","shell.execute_reply":"2023-01-16T13:51:48.895055Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.to_csv('XgbRegressor_tuned_meter_wise.csv',index=False)","metadata":{"execution":{"iopub.status.busy":"2023-01-16T14:12:06.072041Z","iopub.execute_input":"2023-01-16T14:12:06.072574Z","iopub.status.idle":"2023-01-16T14:13:21.540621Z","shell.execute_reply.started":"2023-01-16T14:12:06.072510Z","shell.execute_reply":"2023-01-16T14:13:21.538181Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Submission Score:
\n","metadata":{}},{"cell_type":"markdown","source":"Surely we need to change some strategy. We should select some model in which we can give building_id as categorical column, because there is huge variability in target columns for different building ids. One hot encoding these values lead our systems to go out of memory.\n
\n
\nWe are getting very higher scores on test dataset as compared to validation sets i.e. our data is predicting much higher results. There can be two reasons\n- Either we are removing many of the low lying values as outliers. We removed around 9% of the values that were 0. We should chek 0 meter reading in depth. That can also be the reason of getting lesser score on train and validation sets even without overfitting.\n- We are not removing high lying outliers.\n
\nThe latter reason does not make much sense as we had already removed high outliers that too after very deeply checking the values. If we would remove more of them may be our model does not generalize well. Let's try removing only and only very high outliers this time.\n
\nThe former reason seems to be very very strong as it was not mentioned anywhere that all 0 readings were erratic and will not persist for coming years. We need to check some discussions.\n
\nIt has been found that all 0 values should not be removed but if they persist for long period for any building then only we should consider removing those values as such error will not occur in coming years (may be they are technical faults which would surely be corrected while gather test data ).
\nSo Let's check for such sequences and remove those and exceptional building ids that we found during our EDA.","metadata":{}},{"cell_type":"markdown","source":"# More EDA on Training Data","metadata":{}},{"cell_type":"markdown","source":"Let's start with making some visualization which can tell us something abount density of 0 values.","metadata":{}},{"cell_type":"code","source":"# Reading the File\ntrain_df=pd.read_csv(f'/kaggle/input/ashrae-energy-prediction/train.csv',dtype=training_dtypes)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T06:11:36.793424Z","iopub.execute_input":"2023-01-27T06:11:36.794233Z","iopub.status.idle":"2023-01-27T06:11:44.785702Z","shell.execute_reply.started":"2023-01-27T06:11:36.794191Z","shell.execute_reply":"2023-01-27T06:11:44.784841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Defining a function over which the meter_reading column can be aggregated\ndef count_zeros(col):\n count = 0\n for _ in col:\n if _ == 0:\n count += 1\n return count","metadata":{"execution":{"iopub.status.busy":"2023-01-27T05:29:16.730244Z","iopub.execute_input":"2023-01-27T05:29:16.730567Z","iopub.status.idle":"2023-01-27T05:29:16.735974Z","shell.execute_reply.started":"2023-01-27T05:29:16.730540Z","shell.execute_reply":"2023-01-27T05:29:16.734899Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.timestamp=pd.to_datetime(train_df.timestamp)\ntrain_df['dayofyear'] = train_df['timestamp'].dt.dayofyear\ntrainx_df=train_df.drop('timestamp', axis=1)\ntrainx_df = trainx_df.groupby(['dayofyear', 'building_id', 'meter']).agg([count_zeros]).dropna()\ntrainx_df.reset_index(inplace=True)\ntrainx_df.columns=trainx_df.columns.droplevel(1)\ntrainx_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-01-27T05:29:16.737528Z","iopub.execute_input":"2023-01-27T05:29:16.737879Z","iopub.status.idle":"2023-01-27T05:29:35.192158Z","shell.execute_reply.started":"2023-01-27T05:29:16.737788Z","shell.execute_reply":"2023-01-27T05:29:35.191185Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So, we have well prepared our required sub dataframe which we will will use to plot a density heatmap. Also let's rename the columns meter_reading as it is no more showing us meter_reading.","metadata":{}},{"cell_type":"code","source":"trainx_df.rename(columns={'meter_reading':'count 0'}, inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T05:29:35.194033Z","iopub.execute_input":"2023-01-27T05:29:35.194331Z","iopub.status.idle":"2023-01-27T05:29:35.200174Z","shell.execute_reply.started":"2023-01-27T05:29:35.194304Z","shell.execute_reply":"2023-01-27T05:29:35.199096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.density_heatmap(trainx_df,\n x='dayofyear',\n y='building_id',\n z='count 0',\n histfunc='sum',\n nbinsx=366,\n nbinsy=1449,\n height=1000,facet_col='meter')\nfig.update_layout(xaxis={'type': 'category'})\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2023-01-27T05:29:35.201499Z","iopub.execute_input":"2023-01-27T05:29:35.201915Z","iopub.status.idle":"2023-01-27T05:29:36.741129Z","shell.execute_reply.started":"2023-01-27T05:29:35.201885Z","shell.execute_reply":"2023-01-27T05:29:36.740261Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We got huge chunk of 0 values for first 141 days of the year for building ids 0 to 104. So let's remove these values these are surely erratic. **So now considering month as category columns may not make much sense**. It may make sense if we are not considering to train different models based on meter and site id.","metadata":{}},{"cell_type":"markdown","source":"Let's check trend of meter readings for different building ids for different meters and sites.","metadata":{}},{"cell_type":"code","source":"building_df=pd.read_csv('/kaggle/input/ashrae-energy-prediction/building_metadata.csv')\n# using same variable to keep memory in control. Will delete train_df and reload train.csv\ntrain_df=train_df.merge(building_df,how='left',on='building_id')\ntrain_df=train_df[['site_id','building_id','meter','timestamp','meter_reading']] #keeping only required columns","metadata":{"execution":{"iopub.status.busy":"2023-01-27T06:12:30.530489Z","iopub.execute_input":"2023-01-27T06:12:30.530898Z","iopub.status.idle":"2023-01-27T06:12:35.915869Z","shell.execute_reply.started":"2023-01-27T06:12:30.530862Z","shell.execute_reply":"2023-01-27T06:12:35.914867Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> Defining a recursive function to have nice visualisation with plotly for all buildings in a form of grid.","metadata":{}},{"cell_type":"code","source":"def plot_reading_trend(train_df,site=0,meter=0,x=32):\n trainx_df = train_df.query(f'meter == {meter} & site_id=={site}')\n buildings= trainx_df.building_id.unique()\n # keeping data of 32 buildings only to have better visualisations\n trainy_df=trainx_df.query(f'building_id < {x} & building_id>={x-32}')\n # taking log of meter readings\n trainy_df['meter_reading_log1p']=np.log1p(trainy_df['meter_reading'])\n #grouping with sum aggregation\n trainy_df = trainy_df.groupby(['building_id','timestamp'])['meter_reading_log1p'].sum()\n #resetting the index to give parameters to plotly\n trainy_df = trainy_df.to_frame().reset_index()\n # ploting line graphs for each building id showing graphs of 8 buildings in 1 row\n fig = px.line(trainy_df, x='timestamp', y='meter_reading_log1p',facet_col='building_id',facet_col_wrap=8)\n fig.update_yaxes(title_text=\"\"); # removing x,y titles\n fig.update_xaxes(title_text=\"\");\n fig.show()\n # if there are any buildings left we again call this function\n if x= \"2016-11-3\" & timestamp <= \"2016-11-28\" )')\n df = df.merge(weather_df,how='left',left_on=['site_id','timestamp'],right_on=['site_id','timestamp'])\n df.reset_index(drop=True)\n df['timestamp']=pd.to_datetime(df['timestamp'],format=\"%Y-%m-%d %H:%M:%S\")\n \n df[\"hour\"] = df[\"timestamp\"].dt.hour\n df[\"weekday\"] = df[\"timestamp\"].dt.dayofweek\n \n return df","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:01:01.583387Z","iopub.execute_input":"2023-01-27T21:01:01.583857Z","iopub.status.idle":"2023-01-27T21:01:01.592811Z","shell.execute_reply.started":"2023-01-27T21:01:01.583809Z","shell.execute_reply":"2023-01-27T21:01:01.591473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def prepare_building(b_df):\n b_df.drop(['floor_count','year_built'],axis=1,inplace=True)\n b_df['square_feet'] = np.log1p(b_df['square_feet'])\n encoder=LabelEncoder()\n b_df['primary_use']=encoder.fit_transform(b_df.primary_use)\n return b_df","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:01:02.817967Z","iopub.execute_input":"2023-01-27T21:01:02.818339Z","iopub.status.idle":"2023-01-27T21:01:02.824970Z","shell.execute_reply.started":"2023-01-27T21:01:02.818308Z","shell.execute_reply":"2023-01-27T21:01:02.823847Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Original code from https://www.kaggle.com/gemartin/load-data-reduce-memory-usage by @gemartin\n\nfrom pandas.api.types import is_datetime64_any_dtype as is_datetime\nfrom pandas.api.types import is_categorical_dtype\n\ndef reduce_mem_usage(df, use_float16=False):\n \"\"\"\n Iterate through all the columns of a dataframe and modify the data type to reduce memory usage. \n \"\"\"\n \n start_mem = df.memory_usage().sum() / 1024**2\n print(\"Memory usage of dataframe is {:.2f} MB\".format(start_mem))\n \n for col in df.columns:\n if is_datetime(df[col]) or is_categorical_dtype(df[col]):\n continue\n col_type = df[col].dtype\n \n if col_type != object:\n c_min = df[col].min()\n c_max = df[col].max()\n if str(col_type)[:3] == \"int\":\n if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n df[col] = df[col].astype(np.int8)\n elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n df[col] = df[col].astype(np.int16)\n elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n df[col] = df[col].astype(np.int32)\n elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n df[col] = df[col].astype(np.int64) \n else:\n if use_float16 and c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n df[col] = df[col].astype(np.float16)\n elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n df[col] = df[col].astype(np.float32)\n else:\n df[col] = df[col].astype(np.float64)\n else:\n df[col] = df[col].astype(\"category\")\n\n end_mem = df.memory_usage().sum() / 1024**2\n print(\"Memory usage after optimization is: {:.2f} MB\".format(end_mem))\n print(\"Decreased by {:.1f}%\".format(100 * (start_mem - end_mem) / start_mem))\n \n return df\n","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:01:04.059554Z","iopub.execute_input":"2023-01-27T21:01:04.059951Z","iopub.status.idle":"2023-01-27T21:01:04.075777Z","shell.execute_reply.started":"2023-01-27T21:01:04.059920Z","shell.execute_reply":"2023-01-27T21:01:04.074320Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def prepare_weather(w_df):\n \n # Finding missing timestamps and adding them to weather data\n \n all_hours=list(w_df.timestamp.unique()) # getting unique timestamps\n for site_id in range(16):\n # getting hours/timestamps for the site\n site_hours = w_df[w_df['site_id'] == site_id]['timestamp']\n # Finding missing timestamps using setdiff1d\n missing_hours_df= pd.DataFrame(np.setdiff1d(all_hours,site_hours),columns=['timestamp'])\n missing_hours_df['site_id'] = site_id\n # concating missing timestaamps with weather data\n w_df = pd.concat([w_df,missing_hours_df])\n w_df = w_df.reset_index(drop=True) \n # deleting unrequired features to reduce memory\n del site_hours,missing_hours_df \n \n # converting timestamp to datetime and adding features so that we can impute missing values\n w_df['timestamp1']=pd.to_datetime(w_df.timestamp)\n w_df['hour']=w_df.timestamp1.dt.hour\n w_df['day']=w_df.timestamp1.dt.day\n w_df['month']=w_df.timestamp1.dt.month\n w_df.drop('timestamp1',axis=1,inplace=True)\n # changing index so that grouping operation can be don efficiently\n w_df.set_index(['site_id','month','day'],inplace=True)\n # Listing all weather features we want to give to our training data\n w_features=['cloud_coverage','air_temperature','dew_temperature','sea_level_pressure','precip_depth_1_hr']\n # Filling missing values based on mean for that day \n for i in (w_features):\n filler = w_df.groupby(['site_id','month','day'])[f'{i}'].mean()\n filler = pd.DataFrame(filler.fillna(method='ffill'),columns=[f\"{i}\"])\n w_df.update(filler,overwrite=False)\n w_df.reset_index(inplace=True)#resetting index \n w_features+=['site_id','timestamp'] \n w_df=w_df[w_features] # keeping only useful features \n \n return w_df","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:01:05.298191Z","iopub.execute_input":"2023-01-27T21:01:05.298576Z","iopub.status.idle":"2023-01-27T21:01:05.309815Z","shell.execute_reply.started":"2023-01-27T21:01:05.298544Z","shell.execute_reply":"2023-01-27T21:01:05.308978Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nweather_train_df=prepare_weather(weather_train_df)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:01:07.432916Z","iopub.execute_input":"2023-01-27T21:01:07.433658Z","iopub.status.idle":"2023-01-27T21:03:44.374062Z","shell.execute_reply.started":"2023-01-27T21:01:07.433577Z","shell.execute_reply":"2023-01-27T21:03:44.372838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nbuildings_df=prepare_building(building_df)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:03:44.376661Z","iopub.execute_input":"2023-01-27T21:03:44.377474Z","iopub.status.idle":"2023-01-27T21:03:44.386421Z","shell.execute_reply.started":"2023-01-27T21:03:44.377412Z","shell.execute_reply":"2023-01-27T21:03:44.385070Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\ntrain_df=prepare_main_df(train_df,weather_train_df, building_df)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:03:44.387817Z","iopub.execute_input":"2023-01-27T21:03:44.388157Z","iopub.status.idle":"2023-01-27T21:04:09.332374Z","shell.execute_reply.started":"2023-01-27T21:03:44.388127Z","shell.execute_reply":"2023-01-27T21:04:09.331073Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"useful_features=['building_id', 'meter','meter_reading', 'site_id',\n 'primary_use', 'square_feet', 'air_temperature', 'cloud_coverage',\n 'dew_temperature', 'precip_depth_1_hr','sea_level_pressure',\n 'hour', 'weekday']\ntrain_df=train_df[useful_features]","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:04:09.336008Z","iopub.execute_input":"2023-01-27T21:04:09.337006Z","iopub.status.idle":"2023-01-27T21:04:10.382742Z","shell.execute_reply.started":"2023-01-27T21:04:09.336962Z","shell.execute_reply":"2023-01-27T21:04:10.381691Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df=reduce_mem_usage(train_df, use_float16=False)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:04:10.384087Z","iopub.execute_input":"2023-01-27T21:04:10.384384Z","iopub.status.idle":"2023-01-27T21:04:12.613973Z","shell.execute_reply.started":"2023-01-27T21:04:10.384356Z","shell.execute_reply":"2023-01-27T21:04:12.612837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del weather_train_df\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:04:12.615493Z","iopub.execute_input":"2023-01-27T21:04:12.615946Z","iopub.status.idle":"2023-01-27T21:04:12.758987Z","shell.execute_reply.started":"2023-01-27T21:04:12.615903Z","shell.execute_reply":"2023-01-27T21:04:12.757842Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"target = train_df[\"meter_reading\"]\ninputs = train_df.drop('meter_reading', axis = 1)\ndel train_df\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:04:12.760656Z","iopub.execute_input":"2023-01-27T21:04:12.760972Z","iopub.status.idle":"2023-01-27T21:04:13.732857Z","shell.execute_reply.started":"2023-01-27T21:04:12.760943Z","shell.execute_reply":"2023-01-27T21:04:13.731684Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cat_cols = [\"building_id\", \"site_id\", \"meter\", \"primary_use\", \"weekday\"]\nparams = {\"learning_rate\": 0.05,\"num_leaves\": 1300,\"feature_fraction\": 0.85,\"subsample\":0.9,\n \"reg_lambda\": 2,\"metric\": \"rmse\"}\n\nkf = KFold(n_splits=3)\nmodels = []\nfor train_index,val_index in kf.split(inputs):\n train_inputs = inputs.loc[train_index]\n train_target = target.loc[train_index]\n \n val_inputs = inputs.loc[val_index]\n val_target = target.loc[val_index]\n \n train_d = lgb.Dataset(train_inputs, label=train_target,categorical_feature=cat_cols, free_raw_data=False)\n val_d = lgb.Dataset(val_inputs, label=val_target,categorical_feature=cat_cols, free_raw_data=False)\n \n model = lgb.train(params, train_set=train_d, num_boost_round=1000, valid_sets=[train_d,val_d], verbose_eval=25, early_stopping_rounds=50)\n models.append(model)\n del train_inputs, train_target, val_inputs, val_target, train_d, val_d\n gc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:04:13.734141Z","iopub.execute_input":"2023-01-27T21:04:13.734992Z","iopub.status.idle":"2023-01-27T21:20:27.922700Z","shell.execute_reply.started":"2023-01-27T21:04:13.734947Z","shell.execute_reply":"2023-01-27T21:20:27.921821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for model in models:\n lgb.plot_importance(model)\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:20:27.923821Z","iopub.execute_input":"2023-01-27T21:20:27.924710Z","iopub.status.idle":"2023-01-27T21:20:28.764752Z","shell.execute_reply.started":"2023-01-27T21:20:27.924675Z","shell.execute_reply":"2023-01-27T21:20:28.763592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df = pd.read_csv('/kaggle/input/ashrae-energy-prediction/test.csv')\nrow_ids = test_df[\"row_id\"]\ntest_df.drop(\"row_id\", axis=1, inplace=True)\ntest_df = reduce_mem_usage(test_df)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:20:28.768480Z","iopub.execute_input":"2023-01-27T21:20:28.769200Z","iopub.status.idle":"2023-01-27T21:21:05.705735Z","shell.execute_reply.started":"2023-01-27T21:20:28.769155Z","shell.execute_reply":"2023-01-27T21:21:05.704527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"weather_test_df = pd.read_csv('/kaggle/input/ashrae-energy-prediction/weather_test.csv')\nweather_test_df = prepare_weather(weather_test_df)\nweather_test_df = reduce_mem_usage(weather_test_df)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:21:05.706983Z","iopub.execute_input":"2023-01-27T21:21:05.707294Z","iopub.status.idle":"2023-01-27T21:31:33.666502Z","shell.execute_reply.started":"2023-01-27T21:21:05.707266Z","shell.execute_reply":"2023-01-27T21:31:33.665236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df=prepare_main_df(test_df, weather_test_df, building_df,train=False)\ndel weather_test_df,building_df","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"useful_features.remove('meter_reading')\nrow_id=test_df.row_id\ntest_df=test_df[useful_features]","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:35:35.064583Z","iopub.execute_input":"2023-01-27T21:35:35.065045Z","iopub.status.idle":"2023-01-27T21:35:37.690395Z","shell.execute_reply.started":"2023-01-27T21:35:35.065005Z","shell.execute_reply":"2023-01-27T21:35:37.689190Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preds = []\nfor model in models:\n if preds == []:\n preds = np.expm1(model.predict(test_df, num_iteration=model.best_iteration))/3\n preds=np.array(preds)\n else:\n preds += np.expm1(model.predict(test_df, num_iteration=model.best_iteration))/3\n del model\n gc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-01-27T21:37:41.524456Z","iopub.execute_input":"2023-01-27T21:37:41.524914Z","iopub.status.idle":"2023-01-27T22:07:28.568479Z","shell.execute_reply.started":"2023-01-27T21:37:41.524877Z","shell.execute_reply":"2023-01-27T22:07:28.566996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df=pd.read_csv(\"/kaggle/input/ashrae-energy-prediction/sample_submission.csv\")\nsubmission_df['meter_reading'] = preds","metadata":{"execution":{"iopub.status.busy":"2023-01-27T22:12:37.485411Z","iopub.execute_input":"2023-01-27T22:12:37.485875Z","iopub.status.idle":"2023-01-27T22:12:49.860249Z","shell.execute_reply.started":"2023-01-27T22:12:37.485839Z","shell.execute_reply":"2023-01-27T22:12:49.859000Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission_df.head(10)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T22:54:31.966943Z","iopub.execute_input":"2023-01-27T22:54:31.969078Z","iopub.status.idle":"2023-01-27T22:54:31.991848Z","shell.execute_reply.started":"2023-01-27T22:54:31.969013Z","shell.execute_reply":"2023-01-27T22:54:31.990918Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nsubmission_df.to_csv(\"LGBM Submissionn.csv\",index=False)","metadata":{"execution":{"iopub.status.busy":"2023-01-27T22:13:22.044696Z","iopub.execute_input":"2023-01-27T22:13:22.045108Z","iopub.status.idle":"2023-01-27T22:15:01.245757Z","shell.execute_reply.started":"2023-01-27T22:13:22.045074Z","shell.execute_reply":"2023-01-27T22:15:01.243637Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Submission Score:-
\n","metadata":{}},{"cell_type":"markdown","source":"# Summary\nBest results were obtained using **Light GBM which lands us in top 38% of Leaderboard** of this Kaggle competition.\n
\nThree different Regression models were used\n- Random Forest Regressor\n- XGBoost Regressor\n- Light GBM Regressor\n
After training each model different techniques were used to modify and improve Training Data to get best results on Test Data and attempts remained successful each time.\n
\n- It was observed that with lower learning rates and increasing other hyperparameters of the model resulted in improvement of results.\n- LGBM proved out to be Most effective in terms of time cost.\n- It was huge dataset we had regularly remove variables occupying huge size","metadata":{}},{"cell_type":"markdown","source":"# Future Work\n- More EDA can be performed to spot and remove erratic target values for different buildings\n- More Feature Engineering can be done. Values in weather data are highly fluctuating, treating those columns may help in model improvement. [See here](https://towardsdatascience.com/splines-for-feature-selection-and-data-smoothing-python-ed5ad31436d5)\n- More Hyperparameter Tuning in deeper levels can be implemented to Final model. (Gridsearch CV can be used to tune more than parameters at Once )\n- After final modifications, training different Models for different meters and different sites may lead to significant improvement in score.\n- Other GBM's such as Catboost,Adaboost etc. can also be tried for more accuracy.\n","metadata":{}},{"cell_type":"markdown","source":"# References\n- https://jovian.ai/learn/zero-to-data-analyst-bootcamp/lesson/workshop-machine-learning-project-from-scratch\n- https://medium.com/analytics-vidhya/root-mean-square-log-error-rmse-vs-rmlse-935c6cc1802a\n- http://seaborn.pydata.org/\n- https://plotly.com/python/\n- https://www.geeksforgeeks.org/\n- https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor\n- https://jovian.ai/learn/zero-to-data-analyst-bootcamp/lesson/decision-trees-and-random-forests\n- https://jovian.ai/learn/zero-to-data-analyst-bootcamp/lesson/gradient-boosting-machines-with-xgboost\n- https://xgboost.readthedocs.io/en/stable/\n- https://machinelearningmastery.com/xgboost-for-regression/\n- https://lightgbm.readthedocs.io/en/v3.3.2/\n- https://towardsdatascience.com/hyperparameter-tuning-to-reduce-overfitting-lightgbm-5eb81a0b464e\n- https://neptune.ai/blog/lightgbm-parameters-guide\n- https://www.kaggle.com/competitions/ashrae-energy-prediction/discussion\n- https://towardsdatascience.com/splines-for-feature-selection-and-data-smoothing-python-ed5ad31436d5","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}