{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np \nimport pandas as pd \nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom plotnine import *\nfrom sklearn.preprocessing import StandardScaler, LabelEncoder\nfrom sklearn.model_selection import train_test_split, KFold\nimport lightgbm as lgb\nfrom bayes_opt import BayesianOptimization\nfrom sklearn.metrics import mean_squared_log_error\nimport os\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dirc = \"/kaggle/input/ashrae-energy-prediction/\"\ntrain = pd.read_csv(dirc + 'train.csv')\nweather_train = pd.read_csv(dirc + 'weather_train.csv')\ndf_building = pd.read_csv(dirc + 'building_metadata.csv')\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Train data shape:', train.shape,'\\nTrain weather data shape:', weather_train.shape, '\\nBuilding data shape:', df_building.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### train.csv\n- `building_id` - Foreign key for the building metadata.\n- `meter` - The meter id code. Read as <span style='color:blue;background:yellow'>`{0: electricity, 1: chilledwater, 2: steam, hotwater: 3}`. Not every building has all meter types.\n- `timestamp` - When the measurement was taken\n- `meter_reading` - The target variable. Energy consumption in kWh (or equivalent). Note that this is real data with measurement error, which we expect will impose a baseline level of modeling error.\n### building_meta.csv\n- `site_id` - Foreign key for the weather files.\n- `building_id` - Foreign key for training.csv\n- `primary_use` - Indicator of the primary category of activities for the building based on EnergyStar property type definitions\n- `square_feet` - Gross floor area of the building\n- `year_built` - Year building was opened\n- `floor_count` - Number of floors of the building\n### weather_[train/test].csv\n- Weather data from a meteorological station as close as possible to the site.\n\n- `site_id`\n- `air_temperature` - Degrees Celsius\n- `cloud_coverage` - Portion of the sky covered in clouds, in oktas\n- `dew_temperature` - Degrees Celsius\n- `precip_depth_1_hr` - Millimeters\n- `sea_level_pressure` - Millibar/hectopascals\n- `wind_direction` - Compass direction (0-360)\n- `wind_speed` - Meters per second\n\n### test.csv\n- The submission files use row numbers for ID codes in order to save space on the file uploads. test.csv has no feature data; it exists so you can get your predictions into the correct order.\n\n- `row_id` - Row id for your submission file\n- `building_id` - Building id code\n- `meter` - The meter id code\n- `timestamp` - Timestamps for the test data period\n### sample_submission.csv\n- A valid sample submission.\n\n- All floats in the solution file were truncated to four decimal places; we recommend you do the same to save space on your file upload.\nThere are gaps in some of the meter readings for both the train and test sets. Gaps in the test set are not revealed or scored."},{"metadata":{},"cell_type":"markdown","source":"## Train Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.meter.unique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"(train.meter.value_counts('percent')*100).plot(kind='bar')\nprint((train.meter.value_counts('percent')*100))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- 59.66 % meters are type 0 (electricity)\n- Type 3 (hotwater) meters are used least. "},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12,9))\nsns.boxplot(x='meter', y='meter_reading', data=train)\nplt.xlabel('Meter type', fontsize=19, color='blue')\nplt.ylabel('Meter Reading', fontsize=19, color='blue')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Meter type 2 (steam) has maximum meter reading."},{"metadata":{},"cell_type":"markdown","source":"> ## Building data"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_building.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_building.isnull().sum(axis=0)/df_building.shape[0]*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- 53.41%  half values of `year_build` column and 75.50% values of `floor_count` are missing."},{"metadata":{"trusted":true},"cell_type":"code","source":"data = df_building.primary_use.value_counts('percent')*100\ndata.sort_values(ascending=True, inplace=True)\nfig, ax = plt.subplots(figsize=(12,9))\ndata.plot(kind='barh')\nfor i, j in enumerate(data.values):\n    plt.text(j, i-0.25, str(round(j,2))+ '%', fontsize=16, color='k')\nplt.yticks(fontsize=17, color='b')\nfig.set_facecolor('yellow')\nax.set_facecolor('pink')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- 57% buildings are used for Educational and office purpose. "},{"metadata":{"trusted":true},"cell_type":"code","source":"df_building.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12,9))\nsns.boxplot(x='primary_use', y='square_feet', data=df_building)\nplt.xticks(rotation=90)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Median of area for parking is highest."},{"metadata":{},"cell_type":"markdown","source":"## Train weather data"},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_train.isnull().sum(axis=0)/weather_train.shape[0]*100","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Significant values are missing in `cloud_coverage` and `precip_depth_1_hr` columns.\n"},{"metadata":{},"cell_type":"markdown","source":"# Merge data\n### 1. Merge train and building data on `building_id`\n### 3. Merge 1 with train weather data on `site_id` and `timestamp`\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.merge(train, df_building, on='building_id', how='left')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_train['timestamp1'] = pd.to_datetime(weather_train.timestamp)\nweather_train['month'] = np.uint8(weather_train.timestamp1.apply(lambda x:x.month))\nweather_train['dom'] = np.uint8(weather_train.timestamp1.apply(lambda x:x.day))\nweather_train['dow'] = np.uint8(weather_train.timestamp1.apply(lambda x:x.weekday()))\nweather_train['hour'] = np.uint8(weather_train.timestamp1.apply(lambda x:x.hour))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del weather_train['timestamp1'] ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.merge(train, weather_train, on=['site_id', 'timestamp'], how='left')\ndel weather_train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from bokeh.models import Panel, Tabs\nfrom bokeh.plotting import figure\nfrom bokeh.io import output_notebook, show\noutput_notebook()\n\ncolor_map = {\n    \"air_temperature\": \"yellow\",\n    \"dew_temperature\": \"brown\",\n    \"sea_level_pressure\": \"green\",\n    \"wind_speed\": \"red\",\n    \"cloud_coverage\": \"blue\",\n}\n\ncol_map = {\n    \"air_temperature\": \"Air Temperature\",\n    \"dew_temperature\": \"Dew Temperature\",\n    \"sea_level_pressure\": \"Sea Level Pressure\",\n    \"wind_speed\": \"Wind Speed\",\n    \"cloud_coverage\": \"Cloud Coverage\",\n}\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_bar_plot_by_site(df,By):\n    def get_plots(data, col, color, By):\n        p = figure(plot_width=1000, plot_height=350, title=f\"Mean of {col} by {By}\")\n        p.vbar(data[By], top=data[col], color=color, width=0.5)\n        return p\n    main_tabs_list = []\n    cols = [\"air_temperature\",\"dew_temperature\", \"sea_level_pressure\", \"wind_speed\", \"cloud_coverage\"]\n    for col in cols:\n        tab_list = []\n        for site in range(16):\n            temp = df[df[\"site_id\"]==site]\n            temp = temp.groupby(['site_id', By])[col].agg({col:'mean'})\n            temp.reset_index(inplace=True)\n            p = get_plots(temp, col, color_map[col],By)\n            tab = Panel(child=p, title=f\"Site:{site}\")\n            tab_list.append(tab)\n        tabs = Tabs(tabs=tab_list)\n        panel = Panel(child=tabs, title=col_map[col])\n        main_tabs_list.append(panel)\n\n    tabs = Tabs(tabs=main_tabs_list)\n    show(tabs)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"get_bar_plot_by_site(train, 'month')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"get_bar_plot_by_site(train, 'dom')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# <center><span style='color:red;backgroud:yellow'>  Stay tunned! More coming up </span></center>\n#  <center><span style='color:red;backgroud:yellow'>  ||-------------------------------If like it, Upvote Please -------------------------------||</span></center>"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}