{"cells":[{"metadata":{},"cell_type":"markdown","source":"Hello\n\nThe Kernel will conduct Data Loading and a simple Data EDA of the ASHRAE-Great Energy Predictor III competition.\n\nI will keep working on various tasks later, and hope it will help all of you."},{"metadata":{},"cell_type":"markdown","source":"## 0. Config"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"#-*- coding: CP949 -*-\n%matplotlib inline\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\nimport warnings\nwarnings.filterwarnings('ignore')\n\nimport matplotlib\nfrom IPython.display import set_matplotlib_formats\n\nmatplotlib.rc('font', family='Malgun Gothic')\n\nmatplotlib.rc('axes', unicode_minus=False)\n\nset_matplotlib_formats('retina')\n\nfrom IPython.core.interactiveshell import InteractiveShell\nInteractiveShell.ast_node_interactivity = \"all\"\n\n%config IPCompleter.greedy=True","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## 1. Loading Data\n\n### 1.1. Loading Train Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"../input/ashrae-energy-prediction/train.csv\")\ntrain.shape\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Building ID Count : \",train['building_id'].nunique())\nprint(\"Building ID Min Value : \",train['building_id'].min())\nprint(\"Building ID Max Value : \",train['building_id'].max())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 'building_id' is a unique ID of each building, and there are 1449 building information in Train Data."},{"metadata":{"trusted":true},"cell_type":"code","source":"tmp = train['meter'].value_counts(dropna=False)\ntmp.plot.bar()\ndel tmp","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 'meter' feature is an integer value between 0 and 3, and each number represents the amount of power shown below.\n  - 0 : electricity\n  - 1 : chilledwater\n  - 2 : steam\n  - 3 : hotwater\n  \n\n\n* It has the most information on 'electricity' and the least amount of 'hotwater'."},{"metadata":{"trusted":true},"cell_type":"code","source":"tmp = train.sort_values(by = 'timestamp' )\ntmp.head()\ntmp.tail()\ndel tmp","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 'timestamp' column is the time when each measurement data is acquired.\n* There is data from January 1, 2016 00:00 to December 31, 2016 23:00.\n* You can see that there is 1 year data in 2016.\n\n\n* The 'meter_reading' column is the target value we want to find, the building ID, the meter and the power consumption at that time"},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"### 1.2. Loading Building Metadata"},{"metadata":{"trusted":true},"cell_type":"code","source":"building_metadata = pd.read_csv(\"../input/ashrae-energy-prediction/building_metadata.csv\")\n\nbuilding_metadata.shape\nbuilding_metadata.head()\n\nprint(\"SIte ID Count : \",building_metadata['site_id'].nunique())\nprint(\"SIte ID Min Value : \",building_metadata['site_id'].min())\nprint(\"SIte ID Max Value : \",building_metadata['site_id'].max())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 'site_id' is the ID of the area where each building is located.\n* It is divided into 16 regions"},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Building ID Count : \",building_metadata['building_id'].nunique())\nprint(\"Building ID Min Value : \",building_metadata['building_id'].min())\nprint(\"Building ID Max Value : \",building_metadata['building_id'].max())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 'building_id' has a total of 1449 building information identical to the building ID of train data.\n* Using this column, we can integrate the building information into the Train Data later."},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.rcParams[\"figure.figsize\"] = (18,8)\ntmp = building_metadata['primary_use'].value_counts(dropna=False)\ntmp\ntmp.plot.bar(rot = 45)\ndel tmp","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 'primary_use' is for each building use.\n* The use of the building can be used as a feature because it can affect the power consumption."},{"metadata":{"trusted":true},"cell_type":"code","source":"pd.isnull( building_metadata[\"square_feet\"] ).sum()\nbuilding_metadata[\"square_feet\"].describe()\n\nsns.distplot( building_metadata[\"square_feet\"] , bins = 500 )\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 'square_feet' is the gross floor area for each building.\n* The values below 200000 are the majority.\n* I think the total floor area will also be an important feature."},{"metadata":{"trusted":true},"cell_type":"code","source":"print( \"No Data Row Count :\" , pd.isnull( building_metadata[\"year_built\"] ).sum() )\nbuilding_metadata[\"year_built\"].describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tmp = building_metadata[\"year_built\"].dropna()\nsns.distplot( tmp , bins = 120 , kde=False)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 'year_built' is the year the building was built.\n* A lot of buildings built in the 60's and 70's.\n* You can calculate the built year and use it as a feature."},{"metadata":{"trusted":true},"cell_type":"code","source":"print( \"No Data Row Count :\" , pd.isnull( building_metadata[\"floor_count\"] ).sum() )\nbuilding_metadata[\"floor_count\"].describe()\n\nbuilding_metadata[\"floor_count\"].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tmp = building_metadata[\"floor_count\"].dropna()\nsns.distplot( tmp  , kde=False)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* There are many buildings with no floor_count.\n* Let's check out the features of buildings without floors later.\n* Most buildings have with 10 floors or less."},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"### 1.3. Loading Weather Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"weather_test = pd.read_csv(\"../input/ashrae-energy-prediction/weather_test.csv\")\nweather_train = pd.read_csv(\"../input/ashrae-energy-prediction/weather_train.csv\")\n\nweather_train.shape\nweather_train.head()\n\nweather_test.shape\nweather_test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Site ID Count : \" , weather_train['site_id'].nunique() )\nweather_train['site_id'].value_counts(dropna=False)\n\nprint(\"Site ID Count : \" , weather_test['site_id'].nunique() )\nweather_test['site_id'].value_counts(dropna=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* There are 16 Site IDs, same as the Site ID in the Building Metadata above.\n* It is data recorded for one year, surveying the climate data at every hour hour for each Site ID.\n* Climate can play an important role in power consumption, so this data can be an important feature."},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"   "},{"metadata":{},"cell_type":"markdown","source":"### 1.4. Loading Test & Submission Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission = pd.read_csv(\"../input/ashrae-energy-prediction/sample_submission.csv\")\ntest = pd.read_csv(\"../input/ashrae-energy-prediction/test.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.shape\ntest.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tmp = test.sort_values(by = 'timestamp')\ntmp.head()\ntmp.tail()\ndel tmp","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission.shape\nsample_submission.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 'test' data has the building ID, meter and time data that we are going to predict in this competition.\n* It has the same structure as the 'train' Data we saw earlier, but, of course, there is no 'meterr_reading' column and the timestamp has a range from 00 January 1, 2017 to 23 December 2018.\n* We have to predict power consumption for two years based on the previous train data and other data."},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Test Data Building ID Count : \",test['building_id'].nunique())\nprint(\"Test Data Building ID Min Value : \",test['building_id'].min())\nprint(\"Test Data Building ID Max Value : \",test['building_id'].max())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 'Building ID' of 'train' and 'test' data also has the same value as we expected."},{"metadata":{},"cell_type":"markdown","source":"The next Kernel will carry out Data Cleaning and Feature Engineering.\n\nPlease wait for the next Kernel\n\nThank you for reading"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}