{"metadata":{"kernelspec":{"display_name":"Python 3 (ipykernel)","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.3"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":9994,"databundleVersionId":752467,"sourceType":"competition"},{"sourceId":10807989,"sourceType":"datasetVersion","datasetId":6708953}],"dockerImageVersionId":30886,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **Checkpoint 2**\n\nGroup Name: **Eagle_USU**","metadata":{}},{"cell_type":"markdown","source":"Substantial investments are being made to enhance building efficiency, reducing costs and emissions. But are these improvements truly effective? That’s where this competition plays a role. Under pay-for-performance financing, building owners pay based on the difference between their actual energy consumption and what they would have used without retrofits. This baseline consumption must be estimated using a model.\n\nIn this competition, our goal is to develop precise models for metered energy usage in buildings, covering chilled water, electricity, hot water, and steam meters. The dataset includes information from over 1,000 buildings spanning three years. More accurate estimates of energy savings will encourage large-scale investors and financial institutions to support advancements in building efficiency.\n\nParticipants will work with two primary datasets—train.csv and test.csv—similar to other competitions. Additionally, the following supporting files are provided:\n\t•\tbuilding_metadata.csv – Contains metadata about the buildings\n\t•\tweather.csv – Provides weather-related information\n\nNotebook Objective:\n\nThis notebook aims to explore the dataset, perform preprocessing, and generate initial predictions.","metadata":{}},{"cell_type":"markdown","source":"## Loading the dataset","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\n\npath_data = \"ashrae_data/\"\n\ntrain_data = pd.read_csv(path_data + 'train.csv')\ntest_data = pd.read_csv(path_data + 'test.csv')\nbuilding_metadata = pd.read_csv(path_data + 'building_metadata.csv')\nweather_train = pd.read_csv(path_data + 'weather_train.csv')\nweather_test = pd.read_csv(path_data + 'weather_test.csv')\nsample_submission = pd.read_csv(path_data + 'sample_submission.csv')","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.execute_input":"2025-03-18T00:45:43.399918Z","iopub.status.busy":"2025-03-18T00:45:43.399006Z","iopub.status.idle":"2025-03-18T00:46:18.340818Z","shell.execute_reply":"2025-03-18T00:46:18.339472Z","shell.execute_reply.started":"2025-03-18T00:45:43.399879Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Showing the dataset","metadata":{}},{"cell_type":"code","source":"display(train_data.head())\ndisplay(test_data.head())\ndisplay(sample_submission)","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:46:18.342781Z","iopub.status.busy":"2025-03-18T00:46:18.342430Z","iopub.status.idle":"2025-03-18T00:46:18.367387Z","shell.execute_reply":"2025-03-18T00:46:18.366360Z","shell.execute_reply.started":"2025-03-18T00:46:18.342737Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"display(building_metadata.head())","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:46:18.369573Z","iopub.status.busy":"2025-03-18T00:46:18.369192Z","iopub.status.idle":"2025-03-18T00:46:18.388800Z","shell.execute_reply":"2025-03-18T00:46:18.387380Z","shell.execute_reply.started":"2025-03-18T00:46:18.369526Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"display(weather_train.head())","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:46:18.390960Z","iopub.status.busy":"2025-03-18T00:46:18.390515Z","iopub.status.idle":"2025-03-18T00:46:18.421648Z","shell.execute_reply":"2025-03-18T00:46:18.420575Z","shell.execute_reply.started":"2025-03-18T00:46:18.390928Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"display(weather_test.head())","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:46:18.423084Z","iopub.status.busy":"2025-03-18T00:46:18.422718Z","iopub.status.idle":"2025-03-18T00:46:18.452823Z","shell.execute_reply":"2025-03-18T00:46:18.451577Z","shell.execute_reply.started":"2025-03-18T00:46:18.423046Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"display(sample_submission.head())","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:46:18.454466Z","iopub.status.busy":"2025-03-18T00:46:18.454003Z","iopub.status.idle":"2025-03-18T00:46:18.472625Z","shell.execute_reply":"2025-03-18T00:46:18.471535Z","shell.execute_reply.started":"2025-03-18T00:46:18.454420Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Exploratory Data Analysis","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Set style for better visualization\nsns.set_style(\"whitegrid\")\n\n### 1. Class Distribution of Unique Buildings\nplt.figure(figsize=(12, 5))\nsns.countplot(x=building_metadata[\"site_id\"], palette=\"coolwarm\")\nplt.xlabel(\"Site ID\")\nplt.ylabel(\"Count\")\nplt.title(\"Class Distribution of Unique Buildings\")\nplt.show()\n\n### 2. Class Distribution of Unique Meters\n# Mapping meter types to categories\nmeter_mapping = {0: \"Electricity\", 1: \"ChilledWater\", 2: \"Steam\", 3: \"HotWater\"}\ntrain_data[\"meter_category\"] = train_data[\"meter\"].map(meter_mapping)\ntest_data[\"meter_category\"] = test_data[\"meter\"].map(meter_mapping)\ntrain_data.drop(columns=[\"meter\"], inplace=True)\ntest_data.drop(columns=[\"meter\"], inplace=True)\n\nplt.figure(figsize=(8, 5))\nsns.countplot(x=train_data[\"meter_category\"], palette=\"viridis\", order=[\"Electricity\", \"ChilledWater\", \"Steam\", \"HotWater\"])\nplt.xlabel(\"Meter Type\")\nplt.ylabel(\"Count\")\nplt.title(\"Class Distribution of Unique Meters\")\nplt.show()\n\n### 3. Class Distribution of Unique Primary Use of Buildings\nplt.figure(figsize=(12, 5))\nsns.countplot(y=building_metadata[\"primary_use\"], palette=\"magma\", order=building_metadata[\"primary_use\"].value_counts().index)\nplt.xlabel(\"Count\")\nplt.ylabel(\"Primary Use Category\")\nplt.title(\"Class Distribution of Unique Primary Use of Buildings\")\nplt.show()","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:46:18.474259Z","iopub.status.busy":"2025-03-18T00:46:18.473836Z","iopub.status.idle":"2025-03-18T00:46:28.939478Z","shell.execute_reply":"2025-03-18T00:46:28.938223Z","shell.execute_reply.started":"2025-03-18T00:46:18.474219Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# 1. Number of unique buildings\nnum_unique_buildings = building_metadata[\"building_id\"].nunique()\nprint(f\"Number of Unique Buildings: {num_unique_buildings}\")\n\n# 2. Number of unique meters\n# num_unique_meters = train_data[\"meter\"].nunique()\n# print(f\"Number of Unique Meters: {num_unique_meters}\")\n\n# 3. Number of unique primary use of buildings\nnum_unique_primary_use = building_metadata[\"primary_use\"].nunique()\nprint(f\"Number of Unique Primary Use Categories: {num_unique_primary_use}\")\n\n# 4. Distribution of square footage of buildings\nplt.figure(figsize=(12, 5))\nsns.histplot(building_metadata[\"square_feet\"].dropna(), bins=50, kde=True, color=\"blue\")\nplt.xlabel(\"Square Feet\")\nplt.ylabel(\"Frequency\")\nplt.title(\"Distribution of Building Square Footage\")\nplt.show()\n\n# 5. Distribution of year built of buildings\nplt.figure(figsize=(12, 5))\nsns.histplot(building_metadata[\"year_built\"].dropna(), bins=50, kde=True, color=\"green\")\nplt.xlabel(\"Year Built\")\nplt.ylabel(\"Frequency\")\nplt.title(\"Distribution of Building Construction Year\")\nplt.show()\n\n# 6. Distribution of unique site IDs\nplt.figure(figsize=(8, 5))\nsns.countplot(x=building_metadata[\"site_id\"], palette=\"coolwarm\")\nplt.xlabel(\"Site ID\")\nplt.ylabel(\"Count\")\nplt.title(\"Distribution of Unique Site IDs\")\nplt.show()\n\n# 7. Correlation Heatmap for Weather Data (Dropping Non-Numeric Columns)\nweather_train_numeric = weather_train.select_dtypes(include=[np.number])  # Keep only numeric columns\n\nplt.figure(figsize=(10, 6))\nsns.heatmap(weather_train_numeric.corr(), annot=True, cmap=\"coolwarm\", fmt=\".2f\")\nplt.title(\"Correlation Heatmap of Weather Variables\")\nplt.show()\n\n# 8. Distribution plots for selected weather features\nweather_features = [\"air_temperature\", \"cloud_coverage\", \"dew_temperature\", \"precip_depth_1_hr\",\n                    \"sea_level_pressure\", \"wind_direction\", \"wind_speed\"]\n\nfor feature in weather_features:\n    plt.figure(figsize=(8, 5))\n    sns.histplot(weather_train[feature].dropna(), bins=50, kde=True, color=\"purple\")\n    plt.xlabel(feature.replace(\"_\", \" \").title())\n    plt.ylabel(\"Frequency\")\n    plt.title(f\"Distribution of {feature.replace('_', ' ').title()}\")\n    plt.show()","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:46:28.942969Z","iopub.status.busy":"2025-03-18T00:46:28.942432Z","iopub.status.idle":"2025-03-18T00:46:36.949108Z","shell.execute_reply":"2025-03-18T00:46:36.947946Z","shell.execute_reply.started":"2025-03-18T00:46:28.942936Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Preprocessing","metadata":{}},{"cell_type":"markdown","source":"### Merging the data","metadata":{}},{"cell_type":"code","source":"def prepare_data(X, building_data, weather_data, test=False):\n    \"\"\"\n    Preparing vfinal dataset with all features.\n    \"\"\"\n    \n    X = X.merge(building_data, on=\"building_id\", how=\"left\")\n    X = X.merge(weather_data, on=[\"site_id\", \"timestamp\"], how=\"left\")\n    \n    X.timestamp = pd.to_datetime(X.timestamp, format=\"%Y-%m-%d %H:%M:%S\")\n    X.square_feet = np.log1p(X.square_feet)\n    \n    if not test:\n        X.sort_values(\"timestamp\", inplace=True)\n        X.reset_index(drop=True, inplace=True)\n        \n    holidays = [\"2016-01-01\", \"2016-01-18\", \"2016-02-15\", \"2016-05-30\", \"2016-07-04\",\n                \"2016-09-05\", \"2016-10-10\", \"2016-11-11\", \"2016-11-24\", \"2016-12-26\",\n                \"2017-01-01\", \"2017-01-16\", \"2017-02-20\", \"2017-05-29\", \"2017-07-04\",\n                \"2017-09-04\", \"2017-10-09\", \"2017-11-10\", \"2017-11-23\", \"2017-12-25\",\n                \"2018-01-01\", \"2018-01-15\", \"2018-02-19\", \"2018-05-28\", \"2018-07-04\",\n                \"2018-09-03\", \"2018-10-08\", \"2018-11-12\", \"2018-11-22\", \"2018-12-25\",\n                \"2019-01-01\"]\n    \n    X[\"hour\"] = X.timestamp.dt.hour\n    X[\"weekday\"] = X.timestamp.dt.weekday\n    X[\"is_holiday\"] = (X.timestamp.dt.date.astype(\"str\").isin(holidays)).astype(int)\n    \n    drop_features = [\"timestamp\", \"sea_level_pressure\", \"wind_direction\", \"wind_speed\"]\n\n    X.drop(drop_features, axis=1, inplace=True)\n\n    if test:\n        row_ids = X.row_id\n        X.drop(\"row_id\", axis=1, inplace=True)\n        return X, row_ids\n    else:\n        y = np.log1p(X.meter_reading)\n        X.drop(\"meter_reading\", axis=1, inplace=True)\n        return X, y","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:46:36.951090Z","iopub.status.busy":"2025-03-18T00:46:36.950735Z","iopub.status.idle":"2025-03-18T00:46:36.960646Z","shell.execute_reply":"2025-03-18T00:46:36.959421Z","shell.execute_reply.started":"2025-03-18T00:46:36.951060Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_full, y_train_full = prepare_data(train_data, building_metadata, weather_train)\n\n# Applying One Hot Encoding to Categorical Features\ntrain_full = pd.get_dummies(train_full, columns=[\"primary_use\"])\ntrain_full = pd.get_dummies(train_full, columns=[\"meter_category\"])\n\ndisplay(y_train_full)","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:46:36.961875Z","iopub.status.busy":"2025-03-18T00:46:36.961598Z","iopub.status.idle":"2025-03-18T00:47:27.886949Z","shell.execute_reply":"2025-03-18T00:47:27.885687Z","shell.execute_reply.started":"2025-03-18T00:46:36.961850Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"display(train_full)","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_full, row_ids = prepare_data(test_data, building_metadata, weather_test, test=True)\n\n# Applying One Hot Encoding to Categorical Features\ntest_full = pd.get_dummies(test_full, columns=[\"primary_use\"])\ntest_full = pd.get_dummies(test_full, columns=[\"meter_category\"])\n\ndisplay(test_full)","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:47:27.888689Z","iopub.status.busy":"2025-03-18T00:47:27.888313Z","iopub.status.idle":"2025-03-18T00:48:58.042968Z","shell.execute_reply":"2025-03-18T00:48:58.041775Z","shell.execute_reply.started":"2025-03-18T00:47:27.888660Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Missing Value Imputation, Normalization, and Feature Selection","metadata":{}},{"cell_type":"code","source":"from scipy.stats import zscore\n\ndef normalize_features(df):\n    \"\"\"\n    Normalize each feature individually using Z-score normalization.\n    \"\"\"\n    numeric_cols = df.select_dtypes(include=[np.number]).columns  # Select numerical columns\n    df[numeric_cols] = df[numeric_cols].apply(zscore)  # Apply Z-score normalization\n    return df\n\n\ndef impute_missing_values(df):\n    \"\"\"\n    Perform mean imputation for missing values in numeric features.\n    \"\"\"\n    numeric_cols = df.select_dtypes(include=[np.number]).columns  # Select numerical columns\n    df[numeric_cols] = df[numeric_cols].apply(lambda x: x.fillna(x.mean()))  # Mean imputation\n    return df\n\n\ndef remove_dew_temperature(df):\n    \"\"\"\n    Remove the feature 'dew_temperature' since it has correlation with 'site_id'.\n    \"\"\"\n    if \"dew_temperature\" in df.columns:\n        df.drop(columns=[\"dew_temperature\"], inplace=True)\n    return df\n\n\n# Apply preprocessing functions to train and test datasets\ntrain_full = impute_missing_values(train_full)\ntrain_full = normalize_features(train_full)\ntrain_full = remove_dew_temperature(train_full)\n\ntest_full = impute_missing_values(test_full)\ntest_full = normalize_features(test_full)\ntest_full = remove_dew_temperature(test_full)","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:48:58.044438Z","iopub.status.busy":"2025-03-18T00:48:58.044099Z","iopub.status.idle":"2025-03-18T00:49:53.837970Z","shell.execute_reply":"2025-03-18T00:49:53.836807Z","shell.execute_reply.started":"2025-03-18T00:48:58.044407Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"display(train_full)","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Dummy Regression","metadata":{}},{"cell_type":"code","source":"mean_meter_reading = y_train_full.mean()\nprint(mean_meter_reading)","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:49:53.839618Z","iopub.status.busy":"2025-03-18T00:49:53.839332Z","iopub.status.idle":"2025-03-18T00:49:53.937114Z","shell.execute_reply":"2025-03-18T00:49:53.935923Z","shell.execute_reply.started":"2025-03-18T00:49:53.839591Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"num_rows = len(test_full)  # Get the number of rows in test_full\nmean_meter_reading = y_train_full.mean()  # Compute the mean meter reading\n\n# Create the submission DataFrame\nsubmission_df = pd.DataFrame({\n    \"row_id\": range(num_rows),  # Sequential row IDs\n    \"meter_reading\": mean_meter_reading  # Fill with mean meter reading\n})\n\n# Save to CSV file\nsubmission_df.to_csv(\"submission_dummy.csv\", index=False)\n\nprint(\"CSV file 'submission_dummy.csv' created successfully!\")","metadata":{"execution":{"iopub.execute_input":"2025-03-18T00:49:53.938576Z","iopub.status.busy":"2025-03-18T00:49:53.938260Z","iopub.status.idle":"2025-03-18T00:51:13.355360Z","shell.execute_reply":"2025-03-18T00:51:13.354336Z","shell.execute_reply.started":"2025-03-18T00:49:53.938549Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Simple Linear Regression","metadata":{}},{"cell_type":"code","source":"train_full_np = np.array(train_full, dtype=float)\ny_train_full_np = np.array(y_train_full, dtype=float)\ntest_full_np = np.array(test_full, dtype=float)","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print (train_full_np.shape[1])","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Building the model\nm = np.zeros(train_full_np.shape[1])\nc = 0\n\nL = 0.01  # The learning Rate\nepochs = 30  # The number of iterations to perform gradient descent\n\nn = float(len(y_train_full_np))  # Number of elements in y_train_full_np\n\n# Performing Gradient Descent \nfor i in range(epochs): \n    Y_pred = np.dot(train_full_np, m) + c  # The current predicted value of Y\n    D_m = (-2/n) * np.dot(train_full_np.T, (y_train_full_np - Y_pred))  # Derivative wrt m\n    D_c = (-2/n) * np.sum(y_train_full_np - Y_pred)  # Derivative wrt c\n    m = m - L * D_m  # Update m\n    c = c - L * D_c  # Update c\n    \nprint(m, c)\n","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Predicting the meter readings for the test set\nY_pred_test = np.dot(test_full_np, m) + c\n\n# Converting the predictions back from log scale\nY_pred_test = np.expm1(Y_pred_test)\n\n# Calculating the Mean Squared Error for the training set\nY_pred_train = np.dot(train_full_np, m) + c\ntrain_error = np.mean((y_train_full_np - Y_pred_train)**2)\nprint(\"Training MSE:\", train_error)\n\n# Creating the submission DataFrame\nsubmission_df = pd.DataFrame({\n    \"row_id\": row_ids,\n    \"meter_reading\": Y_pred_test\n})\n\n# Save to CSV file\nsubmission_df.to_csv(\"submission_simpleLinear.csv\", index=False)\n\nprint(\"CSV file 'submission_simpleLinear.csv' created successfully!\")","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Lasso Regression","metadata":{}},{"cell_type":"code","source":"from sklearn.linear_model import SGDRegressor\n\n# Lasso using Stochastic Gradient Descent\nlasso_sgd = SGDRegressor(penalty=\"l1\", alpha=0.001, eta0=0.01, learning_rate=\"constant\", max_iter=30)\nlasso_sgd.fit(train_full_np, y_train_full_np)\n\nprint(\"Coefficients:\", lasso_sgd.coef_)\nprint(\"Intercept:\", lasso_sgd.intercept_)\n\n# Predicting the meter readings for the training set\nY_pred_train = lasso_sgd.predict(train_full_np)\n\n# Calculating the Mean Squared Error for the training set\ntrain_error = np.mean((y_train_full_np - Y_pred_train)**2)\nprint(\"Training MSE:\", train_error)\n\n\n# Predicting the meter readings for the test set\nY_pred_test = lasso_sgd.predict(test_full_np)\n\n# Converting the predictions back from log scale\nY_pred_test = np.expm1(Y_pred_test)\n\n# Creating the submission DataFrame\nsubmission_df = pd.DataFrame({\n    \"row_id\": row_ids,\n    \"meter_reading\": Y_pred_test\n})\n\n# Save to CSV file\nsubmission_df.to_csv(\"submission_lasso.csv\", index=False)\n\nprint(\"CSV file 'submission_lasso.csv' created successfully!\")","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Ridge Regression","metadata":{}},{"cell_type":"code","source":"from sklearn.linear_model import SGDRegressor\n\nridge_sgd = SGDRegressor(penalty=\"l2\", alpha=0.001, learning_rate=\"constant\", eta0=0.01, max_iter=30)\nridge_sgd.fit(train_full_np, y_train_full_np)\n\nprint(\"Coefficients:\", ridge_sgd.coef_)\nprint(\"Intercept:\", ridge_sgd.intercept_)\n# Predicting the meter readings for the training set\nY_pred_train = ridge_sgd.predict(train_full_np)\n\n# Calculating the Mean Squared Error for the training set\ntrain_error = np.mean((y_train_full_np - Y_pred_train)**2)\nprint(\"Training MSE:\", train_error)\n\n# Predicting the meter readings for the test set\nY_pred_test = ridge_sgd.predict(test_full_np)\n\n# Converting the predictions back from log scale\nY_pred_test = np.expm1(Y_pred_test)\n\n# Creating the submission DataFrame\nsubmission_df = pd.DataFrame({\n    \"row_id\": row_ids,\n    \"meter_reading\": Y_pred_test\n})\n\n# Save to CSV file\nsubmission_df.to_csv(\"submission_ridge.csv\", index=False)\n\nprint(\"CSV file 'submission_ridge.csv' created successfully!\")","metadata":{},"outputs":[],"execution_count":null}]}