{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Building energy consumption prediction using linear regression\n## The main goal of this Notebook is to predict the energy consumption of different buildings using linear regression. "},{"metadata":{},"cell_type":"markdown","source":"### First all packages are imported"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport tensorflow as tf #AI models\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### The following function is used to reduce the RAM usage of the notebook. Initially it was discovered that the notebook would run out of RAM and reset. This function in combination with deleting unwanted variables, choosing the right data to load and adding/combining data carefully solved the issue of RAM usage. The reduce_mem_usage function is taken from this source: https://www.kaggle.com/gemartin/load-data-reduce-memory-usage "},{"metadata":{"trusted":true},"cell_type":"code","source":"def reduce_mem_usage(df):\n    \"\"\" iterate through all the columns of a dataframe and modify the data type\n        to reduce memory usage.        \n    \"\"\"\n    start_mem = df.memory_usage().sum() / 1024**2\n    print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))\n    \n    for col in df.columns:\n        col_type = df[col].dtype\n        \n        if col_type != object:\n            c_min = df[col].min()\n            c_max = df[col].max()\n            if str(col_type)[:3] == 'int':\n                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                    df[col] = df[col].astype(np.int8)\n                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                    df[col] = df[col].astype(np.int16)\n                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                    df[col] = df[col].astype(np.int32)\n                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                    df[col] = df[col].astype(np.int64)  \n            else:\n                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n                    df[col] = df[col].astype(np.float16)\n                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n                    df[col] = df[col].astype(np.float32)\n                else:\n                    df[col] = df[col].astype(np.float64)\n        #else:\n            #df[col] = df[col].astype('category')\n\n    end_mem = df.memory_usage().sum() / 1024**2\n    print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))\n    print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))\n    \n    return df\n\n\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Next, all necesssary files needed to train the model are loaded. train_data contains the meter_readings, which we want to predict. It also contains the different meater types:\n* 0: electricity\n* 1: chilled water\n* 2: steam\n* 3: hot water\n\n### The file building_metadata contains information about each site and all the buildings in each site. Here only the columns that are needed to merge the file with train_data, as well as columns that are believed to have predictive power, are loaded. \"primary_use\", \"square_feet\" and \"year built\" are believed to have predictive power. The primary use of the buildings, such as education, office, public service etc.  most likely all have different energy consumption patters. As I discovered in the EDA, there are mostly buildings used in educational purposes (see figure below). \n\n### The weather train data contains all conditions present for every meater reading. Just like before, only columns thought to have predictive power are loaded to save RAM.\n\n### Please note that I have tried to decrease the RAM usage as much as possible, but I still had to exclude some variables in order to not restart the notebook before results could be obtained."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train_data = reduce_mem_usage(pd.read_csv(\"../input/ashrae-energy-prediction/train.csv\"))\nbuilding_metadata = reduce_mem_usage(pd.read_csv(\"../input/ashrae-energy-prediction/building_metadata.csv\", usecols=[\"site_id\", \"building_id\", \"primary_use\", \"square_feet\", \"year_built\"]))\nweather_train_data = reduce_mem_usage(pd.read_csv(\"../input/ashrae-energy-prediction/weather_train.csv\", usecols= [\"site_id\", \"timestamp\", \"air_temperature\"]))\n\n\n\n#CHANGE THIS CELL\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"building_metadata[\"primary_use\"].value_counts().plot.bar(figsize = (14,5), xlabel = \"Primary use\", ylabel = \"Number of buildings\", fontsize = 10, rot = 90, title = \"Number of buildings used for a particual reason\")\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### The next step is to merge the three files in to one. The \"meter\" values (0, 1, 2, 3) are changed to string-type so that it can be one-hot encoded and used as a catergorical feature. Note that there is a lot of code which is not in use. This is code that I have expermiented with and want to keep in case I figure out new and better ways of creating my model."},{"metadata":{"trusted":true},"cell_type":"code","source":"#Merging building_metadata and weather                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              _train in 2 steps\nmerged_train = train_data.merge(building_metadata, left_on = \"building_id\", right_on = \"building_id\", how = \"left\")\nmerged_train = merged_train.merge(weather_train_data, left_on = [\"site_id\", \"timestamp\"], right_on = [\"site_id\", \"timestamp\"], how = \"left\")\n\nmerged_train[\"meter\"] = merged_train[\"meter\"].astype(str)\n#merged_train[\"meter\"].replace({0: 1, 1: 2, 2: 3, 3: 4}, inplace=True)\n#merged_train.head()\n\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### In my exploratory data analysis, I discovered that some measurements were suspiciously high. After analyzing each meter type, I found that meter 2 (steam) was responsible for the unusually high values. After this discovery, each site was analyzed, and I found that site 13 was the anomay. The first graph shows the mean hourly steam readings for site 13. The second graph shows all meter readings for all sites. It is clear that the readings are much larger in the first graph. Also, the shape of the graph is dictated by site 13."},{"metadata":{"trusted":true},"cell_type":"code","source":"suspicious_data = merged_train_1 = merged_train.loc[(merged_train[\"site_id\"]==13) & (merged_train[\"meter\"]==\"2\")]\nsuspicious_data.groupby(by = \"timestamp\").mean().filter([\"timestamp\", \"meter_reading\"]).plot(figsize =(15,7), ylabel = \"mean meter readings\", title = \"mean hourly steam readings, site 13 (index 2)\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"merged_train.groupby(by = \"timestamp\").mean().filter([\"timestamp\", \"meter_reading\"]).plot(figsize =(15,7), ylabel = \"mean meter readings\", title = \"mean hourly meter readings (all meters)\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Now, the suspicious site_id is removed, as it is believed that the model will perform better overall."},{"metadata":{"trusted":true},"cell_type":"code","source":"merged_train = merged_train.loc[(merged_train[\"site_id\"]!=13) & (merged_train[\"meter\"]!=3)]\nmerged_train.groupby(by = \"timestamp\").mean().filter([\"timestamp\", \"meter_reading\"]).plot(figsize =(15,7), ylabel = \"mean hourly meter readings\", title = \"mean hourly meter readings (all meters)\")\n\n\n#divide the timestamp column in to 4 new columns\n#merged_train[\"timestamp\"] = (merged_train[\"timestamp\"].dt.month.astype(str) + merged_train[\"timestamp\"].dt.weekday.astype(str) + merged_train[\"timestamp\"].dt.day.astype(str) + merged_train[\"timestamp\"].dt.hour.astype(str))\n#merged_train[\"week\"] = merged_train[\"timestamp\"].dt.weekday.astype(str)\n#merged_train[\"day\"] = merged_train[\"timestamp\"].dt.day.astype(str)\n#merged_train[\"hour\"] = merged_train[\"timestamp\"].dt.hour.astype(str)\n\n#merged_train.drop([\"timestamp\"], axis =1, inplace=True)\n#merged_test[\"month\"] = merged_test[\"timestamp\"].dt.month\n#merged_test[\"weekday\"] = merged_test[\"timestamp\"].dt.weekday\n#merged_test[\"day\"] = merged_test[\"timestamp\"].dt.day\n#merged_test[\"hour\"] = merged_test[\"timestamp\"].dt.hour","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Now it can be seen that the meter readings are not affected by site 13"},{"metadata":{},"cell_type":"markdown","source":"### Next, the column with all the timestamps is changed to datetime datatype. This is done so that the timestamps can be divided in to month, weekday and hour. This is done because energy consumption can vary depending on the month (empty office buildings during vacation times, empty buildings used for educational purposes inbetween semester periods etc.). Energy consumption can also vary depending on the weekday, e.g. office/education buildings that are empty in the weekend. The daily energy consumption is most likely lower during the night."},{"metadata":{"trusted":true},"cell_type":"code","source":"print(merged_train[\"timestamp\"].dtype)\nmerged_train[\"timestamp\"] = pd.to_datetime(merged_train[\"timestamp\"])\nprint(merged_train[\"timestamp\"].dtype)\n\nmerged_train[\"month\"] = merged_train[\"timestamp\"].dt.month\nmerged_train[\"hour\"] = merged_train[\"timestamp\"].dt.hour\nmerged_train[\"weekday\"] = merged_train[\"timestamp\"].dt.weekday","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### The following funtion is taken from class, and is used to convert the dataframe to a tensor.\n\nSource: https://www.kaggle.com/christophertessum/module-9-class-2-airplanes"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Convert a Pandas series to a tensor.\ndef convert_to_tensor(s):\n    dt = s.dtype\n    if dt == \"float64\" or dt == \"int64\" or dt == \"float32\" or dt == \"float16\" or dt == \"int32\" or dt == \"int16\" or dt == \"int8\":\n        a = np.asarray(s).astype(\"float32\")\n        a = np.nan_to_num(a, nan=a[~np.isnan(a)].mean())\n        return (a - a.mean()) / a.std()\n    elif dt == \"object\":\n        return s\n    return None\n    del dt\n    del s\n    del a\n    import gc\n    gc.collect()\n# A utility method to create a tf.data dataset from a Pandas Dataframe\n# Adapted from https://www.tensorflow.org/tutorials/structured_data/feature_columns\ndef df_to_dataset(dataframe, target_name):\n    data_dict = {}\n    for col in dataframe.columns:\n        t = convert_to_tensor(dataframe[col])\n        if col == target_name:\n            labels = t\n        elif t is not None:\n            data_dict[col] = t\n    ds = tf.data.Dataset.from_tensor_slices((data_dict, labels))\n    return ds\n    del ds\n    del dat_dict\n    del t\n    del labels\n    import gc\n    gc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### In the following cell, the different columns are prepared to be used in the model. Initially, the following columns were used:\n* meter\n* weekday\n* meter_reading (the value we want to predict)\n* square_feet\n* year_built\n* air_temperature\n* hour\n* month\n\n### These values worked during the model training, but when new values were predicted, categorical values  did not work with model.predict(*test data*). If I put it this way: after trying to predict new values with categorical columns, I found out that the maximum runtime for a Kaggle notebook is 9 hours..\n\n### After this setback, I tried to use only numerical values. "},{"metadata":{"trusted":true},"cell_type":"code","source":"\n#Preparing the data\n#df_subset = merged_train[[\"meter\", \"weekday\", \"meter_reading\", \"square_feet\", \"year_built\", \"air_temperature\", \"hour\", \"month\"]]\ndf_subset = merged_train[[\"air_temperature\", \"meter_reading\", \"square_feet\", \"year_built\", \"weekday\", \"hour\", \"month\"]]\ndata = df_to_dataset(df_subset, \"meter_reading\")\n\n#Here the categorical data is one-hot encoded\n\n#primary_use = tf.feature_column.categorical_column_with_vocabulary_list(key = \"primary_use\", vocabulary_list = df_subset[\"primary_use\"].unique())\n#primary_use = tf.feature_column.indicator_column(primary_use)\n\n#meter = tf.feature_column.categorical_column_with_vocabulary_list(\"meter\", df_subset[\"meter\"].unique())\n#meter = tf.feature_column.indicator_column(meter)\n\n#timestamp = tf.feature_column.categorical_column_with_vocabulary_list(\"timestamp\", df_subset[\"timestamp\"].unique())\n#timestamp = tf.feature_column.indicator_column(timestamp)\n\n#Here numerical columns are added\nmeter = tf.feature_column.numeric_column(\"meter\")\nweekday = tf.feature_column.numeric_column(\"weekday\")\nhour = tf.feature_column.numeric_column(\"hour\")\nsquare_feet = tf.feature_column.numeric_column(\"square_feet\")\nyear_built =tf.feature_column.numeric_column(\"year_built\")\nair_temperature = tf.feature_column.numeric_column(\"air_temperature\")\nmonth = tf.feature_column.numeric_column(\"month\")\n\nfeature_layer = tf.keras.layers.DenseFeatures([weekday, square_feet, year_built, air_temperature, hour, month])\n#feature_layer = tf.keras.layers.DenseFeatures([square_feet, year_built, air_temperature])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### This function for creating a linear regression model is taken from class. I tried to add a kernel regularizer, but it did not seem to make a difference.\n\nSource: https://www.kaggle.com/christophertessum/module-9-class-2-airplanes"},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\ndef create_model(learning_rate, feature_layer):\n    # Sequential model\n    model = tf.keras.models.Sequential()\n\n    # Here the feature_layer is added\n    model.add(feature_layer)\n     \n    # Here another layer is added to create a linear regression model\n    model.add(tf.keras.layers.Dense(units=1))    \n    \n    #Optional L2 kernel regularizer\n    #model.add(tf.keras.layers.Dense(1, kernel_regularizer=tf.keras.regularizers.L2(0.2)))\n\n    # Construct the layers into a model that TensorFlow can execute.\n    model.compile(optimizer=tf.keras.optimizers.Adam(lr=learning_rate),\n                loss=\"mean_squared_error\",\n                metrics=[tf.keras.metrics.RootMeanSquaredError()])\n    return model\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Hyperparameters\nlearning_rate = 0.001\nepochs = 10\nbatch_size = 10000\n\nmodel = create_model(learning_rate, feature_layer)\nmodel.fit(data.batch(batch_size),\n                  epochs=epochs, shuffle=True)\n\n# The list of epochs is stored separately from the rest of history.\n#epochs = history.epoch\n\n# Isolate the mean absolute error for each epoch.\n#hist = pd.DataFrame(history.history)\n#rmse = hist[\"root_mean_squared_error\"]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### The following cell deletes unnecessary variables to free up RAM. However, depending on the size of the feature layer used in the model, this might not be necessary."},{"metadata":{"trusted":true},"cell_type":"code","source":"#Delete unnecessary data\nimport gc\n\ndel feature_layer\ndel data\ndel meter\ndel weekday\n#del primary_use\ndel square_feet\ndel year_built\ndel air_temperature\n\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### The following cell repeats the previous steps, but with the test datasets. The names have been kept to make it easier to copy and paste, since I changed many things to try and get a better score."},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data = reduce_mem_usage(pd.read_csv(\"../input/ashrae-energy-prediction/test.csv\"))\nbuilding_metadata = reduce_mem_usage(pd.read_csv(\"../input/ashrae-energy-prediction/building_metadata.csv\", usecols=[\"site_id\", \"building_id\", \"primary_use\", \"square_feet\", \"year_built\"]))\nweather_train_data = reduce_mem_usage(pd.read_csv(\"../input/ashrae-energy-prediction/weather_test.csv\", usecols= [\"site_id\", \"timestamp\", \"air_temperature\"]))\n\n#Merging building_metadata and weather                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              _train in 2 steps\nmerged_train = train_data.merge(building_metadata, left_on = \"building_id\", right_on = \"building_id\", how = \"left\")\nmerged_train = merged_train.merge(weather_train_data, left_on = [\"site_id\", \"timestamp\"], right_on = [\"site_id\", \"timestamp\"], how = \"left\")\n\nmerged_train[\"meter\"].replace({0: 1, 1: 2, 2: 3, 3: 4}, inplace=True)\n#merged_train[\"meter\"] = merged_train[\"meter\"].astype(str)\n#merged_train.head()\n\n\n#merged_train = merged_train.loc[(merged_train[\"site_id\"]!=13) & (merged_train[\"meter\"]!=\"2\")]\n#merged_train.groupby(by = \"timestamp\").mean().filter([\"timestamp\", \"meter_reading\"]).plot(figsize =(15,7), ylabel = \"mean meter readings\", title = \"mean hourly chilled water readings (index 1)\")\n#print(merged_train[\"timestamp\"].dtype)\nmerged_train[\"timestamp\"] = pd.to_datetime(merged_train[\"timestamp\"])\n#print(merged_train[\"timestamp\"].dtype)\n\nmerged_train[\"month\"] = merged_train[\"timestamp\"].dt.month\nmerged_train[\"hour\"] = merged_train[\"timestamp\"].dt.hour\nmerged_train[\"weekday\"] = merged_train[\"timestamp\"].dt.weekday\n\n\n#divide the timestamp column in to 4 new columns\n#merged_train[\"timestamp\"] = (merged_train[\"timestamp\"].dt.month.astype(str) + merged_train[\"timestamp\"].dt.weekday.astype(str) + merged_train[\"timestamp\"].dt.day.astype(str) + merged_train[\"timestamp\"].dt.hour.astype(str))\n#merged_train[\"week\"] = merged_train[\"timestamp\"].dt.weekday.astype(str)\n#merged_train[\"day\"] = merged_train[\"timestamp\"].dt.day.astype(str)\n#merged_train[\"hour\"] = merged_train[\"timestamp\"].dt.hour.astype(str)\n\n#merged_train.drop([\"timestamp\"], axis =1, inplace=True)\n#merged_test[\"month\"] = merged_test[\"timestamp\"].dt.month\n#merged_test[\"weekday\"] = merged_test[\"timestamp\"].dt.weekday\n#merged_test[\"day\"] = merged_test[\"timestamp\"].dt.day\n#merged_test[\"hour\"] = merged_test[\"timestamp\"].dt.hour\n\n\ndel train_data\ndel building_metadata\ndel weather_train_data\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Option to save the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"#model.save(\"./\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### The final steps to make a Kaggle submission. Like I stated previously, I could not figure out how to include categorical values in my model.predict function. "},{"metadata":{"trusted":true},"cell_type":"code","source":"#merged_train[\"row_id\"] =merged_train.index\nids = merged_train[\"row_id\"]\n#df_subset = merged_train[[\"meter\", \"timestamp\", \"row_id\", \"primary_use\", \"square_feet\", \"year_built\", \"air_temperature\"]]\n#data = df_to_dataset(df_subset, \"row_id\")\n#[[\"meter\", \"timestamp\", \"meter_reading\", \"primary_use\", \"square_feet\", \"year_built\", \"air_temperature\", \"hour\"]]\nmerged_train[\"meter\"] = convert_to_tensor(merged_train[\"meter\"])\nmerged_train[\"weekday\"] = convert_to_tensor(merged_train[\"weekday\"])\n#merged_train[\"primary_use\"] = convert_to_tensor(merged_train[\"primary_use\"])\nmerged_train[\"square_feet\"] = convert_to_tensor(merged_train[\"square_feet\"])\nmerged_train[\"year_built\"] = convert_to_tensor(merged_train[\"year_built\"])\nmerged_train[\"air_temperature\"] = convert_to_tensor(merged_train[\"air_temperature\"])\nmerged_train[\"hour\"] = convert_to_tensor(merged_train[\"hour\"])\nmerged_train[\"month\"] = convert_to_tensor(merged_train[\"month\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predicted_readings = model.predict({\"weekday\": merged_train.weekday, \"square_feet\":merged_train.square_feet, \"year_built\": merged_train.year_built,\"air_temperature\":merged_train.air_temperature, \"hour\":merged_train.hour, \"month\": merged_train.month})\n#predicted_readings = model.predict({\"air_temperature\":merged_train.air_temperature, \"square_feet\": merged_train.square_feet, \"year_built\":merged_train.year_built})\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"predict = pd.DataFrame(predicted_readings)\npredict[\"row_id\"] = ids\npredict[\"meter_reading\"] = pd.DataFrame(predicted_readings)\n\npredict.drop([0], axis = 1, inplace = True)\npredict.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Here the submission file is created that I manually submit to the competition"},{"metadata":{"trusted":true},"cell_type":"code","source":"predict.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Final conclusion: My score obtained is approx 4,3 no matter what changes I make. Is this the limit for linear regression? Most likely not. In this notebook, several things can be made better. First, the datasets can be optimized even better. I have removed site_id 13 because the values were suspiciously high. In this case, an even more in-depth \"cleaning\" can be made and locade the exact building_id (or several building_ids) that is responsible for the data anomaly. \n\n### Furthermore, if I can manage to pretict a model that includes categorical values, perhaps the score can be improved. \n"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}