{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":61145,"databundleVersionId":7047669,"sourceType":"competition"}],"dockerImageVersionId":30558,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Example Notebook 2: Minimal Viable ML Example\n \nThis notebook shows a simple example how the values of all four columns can be predicted. This example shows the complete process from reading the data to submitting the predctions. Remember these are just some ideas, feel free to play with the data try different combination of features!\n \n ## Table of Content\n \n * [Read data](#Read_data)\n * [Interpolate health data](#Interpolate_health_data)\n * [Interpolate weather data](#Interpolate_weather_data)\n * [Remove rows and columns](#Remove_rows_and_columns)\n * [Split data](#Split_data)\n * [Data preparation](#Data_preparation)\n * [Example prediction](#Example_prediction)\n     * [Prediction `q_noise_nearby`](#Predict_q_noise_nearby)\n     * [Prediction `q_noise_kind`](#Predict_q_noise_kind)\n     * [Prediction `q_earphones`](#Predict_q_earphones)\n     * [Prediction `q_thermal_preference`](#Predict_q_thermal_preference)\n * [Predict any column](#Predict_any_column)\n * [Compute overall score](#Compute_overall_score)\n * [Create submission file](#Create_submission_file)","metadata":{"papermill":{"duration":0.017817,"end_time":"2023-10-23T14:39:35.343608","exception":false,"start_time":"2023-10-23T14:39:35.325791","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import pandas as pd \nimport numpy as np\nfrom xgboost import XGBClassifier\nfrom sklearn.metrics import accuracy_score, f1_score\nfrom math import cos, asin, sqrt, pi\npd.set_option('display.max_columns', None)","metadata":{"papermill":{"duration":2.203153,"end_time":"2023-10-23T14:39:37.563797","exception":false,"start_time":"2023-10-23T14:39:35.360644","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:12:43.554931Z","iopub.execute_input":"2023-11-13T22:12:43.555347Z","iopub.status.idle":"2023-11-13T22:12:45.723770Z","shell.execute_reply.started":"2023-11-13T22:12:43.555312Z","shell.execute_reply":"2023-11-13T22:12:45.722272Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Read_data\"></a>\n# Read data\n\nRead all csv file into dataframes. (This might take a few moments)","metadata":{"papermill":{"duration":0.016124,"end_time":"2023-10-23T14:39:37.596445","exception":false,"start_time":"2023-10-23T14:39:37.580321","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df_train = pd.read_csv('/kaggle/input/cool-quiet-city-competition/cozie_responses_and_physiological_data_training.csv', index_col=[0], keep_default_na=False, na_values=[''], parse_dates=['time', 'ws_timestamp_location', 'ws_timestamp_start'], date_format='%Y-%m-%d %H:%M:%S.%f%z')\ndf_solution = pd.read_csv('/kaggle/input/cool-quiet-city-competition/cozie_responses_and_physiological_data_test_public.csv', index_col=[0], keep_default_na=False, na_values=[''], parse_dates=['time', 'ws_timestamp_location', 'ws_timestamp_start'], date_format='%Y-%m-%d %H:%M:%S.%f%z')\ndf_rain = pd.read_csv('/kaggle/input/cool-quiet-city-competition/weather_rainfall.csv', parse_dates=True, index_col=[0])\ndf_temperature = pd.read_csv('/kaggle/input/cool-quiet-city-competition/weather_air-temperature.csv', parse_dates=True, index_col=[0])\ndf_humidity = pd.read_csv('/kaggle/input/cool-quiet-city-competition/weather_relative-humidity.csv', parse_dates=True, index_col=[0])\ndf_wind_speed = pd.read_csv('/kaggle/input/cool-quiet-city-competition/weather_wind-speed.csv', parse_dates=True, index_col=[0])\ndf_wind_direction = pd.read_csv('/kaggle/input/cool-quiet-city-competition/weather_wind-direction.csv', parse_dates=True, index_col=[0])\ndf_stations = pd.read_csv('/kaggle/input/cool-quiet-city-competition/weather_stations.csv').drop(columns=['Unnamed: 0'])","metadata":{"papermill":{"duration":96.702776,"end_time":"2023-10-23T14:41:14.315418","exception":false,"start_time":"2023-10-23T14:39:37.612642","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:12:45.726201Z","iopub.execute_input":"2023-11-13T22:12:45.727097Z","iopub.status.idle":"2023-11-13T22:15:31.399081Z","shell.execute_reply.started":"2023-11-13T22:12:45.727045Z","shell.execute_reply":"2023-11-13T22:15:31.397763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Combine training data (`df_train`) and data needed to predict the solution (`df_solution`) for more convenient data processing.","metadata":{"papermill":{"duration":0.017612,"end_time":"2023-10-23T14:41:14.351574","exception":false,"start_time":"2023-10-23T14:41:14.333962","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df_train['source']='train'\ndf_solution['source']='solution'\ndf_all = pd.concat([df_train, df_solution])","metadata":{"papermill":{"duration":0.457812,"end_time":"2023-10-23T14:41:14.827405","exception":false,"start_time":"2023-10-23T14:41:14.369593","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:15:31.400554Z","iopub.execute_input":"2023-11-13T22:15:31.400924Z","iopub.status.idle":"2023-11-13T22:15:32.685927Z","shell.execute_reply.started":"2023-11-13T22:15:31.400890Z","shell.execute_reply":"2023-11-13T22:15:32.684924Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Interpolate_health_data\"></a>\n# Interpolate health data\n\nHealth data, e.g., `ts_heart_reate`, and the watch survey responses, e.g., `q_thermal_preference`, do not have coinciding timestamps. Hence, the health data needs to be interpolated at the timestamps of the watch survey responses.\n\nThe interpolated values are stored in columns with the prefix `ws_` which stands for 'watch survey', e.g., the interpolated values of `ts_heart_rate` are stored in `ws_heart_rate`.\n\n\n**Improvement suggestion 1 :** \n\nIt might be worth debating whether some of the health data, e.g., `ts_stand_time`, `ts_step_count`, and `ts_walking_distance` should be aggregated, e.g., summed up over time (cumsum) before being interpolated.\nAlternatively an hourly mean could also be an interesting apporach.\n\n**Improvement suggestion 2 :** \n\nCurrently, the interpolation is looking for the nearest value. However, other approaches could be used, e.g., linear interpolation.\n","metadata":{"papermill":{"duration":0.016358,"end_time":"2023-10-23T14:41:14.861107","exception":false,"start_time":"2023-10-23T14:41:14.844749","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Inspect dataset with raw values\ndf_all[(df_all.id_participant=='xesh001') & ((df_all.ws_survey_count.notna()) | (df_all.ts_heart_rate))][['id_participant', 'ws_survey_count', 'ts_heart_rate']].head(10)","metadata":{"papermill":{"duration":0.245946,"end_time":"2023-10-23T14:41:15.124005","exception":false,"start_time":"2023-10-23T14:41:14.878059","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:15:32.688532Z","iopub.execute_input":"2023-11-13T22:15:32.688978Z","iopub.status.idle":"2023-11-13T22:15:33.430090Z","shell.execute_reply.started":"2023-11-13T22:15:32.688948Z","shell.execute_reply":"2023-11-13T22:15:33.428864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Note, in the cell above the column `ws_survey_count` used as a place holder for watch survey responses. `ws_survey_count` is present in all watch survey response rows. This is not necessarily true for other columns of watch survey responses. Hence, you can filter for watch survey responses with `df[df.ws_survey_response.notna()]`, which will show all rows with watch survey responses.","metadata":{"papermill":{"duration":0.016178,"end_time":"2023-10-23T14:41:15.157065","exception":false,"start_time":"2023-10-23T14:41:15.140887","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Function to resample health data at the timestamps of watch survey responses.\ndef resample_health(df, col_name, col_name_new):\n    df[col_name_new] = df[col_name].interpolate(method='nearest').bfill().ffill().mask(df['ws_survey_count'].isna())\n    return df","metadata":{"papermill":{"duration":0.028328,"end_time":"2023-10-23T14:41:15.202082","exception":false,"start_time":"2023-10-23T14:41:15.173754","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:15:33.432008Z","iopub.execute_input":"2023-11-13T22:15:33.432498Z","iopub.status.idle":"2023-11-13T22:15:33.439780Z","shell.execute_reply.started":"2023-11-13T22:15:33.432456Z","shell.execute_reply":"2023-11-13T22:15:33.438310Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Resample health data\ndf_all = df_all.groupby('id_participant').apply(resample_health, col_name='ts_heart_rate', col_name_new='ws_heart_rate').reset_index(level=[0], drop=True)\ndf_all = df_all.groupby('id_participant').apply(resample_health, col_name='ts_audio_exposure_environment', col_name_new='ws_audio_exposure_environment').reset_index(level=[0], drop=True)\ndf_all = df_all.groupby('id_participant').apply(resample_health, col_name='ts_oxygen_saturation', col_name_new='ws_oxygen_saturation').reset_index(level=[0], drop=True)\ndf_all = df_all.groupby('id_participant').apply(resample_health, col_name='ts_resting_heart_rate', col_name_new='ws_resting_heart_rate').reset_index(level=[0], drop=True)\ndf_all = df_all.groupby('id_participant').apply(resample_health, col_name='ts_stand_time', col_name_new='ws_stand_time').reset_index(level=[0], drop=True)\ndf_all = df_all.groupby('id_participant').apply(resample_health, col_name='ts_step_count', col_name_new='ws_step_count').reset_index(level=[0], drop=True)\ndf_all = df_all.groupby('id_participant').apply(resample_health, col_name='ts_walking_distance', col_name_new='ws_walking_distance').reset_index(level=[0], drop=True)","metadata":{"papermill":{"duration":15.407148,"end_time":"2023-10-23T14:41:30.625612","exception":false,"start_time":"2023-10-23T14:41:15.218464","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:15:33.441751Z","iopub.execute_input":"2023-11-13T22:15:33.442792Z","iopub.status.idle":"2023-11-13T22:16:17.587764Z","shell.execute_reply.started":"2023-11-13T22:15:33.442756Z","shell.execute_reply":"2023-11-13T22:16:17.586631Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Inspect dataset with resampled health values\ndf_all[df_all.id_participant.notna() & (df_all.ws_survey_count.notna() | df_all.ts_heart_rate.notna())][['id_participant', 'ws_survey_count', 'ws_heart_rate', 'ts_heart_rate']].head(10)","metadata":{"papermill":{"duration":0.37631,"end_time":"2023-10-23T14:41:31.018358","exception":false,"start_time":"2023-10-23T14:41:30.642048","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:16:17.589265Z","iopub.execute_input":"2023-11-13T22:16:17.589809Z","iopub.status.idle":"2023-11-13T22:16:18.355572Z","shell.execute_reply.started":"2023-11-13T22:16:17.589767Z","shell.execute_reply":"2023-11-13T22:16:18.354433Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Interpolate_weather_data\"></a>\n# Interpolate weather data\n\nThe weather data is distributed over several dataframes. Each measurement, e.g., rain or air temperature, is in a separate dataframe. The column names are the IDs of the weather stations.\nThere is one dataframe (`df_stations`), which contains meta information to the weather stations, e.g., coordinates and names. \n\nTo make the weather data usable, we need to find the nearest weather stations for each watch survey response in `df_all`. Then we can transfer the measurement values with the same timestamp to from the weather data dataframe, e.g., `df_rain`, to the dataframe with the Cozie data (`df_all`).\n\n**Improvement suggestion 1**\n\nIt might be worth debating whether the weather data from larger a time frame than the minute of the watch survey response should be considered. This could for example be achieved by hourly averaging the weather data.\n\n**Improvement suggestion 2**\n\nSome values of the new weather data columns, e.g., `ws_air_temperature`, are still NaN values. The reason is that either there is no nearby weather station available or the nearest weather station as no data at that particular point in time. There are several ways to improve the situation:\n - Find the second nearest weather station with data\n - Look further forward or back in time than one minute to find suitable weather data\n - Aggregate the weather data, e.g., hourly or daily mean values\n - Remove weather stations that have data with many missing values","metadata":{"papermill":{"duration":0.01695,"end_time":"2023-10-23T14:41:31.053397","exception":false,"start_time":"2023-10-23T14:41:31.036447","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Function to get weather data for a given timestamp and location from the nearest weather station\ndef get_weather_data(df_weather_data, lat, lon, timestamp):    \n    # Get timestamp with no seconds or microseconds\n    timestamp = timestamp.replace(second=0, microsecond=0)\n    \n    # Get weather data for timestamp above\n    weather_data_row = df_weather_data[df_weather_data.index == timestamp]\n\n    # If no weather data is available for timestamp above, get weather data for next minute\n    if (weather_data_row.empty):\n        timestamp = timestamp + pd.Timedelta(minutes=1)\n        weather_data_row = df_weather_data[df_weather_data.index == timestamp]\n        # If there is still no weather data available, return NaN\n        if (weather_data_row.empty):\n            return np.NaN\n\n    # Get weather station ids from weather stations with with data\n    station_ids_with_data = weather_data_row.columns[weather_data_row.count() > 0].tolist()\n\n    # Get weather stations with data\n    df_stations2 = df_stations[df_stations.id.isin(station_ids_with_data)]\n\n    # Compute distance to each weather station\n    df_stations2['distance'] = df_stations2.apply(lambda x: distance(x['latitude'], x['longitude'], lat, lon), axis=1)\n\n    # Get nearest weather station\n    station_id_nearest = df_stations2[df_stations2.distance==df_stations2.distance.min()].id.values[0]\n\n    # Retrun NaN for cases where the nearest weather station is more than 10km away\n    if df_stations2.distance.min()>10:\n        return np.NaN\n\n    # Return weather data from nearest weather station\n    return weather_data_row[station_id_nearest].values[0]\n\n# Function to compute distnace on spherical earth\ndef distance(lat1, lon1, lat2, lon2):\n    r = 6371 # km\n    p = pi / 180\n\n    a = 0.5 - cos((lat2-lat1)*p)/2 + cos(lat1*p) * cos(lat2*p) * (1-cos((lon2-lon1)*p))/2\n    return 2 * r * asin(sqrt(a))","metadata":{"papermill":{"duration":0.037753,"end_time":"2023-10-23T14:41:31.108776","exception":false,"start_time":"2023-10-23T14:41:31.071023","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:16:18.357102Z","iopub.execute_input":"2023-11-13T22:16:18.357470Z","iopub.status.idle":"2023-11-13T22:16:18.370321Z","shell.execute_reply.started":"2023-11-13T22:16:18.357440Z","shell.execute_reply":"2023-11-13T22:16:18.368949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Apply above function to the dataset for all weather data\ndf_all = df_all.reset_index()\ndf_all['ws_rainfall'] = df_all[df_all.ws_survey_count.notna()].apply(lambda row: get_weather_data(df_rain, row['ws_latitude'], row['ws_longitude'], row['time']), axis=1)\ndf_all['ws_air_temperature'] = df_all[df_all.ws_survey_count.notna()].apply(lambda row: get_weather_data(df_temperature, row['ws_latitude'], row['ws_longitude'], row['time']), axis=1)\ndf_all['ws_relative_humidity'] = df_all[df_all.ws_survey_count.notna()].apply(lambda row: get_weather_data(df_humidity, row['ws_latitude'], row['ws_longitude'], row['time']), axis=1)\ndf_all['ws_wind_speed'] = df_all[df_all.ws_survey_count.notna()].apply(lambda row: get_weather_data(df_wind_speed, row['ws_latitude'], row['ws_longitude'], row['time']), axis=1)\ndf_all['ws_wind_direction'] = df_all[df_all.ws_survey_count.notna()].apply(lambda row: get_weather_data(df_wind_direction, row['ws_latitude'], row['ws_longitude'], row['time']), axis=1)\ndf_all = df_all.set_index('time', drop=True)","metadata":{"papermill":{"duration":57.025298,"end_time":"2023-10-23T14:42:28.151621","exception":false,"start_time":"2023-10-23T14:41:31.126323","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:16:18.372069Z","iopub.execute_input":"2023-11-13T22:16:18.372711Z","iopub.status.idle":"2023-11-13T22:19:49.083741Z","shell.execute_reply.started":"2023-11-13T22:16:18.372661Z","shell.execute_reply":"2023-11-13T22:19:49.082658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Inspect dataframe that now includes weather data\n# Inspect dataset with resampled health values\ndf_all[df_all.id_participant.notna() & df_all.ws_survey_count.notna()][['id_participant', 'ws_survey_count', 'ws_air_temperature', 'ws_heart_rate']].head()","metadata":{"papermill":{"duration":0.110658,"end_time":"2023-10-23T14:42:28.279775","exception":false,"start_time":"2023-10-23T14:42:28.169117","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:49.088290Z","iopub.execute_input":"2023-11-13T22:19:49.088681Z","iopub.status.idle":"2023-11-13T22:19:49.347433Z","shell.execute_reply.started":"2023-11-13T22:19:49.088649Z","shell.execute_reply":"2023-11-13T22:19:49.346241Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Remove_rows_and_columns\"></a>\n# Remove rows and columns that we are not going to use\nWe are now at a point where we have completed the (basic) feature engineering and start preparing the data for the machine learning. For that, we are going to remove some of the rows and columns that we no longer need as it reduces the amount of data that we need to handle. More importantly it makes the dataframes easier to read for humans.\n\n**Improvement suggestion 1 :** \n\nFeel free to add or remove more columns. Ask yourself for each column if you it reasonable to expect any positive contribution to the model performance from the column. Also, what column could you create and add to improve the model performance. One way to find this out is be making changes the data and see whether the model performance improves or declines.","metadata":{"papermill":{"duration":0.016751,"end_time":"2023-10-23T14:42:28.313571","exception":false,"start_time":"2023-10-23T14:42:28.296820","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Remove all rows that are not a watch surve response\ndf_all = df_all[df_all.ws_survey_count.notna()]\n\n# Remove some timestamp columns\ndf_all = df_all.drop(columns=['ws_timestamp_location', 'ws_timestamp_start'])\n\n# Remove columns of the original health data those are no longer needed since we are going to use the interpolated values\ndf_all = df_all.drop(columns=['ts_heart_rate',\n                              'ts_audio_exposure_environment',\n                              'ts_oxygen_saturation',\n                              'ts_resting_heart_rate',\n                              'ts_stand_time', \n                              'ts_step_count',\n                              'ts_walking_distance'])\n\n# Inspect dataframe\ndf_all.head()","metadata":{"papermill":{"duration":0.145382,"end_time":"2023-10-23T14:42:28.476559","exception":false,"start_time":"2023-10-23T14:42:28.331177","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:49.349419Z","iopub.execute_input":"2023-11-13T22:19:49.349859Z","iopub.status.idle":"2023-11-13T22:19:49.638182Z","shell.execute_reply.started":"2023-11-13T22:19:49.349817Z","shell.execute_reply":"2023-11-13T22:19:49.637125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Split_data\"></a>\n# Split data\nWe split the data into the training set and solution set according to the original data (before we combined `df_train` and `df_solution` to `df_all` in the section [Read data](#Read_data)).\nFurther, we are going to take some data from the training set `df_train` and create a test set (`df_test`) in order to score the quality of our model. The split into training and testing data is need because the submission to the public leader board is limited to two per day and we want to avoid overfitting of the model.\n\nThe current split is 28 watch survey responses for the training set and the rest of responses for the test set. Feel free to change this ratio.","metadata":{"papermill":{"duration":0.017825,"end_time":"2023-10-23T14:42:28.512273","exception":false,"start_time":"2023-10-23T14:42:28.494448","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Separte training and solution data again\ndf_train = df_all[df_all.source == 'train'].copy()\ndf_solution = df_all[df_all.source == 'solution'].copy()\n\n# ws_survey_count might contain missing values and might reset. \n# To avoid any issue we are creating a similar column ws_survey_count_2 without missing and duplicate values\ndf_train.loc[(df_train.ws_survey_count.notna()), 'ws_survey_count_2'] = df_train.loc[(df_train.ws_survey_count.notna()),].groupby('id_participant').cumcount()\n\n# Add 1 to start at 1 instead of 0\ndf_train['ws_survey_count_2'] = df_train['ws_survey_count_2']+1 \n\n# Split data into training and testing sets\ndf_test = df_train[(df_train.ws_survey_count_2>28)]\ndf_train = df_train[df_train.ws_survey_count_2<=28]\n\n# Drop the helper columns ws_surve_count_2 and source again\ndf_train = df_train.drop(columns=['ws_survey_count_2', 'source'])\ndf_test = df_test.drop(columns=['ws_survey_count_2', 'source'])\ndf_solution = df_solution.drop(columns=['source'])","metadata":{"papermill":{"duration":0.076241,"end_time":"2023-10-23T14:42:28.606557","exception":false,"start_time":"2023-10-23T14:42:28.530316","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:49.639509Z","iopub.execute_input":"2023-11-13T22:19:49.639826Z","iopub.status.idle":"2023-11-13T22:19:49.767823Z","shell.execute_reply.started":"2023-11-13T22:19:49.639798Z","shell.execute_reply":"2023-11-13T22:19:49.766560Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_train.to_csv(\"nb02_train.csv\")\ndf_test.to_csv(\"nb02_test.csv\")\ndf_solution.to_csv(\"nb02_solution.csv\")","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Data_preparation\"></a>\n# Data preparation","metadata":{"papermill":{"duration":0.020826,"end_time":"2023-10-23T14:42:28.645696","exception":false,"start_time":"2023-10-23T14:42:28.624870","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"For the machine learning model below, all input and output values need to be numeric. At this point, the watch survey the response columns, e.g., `q_noise_nearby`, are of type string. Hence, we need convert the values in these columns into numbers.","metadata":{"papermill":{"duration":0.019053,"end_time":"2023-10-23T14:42:28.685245","exception":false,"start_time":"2023-10-23T14:42:28.666192","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Inspect dataframe still containing strings\ndf_train.head()","metadata":{"papermill":{"duration":0.097134,"end_time":"2023-10-23T14:42:28.801520","exception":false,"start_time":"2023-10-23T14:42:28.704386","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:49.769343Z","iopub.execute_input":"2023-11-13T22:19:49.769734Z","iopub.status.idle":"2023-11-13T22:19:49.846251Z","shell.execute_reply.started":"2023-11-13T22:19:49.769704Z","shell.execute_reply":"2023-11-13T22:19:49.845118Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Find columns with values that are not numeric\ncat_vars = list(df_train.select_dtypes(include=['object']).columns)\ncat_vars","metadata":{"papermill":{"duration":0.035001,"end_time":"2023-10-23T14:42:28.856173","exception":false,"start_time":"2023-10-23T14:42:28.821172","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:49.847580Z","iopub.execute_input":"2023-11-13T22:19:49.848028Z","iopub.status.idle":"2023-11-13T22:19:49.858470Z","shell.execute_reply.started":"2023-11-13T22:19:49.847988Z","shell.execute_reply":"2023-11-13T22:19:49.857289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# encoding categorical variables\ndict_cat = {}\n\nfor var in cat_vars:\n    print(var)\n    if var == 'id_participant': # no need to encode the participant ID\n        continue\n    # convert categorical variables into 'category' type\n    df_train[var] = df_train[var].astype('category')\n    df_test[var] = df_test[var].astype('category')\n    df_solution[var] = df_solution[var].astype('category')\n    \n    # dictionary to keep track of the variable mapping: {variable-name: {number1: string1, number2: string2, ...}\n    dict_cat[var] = dict(enumerate(df_train[var].cat.categories))\n\n    # encode variables, missing values are encoded as -1\n    df_train[var] = df_train[var].cat.codes\n    df_test[var] = df_test[var].cat.codes\n    df_solution[var] = df_solution[var].cat.codes\n\n#dict_cat","metadata":{"papermill":{"duration":0.072299,"end_time":"2023-10-23T14:42:28.948783","exception":false,"start_time":"2023-10-23T14:42:28.876484","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:49.859896Z","iopub.execute_input":"2023-11-13T22:19:49.860243Z","iopub.status.idle":"2023-11-13T22:19:49.917235Z","shell.execute_reply.started":"2023-11-13T22:19:49.860214Z","shell.execute_reply":"2023-11-13T22:19:49.915916Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Inspect dataframe where all strings were converted into integers\ndf_train.head()","metadata":{"papermill":{"duration":0.097399,"end_time":"2023-10-23T14:42:29.066359","exception":false,"start_time":"2023-10-23T14:42:28.968960","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:49.918940Z","iopub.execute_input":"2023-11-13T22:19:49.919301Z","iopub.status.idle":"2023-11-13T22:19:49.993291Z","shell.execute_reply.started":"2023-11-13T22:19:49.919270Z","shell.execute_reply":"2023-11-13T22:19:49.992046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Example_prediction\"></a>\n# Example prediction of `q_noise_nearby`\nBelow we are going to predict the column `q_noise_nearby` step by step.\nLater on, we are going to wrap all steps up in a function for more convenience.","metadata":{"papermill":{"duration":0.021581,"end_time":"2023-10-23T14:42:29.110507","exception":false,"start_time":"2023-10-23T14:42:29.088926","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Original size of the train dataset\ndf_train.shape","metadata":{"papermill":{"duration":0.033424,"end_time":"2023-10-23T14:42:29.165255","exception":false,"start_time":"2023-10-23T14:42:29.131831","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:49.995439Z","iopub.execute_input":"2023-11-13T22:19:49.995920Z","iopub.status.idle":"2023-11-13T22:19:50.003745Z","shell.execute_reply.started":"2023-11-13T22:19:49.995865Z","shell.execute_reply":"2023-11-13T22:19:50.002268Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Original size of the test dataset\ndf_test.shape","metadata":{"papermill":{"duration":0.033694,"end_time":"2023-10-23T14:42:29.220686","exception":false,"start_time":"2023-10-23T14:42:29.186992","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:50.005236Z","iopub.execute_input":"2023-11-13T22:19:50.005635Z","iopub.status.idle":"2023-11-13T22:19:50.017144Z","shell.execute_reply.started":"2023-11-13T22:19:50.005602Z","shell.execute_reply":"2023-11-13T22:19:50.015867Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Train: remove rows where the predicting variable is missing\n# In this example this doesn't remove any rows, but this will \n# be necessary for the column 'q_noise_kind'.\ndf_train_noise_nearby = df_train[df_train['q_noise_nearby'] != -1]\nprint(df_train_noise_nearby.shape)","metadata":{"papermill":{"duration":0.037545,"end_time":"2023-10-23T14:42:29.279745","exception":false,"start_time":"2023-10-23T14:42:29.242200","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:50.018663Z","iopub.execute_input":"2023-11-13T22:19:50.018990Z","iopub.status.idle":"2023-11-13T22:19:50.029064Z","shell.execute_reply.started":"2023-11-13T22:19:50.018962Z","shell.execute_reply":"2023-11-13T22:19:50.027818Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Test: remove rows where the predicting variable is missing\ndf_test_noise_nearby = df_test[df_test['q_noise_nearby'] != -1]\nprint(df_test_noise_nearby.shape)","metadata":{"papermill":{"duration":0.034884,"end_time":"2023-10-23T14:42:29.336720","exception":false,"start_time":"2023-10-23T14:42:29.301836","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:50.030654Z","iopub.execute_input":"2023-11-13T22:19:50.030965Z","iopub.status.idle":"2023-11-13T22:19:50.040150Z","shell.execute_reply.started":"2023-11-13T22:19:50.030938Z","shell.execute_reply":"2023-11-13T22:19:50.038965Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Train: remove the id_participant column\ndf_train_noise_nearby = df_train_noise_nearby.drop(['id_participant'], axis=1)\ndf_train_noise_nearby.shape","metadata":{"papermill":{"duration":0.03767,"end_time":"2023-10-23T14:42:29.395458","exception":false,"start_time":"2023-10-23T14:42:29.357788","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:50.041881Z","iopub.execute_input":"2023-11-13T22:19:50.042498Z","iopub.status.idle":"2023-11-13T22:19:50.056397Z","shell.execute_reply.started":"2023-11-13T22:19:50.042455Z","shell.execute_reply":"2023-11-13T22:19:50.055201Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Test: remove the id_participant column\ndf_test_noise_nearby = df_test_noise_nearby.drop(['id_participant'], axis=1)\ndf_test_noise_nearby.shape","metadata":{"papermill":{"duration":0.033746,"end_time":"2023-10-23T14:42:29.450522","exception":false,"start_time":"2023-10-23T14:42:29.416776","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:50.058430Z","iopub.execute_input":"2023-11-13T22:19:50.058921Z","iopub.status.idle":"2023-11-13T22:19:50.071768Z","shell.execute_reply.started":"2023-11-13T22:19:50.058868Z","shell.execute_reply":"2023-11-13T22:19:50.070290Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Count classes in predicting variable\nnum_class = df_train_noise_nearby['q_noise_nearby'].unique()\nprint(num_class)\ndict_cat['q_noise_nearby']","metadata":{"papermill":{"duration":0.03891,"end_time":"2023-10-23T14:42:29.510647","exception":false,"start_time":"2023-10-23T14:42:29.471737","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:50.072990Z","iopub.execute_input":"2023-11-13T22:19:50.073486Z","iopub.status.idle":"2023-11-13T22:19:50.084620Z","shell.execute_reply.started":"2023-11-13T22:19:50.073442Z","shell.execute_reply":"2023-11-13T22:19:50.083457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Break dataframe into variables and ground truth\ny_train = df_train_noise_nearby['q_noise_nearby']\nX_train = df_train_noise_nearby.drop(['q_noise_nearby', 'q_noise_kind', 'q_earphones', 'q_thermal_preference'], axis=1)\n\ny_test = df_test_noise_nearby['q_noise_nearby']\nX_test = df_test_noise_nearby.drop(['q_noise_nearby', 'q_noise_kind', 'q_earphones', 'q_thermal_preference'], axis=1)","metadata":{"papermill":{"duration":0.03664,"end_time":"2023-10-23T14:42:29.569771","exception":false,"start_time":"2023-10-23T14:42:29.533131","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:50.087264Z","iopub.execute_input":"2023-11-13T22:19:50.087794Z","iopub.status.idle":"2023-11-13T22:19:50.099112Z","shell.execute_reply.started":"2023-11-13T22:19:50.087751Z","shell.execute_reply":"2023-11-13T22:19:50.098222Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Initialize model\nclf = XGBClassifier(num_class=num_class, objective='multi:softmax', enable_categorical=False)","metadata":{"papermill":{"duration":0.032692,"end_time":"2023-10-23T14:42:29.624273","exception":false,"start_time":"2023-10-23T14:42:29.591581","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:50.101082Z","iopub.execute_input":"2023-11-13T22:19:50.101963Z","iopub.status.idle":"2023-11-13T22:19:50.116713Z","shell.execute_reply.started":"2023-11-13T22:19:50.101917Z","shell.execute_reply":"2023-11-13T22:19:50.115780Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Train model\nclf.fit(X_train, y_train)","metadata":{"papermill":{"duration":2.403859,"end_time":"2023-10-23T14:42:32.051666","exception":false,"start_time":"2023-10-23T14:42:29.647807","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:50.118652Z","iopub.execute_input":"2023-11-13T22:19:50.119522Z","iopub.status.idle":"2023-11-13T22:19:53.294696Z","shell.execute_reply.started":"2023-11-13T22:19:50.119476Z","shell.execute_reply":"2023-11-13T22:19:53.293494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Predict and evaluate\ny_pred = clf.predict(X_test)\nprint('XGBoost model accuracy score: {0:0.4f}'. format(accuracy_score(y_test, y_pred)))","metadata":{"papermill":{"duration":0.045012,"end_time":"2023-10-23T14:42:32.121428","exception":false,"start_time":"2023-10-23T14:42:32.076416","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:53.296226Z","iopub.execute_input":"2023-11-13T22:19:53.296707Z","iopub.status.idle":"2023-11-13T22:19:53.321471Z","shell.execute_reply.started":"2023-11-13T22:19:53.296676Z","shell.execute_reply":"2023-11-13T22:19:53.320399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Copy test data into new dataframe\ndf_comparison = df_test.copy()\n\n# Assign test prediction to dataframe\ndf_comparison['q_noise_nearby (original)'] = y_test\ndf_comparison['q_noise_nearby (predicted)'] = y_pred\n\n# Convert labels from integers back to strings\ndf_comparison['q_noise_nearby (original)'] = df_comparison['q_noise_nearby (original)'].replace(dict_cat['q_noise_nearby'])\ndf_comparison['q_noise_nearby (predicted)'] = df_comparison['q_noise_nearby (predicted)'].replace(dict_cat['q_noise_nearby'])\n\n# Show dataframe\ndf_comparison[['id_participant', 'q_noise_nearby (original)', 'q_noise_nearby (predicted)', 'ws_heart_rate']]","metadata":{"papermill":{"duration":0.053726,"end_time":"2023-10-23T14:42:32.198299","exception":false,"start_time":"2023-10-23T14:42:32.144573","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:53.331014Z","iopub.execute_input":"2023-11-13T22:19:53.334178Z","iopub.status.idle":"2023-11-13T22:19:53.361989Z","shell.execute_reply.started":"2023-11-13T22:19:53.334126Z","shell.execute_reply":"2023-11-13T22:19:53.360765Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Show rows with wrong predictions\ndf_comparison[df_comparison['q_noise_nearby (original)'] != df_comparison['q_noise_nearby (predicted)']][['id_participant', 'q_noise_nearby (original)', 'q_noise_nearby (predicted)', 'ws_heart_rate']]","metadata":{"papermill":{"duration":0.048237,"end_time":"2023-10-23T14:42:32.270149","exception":false,"start_time":"2023-10-23T14:42:32.221912","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:53.363574Z","iopub.execute_input":"2023-11-13T22:19:53.364003Z","iopub.status.idle":"2023-11-13T22:19:53.386064Z","shell.execute_reply.started":"2023-11-13T22:19:53.363969Z","shell.execute_reply":"2023-11-13T22:19:53.385025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This is a good moment to look at `df_comparison` and see whether you can recognise a pattern in the values that were predicted wrongly.\nThis might inspire ideas for new features or data pre-processing steps.","metadata":{"papermill":{"duration":0.024625,"end_time":"2023-10-23T14:42:32.321637","exception":false,"start_time":"2023-10-23T14:42:32.297012","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"<a id=\"Predict_any_column\"></a>\n# Predict any column\nWe are going to wrap up above code into a function in order to make the prediction of the other columns a bit more convenient. The steps are mostly the same as above and might feel a bit repetitive.\n\n**Improvement suggestion 1**\n\nCurrently, one column is predicted at a time and the same model is used for all participants. In order, to improve the performance, a separate model could be trained for each participant.\n\n**Improvement suggestion 2**\n\nLook into other algorithms than XGBoost.\n\n**Improvement suggestion 3**\n\nSome of the responses depend on previous responses, e.g., if the answer to the question 'Noise distractions nearby?' is 'None', then the response to 'What kind of noise?' and 'Wearing earphones' has to be 'Not applicable'. \nThese structures of the watch survey is currently not checked nor enforced.\n","metadata":{"papermill":{"duration":0.024201,"end_time":"2023-10-23T14:42:32.370028","exception":false,"start_time":"2023-10-23T14:42:32.345827","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def predict_any_column(column_name, df_train, df_test, df_solution):\n    # Remove rows where the predicting variable is missing\n    df_train = df_train[df_train[column_name] != -1]\n    df_test = df_test[df_test[column_name] != -1]\n    \n    # Remove rows where the predicting variable is missing\n    df_train = df_train.drop(columns=['id_participant'])\n    df_test = df_test.drop(columns=['id_participant'])\n    df_solution = df_solution.drop(columns=['id_participant'])\n\n    # Count classes in predicting variable\n    num_class = df_train[column_name].unique()\n    \n    # Break dataframe into variables and ground truth\n    y_train = df_train[column_name]\n    X_train = df_train.drop(['q_noise_nearby', 'q_noise_kind', 'q_earphones', 'q_thermal_preference'], axis=1)\n\n    y_test = df_test[column_name]\n    X_test = df_test.drop(['q_noise_nearby', 'q_noise_kind', 'q_earphones', 'q_thermal_preference'], axis=1)\n    \n    X_solution = df_solution.drop(['q_noise_nearby', 'q_noise_kind', 'q_earphones', 'q_thermal_preference'], axis=1)\n    \n    # Initialize model\n    clf = XGBClassifier(num_class=num_class, objective='multi:softmax', enable_categorical=False)\n    \n    # Train model\n    clf.fit(X_train, y_train)\n\n    # Predict and evaluate\n    y_pred = clf.predict(X_test)\n    score = accuracy_score(y_test, y_pred)\n    print('XGBoost model accuracy score: {0:0.4f}'. format(score))\n\n    # Predict solution\n    X_solution.shape\n    y_solution = clf.predict(X_solution)\n    \n    return y_solution, score","metadata":{"papermill":{"duration":0.040227,"end_time":"2023-10-23T14:42:32.434348","exception":false,"start_time":"2023-10-23T14:42:32.394121","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:53.387666Z","iopub.execute_input":"2023-11-13T22:19:53.387991Z","iopub.status.idle":"2023-11-13T22:19:53.399712Z","shell.execute_reply.started":"2023-11-13T22:19:53.387963Z","shell.execute_reply":"2023-11-13T22:19:53.398471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create an empty list to collect the scores\nscore = []\n\n# Create a copy of the solution dataframe for submission\ndf_submission = df_solution.copy()","metadata":{"papermill":{"duration":0.03541,"end_time":"2023-10-23T14:42:32.494291","exception":false,"start_time":"2023-10-23T14:42:32.458881","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:53.401232Z","iopub.execute_input":"2023-11-13T22:19:53.401597Z","iopub.status.idle":"2023-11-13T22:19:53.416172Z","shell.execute_reply.started":"2023-11-13T22:19:53.401567Z","shell.execute_reply":"2023-11-13T22:19:53.415297Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Predict_q_noise_nearby\"></a>\n## Prediction `q_noise_nearby`\n\nWe've already prediced the values for the column `q_noise_nearby`. We are going to repeat this step in order to collect the score so that we can compute the overall score for all four columns that need to be predicted.","metadata":{"papermill":{"duration":0.022925,"end_time":"2023-10-23T14:42:32.541066","exception":false,"start_time":"2023-10-23T14:42:32.518141","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df_solution['q_noise_nearby'], this_score = predict_any_column('q_noise_nearby', df_train, df_test, df_solution)\nscore.append(this_score)","metadata":{"papermill":{"duration":2.299523,"end_time":"2023-10-23T14:42:34.863581","exception":false,"start_time":"2023-10-23T14:42:32.564058","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:19:53.417782Z","iopub.execute_input":"2023-11-13T22:19:53.418260Z","iopub.status.idle":"2023-11-13T22:20:06.104239Z","shell.execute_reply.started":"2023-11-13T22:19:53.418222Z","shell.execute_reply":"2023-11-13T22:20:06.103158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Predict_q_noise_kind\"></a>\n## Prediction `q_noise_kind`","metadata":{"papermill":{"duration":0.023142,"end_time":"2023-10-23T14:42:34.911438","exception":false,"start_time":"2023-10-23T14:42:34.888296","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df_submission['q_noise_kind'], this_score  = predict_any_column('q_noise_kind', df_train, df_test, df_solution)\nscore.append(this_score)","metadata":{"papermill":{"duration":4.265164,"end_time":"2023-10-23T14:42:39.201188","exception":false,"start_time":"2023-10-23T14:42:34.936024","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:20:06.108498Z","iopub.execute_input":"2023-11-13T22:20:06.111310Z","iopub.status.idle":"2023-11-13T22:20:13.582702Z","shell.execute_reply.started":"2023-11-13T22:20:06.111261Z","shell.execute_reply":"2023-11-13T22:20:13.581475Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Predict_q_earphones\"></a>\n## Prediction `q_earphones`","metadata":{"papermill":{"duration":0.024181,"end_time":"2023-10-23T14:42:39.249755","exception":false,"start_time":"2023-10-23T14:42:39.225574","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df_submission['q_earphones'],this_score = predict_any_column('q_earphones', df_train, df_test, df_solution)\nscore.append(this_score)","metadata":{"papermill":{"duration":2.477977,"end_time":"2023-10-23T14:42:41.751853","exception":false,"start_time":"2023-10-23T14:42:39.273876","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:20:13.587745Z","iopub.execute_input":"2023-11-13T22:20:13.588964Z","iopub.status.idle":"2023-11-13T22:20:17.403763Z","shell.execute_reply.started":"2023-11-13T22:20:13.588913Z","shell.execute_reply":"2023-11-13T22:20:17.402681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Predict_q_thermal_preference\"></a>\n## Prediction `q_thermal_preference`","metadata":{"papermill":{"duration":0.02382,"end_time":"2023-10-23T14:42:41.801641","exception":false,"start_time":"2023-10-23T14:42:41.777821","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df_submission['q_thermal_preference'],this_score = predict_any_column('q_thermal_preference', df_train, df_test, df_solution)\nscore.append(this_score)","metadata":{"papermill":{"duration":3.356661,"end_time":"2023-10-23T14:42:45.182418","exception":false,"start_time":"2023-10-23T14:42:41.825757","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:20:17.408085Z","iopub.execute_input":"2023-11-13T22:20:17.411086Z","iopub.status.idle":"2023-11-13T22:20:20.456040Z","shell.execute_reply.started":"2023-11-13T22:20:17.411036Z","shell.execute_reply":"2023-11-13T22:20:20.455057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Compute_overall_score\"></a>\n# Compute overall score\nThe model for this score used only part of the training dataset. This is because we need some reference data to compute a score. This also means that the score computed in this notebook will not necessarily be the same score as in the leaderboard.\n\n**Improvement suggestion**\n\nFit the model again using the entire training dataset and submit the result from that model.","metadata":{"papermill":{"duration":0.024838,"end_time":"2023-10-23T14:42:45.232740","exception":false,"start_time":"2023-10-23T14:42:45.207902","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# The overall score is just the averge of the scores for each column\nscore_total = np.mean(score)\nprint('Overall accuracy score: {0:0.4f}'. format(score_total))","metadata":{"papermill":{"duration":0.035295,"end_time":"2023-10-23T14:42:45.292518","exception":false,"start_time":"2023-10-23T14:42:45.257223","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:20:20.460779Z","iopub.execute_input":"2023-11-13T22:20:20.463718Z","iopub.status.idle":"2023-11-13T22:20:20.471445Z","shell.execute_reply.started":"2023-11-13T22:20:20.463668Z","shell.execute_reply":"2023-11-13T22:20:20.469899Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"Create_submission_file\"></a>\n# Create submission file\nWe are almost done. We now need to convert the integers in the watch survey response columns back to the original strings and create a file that can be submitted to the competition.","metadata":{"papermill":{"duration":0.024578,"end_time":"2023-10-23T14:42:45.341415","exception":false,"start_time":"2023-10-23T14:42:45.316837","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Convert classes back form integers to strings\nfor key in dict_cat:\n    if key in df_solution.columns:\n        df_submission[key] = df_submission[key].replace(dict_cat[key])\n\n# Select only necessary columns for submission\ndf_submission = df_submission[['id_unique', 'q_noise_nearby', 'q_noise_kind', 'q_earphones', 'q_thermal_preference']].set_index('id_unique')\n\n# Write dataframe to csv file\ndf_submission.to_csv('submission.csv')\n\nprint('Congrats! The Notebook exectued successfully!')","metadata":{"papermill":{"duration":0.060824,"end_time":"2023-10-23T14:42:45.427439","exception":false,"start_time":"2023-10-23T14:42:45.366615","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-11-13T22:20:20.473492Z","iopub.execute_input":"2023-11-13T22:20:20.474779Z","iopub.status.idle":"2023-11-13T22:20:20.532388Z","shell.execute_reply.started":"2023-11-13T22:20:20.474742Z","shell.execute_reply":"2023-11-13T22:20:20.531100Z"},"trusted":true},"execution_count":null,"outputs":[]}]}