{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Example Notebook 3: Exploratory Data Analysis\n\nIn this notebook we will be exploring and visualizing the data available. As explained in the [data description page](https://www.kaggle.com/competitions/cool-quiet-city-competition/data), there are different types of features you can find. We will be using the same clasification detailed there and explore each feature related to each target feature.\n\n#### Table of content\n- Micro-survey data\n- Physiological / Environmental / Activity data\n- Spatial indicators\n- Weather data","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nfrom matplotlib.cm import get_cmap\nplt.style.use('ggplot')\nimport seaborn as sns\nimport gc\n\n# What do we say to python warnings? NOT TODAY\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:18.208190Z","iopub.execute_input":"2023-11-13T22:56:18.208920Z","iopub.status.idle":"2023-11-13T22:56:18.886186Z","shell.execute_reply.started":"2023-11-13T22:56:18.208888Z","shell.execute_reply":"2023-11-13T22:56:18.884716Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As the purporse of the competition is to predict the four target columns, we will explore the available features grouped by the targets. We also have a column for the participant id and the timestamp, we'll have them handy here:","metadata":{}},{"cell_type":"code","source":"target_cols = [\"q_noise_nearby\", \"q_noise_kind\", \"q_earphones\", \"q_thermal_preference\"]\nid_cols = [\"id_participant\"]\ntime_cols = [\"time\"]","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:18.888500Z","iopub.execute_input":"2023-11-13T22:56:18.889064Z","iopub.status.idle":"2023-11-13T22:56:18.895142Z","shell.execute_reply.started":"2023-11-13T22:56:18.889017Z","shell.execute_reply":"2023-11-13T22:56:18.893929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Micro-survey data\nIn this section we will be exploring the answers to the micro-survey questions; four of them are the target and they are already declared in `target_cols`.","metadata":{}},{"cell_type":"code","source":"q_cols = [\n    \"q_activity_category_alone\",\n    \"q_activity_category_group\",\n    \"q_alone_group\",\n    \"q_location\",\n    \"q_location_office\",\n    \"q_location_transport\"\n]","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:18.896858Z","iopub.execute_input":"2023-11-13T22:56:18.897243Z","iopub.status.idle":"2023-11-13T22:56:18.910585Z","shell.execute_reply.started":"2023-11-13T22:56:18.897209Z","shell.execute_reply":"2023-11-13T22:56:18.909246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ms_data = pd.read_csv(\"/kaggle/input/cool-quiet-city-competition/cozie_responses_and_physiological_data_training.csv\", usecols=q_cols+target_cols)\nms_data.info()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:18.911735Z","iopub.execute_input":"2023-11-13T22:56:18.912068Z","iopub.status.idle":"2023-11-13T22:56:20.789901Z","shell.execute_reply.started":"2023-11-13T22:56:18.912042Z","shell.execute_reply":"2023-11-13T22:56:20.788782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Remember that this is an sparse data set; in this section we are explorying categorical feature, will keep only the non-null values.","metadata":{}},{"cell_type":"code","source":"ms_data.dropna(inplace=True)","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:20.792318Z","iopub.execute_input":"2023-11-13T22:56:20.792655Z","iopub.status.idle":"2023-11-13T22:56:21.076244Z","shell.execute_reply.started":"2023-11-13T22:56:20.792628Z","shell.execute_reply":"2023-11-13T22:56:21.074923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ms_data.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:21.077291Z","iopub.execute_input":"2023-11-13T22:56:21.077599Z","iopub.status.idle":"2023-11-13T22:56:21.095191Z","shell.execute_reply.started":"2023-11-13T22:56:21.077573Z","shell.execute_reply":"2023-11-13T22:56:21.094448Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As mentioned above, these are all categorical features. We will create a [FacetGrid](https://seaborn.pydata.org/generated/seaborn.FacetGrid.html) with [countplots](https://seaborn.pydata.org/generated/seaborn.countplot.html) to visualize the distribution among categories. We will use different colors to make the visualization easier. For each target column the same plot will be created.","metadata":{}},{"cell_type":"code","source":"# This is the discrete color palette we will use\ncmap = get_cmap(\"Set2\")\ncolors = cmap.colors","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:21.096305Z","iopub.execute_input":"2023-11-13T22:56:21.096749Z","iopub.status.idle":"2023-11-13T22:56:21.101194Z","shell.execute_reply.started":"2023-11-13T22:56:21.096724Z","shell.execute_reply":"2023-11-13T22:56:21.100148Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `q_noise_nearby`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[0]\n\nfor question, color in zip(q_cols, colors):\n    # Create the grid\n    grid = sns.FacetGrid(ms_data, col=target_col, sharey=True, sharex=True)\n    # Map the countplot\n    plot = grid.map_dataframe(sns.countplot, x=question, color=color)\n    #This is for rotating x-axis ticks labels\n    for axes in plot.axes.flat:\n        _ = axes.set_xticklabels(axes.get_xticklabels(), rotation=90)\n        _ = axes.set_xlabel(\"\")\n        _ = axes.set_ylabel(\"\")\n        \n    plot.fig.suptitle(f\"Target: {target_col} | Feature: {question}\")\n    plt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:21.103154Z","iopub.execute_input":"2023-11-13T22:56:21.103639Z","iopub.status.idle":"2023-11-13T22:56:23.934092Z","shell.execute_reply.started":"2023-11-13T22:56:21.103604Z","shell.execute_reply":"2023-11-13T22:56:23.932626Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `q_noise_kind`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[1]\n\nfor question, color in zip(q_cols, colors):\n    # Create the grid\n    grid = sns.FacetGrid(ms_data, col=target_col, sharey=True, sharex=True)\n    # Map the countplot\n    plot = grid.map_dataframe(sns.countplot, x=question, color=color)\n    #This is for rotating x-axis ticks labels\n    for axes in plot.axes.flat:\n        _ = axes.set_xticklabels(axes.get_xticklabels(), rotation=90)\n        _ = axes.set_xlabel(\"\")\n        _ = axes.set_ylabel(\"\")\n        \n    plot.fig.suptitle(f\"Target: {target_col} | Feature: {question}\")\n    plt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:23.936034Z","iopub.execute_input":"2023-11-13T22:56:23.936466Z","iopub.status.idle":"2023-11-13T22:56:31.498842Z","shell.execute_reply.started":"2023-11-13T22:56:23.936430Z","shell.execute_reply":"2023-11-13T22:56:31.497528Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `q_noise_nearby`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[2]\n\nfor question, color in zip(q_cols, colors):\n    # Create the grid\n    grid = sns.FacetGrid(ms_data, col=target_col, sharey=True, sharex=True)\n    # Map the countplot\n    plot = grid.map_dataframe(sns.countplot, x=question, color=color)\n    #This is for rotating x-axis ticks labels\n    for axes in plot.axes.flat:\n        _ = axes.set_xticklabels(axes.get_xticklabels(), rotation=90)\n        _ = axes.set_xlabel(\"\")\n        _ = axes.set_ylabel(\"\")\n        \n    plot.fig.suptitle(f\"Target: {target_col} | Feature: {question}\")\n    plt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:31.500540Z","iopub.execute_input":"2023-11-13T22:56:31.501588Z","iopub.status.idle":"2023-11-13T22:56:35.436513Z","shell.execute_reply.started":"2023-11-13T22:56:31.501554Z","shell.execute_reply":"2023-11-13T22:56:35.434457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `q_thermal_preference`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[3]\n\nfor question, color in zip(q_cols, colors):\n    # Create the grid\n    grid = sns.FacetGrid(ms_data, col=target_col, sharey=True, sharex=True)\n    # Map the countplot\n    plot = grid.map_dataframe(sns.countplot, x=question, color=color)\n    #This is for rotating x-axis ticks labels\n    for axes in plot.axes.flat:\n        _ = axes.set_xticklabels(axes.get_xticklabels(), rotation=90)\n        _ = axes.set_xlabel(\"\")\n        _ = axes.set_ylabel(\"\")\n        \n    plot.fig.suptitle(f\"Target: {target_col} | Feature: {question}\")\n    plt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:35.438131Z","iopub.execute_input":"2023-11-13T22:56:35.438620Z","iopub.status.idle":"2023-11-13T22:56:39.589624Z","shell.execute_reply.started":"2023-11-13T22:56:35.438585Z","shell.execute_reply":"2023-11-13T22:56:39.588544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del(ms_data, grid, plot)\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:39.591035Z","iopub.execute_input":"2023-11-13T22:56:39.591287Z","iopub.status.idle":"2023-11-13T22:56:39.703573Z","shell.execute_reply.started":"2023-11-13T22:56:39.591265Z","shell.execute_reply":"2023-11-13T22:56:39.702527Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Physiological / Environmental / Activity data\nIn the case of activity data we are dealing with time series. We will try out another kind of visualizations.","metadata":{}},{"cell_type":"code","source":"pea_cols = [\n    \"ts_audio_exposure_environment\",\n    \"ts_heart_rate\",\n    \"ts_oxygen_saturation\",\n    \"ts_resting_heart_rate\",\n    \"ts_stand_time\",\n    \"ts_step_count\",\n    \"ts_walking_distance\"\n]","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:39.704727Z","iopub.execute_input":"2023-11-13T22:56:39.704959Z","iopub.status.idle":"2023-11-13T22:56:39.711172Z","shell.execute_reply.started":"2023-11-13T22:56:39.704931Z","shell.execute_reply":"2023-11-13T22:56:39.710599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pea_data = pd.read_csv(\"/kaggle/input/cool-quiet-city-competition/cozie_responses_and_physiological_data_training.csv\", usecols=time_cols+id_cols+target_cols+pea_cols, parse_dates=[\"time\"])\npea_data.info()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:56:39.715313Z","iopub.execute_input":"2023-11-13T22:56:39.715977Z","iopub.status.idle":"2023-11-13T22:57:03.032085Z","shell.execute_reply.started":"2023-11-13T22:56:39.715953Z","shell.execute_reply":"2023-11-13T22:57:03.031262Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, there is an issue here: we have questions answers from time to time, and we have activity and enviromental reading from time to time. The readings not always (or almost never) are at the same time. In order to explore this date and get an idea of how these timeseries relate to the target, we are going to do the following:\n\n- Each question answers, when missing, will be repeated from the previous one. This asume that the participant is feeling the same during the periods between surveys.\n- We will resample by hour the survey answers and the activity, enviromental and physiological readings\n- We will merge both data sets to have a question answers per hourly reading\n- We will do this by participant and concat all the resulting dataframes at the end\n\n> *This is only one -of infinite- strategies, with te main purpose of visualizing the data before developing a model. What other strategy would you try?*\n\nBelow is the code to process the data set and \"sync\" all the readings:","metadata":{}},{"cell_type":"code","source":"dfs = []\n\nfor participant in pea_data.id_participant.unique():\n    \n    # Get target columns per participant\n    temp_target = pea_data.loc[pea_data.id_participant == participant, time_cols+target_cols]\n    # Forward fill and resample by hour, keeping only first value\n    temp_target = temp_target.ffill().set_index(\"time\").resample(\"h\").first()\n    \n    # Get activity columns\n    temp_pea = pea_data.loc[pea_data.id_participant == participant, time_cols+pea_cols]\n    # Resample hourly\n    temp_pea = temp_pea.set_index(\"time\").resample(\"h\").mean()\n    \n    # Merge both datasets\n    temp = pd.merge(temp_target, temp_pea, how=\"inner\", left_index=True, right_index=True)\n    temp[\"id_participant\"] = participant\n    \n    dfs.append(temp)","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:03.033155Z","iopub.execute_input":"2023-11-13T22:57:03.034174Z","iopub.status.idle":"2023-11-13T22:57:18.390999Z","shell.execute_reply.started":"2023-11-13T22:57:03.034145Z","shell.execute_reply":"2023-11-13T22:57:18.389894Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pea_hourly = pd.concat(dfs).reset_index()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:18.392332Z","iopub.execute_input":"2023-11-13T22:57:18.392641Z","iopub.status.idle":"2023-11-13T22:57:18.426304Z","shell.execute_reply.started":"2023-11-13T22:57:18.392618Z","shell.execute_reply":"2023-11-13T22:57:18.425375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pea_hourly.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:18.427484Z","iopub.execute_input":"2023-11-13T22:57:18.427819Z","iopub.status.idle":"2023-11-13T22:57:18.446267Z","shell.execute_reply.started":"2023-11-13T22:57:18.427792Z","shell.execute_reply":"2023-11-13T22:57:18.445163Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And here we have our \"synced\" dataset. There are still missing values, but is not as sparse as the original one.\n\nNow, as we are dealing with time series, we will explore the features related to the targets, but with an addition: we will be exploring the date grouped by different periods of  time. This is an strategy to find seasonality and use that information for feature engineer.","metadata":{}},{"cell_type":"code","source":"del(pea_data, temp_target, temp_pea, temp)","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:18.447736Z","iopub.execute_input":"2023-11-13T22:57:18.448116Z","iopub.status.idle":"2023-11-13T22:57:18.475618Z","shell.execute_reply.started":"2023-11-13T22:57:18.448086Z","shell.execute_reply":"2023-11-13T22:57:18.474638Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `q_noise_nearby`","metadata":{}},{"cell_type":"markdown","source":"#### By date","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[0]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.date, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,14), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:18.476696Z","iopub.execute_input":"2023-11-13T22:57:18.476947Z","iopub.status.idle":"2023-11-13T22:57:21.644639Z","shell.execute_reply.started":"2023-11-13T22:57:18.476924Z","shell.execute_reply":"2023-11-13T22:57:21.643943Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### By hour of the day","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[0]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.hour, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,14), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:21.645537Z","iopub.execute_input":"2023-11-13T22:57:21.645782Z","iopub.status.idle":"2023-11-13T22:57:23.691332Z","shell.execute_reply.started":"2023-11-13T22:57:21.645759Z","shell.execute_reply":"2023-11-13T22:57:23.690404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### By week day\nThe day of the week with Monday=0, Sunday=6.","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[0]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.weekday, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,14), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:23.692885Z","iopub.execute_input":"2023-11-13T22:57:23.693469Z","iopub.status.idle":"2023-11-13T22:57:25.839099Z","shell.execute_reply.started":"2023-11-13T22:57:23.693434Z","shell.execute_reply":"2023-11-13T22:57:25.838414Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `q_noise_kind`","metadata":{}},{"cell_type":"markdown","source":"#### By date","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[1]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.date, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,18), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:25.840306Z","iopub.execute_input":"2023-11-13T22:57:25.840810Z","iopub.status.idle":"2023-11-13T22:57:30.056354Z","shell.execute_reply.started":"2023-11-13T22:57:25.840785Z","shell.execute_reply":"2023-11-13T22:57:30.055416Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### By hour of the day","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[1]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.hour, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,14), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:30.057444Z","iopub.execute_input":"2023-11-13T22:57:30.058225Z","iopub.status.idle":"2023-11-13T22:57:32.831637Z","shell.execute_reply.started":"2023-11-13T22:57:30.058199Z","shell.execute_reply":"2023-11-13T22:57:32.830288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### By week day","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[1]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.weekday, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,14), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:32.832931Z","iopub.execute_input":"2023-11-13T22:57:32.833352Z","iopub.status.idle":"2023-11-13T22:57:35.797434Z","shell.execute_reply.started":"2023-11-13T22:57:32.833187Z","shell.execute_reply":"2023-11-13T22:57:35.795910Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `q_earphones`","metadata":{}},{"cell_type":"markdown","source":"#### By date","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[2]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.date, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,18), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:35.798669Z","iopub.execute_input":"2023-11-13T22:57:35.798942Z","iopub.status.idle":"2023-11-13T22:57:39.471063Z","shell.execute_reply.started":"2023-11-13T22:57:35.798919Z","shell.execute_reply":"2023-11-13T22:57:39.469775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### By hour of the day","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[2]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.hour, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,14), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:39.472941Z","iopub.execute_input":"2023-11-13T22:57:39.473272Z","iopub.status.idle":"2023-11-13T22:57:41.860743Z","shell.execute_reply.started":"2023-11-13T22:57:39.473244Z","shell.execute_reply":"2023-11-13T22:57:41.859681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### By week day","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[2]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.weekday, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,14), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:41.862068Z","iopub.execute_input":"2023-11-13T22:57:41.862393Z","iopub.status.idle":"2023-11-13T22:57:44.705452Z","shell.execute_reply.started":"2023-11-13T22:57:41.862349Z","shell.execute_reply":"2023-11-13T22:57:44.704190Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### `q_thermal_preference`","metadata":{}},{"cell_type":"markdown","source":"#### By date","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[3]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.date, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,18), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:44.706723Z","iopub.execute_input":"2023-11-13T22:57:44.706978Z","iopub.status.idle":"2023-11-13T22:57:48.242984Z","shell.execute_reply.started":"2023-11-13T22:57:44.706956Z","shell.execute_reply":"2023-11-13T22:57:48.242123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### By hour of the day","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[3]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.hour, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,14), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:48.244139Z","iopub.execute_input":"2023-11-13T22:57:48.244664Z","iopub.status.idle":"2023-11-13T22:57:50.440184Z","shell.execute_reply.started":"2023-11-13T22:57:48.244638Z","shell.execute_reply":"2023-11-13T22:57:50.439150Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### By week day","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[3]\ntemp = pea_hourly.groupby([pea_hourly.time.dt.weekday, target_col])[pea_cols].mean().reset_index()\n\ncols = 1\nrows = len(pea_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(20,14), sharex=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pea_cols):\n    ax = axes[i]\n    f = sns.lineplot(temp, x=\"time\", y=col, hue=target_col, palette=\"Set2\", ax=ax)\n    ax.set_title(col)\n    ax.set_ylabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:50.441312Z","iopub.execute_input":"2023-11-13T22:57:50.442195Z","iopub.status.idle":"2023-11-13T22:57:53.104174Z","shell.execute_reply.started":"2023-11-13T22:57:50.442167Z","shell.execute_reply":"2023-11-13T22:57:53.103419Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del(pea_hourly, temp, fig, f)\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:53.105335Z","iopub.execute_input":"2023-11-13T22:57:53.105865Z","iopub.status.idle":"2023-11-13T22:57:53.306200Z","shell.execute_reply.started":"2023-11-13T22:57:53.105835Z","shell.execute_reply":"2023-11-13T22:57:53.305505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Spatial indicators\nIn the case of spatial indicators we are dealing with continuous features (proportions, counts, means). One suitable plot to visualize this kind of data are histograms. As there are a lot of features here, we will group them by:\n- Building morphology\n- Population\n- Points of interes","metadata":{}},{"cell_type":"markdown","source":"### Building morphology","metadata":{}},{"cell_type":"code","source":"bdg_cols = [\n    \"Footprint Proportion\",\n    \"Footprint Mean\",\n    \"Footprint Stdev\",\n    \"Perimeter Total\",\n    \"Perimeter Mean\",\n    \"Perimeter Stdev\",\n    \"Complexity Mean\",\n    \"Complexity Stdev\",\n    \"Building Count\",\n]","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:53.307513Z","iopub.execute_input":"2023-11-13T22:57:53.308049Z","iopub.status.idle":"2023-11-13T22:57:53.318625Z","shell.execute_reply.started":"2023-11-13T22:57:53.308023Z","shell.execute_reply":"2023-11-13T22:57:53.317342Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"bdg_data = pd.read_csv(\"/kaggle/input/cool-quiet-city-competition/cozie_responses_and_physiological_data_training.csv\", usecols=target_cols+bdg_cols)\nbdg_data.info()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:53.319774Z","iopub.execute_input":"2023-11-13T22:57:53.320123Z","iopub.status.idle":"2023-11-13T22:57:54.966066Z","shell.execute_reply.started":"2023-11-13T22:57:53.320036Z","shell.execute_reply":"2023-11-13T22:57:54.964864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_noise_nearby`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[0]\ncols = 1\nrows = len(bdg_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,20), sharey=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(bdg_cols):\n    ax = axes[i]\n    f = sns.histplot(bdg_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:57:54.967691Z","iopub.execute_input":"2023-11-13T22:57:54.967959Z","iopub.status.idle":"2023-11-13T22:58:04.196697Z","shell.execute_reply.started":"2023-11-13T22:57:54.967936Z","shell.execute_reply":"2023-11-13T22:58:04.195665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_noise_kind`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[1]\ncols = 1\nrows = len(bdg_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,20), sharey=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(bdg_cols):\n    ax = axes[i]\n    f = sns.histplot(bdg_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:58:04.198620Z","iopub.execute_input":"2023-11-13T22:58:04.199282Z","iopub.status.idle":"2023-11-13T22:58:17.005790Z","shell.execute_reply.started":"2023-11-13T22:58:04.199227Z","shell.execute_reply":"2023-11-13T22:58:17.004542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_earphones`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[2]\ncols = 1\nrows = len(bdg_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,20), sharey=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(bdg_cols):\n    ax = axes[i]\n    f = sns.histplot(bdg_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:58:17.007253Z","iopub.execute_input":"2023-11-13T22:58:17.007599Z","iopub.status.idle":"2023-11-13T22:58:28.136379Z","shell.execute_reply.started":"2023-11-13T22:58:17.007561Z","shell.execute_reply":"2023-11-13T22:58:28.135434Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_thermal_preference`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[3]\ncols = 1\nrows = len(bdg_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,20), sharey=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(bdg_cols):\n    ax = axes[i]\n    f = sns.histplot(bdg_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:58:28.137438Z","iopub.execute_input":"2023-11-13T22:58:28.137999Z","iopub.status.idle":"2023-11-13T22:58:38.394939Z","shell.execute_reply.started":"2023-11-13T22:58:28.137973Z","shell.execute_reply":"2023-11-13T22:58:38.393474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del(bdg_data, fig, f)\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:58:38.396773Z","iopub.execute_input":"2023-11-13T22:58:38.397078Z","iopub.status.idle":"2023-11-13T22:58:38.560806Z","shell.execute_reply.started":"2023-11-13T22:58:38.397055Z","shell.execute_reply":"2023-11-13T22:58:38.559193Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Population","metadata":{}},{"cell_type":"code","source":"pop_cols = [\n    \"PopSum\",\n    \"Men\",\n    \"Women\",\n    \"Elderly\",\n    \"Youth\",\n    \"Children\"\n]","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:58:38.562951Z","iopub.execute_input":"2023-11-13T22:58:38.563357Z","iopub.status.idle":"2023-11-13T22:58:38.570286Z","shell.execute_reply.started":"2023-11-13T22:58:38.563325Z","shell.execute_reply":"2023-11-13T22:58:38.569291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pop_data = pd.read_csv(\"/kaggle/input/cool-quiet-city-competition/cozie_responses_and_physiological_data_training.csv\", usecols=id_cols+target_cols+pop_cols)\npop_data.info()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:58:38.571630Z","iopub.execute_input":"2023-11-13T22:58:38.571936Z","iopub.status.idle":"2023-11-13T22:58:40.224440Z","shell.execute_reply.started":"2023-11-13T22:58:38.571912Z","shell.execute_reply":"2023-11-13T22:58:40.222702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_noise_nearby`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[0]\ncols = 1\nrows = len(pop_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,20), sharey=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pop_cols):\n    ax = axes[i]\n    f = sns.histplot(pop_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:58:40.232155Z","iopub.execute_input":"2023-11-13T22:58:40.232537Z","iopub.status.idle":"2023-11-13T22:58:46.507438Z","shell.execute_reply.started":"2023-11-13T22:58:40.232507Z","shell.execute_reply":"2023-11-13T22:58:46.506239Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_noise_kind`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[1]\ncols = 1\nrows = len(pop_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,20), sharey=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pop_cols):\n    ax = axes[i]\n    f = sns.histplot(pop_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:58:46.508519Z","iopub.execute_input":"2023-11-13T22:58:46.508767Z","iopub.status.idle":"2023-11-13T22:58:55.186941Z","shell.execute_reply.started":"2023-11-13T22:58:46.508746Z","shell.execute_reply":"2023-11-13T22:58:55.185687Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_earphones`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[2]\ncols = 1\nrows = len(pop_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,20), sharey=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pop_cols):\n    ax = axes[i]\n    f = sns.histplot(pop_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:58:55.188256Z","iopub.execute_input":"2023-11-13T22:58:55.188610Z","iopub.status.idle":"2023-11-13T22:59:02.275121Z","shell.execute_reply.started":"2023-11-13T22:58:55.188580Z","shell.execute_reply":"2023-11-13T22:59:02.274435Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_thermal_preference`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[3]\ncols = 1\nrows = len(pop_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,20), sharey=True)\naxes = axes.flatten()\n    \nfor i, col in enumerate(pop_cols):\n    ax = axes[i]\n    f = sns.histplot(pop_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:59:02.276249Z","iopub.execute_input":"2023-11-13T22:59:02.277441Z","iopub.status.idle":"2023-11-13T22:59:09.342183Z","shell.execute_reply.started":"2023-11-13T22:59:02.277390Z","shell.execute_reply":"2023-11-13T22:59:09.341321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del(pop_data, fig, f)\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:59:09.343491Z","iopub.execute_input":"2023-11-13T22:59:09.343733Z","iopub.status.idle":"2023-11-13T22:59:09.660577Z","shell.execute_reply.started":"2023-11-13T22:59:09.343712Z","shell.execute_reply":"2023-11-13T22:59:09.658936Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Point of interest","metadata":{}},{"cell_type":"code","source":"interest_cols = [\n    \"Civic\",\n    \"Commercial\",\n    \"Entertainment\",\n    \"Food\",\n    \"Healthcare\",\n    \"Institutional\",\n    \"Recreational\",\n    \"Social\",\n    \"Green View Mean\",\n    \"Green View Stdev\",\n    \"Sky View Mean\",\n    \"Sky View Stdev\",\n    \"Building View Mean\",\n    \"Building View Stdev\",\n    \"Road View Mean\",\n    \"Road View Stdev\",\n    \"Visual Complexity Mean\",\n    \"Visual Complexity Stdev\"\n]","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:59:09.662075Z","iopub.execute_input":"2023-11-13T22:59:09.662644Z","iopub.status.idle":"2023-11-13T22:59:09.670447Z","shell.execute_reply.started":"2023-11-13T22:59:09.662610Z","shell.execute_reply":"2023-11-13T22:59:09.668920Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"interest_data = pd.read_csv(\"/kaggle/input/cool-quiet-city-competition/cozie_responses_and_physiological_data_training.csv\", usecols=target_cols+interest_cols)\ninterest_data.info()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:59:09.673026Z","iopub.execute_input":"2023-11-13T22:59:09.673501Z","iopub.status.idle":"2023-11-13T22:59:11.512940Z","shell.execute_reply.started":"2023-11-13T22:59:09.673469Z","shell.execute_reply":"2023-11-13T22:59:11.511814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_noise_nearby`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[0]\ncols = 1\nrows = len(interest_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,30))\naxes = axes.flatten()\n    \nfor i, col in enumerate(interest_cols):\n    ax = axes[i]\n    f = sns.histplot(interest_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:59:11.514415Z","iopub.execute_input":"2023-11-13T22:59:11.515285Z","iopub.status.idle":"2023-11-13T22:59:29.614870Z","shell.execute_reply.started":"2023-11-13T22:59:11.515253Z","shell.execute_reply":"2023-11-13T22:59:29.613268Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_noise_kind`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[1]\ncols = 1\nrows = len(interest_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,50))\naxes = axes.flatten()\n    \nfor i, col in enumerate(interest_cols):\n    ax = axes[i]\n    f = sns.histplot(interest_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:59:29.616263Z","iopub.execute_input":"2023-11-13T22:59:29.616562Z","iopub.status.idle":"2023-11-13T22:59:55.181332Z","shell.execute_reply.started":"2023-11-13T22:59:29.616539Z","shell.execute_reply":"2023-11-13T22:59:55.179781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_earphones`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[2]\ncols = 1\nrows = len(interest_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,50))\naxes = axes.flatten()\n    \nfor i, col in enumerate(interest_cols):\n    ax = axes[i]\n    f = sns.histplot(interest_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T22:59:55.182810Z","iopub.execute_input":"2023-11-13T22:59:55.183113Z","iopub.status.idle":"2023-11-13T23:00:16.955917Z","shell.execute_reply.started":"2023-11-13T22:59:55.183081Z","shell.execute_reply":"2023-11-13T23:00:16.954689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_thermal_preference`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[3]\ncols = 1\nrows = len(interest_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,50))\naxes = axes.flatten()\n    \nfor i, col in enumerate(interest_cols):\n    ax = axes[i]\n    f = sns.histplot(interest_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T23:00:16.957350Z","iopub.execute_input":"2023-11-13T23:00:16.957644Z","iopub.status.idle":"2023-11-13T23:00:37.001516Z","shell.execute_reply.started":"2023-11-13T23:00:16.957621Z","shell.execute_reply":"2023-11-13T23:00:37.000524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"del(interest_data, fig, f)\ngc.collect()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T23:00:37.002772Z","iopub.execute_input":"2023-11-13T23:00:37.003047Z","iopub.status.idle":"2023-11-13T23:00:37.297743Z","shell.execute_reply.started":"2023-11-13T23:00:37.003022Z","shell.execute_reply":"2023-11-13T23:00:37.296158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Weather data\nFor this part we are going to use the train data set resulting from [notebook 2](https://www.kaggle.com/code/mariofrei/02-example-base-analysis-and-model-example); in this data set each row is already mapped to the closest weather station and the weather features obtained from it. We are using histograms again, as we are dealing with continuous features.","metadata":{}},{"cell_type":"code","source":"weather_cols = [\n    \"ws_rainfall\",          \n    \"ws_air_temperature\", \n    \"ws_relative_humidity\", \n    \"ws_wind_speed\",\n    \"ws_wind_direction\"  \n]","metadata":{"execution":{"iopub.status.busy":"2023-11-13T23:00:37.299345Z","iopub.execute_input":"2023-11-13T23:00:37.299849Z","iopub.status.idle":"2023-11-13T23:00:37.308594Z","shell.execute_reply.started":"2023-11-13T23:00:37.299819Z","shell.execute_reply":"2023-11-13T23:00:37.306941Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"weather_data = pd.read_csv(\"/kaggle/input/cqc-competition-nb02-output/nb02_train.csv\", usecols=target_cols+weather_cols)\nweather_data.info()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T23:00:37.310968Z","iopub.execute_input":"2023-11-13T23:00:37.311303Z","iopub.status.idle":"2023-11-13T23:00:37.974220Z","shell.execute_reply.started":"2023-11-13T23:00:37.311276Z","shell.execute_reply":"2023-11-13T23:00:37.972479Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_noise_nearby`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[0]\ncols = 1\nrows = len(weather_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,14))\naxes = axes.flatten()\n    \nfor i, col in enumerate(weather_cols):\n    ax = axes[i]\n    f = sns.histplot(weather_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T23:00:37.975306Z","iopub.status.idle":"2023-11-13T23:00:37.975706Z","shell.execute_reply.started":"2023-11-13T23:00:37.975515Z","shell.execute_reply":"2023-11-13T23:00:37.975533Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_noise_kind`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[1]\ncols = 1\nrows = len(weather_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,14))\naxes = axes.flatten()\n    \nfor i, col in enumerate(weather_cols):\n    ax = axes[i]\n    f = sns.histplot(weather_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T23:00:37.977692Z","iopub.status.idle":"2023-11-13T23:00:37.978144Z","shell.execute_reply.started":"2023-11-13T23:00:37.977914Z","shell.execute_reply":"2023-11-13T23:00:37.977935Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_earphones`","metadata":{}},{"cell_type":"code","source":"#### target_col = target_cols[2]\ncols = 1\nrows = len(weather_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,14))\naxes = axes.flatten()\n    \nfor i, col in enumerate(weather_cols):\n    ax = axes[i]\n    f = sns.histplot(weather_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T23:00:37.980231Z","iopub.status.idle":"2023-11-13T23:00:37.980692Z","shell.execute_reply.started":"2023-11-13T23:00:37.980466Z","shell.execute_reply":"2023-11-13T23:00:37.980486Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### `q_thermal_preference`","metadata":{}},{"cell_type":"code","source":"target_col = target_cols[3]\ncols = 1\nrows = len(weather_cols)\nfig, axes = plt.subplots(rows, cols, figsize=(6,14))\naxes = axes.flatten()\n    \nfor i, col in enumerate(weather_cols):\n    ax = axes[i]\n    f = sns.histplot(weather_data, x=col, hue=target_col, palette=\"Set2\", ax=ax, bins=30)\n    ax.set_title(col)\n    ax.set_xlabel(\"\")\n\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T23:00:37.981938Z","iopub.status.idle":"2023-11-13T23:00:37.982341Z","shell.execute_reply.started":"2023-11-13T23:00:37.982138Z","shell.execute_reply":"2023-11-13T23:00:37.982157Z"},"trusted":true},"execution_count":null,"outputs":[]}]}