{"metadata":{"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":61145,"databundleVersionId":7047669,"sourceType":"competition"}],"dockerImageVersionId":30646,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"papermill":{"default_parameters":{},"duration":8816.051734,"end_time":"2024-01-29T07:16:03.940045","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-01-29T04:49:07.888311","version":"2.4.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Loading Libraries and Datasets","metadata":{"papermill":{"duration":0.013798,"end_time":"2024-01-29T04:49:11.324968","exception":false,"start_time":"2024-01-29T04:49:11.31117","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# En primer lugar, se instala el paquete iterative-stratification que será de utilidad para realizar procesos iterativos que ayudarán a clasificar y validar información.","metadata":{}},{"cell_type":"code","source":"!pip install -q iterative-stratification","metadata":{"papermill":{"duration":13.360127,"end_time":"2024-01-29T04:49:24.69885","exception":false,"start_time":"2024-01-29T04:49:11.338723","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:09:36.267547Z","iopub.execute_input":"2024-03-14T17:09:36.267916Z","iopub.status.idle":"2024-03-14T17:09:51.127833Z","shell.execute_reply.started":"2024-03-14T17:09:36.267887Z","shell.execute_reply":"2024-03-14T17:09:51.126684Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Luego, se importarán las bibliotecas y módulos a utilizar en el código","metadata":{}},{"cell_type":"code","source":"#numpy permite realizar operaciones numéricas y trabajar con matrices\nimport numpy as np\n#pandas sirve para manipular y analizar información\nimport pandas as pd\n#polars analiza los datos de manera eficiente\nimport polars as pl\n#pyplot implementa gráficos y visualizaciones\nimport matplotlib.pyplot as plt\n#seaborn dibuja los gráficos estadísticos\nimport seaborn as sns\n#tensorflow se utliza pra permitir el aprendizaje automático y el desarrollo de modelos de IA\nimport tensorflow as tf\n#os permite manipular la ruta de los archivos e interactuar con el sistema operativo\nimport os\n#optuna es útil para la optimización automática de hiperparámetros\nimport optuna\n#holidays proporciona la información de festividades en todo el mundo\nimport holidays\n#copy permite copiar objetos\nimport copy\n\n#se importan más clases y funciones que proporcionan herramientas para poder trabajar con datos y modelos de aprendizaje automático\nfrom category_encoders import OneHotEncoder, MEstimateEncoder, CatBoostEncoder, BinaryEncoder\nfrom iterstrat.ml_stratifiers import MultilabelStratifiedKFold, RepeatedMultilabelStratifiedKFold\nfrom sklearn.cluster import KMeans\nfrom sklearn.compose import ColumnTransformer\nfrom sklearn.model_selection import StratifiedKFold, RepeatedStratifiedKFold, TimeSeriesSplit\nfrom sklearn.decomposition import PCA\nfrom sklearn.ensemble import RandomForestClassifier, ExtraTreesClassifier\nfrom sklearn.ensemble import GradientBoostingClassifier, HistGradientBoostingClassifier\nfrom sklearn.ensemble import VotingClassifier\nfrom sklearn.impute import KNNImputer, SimpleImputer\nfrom sklearn.metrics import accuracy_score, roc_auc_score, roc_curve, make_scorer\nfrom sklearn.metrics.pairwise import euclidean_distances\nfrom sklearn.naive_bayes import GaussianNB, BernoulliNB\nfrom sklearn.pipeline import make_pipeline, Pipeline\nfrom sklearn.base import BaseEstimator, TransformerMixin, ClassifierMixin, clone\nfrom sklearn.multioutput import ClassifierChain\nfrom sklearn.preprocessing import FunctionTransformer, StandardScaler, PowerTransformer, LabelEncoder\nfrom sklearn.linear_model import LogisticRegression, RidgeClassifier\nfrom sklearn.svm import SVC\nfrom sklearn.discriminant_analysis import LinearDiscriminantAnalysis\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.inspection import permutation_importance\nfrom scipy.cluster.hierarchy import dendrogram, linkage\nfrom scipy.spatial.distance import squareform\nfrom xgboost import XGBClassifier, XGBRFClassifier\nfrom lightgbm import LGBMClassifier\nfrom catboost import CatBoostClassifier\nfrom sklearn.feature_selection import SequentialFeatureSelector\n\nsns.set_theme(style = 'white', palette = 'viridis')\npal = sns.color_palette('viridis')\n\npd.set_option('display.max_rows', 100)","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":19.724847,"end_time":"2024-01-29T04:49:44.43698","exception":false,"start_time":"2024-01-29T04:49:24.712133","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:25:29.693885Z","iopub.execute_input":"2024-03-14T17:25:29.694284Z","iopub.status.idle":"2024-03-14T17:25:52.629660Z","shell.execute_reply.started":"2024-03-14T17:25:29.694249Z","shell.execute_reply":"2024-03-14T17:25:52.628570Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Se preprocesan los datos importados con la ayuda de polars","metadata":{}},{"cell_type":"code","source":"#Se convierten los datos flotantes y fechas. Además, se extraen los componentes de fechas.\npreprocessor = [\n    pl.col('dT').cast(pl.Float32),\n    \n    pl.col('ts_oxygen_saturation').cast(pl.Float32),\n    pl.col('ts_resting_heart_rate').cast(pl.Float32),\n    \n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z'),\n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.year().alias('time_year').cast(pl.UInt16),\n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.month().alias('time_month').cast(pl.UInt8),\n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.week().alias('time_week').cast(pl.UInt8),\n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.weekday().alias('time_weekday').cast(pl.UInt8),\n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.ordinal_day().alias('time_day_of_year').cast(pl.UInt16),\n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.day().alias('time_day').cast(pl.UInt8),\n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.hour().alias('time_hour').cast(pl.UInt8),\n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.minute().alias('time_minute').cast(pl.UInt8),\n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.second().alias('time_second').cast(pl.UInt8),\n    pl.col('time').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.microsecond().alias('time_micro').cast(pl.UInt32),\n    \n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z'),\n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.year().alias('loc_year').cast(pl.UInt16),\n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.month().alias('loc_month').cast(pl.UInt8),\n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.week().alias('loc_week').cast(pl.UInt8),\n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.weekday().alias('loc_weekday').cast(pl.UInt8),\n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.ordinal_day().alias('loc_day_of_year').cast(pl.UInt16),\n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.day().alias('loc_day').cast(pl.UInt8),\n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.hour().alias('loc_hour').cast(pl.UInt8),\n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.minute().alias('loc_minute').cast(pl.UInt8),\n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.second().alias('loc_second').cast(pl.UInt8),\n    pl.col('ws_timestamp_location').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.microsecond().alias('loc_micro').cast(pl.UInt32),\n    \n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z'),\n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.year().alias('start_year').cast(pl.UInt16),\n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.month().alias('start_month').cast(pl.UInt8),\n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.week().alias('start_week').cast(pl.UInt8),\n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.weekday().alias('start_weekday').cast(pl.UInt8),\n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.ordinal_day().alias('start_day_of_year').cast(pl.UInt16),\n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.day().alias('start_day').cast(pl.UInt8),\n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.hour().alias('start_hour').cast(pl.UInt8),\n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.minute().alias('start_minute').cast(pl.UInt8),\n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.second().alias('start_second').cast(pl.UInt8),\n    pl.col('ws_timestamp_start').str.to_datetime('%Y-%m-%d %H:%M:%S%.6f%Z').dt.microsecond().alias('start_micro').cast(pl.UInt32)\n]\n\npreprocessor_2 = [\n    pl.col('').str.to_datetime('%Y-%m-%d %H:%M:%S%Z'),\n    pl.col('').str.to_datetime('%Y-%m-%d %H:%M:%S%Z').dt.year().alias('year').cast(pl.UInt16),\n    pl.col('').str.to_datetime('%Y-%m-%d %H:%M:%S%Z').dt.month().alias('month').cast(pl.UInt8),\n    pl.col('').str.to_datetime('%Y-%m-%d %H:%M:%S%Z').dt.day().alias('day').cast(pl.UInt8),\n    pl.col('').str.to_datetime('%Y-%m-%d %H:%M:%S%Z').dt.hour().alias('hour').cast(pl.UInt8),\n    pl.col('').str.to_datetime('%Y-%m-%d %H:%M:%S%Z').dt.minute().alias('minute').cast(pl.UInt8),\n    \n    pl.col('value').cast(pl.Float32)\n]\n\n#Calcula las diferencias de tiempos entre las columnas relacionadas con las horas\nduration_calculator = [\n    (pl.duration(days = pl.col('time_day_of_year'), hours = pl.col('time_hour'), minutes = pl.col('time_minute'), seconds = pl.col('time_second'), microseconds = pl.col('time_micro')) -\n    pl.duration(days = pl.col('start_day_of_year'), hours = pl.col('start_hour'), minutes = pl.col('start_minute'), seconds = pl.col('start_second'), microseconds = pl.col('start_micro'))).dt.total_microseconds().alias('time_start_difference'),\n    (pl.duration(days = pl.col('time_day_of_year'), hours = pl.col('time_hour'), minutes = pl.col('time_minute'), seconds = pl.col('time_second'), microseconds = pl.col('time_micro')) -\n    pl.duration(days = pl.col('loc_day_of_year'), hours = pl.col('loc_hour'), minutes = pl.col('loc_minute'), seconds = pl.col('loc_second'), microseconds = pl.col('loc_micro'))).dt.total_microseconds().alias('time_loc_difference'),\n    (pl.duration(days = pl.col('loc_day_of_year'), hours = pl.col('loc_hour'), minutes = pl.col('loc_minute'), seconds = pl.col('loc_second'), microseconds = pl.col('loc_micro')) -\n     pl.duration(days = pl.col('start_day_of_year'), hours = pl.col('start_hour'), minutes = pl.col('start_minute'), seconds = pl.col('start_second'), microseconds = pl.col('start_micro'))).dt.total_microseconds().alias('start_loc_difference')\n]","metadata":{"papermill":{"duration":0.064479,"end_time":"2024-01-29T04:49:44.521031","exception":false,"start_time":"2024-01-29T04:49:44.456552","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:29:42.848257Z","iopub.execute_input":"2024-03-14T17:29:42.848769Z","iopub.status.idle":"2024-03-14T17:29:42.905425Z","shell.execute_reply.started":"2024-03-14T17:29:42.848739Z","shell.execute_reply":"2024-03-14T17:29:42.904496Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Se cargan los conjuntos de datos de entrenamiento  y se aplica el preprocesamiento anterior a los nuevos datos.","metadata":{}},{"cell_type":"code","source":"train = pl.scan_csv('/kaggle/input/cool-quiet-city-competition/cozie_responses_and_physiological_data_training.csv').\\\n        with_columns(preprocessor).with_columns(duration_calculator)\ntest = pl.scan_csv('/kaggle/input/cool-quiet-city-competition/cozie_responses_and_physiological_data_test_public.csv').\\\n       with_columns(preprocessor).with_columns(duration_calculator)","metadata":{"papermill":{"duration":0.082503,"end_time":"2024-01-29T04:49:44.616906","exception":false,"start_time":"2024-01-29T04:49:44.534403","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:34:29.625233Z","iopub.execute_input":"2024-03-14T17:34:29.626017Z","iopub.status.idle":"2024-03-14T17:34:29.697377Z","shell.execute_reply.started":"2024-03-14T17:34:29.625979Z","shell.execute_reply":"2024-03-14T17:34:29.696373Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Se cargan y procesan mútiples conjuntos de datos meteorológicos y se almacenan en variables","metadata":{}},{"cell_type":"code","source":"#temperatura del aire\nair_temp = pl.scan_csv('/kaggle/input/cool-quiet-city-competition/weather_air-temperature.csv').\\\n           melt('').\\\n           with_columns(preprocessor_2).\\\n           group_by(['year', 'month', 'day', 'hour', 'minute']).\\\n           agg(\n               pl.col('value').mean().alias('mean_air_temp'),\n               pl.col('value').std().alias('std_air_temp'),\n           )\n#lluvias\nrainfall = pl.scan_csv('/kaggle/input/cool-quiet-city-competition/weather_rainfall.csv').\\\n           melt('').\\\n           with_columns(preprocessor_2).\\\n           group_by(['year', 'month', 'day', 'hour', 'minute']).\\\n           agg(\n               pl.col('value').mean().alias('mean_rainfall'),\n               pl.col('value').std().alias('std_rainfall'),\n           )\n#humedad\nhumidity = pl.scan_csv('/kaggle/input/cool-quiet-city-competition/weather_relative-humidity.csv').\\\n           melt('').\\\n           with_columns(preprocessor_2).\\\n           group_by(['year', 'month', 'day', 'hour', 'minute']).\\\n           agg(\n               pl.col('value').mean().alias('mean_humidity'),\n               pl.col('value').std().alias('std_humidity'),\n           )\n#estaciones\nstations = pl.read_csv('/kaggle/input/cool-quiet-city-competition/weather_stations.csv')\n#dirección del viento\nwind_direction = pl.scan_csv('/kaggle/input/cool-quiet-city-competition/weather_wind-direction.csv').\\\n                 melt('').\\\n                 with_columns(preprocessor_2).\\\n                 group_by(['year', 'month', 'day', 'hour', 'minute']).\\\n                 agg(\n                     pl.col('value').mean().alias('mean_wind_direction'),\n                     pl.col('value').std().alias('std_wind_direction')\n                 )\n#velocidad del viento\nwind_speed = pl.scan_csv('/kaggle/input/cool-quiet-city-competition/weather_wind-speed.csv').\\\n             melt('').\\\n             with_columns(preprocessor_2).\\\n             group_by(['year', 'month', 'day', 'hour', 'minute']).\\\n             agg(\n                 pl.col('value').mean().alias('mean_wind_speed'),\n                 pl.col('value').std().alias('std_wind_speed'),\n             )","metadata":{"papermill":{"duration":0.102406,"end_time":"2024-01-29T04:49:44.732804","exception":false,"start_time":"2024-01-29T04:49:44.630398","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:43:30.696875Z","iopub.execute_input":"2024-03-14T17:43:30.697273Z","iopub.status.idle":"2024-03-14T17:43:30.727459Z","shell.execute_reply.started":"2024-03-14T17:43:30.697245Z","shell.execute_reply":"2024-03-14T17:43:30.726502Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Dataset Information","metadata":{"papermill":{"duration":0.01312,"end_time":"2024-01-29T04:49:44.759911","exception":false,"start_time":"2024-01-29T04:49:44.746791","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# Se realizan estadísticas descriptivas sobre la información obtenida","metadata":{}},{"cell_type":"code","source":"#Se obtienen valores estadísticos como la mediana, desviación estándar, valores min y max, etc.\ndef desc_dataset_pl(df):\n    return pl.concat([\n        pl.DataFrame({'key' : df.columns, 'dataset_length' : len(df)}),\n        df.null_count().transpose().rename({'column_0' : 'null'}),\n        df.approx_n_unique().transpose().rename({'column_0' : 'nunique'}),\n        df.mean().transpose().rename({'column_0' : 'mean'}),\n        df.std().transpose().rename({'column_0' : 'std'}),\n        df.median().transpose().rename({'column_0' : 'median'}),\n        df.min().transpose().rename({'column_0' : 'min'}),\n        df.max().transpose().rename({'column_0' : 'max'})\n    ], how = 'horizontal').with_columns(\n        (pl.col('dataset_length') - pl.col('null')).alias('count'),\n        (pl.col('null') / pl.col('dataset_length') * 100).alias('%null'),\n        (pl.col('nunique') / pl.col('dataset_length') * 100).alias('%unique')\n    ).drop('dataset_length').to_pandas()","metadata":{"papermill":{"duration":0.025358,"end_time":"2024-01-29T04:49:44.798669","exception":false,"start_time":"2024-01-29T04:49:44.773311","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:43:36.075560Z","iopub.execute_input":"2024-03-14T17:43:36.075968Z","iopub.status.idle":"2024-03-14T17:43:36.085680Z","shell.execute_reply.started":"2024-03-14T17:43:36.075937Z","shell.execute_reply":"2024-03-14T17:43:36.084482Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#desc_dataset_pl(train)","metadata":{"papermill":{"duration":0.0217,"end_time":"2024-01-29T04:49:44.834261","exception":false,"start_time":"2024-01-29T04:49:44.812561","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:38:14.179240Z","iopub.execute_input":"2024-03-14T17:38:14.180372Z","iopub.status.idle":"2024-03-14T17:38:14.184418Z","shell.execute_reply.started":"2024-03-14T17:38:14.180338Z","shell.execute_reply":"2024-03-14T17:38:14.183436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#desc_dataset_pl(test)","metadata":{"papermill":{"duration":0.021945,"end_time":"2024-01-29T04:49:44.869571","exception":false,"start_time":"2024-01-29T04:49:44.847626","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:38:17.626673Z","iopub.execute_input":"2024-03-14T17:38:17.627244Z","iopub.status.idle":"2024-03-14T17:38:17.634117Z","shell.execute_reply.started":"2024-03-14T17:38:17.627197Z","shell.execute_reply":"2024-03-14T17:38:17.631203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# TS Features Aggregation","metadata":{"papermill":{"duration":0.013129,"end_time":"2024-01-29T04:49:44.960247","exception":false,"start_time":"2024-01-29T04:49:44.947118","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Se combinan los DataFrames \"train\" y \"test\" en uno solo : \"how\"","metadata":{}},{"cell_type":"code","source":"df_all = pl.concat([train, test], how = 'vertical')","metadata":{"papermill":{"duration":0.025587,"end_time":"2024-01-29T04:49:44.999204","exception":false,"start_time":"2024-01-29T04:49:44.973617","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:43:39.230913Z","iopub.execute_input":"2024-03-14T17:43:39.231283Z","iopub.status.idle":"2024-03-14T17:43:39.236237Z","shell.execute_reply.started":"2024-03-14T17:43:39.231253Z","shell.execute_reply":"2024-03-14T17:43:39.235343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Se crean nuevos DataFrames que registran la cantidad de encuestas por día, hora y minuto","metadata":{}},{"cell_type":"code","source":"#registro de la cantidad de encuestas por día, hora y minuto\nsurvey_counter_day = df_all.group_by(\n    ['id_participant', 'time_year', 'time_month', 'time_day']\n).agg(\n    pl.col('id_participant').count().alias('survey_count_per_day').cast(pl.UInt16)\n)\nsurvey_counter_hour = df_all.group_by(\n    ['id_participant', 'time_year', 'time_month', 'time_day', 'time_hour']\n).agg(\n    pl.col('id_participant').count().alias('survey_count_per_hour').cast(pl.UInt16)\n)\nsurvey_counter_minute = df_all.group_by(\n    ['id_participant', 'time_year', 'time_month', 'time_day', 'time_hour', 'time_minute']\n).agg(\n    pl.col('id_participant').count().alias('survey_count_per_minute').cast(pl.UInt8)\n)","metadata":{"papermill":{"duration":0.025501,"end_time":"2024-01-29T04:49:45.038174","exception":false,"start_time":"2024-01-29T04:49:45.012673","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:43:40.774631Z","iopub.execute_input":"2024-03-14T17:43:40.775738Z","iopub.status.idle":"2024-03-14T17:43:40.784610Z","shell.execute_reply.started":"2024-03-14T17:43:40.775690Z","shell.execute_reply":"2024-03-14T17:43:40.783565Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Se crean nuevas estadísticas para nuevas variables, esta vez relacionadas con el comportamiento fisiológico de las personas","metadata":{}},{"cell_type":"code","source":"#ritmo cardiaco en reposo\nrest_heart_rate_agg = df_all.group_by(\n    ['time_year', 'time_week', 'id_participant']\n).agg(\n    pl.col('ts_resting_heart_rate').mean().alias('mean_resting_heart_rate'),\n    pl.col('ts_resting_heart_rate').std().alias('std_resting_heart_rate')\n)\n#saturación de oxigeno\noxygen_saturation_agg = df_all.group_by(\n    ['time_year', 'time_week', 'id_participant']\n).agg(\n    pl.col('ts_oxygen_saturation').mean().alias('mean_oxygen_saturation'),\n    pl.col('ts_oxygen_saturation').std().alias('std_oxygen_saturation')\n)\n#tiempo de pie\nstand_time_agg = df_all.group_by(\n    ['time_year', 'time_month', 'time_day', 'id_participant']\n).agg(\n    pl.col('ts_stand_time').mean().alias('mean_stand_time'),\n    pl.col('ts_stand_time').std().alias('std_stand_time')\n)\n#conteo de pasos\nstep_count_agg = df_all.group_by(\n    ['time_year', 'time_month', 'time_day', 'id_participant']\n).agg(\n    pl.col('ts_step_count').mean().alias('mean_step_count'),\n    pl.col('ts_step_count').std().alias('std_step_count')\n)\n#distancia caminada\nwalking_distance_agg = df_all.group_by(\n    ['time_year', 'time_month', 'time_day', 'id_participant']\n).agg(\n    pl.col('ts_walking_distance').mean().alias('mean_walking_distance'),\n    pl.col('ts_walking_distance').std().alias('std_walking_distance')\n)\n#exposición al ambiente sonoro\naudio_exposure_environment_agg = df_all.group_by(\n    ['time_year', 'time_month', 'time_day', 'id_participant']\n).agg(\n    pl.col('ts_audio_exposure_environment').mean().alias('mean_audio_exposure_environment'),\n    pl.col('ts_audio_exposure_environment').std().alias('std_audio_exposure_environment')\n)\n#ritmo cardiaco\nheart_rate_agg = df_all.group_by(\n    ['time_year', 'time_month', 'time_day', 'id_participant']\n).agg(\n    pl.col('ts_heart_rate').mean().alias('mean_heart_rate'),\n    pl.col('ts_heart_rate').std().alias('std_heart_rate')\n)","metadata":{"papermill":{"duration":0.030034,"end_time":"2024-01-29T04:49:45.081786","exception":false,"start_time":"2024-01-29T04:49:45.051752","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:47:39.581621Z","iopub.execute_input":"2024-03-14T17:47:39.582073Z","iopub.status.idle":"2024-03-14T17:47:39.595721Z","shell.execute_reply.started":"2024-03-14T17:47:39.582041Z","shell.execute_reply":"2024-03-14T17:47:39.594634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Los datos fisiológicos obtenidos se unen con los DataFrames de entrenamiento y prueba, lo que enriquece el conjunto de datos inicial","metadata":{}},{"cell_type":"code","source":"#unión de los DataFrames\ntrain = train.join(\n    rest_heart_rate_agg, on = ['time_year', 'time_week', 'id_participant'], how = 'left'\n).join(\n    oxygen_saturation_agg, on = ['time_year', 'time_week', 'id_participant'], how = 'left'\n).join(\n    stand_time_agg, on = ['time_year', 'time_month', 'time_day', 'id_participant'], how = 'left'\n).join(\n    step_count_agg, on = ['time_year', 'time_month', 'time_day', 'id_participant'], how = 'left'\n).join(\n    walking_distance_agg, on = ['time_year', 'time_month', 'time_day', 'id_participant'], how = 'left'\n).join(\n    audio_exposure_environment_agg, on = ['time_year', 'time_month', 'time_day', 'id_participant'], how = 'left'\n).join(\n    heart_rate_agg, on = ['time_year', 'time_month', 'time_day', 'id_participant'], how = 'left'\n).join(\n    survey_counter_day, on = ['id_participant', 'time_year', 'time_month', 'time_day'], how = 'left'\n).join(\n    survey_counter_hour, on = ['id_participant', 'time_year', 'time_month', 'time_day', 'time_hour'], how = 'left'\n).join(\n    survey_counter_minute, on = ['id_participant', 'time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'], how = 'left'\n)\ntest = test.join(\n    rest_heart_rate_agg, on = ['time_year', 'time_week', 'id_participant'], how = 'left'\n).join(\n    oxygen_saturation_agg, on = ['time_year', 'time_week', 'id_participant'], how = 'left'\n).join(\n    stand_time_agg, on = ['time_year', 'time_month', 'time_day', 'id_participant'], how = 'left'\n).join(\n    step_count_agg, on = ['time_year', 'time_month', 'time_day', 'id_participant'], how = 'left'\n).join(\n    walking_distance_agg, on = ['time_year', 'time_month', 'time_day', 'id_participant'], how = 'left'\n).join(\n    audio_exposure_environment_agg, on = ['time_year', 'time_month', 'time_day', 'id_participant'], how = 'left'\n).join(\n    heart_rate_agg, on = ['time_year', 'time_month', 'time_day', 'id_participant'], how = 'left'\n).join(\n    survey_counter_day, on = ['id_participant', 'time_year', 'time_month', 'time_day'], how = 'left'\n).join(\n    survey_counter_hour, on = ['id_participant', 'time_year', 'time_month', 'time_day', 'time_hour'], how = 'left'\n).join(\n    survey_counter_minute, on = ['id_participant', 'time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'], how = 'left'\n)","metadata":{"papermill":{"duration":0.040233,"end_time":"2024-01-29T04:49:45.135455","exception":false,"start_time":"2024-01-29T04:49:45.095222","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:47:41.723800Z","iopub.execute_input":"2024-03-14T17:47:41.724240Z","iopub.status.idle":"2024-03-14T17:47:41.748903Z","shell.execute_reply.started":"2024-03-14T17:47:41.724209Z","shell.execute_reply":"2024-03-14T17:47:41.747871Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Station and Weather","metadata":{"papermill":{"duration":0.012909,"end_time":"2024-01-29T04:49:45.16168","exception":false,"start_time":"2024-01-29T04:49:45.148771","status":"completed"},"tags":[]}},{"cell_type":"code","source":"#Se busca la estación meteorológica más cercana a la ubicación proporcionada mediante coordenadas(latitud y longitud) en \"x\"\ndef find_closest_station(x):\n    if x[0] == None: return None\n    latitude = float(x[0][:18])\n    longitude = float(x[0][19:])\n    return stations.row(stations.select(\n        distance = (pl.col('latitude') - latitude) ** 2 + (pl.col('longitude') - longitude) ** 2\n    )['distance'].arg_min())[1]","metadata":{"papermill":{"duration":0.022635,"end_time":"2024-01-29T04:49:45.197578","exception":false,"start_time":"2024-01-29T04:49:45.174943","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:55:00.225209Z","iopub.execute_input":"2024-03-14T17:55:00.225644Z","iopub.status.idle":"2024-03-14T17:55:00.232955Z","shell.execute_reply.started":"2024-03-14T17:55:00.225613Z","shell.execute_reply":"2024-03-14T17:55:00.231898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Encuentra la estación más cercana y proporciona una columna con estos datos\ndef map_station(df):\n    return df.select(pl.col('ws_latitude').cast(pl.Utf8).map_elements(\n        lambda x: x + '0' * (18 - len(x))\n    ) + ' ' + pl.col('ws_longitude').cast(pl.Utf8).map_elements(\n        lambda x: x + '0' * (18 - len(x))\n    )).map_rows(find_closest_station)['map']","metadata":{"papermill":{"duration":0.02323,"end_time":"2024-01-29T04:49:45.234313","exception":false,"start_time":"2024-01-29T04:49:45.211083","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T17:55:02.699858Z","iopub.execute_input":"2024-03-14T17:55:02.700260Z","iopub.status.idle":"2024-03-14T17:55:02.706665Z","shell.execute_reply.started":"2024-03-14T17:55:02.700229Z","shell.execute_reply":"2024-03-14T17:55:02.705626Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Se unen los datos meteorológicos obtenidos y se alimenta al modelo de aprendizaje automático","metadata":{}},{"cell_type":"code","source":"#unión de los DataFrames meteorológicos con los iniciales\ntrain = train.join(\n    air_temp,\n    left_on = ['time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'],\n    right_on = ['year', 'month', 'day', 'hour', 'minute'],\n    how = 'left'\n).drop(['year', 'month', 'day', 'hour', 'minute']).join(\n    rainfall,\n    left_on = ['time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'],\n    right_on = ['year', 'month', 'day', 'hour', 'minute'],\n    how = 'left'\n).drop(['year', 'month', 'day', 'hour', 'minute']).join(\n    humidity,\n    left_on = ['time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'],\n    right_on = ['year', 'month', 'day', 'hour', 'minute'],\n    how = 'left'\n).drop(['year', 'month', 'day', 'hour', 'minute']).join(\n    wind_direction,\n    left_on = ['time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'],\n    right_on = ['year', 'month', 'day', 'hour', 'minute'],\n    how = 'left'\n).drop(['year', 'month', 'day', 'hour', 'minute']).join(\n    wind_speed,\n    left_on = ['time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'],\n    right_on = ['year', 'month', 'day', 'hour', 'minute'],\n    how = 'left'\n).drop(['year', 'month', 'day', 'hour', 'minute'])\n\ntest = test.join(\n    air_temp,\n    left_on = ['time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'],\n    right_on = ['year', 'month', 'day', 'hour', 'minute'],\n    how = 'left'\n).drop(['year', 'month', 'day', 'hour', 'minute']).join(\n    rainfall,\n    left_on = ['time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'],\n    right_on = ['year', 'month', 'day', 'hour', 'minute'],\n    how = 'left'\n).drop(['year', 'month', 'day', 'hour', 'minute']).join(\n    humidity,\n    left_on = ['time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'],\n    right_on = ['year', 'month', 'day', 'hour', 'minute'],\n    how = 'left'\n).drop(['year', 'month', 'day', 'hour', 'minute']).join(\n    wind_direction,\n    left_on = ['time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'],\n    right_on = ['year', 'month', 'day', 'hour', 'minute'],\n    how = 'left'\n).drop(['year', 'month', 'day', 'hour', 'minute']).join(\n    wind_speed,\n    left_on = ['time_year', 'time_month', 'time_day', 'time_hour', 'time_minute'],\n    right_on = ['year', 'month', 'day', 'hour', 'minute'],\n    how = 'left'\n).drop(['year', 'month', 'day', 'hour', 'minute'])","metadata":{"execution":{"iopub.execute_input":"2024-01-29T04:49:45.263619Z","iopub.status.busy":"2024-01-29T04:49:45.263154Z","iopub.status.idle":"2024-01-29T04:49:45.292589Z","shell.execute_reply":"2024-01-29T04:49:45.291397Z"},"papermill":{"duration":0.04683,"end_time":"2024-01-29T04:49:45.295134","exception":false,"start_time":"2024-01-29T04:49:45.248304","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Correlation","metadata":{"papermill":{"duration":0.01345,"end_time":"2024-01-29T04:49:45.322014","exception":false,"start_time":"2024-01-29T04:49:45.308564","status":"completed"},"tags":[]}},{"cell_type":"code","source":"#Se genera un mapa de calor desde el dataset, mostrando la correlación entre las características de los datos\ndef heatmap(dataset, label = None):\n    corr = dataset.corr(method = 'pearson')\n    plt.figure(figsize = (45, 45), dpi = 300)\n    mask = np.zeros_like(corr)\n    mask[np.triu_indices_from(mask)] = True\n    sns.heatmap(corr, mask = mask, cmap = 'viridis', annot = True, annot_kws = {'size' : 11})\n    plt.title(f'{label} Dataset Correlation Matrix\\n', fontsize = 25, weight = 'bold')\n    plt.show()","metadata":{"papermill":{"duration":0.022915,"end_time":"2024-01-29T04:49:45.358148","exception":false,"start_time":"2024-01-29T04:49:45.335233","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T18:02:46.196586Z","iopub.execute_input":"2024-03-14T18:02:46.197012Z","iopub.status.idle":"2024-03-14T18:02:46.204370Z","shell.execute_reply.started":"2024-03-14T18:02:46.196981Z","shell.execute_reply":"2024-03-14T18:02:46.203346Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preparation ","metadata":{"papermill":{"duration":0.013031,"end_time":"2024-01-29T04:49:45.384604","exception":false,"start_time":"2024-01-29T04:49:45.371573","status":"completed"},"tags":[]}},{"cell_type":"code","source":"#Selecciona valores no nulos y elimina columnas innecesarias. Luego, asigna la estación meteorológica más cercano a cada registro. \nX = train.filter(pl.col('q_location').is_not_null()).drop('id_unique', 'time', 'ws_timestamp_location', 'ws_timestamp_start', 'ts_oxygen_saturation', 'ts_resting_heart_rate', 'ts_stand_time', 'ts_step_count', 'ts_walking_distance', 'ts_heart_rate', 'ts_audio_exposure_environment').collect()\nX = X.with_columns(stations = map_station(X)).to_pandas()\n\n#Se le asignan variables a \"y\" y se transforman a valores numéricos\ny = X[['q_earphones', 'q_noise_kind', 'q_noise_nearby', 'q_thermal_preference']]\nX = X.drop(['q_earphones', 'q_noise_kind', 'q_noise_nearby', 'q_thermal_preference'], axis = 1)\n\n#Se filtra el conjunto de prueba y se le asigna nuevamente la estación más cercana\nsubmission = test.filter(pl.col('q_location').is_not_null()).drop('q_earphones', 'q_noise_kind', 'q_noise_nearby', 'q_thermal_preference', 'time', 'ws_timestamp_location', 'ws_timestamp_start', 'ts_oxygen_saturation', 'ts_resting_heart_rate', 'ts_stand_time', 'ts_step_count', 'ts_walking_distance', 'ts_heart_rate', 'ts_audio_exposure_environment').collect()\nsubmission = submission.with_columns(stations = map_station(submission)).to_pandas()\n\n#Se ajusta el dato correspondiente a la duración del primer día para establecer una referencia y evitar errores\ntest_pd = submission.drop('id_unique', axis = 1)\ntest_pd.loc[3178, 'time_loc_difference'] += 1e6 * 3600 * 24 * 365\ntest_pd.loc[3178, 'start_loc_difference'] -= 1e6 * 3600 * 24 * 365\n\n#Identifica las características de los datos\ncategorical_features = list(X.select_dtypes('object'))\n\n#Se transforman las variables dependientes a valores numéricos\nle = {}\nfor col in y.columns:\n    le[col] = LabelEncoder().fit(y[col])\n    y[col] = le[col].transform(y[col])   \nacc_scorer = make_scorer(accuracy_score)\n\n#Se establece una semilla aleatoria\nseed = 42\ntf.keras.utils.set_random_seed(seed)\n\n#Se configura la validación cruzada\nkf = MultilabelStratifiedKFold(n_splits = 5, random_state = seed, shuffle = True)\n#kf = TimeSeriesSplit()","metadata":{"papermill":{"duration":45.476571,"end_time":"2024-01-29T04:50:30.874598","exception":false,"start_time":"2024-01-29T04:49:45.398027","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T18:15:17.554856Z","iopub.execute_input":"2024-03-14T18:15:17.555343Z","iopub.status.idle":"2024-03-14T18:15:32.249030Z","shell.execute_reply.started":"2024-03-14T18:15:17.555298Z","shell.execute_reply":"2024-03-14T18:15:32.248166Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Feature Engineering","metadata":{"papermill":{"duration":0.013352,"end_time":"2024-01-29T04:50:30.901792","exception":false,"start_time":"2024-01-29T04:50:30.88844","status":"completed"},"tags":[]}},{"cell_type":"code","source":"#Se impùta valores faltantes y automatiza el preprocesamiento de datos\nclass NumericalImputer(BaseEstimator, TransformerMixin):\n    \n    def fit(self, x, y = None):\n        self.imputer = SimpleImputer().fit(x.select_dtypes(['int', 'float']), y)\n        return self\n    \n    def transform(self, x):\n        x_copy = x.copy()\n        x_copy.loc[:, list(x_copy.select_dtypes(['int', 'float']))] = self.imputer.transform(x_copy.loc[:, list(x_copy.select_dtypes(['int', 'float']))])\n        return x_copy","metadata":{"papermill":{"duration":0.024644,"end_time":"2024-01-29T04:50:31.195005","exception":false,"start_time":"2024-01-29T04:50:31.170361","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-03-14T18:18:19.866214Z","iopub.execute_input":"2024-03-14T18:18:19.866668Z","iopub.status.idle":"2024-03-14T18:18:19.874433Z","shell.execute_reply.started":"2024-03-14T18:18:19.866637Z","shell.execute_reply":"2024-03-14T18:18:19.873307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Adversarial Validation","metadata":{"papermill":{"duration":0.013686,"end_time":"2024-01-29T04:50:31.222264","exception":false,"start_time":"2024-01-29T04:50:31.208578","status":"completed"},"tags":[]}},{"cell_type":"code","source":"#thanks to @carlmcbrideellis\n#https://www.kaggle.com/code/carlmcbrideellis/what-is-adversarial-validation\n\n#Se realiza una validación que permite identificar si dos datos vienen de una misma distribución. (No entendí cómo, pero sé que hace eso)\ndef adversarial_validation(dataset_1 = train, dataset_2 = test, label = 'Train-Test'):\n\n    adv_train = dataset_1.copy()\n    adv_test = dataset_2.copy()\n\n    adv_train['is_test'] = 0\n    adv_test['is_test'] = 1\n\n    adv = pd.concat([adv_train, adv_test], ignore_index = True)\n\n    adv_shuffled = adv.sample(frac = 1)\n\n    adv_X = adv_shuffled.drop('is_test', axis = 1)\n    adv_y = adv_shuffled.is_test\n\n    skf = StratifiedKFold(n_splits = 5, random_state = 42, shuffle = True)\n\n    val_scores = []\n    predictions = np.zeros(len(adv))\n\n    for fold, (train_idx, val_idx) in enumerate(skf.split(adv_X, adv_y)):\n    \n        adv_lr = make_pipeline(\n            MEstimateEncoder(cols = list(categorical_features)), \n            #FeatureDropper(cols = ['ws_survey_count', 'id_participant']),\n            SimpleImputer(),\n            XGBClassifier(random_state = seed, tree_method = 'hist')\n        )\n        adv_lr.fit(adv_X.iloc[train_idx], adv_y.iloc[train_idx])\n        \n        val_preds = adv_lr.predict_proba(adv_X.iloc[val_idx])[:, 1]\n        predictions[val_idx] = val_preds\n        val_score = roc_auc_score(adv_y.iloc[val_idx], val_preds)\n        val_scores.append(val_score)\n    \n    fpr, tpr, _ = roc_curve(adv['is_test'], predictions)\n    \n    plt.figure(figsize = (10, 10), dpi = 300)\n    sns.lineplot(x=[0, 1], y=[0, 1], linestyle=\"--\", label=\"Indistinguishable Datasets\")\n    sns.lineplot(x=fpr, y=tpr, label=\"Adversarial Validation Classifier\")\n    plt.title(f'{label} Validation = {np.mean(val_scores):.5f}', weight = 'bold', size = 17)\n    plt.xlabel('False Positive Rate')\n    plt.ylabel('True Positive Rate')\n    plt.show()","metadata":{"execution":{"iopub.execute_input":"2024-01-29T04:50:31.25084Z","iopub.status.busy":"2024-01-29T04:50:31.250391Z","iopub.status.idle":"2024-01-29T04:50:31.261576Z","shell.execute_reply":"2024-01-29T04:50:31.260396Z"},"papermill":{"duration":0.028635,"end_time":"2024-01-29T04:50:31.264265","exception":false,"start_time":"2024-01-29T04:50:31.23563","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#adversarial_validation(X, test_pd)","metadata":{"execution":{"iopub.execute_input":"2024-01-29T04:50:31.293053Z","iopub.status.busy":"2024-01-29T04:50:31.292603Z","iopub.status.idle":"2024-01-29T04:50:31.297277Z","shell.execute_reply":"2024-01-29T04:50:31.296116Z"},"papermill":{"duration":0.021817,"end_time":"2024-01-29T04:50:31.299491","exception":false,"start_time":"2024-01-29T04:50:31.277674","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Cross Validation","metadata":{"papermill":{"duration":0.012986,"end_time":"2024-01-29T04:50:31.325875","exception":false,"start_time":"2024-01-29T04:50:31.312889","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def cross_val_score(estimator, cv = kf, label = '', show_importance = False):\n    \n    val_scores = []\n    \n    feature_importances_table = pd.DataFrame({'value' : 0}, index = list(X.columns))\n    \n    for fold, (train_idx, val_idx) in enumerate(cv.split(X, y)):\n        \n        X_train = X.iloc[train_idx]\n        y_train = y.iloc[train_idx]\n        \n        X_val = X.iloc[val_idx]\n        y_val = y.iloc[val_idx]\n        \n        val_preds = np.zeros((len(X_val), 4))\n        val_score = 0\n        \n        for i in range(4):\n            model = clone(estimator).fit(X_train, y_train.iloc[:, i])\n            val_preds[:, i] = model.predict(X_val)\n            val_score += accuracy_score(y_val.iloc[:, i], val_preds[:, i]) / 4\n            if show_importance:\n                feature_importances_table['value'] += permutation_importance(model, X_val, y_val.iloc[:, i], random_state = seed, scoring = 'accuracy', n_repeats = 5).importances_mean / 4 / cv.get_n_splits()\n            \n        val_scores.append(val_score)\n    \n    print(f'Val Score: {np.mean(val_scores):.5f} ± {np.std(val_scores):.5f} | {label}')\n    \n    if show_importance:\n        plt.figure(figsize = (20, 50))\n        plt.title('Feature Importances', weight = 'bold', size = 25)\n        sns.barplot(feature_importances_table.sort_values('value', ascending = False).T, orient = 'h', palette = 'viridis')\n        plt.show()\n    \n    return val_scores, feature_importances_table","metadata":{"execution":{"iopub.execute_input":"2024-01-29T04:50:31.36077Z","iopub.status.busy":"2024-01-29T04:50:31.360342Z","iopub.status.idle":"2024-01-29T04:50:31.370741Z","shell.execute_reply":"2024-01-29T04:50:31.369556Z"},"papermill":{"duration":0.032302,"end_time":"2024-01-29T04:50:31.373358","exception":false,"start_time":"2024-01-29T04:50:31.341056","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def cross_val_score_proba(estimator, col, cv = kf, label = ''):\n    \n    val_scores = []\n    val_predictions = np.zeros((len(X), y.loc[:, col].nunique()))\n    test_predictions = np.zeros((len(test_pd), y.loc[:, col].nunique()))\n    \n    for fold, (train_idx, val_idx) in enumerate(cv.split(X, y)):\n        \n        X_train = X.iloc[train_idx].reset_index(drop = True)\n        y_train = y.iloc[train_idx].reset_index(drop = True)\n        \n        X_val = X.iloc[val_idx].reset_index(drop = True)\n        y_val = y.iloc[val_idx].reset_index(drop = True)\n        \n        model = clone(estimator).fit(X_train, y_train[col])\n        val_preds = model.predict_proba(X_val)\n        val_score = accuracy_score(y_val[col], val_preds.argmax(axis = 1))\n            \n        val_scores.append(val_score)\n        val_predictions[val_idx] = val_preds\n        test_predictions += model.predict_proba(test_pd) / cv.get_n_splits()\n        \n        #print(f'Fold Score: {val_score:.5f}')\n    \n    print(f'Val Score: {np.mean(val_scores):.5f} ± {np.std(val_scores):.5f} | {label}')\n    \n    return val_scores, val_predictions, test_predictions","metadata":{"execution":{"iopub.execute_input":"2024-01-29T04:50:31.402929Z","iopub.status.busy":"2024-01-29T04:50:31.402469Z","iopub.status.idle":"2024-01-29T04:50:31.411459Z","shell.execute_reply":"2024-01-29T04:50:31.410289Z"},"papermill":{"duration":0.026822,"end_time":"2024-01-29T04:50:31.413927","exception":false,"start_time":"2024-01-29T04:50:31.387105","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"oof_list = {col : [] for col in list(y)}\npredict_list = {col : [] for col in list(y)}","metadata":{"execution":{"iopub.execute_input":"2024-01-29T04:50:31.443615Z","iopub.status.busy":"2024-01-29T04:50:31.442567Z","iopub.status.idle":"2024-01-29T04:50:31.44809Z","shell.execute_reply":"2024-01-29T04:50:31.446878Z"},"papermill":{"duration":0.022826,"end_time":"2024-01-29T04:50:31.450679","exception":false,"start_time":"2024-01-29T04:50:31.427853","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Logistic Regression","metadata":{"papermill":{"duration":0.013066,"end_time":"2024-01-29T04:50:31.477414","exception":false,"start_time":"2024-01-29T04:50:31.464348","status":"completed"},"tags":[]}},{"cell_type":"code","source":"log_models = {}\n\nfor col in list(y):\n    log_models[col] = make_pipeline(\n        MEstimateEncoder(cols = categorical_features),\n        SimpleImputer(),\n        StandardScaler(),\n        PowerTransformer(),\n        LogisticRegression(random_state = seed, max_iter = 10000000)\n    )\n    _, oof_preds, _ = cross_val_score_proba(log_models[col], col)\n    oof_list[col].append(oof_preds)\n    \nprint('')","metadata":{"execution":{"iopub.execute_input":"2024-01-29T04:50:31.505844Z","iopub.status.busy":"2024-01-29T04:50:31.505452Z","iopub.status.idle":"2024-01-29T04:51:07.051792Z","shell.execute_reply":"2024-01-29T04:51:07.050549Z"},"papermill":{"duration":35.58775,"end_time":"2024-01-29T04:51:07.07856","exception":false,"start_time":"2024-01-29T04:50:31.49081","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# SVM","metadata":{"papermill":{"duration":0.022879,"end_time":"2024-01-29T04:51:07.125295","exception":false,"start_time":"2024-01-29T04:51:07.102416","status":"completed"},"tags":[]}},{"cell_type":"code","source":"svm_models = {}\n\nfor col in list(y):\n    svm_models[col] = make_pipeline(\n        MEstimateEncoder(cols = categorical_features),\n        SimpleImputer(),\n        StandardScaler(),\n        PowerTransformer(),\n        SVC(random_state = seed, probability = True, max_iter = 1000000)\n    )\n    _, oof_preds, _ = cross_val_score_proba(svm_models[col], col)\n    oof_list[col].append(oof_preds)\n    \nprint('')","metadata":{"execution":{"iopub.execute_input":"2024-01-29T04:51:07.171536Z","iopub.status.busy":"2024-01-29T04:51:07.170838Z","iopub.status.idle":"2024-01-29T04:54:49.455731Z","shell.execute_reply":"2024-01-29T04:54:49.454592Z"},"papermill":{"duration":222.322245,"end_time":"2024-01-29T04:54:49.47075","exception":false,"start_time":"2024-01-29T04:51:07.148505","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# LightGBM","metadata":{"papermill":{"duration":0.014521,"end_time":"2024-01-29T04:54:49.49914","exception":false,"start_time":"2024-01-29T04:54:49.484619","status":"completed"},"tags":[]}},{"cell_type":"code","source":"lgb_models = {}\n\nfor col in list(y):\n    lgb_models[col] = make_pipeline(\n        MEstimateEncoder(cols = categorical_features),\n        SimpleImputer(),\n        LGBMClassifier(random_state = seed)\n    )\n    _, oof_preds, _ = cross_val_score_proba(lgb_models[col], col)\n    oof_list[col].append(oof_preds)\n    \nprint('')","metadata":{"execution":{"iopub.execute_input":"2024-01-29T04:54:49.606797Z","iopub.status.busy":"2024-01-29T04:54:49.606385Z","iopub.status.idle":"2024-01-29T04:56:38.351815Z","shell.execute_reply":"2024-01-29T04:56:38.350504Z"},"papermill":{"duration":108.777442,"end_time":"2024-01-29T04:56:38.368714","exception":false,"start_time":"2024-01-29T04:54:49.591272","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# CatBoost","metadata":{"papermill":{"duration":0.014134,"end_time":"2024-01-29T04:56:38.397087","exception":false,"start_time":"2024-01-29T04:56:38.382953","status":"completed"},"tags":[]}},{"cell_type":"code","source":"cb_models = {}\n\nfor col in list(y):\n    cb_models[col] = make_pipeline(\n        NumericalImputer(),\n        CatBoostClassifier(random_state = seed, verbose = 0, cat_features = categorical_features)\n    )\n    _, oof_preds, _ = cross_val_score_proba(cb_models[col], col)\n    oof_list[col].append(oof_preds)\n\nprint('')","metadata":{"execution":{"iopub.execute_input":"2024-01-29T04:56:38.427938Z","iopub.status.busy":"2024-01-29T04:56:38.427559Z","iopub.status.idle":"2024-01-29T05:33:22.136091Z","shell.execute_reply":"2024-01-29T05:33:22.134848Z"},"papermill":{"duration":2203.741797,"end_time":"2024-01-29T05:33:22.153384","exception":false,"start_time":"2024-01-29T04:56:38.411587","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cb_bayes_models = {}\n\nfor col in list(y):\n    cb_bayes_models[col] = make_pipeline(\n        NumericalImputer(),\n        CatBoostClassifier(random_state = seed, verbose = 0, bootstrap_type = 'Bayesian', cat_features = categorical_features)\n    )\n    _, oof_preds, _ = cross_val_score_proba(cb_bayes_models[col], col)\n    oof_list[col].append(oof_preds)\n    \nprint('')","metadata":{"execution":{"iopub.execute_input":"2024-01-29T05:33:22.184556Z","iopub.status.busy":"2024-01-29T05:33:22.183677Z","iopub.status.idle":"2024-01-29T06:09:43.783276Z","shell.execute_reply":"2024-01-29T06:09:43.781812Z"},"papermill":{"duration":2181.626697,"end_time":"2024-01-29T06:09:43.794465","exception":false,"start_time":"2024-01-29T05:33:22.167768","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cb_bernoulli_models = {}\n\nfor col in list(y):\n    cb_bernoulli_models[col] = make_pipeline(\n        NumericalImputer(),\n        CatBoostClassifier(random_state = seed, verbose = 0, bootstrap_type = 'Bernoulli', cat_features = categorical_features)\n    )\n    _, oof_preds, _ = cross_val_score_proba(cb_bernoulli_models[col], col)\n    oof_list[col].append(oof_preds)\n    \nprint('')","metadata":{"execution":{"iopub.execute_input":"2024-01-29T06:09:43.826052Z","iopub.status.busy":"2024-01-29T06:09:43.824999Z","iopub.status.idle":"2024-01-29T06:43:16.970497Z","shell.execute_reply":"2024-01-29T06:43:16.969416Z"},"papermill":{"duration":2013.17832,"end_time":"2024-01-29T06:43:16.987367","exception":false,"start_time":"2024-01-29T06:09:43.809047","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# XGBoost","metadata":{"papermill":{"duration":0.014571,"end_time":"2024-01-29T06:43:17.016813","exception":false,"start_time":"2024-01-29T06:43:17.002242","status":"completed"},"tags":[]}},{"cell_type":"code","source":"xgb_models = {}\n\nfor col in list(y):\n    xgb_models[col] = make_pipeline(\n        MEstimateEncoder(cols = categorical_features),\n        SimpleImputer(),\n        XGBClassifier(random_state = seed, tree_method = 'hist')\n    )\n    _, oof_preds, _ = cross_val_score_proba(xgb_models[col], col)\n    oof_list[col].append(oof_preds)\n    \nprint('')","metadata":{"execution":{"iopub.execute_input":"2024-01-29T06:43:17.129141Z","iopub.status.busy":"2024-01-29T06:43:17.128714Z","iopub.status.idle":"2024-01-29T06:45:30.799625Z","shell.execute_reply":"2024-01-29T06:45:30.798193Z"},"papermill":{"duration":133.699844,"end_time":"2024-01-29T06:45:30.812195","exception":false,"start_time":"2024-01-29T06:43:17.112351","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Ensemble NN","metadata":{"papermill":{"duration":0.014948,"end_time":"2024-01-29T06:45:30.842435","exception":false,"start_time":"2024-01-29T06:45:30.827487","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def cross_val_score_nn(estimator, col, cv = kf, label = ''):\n    \n    X_oof = oof_list[col]\n        \n    model = clone(estimator).fit(X_oof, y[col])\n    val_preds = model.predict_proba(X_oof)\n    val_score = accuracy_score(y[col], val_preds.argmax(axis = 1))\n    \n    print(f'Val Score: {val_score:.5f} | {label}')\n    \n    return val_score","metadata":{"execution":{"iopub.execute_input":"2024-01-29T06:45:30.875169Z","iopub.status.busy":"2024-01-29T06:45:30.874371Z","iopub.status.idle":"2024-01-29T06:45:30.880556Z","shell.execute_reply":"2024-01-29T06:45:30.879807Z"},"papermill":{"duration":0.025169,"end_time":"2024-01-29T06:45:30.88277","exception":false,"start_time":"2024-01-29T06:45:30.857601","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class TensorFlower(BaseEstimator, ClassifierMixin):\n    \n    def fit(self, x, y):\n        \n        tf.keras.utils.set_random_seed(seed)\n        tf.config.experimental.enable_op_determinism()\n        \n        inputs = []\n        norm_inputs = []\n        \n        multiply = []\n        \n        for i in range(len(x)):\n            inputs.append(tf.keras.Input((len(np.unique(y)), )))\n            norm_inputs.append(tf.keras.layers.BatchNormalization()(inputs[i]))\n            \n        concat = tf.keras.layers.Concatenate()(norm_inputs)\n        \n        for i in range(len(x)):\n            dense = tf.keras.layers.Dense(len(np.unique(y)))(concat)\n            norm = tf.keras.layers.BatchNormalization()(dense)\n            act = tf.keras.activations.softmax(norm)\n            multiply.append(tf.keras.layers.Multiply()([inputs[i], act]))\n            \n        outputs = tf.keras.layers.Add()(multiply)\n        \n        self.model = tf.keras.Model(inputs, outputs)\n        self.model.compile(loss = 'sparse_categorical_crossentropy', optimizer = tf.keras.optimizers.AdamW(1e-4))\n        \n        self.model.fit(x, y, epochs = 50, verbose = 0)\n        \n        return self\n    \n    def predict_proba(self, x):\n        \n        return self.model.predict(x, verbose = 0)\n    \n    def predict(self, x):\n        \n        return np.argmax(self.predict_proba(x), axis = 1)","metadata":{"execution":{"iopub.execute_input":"2024-01-29T06:45:30.915267Z","iopub.status.busy":"2024-01-29T06:45:30.914483Z","iopub.status.idle":"2024-01-29T06:45:30.924814Z","shell.execute_reply":"2024-01-29T06:45:30.924047Z"},"papermill":{"duration":0.029221,"end_time":"2024-01-29T06:45:30.927107","exception":false,"start_time":"2024-01-29T06:45:30.897886","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for col in list(y):\n    cross_val_score_nn(TensorFlower(), col)\n    \nprint('\\n')","metadata":{"execution":{"iopub.execute_input":"2024-01-29T06:45:30.959625Z","iopub.status.busy":"2024-01-29T06:45:30.958636Z","iopub.status.idle":"2024-01-29T06:48:14.201221Z","shell.execute_reply":"2024-01-29T06:48:14.200024Z"},"papermill":{"duration":163.27622,"end_time":"2024-01-29T06:48:14.218501","exception":false,"start_time":"2024-01-29T06:45:30.942281","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prediction and Submission","metadata":{"papermill":{"duration":0.015352,"end_time":"2024-01-29T06:48:14.322618","exception":false,"start_time":"2024-01-29T06:48:14.307266","status":"completed"},"tags":[]}},{"cell_type":"code","source":"for col in list(y):\n    predict_list[col].append(log_models[col].fit(X, y[col]).predict_proba(test_pd))\n    predict_list[col].append(svm_models[col].fit(X, y[col]).predict_proba(test_pd))\n    predict_list[col].append(lgb_models[col].fit(X, y[col]).predict_proba(test_pd))\n    predict_list[col].append(cb_models[col].fit(X, y[col]).predict_proba(test_pd))\n    predict_list[col].append(cb_bayes_models[col].fit(X, y[col]).predict_proba(test_pd))\n    predict_list[col].append(cb_bernoulli_models[col].fit(X, y[col]).predict_proba(test_pd))\n    predict_list[col].append(xgb_models[col].fit(X, y[col]).predict_proba(test_pd))","metadata":{"execution":{"iopub.execute_input":"2024-01-29T06:48:14.355437Z","iopub.status.busy":"2024-01-29T06:48:14.355036Z","iopub.status.idle":"2024-01-29T07:13:06.847387Z","shell.execute_reply":"2024-01-29T07:13:06.846229Z"},"papermill":{"duration":1492.512251,"end_time":"2024-01-29T07:13:06.850299","exception":false,"start_time":"2024-01-29T06:48:14.338048","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for col in list(y):\n    ensembler = TensorFlower().fit(oof_list[col], y[col])\n    submission[col] = ensembler.predict(predict_list[col])\n    submission[col] = le[col].inverse_transform(submission[col].astype(np.int8))\n\nsubmission[['id_unique'] + list(y)].to_csv('cool_city.csv', index = False)","metadata":{"execution":{"iopub.execute_input":"2024-01-29T07:13:06.888262Z","iopub.status.busy":"2024-01-29T07:13:06.887491Z","iopub.status.idle":"2024-01-29T07:16:00.598198Z","shell.execute_reply":"2024-01-29T07:16:00.597125Z"},"papermill":{"duration":173.734399,"end_time":"2024-01-29T07:16:00.601369","exception":false,"start_time":"2024-01-29T07:13:06.86697","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Citation\n\n- Clayton Miller, Matias Quintana, Mario Frei, Yun Xuan Chua, Chun Fu, Bianca Picchetti, Winston Yap, Adrian Chong, and Filip Biljecki. 2023. Introducing the Cool, Quiet City Competition: Predicting Smartwatch-Reported Heat and Noise with Digital Twin Metrics. In Proceedings of the 10th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys '23). Association for Computing Machinery, New York, NY, USA, 298–299. https://doi.org/10.1145/3600100.3626269","metadata":{"papermill":{"duration":0.015561,"end_time":"2024-01-29T07:16:00.632974","exception":false,"start_time":"2024-01-29T07:16:00.617413","status":"completed"},"tags":[]}}]}