Introduce
","metadata":{}},{"cell_type":"markdown","source":"This notebook is based on currently publicly available feature engineering. It was found that most of the notebooks in the current competition combine training and infer into the same notebook, and the model was not saved. Therefore, errors such as timeout occurred during model integration. This notebook can only perform inference. You can load a trained model and use a custom model integration class for inference. At present, the integration method only uses the average result. You can easily replace it with other integration methods you want, and be careful not to overfit the results for the sake of the lb score. Wishing everyone a happy competition 😁 ","metadata":{}},{"cell_type":"markdown","source":"#Train Recommendation
","metadata":{}},{"cell_type":"markdown","source":"Random Forest Training Notebook:\n \nhttps://www.kaggle.com/code/lccburk/feature-engineering-and-random-forest-prediction\nhttps://www.kaggle.com/code/siddhvr/cmi-feature-engg-with-ensemble-models\n\nTraining of the lgb model:\n \n https://www.kaggle.com/code/zhukovoleksiy/detect-sleep-states-baseline-solution","metadata":{}},{"cell_type":"markdown","source":"You can use the joblib.dump method to save the model","metadata":{}},{"cell_type":"markdown","source":"#Libraries
","metadata":{}},{"cell_type":"code","source":"# Misc\nimport numpy as np\nimport pandas as pd\nimport plotly.express as px\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport random\nimport os\nimport gc\nfrom copy import deepcopy\nfrom functools import partial\nfrom itertools import combinations\nfrom itertools import groupby\nfrom tqdm import tqdm\nimport polars as pl\nimport datetime\n\n# Import sklearn classes for model selection, cross validation, and performance evaluation\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.model_selection import StratifiedKFold, KFold\nfrom sklearn.metrics import roc_auc_score, accuracy_score, log_loss\nfrom sklearn.model_selection import cross_validate\nfrom sklearn.metrics import RocCurveDisplay\nfrom sklearn.metrics import confusion_matrix\nfrom sklearn.metrics import ConfusionMatrixDisplay\nfrom sklearn.metrics import precision_score\nfrom sklearn.metrics import average_precision_score\n\n# Import libraries for Hypertuning\nimport optuna\n\n#Import libraries for gradient boosting\nimport xgboost as xgb\nimport lightgbm as lgb\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.ensemble import GradientBoostingClassifier\nfrom sklearn.pipeline import Pipeline\nfrom catboost import Pool\nimport ctypes\nlibc = ctypes.CDLL(\"libc.so.6\")\n\nfrom metric import score # Import event detection ap score function\n\n# These are variables to be used by the score function\ncolumn_names = {\n 'series_id_column_name': 'series_id',\n 'time_column_name': 'step',\n 'event_column_name': 'event',\n 'score_column_name': 'score',\n}\n\ntolerances = {\n 'onset': [12, 36, 60, 90, 120, 150, 180, 240, 300, 360], \n 'wakeup': [12, 36, 60, 90, 120, 150, 180, 240, 300, 360]\n}\n\n\n# Useful line of code to set the display option so we could see all the columns in pd dataframe\npd.set_option('display.max_columns', None)\n\n# Suppress warnings\nimport warnings\nwarnings.filterwarnings(\"ignore\", category=UserWarning)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-09-27T04:55:35.076055Z","iopub.execute_input":"2023-09-27T04:55:35.077038Z","iopub.status.idle":"2023-09-27T04:55:42.773067Z","shell.execute_reply.started":"2023-09-27T04:55:35.077001Z","shell.execute_reply":"2023-09-27T04:55:42.771781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Dataset
","metadata":{}},{"cell_type":"code","source":"# Importing data \n\n# Column transformations\n\ndt_transforms = [\n pl.col('timestamp').str.to_datetime(), \n (pl.col('timestamp').str.to_datetime().dt.year()-2000).cast(pl.UInt8).alias('year'), \n pl.col('timestamp').str.to_datetime().dt.month().cast(pl.UInt8).alias('month'),\n pl.col('timestamp').str.to_datetime().dt.day().cast(pl.UInt8).alias('day'), \n pl.col('timestamp').str.to_datetime().dt.hour().cast(pl.UInt8).alias('hour')\n]\n\ndata_transforms = [\n pl.col('anglez').cast(pl.Int16), # Casting anglez to 16 bit integer\n (pl.col('enmo')*1000).cast(pl.UInt16), # Convert enmo to 16 bit uint\n]\n\ntrain_series = pl.scan_parquet('/kaggle/input/child-mind-institute-detect-sleep-states/train_series.parquet').with_columns(\n dt_transforms + data_transforms\n )\n\ntrain_events = pl.read_csv('/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv').with_columns(\n dt_transforms\n )\n\ntest_series = pl.scan_parquet('/kaggle/input/child-mind-institute-detect-sleep-states/test_series.parquet').with_columns(\n dt_transforms + data_transforms\n )\n\n# Getting series ids as a list for convenience\nseries_ids = train_events['series_id'].unique(maintain_order=True).to_list()\n\n# Removing series with mismatched counts: \nonset_counts = train_events.filter(pl.col('event')=='onset').group_by('series_id').count().sort('series_id')['count']\nwakeup_counts = train_events.filter(pl.col('event')=='wakeup').group_by('series_id').count().sort('series_id')['count']\n\ncounts = pl.DataFrame({'series_id':sorted(series_ids), 'onset_counts':onset_counts, 'wakeup_counts':wakeup_counts})\ncount_mismatches = counts.filter(counts['onset_counts'] != counts['wakeup_counts'])\n\ntrain_series = train_series.filter(~pl.col('series_id').is_in(count_mismatches['series_id']))\ntrain_events = train_events.filter(~pl.col('series_id').is_in(count_mismatches['series_id']))\n\n# Updating list of series ids, not including series with no non-null values.\nseries_ids = train_events.drop_nulls()['series_id'].unique(maintain_order=True).to_list()","metadata":{"execution":{"iopub.status.busy":"2023-09-27T04:55:42.774841Z","iopub.execute_input":"2023-09-27T04:55:42.775173Z","iopub.status.idle":"2023-09-27T04:55:43.047941Z","shell.execute_reply.started":"2023-09-27T04:55:42.775144Z","shell.execute_reply":"2023-09-27T04:55:43.046885Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"features, feature_cols = [pl.col('hour')], ['hour']\n\nfor mins in [5, 30, 60*2, 60*8] :\n features += [\n pl.col('enmo').rolling_mean(12 * mins, center=True, min_periods=1).abs().cast(pl.UInt16).alias(f'enmo_{mins}m_mean'),\n pl.col('enmo').rolling_max(12 * mins, center=True, min_periods=1).abs().cast(pl.UInt16).alias(f'enmo_{mins}m_max')\n ]\n\n feature_cols += [ \n f'enmo_{mins}m_mean', f'enmo_{mins}m_max'\n ]\n\n # Getting first variations\n for var in ['enmo', 'anglez'] :\n features += [\n (pl.col(var).diff().abs().rolling_mean(12 * mins, center=True, min_periods=1)*10).abs().cast(pl.UInt32).alias(f'{var}_1v_{mins}m_mean'),\n (pl.col(var).diff().abs().rolling_max(12 * mins, center=True, min_periods=1)*10).abs().cast(pl.UInt32).alias(f'{var}_1v_{mins}m_max')\n ]\n\n feature_cols += [ \n f'{var}_1v_{mins}m_mean', f'{var}_1v_{mins}m_max'\n ]\n\nid_cols = ['series_id', 'step', 'timestamp']\n\ntrain_series = train_series.with_columns(\n features\n).select(id_cols + feature_cols)\n\ntest_series = test_series.with_columns(\n features\n).select(id_cols + feature_cols)","metadata":{"execution":{"iopub.status.busy":"2023-09-27T04:55:43.049398Z","iopub.execute_input":"2023-09-27T04:55:43.049839Z","iopub.status.idle":"2023-09-27T04:55:43.065593Z","shell.execute_reply.started":"2023-09-27T04:55:43.049798Z","shell.execute_reply":"2023-09-27T04:55:43.064693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def make_train_dataset(train_data, train_events, drop_nulls=False) :\n \n series_ids = train_data['series_id'].unique(maintain_order=True).to_list()\n X, y = pl.DataFrame(), pl.DataFrame()\n for idx in tqdm(series_ids) : \n \n # Normalizing sample features\n sample = train_data.filter(pl.col('series_id')==idx).with_columns(\n [(pl.col(col) / pl.col(col).std()).cast(pl.Float32) for col in feature_cols if col != 'hour']\n )\n \n events = train_events.filter(pl.col('series_id')==idx)\n \n if drop_nulls : \n # Removing datapoints on dates where no data was recorded\n sample = sample.filter(\n pl.col('timestamp').dt.date().is_in(events['timestamp'].dt.date())\n )\n \n X = X.vstack(sample[id_cols + feature_cols])\n\n onsets = events.filter((pl.col('event') == 'onset') & (pl.col('step') != None))['step'].to_list()\n wakeups = events.filter((pl.col('event') == 'wakeup') & (pl.col('step') != None))['step'].to_list()\n\n # NOTE: This will break if there are event series without any recorded onsets or wakeups\n y = y.vstack(sample.with_columns(\n sum([(onset <= pl.col('step')) & (pl.col('step') <= wakeup) for onset, wakeup in zip(onsets, wakeups)]).cast(pl.Boolean).alias('asleep')\n ).select('asleep')\n )\n \n y = y.to_numpy().ravel()\n \n return X, y","metadata":{"execution":{"iopub.status.busy":"2023-09-27T04:55:43.069292Z","iopub.execute_input":"2023-09-27T04:55:43.070277Z","iopub.status.idle":"2023-09-27T04:55:43.082497Z","shell.execute_reply.started":"2023-09-27T04:55:43.070228Z","shell.execute_reply":"2023-09-27T04:55:43.08162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_events(series, classifier) :\n '''\n Takes a time series and two classifiers and returns a formatted submission dataframe.\n '''\n \n series_ids = series['series_id'].unique(maintain_order=True).to_list()\n events = pl.DataFrame(schema={'series_id':str, 'step':int, 'event':str, 'score':float})\n\n for idx in tqdm(series_ids) : \n\n # Collecting sample and normalizing features\n scale_cols = [col for col in feature_cols if (col != 'hour') & (series[col].std() !=0)]\n X = series.filter(pl.col('series_id') == idx).select(id_cols + feature_cols).with_columns(\n [(pl.col(col) / series[col].std()).cast(pl.Float32) for col in scale_cols]\n )\n\n # Applying classifiers to get predictions and scores\n preds, probs = classifier.predict(X[feature_cols]), classifier.predict_proba(X[feature_cols])[:, 1]\n\n X = X.with_columns(\n pl.lit(preds).cast(pl.Int8).alias('prediction'), \n pl.lit(probs).alias('probability')\n )\n \n # Getting predicted onset and wakeup time steps\n pred_onsets = X.filter(X['prediction'].diff() > 0)['step'].to_list()\n pred_wakeups = X.filter(X['prediction'].diff() < 0)['step'].to_list()\n \n if len(pred_onsets) > 0 : \n \n # Ensuring all predicted sleep periods begin and end\n if min(pred_wakeups) < min(pred_onsets) : \n pred_wakeups = pred_wakeups[1:]\n\n if max(pred_onsets) > max(pred_wakeups) :\n pred_onsets = pred_onsets[:-1]\n\n # Keeping sleep periods longer than 30 minutes\n sleep_periods = [(onset, wakeup) for onset, wakeup in zip(pred_onsets, pred_wakeups) if wakeup - onset >= 12 * 30]\n\n for onset, wakeup in sleep_periods :\n # Scoring using mean probability over period\n score = X.filter((pl.col('step') >= onset) & (pl.col('step') <= wakeup))['probability'].mean()\n\n # Adding sleep event to dataframe\n events = events.vstack(pl.DataFrame().with_columns(\n pl.Series([idx, idx]).alias('series_id'), \n pl.Series([onset, wakeup]).alias('step'),\n pl.Series(['onset', 'wakeup']).alias('event'),\n pl.Series([score, score]).alias('score')\n ))\n\n # Adding row id column\n events = events.to_pandas().reset_index().rename(columns={'index':'row_id'})\n\n return events\n","metadata":{"execution":{"iopub.status.busy":"2023-09-27T04:55:43.084516Z","iopub.execute_input":"2023-09-27T04:55:43.085067Z","iopub.status.idle":"2023-09-27T04:55:43.102912Z","shell.execute_reply.started":"2023-09-27T04:55:43.085026Z","shell.execute_reply":"2023-09-27T04:55:43.101655Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gc.collect()\nlibc.malloc_trim(0)","metadata":{"execution":{"iopub.status.busy":"2023-09-27T04:55:43.104326Z","iopub.execute_input":"2023-09-27T04:55:43.105526Z","iopub.status.idle":"2023-09-27T04:55:43.254834Z","shell.execute_reply.started":"2023-09-27T04:55:43.105493Z","shell.execute_reply":"2023-09-27T04:55:43.253827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Load Model
","metadata":{}},{"cell_type":"code","source":"import joblib\nwith open('/kaggle/input/cmi-feature-engg-with-ensemble-models/rf_model.pkl', 'rb') as f:\n rf_classifier_1 = joblib.load(f)","metadata":{"execution":{"iopub.status.busy":"2023-09-27T04:55:55.543601Z","iopub.execute_input":"2023-09-27T04:55:55.543997Z","iopub.status.idle":"2023-09-27T04:55:55.828977Z","shell.execute_reply.started":"2023-09-27T04:55:55.543967Z","shell.execute_reply":"2023-09-27T04:55:55.827981Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import joblib\nwith open('/kaggle/input/train-infer-notebook-lgb/lgb_model.pkl', 'rb') as f:\n lgb_classifier = joblib.load(f)","metadata":{"execution":{"iopub.status.busy":"2023-09-27T04:55:55.830724Z","iopub.execute_input":"2023-09-27T04:55:55.831258Z","iopub.status.idle":"2023-09-27T04:55:55.889836Z","shell.execute_reply.started":"2023-09-27T04:55:55.831225Z","shell.execute_reply":"2023-09-27T04:55:55.888906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Submit by Ensemable
","metadata":{}},{"cell_type":"code","source":"class EnsembleAvgProba():\n \n def __init__(self, classifiers):\n self.classifiers = classifiers\n \n def fit(self,X,y):\n for classifier in self.classifiers: \n classifier.fit(X, y)\n gc.collect()\n \n def predict_proba(self, X):\n probs = []\n for m in self.classifiers:\n probs.append(m.predict_proba(X))\n \n probabilities = np.stack(probs)\n p = np.mean(probabilities, axis=0)\n \n return p\n\n def predict(self, X):\n proba = self.predict_proba(X)\n return np.argmax(proba, axis=1)\n","metadata":{"execution":{"iopub.status.busy":"2023-09-27T04:55:57.226078Z","iopub.execute_input":"2023-09-27T04:55:57.226434Z","iopub.status.idle":"2023-09-27T04:55:57.234936Z","shell.execute_reply.started":"2023-09-27T04:55:57.226405Z","shell.execute_reply":"2023-09-27T04:55:57.233787Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"models = EnsembleAvgProba(classifiers=[\n lgb_classifier,\n rf_classifier_1,\n ]\n )","metadata":{"execution":{"iopub.status.busy":"2023-09-27T04:56:10.522837Z","iopub.execute_input":"2023-09-27T04:56:10.523249Z","iopub.status.idle":"2023-09-27T04:56:10.528669Z","shell.execute_reply.started":"2023-09-27T04:56:10.523213Z","shell.execute_reply":"2023-09-27T04:56:10.52746Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Getting event predictions for test set and saving submission\nsubmission = get_events(test_series.collect(), models)\nsubmission.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2023-09-27T04:56:10.786955Z","iopub.execute_input":"2023-09-27T04:56:10.787342Z","iopub.status.idle":"2023-09-27T04:56:10.888293Z","shell.execute_reply.started":"2023-09-27T04:56:10.787313Z","shell.execute_reply":"2023-09-27T04:56:10.887102Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}