{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### Random Forest multiclass classifier submission\n**Many thanks to the original author: [Carl McBride Ellis](https://www.kaggle.com/carlmcbrideellis)** ([LinkedIn](https://www.linkedin.com/in/carl-mcbride-ellis/))\n\nAlso took some code from https://www.kaggle.com/code/carlmcbrideellis/zzzs-playing-with-the-competition-metric\n\nWe shall be using the multiclass training data file (`Zzzs_train_multi.parquet`) from the reduced dataset [\"Zzzs: Lightweight training dataset + target\"](https://www.kaggle.com/datasets/carlmcbrideellis/zzzs-lightweight-training-dataset-target)","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nfrom itertools import groupby\nimport gc\nfrom sklearn.metrics import log_loss","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-08T18:41:32.017016Z","iopub.execute_input":"2023-11-08T18:41:32.017524Z","iopub.status.idle":"2023-11-08T18:41:32.025491Z","shell.execute_reply.started":"2023-11-08T18:41:32.017486Z","shell.execute_reply":"2023-11-08T18:41:32.023657Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# binary class data\n#train = pd.read_parquet(\"/kaggle/input/zzzs-lightweight-training-dataset-target/Zzzs_train.parquet\")\n\n# multiclass data\ntrain = pd.read_parquet(\"/kaggle/input/zzzs-lightweight-training-dataset-target/Zzzs_train_multi.parquet\")","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:32.027914Z","iopub.execute_input":"2023-11-08T18:41:32.028838Z","iopub.status.idle":"2023-11-08T18:41:41.861925Z","shell.execute_reply.started":"2023-11-08T18:41:32.028773Z","shell.execute_reply":"2023-11-08T18:41:41.860661Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Gebruik eerst 1% van de data. Dit kunnen we later uitbreiden naar max 10%.","metadata":{}},{"cell_type":"markdown","source":"Validatieset om mee te gaan experimenteren","metadata":{}},{"cell_type":"code","source":"val = train.iloc[162898:162898*2]","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:41.863206Z","iopub.execute_input":"2023-11-08T18:41:41.864263Z","iopub.status.idle":"2023-11-08T18:41:41.870579Z","shell.execute_reply.started":"2023-11-08T18:41:41.864217Z","shell.execute_reply":"2023-11-08T18:41:41.869001Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Train om mee te gaan trainen","metadata":{}},{"cell_type":"code","source":"train = train.iloc[:162898]","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:41.874962Z","iopub.execute_input":"2023-11-08T18:41:41.875474Z","iopub.status.idle":"2023-11-08T18:41:41.886623Z","shell.execute_reply.started":"2023-11-08T18:41:41.875437Z","shell.execute_reply":"2023-11-08T18:41:41.884967Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Feature engineering","metadata":{}},{"cell_type":"markdown","source":"Hieronder valt heel veel te optimaliseren. Laten we er samen eens naar kijken.\n\nJe kan bijvoorbeeld een andere periode nemen om lopende statistieken van te nemen, periodelengten combineren, perioden uitdrukken in tijd, andere statistieken nemen, andere tijdsfeatures, etc. En hebben we nog andere kolommen? ","metadata":{}},{"cell_type":"code","source":"def make_features(df):\n    # parse the timestamp and create an \"hour\" feature\n    df['timestamp'] = pd.to_datetime(df['timestamp']).apply(lambda t: t.tz_localize(None))\n    df[\"hour\"] = df[\"timestamp\"].dt.hour\n    \n    periods = 5\n    df[\"anglez\"] = abs(df[\"anglez\"])\n    df[\"anglez_diff\"] = df.groupby('series_id')['anglez'].diff(periods=periods).fillna(method=\"bfill\").astype('float16')\n    df[\"enmo_diff\"] = df.groupby('series_id')['enmo'].diff(periods=periods).fillna(method=\"bfill\").astype('float16')\n    df[\"anglez_rolling_mean\"] = df[\"anglez\"].rolling(periods,center=True).mean().fillna(method=\"bfill\").fillna(method=\"ffill\").astype('float16')\n    df[\"enmo_rolling_mean\"] = df[\"enmo\"].rolling(periods,center=True).mean().fillna(method=\"bfill\").fillna(method=\"ffill\").astype('float16')\n    return df\n\nfeatures = [\"hour\",\n            \"anglez\",\n            \"anglez_diff\",\n            \"enmo_diff\",\n            \"anglez_rolling_mean\",\n            \"enmo_rolling_mean\",\n           ]","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:41.888007Z","iopub.execute_input":"2023-11-08T18:41:41.888508Z","iopub.status.idle":"2023-11-08T18:41:41.902913Z","shell.execute_reply.started":"2023-11-08T18:41:41.888469Z","shell.execute_reply":"2023-11-08T18:41:41.901304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Training","metadata":{}},{"cell_type":"code","source":"train   = make_features(train)\n\nX_train = train[features]\ny_train = train[\"awake\"]\n\n# save some memory\n# del train\ngc.collect();","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:41.906761Z","iopub.execute_input":"2023-11-08T18:41:41.908551Z","iopub.status.idle":"2023-11-08T18:41:45.222244Z","shell.execute_reply.started":"2023-11-08T18:41:41.908463Z","shell.execute_reply":"2023-11-08T18:41:45.220645Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Je kunt de volgende parameters hieronder optimaliseren:\n* Het type model (nu RandomForestClassifier, hier staat een aantal voorbeelden: https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html)\n* Daarbinnen alle zogenaamde 'hyperparameters'. Hier zijn dat onder andere:\n  * n_estimators (aantal bomen)\n  * min_samples_leaf (minimaal aantal datapunten dat in de laatste tak valt)","metadata":{}},{"cell_type":"code","source":"%%time\n\nfrom sklearn.ensemble import RandomForestClassifier\nclassifier = RandomForestClassifier(n_estimators=50,\n                                    min_samples_leaf=300,\n                                    random_state=42,n_jobs=-1)\n\n\nclassifier.fit(X_train, y_train)\n\n# save some memory\ndel X_train, y_train\ngc.collect();","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:45.223544Z","iopub.execute_input":"2023-11-08T18:41:45.223885Z","iopub.status.idle":"2023-11-08T18:41:49.889147Z","shell.execute_reply.started":"2023-11-08T18:41:45.223857Z","shell.execute_reply":"2023-11-08T18:41:49.887271Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Prediction on validation set**","metadata":{}},{"cell_type":"code","source":"val   = make_features(val)\nX_val = val[features]","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:49.891084Z","iopub.execute_input":"2023-11-08T18:41:49.891599Z","iopub.status.idle":"2023-11-08T18:41:51.86554Z","shell.execute_reply.started":"2023-11-08T18:41:49.891558Z","shell.execute_reply":"2023-11-08T18:41:51.864331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"val_pred = pd.DataFrame(index=val.index)\nval_pred[\"not_awake\"] = classifier.predict_proba(X_val)[:,0]\nval_pred[\"awake\"]     = classifier.predict_proba(X_val)[:,1]","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:51.867138Z","iopub.execute_input":"2023-11-08T18:41:51.867508Z","iopub.status.idle":"2023-11-08T18:41:52.274122Z","shell.execute_reply.started":"2023-11-08T18:41:51.867476Z","shell.execute_reply":"2023-11-08T18:41:52.272315Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Dit is je logloss score. Lager is beter, maar onder 0 kan niet:","metadata":{}},{"cell_type":"code","source":"log_loss(val['awake'], val_pred['awake'])","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:52.279241Z","iopub.execute_input":"2023-11-08T18:41:52.279646Z","iopub.status.idle":"2023-11-08T18:41:52.318261Z","shell.execute_reply.started":"2023-11-08T18:41:52.279614Z","shell.execute_reply":"2023-11-08T18:41:52.316706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let op: de echte score voor de competitie is anders (en een stuk ingewikkelder, zie: https://www.kaggle.com/code/metric/event-detection-ap/notebook). Daar telt de hoogste! Deze score kun je zien als je de code hieronder runt en hem submit.\nJe kan pp tot 5 keer per dag inleveren. Zorg dat je wel alles openbaar doet, want anders kan deze interne competitie per ongeluk niet helemaal volgens de regels van Kaggle gaan!","metadata":{}},{"cell_type":"markdown","source":"#### Predictions on test","metadata":{}},{"cell_type":"code","source":"test  = pd.read_parquet(\"/kaggle/input/child-mind-institute-detect-sleep-states/test_series.parquet\")\n\ntest  = make_features(test)\n\nX_test = test[features]\n\ntest[\"not_awake\"] = classifier.predict_proba(X_test)[:,0]\ntest[\"awake\"]     = classifier.predict_proba(X_test)[:,1]","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:52.320238Z","iopub.execute_input":"2023-11-08T18:41:52.32063Z","iopub.status.idle":"2023-11-08T18:41:52.414376Z","shell.execute_reply.started":"2023-11-08T18:41:52.320599Z","shell.execute_reply":"2023-11-08T18:41:52.413176Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test[['not_awake', 'awake']]","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:52.415835Z","iopub.execute_input":"2023-11-08T18:41:52.416959Z","iopub.status.idle":"2023-11-08T18:41:52.437902Z","shell.execute_reply.started":"2023-11-08T18:41:52.416887Z","shell.execute_reply":"2023-11-08T18:41:52.436065Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from typing import Dict, List, Tuple\n\n# tolerances in steps\ntolerances = {\n    \"onset\":  [12, 36, 60, 90, 120, 150, 180, 240, 300, 360],\n    \"wakeup\": [12, 36, 60, 90, 120, 150, 180, 240, 300, 360],\n}","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:52.44109Z","iopub.execute_input":"2023-11-08T18:41:52.44168Z","iopub.status.idle":"2023-11-08T18:41:52.452453Z","shell.execute_reply.started":"2023-11-08T18:41:52.441633Z","shell.execute_reply":"2023-11-08T18:41:52.451198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"series_id_column_name = \"series_id\"\ntime_column_name = \"step\"\nevent_column_name = \"event\"\nscore_column_name = \"score\"\nuse_scoring_intervals = None","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:52.454087Z","iopub.execute_input":"2023-11-08T18:41:52.455629Z","iopub.status.idle":"2023-11-08T18:41:52.468289Z","shell.execute_reply.started":"2023-11-08T18:41:52.455594Z","shell.execute_reply":"2023-11-08T18:41:52.466108Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Processing","metadata":{}},{"cell_type":"code","source":"# smoothing the predictions\nsmoothing_length = 2*230\ntest[\"score\"]  = test[\"awake\"].rolling(smoothing_length,center=True).mean().fillna(method=\"bfill\").fillna(method=\"ffill\")\ntest[\"smooth\"] = test[\"not_awake\"].rolling(smoothing_length,center=True).mean().fillna(method=\"bfill\").fillna(method=\"ffill\")\n# re-binarize\ntest[\"smooth\"] = test[\"smooth\"].round()\n\n# https://stackoverflow.com/questions/73777727/how-to-mark-start-end-of-a-series-of-non-null-and-non-0-values-in-a-column-of-a\ndef get_event(df):\n    lstCV = zip(df.series_id, df.smooth)\n    lstPOI = []\n    for (c, v), g in groupby(lstCV, lambda cv: \n                            (cv[0], cv[1]!=0 and not pd.isnull(cv[1]))):\n        llg = sum(1 for item in g)\n        if v is False: \n            lstPOI.extend([0]*llg)\n        else: \n            lstPOI.extend(['onset']+(llg-2)*[0]+['wakeup'] if llg > 1 else [0])\n    return lstPOI\n\ntest[\"event\"] = get_event(test)","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:52.470469Z","iopub.execute_input":"2023-11-08T18:41:52.470905Z","iopub.status.idle":"2023-11-08T18:41:52.488732Z","shell.execute_reply.started":"2023-11-08T18:41:52.470868Z","shell.execute_reply":"2023-11-08T18:41:52.487276Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Write out a `submission.csv`","metadata":{}},{"cell_type":"code","source":"sample_submission = test.loc[test[\"event\"] != 0][[\"series_id\",\"step\",\"event\",\"score\"]].copy().reset_index(drop=True).reset_index(names=\"row_id\")\nsample_submission.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2023-11-08T18:41:52.490705Z","iopub.execute_input":"2023-11-08T18:41:52.491265Z","iopub.status.idle":"2023-11-08T18:41:52.508043Z","shell.execute_reply.started":"2023-11-08T18:41:52.491232Z","shell.execute_reply":"2023-11-08T18:41:52.506459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Related reading\n* [K. Sundararajan *et al. \"Sleep classification from wrist-worn accelerometer data using Random Forests*\", Scientific Reports volume **11**, Article number: 24 (2021)](https://www.nature.com/articles/s41598-020-79217-x.pdf)\n* [K. Sundararajan *et al. \"Supplementary Information for Article: Sleep classification from wrist-worn accelerometer data using Random Forests*\"](https://static-content.springer.com/esm/art%3A10.1038%2Fs41598-020-79217-x/MediaObjects/41598_2020_79217_MOESM1_ESM.pdf)","metadata":{}}]}