{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<div class=\"alert alert-info\" role=\"alert\">\n  <strong>TL;DR</strong>\n    \n  Since there are some similar discussions regarding the `signal_awake` or `signal_onset` stuff (see the list below), I would like to explain the logic behind it, and how is the impact into the public LB score of my team.\n</div>\n\n1. [What's the logic behind `signal_awake` and `signal_onset` in my notebook](https://www.kaggle.com/code/rimbax/lightgbm-feature-implementation-from-paper/comments#2440916)\n2. [What is this - \"signal_awake\" ...](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/441058)\n3. [What inspires me first to do converting hour to periodic signals](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/438298#2434537)","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport matplotlib.pyplot as plt","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-09-17T16:25:14.444204Z","iopub.execute_input":"2023-09-17T16:25:14.444604Z","iopub.status.idle":"2023-09-17T16:25:15.438387Z","shell.execute_reply.started":"2023-09-17T16:25:14.444571Z","shell.execute_reply":"2023-09-17T16:25:15.437075Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"First things first, let's load the data from the competition.\n\nNote that the data will be renewed as per Monday 2023/09/18 to fix the Daylight Time Saving error on the data.","metadata":{}},{"cell_type":"code","source":"events = pd.read_csv('/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv')\nevents","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-09-17T16:25:15.440872Z","iopub.execute_input":"2023-09-17T16:25:15.441555Z","iopub.status.idle":"2023-09-17T16:25:15.487308Z","shell.execute_reply.started":"2023-09-17T16:25:15.441508Z","shell.execute_reply":"2023-09-17T16:25:15.486100Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Then we drop the N/A and convert the timestamp with `tz_localize`","metadata":{}},{"cell_type":"code","source":"events.timestamp = pd.to_datetime(events.timestamp).apply(lambda x: x.tz_localize(None))\nevents = events.dropna()\nevents","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-09-17T16:25:15.489968Z","iopub.execute_input":"2023-09-17T16:25:15.491100Z","iopub.status.idle":"2023-09-17T16:25:15.824913Z","shell.execute_reply.started":"2023-09-17T16:25:15.491051Z","shell.execute_reply":"2023-09-17T16:25:15.823774Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Main Idea","metadata":{}},{"cell_type":"markdown","source":"As quoted [here](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/438298#2432393) and [here](https://www.kaggle.com/code/rimbax/lightgbm-feature-implementation-from-paper/comments#2441028), the idea is to **inform the model something like 23.00 is closer to 0 than to 19**, despite the fact that |23-19| < |0-23|.\n\nWith that, we can actually fit a sine wave into the data, where the peak should fit the highest frequency of an hour being an awake or onset event. It will give some sense of \"cycling\" period to inform the model as mentioned above. By matching to the highest frequency, we provide a magnitude such that hopefully the model knows that it is the time where most likely people will awake/asleep.\n\nNow let's see the distribution of the sleep and awake hour.","metadata":{}},{"cell_type":"code","source":"df_onset = events.loc[events.event == 'onset',  'timestamp'].dt.hour\ndf_awake = events.loc[events.event == 'wakeup', 'timestamp'].dt.hour\n\nsns.histplot(df_onset, discrete=True, bins=24, label='onset')\nsns.histplot(df_awake, discrete=True, bins=24, label='awake')\nplt.legend()\nplt.xlabel('hour')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-09-17T16:25:15.826427Z","iopub.execute_input":"2023-09-17T16:25:15.826834Z","iopub.status.idle":"2023-09-17T16:25:16.267131Z","shell.execute_reply.started":"2023-09-17T16:25:15.826802Z","shell.execute_reply":"2023-09-17T16:25:16.266279Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Matching The Wave","metadata":{}},{"cell_type":"markdown","source":"If we convert the distribution plot above as a line where the peak is equal to 1, the plot will be look like this.","metadata":{}},{"cell_type":"code","source":"dist_onset = pd.Series(range(24), name='hour') \\\n    .to_frame().join(df_onset.value_counts()) \\\n    .set_index('hour') \\\n    .rename(columns={'count': 'proba'})\ndist_onset.proba = dist_onset.proba.fillna(0) / dist_onset.proba.max()\n\ndist_awake = pd.Series(range(24), name='hour') \\\n    .to_frame().join(df_awake.value_counts()) \\\n    .set_index('hour') \\\n    .rename(columns={'count': 'proba'})\ndist_awake.proba = dist_awake.proba.fillna(0) / dist_awake.proba.max()\n\n\nplt.plot(dist_onset, label='onset')\nplt.plot(dist_awake, label='awake')\nplt.xlabel('hour')\nplt.ylabel('probability/distribution')\nplt.legend()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-09-17T16:25:16.269607Z","iopub.execute_input":"2023-09-17T16:25:16.269949Z","iopub.status.idle":"2023-09-17T16:25:16.567836Z","shell.execute_reply.started":"2023-09-17T16:25:16.269918Z","shell.execute_reply":"2023-09-17T16:25:16.566832Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now if we add a plot for a basic sine wave from x = 0 to 23, it will be like this.","metadata":{}},{"cell_type":"code","source":"signal_sine = np.sin(np.linspace(0, np.pi, 24))\n\nplt.plot(dist_onset, label='onset')\nplt.plot(dist_awake, label='awake')\nplt.plot(signal_sine, label='basic_sine')\nplt.xlabel('hour')\nplt.ylabel('probability/distribution')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-09-17T16:25:16.569218Z","iopub.execute_input":"2023-09-17T16:25:16.569628Z","iopub.status.idle":"2023-09-17T16:25:16.896465Z","shell.execute_reply.started":"2023-09-17T16:25:16.569585Z","shell.execute_reply":"2023-09-17T16:25:16.895360Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now our task is how to shift the wave to left or right of x-axis to match the peak.\n\nBasically by simple math, shifting a function $f(x)$ on x-axis is done by adding some constant to the $x$ like $f(x + c)$. Now our task is to find the $c$ for both onset and awake event.","metadata":{}},{"cell_type":"code","source":"# since sine could be negative, let's square it\nsignal_awake = np.sin(np.linspace(0, np.pi, 24) + 0.3 * np.pi) ** 2 # lets start by adding 0.3 * pi here\nsignal_onset = np.sin(np.linspace(0, np.pi, 24) + 0.7 * np.pi) ** 2 # lets start by adding 0.7 * pi here\n\nplt.plot(dist_onset, label='onset')\nplt.plot(dist_awake, label='awake')\nplt.plot(signal_awake, label='signal_awake ($sin(x + 0.3 \\pi)^2$)')\nplt.plot(signal_onset, label='signal_onset ($sin(x + 0.7 \\pi)^2$)')\nplt.xlabel('hour')\nplt.ylabel('probability/distribution')\n# plt.ylim(0, 1.1)\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-09-17T16:25:16.897903Z","iopub.execute_input":"2023-09-17T16:25:16.898641Z","iopub.status.idle":"2023-09-17T16:25:17.385317Z","shell.execute_reply.started":"2023-09-17T16:25:16.898591Z","shell.execute_reply":"2023-09-17T16:25:17.383906Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Nice, we are close to it. Let's adjust it a bit more.\n\nBy trial and error, you can find that the constant that fit the actual distribution is around `0.208 * np.pi` for awake event and `0.555 * np.pi` for the onset event (or something around that).","metadata":{}},{"cell_type":"code","source":"signal_awake = np.sin(np.linspace(0, np.pi, 24) + 0.208 * np.pi) ** 2 \nsignal_onset = np.sin(np.linspace(0, np.pi, 24) + 0.555 * np.pi) ** 2 \n\nplt.plot(dist_onset, label='onset')\nplt.plot(dist_awake, label='awake')\nplt.plot(signal_awake, label='signal_awake')\nplt.plot(signal_onset, label='signal_onset')\nplt.xlabel('hour')\nplt.ylabel('probability/distribution')\n# plt.ylim(0, 1.1)\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-09-17T16:25:17.386791Z","iopub.execute_input":"2023-09-17T16:25:17.387269Z","iopub.status.idle":"2023-09-17T16:25:17.717380Z","shell.execute_reply.started":"2023-09-17T16:25:17.387225Z","shell.execute_reply":"2023-09-17T16:25:17.716156Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now all we need is to \"squeeze\" the wave to match the distribution. In this case, we only need to increase the power `p` in the equation below\n\n`signal_awake = np.sin(np.linspace(0, np.pi, 24) + 0.208 * np.pi) ** p`\n\nHere I choose $p = 24$ as the number.","metadata":{}},{"cell_type":"code","source":"signal_awake = np.sin(np.linspace(0, np.pi, 24) + 0.208 * np.pi) ** 24 \nsignal_onset = np.sin(np.linspace(0, np.pi, 24) + 0.555 * np.pi) ** 24 \n\nplt.plot(dist_onset, label='onset')\nplt.plot(dist_awake, label='awake')\nplt.plot(signal_awake, label='signal_awake')\nplt.plot(signal_onset, label='signal_onset')\nplt.xlabel('hour')\nplt.ylabel('probability/distribution')\n# plt.ylim(0, 1.1)\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-09-17T16:25:17.718792Z","iopub.execute_input":"2023-09-17T16:25:17.719249Z","iopub.status.idle":"2023-09-17T16:25:18.045878Z","shell.execute_reply.started":"2023-09-17T16:25:17.719206Z","shell.execute_reply":"2023-09-17T16:25:18.044700Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Voila!** Until this we are actually done.","metadata":{}},{"cell_type":"markdown","source":"# Expanding to Minutes","metadata":{}},{"cell_type":"markdown","source":"I think rather than only involving hour here, why wouldn't we expand it into the minute level?\n\nInstead of only 24 values, we can actually use 24 x 60 = 1440 values, that time **HH:MM** will be converted into **HH * 60 + MM**.\n\nThis can be achieved simply by modifying the `np.linspace` up to 1440.","metadata":{}},{"cell_type":"code","source":"signal_awake = np.sin(np.linspace(0, np.pi, 1440) + 0.208 * np.pi) ** 24 \nsignal_onset = np.sin(np.linspace(0, np.pi, 1440) + 0.555 * np.pi) ** 24 \n\nplt.plot(signal_awake, label='signal_awake')\nplt.plot(signal_onset, label='signal_onset')\nplt.ylabel('probability/distribution')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-09-17T16:25:18.047717Z","iopub.execute_input":"2023-09-17T16:25:18.048505Z","iopub.status.idle":"2023-09-17T16:25:18.365621Z","shell.execute_reply.started":"2023-09-17T16:25:18.048458Z","shell.execute_reply":"2023-09-17T16:25:18.364406Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Smooth as smoothies.\n\nNow let's simulate to the event we have in the data.","metadata":{}},{"cell_type":"code","source":"signal_awake = dict(zip(range(1440), np.sin(np.linspace(0, np.pi, 1440) + 0.208 * np.pi) ** 24))\nsignal_onset = dict(zip(range(1440), np.sin(np.linspace(0, np.pi, 1440) + 0.555 * np.pi) ** 24))\n\nevents.loc[:, 'signal_awake'] = (events.timestamp.dt.hour * 60 + events.timestamp.dt.minute).map(signal_awake)\nevents.loc[:, 'signal_onset'] = (events.timestamp.dt.hour * 60 + events.timestamp.dt.minute).map(signal_onset)\n\nevents.round(3).head(10)","metadata":{"execution":{"iopub.status.busy":"2023-09-17T16:25:34.408296Z","iopub.execute_input":"2023-09-17T16:25:34.408677Z","iopub.status.idle":"2023-09-17T16:25:34.445669Z","shell.execute_reply.started":"2023-09-17T16:25:34.408647Z","shell.execute_reply":"2023-09-17T16:25:34.444568Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Most of the data have high signal values for related awake/onset event, that means our implementation is somewhat correct.","metadata":{}},{"cell_type":"markdown","source":"# How impactful?","metadata":{}},{"cell_type":"markdown","source":"Unfortunately I havent benchmarking the impact of using these features on the public LB score. All the information I had is on one of my notebook that is initially not using these periodic signal features, it has 0.139 LB. Once I add the signal features (and many more features, hence invalid comparison), the score increased to 0.199.\n\nHowever, in one of my [notebook](https://www.kaggle.com/code/rimbax/lightgbm-feature-implementation-from-paper), both of these two periodic signals are the most important features of my model, followed by LIDS ([Locomotor Inactivity During Sleep](https://www.nature.com/articles/s41598-020-79217-x.pdf)).\n\nAny help for this will be appreciated, as well as tuning the signal constants.","metadata":{}}]}