{
  "id": 459766,
  "title": "5th place solution",
  "url": "/competitions/child-mind-institute-detect-sleep-states/writeups/ruby-5th-place-solution",
  "author_name": "",
  "post_date": "2024-01-24T14:45:03.527Z",
  "votes": 63,
  "comment_count": 10,
  "views": 0,
  "content": "<p>Thanks Kaggle and competition host for this interesting competition that can be approached by various of methods, it is an enjoyable journey to explore this dataset.<br>\nAnd also great thanks to people who sharing insights and ideas during the process of this competition, I learnt a lot from you all.</p>\n<p><strong>overview</strong><br>\nI approach this problem in four stages: </p>\n<ol>\n<li>candidate step generation by heuristic rules </li>\n<li>step correct model to improve step quality </li>\n<li>score model generate score for submission </li>\n<li>post-processing </li>\n</ol>\n<p><strong>candidate generation</strong><br>\nbasic idea of heuristic rules, you can check code for more details and tricks.</p>\n<ul>\n<li>region with |diff(anglez)|&lt;5: start -&gt; onset, end -&gt; wakeup. <br>\nOnset/wakeup is not likely to happen in the middle of an inactive region. Region with suspicious fake data (rules to detect such region is described later) is removed.</li>\n<li>fake region extended by inactive region: start-&gt;wakeup, end-&gt;onset. <br>\nOne explanation is that behavior of removing the watch can only happen when one is awake.</li>\n<li>current candidates shift by +/-720 when no other candidates in range of 720*2.</li>\n</ul>\n<p>The three rules generate 254653 candidate steps in total for all series with best possible score 0.9006.</p>\n<p><strong>step correct model</strong></p>\n<ul>\n<li>Lightgbm L1 regression with (nearest_target_step-step) as target</li>\n<li>since we just interested in the quality of steps that matched with target, weight of data points with |target|&gt;=360 is set to be zero</li>\n<li>data points is weighted by threshold_class_width^(-6) to focusing on ones nearing the target, as I don’t think accurately predict the gap is possible when true target is far away.（threshold_class_width eg: 120&lt;y=127&lt;150, then resp class_width is (150-120)=30）There are also many other choice of weights that can reach similar performance based on my experiments.</li>\n<li>As minute%15 and second are the most important features I think this stage is mainly correct for the bias in label generation process rather than actually improve the step quality. And the most gains of this stage actually come from shift step by ~-11, which is not reflected in feature importance list.</li>\n<li>To validate next stage without leak CV is done in inner fold with 5fold group-k-fold by series_id, so 2* 5* 5 models in total.</li>\n</ul>\n<p>Best possible score after this stage is 0.932.</p>\n<p><strong>score model</strong></p>\n<ul>\n<li>Lightgbm with cross-entropy objective</li>\n<li>target=max(0,1-|nearest_target_step-step|/360) for data points nearest to each target else target is set to zero.</li>\n<li>negative data points (target=0) is down weight by *0.7/0.4 for those with |nearest_target_step-step| &gt;/&lt;360.</li>\n<li>target is created on corrected step given by out of fold prediction in last stage.</li>\n<li>I create different models for onset and wakeup candidates and use 5fold group-k-fold by series_id to do validation, so 2*5 models in total.</li>\n</ul>\n<p>I compute competition metric after post processing, so no score is reported here, but It is easy to add one.</p>\n<p><strong>post-processing</strong><br>\nthis consists of three parts:</p>\n<ul>\n<li>if step%12=0 then step+/- 1 to match more target steps.</li>\n<li>for two candidates with gap &lt; 720 only keep them when:               <br>\n<code>min(|gap|/720,1)*(exp_score1*exp_score2/(exp_score1**2+exp_score2**2))**0.5&gt;0.083</code><br>\nhere <br>\n<code>exp_score=np.exp(np.arctanh(2*score-1))</code><br>\nelse remove the one with smaller score.<br>\nThis avoid too dense candidates and only works when model preference to candidates is clearly biased.</li>\n<li>If score sum in day for some series is larger than 1 then all scores in that day will be divide by score sum.</li>\n</ul>\n<p>After post-processing CV score 0.825, public LB score 0.783, private LB score 0.844.</p>\n<p><strong>more details</strong></p>\n<ul>\n<li><p><strong>fake region</strong></p>\n<ol>\n<li>anglez 5min mean (6 decimal accuracy) appear more than one time in the same series</li>\n<li>anglez 5min std&gt;0.5 to avoid mark some inactive region as fake</li>\n<li>mark regions surrounded by long fake region as fake, as described in GGIR website.</li></ol></li>\n<li><p><strong>data cleaning</strong></p>\n<ol>\n<li>remove some series with many wrong labels by manual inspection</li>\n<li>data points covered by long time unlabeled tail region is set to weight zero</li>\n<li>data points related to target with unexpected anglez distribution is set to weight zero. (onset followed by active region, wakeup followed by inactive region )</li></ol>\n<p>Those cases are only removed from training process, I still include them when compute CV score.</p></li>\n<li><p><strong>features</strong></p>\n<p>2040 in total, I only describe the most important ones, the remaining ones can be duplicated, unimportant or useless. You can check code for more details.</p>\n<ol>\n<li>window based features: <strong>source_column X stats X window_size X window_operations</strong>, not full combination.<br>\n<strong>source_column:</strong>  <em>anglez_abs_diff, enmo_abs_diff, anglez_abs_diff_quantile, smoothed_anglez_abs_diff, \nanglez_abs_diff_in_given_range, fake_mark, sleep_mark by GGIR heuristic rules, …</em><br>\n<strong>stats</strong>:  <em>mean/50 quantile/95 quantile/max/min</em><br>\n<strong>window_size</strong>:  <em>1min,3min,5min,10min,15min,30min,1h,2h,4h,8h,12h</em><br>\n<strong>window_operations</strong>:  <em>left_side_window(lw), right_side_window(rw), (lw-rw), [(lw-rw)/(lw+rw)], concat(lw,rw), [(lw-rw)/ concat(lw,rw))]</em></li>\n<li>time features<br>\nweekday, hour, second, minute, second_in_day, step/max_step<br>\nminute%15 (as discussed in <a href=\"https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/444374\" target=\"_blank\">https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/444374</a>)</li>\n<li>feature stats aggregated in day<br>\nmainly fake_mark and sleep_mark</li>\n<li>step gap to mean event time in series given by heuristic methods</li>\n<li>stats values at the step time</li>\n<li>step gap to nearby candidates and feature value gap with some features that ranked high based on feature importance, manually picked.</li></ol></li>\n</ul>\n<p><strong>Reproduce</strong><br>\nre-run following notebooks in order<br>\n<a href=\"https://www.kaggle.com/code/w5833946/cmi-lgb-v9-train-reproduce\" target=\"_blank\">https://www.kaggle.com/code/w5833946/cmi-lgb-v9-train-reproduce</a><br>\n<a href=\"https://www.kaggle.com/code/w5833946/cmi-lgb-v9-predict-reproduce\" target=\"_blank\">https://www.kaggle.com/code/w5833946/cmi-lgb-v9-predict-reproduce</a></p>",
  "messages": [
    {
      "id": "2551210",
      "postDate": "12/06/2023 15:01:28",
      "content": "<p>Thanks Kaggle and competition host for this interesting competition that can be approached by various of methods, it is an enjoyable journey to explore this dataset.<br>\nAnd also great thanks to people who sharing insights and ideas during the process of this competition, I learnt a lot from you all.</p>\n<p><strong>overview</strong><br>\nI approach this problem in four stages: </p>\n<ol>\n<li>candidate step generation by heuristic rules </li>\n<li>step correct model to improve step quality </li>\n<li>score model generate score for submission </li>\n<li>post-processing </li>\n</ol>\n<p><strong>candidate generation</strong><br>\nbasic idea of heuristic rules, you can check code for more details and tricks.</p>\n<ul>\n<li>region with |diff(anglez)|&lt;5: start -&gt; onset, end -&gt; wakeup. <br>\nOnset/wakeup is not likely to happen in the middle of an inactive region. Region with suspicious fake data (rules to detect such region is described later) is removed.</li>\n<li>fake region extended by inactive region: start-&gt;wakeup, end-&gt;onset. <br>\nOne explanation is that behavior of removing the watch can only happen when one is awake.</li>\n<li>current candidates shift by +/-720 when no other candidates in range of 720*2.</li>\n</ul>\n<p>The three rules generate 254653 candidate steps in total for all series with best possible score 0.9006.</p>\n<p><strong>step correct model</strong></p>\n<ul>\n<li>Lightgbm L1 regression with (nearest_target_step-step) as target</li>\n<li>since we just interested in the quality of steps that matched with target, weight of data points with |target|&gt;=360 is set to be zero</li>\n<li>data points is weighted by threshold_class_width^(-6) to focusing on ones nearing the target, as I don’t think accurately predict the gap is possible when true target is far away.（threshold_class_width eg: 120&lt;y=127&lt;150, then resp class_width is (150-120)=30）There are also many other choice of weights that can reach similar performance based on my experiments.</li>\n<li>As minute%15 and second are the most important features I think this stage is mainly correct for the bias in label generation process rather than actually improve the step quality. And the most gains of this stage actually come from shift step by ~-11, which is not reflected in feature importance list.</li>\n<li>To validate next stage without leak CV is done in inner fold with 5fold group-k-fold by series_id, so 2* 5* 5 models in total.</li>\n</ul>\n<p>Best possible score after this stage is 0.932.</p>\n<p><strong>score model</strong></p>\n<ul>\n<li>Lightgbm with cross-entropy objective</li>\n<li>target=max(0,1-|nearest_target_step-step|/360) for data points nearest to each target else target is set to zero.</li>\n<li>negative data points (target=0) is down weight by *0.7/0.4 for those with |nearest_target_step-step| &gt;/&lt;360.</li>\n<li>target is created on corrected step given by out of fold prediction in last stage.</li>\n<li>I create different models for onset and wakeup candidates and use 5fold group-k-fold by series_id to do validation, so 2*5 models in total.</li>\n</ul>\n<p>I compute competition metric after post processing, so no score is reported here, but It is easy to add one.</p>\n<p><strong>post-processing</strong><br>\nthis consists of three parts:</p>\n<ul>\n<li>if step%12=0 then step+/- 1 to match more target steps.</li>\n<li>for two candidates with gap &lt; 720 only keep them when:               <br>\n<code>min(|gap|/720,1)*(exp_score1*exp_score2/(exp_score1**2+exp_score2**2))**0.5&gt;0.083</code><br>\nhere <br>\n<code>exp_score=np.exp(np.arctanh(2*score-1))</code><br>\nelse remove the one with smaller score.<br>\nThis avoid too dense candidates and only works when model preference to candidates is clearly biased.</li>\n<li>If score sum in day for some series is larger than 1 then all scores in that day will be divide by score sum.</li>\n</ul>\n<p>After post-processing CV score 0.825, public LB score 0.783, private LB score 0.844.</p>\n<p><strong>more details</strong></p>\n<ul>\n<li><p><strong>fake region</strong></p>\n<ol>\n<li>anglez 5min mean (6 decimal accuracy) appear more than one time in the same series</li>\n<li>anglez 5min std&gt;0.5 to avoid mark some inactive region as fake</li>\n<li>mark regions surrounded by long fake region as fake, as described in GGIR website.</li></ol></li>\n<li><p><strong>data cleaning</strong></p>\n<ol>\n<li>remove some series with many wrong labels by manual inspection</li>\n<li>data points covered by long time unlabeled tail region is set to weight zero</li>\n<li>data points related to target with unexpected anglez distribution is set to weight zero. (onset followed by active region, wakeup followed by inactive region )</li></ol>\n<p>Those cases are only removed from training process, I still include them when compute CV score.</p></li>\n<li><p><strong>features</strong></p>\n<p>2040 in total, I only describe the most important ones, the remaining ones can be duplicated, unimportant or useless. You can check code for more details.</p>\n<ol>\n<li>window based features: <strong>source_column X stats X window_size X window_operations</strong>, not full combination.<br>\n<strong>source_column:</strong>  <em>anglez_abs_diff, enmo_abs_diff, anglez_abs_diff_quantile, smoothed_anglez_abs_diff, \nanglez_abs_diff_in_given_range, fake_mark, sleep_mark by GGIR heuristic rules, …</em><br>\n<strong>stats</strong>:  <em>mean/50 quantile/95 quantile/max/min</em><br>\n<strong>window_size</strong>:  <em>1min,3min,5min,10min,15min,30min,1h,2h,4h,8h,12h</em><br>\n<strong>window_operations</strong>:  <em>left_side_window(lw), right_side_window(rw), (lw-rw), [(lw-rw)/(lw+rw)], concat(lw,rw), [(lw-rw)/ concat(lw,rw))]</em></li>\n<li>time features<br>\nweekday, hour, second, minute, second_in_day, step/max_step<br>\nminute%15 (as discussed in <a href=\"https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/444374\" target=\"_blank\">https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/444374</a>)</li>\n<li>feature stats aggregated in day<br>\nmainly fake_mark and sleep_mark</li>\n<li>step gap to mean event time in series given by heuristic methods</li>\n<li>stats values at the step time</li>\n<li>step gap to nearby candidates and feature value gap with some features that ranked high based on feature importance, manually picked.</li></ol></li>\n</ul>\n<p><strong>Reproduce</strong><br>\nre-run following notebooks in order<br>\n<a href=\"https://www.kaggle.com/code/w5833946/cmi-lgb-v9-train-reproduce\" target=\"_blank\">https://www.kaggle.com/code/w5833946/cmi-lgb-v9-train-reproduce</a><br>\n<a href=\"https://www.kaggle.com/code/w5833946/cmi-lgb-v9-predict-reproduce\" target=\"_blank\">https://www.kaggle.com/code/w5833946/cmi-lgb-v9-predict-reproduce</a></p>",
      "rawMarkdown": "Thanks Kaggle and competition host for this interesting competition that can be approached by various of methods, it is an enjoyable journey to explore this dataset.\nAnd also great thanks to people who sharing insights and ideas during the process of this competition, I learnt a lot from you all.\n\n**overview**\nI approach this problem in four stages: \n1. candidate step generation by heuristic rules \n2. step correct model to improve step quality \n3. score model generate score for submission \n4. post-processing \n\n\n**candidate generation**\nbasic idea of heuristic rules, you can check code for more details and tricks.\n- region with |diff(anglez)|<5: start -> onset, end -> wakeup. \nOnset/wakeup is not likely to happen in the middle of an inactive region. Region with suspicious fake data (rules to detect such region is described later) is removed.\n- fake region extended by inactive region: start->wakeup, end->onset. \nOne explanation is that behavior of removing the watch can only happen when one is awake.\n- current candidates shift by +/-720 when no other candidates in range of 720*2.\n\nThe three rules generate 254653 candidate steps in total for all series with best possible score 0.9006.\n\n\n**step correct model**\n- Lightgbm L1 regression with (nearest_target_step-step) as target\n- since we just interested in the quality of steps that matched with target, weight of data points with |target|>=360 is set to be zero\n- data points is weighted by threshold_class_width^(-6) to focusing on ones nearing the target, as I don’t think accurately predict the gap is possible when true target is far away.（threshold_class_width eg: 120<y=127<150, then resp class_width is (150-120)=30）There are also many other choice of weights that can reach similar performance based on my experiments.\n- As minute%15 and second are the most important features I think this stage is mainly correct for the bias in label generation process rather than actually improve the step quality. And the most gains of this stage actually come from shift step by ~-11, which is not reflected in feature importance list.\n- To validate next stage without leak CV is done in inner fold with 5fold group-k-fold by series_id, so 2* 5* 5 models in total.\n\nBest possible score after this stage is 0.932.\n\n\n**score model**\n- Lightgbm with cross-entropy objective\n- target=max(0,1-|nearest_target_step-step|/360) for data points nearest to each target else target is set to zero.\n- negative data points (target=0) is down weight by *0.7/0.4 for those with |nearest_target_step-step| >/<360.\n- target is created on corrected step given by out of fold prediction in last stage.\n- I create different models for onset and wakeup candidates and use 5fold group-k-fold by series_id to do validation, so 2*5 models in total.\n\nI compute competition metric after post processing, so no score is reported here, but It is easy to add one.\n\n\n**post-processing**\nthis consists of three parts:\n- if step%12=0 then step+/- 1 to match more target steps.\n- for two candidates with gap < 720 only keep them when:               \n`min(|gap|/720,1)*(exp_score1*exp_score2/(exp_score1**2+exp_score2**2))**0.5>0.083`\nhere \n`exp_score=np.exp(np.arctanh(2*score-1))`\nelse remove the one with smaller score.\nThis avoid too dense candidates and only works when model preference to candidates is clearly biased.\n- If score sum in day for some series is larger than 1 then all scores in that day will be divide by score sum.\n\nAfter post-processing CV score 0.825, public LB score 0.783, private LB score 0.844.\n\n\n**more details**\n- **fake region**\n  1. anglez 5min mean (6 decimal accuracy) appear more than one time in the same series\n  2. anglez 5min std>0.5 to avoid mark some inactive region as fake\n  3. mark regions surrounded by long fake region as fake, as described in GGIR website.\n- **data cleaning**\n  1. remove some series with many wrong labels by manual inspection\n  2. data points covered by long time unlabeled tail region is set to weight zero\n  3. data points related to target with unexpected anglez distribution is set to weight zero. (onset followed by active region, wakeup followed by inactive region )\n  \n  Those cases are only removed from training process, I still include them when compute CV score.\n- **features**\n  \n  2040 in total, I only describe the most important ones, the remaining ones can be duplicated, unimportant or useless. You can check code for more details.\n  1. window based features: **source_column X stats X window_size X window_operations**, not full combination.\n  **source_column:**  *anglez_abs_diff, enmo_abs_diff, anglez_abs_diff_quantile, smoothed_anglez_abs_diff, \n  anglez_abs_diff_in_given_range, fake_mark, sleep_mark by GGIR heuristic rules, …*\n  **stats**:  *mean/50 quantile/95 quantile/max/min*\n  **window_size**:  *1min,3min,5min,10min,15min,30min,1h,2h,4h,8h,12h*\n  **window_operations**:  *left_side_window(lw), right_side_window(rw), (lw-rw), [(lw-rw)/(lw+rw)], concat(lw,rw), [(lw-rw)/ concat(lw,rw))]*\n  2. time features\nweekday, hour, second, minute, second_in_day, step/max_step\nminute%15 (as discussed in https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/444374)\n  3. feature stats aggregated in day\nmainly fake_mark and sleep_mark\n  4. step gap to mean event time in series given by heuristic methods\n  5. stats values at the step time\n  6. step gap to nearby candidates and feature value gap with some features that ranked high based on feature importance, manually picked.\n\n\n**Reproduce**\nre-run following notebooks in order\nhttps://www.kaggle.com/code/w5833946/cmi-lgb-v9-train-reproduce\nhttps://www.kaggle.com/code/w5833946/cmi-lgb-v9-predict-reproduce",
      "votes": null
    },
    {
      "id": "2551269",
      "postDate": "12/06/2023 15:40:16",
      "content": "<p>I really regret giving up on LGBMs so early it seems many of the top solutions, like yourself, were able to have very creative implementations for it! Great work!</p>",
      "rawMarkdown": "I really regret giving up on LGBMs so early it seems many of the top solutions, like yourself, were able to have very creative implementations for it! Great work!",
      "votes": null
    },
    {
      "id": "2551753",
      "postDate": "12/06/2023 23:52:49",
      "content": "<p>Wow thank you for sharing your solution and the notebooks! I really liked the idea of sample weighting and target construction. Very elegant 👌</p>\n<p>Unfortunately, I also gave up on the tabular models too early.. But seeing them work, amazing!</p>",
      "rawMarkdown": "Wow thank you for sharing your solution and the notebooks! I really liked the idea of sample weighting and target construction. Very elegant 👌\n\nUnfortunately, I also gave up on the tabular models too early.. But seeing them work, amazing!",
      "votes": null
    },
    {
      "id": "2551945",
      "postDate": "12/07/2023 04:41:55",
      "content": "<p>GPU deep learning</p>",
      "rawMarkdown": "GPU deep learning",
      "votes": null
    },
    {
      "id": "2551961",
      "postDate": "12/07/2023 05:08:15",
      "content": "<p>What a great achievement!! Let me learn from these notebooks.</p>",
      "rawMarkdown": "What a great achievement!! Let me learn from these notebooks.",
      "votes": null
    },
    {
      "id": "2552436",
      "postDate": "12/07/2023 13:11:55",
      "content": "<p>great achievement <a href=\"https://www.kaggle.com/w5833946\" target=\"_blank\">@w5833946</a> </p>",
      "rawMarkdown": "great achievement @w5833946",
      "votes": null
    },
    {
      "id": "2553966",
      "postDate": "12/08/2023 16:56:59",
      "content": "<p>Congrats on coming 5th place <a href=\"https://www.kaggle.com/w5833946\" target=\"_blank\">@w5833946</a> </p>",
      "rawMarkdown": "Congrats on coming 5th place @w5833946",
      "votes": null
    },
    {
      "id": "2555792",
      "postDate": "12/10/2023 07:51:53",
      "content": "<p>I didn't understand it clearly , congra to u</p>",
      "rawMarkdown": "I didn't understand it clearly , congra to u",
      "votes": null
    },
    {
      "id": "2556766",
      "postDate": "12/11/2023 01:15:04",
      "content": "<p><a href=\"https://www.kaggle.com/w5833946\" target=\"_blank\">@w5833946</a> Congratulations. This is an amazing LGBM model! Congrats on solo cash gold 5th place!</p>",
      "rawMarkdown": "w5833946 Congratulations. This is an amazing LGBM model! Congrats on solo cash gold 5th place!",
      "votes": null
    },
    {
      "id": "2567120",
      "postDate": "12/19/2023 12:35:58",
      "content": "<p>Congratulations for this masterpiece and thanks to you for sharing your solution</p>",
      "rawMarkdown": "Congratulations for this masterpiece and thanks to you for sharing your solution",
      "votes": null
    },
    {
      "id": "2573907",
      "postDate": "12/25/2023 13:14:42",
      "content": "<p>Love your idea most!</p>",
      "rawMarkdown": "Love your idea most!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2551269,
      "author_name": "cody11null",
      "author_url": "",
      "post_date": "12/06/2023 15:40:16",
      "content": "<p>I really regret giving up on LGBMs so early it seems many of the top solutions, like yourself, were able to have very creative implementations for it! Great work!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2551753,
      "author_name": "snnclsr",
      "author_url": "",
      "post_date": "12/06/2023 23:52:49",
      "content": "<p>Wow thank you for sharing your solution and the notebooks! I really liked the idea of sample weighting and target construction. Very elegant 👌</p>\n<p>Unfortunately, I also gave up on the tabular models too early.. But seeing them work, amazing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2551945,
      "author_name": "kallolmohammad",
      "author_url": "",
      "post_date": "12/07/2023 04:41:55",
      "content": "<p>GPU deep learning</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2551961,
      "author_name": "mitsuyasuhoshino",
      "author_url": "",
      "post_date": "12/07/2023 05:08:15",
      "content": "<p>What a great achievement!! Let me learn from these notebooks.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2552436,
      "author_name": "nithinreddy90",
      "author_url": "",
      "post_date": "12/07/2023 13:11:55",
      "content": "<p>great achievement <a href=\"https://www.kaggle.com/w5833946\" target=\"_blank\">@w5833946</a> </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2553966,
      "author_name": "malabhbakshi",
      "author_url": "",
      "post_date": "12/08/2023 16:56:59",
      "content": "<p>Congrats on coming 5th place <a href=\"https://www.kaggle.com/w5833946\" target=\"_blank\">@w5833946</a> </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2555792,
      "author_name": "luficergfree",
      "author_url": "",
      "post_date": "12/10/2023 07:51:53",
      "content": "<p>I didn't understand it clearly , congra to u</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2556766,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "12/11/2023 01:15:04",
      "content": "<p><a href=\"https://www.kaggle.com/w5833946\" target=\"_blank\">@w5833946</a> Congratulations. This is an amazing LGBM model! Congrats on solo cash gold 5th place!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2567120,
      "author_name": "jler35",
      "author_url": "",
      "post_date": "12/19/2023 12:35:58",
      "content": "<p>Congratulations for this masterpiece and thanks to you for sharing your solution</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2573907,
      "author_name": "clas233",
      "author_url": "",
      "post_date": "12/25/2023 13:14:42",
      "content": "<p>Love your idea most!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2551210": "Thanks Kaggle and competition host for this interesting competition that can be approached by various of methods, it is an enjoyable journey to explore this dataset.\nAnd also great thanks to people who sharing insights and ideas during the process of this competition, I learnt a lot from you all.\n\n**overview**\nI approach this problem in four stages: \n1. candidate step generation by heuristic rules \n2. step correct model to improve step quality \n3. score model generate score for submission \n4. post-processing \n\n\n**candidate generation**\nbasic idea of heuristic rules, you can check code for more details and tricks.\n- region with |diff(anglez)|<5: start -> onset, end -> wakeup. \nOnset/wakeup is not likely to happen in the middle of an inactive region. Region with suspicious fake data (rules to detect such region is described later) is removed.\n- fake region extended by inactive region: start->wakeup, end->onset. \nOne explanation is that behavior of removing the watch can only happen when one is awake.\n- current candidates shift by +/-720 when no other candidates in range of 720*2.\n\nThe three rules generate 254653 candidate steps in total for all series with best possible score 0.9006.\n\n\n**step correct model**\n- Lightgbm L1 regression with (nearest_target_step-step) as target\n- since we just interested in the quality of steps that matched with target, weight of data points with |target|>=360 is set to be zero\n- data points is weighted by threshold_class_width^(-6) to focusing on ones nearing the target, as I don’t think accurately predict the gap is possible when true target is far away.（threshold_class_width eg: 120<y=127<150, then resp class_width is (150-120)=30）There are also many other choice of weights that can reach similar performance based on my experiments.\n- As minute%15 and second are the most important features I think this stage is mainly correct for the bias in label generation process rather than actually improve the step quality. And the most gains of this stage actually come from shift step by ~-11, which is not reflected in feature importance list.\n- To validate next stage without leak CV is done in inner fold with 5fold group-k-fold by series_id, so 2* 5* 5 models in total.\n\nBest possible score after this stage is 0.932.\n\n\n**score model**\n- Lightgbm with cross-entropy objective\n- target=max(0,1-|nearest_target_step-step|/360) for data points nearest to each target else target is set to zero.\n- negative data points (target=0) is down weight by *0.7/0.4 for those with |nearest_target_step-step| >/<360.\n- target is created on corrected step given by out of fold prediction in last stage.\n- I create different models for onset and wakeup candidates and use 5fold group-k-fold by series_id to do validation, so 2*5 models in total.\n\nI compute competition metric after post processing, so no score is reported here, but It is easy to add one.\n\n\n**post-processing**\nthis consists of three parts:\n- if step%12=0 then step+/- 1 to match more target steps.\n- for two candidates with gap < 720 only keep them when:               \n`min(|gap|/720,1)*(exp_score1*exp_score2/(exp_score1**2+exp_score2**2))**0.5>0.083`\nhere \n`exp_score=np.exp(np.arctanh(2*score-1))`\nelse remove the one with smaller score.\nThis avoid too dense candidates and only works when model preference to candidates is clearly biased.\n- If score sum in day for some series is larger than 1 then all scores in that day will be divide by score sum.\n\nAfter post-processing CV score 0.825, public LB score 0.783, private LB score 0.844.\n\n\n**more details**\n- **fake region**\n  1. anglez 5min mean (6 decimal accuracy) appear more than one time in the same series\n  2. anglez 5min std>0.5 to avoid mark some inactive region as fake\n  3. mark regions surrounded by long fake region as fake, as described in GGIR website.\n- **data cleaning**\n  1. remove some series with many wrong labels by manual inspection\n  2. data points covered by long time unlabeled tail region is set to weight zero\n  3. data points related to target with unexpected anglez distribution is set to weight zero. (onset followed by active region, wakeup followed by inactive region )\n  \n  Those cases are only removed from training process, I still include them when compute CV score.\n- **features**\n  \n  2040 in total, I only describe the most important ones, the remaining ones can be duplicated, unimportant or useless. You can check code for more details.\n  1. window based features: **source_column X stats X window_size X window_operations**, not full combination.\n  **source_column:**  *anglez_abs_diff, enmo_abs_diff, anglez_abs_diff_quantile, smoothed_anglez_abs_diff, \n  anglez_abs_diff_in_given_range, fake_mark, sleep_mark by GGIR heuristic rules, …*\n  **stats**:  *mean/50 quantile/95 quantile/max/min*\n  **window_size**:  *1min,3min,5min,10min,15min,30min,1h,2h,4h,8h,12h*\n  **window_operations**:  *left_side_window(lw), right_side_window(rw), (lw-rw), [(lw-rw)/(lw+rw)], concat(lw,rw), [(lw-rw)/ concat(lw,rw))]*\n  2. time features\nweekday, hour, second, minute, second_in_day, step/max_step\nminute%15 (as discussed in https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/444374)\n  3. feature stats aggregated in day\nmainly fake_mark and sleep_mark\n  4. step gap to mean event time in series given by heuristic methods\n  5. stats values at the step time\n  6. step gap to nearby candidates and feature value gap with some features that ranked high based on feature importance, manually picked.\n\n\n**Reproduce**\nre-run following notebooks in order\nhttps://www.kaggle.com/code/w5833946/cmi-lgb-v9-train-reproduce\nhttps://www.kaggle.com/code/w5833946/cmi-lgb-v9-predict-reproduce",
    "2551269": "I really regret giving up on LGBMs so early it seems many of the top solutions, like yourself, were able to have very creative implementations for it! Great work!",
    "2551753": "Wow thank you for sharing your solution and the notebooks! I really liked the idea of sample weighting and target construction. Very elegant 👌\n\nUnfortunately, I also gave up on the tabular models too early.. But seeing them work, amazing!",
    "2551945": "GPU deep learning",
    "2551961": "What a great achievement!! Let me learn from these notebooks.",
    "2552436": "great achievement @w5833946",
    "2553966": "Congrats on coming 5th place @w5833946",
    "2555792": "I didn't understand it clearly , congra to u",
    "2556766": "w5833946 Congratulations. This is an amazing LGBM model! Congrats on solo cash gold 5th place!",
    "2567120": "Congratulations for this masterpiece and thanks to you for sharing your solution",
    "2573907": "Love your idea most!"
  },
  "source": "meta"
}