{
  "id": 459627,
  "title": "2nd Place Solution",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/459627",
  "author_name": "K_mat",
  "post_date": "2023-12-06T03:22:42.754000",
  "votes": 177,
  "comment_count": 27,
  "views": 0,
  "content": "<p>Thanks to organizers for this interesting challenge and congrats everyone who enjoyed it! It was a fun competition though the data is very simple 1D data. I look forward to seeing the various solutions.</p>\n<h1>Overview of my solution</h1>\n<p>My pipeline consists of:</p>\n<ul>\n<li><strong>First Stage</strong><ul>\n<li>Preprocess and Feature engineering</li>\n<li>Event Detection and Sleep/Awake Classification by 1DCNN(U-NET)</li>\n<li>PostProcess of 1D CNN (such as Peak Detection)</li></ul></li>\n<li><strong>Second Stage</strong><ul>\n<li>Use ML instead of the Post-processing that is difficult to adjust</li>\n<li>Rescoring by LGBM that can consider the limitation of 2evets/day</li></ul></li>\n<li><strong>Third Stage</strong><ul>\n<li>Predict as many events as possible just by shifting the step of the base prediction above.</li>\n<li>Scoring the generated predictions by LGBM</li></ul></li>\n<li><strong>Ensemble</strong><ul>\n<li>combination of averaging and WBF ensemble</li></ul></li>\n</ul>\n<p>I think the 2nd stage is the most unique. <br>\nThis model predict the daily accuracy curve of candidates received from 1st stage. </p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2Fe307a878f4ca758835ff897f401a7008%2Foverview.png?generation=1701831890750504&amp;alt=media\" alt=\"\"></p>\n<p>Details are below.</p>\n<h1>1st Stage: Event Detection and Sleep/Awake Classification</h1>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F08d268f6a008646b39b272594226df16%2F1ststage.png?generation=1701832322797003&amp;alt=media\" alt=\"\"><br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F94bdf0ce5785ba775f2a818c769d3a9d%2F1ststagepp.png?generation=1701832338347452&amp;alt=media\" alt=\"\"></p>\n<h1>2nd Stage: Rescoring the confidence considering the limitation of 2 events/day</h1>\n<h2>Concept</h2>\n<p>I applied 2nd stage for the following 2 reasons.</p>\n<h3>1. Less than 2 events/day</h3>\n<p>The first stage NN cannot consider the very important limitation of this competition that the event occurs at most twice in a day. Which do you think is important?</p>\n<blockquote>\n  <p>A) The third candidate in a day with a confidence score of 0.20.<br>\n  B) The first candidate in a day with a confidence score of 0.19.</p>\n</blockquote>\n<p>In many cases, the latter is more important than the former though the score is lower.</p>\n<h3>Long-term features</h3>\n<p>NN is really strong. However, it’s not good at handling meta features and long-term cyclic features. For example,<br>\n“He can be sleeping longer than the usual. This might not actual wakeup.”<br>\n“This onset is relatively unclear compared to the other days.”</p>\n<h2>How to train 2nd stage model</h2>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2Ff6b0889e818ed24add13fd5d74580884%2F2ndstage_r.png?generation=1701861341160280&amp;alt=media\" alt=\"\"></p>\n<h1>3rd Stage: Add as many events as possible</h1>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F049000b0c98a2e5f8bd5f2692a1c2986%2F3rdstage.png?generation=1701832658530598&amp;alt=media\" alt=\"\"></p>\n<h1>Ensemble</h1>\n<ul>\n<li>prepare 2 CNN models</li>\n<li>Averaging 10(5fold x 2seed) predictions at 1st stage for each model</li>\n<li>Run 2nd stage for each model</li>\n<li>WBF-like Ensemble of 2 models</li>\n<li>Run 3rd stage<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F8f6a3cf08dfcf72067db9d7a3db5d849%2FEnsemble.png?generation=1701832240218774&amp;alt=media\" alt=\"\"></li>\n</ul>\n<h1>Ablation Study</h1>\n<p>not yet</p>\n<ul>\n<li>CV Scores<ul>\n<li>1st stage: 0.826 (-0.019)</li>\n<li>add 2nd stage: 0.832 (-0.012)</li>\n<li>add third stage: 0.842 (-0.002)</li>\n<li>add model ensemble(2models): 0.844 (baseline. final submit)</li></ul></li>\n</ul>\n<h3>Inference</h3>\n<p><a href=\"https://www.kaggle.com/code/kmat2019/cmi-sleep-2ndplace\" target=\"_blank\">https://www.kaggle.com/code/kmat2019/cmi-sleep-2ndplace</a></p>\n<h3>Training</h3>\n<p><a href=\"https://www.kaggle.com/code/kmat2019/cmisleep-training-sample-2ndplace-kmat\" target=\"_blank\">https://www.kaggle.com/code/kmat2019/cmisleep-training-sample-2ndplace-kmat</a></p>",
  "messages": [
    {
      "id": 2550443,
      "postDate": "2023-12-06T03:22:42.753Z",
      "content": "<p>Thanks to organizers for this interesting challenge and congrats everyone who enjoyed it! It was a fun competition though the data is very simple 1D data. I look forward to seeing the various solutions.</p>\n<h1>Overview of my solution</h1>\n<p>My pipeline consists of:</p>\n<ul>\n<li><strong>First Stage</strong><ul>\n<li>Preprocess and Feature engineering</li>\n<li>Event Detection and Sleep/Awake Classification by 1DCNN(U-NET)</li>\n<li>PostProcess of 1D CNN (such as Peak Detection)</li></ul></li>\n<li><strong>Second Stage</strong><ul>\n<li>Use ML instead of the Post-processing that is difficult to adjust</li>\n<li>Rescoring by LGBM that can consider the limitation of 2evets/day</li></ul></li>\n<li><strong>Third Stage</strong><ul>\n<li>Predict as many events as possible just by shifting the step of the base prediction above.</li>\n<li>Scoring the generated predictions by LGBM</li></ul></li>\n<li><strong>Ensemble</strong><ul>\n<li>combination of averaging and WBF ensemble</li></ul></li>\n</ul>\n<p>I think the 2nd stage is the most unique. <br>\nThis model predict the daily accuracy curve of candidates received from 1st stage. </p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2Fe307a878f4ca758835ff897f401a7008%2Foverview.png?generation=1701831890750504&amp;alt=media\" alt=\"\"></p>\n<p>Details are below.</p>\n<h1>1st Stage: Event Detection and Sleep/Awake Classification</h1>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F08d268f6a008646b39b272594226df16%2F1ststage.png?generation=1701832322797003&amp;alt=media\" alt=\"\"><br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F94bdf0ce5785ba775f2a818c769d3a9d%2F1ststagepp.png?generation=1701832338347452&amp;alt=media\" alt=\"\"></p>\n<h1>2nd Stage: Rescoring the confidence considering the limitation of 2 events/day</h1>\n<h2>Concept</h2>\n<p>I applied 2nd stage for the following 2 reasons.</p>\n<h3>1. Less than 2 events/day</h3>\n<p>The first stage NN cannot consider the very important limitation of this competition that the event occurs at most twice in a day. Which do you think is important?</p>\n<blockquote>\n  <p>A) The third candidate in a day with a confidence score of 0.20.<br>\n  B) The first candidate in a day with a confidence score of 0.19.</p>\n</blockquote>\n<p>In many cases, the latter is more important than the former though the score is lower.</p>\n<h3>Long-term features</h3>\n<p>NN is really strong. However, it’s not good at handling meta features and long-term cyclic features. For example,<br>\n“He can be sleeping longer than the usual. This might not actual wakeup.”<br>\n“This onset is relatively unclear compared to the other days.”</p>\n<h2>How to train 2nd stage model</h2>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2Ff6b0889e818ed24add13fd5d74580884%2F2ndstage_r.png?generation=1701861341160280&amp;alt=media\" alt=\"\"></p>\n<h1>3rd Stage: Add as many events as possible</h1>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F049000b0c98a2e5f8bd5f2692a1c2986%2F3rdstage.png?generation=1701832658530598&amp;alt=media\" alt=\"\"></p>\n<h1>Ensemble</h1>\n<ul>\n<li>prepare 2 CNN models</li>\n<li>Averaging 10(5fold x 2seed) predictions at 1st stage for each model</li>\n<li>Run 2nd stage for each model</li>\n<li>WBF-like Ensemble of 2 models</li>\n<li>Run 3rd stage<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F8f6a3cf08dfcf72067db9d7a3db5d849%2FEnsemble.png?generation=1701832240218774&amp;alt=media\" alt=\"\"></li>\n</ul>\n<h1>Ablation Study</h1>\n<p>not yet</p>\n<ul>\n<li>CV Scores<ul>\n<li>1st stage: 0.826 (-0.019)</li>\n<li>add 2nd stage: 0.832 (-0.012)</li>\n<li>add third stage: 0.842 (-0.002)</li>\n<li>add model ensemble(2models): 0.844 (baseline. final submit)</li></ul></li>\n</ul>\n<h3>Inference</h3>\n<p><a href=\"https://www.kaggle.com/code/kmat2019/cmi-sleep-2ndplace\" target=\"_blank\">https://www.kaggle.com/code/kmat2019/cmi-sleep-2ndplace</a></p>\n<h3>Training</h3>\n<p><a href=\"https://www.kaggle.com/code/kmat2019/cmisleep-training-sample-2ndplace-kmat\" target=\"_blank\">https://www.kaggle.com/code/kmat2019/cmisleep-training-sample-2ndplace-kmat</a></p>",
      "rawMarkdown": "Thanks to organizers for this interesting challenge and congrats everyone who enjoyed it! It was a fun competition though the data is very simple 1D data. I look forward to seeing the various solutions.\n\n# Overview of my solution\nMy pipeline consists of:\n- **First Stage**\n  - Preprocess and Feature engineering\n  - Event Detection and Sleep/Awake Classification by 1DCNN(U-NET)\n  - PostProcess of 1D CNN (such as Peak Detection)\n- **Second Stage**\n  - Use ML instead of the Post-processing that is difficult to adjust\n  - Rescoring by LGBM that can consider the limitation of 2evets/day\n- **Third Stage**\n  - Predict as many events as possible just by shifting the step of the base prediction above.\n  - Scoring the generated predictions by LGBM\n- **Ensemble**\n  - combination of averaging and WBF ensemble\n\nI think the 2nd stage is the most unique. \nThis model predict the daily accuracy curve of candidates received from 1st stage. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2Fe307a878f4ca758835ff897f401a7008%2Foverview.png?generation=1701831890750504&alt=media)\n\nDetails are below.\n\n# 1st Stage: Event Detection and Sleep/Awake Classification\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F08d268f6a008646b39b272594226df16%2F1ststage.png?generation=1701832322797003&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F94bdf0ce5785ba775f2a818c769d3a9d%2F1ststagepp.png?generation=1701832338347452&alt=media)\n\n# 2nd Stage: Rescoring the confidence considering the limitation of 2 events/day\n## Concept\nI applied 2nd stage for the following 2 reasons.\n### 1. Less than 2 events/day\nThe first stage NN cannot consider the very important limitation of this competition that the event occurs at most twice in a day. Which do you think is important?\n> A) The third candidate in a day with a confidence score of 0.20.\n> B) The first candidate in a day with a confidence score of 0.19.\n\nIn many cases, the latter is more important than the former though the score is lower.\n\n### Long-term features\nNN is really strong. However, it’s not good at handling meta features and long-term cyclic features. For example,\n“He can be sleeping longer than the usual. This might not actual wakeup.”\n“This onset is relatively unclear compared to the other days.”\n\n## How to train 2nd stage model\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2Ff6b0889e818ed24add13fd5d74580884%2F2ndstage_r.png?generation=1701861341160280&alt=media)\n# 3rd Stage: Add as many events as possible\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F049000b0c98a2e5f8bd5f2692a1c2986%2F3rdstage.png?generation=1701832658530598&alt=media)\n\n# Ensemble\n  - prepare 2 CNN models\n  - Averaging 10(5fold x 2seed) predictions at 1st stage for each model\n  - Run 2nd stage for each model\n  - WBF-like Ensemble of 2 models\n  - Run 3rd stage\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F8f6a3cf08dfcf72067db9d7a3db5d849%2FEnsemble.png?generation=1701832240218774&alt=media)\n\n# Ablation Study\nnot yet\n- CV Scores\n  - 1st stage: 0.826 (-0.019)\n  - add 2nd stage: 0.832 (-0.012)\n  - add third stage: 0.842 (-0.002)\n  - add model ensemble(2models): 0.844 (baseline. final submit)\n\n### Inference\nhttps://www.kaggle.com/code/kmat2019/cmi-sleep-2ndplace\n\n### Training\nhttps://www.kaggle.com/code/kmat2019/cmisleep-training-sample-2ndplace-kmat\n",
      "votes": 177
    },
    {
      "id": 2675411,
      "postDate": "2024-02-29T21:57:22.263Z",
      "content": "<p>Hi K_mat! I really enjoyed reading your solutions, and congratulation for getting the first place in solo. However, I have some confusions regarding your first phase, which is training the 1D CNN U net. </p>\n<ol>\n<li><p>May I ask what this wide range refers to? What is the difference between a wide range target and short range target?</p></li>\n<li><p>Can I ask the specific output of this 1D CNN Unet model, from my understanding, it should be 2 dimensions, one for \"event or not\" and one \"sleep or awake.\"  However, when I look into your source code, I see you set up the output dimension as [seq_length,1].</p></li>\n<li><p>May I ask why you chose to use 20 hours instead of 24 hours?</p></li>\n</ol>\n<p>Thanks for the help in advance. </p>",
      "rawMarkdown": "Hi K_mat! I really enjoyed reading your solutions, and congratulation for getting the first place in solo. However, I have some confusions regarding your first phase, which is training the 1D CNN U net. \n\n1. May I ask what this wide range refers to? What is the difference between a wide range target and short range target?\n\n2. Can I ask the specific output of this 1D CNN Unet model, from my understanding, it should be 2 dimensions, one for \"event or not\" and one \"sleep or awake.\"  However, when I look into your source code, I see you set up the output dimension as [seq_length,1].\n\n3. May I ask why you chose to use 20 hours instead of 24 hours?\n\nThanks for the help in advance. ",
      "votes": 3
    },
    {
      "id": 2550499,
      "postDate": "2023-12-06T04:44:25.380Z",
      "content": "<p>I loved reading your solution, especially  the 2nd stage LightGBM. You dominated the competition from the start to end. Congratulations !!</p>",
      "rawMarkdown": "I loved reading your solution, especially  the 2nd stage LightGBM. You dominated the competition from the start to end. Congratulations !!",
      "votes": 4,
      "replies": [
        {
          "id": 2553349,
          "postDate": "2023-12-08T07:14:39.517Z",
          "content": "<p>Thank you. Congratulations to you too !</p>",
          "rawMarkdown": "Thank you. Congratulations to you too !"
        }
      ]
    },
    {
      "id": 2556721,
      "postDate": "2023-12-11T00:33:51.057Z",
      "content": "<p>Congratulations K_MAT on solo cash gold finish! (And congratulations on your previous solo Kaggle comp top1 and top2 object detection finishes!) Thanks for the beautiful pictures in your writeup.</p>\n<p>Your second stage is very ingenious! I made 30 predictions per night but i put all my top 1 predictions before my top2 before my top 3. Your method of using using the Top-K Accuracy minus the previous Top-K Accuracy to determine the importance of a prediction is great. This lets you decide when a top2 prediction should come before a top1 prediction. Do you have any idea how much this boost CV and LB? (i.e. compared to just putting all top1 before top2 before top3)</p>\n<p>When you say \"Top-K Accuracy\", are you referring to the 10 thresholds? And \"Accuracy = number of true positive thresholds satisfied / 10\"?</p>",
      "rawMarkdown": "Congratulations K_MAT on solo cash gold finish! (And congratulations on your previous solo Kaggle comp top1 and top2 object detection finishes!) Thanks for the beautiful pictures in your writeup.\n\nYour second stage is very ingenious! I made 30 predictions per night but i put all my top 1 predictions before my top2 before my top 3. Your method of using using the Top-K Accuracy minus the previous Top-K Accuracy to determine the importance of a prediction is great. This lets you decide when a top2 prediction should come before a top1 prediction. Do you have any idea how much this boost CV and LB? (i.e. compared to just putting all top1 before top2 before top3)\n\nWhen you say \"Top-K Accuracy\", are you referring to the 10 thresholds? And \"Accuracy = number of true positive thresholds satisfied / 10\"?",
      "votes": 1,
      "replies": [
        {
          "id": 2561495,
          "postDate": "2023-12-14T14:35:02.870Z",
          "content": "<p>Thank you Chris for mentioned the other results as well :) Congratulations on your solo gold too <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a>! </p>\n<blockquote>\n  <p>Do you have any idea how much this boost CV and LB?</p>\n</blockquote>\n<p>I tested your postprocessing, but I couldn't make it work for my model. So I compared it with my former postprocessing which decreases the score of others if the top1 score is enough high. CV score increased by 0.006 with the 2nd stage.</p>\n<blockquote>\n  <p>When you say \"Top-K Accuracy\", are you referring to the 10 thresholds? And \"Accuracy = number of true positive thresholds satisfied / 10\"?</p>\n</blockquote>\n<p>Really clever! I also thought the expected value (10 predictions average) would be the best in theory. Yet it proved that it didn't affects the results so much. Threshold of 0.5 is enough good. Averaging 0.3,0.5,0.7 is slightly better. Averaging 10 predictions is almost same as the 3 predictions average.</p>",
          "rawMarkdown": "Thank you Chris for mentioned the other results as well :) Congratulations on your solo gold too @cdeotte! \n\n> Do you have any idea how much this boost CV and LB?\n\nI tested your postprocessing, but I couldn't make it work for my model. So I compared it with my former postprocessing which decreases the score of others if the top1 score is enough high. CV score increased by 0.006 with the 2nd stage.\n\n> When you say \"Top-K Accuracy\", are you referring to the 10 thresholds? And \"Accuracy = number of true positive thresholds satisfied / 10\"?\n\nReally clever! I also thought the expected value (10 predictions average) would be the best in theory. Yet it proved that it didn't affects the results so much. Threshold of 0.5 is enough good. Averaging 0.3,0.5,0.7 is slightly better. Averaging 10 predictions is almost same as the 3 predictions average.",
          "votes": 1
        }
      ]
    },
    {
      "id": 2555041,
      "postDate": "2023-12-09T16:12:01.783Z",
      "content": "<p>Congratulation for the solo gold medal you have got this time! The second part of your solution is unique indeed, and the other parts are also impressive to me. </p>",
      "rawMarkdown": "Congratulation for the solo gold medal you have got this time! The second part of your solution is unique indeed, and the other parts are also impressive to me. ",
      "votes": 1
    },
    {
      "id": 2552192,
      "postDate": "2023-12-07T09:26:30.640Z",
      "content": "<p>Congratulations on securing the second position in this competition. Appreciate your sharing of the solution. </p>",
      "rawMarkdown": "Congratulations on securing the second position in this competition. Appreciate your sharing of the solution. ",
      "votes": 1
    },
    {
      "id": 2552102,
      "postDate": "2023-12-07T07:37:17.363Z",
      "content": "<p>It is amazing how you did the second stage model (LGB). we tried to do it but we didnt make it work.<br>\nI should test your approach.<br>\nCongratulations! Nice solution</p>",
      "rawMarkdown": "It is amazing how you did the second stage model (LGB). we tried to do it but we didnt make it work.\nI should test your approach.\nCongratulations! Nice solution",
      "votes": 1
    },
    {
      "id": 2551494,
      "postDate": "2023-12-06T18:04:50.260Z",
      "content": "<p>Congratulations for the solution and great explanations</p>",
      "rawMarkdown": "Congratulations for the solution and great explanations",
      "votes": 1
    },
    {
      "id": 2551031,
      "postDate": "2023-12-06T13:21:21.820Z",
      "content": "<p>Great write up! Congrats on your finish. I'm really interested in your 3rd stage. We spent a while trying to figure out post processing tricks to account for the 15 minute pattern in the labels with no success. Can't wait to see how you made that work. Thanks for sharing!</p>",
      "rawMarkdown": "Great write up! Congrats on your finish. I'm really interested in your 3rd stage. We spent a while trying to figure out post processing tricks to account for the 15 minute pattern in the labels with no success. Can't wait to see how you made that work. Thanks for sharing!",
      "votes": 1,
      "replies": [
        {
          "id": 2553357,
          "postDate": "2023-12-08T07:31:49.313Z",
          "content": "<p>I could not account for the 15 minute pattern as explicitly as the 1st place solution. I just added predictions offset from base prediction by 2, 4, 8, 16 minutes, believing that they would cover the small error due to the 15 min pattern.<br>\nIn third stage, I labeled the added prediction positive if it improved the score. As the inputs to the model include the features of score distribution and step values, it might slightly effective to the 15 minutes pattern.</p>",
          "rawMarkdown": "I could not account for the 15 minute pattern as explicitly as the 1st place solution. I just added predictions offset from base prediction by 2, 4, 8, 16 minutes, believing that they would cover the small error due to the 15 min pattern.\nIn third stage, I labeled the added prediction positive if it improved the score. As the inputs to the model include the features of score distribution and step values, it might slightly effective to the 15 minutes pattern."
        }
      ]
    },
    {
      "id": 2550811,
      "postDate": "2023-12-06T09:40:32.617Z",
      "content": "<p>Great work, Love your detailed solution</p>",
      "rawMarkdown": "Great work, Love your detailed solution",
      "votes": 1
    },
    {
      "id": 2550712,
      "postDate": "2023-12-06T08:26:26.187Z",
      "content": "<p>Congrats!! Amazing, well - structured write up!</p>",
      "rawMarkdown": "Congrats!! Amazing, well - structured write up!",
      "votes": 1
    },
    {
      "id": 2550679,
      "postDate": "2023-12-06T07:59:42.830Z",
      "content": "<p>Very interesting, congrats and thank you for sharing :)</p>",
      "rawMarkdown": "Very interesting, congrats and thank you for sharing :)",
      "votes": 1
    },
    {
      "id": 2550649,
      "postDate": "2023-12-06T07:19:26.927Z",
      "content": "<p>Very elegant solution and lots to learn from! congrats <a href=\"https://www.kaggle.com/kmat2019\" target=\"_blank\">@kmat2019</a> </p>",
      "rawMarkdown": "Very elegant solution and lots to learn from! congrats @kmat2019 ",
      "votes": 1
    },
    {
      "id": 2550616,
      "postDate": "2023-12-06T06:35:24.217Z",
      "content": "<p>Really awesome, love how the first stage itself performs great already. Looking forward to details</p>",
      "rawMarkdown": "Really awesome, love how the first stage itself performs great already. Looking forward to details",
      "votes": 1
    },
    {
      "id": 2550613,
      "postDate": "2023-12-06T06:29:02.737Z",
      "content": "<p>Congrats <a href=\"https://www.kaggle.com/kmat2019\" target=\"_blank\">@kmat2019</a>. Keep achieving more. </p>",
      "rawMarkdown": "Congrats @kmat2019. Keep achieving more. ",
      "votes": 1
    },
    {
      "id": 2550558,
      "postDate": "2023-12-06T05:31:21.123Z",
      "content": "<p>😍😍😍😍😍</p>",
      "rawMarkdown": "😍😍😍😍😍",
      "votes": 1
    },
    {
      "id": 2550538,
      "postDate": "2023-12-06T05:16:16.560Z",
      "content": "<p>This solution is awesome!! Thanks for your writing. I got inspired and learned a lot!🤗</p>",
      "rawMarkdown": "This solution is awesome!! Thanks for your writing. I got inspired and learned a lot!🤗",
      "votes": 1
    },
    {
      "id": 2550447,
      "postDate": "2023-12-06T03:27:31.077Z",
      "content": "<p>Wow how amazing work, congratulations!</p>",
      "rawMarkdown": "Wow how amazing work, congratulations!",
      "votes": 1
    },
    {
      "id": 2553127,
      "postDate": "2023-12-08T03:14:50.803Z",
      "content": "<p>Thanks for sharing and congratulations on the 2nd place and great ideas!!!</p>\n<p>A few questions:</p>\n<ul>\n<li>How did you handle labeling errors in the training data? did you use any filtering rules (manual or automatic)?</li>\n<li>What exactly does the positional encoding step represent? Is it a cyclical representation of time of day or of step % (24 * 60 * 12)?</li>\n<li>I understood that your event targets don't discriminate between onset and wakeup; instead postprocessing rules make that determination. Is that right? Does it work better than 5 targets for each event (total of 10 event targets)? Did you apply any weights for the event or state (sleep/awake) targets?</li>\n<li>In L1 postprocessing, do you derive step candidates separately for each event target or do you consider them all in some way to derive each step candidate? </li>\n<li>What criteria did you use to break data into nights for L2? Did you break data at 2pm, 12pm, or some other time of day? Did you use a different criteria for the two types of events (onset / wakeup)?</li>\n<li>In addition to L1 predictions and series aggregated features, what type of features did you  use in L2?</li>\n<li>I understood the target for the L2 model is either 0 or 1 based on whether the TopN accuracy is &gt; 0.5. Is that right?</li>\n<li>How many offsets on L2 predictions did you use in L3? just -2 and +2?</li>\n<li>What target did you use for the L3 model?</li>\n</ul>",
      "rawMarkdown": "Thanks for sharing and congratulations on the 2nd place and great ideas!!!\n\nA few questions:\n- How did you handle labeling errors in the training data? did you use any filtering rules (manual or automatic)?\n- What exactly does the positional encoding step represent? Is it a cyclical representation of time of day or of step % (24 * 60 * 12)?\n- I understood that your event targets don't discriminate between onset and wakeup; instead postprocessing rules make that determination. Is that right? Does it work better than 5 targets for each event (total of 10 event targets)? Did you apply any weights for the event or state (sleep/awake) targets?\n- In L1 postprocessing, do you derive step candidates separately for each event target or do you consider them all in some way to derive each step candidate? \n- What criteria did you use to break data into nights for L2? Did you break data at 2pm, 12pm, or some other time of day? Did you use a different criteria for the two types of events (onset / wakeup)?\n- In addition to L1 predictions and series aggregated features, what type of features did you  use in L2?\n- I understood the target for the L2 model is either 0 or 1 based on whether the TopN accuracy is > 0.5. Is that right?\n- How many offsets on L2 predictions did you use in L3? just -2 and +2?\n- What target did you use for the L3 model?",
      "votes": 2,
      "replies": [
        {
          "id": 2553379,
          "postDate": "2023-12-08T08:07:58.367Z",
          "content": "<p>Thank you for the questions but this is not \"few\" questions for me. 😂</p>\n<ul>\n<li><p>How did you handle labeling errors?<br>\nAre you talking about nan in ground truth? I added the binary mask  to the inputs of 1st stage. If the part is repeated in the other day, the value is 1 else 0.</p></li>\n<li><p>Is it a cyclical representation of time of day or of step % (24 * 60 * 12)?<br>\nExactly.</p></li>\n<li><p>Does it work better than 5 targets for each event?<br>\nI've not tested 10 targets. But I prefer the current model, as the sleep/awake mask is easier to learn than the sparse event label. Learning the former as sub task can help learn the latter.</p></li>\n<li><p>apply any weights for the event or state (sleep/awake) targets?<br>\nYes. Loss weight of the 5 event predictions are a little different. (but it did not affect the score so much)</p></li>\n<li><p>L1 postprocessing, do you derive step candidates separately for each event target or do you consider them all in some way to derive each step candidate?<br>\nI can't understand your question. I applied the peak-detection to all predictions to find the candidates, then separated them into onset/wakeup.</p></li>\n<li><p>What criteria did you use to break data into nights for L2? Did you break data at 2pm, 12pm, or some other time of day? Did you use a different criteria for the two types of events (onset / wakeup)?<br>\napprox. 2pm.</p></li>\n<li><p>In addition to L1 predictions and series aggregated features, what type of features did you use in L2?<br>\nAnd the inputs to first stage. Sum of scores over itself is the most important feature.</p></li>\n<li><p>I understood the target for the L2 model is either 0 or 1 based on whether the TopN accuracy is &gt; 0.5. Is that right?<br>\nRight. In fact, I tried other cases of 0.3 or 0.7 and the final submission was the average of them.</p></li>\n<li><p>How many offsets on L2 predictions did you use in L3? just -2 and +2?<br>\n-16, -8, -4, -2, 2, 4, 8, 16 minutes  </p></li>\n<li><p>What target did you use for the L3 model?<br>\nIn third stage, I labeled the added prediction positive if it improved the score. </p></li>\n</ul>",
          "rawMarkdown": "Thank you for the questions but this is not \"few\" questions for me. 😂\n\n- How did you handle labeling errors?\nAre you talking about nan in ground truth? I added the binary mask  to the inputs of 1st stage. If the part is repeated in the other day, the value is 1 else 0.\n\n- Is it a cyclical representation of time of day or of step % (24 * 60 * 12)?\nExactly.\n\n- Does it work better than 5 targets for each event?\nI've not tested 10 targets. But I prefer the current model, as the sleep/awake mask is easier to learn than the sparse event label. Learning the former as sub task can help learn the latter.\n\n- apply any weights for the event or state (sleep/awake) targets?\nYes. Loss weight of the 5 event predictions are a little different. (but it did not affect the score so much)\n\n- L1 postprocessing, do you derive step candidates separately for each event target or do you consider them all in some way to derive each step candidate?\nI can't understand your question. I applied the peak-detection to all predictions to find the candidates, then separated them into onset/wakeup.\n\n- What criteria did you use to break data into nights for L2? Did you break data at 2pm, 12pm, or some other time of day? Did you use a different criteria for the two types of events (onset / wakeup)?\napprox. 2pm.\n\n- In addition to L1 predictions and series aggregated features, what type of features did you use in L2?\nAnd the inputs to first stage. Sum of scores over itself is the most important feature.\n\n- I understood the target for the L2 model is either 0 or 1 based on whether the TopN accuracy is > 0.5. Is that right?\nRight. In fact, I tried other cases of 0.3 or 0.7 and the final submission was the average of them.\n\n- How many offsets on L2 predictions did you use in L3? just -2 and +2?\n-16, -8, -4, -2, 2, 4, 8, 16 minutes  \n\n- What target did you use for the L3 model?\nIn third stage, I labeled the added prediction positive if it improved the score. ",
          "votes": 6,
          "replies": [
            {
              "id": 2554078,
              "postDate": "2023-12-08T19:04:23.433Z",
              "content": "<p>Thanks for taking the time to answer my \"few\" questions 😁</p>\n<p>I failed to explain myself in my first question. What I meant to ask was how you had handled the errors in the labels of the training data. In many cases the label is in the wrong place (e.g., onset followed by a long period of activity or wakeup in the middle of a long period of inactivity) or is missing (e.g., days with no events when the device was clearly being used and the child was sleeping). Did you consider all events in your training or did you exclude nights with incorrect labels?</p>",
              "rawMarkdown": "Thanks for taking the time to answer my \"few\" questions 😁\n\nI failed to explain myself in my first question. What I meant to ask was how you had handled the errors in the labels of the training data. In many cases the label is in the wrong place (e.g., onset followed by a long period of activity or wakeup in the middle of a long period of inactivity) or is missing (e.g., days with no events when the device was clearly being used and the child was sleeping). Did you consider all events in your training or did you exclude nights with incorrect labels?",
              "votes": 2
            },
            {
              "id": 2561499,
              "postDate": "2023-12-14T14:41:19.193Z",
              "content": "<p>I used all events as removing them didn't affect the score so much. And I finally add the 60 minutes and 1 day offset predictions after 3rd stage with very low score to cope with the annotation errors though the increase in cv score is very small.</p>",
              "rawMarkdown": "I used all events as removing them didn't affect the score so much. And I finally add the 60 minutes and 1 day offset predictions after 3rd stage with very low score to cope with the annotation errors though the increase in cv score is very small."
            }
          ]
        }
      ]
    },
    {
      "id": 2550475,
      "postDate": "2023-12-06T03:58:08.547Z",
      "content": "<p>Great writeup, excited to look through this. I tried multi stage after my CNN so I am curious to see how you got it to work!</p>",
      "rawMarkdown": "Great writeup, excited to look through this. I tried multi stage after my CNN so I am curious to see how you got it to work!",
      "votes": 2
    },
    {
      "id": 2556228,
      "postDate": "2023-12-10T14:52:42.397Z",
      "content": "<p>Thanks for sharing and Congratulations 🎉</p>",
      "rawMarkdown": "Thanks for sharing and Congratulations 🎉"
    },
    {
      "id": 2577005,
      "postDate": "2023-12-28T09:39:28.903Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2675411,
      "author_name": "JiachengCui123",
      "author_url": "",
      "post_date": "2024-02-29T21:57:22.263000",
      "content": "<p>Hi K_mat! I really enjoyed reading your solutions, and congratulation for getting the first place in solo. However, I have some confusions regarding your first phase, which is training the 1D CNN U net. </p>\n<ol>\n<li><p>May I ask what this wide range refers to? What is the difference between a wide range target and short range target?</p></li>\n<li><p>Can I ask the specific output of this 1D CNN Unet model, from my understanding, it should be 2 dimensions, one for \"event or not\" and one \"sleep or awake.\"  However, when I look into your source code, I see you set up the output dimension as [seq_length,1].</p></li>\n<li><p>May I ask why you chose to use 20 hours instead of 24 hours?</p></li>\n</ol>\n<p>Thanks for the help in advance. </p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 2550499,
      "author_name": "NikhilMishra",
      "author_url": "",
      "post_date": "2023-12-06T04:44:25.380000",
      "content": "<p>I loved reading your solution, especially  the 2nd stage LightGBM. You dominated the competition from the start to end. Congratulations !!</p>",
      "votes": 4,
      "replies": [
        {
          "id": 2553349,
          "author_name": "K_mat",
          "author_url": "",
          "post_date": "2023-12-08T07:14:39.517000",
          "content": "<p>Thank you. Congratulations to you too !</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2556721,
      "author_name": "Chris Deotte",
      "author_url": "",
      "post_date": "2023-12-11T00:33:51.057000",
      "content": "<p>Congratulations K_MAT on solo cash gold finish! (And congratulations on your previous solo Kaggle comp top1 and top2 object detection finishes!) Thanks for the beautiful pictures in your writeup.</p>\n<p>Your second stage is very ingenious! I made 30 predictions per night but i put all my top 1 predictions before my top2 before my top 3. Your method of using using the Top-K Accuracy minus the previous Top-K Accuracy to determine the importance of a prediction is great. This lets you decide when a top2 prediction should come before a top1 prediction. Do you have any idea how much this boost CV and LB? (i.e. compared to just putting all top1 before top2 before top3)</p>\n<p>When you say \"Top-K Accuracy\", are you referring to the 10 thresholds? And \"Accuracy = number of true positive thresholds satisfied / 10\"?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2561495,
          "author_name": "K_mat",
          "author_url": "",
          "post_date": "2023-12-14T14:35:02.870000",
          "content": "<p>Thank you Chris for mentioned the other results as well :) Congratulations on your solo gold too <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a>! </p>\n<blockquote>\n  <p>Do you have any idea how much this boost CV and LB?</p>\n</blockquote>\n<p>I tested your postprocessing, but I couldn't make it work for my model. So I compared it with my former postprocessing which decreases the score of others if the top1 score is enough high. CV score increased by 0.006 with the 2nd stage.</p>\n<blockquote>\n  <p>When you say \"Top-K Accuracy\", are you referring to the 10 thresholds? And \"Accuracy = number of true positive thresholds satisfied / 10\"?</p>\n</blockquote>\n<p>Really clever! I also thought the expected value (10 predictions average) would be the best in theory. Yet it proved that it didn't affects the results so much. Threshold of 0.5 is enough good. Averaging 0.3,0.5,0.7 is slightly better. Averaging 10 predictions is almost same as the 3 predictions average.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 2555041,
      "author_name": "Co_yamax",
      "author_url": "",
      "post_date": "2023-12-09T16:12:01.783000",
      "content": "<p>Congratulation for the solo gold medal you have got this time! The second part of your solution is unique indeed, and the other parts are also impressive to me. </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2552192,
      "author_name": "C R Suthikshn Kumar",
      "author_url": "",
      "post_date": "2023-12-07T09:26:30.640000",
      "content": "<p>Congratulations on securing the second position in this competition. Appreciate your sharing of the solution. </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2552102,
      "author_name": "Fnoa",
      "author_url": "",
      "post_date": "2023-12-07T07:37:17.363000",
      "content": "<p>It is amazing how you did the second stage model (LGB). we tried to do it but we didnt make it work.<br>\nI should test your approach.<br>\nCongratulations! Nice solution</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2551494,
      "author_name": "Ozkan",
      "author_url": "",
      "post_date": "2023-12-06T18:04:50.260000",
      "content": "<p>Congratulations for the solution and great explanations</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2551031,
      "author_name": "Rob Mulla",
      "author_url": "",
      "post_date": "2023-12-06T13:21:21.820000",
      "content": "<p>Great write up! Congrats on your finish. I'm really interested in your 3rd stage. We spent a while trying to figure out post processing tricks to account for the 15 minute pattern in the labels with no success. Can't wait to see how you made that work. Thanks for sharing!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2553357,
          "author_name": "K_mat",
          "author_url": "",
          "post_date": "2023-12-08T07:31:49.313000",
          "content": "<p>I could not account for the 15 minute pattern as explicitly as the 1st place solution. I just added predictions offset from base prediction by 2, 4, 8, 16 minutes, believing that they would cover the small error due to the 15 min pattern.<br>\nIn third stage, I labeled the added prediction positive if it improved the score. As the inputs to the model include the features of score distribution and step values, it might slightly effective to the 15 minutes pattern.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2550811,
      "author_name": "Amir Mohammadikarbalaei",
      "author_url": "",
      "post_date": "2023-12-06T09:40:32.617000",
      "content": "<p>Great work, Love your detailed solution</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2550712,
      "author_name": "narsil (jobs-in-data.com)",
      "author_url": "",
      "post_date": "2023-12-06T08:26:26.187000",
      "content": "<p>Congrats!! Amazing, well - structured write up!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2550679,
      "author_name": "FabioDevIsTyping",
      "author_url": "",
      "post_date": "2023-12-06T07:59:42.830000",
      "content": "<p>Very interesting, congrats and thank you for sharing :)</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2550649,
      "author_name": "Catharis",
      "author_url": "",
      "post_date": "2023-12-06T07:19:26.927000",
      "content": "<p>Very elegant solution and lots to learn from! congrats <a href=\"https://www.kaggle.com/kmat2019\" target=\"_blank\">@kmat2019</a> </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2550616,
      "author_name": "Yijie Xu",
      "author_url": "",
      "post_date": "2023-12-06T06:35:24.217000",
      "content": "<p>Really awesome, love how the first stage itself performs great already. Looking forward to details</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2550613,
      "author_name": "Indra Sonowal",
      "author_url": "",
      "post_date": "2023-12-06T06:29:02.737000",
      "content": "<p>Congrats <a href=\"https://www.kaggle.com/kmat2019\" target=\"_blank\">@kmat2019</a>. Keep achieving more. </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2550558,
      "author_name": "不眠症🍳",
      "author_url": "",
      "post_date": "2023-12-06T05:31:21.123000",
      "content": "<p>😍😍😍😍😍</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2550538,
      "author_name": "Cotmd",
      "author_url": "",
      "post_date": "2023-12-06T05:16:16.560000",
      "content": "<p>This solution is awesome!! Thanks for your writing. I got inspired and learned a lot!🤗</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2550447,
      "author_name": "kerry sun",
      "author_url": "",
      "post_date": "2023-12-06T03:27:31.077000",
      "content": "<p>Wow how amazing work, congratulations!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2553127,
      "author_name": "vialactea",
      "author_url": "",
      "post_date": "2023-12-08T03:14:50.803000",
      "content": "<p>Thanks for sharing and congratulations on the 2nd place and great ideas!!!</p>\n<p>A few questions:</p>\n<ul>\n<li>How did you handle labeling errors in the training data? did you use any filtering rules (manual or automatic)?</li>\n<li>What exactly does the positional encoding step represent? Is it a cyclical representation of time of day or of step % (24 * 60 * 12)?</li>\n<li>I understood that your event targets don't discriminate between onset and wakeup; instead postprocessing rules make that determination. Is that right? Does it work better than 5 targets for each event (total of 10 event targets)? Did you apply any weights for the event or state (sleep/awake) targets?</li>\n<li>In L1 postprocessing, do you derive step candidates separately for each event target or do you consider them all in some way to derive each step candidate? </li>\n<li>What criteria did you use to break data into nights for L2? Did you break data at 2pm, 12pm, or some other time of day? Did you use a different criteria for the two types of events (onset / wakeup)?</li>\n<li>In addition to L1 predictions and series aggregated features, what type of features did you  use in L2?</li>\n<li>I understood the target for the L2 model is either 0 or 1 based on whether the TopN accuracy is &gt; 0.5. Is that right?</li>\n<li>How many offsets on L2 predictions did you use in L3? just -2 and +2?</li>\n<li>What target did you use for the L3 model?</li>\n</ul>",
      "votes": 2,
      "replies": [
        {
          "id": 2553379,
          "author_name": "K_mat",
          "author_url": "",
          "post_date": "2023-12-08T08:07:58.367000",
          "content": "<p>Thank you for the questions but this is not \"few\" questions for me. 😂</p>\n<ul>\n<li><p>How did you handle labeling errors?<br>\nAre you talking about nan in ground truth? I added the binary mask  to the inputs of 1st stage. If the part is repeated in the other day, the value is 1 else 0.</p></li>\n<li><p>Is it a cyclical representation of time of day or of step % (24 * 60 * 12)?<br>\nExactly.</p></li>\n<li><p>Does it work better than 5 targets for each event?<br>\nI've not tested 10 targets. But I prefer the current model, as the sleep/awake mask is easier to learn than the sparse event label. Learning the former as sub task can help learn the latter.</p></li>\n<li><p>apply any weights for the event or state (sleep/awake) targets?<br>\nYes. Loss weight of the 5 event predictions are a little different. (but it did not affect the score so much)</p></li>\n<li><p>L1 postprocessing, do you derive step candidates separately for each event target or do you consider them all in some way to derive each step candidate?<br>\nI can't understand your question. I applied the peak-detection to all predictions to find the candidates, then separated them into onset/wakeup.</p></li>\n<li><p>What criteria did you use to break data into nights for L2? Did you break data at 2pm, 12pm, or some other time of day? Did you use a different criteria for the two types of events (onset / wakeup)?<br>\napprox. 2pm.</p></li>\n<li><p>In addition to L1 predictions and series aggregated features, what type of features did you use in L2?<br>\nAnd the inputs to first stage. Sum of scores over itself is the most important feature.</p></li>\n<li><p>I understood the target for the L2 model is either 0 or 1 based on whether the TopN accuracy is &gt; 0.5. Is that right?<br>\nRight. In fact, I tried other cases of 0.3 or 0.7 and the final submission was the average of them.</p></li>\n<li><p>How many offsets on L2 predictions did you use in L3? just -2 and +2?<br>\n-16, -8, -4, -2, 2, 4, 8, 16 minutes  </p></li>\n<li><p>What target did you use for the L3 model?<br>\nIn third stage, I labeled the added prediction positive if it improved the score. </p></li>\n</ul>",
          "votes": 6,
          "replies": [
            {
              "id": 2554078,
              "author_name": "vialactea",
              "author_url": "",
              "post_date": "2023-12-08T19:04:23.433000",
              "content": "<p>Thanks for taking the time to answer my \"few\" questions 😁</p>\n<p>I failed to explain myself in my first question. What I meant to ask was how you had handled the errors in the labels of the training data. In many cases the label is in the wrong place (e.g., onset followed by a long period of activity or wakeup in the middle of a long period of inactivity) or is missing (e.g., days with no events when the device was clearly being used and the child was sleeping). Did you consider all events in your training or did you exclude nights with incorrect labels?</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 2561499,
              "author_name": "K_mat",
              "author_url": "",
              "post_date": "2023-12-14T14:41:19.193000",
              "content": "<p>I used all events as removing them didn't affect the score so much. And I finally add the 60 minutes and 1 day offset predictions after 3rd stage with very low score to cope with the annotation errors though the increase in cv score is very small.</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2550475,
      "author_name": "Cody_Null",
      "author_url": "",
      "post_date": "2023-12-06T03:58:08.547000",
      "content": "<p>Great writeup, excited to look through this. I tried multi stage after my CNN so I am curious to see how you got it to work!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2556228,
      "author_name": "Ankita Nain",
      "author_url": "",
      "post_date": "2023-12-10T14:52:42.397000",
      "content": "<p>Thanks for sharing and Congratulations 🎉</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2577005,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-12-28T09:39:28.903000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2550443": "Thanks to organizers for this interesting challenge and congrats everyone who enjoyed it! It was a fun competition though the data is very simple 1D data. I look forward to seeing the various solutions.\n\n# Overview of my solution\nMy pipeline consists of:\n- **First Stage**\n  - Preprocess and Feature engineering\n  - Event Detection and Sleep/Awake Classification by 1DCNN(U-NET)\n  - PostProcess of 1D CNN (such as Peak Detection)\n- **Second Stage**\n  - Use ML instead of the Post-processing that is difficult to adjust\n  - Rescoring by LGBM that can consider the limitation of 2evets/day\n- **Third Stage**\n  - Predict as many events as possible just by shifting the step of the base prediction above.\n  - Scoring the generated predictions by LGBM\n- **Ensemble**\n  - combination of averaging and WBF ensemble\n\nI think the 2nd stage is the most unique. \nThis model predict the daily accuracy curve of candidates received from 1st stage. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2Fe307a878f4ca758835ff897f401a7008%2Foverview.png?generation=1701831890750504&alt=media)\n\nDetails are below.\n\n# 1st Stage: Event Detection and Sleep/Awake Classification\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F08d268f6a008646b39b272594226df16%2F1ststage.png?generation=1701832322797003&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F94bdf0ce5785ba775f2a818c769d3a9d%2F1ststagepp.png?generation=1701832338347452&alt=media)\n\n# 2nd Stage: Rescoring the confidence considering the limitation of 2 events/day\n## Concept\nI applied 2nd stage for the following 2 reasons.\n### 1. Less than 2 events/day\nThe first stage NN cannot consider the very important limitation of this competition that the event occurs at most twice in a day. Which do you think is important?\n> A) The third candidate in a day with a confidence score of 0.20.\n> B) The first candidate in a day with a confidence score of 0.19.\n\nIn many cases, the latter is more important than the former though the score is lower.\n\n### Long-term features\nNN is really strong. However, it’s not good at handling meta features and long-term cyclic features. For example,\n“He can be sleeping longer than the usual. This might not actual wakeup.”\n“This onset is relatively unclear compared to the other days.”\n\n## How to train 2nd stage model\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2Ff6b0889e818ed24add13fd5d74580884%2F2ndstage_r.png?generation=1701861341160280&alt=media)\n# 3rd Stage: Add as many events as possible\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F049000b0c98a2e5f8bd5f2692a1c2986%2F3rdstage.png?generation=1701832658530598&alt=media)\n\n# Ensemble\n  - prepare 2 CNN models\n  - Averaging 10(5fold x 2seed) predictions at 1st stage for each model\n  - Run 2nd stage for each model\n  - WBF-like Ensemble of 2 models\n  - Run 3rd stage\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2938236%2F8f6a3cf08dfcf72067db9d7a3db5d849%2FEnsemble.png?generation=1701832240218774&alt=media)\n\n# Ablation Study\nnot yet\n- CV Scores\n  - 1st stage: 0.826 (-0.019)\n  - add 2nd stage: 0.832 (-0.012)\n  - add third stage: 0.842 (-0.002)\n  - add model ensemble(2models): 0.844 (baseline. final submit)\n\n### Inference\nhttps://www.kaggle.com/code/kmat2019/cmi-sleep-2ndplace\n\n### Training\nhttps://www.kaggle.com/code/kmat2019/cmisleep-training-sample-2ndplace-kmat\n",
    "2675411": "Hi K_mat! I really enjoyed reading your solutions, and congratulation for getting the first place in solo. However, I have some confusions regarding your first phase, which is training the 1D CNN U net. \n\n1. May I ask what this wide range refers to? What is the difference between a wide range target and short range target?\n\n2. Can I ask the specific output of this 1D CNN Unet model, from my understanding, it should be 2 dimensions, one for \"event or not\" and one \"sleep or awake.\"  However, when I look into your source code, I see you set up the output dimension as [seq_length,1].\n\n3. May I ask why you chose to use 20 hours instead of 24 hours?\n\nThanks for the help in advance. ",
    "2550499": "I loved reading your solution, especially  the 2nd stage LightGBM. You dominated the competition from the start to end. Congratulations !!",
    "2556721": "Congratulations K_MAT on solo cash gold finish! (And congratulations on your previous solo Kaggle comp top1 and top2 object detection finishes!) Thanks for the beautiful pictures in your writeup.\n\nYour second stage is very ingenious! I made 30 predictions per night but i put all my top 1 predictions before my top2 before my top 3. Your method of using using the Top-K Accuracy minus the previous Top-K Accuracy to determine the importance of a prediction is great. This lets you decide when a top2 prediction should come before a top1 prediction. Do you have any idea how much this boost CV and LB? (i.e. compared to just putting all top1 before top2 before top3)\n\nWhen you say \"Top-K Accuracy\", are you referring to the 10 thresholds? And \"Accuracy = number of true positive thresholds satisfied / 10\"?",
    "2555041": "Congratulation for the solo gold medal you have got this time! The second part of your solution is unique indeed, and the other parts are also impressive to me. ",
    "2552192": "Congratulations on securing the second position in this competition. Appreciate your sharing of the solution. ",
    "2552102": "It is amazing how you did the second stage model (LGB). we tried to do it but we didnt make it work.\nI should test your approach.\nCongratulations! Nice solution",
    "2551494": "Congratulations for the solution and great explanations",
    "2551031": "Great write up! Congrats on your finish. I'm really interested in your 3rd stage. We spent a while trying to figure out post processing tricks to account for the 15 minute pattern in the labels with no success. Can't wait to see how you made that work. Thanks for sharing!",
    "2550811": "Great work, Love your detailed solution",
    "2550712": "Congrats!! Amazing, well - structured write up!",
    "2550679": "Very interesting, congrats and thank you for sharing :)",
    "2550649": "Very elegant solution and lots to learn from! congrats @kmat2019 ",
    "2550616": "Really awesome, love how the first stage itself performs great already. Looking forward to details",
    "2550613": "Congrats @kmat2019. Keep achieving more. ",
    "2550558": "😍😍😍😍😍",
    "2550538": "This solution is awesome!! Thanks for your writing. I got inspired and learned a lot!🤗",
    "2550447": "Wow how amazing work, congratulations!",
    "2553127": "Thanks for sharing and congratulations on the 2nd place and great ideas!!!\n\nA few questions:\n- How did you handle labeling errors in the training data? did you use any filtering rules (manual or automatic)?\n- What exactly does the positional encoding step represent? Is it a cyclical representation of time of day or of step % (24 * 60 * 12)?\n- I understood that your event targets don't discriminate between onset and wakeup; instead postprocessing rules make that determination. Is that right? Does it work better than 5 targets for each event (total of 10 event targets)? Did you apply any weights for the event or state (sleep/awake) targets?\n- In L1 postprocessing, do you derive step candidates separately for each event target or do you consider them all in some way to derive each step candidate? \n- What criteria did you use to break data into nights for L2? Did you break data at 2pm, 12pm, or some other time of day? Did you use a different criteria for the two types of events (onset / wakeup)?\n- In addition to L1 predictions and series aggregated features, what type of features did you  use in L2?\n- I understood the target for the L2 model is either 0 or 1 based on whether the TopN accuracy is > 0.5. Is that right?\n- How many offsets on L2 predictions did you use in L3? just -2 and +2?\n- What target did you use for the L3 model?",
    "2550475": "Great writeup, excited to look through this. I tried multi stage after my CNN so I am curious to see how you got it to work!",
    "2556228": "Thanks for sharing and Congratulations 🎉",
    "2577005": ""
  },
  "complete": true,
  "source": "existing_discussion",
  "competition_id": 53666,
  "competition_slug": "child-mind-institute-detect-sleep-states",
  "team_id": 11008610,
  "solution_url": "/competitions/child-mind-institute-detect-sleep-states/writeups/k-mat-2nd-place-solution"
}