{
  "id": 459769,
  "title": "108th place brief solution",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/459769",
  "author_name": "Cotmd",
  "post_date": "2023-12-06T15:31:05.572000",
  "votes": 4,
  "comment_count": 0,
  "views": 0,
  "content": "<p>First of all, thanks for opening this competition and want to say everyone who competed with passion has done a good job. I've got inspired by lots of cool solutions and made more desire to be a stronger kaggler. </p>\n<h3>Solution</h3>\n<p>Here let me introduce my solution shortly, </p>\n<p>My submissions are all based on the model of the super high quality <a href=\"https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states\" target=\"_blank\">repository</a> of <a href=\"https://www.kaggle.com/tubotubo\" target=\"_blank\">@tubotubo</a> . Special thanks.</p>\n<ul>\n<li>Public Notebook : <a href=\"https://www.kaggle.com/code/tubotubo/cmi-submit\" target=\"_blank\">https://www.kaggle.com/code/tubotubo/cmi-submit</a></li>\n<li>Shared approach : <a href=\"https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940\" target=\"_blank\">https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940</a></li>\n</ul>\n<p>I made an ensemble of 3 types of models and used seeds that made a relative good CV and LB at the same time.</p>\n<ol>\n<li>CNN Spectrogram Feature Extractor, U-Net 1D Decoder </li>\n<li>CNN Spectrogram Feature Extractor, LSTM Decoder</li>\n<li>PaNNs Feature Extractor, LSTM Decoder</li>\n</ol>\n<p>The main features used are:</p>\n<ul>\n<li>anglez</li>\n<li>enmo</li>\n<li>hour_cos</li>\n<li>hour_sin</li>\n<li>angelz_diff (not normalized)</li>\n<li>enmo_diffx10 (not normalized)</li>\n</ul>\n<p>Also tried some different types of the same features, but not that much and I don't think it is a meaningful. (For example, enmo_diffx25 or anglez_diff normalized)<br>\nWith different seeds and hyperparameters, my submission is an ensemble of 20+ models. <br>\nI used the models that only made the CV score 0.753 or over.</p>\n<h3>What worked</h3>\n<ul>\n<li>sigma=12</li>\n<li>anglez_diff</li>\n<li>enmo_diff</li>\n</ul>\n<h3>What didn't work</h3>\n<ul>\n<li>Transformer Decoder</li>\n<li>Normalizing enmo_diff (Probably the average value is so small, it gets sensitive to even a small change)</li>\n</ul>\n<p>My final score is Public 0.751, Private 0.775</p>\n<p>I hope I could be a better kaggler at the next competition. Good luck guys 😀<br>\nThanks for reading. </p>",
  "messages": [
    {
      "id": 2551249,
      "postDate": "2023-12-06T15:31:05.573Z",
      "content": "<p>First of all, thanks for opening this competition and want to say everyone who competed with passion has done a good job. I've got inspired by lots of cool solutions and made more desire to be a stronger kaggler. </p>\n<h3>Solution</h3>\n<p>Here let me introduce my solution shortly, </p>\n<p>My submissions are all based on the model of the super high quality <a href=\"https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states\" target=\"_blank\">repository</a> of <a href=\"https://www.kaggle.com/tubotubo\" target=\"_blank\">@tubotubo</a> . Special thanks.</p>\n<ul>\n<li>Public Notebook : <a href=\"https://www.kaggle.com/code/tubotubo/cmi-submit\" target=\"_blank\">https://www.kaggle.com/code/tubotubo/cmi-submit</a></li>\n<li>Shared approach : <a href=\"https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940\" target=\"_blank\">https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940</a></li>\n</ul>\n<p>I made an ensemble of 3 types of models and used seeds that made a relative good CV and LB at the same time.</p>\n<ol>\n<li>CNN Spectrogram Feature Extractor, U-Net 1D Decoder </li>\n<li>CNN Spectrogram Feature Extractor, LSTM Decoder</li>\n<li>PaNNs Feature Extractor, LSTM Decoder</li>\n</ol>\n<p>The main features used are:</p>\n<ul>\n<li>anglez</li>\n<li>enmo</li>\n<li>hour_cos</li>\n<li>hour_sin</li>\n<li>angelz_diff (not normalized)</li>\n<li>enmo_diffx10 (not normalized)</li>\n</ul>\n<p>Also tried some different types of the same features, but not that much and I don't think it is a meaningful. (For example, enmo_diffx25 or anglez_diff normalized)<br>\nWith different seeds and hyperparameters, my submission is an ensemble of 20+ models. <br>\nI used the models that only made the CV score 0.753 or over.</p>\n<h3>What worked</h3>\n<ul>\n<li>sigma=12</li>\n<li>anglez_diff</li>\n<li>enmo_diff</li>\n</ul>\n<h3>What didn't work</h3>\n<ul>\n<li>Transformer Decoder</li>\n<li>Normalizing enmo_diff (Probably the average value is so small, it gets sensitive to even a small change)</li>\n</ul>\n<p>My final score is Public 0.751, Private 0.775</p>\n<p>I hope I could be a better kaggler at the next competition. Good luck guys 😀<br>\nThanks for reading. </p>",
      "rawMarkdown": "First of all, thanks for opening this competition and want to say everyone who competed with passion has done a good job. I've got inspired by lots of cool solutions and made more desire to be a stronger kaggler. \n\n### Solution\nHere let me introduce my solution shortly, \n\nMy submissions are all based on the model of the super high quality [repository](https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states) of @tubotubo . Special thanks.\n- Public Notebook : https://www.kaggle.com/code/tubotubo/cmi-submit\n- Shared approach : https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940\n\nI made an ensemble of 3 types of models and used seeds that made a relative good CV and LB at the same time.\n\n1. CNN Spectrogram Feature Extractor, U-Net 1D Decoder \n2. CNN Spectrogram Feature Extractor, LSTM Decoder\n3. PaNNs Feature Extractor, LSTM Decoder\n\nThe main features used are:\n- anglez\n- enmo\n- hour_cos\n- hour_sin\n- angelz_diff (not normalized)\n- enmo_diffx10 (not normalized)\n\n\n\nAlso tried some different types of the same features, but not that much and I don't think it is a meaningful. (For example, enmo_diffx25 or anglez_diff normalized)\nWith different seeds and hyperparameters, my submission is an ensemble of 20+ models. \nI used the models that only made the CV score 0.753 or over.\n\n### What worked\n- sigma=12\n- anglez_diff\n- enmo_diff\n\n\n### What didn't work\n- Transformer Decoder\n- Normalizing enmo_diff (Probably the average value is so small, it gets sensitive to even a small change)\n\nMy final score is Public 0.751, Private 0.775\n\nI hope I could be a better kaggler at the next competition. Good luck guys 😀\nThanks for reading. ",
      "votes": 4
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2551249": "First of all, thanks for opening this competition and want to say everyone who competed with passion has done a good job. I've got inspired by lots of cool solutions and made more desire to be a stronger kaggler. \n\n### Solution\nHere let me introduce my solution shortly, \n\nMy submissions are all based on the model of the super high quality [repository](https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states) of @tubotubo . Special thanks.\n- Public Notebook : https://www.kaggle.com/code/tubotubo/cmi-submit\n- Shared approach : https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940\n\nI made an ensemble of 3 types of models and used seeds that made a relative good CV and LB at the same time.\n\n1. CNN Spectrogram Feature Extractor, U-Net 1D Decoder \n2. CNN Spectrogram Feature Extractor, LSTM Decoder\n3. PaNNs Feature Extractor, LSTM Decoder\n\nThe main features used are:\n- anglez\n- enmo\n- hour_cos\n- hour_sin\n- angelz_diff (not normalized)\n- enmo_diffx10 (not normalized)\n\n\n\nAlso tried some different types of the same features, but not that much and I don't think it is a meaningful. (For example, enmo_diffx25 or anglez_diff normalized)\nWith different seeds and hyperparameters, my submission is an ensemble of 20+ models. \nI used the models that only made the CV score 0.753 or over.\n\n### What worked\n- sigma=12\n- anglez_diff\n- enmo_diff\n\n\n### What didn't work\n- Transformer Decoder\n- Normalizing enmo_diff (Probably the average value is so small, it gets sensitive to even a small change)\n\nMy final score is Public 0.751, Private 0.775\n\nI hope I could be a better kaggler at the next competition. Good luck guys 😀\nThanks for reading. "
  }
}