{
  "id": 460177,
  "title": "15th Place Solution",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/460177",
  "author_name": "kaerururu",
  "post_date": "2023-12-08T05:28:57.310000",
  "votes": 24,
  "comment_count": 1,
  "views": 0,
  "content": "<p>First of all, I'd like to thank kaggle team, host for hosting this competition and thank many other participants to share codes, ideas and datasets. Special thanks to my teammates (<a href=\"https://www.kaggle.com/kunihikofurugori\" target=\"_blank\">furu</a>, <a href=\"https://www.kaggle.com/yukio0201\" target=\"_blank\">mizoo</a>, <a href=\"https://www.kaggle.com/yamsam\" target=\"_blank\">isamu</a>, <a href=\"https://www.kaggle.com/tereka\" target=\"_blank\">tereka</a>) who worked together on the competition and <a href=\"https://www.kaggle.com/tubotubo\" target=\"_blank\">213tubo</a> for sharing such a great <a href=\"https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states\" target=\"_blank\">pipeline</a>.</p>\n<h3>Overview</h3>\n<p>Our final submission is ensemble of 55 models in total, including eleven 5fold models.</p>\n<p>Our solution has four main parts. <br>\nThey are furu, mizoo, kaerururu part, isamu part, tereka part, and postprocess part.</p>\n<h3>furu, mizoo, kaerururu part</h3>\n<p>Our pipeline is based on 213tubo's one.<br>\nThe architecture is various Feature Extractor -&gt; Unet Encoder -&gt; Unet1d Decoder.<br>\nOnset cv best model and wakeup cv best model are <code>saved separately</code>.</p>\n<ul>\n<li><p>Feature Engineering</p>\n<ul>\n<li>normalize anglez, enmo with mean, std</li>\n<li>anglez, enmo, hour feature (sin, cos)</li></ul></li>\n<li><p>Feature Extractor</p>\n<ul>\n<li>LSTMFeatureExtractor</li>\n<li>CNNSpectrogram</li>\n<li>GRUFeatureExtractor</li></ul></li>\n<li><p>Unet Encoder</p>\n<ul>\n<li>various pretrained weight<ul>\n<li>resnet18</li>\n<li>resnet34</li>\n<li>inceptionv4</li></ul></li></ul></li>\n<li><p>Unet1d Decoder</p></li>\n<li><p>various duration</p>\n<ul>\n<li>17280 (24h)</li>\n<li>5760 (8h)</li></ul></li>\n</ul>\n<h3>isamu part</h3>\n<p>Isamu has two model architectures. The one is MultiResidualBiGRU model based on <a href=\"https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/416410\" target=\"_blank\">4th place solution of parkinson competition</a> and the other is Wavenet based.</p>\n<ul>\n<li>Feature Engineering<ul>\n<li>normalize anglez, enmo with mean, std</li>\n<li>stack sequences down sampled with various way (min, max, mean, std, median) </li></ul></li>\n</ul>\n<h3>tereka part</h3>\n<p>Tereka has two model architectures. The one is Unet based and the other is Transformer based.</p>\n<ul>\n<li><p>Feature Engineering</p>\n<ul>\n<li>normalize anglez, enmo with mean, std</li>\n<li>anglez, enmo, diff (anglez, enmo), hour feature (sin, cos)</li></ul></li>\n<li><p>Use a moving average  (window_size=12) of the model output.</p></li>\n</ul>\n<h3>post processing</h3>\n<ul>\n<li>Unwear detection<ul>\n<li>Both rule base (focus on daily cycle) and random forest model base are worked.</li>\n<li>One of the final submissions was with rule base and the other was with random forest model base.</li></ul></li>\n<li>When predicted value step=0, set step value to +1</li>\n<li>Boost candidates within 24 hours<ul>\n<li>Considering the periodicity of event being one within 24 hours, candidates with the greatest probability within 24 hours were boosted</li></ul></li>\n</ul>\n<h3>What Worked</h3>\n<ul>\n<li>Remove noisy 8 sample in train and valid phase</li>\n<li>Add fulltrain</li>\n<li>The lower threshold, the better score<ul>\n<li>The lower threshold, the higher Probability of submission scoring error :(</li></ul></li>\n<li>Exponential Moving Average</li>\n<li>Label Smoothing</li>\n<li>combine bce loss with dice loss(only apply peak)</li>\n</ul>\n<h3>What Didn’t Work</h3>\n<ul>\n<li>Larger model backbone</li>\n<li>positional encoding</li>\n<li>date embeddings</li>\n<li>2nd stage NN model</li>\n<li>layer lr tunning</li>\n<li>Transformer decoder</li>\n<li>The 0:3000 steps of onset predicted value array and the -3000: steps of wakeup predict array are set to 0.</li>\n</ul>\n<h3>Comparison of CV, LB, and Private for the final submissions</h3>\n<table>\n<thead>\n<tr>\n<th>Name</th>\n<th>CV</th>\n<th>LB</th>\n<th>Private</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Submission 1 (cv best)</td>\n<td>0.8125</td>\n<td>0.773</td>\n<td>0.823</td>\n</tr>\n<tr>\n<td>Submission 2 (lb best)</td>\n<td>-</td>\n<td>0.778</td>\n<td>0.822</td>\n</tr>\n</tbody>\n</table>\n<h3>Comparison of CV, LB, and Private with and without post-processing</h3>\n<table>\n<thead>\n<tr>\n<th>Name</th>\n<th>CV</th>\n<th>LB</th>\n<th>Private</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>With post-processing</td>\n<td>0.8125</td>\n<td>0.773</td>\n<td>0.823(+0.011)</td>\n</tr>\n<tr>\n<td>Without post-processing</td>\n<td>-</td>\n<td>0.767</td>\n<td>0.812</td>\n</tr>\n</tbody>\n</table>\n<h3>Best scores for each single model</h3>\n<table>\n<thead>\n<tr>\n<th>Name</th>\n<th>Model</th>\n<th>CV</th>\n<th>LB</th>\n<th>Private</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>furu model</td>\n<td>GRUINITFeatureExtractor-UNet1DDecoder</td>\n<td>0.7689665</td>\n<td>0.746</td>\n<td>0.8</td>\n</tr>\n<tr>\n<td>kaerururu model1</td>\n<td>GRUFeatureExtractor-UNet1DDecoder</td>\n<td>0.743</td>\n<td>-</td>\n<td>-</td>\n</tr>\n<tr>\n<td>kaerururu model2</td>\n<td>GRUFeatureExtractor-UNet1DDecoder</td>\n<td>0.733</td>\n<td>-</td>\n<td>-</td>\n</tr>\n<tr>\n<td>mizoo model</td>\n<td>LSTMFeatureExtractor-UNet1DDecoder</td>\n<td>0.7508</td>\n<td>0.736</td>\n<td>0.784</td>\n</tr>\n<tr>\n<td>Isamu model1</td>\n<td>wavenet_lstm</td>\n<td>0.773</td>\n<td>-</td>\n<td>-</td>\n</tr>\n<tr>\n<td>Isamu model2</td>\n<td>1d-gru</td>\n<td>0.775</td>\n<td>-</td>\n<td>-</td>\n</tr>\n<tr>\n<td>tereka model1</td>\n<td>unet base</td>\n<td>0.764</td>\n<td>0.752</td>\n<td>0.803</td>\n</tr>\n<tr>\n<td>tereka model2</td>\n<td>transformer base</td>\n<td>0.773</td>\n<td>0.729</td>\n<td>0.797</td>\n</tr>\n</tbody>\n</table>\n<h3>Important Citations</h3>\n<ul>\n<li>213 tubo's pipeline<ul>\n<li><a href=\"https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states\" target=\"_blank\">https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states</a></li></ul></li>\n<li>parkinson 4th place soludion<ul>\n<li><a href=\"https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/416410\" target=\"_blank\">https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/416410</a></li></ul></li>\n</ul>",
  "messages": [
    {
      "id": 2553236,
      "postDate": "2023-12-08T05:28:57.310Z",
      "content": "<p>First of all, I'd like to thank kaggle team, host for hosting this competition and thank many other participants to share codes, ideas and datasets. Special thanks to my teammates (<a href=\"https://www.kaggle.com/kunihikofurugori\" target=\"_blank\">furu</a>, <a href=\"https://www.kaggle.com/yukio0201\" target=\"_blank\">mizoo</a>, <a href=\"https://www.kaggle.com/yamsam\" target=\"_blank\">isamu</a>, <a href=\"https://www.kaggle.com/tereka\" target=\"_blank\">tereka</a>) who worked together on the competition and <a href=\"https://www.kaggle.com/tubotubo\" target=\"_blank\">213tubo</a> for sharing such a great <a href=\"https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states\" target=\"_blank\">pipeline</a>.</p>\n<h3>Overview</h3>\n<p>Our final submission is ensemble of 55 models in total, including eleven 5fold models.</p>\n<p>Our solution has four main parts. <br>\nThey are furu, mizoo, kaerururu part, isamu part, tereka part, and postprocess part.</p>\n<h3>furu, mizoo, kaerururu part</h3>\n<p>Our pipeline is based on 213tubo's one.<br>\nThe architecture is various Feature Extractor -&gt; Unet Encoder -&gt; Unet1d Decoder.<br>\nOnset cv best model and wakeup cv best model are <code>saved separately</code>.</p>\n<ul>\n<li><p>Feature Engineering</p>\n<ul>\n<li>normalize anglez, enmo with mean, std</li>\n<li>anglez, enmo, hour feature (sin, cos)</li></ul></li>\n<li><p>Feature Extractor</p>\n<ul>\n<li>LSTMFeatureExtractor</li>\n<li>CNNSpectrogram</li>\n<li>GRUFeatureExtractor</li></ul></li>\n<li><p>Unet Encoder</p>\n<ul>\n<li>various pretrained weight<ul>\n<li>resnet18</li>\n<li>resnet34</li>\n<li>inceptionv4</li></ul></li></ul></li>\n<li><p>Unet1d Decoder</p></li>\n<li><p>various duration</p>\n<ul>\n<li>17280 (24h)</li>\n<li>5760 (8h)</li></ul></li>\n</ul>\n<h3>isamu part</h3>\n<p>Isamu has two model architectures. The one is MultiResidualBiGRU model based on <a href=\"https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/416410\" target=\"_blank\">4th place solution of parkinson competition</a> and the other is Wavenet based.</p>\n<ul>\n<li>Feature Engineering<ul>\n<li>normalize anglez, enmo with mean, std</li>\n<li>stack sequences down sampled with various way (min, max, mean, std, median) </li></ul></li>\n</ul>\n<h3>tereka part</h3>\n<p>Tereka has two model architectures. The one is Unet based and the other is Transformer based.</p>\n<ul>\n<li><p>Feature Engineering</p>\n<ul>\n<li>normalize anglez, enmo with mean, std</li>\n<li>anglez, enmo, diff (anglez, enmo), hour feature (sin, cos)</li></ul></li>\n<li><p>Use a moving average  (window_size=12) of the model output.</p></li>\n</ul>\n<h3>post processing</h3>\n<ul>\n<li>Unwear detection<ul>\n<li>Both rule base (focus on daily cycle) and random forest model base are worked.</li>\n<li>One of the final submissions was with rule base and the other was with random forest model base.</li></ul></li>\n<li>When predicted value step=0, set step value to +1</li>\n<li>Boost candidates within 24 hours<ul>\n<li>Considering the periodicity of event being one within 24 hours, candidates with the greatest probability within 24 hours were boosted</li></ul></li>\n</ul>\n<h3>What Worked</h3>\n<ul>\n<li>Remove noisy 8 sample in train and valid phase</li>\n<li>Add fulltrain</li>\n<li>The lower threshold, the better score<ul>\n<li>The lower threshold, the higher Probability of submission scoring error :(</li></ul></li>\n<li>Exponential Moving Average</li>\n<li>Label Smoothing</li>\n<li>combine bce loss with dice loss(only apply peak)</li>\n</ul>\n<h3>What Didn’t Work</h3>\n<ul>\n<li>Larger model backbone</li>\n<li>positional encoding</li>\n<li>date embeddings</li>\n<li>2nd stage NN model</li>\n<li>layer lr tunning</li>\n<li>Transformer decoder</li>\n<li>The 0:3000 steps of onset predicted value array and the -3000: steps of wakeup predict array are set to 0.</li>\n</ul>\n<h3>Comparison of CV, LB, and Private for the final submissions</h3>\n<table>\n<thead>\n<tr>\n<th>Name</th>\n<th>CV</th>\n<th>LB</th>\n<th>Private</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Submission 1 (cv best)</td>\n<td>0.8125</td>\n<td>0.773</td>\n<td>0.823</td>\n</tr>\n<tr>\n<td>Submission 2 (lb best)</td>\n<td>-</td>\n<td>0.778</td>\n<td>0.822</td>\n</tr>\n</tbody>\n</table>\n<h3>Comparison of CV, LB, and Private with and without post-processing</h3>\n<table>\n<thead>\n<tr>\n<th>Name</th>\n<th>CV</th>\n<th>LB</th>\n<th>Private</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>With post-processing</td>\n<td>0.8125</td>\n<td>0.773</td>\n<td>0.823(+0.011)</td>\n</tr>\n<tr>\n<td>Without post-processing</td>\n<td>-</td>\n<td>0.767</td>\n<td>0.812</td>\n</tr>\n</tbody>\n</table>\n<h3>Best scores for each single model</h3>\n<table>\n<thead>\n<tr>\n<th>Name</th>\n<th>Model</th>\n<th>CV</th>\n<th>LB</th>\n<th>Private</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>furu model</td>\n<td>GRUINITFeatureExtractor-UNet1DDecoder</td>\n<td>0.7689665</td>\n<td>0.746</td>\n<td>0.8</td>\n</tr>\n<tr>\n<td>kaerururu model1</td>\n<td>GRUFeatureExtractor-UNet1DDecoder</td>\n<td>0.743</td>\n<td>-</td>\n<td>-</td>\n</tr>\n<tr>\n<td>kaerururu model2</td>\n<td>GRUFeatureExtractor-UNet1DDecoder</td>\n<td>0.733</td>\n<td>-</td>\n<td>-</td>\n</tr>\n<tr>\n<td>mizoo model</td>\n<td>LSTMFeatureExtractor-UNet1DDecoder</td>\n<td>0.7508</td>\n<td>0.736</td>\n<td>0.784</td>\n</tr>\n<tr>\n<td>Isamu model1</td>\n<td>wavenet_lstm</td>\n<td>0.773</td>\n<td>-</td>\n<td>-</td>\n</tr>\n<tr>\n<td>Isamu model2</td>\n<td>1d-gru</td>\n<td>0.775</td>\n<td>-</td>\n<td>-</td>\n</tr>\n<tr>\n<td>tereka model1</td>\n<td>unet base</td>\n<td>0.764</td>\n<td>0.752</td>\n<td>0.803</td>\n</tr>\n<tr>\n<td>tereka model2</td>\n<td>transformer base</td>\n<td>0.773</td>\n<td>0.729</td>\n<td>0.797</td>\n</tr>\n</tbody>\n</table>\n<h3>Important Citations</h3>\n<ul>\n<li>213 tubo's pipeline<ul>\n<li><a href=\"https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states\" target=\"_blank\">https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states</a></li></ul></li>\n<li>parkinson 4th place soludion<ul>\n<li><a href=\"https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/416410\" target=\"_blank\">https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/416410</a></li></ul></li>\n</ul>",
      "rawMarkdown": "First of all, I'd like to thank kaggle team, host for hosting this competition and thank many other participants to share codes, ideas and datasets. Special thanks to my teammates ([furu](https://www.kaggle.com/kunihikofurugori), [mizoo](https://www.kaggle.com/yukio0201), [isamu](https://www.kaggle.com/yamsam), [tereka](https://www.kaggle.com/tereka)) who worked together on the competition and [213tubo](https://www.kaggle.com/tubotubo) for sharing such a great [pipeline](https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states).\n\n### Overview\n\nOur final submission is ensemble of 55 models in total, including eleven 5fold models.\n\nOur solution has four main parts. \nThey are furu, mizoo, kaerururu part, isamu part, tereka part, and postprocess part.\n\n### furu, mizoo, kaerururu part\n\nOur pipeline is based on 213tubo's one.\nThe architecture is various Feature Extractor -> Unet Encoder -> Unet1d Decoder.\nOnset cv best model and wakeup cv best model are `saved separately`.\n\n- Feature Engineering\n    - normalize anglez, enmo with mean, std\n    - anglez, enmo, hour feature (sin, cos)\n\n- Feature Extractor\n    - LSTMFeatureExtractor\n    - CNNSpectrogram\n    - GRUFeatureExtractor\n- Unet Encoder\n    - various pretrained weight\n        - resnet18\n        - resnet34\n        - inceptionv4\n- Unet1d Decoder\n- various duration\n    - 17280 (24h)\n    - 5760 (8h)\n\n### isamu part\n\nIsamu has two model architectures. The one is MultiResidualBiGRU model based on [4th place solution of parkinson competition](https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/416410) and the other is Wavenet based.\n\n- Feature Engineering\n    - normalize anglez, enmo with mean, std\n    - stack sequences down sampled with various way (min, max, mean, std, median) \n\n### tereka part\n\nTereka has two model architectures. The one is Unet based and the other is Transformer based.\n\n- Feature Engineering\n    - normalize anglez, enmo with mean, std\n    - anglez, enmo, diff (anglez, enmo), hour feature (sin, cos)\n\n- Use a moving average  (window_size=12) of the model output.\n\n### post processing\n\n- Unwear detection\n    - Both rule base (focus on daily cycle) and random forest model base are worked.\n    - One of the final submissions was with rule base and the other was with random forest model base.\n- When predicted value step=0, set step value to +1\n- Boost candidates within 24 hours\n    - Considering the periodicity of event being one within 24 hours, candidates with the greatest probability within 24 hours were boosted\n\n### What Worked\n- Remove noisy 8 sample in train and valid phase\n- Add fulltrain\n- The lower threshold, the better score\n   - The lower threshold, the higher Probability of submission scoring error :(\n- Exponential Moving Average\n- Label Smoothing\n- combine bce loss with dice loss(only apply peak)\n\n\n### What Didn’t Work\n\n- Larger model backbone\n- positional encoding\n- date embeddings\n- 2nd stage NN model\n- layer lr tunning\n- Transformer decoder\n- The 0:3000 steps of onset predicted value array and the -3000: steps of wakeup predict array are set to 0.\n\n### Comparison of CV, LB, and Private for the final submissions\n\n| Name | CV | LB | Private |\n| ---- | ---- | ---- | ---- |\n| Submission 1 (cv best)  | 0.8125 | 0.773 | 0.823 |\n| Submission 2 (lb best)  | - | 0.778 | 0.822 |\n\n### Comparison of CV, LB, and Private with and without post-processing\n\n| Name | CV | LB | Private |\n| ---- | ---- | ---- | ---- |\n| With post-processing  |  0.8125 | 0.773 | 0.823(+0.011) |\n| Without post-processing  | - | 0.767 | 0.812 |\n\n### Best scores for each single model\n\n| Name | Model | CV | LB | Private |\n| ---- | ---- | ---- | ---- | ---- |\n| furu model  |  GRUINITFeatureExtractor-UNet1DDecoder  | 0.7689665 |  0.746  |  0.8  |\n| kaerururu model1  |  GRUFeatureExtractor-UNet1DDecoder  | 0.743 | - | - |\n| kaerururu model2  |  GRUFeatureExtractor-UNet1DDecoder  | 0.733 | - | - |\n| mizoo model  |  LSTMFeatureExtractor-UNet1DDecoder  | 0.7508 | 0.736 | 0.784 |\n| Isamu model1 |  wavenet_lstm  | 0.773 | - | - | \n| Isamu model2 |  1d-gru  | 0.775 | - | - | \n| tereka model1 |  unet base  | 0.764 | 0.752 | 0.803 |\n| tereka model2 |  transformer base  | 0.773 | 0.729 | 0.797 |\n\n\n### Important Citations\n- 213 tubo's pipeline\n    - https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states\n- parkinson 4th place soludion\n    - https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/416410",
      "votes": 24
    },
    {
      "id": 2554168,
      "postDate": "2023-12-08T20:57:27.693Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2554168,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-12-08T20:57:27.693000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2553236": "First of all, I'd like to thank kaggle team, host for hosting this competition and thank many other participants to share codes, ideas and datasets. Special thanks to my teammates ([furu](https://www.kaggle.com/kunihikofurugori), [mizoo](https://www.kaggle.com/yukio0201), [isamu](https://www.kaggle.com/yamsam), [tereka](https://www.kaggle.com/tereka)) who worked together on the competition and [213tubo](https://www.kaggle.com/tubotubo) for sharing such a great [pipeline](https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states).\n\n### Overview\n\nOur final submission is ensemble of 55 models in total, including eleven 5fold models.\n\nOur solution has four main parts. \nThey are furu, mizoo, kaerururu part, isamu part, tereka part, and postprocess part.\n\n### furu, mizoo, kaerururu part\n\nOur pipeline is based on 213tubo's one.\nThe architecture is various Feature Extractor -> Unet Encoder -> Unet1d Decoder.\nOnset cv best model and wakeup cv best model are `saved separately`.\n\n- Feature Engineering\n    - normalize anglez, enmo with mean, std\n    - anglez, enmo, hour feature (sin, cos)\n\n- Feature Extractor\n    - LSTMFeatureExtractor\n    - CNNSpectrogram\n    - GRUFeatureExtractor\n- Unet Encoder\n    - various pretrained weight\n        - resnet18\n        - resnet34\n        - inceptionv4\n- Unet1d Decoder\n- various duration\n    - 17280 (24h)\n    - 5760 (8h)\n\n### isamu part\n\nIsamu has two model architectures. The one is MultiResidualBiGRU model based on [4th place solution of parkinson competition](https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/416410) and the other is Wavenet based.\n\n- Feature Engineering\n    - normalize anglez, enmo with mean, std\n    - stack sequences down sampled with various way (min, max, mean, std, median) \n\n### tereka part\n\nTereka has two model architectures. The one is Unet based and the other is Transformer based.\n\n- Feature Engineering\n    - normalize anglez, enmo with mean, std\n    - anglez, enmo, diff (anglez, enmo), hour feature (sin, cos)\n\n- Use a moving average  (window_size=12) of the model output.\n\n### post processing\n\n- Unwear detection\n    - Both rule base (focus on daily cycle) and random forest model base are worked.\n    - One of the final submissions was with rule base and the other was with random forest model base.\n- When predicted value step=0, set step value to +1\n- Boost candidates within 24 hours\n    - Considering the periodicity of event being one within 24 hours, candidates with the greatest probability within 24 hours were boosted\n\n### What Worked\n- Remove noisy 8 sample in train and valid phase\n- Add fulltrain\n- The lower threshold, the better score\n   - The lower threshold, the higher Probability of submission scoring error :(\n- Exponential Moving Average\n- Label Smoothing\n- combine bce loss with dice loss(only apply peak)\n\n\n### What Didn’t Work\n\n- Larger model backbone\n- positional encoding\n- date embeddings\n- 2nd stage NN model\n- layer lr tunning\n- Transformer decoder\n- The 0:3000 steps of onset predicted value array and the -3000: steps of wakeup predict array are set to 0.\n\n### Comparison of CV, LB, and Private for the final submissions\n\n| Name | CV | LB | Private |\n| ---- | ---- | ---- | ---- |\n| Submission 1 (cv best)  | 0.8125 | 0.773 | 0.823 |\n| Submission 2 (lb best)  | - | 0.778 | 0.822 |\n\n### Comparison of CV, LB, and Private with and without post-processing\n\n| Name | CV | LB | Private |\n| ---- | ---- | ---- | ---- |\n| With post-processing  |  0.8125 | 0.773 | 0.823(+0.011) |\n| Without post-processing  | - | 0.767 | 0.812 |\n\n### Best scores for each single model\n\n| Name | Model | CV | LB | Private |\n| ---- | ---- | ---- | ---- | ---- |\n| furu model  |  GRUINITFeatureExtractor-UNet1DDecoder  | 0.7689665 |  0.746  |  0.8  |\n| kaerururu model1  |  GRUFeatureExtractor-UNet1DDecoder  | 0.743 | - | - |\n| kaerururu model2  |  GRUFeatureExtractor-UNet1DDecoder  | 0.733 | - | - |\n| mizoo model  |  LSTMFeatureExtractor-UNet1DDecoder  | 0.7508 | 0.736 | 0.784 |\n| Isamu model1 |  wavenet_lstm  | 0.773 | - | - | \n| Isamu model2 |  1d-gru  | 0.775 | - | - | \n| tereka model1 |  unet base  | 0.764 | 0.752 | 0.803 |\n| tereka model2 |  transformer base  | 0.773 | 0.729 | 0.797 |\n\n\n### Important Citations\n- 213 tubo's pipeline\n    - https://github.com/tubo213/kaggle-child-mind-institute-detect-sleep-states\n- parkinson 4th place soludion\n    - https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/416410",
    "2554168": ""
  }
}