{
  "id": 460021,
  "title": "29th Place Solution and Initial Solution",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/460021",
  "author_name": "Maruichi01",
  "post_date": "2023-12-07T13:18:11.585000",
  "votes": 10,
  "comment_count": 0,
  "views": 0,
  "content": "<h2>29th Solution Overview:</h2>\n<h3>Model Architecture (from tubo's GitHub Repository):</h3>\n<ul>\n<li>2DCNN Feature Extractor -&gt; Unet -&gt; Unet Decoder</li>\n<li>LSTM Feature Extractor -&gt; 1DCNN -&gt; LSTM Decoder</li>\n</ul>\n<h3>Input Features:</h3>\n<ul>\n<li>enmo, anglez, anglez_diff, hour, weekday, and a noise flag.<ul>\n<li>Noise is flagged if identical (enmo + anglez) occur within a 5-minute span in the same series_id.</li>\n<li>EDA indicated that the day before a holiday typically shows later onset times, prompting the inclusion of weekday and holiday features.</li></ul></li>\n<li>Input steps of 5760 and 11520 were used, with four models based on ensemble model and input step pairs.</li>\n</ul>\n<h3>Target Size:</h3>\n<ul>\n<li>The target size is identical to the input size.</li>\n</ul>\n<h3>Training Targets:</h3>\n<ul>\n<li>Events with Gaussian soft labels (2 channels)</li>\n<li>Sleep flag (1 channel)</li>\n</ul>\n<h3>Inference Process:</h3>\n<ul>\n<li>Chunks are created by sliding the input_step/4.<ul>\n<li>For example, in the 5760 model, chunk steps begin from 0, 1440, 2880, etc.</li></ul></li>\n<li>All chunks are predicted and averaged by step.</li>\n<li>In CNN-based models, edge predictions are trimmed by 12.5%.</li>\n</ul>\n<h3>Postprocessing:</h3>\n<ul>\n<li>Use only events prediction (sleep flag is not used).</li>\n<li>Utilizes find_peaks from SciPy signals, and applies a weighted average to steps near detected signals based on the prediction scores.<ul>\n<li>For instance, if the candidate step is 5000, the weighted average of steps 4998-5002 is calculated.</li></ul></li>\n</ul>\n<hr>\n<p>Final solution is just a small modification from tubo's approach, so let me share my initial solution. (I took most of my competition time in this approach…)</p>\n<h2>Initial Solution (LB 0.677):</h2>\n<h3>Based on a two-stage model:</h3>\n<ul>\n<li>Stage 1: Chunk Detection by binary classification <ul>\n<li>Segmenting the dataset into 1-hour chunks with weak labels to identify useful segments. Label 1 is added if an event occurs within a chunk.</li></ul></li>\n<li>Stage 2: L1 Regression <ul>\n<li>Pinpointing the event position within the candidate chunks from Stage 1, followed by post-processing.</li></ul></li>\n</ul>\n<h3>Challenges and Learning Points:</h3>\n<p>As a newcomer to neural networks, my approach lacked certain elements:</p>\n<ul>\n<li>Random Dataloader: No ideas to implement a random chunk dataloader. Each epoch started from the same position, leading to overfitting and lack of robustness.</li>\n<li>Model Exploration: I used a very basic model (a single MLP, attention layer, LSTM, and head). Exploring more sophisticated models, as seen in tubo's work, would have been beneficial.</li>\n</ul>\n<hr>\n<h2>Concluding Thoughts:</h2>\n<p>This competition was a tremendous learning experience. My thanks to Kaggle for this opportunity, and to all the participants for their contributions and shared insights</p>",
  "messages": [
    {
      "id": 2552442,
      "postDate": "2023-12-07T13:18:11.587Z",
      "content": "<h2>29th Solution Overview:</h2>\n<h3>Model Architecture (from tubo's GitHub Repository):</h3>\n<ul>\n<li>2DCNN Feature Extractor -&gt; Unet -&gt; Unet Decoder</li>\n<li>LSTM Feature Extractor -&gt; 1DCNN -&gt; LSTM Decoder</li>\n</ul>\n<h3>Input Features:</h3>\n<ul>\n<li>enmo, anglez, anglez_diff, hour, weekday, and a noise flag.<ul>\n<li>Noise is flagged if identical (enmo + anglez) occur within a 5-minute span in the same series_id.</li>\n<li>EDA indicated that the day before a holiday typically shows later onset times, prompting the inclusion of weekday and holiday features.</li></ul></li>\n<li>Input steps of 5760 and 11520 were used, with four models based on ensemble model and input step pairs.</li>\n</ul>\n<h3>Target Size:</h3>\n<ul>\n<li>The target size is identical to the input size.</li>\n</ul>\n<h3>Training Targets:</h3>\n<ul>\n<li>Events with Gaussian soft labels (2 channels)</li>\n<li>Sleep flag (1 channel)</li>\n</ul>\n<h3>Inference Process:</h3>\n<ul>\n<li>Chunks are created by sliding the input_step/4.<ul>\n<li>For example, in the 5760 model, chunk steps begin from 0, 1440, 2880, etc.</li></ul></li>\n<li>All chunks are predicted and averaged by step.</li>\n<li>In CNN-based models, edge predictions are trimmed by 12.5%.</li>\n</ul>\n<h3>Postprocessing:</h3>\n<ul>\n<li>Use only events prediction (sleep flag is not used).</li>\n<li>Utilizes find_peaks from SciPy signals, and applies a weighted average to steps near detected signals based on the prediction scores.<ul>\n<li>For instance, if the candidate step is 5000, the weighted average of steps 4998-5002 is calculated.</li></ul></li>\n</ul>\n<hr>\n<p>Final solution is just a small modification from tubo's approach, so let me share my initial solution. (I took most of my competition time in this approach…)</p>\n<h2>Initial Solution (LB 0.677):</h2>\n<h3>Based on a two-stage model:</h3>\n<ul>\n<li>Stage 1: Chunk Detection by binary classification <ul>\n<li>Segmenting the dataset into 1-hour chunks with weak labels to identify useful segments. Label 1 is added if an event occurs within a chunk.</li></ul></li>\n<li>Stage 2: L1 Regression <ul>\n<li>Pinpointing the event position within the candidate chunks from Stage 1, followed by post-processing.</li></ul></li>\n</ul>\n<h3>Challenges and Learning Points:</h3>\n<p>As a newcomer to neural networks, my approach lacked certain elements:</p>\n<ul>\n<li>Random Dataloader: No ideas to implement a random chunk dataloader. Each epoch started from the same position, leading to overfitting and lack of robustness.</li>\n<li>Model Exploration: I used a very basic model (a single MLP, attention layer, LSTM, and head). Exploring more sophisticated models, as seen in tubo's work, would have been beneficial.</li>\n</ul>\n<hr>\n<h2>Concluding Thoughts:</h2>\n<p>This competition was a tremendous learning experience. My thanks to Kaggle for this opportunity, and to all the participants for their contributions and shared insights</p>",
      "rawMarkdown": "## 29th Solution Overview:\n### Model Architecture (from tubo's GitHub Repository):\n- 2DCNN Feature Extractor -> Unet -> Unet Decoder\n- LSTM Feature Extractor -> 1DCNN -> LSTM Decoder\n\n### Input Features:\n- enmo, anglez, anglez_diff, hour, weekday, and a noise flag.\n  - Noise is flagged if identical (enmo + anglez) occur within a 5-minute span in the same series_id.\n  - EDA indicated that the day before a holiday typically shows later onset times, prompting the inclusion of weekday and holiday features.\n- Input steps of 5760 and 11520 were used, with four models based on ensemble model and input step pairs.\n\n### Target Size:\n\n- The target size is identical to the input size.\n\n### Training Targets:\n\n- Events with Gaussian soft labels (2 channels)\n- Sleep flag (1 channel)\n\n### Inference Process:\n\n- Chunks are created by sliding the input_step/4.\n  - For example, in the 5760 model, chunk steps begin from 0, 1440, 2880, etc.\n- All chunks are predicted and averaged by step.\n- In CNN-based models, edge predictions are trimmed by 12.5%.\n\n### Postprocessing:\n\n- Use only events prediction (sleep flag is not used).\n- Utilizes find_peaks from SciPy signals, and applies a weighted average to steps near detected signals based on the prediction scores.\n  - For instance, if the candidate step is 5000, the weighted average of steps 4998-5002 is calculated.\n\n<hr>\n\nFinal solution is just a small modification from tubo's approach, so let me share my initial solution. (I took most of my competition time in this approach...)\n\n## Initial Solution (LB 0.677):\n\n### Based on a two-stage model:\n- Stage 1: Chunk Detection by binary classification \n  - Segmenting the dataset into 1-hour chunks with weak labels to identify useful segments. Label 1 is added if an event occurs within a chunk.\n- Stage 2: L1 Regression \n  - Pinpointing the event position within the candidate chunks from Stage 1, followed by post-processing.\n\n### Challenges and Learning Points:\n\nAs a newcomer to neural networks, my approach lacked certain elements:\n- Random Dataloader: No ideas to implement a random chunk dataloader. Each epoch started from the same position, leading to overfitting and lack of robustness.\n- Model Exploration: I used a very basic model (a single MLP, attention layer, LSTM, and head). Exploring more sophisticated models, as seen in tubo's work, would have been beneficial.\n\n<hr>\n\n## Concluding Thoughts:\nThis competition was a tremendous learning experience. My thanks to Kaggle for this opportunity, and to all the participants for their contributions and shared insights",
      "votes": 10
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2552442": "## 29th Solution Overview:\n### Model Architecture (from tubo's GitHub Repository):\n- 2DCNN Feature Extractor -> Unet -> Unet Decoder\n- LSTM Feature Extractor -> 1DCNN -> LSTM Decoder\n\n### Input Features:\n- enmo, anglez, anglez_diff, hour, weekday, and a noise flag.\n  - Noise is flagged if identical (enmo + anglez) occur within a 5-minute span in the same series_id.\n  - EDA indicated that the day before a holiday typically shows later onset times, prompting the inclusion of weekday and holiday features.\n- Input steps of 5760 and 11520 were used, with four models based on ensemble model and input step pairs.\n\n### Target Size:\n\n- The target size is identical to the input size.\n\n### Training Targets:\n\n- Events with Gaussian soft labels (2 channels)\n- Sleep flag (1 channel)\n\n### Inference Process:\n\n- Chunks are created by sliding the input_step/4.\n  - For example, in the 5760 model, chunk steps begin from 0, 1440, 2880, etc.\n- All chunks are predicted and averaged by step.\n- In CNN-based models, edge predictions are trimmed by 12.5%.\n\n### Postprocessing:\n\n- Use only events prediction (sleep flag is not used).\n- Utilizes find_peaks from SciPy signals, and applies a weighted average to steps near detected signals based on the prediction scores.\n  - For instance, if the candidate step is 5000, the weighted average of steps 4998-5002 is calculated.\n\n<hr>\n\nFinal solution is just a small modification from tubo's approach, so let me share my initial solution. (I took most of my competition time in this approach...)\n\n## Initial Solution (LB 0.677):\n\n### Based on a two-stage model:\n- Stage 1: Chunk Detection by binary classification \n  - Segmenting the dataset into 1-hour chunks with weak labels to identify useful segments. Label 1 is added if an event occurs within a chunk.\n- Stage 2: L1 Regression \n  - Pinpointing the event position within the candidate chunks from Stage 1, followed by post-processing.\n\n### Challenges and Learning Points:\n\nAs a newcomer to neural networks, my approach lacked certain elements:\n- Random Dataloader: No ideas to implement a random chunk dataloader. Each epoch started from the same position, leading to overfitting and lack of robustness.\n- Model Exploration: I used a very basic model (a single MLP, attention layer, LSTM, and head). Exploring more sophisticated models, as seen in tubo's work, would have been beneficial.\n\n<hr>\n\n## Concluding Thoughts:\nThis competition was a tremendous learning experience. My thanks to Kaggle for this opportunity, and to all the participants for their contributions and shared insights"
  },
  "complete": true,
  "source": "existing_discussion",
  "competition_id": 53666,
  "competition_slug": "child-mind-institute-detect-sleep-states",
  "team_id": 10873018,
  "solution_url": "/competitions/child-mind-institute-detect-sleep-states/writeups/maruichi01-29th-place-solution-and-initial-solutio"
}