![\"Comparison\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F10671485%2Fec42c865fae37d8815cc81fff8521184%2F_20231205195307.png?generation=1701903784520525&alt=media\")
Thanks to Kaggle for hosting this meaningful competition. Thanks to all the Kagglers for your discussions and shared perspectives. This was our first time participating in a formal tabular competition, and we've learned a lot from the experience.
\nTeam Avengers will always continue the journey on Kaggle.
\nMain GitHub Repo: Here
\nPrecTime GitHub Repo: Here
\nHere, we need to thank tubotubo for providing the baseline code. We didn't join the competition from the very beginning, this baseline code provided us with some ideas and basic model structures.
\nWe didn't use any methods to handle the dirty data, which might be one reason why we couldn't improve our scores anymore.
On the evening before the competition ended, my teammate found this discussion. Consequently, we attempted to clean the data by removing the data for the days where the event was empty. However, due to the time limitation, we didn't make significant progress.
We believe data cleaning should be helpful because the model using this method showed a smaller difference in scores on the private leaderboard.
We generated new features using shift, different and rolling window functions.
\nThe final set we utilized 24 rolling features in addition to the original 4, making a total of 28 features. The use of new features did not significantly improve the model's score, which was somewhat unexpected for us.
\nCode: Here
\n*[pl.().(i).(f) i (diff_start, diff_end, diff_step)],\n*[pl.().shift(i).(f)\n i (shift_start, shift_end, shift_step) i != ],\n*[pl.().rolling_mean(window_size).(\n f) window_size window_steps],\n*[pl.().rolling_min(window_size).(\n f) window_size window_steps],\n*[pl.().rolling_max(window_size).(\n f) window_size window_steps],\n*[pl.().rolling_std(window_size).(\n f) window_size window_steps]\nWandb sweep is a hyperparameter optimization tool provided by the Wandb machine learning platform. It allows automatic exploration of different hyperparameter combinations to enhance a model's performance.
\nImplementation Code: Here
\nWe used dynamic programming algorithm to deal with overlap problem.
\nPrinciple behind this method: To achieve a high AP (Average Precision), three main criteria need to be met: Firstly, the predicted label should be sufficiently close to the actual label. Secondly, within a positive or negative tolerance range around the actual label, there should only be one predicted point. Thirdly, the score of other predicted points outside the actual label range should be lower than those within the range.
\ndef get_results_slide_window(pred, gap):\n scores = (pred)\n stack = []\n = [-] * (scores)\n [] = \n i in (,(scores)):\n i - stack[-] < gap:\n scores[i] >= scores[stack[-]]:\n stack.()\n i - gap >= :\n stack:\n [i - gap] != stack[-]:\n stack [i - gap] - stack[-] < gap:\n stack.()\n stack.([i - gap])\n :\n stack.([i - gap])\n stack.(i)\n :\n stack.(i)\n [i] = stack[-]\n stack\nOur final ensemble method essentially involved averaging different outputs. With post-processing and this ensemble method combined, our results generally follow the pattern that the more models we use or the greater the variety of models, the higher the score.
\nSubmission Code1: Here
\nSubmission Code2: Here
Our submissions:
\n| Models | \nLB Score | \nPB Score | \nSelected | \n
|---|---|---|---|
2 * 5 folds PrecTime + 1 * 5 folds LSTM-Unet | \n0.75 | \n0.8 | \nYes | \n
2 * 5 folds PrecTime + 2 * 5 folds LSTM-Unet + 10 single models | \n0.759 | \n0.803 | \nYes | \n
1 * 5 folds PrecTime + 1 fold LSTM-Unet + 10 single models | \n0.761 | \n0.804 | \nNo | \n
1 * 5 folds PrecTime + 1 * 5 folds LSTM-Unet + 10 single models | \n0.759 | \n0.803 | \nNo | \n
Welcome everyone to check our GitHub code, looking forward to any discussions.
\nThanks to Kaggle for hosting this meaningful competition. Thanks to all the Kagglers for your discussions and shared perspectives. This was our first time participating in a formal tabular competition, and we've learned a lot from the experience.
\nTeam Avengers will always continue the journey on Kaggle.
\nMain GitHub Repo: Here
\nPrecTime GitHub Repo: Here
\nHere, we need to thank tubotubo for providing the baseline code. We didn't join the competition from the very beginning, this baseline code provided us with some ideas and basic model structures.
\nWe didn't use any methods to handle the dirty data, which might be one reason why we couldn't improve our scores anymore.
On the evening before the competition ended, my teammate found this discussion. Consequently, we attempted to clean the data by removing the data for the days where the event was empty. However, due to the time limitation, we didn't make significant progress.
We believe data cleaning should be helpful because the model using this method showed a smaller difference in scores on the private leaderboard.
We generated new features using shift, different and rolling window functions.
\nThe final set we utilized 24 rolling features in addition to the original 4, making a total of 28 features. The use of new features did not significantly improve the model's score, which was somewhat unexpected for us.
\nCode: Here
\n*[pl.().(i).(f) i (diff_start, diff_end, diff_step)],\n*[pl.().shift(i).(f)\n i (shift_start, shift_end, shift_step) i != ],\n*[pl.().rolling_mean(window_size).(\n f) window_size window_steps],\n*[pl.().rolling_min(window_size).(\n f) window_size window_steps],\n*[pl.().rolling_max(window_size).(\n f) window_size window_steps],\n*[pl.().rolling_std(window_size).(\n f) window_size window_steps]\nWandb sweep is a hyperparameter optimization tool provided by the Wandb machine learning platform. It allows automatic exploration of different hyperparameter combinations to enhance a model's performance.
\nImplementation Code: Here
\nWe used dynamic programming algorithm to deal with overlap problem.
\nPrinciple behind this method: To achieve a high AP (Average Precision), three main criteria need to be met: Firstly, the predicted label should be sufficiently close to the actual label. Secondly, within a positive or negative tolerance range around the actual label, there should only be one predicted point. Thirdly, the score of other predicted points outside the actual label range should be lower than those within the range.
\ndef get_results_slide_window(pred, gap):\n scores = (pred)\n stack = []\n = [-] * (scores)\n [] = \n i in (,(scores)):\n i - stack[-] < gap:\n scores[i] >= scores[stack[-]]:\n stack.()\n i - gap >= :\n stack:\n [i - gap] != stack[-]:\n stack [i - gap] - stack[-] < gap:\n stack.()\n stack.([i - gap])\n :\n stack.([i - gap])\n stack.(i)\n :\n stack.(i)\n [i] = stack[-]\n stack\nOur final ensemble method essentially involved averaging different outputs. With post-processing and this ensemble method combined, our results generally follow the pattern that the more models we use or the greater the variety of models, the higher the score.
\nSubmission Code1: Here
\nSubmission Code2: Here
Our submissions:
\n| Models | \nLB Score | \nPB Score | \nSelected | \n
|---|---|---|---|
2 * 5 folds PrecTime + 1 * 5 folds LSTM-Unet | \n0.75 | \n0.8 | \nYes | \n
2 * 5 folds PrecTime + 2 * 5 folds LSTM-Unet + 10 single models | \n0.759 | \n0.803 | \nYes | \n
1 * 5 folds PrecTime + 1 fold LSTM-Unet + 10 single models | \n0.761 | \n0.804 | \nNo | \n
1 * 5 folds PrecTime + 1 * 5 folds LSTM-Unet + 10 single models | \n0.759 | \n0.803 | \nNo | \n
Welcome everyone to check our GitHub code, looking forward to any discussions.
\nexcellent work
", "rawMarkdown": "excellent work", "votes": null }, { "id": "2551916", "postDate": "12/07/2023 04:10:37", "content": "excellent teammates
", "rawMarkdown": "excellent teammates", "votes": null }, { "id": "2552974", "postDate": "12/07/2023 23:41:42", "content": "Data Structures and Algorithms implementation in play!
", "rawMarkdown": "Data Structures and Algorithms implementation in play!", "votes": null } ], "comments": [ { "id": 2551906, "author_name": "rdxsun", "author_url": "", "post_date": "12/07/2023 03:56:12", "content": "excellent work
", "votes": null, "replies": [ { "id": 2551916, "author_name": "lizhecheng", "author_url": "", "post_date": "12/07/2023 04:10:37", "content": "excellent teammates
", "votes": null, "replies": [] } ] }, { "id": 2552974, "author_name": "yeoyunsianggeremie", "author_url": "", "post_date": "12/07/2023 23:41:42", "content": "Data Structures and Algorithms implementation in play!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2551723": "# Here is 39th Detail Solution Child Mind Institute - Detect Sleep States\n\nThanks to Kaggle for hosting this meaningful competition. Thanks to all the Kagglers for your discussions and shared perspectives. This was our first time participating in a formal tabular competition, and we've learned a lot from the experience.\n\n**Team Avengers will always continue the journey on Kaggle.**\n\nMain GitHub Repo: [Here](https://github.com/lullabies777/kaggle-detect-sleep)\n\nPrecTime GitHub Repo: [Here](https://github.com/Lizhecheng02/Kaggle-Detect_Sleep_States)\n\n## Here is the Detail Solution\n\n### Baseline Code\n\nHere, we need to thank [tubotubo](https://www.kaggle.com/tubotubo) for providing the baseline code. We didn't join the competition from the very beginning, this baseline code provided us with some ideas and basic model structures.\n\n### Dataset Preparation\n\n- We didn't use any methods to handle the dirty data, which might be one reason why we couldn't improve our scores anymore.\n\n- On the evening before the competition ended, my teammate found this [discussion](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/456177). Consequently, we attempted to clean the data by removing the data for the days where the event was empty. However, due to the time limitation, we didn't make significant progress. \n\n- We believe data cleaning should be helpful because the model using this method showed a smaller difference in scores on the private leaderboard.\n\n\n\n### Feature Engineering\n\nWe generated new features using shift, different and rolling window functions. \n\nThe final set we utilized 24 rolling features in addition to the original 4, making a total of 28 features. The use of new features did not significantly improve the model's score, which was somewhat unexpected for us.\n\nCode: [Here](https://github.com/lullabies777/kaggle-detect-sleep/blob/main/run/prepare_data.py)\n\n```\n*[pl.col(\"anglez\").diff(i).alias(f\"anglez_diff_{i}\") for i in range(diff_start, diff_end, diff_step)],\n*[pl.col(\"anglez\").shift(i).alias(f\"anglez_lag_{i}\")\n for i in range(shift_start, shift_end, shift_step) if i != 0],\n*[pl.col(\"anglez\").rolling_mean(window_size).alias(\n f\"anglez_mean_{window_size}\") for window_size in window_steps],\n*[pl.col(\"anglez\").rolling_min(window_size).alias(\n f\"anglez_min_{window_size}\") for window_size in window_steps],\n*[pl.col(\"anglez\").rolling_max(window_size).alias(\n f\"anglez_max_{window_size}\") for window_size in window_steps],\n*[pl.col(\"anglez\").rolling_std(window_size).alias(\n f\"anglez_std_{window_size}\") for window_size in window_steps]\n```\n\n### Wandb sweep\n\nWandb sweep is a hyperparameter optimization tool provided by the Wandb machine learning platform. It allows automatic exploration of different hyperparameter combinations to enhance a model's performance.\n\nImplementation Code: [Here](https://github.com/lullabies777/kaggle-detect-sleep/tree/main/run/sweep)\n\n### Models\n\n- Used overlap - To enhance accuracy in predicting sequence edges, we utilized overlap by using a 10000 length sequence to predict an 8000 length sequence.\n- Implementation of PrecTime Model - You can find details in this [discussion](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/459616). We also made modifications to its structure, including the addition of transformer architecture and residual connection structures. The experiments had shown that these modifications contribute to a certain improvement in the model's performance.\n\n### Post-preprocessing Trick\n\nWe used dynamic programming algorithm to deal with overlap problem. \n\nPrinciple behind this method: To achieve a high AP (Average Precision), three main criteria need to be met: Firstly, the predicted label should be sufficiently close to the actual label. Secondly, within a positive or negative tolerance range around the actual label, there should only be one predicted point. Thirdly, the score of other predicted points outside the actual label range should be lower than those within the range.\n\n```\ndef get_results_slide_window(pred, gap):\n scores = list(pred)\n stack = [0]\n dp = [-1] * len(scores)\n dp[0] = 0\n for i in range(1,len(scores)):\n if i - stack[-1] < gap:\n if scores[i] >= scores[stack[-1]]:\n stack.pop()\n if i - gap >= 0:\n if stack:\n if dp[i - gap] != stack[-1]:\n while stack and dp[i - gap] - stack[-1] < gap:\n stack.pop()\n stack.append(dp[i - gap])\n else:\n stack.append(dp[i - gap])\n stack.append(i)\n else:\n stack.append(i)\n dp[i] = stack[-1]\n return stack\n```\n\n### Ensemble\n\nOur final ensemble method essentially involved averaging different outputs. With post-processing and this ensemble method combined, our results generally follow the pattern that the more models we use or the greater the variety of models, the higher the score.\n\nSubmission Code1: [Here](https://www.kaggle.com/code/lizhecheng/detect-sleep-states-ensemble-lb-0-761-pb-0-804)\nSubmission Code2: [Here](https://www.kaggle.com/code/lizhecheng/detect-sleep-states-ensemble-lb-0-75-pb-0-8)\n\nOur submissions:\n\n| Models | LB Score | PB Score | Selected |\n| ------------------------------------------------------------ | --------- | --------- | -------- |\n| ``2 * 5 folds PrecTime + 1 * 5 folds LSTM-Unet`` | ``0.75`` | ``0.8`` | **Yes** |\n| ``2 * 5 folds PrecTime + 2 * 5 folds LSTM-Unet + 10 single models`` | ``0.759`` | ``0.803`` | **Yes** |\n| ``1 * 5 folds PrecTime + 1 fold LSTM-Unet + 10 single models`` | ``0.761`` | ``0.804`` | **No** |\n| ``1 * 5 folds PrecTime + 1 * 5 folds LSTM-Unet + 10 single models`` | ``0.759`` | ``0.803`` | **No** |\n\n\n### Other\n\nWelcome everyone to check our GitHub code, looking forward to any discussions.\n\n### Conclusion\n\n- Data Cleaning.\n- Generate New Features. \n- Use Architecture Like Conv1d, RNN, GRU, LSTM or Transformer.\n- Write Post-preprocessing for Special Metrics.\n\n## Thanks to all of my teammates for working together to gain this Silver Medal.", "2551906": "excellent work", "2551916": "excellent teammates", "2552974": "Data Structures and Algorithms implementation in play!" }, "source": "existing_discussion", "complete": true, "competition_id": 53666, "competition_slug": "child-mind-institute-detect-sleep-states", "team_id": 11061227, "solution_url": "/competitions/child-mind-institute-detect-sleep-states/writeups/avengers-39th-place-solution" }