I started to working on this competition just after @tubotubo published his solution to learn how to solve this kind of time series problem.
\nI learned a lot from his approach and appreciate sharing great ideas!
https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940
\nMy solution is basically same as the referenced one.
\nHere are my ideas to improve the score.
The original code uses hour_sin/hour_cos features to teach the model when of the day it is looking at.
\nI changed this feature to seconds in a day so the feature has continuous value rather than digital rapidly change on the border of hours. This boosted my cv score about ~ 0.01
Considering the competition metric, I changed post processing parameters.
\nThe Larger peak distance decreases false positives that is too close to each other.
\nThe smaller peak threshold increases true positives without increasing false positives that have effects on the MAP metric (additional peaks that have relatively small scores are never referenced before the higher peaks are counted as TP/FP when calculating MAP score so the setting smaller threshold is a reasonable strategy.)
There are periodic times without any label events in training data.
\nPredictions in these period should be removed so that the number of FP decreases.
\nI compared the series features around each predicted events with that of preceding and following day, and excluded the event if it is identical to at least one of them.
\nThis had little effect on my cv score but boosted the public score about +0.007 (private: +0.016)!
Actually I have no Idea about how to improve the model architectures that is already sophisticated and worked on minor changes like post processing. I will catch up the high ranker solutions and learn how to construct models!
", "messages": [ { "id": 2550932, "postDate": "2023-12-06T12:07:17.947Z", "content": "I started to working on this competition just after @tubotubo published his solution to learn how to solve this kind of time series problem.
\nI learned a lot from his approach and appreciate sharing great ideas!
https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940
\nMy solution is basically same as the referenced one.
\nHere are my ideas to improve the score.
The original code uses hour_sin/hour_cos features to teach the model when of the day it is looking at.
\nI changed this feature to seconds in a day so the feature has continuous value rather than digital rapidly change on the border of hours. This boosted my cv score about ~ 0.01
Considering the competition metric, I changed post processing parameters.
\nThe Larger peak distance decreases false positives that is too close to each other.
\nThe smaller peak threshold increases true positives without increasing false positives that have effects on the MAP metric (additional peaks that have relatively small scores are never referenced before the higher peaks are counted as TP/FP when calculating MAP score so the setting smaller threshold is a reasonable strategy.)
There are periodic times without any label events in training data.
\nPredictions in these period should be removed so that the number of FP decreases.
\nI compared the series features around each predicted events with that of preceding and following day, and excluded the event if it is identical to at least one of them.
\nThis had little effect on my cv score but boosted the public score about +0.007 (private: +0.016)!
Actually I have no Idea about how to improve the model architectures that is already sophisticated and worked on minor changes like post processing. I will catch up the high ranker solutions and learn how to construct models!
", "rawMarkdown": "I started to working on this competition just after @tubotubo published his solution to learn how to solve this kind of time series problem.\nI learned a lot from his approach and appreciate sharing great ideas!\n\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940\n\nMy solution is basically same as the referenced one.\nHere are my ideas to improve the score.\n\n# Continuous time features\nThe original code uses hour_sin/hour_cos features to teach the model when of the day it is looking at.\nI changed this feature to seconds in a day so the feature has continuous value rather than digital rapidly change on the border of hours. This boosted my cv score about ~ 0.01\n\n# Smaller threshold and larger distance of the peaks\nConsidering the competition metric, I changed post processing parameters.\n\n- peak distance: 100steps\n- peak threshold: 0.005\n\nThe Larger peak distance decreases false positives that is too close to each other.\nThe smaller peak threshold increases true positives without increasing false positives that have effects on the MAP metric (additional peaks that have relatively small scores are never referenced before the higher peaks are counted as TP/FP when calculating MAP score so the setting smaller threshold is a reasonable strategy.)\n\n# Excluding predictions in periodic times\nThere are periodic times without any label events in training data.\nPredictions in these period should be removed so that the number of FP decreases.\nI compared the series features around each predicted events with that of preceding and following day, and excluded the event if it is identical to at least one of them.\nThis had little effect on my cv score but boosted the public score about +0.007 (private: +0.016)!\n\n# etc.\nActually I have no Idea about how to improve the model architectures that is already sophisticated and worked on minor changes like post processing. I will catch up the high ranker solutions and learn how to construct models!", "votes": 11 } ], "comments": [], "raw_markdown_by_id": { "2550932": "I started to working on this competition just after @tubotubo published his solution to learn how to solve this kind of time series problem.\nI learned a lot from his approach and appreciate sharing great ideas!\n\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/452940\n\nMy solution is basically same as the referenced one.\nHere are my ideas to improve the score.\n\n# Continuous time features\nThe original code uses hour_sin/hour_cos features to teach the model when of the day it is looking at.\nI changed this feature to seconds in a day so the feature has continuous value rather than digital rapidly change on the border of hours. This boosted my cv score about ~ 0.01\n\n# Smaller threshold and larger distance of the peaks\nConsidering the competition metric, I changed post processing parameters.\n\n- peak distance: 100steps\n- peak threshold: 0.005\n\nThe Larger peak distance decreases false positives that is too close to each other.\nThe smaller peak threshold increases true positives without increasing false positives that have effects on the MAP metric (additional peaks that have relatively small scores are never referenced before the higher peaks are counted as TP/FP when calculating MAP score so the setting smaller threshold is a reasonable strategy.)\n\n# Excluding predictions in periodic times\nThere are periodic times without any label events in training data.\nPredictions in these period should be removed so that the number of FP decreases.\nI compared the series features around each predicted events with that of preceding and following day, and excluded the event if it is identical to at least one of them.\nThis had little effect on my cv score but boosted the public score about +0.007 (private: +0.016)!\n\n# etc.\nActually I have no Idea about how to improve the model architectures that is already sophisticated and worked on minor changes like post processing. I will catch up the high ranker solutions and learn how to construct models!" } }