In this competition we are given 277 series. However, for 8 of these series we have absolutely no annotations of event data (onset or wakeup timestamps) whatsoever. Indeed, there are only 37 series of events data for which there are no missing events at all, and two of these seem to be truncated.
Without the corresponding event data it is particularly hard to untangle the sleeping from the awake data contained in the train_series data (¿maybe this could be achieved via pseudolabelling?). With this in mind I have created a small \"reduced\" training dataset consisting of a selection of just 35 of the series. A binary target column called awake has also been added. This reduced dataset could be used for prototyping/testing before progressing to the full dataset.
The reduced dataset (parquet, 180M with 13 million rows of data) can be found here:
\n\nand was created via the notebook
\nFor an EDA plotting all of the 277 series see
\n\nAll the best,
\ncarl
", "messages": [ { "id": "2426163", "postDate": "09/06/2023 12:52:08", "content": "\nIn this competition we are given 277 series. However, for 8 of these series we have absolutely no annotations of event data (onset or wakeup timestamps) whatsoever. Indeed, there are only 37 series of events data for which there are no missing events at all, and two of these seem to be truncated.
Without the corresponding event data it is particularly hard to untangle the sleeping from the awake data contained in the train_series data (¿maybe this could be achieved via pseudolabelling?). With this in mind I have created a small \"reduced\" training dataset consisting of a selection of just 35 of the series. A binary target column called awake has also been added. This reduced dataset could be used for prototyping/testing before progressing to the full dataset.
The reduced dataset (parquet, 180M with 13 million rows of data) can be found here:
\n\nand was created via the notebook
\nFor an EDA plotting all of the 277 series see
\n\nAll the best,
\ncarl
", "rawMarkdown": "### ~~**Update (18 September 2023):** The datasets have been updated with the new [Offset Events](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697)~~\n### **Update (20 September 2023):** The datasets have been updated with the new [Offset Events](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697)\n\nIn this competition we are given 277 series. However, for 8 of these series we have absolutely no annotations of event data (`onset` or `wakeup` timestamps) whatsoever. Indeed, there are only 37 series of events data for which there are no missing events at all, and two of these seem to be truncated.\n\nWithout the corresponding event data it is particularly hard to untangle the sleeping from the awake data contained in the `train_series` data (¿maybe this could be achieved via pseudolabelling?). With this in mind I have created a small \"reduced\" training dataset consisting of a selection of just 35 of the series. A binary target column called `awake` has also been added. This reduced dataset could be used for prototyping/testing before progressing to the full dataset.\n\nThe reduced dataset (parquet, 180M with 13 million rows of data) can be found here:\n\n* [Zzzs: Lightweight training dataset + target](https://www.kaggle.com/datasets/carlmcbrideellis/zzzs-lightweight-training-dataset-target)\n\nand was created via the notebook\n\n* [Zzzs: Make small starter datasets + target](https://www.kaggle.com/code/carlmcbrideellis/zzzs-make-small-starter-datasets-target) (also contains plots of the 35 selected series)\n\nFor an EDA plotting all of the 277 series see \n\n* [Zzzs: Plots of all 277 anglez and enmo series](https://www.kaggle.com/code/carlmcbrideellis/zzzs-plots-of-all-277-anglez-and-enmo-series)\n\nAll the best,\n\ncarl", "votes": null }, { "id": "2426340", "postDate": "09/06/2023 14:56:43", "content": "Cool, thanks for putting together a pruned dataset! One thing to note: detecting when events cannot be found is an important part of the challenge, so be careful about getting too comfortable working on this clean sample.
\nA possible rationale for the 8 series with no events is that, for example, the child could have taken the watch off before bed each night and so we have no meaningful data over that period. For many participants (as you note, all but 37), this is the sort of action that occurred at least once in a while, and ideally shouldn't prevent us from learning from the nights they have recorded.
\nThanks again & happy hacking!
", "rawMarkdown": "Cool, thanks for putting together a pruned dataset! One thing to note: detecting when events cannot be found is an important part of the challenge, so be careful about getting too comfortable working on this clean sample.\n\nA possible rationale for the 8 series with no events is that, for example, the child could have taken the watch off before bed each night and so we have no meaningful data over that period. For many participants (as you note, all but 37), this is the sort of action that occurred at least once in a while, and ideally shouldn't prevent us from learning from the nights they have recorded.\n\nThanks again & happy hacking!", "votes": null }, { "id": "2439376", "postDate": "09/14/2023 19:39:19", "content": "Thanks for your comments. They are helpful
", "rawMarkdown": "Thanks for your comments. They are helpful", "votes": null }, { "id": "2445534", "postDate": "09/18/2023 23:01:58", "content": "Thank you for immediately responding to the new offset event!
", "rawMarkdown": "Thank you for immediately responding to the new offset event!", "votes": null } ], "comments": [ { "id": 2426340, "author_name": "gkiar07", "author_url": "", "post_date": "09/06/2023 14:56:43", "content": "Cool, thanks for putting together a pruned dataset! One thing to note: detecting when events cannot be found is an important part of the challenge, so be careful about getting too comfortable working on this clean sample.
\nA possible rationale for the 8 series with no events is that, for example, the child could have taken the watch off before bed each night and so we have no meaningful data over that period. For many participants (as you note, all but 37), this is the sort of action that occurred at least once in a while, and ideally shouldn't prevent us from learning from the nights they have recorded.
\nThanks again & happy hacking!
", "votes": null, "replies": [ { "id": 2439376, "author_name": "pehahn", "author_url": "", "post_date": "09/14/2023 19:39:19", "content": "Thanks for your comments. They are helpful
", "votes": null, "replies": [] } ] }, { "id": 2445534, "author_name": "ryotapy", "author_url": "", "post_date": "09/18/2023 23:01:58", "content": "Thank you for immediately responding to the new offset event!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2426163": "### ~~**Update (18 September 2023):** The datasets have been updated with the new [Offset Events](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697)~~\n### **Update (20 September 2023):** The datasets have been updated with the new [Offset Events](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697)\n\nIn this competition we are given 277 series. However, for 8 of these series we have absolutely no annotations of event data (`onset` or `wakeup` timestamps) whatsoever. Indeed, there are only 37 series of events data for which there are no missing events at all, and two of these seem to be truncated.\n\nWithout the corresponding event data it is particularly hard to untangle the sleeping from the awake data contained in the `train_series` data (¿maybe this could be achieved via pseudolabelling?). With this in mind I have created a small \"reduced\" training dataset consisting of a selection of just 35 of the series. A binary target column called `awake` has also been added. This reduced dataset could be used for prototyping/testing before progressing to the full dataset.\n\nThe reduced dataset (parquet, 180M with 13 million rows of data) can be found here:\n\n* [Zzzs: Lightweight training dataset + target](https://www.kaggle.com/datasets/carlmcbrideellis/zzzs-lightweight-training-dataset-target)\n\nand was created via the notebook\n\n* [Zzzs: Make small starter datasets + target](https://www.kaggle.com/code/carlmcbrideellis/zzzs-make-small-starter-datasets-target) (also contains plots of the 35 selected series)\n\nFor an EDA plotting all of the 277 series see \n\n* [Zzzs: Plots of all 277 anglez and enmo series](https://www.kaggle.com/code/carlmcbrideellis/zzzs-plots-of-all-277-anglez-and-enmo-series)\n\nAll the best,\n\ncarl", "2426340": "Cool, thanks for putting together a pruned dataset! One thing to note: detecting when events cannot be found is an important part of the challenge, so be careful about getting too comfortable working on this clean sample.\n\nA possible rationale for the 8 series with no events is that, for example, the child could have taken the watch off before bed each night and so we have no meaningful data over that period. For many participants (as you note, all but 37), this is the sort of action that occurred at least once in a while, and ideally shouldn't prevent us from learning from the nights they have recorded.\n\nThanks again & happy hacking!", "2439376": "Thanks for your comments. They are helpful", "2445534": "Thank you for immediately responding to the new offset event!" }, "source": "meta" }