{ "id": 438563, "title": "Bad data and labels", "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/438563", "author_name": "", "post_date": "2023-09-11T18:06:38.708841600Z", "votes": 6, "comment_count": 3, "views": 0, "content": "

As far as Ive seen it, the data belonging to nan events days are useless. They just have duplicated data.

\n
    \n
  1. Will we get such cases in test set?
    2. \n
  2. Are these just kept to make the dataset continuous?
    3. \n
", "messages": [ { "id": "2433679", "postDate": "09/11/2023 18:06:38", "content": "

As far as Ive seen it, the data belonging to nan events days are useless. They just have duplicated data.

\n
    \n
  1. Will we get such cases in test set?
    2. \n
  2. Are these just kept to make the dataset continuous?
    3. \n
", "rawMarkdown": "As far as Ive seen it, the data belonging to nan events days are useless. They just have duplicated data.\n1. Will we get such cases in test set?\n2. Are these just kept to make the dataset continuous?", "votes": null }, { "id": "2434966", "postDate": "09/12/2023 16:10:36", "content": "

Hi! The NaN cases are those when no meaningful sleep events were detected in the night, often due to non-wear (e.g., the participant took off the watch at some point). These are a common occurrence, and may exist within the test sets. The reason we coded these as NaNs rather than removing the days was so that your algorithms can also be challenged to identify when and when not to make a prediction about sleep events.

\n

Does this help?

", "rawMarkdown": "Hi! The `NaN` cases are those when no meaningful sleep events were detected in the night, often due to non-wear (e.g., the participant took off the watch at some point). These are a common occurrence, and may exist within the test sets. The reason we coded these as `NaN`s rather than removing the days was so that your algorithms can also be challenged to identify _when_ and _when not_ to make a prediction about sleep events.\n\nDoes this help?", "votes": null }, { "id": "2480603", "postDate": "10/13/2023 12:33:03", "content": "

There are also unlabeled nights that could have been labeled. So be careful when creating labels. You may assign non-wear nights to wear nights.

", "rawMarkdown": "There are also unlabeled nights that could have been labeled. So be careful when creating labels. You may assign non-wear nights to wear nights.", "votes": null }, { "id": "2483978", "postDate": "10/16/2023 06:40:59", "content": "
\n

The reason we coded these as NaNs rather than removing the days was so that your algorithms can also be challenged to identify when and when not to make a prediction about sleep events.

\n
\n

@gkiar07 Should the model predict similar NaN that exist in the test dataset for the submission?

", "rawMarkdown": "> The reason we coded these as NaNs rather than removing the days was so that your algorithms can also be challenged to identify when and when not to make a prediction about sleep events.\n\n@gkiar07 Should the model predict similar `NaN` that exist in the test dataset for the submission?", "votes": null } ], "comments": [ { "id": 2434966, "author_name": "gkiar07", "author_url": "", "post_date": "09/12/2023 16:10:36", "content": "

Hi! The NaN cases are those when no meaningful sleep events were detected in the night, often due to non-wear (e.g., the participant took off the watch at some point). These are a common occurrence, and may exist within the test sets. The reason we coded these as NaNs rather than removing the days was so that your algorithms can also be challenged to identify when and when not to make a prediction about sleep events.

\n

Does this help?

", "votes": null, "replies": [ { "id": 2483978, "author_name": "bkowshik", "author_url": "", "post_date": "10/16/2023 06:40:59", "content": "
\n

The reason we coded these as NaNs rather than removing the days was so that your algorithms can also be challenged to identify when and when not to make a prediction about sleep events.

\n
\n

@gkiar07 Should the model predict similar NaN that exist in the test dataset for the submission?

", "votes": null, "replies": [] } ] }, { "id": 2480603, "author_name": "tolgadincer", "author_url": "", "post_date": "10/13/2023 12:33:03", "content": "

There are also unlabeled nights that could have been labeled. So be careful when creating labels. You may assign non-wear nights to wear nights.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2433679": "As far as Ive seen it, the data belonging to nan events days are useless. They just have duplicated data.\n1. Will we get such cases in test set?\n2. Are these just kept to make the dataset continuous?", "2434966": "Hi! The `NaN` cases are those when no meaningful sleep events were detected in the night, often due to non-wear (e.g., the participant took off the watch at some point). These are a common occurrence, and may exist within the test sets. The reason we coded these as `NaN`s rather than removing the days was so that your algorithms can also be challenged to identify _when_ and _when not_ to make a prediction about sleep events.\n\nDoes this help?", "2480603": "There are also unlabeled nights that could have been labeled. So be careful when creating labels. You may assign non-wear nights to wear nights.", "2483978": "> The reason we coded these as NaNs rather than removing the days was so that your algorithms can also be challenged to identify when and when not to make a prediction about sleep events.\n\n@gkiar07 Should the model predict similar `NaN` that exist in the test dataset for the submission?" }, "source": "meta" }