![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2Fe04f80f8cfa4f7c159e7ffe2ad894e12%2FUntitled%20(3).png?generation=1698516497196432&alt=media\")
The dataset description says it defines concrete rules for the ground-truth labelling:
\nBut I cannot see a definition for a \"night\" (e.g. a time window 20:00 - 06:00).
\nHas anyone spotted a concrete definition for this, or was it left up to the raters to decide what they thought constituted a night?
", "messages": [ { "id": 2429625, "postDate": "2023-09-08T17:25:18.983Z", "content": "The dataset description says it defines concrete rules for the ground-truth labelling:
\nBut I cannot see a definition for a \"night\" (e.g. a time window 20:00 - 06:00).
\nHas anyone spotted a concrete definition for this, or was it left up to the raters to decide what they thought constituted a night?
", "rawMarkdown": "The [dataset description](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/data) says it defines concrete rules for the ground-truth labelling:\n\n- The longest sleep window during the **night** is the only one which is recorded.\n- no more than one window should be assigned per **night**.\n- ...\n\nBut I cannot see a definition for a \"night\" (e.g. a time window 20:00 - 06:00).\n\nHas anyone spotted a concrete definition for this, or was it left up to the raters to decide what *they* thought constituted a night? \n\n\n\n", "votes": 8 }, { "id": 2432260, "postDate": "2023-09-10T17:37:27.497Z", "content": "Hi @jagofc
\nWe have labeled data only for train events, so for test set it is upto us to define night,
\nI think we should go with definition of night as given in train_events.csv file description.
\nSo extended nap is to be recorded as night.
\nI understand the ambiguity you are refering to but i guess they only want us to predict approx as many event pairs as there are 24 hr periods in test set. That is the only takeaway.
@maggiemd can you elaborate more on \"definition of night\" as asked by @jagofc ?
", "rawMarkdown": "Hi @jagofc \n\nWe have labeled data only for train events, so for test set it is upto us to define night,\nI think we should go with definition of night as given in train_events.csv file description.\nSo extended nap is to be recorded as night.\nI understand the ambiguity you are refering to but i guess they only want us to predict approx as many event pairs as there are 24 hr periods in test set. That is the only takeaway.\n\n@maggiemd can you elaborate more on \"definition of night\" as asked by @jagofc ?", "votes": 1 }, { "id": 2430727, "postDate": "2023-09-09T14:34:12.667Z", "content": "I don't really know the definition of \"night\". Although it only appears in the training data, my EDA divides the \"night\" in an arbitrary manner (namely divide time in the middle between events). Since there is no \"night\" column in the test data, we need to predict one set of \"events\" for one \"series_id\" during inference (however we cannot rule out the possibility that multiple \"nights\" are connected).
", "rawMarkdown": "I don't really know the definition of \"night\". Although it only appears in the training data, [my EDA](https://www.kaggle.com/code/mitsuyasuhoshino/cmi-detectsleepstates-eda-visualization) divides the \"night\" in an arbitrary manner (namely divide time in the middle between events). Since there is no \"night\" column in the test data, we need to predict one set of \"events\" for one \"series_id\" during inference (however we cannot rule out the possibility that multiple \"nights\" are connected).", "votes": 1, "replies": [ { "id": 2430744, "postDate": "2023-09-09T14:49:22.857Z", "content": "Thanks for your comment @mitsuyasuhoshino :) - I roughly understand the motivation behind the labelling, but there is some lack of precision there at the moment, which is what I'm hoping to get clarified.
\nE.g. if the child has an extended nap for say four hours 12:00-16:00, and then sleeps for just three hours 23:00 - 02:00 subsequently, does that mean that the onset would be 12:00 and wakeup at 16:00 in this circumstance?
\nIf I were a labeller and I were given the instructions above, then I'd need clarification from the competition team about what a night is. After all, the labels form the ground truth that our test predictions are assessed against.
", "rawMarkdown": "Thanks for your comment @mitsuyasuhoshino :) - I roughly understand the motivation behind the labelling, but there is some lack of precision there at the moment, which is what I'm hoping to get clarified.\n\nE.g. if the child has an extended nap for say four hours 12:00-16:00, and then sleeps for just three hours 23:00 - 02:00 subsequently, does that mean that the onset would be 12:00 and wakeup at 16:00 in this circumstance?\n\nIf I were a labeller and I were given the instructions above, then I'd need clarification from the competition team about what a night is. After all, the labels form the ground truth that our test predictions are assessed against.", "votes": 1, "replies": [ { "id": 2430760, "postDate": "2023-09-09T15:05:50.460Z", "content": "I understand the intent of your question @jagofc. However, \"night\" as a feature is the sleep time count information for \"series_id\", so I don't think you need to worry about the time period defined as sleep. I haven't seen it yet, but line 14510 of train_events may include a nap…
", "rawMarkdown": "I understand the intent of your question @jagofc. However, \"night\" as a feature is the sleep time count information for \"series_id\", so I don't think you need to worry about the time period defined as sleep. I haven't seen it yet, but line 14510 of train_events may include a nap..." }, { "id": 2430783, "postDate": "2023-09-09T15:15:52.083Z", "content": "I think we might be talking about different things @mitsuyasuhoshino - I'm not referring to night as a feature, or as it occurs in train_events.csv. I'm referring to the original labelling instructions and how it was conducted in the first place. They mention a reference to night (as highlighted in bold in the original question), but there is no associated definition.
Understood @jagofc. That's all I've answered.
", "rawMarkdown": "Understood @jagofc. That's all I've answered." } ] } ] }, { "id": 2503078, "postDate": "2023-10-28T18:12:46.030Z", "content": "Looked at the relationship between night and timestamp from a whether there it's NULL or not.
First looked at what percentage of series IDs have event annotations for the different nights. Roughly, there are 3 groups here:
\n23 days.43 days.44 days and beyond.
Next, calculated the number of events annotated per night split into NULL and Not NULL. When the value of step is NULL, the column step_is_null is set to True and False otherwise.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2F6a7ce371e12b7f300461a19e423d1ef6%2FUntitled%20(1).png?generation=1698516592662762&alt=media\")
Finally, converted the count to percentage of events annotated per night split into NULL and Not NULL. Clearly, as the number of days increases, percentage of event annotations with NULL value in step increases too.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2F528815e6cff457497d6d6a23311c0469%2FUntitled%20(2).png?generation=1698516716413690&alt=media\")
Only 3 out of 14507 samples had multiple events recorded per night. Which means, along the expectation, a combination of [series_id, night, event] has just one row in the events dataset. The 3 incorrect samples are below:
,night ,event ,samples\n , ,onset , \n , ,onset , \n , ,onset , \nOne of the incorrect samples is below. There is a duplicate onset event with the difference being in the step and the timezone.
,night ,event ,step ,timestamp ,hour\n , ,onset ,. ,-- ::-: , .\n , ,onset ,. ,-- ::-: , .\n , ,wakeup ,. ,-- ::-: , .\nYes. And there is a 4th case where two wakeups do not have an onset between them. Below is the code I'm using to correct these in the events dataframe. \"Buyer beware\": I may not have picked the best of each onset pair, nor patched in a best step/time for the missing onset. 😉
\n\ntrain_events = pd.read_csv(train_events_file)\n(, len(train_events.series_id.unique()),\n )\n\n\n\n\ntrain_events = train_events.drop([5362],=0)\n\n\ntrain_events = train_events.drop([7358],=0)\n\n\n\n\ntrain_events.loc[10131.5] = [, 20, , 339000, \n ]\n\ntrain_events = train_events.sort_index().reset_index(=)\n\n\ntrain_events = train_events.drop([10348],=0)\n\ntrain_events = train_events.sort_index().reset_index(=)\n\ntrain_events\nHi @jagofc
\nWe have labeled data only for train events, so for test set it is upto us to define night,
\nI think we should go with definition of night as given in train_events.csv file description.
\nSo extended nap is to be recorded as night.
\nI understand the ambiguity you are refering to but i guess they only want us to predict approx as many event pairs as there are 24 hr periods in test set. That is the only takeaway.
@maggiemd can you elaborate more on \"definition of night\" as asked by @jagofc ?
", "votes": 1, "replies": [] }, { "id": 2430727, "author_name": "mh", "author_url": "", "post_date": "2023-09-09T14:34:12.667000", "content": "I don't really know the definition of \"night\". Although it only appears in the training data, my EDA divides the \"night\" in an arbitrary manner (namely divide time in the middle between events). Since there is no \"night\" column in the test data, we need to predict one set of \"events\" for one \"series_id\" during inference (however we cannot rule out the possibility that multiple \"nights\" are connected).
", "votes": 1, "replies": [ { "id": 2430744, "author_name": "jagofc", "author_url": "", "post_date": "2023-09-09T14:49:22.857000", "content": "Thanks for your comment @mitsuyasuhoshino :) - I roughly understand the motivation behind the labelling, but there is some lack of precision there at the moment, which is what I'm hoping to get clarified.
\nE.g. if the child has an extended nap for say four hours 12:00-16:00, and then sleeps for just three hours 23:00 - 02:00 subsequently, does that mean that the onset would be 12:00 and wakeup at 16:00 in this circumstance?
\nIf I were a labeller and I were given the instructions above, then I'd need clarification from the competition team about what a night is. After all, the labels form the ground truth that our test predictions are assessed against.
", "votes": 1, "replies": [ { "id": 2430760, "author_name": "mh", "author_url": "", "post_date": "2023-09-09T15:05:50.460000", "content": "I understand the intent of your question @jagofc. However, \"night\" as a feature is the sleep time count information for \"series_id\", so I don't think you need to worry about the time period defined as sleep. I haven't seen it yet, but line 14510 of train_events may include a nap…
", "votes": 0, "replies": [] }, { "id": 2430783, "author_name": "jagofc", "author_url": "", "post_date": "2023-09-09T15:15:52.083000", "content": "I think we might be talking about different things @mitsuyasuhoshino - I'm not referring to night as a feature, or as it occurs in train_events.csv. I'm referring to the original labelling instructions and how it was conducted in the first place. They mention a reference to night (as highlighted in bold in the original question), but there is no associated definition.
Understood @jagofc. That's all I've answered.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2503078, "author_name": "Bhargav Kowshik", "author_url": "", "post_date": "2023-10-28T18:12:46.030000", "content": "Looked at the relationship between night and timestamp from a whether there it's NULL or not.
First looked at what percentage of series IDs have event annotations for the different nights. Roughly, there are 3 groups here:
\n23 days.43 days.44 days and beyond.
Next, calculated the number of events annotated per night split into NULL and Not NULL. When the value of step is NULL, the column step_is_null is set to True and False otherwise.
Finally, converted the count to percentage of events annotated per night split into NULL and Not NULL. Clearly, as the number of days increases, percentage of event annotations with NULL value in step increases too.
Only 3 out of 14507 samples had multiple events recorded per night. Which means, along the expectation, a combination of [series_id, night, event] has just one row in the events dataset. The 3 incorrect samples are below:
,night ,event ,samples\n , ,onset , \n , ,onset , \n , ,onset , \nOne of the incorrect samples is below. There is a duplicate onset event with the difference being in the step and the timezone.
,night ,event ,step ,timestamp ,hour\n , ,onset ,. ,-- ::-: , .\n , ,onset ,. ,-- ::-: , .\n , ,wakeup ,. ,-- ::-: , .\nYes. And there is a 4th case where two wakeups do not have an onset between them. Below is the code I'm using to correct these in the events dataframe. \"Buyer beware\": I may not have picked the best of each onset pair, nor patched in a best step/time for the missing onset. 😉
\n\ntrain_events = pd.read_csv(train_events_file)\n(, len(train_events.series_id.unique()),\n )\n\n\n\n\ntrain_events = train_events.drop([5362],=0)\n\n\ntrain_events = train_events.drop([7358],=0)\n\n\n\n\ntrain_events.loc[10131.5] = [, 20, , 339000, \n ]\n\ntrain_events = train_events.sort_index().reset_index(=)\n\n\ntrain_events = train_events.drop([10348],=0)\n\ntrain_events = train_events.sort_index().reset_index(=)\n\ntrain_events\n