![\"img1\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1029534%2F08afe5b77341e9b0d015f1d245a3e3f6%2FScreenshot%202023-09-24%20202953.png?generation=1695562623338597&alt=media\")
Hi all participants and Kaggle team,
\nFrom here and here, we know that the 2nd data update (+ rescoring) has been happened
\nBut according to my exploration here, there still some \"onset but not wakeup\" and \"not onset but wakeup\" events, even after the data update round 2
\n\n\n\"onset but not wakeup\" events:
\n\n
\n- Series ID 0ce74d6d2106, night 20
\n
\n\nnot onset but wakeup events:
\n\n
\n- Series ID 154fe824ed87, night 30
\n- Series ID 44a41bba1ee7, night 10
\n- Series ID efbfc4526d58, night 7
\n- Series ID f8a8da8bdd00, night 17
\n
![\"img2\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1029534%2F0fd202e721e723f85568bde72fed6a41%2FUntitled.png?generation=1695562637942255&alt=media\")
\n\nQuestions:
\n\n
\n- Will this be fixed from the Kaggle team side?
\n- How do you guys handle it to your solution pipeline?
\n
Many thanks
", "messages": [ { "id": "2454008", "postDate": "09/24/2023 13:38:44", "content": "Hi all participants and Kaggle team,
\nFrom here and here, we know that the 2nd data update (+ rescoring) has been happened
\nBut according to my exploration here, there still some \"onset but not wakeup\" and \"not onset but wakeup\" events, even after the data update round 2
\n\n\n\"onset but not wakeup\" events:
\n\n
\n- Series ID 0ce74d6d2106, night 20
\n
\n\nnot onset but wakeup events:
\n\n
\n- Series ID 154fe824ed87, night 30
\n- Series ID 44a41bba1ee7, night 10
\n- Series ID efbfc4526d58, night 7
\n- Series ID f8a8da8bdd00, night 17
\n
\n\nQuestions:
\n\n
\n- Will this be fixed from the Kaggle team side?
\n- How do you guys handle it to your solution pipeline?
\n
Many thanks
", "rawMarkdown": "Hi all participants and Kaggle team,\n\nFrom [here](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697#2448756) and [here](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697#2446620), we know that the 2nd data update (+ rescoring) has been happened\n\nBut according to my exploration [here](https://www.kaggle.com/code/rimbax/detection-plots-of-all-277-series-with-flags/notebook#%22sleep-but-not-wakeup%22-Events), there still some **\"onset but not wakeup\"** and **\"not onset but wakeup\"** events, even after the data update round 2\n\n> **\"onset but not wakeup\"** events:\n1. Series ID 0ce74d6d2106, night 20\n\n\n\n> **not onset but wakeup** events:\n- Series ID 154fe824ed87, night 30\n- Series ID 44a41bba1ee7, night 10\n- Series ID efbfc4526d58, night 7\n- Series ID f8a8da8bdd00, night 17\n\n\n\n> Questions:\n1. Will this be fixed from the Kaggle team side?\n2. How do you guys handle it to your solution pipeline?\n\nMany thanks", "votes": null }, { "id": "2456062", "postDate": "09/26/2023 00:07:27", "content": "The train events file does list a wakeup for night 20 of series 0ce74d6d2106 -- it just assigns NaN to its step and timestamp. (I agree that it looks like that wakeup is behaving 'normally' and should be listed around step 335835.) Series f8a8da8bdd00 night 17 onset also looks 'clean' (around 284158) but is assigned NaN.
\nThese are not the kind of errors that the updates were fixing, e.g., where an onset was listed in events right after another onset. Instead this is more like the case where a reasonable looking night is assigned NaNs for onset and wakeup. In the cases here, for whatever reason, the rater did not enter a value for the wakeup of night 20 or onset of night 17. Correcting this is really asking for a re-rating which I don't think the sponsors are intending to do.
\nThanks for pointing out that there are some cases of train events listing only one of an onset-wakeup pair as being NaN.
", "rawMarkdown": "The train events file *does* list a wakeup for night 20 of series 0ce74d6d2106 -- it just assigns NaN to its step and timestamp. (I agree that it looks like that wakeup is behaving 'normally' and should be listed around step 335835.) Series f8a8da8bdd00 night 17 onset also looks 'clean' (around 284158) but is assigned NaN. \n\nThese are not the kind of errors that the updates were fixing, e.g., where an onset was listed in events right after another onset. Instead this is more like the case where a reasonable looking night is assigned NaNs for onset and wakeup. In the cases here, for whatever reason, the rater did not enter a value for the wakeup of night 20 or onset of night 17. Correcting this is really asking for a re-rating which I don't think the sponsors are intending to do.\n\nThanks for pointing out that there are some cases of train events listing only one of an onset-wakeup pair as being NaN.", "votes": null }, { "id": "2456191", "postDate": "09/26/2023 04:13:56", "content": "This is my personal opinion, but I don't think filling in NaN values in correct data is prohibited (on the contrary, any changes to existing event data are prohibited). There are about 5,000 of them, so it's quite a laborious process, but it may improve accuracy.
", "rawMarkdown": "This is my personal opinion, but I don't think filling in NaN values in correct data is prohibited (on the contrary, any changes to existing event data are prohibited). There are about 5,000 of them, so it's quite a laborious process, but it may improve accuracy.", "votes": null }, { "id": "2456560", "postDate": "09/26/2023 09:22:20", "content": "Another techniques that can be leveraged is semi-supervised learning, i.e. use the trained model to predict some NaN events, and then reuse it for training.
", "rawMarkdown": "Another techniques that can be leveraged is semi-supervised learning, i.e. use the trained model to predict some NaN events, and then reuse it for training.", "votes": null }, { "id": "2456581", "postDate": "09/26/2023 09:47:45", "content": "@rimbax That's right. I'll share if I get any good results.
", "rawMarkdown": "rimbax That's right. I'll share if I get any good results.", "votes": null }, { "id": "2460052", "postDate": "09/28/2023 14:49:47", "content": "Hi all, I attached the incorrect events I mentioned above. My current treatment is to remove them in the training set. Complete charts can be seen here: LINK
\n![\"incorrect-events\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1029534%2F9d146114e4978c9d11ac3c4156bf2e98%2Fincorrect-events.png?generation=1695912577187845&alt=media\")
Hi @dan3dewey ,
\nI have attached the chart of incorrect events I mentioned above to ignite more discussions. Thank you!
", "rawMarkdown": "Hi @dan3dewey ,\n\nI have attached the chart of incorrect events I mentioned above to ignite more discussions. Thank you!", "votes": null }, { "id": "2460269", "postDate": "09/28/2023 17:01:14", "content": "Nice visuals :) I see there is a missing green line and 4 missing red ones are missing. See @ishicamo 's post quoting the competition guidlines - it seems this is not supposed to happen. However it is only 5 events so not a large effect on AP values. Or do you have an analysis scheme that gets messed up by these missing events?
", "rawMarkdown": "Nice visuals :) I see there is a missing green line and 4 missing red ones are missing. See @ishicamo 's [post quoting the competition guidlines](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697#2457459) - it seems this is not supposed to happen. However it is only 5 events so not a large effect on AP values. Or do you have an analysis scheme that gets messed up by these missing events?", "votes": null } ], "comments": [ { "id": 2456062, "author_name": "dan3dewey", "author_url": "", "post_date": "09/26/2023 00:07:27", "content": "The train events file does list a wakeup for night 20 of series 0ce74d6d2106 -- it just assigns NaN to its step and timestamp. (I agree that it looks like that wakeup is behaving 'normally' and should be listed around step 335835.) Series f8a8da8bdd00 night 17 onset also looks 'clean' (around 284158) but is assigned NaN.
\nThese are not the kind of errors that the updates were fixing, e.g., where an onset was listed in events right after another onset. Instead this is more like the case where a reasonable looking night is assigned NaNs for onset and wakeup. In the cases here, for whatever reason, the rater did not enter a value for the wakeup of night 20 or onset of night 17. Correcting this is really asking for a re-rating which I don't think the sponsors are intending to do.
\nThanks for pointing out that there are some cases of train events listing only one of an onset-wakeup pair as being NaN.
", "votes": null, "replies": [ { "id": 2460057, "author_name": "rimbax", "author_url": "", "post_date": "09/28/2023 14:51:29", "content": "Hi @dan3dewey ,
\nI have attached the chart of incorrect events I mentioned above to ignite more discussions. Thank you!
", "votes": null, "replies": [] } ] }, { "id": 2456191, "author_name": "mitsuyasuhoshino", "author_url": "", "post_date": "09/26/2023 04:13:56", "content": "This is my personal opinion, but I don't think filling in NaN values in correct data is prohibited (on the contrary, any changes to existing event data are prohibited). There are about 5,000 of them, so it's quite a laborious process, but it may improve accuracy.
", "votes": null, "replies": [ { "id": 2456560, "author_name": "rimbax", "author_url": "", "post_date": "09/26/2023 09:22:20", "content": "Another techniques that can be leveraged is semi-supervised learning, i.e. use the trained model to predict some NaN events, and then reuse it for training.
", "votes": null, "replies": [ { "id": 2456581, "author_name": "mitsuyasuhoshino", "author_url": "", "post_date": "09/26/2023 09:47:45", "content": "@rimbax That's right. I'll share if I get any good results.
", "votes": null, "replies": [] } ] } ] }, { "id": 2460052, "author_name": "rimbax", "author_url": "", "post_date": "09/28/2023 14:49:47", "content": "Hi all, I attached the incorrect events I mentioned above. My current treatment is to remove them in the training set. Complete charts can be seen here: LINK
\n
Nice visuals :) I see there is a missing green line and 4 missing red ones are missing. See @ishicamo 's post quoting the competition guidlines - it seems this is not supposed to happen. However it is only 5 events so not a large effect on AP values. Or do you have an analysis scheme that gets messed up by these missing events?
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2454008": "Hi all participants and Kaggle team,\n\nFrom [here](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697#2448756) and [here](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697#2446620), we know that the 2nd data update (+ rescoring) has been happened\n\nBut according to my exploration [here](https://www.kaggle.com/code/rimbax/detection-plots-of-all-277-series-with-flags/notebook#%22sleep-but-not-wakeup%22-Events), there still some **\"onset but not wakeup\"** and **\"not onset but wakeup\"** events, even after the data update round 2\n\n> **\"onset but not wakeup\"** events:\n1. Series ID 0ce74d6d2106, night 20\n\n\n\n> **not onset but wakeup** events:\n- Series ID 154fe824ed87, night 30\n- Series ID 44a41bba1ee7, night 10\n- Series ID efbfc4526d58, night 7\n- Series ID f8a8da8bdd00, night 17\n\n\n\n> Questions:\n1. Will this be fixed from the Kaggle team side?\n2. How do you guys handle it to your solution pipeline?\n\nMany thanks", "2456062": "The train events file *does* list a wakeup for night 20 of series 0ce74d6d2106 -- it just assigns NaN to its step and timestamp. (I agree that it looks like that wakeup is behaving 'normally' and should be listed around step 335835.) Series f8a8da8bdd00 night 17 onset also looks 'clean' (around 284158) but is assigned NaN. \n\nThese are not the kind of errors that the updates were fixing, e.g., where an onset was listed in events right after another onset. Instead this is more like the case where a reasonable looking night is assigned NaNs for onset and wakeup. In the cases here, for whatever reason, the rater did not enter a value for the wakeup of night 20 or onset of night 17. Correcting this is really asking for a re-rating which I don't think the sponsors are intending to do.\n\nThanks for pointing out that there are some cases of train events listing only one of an onset-wakeup pair as being NaN.", "2456191": "This is my personal opinion, but I don't think filling in NaN values in correct data is prohibited (on the contrary, any changes to existing event data are prohibited). There are about 5,000 of them, so it's quite a laborious process, but it may improve accuracy.", "2456560": "Another techniques that can be leveraged is semi-supervised learning, i.e. use the trained model to predict some NaN events, and then reuse it for training.", "2456581": "rimbax That's right. I'll share if I get any good results.", "2460052": "Hi all, I attached the incorrect events I mentioned above. My current treatment is to remove them in the training set. Complete charts can be seen here: [LINK](https://www.kaggle.com/code/rimbax/detection-plots-of-all-277-series-with-flags)\n\n", "2460057": "Hi @dan3dewey ,\n\nI have attached the chart of incorrect events I mentioned above to ignite more discussions. Thank you!", "2460269": "Nice visuals :) I see there is a missing green line and 4 missing red ones are missing. See @ishicamo 's [post quoting the competition guidlines](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697#2457459) - it seems this is not supposed to happen. However it is only 5 events so not a large effect on AP values. Or do you have an analysis scheme that gets messed up by these missing events?" }, "source": "meta" }