![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Fd338a9bbdd5efe4ab2d456d8eaff86ab%2FScreenshot%202023-11-06%20at%201.06.17.png?generation=1699200744424351&alt=media\")
Hi.
\nI will share a result by pure heuristic approach called HDCZA. This approach is presented by Hees et. al.[1].
\nWithout training any models, and only utilizing z-angle feature, it scored LB=0.447, seems not bad for me.
\nLooking at the success cases and failure cases, there are a lot of FPs where no sleep logs exists.
\nSince ~37% of nights in training sequences has no sleep logs, I believe training model of predicting existence of sleep logs in the target night will get higher score in LB.
The notebook is available in [2].
\nSorry for bad visualization, but blue area shows ground truth sleep logs and orange bar shows the predicted sleep episode by HDCZA. Though it's performance is good for healthy sleep schedule, it fails to successfully predict unhealthy sleep schedule (and it produces a lot of FPs).
\nsuccess case:
\n
failure case:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Ffd678be30f6b2b9c2989c82c6fb411ce%2FScreenshot%202023-11-06%20at%201.06.24.png?generation=1699200702694394&alt=media\")
Since this approach is heuristic, there exists systematic bias to the target label.
\nCorrecting these bias with general mean of error, I can get some boost both on local AP and LB score.
\n(You can show this result in the latest version of the notebook.)
By the way, considering LB score is much higher than the local AP score, the public data might consists of easier samples comparing to the training samples.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Fdf92efe193e4df2101d0d35c094f4c84%2Ferror_hist.png?generation=1699452116883581&alt=media\")
Hi.
\nI will share a result by pure heuristic approach called HDCZA. This approach is presented by Hees et. al.[1].
\nWithout training any models, and only utilizing z-angle feature, it scored LB=0.447, seems not bad for me.
\nLooking at the success cases and failure cases, there are a lot of FPs where no sleep logs exists.
\nSince ~37% of nights in training sequences has no sleep logs, I believe training model of predicting existence of sleep logs in the target night will get higher score in LB.
The notebook is available in [2].
\nSorry for bad visualization, but blue area shows ground truth sleep logs and orange bar shows the predicted sleep episode by HDCZA. Though it's performance is good for healthy sleep schedule, it fails to successfully predict unhealthy sleep schedule (and it produces a lot of FPs).
\nsuccess case:
\n
failure case:
\n
Since this approach is heuristic, there exists systematic bias to the target label.
\nCorrecting these bias with general mean of error, I can get some boost both on local AP and LB score.
\n(You can show this result in the latest version of the notebook.)
By the way, considering LB score is much higher than the local AP score, the public data might consists of easier samples comparing to the training samples.
\n
Nice approach! A pity that this competition doesn't have an Efficiency Prize - this solution would be a strong contender.
", "rawMarkdown": "Nice approach! A pity that this competition doesn't have an Efficiency Prize - this solution would be a strong contender.", "votes": 1 }, { "id": 2514890, "postDate": "2023-11-06T15:35:58.617Z", "content": "@tatamikenn I think it makes sense, due to the fact that private LB is tested on 75% of the original dataset, whilst public LB only take 25% of that, so to be honest, possible overfitting to the public may lead to huge shakeup, which is a concern..
", "rawMarkdown": "@tatamikenn I think it makes sense, due to the fact that private LB is tested on 75% of the original dataset, whilst public LB only take 25% of that, so to be honest, possible overfitting to the public may lead to huge shakeup, which is a concern.." }, { "id": 2522199, "postDate": "2023-11-12T13:38:48.250Z", "content": "\"Since ~37% of nights in training sequences has no sleep logs, I believe training model of predicting existence of sleep logs in the target night will get higher score in LB.\" Yeah, that is something I find really weird in this completion
", "rawMarkdown": "\"Since ~37% of nights in training sequences has no sleep logs, I believe training model of predicting existence of sleep logs in the target night will get higher score in LB.\" Yeah, that is something I find really weird in this completion " }, { "id": 2517083, "postDate": "2023-11-08T07:52:40.850Z", "rawMarkdown": "", "votes": -1, "isDeleted": true, "replies": [ { "id": 2517086, "postDate": "2023-11-08T07:56:26.043Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 2514289, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-11-06T06:59:31.967000", "content": "Nice approach! A pity that this competition doesn't have an Efficiency Prize - this solution would be a strong contender.
", "votes": 1, "replies": [] }, { "id": 2514890, "author_name": "Lamal", "author_url": "", "post_date": "2023-11-06T15:35:58.617000", "content": "@tatamikenn I think it makes sense, due to the fact that private LB is tested on 75% of the original dataset, whilst public LB only take 25% of that, so to be honest, possible overfitting to the public may lead to huge shakeup, which is a concern..
", "votes": 0, "replies": [] }, { "id": 2522199, "author_name": "n00b", "author_url": "", "post_date": "2023-11-12T13:38:48.250000", "content": "\"Since ~37% of nights in training sequences has no sleep logs, I believe training model of predicting existence of sleep logs in the target night will get higher score in LB.\" Yeah, that is something I find really weird in this completion
", "votes": 0, "replies": [] }, { "id": 2517083, "author_name": "", "author_url": "", "post_date": "2023-11-08T07:52:40.850000", "content": "", "votes": -1, "replies": [ { "id": 2517086, "author_name": "", "author_url": "", "post_date": "2023-11-08T07:56:26.043000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2513723": "Hi. \n\nI will share a result by pure heuristic approach called HDCZA. This approach is presented by Hees et. al.[1]. \n\nWithout training any models, and only utilizing z-angle feature, it scored LB=0.447, seems not bad for me.\n\nLooking at the success cases and failure cases, there are a lot of FPs where no sleep logs exists.\nSince ~37% of nights in training sequences has no sleep logs, I believe training model of predicting existence of sleep logs in the target night will get higher score in LB.\n\nThe notebook is available in [2].\n\n## Visualization of success/failure cases\n\nSorry for bad visualization, but blue area shows ground truth sleep logs and orange bar shows the predicted sleep episode by HDCZA. Though it's performance is good for healthy sleep schedule, it fails to successfully predict unhealthy sleep schedule (and it produces a lot of FPs).\n\nsuccess case:\n\n\n\nfailure case:\n\n\n\n## Updates\n\nSince this approach is heuristic, there exists systematic bias to the target label.\nCorrecting these bias with general mean of error, I can get some boost both on local AP and LB score.\n(You can show this result in the latest version of the notebook.)\n\n- AP(local): 0.306 -> 0.314 (+0.008)\n- AP(LB): 0.437 -> 0.447 (+0.010)\n\nBy the way, considering LB score is much higher than the local AP score, the public data might consists of easier samples comparing to the training samples.\n\n\n\n## Reference\n\n- [1] [Estimating sleep parameters using an accelerometer without sleep diary](https://www.nature.com/articles/s41598-018-31266-z)\n- [2] [sleep: HDCZA - a pure heuristic approach LB=0.447](https://www.kaggle.com/code/tatamikenn/sleep-hdcza-a-pure-heuristic-approach-lb-0-447?scriptVersionId=149818796)", "2514289": "Nice approach! A pity that this competition doesn't have an Efficiency Prize - this solution would be a strong contender.", "2514890": "@tatamikenn I think it makes sense, due to the fact that private LB is tested on 75% of the original dataset, whilst public LB only take 25% of that, so to be honest, possible overfitting to the public may lead to huge shakeup, which is a concern..", "2522199": "\"Since ~37% of nights in training sequences has no sleep logs, I believe training model of predicting existence of sleep logs in the target night will get higher score in LB.\" Yeah, that is something I find really weird in this completion ", "2517083": "" } }