{ "id": 438438, "title": "Multiclass dataset (211M)", "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/438438", "author_name": "Carl McBride Ellis", "post_date": "2023-09-11T07:23:25.773000", "votes": 32, "comment_count": 17, "views": 0, "content": "

\n

Update (20 September 2023): The datasets have been updated with the new Offset Events

\n

When posting a link to the Lightweight training dataset Greg Kiar, the competition host, kindly pointed out that there will be periods during which the child has taken off their watch(!) Indeed we can see in the anglez data in this example series (03d92c9f6f8a)

\n

\"\"

\n

that could well be such watch not worn (nonwear) episodes on the right hand half of this series.

\n

Furthermore, during EDA it was noticed that for 8 of the 277 series we have absolutely no annotations of event data (i.e. no onset or wakeup timestamps) whatsoever. Hypothesis: ¿What if these 8 series are actually provided as \"baseline\" series for the nonwear class?

\n

In view of this I have created a new dataset with the name Zzzs_train_multi.parquet (size 211M) which contains the original 35 series of the dataset Zzzs_train.parquet along with the 8 potentially \"baseline\" nonwear series, which have been labelled as the new class awake=2 (thus making this a multiclass classification problem).

\n

Both the original Zzzs_train.parquet and the new Zzzs_train_multi.parquet can be found in the dataset \"Zzzs: Lightweight training dataset + target\".

\n

All the best,
\ncarl

\n

Annex: The 8 series that have been assigned the new class awake=2 are

\n
[,\n ,\n ,\n ,\n ,\n ,\n ,\n ]\n
", "messages": [ { "id": 2432876, "postDate": "2023-09-11T07:23:25.773Z", "content": "

\n

Update (20 September 2023): The datasets have been updated with the new Offset Events

\n

When posting a link to the Lightweight training dataset Greg Kiar, the competition host, kindly pointed out that there will be periods during which the child has taken off their watch(!) Indeed we can see in the anglez data in this example series (03d92c9f6f8a)

\n

\"\"

\n

that could well be such watch not worn (nonwear) episodes on the right hand half of this series.

\n

Furthermore, during EDA it was noticed that for 8 of the 277 series we have absolutely no annotations of event data (i.e. no onset or wakeup timestamps) whatsoever. Hypothesis: ¿What if these 8 series are actually provided as \"baseline\" series for the nonwear class?

\n

In view of this I have created a new dataset with the name Zzzs_train_multi.parquet (size 211M) which contains the original 35 series of the dataset Zzzs_train.parquet along with the 8 potentially \"baseline\" nonwear series, which have been labelled as the new class awake=2 (thus making this a multiclass classification problem).

\n

Both the original Zzzs_train.parquet and the new Zzzs_train_multi.parquet can be found in the dataset \"Zzzs: Lightweight training dataset + target\".

\n

All the best,
\ncarl

\n

Annex: The 8 series that have been assigned the new class awake=2 are

\n
[,\n ,\n ,\n ,\n ,\n ,\n ,\n ]\n
", "rawMarkdown": "### ~~**Update (18 September 2023):** The datasets have been updated with the new [Offset Events](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697)~~\n### **Update (20 September 2023):** The datasets have been updated with the new [Offset Events](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697)\n\nWhen posting a link to the [Lightweight training dataset](https://www.kaggle.com/datasets/carlmcbrideellis/zzzs-lightweight-training-dataset-target) Greg Kiar, the competition host, kindly pointed out that there will be periods during which the child has taken off their watch(!) Indeed we can see in the `anglez` data in this example series (`03d92c9f6f8a`) \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4051350%2Fe7e8ee9ae0f2d51afc0e728e275b11e8%2Fanglez_03d92c9f6f8a.png?generation=1694416253142436&alt=media)\n\nthat could well be such watch not worn (`nonwear`) episodes on the right hand half of this series.\n\nFurthermore, during EDA it was noticed that for 8 of the 277 series we have absolutely no annotations of event data (*i.e.* no `onset` or `wakeup` timestamps) whatsoever. **Hypothesis**: ¿What if these 8 series are actually provided as \"baseline\" series for the `nonwear` class?\n\nIn view of this I have created a new dataset with the name `Zzzs_train_multi.parquet` (size 211M) which contains the original 35 series of the dataset `Zzzs_train.parquet` along with the 8 potentially \"baseline\" nonwear series, which have been labelled as the new class `awake=2` (thus making this a multiclass classification problem).\n\nBoth the original `Zzzs_train.parquet` and the new `Zzzs_train_multi.parquet` can be found in the dataset [\"Zzzs: Lightweight training dataset + target\"](https://www.kaggle.com/datasets/carlmcbrideellis/zzzs-lightweight-training-dataset-target).\n\nAll the best,\ncarl\n\n\n**Annex**: The 8 series that have been assigned the new class `awake=2` are \n\n```python\n['0f9e60a8e56d',\n '390b487231ce',\n '2fc653ca75c7',\n 'c7b1283bb7eb',\n '89c7daa72eee',\n 'e11b9d69f856',\n 'c5d08fc3e040',\n 'a3e59c2ce3f6']\n```\n", "votes": 32 }, { "id": 2439165, "postDate": "2023-09-14T17:20:47.683Z", "content": "

To be honest, I find your approach a bit strange. Why don't just focus on the sleeping period (label=1) and treat what left as background (label=0)?

", "rawMarkdown": "To be honest, I find your approach a bit strange. Why don't just focus on the sleeping period (label=1) and treat what left as background (label=0)?", "votes": 4, "replies": [ { "id": 2439236, "postDate": "2023-09-14T17:59:18.570Z", "content": "

Dear @aphysict

\n

Indeed I have also created a binary dataset (see \"Lightweight training dataset (180M)\" with the target of awake/not awake (Zzzs_train.parquet). However looking at the data one can see there are three types of data in the dataset; awake, asleep and watch-not-worn. I posit that a classifier may be able to learn better if it is trained on clear examples of all three states, even though one is only interested in one of those states, rather than having to model the awake+notworn states as one unified class.

\n

All the best,
\ncarl

", "rawMarkdown": "Dear @aphysict \n\nIndeed I have also created a binary dataset (see [\"Lightweight training dataset (180M)\"](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/437345) with the target of awake/not awake (`Zzzs_train.parquet`). However looking at the data one can see there are three types of data in the dataset; awake, asleep and watch-not-worn. I posit that a classifier may be able to learn better if it is trained on clear examples of all three states, even though one is only interested in one of those states, rather than having to model the awake+notworn states as one unified class.\n\nAll the best,\ncarl", "votes": 5, "replies": [ { "id": 2439472, "postDate": "2023-09-14T21:37:37.533Z", "content": "

Dear @carlmcbrideellis
\nCan that hypothesis be demonstrated by the correlation between CV and LB?

", "rawMarkdown": "Dear @carlmcbrideellis\nCan that hypothesis be demonstrated by the correlation between CV and LB?", "votes": 2 }, { "id": 2439815, "postDate": "2023-09-15T06:12:26.283Z", "content": "

Dear @mitsuyasuhoshino

\n

I think one would be better off comparing the results of the confusion matrix.

\n

All the best,
\ncarl

", "rawMarkdown": "Dear @mitsuyasuhoshino \n\nI think one would be better off comparing the results of the confusion matrix.\n\nAll the best,\ncarl", "votes": 2 }, { "id": 2439838, "postDate": "2023-09-15T06:25:19.403Z", "content": "

Thank you @carlmcbrideellis. The metrics certainly seemed unique, so I'll look into the confusion matrix as well.

", "rawMarkdown": "Thank you @carlmcbrideellis. The metrics certainly seemed unique, so I'll look into the confusion matrix as well.", "votes": 2 }, { "id": 2439892, "postDate": "2023-09-15T06:33:17.077Z", "content": "

Dear @mitsuyasuhoshino

\n
\n

\"The metrics certainly seemed unique\"

\n
\n

Indeed, to be honest I am still trying to work out the best way of writing out the submission file and correctly calculating a score column so as to best work with this particular LB metric.

\n

With this in mind both the binary Zzzs_train.parquet and the multiclass Zzzs_train_multi.parquet can both be found in the dataset and choosing which to use (if any of them!) is an open question.

\n

All the best and good luck!
\ncarl

", "rawMarkdown": "Dear @mitsuyasuhoshino \n\n> \"*The metrics certainly seemed unique*\"\n\nIndeed, to be honest I am still trying to work out the best way of writing out the submission file and correctly calculating a `score` column so as to best work with this particular LB metric.\n\nWith this in mind both the binary `Zzzs_train.parquet` and the multiclass `Zzzs_train_multi.parquet` can both be found in the [dataset](https://www.kaggle.com/datasets/carlmcbrideellis/zzzs-lightweight-training-dataset-target) and choosing which to use (if any of them!) is an open question.\n\nAll the best and good luck!\ncarl\n", "votes": 2 }, { "id": 2439926, "postDate": "2023-09-15T06:53:53.203Z", "content": "

Thank you for the additional information @carlmcbrideellis. Indeed, when I saw this post, I thought that if it is enough to make predictions within 11 steps, then 5 seconds of data is too much.

", "rawMarkdown": "Thank you for the additional information @carlmcbrideellis. Indeed, when I saw [this post](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/438936), I thought that if it is enough to make predictions within 11 steps, then 5 seconds of data is too much.", "votes": 1 }, { "id": 2439934, "postDate": "2023-09-15T07:03:51.620Z", "content": "

Dear @mitsuyasuhoshino

\n

Indeed, one thing I have not tried yet is periodically undersampling the train/est datasets, for example every 5th row:

\n
df = df.iloc[::]\n
\n

which would also reduce the memory requirements.

\n

There is so much work still to do and we are only just getting started!

\n

All the best,
\ncarl

", "rawMarkdown": "Dear @mitsuyasuhoshino \n\nIndeed, one thing I have not tried yet is periodically undersampling the train/est datasets, for example every 5th row:\n\n```python\ndf = df.iloc[::5]\n```\nwhich would also reduce the memory requirements.\n\nThere is so much work still to do and we are only just getting started!\n\nAll the best,\ncarl", "votes": 2 } ] } ] }, { "id": 2438140, "postDate": "2023-09-14T04:50:57.270Z", "content": "

Thanks again Carl for your insights.

\n

It is unfortunate (or maybe part of the challenge) that the unworn sections of the timeseries are not directly labelled as such. Yours is a good idea to use the totally unworn series as baselines for predicting this. We could run a preliminary model to label unworn/worn sections of the timeseries in the first instance, and only then try to model awake/asleep over the worn sections.

\n

It is interesting to see the (regular) variability during the unworn sections of the timeseries. I wonder what that is and whether filtering this out in the worn sections of the timeseries would help.

", "rawMarkdown": "Thanks again Carl for your insights.\n\nIt is unfortunate (or maybe part of the challenge) that the unworn sections of the timeseries are not directly labelled as such. Yours is a good idea to use the totally unworn series as baselines for predicting this. We could run a preliminary model to label unworn/worn sections of the timeseries in the first instance, and only then try to model awake/asleep over the worn sections.\n\nIt is interesting to see the (regular) variability during the unworn sections of the timeseries. I wonder what that is and whether filtering this out in the worn sections of the timeseries would help.", "votes": 2, "replies": [ { "id": 2438158, "postDate": "2023-09-14T04:59:32.743Z", "content": "

Dear @nnjjpp

\n
\n

\"It is interesting to see the (regular) variability during the unworn sections of the timeseries.

\n
\n

Indeed, in particular here is series anglez for series 137b99e936ab

\n

\"\"

\n

and looking at the far RHS I would suggest that one can clearly see an almost smooth underlying diurnal component. This is something I have penciled in to look at in the future, and perhaps this could be fitted with a sinusoidal function and then subtracted from all of the anglez data?

\n

As for the periodic variations in the enmo accelerometer ¿noise due to ambient vibrations such as traffic/subway trains etc?

\n

All the best,
\ncarl

", "rawMarkdown": "Dear @nnjjpp \n\n> \"*It is interesting to see the (regular) variability during the unworn sections of the timeseries.*\n\nIndeed, in particular here is series `anglez` for series `137b99e936ab`\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4051350%2F05bda46b554363b2d7d578f594570205%2Fdiurnal.png?generation=1694667333701587&alt=media)\n\nand looking at the far RHS I would suggest that one can clearly see an almost smooth underlying diurnal component. This is something I have penciled in to look at in the future, and perhaps this could be fitted with a sinusoidal function and then subtracted from all of the `anglez` data?\n\nAs for the periodic variations in the `enmo` accelerometer ¿noise due to ambient vibrations such as traffic/subway trains *etc*? \n\nAll the best,\ncarl ", "votes": 3, "replies": [ { "id": 2488783, "postDate": "2023-10-19T13:51:39.427Z", "content": "

Hi @carlmcbrideellis I use FFT to try to denoise the data and looking at series 0f9e60a8e56d get the following:

\n

\"Denoised

\n

\"Denoised

\n

I am very new to signal processing, but the underlying pattern looks far too regular to be due to random traffic. If the device was taken off during those time periods, could it be some underlying pattern inherent to the device itself?

", "rawMarkdown": "Hi @carlmcbrideellis I use FFT to try to denoise the data and looking at series 0f9e60a8e56d get the following:\n\n![Denoised enmo](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F557179%2Faf0d390f6bea9375735774efd8f1a1bf%2FScreenshot%202023-10-19%20at%209.40.34%20AM.png?generation=1697722986719133&alt=media)\n\n![Denoised anglez](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F557179%2F506466ceecd220f5c5dfcaf01cd343dc%2FScreenshot%202023-10-19%20at%209.41.12%20AM.png?generation=1697723002214196&alt=media)\n\nI am very new to signal processing, but the underlying pattern looks far too regular to be due to random traffic. If the device was taken off during those time periods, could it be some underlying pattern inherent to the device itself?", "votes": 3 } ] } ] }, { "id": 2485403, "postDate": "2023-10-17T06:59:50.760Z", "content": "

Ref: Jupyter notebook

\n

Thank you for the data refresh @carlmcbrideellis Confirming the same series ID that don't have any events. Also, looked at how the different series were from a count of null events vs count of not-null events with a scatter plot as in the chart below.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Series with only null eventsnull events vs not-null events
\"\"\"\"
", "rawMarkdown": "Ref: [Jupyter notebook](https://www.kaggle.com/code/bkowshik/annotations-per-day-detect-sleep-states)\n\nThank you for the data refresh @carlmcbrideellis Confirming the same series ID that don't have any events. Also, looked at how the different series were from a count of `null` events vs count of `not-null` events with a scatter plot as in the chart below.\n\n| Series with only `null` events | `null` events vs `not-null` events |\n| --- | --- |\n| ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2Fce0cd4b4f94ae814a4556eb8775c4a08%2FScreenshot%202023-10-17%20at%2012.26.17.png?generation=1697525944298151&alt=media) | ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2F0327146140d14cbe74f3d85108abdfe9%2FScreenshot%202023-10-17%20at%2012.25.52.png?generation=1697525882567131&alt=media) |" }, { "id": 2435808, "postDate": "2023-09-13T07:16:01.260Z", "content": "

Hi @carlmcbrideellis. I'm starting to consider the logic of an inference model, but is there any basis for thinking that multi-class classification is better than 0/1? (namely is there a problem if I include non-sleep and non-measurement periods in 1?) Thank you.

", "rawMarkdown": "Hi @carlmcbrideellis. I'm starting to consider the logic of an inference model, but is there any basis for thinking that multi-class classification is better than 0/1? (namely is there a problem if I include non-sleep and non-measurement periods in 1?) Thank you." }, { "id": 2434450, "postDate": "2023-09-12T09:51:38.730Z", "content": "

Want some clarity on what is the expectation when there is a series like 03d92c9f6f8a in the test dataset. What I mean is that from a time-series point of view, it has the right properties for the model to detect onset / awake events but, these don't exist in the annotations and thus the model could be penalized as false positives.

\n

But, in reality these are actually good predictions from a data point of view; not sure how to handle these in the test dataset. Below are a couple of scenarios for the series 03d92c9f6f8a

\n

All events are properly annotated

\n

\"\"

\n

Only a subset of events are annotated

\n

The events on the left and right are missed during annotation.

\n

\"\"

\n

None of the events are annotated

\n

The the events predicted look legitimate. But, we don't have the annotations for them.

\n

\"\"

", "rawMarkdown": "Want some clarity on what is the expectation when there is a series like `03d92c9f6f8a` in the test dataset. What I mean is that from a time-series point of view, it has the right properties for the model to detect onset / awake events but, these don't exist in the annotations and thus the model could be penalized as false positives.\n\nBut, in reality these are actually good predictions from a data point of view; not sure how to handle these in the `test` dataset. Below are a couple of scenarios for the series `03d92c9f6f8a`\n\n### All events are properly annotated\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2F3d142b558ad716fe39e82c7267f3231e%2FScreenshot%202023-09-12%20at%2012.49.27.png?generation=1694503242189367&alt=media)\n\n### Only a subset of events are annotated\n\nThe events on the left and right are missed during annotation.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2Fc1b6c3345ab2f483d458317fb68e7dca%2FScreenshot%202023-09-12%20at%2012.49.44.png?generation=1694503263024406&alt=media)\n\n### None of the events are annotated\n\nThe the events predicted look legitimate. But, we don't have the annotations for them.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2F6f777fba4a8304113cc2437ff156656d%2FScreenshot%202023-09-12%20at%2012.49.54.png?generation=1694503301900573&alt=media)", "votes": 7, "isDeleted": true, "replies": [ { "id": 2434543, "postDate": "2023-09-12T11:00:56.570Z", "content": "

Dear @bkowshik

\n

Indeed in any classification problem metric score is only as good as its annotations/labels. This goes for the training data as well as for the hidden test data (with its corresponding hidden ground truth labels).

\n

I think at the end of the day there is little we can do about the false positives in the test data…

\n

(Note that series 03d92c9f6f8a does not form part of the Zzzs_train.parquet training set I created!)

\n

All the best,
\ncarl

", "rawMarkdown": "Dear @bkowshik \n\nIndeed in any classification problem metric score is only as good as its annotations/labels. This goes for the training data as well as for the hidden test data (with its corresponding hidden ground truth labels). \n\nI think at the end of the day there is little we can do about the false positives in the test data...\n\n(Note that series `03d92c9f6f8a` does not form part of the `Zzzs_train.parquet` training set I created!)\n\nAll the best,\ncarl", "votes": 5, "replies": [ { "id": 2435758, "postDate": "2023-09-13T06:36:39.397Z", "content": "
\n

I think at the end of the day there is little we can do about the false positives in the test data…

\n
\n

Makes so much sense. Thank you @carlmcbrideellis

", "rawMarkdown": "> I think at the end of the day there is little we can do about the false positives in the test data…\n\nMakes so much sense. Thank you @carlmcbrideellis ", "votes": 2, "isDeleted": true } ] }, { "id": 2435204, "postDate": "2023-09-12T19:08:30.350Z", "content": "

These are only ENMOs. Have you checked the anglez activity?

", "rawMarkdown": "These are only ENMOs. Have you checked the anglez activity?", "votes": 2 } ] } ], "comments": [ { "id": 2439165, "author_name": "Aphysict", "author_url": "", "post_date": "2023-09-14T17:20:47.683000", "content": "

To be honest, I find your approach a bit strange. Why don't just focus on the sleeping period (label=1) and treat what left as background (label=0)?

", "votes": 4, "replies": [ { "id": 2439236, "author_name": "Carl McBride Ellis", "author_url": "", "post_date": "2023-09-14T17:59:18.570000", "content": "

Dear @aphysict

\n

Indeed I have also created a binary dataset (see \"Lightweight training dataset (180M)\" with the target of awake/not awake (Zzzs_train.parquet). However looking at the data one can see there are three types of data in the dataset; awake, asleep and watch-not-worn. I posit that a classifier may be able to learn better if it is trained on clear examples of all three states, even though one is only interested in one of those states, rather than having to model the awake+notworn states as one unified class.

\n

All the best,
\ncarl

", "votes": 5, "replies": [ { "id": 2439472, "author_name": "mh", "author_url": "", "post_date": "2023-09-14T21:37:37.533000", "content": "

Dear @carlmcbrideellis
\nCan that hypothesis be demonstrated by the correlation between CV and LB?

", "votes": 2, "replies": [] }, { "id": 2439815, "author_name": "Carl McBride Ellis", "author_url": "", "post_date": "2023-09-15T06:12:26.283000", "content": "

Dear @mitsuyasuhoshino

\n

I think one would be better off comparing the results of the confusion matrix.

\n

All the best,
\ncarl

", "votes": 2, "replies": [] }, { "id": 2439838, "author_name": "mh", "author_url": "", "post_date": "2023-09-15T06:25:19.403000", "content": "

Thank you @carlmcbrideellis. The metrics certainly seemed unique, so I'll look into the confusion matrix as well.

", "votes": 2, "replies": [] }, { "id": 2439892, "author_name": "Carl McBride Ellis", "author_url": "", "post_date": "2023-09-15T06:33:17.077000", "content": "

Dear @mitsuyasuhoshino

\n
\n

\"The metrics certainly seemed unique\"

\n
\n

Indeed, to be honest I am still trying to work out the best way of writing out the submission file and correctly calculating a score column so as to best work with this particular LB metric.

\n

With this in mind both the binary Zzzs_train.parquet and the multiclass Zzzs_train_multi.parquet can both be found in the dataset and choosing which to use (if any of them!) is an open question.

\n

All the best and good luck!
\ncarl

", "votes": 2, "replies": [] }, { "id": 2439926, "author_name": "mh", "author_url": "", "post_date": "2023-09-15T06:53:53.203000", "content": "

Thank you for the additional information @carlmcbrideellis. Indeed, when I saw this post, I thought that if it is enough to make predictions within 11 steps, then 5 seconds of data is too much.

", "votes": 1, "replies": [] }, { "id": 2439934, "author_name": "Carl McBride Ellis", "author_url": "", "post_date": "2023-09-15T07:03:51.620000", "content": "

Dear @mitsuyasuhoshino

\n

Indeed, one thing I have not tried yet is periodically undersampling the train/est datasets, for example every 5th row:

\n
df = df.iloc[::]\n
\n

which would also reduce the memory requirements.

\n

There is so much work still to do and we are only just getting started!

\n

All the best,
\ncarl

", "votes": 2, "replies": [] } ] } ] }, { "id": 2438140, "author_name": "Nick Potter", "author_url": "", "post_date": "2023-09-14T04:50:57.270000", "content": "

Thanks again Carl for your insights.

\n

It is unfortunate (or maybe part of the challenge) that the unworn sections of the timeseries are not directly labelled as such. Yours is a good idea to use the totally unworn series as baselines for predicting this. We could run a preliminary model to label unworn/worn sections of the timeseries in the first instance, and only then try to model awake/asleep over the worn sections.

\n

It is interesting to see the (regular) variability during the unworn sections of the timeseries. I wonder what that is and whether filtering this out in the worn sections of the timeseries would help.

", "votes": 2, "replies": [ { "id": 2438158, "author_name": "Carl McBride Ellis", "author_url": "", "post_date": "2023-09-14T04:59:32.743000", "content": "

Dear @nnjjpp

\n
\n

\"It is interesting to see the (regular) variability during the unworn sections of the timeseries.

\n
\n

Indeed, in particular here is series anglez for series 137b99e936ab

\n

\"\"

\n

and looking at the far RHS I would suggest that one can clearly see an almost smooth underlying diurnal component. This is something I have penciled in to look at in the future, and perhaps this could be fitted with a sinusoidal function and then subtracted from all of the anglez data?

\n

As for the periodic variations in the enmo accelerometer ¿noise due to ambient vibrations such as traffic/subway trains etc?

\n

All the best,
\ncarl

", "votes": 3, "replies": [ { "id": 2488783, "author_name": "JJ", "author_url": "", "post_date": "2023-10-19T13:51:39.427000", "content": "

Hi @carlmcbrideellis I use FFT to try to denoise the data and looking at series 0f9e60a8e56d get the following:

\n

\"Denoised

\n

\"Denoised

\n

I am very new to signal processing, but the underlying pattern looks far too regular to be due to random traffic. If the device was taken off during those time periods, could it be some underlying pattern inherent to the device itself?

", "votes": 3, "replies": [] } ] } ] }, { "id": 2485403, "author_name": "Bhargav Kowshik", "author_url": "", "post_date": "2023-10-17T06:59:50.760000", "content": "

Ref: Jupyter notebook

\n

Thank you for the data refresh @carlmcbrideellis Confirming the same series ID that don't have any events. Also, looked at how the different series were from a count of null events vs count of not-null events with a scatter plot as in the chart below.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Series with only null eventsnull events vs not-null events
\"\"\"\"
", "votes": 0, "replies": [] }, { "id": 2435808, "author_name": "mh", "author_url": "", "post_date": "2023-09-13T07:16:01.260000", "content": "

Hi @carlmcbrideellis. I'm starting to consider the logic of an inference model, but is there any basis for thinking that multi-class classification is better than 0/1? (namely is there a problem if I include non-sleep and non-measurement periods in 1?) Thank you.

", "votes": 0, "replies": [] }, { "id": 2434450, "author_name": "Bhargav Kowshik", "author_url": "", "post_date": "2023-09-12T09:51:38.730000", "content": "

Want some clarity on what is the expectation when there is a series like 03d92c9f6f8a in the test dataset. What I mean is that from a time-series point of view, it has the right properties for the model to detect onset / awake events but, these don't exist in the annotations and thus the model could be penalized as false positives.

\n

But, in reality these are actually good predictions from a data point of view; not sure how to handle these in the test dataset. Below are a couple of scenarios for the series 03d92c9f6f8a

\n

All events are properly annotated

\n

\"\"

\n

Only a subset of events are annotated

\n

The events on the left and right are missed during annotation.

\n

\"\"

\n

None of the events are annotated

\n

The the events predicted look legitimate. But, we don't have the annotations for them.

\n

\"\"

", "votes": 7, "replies": [ { "id": 2434543, "author_name": "Carl McBride Ellis", "author_url": "", "post_date": "2023-09-12T11:00:56.570000", "content": "

Dear @bkowshik

\n

Indeed in any classification problem metric score is only as good as its annotations/labels. This goes for the training data as well as for the hidden test data (with its corresponding hidden ground truth labels).

\n

I think at the end of the day there is little we can do about the false positives in the test data…

\n

(Note that series 03d92c9f6f8a does not form part of the Zzzs_train.parquet training set I created!)

\n

All the best,
\ncarl

", "votes": 5, "replies": [ { "id": 2435758, "author_name": "Bhargav Kowshik", "author_url": "", "post_date": "2023-09-13T06:36:39.397000", "content": "
\n

I think at the end of the day there is little we can do about the false positives in the test data…

\n
\n

Makes so much sense. Thank you @carlmcbrideellis

", "votes": 2, "replies": [] } ] }, { "id": 2435204, "author_name": "Tolga", "author_url": "", "post_date": "2023-09-12T19:08:30.350000", "content": "

These are only ENMOs. Have you checked the anglez activity?

", "votes": 2, "replies": [] } ] } ], "raw_markdown_by_id": { "2432876": "### ~~**Update (18 September 2023):** The datasets have been updated with the new [Offset Events](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697)~~\n### **Update (20 September 2023):** The datasets have been updated with the new [Offset Events](https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697)\n\nWhen posting a link to the [Lightweight training dataset](https://www.kaggle.com/datasets/carlmcbrideellis/zzzs-lightweight-training-dataset-target) Greg Kiar, the competition host, kindly pointed out that there will be periods during which the child has taken off their watch(!) Indeed we can see in the `anglez` data in this example series (`03d92c9f6f8a`) \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4051350%2Fe7e8ee9ae0f2d51afc0e728e275b11e8%2Fanglez_03d92c9f6f8a.png?generation=1694416253142436&alt=media)\n\nthat could well be such watch not worn (`nonwear`) episodes on the right hand half of this series.\n\nFurthermore, during EDA it was noticed that for 8 of the 277 series we have absolutely no annotations of event data (*i.e.* no `onset` or `wakeup` timestamps) whatsoever. **Hypothesis**: ¿What if these 8 series are actually provided as \"baseline\" series for the `nonwear` class?\n\nIn view of this I have created a new dataset with the name `Zzzs_train_multi.parquet` (size 211M) which contains the original 35 series of the dataset `Zzzs_train.parquet` along with the 8 potentially \"baseline\" nonwear series, which have been labelled as the new class `awake=2` (thus making this a multiclass classification problem).\n\nBoth the original `Zzzs_train.parquet` and the new `Zzzs_train_multi.parquet` can be found in the dataset [\"Zzzs: Lightweight training dataset + target\"](https://www.kaggle.com/datasets/carlmcbrideellis/zzzs-lightweight-training-dataset-target).\n\nAll the best,\ncarl\n\n\n**Annex**: The 8 series that have been assigned the new class `awake=2` are \n\n```python\n['0f9e60a8e56d',\n '390b487231ce',\n '2fc653ca75c7',\n 'c7b1283bb7eb',\n '89c7daa72eee',\n 'e11b9d69f856',\n 'c5d08fc3e040',\n 'a3e59c2ce3f6']\n```\n", "2439165": "To be honest, I find your approach a bit strange. Why don't just focus on the sleeping period (label=1) and treat what left as background (label=0)?", "2438140": "Thanks again Carl for your insights.\n\nIt is unfortunate (or maybe part of the challenge) that the unworn sections of the timeseries are not directly labelled as such. Yours is a good idea to use the totally unworn series as baselines for predicting this. We could run a preliminary model to label unworn/worn sections of the timeseries in the first instance, and only then try to model awake/asleep over the worn sections.\n\nIt is interesting to see the (regular) variability during the unworn sections of the timeseries. I wonder what that is and whether filtering this out in the worn sections of the timeseries would help.", "2485403": "Ref: [Jupyter notebook](https://www.kaggle.com/code/bkowshik/annotations-per-day-detect-sleep-states)\n\nThank you for the data refresh @carlmcbrideellis Confirming the same series ID that don't have any events. Also, looked at how the different series were from a count of `null` events vs count of `not-null` events with a scatter plot as in the chart below.\n\n| Series with only `null` events | `null` events vs `not-null` events |\n| --- | --- |\n| ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2Fce0cd4b4f94ae814a4556eb8775c4a08%2FScreenshot%202023-10-17%20at%2012.26.17.png?generation=1697525944298151&alt=media) | ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2F0327146140d14cbe74f3d85108abdfe9%2FScreenshot%202023-10-17%20at%2012.25.52.png?generation=1697525882567131&alt=media) |", "2435808": "Hi @carlmcbrideellis. I'm starting to consider the logic of an inference model, but is there any basis for thinking that multi-class classification is better than 0/1? (namely is there a problem if I include non-sleep and non-measurement periods in 1?) Thank you.", "2434450": "Want some clarity on what is the expectation when there is a series like `03d92c9f6f8a` in the test dataset. What I mean is that from a time-series point of view, it has the right properties for the model to detect onset / awake events but, these don't exist in the annotations and thus the model could be penalized as false positives.\n\nBut, in reality these are actually good predictions from a data point of view; not sure how to handle these in the `test` dataset. Below are a couple of scenarios for the series `03d92c9f6f8a`\n\n### All events are properly annotated\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2F3d142b558ad716fe39e82c7267f3231e%2FScreenshot%202023-09-12%20at%2012.49.27.png?generation=1694503242189367&alt=media)\n\n### Only a subset of events are annotated\n\nThe events on the left and right are missed during annotation.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2Fc1b6c3345ab2f483d458317fb68e7dca%2FScreenshot%202023-09-12%20at%2012.49.44.png?generation=1694503263024406&alt=media)\n\n### None of the events are annotated\n\nThe the events predicted look legitimate. But, we don't have the annotations for them.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5210536%2F6f777fba4a8304113cc2437ff156656d%2FScreenshot%202023-09-12%20at%2012.49.54.png?generation=1694503301900573&alt=media)" } }