{ "id": 459266, "title": "First Timer at Kaggle: train_series.parquet and test_series.parquet", "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/459266", "author_name": "", "post_date": "2023-12-04T13:32:49.825314800Z", "votes": 1, "comment_count": 2, "views": 0, "content": "

I have trained the model using data from the train_series.parquet. Now when I am assessing the test_series, following are my findings.

\n
    \n
  1. It has got 450 rows comprising of 3 series_id's
    \n2a. Series 'A' is of total 12 mins 2018-08-14T15:30:25-0400 - 2018-08-14T15:42:25-0400
    \n2b. Series 'B' is of total 25 mins 2018-05-31T12:12:00-0400 - 2018-05-31T12:12:25-0400 majority of rows will not be considered because record no activity (i-e device was not worn during those timesteps)
    \n2c. Series 'C' is also of only 12 mins 2018-12-18T12:45:25-0500 - 2018-12-18T12:57:25-0500
    2. \n
\n

As per the instructions; single sleep period must be 30 mins in length with allowable disturbances of 30 mins. max.

\n

That's understandable for the train_series.parquet as the series'es there are longer than 30 minutes but for the test_series either you can record onset event or wakeup event, as the other value will be considered as disturbance or not sleep being lesser than 30 mins consecutive. But even for that we have two series with only 12 mins of data.
\nAm I missing something here?

\n

which file will the submission.csv be made from? test_series.parquet or the train_series.parquet?

\n

Or Do we need to find out the events and steps for the series_ids without the information in train_event.csv?

", "messages": [ { "id": "2548500", "postDate": "12/04/2023 13:32:49", "content": "

I have trained the model using data from the train_series.parquet. Now when I am assessing the test_series, following are my findings.

\n
    \n
  1. It has got 450 rows comprising of 3 series_id's
    \n2a. Series 'A' is of total 12 mins 2018-08-14T15:30:25-0400 - 2018-08-14T15:42:25-0400
    \n2b. Series 'B' is of total 25 mins 2018-05-31T12:12:00-0400 - 2018-05-31T12:12:25-0400 majority of rows will not be considered because record no activity (i-e device was not worn during those timesteps)
    \n2c. Series 'C' is also of only 12 mins 2018-12-18T12:45:25-0500 - 2018-12-18T12:57:25-0500
    2. \n
\n

As per the instructions; single sleep period must be 30 mins in length with allowable disturbances of 30 mins. max.

\n

That's understandable for the train_series.parquet as the series'es there are longer than 30 minutes but for the test_series either you can record onset event or wakeup event, as the other value will be considered as disturbance or not sleep being lesser than 30 mins consecutive. But even for that we have two series with only 12 mins of data.
\nAm I missing something here?

\n

which file will the submission.csv be made from? test_series.parquet or the train_series.parquet?

\n

Or Do we need to find out the events and steps for the series_ids without the information in train_event.csv?

", "rawMarkdown": "I have trained the model using data from the train_series.parquet. Now when I am assessing the test_series, following are my findings.\n1. It has got 450 rows comprising of 3 series_id's\n2a. Series 'A' is of total 12 mins 2018-08-14T15:30:25-0400 - 2018-08-14T15:42:25-0400\n2b. Series 'B' is of total 25 mins 2018-05-31T12:12:00-0400 - 2018-05-31T12:12:25-0400 majority of rows will not be considered because record no activity (i-e device was not worn during those timesteps)\n2c. Series 'C' is also of only 12 mins 2018-12-18T12:45:25-0500 - 2018-12-18T12:57:25-0500\n\nAs per the instructions; single sleep period must be 30 mins in length with allowable disturbances of 30 mins. max. \n\nThat's understandable for the train_series.parquet as the series'es there are longer than 30 minutes but for the test_series either you can record onset event or wakeup event, as the other value will be considered as disturbance or not sleep being lesser than 30 mins consecutive. But even for that we have two series with only 12 mins of data.\nAm I missing something here?\n\nwhich file will the submission.csv be made from? test_series.parquet or the train_series.parquet?\n\nOr Do we need to find out the events and steps for the series_ids without the information in train_event.csv?", "votes": null }, { "id": "2548673", "postDate": "12/04/2023 15:43:27", "content": "

The test_series.parquet you have during saving is very short, but once you actually submit your notebook, it will be run on a different, hidden test_series.parquet, which has series like you would expect (longer).

\n

So yes, use test_series.parquet, but also yes, the one that is there during saving has short series.

\n

Also see here: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/447937

", "rawMarkdown": "The test_series.parquet you have during saving is very short, but once you actually submit your notebook, it will be run on a different, hidden test_series.parquet, which has series like you would expect (longer).\n\nSo yes, use test_series.parquet, but also yes, the one that is there during saving has short series.\n\nAlso see here: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/447937", "votes": null }, { "id": "2548947", "postDate": "12/04/2023 21:46:26", "content": "

Thanks for the answer @chris62

", "rawMarkdown": "Thanks for the answer @chris62", "votes": null } ], "comments": [ { "id": 2548673, "author_name": "chris62", "author_url": "", "post_date": "12/04/2023 15:43:27", "content": "

The test_series.parquet you have during saving is very short, but once you actually submit your notebook, it will be run on a different, hidden test_series.parquet, which has series like you would expect (longer).

\n

So yes, use test_series.parquet, but also yes, the one that is there during saving has short series.

\n

Also see here: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/447937

", "votes": null, "replies": [ { "id": 2548947, "author_name": "najeebz", "author_url": "", "post_date": "12/04/2023 21:46:26", "content": "

Thanks for the answer @chris62

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2548500": "I have trained the model using data from the train_series.parquet. Now when I am assessing the test_series, following are my findings.\n1. It has got 450 rows comprising of 3 series_id's\n2a. Series 'A' is of total 12 mins 2018-08-14T15:30:25-0400 - 2018-08-14T15:42:25-0400\n2b. Series 'B' is of total 25 mins 2018-05-31T12:12:00-0400 - 2018-05-31T12:12:25-0400 majority of rows will not be considered because record no activity (i-e device was not worn during those timesteps)\n2c. Series 'C' is also of only 12 mins 2018-12-18T12:45:25-0500 - 2018-12-18T12:57:25-0500\n\nAs per the instructions; single sleep period must be 30 mins in length with allowable disturbances of 30 mins. max. \n\nThat's understandable for the train_series.parquet as the series'es there are longer than 30 minutes but for the test_series either you can record onset event or wakeup event, as the other value will be considered as disturbance or not sleep being lesser than 30 mins consecutive. But even for that we have two series with only 12 mins of data.\nAm I missing something here?\n\nwhich file will the submission.csv be made from? test_series.parquet or the train_series.parquet?\n\nOr Do we need to find out the events and steps for the series_ids without the information in train_event.csv?", "2548673": "The test_series.parquet you have during saving is very short, but once you actually submit your notebook, it will be run on a different, hidden test_series.parquet, which has series like you would expect (longer).\n\nSo yes, use test_series.parquet, but also yes, the one that is there during saving has short series.\n\nAlso see here: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/447937", "2548947": "Thanks for the answer @chris62" }, "source": "meta" }