{ "id": 450468, "title": "Using CNN for Time Series data", "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/450468", "author_name": "Cody_Null", "post_date": "2023-10-24T14:10:53.098000", "votes": 12, "comment_count": 6, "views": 0, "content": "

As has been pointed out a number of times this competition seems to be more of a segmentation problem than a problem of prediction as we do not need to predict in real time. As we can see here, it would be much simpler to segment based on visuals like this than with the numeric data itself.
\n\"\"
\nFor this kagglers will have to shift their mindset in this competition away from gradient boosted models and to 1D CNN models. Of course there are many interesting ways to feed in data for this that will separate your models from the pack, I will reserve my thoughts on this for now. But for those just getting started here are a few article links I passed by in basic research for this competition.
\nhttps://towardsdatascience.com/how-to-use-convolutional-neural-networks-for-time-series-classification-56b1b0a07a57#:~:text=Research%20has%20shown%20that%20using,which%20are%20independent%20from%20time.
\nhttps://machinelearningmastery.com/cnn-models-for-human-activity-recognition-time-series-classification/
\nThese would ideally be to lay down the concept idea. Also I would highly recommend looking into @itsuki9180 's work released yesterday :
\nhttps://www.kaggle.com/code/itsuki9180/detect-sleep-states-inference
\nHere I should note he links each of his notebooks, data prep, training, and inference.

", "messages": [ { "id": 2497199, "postDate": "2023-10-24T14:10:53.100Z", "content": "

As has been pointed out a number of times this competition seems to be more of a segmentation problem than a problem of prediction as we do not need to predict in real time. As we can see here, it would be much simpler to segment based on visuals like this than with the numeric data itself.
\n\"\"
\nFor this kagglers will have to shift their mindset in this competition away from gradient boosted models and to 1D CNN models. Of course there are many interesting ways to feed in data for this that will separate your models from the pack, I will reserve my thoughts on this for now. But for those just getting started here are a few article links I passed by in basic research for this competition.
\nhttps://towardsdatascience.com/how-to-use-convolutional-neural-networks-for-time-series-classification-56b1b0a07a57#:~:text=Research%20has%20shown%20that%20using,which%20are%20independent%20from%20time.
\nhttps://machinelearningmastery.com/cnn-models-for-human-activity-recognition-time-series-classification/
\nThese would ideally be to lay down the concept idea. Also I would highly recommend looking into @itsuki9180 's work released yesterday :
\nhttps://www.kaggle.com/code/itsuki9180/detect-sleep-states-inference
\nHere I should note he links each of his notebooks, data prep, training, and inference.

", "rawMarkdown": "As has been pointed out a number of times this competition seems to be more of a segmentation problem than a problem of prediction as we do not need to predict in real time. As we can see here, it would be much simpler to segment based on visuals like this than with the numeric data itself.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13672938%2Ff18fe01c2c484d1f8ebabd0f974b54e0%2FCapture.PNG?generation=1698156353810283&alt=media)\nFor this kagglers will have to shift their mindset in this competition away from gradient boosted models and to 1D CNN models. Of course there are many interesting ways to feed in data for this that will separate your models from the pack, I will reserve my thoughts on this for now. But for those just getting started here are a few article links I passed by in basic research for this competition. \nhttps://towardsdatascience.com/how-to-use-convolutional-neural-networks-for-time-series-classification-56b1b0a07a57#:~:text=Research%20has%20shown%20that%20using,which%20are%20independent%20from%20time.\nhttps://machinelearningmastery.com/cnn-models-for-human-activity-recognition-time-series-classification/\nThese would ideally be to lay down the concept idea. Also I would highly recommend looking into @itsuki9180 's work released yesterday : \nhttps://www.kaggle.com/code/itsuki9180/detect-sleep-states-inference \nHere I should note he links each of his notebooks, data prep, training, and inference. ", "votes": 12 }, { "id": 2497294, "postDate": "2023-10-24T15:02:24.453Z", "content": "

Thanks for sharing @cody11null ! That’s a really cool idea, will check this out. We are struggling to improve our score with tree-based boosting techniques, just getting the CV takes ages to run with due to the size of the dataset and the high amounts of feature engineering 😢

", "rawMarkdown": "Thanks for sharing @cody11null ! That’s a really cool idea, will check this out. We are struggling to improve our score with tree-based boosting techniques, just getting the CV takes ages to run with due to the size of the dataset and the high amounts of feature engineering 😢", "votes": 1, "replies": [ { "id": 2497303, "postDate": "2023-10-24T15:05:44.450Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 2497422, "postDate": "2023-10-24T16:04:10.083Z", "content": "

It's possible to get good score in these problems with gradient boosting models if you use optimal data aggregated features and model parameters, but judging by my previous experience RNN models still outperform them.

", "rawMarkdown": "It's possible to get good score in these problems with gradient boosting models if you use optimal data aggregated features and model parameters, but judging by my previous experience RNN models still outperform them.", "votes": 1 } ] } ] }, { "id": 2497398, "postDate": "2023-10-24T15:50:06.280Z", "content": "

I found something close to your post here - it's one of the best solutions of Parkinson FOG competition: https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/418275

", "rawMarkdown": "I found something close to your post here - it's one of the best solutions of Parkinson FOG competition: https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/418275", "votes": 2, "replies": [ { "id": 2497404, "postDate": "2023-10-24T15:54:05.150Z", "content": "

Thank you for sharing this, this type of strategy would be very relevant here I think.

", "rawMarkdown": "Thank you for sharing this, this type of strategy would be very relevant here I think.", "votes": 1, "replies": [ { "id": 2497411, "postDate": "2023-10-24T15:57:08.503Z", "content": "

I tried this model in my earlier pipeline, and got LB 0.63 without any tuning.

", "rawMarkdown": "I tried this model in my earlier pipeline, and got LB 0.63 without any tuning.", "votes": 1 } ] } ] } ], "comments": [ { "id": 2497294, "author_name": "Geremie Yeo", "author_url": "", "post_date": "2023-10-24T15:02:24.453000", "content": "

Thanks for sharing @cody11null ! That’s a really cool idea, will check this out. We are struggling to improve our score with tree-based boosting techniques, just getting the CV takes ages to run with due to the size of the dataset and the high amounts of feature engineering 😢

", "votes": 1, "replies": [ { "id": 2497303, "author_name": "", "author_url": "", "post_date": "2023-10-24T15:05:44.450000", "content": "", "votes": 0, "replies": [ { "id": 2497422, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-10-24T16:04:10.083000", "content": "

It's possible to get good score in these problems with gradient boosting models if you use optimal data aggregated features and model parameters, but judging by my previous experience RNN models still outperform them.

", "votes": 1, "replies": [] } ] } ] }, { "id": 2497398, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-10-24T15:50:06.280000", "content": "

I found something close to your post here - it's one of the best solutions of Parkinson FOG competition: https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/418275

", "votes": 2, "replies": [ { "id": 2497404, "author_name": "Cody_Null", "author_url": "", "post_date": "2023-10-24T15:54:05.150000", "content": "

Thank you for sharing this, this type of strategy would be very relevant here I think.

", "votes": 1, "replies": [ { "id": 2497411, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-10-24T15:57:08.503000", "content": "

I tried this model in my earlier pipeline, and got LB 0.63 without any tuning.

", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2497199": "As has been pointed out a number of times this competition seems to be more of a segmentation problem than a problem of prediction as we do not need to predict in real time. As we can see here, it would be much simpler to segment based on visuals like this than with the numeric data itself.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13672938%2Ff18fe01c2c484d1f8ebabd0f974b54e0%2FCapture.PNG?generation=1698156353810283&alt=media)\nFor this kagglers will have to shift their mindset in this competition away from gradient boosted models and to 1D CNN models. Of course there are many interesting ways to feed in data for this that will separate your models from the pack, I will reserve my thoughts on this for now. But for those just getting started here are a few article links I passed by in basic research for this competition. \nhttps://towardsdatascience.com/how-to-use-convolutional-neural-networks-for-time-series-classification-56b1b0a07a57#:~:text=Research%20has%20shown%20that%20using,which%20are%20independent%20from%20time.\nhttps://machinelearningmastery.com/cnn-models-for-human-activity-recognition-time-series-classification/\nThese would ideally be to lay down the concept idea. Also I would highly recommend looking into @itsuki9180 's work released yesterday : \nhttps://www.kaggle.com/code/itsuki9180/detect-sleep-states-inference \nHere I should note he links each of his notebooks, data prep, training, and inference. ", "2497294": "Thanks for sharing @cody11null ! That’s a really cool idea, will check this out. We are struggling to improve our score with tree-based boosting techniques, just getting the CV takes ages to run with due to the size of the dataset and the high amounts of feature engineering 😢", "2497398": "I found something close to your post here - it's one of the best solutions of Parkinson FOG competition: https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/418275" } }