Anyone has prior experience of what features should be used in time series data?
\nPrimarily when the angle of attack is by using tree-based models rather than sequential model (e.g. LSTM).
\nWhat kind of features have big predictive power?
\nIn this case, hour of the timestamp should be very predictive since it is most likely related with sleep time.
But here, anglez_x_enmo is used, as well as some rolling statistics. And yet it's very powerful by looking into the public LB score.
Any reason/mathematical intuition behind this?
\nWhat about rolling stdev and any other rolling statistics?
\nLet's discuss.
\nP.S. I have also created a notebook for Statistical Feature Engineering here, containing some possible statistical features that can be crafted. One can try to use it and tell us here what is the resulting score in public LB.
", "messages": [ { "id": "2431815", "postDate": "09/10/2023 12:29:18", "content": "Anyone has prior experience of what features should be used in time series data?
\nPrimarily when the angle of attack is by using tree-based models rather than sequential model (e.g. LSTM).
\nWhat kind of features have big predictive power?
\nIn this case, hour of the timestamp should be very predictive since it is most likely related with sleep time.
But here, anglez_x_enmo is used, as well as some rolling statistics. And yet it's very powerful by looking into the public LB score.
Any reason/mathematical intuition behind this?
\nWhat about rolling stdev and any other rolling statistics?
\nLet's discuss.
\nP.S. I have also created a notebook for Statistical Feature Engineering here, containing some possible statistical features that can be crafted. One can try to use it and tell us here what is the resulting score in public LB.
", "rawMarkdown": "Anyone has prior experience of what features should be used in time series data?\n\nPrimarily when the angle of attack is by using tree-based models rather than sequential model (e.g. LSTM).\n\nWhat kind of features have big predictive power?\n\nIn this case, `hour` of the timestamp should be very predictive since it is most likely related with sleep time.\n\nBut [here](https://www.kaggle.com/code/carlmcbrideellis/zzzs-random-forest-model-starter), `anglez_x_enmo` is used, as well as some rolling statistics. And yet it's very powerful by looking into the public LB score.\n\nAny reason/mathematical intuition behind this?\n\nWhat about rolling stdev and any other rolling statistics?\n\nLet's discuss.\n\nP.S. I have also created a notebook for [Statistical Feature Engineering here](https://www.kaggle.com/code/rimbax/statistical-feature-engineering), containing some possible statistical features that can be crafted. One can try to use it and tell us here what is the resulting score in public LB.", "votes": null }, { "id": "2431894", "postDate": "09/10/2023 13:39:45", "content": "Calendar features are a commonly used method for predicting time series data. Is it a weekday or a holiday? What day of the week is it? Are there any other events? is. Even when it comes to children's sleep, it is thought that it is influenced by the lifestyle of adults (parents).
\nRegarding onset and wakeup, \"what time\" is more likely to be effective.
", "rawMarkdown": "Calendar features are a commonly used method for predicting time series data. Is it a weekday or a holiday? What day of the week is it? Are there any other events? is. Even when it comes to children's sleep, it is thought that it is influenced by the lifestyle of adults (parents).\n\nRegarding onset and wakeup, \"what time\" is more likely to be effective.", "votes": null }, { "id": "2431900", "postDate": "09/10/2023 13:42:38", "content": "Nice insight. What about a flag whether the timestamp is in daylight time saving? Is it useful?
\nI am not living in country with DST, no clue about it.
", "rawMarkdown": "Nice insight. What about a flag whether the timestamp is in daylight time saving? Is it useful?\n\nI am not living in country with DST, no clue about it.", "votes": null }, { "id": "2431913", "postDate": "09/10/2023 13:47:12", "content": "If your dataset has enough memory, I think it would be beneficial to increase the number of features for huge time series data like this. Good luck @rimbax.
", "rawMarkdown": "If your dataset has enough memory, I think it would be beneficial to increase the number of features for huge time series data like this. Good luck @rimbax.", "votes": null }, { "id": "2431916", "postDate": "09/10/2023 13:49:56", "content": "Basically the more data you feed, the stronger the model, right? Haha
\nThanks. Good luck for you too @mitsuyasuhoshino
", "rawMarkdown": "Basically the more data you feed, the stronger the model, right? Haha\n\nThanks. Good luck for you too @mitsuyasuhoshino", "votes": null }, { "id": "2432393", "postDate": "09/10/2023 19:58:14", "content": "Some features derived from timestamps (e.g., hour) may work best if they're transformed into periodic signals because your model needs to know that hour 23 is more similar to hour 0 than hour 19.
See the time subheading under \"Feature Engineering\" of this tensorflow tutorial for an example.
", "rawMarkdown": "Some features derived from timestamps (e.g., `hour`) may work best if they're transformed into periodic signals because your model needs to know that hour 23 is more similar to hour 0 than hour 19. \n\nSee the time subheading under \"Feature Engineering\" of [this tensorflow tutorial](https://www.tensorflow.org/tutorials/structured_data/time_series#feature_engineering) for an example.", "votes": null }, { "id": "2432477", "postDate": "09/10/2023 21:26:53", "content": "I did this notebook with some ideas for feature engineering:
\nfeat eng ideas 60% memory reduction
\nWith statistics, lag and lead transformations and reduce memory usage.
In ML table data, it is said that overfitting occurs unless the number of features is approximately 1/10 or less of the number of rows. In this case, the number of rows is huge, so I think there will be no problem even if the number of features is increased considerably as long as the validity of the CV-LB correlation is checked.
", "rawMarkdown": "In ML table data, it is said that overfitting occurs unless the number of features is approximately 1/10 or less of the number of rows. In this case, the number of rows is huge, so I think there will be no problem even if the number of features is increased considerably as long as the validity of the CV-LB correlation is checked.", "votes": null }, { "id": "2433545", "postDate": "09/11/2023 16:45:30", "content": "@renatoreggiani - I really like the work you did with the extraction of additional features. May I ask how you were able to pull the data into memory (first two lines)? When I try to read the Parquet data, I run out of memory on my notebook on Kaggle. Have you migrated outside of Kaggle and onto some other platform?
\nThanks
", "rawMarkdown": "renatoreggiani - I really like the work you did with the extraction of additional features. May I ask how you were able to pull the data into memory (first two lines)? When I try to read the Parquet data, I run out of memory on my notebook on Kaggle. Have you migrated outside of Kaggle and onto some other platform?\n\nThanks", "votes": null }, { "id": "2433854", "postDate": "09/11/2023 22:46:10", "content": "Hi @nadereafshar , the train data is a modified dataset from this idea
", "rawMarkdown": "Hi @nadereafshar , the train data is a modified dataset from this [idea](https://www.kaggle.com/datasets/carlmcbrideellis/zzzs-lightweight-training-dataset-target)", "votes": null }, { "id": "2434537", "postDate": "09/12/2023 10:57:52", "content": "for this case you can transform to:
\ndf['hour_sin'] = np.sin(2 * np.pi * df['hour'] / 24)
\ndf['hour_cos'] = np.cos(2 * np.pi * df['hour'] / 24)
\nalas, for me they do not improve the score much
Calendar features are a commonly used method for predicting time series data. Is it a weekday or a holiday? What day of the week is it? Are there any other events? is. Even when it comes to children's sleep, it is thought that it is influenced by the lifestyle of adults (parents).
\nRegarding onset and wakeup, \"what time\" is more likely to be effective.
", "votes": null, "replies": [ { "id": 2431900, "author_name": "rimbax", "author_url": "", "post_date": "09/10/2023 13:42:38", "content": "Nice insight. What about a flag whether the timestamp is in daylight time saving? Is it useful?
\nI am not living in country with DST, no clue about it.
", "votes": null, "replies": [ { "id": 2431913, "author_name": "mitsuyasuhoshino", "author_url": "", "post_date": "09/10/2023 13:47:12", "content": "If your dataset has enough memory, I think it would be beneficial to increase the number of features for huge time series data like this. Good luck @rimbax.
", "votes": null, "replies": [ { "id": 2431916, "author_name": "rimbax", "author_url": "", "post_date": "09/10/2023 13:49:56", "content": "Basically the more data you feed, the stronger the model, right? Haha
\nThanks. Good luck for you too @mitsuyasuhoshino
", "votes": null, "replies": [ { "id": 2432512, "author_name": "mitsuyasuhoshino", "author_url": "", "post_date": "09/10/2023 23:16:36", "content": "In ML table data, it is said that overfitting occurs unless the number of features is approximately 1/10 or less of the number of rows. In this case, the number of rows is huge, so I think there will be no problem even if the number of features is increased considerably as long as the validity of the CV-LB correlation is checked.
", "votes": null, "replies": [] } ] } ] } ] } ] }, { "id": 2432393, "author_name": "quillen", "author_url": "", "post_date": "09/10/2023 19:58:14", "content": "Some features derived from timestamps (e.g., hour) may work best if they're transformed into periodic signals because your model needs to know that hour 23 is more similar to hour 0 than hour 19.
See the time subheading under \"Feature Engineering\" of this tensorflow tutorial for an example.
", "votes": null, "replies": [ { "id": 2434537, "author_name": "josipvrdoljak", "author_url": "", "post_date": "09/12/2023 10:57:52", "content": "for this case you can transform to:
\ndf['hour_sin'] = np.sin(2 * np.pi * df['hour'] / 24)
\ndf['hour_cos'] = np.cos(2 * np.pi * df['hour'] / 24)
\nalas, for me they do not improve the score much
I did this notebook with some ideas for feature engineering:
\nfeat eng ideas 60% memory reduction
\nWith statistics, lag and lead transformations and reduce memory usage.
@renatoreggiani - I really like the work you did with the extraction of additional features. May I ask how you were able to pull the data into memory (first two lines)? When I try to read the Parquet data, I run out of memory on my notebook on Kaggle. Have you migrated outside of Kaggle and onto some other platform?
\nThanks
", "votes": null, "replies": [ { "id": 2433854, "author_name": "renatoreggiani", "author_url": "", "post_date": "09/11/2023 22:46:10", "content": "Hi @nadereafshar , the train data is a modified dataset from this idea
", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2431815": "Anyone has prior experience of what features should be used in time series data?\n\nPrimarily when the angle of attack is by using tree-based models rather than sequential model (e.g. LSTM).\n\nWhat kind of features have big predictive power?\n\nIn this case, `hour` of the timestamp should be very predictive since it is most likely related with sleep time.\n\nBut [here](https://www.kaggle.com/code/carlmcbrideellis/zzzs-random-forest-model-starter), `anglez_x_enmo` is used, as well as some rolling statistics. And yet it's very powerful by looking into the public LB score.\n\nAny reason/mathematical intuition behind this?\n\nWhat about rolling stdev and any other rolling statistics?\n\nLet's discuss.\n\nP.S. I have also created a notebook for [Statistical Feature Engineering here](https://www.kaggle.com/code/rimbax/statistical-feature-engineering), containing some possible statistical features that can be crafted. One can try to use it and tell us here what is the resulting score in public LB.", "2431894": "Calendar features are a commonly used method for predicting time series data. Is it a weekday or a holiday? What day of the week is it? Are there any other events? is. Even when it comes to children's sleep, it is thought that it is influenced by the lifestyle of adults (parents).\n\nRegarding onset and wakeup, \"what time\" is more likely to be effective.", "2431900": "Nice insight. What about a flag whether the timestamp is in daylight time saving? Is it useful?\n\nI am not living in country with DST, no clue about it.", "2431913": "If your dataset has enough memory, I think it would be beneficial to increase the number of features for huge time series data like this. Good luck @rimbax.", "2431916": "Basically the more data you feed, the stronger the model, right? Haha\n\nThanks. Good luck for you too @mitsuyasuhoshino", "2432393": "Some features derived from timestamps (e.g., `hour`) may work best if they're transformed into periodic signals because your model needs to know that hour 23 is more similar to hour 0 than hour 19. \n\nSee the time subheading under \"Feature Engineering\" of [this tensorflow tutorial](https://www.tensorflow.org/tutorials/structured_data/time_series#feature_engineering) for an example.", "2432477": "I did this notebook with some ideas for feature engineering:\n[feat eng ideas 60% memory reduction](https://www.kaggle.com/code/renatoreggiani/feat-eng-ideas-60-memory-reduction-cmi?scriptVersionId=142568316)\nWith statistics, lag and lead transformations and reduce memory usage.", "2432512": "In ML table data, it is said that overfitting occurs unless the number of features is approximately 1/10 or less of the number of rows. In this case, the number of rows is huge, so I think there will be no problem even if the number of features is increased considerably as long as the validity of the CV-LB correlation is checked.", "2433545": "renatoreggiani - I really like the work you did with the extraction of additional features. May I ask how you were able to pull the data into memory (first two lines)? When I try to read the Parquet data, I run out of memory on my notebook on Kaggle. Have you migrated outside of Kaggle and onto some other platform?\n\nThanks", "2433854": "Hi @nadereafshar , the train data is a modified dataset from this [idea](https://www.kaggle.com/datasets/carlmcbrideellis/zzzs-lightweight-training-dataset-target)", "2434537": "for this case you can transform to: \ndf['hour_sin'] = np.sin(2 * np.pi * df['hour'] / 24)\ndf['hour_cos'] = np.cos(2 * np.pi * df['hour'] / 24)\nalas, for me they do not improve the score much" }, "source": "meta" }