It's no secret that sleep patterns are highly dependent on the length of daylight hours, which in turn varies from season to season. In one of my experiments, I added season (and month as well) as a feature, and its feature importance was quite high (among the top 10 for lightgbm). However, the LB score of the new model was lower than that of a similar model without season. This led me to speculate about the dissimilarity of the data distributions by season on the train and test datasets.
\nAfter examining the distribution of data in train by season, I noticed that they are not evenly distributed: we have significantly less data for the summer period. I would venture to guess that the remaining summer data could end up in test and make up a significant proportion of it.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6366906%2F27ff682a65325cfd726a6313f528f49a%2Fseason_imbalance.png?generation=1697314945031551&alt=media\")
Have you tried undersampling based on month/season? If so, how did it affect the LB score?
", "messages": [ { "id": 2482277, "postDate": "2023-10-14T20:30:08.803Z", "content": "It's no secret that sleep patterns are highly dependent on the length of daylight hours, which in turn varies from season to season. In one of my experiments, I added season (and month as well) as a feature, and its feature importance was quite high (among the top 10 for lightgbm). However, the LB score of the new model was lower than that of a similar model without season. This led me to speculate about the dissimilarity of the data distributions by season on the train and test datasets.
\nAfter examining the distribution of data in train by season, I noticed that they are not evenly distributed: we have significantly less data for the summer period. I would venture to guess that the remaining summer data could end up in test and make up a significant proportion of it.
\n
Have you tried undersampling based on month/season? If so, how did it affect the LB score?
", "rawMarkdown": "It's no secret that sleep patterns are highly dependent on the length of daylight hours, which in turn varies from season to season. In one of my experiments, I added season (and month as well) as a feature, and its feature importance was quite high (among the top 10 for lightgbm). However, the LB score of the new model was lower than that of a similar model without season. This led me to speculate about the dissimilarity of the data distributions by season on the train and test datasets. \nAfter [examining](https://www.kaggle.com/code/andreytikhomirov/eda-fe-introducing-seasons) the distribution of data in train by season, I noticed that they are not evenly distributed: we have significantly less data for the summer period. I would venture to guess that the remaining summer data could end up in test and make up a significant proportion of it.\n\n\n\nHave you tried undersampling based on month/season? If so, how did it affect the LB score?", "votes": 16 }, { "id": 2485422, "postDate": "2023-10-17T07:23:05.310Z", "content": "Very interesting insight. Thank you for sharing @andreytikhomirov
\nIt's only in this competition that I learnt and have been using Polars, previously I worked only with Pandas. I see that you use Dask and I looked up quickly for the differences between the libraries and found the following post. My question is since submission on Kaggle does not have distributed computing support, wouldn't Polars be a better choice for this competition?
\n\nI agree, Polars seems to be the best choice to use in the solution. In my EDA I used Dask only for specific histograms, because the same code written in pandas was much slower.
", "rawMarkdown": "I agree, Polars seems to be the best choice to use in the solution. In my EDA I used Dask only for specific histograms, because the same code written in pandas was much slower. " } ] }, { "id": 2482557, "postDate": "2023-10-15T05:57:38.390Z", "content": "This result is interesting @andreytikhomirov. There seems to be a reason for this, but since the target is children, it is possible that their daily rhythms have changed during summer vacation and they have more opportunities to take off their watches. It may also have an impact on test data.
", "rawMarkdown": "This result is interesting @andreytikhomirov. There seems to be a reason for this, but since the target is children, it is possible that their daily rhythms have changed during summer vacation and they have more opportunities to take off their watches. It may also have an impact on test data.", "votes": 1, "replies": [ { "id": 2482838, "postDate": "2023-10-15T10:17:34.077Z", "content": "I agree that their daily cycle has changed: my analysis showed that the average wake-up time is the latest in the summer (and bedtime is the earliest, which is surprising). However, I'm not sure that the time of wearing the devices should change because of this. The organizer of the competition mentions that we are talking about children \"with mood and behavior difficulties\", so it is logical that such children should be monitored regardless of the season.
", "rawMarkdown": "I agree that their daily cycle has changed: [my analysis showed](https://www.kaggle.com/code/andreytikhomirov/eda-fe-introducing-seasons) that the average wake-up time is the latest in the summer (and bedtime is the earliest, which is surprising). However, I'm not sure that the time of wearing the devices should change because of this. The organizer of the competition mentions that we are talking about children \"with mood and behavior difficulties\", so it is logical that such children should be monitored regardless of the season.", "votes": 1 } ] }, { "id": 2487655, "postDate": "2023-10-18T17:42:08.537Z", "content": "Key insights on seasonal effects @andreytikhomirov , I deem the weights of every season class should be re-distributed. What’s your idea?
", "rawMarkdown": "Key insights on seasonal effects @andreytikhomirov , I deem the weights of every season class should be re-distributed. What’s your idea?" }, { "id": 2482538, "postDate": "2023-10-15T05:36:13.850Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2485422, "author_name": "Bhargav Kowshik", "author_url": "", "post_date": "2023-10-17T07:23:05.310000", "content": "Very interesting insight. Thank you for sharing @andreytikhomirov
\nIt's only in this competition that I learnt and have been using Polars, previously I worked only with Pandas. I see that you use Dask and I looked up quickly for the differences between the libraries and found the following post. My question is since submission on Kaggle does not have distributed computing support, wouldn't Polars be a better choice for this competition?
\n\nI agree, Polars seems to be the best choice to use in the solution. In my EDA I used Dask only for specific histograms, because the same code written in pandas was much slower.
", "votes": 0, "replies": [] } ] }, { "id": 2482557, "author_name": "mh", "author_url": "", "post_date": "2023-10-15T05:57:38.390000", "content": "This result is interesting @andreytikhomirov. There seems to be a reason for this, but since the target is children, it is possible that their daily rhythms have changed during summer vacation and they have more opportunities to take off their watches. It may also have an impact on test data.
", "votes": 1, "replies": [ { "id": 2482838, "author_name": "Andrey Tikhomirov", "author_url": "", "post_date": "2023-10-15T10:17:34.077000", "content": "I agree that their daily cycle has changed: my analysis showed that the average wake-up time is the latest in the summer (and bedtime is the earliest, which is surprising). However, I'm not sure that the time of wearing the devices should change because of this. The organizer of the competition mentions that we are talking about children \"with mood and behavior difficulties\", so it is logical that such children should be monitored regardless of the season.
", "votes": 1, "replies": [] } ] }, { "id": 2487655, "author_name": "Lamal", "author_url": "", "post_date": "2023-10-18T17:42:08.537000", "content": "Key insights on seasonal effects @andreytikhomirov , I deem the weights of every season class should be re-distributed. What’s your idea?
", "votes": 0, "replies": [] }, { "id": 2482538, "author_name": "", "author_url": "", "post_date": "2023-10-15T05:36:13.850000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2482277": "It's no secret that sleep patterns are highly dependent on the length of daylight hours, which in turn varies from season to season. In one of my experiments, I added season (and month as well) as a feature, and its feature importance was quite high (among the top 10 for lightgbm). However, the LB score of the new model was lower than that of a similar model without season. This led me to speculate about the dissimilarity of the data distributions by season on the train and test datasets. \nAfter [examining](https://www.kaggle.com/code/andreytikhomirov/eda-fe-introducing-seasons) the distribution of data in train by season, I noticed that they are not evenly distributed: we have significantly less data for the summer period. I would venture to guess that the remaining summer data could end up in test and make up a significant proportion of it.\n\n\n\nHave you tried undersampling based on month/season? If so, how did it affect the LB score?", "2485422": "Very interesting insight. Thank you for sharing @andreytikhomirov\n\n\nIt's only in this competition that I learnt and have been using Polars, previously I worked only with Pandas. I see that you use Dask and I looked up quickly for the differences between the libraries and found the following post. My question is since submission on Kaggle does not have distributed computing support, **wouldn't Polars be a better choice for this competition**?\n\nRef: https://medium.com/@nandeda.narayan/data-processing-at-scale-comparison-of-pandas-polars-and-dask-333ae65c0a45\n\n1. Pandas remains a solid choice for smaller datasets that fit into memory, offering a rich ecosystem and extensive community support.\n2. Polars is a promising alternative that excels in performance for large-scale data processing while maintaining a Pandas-like API.\n3. On the other hand, Dask shines when it comes to distributed computing and seamlessly scaling computations across clusters.\n\n", "2482557": "This result is interesting @andreytikhomirov. There seems to be a reason for this, but since the target is children, it is possible that their daily rhythms have changed during summer vacation and they have more opportunities to take off their watches. It may also have an impact on test data.", "2487655": "Key insights on seasonal effects @andreytikhomirov , I deem the weights of every season class should be re-distributed. What’s your idea?", "2482538": "" } }