![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2051313%2F041eefe3201b57da4aea75c6ccdca150%2Fblood_glucose_000_100.png?generation=1729804497164765&alt=media\")
An interesting tidbit of information - want to beat the sample submission in 3 lines of code? If you take a quick look at kernel density estimates for bg-0:00 and bg+1:00, you see some interesting similarities:
While there are some differences, their density estimates are very very similar. Taking a quick look at basic properties of the columns reveals the following:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2051313%2Ffb821c7a01e89bf84bce506d5f614f25%2Fproperties_bg_000_100.png?generation=1729804586613123&alt=media\")
As we can see, min, max, mean, and standard deviations are very very similar. So, taking a simple approach, let's just use bg-0:00 as the prediction for bg+1:00. Since we know the data has a lot of missing values, we can fall back to using bg-0:05 or bg-0:10 if one of those are missing. If all three are missing, we can just use the mean value of bg-0:00 as our guess. Doing so reveals a surprising result for the training set:
$$RMSE = 2.3277874025246774$$
\nGenerating a simple solution using this approach for the test set yields a public LB score of 2.9156 - which is better than the sample submission (i.e. the same value for each patient). It is worse than the current first place public notebook of 2.4738, but still - it's a simple baseline for performance comparison purposes. Anyhow, the \"so what\" of this post is that bg-0:00 is a very strong predictor for bg+1:00 with no machine learning or domain knowledge.
An interesting tidbit of information - want to beat the sample submission in 3 lines of code? If you take a quick look at kernel density estimates for bg-0:00 and bg+1:00, you see some interesting similarities:
While there are some differences, their density estimates are very very similar. Taking a quick look at basic properties of the columns reveals the following:
\n
As we can see, min, max, mean, and standard deviations are very very similar. So, taking a simple approach, let's just use bg-0:00 as the prediction for bg+1:00. Since we know the data has a lot of missing values, we can fall back to using bg-0:05 or bg-0:10 if one of those are missing. If all three are missing, we can just use the mean value of bg-0:00 as our guess. Doing so reveals a surprising result for the training set:
$$RMSE = 2.3277874025246774$$
\nGenerating a simple solution using this approach for the test set yields a public LB score of 2.9156 - which is better than the sample submission (i.e. the same value for each patient). It is worse than the current first place public notebook of 2.4738, but still - it's a simple baseline for performance comparison purposes. Anyhow, the \"so what\" of this post is that bg-0:00 is a very strong predictor for bg+1:00 with no machine learning or domain knowledge.
Thumbs up for doing the analysis and sharing it @craigmthomas. Just mentioning that the reason that the distributions are almost identical is that the same values (literally the same values, except the ends of the series) are included: the train dataset consists of 6 hour windows shifted by 15 minutes. The histograms probably look so different just because of approximate binning.
", "rawMarkdown": "Thumbs up for doing the analysis and sharing it @craigmthomas. Just mentioning that the reason that the distributions are almost identical is that the same values (literally the same values, except the ends of the series) are included: the train dataset consists of 6 hour windows shifted by 15 minutes. The histograms probably look so different just because of approximate binning.", "votes": null }, { "id": "3028243", "postDate": "10/25/2024 18:38:38", "content": "@jagofc ha, thank you for this. I hadn't looked at the time windows yet (or the ID fields as part of my EDA) - I'm late to this competition - so I hadn't yet clicked into this. It helps explain some of the repeating patterns that I saw when I was looking at correlation between features. This is especially prominent when you look at correlation between carbs- features:
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2051313%2F8aa13d3c37866ed5ef0c5defed65dd74%2Fblood_glucose_carbs_repeat.png?generation=1729859181525742&alt=media\")
The repeating patterns on the correlation threw me, but now it makes sense - if there is a shift by 15 minute increments of the same values, then there is going to be correlation among the various time related features.
", "rawMarkdown": "jagofc ha, thank you for this. I hadn't looked at the time windows yet (or the ID fields as part of my EDA) - I'm late to this competition - so I hadn't yet clicked into this. It helps explain some of the repeating patterns that I saw when I was looking at correlation between features. This is especially prominent when you look at correlation between `carbs-` features:\n\n\n\nThe repeating patterns on the correlation threw me, but now it makes sense - if there is a shift by 15 minute increments of the same values, then there is going to be correlation among the various time related features.", "votes": null }, { "id": "3033524", "postDate": "11/01/2024 08:41:47", "content": "thanks a lot for sharing
", "rawMarkdown": "thanks a lot for sharing", "votes": null }, { "id": "3037188", "postDate": "11/05/2024 13:10:26", "content": "This was a great help as a starting point. Thanks!
", "rawMarkdown": "This was a great help as a starting point. Thanks!", "votes": null } ], "comments": [ { "id": 3027854, "author_name": "jagofc", "author_url": "", "post_date": "10/25/2024 11:08:09", "content": "Thumbs up for doing the analysis and sharing it @craigmthomas. Just mentioning that the reason that the distributions are almost identical is that the same values (literally the same values, except the ends of the series) are included: the train dataset consists of 6 hour windows shifted by 15 minutes. The histograms probably look so different just because of approximate binning.
", "votes": null, "replies": [ { "id": 3028243, "author_name": "craigmthomas", "author_url": "", "post_date": "10/25/2024 18:38:38", "content": "@jagofc ha, thank you for this. I hadn't looked at the time windows yet (or the ID fields as part of my EDA) - I'm late to this competition - so I hadn't yet clicked into this. It helps explain some of the repeating patterns that I saw when I was looking at correlation between features. This is especially prominent when you look at correlation between carbs- features:
The repeating patterns on the correlation threw me, but now it makes sense - if there is a shift by 15 minute increments of the same values, then there is going to be correlation among the various time related features.
", "votes": null, "replies": [] } ] }, { "id": 3033524, "author_name": "mhhegazy", "author_url": "", "post_date": "11/01/2024 08:41:47", "content": "thanks a lot for sharing
", "votes": null, "replies": [] }, { "id": 3037188, "author_name": "jakebrannigan", "author_url": "", "post_date": "11/05/2024 13:10:26", "content": "This was a great help as a starting point. Thanks!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3027470": "An interesting tidbit of information - want to beat the sample submission in 3 lines of code? If you take a quick look at kernel density estimates for `bg-0:00` and `bg+1:00`, you see some interesting similarities:\n\n\n\nWhile there are some differences, their density estimates are very very similar. Taking a quick look at basic properties of the columns reveals the following:\n\n\n\nAs we can see, min, max, mean, and standard deviations are very very similar. So, taking a simple approach, let's just use `bg-0:00` as the prediction for `bg+1:00`. Since we know the data has a lot of missing values, we can fall back to using `bg-0:05` or `bg-0:10` if one of those are missing. If all three are missing, we can just use the mean value of `bg-0:00` as our guess. Doing so reveals a surprising result for the training set:\n\n$$RMSE = 2.3277874025246774$$\n\nGenerating a [simple solution using this approach](https://www.kaggle.com/code/craigmthomas/brist1d-simple-prediction/notebook) for the test set yields a public LB score of 2.9156 - which is better than the sample submission (i.e. the same value for each patient). It is worse than the current first place public notebook of 2.4738, but still - it's a simple baseline for performance comparison purposes. Anyhow, the \"so what\" of this post is that `bg-0:00` is a very strong predictor for `bg+1:00` with no machine learning or domain knowledge.", "3027854": "Thumbs up for doing the analysis and sharing it @craigmthomas. Just mentioning that the reason that the distributions are almost identical is that the same values (literally the same values, except the ends of the series) are included: the train dataset consists of 6 hour windows shifted by 15 minutes. The histograms probably look so different just because of approximate binning.", "3028243": "jagofc ha, thank you for this. I hadn't looked at the time windows yet (or the ID fields as part of my EDA) - I'm late to this competition - so I hadn't yet clicked into this. It helps explain some of the repeating patterns that I saw when I was looking at correlation between features. This is especially prominent when you look at correlation between `carbs-` features:\n\n\n\nThe repeating patterns on the correlation threw me, but now it makes sense - if there is a shift by 15 minute increments of the same values, then there is going to be correlation among the various time related features.", "3033524": "thanks a lot for sharing", "3037188": "This was a great help as a starting point. Thanks!" }, "source": "meta" }