![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F337744%2F8522dbce4ebae9e241c276e4a27a4307%2FSample.png?generation=1695297333962220&alt=media\")
Instead of segmenting the time series into wake and sleep periods, one may directly focus on the transition times. Here is one way to frame it:
\nTransform the inputs into (batch, 17280, features) shape. 17280 is the number of 5s in 24 hr with index 0 corresponding to the same point of the day for each 24 hr. There may be a small amount of unused portions at the beginning and/or the end of the series, just throw them away or pad with 0s.
\nThe output will be in output=(17280, 2) shape. It will have Gaussians centered on onset positions (output[:,0]) and wake positions (output[:, 1]) with fixed sigma (something to tune). The outputs should be normalized appropriately.
\nThis approach is similar to Heatmap regression method used in key-point prediction problems.
\n
Instead of segmenting the time series into wake and sleep periods, one may directly focus on the transition times. Here is one way to frame it:
\nTransform the inputs into (batch, 17280, features) shape. 17280 is the number of 5s in 24 hr with index 0 corresponding to the same point of the day for each 24 hr. There may be a small amount of unused portions at the beginning and/or the end of the series, just throw them away or pad with 0s.
\nThe output will be in output=(17280, 2) shape. It will have Gaussians centered on onset positions (output[:,0]) and wake positions (output[:, 1]) with fixed sigma (something to tune). The outputs should be normalized appropriately.
\nThis approach is similar to Heatmap regression method used in key-point prediction problems.
\n
I also tried this approach, and I got LB 0.617/CV 0.526 with just one fold.
", "rawMarkdown": "I also tried this approach, and I got LB 0.617/CV 0.526 with just one fold.", "votes": 1 }, { "id": 2473227, "postDate": "2023-10-08T05:21:37.713Z", "content": "Hi!
\nI tried a form of this approach and it looks like it works!
\nI did it without separating the ts into separate days, though, just downsampling.
Not separating into days makes sense.
", "rawMarkdown": "Not separating into days makes sense. ", "votes": 1 }, { "id": 2476750, "postDate": "2023-10-10T23:03:14.380Z", "content": "@werus23 I'm getting amazing results only with 1 epoch of training.
", "rawMarkdown": "@werus23 I'm getting amazing results only with 1 epoch of training.", "votes": 2, "replies": [ { "id": 2479686, "postDate": "2023-10-12T19:29:13.997Z", "content": "Indeed, after just one epoch, the results look promising enough to warrant dedicating another weekend to exploring this model's full potential. I anticipate there will be plenty of fine-tuning involved, from adjusting standard deviations to determining how to split the training data into batches, and many other things to experiment with. Thank you for the inspiration!
", "rawMarkdown": "Indeed, after just one epoch, the results look promising enough to warrant dedicating another weekend to exploring this model's full potential. I anticipate there will be plenty of fine-tuning involved, from adjusting standard deviations to determining how to split the training data into batches, and many other things to experiment with. Thank you for the inspiration!", "votes": 1 } ] }, { "id": 2488183, "postDate": "2023-10-19T05:27:38.943Z", "content": "\n\nI tried a form of this approach and it looks like it works!
\n
@werus23 The link does not work for me, get a HTTP 404.
Hi @tolgadincer , interesting approach
\nI am thinking of using encoder decoder approach instead, i.e. squeezing the sequence into smaller latent space like 50 steps then decode it back to the original length with logit values, but I think it will be overkill. What do you think?
", "rawMarkdown": "Hi @tolgadincer , interesting approach\n\nI am thinking of using encoder decoder approach instead, i.e. squeezing the sequence into smaller latent space like 50 steps then decode it back to the original length with logit values, but I think it will be overkill. What do you think?", "votes": 2, "replies": [ { "id": 2447264, "postDate": "2023-09-20T01:14:34.510Z", "content": "Try and see.
", "rawMarkdown": "Try and see.", "votes": 2 } ] }, { "id": 2987980, "postDate": "2024-09-13T08:57:08.543Z", "content": "Thanks a lot for your approach, @tolgadincer! It helped me figure out this competition 😀
", "rawMarkdown": "Thanks a lot for your approach, @tolgadincer! It helped me figure out this competition 😀" }, { "id": 2487775, "postDate": "2023-10-18T18:41:44.463Z", "content": "@tolgadincer i am considering using an LSTM model with inputs of shape (batch_size, sequence_length, input_size) = (8, 2, 17280) since we have 2 daily time series of enmo and anglez. However, I don't know which way should the labels be structured. I was thinking of creating binary labels which are arrays of length 17280 with 1s between the onset and the wakeup times and 0s elsewhere. What was your approach for the labels? Any direction is much appreciated!
You can either do segmentation with binary labels or the regression method I suggested here. I'm doing everything possible.
", "rawMarkdown": "You can either do segmentation with binary labels or the regression method I suggested here. I'm doing everything possible.", "votes": 1 } ] }, { "id": 2486983, "postDate": "2023-10-18T09:26:07.907Z", "content": "Hi, what's an example of a neural network model that can do this? A lot of the examples I have researched do heatmap regression for images using CNNs. I'm wondering how someone would transform that approach to a Time Series problem in change point detection context
", "rawMarkdown": "Hi, what's an example of a neural network model that can do this? A lot of the examples I have researched do heatmap regression for images using CNNs. I'm wondering how someone would transform that approach to a Time Series problem in change point detection context\n\n", "replies": [ { "id": 2487003, "postDate": "2023-10-18T09:50:19.913Z", "content": "For instance, take a CNN for imaging and convert it to 1d. The inputs and outputs would look like the figure in my post above.
", "rawMarkdown": "For instance, take a CNN for imaging and convert it to 1d. The inputs and outputs would look like the figure in my post above." } ] }, { "id": 2480980, "postDate": "2023-10-13T17:21:07.093Z", "content": "I am thinking about whether we should punish the null onset and wakeup event by setting an -1 * Gaussian centered on those date. But I guess it will be hard to decide where it should center since we only know on which date the event is not recorded, and setting it centered to 17820 / 2 step on that date might be too biased. Any suggestion?
", "rawMarkdown": "I am thinking about whether we should punish the null onset and wakeup event by setting an -1 * Gaussian centered on those date. But I guess it will be hard to decide where it should center since we only know on which date the event is not recorded, and setting it centered to 17820 / 2 step on that date might be too biased. Any suggestion?", "replies": [ { "id": 2481050, "postDate": "2023-10-13T18:11:54.727Z", "content": "I'm thinking of either completely discarding them or training without them, predicting them, and then retraining with the predicted labels.
", "rawMarkdown": "I'm thinking of either completely discarding them or training without them, predicting them, and then retraining with the predicted labels." }, { "id": 2483279, "postDate": "2023-10-15T16:06:26.967Z", "content": "I'm currently just discarding them.
", "rawMarkdown": "I'm currently just discarding them.", "replies": [ { "id": 2483349, "postDate": "2023-10-15T16:33:51.963Z", "content": "ok, I might try that as well. I was working with null days and have just setup a working cv, but the CV is really bad (only 0.32), compared to my lgbm models(0.56).
\nEDIT: ignore the lines above, it turns out that I have set a std that is too large. for anyone who is trying to attempt this approach, a std of 666 is too large.
", "rawMarkdown": "ok, I might try that as well. I was working with null days and have just setup a working cv, but the CV is really bad (only 0.32), compared to my lgbm models(0.56).\n\nEDIT: ignore the lines above, it turns out that I have set a std that is too large. **for anyone who is trying to attempt this approach, a std of 666 is too large.**", "votes": 1 } ] } ] }, { "id": 2477343, "postDate": "2023-10-11T07:58:05.893Z", "content": "Hi @tolgadincer
\nSo is it then for the test data, all the inputs would need to be padded with 0s in front? since it only has 450 rows (<17280)
", "rawMarkdown": "Hi @tolgadincer\n\nSo is it then for the test data, all the inputs would need to be padded with 0s in front? since it only has 450 rows (<17280)", "replies": [ { "id": 2477812, "postDate": "2023-10-11T15:22:19.443Z", "content": "Try and see. I don't know what works best.
", "rawMarkdown": "Try and see. I don't know what works best." } ] }, { "id": 2524094, "postDate": "2023-11-14T01:50:04.563Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2487699, "postDate": "2023-10-18T18:04:36.297Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 2487896, "postDate": "2023-10-18T20:39:32.717Z", "content": "I don't quite understand your question. Could you clarify a little bit?
", "rawMarkdown": "I don't quite understand your question. Could you clarify a little bit?", "votes": 1, "replies": [ { "id": 2488088, "postDate": "2023-10-19T03:07:28.527Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2488993, "postDate": "2023-10-19T16:07:35.743Z", "content": "@liushundawang We are assuming that the data around the onset and wakeup transition times have a common pattern. If the assumption is correct, we should be able to predict whatever shape we put there. Gaussian has this nice peak. If your model finds the peak, it also finds its centroid. The idea is very simple. It's widely used in facial landmark and (human, animal body) keypoint predictions.
\nWe could have labeled these transition times as 1s and all other bins as 0s. But then it would be extremely hard to find those very few 1s among a large amount of 0s.
", "rawMarkdown": "@liushundawang We are assuming that the data around the onset and wakeup transition times have a common pattern. If the assumption is correct, we should be able to predict whatever shape we put there. Gaussian has this nice peak. If your model finds the peak, it also finds its centroid. The idea is very simple. It's widely used in facial landmark and (human, animal body) keypoint predictions.\n\nWe could have labeled these transition times as 1s and all other bins as 0s. But then it would be extremely hard to find those very few 1s among a large amount of 0s.", "votes": 4 }, { "id": 2489001, "postDate": "2023-10-19T16:14:52.437Z", "content": "Got it, thank very much @tolgadincer
", "rawMarkdown": "Got it, thank very much @tolgadincer ", "votes": 1 } ] } ] }, { "id": 2477342, "postDate": "2023-10-11T07:56:20.073Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2445567, "postDate": "2023-09-19T00:26:21.343Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ], "comments": [ { "id": 2481668, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-10-14T10:49:23.823000", "content": "I also tried this approach, and I got LB 0.617/CV 0.526 with just one fold.
", "votes": 1, "replies": [] }, { "id": 2473227, "author_name": "werus23", "author_url": "", "post_date": "2023-10-08T05:21:37.713000", "content": "Hi!
\nI tried a form of this approach and it looks like it works!
\nI did it without separating the ts into separate days, though, just downsampling.
Not separating into days makes sense.
", "votes": 1, "replies": [] }, { "id": 2476750, "author_name": "Tolga", "author_url": "", "post_date": "2023-10-10T23:03:14.380000", "content": "@werus23 I'm getting amazing results only with 1 epoch of training.
", "votes": 2, "replies": [ { "id": 2479686, "author_name": "Yuchen20", "author_url": "", "post_date": "2023-10-12T19:29:13.997000", "content": "Indeed, after just one epoch, the results look promising enough to warrant dedicating another weekend to exploring this model's full potential. I anticipate there will be plenty of fine-tuning involved, from adjusting standard deviations to determining how to split the training data into batches, and many other things to experiment with. Thank you for the inspiration!
", "votes": 1, "replies": [] } ] }, { "id": 2488183, "author_name": "Bhargav Kowshik", "author_url": "", "post_date": "2023-10-19T05:27:38.943000", "content": "\n\nI tried a form of this approach and it looks like it works!
\n
@werus23 The link does not work for me, get a HTTP 404.
Hi @tolgadincer , interesting approach
\nI am thinking of using encoder decoder approach instead, i.e. squeezing the sequence into smaller latent space like 50 steps then decode it back to the original length with logit values, but I think it will be overkill. What do you think?
", "votes": 2, "replies": [ { "id": 2447264, "author_name": "Tolga", "author_url": "", "post_date": "2023-09-20T01:14:34.510000", "content": "Try and see.
", "votes": 2, "replies": [] } ] }, { "id": 2987980, "author_name": "Emil Aflatunov", "author_url": "", "post_date": "2024-09-13T08:57:08.543000", "content": "Thanks a lot for your approach, @tolgadincer! It helped me figure out this competition 😀
", "votes": 0, "replies": [] }, { "id": 2487775, "author_name": "moth", "author_url": "", "post_date": "2023-10-18T18:41:44.463000", "content": "@tolgadincer i am considering using an LSTM model with inputs of shape (batch_size, sequence_length, input_size) = (8, 2, 17280) since we have 2 daily time series of enmo and anglez. However, I don't know which way should the labels be structured. I was thinking of creating binary labels which are arrays of length 17280 with 1s between the onset and the wakeup times and 0s elsewhere. What was your approach for the labels? Any direction is much appreciated!
You can either do segmentation with binary labels or the regression method I suggested here. I'm doing everything possible.
", "votes": 1, "replies": [] } ] }, { "id": 2486983, "author_name": "learningmonkey", "author_url": "", "post_date": "2023-10-18T09:26:07.907000", "content": "Hi, what's an example of a neural network model that can do this? A lot of the examples I have researched do heatmap regression for images using CNNs. I'm wondering how someone would transform that approach to a Time Series problem in change point detection context
", "votes": 0, "replies": [ { "id": 2487003, "author_name": "Tolga", "author_url": "", "post_date": "2023-10-18T09:50:19.913000", "content": "For instance, take a CNN for imaging and convert it to 1d. The inputs and outputs would look like the figure in my post above.
", "votes": 0, "replies": [] } ] }, { "id": 2480980, "author_name": "Yuchen20", "author_url": "", "post_date": "2023-10-13T17:21:07.093000", "content": "I am thinking about whether we should punish the null onset and wakeup event by setting an -1 * Gaussian centered on those date. But I guess it will be hard to decide where it should center since we only know on which date the event is not recorded, and setting it centered to 17820 / 2 step on that date might be too biased. Any suggestion?
", "votes": 0, "replies": [ { "id": 2481050, "author_name": "Tolga", "author_url": "", "post_date": "2023-10-13T18:11:54.727000", "content": "I'm thinking of either completely discarding them or training without them, predicting them, and then retraining with the predicted labels.
", "votes": 0, "replies": [] }, { "id": 2483279, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2023-10-15T16:06:26.967000", "content": "I'm currently just discarding them.
", "votes": 0, "replies": [ { "id": 2483349, "author_name": "Yuchen20", "author_url": "", "post_date": "2023-10-15T16:33:51.963000", "content": "ok, I might try that as well. I was working with null days and have just setup a working cv, but the CV is really bad (only 0.32), compared to my lgbm models(0.56).
\nEDIT: ignore the lines above, it turns out that I have set a std that is too large. for anyone who is trying to attempt this approach, a std of 666 is too large.
", "votes": 1, "replies": [] } ] } ] }, { "id": 2477343, "author_name": "EvanNie006", "author_url": "", "post_date": "2023-10-11T07:58:05.893000", "content": "Hi @tolgadincer
\nSo is it then for the test data, all the inputs would need to be padded with 0s in front? since it only has 450 rows (<17280)
", "votes": 0, "replies": [ { "id": 2477812, "author_name": "Tolga", "author_url": "", "post_date": "2023-10-11T15:22:19.443000", "content": "Try and see. I don't know what works best.
", "votes": 0, "replies": [] } ] }, { "id": 2524094, "author_name": "", "author_url": "", "post_date": "2023-11-14T01:50:04.563000", "content": "", "votes": 0, "replies": [] }, { "id": 2487699, "author_name": "", "author_url": "", "post_date": "2023-10-18T18:04:36.297000", "content": "", "votes": 0, "replies": [ { "id": 2487896, "author_name": "Tolga", "author_url": "", "post_date": "2023-10-18T20:39:32.717000", "content": "I don't quite understand your question. Could you clarify a little bit?
", "votes": 1, "replies": [ { "id": 2488088, "author_name": "", "author_url": "", "post_date": "2023-10-19T03:07:28.527000", "content": "", "votes": 0, "replies": [] }, { "id": 2488993, "author_name": "Tolga", "author_url": "", "post_date": "2023-10-19T16:07:35.743000", "content": "@liushundawang We are assuming that the data around the onset and wakeup transition times have a common pattern. If the assumption is correct, we should be able to predict whatever shape we put there. Gaussian has this nice peak. If your model finds the peak, it also finds its centroid. The idea is very simple. It's widely used in facial landmark and (human, animal body) keypoint predictions.
\nWe could have labeled these transition times as 1s and all other bins as 0s. But then it would be extremely hard to find those very few 1s among a large amount of 0s.
", "votes": 4, "replies": [] }, { "id": 2489001, "author_name": "Lamal", "author_url": "", "post_date": "2023-10-19T16:14:52.437000", "content": "Got it, thank very much @tolgadincer
", "votes": 1, "replies": [] } ] } ] }, { "id": 2477342, "author_name": "", "author_url": "", "post_date": "2023-10-11T07:56:20.073000", "content": "", "votes": 0, "replies": [] }, { "id": 2445567, "author_name": "", "author_url": "", "post_date": "2023-09-19T00:26:21.343000", "content": "", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "2445559": "Instead of segmenting the time series into wake and sleep periods, one may directly focus on the transition times. Here is one way to frame it:\n\nTransform the inputs into (batch, 17280, features) shape. 17280 is the number of 5s in 24 hr with index 0 corresponding to the same point of the day for each 24 hr. There may be a small amount of unused portions at the beginning and/or the end of the series, just throw them away or pad with 0s.\n\nThe output will be in output=(17280, 2) shape. It will have Gaussians centered on onset positions (output[:,0]) and wake positions (output[:, 1]) with fixed sigma (something to tune). The outputs should be normalized appropriately.\n\nThis approach is similar to Heatmap regression method used in key-point prediction problems.\n\n", "2481668": "I also tried this approach, and I got LB 0.617/CV 0.526 with just one fold.", "2473227": "Hi!\nI tried a form of this approach and it looks like [it](https://www.kaggle.com/code/werus23/sleep-critical-point-infer) works! \nI did it without separating the ts into separate days, though, just downsampling.", "2445575": "Hi @tolgadincer , interesting approach\n\nI am thinking of using encoder decoder approach instead, i.e. squeezing the sequence into smaller latent space like 50 steps then decode it back to the original length with logit values, but I think it will be overkill. What do you think?", "2987980": "Thanks a lot for your approach, @tolgadincer! It helped me figure out this competition 😀", "2487775": "@tolgadincer i am considering using an LSTM model with inputs of shape `(batch_size, sequence_length, input_size)` = `(8, 2, 17280)` since we have 2 daily time series of `enmo` and `anglez`. However, I don't know which way should the labels be structured. I was thinking of creating binary labels which are arrays of length 17280 with 1s between the `onset` and the `wakeup` times and 0s elsewhere. What was your approach for the labels? Any direction is much appreciated!", "2486983": "Hi, what's an example of a neural network model that can do this? A lot of the examples I have researched do heatmap regression for images using CNNs. I'm wondering how someone would transform that approach to a Time Series problem in change point detection context\n\n", "2480980": "I am thinking about whether we should punish the null onset and wakeup event by setting an -1 * Gaussian centered on those date. But I guess it will be hard to decide where it should center since we only know on which date the event is not recorded, and setting it centered to 17820 / 2 step on that date might be too biased. Any suggestion?", "2477343": "Hi @tolgadincer\n\nSo is it then for the test data, all the inputs would need to be padded with 0s in front? since it only has 450 rows (<17280)", "2524094": "", "2487699": "", "2477342": "", "2445567": "" } }