While I understand that yes we are in fact dealing with time series data, is the problem itself actually time series? If the problem was being given continuous inputs detect the start and end time of an event, and the data itself was provided step by step rather than a whole sequence at once, I could see this being considered a time series problem. But given we have all the data at once, it feels more like a segmentation problem. That is given an array of data, slice the data into these event periods.
\nI'm not sure if the authors intend for the solution to be used to detect on live data, or if they just need this detect the data after the fact. If the intention is to detect live data, I think this problem needs restructured, because then this would be a data leak, and I think a lot of potentially strong solutions can be looking at the whole series, or at least a large part of it, rather than step by step.
", "messages": [ { "id": 2427761, "postDate": "2023-09-07T13:17:48.497Z", "content": "Thanks for the thoughtful comment! I agree with your observation that this can be considered a segmentation problem, rather than live timeseries analysis. Our goal would not necessarily be to provide immediate real-time detection of sleep events, but rather to be able to segment them reliably after the fact for consideration or analysis alongside other data.
", "rawMarkdown": "Thanks for the thoughtful comment! I agree with your observation that this can be considered a segmentation problem, rather than live timeseries analysis. Our goal would not necessarily be to provide immediate real-time detection of sleep events, but rather to be able to segment them reliably after the fact for consideration or analysis alongside other data.", "votes": 23 }, { "id": 2427263, "postDate": "2023-09-07T06:16:16.263Z", "content": "While I understand that yes we are in fact dealing with time series data, is the problem itself actually time series? If the problem was being given continuous inputs detect the start and end time of an event, and the data itself was provided step by step rather than a whole sequence at once, I could see this being considered a time series problem. But given we have all the data at once, it feels more like a segmentation problem. That is given an array of data, slice the data into these event periods.
\nI'm not sure if the authors intend for the solution to be used to detect on live data, or if they just need this detect the data after the fact. If the intention is to detect live data, I think this problem needs restructured, because then this would be a data leak, and I think a lot of potentially strong solutions can be looking at the whole series, or at least a large part of it, rather than step by step.
", "rawMarkdown": "While I understand that yes we are in fact dealing with time series data, is the problem itself actually time series? If the problem was being given continuous inputs detect the start and end time of an event, and the data itself was provided step by step rather than a whole sequence at once, I could see this being considered a time series problem. But given we have all the data at once, it feels more like a segmentation problem. That is given an array of data, slice the data into these event periods.\n\nI'm not sure if the authors intend for the solution to be used to detect on live data, or if they just need this detect the data after the fact. If the intention is to detect live data, I think this problem needs restructured, because then this would be a data leak, and I think a lot of potentially strong solutions can be looking at the whole series, or at least a large part of it, rather than step by step.", "votes": 25 }, { "id": 2439365, "postDate": "2023-09-14T19:32:14.123Z", "content": "Interesting discussion. Here is a beginner-friendly introduction into the problem of change point detection
\nChange-point-detection
Thank you!
", "rawMarkdown": "Thank you!" } ] }, { "id": 2429459, "postDate": "2023-09-08T15:59:43.927Z", "content": "I think the econometricians call this the regime shift detection problem. It's also called breakpoint or change point detection. There are lots of papers on the subject.
", "rawMarkdown": "I think the econometricians call this the regime shift detection problem. It's also called breakpoint or change point detection. There are lots of papers on the subject.", "votes": 7, "replies": [ { "id": 2457463, "postDate": "2023-09-26T23:59:35.687Z", "content": "I'm also thinking about it as an anomaly detection
", "rawMarkdown": "I'm also thinking about it as an anomaly detection", "votes": 3 } ] }, { "id": 2427439, "postDate": "2023-09-07T08:12:15.940Z", "content": "Indeed, you're right in recognizing this as a segmentation problem. In my experience, data from all the research-grade accelerometers I'm familiar with isn't provided as a continuous stream. Instead, researchers often have extensive databases of accelerometer data they're looking to analyze, notably for patterns like sleep among other things.
\nHowever, it's crucial to remember the inherent time dependency in these data sets. The importance of circadian rhythm is one such element that shouldn't be overlooked. I'd like to point out this paper (doi: 10.1093/sleep/zsz180) that incorporates sensor-independent features, an approach I've integrated into my own research as well.
\nFor tasks like these, a bidirectional LSTM would indeed be fitting, given its ability to understand sequences from both forward and backward directions, potentially capturing patterns more comprehensively.
", "rawMarkdown": "Indeed, you're right in recognizing this as a segmentation problem. In my experience, data from all the research-grade accelerometers I'm familiar with isn't provided as a continuous stream. Instead, researchers often have extensive databases of accelerometer data they're looking to analyze, notably for patterns like sleep among other things.\n\nHowever, it's crucial to remember the inherent time dependency in these data sets. The importance of circadian rhythm is one such element that shouldn't be overlooked. I'd like to point out this paper (doi: 10.1093/sleep/zsz180) that incorporates sensor-independent features, an approach I've integrated into my own research as well.\n\nFor tasks like these, a bidirectional LSTM would indeed be fitting, given its ability to understand sequences from both forward and backward directions, potentially capturing patterns more comprehensively.", "votes": 3, "replies": [ { "id": 2429244, "postDate": "2023-09-08T13:32:50.237Z", "content": "It is interesting to me, whether or not some sort of wavelets or CNN could be of use here
", "rawMarkdown": "It is interesting to me, whether or not some sort of wavelets or CNN could be of use here", "votes": 3 } ] }, { "id": 2427356, "postDate": "2023-09-07T07:16:55.410Z", "rawMarkdown": "", "votes": -7, "isDeleted": true, "replies": [ { "id": 2427438, "postDate": "2023-09-07T08:11:45.987Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 2427340, "postDate": "2023-09-07T07:05:25.360Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2427761, "author_name": "Greg Kiar", "author_url": "", "post_date": "2023-09-07T13:17:48.497000", "content": "Thanks for the thoughtful comment! I agree with your observation that this can be considered a segmentation problem, rather than live timeseries analysis. Our goal would not necessarily be to provide immediate real-time detection of sleep events, but rather to be able to segment them reliably after the fact for consideration or analysis alongside other data.
", "votes": 23, "replies": [] }, { "id": 2439365, "author_name": "Peter Hahn", "author_url": "", "post_date": "2023-09-14T19:32:14.123000", "content": "Interesting discussion. Here is a beginner-friendly introduction into the problem of change point detection
\nChange-point-detection
Thank you!
", "votes": 0, "replies": [] } ] }, { "id": 2429459, "author_name": "grayjay", "author_url": "", "post_date": "2023-09-08T15:59:43.927000", "content": "I think the econometricians call this the regime shift detection problem. It's also called breakpoint or change point detection. There are lots of papers on the subject.
", "votes": 7, "replies": [ { "id": 2457463, "author_name": "HinePo", "author_url": "", "post_date": "2023-09-26T23:59:35.687000", "content": "I'm also thinking about it as an anomaly detection
", "votes": 3, "replies": [] } ] }, { "id": 2427439, "author_name": "sesben", "author_url": "", "post_date": "2023-09-07T08:12:15.940000", "content": "Indeed, you're right in recognizing this as a segmentation problem. In my experience, data from all the research-grade accelerometers I'm familiar with isn't provided as a continuous stream. Instead, researchers often have extensive databases of accelerometer data they're looking to analyze, notably for patterns like sleep among other things.
\nHowever, it's crucial to remember the inherent time dependency in these data sets. The importance of circadian rhythm is one such element that shouldn't be overlooked. I'd like to point out this paper (doi: 10.1093/sleep/zsz180) that incorporates sensor-independent features, an approach I've integrated into my own research as well.
\nFor tasks like these, a bidirectional LSTM would indeed be fitting, given its ability to understand sequences from both forward and backward directions, potentially capturing patterns more comprehensively.
", "votes": 3, "replies": [ { "id": 2429244, "author_name": "Tim Gerasin", "author_url": "", "post_date": "2023-09-08T13:32:50.237000", "content": "It is interesting to me, whether or not some sort of wavelets or CNN could be of use here
", "votes": 3, "replies": [] } ] }, { "id": 2427356, "author_name": "", "author_url": "", "post_date": "2023-09-07T07:16:55.410000", "content": "", "votes": -7, "replies": [ { "id": 2427438, "author_name": "", "author_url": "", "post_date": "2023-09-07T08:11:45.987000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 2427340, "author_name": "", "author_url": "", "post_date": "2023-09-07T07:05:25.360000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2427761": "Thanks for the thoughtful comment! I agree with your observation that this can be considered a segmentation problem, rather than live timeseries analysis. Our goal would not necessarily be to provide immediate real-time detection of sleep events, but rather to be able to segment them reliably after the fact for consideration or analysis alongside other data.", "2427263": "While I understand that yes we are in fact dealing with time series data, is the problem itself actually time series? If the problem was being given continuous inputs detect the start and end time of an event, and the data itself was provided step by step rather than a whole sequence at once, I could see this being considered a time series problem. But given we have all the data at once, it feels more like a segmentation problem. That is given an array of data, slice the data into these event periods.\n\nI'm not sure if the authors intend for the solution to be used to detect on live data, or if they just need this detect the data after the fact. If the intention is to detect live data, I think this problem needs restructured, because then this would be a data leak, and I think a lot of potentially strong solutions can be looking at the whole series, or at least a large part of it, rather than step by step.", "2439365": "Interesting discussion. Here is a beginner-friendly introduction into the problem of change point detection\n[Change-point-detection](https://www.iese.fraunhofer.de/blog/change-point-detection/)", "2429459": "I think the econometricians call this the regime shift detection problem. It's also called breakpoint or change point detection. There are lots of papers on the subject.", "2427439": "Indeed, you're right in recognizing this as a segmentation problem. In my experience, data from all the research-grade accelerometers I'm familiar with isn't provided as a continuous stream. Instead, researchers often have extensive databases of accelerometer data they're looking to analyze, notably for patterns like sleep among other things.\n\nHowever, it's crucial to remember the inherent time dependency in these data sets. The importance of circadian rhythm is one such element that shouldn't be overlooked. I'd like to point out this paper (doi: 10.1093/sleep/zsz180) that incorporates sensor-independent features, an approach I've integrated into my own research as well.\n\nFor tasks like these, a bidirectional LSTM would indeed be fitting, given its ability to understand sequences from both forward and backward directions, potentially capturing patterns more comprehensively.", "2427356": "", "2427340": "" } }