May someone tell me why this may not work? I am new.
\nThe idea is that since the onset/wakeup events in the validation data are labeled by an expert human labeler, presumably using their eyes, wouldn't it be feasible to take a computer vision approach to the charts/graphs from the data we are provided with? This approach might already be very similar to https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/441470
Yes, you could try a vision approach. Usually, that involves taking an fft of the signal (look up \"melspectrogram librosa\"), then running that through a pre-trained vision model and putting a custom head on top of that. There are several previous competitions you can take a look at to get ideas like: https://www.kaggle.com/competitions/g2net-detecting-continuous-gravitational-waves
", "rawMarkdown": "Yes, you could try a vision approach. Usually, that involves taking an fft of the signal (look up \"melspectrogram librosa\"), then running that through a pre-trained vision model and putting a custom head on top of that. There are several previous competitions you can take a look at to get ideas like: https://www.kaggle.com/competitions/g2net-detecting-continuous-gravitational-waves", "votes": 8, "replies": [ { "id": 2500541, "postDate": "2023-10-26T18:39:27.717Z", "content": "Thanks Chris! Have you already considered this approach? How do you think it will fare well compared to the approaches already established? I am asking because it will probably take a long time for me to implement.
", "rawMarkdown": "Thanks Chris! Have you already considered this approach? How do you think it will fare well compared to the approaches already established? I am asking because it will probably take a long time for me to implement.", "votes": 1, "replies": [ { "id": 2500567, "postDate": "2023-10-26T19:25:53.423Z", "content": "Yes, I've tried a melspectrogram approach and it did look promising (the results were about as good as the current public models), but as you say, the pipeline is more complex, the models are bigger, and the runtime is longer.
\nSo if you feel like you want to try a vision approach just because you're interested to try it, then I think it's a valid thing to try. If you'd just like to get started and aren't tied to vision though, then I'd probably start by looking at a few of the public code samples and go from there - then you'll at least start with an end-to-end working sample. Good luck!
", "rawMarkdown": "Yes, I've tried a melspectrogram approach and it did look promising (the results were about as good as the current public models), but as you say, the pipeline is more complex, the models are bigger, and the runtime is longer.\n\nSo if you feel like you want to try a vision approach just because you're interested to try it, then I think it's a valid thing to try. If you'd just like to get started and aren't tied to vision though, then I'd probably start by looking at a few of the public code samples and go from there - then you'll at least start with an end-to-end working sample. Good luck!", "votes": 4 }, { "id": 2500571, "postDate": "2023-10-26T19:31:52.983Z", "content": "Will do! Thanks a lot for your insight, Chris!
", "rawMarkdown": "Will do! Thanks a lot for your insight, Chris!" }, { "id": 2510218, "postDate": "2023-11-02T19:49:14.787Z", "content": "I also see a potential in this approach, I thought about it during the night time. Didn't find any scientific papers on this topic. However, there is one thing that I do not understand. How should we train a CV model, which target could be used? As I see it, we can imagine vertical lines, that divide images of the series on states \"sleep\" and \"wakeup\". Ok, let's say, that now we are able to segment the picture. But how do we find the \"step\" number after this? Any ideas?
", "rawMarkdown": "I also see a potential in this approach, I thought about it during the night time. Didn't find any scientific papers on this topic. However, there is one thing that I do not understand. How should we train a CV model, which target could be used? As I see it, we can imagine vertical lines, that divide images of the series on states \"sleep\" and \"wakeup\". Ok, let's say, that now we are able to segment the picture. But how do we find the \"step\" number after this? Any ideas?" }, { "id": 2510225, "postDate": "2023-11-02T19:59:00.370Z", "content": "We could scale the pixel values to be proportional to a step, or to an interval of time. From there, we can just calculate where that vertical line is in time, or just calculate the step, based on the pixel coordinate. Does that make sense? Not sure if I fully understood your question
", "rawMarkdown": "We could scale the pixel values to be proportional to a step, or to an interval of time. From there, we can just calculate where that vertical line is in time, or just calculate the step, based on the pixel coordinate. Does that make sense? Not sure if I fully understood your question", "votes": 1 }, { "id": 2510232, "postDate": "2023-11-02T20:03:09.167Z", "content": "Yes, also thought about it. Do you think, that a segmentation or a \"bounding box\" approach here are the best options? Or did you mean other approaches in your idea?
", "rawMarkdown": "Yes, also thought about it. Do you think, that a segmentation or a \"bounding box\" approach here are the best options? Or did you mean other approaches in your idea?" }, { "id": 2510236, "postDate": "2023-11-02T20:08:55.887Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2510239, "postDate": "2023-11-02T20:13:28.833Z", "content": "(I replied then deleted my reply realizing I answered the wrong question):
\nThe current public approaches use either a gaussian signal for onset and wake, or a 1/0 signal for awake/sleep. A \"bounding box\" approach is interesting! It would be quite similar to the 1/0 signal for awake/sleep I believe though, since the bounding box would always be full height.
", "rawMarkdown": "(I replied then deleted my reply realizing I answered the wrong question):\n\nThe current public approaches use either a gaussian signal for onset and wake, or a 1/0 signal for awake/sleep. A \"bounding box\" approach is interesting! It would be quite similar to the 1/0 signal for awake/sleep I believe though, since the bounding box would always be full height.", "votes": 1 }, { "id": 2510241, "postDate": "2023-11-02T20:20:42.110Z", "content": "Yeah, correct. I thought of a segmentation firstly, since you want to detect the area of the sleep state. But then I realized, that it is usually heavy in terms of computational power. And it seems that here we basically want a straight line, so a \"bounding box\" might be enough.
\n@chris62, what approach (task type or target) did you suggest in your idea, when you described the melspectrogram? I do not understand, how to get the interval from it with the CV model.
", "rawMarkdown": "Yeah, correct. I thought of a segmentation firstly, since you want to detect the area of the sleep state. But then I realized, that it is usually heavy in terms of computational power. And it seems that here we basically want a straight line, so a \"bounding box\" might be enough. \n\n@chris62, what approach (task type or target) did you suggest in your idea, when you described the melspectrogram? I do not understand, how to get the interval from it with the CV model." }, { "id": 2510258, "postDate": "2023-11-02T20:38:24.853Z", "content": "I tried both approaches that are popular in the current public notebooks (both performed about the same as far as I could tell, without a lot of trials). That is:
\noutput a 1xSTEP tensor with 0s where the person is awake, and 1s where the person is sleeping, then detect the transition.
output a 2xSTEP tensor, where channel 0 has a gaussian spike every time there is an \"onset\" event, and channel 1 has a gaussian spike every time there is a \"wakeup\" event.
[I also tried simple regression - where I broke the signal into days and predicted just 2 float values: one for the onset time and one for the wakeup time. That didn't perform very well, even after adding positional encodings, etc :) ]
", "rawMarkdown": "I tried both approaches that are popular in the current public notebooks (both performed about the same as far as I could tell, without a lot of trials). That is:\n\n1. output a 1xSTEP tensor with 0s where the person is awake, and 1s where the person is sleeping, then detect the transition.\n\n2. output a 2xSTEP tensor, where channel 0 has a gaussian spike every time there is an \"onset\" event, and channel 1 has a gaussian spike every time there is a \"wakeup\" event.\n\n[I also tried simple regression - where I broke the signal into days and predicted just 2 float values: one for the onset time and one for the wakeup time. That didn't perform very well, even after adding positional encodings, etc :) ]", "votes": 2 }, { "id": 2510288, "postDate": "2023-11-02T21:23:13.843Z", "content": "Oh, ok… So do you mean, that we can use \"melspectrogram\" with just the CV model and then just add a custom head that outputs the tensor with 0 and 1 for the states? If I understood you correctly, this might be even better than segmentation and bounding box borders. I didn't really realize that CV models can learn these things and get the nice performance. So that's why I tried to think about classic problem types like classification, segmentation etc.
", "rawMarkdown": "Oh, ok... So do you mean, that we can use \"melspectrogram\" with just the CV model and then just add a custom head that outputs the tensor with 0 and 1 for the states? If I understood you correctly, this might be even better than segmentation and bounding box borders. I didn't really realize that CV models can learn these things and get the nice performance. So that's why I tried to think about classic problem types like classification, segmentation etc." }, { "id": 2510297, "postDate": "2023-11-02T21:32:07.673Z", "content": "yep! put a custom head on it and see how things go. It would also be interesting to compare to more traditional existing models, but I didn't experiment with that.
", "rawMarkdown": "yep! put a custom head on it and see how things go. It would also be interesting to compare to more traditional existing models, but I didn't experiment with that.", "votes": 1 } ] }, { "id": 2501940, "postDate": "2023-10-27T19:20:29.300Z", "content": "I thought giving a try to Unet (1D or 2D not sure yet) but is still down on the list of models I will try. Good to know CV does work for this data
", "rawMarkdown": "I thought giving a try to Unet (1D or 2D not sure yet) but is still down on the list of models I will try. Good to know CV does work for this data ", "votes": 1, "replies": [ { "id": 2502186, "postDate": "2023-10-28T02:06:07.083Z", "content": "I tried to implement Unet but ended giving up because the shape of input and output sequences doesn't match. Too poor at engineering skill to do it. So looking forward to learning from others output…
", "rawMarkdown": "I tried to implement Unet but ended giving up because the shape of input and output sequences doesn't match. Too poor at engineering skill to do it. So looking forward to learning from others output...", "votes": 1 }, { "id": 2502735, "postDate": "2023-10-28T12:54:54.500Z", "content": "If you implemented the rest of the unet - then you're almost there! You just need another layer (either at the start or the end) to match the shapes.
", "rawMarkdown": "If you implemented the rest of the unet - then you're almost there! You just need another layer (either at the start or the end) to match the shapes." }, { "id": 2508733, "postDate": "2023-11-02T00:27:49.073Z", "content": "@econjt x_batch.permute(0, 2, 1) is the magic ;)
", "rawMarkdown": "@econjt x_batch.permute(0, 2, 1) is the magic ;)", "votes": 1 } ] }, { "id": 2515612, "postDate": "2023-11-07T04:14:17.163Z", "content": "What type of features would you be looking to pull out from the pre-trained vision model for this task? Those features that are created will then be fed into a \"custom head\"?
", "rawMarkdown": "What type of features would you be looking to pull out from the pre-trained vision model for this task? Those features that are created will then be fed into a \"custom head\"?" } ] }, { "id": 2500389, "postDate": "2023-10-26T16:17:02.640Z", "content": "May someone tell me why this may not work? I am new.
\nThe idea is that since the onset/wakeup events in the validation data are labeled by an expert human labeler, presumably using their eyes, wouldn't it be feasible to take a computer vision approach to the charts/graphs from the data we are provided with? This approach might already be very similar to https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/441470
As you can find here: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/450468 The reason this makes sense is for 2 reasons. 1 we do not have to make predictions in real time, we get to see the entire series before making predictions. The second reason is because from a logical standpoint CNNs have the ability to see far finer patterns than a LGBM would be able to see in time series data. Finding a pattern in a sequence of numbers vs a pattern in the image of the linked post have different levels of difficulty that the model picks up on. Of course some will be able to make gradient boosted models preform well, but I would be quite sure it wont be the winning strategy here.
", "rawMarkdown": "As you can find here: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/450468 The reason this makes sense is for 2 reasons. 1 we do not have to make predictions in real time, we get to see the entire series before making predictions. The second reason is because from a logical standpoint CNNs have the ability to see far finer patterns than a LGBM would be able to see in time series data. Finding a pattern in a sequence of numbers vs a pattern in the image of the linked post have different levels of difficulty that the model picks up on. Of course some will be able to make gradient boosted models preform well, but I would be quite sure it wont be the winning strategy here. ", "votes": 2 } ], "comments": [ { "id": 2500463, "author_name": "chris", "author_url": "", "post_date": "2023-10-26T17:18:32.393000", "content": "Yes, you could try a vision approach. Usually, that involves taking an fft of the signal (look up \"melspectrogram librosa\"), then running that through a pre-trained vision model and putting a custom head on top of that. There are several previous competitions you can take a look at to get ideas like: https://www.kaggle.com/competitions/g2net-detecting-continuous-gravitational-waves
", "votes": 8, "replies": [ { "id": 2500541, "author_name": "Thomas Wynn", "author_url": "", "post_date": "2023-10-26T18:39:27.717000", "content": "Thanks Chris! Have you already considered this approach? How do you think it will fare well compared to the approaches already established? I am asking because it will probably take a long time for me to implement.
", "votes": 1, "replies": [ { "id": 2500567, "author_name": "chris", "author_url": "", "post_date": "2023-10-26T19:25:53.423000", "content": "Yes, I've tried a melspectrogram approach and it did look promising (the results were about as good as the current public models), but as you say, the pipeline is more complex, the models are bigger, and the runtime is longer.
\nSo if you feel like you want to try a vision approach just because you're interested to try it, then I think it's a valid thing to try. If you'd just like to get started and aren't tied to vision though, then I'd probably start by looking at a few of the public code samples and go from there - then you'll at least start with an end-to-end working sample. Good luck!
", "votes": 4, "replies": [] }, { "id": 2500571, "author_name": "Thomas Wynn", "author_url": "", "post_date": "2023-10-26T19:31:52.983000", "content": "Will do! Thanks a lot for your insight, Chris!
", "votes": 0, "replies": [] }, { "id": 2510218, "author_name": "Man of the year", "author_url": "", "post_date": "2023-11-02T19:49:14.787000", "content": "I also see a potential in this approach, I thought about it during the night time. Didn't find any scientific papers on this topic. However, there is one thing that I do not understand. How should we train a CV model, which target could be used? As I see it, we can imagine vertical lines, that divide images of the series on states \"sleep\" and \"wakeup\". Ok, let's say, that now we are able to segment the picture. But how do we find the \"step\" number after this? Any ideas?
", "votes": 0, "replies": [] }, { "id": 2510225, "author_name": "Thomas Wynn", "author_url": "", "post_date": "2023-11-02T19:59:00.370000", "content": "We could scale the pixel values to be proportional to a step, or to an interval of time. From there, we can just calculate where that vertical line is in time, or just calculate the step, based on the pixel coordinate. Does that make sense? Not sure if I fully understood your question
", "votes": 1, "replies": [] }, { "id": 2510232, "author_name": "Man of the year", "author_url": "", "post_date": "2023-11-02T20:03:09.167000", "content": "Yes, also thought about it. Do you think, that a segmentation or a \"bounding box\" approach here are the best options? Or did you mean other approaches in your idea?
", "votes": 0, "replies": [] }, { "id": 2510236, "author_name": "", "author_url": "", "post_date": "2023-11-02T20:08:55.887000", "content": "", "votes": 0, "replies": [] }, { "id": 2510239, "author_name": "chris", "author_url": "", "post_date": "2023-11-02T20:13:28.833000", "content": "(I replied then deleted my reply realizing I answered the wrong question):
\nThe current public approaches use either a gaussian signal for onset and wake, or a 1/0 signal for awake/sleep. A \"bounding box\" approach is interesting! It would be quite similar to the 1/0 signal for awake/sleep I believe though, since the bounding box would always be full height.
", "votes": 1, "replies": [] }, { "id": 2510241, "author_name": "Man of the year", "author_url": "", "post_date": "2023-11-02T20:20:42.110000", "content": "Yeah, correct. I thought of a segmentation firstly, since you want to detect the area of the sleep state. But then I realized, that it is usually heavy in terms of computational power. And it seems that here we basically want a straight line, so a \"bounding box\" might be enough.
\n@chris62, what approach (task type or target) did you suggest in your idea, when you described the melspectrogram? I do not understand, how to get the interval from it with the CV model.
", "votes": 0, "replies": [] }, { "id": 2510258, "author_name": "chris", "author_url": "", "post_date": "2023-11-02T20:38:24.853000", "content": "I tried both approaches that are popular in the current public notebooks (both performed about the same as far as I could tell, without a lot of trials). That is:
\noutput a 1xSTEP tensor with 0s where the person is awake, and 1s where the person is sleeping, then detect the transition.
output a 2xSTEP tensor, where channel 0 has a gaussian spike every time there is an \"onset\" event, and channel 1 has a gaussian spike every time there is a \"wakeup\" event.
[I also tried simple regression - where I broke the signal into days and predicted just 2 float values: one for the onset time and one for the wakeup time. That didn't perform very well, even after adding positional encodings, etc :) ]
", "votes": 2, "replies": [] }, { "id": 2510288, "author_name": "Man of the year", "author_url": "", "post_date": "2023-11-02T21:23:13.843000", "content": "Oh, ok… So do you mean, that we can use \"melspectrogram\" with just the CV model and then just add a custom head that outputs the tensor with 0 and 1 for the states? If I understood you correctly, this might be even better than segmentation and bounding box borders. I didn't really realize that CV models can learn these things and get the nice performance. So that's why I tried to think about classic problem types like classification, segmentation etc.
", "votes": 0, "replies": [] }, { "id": 2510297, "author_name": "chris", "author_url": "", "post_date": "2023-11-02T21:32:07.673000", "content": "yep! put a custom head on it and see how things go. It would also be interesting to compare to more traditional existing models, but I didn't experiment with that.
", "votes": 1, "replies": [] } ] }, { "id": 2501940, "author_name": "moth", "author_url": "", "post_date": "2023-10-27T19:20:29.300000", "content": "I thought giving a try to Unet (1D or 2D not sure yet) but is still down on the list of models I will try. Good to know CV does work for this data
", "votes": 1, "replies": [ { "id": 2502186, "author_name": "JumpeiT", "author_url": "", "post_date": "2023-10-28T02:06:07.083000", "content": "I tried to implement Unet but ended giving up because the shape of input and output sequences doesn't match. Too poor at engineering skill to do it. So looking forward to learning from others output…
", "votes": 1, "replies": [] }, { "id": 2502735, "author_name": "chris", "author_url": "", "post_date": "2023-10-28T12:54:54.500000", "content": "If you implemented the rest of the unet - then you're almost there! You just need another layer (either at the start or the end) to match the shapes.
", "votes": 0, "replies": [] }, { "id": 2508733, "author_name": "Tolga", "author_url": "", "post_date": "2023-11-02T00:27:49.073000", "content": "@econjt x_batch.permute(0, 2, 1) is the magic ;)
", "votes": 1, "replies": [] } ] }, { "id": 2515612, "author_name": "Thomas Wynn", "author_url": "", "post_date": "2023-11-07T04:14:17.163000", "content": "What type of features would you be looking to pull out from the pre-trained vision model for this task? Those features that are created will then be fed into a \"custom head\"?
", "votes": 0, "replies": [] } ] }, { "id": 2500465, "author_name": "Cody_Null", "author_url": "", "post_date": "2023-10-26T17:19:20.247000", "content": "As you can find here: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/450468 The reason this makes sense is for 2 reasons. 1 we do not have to make predictions in real time, we get to see the entire series before making predictions. The second reason is because from a logical standpoint CNNs have the ability to see far finer patterns than a LGBM would be able to see in time series data. Finding a pattern in a sequence of numbers vs a pattern in the image of the linked post have different levels of difficulty that the model picks up on. Of course some will be able to make gradient boosted models preform well, but I would be quite sure it wont be the winning strategy here.
", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "2500463": "Yes, you could try a vision approach. Usually, that involves taking an fft of the signal (look up \"melspectrogram librosa\"), then running that through a pre-trained vision model and putting a custom head on top of that. There are several previous competitions you can take a look at to get ideas like: https://www.kaggle.com/competitions/g2net-detecting-continuous-gravitational-waves", "2500389": "May someone tell me why this may not work? I am new.\nThe idea is that since the onset/wakeup events in the validation data are labeled by an expert human labeler, presumably using their eyes, wouldn't it be feasible to take a computer vision approach to the charts/graphs from the data we are provided with? This approach might already be very similar to [https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/441470](url)", "2500465": "As you can find here: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/450468 The reason this makes sense is for 2 reasons. 1 we do not have to make predictions in real time, we get to see the entire series before making predictions. The second reason is because from a logical standpoint CNNs have the ability to see far finer patterns than a LGBM would be able to see in time series data. Finding a pattern in a sequence of numbers vs a pattern in the image of the linked post have different levels of difficulty that the model picks up on. Of course some will be able to make gradient boosted models preform well, but I would be quite sure it wont be the winning strategy here. " } }