How can I put 4d data into an LSTM? Lets say my data has 30 sequence, 100000-100000 subsequences in each, 20 steps per subsequence, 10 features per step. So (30,100k-200k,20,10). Should I make every sequence its own epoch? Train in steps?
", "messages": [ { "id": 2452109, "postDate": "2023-09-23T05:24:28.383Z", "content": "I'm wondering why is it necessary to involve subsequences?
\nGenerally for time series data, the input for LSTM is 3D: (batch_size, max_seq_len, num_feats)
3D input of (batch_size, max_seq_len, num_feats) doesn't make sense to me in this situation because I would only get one output per sequence. With subsequences, I can predict the sleep state at each step of the sequence. The \"awake\" value of every step in a sequence would have a subsequence. I guess I could lump all the subsequences of each sequence together, making it 3d. This leads to a deeper question I have about how to maintain continuity between subsequences. When I pass each subsequence into an LSTM layer, I get an output. But next the subsequence I process will have no connection to the previous. I guess my question really is, is how can I process each subsequence so that the next subsequence will take into account the previous subsequence that was just processed?
", "rawMarkdown": "3D input of (batch_size, max_seq_len, num_feats) doesn't make sense to me in this situation because I would only get one output per sequence. With subsequences, I can predict the sleep state at each step of the sequence. The \"awake\" value of every step in a sequence would have a subsequence. I guess I could lump all the subsequences of each sequence together, making it 3d. This leads to a deeper question I have about how to maintain continuity between subsequences. When I pass each subsequence into an LSTM layer, I get an output. But next the subsequence I process will have no connection to the previous. I guess my question really is, is how can I process each subsequence so that the next subsequence will take into account the previous subsequence that was just processed?", "votes": 1 }, { "id": 2453057, "postDate": "2023-09-23T19:06:45.747Z", "content": "@aleksanderklimczyk CMIIW but if your implementation is in Tensorflow, you can pass a return_sequences=True to the constructor though. By this you get output for each step of the sequence, and avoid \"… I would only get one output per sequence\".
I see. Thank you
", "rawMarkdown": "I see. Thank you" }, { "id": 2453251, "postDate": "2023-09-23T22:52:47.963Z", "content": "After further reading, I will now try to implement a Bidirectional LSTM.
", "rawMarkdown": "After further reading, I will now try to implement a Bidirectional LSTM.", "votes": 1 } ] } ] }, { "id": 2452049, "postDate": "2023-09-23T04:07:45.320Z", "content": "How can I put 4d data into an LSTM? Lets say my data has 30 sequence, 100000-100000 subsequences in each, 20 steps per subsequence, 10 features per step. So (30,100k-200k,20,10). Should I make every sequence its own epoch? Train in steps?
", "rawMarkdown": "How can I put 4d data into an LSTM? Lets say my data has 30 sequence, 100000-100000 subsequences in each, 20 steps per subsequence, 10 features per step. So (30,100k-200k,20,10). Should I make every sequence its own epoch? Train in steps?", "votes": 1 } ], "comments": [ { "id": 2452109, "author_name": "Rimba Erlangga", "author_url": "", "post_date": "2023-09-23T05:24:28.383000", "content": "I'm wondering why is it necessary to involve subsequences?
\nGenerally for time series data, the input for LSTM is 3D: (batch_size, max_seq_len, num_feats)
3D input of (batch_size, max_seq_len, num_feats) doesn't make sense to me in this situation because I would only get one output per sequence. With subsequences, I can predict the sleep state at each step of the sequence. The \"awake\" value of every step in a sequence would have a subsequence. I guess I could lump all the subsequences of each sequence together, making it 3d. This leads to a deeper question I have about how to maintain continuity between subsequences. When I pass each subsequence into an LSTM layer, I get an output. But next the subsequence I process will have no connection to the previous. I guess my question really is, is how can I process each subsequence so that the next subsequence will take into account the previous subsequence that was just processed?
", "votes": 1, "replies": [] }, { "id": 2453057, "author_name": "Rimba Erlangga", "author_url": "", "post_date": "2023-09-23T19:06:45.747000", "content": "@aleksanderklimczyk CMIIW but if your implementation is in Tensorflow, you can pass a return_sequences=True to the constructor though. By this you get output for each step of the sequence, and avoid \"… I would only get one output per sequence\".
I see. Thank you
", "votes": 0, "replies": [] }, { "id": 2453251, "author_name": "Aleksander Klimczyk", "author_url": "", "post_date": "2023-09-23T22:52:47.963000", "content": "After further reading, I will now try to implement a Bidirectional LSTM.
", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2452109": "I'm wondering why is it necessary to involve subsequences?\n\nGenerally for time series data, the input for LSTM is 3D: `(batch_size, max_seq_len, num_feats)`", "2452049": "How can I put 4d data into an LSTM? Lets say my data has 30 sequence, 100000-100000 subsequences in each, 20 steps per subsequence, 10 features per step. So (30,100k-200k,20,10). Should I make every sequence its own epoch? Train in steps?" } }