{ "id": 287533, "title": "Time Series Modeling - Starter Notebooks", "url": "/competitions/g-research-crypto-forecasting/discussion/287533", "author_name": "", "post_date": "2021-11-14T13:43:46.334368500Z", "votes": 10, "comment_count": 1, "views": 0, "content": "

'

\n

Hi Guys,

\n

I extended \"[G-Research] Parallel LSTM training\" by DEVOLNYX [1], and prepared some sequence-based models starter notebooks.

\n

The Basic idea of those models is to re-arrange the data into windows of sequences, and labels.
\nWe basically create a sliding window that builds sequences and labels. So we have sequences of WINDOW_SIZE minutes, that will use to predict the next 15 minutes (from the competition tutorial).
\nBecause of the nature of this competition, we will implement the idea of using the parallel sequence model branches [This is where DEVOLNYX's great notebooks come in]. One branch for each asset then uses the representation learned to output a vector of targets for each timestep.

\n
\n

Summarizing the framework:

\n\n
\n

The Notebooks:

\n\n
\n

I used 1 epoch to train the weights since we are all a bit tight on compute time in this competition. There is much room for improvements, for example, different hyperparameters, more epochs, augmentations, different models, more data…

\n

[1] The original notebook can be found here

\n

Hope it helps fellow Kagglers!
\nHappy Kaggling!

", "messages": [ { "id": "1582044", "postDate": "11/14/2021 13:43:46", "content": "

'

\n

Hi Guys,

\n

I extended \"[G-Research] Parallel LSTM training\" by DEVOLNYX [1], and prepared some sequence-based models starter notebooks.

\n

The Basic idea of those models is to re-arrange the data into windows of sequences, and labels.
\nWe basically create a sliding window that builds sequences and labels. So we have sequences of WINDOW_SIZE minutes, that will use to predict the next 15 minutes (from the competition tutorial).
\nBecause of the nature of this competition, we will implement the idea of using the parallel sequence model branches [This is where DEVOLNYX's great notebooks come in]. One branch for each asset then uses the representation learned to output a vector of targets for each timestep.

\n
\n

Summarizing the framework:

\n\n
\n

The Notebooks:

\n\n
\n

I used 1 epoch to train the weights since we are all a bit tight on compute time in this competition. There is much room for improvements, for example, different hyperparameters, more epochs, augmentations, different models, more data…

\n

[1] The original notebook can be found here

\n

Hope it helps fellow Kagglers!
\nHappy Kaggling!

", "rawMarkdown": "'\n\nHi Guys, \n\nI extended \"[G-Research] Parallel LSTM training\" by DEVOLNYX [1], and prepared some sequence-based models starter notebooks.\n\nThe Basic idea of those models is to re-arrange the data into windows of sequences, and labels. \nWe basically create a sliding window that builds sequences and labels. So we have sequences of `WINDOW_SIZE` minutes, that will use to predict the next 15 minutes (from the competition tutorial).\nBecause of the nature of this competition, we will implement the idea of using the parallel sequence model branches [This is where DEVOLNYX's great notebooks come in]. One branch for each asset then uses the representation learned to output a vector of targets for each timestep.\n____\n**Summarizing the framework:**\n\n* Fill gaps in the training set like is described in the competition tutorial notebook.\n* Use `TimeseriesGenerator` generator class to build the dataset.\n* Build the model structure:\n - Lambda layer needed for assets separation;\n - Masking layer. Generated records (filled gaps) have zeros as features values, so they are not used in the computations.\n - Our sequence model architecture [For example LSTM]\n - Concatenation layer\n - Dense layer\n____\n**The Notebooks:**\n- [LSTM](https://www.kaggle.com/yamqwe/time-series-modeling-lstm)\n- [WaveNet](https://www.kaggle.com/yamqwe/time-series-modeling-wavenet)\n- [Multivariate Time Series Transformer](https://www.kaggle.com/yamqwe/the-multivariate-time-series-transformer)\n- [N-BEATS](https://www.kaggle.com/yamqwe/crypto-forecasting-n-beats)\n- [DeepAR](https://www.kaggle.com/yamqwe/probabilistic-forecasting-deepar/)\n____\n\nI used 1 epoch to train the weights since we are all a bit tight on compute time in this competition. There is much room for improvements, for example, different hyperparameters, more epochs, augmentations, different models, more data…\n\n[1] The original notebook can be found [here](https://www.kaggle.com/vmuzhichenko/g-research-parallel-lstm-training)\n\nHope it helps fellow Kagglers!\nHappy Kaggling!", "votes": null }, { "id": "1630983", "postDate": "12/27/2021 22:52:05", "content": "

404 - https://www.kaggle.com/yamqwe/the-multivariate-time-series-transformer

", "rawMarkdown": "404 - https://www.kaggle.com/yamqwe/the-multivariate-time-series-transformer", "votes": null } ], "comments": [ { "id": 1630983, "author_name": "vladimircape", "author_url": "", "post_date": "12/27/2021 22:52:05", "content": "

404 - https://www.kaggle.com/yamqwe/the-multivariate-time-series-transformer

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1582044": "'\n\nHi Guys, \n\nI extended \"[G-Research] Parallel LSTM training\" by DEVOLNYX [1], and prepared some sequence-based models starter notebooks.\n\nThe Basic idea of those models is to re-arrange the data into windows of sequences, and labels. \nWe basically create a sliding window that builds sequences and labels. So we have sequences of `WINDOW_SIZE` minutes, that will use to predict the next 15 minutes (from the competition tutorial).\nBecause of the nature of this competition, we will implement the idea of using the parallel sequence model branches [This is where DEVOLNYX's great notebooks come in]. One branch for each asset then uses the representation learned to output a vector of targets for each timestep.\n____\n**Summarizing the framework:**\n\n* Fill gaps in the training set like is described in the competition tutorial notebook.\n* Use `TimeseriesGenerator` generator class to build the dataset.\n* Build the model structure:\n - Lambda layer needed for assets separation;\n - Masking layer. Generated records (filled gaps) have zeros as features values, so they are not used in the computations.\n - Our sequence model architecture [For example LSTM]\n - Concatenation layer\n - Dense layer\n____\n**The Notebooks:**\n- [LSTM](https://www.kaggle.com/yamqwe/time-series-modeling-lstm)\n- [WaveNet](https://www.kaggle.com/yamqwe/time-series-modeling-wavenet)\n- [Multivariate Time Series Transformer](https://www.kaggle.com/yamqwe/the-multivariate-time-series-transformer)\n- [N-BEATS](https://www.kaggle.com/yamqwe/crypto-forecasting-n-beats)\n- [DeepAR](https://www.kaggle.com/yamqwe/probabilistic-forecasting-deepar/)\n____\n\nI used 1 epoch to train the weights since we are all a bit tight on compute time in this competition. There is much room for improvements, for example, different hyperparameters, more epochs, augmentations, different models, more data…\n\n[1] The original notebook can be found [here](https://www.kaggle.com/vmuzhichenko/g-research-parallel-lstm-training)\n\nHope it helps fellow Kagglers!\nHappy Kaggling!", "1630983": "404 - https://www.kaggle.com/yamqwe/the-multivariate-time-series-transformer" }, "source": "meta" }