Ensemble of Models Using Two Autoregressive Approaches
\nApproach 1: Predicting One Timestep at a Time
\nIn this approach, the models use 5 timesteps of all input data(wind speed (ustar), temperature (theta), and fire-line location (xi), slope (alpha) and mean wind speed (u10)) to predict xi for the next timestep. The predicted xi is then appended, and the oldest xi is discarded to predict the next timestep. This process is repeated to predict the required number of timesteps.
\nModels:
\n1. Residual CNN:
\nUses residual blocks with skip connections.
\nIt consists of initial convolutions, batch normalization, ReLU activations, residual blocks, and a final convolution for prediction.
\nReshapes the input to [batch_size, num_steps, channels, height, width] for multi-step forecasting.
2. Self-Attention Residual CNN:
\nCombines residual blocks with a self-attention mechanism to capture dependencies across feature maps.
\nResidual blocks extract spatial features, and self-attention highlights important spatial regions.
\nIt applies the self-attention mechanism to the feature maps before producing the output via a convolution layer.
3. Self-Attention ConvLSTM:
\nIntegrates ConvLSTM cells with self-attention.
\nUses stacked ConvLSTM cells to process sequential inputs, with self-attention enhancing spatial focus.
\nThe model outputs the hidden state of the last timestep after applying a convolution to predict xi.
Approach 2: Predicting Multiple Timesteps at Once
\nIn this approach, the models predict xi for the next 20 timesteps using all input data(wind speed (ustar), temperature (theta), and fire-line location (xi), slope (alpha) and mean wind speed (u10)) from the last 5 timesteps. Last 5 timesteps from predicted 20 timesteps are used to predict the next 20 timesteps. This autoregressive process repeats to predict the required number of timesteps.
\nModel:
\n1. Residual CNN (Multi-Timestep Prediction):
\nSimilar structure to Approach 1, but modified to predict multiple future timesteps (20) in one forward pass.
\nUtilizes residual blocks to process input sequences and a final convolutional layer to generate predictions for multiple future steps.
Both approaches are autoregressive and can predict any number of timesteps by iteratively updating predictions. The inference code supports flexible timestep forecasting.
\nTraining and Inference code: https://drive.google.com/drive/folders/1EIcfL37vR9eAaGsANDbbCkZZBf7uwU9Q?usp=drive_link
\nWeights -
\nhttps://www.kaggle.com/datasets/ajayasaithambi/residualcnn20-200-seq-len-5-1050
\nhttps://www.kaggle.com/datasets/ajayasaithambi/selfattconvlstm-seq-len-5
\nhttps://www.kaggle.com/datasets/ajayasaithambi/residualcnnsa-seq-len-5
\nhttps://www.kaggle.com/datasets/ajayasaithambi/residualcnn-seq-len-5
\nhttps://www.kaggle.com/datasets/ajayasaithambi/residualcnnsa20-200-seq-len-5-6
In the labels there is heavily imbalance,Any steps used to overcome
\nWhich loss function you used?
Congratulations!, Kudos for using pure pytorch
", "votes": 0, "replies": [] }, { "id": 3022089, "author_name": "Rafał Pawłowski", "author_url": "", "post_date": "2024-10-19T08:54:58.557000", "content": "Congratulations! Did you check it for long time predictions? I have tried one by one step prediction, but my signal disappeared after some rounds.
", "votes": 0, "replies": [ { "id": 3022124, "author_name": "Ajay Asaithambi", "author_url": "", "post_date": "2024-10-19T10:00:12.130000", "content": "Yes i checked with 145 timesteps in training data. It worked fine(MSE only slightly increased).
\n\n", "votes": 1, "replies": [] } ] } ], "comment_count": 4, "messages": [ { "id": 3022078, "postDate": "2024-10-19T08:35:52.527Z", "content": "\n\nI have tried one by one step prediction, but my signal disappeared after some rounds.
\n
\n Did it do well for 20 timesteps?
Ensemble of Models Using Two Autoregressive Approaches
\nApproach 1: Predicting One Timestep at a Time
\nIn this approach, the models use 5 timesteps of all input data(wind speed (ustar), temperature (theta), and fire-line location (xi), slope (alpha) and mean wind speed (u10)) to predict xi for the next timestep. The predicted xi is then appended, and the oldest xi is discarded to predict the next timestep. This process is repeated to predict the required number of timesteps.
\nModels:
\n1. Residual CNN:
\nUses residual blocks with skip connections.
\nIt consists of initial convolutions, batch normalization, ReLU activations, residual blocks, and a final convolution for prediction.
\nReshapes the input to [batch_size, num_steps, channels, height, width] for multi-step forecasting.
2. Self-Attention Residual CNN:
\nCombines residual blocks with a self-attention mechanism to capture dependencies across feature maps.
\nResidual blocks extract spatial features, and self-attention highlights important spatial regions.
\nIt applies the self-attention mechanism to the feature maps before producing the output via a convolution layer.
3. Self-Attention ConvLSTM:
\nIntegrates ConvLSTM cells with self-attention.
\nUses stacked ConvLSTM cells to process sequential inputs, with self-attention enhancing spatial focus.
\nThe model outputs the hidden state of the last timestep after applying a convolution to predict xi.
Approach 2: Predicting Multiple Timesteps at Once
\nIn this approach, the models predict xi for the next 20 timesteps using all input data(wind speed (ustar), temperature (theta), and fire-line location (xi), slope (alpha) and mean wind speed (u10)) from the last 5 timesteps. Last 5 timesteps from predicted 20 timesteps are used to predict the next 20 timesteps. This autoregressive process repeats to predict the required number of timesteps.
\nModel:
\n1. Residual CNN (Multi-Timestep Prediction):
\nSimilar structure to Approach 1, but modified to predict multiple future timesteps (20) in one forward pass.
\nUtilizes residual blocks to process input sequences and a final convolutional layer to generate predictions for multiple future steps.
Both approaches are autoregressive and can predict any number of timesteps by iteratively updating predictions. The inference code supports flexible timestep forecasting.
\nTraining and Inference code: https://drive.google.com/drive/folders/1EIcfL37vR9eAaGsANDbbCkZZBf7uwU9Q?usp=drive_link
\nWeights -
\nhttps://www.kaggle.com/datasets/ajayasaithambi/residualcnn20-200-seq-len-5-1050
\nhttps://www.kaggle.com/datasets/ajayasaithambi/selfattconvlstm-seq-len-5
\nhttps://www.kaggle.com/datasets/ajayasaithambi/residualcnnsa-seq-len-5
\nhttps://www.kaggle.com/datasets/ajayasaithambi/residualcnn-seq-len-5
\nhttps://www.kaggle.com/datasets/ajayasaithambi/residualcnnsa20-200-seq-len-5-6
In the labels there is heavily imbalance,Any steps used to overcome
\nWhich loss function you used?
Congratulations!, Kudos for using pure pytorch
", "rawMarkdown": "Congratulations!, Kudos for using pure pytorch" }, { "id": 3022089, "postDate": "2024-10-19T08:54:58.557Z", "content": "Congratulations! Did you check it for long time predictions? I have tried one by one step prediction, but my signal disappeared after some rounds.
", "rawMarkdown": "Congratulations! Did you check it for long time predictions? I have tried one by one step prediction, but my signal disappeared after some rounds. ", "replies": [ { "id": 3022124, "postDate": "2024-10-19T10:00:12.130Z", "content": "Yes i checked with 145 timesteps in training data. It worked fine(MSE only slightly increased).
\n\n", "rawMarkdown": "Yes i checked with 145 timesteps in training data. It worked fine(MSE only slightly increased).\n\n>> I have tried one by one step prediction, but my signal disappeared after some rounds.\nDid it do well for 20 timesteps?", "votes": 1 } ] } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T13:44:09Z", "competition_id": 85210, "competition_slug": "2024-flame-ai-challenge", "team_id": 12656596, "solution_url": "/competitions/2024-flame-ai-challenge/writeups/ajay-asaithambi-lb-first-place-solution" }\n\nI have tried one by one step prediction, but my signal disappeared after some rounds.
\n
\n Did it do well for 20 timesteps?