# LB First Place Solution

Ensemble of Models Using Two Autoregressive Approaches
Approach 1: Predicting One Timestep at a Time
In this approach, the models use 5 timesteps of all input data(wind speed (ustar), temperature (theta), and fire-line location (xi), slope (alpha) and mean wind speed (u10)) to predict xi for the next timestep. The predicted xi is then appended, and the oldest xi is discarded to predict the next timestep. This process is repeated to predict the required number of timesteps.
Models:
1. Residual CNN:
Uses residual blocks with skip connections.
It consists of initial convolutions, batch normalization, ReLU activations, residual blocks, and a final convolution for prediction.
Reshapes the input to [batch_size, num_steps, channels, height, width] for multi-step forecasting.
2. Self-Attention Residual CNN:
Combines residual blocks with a self-attention mechanism to capture dependencies across feature maps.
Residual blocks extract spatial features, and self-attention highlights important spatial regions.
It applies the self-attention mechanism to the feature maps before producing the output via a convolution layer.
3. Self-Attention ConvLSTM:
Integrates ConvLSTM cells with self-attention.
Uses stacked ConvLSTM cells to process sequential inputs, with self-attention enhancing spatial focus.
The model outputs the hidden state of the last timestep after applying a convolution to predict xi.
Approach 2: Predicting Multiple Timesteps at Once
In this approach, the models predict xi for the next 20 timesteps using all input data(wind speed (ustar), temperature (theta), and fire-line location (xi), slope (alpha) and mean wind speed (u10)) from the last 5 timesteps. Last 5 timesteps from predicted 20 timesteps are used to predict the next 20 timesteps. This autoregressive process repeats to predict the required number of timesteps.
Model:
1. Residual CNN (Multi-Timestep Prediction):
Similar structure to Approach 1, but modified to predict multiple future timesteps (20) in one forward pass.
Utilizes residual blocks to process input sequences and a final convolutional layer to generate predictions for multiple future steps.
Both approaches are autoregressive and can predict any number of timesteps by iteratively updating predictions. The inference code supports flexible timestep forecasting.
Training and Inference code:
[https://drive.google.com/drive/folders/1EIcfL37vR9eAaGsANDbbCkZZBf7uwU9Q?usp=drive_link](https://drive.google.com/drive/folders/1EIcfL37vR9eAaGsANDbbCkZZBf7uwU9Q?usp=drive_link)
Weights -
[https://www.kaggle.com/datasets/ajayasaithambi/residualcnn20-200-seq-len-5-1050](https://www.kaggle.com/datasets/ajayasaithambi/residualcnn20-200-seq-len-5-1050)
[https://www.kaggle.com/datasets/ajayasaithambi/selfattconvlstm-seq-len-5](https://www.kaggle.com/datasets/ajayasaithambi/selfattconvlstm-seq-len-5)
[https://www.kaggle.com/datasets/ajayasaithambi/residualcnnsa-seq-len-5](https://www.kaggle.com/datasets/ajayasaithambi/residualcnnsa-seq-len-5)
[https://www.kaggle.com/datasets/ajayasaithambi/residualcnn-seq-len-5](https://www.kaggle.com/datasets/ajayasaithambi/residualcnn-seq-len-5)
[https://www.kaggle.com/datasets/ajayasaithambi/residualcnnsa20-200-seq-len-5-6](https://www.kaggle.com/datasets/ajayasaithambi/residualcnnsa20-200-seq-len-5-6)