# 48th Place Solution

First of all, thank you for organizing such a fantastic competition! Kudos to the organizers and all the participants for their hard work!
Here’s a brief overview of my approach.
Models Used
I used an ensemble of transformer, GRU, and squeezeformer models. The models were divided into two categories: IMU only and Full data. For Full data, I used only GRU models. Below is the structure of the GRU model (other models are quite similar).
![GRU_model](assets/f2ab3d7d8e2faa7c5226cf5e.png)
A key feature was the addition of a 1D CNN layer after the input, rather than using GRU alone. This significantly boosted the cross-validation (CV) score.
Best Submission (Overview)
IMU only: 2 transformer models + 2 GRU models + 1 squeezeformer model
Full data: 10 GRU models
Feature Engineering
Since this is time-series data, I mainly added features that incorporate information beyond the current row. For example, applying a rolling standard deviation to acc_x improved CV. I also included centered rolling statistics:
train_df[] = train_df.groupby()[].transform( x: x.rolling(window=, min_periods=).std())
train_df[] = train_df.groupby()[].transform( x: x.rolling(window=, center=, min_periods=).std())
Interestingly, something unexpected happened: I accidentally added a feature from thm twice, and for some reason, CV improved by 0.01. Although the reason is unclear, I kept it since it enhanced performance.
EMA
Incorporating EMA helped improve CV while also enhancing generalization.
Null Handling
The tof data contained a fair amount of missing values, so proper null handling was essential. I used linear interpolation and median imputation to fill in missing values smoothly. Here’s the code used for null handling during training:
group_prefix  tof_groups:
    group_cols = [col  col  train_df.columns  col.startswith(group_prefix)]
    train_df[group_cols] = train_df[group_cols].replace(-, np.nan)
    train_df[group_cols] = train_df[group_cols].apply(
         row: row.interpolate(method=, limit_direction=, limit=), axis=
    )
    train_df[group_cols] = train_df[group_cols].apply(
         row: row.fillna(row.median()), axis=
    )
    train_df[group_cols] = train_df[group_cols].fillna(-)
Mixup
I set the mixup alpha to 1.0. This was intended to improve generalization, and it did lead to better CV.
Best Models and Final Models
Models with early stopping were treated as “best models,” while those without were considered “final models.” Both types were used in the ensemble. Here’s the breakdown:
IMU only:
2 transformer models (best & final)
2 GRU models (best & final)
1 squeezeformer model (final only)
Full data:
10 GRU models (2 versions × 5 seeds: best & final)
Ensemble Method
I used simple averaging for the ensemble.
What Didn’t Work
Auxiliary Loss for orientation: It slightly improved CV but worsened the leaderboard (LB) score, so I didn’t adopt it.
Thanks for reading!