# [1st Place Solution] Two-Stage Models

Hi everyone,
First of all, I want to thank
[@lazylo](https://www.kaggle.com/lazylo)
for organizing this competition. The setup and evaluation process encouraged me to be both creative and precise, and I truly had a great time working on it.
Overview
I think the most important factor in the competition was the evaluation metric. In
[@ambrosm](https://www.kaggle.com/ambrosm)
’s
[explanation](https://www.kaggle.com/code/ambrosm/red-explained-baseline)
(thanks for the insight), he talked about how important the zeros are. For that reason, I first focused on identifying the months and sectors with no sales. After that, I ran regression to predict the amounts for the months that do have sales. In the final step, I ensembled the models and submitted the result. Across all these steps, I trained XGBoost, LightGBM, and CatBoost variants for T+1 to T+12. I did not use any external data beyond the datasets provided by the competition.
Feature Engineering
Feature engineering centered on seasonality and time dynamics. I used sine/cosine encodings for both the base month and the label month (t+h), added short Fourier terms (periods 12/6/4), and created a rich set of sector-wise lags and moving averages (e.g., 1,2,3,6,9,12,13 and MA(3,6,9,12,18)). I included safe YoY deltas with clipping to control outliers, momentum/volatility signals (EMA9, STD9, simple mom/roc), a seasonal anchor via lag12, and ratios such as MA12/MA6. To capture structural similarity across time, I built “analog-year” features: for each sector I compared the last W=9 months to historical windows whose futures lie strictly in the past, then used the (+h) outcomes of the nearest matches. POI variables were compressed with PCA to provide compact sector embeddings.
Modeling & Validation
Modeling followed a direct, per horizon approach with two stages. First, a classifier predicts whether the target is zero or positive; second, a regressor predicts the magnitude on log1p(y) for the positive cases (converted back with expm1). Thresholds for “sale vs. no-sale” were chosen using a blend of PR-curve candidates and a fixed grid. To stabilize outputs I clipped predictions by sector at the 99.5th historical percentile and floored negatives at zero. I did all of this by training separate models for T+1, T+2, …, T+12 and using different validation folds for each target month.
In some models, I used the same feature engineering steps and trained directly for T+1 to T+12. In many CatBoost models, I applied more detailed feature engineering for specific periods, such as tailoring the lag values to the target months. For example, I built a CatBoost notebook that predicts T+4 to T+8, and in these models I adjusted the lags specifically for those horizons to improve performance in that range.
Ensemble
After training the models, I ensembled them based on both classification and regression. I used a majority rule in the ensemble. For example, when combining four models, if the majority predicts zero sales for a given sector and month, I set the final prediction to zero; if there is a tie, I decide according to the weight proportions. Then, for months where sales are predicted, I aggregate the regression outputs using the specified weights to produce the final estimate.
Conclusion
Of the two files I submitted to the competition, one was the solution that achieved my highest public score, and the other was the solution that secured first place on the private leaderboard. I did not have the opportunity to try neural networks (NN) or other methods. Overall, I think CatBoost was more effective in this competition. Without any ensemble and with lighter feature engineering, the CatBoost model I trained for T+1 to T+12 (trained in a for loop, so the same hyperparameters were used across horizons) achieved a 0.51999 private score.
Thanks to the organizers and the community for an excellent competition.