# [5th Public LB | 11th Private LB] Solution

First of all, I would like to thank
[@lazylo](https://www.kaggle.com/lazylo)
for the interesting competition with a really tricky evaluation metric. I would also like to thank
[@ambrosm](https://www.kaggle.com/ambrosm)
for his simple
[baseline](https://www.kaggle.com/code/ambrosm/red-explained-baseline)
and his suggestions to keep it simple and be careful about overfitting to the public LB.
OVERVIEW OF MY SOLUTION
My solution was quite simple: I used 12 logistic regressions (1 for each horizon) along with Isotonic calibration, to classify demand into zero vs non-zero and better calibrate the probabilities; after that, I fit 12 ridge regressions (1 for each horizon) on log1p of the positives to predict positive demand. Technically the ridge predicts demand for all 1152 predictions but they are overwritten by zeros if the calibrated prob_zero exceeds a certain global threshold.
It uses 38 features, which are created from the target itself (so lags, rolling features, etc), and calendar-based features to help learn patterns based on the calendar month. So I didn't end up using any of the other datasets we were given because I wanted to try and keep it simple, but both methods were viable I guess.
For the Logistic C and the Ridge's Alpha I used Optuna to optimize the average comp score across validation folds and got the values I ended up using, but optimizing the average comp score across folds still wasn't a reliable enough way of choosing hyperparameter values. Ironically, the same solution with different values of alpha and C, which I randomly chose, ended up outperforming my current strategy (by a lot) on the private LB.
The key in this solution was all the discounting of the positive magnitudes that was done: specifically, the positive magnitudes are discounted by (1-p0) where p0 is the probability of being zero estimated by the Logistic regression with Isotonic calibration; also, while generating predictions, we apply global multiplicative factors by horizon, to correct any bias that is found; finally, some predictions were capped to not exceed 3x the previous prediction.
The final tweak was to seasonally nudge the predictions based on the average demand by sector-month, which I found to be the only thing that consistently improved CV, Public LB and Private LB.
IMPROVEMENTS OF MY SOLUTION
I also ran this strategy with 53 features, using volatility-related features. That ends up improving the zero-classification substantially but hurts the positive magnitudes. I ended up using these 53 features to get the prob_zero for the predictions, but the key here was not to only consider the absolute values of the probabilities but the increments between months. By considering the increments along with the absolute values, I ended up improving my Public LB score by almost 0.004. I didn't end up using this strategy for the zeros in the Private LB although it would have helped and given me a considerable boost in hindsight, but it's definitely a cool idea that I never thought of.
EVERYTHING I TRIED
I have tried lots of different strategies, and most of them performed really well on Public LB, but very few ended up getting good scores on Private LB. I will list most of the things I tried and then I'll highlight those that worked best in terms of Private LB:
Arithmetic Mean (the approach
[@ambrosm](https://www.kaggle.com/ambrosm)
came up with)
Geometric Mean
Harmonic Mean
Exponentially-weighted variations of the means mentioned above.
Seasonal nudge to the exponentially-weighted variants.
Rolling Means (Arithmetic, Geometric, Harmonic).
Nudging predictions based on Lag-Lead correlations (partial correlations too).
Rolling tree methods (XGBoost, CatBoost, LightGBM)
Static and rolling trimmed mean
Static and rolling median
Classifier + Rolling Median
Classifier + Rolling Harmonic Mean
Using Chinese calendar features ( like Chinese New Year and other festivities).
Rolling ridge, rolling lasso, rolling PLS.
Horizon Ridge
Horizon Lasso
Horizon PLS
Elastic Net
Chronos
LSTM
GAM Spline
Zero-Inflated Negative Binomial
Horizon Catboost
Horizon XGBoost
Tried different classifiers + regressors (so Random Forest, Catboost, etc,)
Tried ensembles of past solutions
MoE with PLS and Ridge
MoE with more experts
There's more but these were the most promising
WHAT ACTUALLY WORKED IN TERMS OF PRIVATE LB
Rolling Median with classifier and a global multiplier (0.9) - Score 0.51866 on Private LB.
Same strategy with no global multiplier - Score around 0.49 on Private LB.
Using chinese festivities tanked the Public LB but actually scored about 0.503 on Private LB.
Horizon-XGBoost with 0.9 global multiplier - About 0.47/0.48 on Private LB.
Same strategy with more regularization scores 0.5 on Private LB.
Horizon-PLS scores around 0.49/0.5 on Private LB.
Simple MLP scores 0.45/0.46 on Private LB.
Lastly, anything that is Horizon-Ridge scored well on Private LB. More regularization on the Ridge's Alpha scored over 0.51 on Private LB.