# 2nd Place Solution – Dig4Bio Raman Transfer Learning Challenge

Notebook: final-submit-kaggle :
[https://www.kaggle.com/code/daosyduyminh/final-submit-kaggle](https://www.kaggle.com/code/daosyduyminh/final-submit-kaggle)
Core idea: build two independent pipelines (one heavy feature-engineering pipeline, one PCA/tree pipeline), train/stack them separately, then do a weighted blend at the end.
Data & Cleaning
Used the official files:
transfer_plate.csv (train, 3 targets: Glucose, Sodium Acetate, Magnesium Sulfate)
96_samples.csv (test)
Renamed spectral columns to numeric (0…2047) and forward-filled sample_id.
Spectral values sometimes came as strings like [ … ], so they were stripped and cast to numeric.
Missing spectral points were interpolated row-wise → guarantees consistent input shape.
Model 1 – Upgraded FE Pipeline
Goal: make the spectra easier for tree models by generating several “views” of the same signal.
Steps:
Advanced spectral preprocessing: baseline correction (airPLS), SNV/standardization, plus Savitzky–Golay derivatives (order 1–2) to highlight edges/peaks.
Feature views built from the cleaned spectra:
baseline+SNV version
1st and 2nd derivative
peak features (count, summed intensity, avg width)
statistical features (mean, std, skew, kurtosis)
→ combined into 3 main feature sets: Combined_All, Combined_Processed, Mean_Processed.
Models: for each of the 3 targets, train tuned ExtraTreesRegressor with shallower depth, larger leaf size, bootstrap and small ccp_alpha to reduce overfitting (because we only have 96 training samples).
5-fold CV to get OOF predictions.
Per-target ensemble inside this pipeline: tried 6 methods (simple avg, performance-weighted, optimal weights, linear stacking, ridge stacking, RF stacking) and picked the best CV R² per target. For Glucose and Sodium Acetate, stacking was clearly better.
This pipeline alone reaches ~0.90+ CV depending on the target.
Model 2 – PCA + Linear / Trees
This is a lighter, more “classical ML” view to complement Model 1.
Steps:
Lighter preprocessing (baseline + SNV) to keep the signal shape but reduce device drift.
Recreated similar feature sets (combined, derivative, stats, peaks) so the two pipelines stay comparable.
Added a PCA → LinearRegression branch to get a low-variance linear model.
Also trained ExtraTrees on the combined features, but with slightly different hyperparams than Model 1.
Again, 5-fold CV + per-target stacking (linear / ridge / RF).
In this notebook, this pipeline actually got slightly higher CV (~0.927), so we’ll give it a higher weight in the final blend.
Final Blending
After both pipelines finished, we had:
model1_linear_stacking.csv
model2_linear_stacking.csv
Both files:
have the same 96 IDs
are already post-processed (non-negative, clipped to the 1–99% range of train)
Final submission:
submission = 0.45 * model1 + 0.55 * model2
Model 2 got 0.55 because its CV was slightly better and more stable, but Model 1 brings useful FE-based diversity.
Tried but not used
PLS / classical spectroscopy regressors → didn’t beat the tree+stacking setup on 5-fold CV.
Very large ensembles → too easy to overfit on 96 samples.
DL → dataset too small, tabular+FE wins here.
The solution wins mainly because of:
careful spectral cleaning,
multiple feature views → tree-friendly input,
OOF-based stacking per target, and
a final 2-model weighted blend (0.45 / 0.55).