# 3rd place solution

Machine Learning Competition Solution
Overview
We would like to thank the organizers for creating such an engaging and challenging competition. This document outlines our comprehensive approach to solving the flight ranking problem.
Our solution began with the foundation of a public notebook (
[XGBoost Ranker](https://www.kaggle.com/code/ka1242/xgboost-ranker-with-polars)
), though by the competition's end, our implementation had evolved significantly beyond the original approach.
Feature Engineering Strategy
Our feature engineering process was methodical and multi-faceted, focusing on capturing the complexity of flight booking behavior and preferences.
1. Core Flight Features
Route Analysis:
Extracted segment counts and connection details for each flight leg
Constructed complete flight paths (origin → connections → destination)
Analyzed route structure and categorized connection types
Temporal Features:
Converted departure and arrival times to minutes for consistent processing
Extracted time-based patterns including time of day, day of week, and seasonality
Identified holidays and weekends to capture booking pattern variations
Analyzed business-friendly timing preferences
2. Geographic & Airport Features
Distance Calculations:
Implemented Haversine formula for precise inter-airport distance measurements
Calculated elevation differences between airports
Identified intercontinental flights for specialized handling
Airport Characteristics:
Classified airports as hub versus regional facilities
Analyzed route popularity and traffic patterns
Implemented continental grouping for geographic insights
3. Advanced User Behavior Features
Historical Patterns:
Analyzed user preferences across airlines, booking timing, and service classes
Incorporated loyalty program analysis
Evaluated corporate policy compliance patterns
Contextual Analysis:
Calculated relative rankings within search sessions (price, duration, convenience)
Identified Pareto-optimal options in the choice set
Analyzed price-quality trade-off preferences
4. Flight Quality & Convenience Metrics
Quality Scoring:
Mapped aircraft quality characteristics (wide-body vs narrow-body)
Assessed schedule convenience (business hours vs red-eye flights)
Analyzed fare flexibility options
Evaluated connection time comfort levels
Pricing Analytics:
Calculated price-per-kilometer and price-per-minute efficiency ratios
Measured deviation from group medians
Analyzed competitive positioning within search results
5. Temporal Split & Historical Counters
Following successful strategies employed by other competition winners, we implemented historical counters with multiple categorical feature combinations. Our approach involved:
Using 50% of the time-sorted training dataset as historical reference
Training on the remaining 50% to maintain temporal validity
Creating interaction features across multiple categorical dimensions
6. Data Sampling Strategy
To ensure training stability and computational efficiency:
Limited group sizes to 64 samples maximum
Preserved all positive examples (selected=1) to maintain signal strength
Applied random sampling to negative examples within each group
Model Architecture
7. Model Training
Our ensemble approach incorporated multiple ranking algorithms:
Primary Models:
CatBoostRanker
with three different loss functions:
QueryCrossEntropy (alpha=1.0) - primary loss function
PairLogitPairwise - secondary approach
YetiRankPairwise - tertiary validation
LightGBM Ranker
implemented by Farukcan
8. Ensemble Strategy
Our final prediction system employed a normalized score ensemble:
Normalized all individual ranker scores to ensure comparable scales
Combined normalized scores using optimized ratios
Selected ensemble weights primarily through leaderboard validation
Maintained simplicity in the ensemble approach for robust generalization
Key Insights
The success of our solution stemmed from several critical discoveries:
Price and Duration Features Were Paramount
: As expected, flight price and duration emerged as the most important predictive features. We experimented with multiple variations of these features including:
Absolute values (raw price and duration)
Relative measures (compared to group averages)
Ranked positions within search results
This comprehensive treatment of the core decision factors proved essential for model performance.
Historical Counters Provided Substantial Gains
: Historical counters delivered significant performance improvements, contributing approximately 200 features to our final dataset. However, this represents only a fraction of the possible categorical combinations available in the data. There remains substantial opportunity for further score improvements by exploring additional feature combinations, though computational constraints limited our exploration.
Multi-Loss Function Ensemble Benefits
: A key discovery was that combining models trained with different loss functions, even when using identical training data, yielded meaningful performance gains. This suggests that different loss functions capture complementary aspects of the ranking problem, making ensemble diversity valuable beyond just data variation.
Thanks for attention