# Solution Overview - Feature Engineering + LightGBM

1. Data Loading and Initial Exploration
The training and test datasets were loaded using Pandas and explored for basic statistics, column distributions, and data types.
Initial inspections revealed duplicate rows, missing values, and certain features needing transformations for consistency.
2. Data Preprocessing
Duplicate Removal:
Duplicate rows were identified by excluding the
id
column and removed to avoid skewing the model with redundant information.
Data Cleaning:
The
user_id
column, which included a prefix (
****
), was cleaned using regular expressions and converted to integers. This transformation ensured consistency and enabled easier feature engineering.
Handling Missing Values:
Missing values in numerical features were imputed using median values to minimize the impact of outliers. For categorical features, missing values were treated as a separate category.
3. Outlier Detection and Treatment
Outliers were evaluated in key numerical columns such as:
age
Received_card_discount_percentage
Received_coupon_discount_percentage
Product_value
The treatment strategy depended on whether similar outliers were present in the test dataset. For instance, outliers were removed from the training data only if they did not appear in the test set, preventing model inaccuracies during inference.
4. Feature Engineering
Encoding Techniques:
Label Encoding
: Applied to ordinal categorical variables, preserving the natural order of categories.
One-Hot Encoding
: Used for nominal categorical variables to avoid imposing artificial relationships between categories.
Benefits
: These encodings enabled LightGBM to handle categorical features natively, leading to faster training and better performance.
Feature Importance Analysis:
Mutual Information (MI) Scores
: Calculated to assess the dependency between features and the target variable. Features with low MI scores were considered for removal or transformation.
Derived Features:
Interaction terms between discount percentages and product values were generated, capturing the effect of combined discounts.
Time-based features, such as day-of-week and hour-of-day, were extracted from timestamps to identify temporal patterns.
5. Model Selection and Training
Choice of LightGBM:
LightGBM was selected for its ability to handle large datasets efficiently and its native support for categorical features.
The framework’s gradient-boosting mechanism provided superior performance for this tabular dataset.
Hyperparameter Tuning:
Hyperparameters were optimized using Bayesian optimization, focusing on:
Number of leaves
Learning rate
Max depth
Feature fraction
Bagging fraction
Cross-validation was employed to validate the model’s performance and minimize overfitting.
6. Model Evaluation and Submission
The model was evaluated on a hold-out validation set, with metrics such as accuracy, F1-score, and AUC monitored to ensure alignment with competition objectives.
Predictions for the test dataset were generated and saved in the required submission format.
Challenges and Learnings
Encoding categorical features effectively was crucial for leveraging LightGBM’s capabilities.
Treating outliers required careful cross-referencing with the test dataset to ensure consistency.
Feature engineering, especially interaction terms and time-based features, significantly boosted model performance.
I am grateful to
[@akinduhiman](https://www.kaggle.com/akinduhiman)
for sharing discussion post that introduced the
received_weekday
feature and its application in capturing temporal patterns. This feature proved highly beneficial in improving model performance.
My notebook :
[predict-the-customer-satisfaction-lightgbm](https://www.kaggle.com/code/bashithashamila/predict-the-customer-satisfaction-lightgbm)