id column and removed to avoid skewing the model with redundant information.user_id column, which included a prefix (****), was cleaned using regular expressions and converted to integers. This transformation ensured consistency and enabled easier feature engineering.Outliers were evaluated in key numerical columns such as:
\nageReceived_card_discount_percentageReceived_coupon_discount_percentageProduct_valueThe treatment strategy depended on whether similar outliers were present in the test dataset. For instance, outliers were removed from the training data only if they did not appear in the test set, preventing model inaccuracies during inference.
Hyperparameters were optimized using Bayesian optimization, focusing on:
\nCross-validation was employed to validate the model’s performance and minimize overfitting.
Encoding categorical features effectively was crucial for leveraging LightGBM’s capabilities.
Treating outliers required careful cross-referencing with the test dataset to ensure consistency.
Feature engineering, especially interaction terms and time-based features, significantly boosted model performance.
I am grateful to @akinduhiman for sharing discussion post that introduced the received_weekday feature and its application in capturing temporal patterns. This feature proved highly beneficial in improving model performance.
My notebook : predict-the-customer-satisfaction-lightgbm
", "comments": [], "comment_count": 0, "messages": [ { "id": 3099241, "postDate": "2025-01-17T13:54:22.380Z", "content": "id column and removed to avoid skewing the model with redundant information.user_id column, which included a prefix (****), was cleaned using regular expressions and converted to integers. This transformation ensured consistency and enabled easier feature engineering.Outliers were evaluated in key numerical columns such as:
\nageReceived_card_discount_percentageReceived_coupon_discount_percentageProduct_valueThe treatment strategy depended on whether similar outliers were present in the test dataset. For instance, outliers were removed from the training data only if they did not appear in the test set, preventing model inaccuracies during inference.
Hyperparameters were optimized using Bayesian optimization, focusing on:
\nCross-validation was employed to validate the model’s performance and minimize overfitting.
Encoding categorical features effectively was crucial for leveraging LightGBM’s capabilities.
Treating outliers required careful cross-referencing with the test dataset to ensure consistency.
Feature engineering, especially interaction terms and time-based features, significantly boosted model performance.
I am grateful to @akinduhiman for sharing discussion post that introduced the received_weekday feature and its application in capturing temporal patterns. This feature proved highly beneficial in improving model performance.
My notebook : predict-the-customer-satisfaction-lightgbm
", "rawMarkdown": "### 1. Data Loading and Initial Exploration \n- The training and test datasets were loaded using Pandas and explored for basic statistics, column distributions, and data types.\n- Initial inspections revealed duplicate rows, missing values, and certain features needing transformations for consistency.\n\n---\n\n### 2. Data Preprocessing \n#### Duplicate Removal:\n- Duplicate rows were identified by excluding the `id` column and removed to avoid skewing the model with redundant information.\n\n#### Data Cleaning:\n- The `user_id` column, which included a prefix (`****`), was cleaned using regular expressions and converted to integers. This transformation ensured consistency and enabled easier feature engineering.\n\n#### Handling Missing Values:\n- Missing values in numerical features were imputed using median values to minimize the impact of outliers. For categorical features, missing values were treated as a separate category.\n\n---\n\n### 3. Outlier Detection and Treatment \n- Outliers were evaluated in key numerical columns such as:\n - `age`\n - `Received_card_discount_percentage`\n - `Received_coupon_discount_percentage`\n - `Product_value`\n\n- The treatment strategy depended on whether similar outliers were present in the test dataset. For instance, outliers were removed from the training data only if they did not appear in the test set, preventing model inaccuracies during inference.\n\n---\n\n### 4. Feature Engineering \n#### Encoding Techniques:\n- **Label Encoding**: Applied to ordinal categorical variables, preserving the natural order of categories.\n- **One-Hot Encoding**: Used for nominal categorical variables to avoid imposing artificial relationships between categories.\n- **Benefits**: These encodings enabled LightGBM to handle categorical features natively, leading to faster training and better performance.\n\n#### Feature Importance Analysis:\n- **Mutual Information (MI) Scores**: Calculated to assess the dependency between features and the target variable. Features with low MI scores were considered for removal or transformation.\n\n#### Derived Features:\n- Interaction terms between discount percentages and product values were generated, capturing the effect of combined discounts.\n- Time-based features, such as day-of-week and hour-of-day, were extracted from timestamps to identify temporal patterns.\n\n---\n\n### 5. Model Selection and Training\n#### Choice of LightGBM:\n- LightGBM was selected for its ability to handle large datasets efficiently and its native support for categorical features.\n- The framework’s gradient-boosting mechanism provided superior performance for this tabular dataset.\n\n#### Hyperparameter Tuning:\n- Hyperparameters were optimized using Bayesian optimization, focusing on:\n - Number of leaves\n - Learning rate\n - Max depth\n - Feature fraction\n - Bagging fraction\n\n- Cross-validation was employed to validate the model’s performance and minimize overfitting.\n\n---\n\n### 6. Model Evaluation and Submission\n- The model was evaluated on a hold-out validation set, with metrics such as accuracy, F1-score, and AUC monitored to ensure alignment with competition objectives.\n- Predictions for the test dataset were generated and saved in the required submission format.\n\n---\n\n### Challenges and Learnings \n\n- Encoding categorical features effectively was crucial for leveraging LightGBM’s capabilities.\n\n- Treating outliers required careful cross-referencing with the test dataset to ensure consistency.\n\n- Feature engineering, especially interaction terms and time-based features, significantly boosted model performance.\n\n- I am grateful to @akinduhiman for sharing discussion post that introduced the `received_weekday` feature and its application in capturing temporal patterns. This feature proved highly beneficial in improving model performance.\n\n\nMy notebook : [predict-the-customer-satisfaction-lightgbm](https://www.kaggle.com/code/bashithashamila/predict-the-customer-satisfaction-lightgbm)\n", "votes": 3 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T13:43:33Z", "competition_id": 90922, "competition_slug": "Predict-the-Customer-Satisfaction-CSE-22", "team_id": 13200872, "solution_url": "/competitions/Predict-the-Customer-Satisfaction-CSE-22/writeups/bashitha-shamila-solution-overview-feature-enginee" }