Rank: 6 / 54\nMetric: RMSE\nApproach: Single TABM (Tabular Deep Learning) model, no ensembling, no boosting models
\nThis solution intentionally avoids heavy ensembling or multiple model stacks.\nThe final rank was achieved using a single TABM model.
\ndf['brand'] = df['model'].str.split().str[0] \ndf['cpu_brand'] = df['cpu_model'].str.split().str[0]I tested boosting models (XGB / CatBoost) and simple blends, but gains were marginal and TABM alone generalized extremely well\nThis competition was a good example where model–feature alignment mattered more than model count.\nI could have done more feature engineering. Did some FE but they reduced the private leaderboard score so dropped that idea very soon. Perhaps should have tried some more hyper-parameterisation but didnt have enough time.\nPlotted cumulative distributions and did a KS-Test to find out ensembling adds very little which is consistent with submission findings.
\nAny form of blending or ensembling (simple averaging, weighted averages, log-space averaging) did not improve CV or LB. Adding multiple boosted models (XGB / LGBM / CatBoost) increased complexity without gains. Incorporating the external laptop_prices dataset also hurt generalization and slightly degraded validation performance.
", "comments": [ { "id": 3384317, "author_name": "yunsuxiaozi", "author_url": "", "post_date": "2026-01-01T01:20:44.370000", "content": "You can try this.
\nfrom pytabkit import TabM_HPO_Regressor\nTabM_HPO_Regressor(\n device='cuda',\n random_state=2025,\n n_cv=1,\n # n_refit=0,\n n_epochs=10, \n verbosity=40,\n n_hyperopt_steps=20,\n )\nRank: 6 / 54\nMetric: RMSE\nApproach: Single TABM (Tabular Deep Learning) model, no ensembling, no boosting models
\nThis solution intentionally avoids heavy ensembling or multiple model stacks.\nThe final rank was achieved using a single TABM model.
\ndf['brand'] = df['model'].str.split().str[0] \ndf['cpu_brand'] = df['cpu_model'].str.split().str[0]I tested boosting models (XGB / CatBoost) and simple blends, but gains were marginal and TABM alone generalized extremely well\nThis competition was a good example where model–feature alignment mattered more than model count.\nI could have done more feature engineering. Did some FE but they reduced the private leaderboard score so dropped that idea very soon. Perhaps should have tried some more hyper-parameterisation but didnt have enough time.\nPlotted cumulative distributions and did a KS-Test to find out ensembling adds very little which is consistent with submission findings.
\nAny form of blending or ensembling (simple averaging, weighted averages, log-space averaging) did not improve CV or LB. Adding multiple boosted models (XGB / LGBM / CatBoost) increased complexity without gains. Incorporating the external laptop_prices dataset also hurt generalization and slightly degraded validation performance.
", "rawMarkdown": "#### 🏆 6th Place Solution (Single Model): TABM Only\n\nRank: 6 / 54\nMetric: RMSE\nApproach: Single TABM (Tabular Deep Learning) model, no ensembling, no boosting models\n\nThis solution intentionally avoids heavy ensembling or multiple model stacks.\nThe final rank was achieved using a single TABM model.\n\n###### Feature Engineering done:\n- Removed weak or noisy columns such as: \n 'bluetooth',\n 'warranty_months',\n 'storage_drive_count',\n 'wifi',\n 'model',\n 'cpu_model',\n 'display_size_in',\n 'weight_kg',\n 'cpu_boost_ghz',\n 'battery_wh'\n- Extracted *brand *and *cpu_brand*\n`df['brand'] = df['model'].str.split().str[0] \n df['cpu_brand'] = df['cpu_model'].str.split().str[0] `\n- I generated pairwise feature interactions by concatenating every unique pair of input features (excluding ID and target). Each interaction represents a specific co-occurrence (e.g. CPU–GPU, brand–form factor). Treating these as categorical tokens works especially well with TABM embeddings, allowing the model to learn non-additive pricing effects directly. This expanded the feature set to 253 features and gave a clear performance boost without ensembling.\n\n\nI tested boosting models (XGB / CatBoost) and simple blends, but gains were marginal and TABM alone generalized extremely well\nThis competition was a good example where model–feature alignment mattered more than model count.\nI could have done more feature engineering. Did some FE but they reduced the private leaderboard score so dropped that idea very soon. Perhaps should have tried some more hyper-parameterisation but didnt have enough time.\nPlotted cumulative distributions and did a KS-Test to find out ensembling adds very little which is consistent with submission findings.\n\n###### What didn’t work:\nAny form of blending or ensembling (simple averaging, weighted averages, log-space averaging) did not improve CV or LB. Adding multiple boosted models (XGB / LGBM / CatBoost) increased complexity without gains. Incorporating the external laptop_prices dataset also hurt generalization and slightly degraded validation performance.", "votes": 8 }, { "id": 3384317, "postDate": "2026-01-01T01:20:44.370Z", "content": "You can try this.
\nfrom pytabkit import TabM_HPO_Regressor\nTabM_HPO_Regressor(\n device='cuda',\n random_state=2025,\n n_cv=1,\n # n_refit=0,\n n_epochs=10, \n verbosity=40,\n n_hyperopt_steps=20,\n )\n