First of all, I would like to thank @lazylo for the interesting competition with a really tricky evaluation metric. I would also like to thank @ambrosm for his simple baseline and his suggestions to keep it simple and be careful about overfitting to the public LB.
\nOVERVIEW OF MY SOLUTION
\nMy solution was quite simple: I used 12 logistic regressions (1 for each horizon) along with Isotonic calibration, to classify demand into zero vs non-zero and better calibrate the probabilities; after that, I fit 12 ridge regressions (1 for each horizon) on log1p of the positives to predict positive demand. Technically the ridge predicts demand for all 1152 predictions but they are overwritten by zeros if the calibrated prob_zero exceeds a certain global threshold.
\nIt uses 38 features, which are created from the target itself (so lags, rolling features, etc), and calendar-based features to help learn patterns based on the calendar month. So I didn't end up using any of the other datasets we were given because I wanted to try and keep it simple, but both methods were viable I guess.
\nFor the Logistic C and the Ridge's Alpha I used Optuna to optimize the average comp score across validation folds and got the values I ended up using, but optimizing the average comp score across folds still wasn't a reliable enough way of choosing hyperparameter values. Ironically, the same solution with different values of alpha and C, which I randomly chose, ended up outperforming my current strategy (by a lot) on the private LB.
\nThe key in this solution was all the discounting of the positive magnitudes that was done: specifically, the positive magnitudes are discounted by (1-p0) where p0 is the probability of being zero estimated by the Logistic regression with Isotonic calibration; also, while generating predictions, we apply global multiplicative factors by horizon, to correct any bias that is found; finally, some predictions were capped to not exceed 3x the previous prediction.
\nThe final tweak was to seasonally nudge the predictions based on the average demand by sector-month, which I found to be the only thing that consistently improved CV, Public LB and Private LB.
\nIMPROVEMENTS OF MY SOLUTION
\nI also ran this strategy with 53 features, using volatility-related features. That ends up improving the zero-classification substantially but hurts the positive magnitudes. I ended up using these 53 features to get the prob_zero for the predictions, but the key here was not to only consider the absolute values of the probabilities but the increments between months. By considering the increments along with the absolute values, I ended up improving my Public LB score by almost 0.004. I didn't end up using this strategy for the zeros in the Private LB although it would have helped and given me a considerable boost in hindsight, but it's definitely a cool idea that I never thought of.
\nEVERYTHING I TRIED
\nI have tried lots of different strategies, and most of them performed really well on Public LB, but very few ended up getting good scores on Private LB. I will list most of the things I tried and then I'll highlight those that worked best in terms of Private LB:
\nThere's more but these were the most promising
\nWHAT ACTUALLY WORKED IN TERMS OF PRIVATE LB
\nhi , i also try your solution with my dataset but adding the new features as pseudo sectors as a lazytrick,it do not improve , probably using a proper implementation can help?
", "votes": 0, "replies": [ { "id": 3311172, "author_name": "Mirko Ferretti", "author_url": "", "post_date": "2025-11-04T12:11:22.163000", "content": "I had also tried CPI, LPR, etc. but I've also seen degradation in terms of performance. Ridge is decent with more features but Logistic Regression is hurt by higher dimensionality. Matter of fact, if you use forward/backward selection you'll find that around 20 features is the sweet spot (instead of the current 38), and you'll get a better Private LB score (worse public LB though). I'm assuming that adding features like LPR, CPI etc increases the amount of noise and hurts both the logistic and ridge; it might also be possible that these new features have good non-linear information but ridge isn't able to capture it properly so it ends up being noisy. Maybe more powerful methods like tree-based strategies could capture this information better.
\nOverall my idea was to keep it simple and robust, underperform on public LB but hope for a solid solution for the private LB, which is the main reason why I used so few features and didn't even use any of the other datasets.
\nTo be completely honest though, the data is really messy so it's rare to find something that works for CV, Public LB and Private LB. So I think adding more and more features will just end up hurting the final score. That's my best guess.
", "votes": 0, "replies": [ { "id": 3312472, "author_name": "michel emil", "author_url": "", "post_date": "2025-11-07T09:32:09.767000", "content": "the data was mostly collected on the china national statistics website , it offers free access but display pictures not tables , so i was having to do ocr and compile , with the help of ai, qwen that also fill and predict the future value for 2025, but it is national level data, and then the correlation can be weak for some sectors and thus hurting the predictions , the key is to group the sectors that follow the national trend and* apply to them only the national macro stats*. This competition dataset is for that point having a lack of production usability , all the sectors shall be at least linked to a region mapping where we can fetch macro data live or predicted as it is the standard way state of the art to predict house market.\nthe low performance achieved by the top 10 and the random flip top10 private lb to public lb demonstrate the lack of predictive power of the original dataset, i suggest strongly to map the sectors to city and regions where we can fetch some macro eco data , it is part of the job to discover when the data lacks predictors and advice the customer to enrich the data.
", "votes": 0, "replies": [] } ] } ] } ], "comment_count": 3, "messages": [ { "id": 3310223, "postDate": "2025-11-02T12:02:29.737Z", "content": "First of all, I would like to thank @lazylo for the interesting competition with a really tricky evaluation metric. I would also like to thank @ambrosm for his simple baseline and his suggestions to keep it simple and be careful about overfitting to the public LB.
\nOVERVIEW OF MY SOLUTION
\nMy solution was quite simple: I used 12 logistic regressions (1 for each horizon) along with Isotonic calibration, to classify demand into zero vs non-zero and better calibrate the probabilities; after that, I fit 12 ridge regressions (1 for each horizon) on log1p of the positives to predict positive demand. Technically the ridge predicts demand for all 1152 predictions but they are overwritten by zeros if the calibrated prob_zero exceeds a certain global threshold.
\nIt uses 38 features, which are created from the target itself (so lags, rolling features, etc), and calendar-based features to help learn patterns based on the calendar month. So I didn't end up using any of the other datasets we were given because I wanted to try and keep it simple, but both methods were viable I guess.
\nFor the Logistic C and the Ridge's Alpha I used Optuna to optimize the average comp score across validation folds and got the values I ended up using, but optimizing the average comp score across folds still wasn't a reliable enough way of choosing hyperparameter values. Ironically, the same solution with different values of alpha and C, which I randomly chose, ended up outperforming my current strategy (by a lot) on the private LB.
\nThe key in this solution was all the discounting of the positive magnitudes that was done: specifically, the positive magnitudes are discounted by (1-p0) where p0 is the probability of being zero estimated by the Logistic regression with Isotonic calibration; also, while generating predictions, we apply global multiplicative factors by horizon, to correct any bias that is found; finally, some predictions were capped to not exceed 3x the previous prediction.
\nThe final tweak was to seasonally nudge the predictions based on the average demand by sector-month, which I found to be the only thing that consistently improved CV, Public LB and Private LB.
\nIMPROVEMENTS OF MY SOLUTION
\nI also ran this strategy with 53 features, using volatility-related features. That ends up improving the zero-classification substantially but hurts the positive magnitudes. I ended up using these 53 features to get the prob_zero for the predictions, but the key here was not to only consider the absolute values of the probabilities but the increments between months. By considering the increments along with the absolute values, I ended up improving my Public LB score by almost 0.004. I didn't end up using this strategy for the zeros in the Private LB although it would have helped and given me a considerable boost in hindsight, but it's definitely a cool idea that I never thought of.
\nEVERYTHING I TRIED
\nI have tried lots of different strategies, and most of them performed really well on Public LB, but very few ended up getting good scores on Private LB. I will list most of the things I tried and then I'll highlight those that worked best in terms of Private LB:
\nThere's more but these were the most promising
\nWHAT ACTUALLY WORKED IN TERMS OF PRIVATE LB
\nhi , i also try your solution with my dataset but adding the new features as pseudo sectors as a lazytrick,it do not improve , probably using a proper implementation can help?
", "rawMarkdown": "hi , i also try your solution with my dataset but adding the new features as pseudo sectors as a lazytrick,it do not improve , probably using a proper implementation can help?", "replies": [ { "id": 3311172, "postDate": "2025-11-04T12:11:22.163Z", "content": "I had also tried CPI, LPR, etc. but I've also seen degradation in terms of performance. Ridge is decent with more features but Logistic Regression is hurt by higher dimensionality. Matter of fact, if you use forward/backward selection you'll find that around 20 features is the sweet spot (instead of the current 38), and you'll get a better Private LB score (worse public LB though). I'm assuming that adding features like LPR, CPI etc increases the amount of noise and hurts both the logistic and ridge; it might also be possible that these new features have good non-linear information but ridge isn't able to capture it properly so it ends up being noisy. Maybe more powerful methods like tree-based strategies could capture this information better.
\nOverall my idea was to keep it simple and robust, underperform on public LB but hope for a solid solution for the private LB, which is the main reason why I used so few features and didn't even use any of the other datasets.
\nTo be completely honest though, the data is really messy so it's rare to find something that works for CV, Public LB and Private LB. So I think adding more and more features will just end up hurting the final score. That's my best guess.
", "rawMarkdown": "I had also tried CPI, LPR, etc. but I've also seen degradation in terms of performance. Ridge is decent with more features but Logistic Regression is hurt by higher dimensionality. Matter of fact, if you use forward/backward selection you'll find that around 20 features is the sweet spot (instead of the current 38), and you'll get a better Private LB score (worse public LB though). I'm assuming that adding features like LPR, CPI etc increases the amount of noise and hurts both the logistic and ridge; it might also be possible that these new features have good non-linear information but ridge isn't able to capture it properly so it ends up being noisy. Maybe more powerful methods like tree-based strategies could capture this information better. \n\nOverall my idea was to keep it simple and robust, underperform on public LB but hope for a solid solution for the private LB, which is the main reason why I used so few features and didn't even use any of the other datasets. \n\nTo be completely honest though, the data is really messy so it's rare to find something that works for CV, Public LB and Private LB. So I think adding more and more features will just end up hurting the final score. That's my best guess.", "replies": [ { "id": 3312472, "postDate": "2025-11-07T09:32:09.767Z", "content": "the data was mostly collected on the china national statistics website , it offers free access but display pictures not tables , so i was having to do ocr and compile , with the help of ai, qwen that also fill and predict the future value for 2025, but it is national level data, and then the correlation can be weak for some sectors and thus hurting the predictions , the key is to group the sectors that follow the national trend and* apply to them only the national macro stats*. This competition dataset is for that point having a lack of production usability , all the sectors shall be at least linked to a region mapping where we can fetch macro data live or predicted as it is the standard way state of the art to predict house market.\nthe low performance achieved by the top 10 and the random flip top10 private lb to public lb demonstrate the lack of predictive power of the original dataset, i suggest strongly to map the sectors to city and regions where we can fetch some macro eco data , it is part of the job to discover when the data lacks predictors and advice the customer to enrich the data.
", "rawMarkdown": "the data was mostly collected on the china national statistics website , it offers free access but display pictures not tables , so i was having to do ocr and compile , with the help of ai, qwen that also fill and predict the future value for 2025, but it is national level data, and then the correlation can be weak for some sectors and thus hurting the predictions , the key is to group the sectors that follow the national trend and*** apply to them only the national macro stats***. This competition dataset is for that point having a lack of production usability , all the sectors shall be at least linked to a region mapping where we can fetch macro data live or predicted as it is the standard way state of the art to predict house market.\nthe low performance achieved by the top 10 and the random flip top10 private lb to public lb demonstrate the lack of predictive power of the original dataset, i suggest strongly to map the sectors to city and regions where we can fetch some macro eco data , it is part of the job to discover when the data lacks predictors and advice the customer to enrich the data." } ] } ] } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T00:52:43Z", "competition_id": 111876, "competition_slug": "china-real-estate-demand-prediction", "team_id": 14240771, "solution_url": "/competitions/china-real-estate-demand-prediction/writeups/5th-public-lb-11th-private-lb-solution" }