I would sincerely want to congratulate all the winners, and thank in particular Bishwarup, Zach and Far0n for setting the pace throughout the competition.
\n\nVladimir, Tilli, Danijel, Scirpus, Aleksandr, and everyone else who posted scripts, hints and insights throughout the competition. Without them, I literally wouldn't have accomplished what I did.
\n\nThis has been my best competition thus far, and even though I dropped quite a few spots in the end, I am still extremely satisfied with what I had accomplished. I have invested a lot of time and effort into this competition from the very beginning for a few main reasons.
\n\nFirst of all, the solutions seemed to be easily \"blendable.\" To me that meant that even if a particular effort of mine doesn't work as well as I had hoped, it could still potentially be used for blending with other solutions. And since this was a regression problem, blending is overall slightly easier (and more mathematically sound) than with classification problems.
\n\nSecond, there seemed to be a lot of consistency between the local CV scores and the LB scores, which meant that I can rely more on local experimentation to find the optimum parameters for my models.
\n\nThird, feature engineering did not seem to be a major factor for the quality of models. I still suck at it, and don't trust my skills in that department at all. To me this also indicated (although you can never be sure, and have to keep your fingers crossed) that major leaks were unlikely. This, thankfully, remained true to the end, and I want to extend special thanks to whoever prepared the datasets.
\n\nFor my final solution I primarily relied, just like everyone else, on a combination of XGB and NN. My XGBs in particular were pretty good, and consistently outperformed all the best reported results on the forum (at the time). Unlike others, I had much success with lowering the learning rate, but had to increase the early stopping rounds to almost 1000. My best 10-fold CV model took almost three days to run. I also discovered that the best R script performed better than the Python scripts, and I lowered the constant of the fair objective function all the way down to 0.4 (instead of 2.0). That enabled my best 10-fold CV score to get all the way down to 1105.02 on the LB. I also discovered that blending the Python scripts' results with the R script results gave me an additional boost, so my simple blend of XGBoost scripts was able to go all the way down to 1103.5 on LB. My Keras scripts were not nearly as good, and I was never able to get below 1111 with any single one of them. I guess I still need to work on my NN optimization skills.
\n\nFinally, I used a combination of simple blending and stacking for the final result. I was pretty rigorous with stacking, using only proper oof sets, but I got sloppy with blending and though I could get away with just very simple locally optimized blends, and was not properly paying attention to leakage. I believe that this was my biggest mistake.
\n\nIn the end, I also want to thank everyone who shared their final solution. Those posts were very enlightening, and I learned about many new tools and approaches. I am looking forward to trying them out in future competitions.
", "comments": [], "comment_count": 0, "messages": [ { "id": 150719, "postDate": "2016-12-16T13:31:30.703Z", "content": "I would sincerely want to congratulate all the winners, and thank in particular Bishwarup, Zach and Far0n for setting the pace throughout the competition.
\n\nVladimir, Tilli, Danijel, Scirpus, Aleksandr, and everyone else who posted scripts, hints and insights throughout the competition. Without them, I literally wouldn't have accomplished what I did.
\n\nThis has been my best competition thus far, and even though I dropped quite a few spots in the end, I am still extremely satisfied with what I had accomplished. I have invested a lot of time and effort into this competition from the very beginning for a few main reasons.
\n\nFirst of all, the solutions seemed to be easily \"blendable.\" To me that meant that even if a particular effort of mine doesn't work as well as I had hoped, it could still potentially be used for blending with other solutions. And since this was a regression problem, blending is overall slightly easier (and more mathematically sound) than with classification problems.
\n\nSecond, there seemed to be a lot of consistency between the local CV scores and the LB scores, which meant that I can rely more on local experimentation to find the optimum parameters for my models.
\n\nThird, feature engineering did not seem to be a major factor for the quality of models. I still suck at it, and don't trust my skills in that department at all. To me this also indicated (although you can never be sure, and have to keep your fingers crossed) that major leaks were unlikely. This, thankfully, remained true to the end, and I want to extend special thanks to whoever prepared the datasets.
\n\nFor my final solution I primarily relied, just like everyone else, on a combination of XGB and NN. My XGBs in particular were pretty good, and consistently outperformed all the best reported results on the forum (at the time). Unlike others, I had much success with lowering the learning rate, but had to increase the early stopping rounds to almost 1000. My best 10-fold CV model took almost three days to run. I also discovered that the best R script performed better than the Python scripts, and I lowered the constant of the fair objective function all the way down to 0.4 (instead of 2.0). That enabled my best 10-fold CV score to get all the way down to 1105.02 on the LB. I also discovered that blending the Python scripts' results with the R script results gave me an additional boost, so my simple blend of XGBoost scripts was able to go all the way down to 1103.5 on LB. My Keras scripts were not nearly as good, and I was never able to get below 1111 with any single one of them. I guess I still need to work on my NN optimization skills.
\n\nFinally, I used a combination of simple blending and stacking for the final result. I was pretty rigorous with stacking, using only proper oof sets, but I got sloppy with blending and though I could get away with just very simple locally optimized blends, and was not properly paying attention to leakage. I believe that this was my biggest mistake.
\n\nIn the end, I also want to thank everyone who shared their final solution. Those posts were very enlightening, and I learned about many new tools and approaches. I am looking forward to trying them out in future competitions.
", "rawMarkdown": "I would sincerely want to congratulate all the winners, and thank in particular Bishwarup, Zach and Far0n for setting the pace throughout the competition. \r\n\r\nVladimir, Tilli, Danijel, Scirpus, Aleksandr, and everyone else who posted scripts, hints and insights throughout the competition. Without them, I literally wouldn't have accomplished what I did.\r\n\r\nThis has been my best competition thus far, and even though I dropped quite a few spots in the end, I am still extremely satisfied with what I had accomplished. I have invested a lot of time and effort into this competition from the very beginning for a few main reasons. \r\n\r\nFirst of all, the solutions seemed to be easily \"blendable.\" To me that meant that even if a particular effort of mine doesn't work as well as I had hoped, it could still potentially be used for blending with other solutions. And since this was a regression problem, blending is overall slightly easier (and more mathematically sound) than with classification problems. \r\n\r\nSecond, there seemed to be a lot of consistency between the local CV scores and the LB scores, which meant that I can rely more on local experimentation to find the optimum parameters for my models. \r\n\r\nThird, feature engineering did not seem to be a major factor for the quality of models. I still suck at it, and don't trust my skills in that department at all. To me this also indicated (although you can never be sure, and have to keep your fingers crossed) that major leaks were unlikely. This, thankfully, remained true to the end, and I want to extend special thanks to whoever prepared the datasets. \r\n\r\nFor my final solution I primarily relied, just like everyone else, on a combination of XGB and NN. My XGBs in particular were pretty good, and consistently outperformed all the best reported results on the forum (at the time). Unlike others, I had much success with lowering the learning rate, but had to increase the early stopping rounds to almost 1000. My best 10-fold CV model took almost three days to run. I also discovered that the best R script performed better than the Python scripts, and I lowered the constant of the fair objective function all the way down to 0.4 (instead of 2.0). That enabled my best 10-fold CV score to get all the way down to 1105.02 on the LB. I also discovered that blending the Python scripts' results with the R script results gave me an additional boost, so my simple blend of XGBoost scripts was able to go all the way down to 1103.5 on LB. My Keras scripts were not nearly as good, and I was never able to get below 1111 with any single one of them. I guess I still need to work on my NN optimization skills. \r\n\r\n\r\nFinally, I used a combination of simple blending and stacking for the final result. I was pretty rigorous with stacking, using only proper oof sets, but I got sloppy with blending and though I could get away with just very simple locally optimized blends, and was not properly paying attention to leakage. I believe that this was my biggest mistake. \r\n\r\nIn the end, I also want to thank everyone who shared their final solution. Those posts were very enlightening, and I learned about many new tools and approaches. I am looking forward to trying them out in future competitions. ", "votes": 16 } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T17:16:22Z", "competition_id": 5325, "competition_slug": "allstate-claims-severity", "team_id": 383409, "solution_url": "/competitions/allstate-claims-severity/writeups/bojan-tunguz-congratulations-thanks-and-what-worke" }