# 2nd place solution

Dear Fellow Kagglers,
We have read your solutions with great interest. Thank you for sharing them with us. It is uplifting to see such an open and supportive data science community. Therefore, we have decided to follow suit. Before we do that though, we would like to say thank you to all the organisers and sponsors for making this competition possible. It was a fun-filled intellectual challenge for us. Congratulations to all the winners and to those as well who fared less well in the end. Going from the public to the private LB was quite a surprise for us, too. So, these are our two cents:
Core model
: We have experimented only with NNs and GBMs. However, after a few – stacked and unstacked – submissions we abandoned the models other than GBMs, mainly in the interest of quicker training and tuning. The concrete implementation we used was xgboost with reg:tweedie objective. Gamma distribution would have been a more intuitive choice, but Tweedie performed better, so we stuck to it. After all, the good distribution is rather the one that fits the data well not our a priori assumptions. Also, we applied monotonicity constraints on a few variables and trained a few dozen trees in parallel to combat overfitting.
Text analysis
: We started with word vectors but they did not prove to be specific enough. After a brief detour with n-grams, we ended up with the following simple approach:
Lemmatization and stemming of claim descriptions
Explicit search for in total approx. 200 words falling into 3 categories: body part, wound type, accident cause/object
Creation of one-hot encoded variables as well as manual clustering of the categories based on intuition, count and average ultimate value (k-means clustering was not conclusive)
Miscellaneous
: We applied a few more tweaks as others did, such as deriving new features based on existing ones, cleaning of features based on common sense, exclusion of a few claims with extreme claim cost, etc.
The submission that secured us the 2nd place was a manual combination of different GBM models instead of a more sophisticated meta learner. We have tried several ways to train classifiers that separate large claims/small claims from the rest. Unfortunately, all our attempts were in vain so the fact remained that our predictions were always a bit high for small claims and low for larger claims. An inevitable concomitant of the loss metric, we thought. This insight was the motivation behind the following actuarial expert judgment :): if the mean of the predictions of the selected models were below a certain threshold, we selected the prediction with the lowest value, otherwise the one with the highest value.
Looking at the differences in the scores on the private LB we have no doubt that lady luck has smiled on us. Thank you once again to all for this competition. We look forward to the next actuarial competition with you. 😃
[Nelvis](https://www.kaggle.com/nelvis)
and
[Attila](https://www.kaggle.com/attiguyas)