Thank you to the actuarial societies for organizing this competition. I hope everyone had as much as I did. Even though my score is not the best score on the LB, I would still like to share a bit of my work just so we can learn from each other.
\nI was concerned about overfitting after so many tries on the public LB. Luckily, my final ranking on the private LB was higher.
\n(A bit about myself, I came from an actuarial background and had some work experience in workers' compensations. So I have seen some of the classical actuarial approaches for actuarial liability reserving, and I have been constantly amazed by how machine learning is applied to actuarial science)
\nHere is my approach:
\nI applied unsupervised machine learning (topic modeling) to classify injuries by using the ClaimDescription column (thanks to many helpful discussions in the forum). I applied both Latent Dirichlet Allocation (LDA)and Non-Negative Matrix Factorization (NMF) in the NLP world to classify injuries. I ran NMF and LDA on different combinations of n-grams and topics and let CV choose the best one (I found the best one can be different for different model).
On the modeling front, I tried: ANN, Random Forest, GBM (scikit learn version), Bagging of models, XGBOOST, lgbm, and catboost. I cv looped through various combinations of hyperparameters, features, and models to see which combinations are the most predictive. I then applied a linear model to ensemble the results based on their out-sample (validation) performances, the ensembling then tries all the possible combinations of the most predictive models to optimize a 10-fold C.V. RMSE (this is another CV).
I noticed that there are gaps between the testing set and the training set, two models, that do equally well in C.V. in the training set, had a comparable difference in the public LB, that is when I reminded myself not to overfit as I suspected there will be similar differences between the public LB and the private one (the shake-up in the final ranking proved this).
\nI also applied the above-mentioned models to engineer features.
I applied the unsupervised clustering methods (K-Means) to group the observations and engineer features. I ended up engineered almost 200 features. With the above-mentioned concerns on overfitting, I then applied CV again for feature selections. It was not surprising to see that different model requires a different set of features. This added another layer of complexity, as I needed to designate different sets of features for different models.
People have been discussing that outliers are the game changers for this competition. I tried some statistical methods to identify outliers (DBSCAN unsupervised method and some anomaly detection methods). But none of them seemed to work well on my end.
Thanks for reading my approach. I look forward to listening to your thoughts.
", "comments": [ { "id": 1276131, "author_name": "Abhishek252", "author_url": "", "post_date": "2021-04-17T06:55:22.310000", "content": "Nicely done buddy :)
", "votes": 0, "replies": [ { "id": 1276372, "author_name": "Yaokun Lin", "author_url": "", "post_date": "2021-04-17T13:26:55.770000", "content": "thanks for the kind words! 👍
", "votes": 0, "replies": [] } ] } ], "comment_count": 2, "messages": [ { "id": 1275105, "postDate": "2021-04-16T00:06:20.303Z", "content": "Thank you to the actuarial societies for organizing this competition. I hope everyone had as much as I did. Even though my score is not the best score on the LB, I would still like to share a bit of my work just so we can learn from each other.
\nI was concerned about overfitting after so many tries on the public LB. Luckily, my final ranking on the private LB was higher.
\n(A bit about myself, I came from an actuarial background and had some work experience in workers' compensations. So I have seen some of the classical actuarial approaches for actuarial liability reserving, and I have been constantly amazed by how machine learning is applied to actuarial science)
\nHere is my approach:
\nI applied unsupervised machine learning (topic modeling) to classify injuries by using the ClaimDescription column (thanks to many helpful discussions in the forum). I applied both Latent Dirichlet Allocation (LDA)and Non-Negative Matrix Factorization (NMF) in the NLP world to classify injuries. I ran NMF and LDA on different combinations of n-grams and topics and let CV choose the best one (I found the best one can be different for different model).
On the modeling front, I tried: ANN, Random Forest, GBM (scikit learn version), Bagging of models, XGBOOST, lgbm, and catboost. I cv looped through various combinations of hyperparameters, features, and models to see which combinations are the most predictive. I then applied a linear model to ensemble the results based on their out-sample (validation) performances, the ensembling then tries all the possible combinations of the most predictive models to optimize a 10-fold C.V. RMSE (this is another CV).
I noticed that there are gaps between the testing set and the training set, two models, that do equally well in C.V. in the training set, had a comparable difference in the public LB, that is when I reminded myself not to overfit as I suspected there will be similar differences between the public LB and the private one (the shake-up in the final ranking proved this).
\nI also applied the above-mentioned models to engineer features.
I applied the unsupervised clustering methods (K-Means) to group the observations and engineer features. I ended up engineered almost 200 features. With the above-mentioned concerns on overfitting, I then applied CV again for feature selections. It was not surprising to see that different model requires a different set of features. This added another layer of complexity, as I needed to designate different sets of features for different models.
People have been discussing that outliers are the game changers for this competition. I tried some statistical methods to identify outliers (DBSCAN unsupervised method and some anomaly detection methods). But none of them seemed to work well on my end.
Thanks for reading my approach. I look forward to listening to your thoughts.
", "rawMarkdown": "Thank you to the actuarial societies for organizing this competition. I hope everyone had as much as I did. Even though my score is not the best score on the LB, I would still like to share a bit of my work just so we can learn from each other. \n\nI was concerned about overfitting after so many tries on the public LB. Luckily, my final ranking on the private LB was higher. \n\n(A bit about myself, I came from an actuarial background and had some work experience in workers' compensations. So I have seen some of the classical actuarial approaches for actuarial liability reserving, and I have been constantly amazed by how machine learning is applied to actuarial science)\n\nHere is my approach:\n\n1. I applied unsupervised machine learning (topic modeling) to classify injuries by using the ClaimDescription column (thanks to many helpful discussions in the forum). I applied both **Latent Dirichlet Allocation (LDA)**and **Non-Negative Matrix Factorization (NMF)** in the NLP world to classify injuries. I ran NMF and LDA on different combinations of n-grams and topics and let CV choose the best one (I found the best one can be different for different model).\n\n \n2. On the modeling front, I tried: **ANN, Random Forest, GBM (scikit learn version), Bagging of models, XGBOOST, lgbm, and catboost**. I cv looped through various combinations of hyperparameters, features, and models to see which combinations are the most predictive. I then applied a linear model to ensemble the results based on their out-sample (validation) performances, the ensembling then tries all the possible combinations of the most predictive models to optimize a 10-fold C.V. RMSE (this is another CV). \n\nI noticed that there are gaps between the testing set and the training set, two models, that do equally well in C.V. in the training set, had a comparable difference in the public LB, **that is when I reminded myself not to overfit as I suspected there will be similar differences between the public LB and the private one (the shake-up in the final ranking proved this)**.\n\n3. I also applied the above-mentioned models to engineer features.\n\n4. I applied the unsupervised clustering methods (K-Means) to group the observations and engineer features. I ended up engineered almost 200 features. With the above-mentioned concerns on overfitting, I then applied CV again for feature selections. It was not surprising to see that different model requires a different set of features. This added another layer of complexity, as I needed to designate different sets of features for different models. \n\n5. People have been discussing that outliers are the game changers for this competition. I tried some statistical methods to identify outliers (DBSCAN unsupervised method and some anomaly detection methods). But none of them seemed to work well on my end. \n\nThanks for reading my approach. I look forward to listening to your thoughts.\n\n\n", "votes": 11 }, { "id": 1276131, "postDate": "2021-04-17T06:55:22.310Z", "content": "Nicely done buddy :)
", "rawMarkdown": "Nicely done buddy :)", "replies": [ { "id": 1276372, "postDate": "2021-04-17T13:26:55.770Z", "content": "thanks for the kind words! 👍
", "rawMarkdown": "thanks for the kind words! 👍" } ] } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T19:14:35Z", "competition_id": 24764, "competition_slug": "actuarial-loss-estimation", "team_id": 6187086, "solution_url": "/competitions/actuarial-loss-estimation/writeups/yaokun-lin-15th-place-solution" }