{
  "id": 232054,
  "title": "25th Place Solution",
  "content": "<h1>Hi all !</h1>\n<p>First of all, congratulations to the winners and to all participants !</p>\n<p>It's been a pleasure taking part in this amazing competition, with actuaries finally getting the recognition they deserve in the data science world 😄</p>\n<p>I may not have made it to the top of the leaderboard, but I still learnt a lot from all the models I built and tested, and I look forward to reading top-ranked solutions.</p>\n<h3>Target Engineering :</h3>\n<p>I focused on trying to find the right target for the problem, and that actually proved to be quite a difficult choice. Of course, the final target is the Ultimate Incurred Claim Cost, but I found that some derivatives were much easier to predict :</p>\n<ul>\n<li>Ultimate Cost divided by a price index depending on the date of accident (that is to say, inflation-neutral ultimate cost) ;</li>\n<li>The ratio Ultimate/Initial,  roughly a development factor from the claim's opening to its close (which also happens to be, tadam!, inflation-neutral) ;</li>\n<li>The log of the previous ratios.</li>\n</ul>\n<h3>What worked :</h3>\n<ul>\n<li><em>Gradient Boosting</em>, more precisely <em>LGBM</em> on standard features ;</li>\n<li><em>Principal Component Analysis</em> of the standard features to extract information ;</li>\n<li>On those PCA features (obtained through unsupervised learning) : on fixed folds, I trained LGBM, NN, various cuML (GPU equivalent of sklearn) regression models ;</li>\n<li>Selecting the best models from the previous stage (based on public leaderboard score, not my private cross-validation…) I stacked them using yet another LGB ;</li>\n<li>Bagging, bagging, bagging, bagging.</li>\n</ul>\n<h3>What didn't work :</h3>\n<ul>\n<li>External Data ;</li>\n<li><em>Hand-Crafted Feature Generation</em> : I tried multiple, very inventive and clever manipulations on inflation, embeddings, time of accident, etc, but they never proved to add any more value than the standard features ;</li>\n<li><em>Neural nets</em>, whether shallow or deep, on standard &amp; handmade input features ;</li>\n<li>Sadly, <em>sentence embeddings</em> of claim descriptions, though <a href=\"https://www.kaggle.com/louise2001/rapids-pca-for-sentenceembeddings\" target=\"_blank\">I put much effort in modeling them</a> ;</li>\n<li><em>Inflation normalization</em> depending on the date of the accident ;</li>\n<li><em>Linear regression</em> on standard &amp; handmade features ;</li>\n<li><em>Generalized Linear Models</em>, no matter what law family, formula, features or target I chose. <br>\nThe only one giving satisfying enough results was predicting log(ultimate/initial) with a Gaussian family, identity link, on very standard features.</li>\n</ul>\n<h3>Summary</h3>\n<p>So, to sum things up, dropping the datetime and text inputs and performing a Principal Component Analysis on the standard features proved to be the best strategy, using the resulting embeddings as inputs to various models (LGBM, simple NN, classic regression techniques) and then stacking the best models and bagging all predictions.</p>\n<p>All my models were run on my NVIDIA powered ZBook, using RAPIDS for optimal performance and use of the RTX5000 GPU.</p>\n<h1>Happy Kaggling 😄</h1>",
  "comments": [
    {
      "id": 1273651,
      "author_name": "Abhishek252",
      "author_url": "",
      "post_date": "2021-04-14T14:18:27.850000",
      "content": "<p>Great work!</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "comment_count": 1,
  "messages": [
    {
      "id": 1270736,
      "postDate": "2021-04-12T00:01:22.007Z",
      "content": "<h1>Hi all !</h1>\n<p>First of all, congratulations to the winners and to all participants !</p>\n<p>It's been a pleasure taking part in this amazing competition, with actuaries finally getting the recognition they deserve in the data science world 😄</p>\n<p>I may not have made it to the top of the leaderboard, but I still learnt a lot from all the models I built and tested, and I look forward to reading top-ranked solutions.</p>\n<h3>Target Engineering :</h3>\n<p>I focused on trying to find the right target for the problem, and that actually proved to be quite a difficult choice. Of course, the final target is the Ultimate Incurred Claim Cost, but I found that some derivatives were much easier to predict :</p>\n<ul>\n<li>Ultimate Cost divided by a price index depending on the date of accident (that is to say, inflation-neutral ultimate cost) ;</li>\n<li>The ratio Ultimate/Initial,  roughly a development factor from the claim's opening to its close (which also happens to be, tadam!, inflation-neutral) ;</li>\n<li>The log of the previous ratios.</li>\n</ul>\n<h3>What worked :</h3>\n<ul>\n<li><em>Gradient Boosting</em>, more precisely <em>LGBM</em> on standard features ;</li>\n<li><em>Principal Component Analysis</em> of the standard features to extract information ;</li>\n<li>On those PCA features (obtained through unsupervised learning) : on fixed folds, I trained LGBM, NN, various cuML (GPU equivalent of sklearn) regression models ;</li>\n<li>Selecting the best models from the previous stage (based on public leaderboard score, not my private cross-validation…) I stacked them using yet another LGB ;</li>\n<li>Bagging, bagging, bagging, bagging.</li>\n</ul>\n<h3>What didn't work :</h3>\n<ul>\n<li>External Data ;</li>\n<li><em>Hand-Crafted Feature Generation</em> : I tried multiple, very inventive and clever manipulations on inflation, embeddings, time of accident, etc, but they never proved to add any more value than the standard features ;</li>\n<li><em>Neural nets</em>, whether shallow or deep, on standard &amp; handmade input features ;</li>\n<li>Sadly, <em>sentence embeddings</em> of claim descriptions, though <a href=\"https://www.kaggle.com/louise2001/rapids-pca-for-sentenceembeddings\" target=\"_blank\">I put much effort in modeling them</a> ;</li>\n<li><em>Inflation normalization</em> depending on the date of the accident ;</li>\n<li><em>Linear regression</em> on standard &amp; handmade features ;</li>\n<li><em>Generalized Linear Models</em>, no matter what law family, formula, features or target I chose. <br>\nThe only one giving satisfying enough results was predicting log(ultimate/initial) with a Gaussian family, identity link, on very standard features.</li>\n</ul>\n<h3>Summary</h3>\n<p>So, to sum things up, dropping the datetime and text inputs and performing a Principal Component Analysis on the standard features proved to be the best strategy, using the resulting embeddings as inputs to various models (LGBM, simple NN, classic regression techniques) and then stacking the best models and bagging all predictions.</p>\n<p>All my models were run on my NVIDIA powered ZBook, using RAPIDS for optimal performance and use of the RTX5000 GPU.</p>\n<h1>Happy Kaggling 😄</h1>",
      "rawMarkdown": "# Hi all !\n\nFirst of all, congratulations to the winners and to all participants !\n\nIt's been a pleasure taking part in this amazing competition, with actuaries finally getting the recognition they deserve in the data science world 😄\n\nI may not have made it to the top of the leaderboard, but I still learnt a lot from all the models I built and tested, and I look forward to reading top-ranked solutions.\n\n###Target Engineering :\nI focused on trying to find the right target for the problem, and that actually proved to be quite a difficult choice. Of course, the final target is the Ultimate Incurred Claim Cost, but I found that some derivatives were much easier to predict :\n- Ultimate Cost divided by a price index depending on the date of accident (that is to say, inflation-neutral ultimate cost) ;\n- The ratio Ultimate/Initial,  roughly a development factor from the claim's opening to its close (which also happens to be, tadam!, inflation-neutral) ;\n- The log of the previous ratios.\n\n### What worked :\n- *Gradient Boosting*, more precisely *LGBM* on standard features ;\n- *Principal Component Analysis* of the standard features to extract information ;\n- On those PCA features (obtained through unsupervised learning) : on fixed folds, I trained LGBM, NN, various cuML (GPU equivalent of sklearn) regression models ;\n- Selecting the best models from the previous stage (based on public leaderboard score, not my private cross-validation...) I stacked them using yet another LGB ;\n- Bagging, bagging, bagging, bagging.\n\n### What didn't work :\n- External Data ;\n- *Hand-Crafted Feature Generation* : I tried multiple, very inventive and clever manipulations on inflation, embeddings, time of accident, etc, but they never proved to add any more value than the standard features ;\n- *Neural nets*, whether shallow or deep, on standard & handmade input features ;\n- Sadly, *sentence embeddings* of claim descriptions, though [I put much effort in modeling them](https://www.kaggle.com/louise2001/rapids-pca-for-sentenceembeddings) ;\n- *Inflation normalization* depending on the date of the accident ;\n- *Linear regression* on standard & handmade features ;\n- *Generalized Linear Models*, no matter what law family, formula, features or target I chose. \nThe only one giving satisfying enough results was predicting log(ultimate/initial) with a Gaussian family, identity link, on very standard features.\n\n### Summary\nSo, to sum things up, dropping the datetime and text inputs and performing a Principal Component Analysis on the standard features proved to be the best strategy, using the resulting embeddings as inputs to various models (LGBM, simple NN, classic regression techniques) and then stacking the best models and bagging all predictions.\n\nAll my models were run on my NVIDIA powered ZBook, using RAPIDS for optimal performance and use of the RTX5000 GPU.\n\n# Happy Kaggling 😄",
      "votes": 16
    },
    {
      "id": 1273651,
      "postDate": "2021-04-14T14:18:27.850Z",
      "content": "<p>Great work!</p>",
      "rawMarkdown": "Great work!"
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T19:14:53Z",
  "competition_id": 24764,
  "competition_slug": "actuarial-loss-estimation",
  "team_id": 6036968,
  "solution_url": "/competitions/actuarial-loss-estimation/writeups/loulou-25th-place-solution"
}