{
  "id": 154951,
  "title": "9th place solution (a lot of LGBMs blended)",
  "content": "<p>My approach was quite straightforward: I have trained a lot of very simple LGBM models, blended their predictions, and applied an additional ~linear transformation to optimize the final metric.</p>\n<p><a href=\"https://www.kaggle.com/ilialar/w5-solution-2\" target=\"_blank\">Notebook with my final solution</a></p>\n<p><strong>Overview</strong></p>\n<ul>\n<li><p>Each model takes as input numbers of new cases and fatalities during the last N days divided by the maximum value of cases/fatalities for this region up to the last day in input and predicts the number of cases or fatalities on Kth day from the last input day (again divided by max value up to date).</p></li>\n<li><p>Each model is trained using all available data: all regions and all possible N+K days frames. But different days had different weights – the later the higher. The weight for each particular day is proportional to <code>(0.9)**(-n)</code> which means, that last days in the train set had much greater significance during training.</p></li>\n<li><p>Each model is just <code>lightgbm.LGBMRegressor()</code> with default parameters. I have trained a lot of different models iterating over N (<code>range(5, 40, 5)</code>) and K (<code>range(1,32)</code>). </p></li>\n<li><p>Then I have used all models to make all possible predictions for each day in the test or validation sets (I have used the last 30 days for validation) and computed min, max, and average prediction for each day.</p></li>\n<li><p>It turned out, these predictions were quite pessimistic – predicted values were very high. That’s why I have applied additional linear transformation and deviation factor and computed the final predictions using formulas below (coefficients are optimized on validation). </p></li>\n</ul>\n<pre><code>pred_0.5 = a[0] * mean + a[1] * min + a[2] * max\npred_0.05 = pred_0.5 + (b[0] * mean + b[1] * min + b[2] * max) * dev(lamb_min)\npred_0.95 = pred_0.5 + (c[0] * mean + c[1] * min + c[2] * max) * dev(lamb_max)\n</code></pre>\n<p>Where <code>a</code>, <code>b</code>, and <code>c</code> are just linear coefficients. \n<code>dev(lamb) = np.array([lamb * n for n in range(num_days_to_predict)])</code> – is used to make predictions for later days less (or more) confident.</p>\n<p>What I have not done but probably should (I have joined quite late and just did not have enough time):</p>\n<ul>\n<li>Optimize hyperparameters of LGBM</li>\n<li>Smooth predictions (I see that I have a lot of outliers at some days) by (for example) using 5% and 95% percentiles of predicted values instead of min and max.</li>\n<li>Use other information as inputs or somehow utilize information about connections between different regions.</li>\n</ul>",
  "comments": [],
  "comment_count": 0,
  "messages": [
    {
      "id": 867719,
      "postDate": "2020-05-30T14:53:16.580Z",
      "content": "<p>My approach was quite straightforward: I have trained a lot of very simple LGBM models, blended their predictions, and applied an additional ~linear transformation to optimize the final metric.</p>\n<p><a href=\"https://www.kaggle.com/ilialar/w5-solution-2\" target=\"_blank\">Notebook with my final solution</a></p>\n<p><strong>Overview</strong></p>\n<ul>\n<li><p>Each model takes as input numbers of new cases and fatalities during the last N days divided by the maximum value of cases/fatalities for this region up to the last day in input and predicts the number of cases or fatalities on Kth day from the last input day (again divided by max value up to date).</p></li>\n<li><p>Each model is trained using all available data: all regions and all possible N+K days frames. But different days had different weights – the later the higher. The weight for each particular day is proportional to <code>(0.9)**(-n)</code> which means, that last days in the train set had much greater significance during training.</p></li>\n<li><p>Each model is just <code>lightgbm.LGBMRegressor()</code> with default parameters. I have trained a lot of different models iterating over N (<code>range(5, 40, 5)</code>) and K (<code>range(1,32)</code>). </p></li>\n<li><p>Then I have used all models to make all possible predictions for each day in the test or validation sets (I have used the last 30 days for validation) and computed min, max, and average prediction for each day.</p></li>\n<li><p>It turned out, these predictions were quite pessimistic – predicted values were very high. That’s why I have applied additional linear transformation and deviation factor and computed the final predictions using formulas below (coefficients are optimized on validation). </p></li>\n</ul>\n<pre><code>pred_0.5 = a[0] * mean + a[1] * min + a[2] * max\npred_0.05 = pred_0.5 + (b[0] * mean + b[1] * min + b[2] * max) * dev(lamb_min)\npred_0.95 = pred_0.5 + (c[0] * mean + c[1] * min + c[2] * max) * dev(lamb_max)\n</code></pre>\n<p>Where <code>a</code>, <code>b</code>, and <code>c</code> are just linear coefficients. \n<code>dev(lamb) = np.array([lamb * n for n in range(num_days_to_predict)])</code> – is used to make predictions for later days less (or more) confident.</p>\n<p>What I have not done but probably should (I have joined quite late and just did not have enough time):</p>\n<ul>\n<li>Optimize hyperparameters of LGBM</li>\n<li>Smooth predictions (I see that I have a lot of outliers at some days) by (for example) using 5% and 95% percentiles of predicted values instead of min and max.</li>\n<li>Use other information as inputs or somehow utilize information about connections between different regions.</li>\n</ul>",
      "rawMarkdown": "My approach was quite straightforward: I have trained a lot of very simple LGBM models, blended their predictions, and applied an additional ~linear transformation to optimize the final metric.\n\n[Notebook with my final solution](https://www.kaggle.com/ilialar/w5-solution-2)\n\n**Overview**\n\n- Each model takes as input numbers of new cases and fatalities during the last N days divided by the maximum value of cases/fatalities for this region up to the last day in input and predicts the number of cases or fatalities on Kth day from the last input day (again divided by max value up to date).\n\n- Each model is trained using all available data: all regions and all possible N+K days frames. But different days had different weights – the later the higher. The weight for each particular day is proportional to `(0.9)**(-n)` which means, that last days in the train set had much greater significance during training.\n\n- Each model is just `lightgbm.LGBMRegressor()` with default parameters. I have trained a lot of different models iterating over N (`range(5, 40, 5)`) and K (`range(1,32)`). \n\n- Then I have used all models to make all possible predictions for each day in the test or validation sets (I have used the last 30 days for validation) and computed min, max, and average prediction for each day.\n\n- It turned out, these predictions were quite pessimistic – predicted values were very high. That’s why I have applied additional linear transformation and deviation factor and computed the final predictions using formulas below (coefficients are optimized on validation). \n\n```\npred_0.5 = a[0] * mean + a[1] * min + a[2] * max\npred_0.05 = pred_0.5 + (b[0] * mean + b[1] * min + b[2] * max) * dev(lamb_min)\npred_0.95 = pred_0.5 + (c[0] * mean + c[1] * min + c[2] * max) * dev(lamb_max)\n\n```\nWhere `a`, `b`, and `c` are just linear coefficients. \n`dev(lamb) = np.array([lamb * n for n in range(num_days_to_predict)])` – is used to make predictions for later days less (or more) confident.\n\n\n\nWhat I have not done but probably should (I have joined quite late and just did not have enough time):\n- Optimize hyperparameters of LGBM\n- Smooth predictions (I see that I have a lot of outliers at some days) by (for example) using 5% and 95% percentiles of predicted values instead of min and max.\n- Use other information as inputs or somehow utilize information about connections between different regions.\n\n",
      "votes": 5
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T12:19:45Z",
  "competition_id": 20475,
  "competition_slug": "covid19-global-forecasting-week-5",
  "team_id": 4891407,
  "solution_url": "/competitions/covid19-global-forecasting-week-5/writeups/ilia-9th-place-solution-a-lot-of-lgbms-blended"
}