{
  "id": 127308,
  "title": "How to gain +1839 places during shakeup - complete code!",
  "content": "<p>Hi all,</p>\n\n<p><a href=\"https://www.kaggle.com/marcogorelli/fork-of-quick-and-dirty-regression?scriptVersionId=27370451\">Here</a>'s the notebook I submitted, which (to my great surprise) got me in the top 12%. Not quite my first bronze, but pretty close :)</p>\n\n<p>It's a fork of Andrew Lukyanenko's excellent kernel (thanks Andrew!), the only thing I changed was that I modified some features to use the <code>expanding</code> function from <code>pandas</code>, hoping this would prevent some leakage.</p>\n\n<p>So for example, instead of</p>\n\n<p><code>\n        df['installation_duration_mean'] = df.groupby(['installation_id'])['duration_mean'].transform('mean')\n</code></p>\n\n<p>I would use</p>\n\n<p><code>\n        df['installation_duration_mean'] = df.groupby(['installation_id'])['duration_mean'].expanding().mean().reset_index(drop=True)\n</code></p>\n\n<p>I also remove his hard-coded thresholds for the QWK.</p>\n\n<p>Still trying to figure out whether my gain was due to that or just sheer luck (I suspect the latter has a lot to do with it).</p>",
  "comments": [
    {
      "id": 727245,
      "author_name": "Abhinand",
      "author_url": "",
      "post_date": "2020-01-23T15:13:58.547000",
      "content": "<p>Congratulations <a href=\"/marcogorelli\">@marcogorelli</a> </p>\n\n<p>We learned some good lessons in this competition after all. </p>\n\n<p>I had no hope of winning a medal in this competition, one of my first ever submissions which did miserably on the Public LB (single model) somehow has managed to get me in the top 8% for a bronze (+1652 places). Some competitions are strange really. It's funny now to look back at the fact that I really believed my other submission (blended) would do better. And that would have pushed me down into the 1500s. 😄 </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 726880,
      "author_name": "Hanjoon Choe",
      "author_url": "",
      "post_date": "2020-01-23T09:48:08.043000",
      "content": "<p>Congratulations!</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "comment_count": 2,
  "messages": [
    {
      "id": 726824,
      "postDate": "2020-01-23T08:58:54.613Z",
      "content": "<p>Hi all,</p>\n\n<p><a href=\"https://www.kaggle.com/marcogorelli/fork-of-quick-and-dirty-regression?scriptVersionId=27370451\">Here</a>'s the notebook I submitted, which (to my great surprise) got me in the top 12%. Not quite my first bronze, but pretty close :)</p>\n\n<p>It's a fork of Andrew Lukyanenko's excellent kernel (thanks Andrew!), the only thing I changed was that I modified some features to use the <code>expanding</code> function from <code>pandas</code>, hoping this would prevent some leakage.</p>\n\n<p>So for example, instead of</p>\n\n<p><code>\n        df['installation_duration_mean'] = df.groupby(['installation_id'])['duration_mean'].transform('mean')\n</code></p>\n\n<p>I would use</p>\n\n<p><code>\n        df['installation_duration_mean'] = df.groupby(['installation_id'])['duration_mean'].expanding().mean().reset_index(drop=True)\n</code></p>\n\n<p>I also remove his hard-coded thresholds for the QWK.</p>\n\n<p>Still trying to figure out whether my gain was due to that or just sheer luck (I suspect the latter has a lot to do with it).</p>",
      "rawMarkdown": "Hi all,\n\n[Here](https://www.kaggle.com/marcogorelli/fork-of-quick-and-dirty-regression?scriptVersionId=27370451)'s the notebook I submitted, which (to my great surprise) got me in the top 12%. Not quite my first bronze, but pretty close :)\n\nIt's a fork of Andrew Lukyanenko's excellent kernel (thanks Andrew!), the only thing I changed was that I modified some features to use the `expanding` function from `pandas`, hoping this would prevent some leakage.\n\nSo for example, instead of\n\n```\n        df['installation_duration_mean'] = df.groupby(['installation_id'])['duration_mean'].transform('mean')\n```\n\nI would use\n\n```\n        df['installation_duration_mean'] = df.groupby(['installation_id'])['duration_mean'].expanding().mean().reset_index(drop=True)\n```\n\nI also remove his hard-coded thresholds for the QWK.\n\nStill trying to figure out whether my gain was due to that or just sheer luck (I suspect the latter has a lot to do with it).",
      "votes": 5
    },
    {
      "id": 727245,
      "postDate": "2020-01-23T15:13:58.547Z",
      "content": "<p>Congratulations <a href=\"/marcogorelli\">@marcogorelli</a> </p>\n\n<p>We learned some good lessons in this competition after all. </p>\n\n<p>I had no hope of winning a medal in this competition, one of my first ever submissions which did miserably on the Public LB (single model) somehow has managed to get me in the top 8% for a bronze (+1652 places). Some competitions are strange really. It's funny now to look back at the fact that I really believed my other submission (blended) would do better. And that would have pushed me down into the 1500s. 😄 </p>",
      "rawMarkdown": "Congratulations @marcogorelli \n\nWe learned some good lessons in this competition after all. \n\nI had no hope of winning a medal in this competition, one of my first ever submissions which did miserably on the Public LB (single model) somehow has managed to get me in the top 8% for a bronze (+1652 places). Some competitions are strange really. It's funny now to look back at the fact that I really believed my other submission (blended) would do better. And that would have pushed me down into the 1500s. 😄 "
    },
    {
      "id": 726880,
      "postDate": "2020-01-23T09:48:08.043Z",
      "content": "<p>Congratulations!</p>",
      "rawMarkdown": "Congratulations!"
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-23T06:53:06Z",
  "competition_id": 16531,
  "competition_slug": "data-science-bowl-2019",
  "team_id": 4198159,
  "solution_url": "/competitions/data-science-bowl-2019/writeups/marco-gorelli-how-to-gain-1839-places-during-shake"
}