{
  "id": 411561,
  "title": "Top 89 - Non Leaky Solution.",
  "content": "<p>My solution is simple and doesn't use any \"magic\" in the data. It uses visit_month as a feature +200 peptides stdscaled features. The idea is to minimize the variance of results running multiple train/validation splits and abusing of BaggingRegressor with this simple algorithm:</p>\n<pre><code> fold  ():\n    valid = &lt;- sample  patients data  trainset\n    train = &lt;- sample remaining  patients  trainset\n    model = BaggingRegressor(\n            estimator=SVR(C=),\n             n_estimators=,\n             max_samples=,\n             max_features=,\n             n_jobs= -)\n    model.fit(tr[features].values, tr[]) \n    valid[] = model.predict(valid[features])\n    VALID_LIST.append(valid)\n</code></pre>\n<p>This is basically bagging with booststrap and helps a lot stabilize the results. The thing is that using peptides features boosted my baseline score by around CV: +0.3 / LB: +0.3. <br>\n After the deadline I added the visit_month leak features into this model and it booosted by a lot my scores, but I wasn't able to beat the baseline anymore. What I concluded is that the leak features are so much stronger than the peptides features that makes the model focus only on the magic feats.<br>\nHosts, in a real world application, without any magic feature, this sampling strategy may work to extract a slightly better signal from the data.</p>",
  "comments": [
    {
      "id": 2268678,
      "author_name": "The Devastator",
      "author_url": "",
      "post_date": "2023-05-21T22:44:06.553000",
      "content": "<p>Any special reason for the <code>SVR</code>?</p>\n<p>The Devastator.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2269011,
          "author_name": "Konstantin Yakovlev",
          "author_url": "",
          "post_date": "2023-05-22T06:59:55.917000",
          "content": "<p>Let me make an assumption</p>\n<p>Giba -&gt; Nvidia -&gt; RapidsAI -&gt; SVMs -&gt; GPU-&gt; Fast as hell (with good results)</p>\n<p>Know Your own Sh… - a widely known concept. <br>\nOne thing I would say - All RapidsAI kaggle team is nailing usage of the own libraries - amazing work you are doing.  </p>",
          "votes": 2,
          "replies": []
        }
      ]
    }
  ],
  "comment_count": 2,
  "messages": [
    {
      "id": 2266098,
      "postDate": "2023-05-19T17:51:55.723Z",
      "content": "<p>My solution is simple and doesn't use any \"magic\" in the data. It uses visit_month as a feature +200 peptides stdscaled features. The idea is to minimize the variance of results running multiple train/validation splits and abusing of BaggingRegressor with this simple algorithm:</p>\n<pre><code> fold  ():\n    valid = &lt;- sample  patients data  trainset\n    train = &lt;- sample remaining  patients  trainset\n    model = BaggingRegressor(\n            estimator=SVR(C=),\n             n_estimators=,\n             max_samples=,\n             max_features=,\n             n_jobs= -)\n    model.fit(tr[features].values, tr[]) \n    valid[] = model.predict(valid[features])\n    VALID_LIST.append(valid)\n</code></pre>\n<p>This is basically bagging with booststrap and helps a lot stabilize the results. The thing is that using peptides features boosted my baseline score by around CV: +0.3 / LB: +0.3. <br>\n After the deadline I added the visit_month leak features into this model and it booosted by a lot my scores, but I wasn't able to beat the baseline anymore. What I concluded is that the leak features are so much stronger than the peptides features that makes the model focus only on the magic feats.<br>\nHosts, in a real world application, without any magic feature, this sampling strategy may work to extract a slightly better signal from the data.</p>",
      "rawMarkdown": "    My solution is simple and doesn't use any \"magic\" in the data. It uses visit_month as a feature +200 peptides stdscaled features. The idea is to minimize the variance of results running multiple train/validation splits and abusing of BaggingRegressor with this simple algorithm:\n```python\nfor fold in range(50):\n        valid = <- sample 20 patients data from trainset\n        train = <- sample remaining  patients from trainset\n        model = BaggingRegressor(\n                estimator=SVR(C=2.00),\n                 n_estimators=100,\n                 max_samples=0.334,\n                 max_features=1.00,\n                 n_jobs= -1)\n        model.fit(tr[features].values, tr['target_1_plus_0']) #  <- put the correct target here\n        valid['pred'] = model.predict(valid[features])\n        VALID_LIST.append(valid)\n```\n    This is basically bagging with booststrap and helps a lot stabilize the results. The thing is that using peptides features boosted my baseline score by around CV: +0.3 / LB: +0.3. \n\n    After the deadline I added the visit_month leak features into this model and it booosted by a lot my scores, but I wasn't able to beat the baseline anymore. What I concluded is that the leak features are so much stronger than the peptides features that makes the model focus only on the magic feats.\nHosts, in a real world application, without any magic feature, this sampling strategy may work to extract a slightly better signal from the data.",
      "votes": 27
    },
    {
      "id": 2268678,
      "postDate": "2023-05-21T22:44:06.553Z",
      "content": "<p>Any special reason for the <code>SVR</code>?</p>\n<p>The Devastator.</p>",
      "rawMarkdown": "Any special reason for the `SVR`?\n\nThe Devastator.\n",
      "votes": 1,
      "replies": [
        {
          "id": 2269011,
          "postDate": "2023-05-22T06:59:55.917Z",
          "content": "<p>Let me make an assumption</p>\n<p>Giba -&gt; Nvidia -&gt; RapidsAI -&gt; SVMs -&gt; GPU-&gt; Fast as hell (with good results)</p>\n<p>Know Your own Sh… - a widely known concept. <br>\nOne thing I would say - All RapidsAI kaggle team is nailing usage of the own libraries - amazing work you are doing.  </p>",
          "rawMarkdown": "Let me make an assumption\n\nGiba -> Nvidia -> RapidsAI -> SVMs -> GPU-> Fast as hell (with good results)\n\nKnow Your own Sh... - a widely known concept. \nOne thing I would say - All RapidsAI kaggle team is nailing usage of the own libraries - amazing work you are doing.  \n",
          "votes": 2
        }
      ]
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T14:13:20Z",
  "competition_id": 37794,
  "competition_slug": "amp-parkinsons-disease-progression-prediction",
  "team_id": 9901644,
  "solution_url": "/competitions/amp-parkinsons-disease-progression-prediction/writeups/giba-top-89-non-leaky-solution"
}