{
  "id": 599233,
  "title": "Would an ensemble whose LB value is 0.853 or 0.852 be at risk of overfitting?",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/599233",
  "author_name": "Zhanyu Su",
  "post_date": "2025-08-15T02:43:09.421000",
  "votes": 11,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Would an ensemble whose LB value is 0.853 or 0.852 be at risk of overfitting? How can I verify it?</p>",
  "messages": [
    {
      "id": 3269765,
      "postDate": "2025-08-15T02:43:09.423Z",
      "content": "<p>Would an ensemble whose LB value is 0.853 or 0.852 be at risk of overfitting? How can I verify it?</p>",
      "rawMarkdown": "Would an ensemble whose LB value is 0.853 or 0.852 be at risk of overfitting? How can I verify it?",
      "votes": 11
    },
    {
      "id": 3269789,
      "postDate": "2025-08-15T03:55:27.190Z",
      "content": "<p><strong>What raises the risk</strong></p>\n<ul>\n<li>Large CV → LB gap (LB ≫ CV mean).</li>\n<li>You “cherry-picked” from many submissions (top-of-N effect).</li>\n<li>Heavy threshold/weight tuning using LB feedback.</li>\n<li>High seed sensitivity (small training changes → big LB swings).</li>\n</ul>",
      "rawMarkdown": "**What raises the risk**\n\n- Large CV → LB gap (LB ≫ CV mean).\n- You “cherry-picked” from many submissions (top-of-N effect).\n- Heavy threshold/weight tuning using LB feedback.\n- High seed sensitivity (small training changes → big LB swings).",
      "votes": 4
    },
    {
      "id": 3269861,
      "postDate": "2025-08-15T07:33:08.360Z",
      "content": "<p>Why downvote this post? Isn't this true?</p>",
      "rawMarkdown": "Why downvote this post? Isn't this true?",
      "votes": 1
    },
    {
      "id": 3269788,
      "postDate": "2025-08-15T03:54:54.373Z",
      "content": "<p>Those guys use the same model to ensemble and adjust the merge weight, The score improve from 0.83-&gt;0.85，such a gap ,just reached by changing  weight, without any new feature/methods adding in. So I think definitely overfitting the public board🤣</p>",
      "rawMarkdown": "Those guys use the same model to ensemble and adjust the merge weight, The score improve from 0.83->0.85，such a gap ,just reached by changing  weight, without any new feature/methods adding in. So I think definitely overfitting the public board🤣",
      "votes": 2
    },
    {
      "id": 3269782,
      "postDate": "2025-08-15T03:34:14.660Z",
      "content": "<p>I think it is overfitting because of different cross-validation strategy used for each models.</p>",
      "rawMarkdown": "I think it is overfitting because of different cross-validation strategy used for each models."
    },
    {
      "id": 3269838,
      "postDate": "2025-08-15T06:14:59.440Z",
      "content": "<p>You can make sure the architectures differ for each notebook. My solution involves different architectures; some achieve high CV scores, while others achieve low CV scores and i keep tracking the oof score.</p>",
      "rawMarkdown": "You can make sure the architectures differ for each notebook. My solution involves different architectures; some achieve high CV scores, while others achieve low CV scores and i keep tracking the oof score."
    },
    {
      "id": 3269787,
      "postDate": "2025-08-15T03:53:38.207Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 3269789,
      "author_name": "Haruto Kotou",
      "author_url": "",
      "post_date": "2025-08-15T03:55:27.190000",
      "content": "<p><strong>What raises the risk</strong></p>\n<ul>\n<li>Large CV → LB gap (LB ≫ CV mean).</li>\n<li>You “cherry-picked” from many submissions (top-of-N effect).</li>\n<li>Heavy threshold/weight tuning using LB feedback.</li>\n<li>High seed sensitivity (small training changes → big LB swings).</li>\n</ul>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 3269861,
      "author_name": "Timmy Juicehouse",
      "author_url": "",
      "post_date": "2025-08-15T07:33:08.360000",
      "content": "<p>Why downvote this post? Isn't this true?</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 3269788,
      "author_name": "XuKong Ji",
      "author_url": "",
      "post_date": "2025-08-15T03:54:54.373000",
      "content": "<p>Those guys use the same model to ensemble and adjust the merge weight, The score improve from 0.83-&gt;0.85，such a gap ,just reached by changing  weight, without any new feature/methods adding in. So I think definitely overfitting the public board🤣</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 3269782,
      "author_name": "proj1234",
      "author_url": "",
      "post_date": "2025-08-15T03:34:14.660000",
      "content": "<p>I think it is overfitting because of different cross-validation strategy used for each models.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3269838,
      "author_name": "EliKal",
      "author_url": "",
      "post_date": "2025-08-15T06:14:59.440000",
      "content": "<p>You can make sure the architectures differ for each notebook. My solution involves different architectures; some achieve high CV scores, while others achieve low CV scores and i keep tracking the oof score.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3269787,
      "author_name": "",
      "author_url": "",
      "post_date": "2025-08-15T03:53:38.207000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3269765": "Would an ensemble whose LB value is 0.853 or 0.852 be at risk of overfitting? How can I verify it?",
    "3269789": "**What raises the risk**\n\n- Large CV → LB gap (LB ≫ CV mean).\n- You “cherry-picked” from many submissions (top-of-N effect).\n- Heavy threshold/weight tuning using LB feedback.\n- High seed sensitivity (small training changes → big LB swings).",
    "3269861": "Why downvote this post? Isn't this true?",
    "3269788": "Those guys use the same model to ensemble and adjust the merge weight, The score improve from 0.83->0.85，such a gap ,just reached by changing  weight, without any new feature/methods adding in. So I think definitely overfitting the public board🤣",
    "3269782": "I think it is overfitting because of different cross-validation strategy used for each models.",
    "3269838": "You can make sure the architectures differ for each notebook. My solution involves different architectures; some achieve high CV scores, while others achieve low CV scores and i keep tracking the oof score.",
    "3269787": ""
  }
}