{
  "id": 596218,
  "title": "Is there a big gap between everyone's IMU-only and full-data CV scores?",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/596218",
  "author_name": "DaoHe Liu",
  "post_date": "2025-08-02T09:53:04.844000",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>My CV score and LB are correlated but not consistent, as I calculate the CV score using imu-only and the average of the full data, which results in my CV score being much lower than LB<br>\nDoes everyone have this kind of situation?<br>\nMy CV score:<br>\nimu-only:0.70<br>\nall data:0.81<br>\navg:0.755<br>\nLB:0.81</p>",
  "messages": [
    {
      "id": 3261831,
      "postDate": "2025-08-02T09:53:04.843Z",
      "content": "<p>My CV score and LB are correlated but not consistent, as I calculate the CV score using imu-only and the average of the full data, which results in my CV score being much lower than LB<br>\nDoes everyone have this kind of situation?<br>\nMy CV score:<br>\nimu-only:0.70<br>\nall data:0.81<br>\navg:0.755<br>\nLB:0.81</p>",
      "rawMarkdown": "My CV score and LB are correlated but not consistent, as I calculate the CV score using imu-only and the average of the full data, which results in my CV score being much lower than LB\nDoes everyone have this kind of situation?\nMy CV score:\nimu-only:0.70\nall data:0.81\navg:0.755\nLB:0.81",
      "votes": 1
    },
    {
      "id": 3261849,
      "postDate": "2025-08-02T11:00:47.237Z",
      "content": "<p><code>Is there a big gap between everyone's IMU-only and full-data CV scores?</code><br>\nA gap between IMU-only and full-data scores is expected, but your IMU-only score seems relatively weakperhaps due to missing engineered IMU features.<br>\nIn my case, I'm seeing an average gap of around ~0.05 between the full and IMUonly scores.<br>\nThe gestures that contribute most to this gap in my case are:</p>\n<ul>\n<li>Eyelash - Pull hair</li>\n<li>Eyebrow - Pull hair</li>\n<li>Cheek - Pinch skin</li>\n<li>Neck - Pinch/Scratch</li>\n</ul>\n<p>And as for gap in CV/LB I am facing same issue my LB is 0.02 higher than my CV, I am computing my CV same as yours.</p>",
      "rawMarkdown": "`Is there a big gap between everyone's IMU-only and full-data CV scores?`\nA gap between IMU-only and full-data scores is expected, but your IMU-only score seems relatively weakperhaps due to missing engineered IMU features.\nIn my case, I'm seeing an average gap of around ~0.05 between the full and IMUonly scores.\nThe gestures that contribute most to this gap in my case are:\n\n- Eyelash - Pull hair\n- Eyebrow - Pull hair\n- Cheek - Pinch skin\n- Neck - Pinch/Scratch\n\n\nAnd as for gap in CV/LB I am facing same issue my LB is 0.02 higher than my CV, I am computing my CV same as yours.\n",
      "replies": [
        {
          "id": 3261853,
          "postDate": "2025-08-02T11:13:23.680Z",
          "content": "<p>if the imu feature engineering is missing, i think the LB score is also be low, but my LB score is even higher than the CV score with full data. The gap between your LB and CV looks normal.</p>",
          "rawMarkdown": "if the imu feature engineering is missing, i think the LB score is also be low, but my LB score is even higher than the CV score with full data. The gap between your LB and CV looks normal."
        }
      ]
    },
    {
      "id": 3261834,
      "postDate": "2025-08-02T10:02:15.660Z",
      "content": "<p>I also found that some public notebooks have CV scores equal to LB, which is consistent with my case, if they test with imu-only data and average it might make CV less than LB?<br>\nOr is there not much difference between the results of imu-only and full data?</p>",
      "rawMarkdown": "I also found that some public notebooks have CV scores equal to LB, which is consistent with my case, if they test with imu-only data and average it might make CV less than LB?\nOr is there not much difference between the results of imu-only and full data?"
    },
    {
      "id": 3261845,
      "postDate": "2025-08-02T10:53:12.143Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 3261849,
      "author_name": "Hanan",
      "author_url": "",
      "post_date": "2025-08-02T11:00:47.237000",
      "content": "<p><code>Is there a big gap between everyone's IMU-only and full-data CV scores?</code><br>\nA gap between IMU-only and full-data scores is expected, but your IMU-only score seems relatively weakperhaps due to missing engineered IMU features.<br>\nIn my case, I'm seeing an average gap of around ~0.05 between the full and IMUonly scores.<br>\nThe gestures that contribute most to this gap in my case are:</p>\n<ul>\n<li>Eyelash - Pull hair</li>\n<li>Eyebrow - Pull hair</li>\n<li>Cheek - Pinch skin</li>\n<li>Neck - Pinch/Scratch</li>\n</ul>\n<p>And as for gap in CV/LB I am facing same issue my LB is 0.02 higher than my CV, I am computing my CV same as yours.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3261853,
          "author_name": "DaoHe Liu",
          "author_url": "",
          "post_date": "2025-08-02T11:13:23.680000",
          "content": "<p>if the imu feature engineering is missing, i think the LB score is also be low, but my LB score is even higher than the CV score with full data. The gap between your LB and CV looks normal.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 3261834,
      "author_name": "DaoHe Liu",
      "author_url": "",
      "post_date": "2025-08-02T10:02:15.660000",
      "content": "<p>I also found that some public notebooks have CV scores equal to LB, which is consistent with my case, if they test with imu-only data and average it might make CV less than LB?<br>\nOr is there not much difference between the results of imu-only and full data?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3261845,
      "author_name": "",
      "author_url": "",
      "post_date": "2025-08-02T10:53:12.143000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3261831": "My CV score and LB are correlated but not consistent, as I calculate the CV score using imu-only and the average of the full data, which results in my CV score being much lower than LB\nDoes everyone have this kind of situation?\nMy CV score:\nimu-only:0.70\nall data:0.81\navg:0.755\nLB:0.81",
    "3261849": "`Is there a big gap between everyone's IMU-only and full-data CV scores?`\nA gap between IMU-only and full-data scores is expected, but your IMU-only score seems relatively weakperhaps due to missing engineered IMU features.\nIn my case, I'm seeing an average gap of around ~0.05 between the full and IMUonly scores.\nThe gestures that contribute most to this gap in my case are:\n\n- Eyelash - Pull hair\n- Eyebrow - Pull hair\n- Cheek - Pinch skin\n- Neck - Pinch/Scratch\n\n\nAnd as for gap in CV/LB I am facing same issue my LB is 0.02 higher than my CV, I am computing my CV same as yours.\n",
    "3261834": "I also found that some public notebooks have CV scores equal to LB, which is consistent with my case, if they test with imu-only data and average it might make CV less than LB?\nOr is there not much difference between the results of imu-only and full data?",
    "3261845": ""
  }
}