{
  "id": 609519,
  "title": "Bad x/y data",
  "url": "/competitions/MABe-mouse-behavior-detection/discussion/609519",
  "author_name": "",
  "post_date": "2025-09-27T12:21:14.184553700Z",
  "votes": 18,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Some x/y data is clearly bad. I went looking for outliers, and found a lot of them. Here is an example:</p>\n<p>det_df = pd.read_parquet(data_dir + '/train_tracking/BoisterousParrot/613246188.parquet')<br>\ndet_df.loc[(det_df['video_frame']==81097) &amp; (det_df['mouse_id']==1)]</p>\n<p>video_frame  mouse_id     bodypart           x           y<br>\n81097         1  body_center  170.328491   33.684620<br>\n81097         1     ear_left  162.863861   26.344772<br>\n81097         1    ear_right  172.397858   22.667568<br>\n81097         1         nose   63.071636  262.708344 &lt;- really bad values!!!<br>\n81097         1    tail_base  169.490662   44.906986</p>\n<p>Either 'nose' coordinates are bad, of this is some very mutated mouse!</p>\n<p>I wonder if this is something that the organizers can fix, or do we just have to live with it.</p>\n<p>And I ran this after the data was just patched, so the patch did not address this issue.</p>",
  "messages": [
    {
      "id": "3295023",
      "postDate": "09/27/2025 12:21:14",
      "content": "<p>Some x/y data is clearly bad. I went looking for outliers, and found a lot of them. Here is an example:</p>\n<p>det_df = pd.read_parquet(data_dir + '/train_tracking/BoisterousParrot/613246188.parquet')<br>\ndet_df.loc[(det_df['video_frame']==81097) &amp; (det_df['mouse_id']==1)]</p>\n<p>video_frame  mouse_id     bodypart           x           y<br>\n81097         1  body_center  170.328491   33.684620<br>\n81097         1     ear_left  162.863861   26.344772<br>\n81097         1    ear_right  172.397858   22.667568<br>\n81097         1         nose   63.071636  262.708344 &lt;- really bad values!!!<br>\n81097         1    tail_base  169.490662   44.906986</p>\n<p>Either 'nose' coordinates are bad, of this is some very mutated mouse!</p>\n<p>I wonder if this is something that the organizers can fix, or do we just have to live with it.</p>\n<p>And I ran this after the data was just patched, so the patch did not address this issue.</p>",
      "rawMarkdown": "Some x/y data is clearly bad. I went looking for outliers, and found a lot of them. Here is an example:\n\ndet_df = pd.read_parquet(data_dir + '/train_tracking/BoisterousParrot/613246188.parquet')\ndet_df.loc[(det_df['video_frame']==81097) & (det_df['mouse_id']==1)]\n\nvideo_frame  mouse_id     bodypart           x           y\n81097         1  body_center  170.328491   33.684620\n81097         1     ear_left  162.863861   26.344772\n81097         1    ear_right  172.397858   22.667568\n81097         1         nose   63.071636  262.708344 <- really bad values!!!\n81097         1    tail_base  169.490662   44.906986\n\nEither 'nose' coordinates are bad, of this is some very mutated mouse!\n\nI wonder if this is something that the organizers can fix, or do we just have to live with it.\n\nAnd I ran this after the data was just patched, so the patch did not address this issue.",
      "votes": null
    },
    {
      "id": "3295044",
      "postDate": "09/27/2025 13:39:51",
      "content": "<p>Indeed, this is one of the challenges of the dataset! Labs track mouse posture using automated pose estimation tools (mostly DeepLabCut or SLEAP)- these are computer vision pipelines that have been trained by the labs to recognize body parts of mice.</p>\n<p>Pose estimation tools work okay, but they do make plenty of mistakes! Often people try postprocessessing (auto encoders, median filters, Kalman smoothers) to try to deal with noise in the poses prior to doing classification.</p>",
      "rawMarkdown": "Indeed, this is one of the challenges of the dataset! Labs track mouse posture using automated pose estimation tools (mostly DeepLabCut or SLEAP)- these are computer vision pipelines that have been trained by the labs to recognize body parts of mice.\n\nPose estimation tools work okay, but they do make plenty of mistakes! Often people try postprocessessing (auto encoders, median filters, Kalman smoothers) to try to deal with noise in the poses prior to doing classification.",
      "votes": null
    },
    {
      "id": "3296155",
      "postDate": "09/30/2025 09:43:53",
      "content": "<p>So does that mean we won't have data updates and will only use data during the compection?</p>",
      "rawMarkdown": "So does that mean we won't have data updates and will only use data during the compection?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3295044,
      "author_name": "annkennedy05",
      "author_url": "",
      "post_date": "09/27/2025 13:39:51",
      "content": "<p>Indeed, this is one of the challenges of the dataset! Labs track mouse posture using automated pose estimation tools (mostly DeepLabCut or SLEAP)- these are computer vision pipelines that have been trained by the labs to recognize body parts of mice.</p>\n<p>Pose estimation tools work okay, but they do make plenty of mistakes! Often people try postprocessessing (auto encoders, median filters, Kalman smoothers) to try to deal with noise in the poses prior to doing classification.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3296155,
          "author_name": "snnguynvnk19hl",
          "author_url": "",
          "post_date": "09/30/2025 09:43:53",
          "content": "<p>So does that mean we won't have data updates and will only use data during the compection?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3295023": "Some x/y data is clearly bad. I went looking for outliers, and found a lot of them. Here is an example:\n\ndet_df = pd.read_parquet(data_dir + '/train_tracking/BoisterousParrot/613246188.parquet')\ndet_df.loc[(det_df['video_frame']==81097) & (det_df['mouse_id']==1)]\n\nvideo_frame  mouse_id     bodypart           x           y\n81097         1  body_center  170.328491   33.684620\n81097         1     ear_left  162.863861   26.344772\n81097         1    ear_right  172.397858   22.667568\n81097         1         nose   63.071636  262.708344 <- really bad values!!!\n81097         1    tail_base  169.490662   44.906986\n\nEither 'nose' coordinates are bad, of this is some very mutated mouse!\n\nI wonder if this is something that the organizers can fix, or do we just have to live with it.\n\nAnd I ran this after the data was just patched, so the patch did not address this issue.",
    "3295044": "Indeed, this is one of the challenges of the dataset! Labs track mouse posture using automated pose estimation tools (mostly DeepLabCut or SLEAP)- these are computer vision pipelines that have been trained by the labs to recognize body parts of mice.\n\nPose estimation tools work okay, but they do make plenty of mistakes! Often people try postprocessessing (auto encoders, median filters, Kalman smoothers) to try to deal with noise in the poses prior to doing classification.",
    "3296155": "So does that mean we won't have data updates and will only use data during the compection?"
  },
  "source": "meta"
}