{
  "id": 609092,
  "title": "Questions on tracking and annotation details",
  "url": "/competitions/MABe-mouse-behavior-detection/discussion/609092",
  "author_name": "",
  "post_date": "2025-09-23T18:07:01.438547400Z",
  "votes": 23,
  "comment_count": 11,
  "views": 0,
  "content": "<p>While exploring the dataset, I noticed a few potential inconsistencies:</p>\n<p>1) fps × duration vs tracked span<br>\nSome videos show shorter or longer tracked spans (last_frame - first_frame + 1 in tracking files) than fps × duration (from train.csv)</p>\n<p>Is this due to tracking can starts late/ends early? I can't figure out how there can be longer tracking than video duration..</p>\n<p>2) Coordinates outside video bounds (153 videos)<br>\nThere are negative or &gt; video width/height coordinates.</p>\n<p>What is the intended origin and scale (top-left (0,0)? all coords are pixels)? Are off-screen coordinates expected?</p>\n<p>3) Arena larger than video (64 videos) - already mentioned in other topic, will be corrected</p>\n<p>4) Very short actions (≤0.1 s across 244 videos)</p>\n<p>Are 1–3-frame events considered valid labels? Do we have such short actions in test?</p>\n<p>all are explored in this notebook: <a href=\"https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues\" target=\"_blank\">https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues</a><br>\nCould the organizers please clarify how we should interpret these cases?</p>",
  "messages": [
    {
      "id": "3293407",
      "postDate": "09/23/2025 18:07:01",
      "content": "<p>While exploring the dataset, I noticed a few potential inconsistencies:</p>\n<p>1) fps × duration vs tracked span<br>\nSome videos show shorter or longer tracked spans (last_frame - first_frame + 1 in tracking files) than fps × duration (from train.csv)</p>\n<p>Is this due to tracking can starts late/ends early? I can't figure out how there can be longer tracking than video duration..</p>\n<p>2) Coordinates outside video bounds (153 videos)<br>\nThere are negative or &gt; video width/height coordinates.</p>\n<p>What is the intended origin and scale (top-left (0,0)? all coords are pixels)? Are off-screen coordinates expected?</p>\n<p>3) Arena larger than video (64 videos) - already mentioned in other topic, will be corrected</p>\n<p>4) Very short actions (≤0.1 s across 244 videos)</p>\n<p>Are 1–3-frame events considered valid labels? Do we have such short actions in test?</p>\n<p>all are explored in this notebook: <a href=\"https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues\" target=\"_blank\">https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues</a><br>\nCould the organizers please clarify how we should interpret these cases?</p>",
      "rawMarkdown": "While exploring the dataset, I noticed a few potential inconsistencies:\n\n1) fps × duration vs tracked span\nSome videos show shorter or longer tracked spans (last_frame - first_frame + 1 in tracking files) than fps × duration (from train.csv)\n\nIs this due to tracking can starts late/ends early? I can't figure out how there can be longer tracking than video duration..\n\n2) Coordinates outside video bounds (153 videos)\nThere are negative or > video width/height coordinates.\n\nWhat is the intended origin and scale (top-left (0,0)? all coords are pixels)? Are off-screen coordinates expected?\n\n3) Arena larger than video (64 videos) - already mentioned in other topic, will be corrected\n\n4) Very short actions (≤0.1 s across 244 videos)\n\nAre 1–3-frame events considered valid labels? Do we have such short actions in test?\n\nall are explored in this notebook: https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues\nCould the organizers please clarify how we should interpret these cases?",
      "votes": null
    },
    {
      "id": "3293409",
      "postDate": "09/23/2025 18:13:13",
      "content": "<p>We're working on a data patch now. Please check back after that's live later this week.</p>",
      "rawMarkdown": "We're working on a data patch now. Please check back after that's live later this week.",
      "votes": null
    },
    {
      "id": "3293523",
      "postDate": "09/24/2025 04:05:12",
      "content": "<p>Thank you for your diligent review of the dataset, this is great! Going through this:</p>\n<ol>\n<li><p>missing frames<br>\nNot in your forum post, but noticed it in your notebook. Sometimes there are frames where no body parts are detected, my suspicion is these are what you're finding as missing. In AdaptableSnail this happens because there are some covered spots where mice can hide from view. In some other datasets, eg JovialSwallow, there's so much motion blur during attack that the mice aren't really discernible to the tracker. I think you are right that the overall amount of this is fairly small.</p></li>\n<li><p>fps x duration vs tracked span<br>\nSmall differences are possible due to dropped frames (eg the video reports being 30 fps, but sometimes the camera fails to save a frame so the actual frame count is shorter.) The larger discrepancies you report are more unusual and I will check them- we used cv2.VideoCapture to get video frame counts, and this is known to sometimes return incorrect values. My instinct would be to trust the fps over the video duration when there is disagreement.</p></li>\n<li><p>Coordinates outside video bounds<br>\nThis is a known property of DeepLabCut (<a href=\"https://forum.image.sc/t/negative-y-coordinates/42499\" target=\"_blank\">https://forum.image.sc/t/negative-y-coordinates/42499</a> ): the model is extrapolating to predict locations of body parts that are outside the video bounds, based on what parts are visible.</p></li>\n<li><p>Arena larger than video<br>\nThis is now resolved, and the new dataset should post tomorrow.</p></li>\n<li><p>Very short actions<br>\nSome annotations we received really were quite short, but they are real. We removed single-frame annotations from the Adjective-Animal labs (both training and test sets), but left the single-frame labels in CalMS21, CRIM13, and MABe22 since these are previously published datasets and we didn't want to create version discrepancies.</p></li>\n</ol>",
      "rawMarkdown": "Thank you for your diligent review of the dataset, this is great! Going through this:\n\n0. missing frames\nNot in your forum post, but noticed it in your notebook. Sometimes there are frames where no body parts are detected, my suspicion is these are what you're finding as missing. In AdaptableSnail this happens because there are some covered spots where mice can hide from view. In some other datasets, eg JovialSwallow, there's so much motion blur during attack that the mice aren't really discernible to the tracker. I think you are right that the overall amount of this is fairly small.\n\n1. fps x duration vs tracked span\nSmall differences are possible due to dropped frames (eg the video reports being 30 fps, but sometimes the camera fails to save a frame so the actual frame count is shorter.) The larger discrepancies you report are more unusual and I will check them- we used cv2.VideoCapture to get video frame counts, and this is known to sometimes return incorrect values. My instinct would be to trust the fps over the video duration when there is disagreement.\n\n2. Coordinates outside video bounds\nThis is a known property of DeepLabCut (https://forum.image.sc/t/negative-y-coordinates/42499 ): the model is extrapolating to predict locations of body parts that are outside the video bounds, based on what parts are visible.\n\n3. Arena larger than video\nThis is now resolved, and the new dataset should post tomorrow.\n\n4. Very short actions\nSome annotations we received really were quite short, but they are real. We removed single-frame annotations from the Adjective-Animal labs (both training and test sets), but left the single-frame labels in CalMS21, CRIM13, and MABe22 since these are previously published datasets and we didn't want to create version discrepancies.",
      "votes": null
    },
    {
      "id": "3294216",
      "postDate": "09/25/2025 15:58:31",
      "content": "<p>Thank you very much for the detailed explanation! I really appreciate it!</p>",
      "rawMarkdown": "Thank you very much for the detailed explanation! I really appreciate it!",
      "votes": null
    },
    {
      "id": "3294448",
      "postDate": "09/26/2025 07:42:38",
      "content": "<p>Many videos are sparse labeled. I would like to ask all frame actions should be theoretically provided / in every frame some action is happening if mice are visible? Why the first mouse (id=1) has the most actions in many cases? </p>",
      "rawMarkdown": "Many videos are sparse labeled. I would like to ask all frame actions should be theoretically provided / in every frame some action is happening if mice are visible? Why the first mouse (id=1) has the most actions in many cases?",
      "votes": null
    },
    {
      "id": "3295078",
      "postDate": "09/27/2025 15:32:50",
      "content": "<p>Indeed, labels are sparse. The behaviors listed in <code>behaviors_labeled</code> in the metadata are the ones that labs annotated for in the video- it is quite likely that mice are doing other stuff on the frames without labels, they just might not be things that labs cared about in their experiments. (For example a lab studying mouse social behavior might not bother to annotate frames when a mouse is grooming.)</p>\n<p>In some experiment types, eg 'resident-intruder',  only the behavior of one mouse is being studied; mouse 2 (the 'intruder') is effectively a stimulus used to see how mouse 1 behaves. In these cases, often only mouse 1's actions will be annotated.</p>\n<p>The general conclusion- this is a collection of scientific datasets where annotations are precise but not exhaustive, and reflect the set of behaviors that researchers often want to quantify in their experiments.</p>",
      "rawMarkdown": "Indeed, labels are sparse. The behaviors listed in `behaviors_labeled` in the metadata are the ones that labs annotated for in the video- it is quite likely that mice are doing other stuff on the frames without labels, they just might not be things that labs cared about in their experiments. (For example a lab studying mouse social behavior might not bother to annotate frames when a mouse is grooming.)\n\nIn some experiment types, eg 'resident-intruder',  only the behavior of one mouse is being studied; mouse 2 (the 'intruder') is effectively a stimulus used to see how mouse 1 behaves. In these cases, often only mouse 1's actions will be annotated.\n\nThe general conclusion- this is a collection of scientific datasets where annotations are precise but not exhaustive, and reflect the set of behaviors that researchers often want to quantify in their experiments.",
      "votes": null
    },
    {
      "id": "3295464",
      "postDate": "09/28/2025 20:44:34",
      "content": "<p>AFTER THE DATA UPDATE PATCH:</p>\n<p>there are still some strange inconsistencies:</p>\n<ul>\n<li><p>If you compare the number of frames derived from train.csv (fps × duration) with the <strong>span</strong> observed in tracking files (last frame minus first frame + 1, <strong>which is invariant to missing frames somewhere in between</strong>), you can see that there are large negative mismatches (may be due to clipping - tracking starts late or ends early?), as well as more strange positive mismatches (more frames in tracking files than expected from train.csv fps × duration). For example:</p></li>\n<li><p>video_id 1377260232 (CalMS21_task1), frames span is 0 to 7023 = 7024, fps = 30, video_duration_sec = 2184.50 which gives 65535 expected frames with tracking., i.e. 58511 frames have no tracking or either of the duration or fps in train.csv is incorrrect</p></li>\n<li><p>video_id 1319309143 (CRIM13), frames span is 4092 to 9547 = 5456 frames, fps = 25, video_duration_sec = 64.84 which gives 1621 expected frames with tracking., i.e. 3835 frames are <strong>extras</strong> indicating that either of the duration or fps in train.csv is incorrrect</p></li>\n<li><p>video_id 991578442 (GroovyShrew), frames span is 1311 to 43258 = 41948, fps = 30, video_duration_sec = 1435.4 which gives 43062 expected frames with tracking., i.e. 1114 frames have no tracking. </p></li>\n</ul>\n<p>more examples are in the notebook here: <a href=\"https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues?scriptVersionId=264558125\" target=\"_blank\">https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues?scriptVersionId=264558125</a></p>\n<blockquote>\n  <p>CautiousGiraffe: corrected arena size metadata from 50x50cm to 30x30cm (impacts both training + test data.)</p>\n</blockquote>\n<p>I see the changes, but still, with pix_per_cm_approx = 21, it's 630 x 630 pix, while video is 600 x 600 pix. Also, e.g.:</p>\n<ul>\n<li>SparklingTapir video 1237008561 have pix_per_cm_approx = 65, arena 19 x 28.5 cm, which gives 1235 pix height, while video has 1200 pix height</li>\n</ul>\n<p>There are still 4 labs with such, albeit small, differences. I guess, these are just rounding issues?</p>\n<p>Additionally, I recently have found occasional counterintuitive reciprocal actions in annotations, like these:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F028cc1b2fa775bee9abefcedf72dff3f%2FScreenshot%202025-09-28%20232005.png?generation=1759090816044573&amp;alt=media\" alt=\"\"></p>\n<p>It's difficult to imagine a mouse approaching while the target mouse sniffs it 😅…<br>\nThese are rare and most likely reflect the normal limits of manual annotation; not a major issue but worth being aware of.</p>",
      "rawMarkdown": "AFTER THE DATA UPDATE PATCH:\n\nthere are still some strange inconsistencies:\n\n- If you compare the number of frames derived from train.csv (fps × duration) with the **span** observed in tracking files (last frame minus first frame + 1, **which is invariant to missing frames somewhere in between**), you can see that there are large negative mismatches (may be due to clipping - tracking starts late or ends early?), as well as more strange positive mismatches (more frames in tracking files than expected from train.csv fps × duration). For example:\n\n- video_id 1377260232 (CalMS21_task1), frames span is 0 to 7023 = 7024, fps = 30, video_duration_sec = 2184.50 which gives 65535 expected frames with tracking., i.e. 58511 frames have no tracking or either of the duration or fps in train.csv is incorrrect\n\n- video_id 1319309143 (CRIM13), frames span is 4092 to 9547 = 5456 frames, fps = 25, video_duration_sec = 64.84 which gives 1621 expected frames with tracking., i.e. 3835 frames are **extras** indicating that either of the duration or fps in train.csv is incorrrect\n\n- video_id 991578442 (GroovyShrew), frames span is 1311 to 43258 = 41948, fps = 30, video_duration_sec = 1435.4 which gives 43062 expected frames with tracking., i.e. 1114 frames have no tracking. \n\nmore examples are in the notebook here: https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues?scriptVersionId=264558125\n\n>CautiousGiraffe: corrected arena size metadata from 50x50cm to 30x30cm (impacts both training + test data.)\n\nI see the changes, but still, with pix_per_cm_approx = 21, it's 630 x 630 pix, while video is 600 x 600 pix. Also, e.g.:\n- SparklingTapir video 1237008561 have pix_per_cm_approx = 65, arena 19 x 28.5 cm, which gives 1235 pix height, while video has 1200 pix height\n\nThere are still 4 labs with such, albeit small, differences. I guess, these are just rounding issues?\n\nAdditionally, I recently have found occasional counterintuitive reciprocal actions in annotations, like these:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F028cc1b2fa775bee9abefcedf72dff3f%2FScreenshot%202025-09-28%20232005.png?generation=1759090816044573&alt=media)\n\nIt's difficult to imagine a mouse approaching while the target mouse sniffs it 😅...\nThese are rare and most likely reflect the normal limits of manual annotation; not a major issue but worth being aware of.",
      "votes": null
    },
    {
      "id": "3296119",
      "postDate": "09/30/2025 08:18:29",
      "content": "<p>Is it expected to have tracked body parts inconsistent between mice in a single lab?<br>\nE.g. AdaptableSnail has no ear_left data only for mouse2 but 100% coverage for other mice.</p>",
      "rawMarkdown": "Is it expected to have tracked body parts inconsistent between mice in a single lab?\nE.g. AdaptableSnail has no ear_left data only for mouse2 but 100% coverage for other mice.",
      "votes": null
    },
    {
      "id": "3297825",
      "postDate": "10/03/2025 21:35:18",
      "content": "<p>Sorry for raising yet another data quality question - maybe this is just my misunderstanding, but I’d like to clarify how the arena metadata should be interpreted <a href=\"https://www.kaggle.com/annkennedy05\" target=\"_blank\">@annkennedy05</a> </p>\n<p>SparklingTapir is the most inconsistent case:</p>\n<ul>\n<li>There is an arena CSDS with sizes 12.7 × 46.0 cm and another one 45.7 × 12.0 cm - swapped dimensions?</li>\n<li>the are videos with the same arena size and resolution but very different pixels/cm values </li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2Fbc55b1fef1190bc0889c42bf3915ac63%2FScreenshot%202025-10-04%20003110.png?generation=1759527104262770&amp;alt=media\" alt=\"\"></p>\n<p>ElegantMink: At the same resolution (720×480) pix_per_cm_approx ranges ~17–20, two arena sizes are reported (33×16 cm and 33×22 cm), and one video is 640×480 but still arena is 33×22 cm with pix_per_cm_approx ≈18.2 matching entries with same arena size and video 720×480. </p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F09ed8b274b8f9ce0e7e5f669d6fa786d%2FScreenshot%202025-10-04%20003319.png?generation=1759527213464690&amp;alt=media\" alt=\"\"></p>\n<p>TranquilPanther: For the same 640×480 resolution and arena size 35x20, pix_per_cm_approx ranges widely (9.0–13.7)</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F2d6876656425c15e8b1ce8ceb2ac2e14%2FScreenshot%202025-10-04%20003430.png?generation=1759527287046774&amp;alt=media\" alt=\"\"></p>\n<p>PleasantMeerkat: pix_per_cm_approx here = 15.8, apparently looks like mean/rounded value applied across all videos 2000 x 2022 to 2048 x 2036</p>\n<p>All other labs use a single arena type and size/pix_per_cm_approx/video size across their videos, or the differences are small</p>",
      "rawMarkdown": "Sorry for raising yet another data quality question - maybe this is just my misunderstanding, but I’d like to clarify how the arena metadata should be interpreted @annkennedy05 \n\nSparklingTapir is the most inconsistent case:\n- There is an arena CSDS with sizes 12.7 × 46.0 cm and another one 45.7 × 12.0 cm - swapped dimensions?\n- the are videos with the same arena size and resolution but very different pixels/cm values \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2Fbc55b1fef1190bc0889c42bf3915ac63%2FScreenshot%202025-10-04%20003110.png?generation=1759527104262770&alt=media)\n\nElegantMink: At the same resolution (720×480) pix_per_cm_approx ranges ~17–20, two arena sizes are reported (33×16 cm and 33×22 cm), and one video is 640×480 but still arena is 33×22 cm with pix_per_cm_approx ≈18.2 matching entries with same arena size and video 720×480. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F09ed8b274b8f9ce0e7e5f669d6fa786d%2FScreenshot%202025-10-04%20003319.png?generation=1759527213464690&alt=media)\n\nTranquilPanther: For the same 640×480 resolution and arena size 35x20, pix_per_cm_approx ranges widely (9.0–13.7)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F2d6876656425c15e8b1ce8ceb2ac2e14%2FScreenshot%202025-10-04%20003430.png?generation=1759527287046774&alt=media)\n\nPleasantMeerkat: pix_per_cm_approx here = 15.8, apparently looks like mean/rounded value applied across all videos 2000 x 2022 to 2048 x 2036\n\nAll other labs use a single arena type and size/pix_per_cm_approx/video size across their videos, or the differences are small",
      "votes": null
    },
    {
      "id": "3297845",
      "postDate": "10/03/2025 21:54:55",
      "content": "<p>Poor mouse 😭</p>",
      "rawMarkdown": "Poor mouse 😭",
      "votes": null
    },
    {
      "id": "3307411",
      "postDate": "10/26/2025 23:05:11",
      "content": "<p><a href=\"https://www.kaggle.com/annkennedy05\" target=\"_blank\">@annkennedy05</a> could you please clarify on the questions above about arena metadata at least? 🙏</p>",
      "rawMarkdown": "annkennedy05 could you please clarify on the questions above about arena metadata at least? 🙏",
      "votes": null
    },
    {
      "id": "3309028",
      "postDate": "10/30/2025 18:59:50",
      "content": "<blockquote>\n  <p><a href=\"https://www.kaggle.com/annkennedy05\" target=\"_blank\">@annkennedy05</a> could you please clarify on the questions above about arena metadata at least? 🙏</p>\n</blockquote>\n<p>Thanks for bringing up this question as well for the deep dive into the data — I’d like to take the opportunity to ask the organizer how pix_per_cm_approx is estimated as well. <a href=\"https://www.kaggle.com/annkennedy05\" target=\"_blank\">@annkennedy05</a> </p>",
      "rawMarkdown": "> @annkennedy05 could you please clarify on the questions above about arena metadata at least? 🙏\n\nThanks for bringing up this question as well for the deep dive into the data — I’d like to take the opportunity to ask the organizer how pix_per_cm_approx is estimated as well. @annkennedy05",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3293409,
      "author_name": "sohier",
      "author_url": "",
      "post_date": "09/23/2025 18:13:13",
      "content": "<p>We're working on a data patch now. Please check back after that's live later this week.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3293523,
      "author_name": "annkennedy05",
      "author_url": "",
      "post_date": "09/24/2025 04:05:12",
      "content": "<p>Thank you for your diligent review of the dataset, this is great! Going through this:</p>\n<ol>\n<li><p>missing frames<br>\nNot in your forum post, but noticed it in your notebook. Sometimes there are frames where no body parts are detected, my suspicion is these are what you're finding as missing. In AdaptableSnail this happens because there are some covered spots where mice can hide from view. In some other datasets, eg JovialSwallow, there's so much motion blur during attack that the mice aren't really discernible to the tracker. I think you are right that the overall amount of this is fairly small.</p></li>\n<li><p>fps x duration vs tracked span<br>\nSmall differences are possible due to dropped frames (eg the video reports being 30 fps, but sometimes the camera fails to save a frame so the actual frame count is shorter.) The larger discrepancies you report are more unusual and I will check them- we used cv2.VideoCapture to get video frame counts, and this is known to sometimes return incorrect values. My instinct would be to trust the fps over the video duration when there is disagreement.</p></li>\n<li><p>Coordinates outside video bounds<br>\nThis is a known property of DeepLabCut (<a href=\"https://forum.image.sc/t/negative-y-coordinates/42499\" target=\"_blank\">https://forum.image.sc/t/negative-y-coordinates/42499</a> ): the model is extrapolating to predict locations of body parts that are outside the video bounds, based on what parts are visible.</p></li>\n<li><p>Arena larger than video<br>\nThis is now resolved, and the new dataset should post tomorrow.</p></li>\n<li><p>Very short actions<br>\nSome annotations we received really were quite short, but they are real. We removed single-frame annotations from the Adjective-Animal labs (both training and test sets), but left the single-frame labels in CalMS21, CRIM13, and MABe22 since these are previously published datasets and we didn't want to create version discrepancies.</p></li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 3294216,
          "author_name": "antoninadolgorukova",
          "author_url": "",
          "post_date": "09/25/2025 15:58:31",
          "content": "<p>Thank you very much for the detailed explanation! I really appreciate it!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3294448,
      "author_name": "jankowalski2000",
      "author_url": "",
      "post_date": "09/26/2025 07:42:38",
      "content": "<p>Many videos are sparse labeled. I would like to ask all frame actions should be theoretically provided / in every frame some action is happening if mice are visible? Why the first mouse (id=1) has the most actions in many cases? </p>",
      "votes": null,
      "replies": [
        {
          "id": 3295078,
          "author_name": "annkennedy05",
          "author_url": "",
          "post_date": "09/27/2025 15:32:50",
          "content": "<p>Indeed, labels are sparse. The behaviors listed in <code>behaviors_labeled</code> in the metadata are the ones that labs annotated for in the video- it is quite likely that mice are doing other stuff on the frames without labels, they just might not be things that labs cared about in their experiments. (For example a lab studying mouse social behavior might not bother to annotate frames when a mouse is grooming.)</p>\n<p>In some experiment types, eg 'resident-intruder',  only the behavior of one mouse is being studied; mouse 2 (the 'intruder') is effectively a stimulus used to see how mouse 1 behaves. In these cases, often only mouse 1's actions will be annotated.</p>\n<p>The general conclusion- this is a collection of scientific datasets where annotations are precise but not exhaustive, and reflect the set of behaviors that researchers often want to quantify in their experiments.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3295464,
      "author_name": "antoninadolgorukova",
      "author_url": "",
      "post_date": "09/28/2025 20:44:34",
      "content": "<p>AFTER THE DATA UPDATE PATCH:</p>\n<p>there are still some strange inconsistencies:</p>\n<ul>\n<li><p>If you compare the number of frames derived from train.csv (fps × duration) with the <strong>span</strong> observed in tracking files (last frame minus first frame + 1, <strong>which is invariant to missing frames somewhere in between</strong>), you can see that there are large negative mismatches (may be due to clipping - tracking starts late or ends early?), as well as more strange positive mismatches (more frames in tracking files than expected from train.csv fps × duration). For example:</p></li>\n<li><p>video_id 1377260232 (CalMS21_task1), frames span is 0 to 7023 = 7024, fps = 30, video_duration_sec = 2184.50 which gives 65535 expected frames with tracking., i.e. 58511 frames have no tracking or either of the duration or fps in train.csv is incorrrect</p></li>\n<li><p>video_id 1319309143 (CRIM13), frames span is 4092 to 9547 = 5456 frames, fps = 25, video_duration_sec = 64.84 which gives 1621 expected frames with tracking., i.e. 3835 frames are <strong>extras</strong> indicating that either of the duration or fps in train.csv is incorrrect</p></li>\n<li><p>video_id 991578442 (GroovyShrew), frames span is 1311 to 43258 = 41948, fps = 30, video_duration_sec = 1435.4 which gives 43062 expected frames with tracking., i.e. 1114 frames have no tracking. </p></li>\n</ul>\n<p>more examples are in the notebook here: <a href=\"https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues?scriptVersionId=264558125\" target=\"_blank\">https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues?scriptVersionId=264558125</a></p>\n<blockquote>\n  <p>CautiousGiraffe: corrected arena size metadata from 50x50cm to 30x30cm (impacts both training + test data.)</p>\n</blockquote>\n<p>I see the changes, but still, with pix_per_cm_approx = 21, it's 630 x 630 pix, while video is 600 x 600 pix. Also, e.g.:</p>\n<ul>\n<li>SparklingTapir video 1237008561 have pix_per_cm_approx = 65, arena 19 x 28.5 cm, which gives 1235 pix height, while video has 1200 pix height</li>\n</ul>\n<p>There are still 4 labs with such, albeit small, differences. I guess, these are just rounding issues?</p>\n<p>Additionally, I recently have found occasional counterintuitive reciprocal actions in annotations, like these:</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F028cc1b2fa775bee9abefcedf72dff3f%2FScreenshot%202025-09-28%20232005.png?generation=1759090816044573&amp;alt=media\" alt=\"\"></p>\n<p>It's difficult to imagine a mouse approaching while the target mouse sniffs it 😅…<br>\nThese are rare and most likely reflect the normal limits of manual annotation; not a major issue but worth being aware of.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3296119,
      "author_name": "elpres1dente",
      "author_url": "",
      "post_date": "09/30/2025 08:18:29",
      "content": "<p>Is it expected to have tracked body parts inconsistent between mice in a single lab?<br>\nE.g. AdaptableSnail has no ear_left data only for mouse2 but 100% coverage for other mice.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3297845,
          "author_name": "willhirsch",
          "author_url": "",
          "post_date": "10/03/2025 21:54:55",
          "content": "<p>Poor mouse 😭</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3297825,
      "author_name": "antoninadolgorukova",
      "author_url": "",
      "post_date": "10/03/2025 21:35:18",
      "content": "<p>Sorry for raising yet another data quality question - maybe this is just my misunderstanding, but I’d like to clarify how the arena metadata should be interpreted <a href=\"https://www.kaggle.com/annkennedy05\" target=\"_blank\">@annkennedy05</a> </p>\n<p>SparklingTapir is the most inconsistent case:</p>\n<ul>\n<li>There is an arena CSDS with sizes 12.7 × 46.0 cm and another one 45.7 × 12.0 cm - swapped dimensions?</li>\n<li>the are videos with the same arena size and resolution but very different pixels/cm values </li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2Fbc55b1fef1190bc0889c42bf3915ac63%2FScreenshot%202025-10-04%20003110.png?generation=1759527104262770&amp;alt=media\" alt=\"\"></p>\n<p>ElegantMink: At the same resolution (720×480) pix_per_cm_approx ranges ~17–20, two arena sizes are reported (33×16 cm and 33×22 cm), and one video is 640×480 but still arena is 33×22 cm with pix_per_cm_approx ≈18.2 matching entries with same arena size and video 720×480. </p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F09ed8b274b8f9ce0e7e5f669d6fa786d%2FScreenshot%202025-10-04%20003319.png?generation=1759527213464690&amp;alt=media\" alt=\"\"></p>\n<p>TranquilPanther: For the same 640×480 resolution and arena size 35x20, pix_per_cm_approx ranges widely (9.0–13.7)</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F2d6876656425c15e8b1ce8ceb2ac2e14%2FScreenshot%202025-10-04%20003430.png?generation=1759527287046774&amp;alt=media\" alt=\"\"></p>\n<p>PleasantMeerkat: pix_per_cm_approx here = 15.8, apparently looks like mean/rounded value applied across all videos 2000 x 2022 to 2048 x 2036</p>\n<p>All other labs use a single arena type and size/pix_per_cm_approx/video size across their videos, or the differences are small</p>",
      "votes": null,
      "replies": [
        {
          "id": 3307411,
          "author_name": "antoninadolgorukova",
          "author_url": "",
          "post_date": "10/26/2025 23:05:11",
          "content": "<p><a href=\"https://www.kaggle.com/annkennedy05\" target=\"_blank\">@annkennedy05</a> could you please clarify on the questions above about arena metadata at least? 🙏</p>",
          "votes": null,
          "replies": [
            {
              "id": 3309028,
              "author_name": "yijiey",
              "author_url": "",
              "post_date": "10/30/2025 18:59:50",
              "content": "<blockquote>\n  <p><a href=\"https://www.kaggle.com/annkennedy05\" target=\"_blank\">@annkennedy05</a> could you please clarify on the questions above about arena metadata at least? 🙏</p>\n</blockquote>\n<p>Thanks for bringing up this question as well for the deep dive into the data — I’d like to take the opportunity to ask the organizer how pix_per_cm_approx is estimated as well. <a href=\"https://www.kaggle.com/annkennedy05\" target=\"_blank\">@annkennedy05</a> </p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3293407": "While exploring the dataset, I noticed a few potential inconsistencies:\n\n1) fps × duration vs tracked span\nSome videos show shorter or longer tracked spans (last_frame - first_frame + 1 in tracking files) than fps × duration (from train.csv)\n\nIs this due to tracking can starts late/ends early? I can't figure out how there can be longer tracking than video duration..\n\n2) Coordinates outside video bounds (153 videos)\nThere are negative or > video width/height coordinates.\n\nWhat is the intended origin and scale (top-left (0,0)? all coords are pixels)? Are off-screen coordinates expected?\n\n3) Arena larger than video (64 videos) - already mentioned in other topic, will be corrected\n\n4) Very short actions (≤0.1 s across 244 videos)\n\nAre 1–3-frame events considered valid labels? Do we have such short actions in test?\n\nall are explored in this notebook: https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues\nCould the organizers please clarify how we should interpret these cases?",
    "3293409": "We're working on a data patch now. Please check back after that's live later this week.",
    "3293523": "Thank you for your diligent review of the dataset, this is great! Going through this:\n\n0. missing frames\nNot in your forum post, but noticed it in your notebook. Sometimes there are frames where no body parts are detected, my suspicion is these are what you're finding as missing. In AdaptableSnail this happens because there are some covered spots where mice can hide from view. In some other datasets, eg JovialSwallow, there's so much motion blur during attack that the mice aren't really discernible to the tracker. I think you are right that the overall amount of this is fairly small.\n\n1. fps x duration vs tracked span\nSmall differences are possible due to dropped frames (eg the video reports being 30 fps, but sometimes the camera fails to save a frame so the actual frame count is shorter.) The larger discrepancies you report are more unusual and I will check them- we used cv2.VideoCapture to get video frame counts, and this is known to sometimes return incorrect values. My instinct would be to trust the fps over the video duration when there is disagreement.\n\n2. Coordinates outside video bounds\nThis is a known property of DeepLabCut (https://forum.image.sc/t/negative-y-coordinates/42499 ): the model is extrapolating to predict locations of body parts that are outside the video bounds, based on what parts are visible.\n\n3. Arena larger than video\nThis is now resolved, and the new dataset should post tomorrow.\n\n4. Very short actions\nSome annotations we received really were quite short, but they are real. We removed single-frame annotations from the Adjective-Animal labs (both training and test sets), but left the single-frame labels in CalMS21, CRIM13, and MABe22 since these are previously published datasets and we didn't want to create version discrepancies.",
    "3294216": "Thank you very much for the detailed explanation! I really appreciate it!",
    "3294448": "Many videos are sparse labeled. I would like to ask all frame actions should be theoretically provided / in every frame some action is happening if mice are visible? Why the first mouse (id=1) has the most actions in many cases?",
    "3295078": "Indeed, labels are sparse. The behaviors listed in `behaviors_labeled` in the metadata are the ones that labs annotated for in the video- it is quite likely that mice are doing other stuff on the frames without labels, they just might not be things that labs cared about in their experiments. (For example a lab studying mouse social behavior might not bother to annotate frames when a mouse is grooming.)\n\nIn some experiment types, eg 'resident-intruder',  only the behavior of one mouse is being studied; mouse 2 (the 'intruder') is effectively a stimulus used to see how mouse 1 behaves. In these cases, often only mouse 1's actions will be annotated.\n\nThe general conclusion- this is a collection of scientific datasets where annotations are precise but not exhaustive, and reflect the set of behaviors that researchers often want to quantify in their experiments.",
    "3295464": "AFTER THE DATA UPDATE PATCH:\n\nthere are still some strange inconsistencies:\n\n- If you compare the number of frames derived from train.csv (fps × duration) with the **span** observed in tracking files (last frame minus first frame + 1, **which is invariant to missing frames somewhere in between**), you can see that there are large negative mismatches (may be due to clipping - tracking starts late or ends early?), as well as more strange positive mismatches (more frames in tracking files than expected from train.csv fps × duration). For example:\n\n- video_id 1377260232 (CalMS21_task1), frames span is 0 to 7023 = 7024, fps = 30, video_duration_sec = 2184.50 which gives 65535 expected frames with tracking., i.e. 58511 frames have no tracking or either of the duration or fps in train.csv is incorrrect\n\n- video_id 1319309143 (CRIM13), frames span is 4092 to 9547 = 5456 frames, fps = 25, video_duration_sec = 64.84 which gives 1621 expected frames with tracking., i.e. 3835 frames are **extras** indicating that either of the duration or fps in train.csv is incorrrect\n\n- video_id 991578442 (GroovyShrew), frames span is 1311 to 43258 = 41948, fps = 30, video_duration_sec = 1435.4 which gives 43062 expected frames with tracking., i.e. 1114 frames have no tracking. \n\nmore examples are in the notebook here: https://www.kaggle.com/code/antoninadolgorukova/mabe-sanity-checks-data-issues?scriptVersionId=264558125\n\n>CautiousGiraffe: corrected arena size metadata from 50x50cm to 30x30cm (impacts both training + test data.)\n\nI see the changes, but still, with pix_per_cm_approx = 21, it's 630 x 630 pix, while video is 600 x 600 pix. Also, e.g.:\n- SparklingTapir video 1237008561 have pix_per_cm_approx = 65, arena 19 x 28.5 cm, which gives 1235 pix height, while video has 1200 pix height\n\nThere are still 4 labs with such, albeit small, differences. I guess, these are just rounding issues?\n\nAdditionally, I recently have found occasional counterintuitive reciprocal actions in annotations, like these:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F028cc1b2fa775bee9abefcedf72dff3f%2FScreenshot%202025-09-28%20232005.png?generation=1759090816044573&alt=media)\n\nIt's difficult to imagine a mouse approaching while the target mouse sniffs it 😅...\nThese are rare and most likely reflect the normal limits of manual annotation; not a major issue but worth being aware of.",
    "3296119": "Is it expected to have tracked body parts inconsistent between mice in a single lab?\nE.g. AdaptableSnail has no ear_left data only for mouse2 but 100% coverage for other mice.",
    "3297825": "Sorry for raising yet another data quality question - maybe this is just my misunderstanding, but I’d like to clarify how the arena metadata should be interpreted @annkennedy05 \n\nSparklingTapir is the most inconsistent case:\n- There is an arena CSDS with sizes 12.7 × 46.0 cm and another one 45.7 × 12.0 cm - swapped dimensions?\n- the are videos with the same arena size and resolution but very different pixels/cm values \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2Fbc55b1fef1190bc0889c42bf3915ac63%2FScreenshot%202025-10-04%20003110.png?generation=1759527104262770&alt=media)\n\nElegantMink: At the same resolution (720×480) pix_per_cm_approx ranges ~17–20, two arena sizes are reported (33×16 cm and 33×22 cm), and one video is 640×480 but still arena is 33×22 cm with pix_per_cm_approx ≈18.2 matching entries with same arena size and video 720×480. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F09ed8b274b8f9ce0e7e5f669d6fa786d%2FScreenshot%202025-10-04%20003319.png?generation=1759527213464690&alt=media)\n\nTranquilPanther: For the same 640×480 resolution and arena size 35x20, pix_per_cm_approx ranges widely (9.0–13.7)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12193662%2F2d6876656425c15e8b1ce8ceb2ac2e14%2FScreenshot%202025-10-04%20003430.png?generation=1759527287046774&alt=media)\n\nPleasantMeerkat: pix_per_cm_approx here = 15.8, apparently looks like mean/rounded value applied across all videos 2000 x 2022 to 2048 x 2036\n\nAll other labs use a single arena type and size/pix_per_cm_approx/video size across their videos, or the differences are small",
    "3297845": "Poor mouse 😭",
    "3307411": "annkennedy05 could you please clarify on the questions above about arena metadata at least? 🙏",
    "3309028": "> @annkennedy05 could you please clarify on the questions above about arena metadata at least? 🙏\n\nThanks for bringing up this question as well for the deep dive into the data — I’d like to take the opportunity to ask the organizer how pix_per_cm_approx is estimated as well. @annkennedy05"
  },
  "source": "meta"
}