{ "id": 612699, "title": "lab_id in evaluation?", "url": "/competitions/MABe-mouse-behavior-detection/discussion/612699", "author_name": "", "post_date": "2025-10-21T12:27:02.061852700Z", "votes": 2, "comment_count": 2, "views": 0, "content": "

I'm confused about whether the scoring depends on lab_id. https://www.kaggle.com/competitions/MABe-mouse-behavior-detection says the scoring is done by https://www.kaggle.com/code/metric/mabe-f-beta
\nbut when I look in the code there; I believe the calculation uses lab_id; and it gets the lab_id from the input. But the sample submissions don't have lab_id.
\nCan someone explain? Thanks for help!

", "messages": [ { "id": "3304867", "postDate": "10/21/2025 12:27:02", "content": "

I'm confused about whether the scoring depends on lab_id. https://www.kaggle.com/competitions/MABe-mouse-behavior-detection says the scoring is done by https://www.kaggle.com/code/metric/mabe-f-beta
\nbut when I look in the code there; I believe the calculation uses lab_id; and it gets the lab_id from the input. But the sample submissions don't have lab_id.
\nCan someone explain? Thanks for help!

", "rawMarkdown": "I'm confused about whether the scoring depends on lab_id. https://www.kaggle.com/competitions/MABe-mouse-behavior-detection says the scoring is done by https://www.kaggle.com/code/metric/mabe-f-beta\nbut when I look in the code there; I believe the calculation uses lab_id; and it gets the lab_id from the input. But the sample submissions don't have lab_id.\nCan someone explain? Thanks for help!", "votes": null }, { "id": "3305090", "postDate": "10/22/2025 01:45:27", "content": "

The scoring function takes two data inputs: the submission which it gets from reading your submission file, and the solution which contains the private solution against which the submission is scored. The relevant part of the function is this:

\n
     lab  solution[].unique():\n        lab_solution = solution.(pl.col() == lab).clone()\n        lab_videos = (lab_solution[].unique())\n        lab_submission = submission.(pl.col().is_in(lab_videos)).clone()\n
\n

The submission does not need to include lab_id because this is read from the solution. The rows of the submission corresponding to each lab are determined by matching on video_id based on that lab's videos in the solution.

", "rawMarkdown": "The scoring function takes two data inputs: the `submission` which it gets from reading your submission file, and the `solution` which contains the private solution against which the submission is scored. The relevant part of the function is this:\n\n```python\n for lab in solution['lab_id'].unique():\n lab_solution = solution.filter(pl.col('lab_id') == lab).clone()\n lab_videos = set(lab_solution['video_id'].unique())\n lab_submission = submission.filter(pl.col('video_id').is_in(lab_videos)).clone()\n```\nThe submission does not need to include `lab_id` because this is read from the `solution`. The rows of the submission corresponding to each lab are determined by matching on `video_id` based on that lab's videos in the solution.", "votes": null }, { "id": "3307047", "postDate": "10/26/2025 02:50:45", "content": "

thank you. very helpful.

", "rawMarkdown": "thank you. very helpful.", "votes": null } ], "comments": [ { "id": 3305090, "author_name": "willhirsch", "author_url": "", "post_date": "10/22/2025 01:45:27", "content": "

The scoring function takes two data inputs: the submission which it gets from reading your submission file, and the solution which contains the private solution against which the submission is scored. The relevant part of the function is this:

\n
     lab  solution[].unique():\n        lab_solution = solution.(pl.col() == lab).clone()\n        lab_videos = (lab_solution[].unique())\n        lab_submission = submission.(pl.col().is_in(lab_videos)).clone()\n
\n

The submission does not need to include lab_id because this is read from the solution. The rows of the submission corresponding to each lab are determined by matching on video_id based on that lab's videos in the solution.

", "votes": null, "replies": [ { "id": 3307047, "author_name": "francisganong", "author_url": "", "post_date": "10/26/2025 02:50:45", "content": "

thank you. very helpful.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3304867": "I'm confused about whether the scoring depends on lab_id. https://www.kaggle.com/competitions/MABe-mouse-behavior-detection says the scoring is done by https://www.kaggle.com/code/metric/mabe-f-beta\nbut when I look in the code there; I believe the calculation uses lab_id; and it gets the lab_id from the input. But the sample submissions don't have lab_id.\nCan someone explain? Thanks for help!", "3305090": "The scoring function takes two data inputs: the `submission` which it gets from reading your submission file, and the `solution` which contains the private solution against which the submission is scored. The relevant part of the function is this:\n\n```python\n for lab in solution['lab_id'].unique():\n lab_solution = solution.filter(pl.col('lab_id') == lab).clone()\n lab_videos = set(lab_solution['video_id'].unique())\n lab_submission = submission.filter(pl.col('video_id').is_in(lab_videos)).clone()\n```\nThe submission does not need to include `lab_id` because this is read from the `solution`. The rows of the submission corresponding to each lab are determined by matching on `video_id` based on that lab's videos in the solution.", "3307047": "thank you. very helpful." }, "source": "meta" }