![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16333308%2F8feba5c0e52f9ca25e37e6c040322c58%2FPicture1.png?generation=1759432737927171&alt=media\")
Hi,
\nThanks for organizing this cool competition!
\nGiven that we don't have access to the videos, except for the published dataset CalMS21, I found it quite difficult to assess where on the rodent's body right_lateral (or left) is located. Is it where I drew the arrow on the attached picture?
And was left_lateral defined uniformly across labs? I noticed that in most datasets, there was only either the right_hip or right_lateral keypoint. In only 1 dataset, there were both. This made me wonder whether some labs used the right_lateral label to describe the right hip location (i.e. where the bulge is)? If that was the case, then it would make sense to fuse the right_lateral and right_hip categories into a single one. This would be convient as one could train a larger model with less missing feature data.
All the best
\nAkseli
Hi,
\nThanks for organizing this cool competition!
\nGiven that we don't have access to the videos, except for the published dataset CalMS21, I found it quite difficult to assess where on the rodent's body right_lateral (or left) is located. Is it where I drew the arrow on the attached picture?
And was left_lateral defined uniformly across labs? I noticed that in most datasets, there was only either the right_hip or right_lateral keypoint. In only 1 dataset, there were both. This made me wonder whether some labs used the right_lateral label to describe the right hip location (i.e. where the bulge is)? If that was the case, then it would make sense to fuse the right_lateral and right_hip categories into a single one. This would be convient as one could train a larger model with less missing feature data.
All the best
\nAkseli
Actually, what would be even more helpful, if we could map a couple of CalMS21 videos to the video_id in this challenge. For example, something like mouse039_task1_annotator1.mp4 in the Task1 of the CalMS21 dataset corresponds to CalMS21_task1/99618595.parquet in the Kaggle challenge.
That right_lateral annotation looks correct to me- it's hard to say whether different labs all annotated it exactly the same, since it's a bit more subjective than things like the location of the nose or the ears.
I'll check with Kaggle on whether they'd mind us releasing the video identifiers for the CalMS21 dataset, and get back to you. We do want to avoid a situation where people use video information (beyond the keypoints) in training their behavior classifiers for this competition, as that tends to turn things into a competition over who has access to the most GPUs.
", "rawMarkdown": "That `right_lateral` annotation looks correct to me- it's hard to say whether different labs all annotated it *exactly* the same, since it's a bit more subjective than things like the location of the nose or the ears.\n\nI'll check with Kaggle on whether they'd mind us releasing the video identifiers for the CalMS21 dataset, and get back to you. We do want to avoid a situation where people use video information (beyond the keypoints) in training their behavior classifiers for this competition, as that tends to turn things into a competition over who has access to the most GPUs.", "votes": null }, { "id": "3297487", "postDate": "10/03/2025 09:42:33", "content": "Thank you! Yes, I understand the GPU concern. For me, a couple of sample videos would be a helpful ground truth to check whether my feature engineering and data pre-processing are working well.
", "rawMarkdown": "Thank you! Yes, I understand the GPU concern. For me, a couple of sample videos would be a helpful ground truth to check whether my feature engineering and data pre-processing are working well.", "votes": null }, { "id": "3298201", "postDate": "10/04/2025 19:41:39", "content": "How does that turn competition to more GPUs wins? So each lab is meant to be trained separately? Can't u just merge datasets anyways?
", "rawMarkdown": "How does that turn competition to more GPUs wins? So each lab is meant to be trained separately? Can't u just merge datasets anyways?", "votes": null }, { "id": "3298270", "postDate": "10/05/2025 03:50:56", "content": "Ah, no we certainly hope you're able to do some merging/learning across labs! The concern is that people might sink a lot of time into trying GPU-intensive video understanding models on CalMS21, rather than working with the pose data provided. We have decent evidence from CalMS21 that you can achieve human-level accuracy in behavior classification (in a single lab setting) just from pose data if you are clever about your feature engineering/training data, so it would be a shame to see that thrown out the window in favor of using expensive video models.
", "rawMarkdown": "Ah, no we certainly hope you're able to do some merging/learning across labs! The concern is that people might sink a lot of time into trying GPU-intensive video understanding models on CalMS21, rather than working with the pose data provided. We have decent evidence from CalMS21 that you can achieve human-level accuracy in behavior classification (in a single lab setting) just from pose data if you are clever about your feature engineering/training data, so it would be a shame to see that thrown out the window in favor of using expensive video models.", "votes": null }, { "id": "3300818", "postDate": "10/11/2025 16:02:24", "content": "Hi @annkennedy05, I wanted to check if releasing the video identifiers for the CalMS21 dataset would be possible? As I mentioned, it would help me with feature engineering. I also made a nice GUI (for another project), where I can see time series of the features with video above. Seeing some hand-crafted features aligned with video snippets would be a very nice ground truth to evaluate whether the feature engineering is going well.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16333308%2Fdb435a3842994cec9c22e2ee3a75911f%2FExample.png?generation=1760200434492874&alt=media\")
For example, if I had a video, I could play a snippet of e.g. an attack behaviour, and explore how different features look like in those frames.
", "rawMarkdown": "Hi @annkennedy05, I wanted to check if releasing the video identifiers for the CalMS21 dataset would be possible? As I mentioned, it would help me with feature engineering. I also made a nice GUI (for another project), where I can see time series of the features with video above. Seeing some hand-crafted features aligned with video snippets would be a very nice ground truth to evaluate whether the feature engineering is going well.\n\n\n\nFor example, if I had a video, I could play a snippet of e.g. an attack behaviour, and explore how different features look like in those frames.", "votes": null } ], "comments": [ { "id": 3297312, "author_name": "akseliilmanen", "author_url": "", "post_date": "10/02/2025 20:36:52", "content": "Actually, what would be even more helpful, if we could map a couple of CalMS21 videos to the video_id in this challenge. For example, something like mouse039_task1_annotator1.mp4 in the Task1 of the CalMS21 dataset corresponds to CalMS21_task1/99618595.parquet in the Kaggle challenge.
That right_lateral annotation looks correct to me- it's hard to say whether different labs all annotated it exactly the same, since it's a bit more subjective than things like the location of the nose or the ears.
I'll check with Kaggle on whether they'd mind us releasing the video identifiers for the CalMS21 dataset, and get back to you. We do want to avoid a situation where people use video information (beyond the keypoints) in training their behavior classifiers for this competition, as that tends to turn things into a competition over who has access to the most GPUs.
", "votes": null, "replies": [ { "id": 3297487, "author_name": "akseliilmanen", "author_url": "", "post_date": "10/03/2025 09:42:33", "content": "Thank you! Yes, I understand the GPU concern. For me, a couple of sample videos would be a helpful ground truth to check whether my feature engineering and data pre-processing are working well.
", "votes": null, "replies": [] }, { "id": 3298201, "author_name": "luciankucera", "author_url": "", "post_date": "10/04/2025 19:41:39", "content": "How does that turn competition to more GPUs wins? So each lab is meant to be trained separately? Can't u just merge datasets anyways?
", "votes": null, "replies": [ { "id": 3298270, "author_name": "annkennedy05", "author_url": "", "post_date": "10/05/2025 03:50:56", "content": "Ah, no we certainly hope you're able to do some merging/learning across labs! The concern is that people might sink a lot of time into trying GPU-intensive video understanding models on CalMS21, rather than working with the pose data provided. We have decent evidence from CalMS21 that you can achieve human-level accuracy in behavior classification (in a single lab setting) just from pose data if you are clever about your feature engineering/training data, so it would be a shame to see that thrown out the window in favor of using expensive video models.
", "votes": null, "replies": [] } ] }, { "id": 3300818, "author_name": "akseliilmanen", "author_url": "", "post_date": "10/11/2025 16:02:24", "content": "Hi @annkennedy05, I wanted to check if releasing the video identifiers for the CalMS21 dataset would be possible? As I mentioned, it would help me with feature engineering. I also made a nice GUI (for another project), where I can see time series of the features with video above. Seeing some hand-crafted features aligned with video snippets would be a very nice ground truth to evaluate whether the feature engineering is going well.
\n
For example, if I had a video, I could play a snippet of e.g. an attack behaviour, and explore how different features look like in those frames.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3297285": "Hi,\n\nThanks for organizing this cool competition!\n\nGiven that we don't have access to the videos, except for the published dataset CalMS21, I found it quite difficult to assess where on the rodent's body `right_lateral` (or left) is located. Is it where I drew the arrow on the attached picture? \n\nAnd was left_lateral defined uniformly across labs? I noticed that in most datasets, there was only either the `right_hip` or `right_lateral` keypoint. In only 1 dataset, there were both. This made me wonder whether some labs used the `right_lateral` label to describe the right hip location (i.e. where the bulge is)? If that was the case, then it would make sense to fuse the right_lateral and right_hip categories into a single one. This would be convient as one could train a larger model with less missing feature data.\n\nAll the best \nAkseli\n\n", "3297312": "Actually, what would be even more helpful, if we could map a couple of CalMS21 videos to the `video_id` in this challenge. For example, something like `mouse039_task1_annotator1.mp4` in the Task1 of the CalMS21 dataset corresponds to `CalMS21_task1/99618595.parquet` in the Kaggle challenge.", "3297324": "That `right_lateral` annotation looks correct to me- it's hard to say whether different labs all annotated it *exactly* the same, since it's a bit more subjective than things like the location of the nose or the ears.\n\nI'll check with Kaggle on whether they'd mind us releasing the video identifiers for the CalMS21 dataset, and get back to you. We do want to avoid a situation where people use video information (beyond the keypoints) in training their behavior classifiers for this competition, as that tends to turn things into a competition over who has access to the most GPUs.", "3297487": "Thank you! Yes, I understand the GPU concern. For me, a couple of sample videos would be a helpful ground truth to check whether my feature engineering and data pre-processing are working well.", "3298201": "How does that turn competition to more GPUs wins? So each lab is meant to be trained separately? Can't u just merge datasets anyways?", "3298270": "Ah, no we certainly hope you're able to do some merging/learning across labs! The concern is that people might sink a lot of time into trying GPU-intensive video understanding models on CalMS21, rather than working with the pose data provided. We have decent evidence from CalMS21 that you can achieve human-level accuracy in behavior classification (in a single lab setting) just from pose data if you are clever about your feature engineering/training data, so it would be a shame to see that thrown out the window in favor of using expensive video models.", "3300818": "Hi @annkennedy05, I wanted to check if releasing the video identifiers for the CalMS21 dataset would be possible? As I mentioned, it would help me with feature engineering. I also made a nice GUI (for another project), where I can see time series of the features with video above. Seeing some hand-crafted features aligned with video snippets would be a very nice ground truth to evaluate whether the feature engineering is going well.\n\n\n\nFor example, if I had a video, I could play a snippet of e.g. an attack behaviour, and explore how different features look like in those frames." }, "source": "meta" }