I understand that the goal of this competition is to develop a model that generalizes across different labs and their respective target behaviors.
\nHowever, according to the following discussion:
https://www.kaggle.com/competitions/MABe-mouse-behavior-detection/discussion/609904
\nthe competition organizers mentioned that all behavior labels present in the test set are already included in the training set. In addition, the test.csv file provides both the lab IDs and the corresponding target behaviors, which are neither new nor unseen.MAYBE the MABe_22 datasets could serve to test generalizability, since they contain behaviors that are not labeled in the training set.
\nBut though, the competition’s objective seems somewhat ambiguous, since one could simply train a separate model for each lab without needing to consider generalization to entirely new labs — a scenario that would better reflect real-world challenges.
I want some description about this topic from competition hosts, thanks @annkennedy05
", "messages": [ { "id": "3304313", "postDate": "10/20/2025 09:35:00", "content": "I understand that the goal of this competition is to develop a model that generalizes across different labs and their respective target behaviors.
\nHowever, according to the following discussion:
https://www.kaggle.com/competitions/MABe-mouse-behavior-detection/discussion/609904
\nthe competition organizers mentioned that all behavior labels present in the test set are already included in the training set. In addition, the test.csv file provides both the lab IDs and the corresponding target behaviors, which are neither new nor unseen.MAYBE the MABe_22 datasets could serve to test generalizability, since they contain behaviors that are not labeled in the training set.
\nBut though, the competition’s objective seems somewhat ambiguous, since one could simply train a separate model for each lab without needing to consider generalization to entirely new labs — a scenario that would better reflect real-world challenges.
I want some description about this topic from competition hosts, thanks @annkennedy05
", "rawMarkdown": "I understand that the goal of this competition is to develop a model that generalizes across different labs and their respective target behaviors.\nHowever, according to the following discussion:\n\n[https://www.kaggle.com/competitions/MABe-mouse-behavior-detection/discussion/609904](url)\n\nthe competition organizers mentioned that all behavior labels present in the test set are already included in the training set. In addition, the test.csv file provides both the lab IDs and the corresponding target behaviors, which are neither new nor unseen.MAYBE the MABe_22 datasets could serve to test generalizability, since they contain behaviors that are not labeled in the training set.\nBut though, the competition’s objective seems somewhat ambiguous, since one could simply train a separate model for each lab without needing to consider generalization to entirely new labs — a scenario that would better reflect real-world challenges. \n\nI want some description about this topic from competition hosts, thanks @annkennedy05", "votes": null }, { "id": "3304366", "postDate": "10/20/2025 12:09:57", "content": "I second this
", "rawMarkdown": "I second this", "votes": null }, { "id": "3304393", "postDate": "10/20/2025 13:04:46", "content": "The dataset provided in the previous competition by the hosts was intended to explore the potential for transfer learning. In this competition, the amount of training data available for each lab is quite limited, so we cannot expect a significant improvement in model performance through simple training alone. In fact, the current leaderboard macro F1 scores have not exceeded 0.5, which is too low for practical real-world applications. I believe the hosts are expecting models and approaches that can achieve scores above 0.7. On the other hand, if such high performance can be achieved through lab-specific training, that would indeed be remarkable.
", "rawMarkdown": "The dataset provided in the previous competition by the hosts was intended to explore the potential for transfer learning. In this competition, the amount of training data available for each lab is quite limited, so we cannot expect a significant improvement in model performance through simple training alone. In fact, the current leaderboard macro F1 scores have not exceeded 0.5, which is too low for practical real-world applications. I believe the hosts are expecting models and approaches that can achieve scores above 0.7. On the other hand, if such high performance can be achieved through lab-specific training, that would indeed be remarkable.", "votes": null }, { "id": "3304397", "postDate": "10/20/2025 13:15:38", "content": "yes. Impressive, but contrary to the purpose of the competition
", "rawMarkdown": "yes. Impressive, but contrary to the purpose of the competition", "votes": null }, { "id": "3304406", "postDate": "10/20/2025 13:29:45", "content": "Even if the training is lab-specific, if there exists a generalizable method that can build a high-performing model from a small amount of data, I believe the goal has already been achieved. Conversely, if transfer learning approaches do not outperform lab-specific ones, that in itself is a valuable insight.
", "rawMarkdown": "Even if the training is lab-specific, if there exists a generalizable method that can build a high-performing model from a small amount of data, I believe the goal has already been achieved. Conversely, if transfer learning approaches do not outperform lab-specific ones, that in itself is a valuable insight.", "votes": null }, { "id": "3304458", "postDate": "10/20/2025 15:24:38", "content": "@kyoumonemui has the gist of it. While every test set behavior appears in the training set, the number of training examples for some behaviors is quite limited- in our experience, the provided data is not sufficient to achieve satisfactory classifier performance if one simply trains a separate model from scratch for each lab. For example, in our hands an XGBoost-based classifier needs to see about 50 bouts of a behavior before its performance begins to plateau. Teaching another human to recognize a behavior, in contrast, takes only a handful of examples.
\nThe hope is that there are tricks to make this limited training data stretch further. A big part of this is learning the \"right\" representation of mouse pose dynamics. Fields like meta-learning have shown that it is possible to learn representations of a task that are better-suited to efficient learning of downstream tasks. Can this conclusion be applied to mouse behavior data? If we learn a good underlying model of how mice move and interact, can we learn to reproduce a given lab's annotation style for a target behavior given only a small number of labeled examples of interest?
", "rawMarkdown": "kyoumonemui has the gist of it. While every test set behavior appears in the training set, the number of training examples for some behaviors is quite limited- in our experience, the provided data is not sufficient to achieve satisfactory classifier performance if one simply trains a separate model from scratch for each lab. For example, in our hands an XGBoost-based classifier needs to see about 50 bouts of a behavior before its performance begins to plateau. Teaching another human to recognize a behavior, in contrast, takes only a handful of examples.\n\nThe hope is that there are tricks to make this limited training data stretch further. A big part of this is learning the \"right\" representation of mouse pose dynamics. Fields like meta-learning have shown that it is possible to learn representations of a task that are better-suited to efficient learning of downstream tasks. Can this conclusion be applied to mouse behavior data? If we learn a good underlying model of how mice move and interact, can we learn to reproduce a given lab's annotation style for a target behavior given only a small number of labeled examples of interest?", "votes": null }, { "id": "3304533", "postDate": "10/20/2025 18:26:54", "content": "Totally agree, The real challenge isn’t just high scores but building data-efficient, generalizable models that work across labs.
", "rawMarkdown": "Totally agree, The real challenge isn’t just high scores but building data-efficient, generalizable models that work across labs.", "votes": null } ], "comments": [ { "id": 3304366, "author_name": "s617603", "author_url": "", "post_date": "10/20/2025 12:09:57", "content": "I second this
", "votes": null, "replies": [] }, { "id": 3304393, "author_name": "kyoumonemui", "author_url": "", "post_date": "10/20/2025 13:04:46", "content": "The dataset provided in the previous competition by the hosts was intended to explore the potential for transfer learning. In this competition, the amount of training data available for each lab is quite limited, so we cannot expect a significant improvement in model performance through simple training alone. In fact, the current leaderboard macro F1 scores have not exceeded 0.5, which is too low for practical real-world applications. I believe the hosts are expecting models and approaches that can achieve scores above 0.7. On the other hand, if such high performance can be achieved through lab-specific training, that would indeed be remarkable.
", "votes": null, "replies": [ { "id": 3304397, "author_name": "doheon114", "author_url": "", "post_date": "10/20/2025 13:15:38", "content": "yes. Impressive, but contrary to the purpose of the competition
", "votes": null, "replies": [ { "id": 3304406, "author_name": "kyoumonemui", "author_url": "", "post_date": "10/20/2025 13:29:45", "content": "Even if the training is lab-specific, if there exists a generalizable method that can build a high-performing model from a small amount of data, I believe the goal has already been achieved. Conversely, if transfer learning approaches do not outperform lab-specific ones, that in itself is a valuable insight.
", "votes": null, "replies": [] } ] } ] }, { "id": 3304458, "author_name": "annkennedy05", "author_url": "", "post_date": "10/20/2025 15:24:38", "content": "@kyoumonemui has the gist of it. While every test set behavior appears in the training set, the number of training examples for some behaviors is quite limited- in our experience, the provided data is not sufficient to achieve satisfactory classifier performance if one simply trains a separate model from scratch for each lab. For example, in our hands an XGBoost-based classifier needs to see about 50 bouts of a behavior before its performance begins to plateau. Teaching another human to recognize a behavior, in contrast, takes only a handful of examples.
\nThe hope is that there are tricks to make this limited training data stretch further. A big part of this is learning the \"right\" representation of mouse pose dynamics. Fields like meta-learning have shown that it is possible to learn representations of a task that are better-suited to efficient learning of downstream tasks. Can this conclusion be applied to mouse behavior data? If we learn a good underlying model of how mice move and interact, can we learn to reproduce a given lab's annotation style for a target behavior given only a small number of labeled examples of interest?
", "votes": null, "replies": [] }, { "id": 3304533, "author_name": "dimpimittal", "author_url": "", "post_date": "10/20/2025 18:26:54", "content": "Totally agree, The real challenge isn’t just high scores but building data-efficient, generalizable models that work across labs.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3304313": "I understand that the goal of this competition is to develop a model that generalizes across different labs and their respective target behaviors.\nHowever, according to the following discussion:\n\n[https://www.kaggle.com/competitions/MABe-mouse-behavior-detection/discussion/609904](url)\n\nthe competition organizers mentioned that all behavior labels present in the test set are already included in the training set. In addition, the test.csv file provides both the lab IDs and the corresponding target behaviors, which are neither new nor unseen.MAYBE the MABe_22 datasets could serve to test generalizability, since they contain behaviors that are not labeled in the training set.\nBut though, the competition’s objective seems somewhat ambiguous, since one could simply train a separate model for each lab without needing to consider generalization to entirely new labs — a scenario that would better reflect real-world challenges. \n\nI want some description about this topic from competition hosts, thanks @annkennedy05", "3304366": "I second this", "3304393": "The dataset provided in the previous competition by the hosts was intended to explore the potential for transfer learning. In this competition, the amount of training data available for each lab is quite limited, so we cannot expect a significant improvement in model performance through simple training alone. In fact, the current leaderboard macro F1 scores have not exceeded 0.5, which is too low for practical real-world applications. I believe the hosts are expecting models and approaches that can achieve scores above 0.7. On the other hand, if such high performance can be achieved through lab-specific training, that would indeed be remarkable.", "3304397": "yes. Impressive, but contrary to the purpose of the competition", "3304406": "Even if the training is lab-specific, if there exists a generalizable method that can build a high-performing model from a small amount of data, I believe the goal has already been achieved. Conversely, if transfer learning approaches do not outperform lab-specific ones, that in itself is a valuable insight.", "3304458": "kyoumonemui has the gist of it. While every test set behavior appears in the training set, the number of training examples for some behaviors is quite limited- in our experience, the provided data is not sufficient to achieve satisfactory classifier performance if one simply trains a separate model from scratch for each lab. For example, in our hands an XGBoost-based classifier needs to see about 50 bouts of a behavior before its performance begins to plateau. Teaching another human to recognize a behavior, in contrast, takes only a handful of examples.\n\nThe hope is that there are tricks to make this limited training data stretch further. A big part of this is learning the \"right\" representation of mouse pose dynamics. Fields like meta-learning have shown that it is possible to learn representations of a task that are better-suited to efficient learning of downstream tasks. Can this conclusion be applied to mouse behavior data? If we learn a good underlying model of how mice move and interact, can we learn to reproduce a given lab's annotation style for a target behavior given only a small number of labeled examples of interest?", "3304533": "Totally agree, The real challenge isn’t just high scores but building data-efficient, generalizable models that work across labs." }, "source": "meta" }