In this topic, I want to discuss the best approach for selecting:
\nSo far, I have experimented with several methods:
\nPredict probabilities for 18 classes using CrossEntropyLoss.
Collapse all 10 non-target classes into a single class and train the model to predict 9 classes using standard CrossEntropyLoss.
Maximize the competition metric ((binary-f1-score + 9-class-f1-score)/2) by training the model to predict 9/18 classes with weighted CrossEntropyLoss.
\nHierarchical modeling:
\nMulti-Task Learning (MTL):
\nHybrid approach:
\n18-class probability prediction:
\nMulti Task Learning with auxiliary tasks like prediction of part of the sequence (prepare/pause/gesture) or predicting the age/gender/height
Results & Open Question
\nDespite testing various non-standard approaches, none have improved my CV or LB scores so far in compare with the simplest algorithm, presented in (1)
\nWhat do you think is the best training/inference strategy for this competition?
", "messages": [ { "id": "3244843", "postDate": "07/08/2025 16:18:33", "content": "In this topic, I want to discuss the best approach for selecting:
\nSo far, I have experimented with several methods:
\nPredict probabilities for 18 classes using CrossEntropyLoss.
Collapse all 10 non-target classes into a single class and train the model to predict 9 classes using standard CrossEntropyLoss.
Maximize the competition metric ((binary-f1-score + 9-class-f1-score)/2) by training the model to predict 9/18 classes with weighted CrossEntropyLoss.
\nHierarchical modeling:
\nMulti-Task Learning (MTL):
\nHybrid approach:
\n18-class probability prediction:
\nMulti Task Learning with auxiliary tasks like prediction of part of the sequence (prepare/pause/gesture) or predicting the age/gender/height
Results & Open Question
\nDespite testing various non-standard approaches, none have improved my CV or LB scores so far in compare with the simplest algorithm, presented in (1)
\nWhat do you think is the best training/inference strategy for this competition?
", "rawMarkdown": "In this topic, I want to discuss the best approach for selecting:\n\n1. The optimal loss function\n2. The best post-processing algorithm\n\nSo far, I have experimented with several **methods**:\n\n1. Predict probabilities for 18 classes using CrossEntropyLoss.\n2. Collapse all 10 non-target classes into a single class and train the model to predict 9 classes using standard CrossEntropyLoss.\n\n3. Maximize the competition metric ((binary-f1-score + 9-class-f1-score)/2) by training the model to predict 9/18 classes with weighted CrossEntropyLoss.\n - Brute-forced the weight for non-target classes while keeping target class weights equal (e.g., non-target-weight=0.5, each target class weight = 0.5/8).\n\n4. Hierarchical modeling:\n - Use two separate models—one for binary classification (target vs. non-target) and another for target-gesture classification.\n - During inference, first classify if the gesture is a target; if yes, use the second model to predict the exact gesture.\n5. Multi-Task Learning (MTL):\n - Build a model with two heads—one for 9/18-class classification and another for binary classification.\n - Drop the binary head during inference.\n - Train using a linear combination of the two losses (tuning coefficients).\n6. Hybrid approach:\n - Use the two-head model but feed features from the second head into the first head as input.\n - Training and inference follow the same logic as in (5).\n7. 18-class probability prediction:\n - Derive target/non-target probability via logsumexp of logits.\n - Use a linear combination of losses; inference matches method (1).\n8. Multi Task Learning with auxiliary tasks like prediction of part of the sequence (prepare/pause/gesture) or predicting the age/gender/height\n\n**Results & Open Question**\n\nDespite testing various non-standard approaches, none have improved my CV or LB scores so far in compare with the simplest algorithm, presented in (1)\n\nWhat do you think is the best training/inference strategy for this competition?", "votes": null }, { "id": "3245518", "postDate": "07/09/2025 13:42:10", "content": "I've tried probabilistic modeling using pyro for hierarchical label structure, it does not work well.
So far the best loss I found is cross entropy loss lol
", "rawMarkdown": "So far the best loss I found is cross entropy loss lol", "votes": null } ], "comments": [ { "id": 3245518, "author_name": "tom99763", "author_url": "", "post_date": "07/09/2025 13:42:10", "content": "I've tried probabilistic modeling using pyro for hierarchical label structure, it does not work well.
So far the best loss I found is cross entropy loss lol
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3244843": "In this topic, I want to discuss the best approach for selecting:\n\n1. The optimal loss function\n2. The best post-processing algorithm\n\nSo far, I have experimented with several **methods**:\n\n1. Predict probabilities for 18 classes using CrossEntropyLoss.\n2. Collapse all 10 non-target classes into a single class and train the model to predict 9 classes using standard CrossEntropyLoss.\n\n3. Maximize the competition metric ((binary-f1-score + 9-class-f1-score)/2) by training the model to predict 9/18 classes with weighted CrossEntropyLoss.\n - Brute-forced the weight for non-target classes while keeping target class weights equal (e.g., non-target-weight=0.5, each target class weight = 0.5/8).\n\n4. Hierarchical modeling:\n - Use two separate models—one for binary classification (target vs. non-target) and another for target-gesture classification.\n - During inference, first classify if the gesture is a target; if yes, use the second model to predict the exact gesture.\n5. Multi-Task Learning (MTL):\n - Build a model with two heads—one for 9/18-class classification and another for binary classification.\n - Drop the binary head during inference.\n - Train using a linear combination of the two losses (tuning coefficients).\n6. Hybrid approach:\n - Use the two-head model but feed features from the second head into the first head as input.\n - Training and inference follow the same logic as in (5).\n7. 18-class probability prediction:\n - Derive target/non-target probability via logsumexp of logits.\n - Use a linear combination of losses; inference matches method (1).\n8. Multi Task Learning with auxiliary tasks like prediction of part of the sequence (prepare/pause/gesture) or predicting the age/gender/height\n\n**Results & Open Question**\n\nDespite testing various non-standard approaches, none have improved my CV or LB scores so far in compare with the simplest algorithm, presented in (1)\n\nWhat do you think is the best training/inference strategy for this competition?", "3245518": "I've tried probabilistic modeling using `pyro` for hierarchical label structure, it does not work well.", "3245549": "So far the best loss I found is cross entropy loss lol" }, "source": "meta" }