Training and k-fold evaluation are nominal:
\nBest fold: , Highest H‑F1: \navg H-F1: \nBest fold: Fold trails baseline HI score by \nAverage score beats baseline AVG score by \nHowever, at inference time, i always get **near-uniform probabilities ** at inference.
\nThey look like this:\n ensembleProb:[ \n \n ]\nSome details
\nI am currently saving model weights with .state_dict() and loading weights with model.load_state_dict() and .eval() before inference.
", "messages": [ { "id": 3251658, "postDate": "2025-07-21T10:25:35.757Z", "content": "Training and k-fold evaluation are nominal:
\nBest fold: , Highest H‑F1: \navg H-F1: \nBest fold: Fold trails baseline HI score by \nAverage score beats baseline AVG score by \nHowever, at inference time, i always get **near-uniform probabilities ** at inference.
\nThey look like this:\n ensembleProb:[ \n \n ]\nSome details
\nI am currently saving model weights with .state_dict() and loading weights with model.load_state_dict() and .eval() before inference.
", "rawMarkdown": "# link to notebook in question. [https://www.kaggle.com/code/hsiaosuan/xiaosuancmiupd/edit](url)\n# Kudos to [https://www.kaggle.com/code/hideyukizushi/cmi25-imu-thm-tof-tf-blendingmodel-lb-82](url) for providing the baseline Keras model and to [https://www.kaggle.com/code/wasupandceacar/lb-0-82-5fold-single-bert-model/comments](url) for pointing out a potential solution\n\n\nTraining and k-fold evaluation are nominal:\n```python\nBest fold: 4, Highest H‑F1: 0.8638\navg H-F1: 0.8537\nBest fold: Fold 4 trails baseline HI score by 0.0032 \nAverage score beats baseline AVG score by 0.0043\n```\n\nHowever, at inference time, i always get **near-uniform probabilities ** at inference. \n```python\nThey look like this:\n ensembleProb:[0.05988789 0.05126978 0.05118179 0.05586503 0.07734321 0.05117348\n0.05118678 0.06303753 0.05117462 0.05120493 0.07548559 0.0511738\n0.05117623 0.05118182 0.05328446 0.05118641 0.05200555 0.05118113]\n```\n## Key question: why does this happen only in torch, not in keras? Is this caused by masking, attention, or maybe the training process?\nSome details\n1. I have implemented masking for TOF and THM features durng training, inspired by forum dicussions, to improve generalization on samples that lack TOF and/or THM data\n2. My model ensemble consists of a series of two-branch models, with separate inputs and attention mechanism for IMU (gyro + acceleration) and THM+TOF.\n3. Inference is always done on full samples. No masking.\n4. Even when single out one model from the ensemble, the output is still uniform\n5. Keras-based model, including the baseline model I imported from user **hideyukizushi**, do not seem to face this problem. While another Torch user encounters a similar problem to mine. I wonder why Torch tends to do this. \n\nI am currently saving model weights with .state_dict() and loading weights with model.load_state_dict() and .eval() before inference.\n\n", "votes": 2 } ], "comments": [], "raw_markdown_by_id": { "3251658": "# link to notebook in question. [https://www.kaggle.com/code/hsiaosuan/xiaosuancmiupd/edit](url)\n# Kudos to [https://www.kaggle.com/code/hideyukizushi/cmi25-imu-thm-tof-tf-blendingmodel-lb-82](url) for providing the baseline Keras model and to [https://www.kaggle.com/code/wasupandceacar/lb-0-82-5fold-single-bert-model/comments](url) for pointing out a potential solution\n\n\nTraining and k-fold evaluation are nominal:\n```python\nBest fold: 4, Highest H‑F1: 0.8638\navg H-F1: 0.8537\nBest fold: Fold 4 trails baseline HI score by 0.0032 \nAverage score beats baseline AVG score by 0.0043\n```\n\nHowever, at inference time, i always get **near-uniform probabilities ** at inference. \n```python\nThey look like this:\n ensembleProb:[0.05988789 0.05126978 0.05118179 0.05586503 0.07734321 0.05117348\n0.05118678 0.06303753 0.05117462 0.05120493 0.07548559 0.0511738\n0.05117623 0.05118182 0.05328446 0.05118641 0.05200555 0.05118113]\n```\n## Key question: why does this happen only in torch, not in keras? Is this caused by masking, attention, or maybe the training process?\nSome details\n1. I have implemented masking for TOF and THM features durng training, inspired by forum dicussions, to improve generalization on samples that lack TOF and/or THM data\n2. My model ensemble consists of a series of two-branch models, with separate inputs and attention mechanism for IMU (gyro + acceleration) and THM+TOF.\n3. Inference is always done on full samples. No masking.\n4. Even when single out one model from the ensemble, the output is still uniform\n5. Keras-based model, including the baseline model I imported from user **hideyukizushi**, do not seem to face this problem. While another Torch user encounters a similar problem to mine. I wonder why Torch tends to do this. \n\nI am currently saving model weights with .state_dict() and loading weights with model.load_state_dict() and .eval() before inference.\n\n" } }