{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.11.13"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":102335,"databundleVersionId":12518947,"sourceType":"competition"},{"sourceId":12139340,"sourceType":"datasetVersion","datasetId":7645099},{"sourceId":12293285,"sourceType":"datasetVersion","datasetId":7748073},{"sourceId":12328761,"sourceType":"datasetVersion","datasetId":7771623},{"sourceId":12411879,"sourceType":"datasetVersion","datasetId":7827797},{"sourceId":12573306,"sourceType":"datasetVersion","datasetId":7932089},{"sourceId":12723935,"sourceType":"datasetVersion","datasetId":7869970},{"sourceId":240649816,"sourceType":"kernelVersion"},{"sourceId":245926974,"sourceType":"kernelVersion"},{"sourceId":246893721,"sourceType":"kernelVersion"},{"sourceId":248244433,"sourceType":"kernelVersion"},{"sourceId":251413288,"sourceType":"kernelVersion"},{"sourceId":470587,"sourceType":"modelInstanceVersion","modelInstanceId":379625,"modelId":398856},{"sourceId":471764,"sourceType":"modelInstanceVersion","modelInstanceId":380358,"modelId":400086}],"dockerImageVersionId":31090,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true},"papermill":{"default_parameters":{},"duration":333.100766,"end_time":"2025-08-22T01:19:49.846142","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2025-08-22T01:14:16.745376","version":"2.6.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🚀Best-Practice Learning for Hierarchical Model Averaging!\n> Author: H-Z-Ning  \n> Date: 2025-08-25  \n> Dataset: [Kaggle Competition Page](https://www.kaggle.com/competitions/cmi-detect-behavior-with-sensor-data)\n\n---\n\n## 📚 Table of Contents\n1. Background\n2. Hierarchical Model Averaging – Optimal Parameter Search Plan\n3. Model Descriptions\n4. Fusion Function Description\n5. Submission\n\n## 🧾 Background\n\nThis notebook builds upon the work in [Enumerate weights for Model.3](https://www.kaggle.com/code/nina2025/cmi-enumerate-weights-for-model-3#Model_1) and continues the Hierarchical Model Averaging pipeline.\n\nIn the referenced notebook, F.A. Nina has already tested the following parameters and achieved excellent results.\n\n| [Model 1](#Model_1) | 0.820                                                        | &nbsp; v.6 &nbsp;  | [CMI25  IMU+THM/TOF TF BlendingModel](https://www.kaggle.com/code/hideyukizushi/cmi25-imu-thm-tof-tf-blendingmodel-lb-82) | &nbsp; grandmaster &nbsp; |      [yukiZ](https://www.kaggle.com/hideyukizushi)      |         Japan |\n| ------------------- | ------------------------------------------------------------ | :----------------: | :----------------------------------------------------------- | :-----------------------: | :-----------------------------------------------------: | ------------: |\n| [Model 2](#Model_2) | 0.829                                                        | &nbsp; v.7 &nbsp;  | [5fold single bert model](https://www.kaggle.com/code/wasupandceacar/lb-0-82-5fold-single-bert-model) |   &nbsp; expert &nbsp;    | [wasupandceacar](https://www.kaggle.com/wasupandceacar) | United States |\n| [Model 3](#Model_3) | 0.835                                                        | &nbsp; v.10 &nbsp; | [Gated GRU + Hybrid Ensemble_v02](https://www.kaggle.com/code/pepushi/gated-gru-hybrid-ensemble-v02) |   &nbsp; expert &nbsp;    |         [uhey](https://www.kaggle.com/pepushi)          |         Japan |\n|                     |                                                              |                    |                                                              |                           |                                                         |               |\n| h-blend             |                                                              |                    | weights.[1,2,3](https://www.kaggle.com/code/nina2025/cmi-ensemble-of-3-solutions-horizontal-blend?scriptVersionId=255523877)&nbsp;=&nbsp; [ 0.271, &nbsp;0.347, &nbsp;**weights**[**.3**](#predict_3) ] |        asc/desc %         |                   correct sub.weights                   |               |\n|                     |                                                              |                    |                                                              |                           |                                                         |               |\n|                     |                                                              |                    |                                                              |                           |                                                         |               |\n|                     | [0.855](https://www.kaggle.com/code/nina2025/cmi-enumerate-weights-for-model-3?scriptVersionId=257776810) |        v.6         | Models.[1,2,3](https://www.kaggle.com/code/nina2025/cmi-ensemble-of-3-solutions-horizontal-blend)&nbsp;=&nbsp; [ 0.2715, &nbsp;0.3465, &nbsp;0.3820 ] <br> **Model**[**.3**](#predict_3) = [[weights](#enumerate) 1, 7, 8, 5] |   [ 70 x 30 ](#predict)   |            [ +0.0021, -0.000526, -0.001574 ]            |               |\n|                     |                                                              |                    |                                                              |                           |                                                         |               |\n|                     | [0.855](https://www.kaggle.com/code/nina2025/cmi-enumerate-weights-for-model-3/notebook?scriptVersionId=257836175) |        v.7         | Models.[1,2,3](https://www.kaggle.com/code/nina2025/cmi-ensemble-of-3-solutions-horizontal-blend)&nbsp;=&nbsp; [ 0.271 - **k**, &nbsp;0.357 + **k**, &nbsp;&nbsp;0.382 ] <br> **Model**[**.3**](#predict_3) = [[weights](#enumerate) **7a**] |   [ 70 x 30 ](#predict)   |             [ +0.0021, -0.00055, -0.00155 ]             |               |\n|                     |                                                              |                    |                                                              |                           |                                                         |               |\n|                     | [0.854](https://www.kaggle.com/code/nina2025/cmi-enumerate-weights-for-model-3?scriptVersionId=257838328) |        v.8         | Models.[1,2,3](https://www.kaggle.com/code/nina2025/cmi-ensemble-of-3-solutions-horizontal-blend)&nbsp;=&nbsp; [ 0.271 + **k**, &nbsp;0.357 - **k**, &nbsp;&nbsp;0.382 ] <br> **Model**[**.3**](#predict_3) = [[weights](#enumerate) **7b**] |   [ 70 x 30 ](#predict)   |             [ +0.0021, -0.00055, -0.00155 ]             |               |\n\n### 🎯  Hierarchical Model Averaging – Optimal Parameter Search Plan\n\n**This notebook will therefore investigate the following aspects of H-Blend on top of the above foundation, aiming to achieve even better results.**\n\n|      score       | version |                            params                            | **weighting** |          **incremental**          |\n| :--------------: | :-----: | :----------------------------------------------------------: | :-----------: | :-------------------------------: |\n|                  |         |                                                              |               |                                   |\n| **0.855** |   v.1   | Models.[1,2,3](https://www.kaggle.com/code/nina2025/cmi-ensemble-of-3-solutions-horizontal-blend)&nbsp;=&nbsp; [ 0.2710, &nbsp;0.3470, &nbsp;0.3820 ] <br> **Model**[**.3**](https://kkb-production.jupyter-proxy.kaggle.net/static/assets/jupyterlab-v4/jupyterlab-index-f1bb73b2c4ad44c1a82e.html?session=eyJhbGciOiJub25lIiwidHlwIjoiSldUIn0.eyJpc3MiOiJrYWdnbGUiLCJhdWQiOiJjbGllbnQiLCJzdWIiOiJoem5pbmciLCJuYnQiOiIyMDI1LTA4LTI1VDEyOjAxOjA0Ljk2MjU3OTJaIiwiaWF0IjoiMjAyNS0wOC0yNVQxMjowMTowNC45NjI1NzkyWiIsImp0aSI6ImVkOWVmZTAwLWI0NWQtNGZmMi1iMTc2LTNhODdhMDI0MmMzMyIsImV4cCI6IjIwMjUtMDktMjVUMTI6MDE6MDQuOTYyNTc5MloiLCJ1aWQiOjI3NjQwOTM3LCJkaXNwbGF5TmFtZSI6IkgtWi1OaW5nIiwiZW1haWwiOiJ6aGluaW5nMTAyNEAxNjMuY29tIiwidGllciI6ImV4cGVydCIsInZlcmlmaWVkIjp0cnVlLCJwcm9maWxlVXJsIjoiL2h6bmluZyIsInRodW1ibmFpbFVybCI6Imh0dHBzOi8vc3RvcmFnZS5nb29nbGVhcGlzLmNvbS9rYWdnbGUtYXZhdGFycy90aHVtYm5haWxzLzI3NjQwOTM3LWtnLnBuZz90PTIwMjUtMDctMDEtMDAtNTctNTMiLCJmZiI6WyJCZW5jaG1hcmtVbmlmaWVkTGVhZGVyYm9hcmREYXRhR3JpZCIsIktlcm5lbHNPcGVuSW5Db2xhYkxvY2FsVXJsIiwiTWV0YXN0b3JlQ2hlY2tBZ2dyZWdhdGVGaWxlSGFzaGVzIiwiVXNlckxpY2Vuc2VBZ3JlZW1lbnRTdGFsZW5lc3NUcmFja2luZyIsIktlcm5lbHNTZXR0aW5nc1RhYiIsIktlcm5lbHNQYXlUb1NjYWxlIiwiTmV3QW5kRXhjaXRpbmdDb250ZW50TG9ocCIsIkhhY2thdGhvbldyaXRldXBSZW1pbmRlckJhbm5lciIsIkdhbWVBcmVuYSIsIlJlZGlyZWN0S2FnZ2xlT3JnQmVuY2htYXJrcyIsIkNvbW11bml0eVdyaXRlVXBzIiwiRmVhdHVyZWRNb2RlbHNTaGVsZiIsIkRhdGFzZXRQb2xhcnNEYXRhTG9hZGVyIiwiS2VybmVsc0ZpcmViYXNlTG9uZ1BvbGxpbmciLCJLZXJuZWxzRHJhZnRVcGxvYWRCbG9iIiwiS2VybmVsc1NhdmVDZWxsT3V0cHV0IiwiRnJvbnRlbmRFcnJvclJlcG9ydGluZyIsIkFsbG93Rm9ydW1BdHRhY2htZW50cyIsIlRlcm1zT2ZTZXJ2aWNlQmFubmVyIiwiRGF0YXNldFVwbG9hZGVyRHVwbGljYXRlRGV0ZWN0aW9uIl0sImZmZCI6eyJNb2RlbElkc0FsbG93SW5mZXJlbmNlIjoiIiwiTW9kZWxJbmZlcmVuY2VQYXJhbWV0ZXJzIjoieyBcIm1heF90b2tlbnNcIjogMTI4LCBcInRlbXBlcmF0dXJlXCI6IDAuNCwgXCJ0b3Bfa1wiOiA1IH0iLCJTcG90bGlnaHRDb21tdW5pdHlDb21wZXRpdGlvbiI6IjEwMTAzOSwxMDU4NzQsOTk2NTAsOTExOTYsOTE0NTEsOTE0NDgsODk4NTAsODg2MTIsOTQ2ODksOTc1NjksOTg0NTAsOTQxNDcsOTQxNDcsOTYxNjQsMTA1Mzk5IiwiUGVyc29uYWxCZW5jaG1hcmtzUHJpb3JpdHlUYXNrSWRzIjoiMTA2LDIyOSwxMDUsMjI3LDExMCwyMjgsMTE2LDIzMCwxNzMsMjMyLDE3NSwyMzksMjY0LDI0OSwyNDciLCJTdHNNaW5GaWxlcyI6Ijc1MDAwIiwiRmVhdHVyZWRCZW5jaG1hcmtzIjoiMSwxNiwxNDcsMTM0IiwiU3RzTWluR2IiOiIxIiwiR2V0dGluZ1N0YXJ0ZWRDb21wZXRpdGlvbnMiOiIzMTM2LDU0MDcsODY1MTgsMzQzNzciLCJDbGllbnRScGNSYXRlTGltaXRRcHMiOiI0MCIsIkNsaWVudFJwY1JhdGVMaW1pdFFwbSI6IjUwMCIsIkFkZEZlYXR1cmVGbGFnc1RvUGFnZUxvYWRUYWciOiJkaXNhYmxlZCIsIktlcm5lbEVkaXRvckF1dG9zYXZlVGhyb3R0bGVNcyI6IjMwMDAwIiwiS2VybmVsc0w0R3B1Q29tcHMiOiI4NjAyMyw4NDc5NSw4ODkyNSw5MTQ5NiIsIkdhbWVBcmVuYUJlbmNobWFya1ZlcnNpb25zIjoiMTI5IiwiRmVhdHVyZWRDb21tdW5pdHlDb21wZXRpdGlvbnMiOiI2MDA5NSw1NDAwMCw1NzE2Myw4MDg3NCw4MTc4Niw4MTcwNCw4MjYxMSw4NTIxMCIsIkVtZXJnZW5jeUFsZXJ0QmFubmVyIjoiIiwiQ29tcGV0aXRpb25NZXRyaWNUaW1lb3V0TWludXRlcyI6IjMwIiwiRW1lcmdlbmN5QWxlcnRCYW5uZXJGb3JVc2VyUHJvZmlsZSI6IiIsIkdhbWVBcmVuYUZlYXR1cmVkU3RyZWFtIjoiIiwiRGF0YXNldHNTZW5kUGVuZGluZ1N1Z2dlc3Rpb25zUmVtaW5kZXJzQmF0Y2hTaXplIjoiMTAwIiwiR2FtZUFyZW5hU3RyZWFtTGlua3MiOiIiLCJLZXJuZWxzUGF5VG9TY2FsZVByb1BsdXNHcHVIb3VycyI6IjMwIiwiS2VybmVsc1BheVRvU2NhbGVQcm9HcHVIb3VycyI6IjE1In0sInBpZCI6ImthZ2dsZS0xNjE2MDciLCJzdmMiOiJ3ZWItZmUiLCJzZGFrIjoiQUl6YVN5QTRlTnFVZFJSc2tKc0NaV1Z6LXFMNjU1WGE1SkVNcmVFIiwiYmxkIjoiYTc1ZmE4ZGY1NTExMWE4M2I1ZTBkMTIxMzg0OGM5NjAzNzFmZDg0ZSJ9.&lsp=true#predict_3)={'A': 0.53,  'B': 0.18,   'C': 0.29} |    70 x 30    | [ +0.0021, -0.000526, -0.001574 ] |\n|                  |         |                                                              |               |                                   |\n| **0.855** |   v.2   | Models.[1,2,3](https://www.kaggle.com/code/nina2025/cmi-ensemble-of-3-solutions-horizontal-blend)&nbsp;=&nbsp; [ 0.2710, &nbsp;0.3470, &nbsp;0.3820 ] <br/> **Model**[**.3**](https://kkb-production.jupyter-proxy.kaggle.net/static/assets/jupyterlab-v4/jupyterlab-index-f1bb73b2c4ad44c1a82e.html?session=eyJhbGciOiJub25lIiwidHlwIjoiSldUIn0.eyJpc3MiOiJrYWdnbGUiLCJhdWQiOiJjbGllbnQiLCJzdWIiOiJoem5pbmciLCJuYnQiOiIyMDI1LTA4LTI1VDEyOjAxOjA0Ljk2MjU3OTJaIiwiaWF0IjoiMjAyNS0wOC0yNVQxMjowMTowNC45NjI1NzkyWiIsImp0aSI6ImVkOWVmZTAwLWI0NWQtNGZmMi1iMTc2LTNhODdhMDI0MmMzMyIsImV4cCI6IjIwMjUtMDktMjVUMTI6MDE6MDQuOTYyNTc5MloiLCJ1aWQiOjI3NjQwOTM3LCJkaXNwbGF5TmFtZSI6IkgtWi1OaW5nIiwiZW1haWwiOiJ6aGluaW5nMTAyNEAxNjMuY29tIiwidGllciI6ImV4cGVydCIsInZlcmlmaWVkIjp0cnVlLCJwcm9maWxlVXJsIjoiL2h6bmluZyIsInRodW1ibmFpbFVybCI6Imh0dHBzOi8vc3RvcmFnZS5nb29nbGVhcGlzLmNvbS9rYWdnbGUtYXZhdGFycy90aHVtYm5haWxzLzI3NjQwOTM3LWtnLnBuZz90PTIwMjUtMDctMDEtMDAtNTctNTMiLCJmZiI6WyJCZW5jaG1hcmtVbmlmaWVkTGVhZGVyYm9hcmREYXRhR3JpZCIsIktlcm5lbHNPcGVuSW5Db2xhYkxvY2FsVXJsIiwiTWV0YXN0b3JlQ2hlY2tBZ2dyZWdhdGVGaWxlSGFzaGVzIiwiVXNlckxpY2Vuc2VBZ3JlZW1lbnRTdGFsZW5lc3NUcmFja2luZyIsIktlcm5lbHNTZXR0aW5nc1RhYiIsIktlcm5lbHNQYXlUb1NjYWxlIiwiTmV3QW5kRXhjaXRpbmdDb250ZW50TG9ocCIsIkhhY2thdGhvbldyaXRldXBSZW1pbmRlckJhbm5lciIsIkdhbWVBcmVuYSIsIlJlZGlyZWN0S2FnZ2xlT3JnQmVuY2htYXJrcyIsIkNvbW11bml0eVdyaXRlVXBzIiwiRmVhdHVyZWRNb2RlbHNTaGVsZiIsIkRhdGFzZXRQb2xhcnNEYXRhTG9hZGVyIiwiS2VybmVsc0ZpcmViYXNlTG9uZ1BvbGxpbmciLCJLZXJuZWxzRHJhZnRVcGxvYWRCbG9iIiwiS2VybmVsc1NhdmVDZWxsT3V0cHV0IiwiRnJvbnRlbmRFcnJvclJlcG9ydGluZyIsIkFsbG93Rm9ydW1BdHRhY2htZW50cyIsIlRlcm1zT2ZTZXJ2aWNlQmFubmVyIiwiRGF0YXNldFVwbG9hZGVyRHVwbGljYXRlRGV0ZWN0aW9uIl0sImZmZCI6eyJNb2RlbElkc0FsbG93SW5mZXJlbmNlIjoiIiwiTW9kZWxJbmZlcmVuY2VQYXJhbWV0ZXJzIjoieyBcIm1heF90b2tlbnNcIjogMTI4LCBcInRlbXBlcmF0dXJlXCI6IDAuNCwgXCJ0b3Bfa1wiOiA1IH0iLCJTcG90bGlnaHRDb21tdW5pdHlDb21wZXRpdGlvbiI6IjEwMTAzOSwxMDU4NzQsOTk2NTAsOTExOTYsOTE0NTEsOTE0NDgsODk4NTAsODg2MTIsOTQ2ODksOTc1NjksOTg0NTAsOTQxNDcsOTQxNDcsOTYxNjQsMTA1Mzk5IiwiUGVyc29uYWxCZW5jaG1hcmtzUHJpb3JpdHlUYXNrSWRzIjoiMTA2LDIyOSwxMDUsMjI3LDExMCwyMjgsMTE2LDIzMCwxNzMsMjMyLDE3NSwyMzksMjY0LDI0OSwyNDciLCJTdHNNaW5GaWxlcyI6Ijc1MDAwIiwiRmVhdHVyZWRCZW5jaG1hcmtzIjoiMSwxNiwxNDcsMTM0IiwiU3RzTWluR2IiOiIxIiwiR2V0dGluZ1N0YXJ0ZWRDb21wZXRpdGlvbnMiOiIzMTM2LDU0MDcsODY1MTgsMzQzNzciLCJDbGllbnRScGNSYXRlTGltaXRRcHMiOiI0MCIsIkNsaWVudFJwY1JhdGVMaW1pdFFwbSI6IjUwMCIsIkFkZEZlYXR1cmVGbGFnc1RvUGFnZUxvYWRUYWciOiJkaXNhYmxlZCIsIktlcm5lbEVkaXRvckF1dG9zYXZlVGhyb3R0bGVNcyI6IjMwMDAwIiwiS2VybmVsc0w0R3B1Q29tcHMiOiI4NjAyMyw4NDc5NSw4ODkyNSw5MTQ5NiIsIkdhbWVBcmVuYUJlbmNobWFya1ZlcnNpb25zIjoiMTI5IiwiRmVhdHVyZWRDb21tdW5pdHlDb21wZXRpdGlvbnMiOiI2MDA5NSw1NDAwMCw1NzE2Myw4MDg3NCw4MTc4Niw4MTcwNCw4MjYxMSw4NTIxMCIsIkVtZXJnZW5jeUFsZXJ0QmFubmVyIjoiIiwiQ29tcGV0aXRpb25NZXRyaWNUaW1lb3V0TWludXRlcyI6IjMwIiwiRW1lcmdlbmN5QWxlcnRCYW5uZXJGb3JVc2VyUHJvZmlsZSI6IiIsIkdhbWVBcmVuYUZlYXR1cmVkU3RyZWFtIjoiIiwiRGF0YXNldHNTZW5kUGVuZGluZ1N1Z2dlc3Rpb25zUmVtaW5kZXJzQmF0Y2hTaXplIjoiMTAwIiwiR2FtZUFyZW5hU3RyZWFtTGlua3MiOiIiLCJLZXJuZWxzUGF5VG9TY2FsZVByb1BsdXNHcHVIb3VycyI6IjMwIiwiS2VybmVsc1BheVRvU2NhbGVQcm9HcHVIb3VycyI6IjE1In0sInBpZCI6ImthZ2dsZS0xNjE2MDciLCJzdmMiOiJ3ZWItZmUiLCJzZGFrIjoiQUl6YVN5QTRlTnFVZFJSc2tKc0NaV1Z6LXFMNjU1WGE1SkVNcmVFIiwiYmxkIjoiYTc1ZmE4ZGY1NTExMWE4M2I1ZTBkMTIxMzg0OGM5NjAzNzFmZDg0ZSJ9.&lsp=true#predict_3)={'A': 0.53,  'B': 0.18,   'C': 0.29} |    70 x 30    | [ +0.0021, -0.000537, -0.001563 ] |\n|                  |         |                                                              |               |                                   |\n| **0.855** |   v.3   | Models.[1,2,3](https://www.kaggle.com/code/nina2025/cmi-ensemble-of-3-solutions-horizontal-blend)&nbsp;=&nbsp; [ 0.271 + **k**, &nbsp;0.357 - **k**, &nbsp;&nbsp;0.382 ] <br>**Model**[**.3**](https://kkb-production.jupyter-proxy.kaggle.net/static/assets/jupyterlab-v4/jupyterlab-index-f1bb73b2c4ad44c1a82e.html?session=eyJhbGciOiJub25lIiwidHlwIjoiSldUIn0.eyJpc3MiOiJrYWdnbGUiLCJhdWQiOiJjbGllbnQiLCJzdWIiOiJoem5pbmciLCJuYnQiOiIyMDI1LTA4LTI1VDEyOjAxOjA0Ljk2MjU3OTJaIiwiaWF0IjoiMjAyNS0wOC0yNVQxMjowMTowNC45NjI1NzkyWiIsImp0aSI6ImVkOWVmZTAwLWI0NWQtNGZmMi1iMTc2LTNhODdhMDI0MmMzMyIsImV4cCI6IjIwMjUtMDktMjVUMTI6MDE6MDQuOTYyNTc5MloiLCJ1aWQiOjI3NjQwOTM3LCJkaXNwbGF5TmFtZSI6IkgtWi1OaW5nIiwiZW1haWwiOiJ6aGluaW5nMTAyNEAxNjMuY29tIiwidGllciI6ImV4cGVydCIsInZlcmlmaWVkIjp0cnVlLCJwcm9maWxlVXJsIjoiL2h6bmluZyIsInRodW1ibmFpbFVybCI6Imh0dHBzOi8vc3RvcmFnZS5nb29nbGVhcGlzLmNvbS9rYWdnbGUtYXZhdGFycy90aHVtYm5haWxzLzI3NjQwOTM3LWtnLnBuZz90PTIwMjUtMDctMDEtMDAtNTctNTMiLCJmZiI6WyJCZW5jaG1hcmtVbmlmaWVkTGVhZGVyYm9hcmREYXRhR3JpZCIsIktlcm5lbHNPcGVuSW5Db2xhYkxvY2FsVXJsIiwiTWV0YXN0b3JlQ2hlY2tBZ2dyZWdhdGVGaWxlSGFzaGVzIiwiVXNlckxpY2Vuc2VBZ3JlZW1lbnRTdGFsZW5lc3NUcmFja2luZyIsIktlcm5lbHNTZXR0aW5nc1RhYiIsIktlcm5lbHNQYXlUb1NjYWxlIiwiTmV3QW5kRXhjaXRpbmdDb250ZW50TG9ocCIsIkhhY2thdGhvbldyaXRldXBSZW1pbmRlckJhbm5lciIsIkdhbWVBcmVuYSIsIlJlZGlyZWN0S2FnZ2xlT3JnQmVuY2htYXJrcyIsIkNvbW11bml0eVdyaXRlVXBzIiwiRmVhdHVyZWRNb2RlbHNTaGVsZiIsIkRhdGFzZXRQb2xhcnNEYXRhTG9hZGVyIiwiS2VybmVsc0ZpcmViYXNlTG9uZ1BvbGxpbmciLCJLZXJuZWxzRHJhZnRVcGxvYWRCbG9iIiwiS2VybmVsc1NhdmVDZWxsT3V0cHV0IiwiRnJvbnRlbmRFcnJvclJlcG9ydGluZyIsIkFsbG93Rm9ydW1BdHRhY2htZW50cyIsIlRlcm1zT2ZTZXJ2aWNlQmFubmVyIiwiRGF0YXNldFVwbG9hZGVyRHVwbGljYXRlRGV0ZWN0aW9uIl0sImZmZCI6eyJNb2RlbElkc0FsbG93SW5mZXJlbmNlIjoiIiwiTW9kZWxJbmZlcmVuY2VQYXJhbWV0ZXJzIjoieyBcIm1heF90b2tlbnNcIjogMTI4LCBcInRlbXBlcmF0dXJlXCI6IDAuNCwgXCJ0b3Bfa1wiOiA1IH0iLCJTcG90bGlnaHRDb21tdW5pdHlDb21wZXRpdGlvbiI6IjEwMTAzOSwxMDU4NzQsOTk2NTAsOTExOTYsOTE0NTEsOTE0NDgsODk4NTAsODg2MTIsOTQ2ODksOTc1NjksOTg0NTAsOTQxNDcsOTQxNDcsOTYxNjQsMTA1Mzk5IiwiUGVyc29uYWxCZW5jaG1hcmtzUHJpb3JpdHlUYXNrSWRzIjoiMTA2LDIyOSwxMDUsMjI3LDExMCwyMjgsMTE2LDIzMCwxNzMsMjMyLDE3NSwyMzksMjY0LDI0OSwyNDciLCJTdHNNaW5GaWxlcyI6Ijc1MDAwIiwiRmVhdHVyZWRCZW5jaG1hcmtzIjoiMSwxNiwxNDcsMTM0IiwiU3RzTWluR2IiOiIxIiwiR2V0dGluZ1N0YXJ0ZWRDb21wZXRpdGlvbnMiOiIzMTM2LDU0MDcsODY1MTgsMzQzNzciLCJDbGllbnRScGNSYXRlTGltaXRRcHMiOiI0MCIsIkNsaWVudFJwY1JhdGVMaW1pdFFwbSI6IjUwMCIsIkFkZEZlYXR1cmVGbGFnc1RvUGFnZUxvYWRUYWciOiJkaXNhYmxlZCIsIktlcm5lbEVkaXRvckF1dG9zYXZlVGhyb3R0bGVNcyI6IjMwMDAwIiwiS2VybmVsc0w0R3B1Q29tcHMiOiI4NjAyMyw4NDc5NSw4ODkyNSw5MTQ5NiIsIkdhbWVBcmVuYUJlbmNobWFya1ZlcnNpb25zIjoiMTI5IiwiRmVhdHVyZWRDb21tdW5pdHlDb21wZXRpdGlvbnMiOiI2MDA5NSw1NDAwMCw1NzE2Myw4MDg3NCw4MTc4Niw4MTcwNCw4MjYxMSw4NTIxMCIsIkVtZXJnZW5jeUFsZXJ0QmFubmVyIjoiIiwiQ29tcGV0aXRpb25NZXRyaWNUaW1lb3V0TWludXRlcyI6IjMwIiwiRW1lcmdlbmN5QWxlcnRCYW5uZXJGb3JVc2VyUHJvZmlsZSI6IiIsIkdhbWVBcmVuYUZlYXR1cmVkU3RyZWFtIjoiIiwiRGF0YXNldHNTZW5kUGVuZGluZ1N1Z2dlc3Rpb25zUmVtaW5kZXJzQmF0Y2hTaXplIjoiMTAwIiwiR2FtZUFyZW5hU3RyZWFtTGlua3MiOiIiLCJLZXJuZWxzUGF5VG9TY2FsZVByb1BsdXNHcHVIb3VycyI6IjMwIiwiS2VybmVsc1BheVRvU2NhbGVQcm9HcHVIb3VycyI6IjE1In0sInBpZCI6ImthZ2dsZS0xNjE2MDciLCJzdmMiOiJ3ZWItZmUiLCJzZGFrIjoiQUl6YVN5QTRlTnFVZFJSc2tKc0NaV1Z6LXFMNjU1WGE1SkVNcmVFIiwiYmxkIjoiYTc1ZmE4ZGY1NTExMWE4M2I1ZTBkMTIxMzg0OGM5NjAzNzFmZDg0ZSJ9.&lsp=true#predict_3)={'A': 0.535,  'B': 0.1775,   'C': 0.2875} |    70 x 30    | [ +0.0021, -0.000537, -0.001563 ] |\n","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"## 🧱 Model 1 (Keras version)\n\n🛠️ **Tools & Features**  \nProvide functions like `remove_gravity_from_acc`, `calculate_angular_velocity_from_quat`, and `calculate_angular_distance` to transform raw IMU, quaternion, and ToF data into linear acceleration, angular velocity, angular displacement, and ToF statistical features.\n\n🏗️ **Network Architecture**  \n`build_two_branch_model()` builds a hybrid network consisting of a two-branch CNN (ResNet-SE) plus Bi-LSTM/GRU and Attention layers for sequence classification.\n\n📦 **One-Shot Loading of 40 Models**  \nRead 10 × 4 = 40 `.h5` weight files from four different experimental directories, load them all into the `models1` list, and simultaneously load the corresponding `feature_cols`, `pad_len`, `scaler`, and `gesture_classes` to ensure consistent preprocessing.\n\n🎯 **Purpose**  \nIn the subsequent `predict1()` call, directly use `models1` to perform parallel inference across all models, then take the median fusion of their outputs to yield the final class probabilities.\n\n🚀 **One-liner**  \n“Prepare 40Keras models along with their preprocessing pipelines for one-click real-time inference.”","metadata":{}},{"cell_type":"code","source":"import os, json, joblib, numpy as np, pandas as pd\nimport random\nfrom pathlib import Path\nimport warnings  \n\nwarnings.filterwarnings(\"ignore\")\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import StandardScaler, LabelEncoder\nfrom sklearn.utils.class_weight import compute_class_weight\nfrom tensorflow.keras.utils import Sequence, to_categorical, pad_sequences\nfrom tensorflow.keras.models import Model, load_model\nfrom tensorflow.keras.layers import (\n    Input, Conv1D, BatchNormalization, Activation, add, MaxPooling1D, Dropout,\n    Bidirectional, LSTM, GlobalAveragePooling1D, Dense, Multiply, Reshape,\n    Lambda, Concatenate, GRU, GaussianNoise\n)\nfrom tensorflow.keras.regularizers import l2\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.callbacks import EarlyStopping\nfrom tensorflow.keras import backend as K\nimport tensorflow as tf\nimport polars as pl\nfrom sklearn.model_selection import StratifiedGroupKFold\nfrom scipy.spatial.transform import Rotation as R\n\ndef seed_everything(seed):\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    random.seed(seed)\n    np.random.seed(seed)\n    tf.random.set_seed(seed)\n    tf.experimental.numpy.random.seed(seed)\n    os.environ['TF_CUDNN_DETERMINISTIC'] = '1'\n    os.environ['TF_DETERMINISTIC_OPS'] = '1'\n\nseed_everything(seed=42)\n# (Competition metric will only be imported when TRAINing)\nTRAIN = False                     # ← set to True when you want to train\nRAW_DIR = Path(\"/kaggle/input/cmi-detect-behavior-with-sensor-data\")\nPRETRAINED_DIR = Path(\"/kaggle/input/cmi-d-111\")\nEXPORT_DIR = Path(\"./\")                                    # artefacts will be saved here\nBATCH_SIZE = 64\nPAD_PERCENTILE = 95\nLR_INIT = 5e-4\nWD = 3e-3\nMIXUP_ALPHA = 0.4\nEPOCHS = 160\nPATIENCE = 40\n\n\nprint(\"▶ imports ready· tensorflow\", tf.__version__)\n#Tensor Manipulations\ndef time_sum(x):\n    return K.sum(x, axis=1)\n\ndef squeeze_last_axis(x):\n    return tf.squeeze(x, axis=-1)\n\ndef expand_last_axis(x):\n    return tf.expand_dims(x, axis=-1)\n\ndef se_block(x, reduction=8):\n    ch = x.shape[-1]\n    se = GlobalAveragePooling1D()(x)\n    se = Dense(ch // reduction, activation='relu')(se)\n    se = Dense(ch, activation='sigmoid')(se)\n    se = Reshape((1, ch))(se)\n    return Multiply()([x, se])\n\n# Residual CNN Block with SE\ndef residual_se_cnn_block(x, filters, kernel_size, pool_size=2, drop=0.3, wd=1e-4):\n    shortcut = x\n    for _ in range(2):\n        x = Conv1D(filters, kernel_size, padding='same', use_bias=False,\n                   kernel_regularizer=l2(wd))(x)\n        x = BatchNormalization()(x)\n        x = Activation('relu')(x)\n    x = se_block(x)\n    if shortcut.shape[-1] != filters:\n        shortcut = Conv1D(filters, 1, padding='same', use_bias=False,\n                          kernel_regularizer=l2(wd))(shortcut)\n        shortcut = BatchNormalization()(shortcut)\n    x = add([x, shortcut])\n    x = Activation('relu')(x)\n    x = MaxPooling1D(pool_size)(x)\n    x = Dropout(drop)(x)\n    return x\n\ndef attention_layer(inputs):\n    score = Dense(1, activation='tanh')(inputs)\n    score = Lambda(squeeze_last_axis)(score)\n    weights = Activation('softmax')(score)\n    weights = Lambda(expand_last_axis)(weights)\n    context = Multiply()([inputs, weights])\n    context = Lambda(time_sum)(context)\n    return context\n\n# Normalizes and cleans the time series sequence. \n\ndef preprocess_sequence(df_seq: pd.DataFrame, feature_cols: list[str], scaler: StandardScaler):\n    mat = df_seq[feature_cols].ffill().bfill().fillna(0).values\n    return scaler.transform(mat).astype('float32')\n\n# MixUp the data argumentation in order to regularize the neural network. \n\nclass MixupGenerator(Sequence):\n    def __init__(self, X, y, batch_size, alpha=0.2):\n        self.X, self.y = X, y\n        self.batch = batch_size\n        self.alpha = alpha\n        self.indices = np.arange(len(X))\n    def __len__(self):\n        return int(np.ceil(len(self.X) / self.batch))\n    def __getitem__(self, i):\n        idx = self.indices[i*self.batch:(i+1)*self.batch]\n        Xb, yb = self.X[idx], self.y[idx]\n        lam = np.random.beta(self.alpha, self.alpha)\n        perm = np.random.permutation(len(Xb))\n        X_mix = lam * Xb + (1-lam) * Xb[perm]\n        y_mix = lam * yb + (1-lam) * yb[perm]\n        return X_mix, y_mix\n    def on_epoch_end(self):\n        np.random.shuffle(self.indices)\n\n\ndef remove_gravity_from_acc(acc_data, rot_data):\n\n    if isinstance(acc_data, pd.DataFrame):\n        acc_values = acc_data[['acc_x', 'acc_y', 'acc_z']].values\n    else:\n        acc_values = acc_data\n\n    if isinstance(rot_data, pd.DataFrame):\n        quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n    else:\n        quat_values = rot_data\n\n    num_samples = acc_values.shape[0]\n    linear_accel = np.zeros_like(acc_values)\n    \n    gravity_world = np.array([0, 0, 9.81])\n\n    for i in range(num_samples):\n        if np.all(np.isnan(quat_values[i])) or np.all(np.isclose(quat_values[i], 0)):\n            linear_accel[i, :] = acc_values[i, :] \n            continue\n\n        try:\n            rotation = R.from_quat(quat_values[i])\n            gravity_sensor_frame = rotation.apply(gravity_world, inverse=True)\n            linear_accel[i, :] = acc_values[i, :] - gravity_sensor_frame\n        except ValueError:\n             linear_accel[i, :] = acc_values[i, :]\n             \n    return linear_accel\n\ndef calculate_angular_velocity_from_quat(rot_data, time_delta=1/200): # Assuming 200Hz sampling rate\n    if isinstance(rot_data, pd.DataFrame):\n        quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n    else:\n        quat_values = rot_data\n\n    num_samples = quat_values.shape[0]\n    angular_vel = np.zeros((num_samples, 3))\n\n    for i in range(num_samples - 1):\n        q_t = quat_values[i]\n        q_t_plus_dt = quat_values[i+1]\n\n        if np.all(np.isnan(q_t)) or np.all(np.isclose(q_t, 0)) or \\\n           np.all(np.isnan(q_t_plus_dt)) or np.all(np.isclose(q_t_plus_dt, 0)):\n            continue\n\n        try:\n            rot_t = R.from_quat(q_t)\n            rot_t_plus_dt = R.from_quat(q_t_plus_dt)\n\n            # Calculate the relative rotation\n            delta_rot = rot_t.inv() * rot_t_plus_dt\n            \n            # Convert delta rotation to angular velocity vector\n            # The rotation vector (Euler axis * angle) scaled by 1/dt\n            # is a good approximation for small delta_rot\n            angular_vel[i, :] = delta_rot.as_rotvec() / time_delta\n        except ValueError:\n            # If quaternion is invalid, angular velocity remains zero\n            pass\n            \n    return angular_vel\n    \ndef calculate_angular_distance(rot_data):\n    if isinstance(rot_data, pd.DataFrame):\n        quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n    else:\n        quat_values = rot_data\n\n    num_samples = quat_values.shape[0]\n    angular_dist = np.zeros(num_samples)\n\n    for i in range(num_samples - 1):\n        q1 = quat_values[i]\n        q2 = quat_values[i+1]\n\n        if np.all(np.isnan(q1)) or np.all(np.isclose(q1, 0)) or \\\n           np.all(np.isnan(q2)) or np.all(np.isclose(q2, 0)):\n            angular_dist[i] = 0 # Или np.nan, в зависимости от желаемого поведения\n            continue\n        try:\n            # Преобразование кватернионов в объекты Rotation\n            r1 = R.from_quat(q1)\n            r2 = R.from_quat(q2)\n\n            # Вычисление углового расстояния: 2 * arccos(|real(p * q*)|)\n            # где p* - сопряженный кватернион q\n            # В scipy.spatial.transform.Rotation, r1.inv() * r2 дает относительное вращение.\n            # Угол этого относительного вращения - это и есть угловое расстояние.\n            relative_rotation = r1.inv() * r2\n            \n            # Угол rotation vector соответствует угловому расстоянию\n            # Норма rotation vector - это угол в радианах\n            angle = np.linalg.norm(relative_rotation.as_rotvec())\n            angular_dist[i] = angle\n        except ValueError:\n            angular_dist[i] = 0 # В случае недействительных кватернионов\n            pass\n            \n    return angular_dist\n\ndef build_two_branch_model(pad_len, imu_dim, tof_dim, n_classes, wd=1e-4):\n    inp = Input(shape=(pad_len, imu_dim+tof_dim))\n    imu = Lambda(lambda t: t[:, :, :imu_dim])(inp)\n    tof = Lambda(lambda t: t[:, :, imu_dim:])(inp)\n\n    # IMU deep branch\n    x1 = residual_se_cnn_block(imu, 64, 3, drop=0.1, wd=wd)\n    x1 = residual_se_cnn_block(x1, 128, 5, drop=0.1, wd=wd)\n\n    # TOF/Thermal lighter branch\n    x2 = Conv1D(64, 3, padding='same', use_bias=False, kernel_regularizer=l2(wd))(tof)\n    x2 = BatchNormalization()(x2); x2 = Activation('relu')(x2)\n    x2 = MaxPooling1D(2)(x2); x2 = Dropout(0.2)(x2)\n    x2 = Conv1D(128, 3, padding='same', use_bias=False, kernel_regularizer=l2(wd))(x2)\n    x2 = BatchNormalization()(x2); x2 = Activation('relu')(x2)\n    x2 = MaxPooling1D(2)(x2); x2 = Dropout(0.2)(x2)\n\n    merged = Concatenate()([x1, x2])\n\n    xa = Bidirectional(LSTM(128, return_sequences=True, kernel_regularizer=l2(wd)))(merged)\n    xb = Bidirectional(GRU(128, return_sequences=True, kernel_regularizer=l2(wd)))(merged)\n    xc = GaussianNoise(0.09)(merged)\n    xc = Dense(16, activation='elu')(xc)\n    \n    x = Concatenate()([xa, xb, xc])\n    x = Dropout(0.4)(x)\n    x = attention_layer(x)\n\n    for units, drop in [(256, 0.5), (128, 0.3)]:\n        x = Dense(units, use_bias=False, kernel_regularizer=l2(wd))(x)\n        x = BatchNormalization()(x); x = Activation('relu')(x)\n        x = Dropout(drop)(x)\n\n    out = Dense(n_classes, activation='softmax', kernel_regularizer=l2(wd))(x)\n    return Model(inp, out)\n\ntmp_model = build_two_branch_model(127,7,325,18)\n\ncustom_objs = {\n    'time_sum': time_sum, 'squeeze_last_axis': squeeze_last_axis, 'expand_last_axis': expand_last_axis,\n    'se_block': se_block, 'residual_se_cnn_block': residual_se_cnn_block, 'attention_layer': attention_layer,\n}\n\n# ----------------------------------------------------------------- #\n# Load any Models\n# * is 2 Train Model Load\n# ----------------------------------------------------------------- #\n\nPRETRAINED_DIR = Path(\"/kaggle/input/cmi-d-111\")\nprint(\"▶ INFERENCE MODE 1,2 – loading artefacts from\", PRETRAINED_DIR)\nfinal_feature_cols = np.load(PRETRAINED_DIR / \"feature_cols.npy\", allow_pickle=True).tolist()\npad_len        = int(np.load(PRETRAINED_DIR / \"sequence_maxlen.npy\"))\nscaler         = joblib.load(PRETRAINED_DIR / \"scaler.pkl\")\ngesture_classes = np.load(PRETRAINED_DIR / \"gesture_classes.npy\", allow_pickle=True)\n\nmodels1 = []\nprint(f\"  Loading models for ensemble inference...\")\nfor fold in range(10):\n    model_path = f\"{PRETRAINED_DIR}/D-111_{fold}.h5\"\n    print(\">>>LoadModel>>>\",model_path)\n    model = load_model(model_path, compile=False, custom_objects=custom_objs)\n    models1.append(model)\nprint(\"-\"*50)\n\nfor fold in range(10):\n    model_path = f\"{PRETRAINED_DIR}/v0629_{fold}.h5\"\n    print(\">>>LoadModel>>>\",model_path)\n    model = load_model(model_path, compile=False, custom_objects=custom_objs)\n    models1.append(model)\nprint(\"-\"*50)\nprint(f\"[INFO]NumUseModels:{len(models1)}\")\n\n\nPRETRAINED_DIR = Path(\"/kaggle/input/n-splits-10\")\nprint(\"▶ INFERENCE MODE 3 – loading artefacts from\", PRETRAINED_DIR)\nfinal_feature_cols = np.load(PRETRAINED_DIR / \"feature_cols.npy\", allow_pickle=True).tolist()\npad_len        = int(np.load(PRETRAINED_DIR / \"sequence_maxlen.npy\"))\nscaler         = joblib.load(PRETRAINED_DIR / \"scaler.pkl\")\ngesture_classes = np.load(PRETRAINED_DIR / \"gesture_classes.npy\", allow_pickle=True)\nfor fold in range(10):\n    model_path = f\"{PRETRAINED_DIR}/gesture_model_fold_{fold}.h5\"\n    print(\">>>LoadModel>>>\",model_path)\n    model = load_model(model_path, compile=False, custom_objects=custom_objs)\n    models1.append(model)\nprint(\"-\"*50)\nprint(f\"[INFO]NumUseModels:{len(models1)}\")\n\nfor fold in range(10):\n    MODEL_DIR = \"/kaggle/input/cmi-data-tensorflow-train\"\n    \n    model_path = f\"{MODEL_DIR}/gesture_model_fold_{fold}.h5\"\n    print(\">>>LoadModel>>>\",model_path)\n    model = load_model(model_path, compile=False, custom_objects=custom_objs)\n    models1.append(model)\nprint(\"-\"*50)\nprint(f\"[INFO]NumUseModels:{len(models1)}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-01T11:23:14.127924Z","iopub.execute_input":"2025-09-01T11:23:14.128126Z","iopub.status.idle":"2025-09-01T11:24:02.557303Z","shell.execute_reply.started":"2025-09-01T11:23:14.128107Z","shell.execute_reply":"2025-09-01T11:24:02.556534Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"###  🔢 Model 1 inference output\n\n**Convert a single sensor sequence into a fixed-dimensional feature matrix, run parallel inference with all loaded Keras models (models1), and finally obtain the class-wise probability vector via median fusion.**","metadata":{}},{"cell_type":"code","source":"def predict1(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\n    df_seq = sequence.to_pandas()\n    linear_accel = remove_gravity_from_acc(df_seq, df_seq)\n    df_seq['linear_acc_x'], df_seq['linear_acc_y'], df_seq['linear_acc_z'] = linear_accel[:, 0], linear_accel[:, 1], linear_accel[:, 2]\n    df_seq['linear_acc_mag'] = np.sqrt(df_seq['linear_acc_x']**2 + df_seq['linear_acc_y']**2 + df_seq['linear_acc_z']**2)\n    df_seq['linear_acc_mag_jerk'] = df_seq['linear_acc_mag'].diff().fillna(0)\n    angular_vel = calculate_angular_velocity_from_quat(df_seq)\n    df_seq['angular_vel_x'], df_seq['angular_vel_y'], df_seq['angular_vel_z'] = angular_vel[:, 0], angular_vel[:, 1], angular_vel[:, 2]\n    df_seq['angular_distance'] = calculate_angular_distance(df_seq)\n    \n    for i in range(1, 6):\n        pixel_cols = [f\"tof_{i}_v{p}\" for p in range(64)]; tof_data = df_seq[pixel_cols].replace(-1, np.nan)\n        df_seq[f'tof_{i}_mean'], df_seq[f'tof_{i}_std'], df_seq[f'tof_{i}_min'], df_seq[f'tof_{i}_max'] = tof_data.mean(axis=1), tof_data.std(axis=1), tof_data.min(axis=1), tof_data.max(axis=1)\n        \n    mat_unscaled = df_seq[final_feature_cols].ffill().bfill().fillna(0).values.astype('float32')\n    mat_scaled = scaler.transform(mat_unscaled)\n    pad_input = pad_sequences([mat_scaled], maxlen=pad_len, padding='post', truncating='post', dtype='float32')\n    \n    all_preds = [model.predict(pad_input, verbose=0)[0] for model in models1]\n    avg_pred = np.median(all_preds, axis=0) \n    return avg_pred","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-01T11:24:02.558808Z","iopub.execute_input":"2025-09-01T11:24:02.559250Z","iopub.status.idle":"2025-09-01T11:24:02.567475Z","shell.execute_reply.started":"2025-09-01T11:24:02.559222Z","shell.execute_reply":"2025-09-01T11:24:02.566694Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 🧱 Model 2(PyTorch version)\n\n📊 **Dataset Preparation**  \nUse CMIFeDataset to convert the raw CSV into IMU / Thermal / ToF three-modality features, handling missing values, standardizing, padding, truncating, and generating 5-fold cross-validation indices.\n\n🧱 **Model Construction**  \nDefine CMIModel:  \n• Three independent ResNet-SE CNN branches for IMU, Thermal, and ToF  \n• Concatenate their features into a BERT-like Transformer for global representation  \n• Final dense layer outputs class logits\n\n📥 **Model Loading**  \nRead the 5-fold pre-trained .pt weights from /kaggle/input/cmi-models-public/... and load them into the models2 list.\n\n⚡ **Inference Functions**  \n• inference_process(): transforms a single Polars sequence into three-modality tensors  \n• avg_predict(): averages the five fold-wise models and returns the final softmax probabilities\n\n🧪 **Evaluation Interface (commented out)**  \nProvides valid() and metric_package for offline validation or IMU-only ablation experiments.\n\n🚀 **One-sentence summary**  \nLoad and ensemble a 5-fold PyTorch three-modality Transformer for rapid inference on new sequences.","metadata":{}},{"cell_type":"code","source":"import os\nimport torch\nimport kagglehub\nfrom pathlib import Path\nimport numpy as np\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom scipy.spatial.transform import Rotation as R\nfrom collections import defaultdict\nfrom torch.utils.data import Dataset, DataLoader, Subset\nfrom tqdm.notebook import tqdm\nfrom torch.amp import autocast\nimport pandas as pd\nimport polars as pl\nfrom sklearn.model_selection import StratifiedKFold\nfrom sklearn.utils.class_weight import compute_class_weight\nfrom sklearn.preprocessing import StandardScaler, LabelEncoder\nfrom transformers import BertConfig, BertModel\n\n\ndef remove_gravity_from_acc(acc_data, rot_data):\n    if isinstance(acc_data, pd.DataFrame):\n        acc_values = acc_data[['acc_x', 'acc_y', 'acc_z']].values\n    else:\n        acc_values = acc_data\n    if isinstance(rot_data, pd.DataFrame):\n        quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n    else:\n        quat_values = rot_data\n    num_samples = acc_values.shape[0]\n    linear_accel = np.zeros_like(acc_values)\n    gravity_world = np.array([0, 0, 9.81])\n    for i in range(num_samples):\n        if np.all(np.isnan(quat_values[i])) or np.all(np.isclose(quat_values[i], 0)):\n            linear_accel[i, :] = acc_values[i, :] \n            continue\n        try:\n            rotation = R.from_quat(quat_values[i])\n            gravity_sensor_frame = rotation.apply(gravity_world, inverse=True)\n            linear_accel[i, :] = acc_values[i, :] - gravity_sensor_frame\n        except ValueError:\n             linear_accel[i, :] = acc_values[i, :]\n    return linear_accel\n\ndef calculate_angular_velocity_from_quat(rot_data, time_delta=1/200): # Assuming 200Hz sampling rate\n    if isinstance(rot_data, pd.DataFrame):\n        quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n    else:\n        quat_values = rot_data\n    num_samples = quat_values.shape[0]\n    angular_vel = np.zeros((num_samples, 3))\n    for i in range(num_samples - 1):\n        q_t = quat_values[i]\n        q_t_plus_dt = quat_values[i+1]\n        if np.all(np.isnan(q_t)) or np.all(np.isclose(q_t, 0)) or \\\n           np.all(np.isnan(q_t_plus_dt)) or np.all(np.isclose(q_t_plus_dt, 0)):\n            continue\n        try:\n            rot_t = R.from_quat(q_t)\n            rot_t_plus_dt = R.from_quat(q_t_plus_dt)\n            delta_rot = rot_t.inv() * rot_t_plus_dt\n            angular_vel[i, :] = delta_rot.as_rotvec() / time_delta\n        except ValueError:\n            pass\n    return angular_vel\n\ndef calculate_angular_distance(rot_data):\n    if isinstance(rot_data, pd.DataFrame):\n        quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n    else:\n        quat_values = rot_data\n    num_samples = quat_values.shape[0]\n    angular_dist = np.zeros(num_samples)\n    for i in range(num_samples - 1):\n        q1 = quat_values[i]\n        q2 = quat_values[i+1]\n        if np.all(np.isnan(q1)) or np.all(np.isclose(q1, 0)) or \\\n           np.all(np.isnan(q2)) or np.all(np.isclose(q2, 0)):\n            angular_dist[i] = 0\n            continue\n        try:\n            r1 = R.from_quat(q1)\n            r2 = R.from_quat(q2)\n            relative_rotation = r1.inv() * r2\n            angle = np.linalg.norm(relative_rotation.as_rotvec())\n            angular_dist[i] = angle\n        except ValueError:\n            angular_dist[i] = 0 # В случае недействительных кватернионов\n            pass\n    return angular_dist\n\n\nclass CMIFeDataset(Dataset):\n    def __init__(self, data_path, config):\n        self.config = config\n        self.init_feature_names(data_path)\n        df = self.generate_features(pd.read_csv(data_path, usecols=set(self.base_cols+self.feature_cols)))\n        self.generate_dataset(df)\n\n    def init_feature_names(self, data_path):\n        self.imu_engineered_features = [\n            'acc_mag', 'rot_angle',\n            'acc_mag_jerk', 'rot_angle_vel',\n            'linear_acc_mag', 'linear_acc_mag_jerk',\n            'angular_vel_x', 'angular_vel_y', 'angular_vel_z',\n            'angular_distance'\n        ]\n\n        self.tof_mode = self.config.get(\"tof_mode\", \"stats\")\n        self.tof_region_stats = ['mean', 'std', 'min', 'max']\n        self.tof_cols = self.generate_tof_feature_names()\n\n        columns = pd.read_csv(data_path, nrows=0).columns.tolist()\n        imu_cols_base = ['linear_acc_x', 'linear_acc_y', 'linear_acc_z']\n        imu_cols_base.extend([c for c in columns if c.startswith('rot_') and c not in ['rot_angle', 'rot_angle_vel']])\n        self.imu_cols = list(dict.fromkeys(imu_cols_base + self.imu_engineered_features))\n        self.thm_cols = [c for c in columns if c.startswith('thm_')]\n        self.feature_cols = self.imu_cols + self.thm_cols + self.tof_cols\n        self.imu_dim = len(self.imu_cols)\n        self.thm_dim = len(self.thm_cols)\n        self.tof_dim = len(self.tof_cols)\n        self.base_cols = ['acc_x', 'acc_y', 'acc_z',\n                          'rot_x', 'rot_y', 'rot_z', 'rot_w',\n                          'sequence_id', 'subject', \n                          'sequence_type', 'gesture', 'orientation'] + [c for c in columns if c.startswith('thm_')] + [f\"tof_{i}_v{p}\" for i in range(1, 6) for p in range(64)]\n        self.fold_cols = ['subject', 'sequence_type', 'gesture', 'orientation']\n\n    def generate_tof_feature_names(self):\n        features = []\n        if self.config.get(\"tof_raw\", False):\n            for i in range(1, 6):\n                features.extend([f\"tof_{i}_v{p}\" for p in range(64)])\n        for i in range(1, 6):\n            if self.tof_mode != 0:\n                for stat in self.tof_region_stats:\n                    features.append(f'tof_{i}_{stat}')\n                if self.tof_mode > 1:\n                    for r in range(self.tof_mode):\n                        for stat in self.tof_region_stats:\n                            features.append(f'tof{self.tof_mode}_{i}_region_{r}_{stat}')\n                if self.tof_mode == -1:\n                    for mode in [2, 4, 8, 16, 32]:\n                        for r in range(mode):\n                            for stat in self.tof_region_stats:\n                                features.append(f'tof{mode}_{i}_region_{r}_{stat}')\n        return features\n\n    def compute_features(self, df):\n        df['acc_mag'] = np.sqrt(df['acc_x']**2 + df['acc_y']**2 + df['acc_z']**2)\n        df['rot_angle'] = 2 * np.arccos(df['rot_w'].clip(-1, 1))\n        df['acc_mag_jerk'] = df.groupby('sequence_id')['acc_mag'].diff().fillna(0)\n        df['rot_angle_vel'] = df.groupby('sequence_id')['rot_angle'].diff().fillna(0)\n            \n        linear_accel_list = []\n        for _, group in df.groupby('sequence_id'):\n            acc_data_group = group[['acc_x', 'acc_y', 'acc_z']]\n            rot_data_group = group[['rot_x', 'rot_y', 'rot_z', 'rot_w']]\n            linear_accel_group = remove_gravity_from_acc(acc_data_group, rot_data_group)\n            linear_accel_list.append(pd.DataFrame(linear_accel_group, columns=['linear_acc_x', 'linear_acc_y', 'linear_acc_z'], index=group.index))\n        df_linear_accel = pd.concat(linear_accel_list)\n        df = pd.concat([df, df_linear_accel], axis=1)\n        df['linear_acc_mag'] = np.sqrt(df['linear_acc_x']**2 + df['linear_acc_y']**2 + df['linear_acc_z']**2)\n        df['linear_acc_mag_jerk'] = df.groupby('sequence_id')['linear_acc_mag'].diff().fillna(0)\n    \n        angular_vel_list = []\n        for _, group in df.groupby('sequence_id'):\n            rot_data_group = group[['rot_x', 'rot_y', 'rot_z', 'rot_w']]\n            angular_vel_group = calculate_angular_velocity_from_quat(rot_data_group)\n            angular_vel_list.append(pd.DataFrame(angular_vel_group, columns=['angular_vel_x', 'angular_vel_y', 'angular_vel_z'], index=group.index))\n        df_angular_vel = pd.concat(angular_vel_list)\n        df = pd.concat([df, df_angular_vel], axis=1)\n    \n        angular_distance_list = []\n        for _, group in df.groupby('sequence_id'):\n            rot_data_group = group[['rot_x', 'rot_y', 'rot_z', 'rot_w']]\n            angular_dist_group = calculate_angular_distance(rot_data_group)\n            angular_distance_list.append(pd.DataFrame(angular_dist_group, columns=['angular_distance'], index=group.index))\n        df_angular_distance = pd.concat(angular_distance_list)\n        df = pd.concat([df, df_angular_distance], axis=1)\n\n        if self.tof_mode != 0:\n            new_columns = {}\n            for i in range(1, 6):\n                pixel_cols = [f\"tof_{i}_v{p}\" for p in range(64)]\n                tof_data = df[pixel_cols].replace(-1, np.nan)\n                new_columns.update({\n                    f'tof_{i}_mean': tof_data.mean(axis=1),\n                    f'tof_{i}_std': tof_data.std(axis=1),\n                    f'tof_{i}_min': tof_data.min(axis=1),\n                    f'tof_{i}_max': tof_data.max(axis=1)\n                })\n                if self.tof_mode > 1:\n                    region_size = 64 // self.tof_mode\n                    for r in range(self.tof_mode):\n                        region_data = tof_data.iloc[:, r*region_size : (r+1)*region_size]\n                        new_columns.update({\n                            f'tof{self.tof_mode}_{i}_region_{r}_mean': region_data.mean(axis=1),\n                            f'tof{self.tof_mode}_{i}_region_{r}_std': region_data.std(axis=1),\n                            f'tof{self.tof_mode}_{i}_region_{r}_min': region_data.min(axis=1),\n                            f'tof{self.tof_mode}_{i}_region_{r}_max': region_data.max(axis=1)\n                        })\n                if self.tof_mode == -1:\n                    for mode in [2, 4, 8, 16, 32]:\n                        region_size = 64 // mode\n                        for r in range(mode):\n                            region_data = tof_data.iloc[:, r*region_size : (r+1)*region_size]\n                            new_columns.update({\n                                f'tof{mode}_{i}_region_{r}_mean': region_data.mean(axis=1),\n                                f'tof{mode}_{i}_region_{r}_std': region_data.std(axis=1),\n                                f'tof{mode}_{i}_region_{r}_min': region_data.min(axis=1),\n                                f'tof{mode}_{i}_region_{r}_max': region_data.max(axis=1)\n                            })\n            df = pd.concat([df, pd.DataFrame(new_columns)], axis=1)\n        return df\n        \n    def generate_features(self, df):\n        self.le = LabelEncoder()\n        df['gesture_int'] = self.le.fit_transform(df['gesture'])\n        self.class_num = len(self.le.classes_)\n        \n        if all(c in df.columns for c in self.imu_engineered_features) and all(c in df.columns for c in self.tof_cols):\n            print(\"Have precomputed, skip compute.\")\n        else:\n            print(\"Not precomputed, do compute.\")\n            df = self.compute_features(df)\n\n        if self.config.get(\"save_precompute\", False):\n            df.to_csv(self.config.get(\"save_filename\", \"train.csv\"))\n        return df\n\n    def scale(self, data_unscaled):\n        scaler_function = self.config.get(\"scaler_function\", StandardScaler())\n        scaler = scaler_function.fit(np.concatenate(data_unscaled, axis=0))\n        return [scaler.transform(x) for x in data_unscaled], scaler\n\n    def pad(self, data_scaled, cols):\n        pad_data = np.zeros((len(data_scaled), self.pad_len, len(cols)), dtype='float32')\n        for i, seq in enumerate(data_scaled):\n            seq_len = min(len(seq), self.pad_len)\n            pad_data[i, :seq_len] = seq[:seq_len]\n        return pad_data\n\n    def get_nan_value(self, data, ratio):\n        max_value = data.max().max()\n        nan_value = -max_value * ratio\n        return nan_value\n\n    def generate_dataset(self, df):\n        seq_gp = df.groupby('sequence_id') \n        imu_unscaled, thm_unscaled, tof_unscaled = [], [], []\n        imu_mask, thm_mask, tof_mask = [], [], []\n        classes, lens = [], []\n        self.imu_nan_value = self.get_nan_value(df[self.imu_cols], self.config[\"nan_ratio\"][\"imu\"])\n        self.thm_nan_value = self.get_nan_value(df[self.thm_cols], self.config[\"nan_ratio\"][\"thm\"])\n        self.tof_nan_value = self.get_nan_value(df[self.tof_cols], self.config[\"nan_ratio\"][\"tof\"])\n\n        self.fold_feats = defaultdict(list)\n        for seq_id, seq_df in seq_gp:\n            imu_data = seq_df[self.imu_cols]\n            if self.config[\"fbfill\"][\"imu\"]:\n                imu_data = imu_data.ffill().bfill()\n            imu_unscaled.append(imu_data.fillna(self.imu_nan_value).values.astype('float32'))\n\n            thm_data = seq_df[self.thm_cols]\n            if self.config[\"fbfill\"][\"thm\"]:\n                thm_data = thm_data.ffill().bfill()\n            thm_unscaled.append(thm_data.fillna(self.thm_nan_value).values.astype('float32'))\n\n            tof_data = seq_df[self.tof_cols]\n            if self.config[\"fbfill\"][\"tof\"]:\n                tof_data = tof_data.ffill().bfill()\n            tof_unscaled.append(tof_data.fillna(self.tof_nan_value).values.astype('float32'))\n            \n            classes.append(seq_df['gesture_int'].iloc[0])\n            lens.append(len(imu_data))\n\n            for col in self.fold_cols:\n                self.fold_feats[col].append(seq_df[col].iloc[0])\n            \n        self.dataset_indices = classes\n        self.pad_len = int(np.percentile(lens, self.config.get(\"percent\", 95)))\n        if self.config.get(\"one_scale\", True):\n            x_unscaled = [np.concatenate([imu, thm, tof], axis=1) for imu, thm, tof in zip(imu_unscaled, thm_unscaled, tof_unscaled)]\n            x_scaled, self.x_scaler = self.scale(x_unscaled)\n            x = self.pad(x_scaled, self.imu_cols+self.thm_cols+self.tof_cols)\n            self.imu = x[..., :self.imu_dim]\n            self.thm = x[..., self.imu_dim:self.imu_dim+self.thm_dim]\n            self.tof = x[..., self.imu_dim+self.thm_dim:self.imu_dim+self.thm_dim+self.tof_dim]\n        else:\n            imu_scaled, self.imu_scaler = self.scale(imu_unscaled)\n            thm_scaled, self.thm_scaler = self.scale(thm_unscaled)\n            tof_scaled, self.tof_scaler = self.scale(tof_unscaled)\n            self.imu = self.pad(imu_scaled, self.imu_cols)\n            self.thm = self.pad(thm_scaled, self.thm_cols)\n            self.tof = self.pad(tof_scaled, self.tof_cols)\n        self.precompute_scaled_nan_values()\n        self.class_ = F.one_hot(torch.from_numpy(np.array(classes)).long(), num_classes=len(self.le.classes_)).float().numpy()\n        self.class_weight = torch.FloatTensor(compute_class_weight('balanced', classes=np.arange(len(self.le.classes_)), y=classes))\n\n    def precompute_scaled_nan_values(self):\n        dummy_df = pd.DataFrame(\n            np.array([[self.imu_nan_value]*len(self.imu_cols) + \n                     [self.thm_nan_value]*len(self.thm_cols) +\n                     [self.tof_nan_value]*len(self.tof_cols)]),\n            columns=self.imu_cols + self.thm_cols + self.tof_cols\n        )\n        \n        if self.config.get(\"one_scale\", True):\n            scaled = self.x_scaler.transform(dummy_df)\n            self.imu_scaled_nan = scaled[0, :self.imu_dim].mean()\n            self.thm_scaled_nan = scaled[0, self.imu_dim:self.imu_dim+self.thm_dim].mean()\n            self.tof_scaled_nan = scaled[0, self.imu_dim+self.thm_dim:self.imu_dim+self.thm_dim+self.tof_dim].mean()\n        else:\n            self.imu_scaled_nan = self.imu_scaler.transform(dummy_df[self.imu_cols])[0].mean()\n            self.thm_scaled_nan = self.thm_scaler.transform(dummy_df[self.thm_cols])[0].mean()\n            self.tof_scaled_nan = self.tof_scaler.transform(dummy_df[self.tof_cols])[0].mean()\n\n    def get_scaled_nan_tensors(self, imu, thm, tof):\n        return torch.full(imu.shape, self.imu_scaled_nan, device=imu.device), \\\n            torch.full(thm.shape, self.thm_scaled_nan, device=thm.device), \\\n            torch.full(tof.shape, self.tof_scaled_nan, device=tof.device)\n\n    def inference_process(self, sequence):\n        df_seq = sequence.to_pandas().copy()\n        if not all(c in df_seq.columns for c in self.imu_engineered_features):\n            df_seq['acc_mag'] = np.sqrt(df_seq['acc_x']**2 + df_seq['acc_y']**2 + df_seq['acc_z']**2)\n            df_seq['rot_angle'] = 2 * np.arccos(df_seq['rot_w'].clip(-1, 1))\n            df_seq['acc_mag_jerk'] = df_seq['acc_mag'].diff().fillna(0)\n            df_seq['rot_angle_vel'] = df_seq['rot_angle'].diff().fillna(0)\n            if all(col in df_seq.columns for col in ['acc_x', 'acc_y', 'acc_z', 'rot_x', 'rot_y', 'rot_z', 'rot_w']):\n                linear_accel = remove_gravity_from_acc(\n                    df_seq[['acc_x', 'acc_y', 'acc_z']], \n                    df_seq[['rot_x', 'rot_y', 'rot_z', 'rot_w']]\n                )\n                df_seq[['linear_acc_x', 'linear_acc_y', 'linear_acc_z']] = linear_accel\n            else:\n                df_seq['linear_acc_x'] = df_seq.get('acc_x', 0)\n                df_seq['linear_acc_y'] = df_seq.get('acc_y', 0)\n                df_seq['linear_acc_z'] = df_seq.get('acc_z', 0)\n            df_seq['linear_acc_mag'] = np.sqrt(df_seq['linear_acc_x']**2 + df_seq['linear_acc_y']**2 + df_seq['linear_acc_z']**2)\n            df_seq['linear_acc_mag_jerk'] = df_seq['linear_acc_mag'].diff().fillna(0)\n            if all(col in df_seq.columns for col in ['rot_x', 'rot_y', 'rot_z', 'rot_w']):\n                angular_vel = calculate_angular_velocity_from_quat(df_seq[['rot_x', 'rot_y', 'rot_z', 'rot_w']])\n                df_seq[['angular_vel_x', 'angular_vel_y', 'angular_vel_z']] = angular_vel\n            else:\n                df_seq[['angular_vel_x', 'angular_vel_y', 'angular_vel_z']] = 0\n            if all(col in df_seq.columns for col in ['rot_x', 'rot_y', 'rot_z', 'rot_w']):\n                df_seq['angular_distance'] = calculate_angular_distance(df_seq[['rot_x', 'rot_y', 'rot_z', 'rot_w']])\n            else:\n                df_seq['angular_distance'] = 0\n\n        if self.tof_mode != 0:\n            new_columns = {} \n            for i in range(1, 6):\n                pixel_cols = [f\"tof_{i}_v{p}\" for p in range(64)]\n                tof_data = df_seq[pixel_cols].replace(-1, np.nan)\n                new_columns.update({\n                    f'tof_{i}_mean': tof_data.mean(axis=1),\n                    f'tof_{i}_std': tof_data.std(axis=1),\n                    f'tof_{i}_min': tof_data.min(axis=1),\n                    f'tof_{i}_max': tof_data.max(axis=1)\n                })\n                if self.tof_mode > 1:\n                    region_size = 64 // self.tof_mode\n                    for r in range(self.tof_mode):\n                        region_data = tof_data.iloc[:, r*region_size : (r+1)*region_size]\n                        new_columns.update({\n                            f'tof{self.tof_mode}_{i}_region_{r}_mean': region_data.mean(axis=1),\n                            f'tof{self.tof_mode}_{i}_region_{r}_std': region_data.std(axis=1),\n                            f'tof{self.tof_mode}_{i}_region_{r}_min': region_data.min(axis=1),\n                            f'tof{self.tof_mode}_{i}_region_{r}_max': region_data.max(axis=1)\n                        })\n                if self.tof_mode == -1:\n                    for mode in [2, 4, 8, 16, 32]:\n                        region_size = 64 // mode\n                        for r in range(mode):\n                            region_data = tof_data.iloc[:, r*region_size : (r+1)*region_size]\n                            new_columns.update({\n                                f'tof{mode}_{i}_region_{r}_mean': region_data.mean(axis=1),\n                                f'tof{mode}_{i}_region_{r}_std': region_data.std(axis=1),\n                                f'tof{mode}_{i}_region_{r}_min': region_data.min(axis=1),\n                                f'tof{mode}_{i}_region_{r}_max': region_data.max(axis=1)\n                            })\n            df_seq = pd.concat([df_seq, pd.DataFrame(new_columns)], axis=1)\n        \n        imu_unscaled = df_seq[self.imu_cols]\n        if self.config[\"fbfill\"][\"imu\"]:\n            imu_unscaled = imu_unscaled.ffill().bfill()\n        imu_unscaled = imu_unscaled.fillna(self.imu_nan_value).values.astype('float32')\n\n        thm_unscaled = df_seq[self.thm_cols]\n        if self.config[\"fbfill\"][\"thm\"]:\n            thm_unscaled = thm_unscaled.ffill().bfill()\n        thm_unscaled = thm_unscaled.fillna(self.thm_nan_value).values.astype('float32')\n\n        tof_unscaled = df_seq[self.tof_cols]\n        if self.config[\"fbfill\"][\"tof\"]:\n            tof_unscaled = tof_unscaled.ffill().bfill()\n        tof_unscaled = tof_unscaled.fillna(self.tof_nan_value).values.astype('float32')\n        \n        if self.config.get(\"one_scale\", True):\n            x_unscaled = np.concatenate([imu_unscaled, thm_unscaled, tof_unscaled], axis=1)\n            x_scaled = self.x_scaler.transform(x_unscaled)\n            imu_scaled = x_scaled[..., :self.imu_dim]\n            thm_scaled = x_scaled[..., self.imu_dim:self.imu_dim+self.thm_dim]\n            tof_scaled = x_scaled[..., self.imu_dim+self.thm_dim:self.imu_dim+self.thm_dim+self.tof_dim]\n        else:\n            imu_scaled = self.imu_scaler.transform(imu_unscaled)\n            thm_scaled = self.thm_scaler.transform(thm_unscaled)\n            tof_scaled = self.tof_scaler.transform(tof_unscaled)\n\n        combined = np.concatenate([imu_scaled, thm_scaled, tof_scaled], axis=1)\n        padded = np.zeros((self.pad_len, combined.shape[1]), dtype='float32')\n        seq_len = min(combined.shape[0], self.pad_len)\n        padded[:seq_len] = combined[:seq_len]\n        imu = padded[..., :self.imu_dim]\n        thm = padded[..., self.imu_dim:self.imu_dim+self.thm_dim]\n        tof = padded[..., self.imu_dim+self.thm_dim:self.imu_dim+self.thm_dim+self.tof_dim]\n        \n        return torch.from_numpy(imu).float().unsqueeze(0), torch.from_numpy(thm).float().unsqueeze(0), torch.from_numpy(tof).float().unsqueeze(0)\n\n    def __getitem__(self, idx):\n        return self.imu[idx], self.thm[idx], self.tof[idx], self.class_[idx]\n\n    def __len__(self):\n        return len(self.class_)\n\nclass CMIFoldDataset:\n    def __init__(self, data_path, config, full_dataset_function, n_folds=5, random_seed=0):\n        self.full_dataset = full_dataset_function(data_path=data_path, config=config)\n        self.imu_dim = self.full_dataset.imu_dim\n        self.thm_dim = self.full_dataset.thm_dim\n        self.tof_dim = self.full_dataset.tof_dim\n        self.le = self.full_dataset.le\n        self.class_names = self.full_dataset.le.classes_\n        self.class_weight = self.full_dataset.class_weight\n        all_indices = np.arange(len(self.full_dataset))\n        self.n_folds = n_folds\n        self.skf = StratifiedKFold(n_splits=n_folds, shuffle=True, random_state=random_seed)\n        self.folds = list(self.skf.split(all_indices, np.array(self.full_dataset.dataset_indices)))\n    \n    def get_fold_datasets(self, fold_idx):\n        if self.folds is None or fold_idx >= self.n_folds:\n            return None, None\n        fold_train_idx, fold_valid_idx = self.folds[fold_idx]\n        return Subset(self.full_dataset, fold_train_idx), Subset(self.full_dataset, fold_valid_idx)\n\n    def print_fold_stats(self):\n        def get_label_counts(subset):\n            counts = {name: 0 for name in self.class_names}\n            if subset is None:\n                return counts\n            for idx in subset.indices:\n                label_idx = self.full_dataset.dataset_indices[idx]\n                counts[self.class_names[label_idx]] += 1\n            return counts\n        \n        print(\"\\n交叉验证折叠统计:\")\n        for fold_idx in range(self.n_folds):\n            train_fold, valid_fold = self.get_fold_datasets(fold_idx)\n            train_counts = get_label_counts(train_fold)\n            valid_counts = get_label_counts(valid_fold)\n                \n            print(f\"\\nFold {fold_idx + 1}:\")\n            print(f\"{'类别':<50} {'训练集':<10} {'验证集':<10}\")\n            for name in self.class_names:\n                print(f\"{name:<50} {train_counts[name]:<10} {valid_counts[name]:<10}\")\n\n\nclass SEBlock(nn.Module):\n    def __init__(self, channels, reduction = 8):\n        super().__init__()\n        self.fc1 = nn.Linear(channels, channels // reduction, bias=True)\n        self.fc2 = nn.Linear(channels // reduction, channels, bias=True)\n        self.sigmoid = nn.Sigmoid()\n\n    def forward(self, x):\n        # x: (B, C, L)\n        se = F.adaptive_avg_pool1d(x, 1).squeeze(-1)      # -> (B, C)\n        se = F.relu(self.fc1(se), inplace=True)          # -> (B, C//r)\n        se = self.sigmoid(self.fc2(se)).unsqueeze(-1)    # -> (B, C, 1)\n        return x * se                \n\nclass ResNetSEBlock(nn.Module):\n    def __init__(self, in_channels, out_channels, wd = 1e-4):\n        super().__init__()\n        self.conv1 = nn.Conv1d(in_channels, out_channels,\n                               kernel_size=3, padding=1, bias=False)\n        self.bn1 = nn.BatchNorm1d(out_channels)\n        self.conv2 = nn.Conv1d(out_channels, out_channels,\n                               kernel_size=3, padding=1, bias=False)\n        self.bn2 = nn.BatchNorm1d(out_channels)\n        # SE\n        self.se = SEBlock(out_channels)\n        \n        if in_channels != out_channels:\n            self.shortcut = nn.Sequential(\n                nn.Conv1d(in_channels, out_channels, kernel_size=1,\n                          padding=0, bias=False),\n                nn.BatchNorm1d(out_channels)\n            )\n        else:\n            self.shortcut = nn.Identity()\n\n        self.relu = nn.ReLU(inplace=True)\n\n    def forward(self, x) :\n        identity = self.shortcut(x)              # (B, out, L)\n        out = self.relu(self.bn1(self.conv1(x)))\n        out = self.bn2(self.conv2(out))\n        out = self.se(out)                       # (B, out, L)\n        out = out + identity\n        return self.relu(out)\n\nclass CMIModel(nn.Module):\n    def __init__(self, imu_dim, thm_dim, tof_dim, n_classes, **kwargs):\n        super().__init__()\n        self.imu_branch = nn.Sequential(\n            self.residual_se_cnn_block(imu_dim, kwargs[\"imu1_channels\"], kwargs[\"imu1_layers\"],\n                                       drop=kwargs[\"imu1_dropout\"]),\n            self.residual_se_cnn_block(kwargs[\"imu1_channels\"], kwargs[\"feat_dim\"], kwargs[\"imu2_layers\"],\n                                       drop=kwargs[\"imu2_dropout\"])\n        )\n\n        self.thm_branch = nn.Sequential(\n            nn.Conv1d(thm_dim, kwargs[\"thm1_channels\"], kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm1d(kwargs[\"thm1_channels\"]),\n            nn.ReLU(inplace=True),\n            nn.MaxPool1d(2, ceil_mode=True),\n            nn.Dropout(kwargs[\"thm1_dropout\"]),\n            \n            nn.Conv1d(kwargs[\"thm1_channels\"], kwargs[\"feat_dim\"], kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm1d(kwargs[\"feat_dim\"]),\n            nn.ReLU(inplace=True),\n            nn.MaxPool1d(2, ceil_mode=True),\n            nn.Dropout(kwargs[\"thm2_dropout\"])\n        )\n        \n        self.tof_branch = nn.Sequential(\n            nn.Conv1d(tof_dim, kwargs[\"tof1_channels\"], kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm1d(kwargs[\"tof1_channels\"]),\n            nn.ReLU(inplace=True),\n            nn.MaxPool1d(2, ceil_mode=True),\n            nn.Dropout(kwargs[\"tof1_dropout\"]),\n            \n            nn.Conv1d(kwargs[\"tof1_channels\"], kwargs[\"feat_dim\"], kernel_size=3, padding=1, bias=False),\n            nn.BatchNorm1d(kwargs[\"feat_dim\"]),\n            nn.ReLU(inplace=True),\n            nn.MaxPool1d(2, ceil_mode=True),\n            nn.Dropout(kwargs[\"tof2_dropout\"])\n        )\n\n        self.cls_token = nn.Parameter(torch.zeros(1, 1, kwargs[\"feat_dim\"]))\n        self.bert = BertModel(BertConfig(\n            hidden_size=kwargs[\"feat_dim\"],\n            num_hidden_layers=kwargs[\"bert_layers\"],\n            num_attention_heads=kwargs[\"bert_heads\"],\n            intermediate_size=kwargs[\"feat_dim\"]*4\n        ))\n        \n        self.classifier = nn.Sequential(\n            nn.Linear(kwargs[\"feat_dim\"], kwargs[\"cls1_channels\"], bias=False),\n            nn.BatchNorm1d(kwargs[\"cls1_channels\"]),\n            nn.ReLU(inplace=True),\n            nn.Dropout(kwargs[\"cls1_dropout\"]),\n            nn.Linear(kwargs[\"cls1_channels\"], kwargs[\"cls2_channels\"], bias=False),\n            nn.BatchNorm1d(kwargs[\"cls2_channels\"]),\n            nn.ReLU(inplace=True),\n            nn.Dropout(kwargs[\"cls2_dropout\"]),\n            nn.Linear(kwargs[\"cls2_channels\"], n_classes)\n        )\n    \n    def residual_se_cnn_block(self, in_channels, out_channels, num_layers, pool_size=2, drop=0.3, wd=1e-4):\n        return nn.Sequential(\n            *[ResNetSEBlock(in_channels=in_channels, out_channels=in_channels) for i in range(num_layers)],\n            ResNetSEBlock(in_channels, out_channels, wd=wd),\n            nn.MaxPool1d(pool_size),\n            nn.Dropout(drop)\n        )\n    \n    def forward(self, imu, thm, tof):\n        imu_feat = self.imu_branch(imu.permute(0, 2, 1))\n        thm_feat = self.thm_branch(thm.permute(0, 2, 1))\n        tof_feat = self.tof_branch(tof.permute(0, 2, 1))\n        \n        bert_input = torch.cat([imu_feat, thm_feat, tof_feat], dim=-1).permute(0, 2, 1)\n        cls_token = self.cls_token.expand(bert_input.size(0), -1, -1)  # (B,1,H)\n        bert_input = torch.cat([cls_token, bert_input], dim=1)  # (B,T+1,H)\n        outputs = self.bert(inputs_embeds=bert_input)\n        pred_cls = outputs.last_hidden_state[:, 0, :]\n\n        return self.classifier(pred_cls)\n\n\nCUDA0 = \"cuda:0\"\nseed = 0\nbatch_size = 64\nnum_workers = 4\nn_folds = 5\n\nroot_dir = Path(\"/kaggle/input/cmi-detect-behavior-with-sensor-data\")\nuniverse_csv_path = Path(\"/kaggle/input/cmi-precompute/pytorch/all/1/tof-1_raw.csv\")\n\ndeterministic = kagglehub.package_import('wasupandceacar/deterministic').deterministic\ndeterministic.init_all(seed)\ndef init_dataset():\n    dataset_config = {\n        \"percent\": 95,\n        \"scaler_config\": StandardScaler(),\n        \"nan_ratio\": {\n            \"imu\": 0,\n            \"thm\": 0,\n            \"tof\": 0,\n        },\n        \"fbfill\": {\n            \"imu\": True,\n            \"thm\": True,\n            \"tof\": True,\n        },\n        \"one_scale\": True,\n        \"tof_raw\": True,\n        \"tof_mode\": 16,\n        \"save_precompute\": False,\n    }\n    dataset = CMIFoldDataset(universe_csv_path, dataset_config,\n                             n_folds=n_folds, random_seed=seed, full_dataset_function=CMIFeDataset)\n    dataset.print_fold_stats()\n    return dataset\n\ndef get_fold_dataset(dataset, fold):\n    _, valid_dataset = dataset.get_fold_datasets(fold)\n    valid_loader = DataLoader(valid_dataset, batch_size=batch_size, num_workers=num_workers, shuffle=False)\n    return valid_loader\n\ndataset = init_dataset()\n\nmodel_function = CMIModel\nmodel_args = {\"feat_dim\": 500,\n              \"imu1_channels\": 219, \"imu1_dropout\": 0.2946731587132302, \"imu2_dropout\": 0.2697745571929592,\n              \"imu1_weight_decay\": 0.0014824054650601245, \"imu2_weight_decay\": 0.002742543773142381,\n              \"imu1_layers\": 0, \"imu2_layers\": 0,\n              \"thm1_channels\": 82, \"thm1_dropout\": 0.2641274454844602, \"thm2_dropout\": 0.302896343020985, \n              \"tof1_channels\": 82, \"tof1_dropout\": 0.2641274454844602, \"tof2_dropout\": 0.3028963430209852, \n              \"bert_layers\": 8, \"bert_heads\": 10,\n              \"cls1_channels\": 937, \"cls2_channels\": 303, \"cls1_dropout\": 0.2281834512100508, \"cls2_dropout\": 0.22502521933558461}\nmodel_args.update({\n    \"imu_dim\": dataset.full_dataset.imu_dim, \n    \"thm_dim\": dataset.full_dataset.thm_dim,\n    \"tof_dim\": dataset.full_dataset.tof_dim,\n    \"n_classes\": dataset.full_dataset.class_num})\nmodel_dir = Path(\"/kaggle/input/cmi-models-public/pytorch/train_fold_model05_tof16_raw/1\")\n\nmodel_dicts = [\n    {\n        \"model_function\": model_function,\n        \"model_args\": model_args,\n        \"model_path\": model_dir / f\"fold{fold}/best_ema.pt\",\n    } for fold in range(n_folds)\n]\n\nmodels2 = list()\nfor model_dict in model_dicts:\n    model_function = model_dict[\"model_function\"]\n    model_args = model_dict[\"model_args\"]\n    model_path = model_dict[\"model_path\"]\n    model = model_function(**model_args).to(CUDA0)\n    state_dict = {k.replace(\"_orig_mod.\", \"\"): v for k,v in torch.load(model_path).items()}\n    model.load_state_dict(state_dict)\n    model = model.eval()\n    models2.append(model)\n\n\nmetric_package = kagglehub.package_import('wasupandceacar/cmi-metric')\n\nmetric = metric_package.Metric()\nimu_only_metric = metric_package.Metric()\n\ndef to_cuda(*tensors):\n    return [tensor.to(CUDA0) for tensor in tensors]\n\ndef predict_fold(model, imu, thm, tof):\n    pred = model(imu, thm, tof)\n    return pred\n\ndef valid(model, valid_bar):\n    with torch.no_grad():\n        for imu, thm, tof, y in valid_bar:\n            imu, thm, tof, y = to_cuda(imu, thm, tof, y)\n            with autocast(device_type='cuda', dtype=torch.bfloat16): \n                logits = predict_fold(model, imu, thm, tof)\n            metric.add(dataset.le.classes_[y.argmax(dim=1).cpu()], dataset.le.classes_[logits.argmax(dim=1).cpu()])\n            _, thm, tof = dataset.full_dataset.get_scaled_nan_tensors(imu, thm, tof)\n            with autocast(device_type='cuda', dtype=torch.bfloat16): \n                logits = model(imu, thm, tof)\n            imu_only_metric.add(dataset.le.classes_[y.argmax(dim=1).cpu()], dataset.le.classes_[logits.argmax(dim=1).cpu()])\n\n# for fold, model in enumerate(models2):\n#     valid_loader = get_fold_dataset(dataset, fold)\n#     valid_bar = tqdm(valid_loader, desc=f\"Valid\", position=0, leave=False)\n#     valid(model, valid_bar)\n\n# print(f\"\"\"\n# Normal score: {metric.score()}\n# IMU only score: {imu_only_metric.score()}\n# \"\"\")\n\ndef avg_predict(models, imu, thm, tof):\n    outputs = []\n    with autocast(device_type='cuda'):\n        for model in models:\n            logits = model(imu, thm, tof)\n        outputs.append(logits)\n    return torch.mean(torch.stack(outputs), dim=0)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-01T11:24:02.568638Z","iopub.execute_input":"2025-09-01T11:24:02.568960Z","iopub.status.idle":"2025-09-01T11:26:47.316692Z","shell.execute_reply.started":"2025-09-01T11:24:02.568933Z","shell.execute_reply":"2025-09-01T11:26:47.316146Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"###  🔢 Model 2 inference output\n\n**Convert the input sequence into IMU / Thermal / ToF three-modality tensors, then average-ensemble the five loaded PyTorch models (models2) and return the resulting softmax probability vector.**","metadata":{}},{"cell_type":"code","source":"def predict2(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\n    imu, thm, tof = dataset.full_dataset.inference_process(sequence)\n    with torch.no_grad():\n        imu, thm, tof = to_cuda(imu, thm, tof)\n        logits = avg_predict(models2, imu, thm, tof)\n        probabilities = F.softmax(logits, dim=1).cpu().numpy()\n    return probabilities","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-01T11:26:47.317499Z","iopub.execute_input":"2025-09-01T11:26:47.317769Z","iopub.status.idle":"2025-09-01T11:26:47.322528Z","shell.execute_reply.started":"2025-09-01T11:26:47.317743Z","shell.execute_reply":"2025-09-01T11:26:47.321773Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 🧱 Model 3(TensorFlow/Keras 和 PyTorch)\n\nHere is the English translation of the summary:\n\n---\n\n### 🔍 **Brief Summary**\n\n#### ✅ **Objective**\n- Predict human activities (e.g., gestures) using multi-sensor data (IMU + TOF/THM).\n- Perform **inference with ensemble modeling**, combining both custom and public high-performing models.\n\n---\n\n### 🧩 **Main Components**\n\n#### 1. **Environment Setup and Dependencies**\n- Imports essential libraries: `tensorflow`, `pytorch`, `sklearn`, `polars`, `scipy`, etc.\n- Sets random seeds for reproducibility.\n- Defines paths for raw data, model weights, and output directories.\n\n#### 2. **Hyperparameter Configuration**\n- Batch size, sequence padding length, learning rate, regularization, Mixup parameters, number of CV folds, etc.\n- Inference mode (`TRAIN = False`) — no training; only model loading and prediction.\n\n#### 3. **PyTorch Feature Extraction Module (ImuFeatureExtractor)**\n- Performs feature engineering on IMU data (accelerometer + gyroscope):\n  - Magnitude\n  - Jerk\n  - Energy\n  - High-/Low-pass filtering (HPF/LPF)\n- **Critical fix**: Adds the missing `self.lpf` layer to ensure compatibility with public model weights.\n\n#### 4. **PyTorch Model Architecture**\n- `SEBlock`: Squeeze-and-Excitation attention module.\n- `ResidualSECNNBlock`: Residual CNN block with SE module.\n- `AttentionLayer`: Temporal attention mechanism.\n- `TwoBranchModel`: Dual-branch model (IMU + TOF), using CNN + BiLSTM + Attention.\n- `PublicTwoBranchModel`: Architecture matching the public PyTorch models exactly, enabling correct weight loading.\n\n#### 5. **TensorFlow/Keras Model Architecture**\n- Custom layers: `residual_se_cnn_block`, `se_block`, `attention_layer`.\n- Builds a **Gated Two-Branch Model**:\n  - IMU branch: Deep CNN\n  - TOF branch: Lightweight CNN + **Gating mechanism** (to assess TOF validity)\n  - Fusion via **BiGRU** (not LSTM) + Attention + multiple dense layers\n  - Dual output: main classification + TOF gate signal\n\n#### 6. **Data Preprocessing & Feature Engineering Functions**\n- Remove gravity from acceleration using quaternions\n- Compute angular velocity and angular distance\n- Sequence padding function (compatible with PyTorch)\n\n#### 7. **Training Components (for TRAIN mode)**\n- `GatedMixupGenerator`: Data generator with Mixup and sensor masking.\n- Class weighting, early stopping, etc. (not used since training is disabled).\n\n---\n\n### 🚀 **Inference Pipeline (Core)**\n\nWhen `TRAIN = False`, the following steps are executed:\n\n#### 📦 Loads three groups of pre-trained models:\n\n| Model Group | Type | Source | Count | Features |\n|-----------|------|--------|-------|----------|\n| A | Custom TensorFlow | User-uploaded | 10-fold | Gated GRU + Mixup + custom features |\n| B | Public TensorFlow | Top LB solution | 1 model | Quaternion-aware, BiLSTM + Attention |\n| C | Public PyTorch | Public models (\"Model Group C\") | 5-fold | CNN + BiLSTM + Attention, requires exact architecture match |\n\n#### ⚙️ Each model group loads:\n- Model weights\n- Normalization `scaler`\n- Feature column names `feature_cols`\n- Maximum sequence length `pad_len`\n- Class label mapping\n\n---\n\n### 📤 Output (implied, not shown)\n- Test data is preprocessed (feature extraction, normalization, padding)\n- Forward pass through all three model groups\n- Final predictions obtained via **weighted averaging of model outputs (ensemble)**\n- Outputs final behavior class probabilities (for submission)\n\n---\n\n### ✅ **Key Features & Design Philosophy**\n1. **Multi-framework integration**: Combines TensorFlow and PyTorch models to leverage strengths of both.\n2. **Ensemble Modeling**: Fuses multiple independently trained models to improve robustness and accuracy.\n3. **Gating Mechanism**: In custom models, dynamically assesses the reliability of TOF signals.\n4. **Rich Feature Engineering**: Includes filtering, magnitude, energy, jerk, angular velocity, etc.\n5. **Precise Architecture Reproduction**: Accurately reconstructs public model structures (e.g., keeping unused but required `lpf` layer) for correct weight loading.\n\n---\n\n### 📌 One-Sentence Summary:\n> This is a **highly engineered, multi-model ensemble inference script for sensor-based activity recognition**, integrating custom and public models via both TensorFlow and PyTorch backends to achieve high-accuracy behavioral classification.\n\n---\n\nSuitable for the final submission phase of Kaggle-style competitions, emphasizing **model ensemble, robustness, and reproducibility**.","metadata":{}},{"cell_type":"code","source":"import os\nimport json\nimport joblib\nimport numpy as np\nimport pandas as pd\nfrom pathlib import Path\nimport warnings\nimport random\nimport math\nimport matplotlib.pyplot as plt\nimport polars as pl\nimport tensorflow as tf\nfrom tensorflow.keras import backend as K\nfrom tensorflow.keras.models import Model, load_model\nfrom tensorflow.keras.layers import (\n    Input, Conv1D, BatchNormalization, Activation, add, MaxPooling1D, Dropout,\n    Bidirectional, GRU, GlobalAveragePooling1D, Dense, Multiply, Reshape,\n    Lambda, Concatenate\n)\nfrom tensorflow.keras.optimizers import Adam as AdamTF\nfrom tensorflow.keras.regularizers import l2\nfrom tensorflow.keras.utils import Sequence, to_categorical, pad_sequences\nfrom tensorflow.keras.callbacks import EarlyStopping\nfrom tensorflow.keras.optimizers.schedules import CosineDecay\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import Dataset, DataLoader\nfrom torch.optim import Adam as AdamTorch\nfrom torch.optim.lr_scheduler import CosineAnnealingWarmRestarts\n\nfrom sklearn.model_selection import StratifiedGroupKFold\nfrom sklearn.preprocessing import StandardScaler, LabelEncoder\nfrom sklearn.utils.class_weight import compute_class_weight\nfrom scipy.spatial.transform import Rotation as R\nfrom scipy.signal import firwin\n\n# 評価メトリクスはローカル検証/学習時にのみインポート\ntry:\n    from cmi_2025_metric_copy_for_import import CompetitionMetric\nexcept ImportError:\n    CompetitionMetric = None\n    print(\"CompetitionMetric could not be imported. OOF/CV score will not be calculated.\")\n\ndef seed_everything(seed=42):\n    \"\"\"\n    実行環境の乱数シードを統一的に設定する関数。\n    \"\"\"\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    random.seed(seed)\n    np.random.seed(2025)\n    tf.random.set_seed(seed)\n    tf.experimental.numpy.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.cuda.manual_seed_all(seed)\n    os.environ['TF_CUDNN_DETERMINISTIC'] = '1'\n    os.environ['TF_DETERMINISTIC_OPS'] = '1'\n    # torch.backends.cudnn.deterministic = True # パフォーマンスが低下する可能性があるためコメントアウト\n    # torch.backends.cudnn.benchmark = False\n\nseed_everything(seed=42)\nwarnings.filterwarnings(\"ignore\")\n\nTRAIN = False\n\n# --- パス設定 ---\nRAW_DIR = Path(\"/kaggle/input/cmi-detect-behavior-with-sensor-data\")\n# YOUR_MODELS_DIRは自分の学習済みモデルが格納されているKaggleデータセットのパスに設定してください\nYOUR_MODELS_DIR = Path(\"/kaggle/input/cmi-data-gated-gru\") # ★★★ 自分のモデルパスに変更 ★★★\nPUBLIC_TF_MODEL_DIR = Path(\"/kaggle/input/lb-0-78-quaternions-tf-bilstm-gru-attention\")\nPUBLIC_PT_MODEL_DIR = Path(\"/kaggle/input/cmi3-models-p\")\nEXPORT_DIR = Path(\"./\") # 学習済みモデルやアーティファクトの保存先\n\n# --- モデル学習ハイパーパラメータ ---\nBATCH_SIZE = 64          # バッチサイズ\nPAD_PERCENTILE = 95      # シーケンス長のパディングを決めるためのパーセンタイル値\nLR_INIT = 4e-4           # 学習率の初期値 (微調整)\nWD = 3e-3                # Weight Decay（L2正則化）の係数\nMIXUP_ALPHA = 0.4        # Mixupのα値\nEPOCHS = 360             # 最大エポック数 (増加)\nPATIENCE = 50            # EarlyStoppingのpatience (増加)\nN_SPLITS = 10             # クロスバリデーションの分割数\nMASKING_PROB = 0.25      # 学習時にTOF/THMデータをマスクする確率\nGATE_LOSS_WEIGHT = 0.2   # Gatedモデルのゲート損失に対する重み\n\nprint(f\"▶ ライブラリのインポート完了\")\nprint(f\"  - TensorFlow: {tf.__version__}\")\nprint(f\"  - PyTorch: {torch.__version__}\")\nprint(f\"▶ TRAINモード: {TRAIN}\")\n\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n\n# PyTorchモデル用の標準化パラメータ\nmean_pt = torch.tensor([\n    0, 0, 0, 0, 0, 0, 9.0319e-03, 1.0849e+00, -2.6186e-03, 3.7651e-03,\n    -5.3660e-03, -2.8177e-03, 1.3318e-03, -1.5876e-04, 6.3495e-01,\n    6.2877e-01, 6.0607e-01, 6.2142e-01, 6.3808e-01, 6.5420e-01,\n    7.4102e-03, -3.4159e-03, -7.5237e-03, -2.6034e-02, 2.9704e-02,\n    -3.1546e-02, -2.0610e-03, -4.6986e-03, -4.7216e-03, -2.6281e-02,\n    1.5799e-02, 1.0016e-02\n], dtype=torch.float32).view(1, -1, 1).to(device)\n\nstd_pt = torch.tensor([\n    1, 1, 1, 1, 1, 1, 0.2067, 0.8583, 0.3162,\n    0.2668, 0.2917, 0.2341, 0.3023, 0.3281, 1.0264, 0.8838, 0.8686, 1.0973,\n    1.0267, 0.9018, 0.4658, 0.2009, 0.2057, 1.2240, 0.9535, 0.6655, 0.2941,\n    0.3421, 0.8156, 0.6565, 1.1034, 1.5577\n], dtype=torch.float32).view(1, -1, 1).to(device) + 1e-8\n\nclass ImuFeatureExtractor(nn.Module):\n    \"\"\"\n    ★★★ PyTorchモデル用の特徴量抽出器 ★★★\n    公開モデルの重みと一致させるため、元の正しい定義に修正。\n    \"\"\"\n    def __init__(self, fs=100., add_quaternion=False):\n        super().__init__()\n        self.fs = fs\n        self.add_quaternion = add_quaternion\n\n        k = 15\n\n        # ▼▼▼【ここが修正点】▼▼▼\n        # 公開モデルの重みファイルに存在する 'self.lpf' 層を再度追加する\n        self.lpf = nn.Conv1d(6, 6, kernel_size=k, padding=k//2,\n                                 groups=6, bias=False)\n        nn.init.kaiming_uniform_(self.lpf.weight, a=math.sqrt(5))\n        # ▲▲▲【ここまでが修正点】▲▲▲\n\n        self.lpf_acc  = nn.Conv1d(3, 3, k, padding=k//2, groups=3, bias=False)\n        self.lpf_gyro = nn.Conv1d(3, 3, k, padding=k//2, groups=3, bias=False)\n\n    def forward(self, imu):\n        acc  = imu[:, 0:3, :]\n        gyro = imu[:, 3:6, :]\n\n        # 1) magnitude\n        acc_mag  = torch.norm(acc,  dim=1, keepdim=True)\n        gyro_mag = torch.norm(gyro, dim=1, keepdim=True)\n\n        # 2) jerk\n        jerk = F.pad(acc[:, :, 1:] - acc[:, :, :-1], (1,0))\n        gyro_delta = F.pad(gyro[:, :, 1:] - gyro[:, :, :-1], (1,0))\n\n        # 3) energy\n        acc_pow  = acc ** 2\n        gyro_pow = gyro ** 2\n\n        # 4) LPF / HPF\n        # self.lpf は forwardパスでは使われていないが、重み読み込みのために定義が必要\n        acc_lpf  = self.lpf_acc(acc)\n        acc_hpf  = acc - acc_lpf\n        gyro_lpf = self.lpf_gyro(gyro)\n        gyro_hpf = gyro - gyro_lpf\n\n        features = [\n            acc, gyro,\n            acc_mag, gyro_mag,\n            jerk, gyro_delta,\n            acc_pow, gyro_pow,\n            acc_lpf, acc_hpf,\n            gyro_lpf, gyro_hpf,\n        ]\n        return torch.cat(features, dim=1)\n\nclass SEBlock(nn.Module):\n    def __init__(self, channels, reduction=8):\n        super().__init__()\n        self.squeeze = nn.AdaptiveAvgPool1d(1)\n        self.excitation = nn.Sequential(\n            nn.Linear(channels, channels // reduction, bias=False), nn.ReLU(inplace=True),\n            nn.Linear(channels // reduction, channels, bias=False), nn.Sigmoid()\n        )\n    def forward(self, x):\n        b, c, _ = x.size()\n        y = self.squeeze(x).view(b, c)\n        y = self.excitation(y).view(b, c, 1)\n        return x * y.expand_as(x)\n\nclass ResidualSECNNBlock(nn.Module):\n    def __init__(self, in_channels, out_channels, kernel_size, pool_size=2, dropout=0.3):\n        super().__init__()\n        self.conv1 = nn.Conv1d(in_channels, out_channels, kernel_size, padding=kernel_size//2, bias=False)\n        self.bn1 = nn.BatchNorm1d(out_channels)\n        self.conv2 = nn.Conv1d(out_channels, out_channels, kernel_size, padding=kernel_size//2, bias=False)\n        self.bn2 = nn.BatchNorm1d(out_channels)\n        self.se = SEBlock(out_channels)\n        self.shortcut = nn.Sequential()\n        if in_channels != out_channels:\n            self.shortcut = nn.Sequential(nn.Conv1d(in_channels, out_channels, 1, bias=False), nn.BatchNorm1d(out_channels))\n        self.pool = nn.MaxPool1d(pool_size)\n        self.dropout = nn.Dropout(dropout)\n    def forward(self, x):\n        out = F.relu(self.bn1(self.conv1(x)))\n        out = self.bn2(self.conv2(out))\n        out = self.se(out)\n        out += self.shortcut(x)\n        return self.dropout(self.pool(F.relu(out)))\n\nclass AttentionLayer(nn.Module):\n    def __init__(self, hidden_dim):\n        super().__init__()\n        self.attention = nn.Linear(hidden_dim, 1)\n    def forward(self, x):\n        scores = torch.tanh(self.attention(x))\n        weights = F.softmax(scores.squeeze(-1), dim=1)\n        return torch.sum(x * weights.unsqueeze(-1), dim=1)\n\nclass TwoBranchModel(nn.Module):\n    def __init__(self, pad_len, imu_dim_raw, tof_dim, n_classes, dropouts=[0.3, 0.3, 0.3, 0.3, 0.4, 0.5, 0.3], feature_engineering=True, **kwargs):\n        super().__init__()\n        self.feature_engineering = feature_engineering\n        imu_dim = 32 if feature_engineering else imu_dim_raw\n        self.imu_fe = ImuFeatureExtractor(**kwargs) if feature_engineering else nn.Identity()\n        self.fir_nchan = 7\n        numtaps = 33\n        fir_kernel = torch.tensor(firwin(numtaps, cutoff=1.0, fs=10.0, pass_zero=False), dtype=torch.float32).view(1, 1, -1).repeat(self.fir_nchan, 1, 1)\n        self.register_buffer(\"fir_kernel\", fir_kernel)\n        self.imu_block1 = ResidualSECNNBlock(imu_dim, 64, 3, dropout=dropouts[0])\n        self.imu_block2 = ResidualSECNNBlock(64, 128, 5, dropout=dropouts[1])\n        self.tof_conv1 = nn.Conv1d(tof_dim, 64, 3, padding=1, bias=False)\n        self.tof_bn1, self.tof_pool1, self.tof_drop1 = nn.BatchNorm1d(64), nn.MaxPool1d(2), nn.Dropout(dropouts[2])\n        self.tof_conv2 = nn.Conv1d(64, 128, 3, padding=1, bias=False)\n        self.tof_bn2, self.tof_pool2, self.tof_drop2 = nn.BatchNorm1d(128), nn.MaxPool1d(2), nn.Dropout(dropouts[3])\n        self.bilstm = nn.LSTM(256, 128, bidirectional=True, batch_first=True)\n        self.lstm_dropout = nn.Dropout(dropouts[4])\n        self.attention = AttentionLayer(256)\n        self.dense1, self.bn_dense1, self.drop1 = nn.Linear(256, 256, bias=False), nn.BatchNorm1d(256), nn.Dropout(dropouts[5])\n        self.dense2, self.bn_dense2, self.drop2 = nn.Linear(256, 128, bias=False), nn.BatchNorm1d(128), nn.Dropout(dropouts[6])\n        self.classifier = nn.Linear(128, n_classes)\n\n    def forward(self, x):\n        imu_raw = x[:, :, :self.fir_nchan].transpose(1, 2)\n        tof = x[:, :, self.fir_nchan:].transpose(1, 2)\n        imu_fe = self.imu_fe(imu_raw)\n        filtered = F.conv1d(imu_fe[:, :self.fir_nchan, :], self.fir_kernel, padding=self.fir_kernel.shape[-1] // 2, groups=self.fir_nchan)\n        imu = (torch.cat([filtered, imu_fe[:, self.fir_nchan:, :]], dim=1) - mean_pt) / std_pt\n        x1 = self.imu_block1(imu); x1 = self.imu_block2(x1)\n        x2 = self.tof_drop1(self.tof_pool1(F.relu(self.tof_bn1(self.tof_conv1(tof)))))\n        x2 = self.tof_drop2(self.tof_pool2(F.relu(self.tof_bn2(self.tof_conv2(x2)))))\n        merged = torch.cat([x1, x2], dim=1).transpose(1, 2)\n        lstm_out, _ = self.bilstm(merged); lstm_out = self.lstm_dropout(lstm_out)\n        attended = self.attention(lstm_out)\n        x = self.drop1(F.relu(self.bn_dense1(self.dense1(attended))))\n        x = self.drop2(F.relu(self.bn_dense2(self.dense2(x))))\n        return self.classifier(x)\n\nclass PublicTwoBranchModel(nn.Module):\n    \"\"\"\n    ★★★ 公開されているPyTorchモデル（モデル群C）を読み込むための、元のアーキテクチャを持つクラス ★★★\n    \"\"\"\n    def __init__(self, pad_len, imu_dim_raw, tof_dim, n_classes, dropouts=[0.3, 0.3, 0.3, 0.3, 0.4, 0.5, 0.3], feature_engineering=True, **kwargs):\n        super().__init__()\n        self.feature_engineering = feature_engineering\n        imu_dim = 32 if feature_engineering else imu_dim_raw\n        self.imu_fe = ImuFeatureExtractor(**kwargs) if feature_engineering else nn.Identity()\n        self.fir_nchan = 7\n        numtaps = 33\n        fir_kernel = torch.tensor(firwin(numtaps, cutoff=1.0, fs=10.0, pass_zero=False), dtype=torch.float32).view(1, 1, -1).repeat(self.fir_nchan, 1, 1)\n        self.register_buffer(\"fir_kernel\", fir_kernel)\n        self.imu_block1 = ResidualSECNNBlock(imu_dim, 64, 3, dropout=dropouts[0])\n        self.imu_block2 = ResidualSECNNBlock(64, 128, 5, dropout=dropouts[1])\n        self.tof_conv1 = nn.Conv1d(tof_dim, 64, 3, padding=1, bias=False)\n        self.tof_bn1, self.tof_pool1, self.tof_drop1 = nn.BatchNorm1d(64), nn.MaxPool1d(2), nn.Dropout(dropouts[2])\n        self.tof_conv2 = nn.Conv1d(64, 128, 3, padding=1, bias=False)\n        self.tof_bn2, self.tof_pool2, self.tof_drop2 = nn.BatchNorm1d(128), nn.MaxPool1d(2), nn.Dropout(dropouts[3])\n        self.bilstm = nn.LSTM(256, 128, bidirectional=True, batch_first=True) # GRUではなくLSTM\n        self.lstm_dropout = nn.Dropout(dropouts[4])\n        self.attention = AttentionLayer(256) # 128*2 for bidirectional\n        self.dense1, self.bn_dense1, self.drop1 = nn.Linear(256, 256, bias=False), nn.BatchNorm1d(256), nn.Dropout(dropouts[5])\n        self.dense2, self.bn_dense2, self.drop2 = nn.Linear(256, 128, bias=False), nn.BatchNorm1d(128), nn.Dropout(dropouts[6])\n        self.classifier = nn.Linear(128, n_classes)\n\n    def forward(self, x):\n        imu_raw = x[:, :, :self.fir_nchan].transpose(1, 2)\n        tof = x[:, :, self.fir_nchan:].transpose(1, 2)\n        imu_fe = self.imu_fe(imu_raw)\n        filtered = F.conv1d(imu_fe[:, :self.fir_nchan, :], self.fir_kernel, padding=self.fir_kernel.shape[-1] // 2, groups=self.fir_nchan)\n        # mean_pt, std_pt は事前に定義されているグローバル変数\n        imu = (torch.cat([filtered, imu_fe[:, self.fir_nchan:, :]], dim=1) - mean_pt) / std_pt\n        x1 = self.imu_block1(imu); x1 = self.imu_block2(x1)\n        x2 = self.tof_drop1(self.tof_pool1(F.relu(self.tof_bn1(self.tof_conv1(tof)))))\n        x2 = self.tof_drop2(self.tof_pool2(F.relu(self.tof_bn2(self.tof_conv2(x2)))))\n        merged = torch.cat([x1, x2], dim=1).transpose(1, 2)\n        lstm_out, _ = self.bilstm(merged); lstm_out = self.lstm_dropout(lstm_out)\n        attended = self.attention(lstm_out)\n        x = self.drop1(F.relu(self.bn_dense1(self.dense1(attended))))\n        x = self.drop2(F.relu(self.bn_dense2(self.dense2(x))))\n        return self.classifier(x)\n\ndef pad_sequences_torch3(sequences, maxlen, padding='post', truncating='post', value=0.0):\n    result = []\n    for seq in sequences:\n        if len(seq) >= maxlen: seq = seq[:maxlen] if truncating == 'post' else seq[-maxlen:]\n        else:\n            pad_len = maxlen - len(seq)\n            pad_array = np.full((pad_len, seq.shape[1]), value)\n            seq = np.concatenate([seq, pad_array]) if padding == 'post' else np.concatenate([pad_array, seq])\n        result.append(seq)\n    return np.array(result, dtype=np.float32)\n\n# =============================================================================\n# ## 特徴量エンジニアリング関数\n# =============================================================================\ndef remove_gravity_from_acc3(acc_data, rot_data):\n    \"\"\"加速度データから重力成分を除去する\"\"\"\n    acc_values = acc_data[['acc_x', 'acc_y', 'acc_z']].values\n    quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n    linear_accel = np.zeros_like(acc_values)\n    gravity_world = np.array([0, 0, 9.81])\n    for i in range(len(acc_values)):\n        if np.all(np.isnan(quat_values[i])):\n            linear_accel[i, :] = acc_values[i, :]\n            continue\n        try:\n            rotation = R.from_quat(quat_values[i])\n            gravity_sensor_frame = rotation.apply(gravity_world, inverse=True)\n            linear_accel[i, :] = acc_values[i, :] - gravity_sensor_frame\n        except (ValueError, IndexError):\n            linear_accel[i, :] = acc_values[i, :]\n    return linear_accel\n\ndef calculate_angular_velocity_from_quat3(rot_data, time_delta=1/200):\n    \"\"\"クォータニオンから角速度を計算する\"\"\"\n    quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n    angular_vel = np.zeros((len(quat_values), 3))\n    for i in range(len(quat_values) - 1):\n        q_t, q_t_plus_dt = quat_values[i], quat_values[i+1]\n        if np.all(np.isnan(q_t)) or np.all(np.isnan(q_t_plus_dt)): continue\n        try:\n            rot_t = R.from_quat(q_t)\n            rot_t_plus_dt = R.from_quat(q_t_plus_dt)\n            delta_rot = rot_t.inv() * rot_t_plus_dt\n            angular_vel[i, :] = delta_rot.as_rotvec() / time_delta\n        except (ValueError, IndexError): pass\n    return angular_vel\n\ndef calculate_angular_distance3(rot_data):\n    \"\"\"クォータニオンから角距離を計算する\"\"\"\n    quat_values = rot_data[['rot_x', 'rot_y', 'rot_z', 'rot_w']].values\n    angular_dist = np.zeros(len(quat_values))\n    for i in range(len(quat_values) - 1):\n        q1, q2 = quat_values[i], quat_values[i+1]\n        if np.all(np.isnan(q1)) or np.all(np.isnan(q2)): continue\n        try:\n            r1 = R.from_quat(q1)\n            r2 = R.from_quat(q2)\n            relative_rotation = r1.inv() * r2\n            angular_dist[i] = np.linalg.norm(relative_rotation.as_rotvec())\n        except (ValueError, IndexError): pass\n    return angular_dist\n\ndef time_sum(x): return K.sum(x, axis=1)\ndef squeeze_last_axis(x): return tf.squeeze(x, axis=-1)\ndef expand_last_axis(x): return tf.expand_dims(x, axis=-1)\n\ndef se_block(x, reduction=8):\n    \"\"\"Squeeze-and-Excitationブロック\"\"\"\n    ch = x.shape[-1]\n    se = GlobalAveragePooling1D()(x)\n    se = Dense(ch // reduction, activation='relu')(se)\n    se = Dense(ch, activation='sigmoid')(se)\n    se = Reshape((1, ch))(se)\n    return Multiply()([x, se])\n\ndef residual_se_cnn_block(x, filters, kernel_size, pool_size=2, drop=0.3, wd=1e-4):\n    \"\"\"Residual SE-CNNブロック\"\"\"\n    shortcut = x\n    # 2層のConv1D\n    for _ in range(2):\n        x = Conv1D(filters, kernel_size, padding='same', use_bias=False, kernel_regularizer=l2(wd))(x)\n        x = BatchNormalization()(x)\n        x = Activation('relu')(x)\n    # SEブロック\n    x = se_block(x)\n    # ショートカット接続\n    if shortcut.shape[-1] != filters:\n        shortcut = Conv1D(filters, 1, padding='same', use_bias=False, kernel_regularizer=l2(wd))(shortcut)\n        shortcut = BatchNormalization()(shortcut)\n    x = add([x, shortcut])\n    x = Activation('relu')(x)\n    x = MaxPooling1D(pool_size)(x)\n    x = Dropout(drop)(x)\n    return x\n\ndef attention_layer(inputs):\n    \"\"\"アテンション層\"\"\"\n    score = Dense(1, activation='tanh')(inputs)\n    score = Lambda(squeeze_last_axis)(score)\n    weights = Activation('softmax')(score)\n    weights = Lambda(expand_last_axis)(weights)\n    context = Multiply()([inputs, weights])\n    context = Lambda(time_sum)(context)\n    return context\n\nclass GatedMixupGenerator(Sequence):\n    \"\"\"Mixupとセンサーマスキングを適用するデータジェネレータ\"\"\"\n    def __init__(self, X, y, batch_size, imu_dim, class_weight=None, alpha=0.2, masking_prob=0.0):\n        self.X, self.y, self.batch, self.imu_dim = X, y, batch_size, imu_dim\n        self.class_weight, self.alpha, self.masking_prob = class_weight, alpha, masking_prob\n        self.indices = np.arange(len(X))\n\n    def __len__(self):\n        return int(np.ceil(len(self.X) / self.batch))\n\n    def __getitem__(self, i):\n        idx = self.indices[i*self.batch:(i+1)*self.batch]\n        Xb, yb = self.X[idx].copy(), self.y[idx].copy()\n\n        sample_weights = np.ones(len(Xb), dtype='float32')\n        if self.class_weight:\n            sample_weights = np.array([self.class_weight.get(i, 1.0) for i in yb.argmax(axis=1)])\n\n        gate_target = np.ones(len(Xb), dtype='float32')\n        if self.masking_prob > 0:\n            for j in range(len(Xb)):\n                if np.random.rand() < self.masking_prob:\n                    Xb[j, :, self.imu_dim:] = 0\n                    gate_target[j] = 0.0\n\n        if self.alpha > 0:\n            lam = np.random.beta(self.alpha, self.alpha)\n            perm = np.random.permutation(len(Xb))\n            X_mix = lam * Xb + (1 - lam) * Xb[perm]\n            y_mix = lam * yb + (1 - lam) * yb[perm]\n            gate_target_mix = lam * gate_target + (1 - lam) * gate_target[perm]\n            sample_weights_mix = lam * sample_weights + (1 - lam) * sample_weights[perm]\n            return X_mix, {'main_output': y_mix, 'tof_gate': gate_target_mix}, sample_weights_mix\n\n        return Xb, {'main_output': yb, 'tof_gate': gate_target}, sample_weights\n\n    def on_epoch_end(self):\n        np.random.shuffle(self.indices)\n\ndef build_gated_two_branch_model(pad_len, imu_dim, tof_dim, n_classes, wd=1e-4):\n    \"\"\"\n    自作のGated Two-Branchモデルを構築する関数。\n    [改良点] LSTMをGRUに変更、全結合層を1層追加。\n    \"\"\"\n    inp = Input(shape=(pad_len, imu_dim + tof_dim))\n    imu = Lambda(lambda t: t[:, :, :imu_dim])(inp)\n    tof = Lambda(lambda t: t[:, :, imu_dim:])(inp)\n\n    # IMUブランチ (Deep)\n    x1 = residual_se_cnn_block(imu, 64, 3, drop=0.1, wd=wd)\n    x1 = residual_se_cnn_block(x1, 128, 5, drop=0.1, wd=wd)\n\n    # TOF/THMブランチ (Light) with Gating\n    x2_base = Conv1D(64, 3, padding='same', use_bias=False, kernel_regularizer=l2(wd))(tof)\n    x2_base = BatchNormalization()(x2_base); x2_base = Activation('relu')(x2_base)\n    x2_base = MaxPooling1D(2)(x2_base); x2_base = Dropout(0.2)(x2_base)\n    x2_base = Conv1D(128, 3, padding='same', use_bias=False, kernel_regularizer=l2(wd))(x2_base)\n    x2_base = BatchNormalization()(x2_base); x2_base = Activation('relu')(x2_base)\n    x2_base = MaxPooling1D(2)(x2_base); x2_base = Dropout(0.2)(x2_base)\n\n    # Gating機構\n    gate_input = GlobalAveragePooling1D()(tof)\n    gate_input = Dense(16, activation='relu')(gate_input)\n    gate = Dense(1, activation='sigmoid', name='tof_gate')(gate_input)\n    x2 = Multiply()([x2_base, gate])\n\n    # ブランチのマージと後続層\n    merged = Concatenate()([x1, x2])\n    # ★改良点: LSTM -> GRU\n    x = Bidirectional(GRU(256, return_sequences=True, kernel_regularizer=l2(wd)))(merged)\n    x = Dropout(0.45)(x)\n    x = attention_layer(x)\n\n    # ★改良点: 全結合層を1層追加して表現力を向上\n    for units, drop in [(512, 0.5), (256, 0.4), (128, 0.3)]:\n        x = Dense(units, use_bias=False, kernel_regularizer=l2(wd))(x)\n        x = BatchNormalization()(x)\n        x = Activation('relu')(x)\n        x = Dropout(drop)(x)\n\n    out = Dense(n_classes, activation='softmax', name='main_output', kernel_regularizer=l2(wd))(x)\n\n    return Model(inputs=inp, outputs=[out, gate])\n\n# -----------------------------------------------------------------------------\n# ### 推論モード (`TRAIN = False`)\n# -----------------------------------------------------------------------------\n\nprint(\"▶ 推論モード開始 – 学習済みモデルとアーティファクトを読み込みます...\")\n\n# --- モデル群A (自作TF/Kerasモデル) の読み込み ---\nprint(\"  モデル群A (自作5-Fold Gated GRUモデル) を読み込み中...\")\nfinal_feature_cols_A = np.load(YOUR_MODELS_DIR / \"final_feature_cols.npy\", allow_pickle=True).tolist()\npad_len_A = int(np.load(YOUR_MODELS_DIR / \"sequence_maxlen.npy\"))\nscaler_A = joblib.load(YOUR_MODELS_DIR / \"scaler.pkl\")\ngesture_classes = np.load(YOUR_MODELS_DIR / \"gesture_classes.npy\", allow_pickle=True)\ncustom_objs_A = {'time_sum': time_sum, 'squeeze_last_axis': squeeze_last_axis, 'expand_last_axis': expand_last_axis,\n                 'se_block': se_block, 'residual_se_cnn_block': residual_se_cnn_block, 'attention_layer': attention_layer}\nmodels_A = [load_model(YOUR_MODELS_DIR / f\"final_model_fold_{f}.h5\", compile=False, custom_objects=custom_objs_A) for f in range(N_SPLITS)]\nprint(f\"  > {len(models_A)}個のモデルを正常に読み込みました。\")\n\n# --- モデル群B (公開TF/Kerasモデル) の読み込み ---\nprint(\"\\n  モデル群B (公開TF/Kerasモデル) を読み込み中...\")\nfinal_feature_cols_B = np.load(PUBLIC_TF_MODEL_DIR / \"feature_cols.npy\", allow_pickle=True).tolist()\npad_len_B = int(np.load(PUBLIC_TF_MODEL_DIR / \"sequence_maxlen.npy\"))\nscaler_B = joblib.load(PUBLIC_TF_MODEL_DIR / \"scaler.pkl\")\ncustom_objs_B = custom_objs_A # public modelも同じカスタムオブジェクトを使用\nmodel_B = load_model(PUBLIC_TF_MODEL_DIR / \"gesture_two_branch_mixup.h5\", compile=False, custom_objects=custom_objs_B)\nprint(\"  > 1個のモデルを正常に読み込みました。\")\n\n# --- モデル群C (公開PyTorchモデル) の読み込み ---\nprint(\"\\n  モデル群C (公開PyTorchモデル) を読み込み中...\")\nfinal_feature_cols_C = np.load(PUBLIC_PT_MODEL_DIR / \"feature_cols.npy\", allow_pickle=True).tolist()\npad_len_C = int(np.load(PUBLIC_PT_MODEL_DIR / \"sequence_maxlen.npy\"))\nscaler_C = joblib.load(PUBLIC_PT_MODEL_DIR / \"scaler.pkl\")\n\npt_models = []\nfor f in range(5):\n    checkpoint = torch.load(PUBLIC_PT_MODEL_DIR / f\"gesture_two_branch_fold{f}.pth\", map_location=device)\n    cfg = {'pad_len': checkpoint['pad_len'], 'imu_dim_raw': checkpoint['imu_dim'],\n           'tof_dim': checkpoint['tof_dim'], 'n_classes': checkpoint['n_classes']}\n    m = PublicTwoBranchModel(**cfg).to(device)\n    m.load_state_dict(checkpoint['model_state_dict'])\n    m.eval()\n    pt_models.append(m)\nprint(f\"  > {len(pt_models)}個のモデルを正常に読み込みました。\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-01T11:26:47.324666Z","iopub.execute_input":"2025-09-01T11:26:47.324922Z","iopub.status.idle":"2025-09-01T11:26:53.733479Z","shell.execute_reply.started":"2025-09-01T11:26:47.324906Z","shell.execute_reply":"2025-09-01T11:26:53.732832Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### 🔢 Model 2 inference output\n\nThe `predict3` function fuses three models (A: Keras, B: TensorFlow, C: PyTorch) to predict on sensor time-series data. It extracts features like acceleration, angular velocity, and TOF/THM, feeds them into each model, then computes a weighted average of predictions  to output the final probability.","metadata":{}},{"cell_type":"code","source":"# --- `predict`関数の定義 ---\ndef predict3(sequence: pl.DataFrame, demographics: pl.DataFrame) -> str:\n    df_seq_orig = sequence.to_pandas()\n    df_seq_A = df_seq_orig.copy()\n    \n    linear_accel_A = remove_gravity_from_acc3(df_seq_A[['acc_x','acc_y','acc_z']], df_seq_A[['rot_x','rot_y','rot_z','rot_w']])\n    df_seq_A['linear_acc_x'], df_seq_A['linear_acc_y'], df_seq_A['linear_acc_z'] = linear_accel_A[:,0], linear_accel_A[:,1], linear_accel_A[:,2]\n    df_seq_A['linear_acc_mag'] = np.linalg.norm(linear_accel_A, axis=1)\n    df_seq_A['linear_acc_mag_jerk'] = df_seq_A['linear_acc_mag'].diff().fillna(0)\n    angular_vel_A = calculate_angular_velocity_from_quat3(df_seq_A[['rot_x','rot_y','rot_z','rot_w']])\n    df_seq_A['angular_vel_x'], df_seq_A['angular_vel_y'], df_seq_A['angular_vel_z'] = angular_vel_A[:,0], angular_vel_A[:,1], angular_vel_A[:,2]\n    df_seq_A['angular_distance'] = calculate_angular_distance3(df_seq_A[['rot_x','rot_y','rot_z','rot_w']])\n    for col in ['rot_x', 'rot_y', 'rot_z', 'rot_w']:\n        df_seq_A[f'{col}_diff'] = df_seq_A[col].diff().fillna(0)\n    cols_for_stats=['linear_acc_mag','linear_acc_mag_jerk','angular_distance']\n    for col in cols_for_stats:\n        df_seq_A[f'{col}_skew'], df_seq_A[f'{col}_kurt'] = df_seq_A[col].skew(), df_seq_A[col].kurtosis()\n    for i in range(1,6):\n        if f'tof_{i}_v0' in df_seq_A.columns:\n            pixel_cols=[f\"tof_{i}_v{p}\" for p in range(64)]; tof_data=df_seq_A[pixel_cols].replace(-1,np.nan)\n            df_seq_A[f'tof_{i}_mean'], df_seq_A[f'tof_{i}_std'], df_seq_A[f'tof_{i}_min'], df_seq_A[f'tof_{i}_max'] = tof_data.mean(axis=1),tof_data.std(axis=1),tof_data.min(axis=1),tof_data.max(axis=1)\n    tof_mean_cols=[f'tof_{i}_mean' for i in range(1,6) if f'tof_{i}_mean' in df_seq_A.columns]\n    if tof_mean_cols:\n        df_seq_A['tof_std_across_sensors']=df_seq_A[tof_mean_cols].std(axis=1)\n        df_seq_A['tof_range_across_sensors']=df_seq_A[tof_mean_cols].max(axis=1)-df_seq_A[tof_mean_cols].min(axis=1)\n    thm_cols=[f'thm_{i}' for i in range(1,6) if f'thm_{i}' in df_seq_A.columns]\n    if thm_cols:\n        df_seq_A['thm_std_across_sensors']=df_seq_A[thm_cols].std(axis=1)\n        df_seq_A['thm_range_across_sensors']=df_seq_A[thm_cols].max(axis=1)-df_seq_A[thm_cols].min(axis=1)\n    # (推論 A)\n    mat_A = df_seq_A[final_feature_cols_A].ffill().bfill().fillna(0).values.astype('float32')\n    mat_A = scaler_A.transform(mat_A)\n    pad_input_A = pad_sequences([mat_A], maxlen=pad_len_A, padding='post', dtype='float32')\n    preds_A_folds = [model.predict(pad_input_A, verbose=0)[0] for model in models_A]\n\n    # --- 2. モデル群B (公開TFモデル) の予測 ---\n    df_seq_B = df_seq_orig.copy()\n    # (特徴量生成 B)\n    df_seq_B['acc_mag']=np.sqrt(df_seq_B['acc_x']**2+df_seq_B['acc_y']**2+df_seq_B['acc_z']**2)\n    df_seq_B['rot_angle']=2*np.arccos(df_seq_B['rot_w'].clip(-1,1))\n    df_seq_B['acc_mag_jerk']=df_seq_B['acc_mag'].diff().fillna(0)\n    df_seq_B['rot_angle_vel']=df_seq_B['rot_angle'].diff().fillna(0)\n    linear_accel_B=remove_gravity_from_acc3(df_seq_B,df_seq_B)\n    df_seq_B['linear_acc_x'],df_seq_B['linear_acc_y'],df_seq_B['linear_acc_z']=linear_accel_B[:,0],linear_accel_B[:,1],linear_accel_B[:,2]\n    df_seq_B['linear_acc_mag']=np.sqrt(df_seq_B['linear_acc_x']**2+df_seq_B['linear_acc_y']**2+df_seq_B['linear_acc_z']**2)\n    df_seq_B['linear_acc_mag_jerk']=df_seq_B['linear_acc_mag'].diff().fillna(0)\n    angular_vel_B=calculate_angular_velocity_from_quat3(df_seq_B)\n    df_seq_B['angular_vel_x'],df_seq_B['angular_vel_y'],df_seq_B['angular_vel_z']=angular_vel_B[:,0],angular_vel_B[:,1],angular_vel_B[:,2]\n    df_seq_B['angular_distance']=calculate_angular_distance3(df_seq_B)\n    for i in range(1,6):\n        if f'tof_{i}_v0' in df_seq_B.columns:\n            pixel_cols=[f\"tof_{i}_v{p}\" for p in range(64)]; tof_data=df_seq_B[pixel_cols].replace(-1,np.nan)\n            df_seq_B[f\"tof_{i}_mean\"],df_seq_B[f\"tof_{i}_std\"],df_seq_B[f\"tof_{i}_min\"],df_seq_B[f\"tof_{i}_max\"]=tof_data.mean(axis=1),tof_data.std(axis=1),tof_data.min(axis=1),tof_data.max(axis=1)\n    # (推論 B)\n    mat_B = df_seq_B[final_feature_cols_B].ffill().bfill().fillna(0).values.astype('float32')\n    mat_B = scaler_B.transform(mat_B)\n    pad_input_B = pad_sequences([mat_B], maxlen=pad_len_B, padding='post', dtype='float32')\n\n    # --- 3. モデル群C (公開PyTorchモデル) の予測 ---\n    df_seq_C = df_seq_orig.copy() # Cは特徴量生成が不要なため、コピーのみ\n    mat_C = df_seq_C[final_feature_cols_C].ffill().bfill().fillna(0).values.astype('float32')\n    mat_C = scaler_C.transform(mat_C)\n    pad_input_C = pad_sequences_torch3([mat_C], maxlen=pad_len_C, padding='pre', truncating='pre')\n   \n    with torch.no_grad():\n        pt_input = torch.from_numpy(pad_input_C).to(device)\n        preds_C_folds = [model(pt_input) for model in pt_models]\n\n    avg_pred_A = np.mean(preds_A_folds, axis=0)\n    avg_pred_C_logits = torch.median(torch.stack(preds_C_folds), dim=0).values\n    avg_pred_C = torch.softmax(avg_pred_C_logits, dim=1).cpu().numpy()\n    pred_B = model_B.predict(pad_input_B, verbose=0)\n    if isinstance(pred_B, list): pred_B = pred_B[0]\n    \n\n    weights = {'A': 0.535,  'B': 0.1775,   'C': 0.2875}\n\n    return (weights['A']* avg_pred_A+\n            weights['B']* pred_B    +\n            weights['C']* avg_pred_C)\n            ","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-01T11:26:53.734289Z","iopub.execute_input":"2025-09-01T11:26:53.734884Z","iopub.status.idle":"2025-09-01T11:26:53.753907Z","shell.execute_reply.started":"2025-09-01T11:26:53.734856Z","shell.execute_reply":"2025-09-01T11:26:53.753348Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 🎯Submission\n\n### Model Ensemble Prediction Code Explanation\n\n#### Overview  \nThis code implements an ensemble prediction system for a classification task, combining outputs from three sub-models (`predict1`, `predict2`, `predict3`) using a custom weighted fusion method `h_blend`. The final prediction is obtained through weighted aggregation, and the corresponding class label is returned via inverse label encoding.\n\n#### Core Method: Order-Aware Weighted Fusion  \nThe `h_blend` function applies not only base model weights (`m_w`) but also correction weights (`c_w`) based on the ascending and descending order of the three predictions per sample. Smaller predictions receive positive adjustments, while larger ones are slightly penalized, improving robustness. The results from both sorting directions are combined using direction weights `da_w` (70% ascending, 30% descending).\n\n#### Prediction Output  \nThe class with the highest fused score is selected via `argmax()` and mapped back to the original label using `dataset.le.classes_`.\n\n#### Note  \nThere is a bug: in the descending sum calculation, the loop variable uses `asc` instead of `desc`, which should be corrected. Despite this, the design reflects a sophisticated, distribution-sensitive ensemble strategy that enhances model stability and performance.","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport copy\nimport os\nimport pandas as pd\ndef h_blend(pred0,pred1,pred2, m_w,da_w,c_w):\n    m_wts, preds = np.asarray(m_w), []\n    for a,b,c in zip(pred0,pred1,pred2):    \n        l_abc = [\n            { 'wts':m_wts[0], 'pred':a, 'res':0 },\n            { 'wts':m_wts[1], 'pred':b, 'res':0 },\n            { 'wts':m_wts[2], 'pred':c, 'res':0 },\n        ]\n        l_asc  = sorted(copy.deepcopy(l_abc), key=lambda _:_['pred'],reverse=False)\n        l_desc = sorted(copy.deepcopy(l_abc), key=lambda _:_['pred'],reverse=True) \n        for asc, c_wts in zip(l_asc, c_w): asc ['res'] = asc ['pred'] * (asc ['wts'] +c_wts)\n        for desc,c_wts in zip(l_desc,c_w): desc['res'] = desc['pred'] * (desc['wts'] +c_wts)\n        result_asc  = sum([asc ['res'] for asc in l_asc])\n        result_desc = sum([desc['res'] for asc in l_desc])\n        result = result_asc * da_w[0] + da_w[1] * result_desc\n        preds.append(result)\n    avg_pred =  np.asarray(preds)\n    return avg_pred\n\n\ndef find_max_val(one_avg_pred):\n    max_val = -float('inf')\n    for i, val in enumerate(one_avg_pred):\n        if val > max_val:\n            max_val = val\n    return max_val\n\n\ndef predict(sequence, demographics):\n    pred0 = predict1(sequence, demographics)[0]\n    pred1 = predict2(sequence, demographics)[0]\n    pred3 = predict3(sequence, demographics)[0]\n    \n    m_w,da_w = [0.271, 0.347, 0.382], [0.70, 0.30]\n\n    avg_p = h_blend(pred0,pred1,pred3, m_w,da_w,c_w=[+0.0021,-0.000537,-0.001563])\n\n    return dataset.le.classes_[avg_p.argmax()]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-01T11:26:53.754786Z","iopub.execute_input":"2025-09-01T11:26:53.755058Z","iopub.status.idle":"2025-09-01T11:26:53.777473Z","shell.execute_reply.started":"2025-09-01T11:26:53.755029Z","shell.execute_reply":"2025-09-01T11:26:53.776960Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 🚀Kaggle CMI Inference Pipeline with Local Testing and Submission Generation","metadata":{}},{"cell_type":"code","source":"import kaggle_evaluation.cmi_inference_server\ninference_server = kaggle_evaluation.cmi_inference_server.CMIInferenceServer(predict)\n\nif os.getenv('KAGGLE_IS_COMPETITION_RERUN'):\n    inference_server.serve()\nelse:\n    inference_server.run_local_gateway(\n        data_paths=(\n            '/kaggle/input/cmi-detect-behavior-with-sensor-data/test.csv',\n            '/kaggle/input/cmi-detect-behavior-with-sensor-data/test_demographics.csv',\n        )\n    )\n\nif not os.getenv('KAGGLE_IS_COMPETITION_RERUN'):\n    print(pd.read_parquet(\"submission.parquet\"))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-09-01T11:26:53.778141Z","iopub.execute_input":"2025-09-01T11:26:53.778349Z","iopub.status.idle":"2025-09-01T11:27:49.980038Z","shell.execute_reply.started":"2025-09-01T11:26:53.778327Z","shell.execute_reply":"2025-09-01T11:27:49.979364Z"}},"outputs":[],"execution_count":null}]}