{
  "id": 603677,
  "title": "7th Place Solution",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/writeups/7th-place-solution",
  "author_name": "",
  "post_date": "2025-09-03T16:22:42.700Z",
  "votes": 39,
  "comment_count": 7,
  "views": 0,
  "content": "<p>First of all, I'd like to thank the organizers for hosting such an interesting competition. There were many different modeling options to explore, and I learned a great deal through trial and error.</p>\n<p>My approach uses a <strong>2D CNN</strong>, inspired by a top solution from the HMS competition I previously participated in.</p>\n<hr>\n<h2><strong>Overview</strong></h2>\n<ul>\n<li><strong>2D CNN Approach</strong>: I converted the time-series data into images and processed them using either EfficientNet or ConvNeXt.</li>\n<li><strong>Hierarchical Multi-task Learning</strong>: The model has a main 9-class classification task and three auxiliary tasks (a detailed 18-class classification, orientation, and behavior).<ul>\n<li>The <code>behavior</code> is predicted as a time series from the backbone's intermediate feature maps. This prediction is then used as an additional feature for the main and other auxiliary classification tasks.</li></ul></li>\n<li><strong>Exponential Moving Average (EMA)</strong>: Used for more stable training.</li>\n<li><strong>Data Augmentation</strong>: I used a variety of augmentations, including time-series specific ones (like Time Warping) and Mixup that was only applied between samples of the same orientation.</li>\n<li><strong>Post-processing</strong>: A final step was applied to ensure that each subject in the test set had an equal number of assigned classes.</li>\n</ul>\n<hr>\n<h2><strong>Preprocessing</strong></h2>\n<h4><strong>IMU Data (12 dims → 41 dims)</strong></h4>\n<ul>\n<li><strong>Base Features</strong>: acc (3), rot (4), thm (5).</li>\n<li><strong>Engineered Features</strong>:<ul>\n<li>Linear acceleration after removing gravity (calculated using quaternions).</li>\n<li>Angular velocity and angular distance.</li>\n<li>Jerk (the derivative of acceleration) and its magnitude.</li>\n<li>Frequency domain features (power in the 0-1Hz, 1-5Hz, and 5-10Hz bands).</li>\n<li>Detection of direction changes (using cosine similarity between consecutive frames).</li></ul></li>\n<li>Corrected for handedness by inverting specific axes.</li>\n<li>Normalized all features using <code>StandardScaler</code>.</li>\n</ul>\n<h4><strong>TOF Data (320 dims)</strong></h4>\n<ul>\n<li>The data structure was <code>(seq_len, 5*64=320)</code>.</li>\n<li>Corrected for sensor rotation and left/right differences.</li>\n<li>I found that arranging the TOF sensor data to match their physical proximity improved accuracy. The best order was:<ul>\n<li>TOF2 → TOF1 → TOF3 → TOF5 → TOF4</li></ul></li>\n</ul>\n<hr>\n<h2><strong>Data Transformation</strong></h2>\n<ol>\n<li><strong>Crop</strong>: Took the last 140 steps from each sequence.</li>\n<li><strong>Resample</strong>: Resampled the 140 steps to 512 steps.</li>\n<li><strong>Resize</strong>: Treated the resulting <code>(512 steps, 361 channels)</code> data as an image and resized it to <code>(512, 1024)</code> using <code>cv2.resize</code>.</li>\n</ol>\n<hr>\n<h2><strong>Model</strong></h2>\n<h4><strong>Architecture</strong></h4>\n<ul>\n<li><strong>IMU only</strong>: ConvNeXt-base (pre-trained on ImageNet).</li>\n<li><strong>IMU+THM+TOF</strong>: EfficientNet-B5 (pre-trained on ImageNet).</li>\n<li>Used multi-scale feature extraction from stages -1, -2, and -3 of the backbone.</li>\n<li>The model has multi-head outputs:<ul>\n<li><strong>Main Cls Head</strong> (9 classes)</li>\n<li><strong>Auxiliary Cls Head</strong> (18 classes)</li>\n<li><strong>Orientation Head</strong> (4 classes)</li>\n<li><strong>Behavior Head</strong> (4 classes x 64 steps, temporal)</li></ul></li>\n<li>The behavior prediction was generated as a 64-step time series from an intermediate feature map. The resulting 4-class probability values for each step were added as features for the main and other aux task.</li>\n</ul>\n<hr>\n<h2><strong>Others</strong></h2>\n<h4><strong>Data Augmentation</strong></h4>\n<ul>\n<li><strong>Time-series Augmentations</strong>: Time Warping, Magnitude Warping, Jittering, Scaling, and Window Slicing.</li>\n<li><strong>Mixup</strong>: Applied only between samples that shared the same orientation.</li>\n<li><strong>Rotation Data Dropout</strong>: Randomly set quaternion features to zero with a 20% probability.</li>\n<li><strong>Variable-length Cropping during Training</strong>: Used shorter sequences (70-200 steps) and longer sequences (120-700 steps) to handle variability.</li>\n</ul>\n<h4><strong>Training Configuration</strong></h4>\n<ul>\n<li><strong>Optimizer</strong>: AdamW (<code>lr=1e-3</code>, <code>weight_decay=1e-4</code>)</li>\n<li><strong>Batch Size</strong>: 32</li>\n<li><strong>Scheduler</strong>: Cosine Annealing with a 150-step warm-up.</li>\n<li><strong>EMA</strong>: decay=0.99.</li>\n</ul>\n<h4><strong>Post-processing</strong></h4>\n<ul>\n<li>I applied a post-processing step to equalize the number of predicted classes for each subject in the test set.</li>\n<li>I did not have time to implement a more complex version that also considered <code>(subject, orientation)</code>.</li>\n<li>The score improvement from this post-processing was <strong>+0.004 CV</strong>, <strong>+0.01 Public LB</strong>, and <strong>+0.005 Private LB</strong>.</li>\n</ul>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>CV</th>\n<th>Public</th>\n<th>Private</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Without postprocess (ensemble)</td>\n<td>0.884</td>\n<td>0.872</td>\n<td>0.864</td>\n</tr>\n<tr>\n<td>With postprocess (ensemble)</td>\n<td>0.888</td>\n<td>0.882</td>\n<td>0.869</td>\n</tr>\n</tbody>\n</table>\n<ul>\n<li>The CV score is calculated excluding <code>SUBJ_045235</code> and <code>SUBJ_019262</code>.</li>\n</ul>",
  "messages": [
    {
      "id": "3281100",
      "postDate": "09/03/2025 16:21:22",
      "content": "<p>First of all, I'd like to thank the organizers for hosting such an interesting competition. There were many different modeling options to explore, and I learned a great deal through trial and error.</p>\n<p>My approach uses a <strong>2D CNN</strong>, inspired by a top solution from the HMS competition I previously participated in.</p>\n<hr>\n<h2><strong>Overview</strong></h2>\n<ul>\n<li><strong>2D CNN Approach</strong>: I converted the time-series data into images and processed them using either EfficientNet or ConvNeXt.</li>\n<li><strong>Hierarchical Multi-task Learning</strong>: The model has a main 9-class classification task and three auxiliary tasks (a detailed 18-class classification, orientation, and behavior).<ul>\n<li>The <code>behavior</code> is predicted as a time series from the backbone's intermediate feature maps. This prediction is then used as an additional feature for the main and other auxiliary classification tasks.</li></ul></li>\n<li><strong>Exponential Moving Average (EMA)</strong>: Used for more stable training.</li>\n<li><strong>Data Augmentation</strong>: I used a variety of augmentations, including time-series specific ones (like Time Warping) and Mixup that was only applied between samples of the same orientation.</li>\n<li><strong>Post-processing</strong>: A final step was applied to ensure that each subject in the test set had an equal number of assigned classes.</li>\n</ul>\n<hr>\n<h2><strong>Preprocessing</strong></h2>\n<h4><strong>IMU Data (12 dims → 41 dims)</strong></h4>\n<ul>\n<li><strong>Base Features</strong>: acc (3), rot (4), thm (5).</li>\n<li><strong>Engineered Features</strong>:<ul>\n<li>Linear acceleration after removing gravity (calculated using quaternions).</li>\n<li>Angular velocity and angular distance.</li>\n<li>Jerk (the derivative of acceleration) and its magnitude.</li>\n<li>Frequency domain features (power in the 0-1Hz, 1-5Hz, and 5-10Hz bands).</li>\n<li>Detection of direction changes (using cosine similarity between consecutive frames).</li></ul></li>\n<li>Corrected for handedness by inverting specific axes.</li>\n<li>Normalized all features using <code>StandardScaler</code>.</li>\n</ul>\n<h4><strong>TOF Data (320 dims)</strong></h4>\n<ul>\n<li>The data structure was <code>(seq_len, 5*64=320)</code>.</li>\n<li>Corrected for sensor rotation and left/right differences.</li>\n<li>I found that arranging the TOF sensor data to match their physical proximity improved accuracy. The best order was:<ul>\n<li>TOF2 → TOF1 → TOF3 → TOF5 → TOF4</li></ul></li>\n</ul>\n<hr>\n<h2><strong>Data Transformation</strong></h2>\n<ol>\n<li><strong>Crop</strong>: Took the last 140 steps from each sequence.</li>\n<li><strong>Resample</strong>: Resampled the 140 steps to 512 steps.</li>\n<li><strong>Resize</strong>: Treated the resulting <code>(512 steps, 361 channels)</code> data as an image and resized it to <code>(512, 1024)</code> using <code>cv2.resize</code>.</li>\n</ol>\n<hr>\n<h2><strong>Model</strong></h2>\n<h4><strong>Architecture</strong></h4>\n<ul>\n<li><strong>IMU only</strong>: ConvNeXt-base (pre-trained on ImageNet).</li>\n<li><strong>IMU+THM+TOF</strong>: EfficientNet-B5 (pre-trained on ImageNet).</li>\n<li>Used multi-scale feature extraction from stages -1, -2, and -3 of the backbone.</li>\n<li>The model has multi-head outputs:<ul>\n<li><strong>Main Cls Head</strong> (9 classes)</li>\n<li><strong>Auxiliary Cls Head</strong> (18 classes)</li>\n<li><strong>Orientation Head</strong> (4 classes)</li>\n<li><strong>Behavior Head</strong> (4 classes x 64 steps, temporal)</li></ul></li>\n<li>The behavior prediction was generated as a 64-step time series from an intermediate feature map. The resulting 4-class probability values for each step were added as features for the main and other aux task.</li>\n</ul>\n<hr>\n<h2><strong>Others</strong></h2>\n<h4><strong>Data Augmentation</strong></h4>\n<ul>\n<li><strong>Time-series Augmentations</strong>: Time Warping, Magnitude Warping, Jittering, Scaling, and Window Slicing.</li>\n<li><strong>Mixup</strong>: Applied only between samples that shared the same orientation.</li>\n<li><strong>Rotation Data Dropout</strong>: Randomly set quaternion features to zero with a 20% probability.</li>\n<li><strong>Variable-length Cropping during Training</strong>: Used shorter sequences (70-200 steps) and longer sequences (120-700 steps) to handle variability.</li>\n</ul>\n<h4><strong>Training Configuration</strong></h4>\n<ul>\n<li><strong>Optimizer</strong>: AdamW (<code>lr=1e-3</code>, <code>weight_decay=1e-4</code>)</li>\n<li><strong>Batch Size</strong>: 32</li>\n<li><strong>Scheduler</strong>: Cosine Annealing with a 150-step warm-up.</li>\n<li><strong>EMA</strong>: decay=0.99.</li>\n</ul>\n<h4><strong>Post-processing</strong></h4>\n<ul>\n<li>I applied a post-processing step to equalize the number of predicted classes for each subject in the test set.</li>\n<li>I did not have time to implement a more complex version that also considered <code>(subject, orientation)</code>.</li>\n<li>The score improvement from this post-processing was <strong>+0.004 CV</strong>, <strong>+0.01 Public LB</strong>, and <strong>+0.005 Private LB</strong>.</li>\n</ul>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>CV</th>\n<th>Public</th>\n<th>Private</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Without postprocess (ensemble)</td>\n<td>0.884</td>\n<td>0.872</td>\n<td>0.864</td>\n</tr>\n<tr>\n<td>With postprocess (ensemble)</td>\n<td>0.888</td>\n<td>0.882</td>\n<td>0.869</td>\n</tr>\n</tbody>\n</table>\n<ul>\n<li>The CV score is calculated excluding <code>SUBJ_045235</code> and <code>SUBJ_019262</code>.</li>\n</ul>",
      "rawMarkdown": "First of all, I'd like to thank the organizers for hosting such an interesting competition. There were many different modeling options to explore, and I learned a great deal through trial and error.\n\nMy approach uses a **2D CNN**, inspired by a top solution from the HMS competition I previously participated in.\n\n---\n## **Overview**\n\n* **2D CNN Approach**: I converted the time-series data into images and processed them using either EfficientNet or ConvNeXt.\n* **Hierarchical Multi-task Learning**: The model has a main 9-class classification task and three auxiliary tasks (a detailed 18-class classification, orientation, and behavior).\n    * The `behavior` is predicted as a time series from the backbone's intermediate feature maps. This prediction is then used as an additional feature for the main and other auxiliary classification tasks.\n* **Exponential Moving Average (EMA)**: Used for more stable training.\n* **Data Augmentation**: I used a variety of augmentations, including time-series specific ones (like Time Warping) and Mixup that was only applied between samples of the same orientation.\n* **Post-processing**: A final step was applied to ensure that each subject in the test set had an equal number of assigned classes.\n\n---\n## **Preprocessing**\n\n#### **IMU Data (12 dims → 41 dims)**\n* **Base Features**: acc (3), rot (4), thm (5).\n* **Engineered Features**:\n    * Linear acceleration after removing gravity (calculated using quaternions).\n    * Angular velocity and angular distance.\n    * Jerk (the derivative of acceleration) and its magnitude.\n    * Frequency domain features (power in the 0-1Hz, 1-5Hz, and 5-10Hz bands).\n    * Detection of direction changes (using cosine similarity between consecutive frames).\n* Corrected for handedness by inverting specific axes.\n* Normalized all features using `StandardScaler`.\n\n#### **TOF Data (320 dims)**\n* The data structure was `(seq_len, 5*64=320)`.\n* Corrected for sensor rotation and left/right differences.\n* I found that arranging the TOF sensor data to match their physical proximity improved accuracy. The best order was:\n    * TOF2 → TOF1 → TOF3 → TOF5 → TOF4\n\n---\n## **Data Transformation**\n\n1.  **Crop**: Took the last 140 steps from each sequence.\n2.  **Resample**: Resampled the 140 steps to 512 steps.\n3.  **Resize**: Treated the resulting `(512 steps, 361 channels)` data as an image and resized it to `(512, 1024)` using `cv2.resize`.\n\n---\n## **Model**\n\n#### **Architecture**\n* **IMU only**: ConvNeXt-base (pre-trained on ImageNet).\n* **IMU+THM+TOF**: EfficientNet-B5 (pre-trained on ImageNet).\n* Used multi-scale feature extraction from stages -1, -2, and -3 of the backbone.\n* The model has multi-head outputs:\n    * **Main Cls Head** (9 classes)\n    * **Auxiliary Cls Head** (18 classes)\n    * **Orientation Head** (4 classes)\n    * **Behavior Head** (4 classes x 64 steps, temporal)\n* The behavior prediction was generated as a 64-step time series from an intermediate feature map. The resulting 4-class probability values for each step were added as features for the main and other aux task.\n\n---\n## **Others**\n\n#### **Data Augmentation**\n* **Time-series Augmentations**: Time Warping, Magnitude Warping, Jittering, Scaling, and Window Slicing.\n* **Mixup**: Applied only between samples that shared the same orientation.\n* **Rotation Data Dropout**: Randomly set quaternion features to zero with a 20% probability.\n* **Variable-length Cropping during Training**: Used shorter sequences (70-200 steps) and longer sequences (120-700 steps) to handle variability.\n\n#### **Training Configuration**\n* **Optimizer**: AdamW (`lr=1e-3`, `weight_decay=1e-4`)\n* **Batch Size**: 32\n* **Scheduler**: Cosine Annealing with a 150-step warm-up.\n* **EMA**: decay=0.99.\n\n#### **Post-processing**\n* I applied a post-processing step to equalize the number of predicted classes for each subject in the test set.\n* I did not have time to implement a more complex version that also considered `(subject, orientation)`.\n* The score improvement from this post-processing was **+0.004 CV**, **+0.01 Public LB**, and **+0.005 Private LB**.\n\n|                     | CV    | Public | Private |\n|---------------------|-------|--------|---------|\n| Without postprocess (ensemble) | 0.884 | 0.872  | 0.864   |\n| With postprocess (ensemble)    | 0.888 | 0.882  | 0.869   |\n\n* The CV score is calculated excluding `SUBJ_045235` and `SUBJ_019262`.",
      "votes": null
    },
    {
      "id": "3281133",
      "postDate": "09/03/2025 17:29:02",
      "content": "<p>Can you please elaborate, how you turn the 1D data to 2D images?</p>",
      "rawMarkdown": "Can you please elaborate, how you turn the 1D data to 2D images?",
      "votes": null
    },
    {
      "id": "3281272",
      "postDate": "09/04/2025 01:04:40",
      "content": "<p>Hi ishikei,<br>\nThanks for the writeup. This is definitely a very elegant approach. Could you please elaborate on how you applied a post-processing step to equalize the number of predicted classes for each subject in the test set? Many thanks!</p>",
      "rawMarkdown": "Hi ishikei,\nThanks for the writeup. This is definitely a very elegant approach. Could you please elaborate on how you applied a post-processing step to equalize the number of predicted classes for each subject in the test set? Many thanks!",
      "votes": null
    },
    {
      "id": "3281323",
      "postDate": "09/04/2025 03:48:48",
      "content": "<p>Simply concatenate IMU and TOF features along the feature_dim, and treat them as image data with ch_size=1.</p>\n<pre><code>\n\nimg = np.concatenate([imu, tof], axis=) \nimg = cv2.resize(img, (, )) \nimg = img.reshape(, , ) \n</code></pre>",
      "rawMarkdown": "Simply concatenate IMU and TOF features along the feature_dim, and treat them as image data with ch_size=1.\n\n```python\n# imu.shape = (seq_len, imu_ch)\n# tof.shape = (seq_len, tof_ch)\nimg = np.concatenate([imu, tof], axis=1) # (seq_len, imu_ch+tof_ch)\nimg = cv2.resize(img, (1024, 512)) # (seq_len=512, feat_size=1024)\nimg = img.reshape(1, 512, 1024) # (1, seq_len, feat_size) as (ch, h, w)\n```",
      "votes": null
    },
    {
      "id": "3281442",
      "postDate": "09/04/2025 10:00:27",
      "content": "<p>Hello, congrats for reaching 7th place.<br>\nWhat is the advantage of turning the channels into a spatial dimension?<br>\nDoes it mean that you have a single channel in the input?<br>\nDo you leave the kernel size of your feature extractors to their original size?</p>",
      "rawMarkdown": "Hello, congrats for reaching 7th place.\nWhat is the advantage of turning the channels into a spatial dimension?\nDoes it mean that you have a single channel in the input?\nDo you leave the kernel size of your feature extractors to their original size?",
      "votes": null
    },
    {
      "id": "3281882",
      "postDate": "09/05/2025 06:40:08",
      "content": "<p>When I saw this competition, I also considered 1D to 2D data, but this way is a time-cost method so I gave up. Could you please reveal how many time will your training cost? <a href=\"https://www.kaggle.com/amanatsu\" target=\"_blank\">@amanatsu</a> And this method is what I considered about 1D to 2D, hoping this could be helpful! <a href=\"https://www.kaggle.com/competitions/birdclef-2024/discussion/511763\" target=\"_blank\">https://www.kaggle.com/competitions/birdclef-2024/discussion/511763</a></p>",
      "rawMarkdown": "When I saw this competition, I also considered 1D to 2D data, but this way is a time-cost method so I gave up. Could you please reveal how many time will your training cost? @amanatsu And this method is what I considered about 1D to 2D, hoping this could be helpful! https://www.kaggle.com/competitions/birdclef-2024/discussion/511763",
      "votes": null
    },
    {
      "id": "3283978",
      "postDate": "09/08/2025 04:00:51",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/amanatsu\" target=\"_blank\">@amanatsu</a> cool solution, do you have code to share?</p>",
      "rawMarkdown": "Hi, @amanatsu cool solution, do you have code to share?",
      "votes": null
    },
    {
      "id": "3285960",
      "postDate": "09/09/2025 01:03:33",
      "content": "<p>Nice, I also participated in HMS challenge and only used pure computer vision models, and now in this competition, I am also using computer vision models. 😆</p>",
      "rawMarkdown": "Nice, I also participated in HMS challenge and only used pure computer vision models, and now in this competition, I am also using computer vision models. 😆",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3281133,
      "author_name": "shlomoron",
      "author_url": "",
      "post_date": "09/03/2025 17:29:02",
      "content": "<p>Can you please elaborate, how you turn the 1D data to 2D images?</p>",
      "votes": null,
      "replies": [
        {
          "id": 3281323,
          "author_name": "amanatsu",
          "author_url": "",
          "post_date": "09/04/2025 03:48:48",
          "content": "<p>Simply concatenate IMU and TOF features along the feature_dim, and treat them as image data with ch_size=1.</p>\n<pre><code>\n\nimg = np.concatenate([imu, tof], axis=) \nimg = cv2.resize(img, (, )) \nimg = img.reshape(, , ) \n</code></pre>",
          "votes": null,
          "replies": [
            {
              "id": 3281442,
              "author_name": "mauroabidalcarrer",
              "author_url": "",
              "post_date": "09/04/2025 10:00:27",
              "content": "<p>Hello, congrats for reaching 7th place.<br>\nWhat is the advantage of turning the channels into a spatial dimension?<br>\nDoes it mean that you have a single channel in the input?<br>\nDo you leave the kernel size of your feature extractors to their original size?</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 3281882,
          "author_name": "kurisew",
          "author_url": "",
          "post_date": "09/05/2025 06:40:08",
          "content": "<p>When I saw this competition, I also considered 1D to 2D data, but this way is a time-cost method so I gave up. Could you please reveal how many time will your training cost? <a href=\"https://www.kaggle.com/amanatsu\" target=\"_blank\">@amanatsu</a> And this method is what I considered about 1D to 2D, hoping this could be helpful! <a href=\"https://www.kaggle.com/competitions/birdclef-2024/discussion/511763\" target=\"_blank\">https://www.kaggle.com/competitions/birdclef-2024/discussion/511763</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3281272,
      "author_name": "hexoaiagent",
      "author_url": "",
      "post_date": "09/04/2025 01:04:40",
      "content": "<p>Hi ishikei,<br>\nThanks for the writeup. This is definitely a very elegant approach. Could you please elaborate on how you applied a post-processing step to equalize the number of predicted classes for each subject in the test set? Many thanks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3283978,
      "author_name": "zhangyue199",
      "author_url": "",
      "post_date": "09/08/2025 04:00:51",
      "content": "<p>Hi, <a href=\"https://www.kaggle.com/amanatsu\" target=\"_blank\">@amanatsu</a> cool solution, do you have code to share?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3285960,
      "author_name": "quan0095",
      "author_url": "",
      "post_date": "09/09/2025 01:03:33",
      "content": "<p>Nice, I also participated in HMS challenge and only used pure computer vision models, and now in this competition, I am also using computer vision models. 😆</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3281100": "First of all, I'd like to thank the organizers for hosting such an interesting competition. There were many different modeling options to explore, and I learned a great deal through trial and error.\n\nMy approach uses a **2D CNN**, inspired by a top solution from the HMS competition I previously participated in.\n\n---\n## **Overview**\n\n* **2D CNN Approach**: I converted the time-series data into images and processed them using either EfficientNet or ConvNeXt.\n* **Hierarchical Multi-task Learning**: The model has a main 9-class classification task and three auxiliary tasks (a detailed 18-class classification, orientation, and behavior).\n    * The `behavior` is predicted as a time series from the backbone's intermediate feature maps. This prediction is then used as an additional feature for the main and other auxiliary classification tasks.\n* **Exponential Moving Average (EMA)**: Used for more stable training.\n* **Data Augmentation**: I used a variety of augmentations, including time-series specific ones (like Time Warping) and Mixup that was only applied between samples of the same orientation.\n* **Post-processing**: A final step was applied to ensure that each subject in the test set had an equal number of assigned classes.\n\n---\n## **Preprocessing**\n\n#### **IMU Data (12 dims → 41 dims)**\n* **Base Features**: acc (3), rot (4), thm (5).\n* **Engineered Features**:\n    * Linear acceleration after removing gravity (calculated using quaternions).\n    * Angular velocity and angular distance.\n    * Jerk (the derivative of acceleration) and its magnitude.\n    * Frequency domain features (power in the 0-1Hz, 1-5Hz, and 5-10Hz bands).\n    * Detection of direction changes (using cosine similarity between consecutive frames).\n* Corrected for handedness by inverting specific axes.\n* Normalized all features using `StandardScaler`.\n\n#### **TOF Data (320 dims)**\n* The data structure was `(seq_len, 5*64=320)`.\n* Corrected for sensor rotation and left/right differences.\n* I found that arranging the TOF sensor data to match their physical proximity improved accuracy. The best order was:\n    * TOF2 → TOF1 → TOF3 → TOF5 → TOF4\n\n---\n## **Data Transformation**\n\n1.  **Crop**: Took the last 140 steps from each sequence.\n2.  **Resample**: Resampled the 140 steps to 512 steps.\n3.  **Resize**: Treated the resulting `(512 steps, 361 channels)` data as an image and resized it to `(512, 1024)` using `cv2.resize`.\n\n---\n## **Model**\n\n#### **Architecture**\n* **IMU only**: ConvNeXt-base (pre-trained on ImageNet).\n* **IMU+THM+TOF**: EfficientNet-B5 (pre-trained on ImageNet).\n* Used multi-scale feature extraction from stages -1, -2, and -3 of the backbone.\n* The model has multi-head outputs:\n    * **Main Cls Head** (9 classes)\n    * **Auxiliary Cls Head** (18 classes)\n    * **Orientation Head** (4 classes)\n    * **Behavior Head** (4 classes x 64 steps, temporal)\n* The behavior prediction was generated as a 64-step time series from an intermediate feature map. The resulting 4-class probability values for each step were added as features for the main and other aux task.\n\n---\n## **Others**\n\n#### **Data Augmentation**\n* **Time-series Augmentations**: Time Warping, Magnitude Warping, Jittering, Scaling, and Window Slicing.\n* **Mixup**: Applied only between samples that shared the same orientation.\n* **Rotation Data Dropout**: Randomly set quaternion features to zero with a 20% probability.\n* **Variable-length Cropping during Training**: Used shorter sequences (70-200 steps) and longer sequences (120-700 steps) to handle variability.\n\n#### **Training Configuration**\n* **Optimizer**: AdamW (`lr=1e-3`, `weight_decay=1e-4`)\n* **Batch Size**: 32\n* **Scheduler**: Cosine Annealing with a 150-step warm-up.\n* **EMA**: decay=0.99.\n\n#### **Post-processing**\n* I applied a post-processing step to equalize the number of predicted classes for each subject in the test set.\n* I did not have time to implement a more complex version that also considered `(subject, orientation)`.\n* The score improvement from this post-processing was **+0.004 CV**, **+0.01 Public LB**, and **+0.005 Private LB**.\n\n|                     | CV    | Public | Private |\n|---------------------|-------|--------|---------|\n| Without postprocess (ensemble) | 0.884 | 0.872  | 0.864   |\n| With postprocess (ensemble)    | 0.888 | 0.882  | 0.869   |\n\n* The CV score is calculated excluding `SUBJ_045235` and `SUBJ_019262`.",
    "3281133": "Can you please elaborate, how you turn the 1D data to 2D images?",
    "3281272": "Hi ishikei,\nThanks for the writeup. This is definitely a very elegant approach. Could you please elaborate on how you applied a post-processing step to equalize the number of predicted classes for each subject in the test set? Many thanks!",
    "3281323": "Simply concatenate IMU and TOF features along the feature_dim, and treat them as image data with ch_size=1.\n\n```python\n# imu.shape = (seq_len, imu_ch)\n# tof.shape = (seq_len, tof_ch)\nimg = np.concatenate([imu, tof], axis=1) # (seq_len, imu_ch+tof_ch)\nimg = cv2.resize(img, (1024, 512)) # (seq_len=512, feat_size=1024)\nimg = img.reshape(1, 512, 1024) # (1, seq_len, feat_size) as (ch, h, w)\n```",
    "3281442": "Hello, congrats for reaching 7th place.\nWhat is the advantage of turning the channels into a spatial dimension?\nDoes it mean that you have a single channel in the input?\nDo you leave the kernel size of your feature extractors to their original size?",
    "3281882": "When I saw this competition, I also considered 1D to 2D data, but this way is a time-cost method so I gave up. Could you please reveal how many time will your training cost? @amanatsu And this method is what I considered about 1D to 2D, hoping this could be helpful! https://www.kaggle.com/competitions/birdclef-2024/discussion/511763",
    "3283978": "Hi, @amanatsu cool solution, do you have code to share?",
    "3285960": "Nice, I also participated in HMS challenge and only used pure computer vision models, and now in this competition, I am also using computer vision models. 😆"
  },
  "source": "meta"
}