{ "id": 603542, "title": "5th place solution", "url": "/competitions/cmi-detect-behavior-with-sensor-data/writeups/5th-solution", "author_name": "", "post_date": "2025-09-03T00:19:28.820Z", "votes": 79, "comment_count": 6, "views": 0, "content": "

First, congrats for all winners and thanks to the organizers for hosting this interesting competition, the community-we learn a lot from your works and sharings. Thanks to my teammates(Rib~) for the excellent teamwork.

\n

Tricks

\n

It is obvious that this competition involves some tricks, and our solution mainly focuses on the following two points.
\n1.Figure out how to use the other sequences' information with the same subject. For convenience in subsequent descriptions, we will refer to this model as the subject-based. We group data from the same subject into a single batch, extract sequential information using CNN, RNN or transformer, and then add a Transformer layer along the subject dimension to capture inter-sequence relationships. During offline validation and online inference, you need to accumulate each subject’s historical sequence information before making predictions on the current sequence.
\n\"\"

\n

Since the evaluation API provides data in batches of single sequence rather than all sequences from the same subject, and additionally shuffles the order of sequences for each submission, this approach leads to some instability. We address this issue in the following ways:

\n\n

2.We prepared 4 types of models:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
modeltraining approch
imu-only_subjectuse imu features only, train subject-base
imu-only_sequenceuse imu features only, train sequence-base
all_subjectuse all features, train subject-base
all_sequenceuse all features, train sequence-base
\n

We adopt the following strategies for model selection and ensemble:

\n\n
df_test = df_hist\nn_hist = (df_hist)\nw_imu_tf = (n_hist,)/\nw_all_tf = (n_hist,)/\nn_samples = sequence\n sequence()/n_samples>=:\n     subject_predictions = (imu-only_subject_model) \n     sequence_predictions = (imu-only_sequence_model)\n     predictions = w_imu_tf*subject_predictions + (-w_imu_tf)*sequence_predictions \n:\n     imu_predictions = (imu-only_subject_model)\n     all_subject_predictions = (all_subject_model)\n     all_sequence_predictions = (all_sequence_model)\n     predictions = w_all_tf*all_subject_predictions + (-w_all_tf)*all_sequence_predictions\n     predictions = predictions*+imu_predictions*\n
\n

Others

\n
    \n
  1. Features & Data augment: we use the features and augments mainly from the public codes, thanks again to those contributors.
    2. \n
  2. Validation: a little different with the public ones, we just split by subject. 5folds cv: imu-only: 0.855, all: 0.898.
    3. \n
  3. The subject-based model improved the performance by ~0.02 compared to the sequence-based model, while the ensemble (average) of both of our individual solutions further increased the performance by ~0.01.
    4. \n
  4. Our sequence-based model may be relatively weaker compared to those of other top teams, which could be one of the reasons for the performance gap.
    5. \n
  5. Due to the special API mechanism I mentioned earlier, our approach results fluctuate with each submission. Therefore, we finalized our solution three days ago and used the last 15 submission opportunities to repeatedly evaluate and select 2 of them.
    6. \n
", "messages": [ { "id": "3280614", "postDate": "09/03/2025 00:17:18", "content": "

First, congrats for all winners and thanks to the organizers for hosting this interesting competition, the community-we learn a lot from your works and sharings. Thanks to my teammates(Rib~) for the excellent teamwork.

\n

Tricks

\n

It is obvious that this competition involves some tricks, and our solution mainly focuses on the following two points.
\n1.Figure out how to use the other sequences' information with the same subject. For convenience in subsequent descriptions, we will refer to this model as the subject-based. We group data from the same subject into a single batch, extract sequential information using CNN, RNN or transformer, and then add a Transformer layer along the subject dimension to capture inter-sequence relationships. During offline validation and online inference, you need to accumulate each subject’s historical sequence information before making predictions on the current sequence.
\n\"\"

\n

Since the evaluation API provides data in batches of single sequence rather than all sequences from the same subject, and additionally shuffles the order of sequences for each submission, this approach leads to some instability. We address this issue in the following ways:

\n\n

2.We prepared 4 types of models:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
modeltraining approch
imu-only_subjectuse imu features only, train subject-base
imu-only_sequenceuse imu features only, train sequence-base
all_subjectuse all features, train subject-base
all_sequenceuse all features, train sequence-base
\n

We adopt the following strategies for model selection and ensemble:

\n\n
df_test = df_hist\nn_hist = (df_hist)\nw_imu_tf = (n_hist,)/\nw_all_tf = (n_hist,)/\nn_samples = sequence\n sequence()/n_samples>=:\n     subject_predictions = (imu-only_subject_model) \n     sequence_predictions = (imu-only_sequence_model)\n     predictions = w_imu_tf*subject_predictions + (-w_imu_tf)*sequence_predictions \n:\n     imu_predictions = (imu-only_subject_model)\n     all_subject_predictions = (all_subject_model)\n     all_sequence_predictions = (all_sequence_model)\n     predictions = w_all_tf*all_subject_predictions + (-w_all_tf)*all_sequence_predictions\n     predictions = predictions*+imu_predictions*\n
\n

Others

\n
    \n
  1. Features & Data augment: we use the features and augments mainly from the public codes, thanks again to those contributors.
    2. \n
  2. Validation: a little different with the public ones, we just split by subject. 5folds cv: imu-only: 0.855, all: 0.898.
    3. \n
  3. The subject-based model improved the performance by ~0.02 compared to the sequence-based model, while the ensemble (average) of both of our individual solutions further increased the performance by ~0.01.
    4. \n
  4. Our sequence-based model may be relatively weaker compared to those of other top teams, which could be one of the reasons for the performance gap.
    5. \n
  5. Due to the special API mechanism I mentioned earlier, our approach results fluctuate with each submission. Therefore, we finalized our solution three days ago and used the last 15 submission opportunities to repeatedly evaluate and select 2 of them.
    6. \n
", "rawMarkdown": "First, congrats for all winners and thanks to the organizers for hosting this interesting competition, the community-we learn a lot from your works and sharings. Thanks to my teammates(Rib~) for the excellent teamwork.\n\n# Tricks\nIt is obvious that this competition involves some tricks, and our solution mainly focuses on the following two points.\n1.Figure out how to use the other sequences' information with the same subject. For convenience in subsequent descriptions, we will refer to this model as the subject-based. We group data from the same subject into a single batch, extract sequential information using CNN, RNN or transformer, and then add a Transformer layer along the subject dimension to capture inter-sequence relationships. During offline validation and online inference, you need to accumulate each subject’s historical sequence information before making predictions on the current sequence.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1069377%2Fb8bcf0438e0a44eca60461df7bf7d730%2F2222222.png?generation=1756860250907385&alt=media)\n\nSince the evaluation API provides data in batches of single sequence rather than all sequences from the same subject, and additionally shuffles the order of sequences for each submission, this approach leads to some instability. We address this issue in the following ways:\n- During the training process, the sequences from the same subject is sampled at different ratios(0.5~0.8), and this process is repeated multiple times.\n- We have observed that the effectiveness of this solution is limited when historical data is insufficient. Therefore, we still need the sequence-based model to correct predictions under conditions of inadequate historical data, that's the second part of our tricks.\n\n2.We prepared 4 types of models: \n| model | training approch |\n| --- | --- |\n| imu-only_subject | use imu features only, train subject-base |\n| imu-only_sequence | use imu features only, train sequence-base |\n| all_subject | use all features, train subject-base |\n| all_sequence | use all features, train sequence-base |\n\nWe adopt the following strategies for model selection and ensemble:\n- When the proportion of missing values exceeds 50%, the imu-only model is selected. Otherwise, the all model is chosen.\n- The imu-only model can further enhance the performance of the all model. Therefore, we apply a weighted fusion of the two.\n- When historical data is insufficient, the sequence-based model is used for ensemble, with its weight adjusted based on the amount of accumulated records.\n```\ndf_test = df_hist[subject]\nn_hist = len(df_hist[subject])\nw_imu_tf = min(n_hist,15)/15\nw_all_tf = min(n_hist,30)/30\nn_samples = sequence.shape[0]\nif sequence['thm_1'].null_count()/n_samples>=0.5:\n subject_predictions = inference(imu-only_subject_model) \n sequence_predictions = inference(imu-only_sequence_model)\n predictions = w_imu_tf*subject_predictions + (1-w_imu_tf)*sequence_predictions \nelse:\n imu_predictions = inference(imu-only_subject_model)\n all_subject_predictions = inference(all_subject_model)\n all_sequence_predictions = inference(all_sequence_model)\n predictions = w_all_tf*all_subject_predictions + (1-w_all_tf)*all_sequence_predictions\n predictions = predictions*0.8+imu_predictions*0.2\n```\n\n# Others\n1. Features & Data augment: we use the features and augments mainly from the public codes, thanks again to those contributors.\n2. Validation: a little different with the public ones, we just split by subject. 5folds cv: imu-only: 0.855, all: 0.898.\n3. The subject-based model improved the performance by ~0.02 compared to the sequence-based model, while the ensemble (average) of both of our individual solutions further increased the performance by ~0.01.\n4. Our sequence-based model may be relatively weaker compared to those of other top teams, which could be one of the reasons for the performance gap.\n5. Due to the special API mechanism I mentioned earlier, our approach results fluctuate with each submission. Therefore, we finalized our solution three days ago and used the last 15 submission opportunities to repeatedly evaluate and select 2 of them.", "votes": null }, { "id": "3280625", "postDate": "09/03/2025 00:35:12", "content": "

Here come the time-series duo💥

", "rawMarkdown": "Here come the time-series duo💥", "votes": null }, { "id": "3280630", "postDate": "09/03/2025 00:50:03", "content": "

I thought the evaluation API was meant to prevent extracting information from other sequences. But it seems the evaluation API doesn't have any effect 🤣

", "rawMarkdown": "I thought the evaluation API was meant to prevent extracting information from other sequences. But it seems the evaluation API doesn't have any effect 🤣", "votes": null }, { "id": "3280633", "postDate": "09/03/2025 00:52:50", "content": "

Wow, I didnt even think about doing it on this level. We just postprocessed, didnt think of storing the whole sequence and using it

", "rawMarkdown": "Wow, I didnt even think about doing it on this level. We just postprocessed, didnt think of storing the whole sequence and using it", "votes": null }, { "id": "3280635", "postDate": "09/03/2025 00:58:32", "content": "

The API indeed had some effect. First, each batch did not necessarily contain the complete data of a subject. Second, the inference order of sequences was randomized each submit time, which introduced instability into our approach. I will elaborate on this in the subsequent section.

", "rawMarkdown": "The API indeed had some effect. First, each batch did not necessarily contain the complete data of a subject. Second, the inference order of sequences was randomized each submit time, which introduced instability into our approach. I will elaborate on this in the subsequent section.", "votes": null }, { "id": "3280787", "postDate": "09/03/2025 06:34:51", "content": "

I also attempted to use API probe methods, such as pseudo-labeling strategies or others, but they ended up failing.

", "rawMarkdown": "I also attempted to use API probe methods, such as pseudo-labeling strategies or others, but they ended up failing.", "votes": null }, { "id": "3280876", "postDate": "09/03/2025 09:27:25", "content": "

I think transformer on subject is doing the similar things to postprocessed, and transformer maybe do better.

", "rawMarkdown": "I think transformer on subject is doing the similar things to postprocessed, and transformer maybe do better.", "votes": null } ], "comments": [ { "id": 3280625, "author_name": "wayne127", "author_url": "", "post_date": "09/03/2025 00:35:12", "content": "

Here come the time-series duo💥

", "votes": null, "replies": [] }, { "id": 3280630, "author_name": "quan0095", "author_url": "", "post_date": "09/03/2025 00:50:03", "content": "

I thought the evaluation API was meant to prevent extracting information from other sequences. But it seems the evaluation API doesn't have any effect 🤣

", "votes": null, "replies": [ { "id": 3280635, "author_name": "chenxin1991", "author_url": "", "post_date": "09/03/2025 00:58:32", "content": "

The API indeed had some effect. First, each batch did not necessarily contain the complete data of a subject. Second, the inference order of sequences was randomized each submit time, which introduced instability into our approach. I will elaborate on this in the subsequent section.

", "votes": null, "replies": [] }, { "id": 3280787, "author_name": "sweetyheehee", "author_url": "", "post_date": "09/03/2025 06:34:51", "content": "

I also attempted to use API probe methods, such as pseudo-labeling strategies or others, but they ended up failing.

", "votes": null, "replies": [] } ] }, { "id": 3280633, "author_name": "ryches", "author_url": "", "post_date": "09/03/2025 00:52:50", "content": "

Wow, I didnt even think about doing it on this level. We just postprocessed, didnt think of storing the whole sequence and using it

", "votes": null, "replies": [ { "id": 3280876, "author_name": "chenxin1991", "author_url": "", "post_date": "09/03/2025 09:27:25", "content": "

I think transformer on subject is doing the similar things to postprocessed, and transformer maybe do better.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3280614": "First, congrats for all winners and thanks to the organizers for hosting this interesting competition, the community-we learn a lot from your works and sharings. Thanks to my teammates(Rib~) for the excellent teamwork.\n\n# Tricks\nIt is obvious that this competition involves some tricks, and our solution mainly focuses on the following two points.\n1.Figure out how to use the other sequences' information with the same subject. For convenience in subsequent descriptions, we will refer to this model as the subject-based. We group data from the same subject into a single batch, extract sequential information using CNN, RNN or transformer, and then add a Transformer layer along the subject dimension to capture inter-sequence relationships. During offline validation and online inference, you need to accumulate each subject’s historical sequence information before making predictions on the current sequence.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1069377%2Fb8bcf0438e0a44eca60461df7bf7d730%2F2222222.png?generation=1756860250907385&alt=media)\n\nSince the evaluation API provides data in batches of single sequence rather than all sequences from the same subject, and additionally shuffles the order of sequences for each submission, this approach leads to some instability. We address this issue in the following ways:\n- During the training process, the sequences from the same subject is sampled at different ratios(0.5~0.8), and this process is repeated multiple times.\n- We have observed that the effectiveness of this solution is limited when historical data is insufficient. Therefore, we still need the sequence-based model to correct predictions under conditions of inadequate historical data, that's the second part of our tricks.\n\n2.We prepared 4 types of models: \n| model | training approch |\n| --- | --- |\n| imu-only_subject | use imu features only, train subject-base |\n| imu-only_sequence | use imu features only, train sequence-base |\n| all_subject | use all features, train subject-base |\n| all_sequence | use all features, train sequence-base |\n\nWe adopt the following strategies for model selection and ensemble:\n- When the proportion of missing values exceeds 50%, the imu-only model is selected. Otherwise, the all model is chosen.\n- The imu-only model can further enhance the performance of the all model. Therefore, we apply a weighted fusion of the two.\n- When historical data is insufficient, the sequence-based model is used for ensemble, with its weight adjusted based on the amount of accumulated records.\n```\ndf_test = df_hist[subject]\nn_hist = len(df_hist[subject])\nw_imu_tf = min(n_hist,15)/15\nw_all_tf = min(n_hist,30)/30\nn_samples = sequence.shape[0]\nif sequence['thm_1'].null_count()/n_samples>=0.5:\n subject_predictions = inference(imu-only_subject_model) \n sequence_predictions = inference(imu-only_sequence_model)\n predictions = w_imu_tf*subject_predictions + (1-w_imu_tf)*sequence_predictions \nelse:\n imu_predictions = inference(imu-only_subject_model)\n all_subject_predictions = inference(all_subject_model)\n all_sequence_predictions = inference(all_sequence_model)\n predictions = w_all_tf*all_subject_predictions + (1-w_all_tf)*all_sequence_predictions\n predictions = predictions*0.8+imu_predictions*0.2\n```\n\n# Others\n1. Features & Data augment: we use the features and augments mainly from the public codes, thanks again to those contributors.\n2. Validation: a little different with the public ones, we just split by subject. 5folds cv: imu-only: 0.855, all: 0.898.\n3. The subject-based model improved the performance by ~0.02 compared to the sequence-based model, while the ensemble (average) of both of our individual solutions further increased the performance by ~0.01.\n4. Our sequence-based model may be relatively weaker compared to those of other top teams, which could be one of the reasons for the performance gap.\n5. Due to the special API mechanism I mentioned earlier, our approach results fluctuate with each submission. Therefore, we finalized our solution three days ago and used the last 15 submission opportunities to repeatedly evaluate and select 2 of them.", "3280625": "Here come the time-series duo💥", "3280630": "I thought the evaluation API was meant to prevent extracting information from other sequences. But it seems the evaluation API doesn't have any effect 🤣", "3280633": "Wow, I didnt even think about doing it on this level. We just postprocessed, didnt think of storing the whole sequence and using it", "3280635": "The API indeed had some effect. First, each batch did not necessarily contain the complete data of a subject. Second, the inference order of sequences was randomized each submit time, which introduced instability into our approach. I will elaborate on this in the subsequent section.", "3280787": "I also attempted to use API probe methods, such as pseudo-labeling strategies or others, but they ended up failing.", "3280876": "I think transformer on subject is doing the similar things to postprocessed, and transformer maybe do better." }, "source": "meta" }