{
  "id": 589904,
  "title": "What do you think of the model structure as follows?",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/589904",
  "author_name": "YYH",
  "post_date": "2025-07-16T09:27:19.016000",
  "votes": -1,
  "comment_count": 12,
  "views": 0,
  "content": "<p>ultiModalModel(<br>\n  (imu_encoder): IMUEncoder(<br>\n    (feat): ImuFeatureExtractor(<br>\n      (lpf): Conv1d(6, 6, kernel_size=(15,), stride=(1,), padding=(7,), groups=6, bias=False)<br>\n      (lpf_acc): Conv1d(3, 3, kernel_size=(15,), stride=(1,), padding=(7,), groups=3, bias=False)<br>\n      (lpf_gyro): Conv1d(3, 3, kernel_size=(15,), stride=(1,), padding=(7,), groups=3, bias=False)<br>\n    )<br>\n    (norm): LayerNorm((62,), eps=1e-05, elementwise_affine=True)<br>\n    (proj): Linear(in_features=62, out_features=64, bias=True)<br>\n    (transformer): TimeTransformer(<br>\n      (proj): Linear(in_features=64, out_features=64, bias=True)<br>\n      (encoder): TransformerEncoder(<br>\n        (layers): ModuleList(<br>\n          (0-1): 2 x TransformerEncoderLayer(<br>\n            (self_attn): MultiheadAttention(<br>\n              (out_proj): NonDynamicallyQuantizableLinear(in_features=64, out_features=64, bias=True)<br>\n            )<br>\n            (linear1): Linear(in_features=64, out_features=2048, bias=True)<br>\n            (dropout): Dropout(p=0.1, inplace=False)<br>\n            (linear2): Linear(in_features=2048, out_features=64, bias=True)<br>\n            (norm1): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n            (norm2): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n            (dropout1): Dropout(p=0.1, inplace=False)<br>\n            (dropout2): Dropout(p=0.1, inplace=False)<br>\n          )<br>\n        )<br>\n      )<br>\n      (norm): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n    )<br>\n  )<br>\n  (tofthm_encoder): TOFTHMBranch(<br>\n    (conv1): Conv1d(10, 32, kernel_size=(3,), stride=(1,), padding=(1,), bias=False)        <br>\n    (conv2): Conv1d(10, 32, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)        <br>\n    (bn1): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  <br>\n    (se_block): SEBlock(<br>\n      (squeeze): AdaptiveAvgPool1d(output_size=1)<br>\n      (excitation): Sequential(<br>\n        (0): Linear(in_features=64, out_features=4, bias=False)<br>\n        (1): ReLU(inplace=True)<br>\n        (2): Linear(in_features=4, out_features=64, bias=False)<br>\n        (3): Sigmoid()<br>\n      )<br>\n    )<br>\n    (res_conv): Conv1d(64, 64, kernel_size=(1,), stride=(1,), bias=False)<br>\n    (bn2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  <br>\n    (pool): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)      <br>\n    (dropout): Dropout(p=0.3, inplace=False)<br>\n    (conv3): Conv1d(64, 128, kernel_size=(5,), stride=(1,), padding=(2,), bias=False)       <br>\n    (bn3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) <br>\n    (dropout2): Dropout(p=0.3, inplace=False)<br>\n    (temporal_model): TimeTransformer(<br>\n      (proj): Linear(in_features=128, out_features=64, bias=True)<br>\n      (encoder): TransformerEncoder(<br>\n        (layers): ModuleList(<br>\n          (0-1): 2 x TransformerEncoderLayer(<br>\n            (self_attn): MultiheadAttention(<br>\n              (out_proj): NonDynamicallyQuantizableLinear(in_features=64, out_features=64, bias=True)<br>\n            )<br>\n            (linear1): Linear(in_features=64, out_features=2048, bias=True)<br>\n            (dropout): Dropout(p=0.1, inplace=False)<br>\n            (linear2): Linear(in_features=2048, out_features=64, bias=True)<br>\n            (norm1): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n            (norm2): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n            (dropout1): Dropout(p=0.1, inplace=False)<br>\n            (dropout2): Dropout(p=0.1, inplace=False)<br>\n          )<br>\n        )<br>\n      )<br>\n      (norm): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n    )<br>\n  )<br>\n  (fusion): MultimodalFusion(<br>\n    (cross_attn): CrossAttention(<br>\n      (q_proj): Linear(in_features=64, out_features=64, bias=True)<br>\n      (k_proj): Linear(in_features=64, out_features=64, bias=True)<br>\n      (v_proj): Linear(in_features=64, out_features=64, bias=True)<br>\n      (out_proj): Linear(in_features=64, out_features=64, bias=True)<br>\n      (dropout): Dropout(p=0.1, inplace=False)<br>\n      (softmax): Softmax(dim=-1)<br>\n    )<br>\n    (norm): LayerNorm((128,), eps=1e-05, elementwise_affine=True)<br>\n    (mlp): Sequential(<br>\n      (0): Linear(in_features=128, out_features=64, bias=True)<br>\n      (1): ReLU()<br>\n      (2): Dropout(p=0.3, inplace=False)<br>\n      (3): Linear(in_features=64, out_features=64, bias=True)<br>\n    )<br>\n  )<br>\n  (classifier): Sequential(<br>\n    (0): Linear(in_features=64, out_features=64, bias=True)<br>\n    (1): ReLU()<br>\n    (2): Dropout(p=0.3, inplace=False)<br>\n    (3): Linear(in_features=64, out_features=7, bias=True)<br>\n  )<br>\n)</p>",
  "messages": [
    {
      "id": 3250008,
      "postDate": "2025-07-17T14:37:05.570Z",
      "content": "<p>666被资本做局了   </p>",
      "rawMarkdown": "666被资本做局了   ",
      "votes": -2
    },
    {
      "id": 3249427,
      "postDate": "2025-07-16T13:03:38.777Z",
      "content": "<ol>\n<li>Dataset is small, so don't use transformer</li>\n<li>InstaceNorm&gt;BatchNorm&gt;LayerNorm</li>\n</ol>",
      "rawMarkdown": "1. Dataset is small, so don't use transformer\n2. InstaceNorm>BatchNorm>LayerNorm",
      "replies": [
        {
          "id": 3249432,
          "postDate": "2025-07-16T13:18:41.320Z",
          "content": "<p>Okay, thanks for the reminder. Then should we use LSTM, GRU, or CNN structures to extract more features?</p>",
          "rawMarkdown": "Okay, thanks for the reminder. Then should we use LSTM, GRU, or CNN structures to extract more features?",
          "votes": 1,
          "replies": [
            {
              "id": 3249722,
              "postDate": "2025-07-17T02:48:46.703Z",
              "rawMarkdown": "",
              "isDeleted": true
            }
          ]
        },
        {
          "id": 3249724,
          "postDate": "2025-07-17T02:50:47.557Z",
          "content": "<p>May I ask how much performance do the InstanceNorm improve? In my experiment, InstanceNorm is much worse than BatchNorm whether in CNN or RNN</p>",
          "rawMarkdown": "May I ask how much performance do the InstanceNorm improve? In my experiment, InstanceNorm is much worse than BatchNorm whether in CNN or RNN",
          "votes": 1,
          "replies": [
            {
              "id": 3249725,
              "postDate": "2025-07-17T02:51:41.013Z",
              "content": "<p><a href=\"https://www.kaggle.com/i2nfinit3y\" target=\"_blank\">@i2nfinit3y</a> which place you put your inst norm in.</p>",
              "rawMarkdown": "@i2nfinit3y which place you put your inst norm in."
            },
            {
              "id": 3249727,
              "postDate": "2025-07-17T02:54:23.100Z",
              "content": "<p>after conv layers and linear layers</p>",
              "rawMarkdown": "after conv layers and linear layers"
            },
            {
              "id": 3249784,
              "postDate": "2025-07-17T04:55:19.463Z",
              "content": "<p>Try partial inst norm on acc and quat position</p>",
              "rawMarkdown": "Try partial inst norm on acc and quat position",
              "votes": 2
            },
            {
              "id": 3249954,
              "postDate": "2025-07-17T12:35:10.197Z",
              "content": "<p>Would you please explain what you mean by partial inst norm?</p>",
              "rawMarkdown": "Would you please explain what you mean by partial inst norm?"
            }
          ]
        }
      ]
    },
    {
      "id": 3249423,
      "postDate": "2025-07-16T13:01:15.850Z",
      "content": "<p>If you use batch size larger than 32, LayerNorm might be not good choice</p>",
      "rawMarkdown": "If you use batch size larger than 32, LayerNorm might be not good choice",
      "replies": [
        {
          "id": 3249435,
          "postDate": "2025-07-16T13:23:20.907Z",
          "content": "<p>Thank you，I'm working on complex features, but it feels like most of them are ineffective.</p>",
          "rawMarkdown": "Thank you，I'm working on complex features, but it feels like most of them are ineffective.",
          "replies": [
            {
              "id": 3249484,
              "postDate": "2025-07-16T15:21:24.017Z",
              "content": "<p>Sample is small so it is hard to goes up</p>",
              "rawMarkdown": "Sample is small so it is hard to goes up"
            }
          ]
        }
      ]
    },
    {
      "id": 3249345,
      "postDate": "2025-07-16T09:27:19.017Z",
      "content": "<p>ultiModalModel(<br>\n  (imu_encoder): IMUEncoder(<br>\n    (feat): ImuFeatureExtractor(<br>\n      (lpf): Conv1d(6, 6, kernel_size=(15,), stride=(1,), padding=(7,), groups=6, bias=False)<br>\n      (lpf_acc): Conv1d(3, 3, kernel_size=(15,), stride=(1,), padding=(7,), groups=3, bias=False)<br>\n      (lpf_gyro): Conv1d(3, 3, kernel_size=(15,), stride=(1,), padding=(7,), groups=3, bias=False)<br>\n    )<br>\n    (norm): LayerNorm((62,), eps=1e-05, elementwise_affine=True)<br>\n    (proj): Linear(in_features=62, out_features=64, bias=True)<br>\n    (transformer): TimeTransformer(<br>\n      (proj): Linear(in_features=64, out_features=64, bias=True)<br>\n      (encoder): TransformerEncoder(<br>\n        (layers): ModuleList(<br>\n          (0-1): 2 x TransformerEncoderLayer(<br>\n            (self_attn): MultiheadAttention(<br>\n              (out_proj): NonDynamicallyQuantizableLinear(in_features=64, out_features=64, bias=True)<br>\n            )<br>\n            (linear1): Linear(in_features=64, out_features=2048, bias=True)<br>\n            (dropout): Dropout(p=0.1, inplace=False)<br>\n            (linear2): Linear(in_features=2048, out_features=64, bias=True)<br>\n            (norm1): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n            (norm2): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n            (dropout1): Dropout(p=0.1, inplace=False)<br>\n            (dropout2): Dropout(p=0.1, inplace=False)<br>\n          )<br>\n        )<br>\n      )<br>\n      (norm): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n    )<br>\n  )<br>\n  (tofthm_encoder): TOFTHMBranch(<br>\n    (conv1): Conv1d(10, 32, kernel_size=(3,), stride=(1,), padding=(1,), bias=False)        <br>\n    (conv2): Conv1d(10, 32, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)        <br>\n    (bn1): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  <br>\n    (se_block): SEBlock(<br>\n      (squeeze): AdaptiveAvgPool1d(output_size=1)<br>\n      (excitation): Sequential(<br>\n        (0): Linear(in_features=64, out_features=4, bias=False)<br>\n        (1): ReLU(inplace=True)<br>\n        (2): Linear(in_features=4, out_features=64, bias=False)<br>\n        (3): Sigmoid()<br>\n      )<br>\n    )<br>\n    (res_conv): Conv1d(64, 64, kernel_size=(1,), stride=(1,), bias=False)<br>\n    (bn2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  <br>\n    (pool): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)      <br>\n    (dropout): Dropout(p=0.3, inplace=False)<br>\n    (conv3): Conv1d(64, 128, kernel_size=(5,), stride=(1,), padding=(2,), bias=False)       <br>\n    (bn3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) <br>\n    (dropout2): Dropout(p=0.3, inplace=False)<br>\n    (temporal_model): TimeTransformer(<br>\n      (proj): Linear(in_features=128, out_features=64, bias=True)<br>\n      (encoder): TransformerEncoder(<br>\n        (layers): ModuleList(<br>\n          (0-1): 2 x TransformerEncoderLayer(<br>\n            (self_attn): MultiheadAttention(<br>\n              (out_proj): NonDynamicallyQuantizableLinear(in_features=64, out_features=64, bias=True)<br>\n            )<br>\n            (linear1): Linear(in_features=64, out_features=2048, bias=True)<br>\n            (dropout): Dropout(p=0.1, inplace=False)<br>\n            (linear2): Linear(in_features=2048, out_features=64, bias=True)<br>\n            (norm1): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n            (norm2): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n            (dropout1): Dropout(p=0.1, inplace=False)<br>\n            (dropout2): Dropout(p=0.1, inplace=False)<br>\n          )<br>\n        )<br>\n      )<br>\n      (norm): LayerNorm((64,), eps=1e-05, elementwise_affine=True)<br>\n    )<br>\n  )<br>\n  (fusion): MultimodalFusion(<br>\n    (cross_attn): CrossAttention(<br>\n      (q_proj): Linear(in_features=64, out_features=64, bias=True)<br>\n      (k_proj): Linear(in_features=64, out_features=64, bias=True)<br>\n      (v_proj): Linear(in_features=64, out_features=64, bias=True)<br>\n      (out_proj): Linear(in_features=64, out_features=64, bias=True)<br>\n      (dropout): Dropout(p=0.1, inplace=False)<br>\n      (softmax): Softmax(dim=-1)<br>\n    )<br>\n    (norm): LayerNorm((128,), eps=1e-05, elementwise_affine=True)<br>\n    (mlp): Sequential(<br>\n      (0): Linear(in_features=128, out_features=64, bias=True)<br>\n      (1): ReLU()<br>\n      (2): Dropout(p=0.3, inplace=False)<br>\n      (3): Linear(in_features=64, out_features=64, bias=True)<br>\n    )<br>\n  )<br>\n  (classifier): Sequential(<br>\n    (0): Linear(in_features=64, out_features=64, bias=True)<br>\n    (1): ReLU()<br>\n    (2): Dropout(p=0.3, inplace=False)<br>\n    (3): Linear(in_features=64, out_features=7, bias=True)<br>\n  )<br>\n)</p>",
      "rawMarkdown": "ultiModalModel(\n  (imu_encoder): IMUEncoder(\n    (feat): ImuFeatureExtractor(\n      (lpf): Conv1d(6, 6, kernel_size=(15,), stride=(1,), padding=(7,), groups=6, bias=False)\n      (lpf_acc): Conv1d(3, 3, kernel_size=(15,), stride=(1,), padding=(7,), groups=3, bias=False)\n      (lpf_gyro): Conv1d(3, 3, kernel_size=(15,), stride=(1,), padding=(7,), groups=3, bias=False)\n    )\n    (norm): LayerNorm((62,), eps=1e-05, elementwise_affine=True)\n    (proj): Linear(in_features=62, out_features=64, bias=True)\n    (transformer): TimeTransformer(\n      (proj): Linear(in_features=64, out_features=64, bias=True)\n      (encoder): TransformerEncoder(\n        (layers): ModuleList(\n          (0-1): 2 x TransformerEncoderLayer(\n            (self_attn): MultiheadAttention(\n              (out_proj): NonDynamicallyQuantizableLinear(in_features=64, out_features=64, bias=True)\n            )\n            (linear1): Linear(in_features=64, out_features=2048, bias=True)\n            (dropout): Dropout(p=0.1, inplace=False)\n            (linear2): Linear(in_features=2048, out_features=64, bias=True)\n            (norm1): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n            (norm2): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n            (dropout1): Dropout(p=0.1, inplace=False)\n            (dropout2): Dropout(p=0.1, inplace=False)\n          )\n        )\n      )\n      (norm): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n    )\n  )\n  (tofthm_encoder): TOFTHMBranch(\n    (conv1): Conv1d(10, 32, kernel_size=(3,), stride=(1,), padding=(1,), bias=False)        \n    (conv2): Conv1d(10, 32, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)        \n    (bn1): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  \n    (se_block): SEBlock(\n      (squeeze): AdaptiveAvgPool1d(output_size=1)\n      (excitation): Sequential(\n        (0): Linear(in_features=64, out_features=4, bias=False)\n        (1): ReLU(inplace=True)\n        (2): Linear(in_features=4, out_features=64, bias=False)\n        (3): Sigmoid()\n      )\n    )\n    (res_conv): Conv1d(64, 64, kernel_size=(1,), stride=(1,), bias=False)\n    (bn2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  \n    (pool): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)      \n    (dropout): Dropout(p=0.3, inplace=False)\n    (conv3): Conv1d(64, 128, kernel_size=(5,), stride=(1,), padding=(2,), bias=False)       \n    (bn3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) \n    (dropout2): Dropout(p=0.3, inplace=False)\n    (temporal_model): TimeTransformer(\n      (proj): Linear(in_features=128, out_features=64, bias=True)\n      (encoder): TransformerEncoder(\n        (layers): ModuleList(\n          (0-1): 2 x TransformerEncoderLayer(\n            (self_attn): MultiheadAttention(\n              (out_proj): NonDynamicallyQuantizableLinear(in_features=64, out_features=64, bias=True)\n            )\n            (linear1): Linear(in_features=64, out_features=2048, bias=True)\n            (dropout): Dropout(p=0.1, inplace=False)\n            (linear2): Linear(in_features=2048, out_features=64, bias=True)\n            (norm1): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n            (norm2): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n            (dropout1): Dropout(p=0.1, inplace=False)\n            (dropout2): Dropout(p=0.1, inplace=False)\n          )\n        )\n      )\n      (norm): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n    )\n  )\n  (fusion): MultimodalFusion(\n    (cross_attn): CrossAttention(\n      (q_proj): Linear(in_features=64, out_features=64, bias=True)\n      (k_proj): Linear(in_features=64, out_features=64, bias=True)\n      (v_proj): Linear(in_features=64, out_features=64, bias=True)\n      (out_proj): Linear(in_features=64, out_features=64, bias=True)\n      (dropout): Dropout(p=0.1, inplace=False)\n      (softmax): Softmax(dim=-1)\n    )\n    (norm): LayerNorm((128,), eps=1e-05, elementwise_affine=True)\n    (mlp): Sequential(\n      (0): Linear(in_features=128, out_features=64, bias=True)\n      (1): ReLU()\n      (2): Dropout(p=0.3, inplace=False)\n      (3): Linear(in_features=64, out_features=64, bias=True)\n    )\n  )\n  (classifier): Sequential(\n    (0): Linear(in_features=64, out_features=64, bias=True)\n    (1): ReLU()\n    (2): Dropout(p=0.3, inplace=False)\n    (3): Linear(in_features=64, out_features=7, bias=True)\n  )\n)\n",
      "votes": -1
    }
  ],
  "comments": [
    {
      "id": 3250008,
      "author_name": "menglei",
      "author_url": "",
      "post_date": "2025-07-17T14:37:05.570000",
      "content": "<p>666被资本做局了   </p>",
      "votes": -2,
      "replies": []
    },
    {
      "id": 3249427,
      "author_name": "Tom",
      "author_url": "",
      "post_date": "2025-07-16T13:03:38.777000",
      "content": "<ol>\n<li>Dataset is small, so don't use transformer</li>\n<li>InstaceNorm&gt;BatchNorm&gt;LayerNorm</li>\n</ol>",
      "votes": 0,
      "replies": [
        {
          "id": 3249432,
          "author_name": "YYH",
          "author_url": "",
          "post_date": "2025-07-16T13:18:41.320000",
          "content": "<p>Okay, thanks for the reminder. Then should we use LSTM, GRU, or CNN structures to extract more features?</p>",
          "votes": 1,
          "replies": [
            {
              "id": 3249722,
              "author_name": "",
              "author_url": "",
              "post_date": "2025-07-17T02:48:46.703000",
              "content": "",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 3249724,
          "author_name": "I2nfinit3y",
          "author_url": "",
          "post_date": "2025-07-17T02:50:47.557000",
          "content": "<p>May I ask how much performance do the InstanceNorm improve? In my experiment, InstanceNorm is much worse than BatchNorm whether in CNN or RNN</p>",
          "votes": 1,
          "replies": [
            {
              "id": 3249725,
              "author_name": "Tom",
              "author_url": "",
              "post_date": "2025-07-17T02:51:41.013000",
              "content": "<p><a href=\"https://www.kaggle.com/i2nfinit3y\" target=\"_blank\">@i2nfinit3y</a> which place you put your inst norm in.</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3249727,
              "author_name": "I2nfinit3y",
              "author_url": "",
              "post_date": "2025-07-17T02:54:23.100000",
              "content": "<p>after conv layers and linear layers</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3249784,
              "author_name": "Tom",
              "author_url": "",
              "post_date": "2025-07-17T04:55:19.463000",
              "content": "<p>Try partial inst norm on acc and quat position</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 3249954,
              "author_name": "Jack",
              "author_url": "",
              "post_date": "2025-07-17T12:35:10.197000",
              "content": "<p>Would you please explain what you mean by partial inst norm?</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3249423,
      "author_name": "CodeHacker",
      "author_url": "",
      "post_date": "2025-07-16T13:01:15.850000",
      "content": "<p>If you use batch size larger than 32, LayerNorm might be not good choice</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3249435,
          "author_name": "YYH",
          "author_url": "",
          "post_date": "2025-07-16T13:23:20.907000",
          "content": "<p>Thank you，I'm working on complex features, but it feels like most of them are ineffective.</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3249484,
              "author_name": "CodeHacker",
              "author_url": "",
              "post_date": "2025-07-16T15:21:24.017000",
              "content": "<p>Sample is small so it is hard to goes up</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3250008": "666被资本做局了   ",
    "3249427": "1. Dataset is small, so don't use transformer\n2. InstaceNorm>BatchNorm>LayerNorm",
    "3249423": "If you use batch size larger than 32, LayerNorm might be not good choice",
    "3249345": "ultiModalModel(\n  (imu_encoder): IMUEncoder(\n    (feat): ImuFeatureExtractor(\n      (lpf): Conv1d(6, 6, kernel_size=(15,), stride=(1,), padding=(7,), groups=6, bias=False)\n      (lpf_acc): Conv1d(3, 3, kernel_size=(15,), stride=(1,), padding=(7,), groups=3, bias=False)\n      (lpf_gyro): Conv1d(3, 3, kernel_size=(15,), stride=(1,), padding=(7,), groups=3, bias=False)\n    )\n    (norm): LayerNorm((62,), eps=1e-05, elementwise_affine=True)\n    (proj): Linear(in_features=62, out_features=64, bias=True)\n    (transformer): TimeTransformer(\n      (proj): Linear(in_features=64, out_features=64, bias=True)\n      (encoder): TransformerEncoder(\n        (layers): ModuleList(\n          (0-1): 2 x TransformerEncoderLayer(\n            (self_attn): MultiheadAttention(\n              (out_proj): NonDynamicallyQuantizableLinear(in_features=64, out_features=64, bias=True)\n            )\n            (linear1): Linear(in_features=64, out_features=2048, bias=True)\n            (dropout): Dropout(p=0.1, inplace=False)\n            (linear2): Linear(in_features=2048, out_features=64, bias=True)\n            (norm1): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n            (norm2): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n            (dropout1): Dropout(p=0.1, inplace=False)\n            (dropout2): Dropout(p=0.1, inplace=False)\n          )\n        )\n      )\n      (norm): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n    )\n  )\n  (tofthm_encoder): TOFTHMBranch(\n    (conv1): Conv1d(10, 32, kernel_size=(3,), stride=(1,), padding=(1,), bias=False)        \n    (conv2): Conv1d(10, 32, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)        \n    (bn1): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  \n    (se_block): SEBlock(\n      (squeeze): AdaptiveAvgPool1d(output_size=1)\n      (excitation): Sequential(\n        (0): Linear(in_features=64, out_features=4, bias=False)\n        (1): ReLU(inplace=True)\n        (2): Linear(in_features=4, out_features=64, bias=False)\n        (3): Sigmoid()\n      )\n    )\n    (res_conv): Conv1d(64, 64, kernel_size=(1,), stride=(1,), bias=False)\n    (bn2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)  \n    (pool): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)      \n    (dropout): Dropout(p=0.3, inplace=False)\n    (conv3): Conv1d(64, 128, kernel_size=(5,), stride=(1,), padding=(2,), bias=False)       \n    (bn3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) \n    (dropout2): Dropout(p=0.3, inplace=False)\n    (temporal_model): TimeTransformer(\n      (proj): Linear(in_features=128, out_features=64, bias=True)\n      (encoder): TransformerEncoder(\n        (layers): ModuleList(\n          (0-1): 2 x TransformerEncoderLayer(\n            (self_attn): MultiheadAttention(\n              (out_proj): NonDynamicallyQuantizableLinear(in_features=64, out_features=64, bias=True)\n            )\n            (linear1): Linear(in_features=64, out_features=2048, bias=True)\n            (dropout): Dropout(p=0.1, inplace=False)\n            (linear2): Linear(in_features=2048, out_features=64, bias=True)\n            (norm1): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n            (norm2): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n            (dropout1): Dropout(p=0.1, inplace=False)\n            (dropout2): Dropout(p=0.1, inplace=False)\n          )\n        )\n      )\n      (norm): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n    )\n  )\n  (fusion): MultimodalFusion(\n    (cross_attn): CrossAttention(\n      (q_proj): Linear(in_features=64, out_features=64, bias=True)\n      (k_proj): Linear(in_features=64, out_features=64, bias=True)\n      (v_proj): Linear(in_features=64, out_features=64, bias=True)\n      (out_proj): Linear(in_features=64, out_features=64, bias=True)\n      (dropout): Dropout(p=0.1, inplace=False)\n      (softmax): Softmax(dim=-1)\n    )\n    (norm): LayerNorm((128,), eps=1e-05, elementwise_affine=True)\n    (mlp): Sequential(\n      (0): Linear(in_features=128, out_features=64, bias=True)\n      (1): ReLU()\n      (2): Dropout(p=0.3, inplace=False)\n      (3): Linear(in_features=64, out_features=64, bias=True)\n    )\n  )\n  (classifier): Sequential(\n    (0): Linear(in_features=64, out_features=64, bias=True)\n    (1): ReLU()\n    (2): Dropout(p=0.3, inplace=False)\n    (3): Linear(in_features=64, out_features=7, bias=True)\n  )\n)\n"
  }
}