{
  "id": 589160,
  "title": "Does Transformer Improve the Score?",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/589160",
  "author_name": "",
  "post_date": "2025-07-10T15:26:27.516371800Z",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Has anyone tried combining CNN and Transformer for this competition?<br>\nI’m wondering how effective this approach is in practice.<br>\nIf you have experimented with this architecture, could you please share whether it improved your scores compared to using only CNN or RNN models?</p>",
  "messages": [
    {
      "id": "3246260",
      "postDate": "07/10/2025 15:26:27",
      "content": "<p>Has anyone tried combining CNN and Transformer for this competition?<br>\nI’m wondering how effective this approach is in practice.<br>\nIf you have experimented with this architecture, could you please share whether it improved your scores compared to using only CNN or RNN models?</p>",
      "rawMarkdown": "Has anyone tried combining CNN and Transformer for this competition?\nI’m wondering how effective this approach is in practice.\nIf you have experimented with this architecture, could you please share whether it improved your scores compared to using only CNN or RNN models?",
      "votes": null
    },
    {
      "id": "3246291",
      "postDate": "07/10/2025 16:31:00",
      "content": "<p>0.01 in some case. But usually same</p>",
      "rawMarkdown": "0.01 in some case. But usually same",
      "votes": null
    },
    {
      "id": "3246302",
      "postDate": "07/10/2025 16:54:51",
      "content": "<p>In my experiments, the model with Transformer added contributes to improving LB.</p>\n<p>However, Transformer generally tends to require a lot of training data.<br>\nIn this competition, the training data is a small dataset of 5100 rows (5100 sequences), so it seems that the dataset is too small to maximize the performance of Transformer.</p>",
      "rawMarkdown": "In my experiments, the model with Transformer added contributes to improving LB.\n\nHowever, Transformer generally tends to require a lot of training data.\nIn this competition, the training data is a small dataset of 5100 rows (5100 sequences), so it seems that the dataset is too small to maximize the performance of Transformer.",
      "votes": null
    },
    {
      "id": "3247764",
      "postDate": "07/13/2025 11:36:55",
      "content": "<p>Thank you so much!</p>",
      "rawMarkdown": "Thank you so much!",
      "votes": null
    },
    {
      "id": "3247765",
      "postDate": "07/13/2025 11:37:07",
      "content": "<p>Thanks a lot</p>",
      "rawMarkdown": "Thanks a lot",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3246291,
      "author_name": "junhanzangai",
      "author_url": "",
      "post_date": "07/10/2025 16:31:00",
      "content": "<p>0.01 in some case. But usually same</p>",
      "votes": null,
      "replies": [
        {
          "id": 3247765,
          "author_name": "xunden",
          "author_url": "",
          "post_date": "07/13/2025 11:37:07",
          "content": "<p>Thanks a lot</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 3246302,
      "author_name": "hideyukizushi",
      "author_url": "",
      "post_date": "07/10/2025 16:54:51",
      "content": "<p>In my experiments, the model with Transformer added contributes to improving LB.</p>\n<p>However, Transformer generally tends to require a lot of training data.<br>\nIn this competition, the training data is a small dataset of 5100 rows (5100 sequences), so it seems that the dataset is too small to maximize the performance of Transformer.</p>",
      "votes": null,
      "replies": [
        {
          "id": 3247764,
          "author_name": "xunden",
          "author_url": "",
          "post_date": "07/13/2025 11:36:55",
          "content": "<p>Thank you so much!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3246260": "Has anyone tried combining CNN and Transformer for this competition?\nI’m wondering how effective this approach is in practice.\nIf you have experimented with this architecture, could you please share whether it improved your scores compared to using only CNN or RNN models?",
    "3246291": "0.01 in some case. But usually same",
    "3246302": "In my experiments, the model with Transformer added contributes to improving LB.\n\nHowever, Transformer generally tends to require a lot of training data.\nIn this competition, the training data is a small dataset of 5100 rows (5100 sequences), so it seems that the dataset is too small to maximize the performance of Transformer.",
    "3247764": "Thank you so much!",
    "3247765": "Thanks a lot"
  },
  "source": "meta"
}