{
  "id": 589282,
  "title": "Adding more layers does not help much",
  "url": "/competitions/cmi-detect-behavior-with-sensor-data/discussion/589282",
  "author_name": "",
  "post_date": "2025-07-11T15:52:12.130359900Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>During my fold 1 training of CV for imu only model((I fixed the random seed to the data splitter), I first used a model that is 10mb, and obtained a highest score of 0.82, I then kept the overall structure of that model, but removed a lot of layers from that, and I also get the a score of 0.822 with more epochs, yet the model is only 380kb now. Therefore, I think a bigger model with same architecture will mostly contribute to less epochs of training (not necessarily less time, since smaller model are also faster to train), it is the feature engineering, missing value handling, data augmentation and architecture design that really help. </p>",
  "messages": [
    {
      "id": "3246820",
      "postDate": "07/11/2025 15:52:12",
      "content": "<p>During my fold 1 training of CV for imu only model((I fixed the random seed to the data splitter), I first used a model that is 10mb, and obtained a highest score of 0.82, I then kept the overall structure of that model, but removed a lot of layers from that, and I also get the a score of 0.822 with more epochs, yet the model is only 380kb now. Therefore, I think a bigger model with same architecture will mostly contribute to less epochs of training (not necessarily less time, since smaller model are also faster to train), it is the feature engineering, missing value handling, data augmentation and architecture design that really help. </p>",
      "rawMarkdown": "During my fold 1 training of CV for imu only model((I fixed the random seed to the data splitter), I first used a model that is 10mb, and obtained a highest score of 0.82, I then kept the overall structure of that model, but removed a lot of layers from that, and I also get the a score of 0.822 with more epochs, yet the model is only 380kb now. Therefore, I think a bigger model with same architecture will mostly contribute to less epochs of training (not necessarily less time, since smaller model are also faster to train), it is the feature engineering, missing value handling, data augmentation and architecture design that really help.",
      "votes": null
    },
    {
      "id": "3246854",
      "postDate": "07/11/2025 16:35:32",
      "content": "<p>One thing to add to this is the risk of overfitting:<br>\nGenerally speaking, larger models are more prone to overfitting; thus the goal would be to maximize the model size without overfitting (e.g. with regularization, augmentation, …)</p>",
      "rawMarkdown": "One thing to add to this is the risk of overfitting:\nGenerally speaking, larger models are more prone to overfitting; thus the goal would be to maximize the model size without overfitting (e.g. with regularization, augmentation, ...)",
      "votes": null
    },
    {
      "id": "3246856",
      "postDate": "07/11/2025 16:37:03",
      "content": "<p>Definitely agree.</p>",
      "rawMarkdown": "Definitely agree.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3246854,
      "author_name": "patrickaiforfun",
      "author_url": "",
      "post_date": "07/11/2025 16:35:32",
      "content": "<p>One thing to add to this is the risk of overfitting:<br>\nGenerally speaking, larger models are more prone to overfitting; thus the goal would be to maximize the model size without overfitting (e.g. with regularization, augmentation, …)</p>",
      "votes": null,
      "replies": [
        {
          "id": 3246856,
          "author_name": "georgewang73",
          "author_url": "",
          "post_date": "07/11/2025 16:37:03",
          "content": "<p>Definitely agree.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3246820": "During my fold 1 training of CV for imu only model((I fixed the random seed to the data splitter), I first used a model that is 10mb, and obtained a highest score of 0.82, I then kept the overall structure of that model, but removed a lot of layers from that, and I also get the a score of 0.822 with more epochs, yet the model is only 380kb now. Therefore, I think a bigger model with same architecture will mostly contribute to less epochs of training (not necessarily less time, since smaller model are also faster to train), it is the feature engineering, missing value handling, data augmentation and architecture design that really help.",
    "3246854": "One thing to add to this is the risk of overfitting:\nGenerally speaking, larger models are more prone to overfitting; thus the goal would be to maximize the model size without overfitting (e.g. with regularization, augmentation, ...)",
    "3246856": "Definitely agree."
  },
  "source": "meta"
}