{
  "id": 12686,
  "title": "Deep convnets overfitting analysis",
  "url": "/competitions/datasciencebowl/discussion/12686",
  "author_name": "",
  "post_date": "2015-03-04T19:05:46.747Z",
  "votes": 1,
  "comment_count": 5,
  "views": 1856,
  "content": "<p>When we say overfitting happens to a convnet, it means the whole network. I'm curious that can we know which layers overfitting happens to?&nbsp;</p>",
  "messages": [
    {
      "id": "65429",
      "postDate": "03/04/2015 19:05:46",
      "content": "<p>When we say overfitting happens to a convnet, it means the whole network. I'm curious that can we know which layers overfitting happens to?&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65774",
      "postDate": "03/09/2015 13:23:19",
      "content": "<p>That's an interesting question which I have never thought. I believe answering this question is really tough since we do not know the intermediate layers 's state(or action) when the overfitting happens. May be a regular research paper should investigate this.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65808",
      "postDate": "03/09/2015 19:19:57",
      "content": "<p>--- I misinterpreted the question --</p>\n<p>How can I&nbsp;delete posts&nbsp;???&nbsp;:)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65851",
      "postDate": "03/10/2015 03:12:21",
      "content": "<p>Maybe you can add regulazation to each conv layer. If the result turns to be better, that the layer you added regulazation overfitting.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65888",
      "postDate": "03/10/2015 16:05:36",
      "content": "<p>It's hard to measure this exactly I think. But as a general rule I'd guess that layers with lots of parameters, tend to overfit more than layers with few parameters. For example, I've read somewhere that the lowest layers of a convnet are &quot;virtually immune&quot; to overfitting, because each filter/kernel is very small yet gets to see a lot of data.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65916",
      "postDate": "03/10/2015 20:37:38",
      "content": "<p>Thanks for your suggestions!</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 65774,
      "author_name": "pengpai",
      "author_url": "",
      "post_date": "03/09/2015 13:23:19",
      "content": "<p>That's an interesting question which I have never thought. I believe answering this question is really tough since we do not know the intermediate layers 's state(or action) when the overfitting happens. May be a regular research paper should investigate this.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65808,
      "author_name": "juliandewit",
      "author_url": "",
      "post_date": "03/09/2015 19:19:57",
      "content": "<p>--- I misinterpreted the question --</p>\n<p>How can I&nbsp;delete posts&nbsp;???&nbsp;:)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65851,
      "author_name": "liangpang",
      "author_url": "",
      "post_date": "03/10/2015 03:12:21",
      "content": "<p>Maybe you can add regulazation to each conv layer. If the result turns to be better, that the layer you added regulazation overfitting.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65888,
      "author_name": "nicolaykorslund",
      "author_url": "",
      "post_date": "03/10/2015 16:05:36",
      "content": "<p>It's hard to measure this exactly I think. But as a general rule I'd guess that layers with lots of parameters, tend to overfit more than layers with few parameters. For example, I've read somewhere that the lowest layers of a convnet are &quot;virtually immune&quot; to overfitting, because each filter/kernel is very small yet gets to see a lot of data.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65916,
      "author_name": "songgc",
      "author_url": "",
      "post_date": "03/10/2015 20:37:38",
      "content": "<p>Thanks for your suggestions!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "65429": "",
    "65774": "",
    "65808": "",
    "65851": "",
    "65888": "",
    "65916": ""
  },
  "source": "meta"
}