{
  "id": 45617,
  "title": "Overfitting and low performance in transfer learning in keras",
  "url": "/competitions/dog-breed-identification/discussion/45617",
  "author_name": "",
  "post_date": "2017-12-13T15:09:14.829772400Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hi,</p>\n\n<p>I am using VGG16 as base model, I've extracted bottlleneck features of last Conv layer and used that as input to my <code>top_model</code>. the architecture of my <code>top_model</code> is:</p>\n\n<hr>\n\n<p>Layer (type)                 Output Shape              Param #   </p>\n\n<p>input_3 (InputLayer)         (None, 4, 4, 512)         0         </p>\n\n<hr>\n\n<p>flatten_2 (Flatten)          (None, 8192)              0         </p>\n\n<hr>\n\n<p>dropout_2 (Dropout)          (None, 8192)              0         </p>\n\n<hr>\n\n<p>dense_3 (Dense)              (None, 1024)              8389632   </p>\n\n<hr>\n\n<p>dropout_3 (Dropout)          (None, 1024)              0         </p>\n\n<hr>\n\n<p>dense_4 (Dense)              (None, 120)               123000    </p>\n\n<p>Total params: 8,512,632\nTrainable params: 8,512,632\nNon-trainable params: 0</p>\n\n<hr>\n\n<p>I've resized image as 124*124*3 and rescaled to 1./255\nThe when I try to train the model, I get a plateau on validation accuracy of about 20%, training acccuracy keeps increasing. Could anyone please help me out.</p>\n\n<p>the accuracy plot is attached.</p>",
  "messages": [
    {
      "id": "257161",
      "postDate": "12/13/2017 15:09:14",
      "content": "<p>Hi,</p>\n\n<p>I am using VGG16 as base model, I've extracted bottlleneck features of last Conv layer and used that as input to my <code>top_model</code>. the architecture of my <code>top_model</code> is:</p>\n\n<hr>\n\n<p>Layer (type)                 Output Shape              Param #   </p>\n\n<p>input_3 (InputLayer)         (None, 4, 4, 512)         0         </p>\n\n<hr>\n\n<p>flatten_2 (Flatten)          (None, 8192)              0         </p>\n\n<hr>\n\n<p>dropout_2 (Dropout)          (None, 8192)              0         </p>\n\n<hr>\n\n<p>dense_3 (Dense)              (None, 1024)              8389632   </p>\n\n<hr>\n\n<p>dropout_3 (Dropout)          (None, 1024)              0         </p>\n\n<hr>\n\n<p>dense_4 (Dense)              (None, 120)               123000    </p>\n\n<p>Total params: 8,512,632\nTrainable params: 8,512,632\nNon-trainable params: 0</p>\n\n<hr>\n\n<p>I've resized image as 124*124*3 and rescaled to 1./255\nThe when I try to train the model, I get a plateau on validation accuracy of about 20%, training acccuracy keeps increasing. Could anyone please help me out.</p>\n\n<p>the accuracy plot is attached.</p>",
      "rawMarkdown": "Hi,\n\nI am using VGG16 as base model, I've extracted bottlleneck features of last Conv layer and used that as input to my `top_model`. the architecture of my `top_model` is:\n_________________________________________________________________\nLayer (type)                 Output Shape              Param #   \n\ninput_3 (InputLayer)         (None, 4, 4, 512)         0         \n_________________________________________________________________\nflatten_2 (Flatten)          (None, 8192)              0         \n_________________________________________________________________\ndropout_2 (Dropout)          (None, 8192)              0         \n_________________________________________________________________\ndense_3 (Dense)              (None, 1024)              8389632   \n_________________________________________________________________\ndropout_3 (Dropout)          (None, 1024)              0         \n_________________________________________________________________\ndense_4 (Dense)              (None, 120)               123000    \n\nTotal params: 8,512,632\nTrainable params: 8,512,632\nNon-trainable params: 0\n_________________________________________________________________\n\nI've resized image as 124*124*3 and rescaled to 1./255\nThe when I try to train the model, I get a plateau on validation accuracy of about 20%, training acccuracy keeps increasing. Could anyone please help me out.\n\nthe accuracy plot is attached.",
      "votes": null
    },
    {
      "id": "258612",
      "postDate": "12/16/2017 15:43:57",
      "content": "<p>Your model is overfitting. Just do more data augmentation. Regularization is another way but you are already using dropout. So do more augmentation. \nAnother approach to try is to use a high-end model like Inception or Resnet. It will definitely improve the score.</p>",
      "rawMarkdown": "Your model is overfitting. Just do more data augmentation. Regularization is another way but you are already using dropout. So do more augmentation. \nAnother approach to try is to use a high-end model like Inception or Resnet. It will definitely improve the score.",
      "votes": null
    },
    {
      "id": "258635",
      "postDate": "12/16/2017 17:04:58",
      "content": "<p>Yes, I used Inception and resnet and it worked :)\nthanks </p>",
      "rawMarkdown": "Yes, I used Inception and resnet and it worked :)\nthanks",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 258612,
      "author_name": "shridhar743",
      "author_url": "",
      "post_date": "12/16/2017 15:43:57",
      "content": "<p>Your model is overfitting. Just do more data augmentation. Regularization is another way but you are already using dropout. So do more augmentation. \nAnother approach to try is to use a high-end model like Inception or Resnet. It will definitely improve the score.</p>",
      "votes": null,
      "replies": [
        {
          "id": 258635,
          "author_name": "rishabhiitbhu",
          "author_url": "",
          "post_date": "12/16/2017 17:04:58",
          "content": "<p>Yes, I used Inception and resnet and it worked :)\nthanks </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "257161": "Hi,\n\nI am using VGG16 as base model, I've extracted bottlleneck features of last Conv layer and used that as input to my `top_model`. the architecture of my `top_model` is:\n_________________________________________________________________\nLayer (type)                 Output Shape              Param #   \n\ninput_3 (InputLayer)         (None, 4, 4, 512)         0         \n_________________________________________________________________\nflatten_2 (Flatten)          (None, 8192)              0         \n_________________________________________________________________\ndropout_2 (Dropout)          (None, 8192)              0         \n_________________________________________________________________\ndense_3 (Dense)              (None, 1024)              8389632   \n_________________________________________________________________\ndropout_3 (Dropout)          (None, 1024)              0         \n_________________________________________________________________\ndense_4 (Dense)              (None, 120)               123000    \n\nTotal params: 8,512,632\nTrainable params: 8,512,632\nNon-trainable params: 0\n_________________________________________________________________\n\nI've resized image as 124*124*3 and rescaled to 1./255\nThe when I try to train the model, I get a plateau on validation accuracy of about 20%, training acccuracy keeps increasing. Could anyone please help me out.\n\nthe accuracy plot is attached.",
    "258612": "Your model is overfitting. Just do more data augmentation. Regularization is another way but you are already using dropout. So do more augmentation. \nAnother approach to try is to use a high-end model like Inception or Resnet. It will definitely improve the score.",
    "258635": "Yes, I used Inception and resnet and it worked :)\nthanks"
  },
  "source": "meta"
}