Hi friends,
\n\nI have implemented CNN model to classify Dog Breed and the model performance is very poor and I am getting 10% accuracy.
\nThen I used transfer learning using ResNet50 and My model getting over fit and accuracy it much less the my basic CNN model. training accuracy is 60% where as validation accuracy 6%.\nCan some please suggest where i am going wrong with transfer learning
Hi friends,
\n\nI have implemented CNN model to classify Dog Breed and the model performance is very poor and I am getting 10% accuracy.
\nThen I used transfer learning using ResNet50 and My model getting over fit and accuracy it much less the my basic CNN model. training accuracy is 60% where as validation accuracy 6%.\nCan some please suggest where i am going wrong with transfer learning
Total noob here, my only cred in this area is coding through Francois Chollet's book \"Deep Learning with Python\". In fact this is the first project I have attempted outside of the book. So, take my comments/opinions with a grain of salt.
\n\nI am also using this data set as a learning project. I took the approach of using Xception as my ConvBase and then layered my classifier on top. Here is the structure of my classifier.
\n\n(Sequential)\nConvBase/ Xception (Remember to set the ConvBase trainable to FALSE! You can fine tune after your initial training of the classifier.)\nFlatten\nDense/ 512/ Relu\nNormalize\nDropout/ 50\nDense/ 512/ Relu\nNormalize\nDropout/ 50\nDense/ 120/ Softmax
\n\nOptimizer - RMSprop (lr=0.0001)\nEpochs=50-150
\n\nResults: Val_loss= 1.98 Val_acc= 51%\nThe results are less than stellar... but not bad. I am actually looking around this discussion forum for help to see what I might do better. How do we get into the 95% accuracy club?
\n\nAccording to F. Chollet:\nThe steps for fine-tuning a network are as follow:\n1. Add you custom network on top of an already-trained base network\n2. Freeze the base network.\n3. Train the part you added.\n4. Unfreeze some layers in the base network. (last few layers)\n5. Jointly train both these layers and the part you added.
\n\nLet me know if you see progress. Good luck!
", "rawMarkdown": "Total noob here, my only cred in this area is coding through Francois Chollet's book \"Deep Learning with Python\". In fact this is the first project I have attempted outside of the book. So, take my comments/opinions with a grain of salt.\n\nI am also using this data set as a learning project. I took the approach of using Xception as my ConvBase and then layered my classifier on top. Here is the structure of my classifier.\n\n(Sequential)\nConvBase/ Xception (Remember to set the ConvBase trainable to FALSE! You can fine tune after your initial training of the classifier.)\nFlatten\nDense/ 512/ Relu\nNormalize\nDropout/ 50\nDense/ 512/ Relu\nNormalize\nDropout/ 50\nDense/ 120/ Softmax\n\nOptimizer - RMSprop (lr=0.0001)\nEpochs=50-150\n\nResults: Val_loss= 1.98 Val_acc= 51%\nThe results are less than stellar... but not bad. I am actually looking around this discussion forum for help to see what I might do better. How do we get into the 95% accuracy club?\n\nAccording to F. Chollet:\nThe steps for fine-tuning a network are as follow:\n1. Add you custom network on top of an already-trained base network\n2. Freeze the base network.\n3. Train the part you added.\n4. Unfreeze some layers in the base network. (last few layers)\n5. Jointly train both these layers and the part you added.\n\nLet me know if you see progress. Good luck!", "votes": null }, { "id": "464580", "postDate": "02/01/2019 05:25:13", "content": "Even I'm facing the same issue, my custom CNN is overfitting and its loss is getting reduced drastically. But, loss in the validation data remains high. No much improvement.
\n\nBefore working on this problem, I worked classifying a dog and cat. My custom model worked fine in that simple case. But in this breed identification, my model performance is poor just 6% on the validation set. Is this because of the 120 classes we have to predict?
\n\nIs there any way to get at least 50% accuracy using my custom CNN without using a pre-trained model?
\n\nThis is my structure of my custom model:
\n\n cls = Sequential()\n\n cls.add(Conv2D(64, (5, 5), activation='relu', input_shape=(64, 64, 3)))\n cls.add(Conv2D(64, (3, 3), activation='relu', input_shape=(64, 64, 3)))\n cls.add(Dropout(0.3))\n\n cls.add(MaxPool2D((2, 2), strides = 2))\n cls.add(Dropout(0.3))\n\n cls.add(Conv2D(64, (3, 3), activation='relu'))\n cls.add(Conv2D(64, (1, 1),strides = 2, activation='relu'))\n\n cls.add(MaxPool2D((2, 2)))\n cls.add(Dropout(0.2))\n\n cls.add(Flatten())\n cls.add(Dense(516, kernel_initializer='uniform'))\n cls.add(Activation('relu'))\n cls.add(Dropout(0.3))\n cls.add(Dense(258, kernel_initializer='uniform'))\n cls.add(Activation('relu'))\n cls.add(Dropout(0.2))\n\n cls. add(Dense(120))\n cls.add(Activation('softmax'))\n\n # configure the learning params of the classifier\n cls.compile(optimizer='SGD', loss='categorical_crossentropy',\n metrics=['accuracy']\n\n )\nTotal noob here, my only cred in this area is coding through Francois Chollet's book \"Deep Learning with Python\". In fact this is the first project I have attempted outside of the book. So, take my comments/opinions with a grain of salt.
\n\nI am also using this data set as a learning project. I took the approach of using Xception as my ConvBase and then layered my classifier on top. Here is the structure of my classifier.
\n\n(Sequential)\nConvBase/ Xception (Remember to set the ConvBase trainable to FALSE! You can fine tune after your initial training of the classifier.)\nFlatten\nDense/ 512/ Relu\nNormalize\nDropout/ 50\nDense/ 512/ Relu\nNormalize\nDropout/ 50\nDense/ 120/ Softmax
\n\nOptimizer - RMSprop (lr=0.0001)\nEpochs=50-150
\n\nResults: Val_loss= 1.98 Val_acc= 51%\nThe results are less than stellar... but not bad. I am actually looking around this discussion forum for help to see what I might do better. How do we get into the 95% accuracy club?
\n\nAccording to F. Chollet:\nThe steps for fine-tuning a network are as follow:\n1. Add you custom network on top of an already-trained base network\n2. Freeze the base network.\n3. Train the part you added.\n4. Unfreeze some layers in the base network. (last few layers)\n5. Jointly train both these layers and the part you added.
\n\nLet me know if you see progress. Good luck!
", "votes": null, "replies": [] }, { "id": 464580, "author_name": "santhosh0304", "author_url": "", "post_date": "02/01/2019 05:25:13", "content": "Even I'm facing the same issue, my custom CNN is overfitting and its loss is getting reduced drastically. But, loss in the validation data remains high. No much improvement.
\n\nBefore working on this problem, I worked classifying a dog and cat. My custom model worked fine in that simple case. But in this breed identification, my model performance is poor just 6% on the validation set. Is this because of the 120 classes we have to predict?
\n\nIs there any way to get at least 50% accuracy using my custom CNN without using a pre-trained model?
\n\nThis is my structure of my custom model:
\n\n cls = Sequential()\n\n cls.add(Conv2D(64, (5, 5), activation='relu', input_shape=(64, 64, 3)))\n cls.add(Conv2D(64, (3, 3), activation='relu', input_shape=(64, 64, 3)))\n cls.add(Dropout(0.3))\n\n cls.add(MaxPool2D((2, 2), strides = 2))\n cls.add(Dropout(0.3))\n\n cls.add(Conv2D(64, (3, 3), activation='relu'))\n cls.add(Conv2D(64, (1, 1),strides = 2, activation='relu'))\n\n cls.add(MaxPool2D((2, 2)))\n cls.add(Dropout(0.2))\n\n cls.add(Flatten())\n cls.add(Dense(516, kernel_initializer='uniform'))\n cls.add(Activation('relu'))\n cls.add(Dropout(0.3))\n cls.add(Dense(258, kernel_initializer='uniform'))\n cls.add(Activation('relu'))\n cls.add(Dropout(0.2))\n\n cls. add(Dense(120))\n cls.add(Activation('softmax'))\n\n # configure the learning params of the classifier\n cls.compile(optimizer='SGD', loss='categorical_crossentropy',\n metrics=['accuracy']\n\n )\n