{"cells":[{"metadata":{"id":"2kIWaR5ZpKlJ"},"cell_type":"markdown","source":"## Dog Breed Classification\n\nIn this project we will use traditional CNN, CNN with data augmentation and finally transfer Learning by VGG16 model with weights pre-trained on Imagenet to solve the dog breed classification problem","execution_count":null},{"metadata":{"id":"F7MDmaAw2xGO"},"cell_type":"markdown","source":"### Load Dataset Files","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n","execution_count":null,"outputs":[]},{"metadata":{"id":"BZWpQv1OwqYK","outputId":"5f6f6c04-3e60-40f2-94d5-7c0a2152d6c6","trusted":true},"cell_type":"code","source":"# Utilities\nimport sys\nimport h5py\nimport warnings\nfrom time import time\nimport os\nfrom os import path\nfrom zipfile import ZipFile\nimport cv2\nimport pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport random\nfrom scipy import ndarray\n\nimport skimage as sk\nfrom skimage import transform\nfrom skimage import util\nfrom sklearn import preprocessing\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import confusion_matrix\n\n# KERAS MODULES\nfrom keras.utils import np_utils\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.models import Sequential,Model\nfrom keras.layers import Dense\nfrom keras.layers import Dropout\nfrom keras.layers import Flatten\nfrom keras.layers.convolutional import Conv2D\nfrom keras.layers.convolutional import MaxPooling2D\nfrom keras.layers import BatchNormalization\nfrom keras.callbacks import ModelCheckpoint,ReduceLROnPlateau,CSVLogger\nfrom keras.preprocessing.image import load_img, img_to_array, array_to_img, ImageDataGenerator\nimport keras\nimport tensorflow as tf \n\nfrom IPython.core.interactiveshell import InteractiveShell\nInteractiveShell.ast_node_interactivity = \"all\"\n\nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{"id":"1q2zzIaUprk_"},"cell_type":"markdown","source":"Now, upload the given dataset file shared with you in your google drive and give its path for the below given `project_path` variable. For example, a path is given below according to the file path in our google drive. You need to change this to match the path of yours.","execution_count":null},{"metadata":{"id":"Tp6FvAToxUFs","trusted":true},"cell_type":"code","source":"project_path = \"/kaggle/input/dog-breed-identification\"","execution_count":null,"outputs":[]},{"metadata":{"id":"aYmJKmDqqpng"},"cell_type":"markdown","source":"### Read labels.csv file using pandas","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nos.chdir(project_path)","execution_count":null,"outputs":[]},{"metadata":{"id":"WmlJ2VMY96IZ","trusted":true},"cell_type":"code","source":"labels = pd.read_csv(\"labels.csv\")","execution_count":null,"outputs":[]},{"metadata":{"id":"hPvb1RSc96If","outputId":"de68a5e6-74c2-4277-bff9-6053ef5c8e57","trusted":true},"cell_type":"code","source":"labels.head()","execution_count":null,"outputs":[]},{"metadata":{"id":"ufz8ujOAAior","outputId":"f5afa3e4-cfa5-4124-f0a8-5824c992125a","trusted":true},"cell_type":"code","source":"labels.shape","execution_count":null,"outputs":[]},{"metadata":{"id":"QP8YAzQvqyK-"},"cell_type":"markdown","source":"### Print the count of each category of Dogs given in the dataset\n\n","execution_count":null},{"metadata":{"id":"3L2naXlr96Im","trusted":true},"cell_type":"code","source":"breed_count = labels['breed'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"id":"CLm3W5RN96Ir","outputId":"b7e75059-0d2c-400c-bb18-30159c982bec","trusted":true},"cell_type":"code","source":"breed_count","execution_count":null,"outputs":[]},{"metadata":{"id":"WI94_Qcc0D4M"},"cell_type":"markdown","source":"### Get one-hot encodings of labels","execution_count":null},{"metadata":{"id":"Q48iAcY196I3","trusted":true},"cell_type":"code","source":"targets_labels = pd.Series(labels['breed'])\none_hot = pd.get_dummies(targets_labels,sparse=True)\none_hot_labels = np.asarray(one_hot)","execution_count":null,"outputs":[]},{"metadata":{"id":"9nlWmRNM96I8","outputId":"d04fa3fa-c422-456b-abeb-aa6ec6a4e634","trusted":true},"cell_type":"code","source":"one_hot_labels[0]","execution_count":null,"outputs":[]},{"metadata":{"id":"VWaJ9naXfoiU"},"cell_type":"markdown","source":"## Preparing training dataset\n1. Write a code which reads each and every id from labels.csv file and loads the corresponding image (in RGB - 128, 128, 3) from the train folder. <br>\n2. Create 2 variables <br> \n     a.  x_train - Should have all the images of the dogs from train folder <br>\n     b.  y_train - Corresponding label of the dog <br>\n<u>Note:</u> The id of the dog images and its corresponding labels are available in labels.csv file   \n<u>Hint:</u> Watch the video shared on \"Preparing the training dataset\" if you face issue on creating the training dataset","execution_count":null},{"metadata":{"id":"aC2f9ecR0XGR","trusted":true},"cell_type":"code","source":"img_rows = 128\nimg_cols = 128","execution_count":null,"outputs":[]},{"metadata":{"id":"nkkZEpOe0ipk","outputId":"e53bacc9-4132-40f3-810e-35cef742ff02","trusted":true},"cell_type":"code","source":"x_features = []\ny_features = []\nfrom tqdm import tqdm\nfor f,img_label in tqdm(labels.values): # f for format ,jpg\n    img = cv2.imread('./train/{}.jpg'.format(f), 1)\n    img_resize = cv2.resize(img, (img_rows, img_cols)) \n    x_features.append(img_resize)\n    y_features.append(img_label)","execution_count":null,"outputs":[]},{"metadata":{"id":"H5DZ6ktU_ibs","outputId":"b80a7d4d-ff05-4182-c125-57a3d594667e","trusted":true},"cell_type":"code","source":"plt.imshow(x_features[0])","execution_count":null,"outputs":[]},{"metadata":{"id":"ngHyB49v_rKB","outputId":"fd303c9c-95d5-465a-a3a4-8b4abdda4cc7","trusted":true},"cell_type":"code","source":"y_features[0]","execution_count":null,"outputs":[]},{"metadata":{"id":"6ioWDEgElBOs"},"cell_type":"markdown","source":"Normalize the training data and convert into 4 dimensions so that it can be used as an input to conv layers in the model","execution_count":null},{"metadata":{"id":"ARn76j3U1CDa","outputId":"81ea25e4-0ace-4c93-af82-f71a83f98217","trusted":true},"cell_type":"code","source":"x_train_data = np.asarray(x_features,np.float32) / 255\nprint(x_train_data.shape)","execution_count":null,"outputs":[]},{"metadata":{"id":"1egyj0sBNfR7","outputId":"fe4e3443-4893-4365-8267-a6b02812c915","trusted":true},"cell_type":"code","source":"#y_train.shape\n\nlabel_encoder = preprocessing.LabelEncoder()\nencodedlabels = label_encoder.fit_transform(y_features)\nprint('Classes'+str(label_encoder.classes_))\ny_train_encoded = np_utils.to_categorical(encodedlabels)\nclasses = y_train_encoded.shape[1]\nprint(str(classes))","execution_count":null,"outputs":[]},{"metadata":{"id":"bdCXuAE11gZL"},"cell_type":"markdown","source":"### Split the training and validation data from `x_train_data` and `y_train_data` obtained from above step","execution_count":null},{"metadata":{"id":"kpWx-pgV96Jv","trusted":true},"cell_type":"code","source":"#Split into train/validation set\n\nx_train,x_val,y_train,y_val = train_test_split(x_train_data,y_train_encoded,test_size=0.2,random_state=17)","execution_count":null,"outputs":[]},{"metadata":{"id":"XkL-N1jDsU8m"},"cell_type":"markdown","source":"### Loading the test data\nRead the id column from the samples_submission.csv and store it in test_img","execution_count":null},{"metadata":{"id":"DnpXdpd9b3E7","outputId":"dfb9d3ad-dc0b-4147-d776-58cf33816760","trusted":true},"cell_type":"code","source":"submission = pd.read_csv(\"sample_submission.csv\")\ntest_img = submission['id']\nsubmission.head()","execution_count":null,"outputs":[]},{"metadata":{"id":"DEJqZIMbm0Jo"},"cell_type":"markdown","source":"Run the below code to load the test image files in x_test_feature","execution_count":null},{"metadata":{"id":"zf7n4WG-b3Hv","outputId":"a6e978ff-4c7b-4507-80ea-1cb90ea97424","trusted":true},"cell_type":"code","source":"x_test_feature = []\ni = 0 # initialisation\nfor f in tqdm(test_img.values): # f for format ,jpg\n    img = cv2.imread('./test/{}.jpg'.format(f), 1)\n    img_resize = cv2.resize(img, (img_rows, img_cols)) \n    x_test_feature.append(img_resize)","execution_count":null,"outputs":[]},{"metadata":{"id":"9My6qSyDnE-_"},"cell_type":"markdown","source":"Normalize the test data and convert it into 4 dimensions","execution_count":null},{"metadata":{"id":"93n-IntMnJGI","trusted":true},"cell_type":"code","source":"x_test_data = np.asarray(x_test_feature,np.float32) / 255","execution_count":null,"outputs":[]},{"metadata":{"id":"COeTqJrcG4hP","outputId":"082fcd16-0f0e-4908-ac5a-fe43202b0ef0","trusted":true},"cell_type":"code","source":"x_train_data[0].shape","execution_count":null,"outputs":[]},{"metadata":{"id":"zKezNJVMsocP"},"cell_type":"markdown","source":"### Build a basic conv neural network with 2 conv layers (kernel sizes - 5 and 3) add layers as mentioned below for classification.\n\n1. Add a Dense layer with 256 neurons with `relu` activation\n\n2. Add a Dense layer with 120 neurons as final layer (as there are 120 classes in the given dataset) with `softmax` activation for classifiaction. ","execution_count":null},{"metadata":{"id":"D2jxTY2S96J4","outputId":"35464531-5768-447a-d07b-0b83c36371e7","trusted":true},"cell_type":"code","source":"class_count = breed_count.size\nIMAGE_DIMS = x_train_data[0].shape\nbase_model = Sequential()\nbase_model.add(Conv2D(filters=32, kernel_size=(5, 5), input_shape=IMAGE_DIMS, activation='relu'))\nbase_model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))\nbase_model.add(BatchNormalization(axis=3))\nbase_model.add(Flatten())\n# Dense layer with 256 neurons with relu activation\nbase_model.add(Dense(256, activation='relu'))\n# Dense layer with 120 neurons as final layer \nbase_model.add(Dense(class_count, activation='softmax'))\nbase_model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"id":"f_BAvCzo96J6","outputId":"7647af99-036c-44b2-cc37-a7b02595c278","trusted":true},"cell_type":"code","source":"base_model.summary()","execution_count":null,"outputs":[]},{"metadata":{"id":"ui8EXw6_oqpR"},"cell_type":"markdown","source":"### Use batch_size = 128 and epochs = 10 and execute the model","execution_count":null},{"metadata":{"id":"bNoT5Af2KnyA","outputId":"cb265c83-ed3b-44d6-887f-0e2d8a69140b","trusted":true},"cell_type":"code","source":"print(y_train[0])\nplt.imshow(x_train[0])","execution_count":null,"outputs":[]},{"metadata":{"id":"0PR9j5_Xozmd","outputId":"ad042fc6-022b-4300-f363-c2f8665bfc8b","trusted":true},"cell_type":"code","source":"EPOCHS=10\nBATCH_SIZE=125\nbase_model.fit(x_train, y_train, batch_size=BATCH_SIZE, nb_epoch=EPOCHS, validation_data=(x_val, y_val))","execution_count":null,"outputs":[]},{"metadata":{"id":"Z8hWaKmjoz69"},"cell_type":"markdown","source":"#The model accuracy is very poor !!!!","execution_count":null},{"metadata":{"id":"agJKkc6xtKiq"},"cell_type":"markdown","source":"### Use Data Augmentation in the above model to see if the accuracy improves\n","execution_count":null},{"metadata":{"id":"ZxhK6G5u3_22","outputId":"cb3f9b17-5b1f-47ca-dda4-65e2831b3507","trusted":true},"cell_type":"code","source":"model_augmented = Sequential()\n\nmodel_augmented.add(Conv2D(filters=64, kernel_size=(5, 5), input_shape=IMAGE_DIMS, activation='relu'))\nmodel_augmented.add(BatchNormalization(axis=3))\nmodel_augmented.add(Conv2D(filters=64, kernel_size=(5, 5), activation='relu'))\nmodel_augmented.add(MaxPooling2D((2, 2)))\nmodel_augmented.add(BatchNormalization(axis=3))\nmodel_augmented.add(Dropout(0.1))\n\nmodel_augmented.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu'))\nmodel_augmented.add(BatchNormalization(axis=3))\nmodel_augmented.add(Conv2D(filters=128, kernel_size=(3, 3), activation='relu'))\nmodel_augmented.add(MaxPooling2D((2, 2)))\nmodel_augmented.add(BatchNormalization(axis=3))\nmodel_augmented.add(Dropout(0.1))\n\nmodel_augmented.add(Flatten())\nmodel_augmented.add(Dense(256, activation='relu'))\nmodel_augmented.add(BatchNormalization())\nmodel_augmented.add(Dropout(0.5))\n\nmodel_augmented.add(Dense(classes, activation='softmax'))\n\nmodel_augmented.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n\nmodel_augmented.summary()","execution_count":null,"outputs":[]},{"metadata":{"id":"gDLQVFDP96KI","trusted":true},"cell_type":"code","source":"# SETTING UP CHECKPOINTS, CALLBACKS AND REDUCING LEARNING RATE\nlrr = ReduceLROnPlateau(monitor='val_acc', \n                        patience=3, \n                        verbose=2, \n                        factor=0.4, \n                        min_lr=0.00001)\n\nfilepath=\"weights.best_{epoch:02d}-{val_acc:.2f}.hdf5\"\ncheckpoints = ModelCheckpoint(filepath, monitor='val_acc', \n                              verbose=2, save_best_only=True, mode='max')\ncallbacks_list = [checkpoints, lrr]","execution_count":null,"outputs":[]},{"metadata":{"id":"31Mn8qnZb3Ru","trusted":true},"cell_type":"code","source":"train_generator = ImageDataGenerator(rotation_range=90, width_shift_range=0.1,\n                     height_shift_range=0.1, shear_range=0.2, zoom_range=0.2,\n                     horizontal_flip=True, fill_mode=\"nearest\")\ntrain_generator.fit(x_train)\nval_generator = ImageDataGenerator(rotation_range=90, width_shift_range=0.1,\n                height_shift_range=0.1, shear_range=0.2, zoom_range=0.2,\n                horizontal_flip=True, fill_mode=\"nearest\")\nval_generator.fit(x_val)","execution_count":null,"outputs":[]},{"metadata":{"id":"6sssbaTfxlkk"},"cell_type":"markdown","source":"### Using the above objects, create the image generators with variable names `train_generator` and `val_generator`\n\nYou need to use train_datagen.flow() and val_datagen.flow()","execution_count":null},{"metadata":{"id":"TVFQJZw3x4-C"},"cell_type":"markdown","source":"### Fit the model using fit_generator() using `train_generator` and `val_generator` from the above step with 10 epochs","execution_count":null},{"metadata":{"id":"J1K2MqHbuPUa","outputId":"937278cb-d37f-406c-ae10-cb0779b0ab2f","trusted":true},"cell_type":"code","source":"model_augmented.fit_generator(train_generator.flow(x_train, y_train, batch_size=BATCH_SIZE), steps_per_epoch=len(x_train) / BATCH_SIZE, \n                    epochs=EPOCHS, validation_data=val_generator.flow(x_val, y_val, batch_size=BATCH_SIZE), callbacks=callbacks_list)","execution_count":null,"outputs":[]},{"metadata":{"id":"Q2zmLztqo5DY"},"cell_type":"markdown","source":"# Model accuracy is still poor!!!","execution_count":null},{"metadata":{"id":"rSTATrhsAo7L"},"cell_type":"markdown","source":"### Lets use Transfer Learning\n\nDownload the vgg wieght file from here : https://github.com/MinerKasch/applied_deep_learning/blob/master/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5","execution_count":null},{"metadata":{"id":"zy5JdbW6pIvD"},"cell_type":"markdown","source":"Use the below code to load VGG16 weights trained on ImageNet","execution_count":null},{"metadata":{"id":"yrqs0zg7ApNw","trusted":true},"cell_type":"code","source":"from keras.applications.vgg16 import VGG16, preprocess_input\n# Instantiate the model with the pre-trained weights (no top)\nbase_model_VGG16= VGG16(weights=('/content/drive/My Drive/greatlakes/Projects/CNN/Project2/data/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5'),\n                 include_top=False, pooling='avg')","execution_count":null,"outputs":[]},{"metadata":{"id":"EItOlRBGpV_A"},"cell_type":"markdown","source":"Print the summary of the base_model","execution_count":null},{"metadata":{"id":"lQsEBgnlpHjH","outputId":"5e721aff-84e7-4308-9d85-b814c8315036","trusted":true},"cell_type":"code","source":"base_model_VGG16.summary()","execution_count":null,"outputs":[]},{"metadata":{"id":"fHpeOyW0qauW"},"cell_type":"markdown","source":"### Add the following classification layers to the imported VGG Model <br>\n1. Flatten Layer\n2. Dense layer with 1024 neurons with activation as Relu\n3. Dense layer with 256 neurons with activation as Relu\n4. Dense layer with 120 neurons with activation as Softmax","execution_count":null},{"metadata":{"id":"9zwK4cWDxhJ0","trusted":true},"cell_type":"code","source":"from keras.layers import Input\ninput = Input(shape=(128,128,3),name = 'image_input')\noutput_vgg16 = base_model_VGG16(input)","execution_count":null,"outputs":[]},{"metadata":{"id":"rGVdR8OXxp_h","trusted":true},"cell_type":"code","source":"transfer_learn_model = Dense(1024, activation='relu')(output_vgg16)\ntransfer_learn_model = BatchNormalization()(transfer_learn_model)\ntransfer_learn_model = Dense(256, activation='relu')(transfer_learn_model)\ntransfer_learn_model=  Dropout(0.3)(transfer_learn_model)\ntransfer_learn_model = Dense(120, activation='softmax', name='predictions',kernel_initializer='uniform')(transfer_learn_model)","execution_count":null,"outputs":[]},{"metadata":{"id":"0BpT4MLkqoaO","outputId":"78fc1a3f-1329-401d-f2ca-64677b5acafe","trusted":true},"cell_type":"code","source":"VGG16_transfer_learn_model = Model(input=input, output=transfer_learn_model)","execution_count":null,"outputs":[]},{"metadata":{"id":"LeQem0pHITIj"},"cell_type":"markdown","source":"### Make all the layers in the base_model (VGG16) to be non-trainable","execution_count":null},{"metadata":{"id":"C7w9CSPvIRnX","outputId":"b126c348-dad0-4abb-9e8b-83602f6b0364","trusted":true},"cell_type":"code","source":"#Freezing layers in the model which don't have 'dense' in their name\nfor layer in base_model_VGG16.layers:\n    #Freezing a layer\n    layer.trainable = False\n\n#Module to print colourful statements\nfrom termcolor import colored\n\n#Check which layers have been frozen \nfor layer in VGG16_transfer_learn_model.layers:\n  print (colored(layer.name, 'blue'))\n  print (colored(layer.trainable, 'red'))","execution_count":null,"outputs":[]},{"metadata":{"id":"kj-BwqgfIkdv"},"cell_type":"markdown","source":"### Fit and compile the model with batch_size = 128 and epochs = 10 and execute the model","execution_count":null},{"metadata":{"id":"YD5fAgVQIpKZ"},"cell_type":"markdown","source":"Try to get training and validation accuracy to be more than 90%","execution_count":null},{"metadata":{"id":"H3RExCEZzZcZ","trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"id":"Gge8pAF7z7xo","outputId":"3f4e9cb4-0552-4116-a0b9-cb2832c73bc7","trusted":true},"cell_type":"code","source":"VGG16_transfer_learn_model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\nVGG16_transfer_learn_model.summary()","execution_count":null,"outputs":[]},{"metadata":{"id":"SZk2SWvjIoRP","outputId":"9de10433-a9cb-4fe6-edbb-eb16c6163b26","trusted":true},"cell_type":"code","source":"EPOCHS=10\nBATCH_SIZE=128\nVGG16_transfer_learn_model.fit(x_train, y_train, batch_size=BATCH_SIZE, nb_epoch=EPOCHS, validation_data=(x_val, y_val), callbacks=callbacks_list)","execution_count":null,"outputs":[]},{"metadata":{"id":"yQh0Qws8da13","trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}