{"cells":[{"metadata":{},"cell_type":"markdown","source":"<font size=\"10\">TRANSFER LEARNING</font>[](http://)"},{"metadata":{},"cell_type":"markdown","source":"It is a technique when in which model pre-trained for one task is reused for another task.In practice, very few people train an entire Convolutional Network from scratch (with random initialization), because it is relatively rare to have a dataset of sufficient size. Instead, it is common to pretrain a ConvNet on a very large dataset (e.g. ImageNet, which contains 1.2 million images with 1000 categories), and then use the ConvNet either as an initialization or a fixed feature extractor for the task of interest. "},{"metadata":{},"cell_type":"markdown","source":"<font size=\"5\">Dog vs cat dataset</font>"},{"metadata":{},"cell_type":"markdown","source":"The training archive contains 25,000 images of dogs and cats.But we will use only 2000 images each of dogs and cats."},{"metadata":{},"cell_type":"markdown","source":"<font size=\"4\">Importing important libraries</font>"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"import cv2#open cv library for image reading\nimport matplotlib.pyplot as plt#for plotting of graph and visualising it \n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import os \nimport random\nimport gc","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"4\">Loading dataset into test and train data</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_dir='../input/train'\ntest_dir='../input/test'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"4\">Getting training images from dataset</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_dogs=['../input/train/{}'.format(i) for i in os.listdir(train_dir) if 'dog' in i]#get dog image\ntrain_cats=['../input/train/{}'.format(i) for i in os.listdir(train_dir) if 'cat' in i]#get dog image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"4\">Getting test images from dataset</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_image=['../input/test/{}'.format(i) for i in os.listdir(test_dir) ]#get test image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">Taking only 2000 images for training anf shuffling it</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_image=train_dogs[:2000]+train_cats[:2000]#slice dataset upto 2000 samples\nrandom.shuffle(train_image)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">Deleting  variables that are no longer required</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"del train_dogs\ndel train_cats","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gc.collect()#collect garbage to save memory","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"4\">Plotting some random training images</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.image as mpimg#importing image into numpy array\nfor ima in train_image[0:3]:\n    img=mpimg.imread(ima)\n    imgplot = plt.imshow(img)#plotting image as numpy array\n    plt.show()\n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Resizing image now**"},{"metadata":{"trusted":true},"cell_type":"code","source":"nrows=150\nncolums=150\nchannel=3","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#a function to read and process our data in acceptable format\ndef read_process(list_of_img):\n    x=[]#features\n    y=[]#labels\n    for image in list_of_img:\n        x.append(cv2.resize(cv2.imread(image,cv2.IMREAD_COLOR),(nrows,ncolums),interpolation=cv2.INTER_CUBIC))#read image\n        #get labels\n        if 'dog' in image:\n            y.append(1)\n        elif 'cat' in image:\n            y.append(0)\n    return x,y\nx, y = read_process(train_image)\n#x is now an array of image pixel values and y is a list of labels","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#display 5 train image \nplt.figure(figsize=(20,10))\ncolums=5\nfor i in range(colums):\n    plt.subplot(5/colums+1,colums,i+1)\n    plt.imshow(x[i])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import seaborn as sns\ndel train_image\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#convert to array \nx=np.array(x)\ny=np.array(y)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">visualising traning images</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.countplot(y)\nplt.title('labels for cats and dogs')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(x.shape)\nprint(y.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"4\">Spilitting into training and validating set</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"#spilit into train and validation set\nfrom sklearn.model_selection import train_test_split\nx_train,x_val,y_train,y_val=train_test_split(x,y,test_size=.20,random_state=2)\nprint(x_train.shape)\nprint(x_val.shape)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del x\ndel y\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"n_train=len(x_train)\nn_val=len(x_val)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"batch_size=32","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"4\">Using Resnet pretrained  model</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.applications import InceptionResNetV2\nconv_base=InceptionResNetV2(weights='imagenet',include_top=False,input_shape=(150,150,3))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras import  layers\nfrom keras import models\nfrom keras import optimizers\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.preprocessing.image import img_to_array,load_img","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"4\">Adding output fully connected layers to pretrained model </font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"#creating last layer and adding to to pre-trained model\nmodel=models.Sequential()\nmodel.add(conv_base)\nmodel.add(layers.Flatten())\nmodel.add(layers.Dense(256,activation='relu'))\nmodel.add(layers.Dense(1,activation='sigmoid'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#freeze conv_base and train our own only\nprint(\"no of trainable weight before freezing:\",len(model.trainable_weights))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">Freezing pretrained model and training only fully connected layers</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"conv_base.trainable=False\nprint(\"no of trainable weight after freezing:\",len(model.trainable_weights))\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">Specify loss and optimizers</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"#specify loss and optimizers\nmodel.compile(loss='binary_crossentropy',optimizer=optimizers.RMSprop(lr=.0002),metrics=['acc'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">Data argumentation</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"#data argumentation\ntrain_datagen=ImageDataGenerator(rescale=1./255, rotation_range=40,width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2,zoom_range=0.2,horizontal_flip=True)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"val_datagen=ImageDataGenerator(rescale=1./255)#only rescaling in validation set","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#creating image generators\ntrain_generator=train_datagen.flow(x_train,y_train,batch_size=batch_size)\nval_generator=val_datagen.flow(x_val,y_val,batch_size=batch_size)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"4\">Training the dataset</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"#training the dataset\nhistory=model.fit_generator(train_generator,steps_per_epoch=n_train//batch_size,epochs=20,validation_data=val_generator,validation_steps=n_val//batch_size)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">Saving the trained model</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"#saving trained models so tey can be reused\nmodel.save_weights('model_wieghts.h5')\nmodel.save('model_keras.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<font size=\"3\">Predicting test images</font>"},{"metadata":{"trusted":true},"cell_type":"code","source":"#predicting first 10 images\nx_test,y_test=read_process(test_image[0:10])\nX=np.array(x_test)\ntest_datagen=ImageDataGenerator(rescale=1./255)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#loop that iterates over the Images from the generator to make predictions. \ni=0\ntext_labels=[]\nplt.figure(figsize=(30,20))\nfor batch in test_datagen.flow(X,batch_size=1):\n    pred=model.predict(batch)\n    if pred>0.5:\n        text_labels.append('dogs')\n    else:\n         text_labels.append('cats')\n    plt.subplot(5/colums+1,colums,i+1)\n    plt.title(text_labels[i])\n    imgplot=plt.imshow(batch[0])\n    i+=1\n    if i%10==0:\n        break\n    plt.show()\n        \n","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}