{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Dogs vs. Cats / InceptionV3 - Transfer Learning \n\n### Transfer Learning (TL)\nÈ transferir o conhecimento de um modelo para resolver outros problemas, ou seja, usamos modelos pré-treinados como ponto de partida na resolução de novos problemas.\n<hr />\n<img src='https://cdn-images-1.medium.com/max/1024/1*x3ldzOAdnUcky3Dqhtnjlw.jpeg' style='height:300px;float:left' />\n\n*\"Transfer learning (TL) is a research problem in machine learning (ML) that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem.[1] <br/>For example, knowledge gained while learning to recognize cars could apply when trying to recognize trucks. <br/>This area of research bears some relation to the long history of psychological literature on transfer of learning, although formal ties between the two fields are limited.\"* - [wikipedia](https://en.wikipedia.org/wiki/Transfer_learning)\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Inception v3 Model\nÈ um modelo convolucional com 48 camadas de profundidade. Esse modelo é um dos mais famosos para o uso de transferencia de aprendizado.\n<hr />\n*\"Inceptionv3[1] is a convolutional neural network for assisting in image analysis and object detection, and got its start as a module for Googlenet. It is the third edition of Google's Inception Convolutional Neural Network, originally introduced during the ImageNet Recognition Challenge. Just as ImageNet can be thought of as a database of classified visual objects, Inception helps classification of objects[2] in the world of computer vision.\" - [wikipedia](https://en.wikipedia.org/wiki/Inceptionv3)*\n\n<img style='width:900px' src='https://camo.githubusercontent.com/8b243e646673dd9234f39cf8bdd5da1c6f051fd9/68747470733a2f2f7777772e50657465724d6f7373416d6c416c6c52657365617263682e636f6d2f6d656469612f696d616765732f7265706f7369746f726965732f5472616e736665722d4c6561726e696e672e6a7067' />","execution_count":null},{"metadata":{"id":"FRv4vb3qaEyF"},"cell_type":"markdown","source":"# Setup","execution_count":null},{"metadata":{"id":"akHV9qWmZ5GA","trusted":true},"cell_type":"code","source":"!unzip -q   /kaggle/input/dogs-vs-cats-redux-kernels-edition/train.zip -d .\n!unzip -q  /kaggle/input/dogs-vs-cats-redux-kernels-edition/test.zip -d .","execution_count":null,"outputs":[]},{"metadata":{"id":"XEFvbXHvasxs","trusted":true},"cell_type":"code","source":"import tensorflow as tf\nfrom tensorflow import keras\nfrom keras.preprocessing.image import load_img,img_to_array\nimport numpy as np\nimport os\nimport matplotlib as mpl\nimport matplotlib.pyplot as plt\nimport pandas as pd\nimport time\nfrom tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping\nimport shutil\nfrom keras.preprocessing.image import ImageDataGenerator\n\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\nprint(\"GPU is {}\".format(tf.config.list_physical_devices('GPU')))\nprint(\"tensorflow version {}\".format(tf.__version__))\n\nprint(os.listdir(\"./\"))\n\n!nvidia-smi\n\nkeras.backend.clear_session()","execution_count":null,"outputs":[]},{"metadata":{"id":"YjeQJ71_apiz"},"cell_type":"markdown","source":"# Data Pre-processing and Visualization","execution_count":null},{"metadata":{"id":"2jWIHsvGgswV","trusted":true},"cell_type":"code","source":"\ndef show_cats_and_dogs(show=\"\",width=150,height=150, images_path ='./train/'):\n  cols = 25\n  limit = 100\n  index = 0\n  images = list()\n  vertical_images=[]\n \n  for path in os.listdir(images_path):\n    if show != \"\" and  (show in path)==False:\n          continue\n    index=index+1\n    if index%limit==0:\n        break\n    #keras.preprocessing.image\n    image = load_img(images_path+path, target_size=(width,height))\n    image= img_to_array(image) #to numpy\n    image_height, image_width, image_channel = image.shape\n    horizontal_side = np.ones((image_height, 5,  image_channel), dtype=np.float32)*255\n    \n    images.append(image)\n    images.append(horizontal_side)\n\n    if index%cols==0:\n      horizontal_image = np.hstack((images))\n      image_height, image_width, image_channel = horizontal_image.shape\n      vertical_side = np.ones((5, image_width,  image_channel), dtype=np.float32)*255\n      vertical_images.append(horizontal_image)\n      vertical_images.append(vertical_side)\n      images=list()\n  gallery=np.vstack((vertical_images)) \n  plt.figure(figsize=(12,12))\n  plt.xticks([])\n  plt.yticks([])\n  title={\"\":\"cães & gatos\",\n          \"cat\": \"gatos\",\n          \"dog\": \"cães\"}\n  plt.title(\"{} imagens de {} [ path {} ] .\".format(limit, title[show],images_path))\n  plt.imshow(gallery.astype(np.uint8))","execution_count":null,"outputs":[]},{"metadata":{"id":"GlcQRxVYetmV","trusted":true},"cell_type":"code","source":"# raw Dataset\nprint(\"O dataset possui {} imagens de gatos e cães para classificação.\".format(len(os.listdir(\"./train\"))))\nprint(\"O dataset de teste possui {}.\".format(len(os.listdir(\"./test\"))))\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Gatos","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"show_cats_and_dogs(show='cat')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Cães","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"show_cats_and_dogs(show='dog')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Ambos","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"show_cats_and_dogs(show='')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Test","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"show_cats_and_dogs(images_path='./test/')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Pre-processing","execution_count":null},{"metadata":{"id":"5VQhzMUJ201D","trusted":true},"cell_type":"code","source":"image_width,image_height = 150,150#299,299\nlabels =['dog','cat']\nfor d in labels:\n  dir_path = './train/' + d\n  if not os.path.exists(dir_path):\n    print('{} criado.'.format(dir_path))\n    os.mkdir(dir_path)\n  else:\n    print('{} já existe.'.format(dir_path))\n\n\ntrain_path =\"./train/\"\nfor  file in  os.listdir(train_path):\n  category = file.split(\".\")[0]\n  if '.jpg' in file:\n    if 'dog'in category: \n      shutil.copyfile(train_path+file,'./train/dog/'+ file)\n    elif 'cat'in category:  \n      shutil.copyfile(train_path+file,'./train/cat/'+ file)\n","execution_count":null,"outputs":[]},{"metadata":{"id":"3BUweZ334bW7","trusted":true},"cell_type":"code","source":"print(\"Total de cães:\\t{}\".format(sum([len(files) for r, d, files in os.walk('./train/dog/')])))\nprint(\"Total de gatos:\\t{}\".format(sum([len(files) for r, d, files in os.walk('./train/cat/')])))","execution_count":null,"outputs":[]},{"metadata":{"id":"jj3aho2Vg2Bj","trusted":true},"cell_type":"code","source":"keras.backend.clear_session()\nbatch_size=64\nvalidation_split=0.3\nval_size = 7500\ndataset_size = 17500 \ntrain_data_generator = ImageDataGenerator(rescale=1./255, horizontal_flip=True, validation_split=validation_split)\n\ntrain_datagenerator = train_data_generator.flow_from_directory(train_path,\n                                                     target_size=(image_width,image_height ),\n                                                     class_mode=\"categorical\",\n                                                     batch_size=batch_size,\n                                                     shuffle=True,\n                                                     subset='training')\n\nval_datagenerator = train_data_generator.flow_from_directory(train_path,\n                                                     target_size=(image_width,image_height),\n                                                     class_mode=\"categorical\",\n                                                     shuffle=True,\n                                                     batch_size=batch_size,\n                                                     subset='validation')\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# InceptionV3 - Using\n*Com seu peso pré-treinado.*","execution_count":null},{"metadata":{"id":"LP2h_fpb6hVr","trusted":true},"cell_type":"code","source":"inception_v3_model = keras.applications.InceptionV3(include_top=False, weights='imagenet',input_shape=(image_width,image_height,3))","execution_count":null,"outputs":[]},{"metadata":{"id":"kx-RidqL7507","trusted":true},"cell_type":"code","source":"keras.backend.clear_session()\nx = inception_v3_model.output\navg_pool2d = keras.layers.GlobalAveragePooling2D()(x)\ndense = keras.layers.Dense(512, activation= keras.activations.relu)(avg_pool2d)\noutput = keras.layers.Dense(2,activation=keras.activations.softmax)(dense)\nmodel = keras.Model(inputs=inception_v3_model.input, outputs=output,name = \"transfer_inception_v3\")\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Fine tune ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"freeze= np.round((len(model.layers)-len(model.layers)*0.3),0).astype('int') \nfor layer in model.layers[:freeze]:\n    layer.trainable =False\nfor layer in model.layers[freeze:]:\n    layer.trainable=True\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualization ","execution_count":null},{"metadata":{"id":"NpCGuVlO2i0H","trusted":true},"cell_type":"code","source":"tf.keras.utils.plot_model( model)","execution_count":null,"outputs":[]},{"metadata":{"id":"8D9uunIc9vlj","trusted":true},"cell_type":"code","source":"plateau_callback = keras.callbacks.ReduceLROnPlateau(monitor='val_accuracy', patience=3, verbose=2, factor=.5, min_lr=.00001)\n\nstart = time.time()\n\nmodel.compile(loss=keras.losses.BinaryCrossentropy(from_logits=True), \n              optimizer=keras.optimizers.RMSprop(lr=0.0005, decay = 1e-6, momentum = 0.9),\n              metrics=['accuracy'])\n\nhistory = model.fit(train_datagenerator,\n                  steps_per_epoch=(dataset_size//batch_size),\n                  epochs= 5, \n                  verbose=1,\n                  validation_data=val_datagenerator,\n                  validation_steps=(val_size//batch_size),\n                  callbacks=[plateau_callback]\n)                                              \n                                                                                          \n\nprint(\"training: \",time.time()-start)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Train Accuracy:{:.3f}\".format(history.history['accuracy'][-1]))\nprint(\"Val Accuracy:{:.3f}\".format(history.history['val_accuracy'][-1]))\nprint('')\nprint(\"Train Loss:{:.3f}\".format(history.history['loss'][-1]))\nprint(\"Val Loss:{:.3f}\".format(history.history['val_loss'][-1]))","execution_count":null,"outputs":[]},{"metadata":{"id":"INPYD9zaO3u2","trusted":true},"cell_type":"code","source":"score = model.evaluate_generator(val_datagenerator,verbose=1)\nprint('Val loss: ', score[0])\nprint('Val accuracy', score[1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Accuracy","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"epochs = list(range(1,len(history.history['accuracy'])+1))\nepochs\nplt.plot(epochs, history.history['accuracy'],epochs,history.history['val_accuracy'])\nplt.legend(('Training','Validation'))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Loss","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"epochs = list(range(1,len(history.history['loss'])+1))\nepochs\nplt.plot(epochs, history.history['loss'],epochs,history.history['val_loss'])\nplt.legend(('Training','Validation'))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"AmLm53_n6Kwr","outputId":"bd0e9d1f-25e7-44c0-ffcc-641d334f6b34","trusted":true},"cell_type":"code","source":"test_path =\"./test/\"\nif not os.path.exists(\"./test\"):\n  os.mkdir(\"./test\")\n  print('./test criado.')\n\ndir_path = \"./test/data\"\nif not os.path.exists(dir_path):\n  print('{} criado.'.format(dir_path))\n  os.mkdir(dir_path)\nelse:\n  print('{} já existe.'.format(dir_path))\nfor file in os.listdir(test_path):\n    if '.jpg' in file:\n        shutil.copyfile(test_path+file,dir_path+'/'+file)\n\nprint(\"Total de gatos:\\t{}\".format(sum([len(files) for r, d, files in os.walk(dir_path+'/')])))\n\ntest_path = dir_path+'/'\ntest_data_generator = ImageDataGenerator(rescale=1./255)\n\ntest_generator = test_data_generator.flow_from_directory(directory ='./test',\n                                                         target_size=(image_width,image_height),\n                                                     batch_size=batch_size,\n                                                     class_mode=None,\n                                                     shuffle=False)","execution_count":null,"outputs":[]},{"metadata":{"id":"7xRyFfM83eyh","trusted":true},"cell_type":"code","source":"predict = model.predict(test_generator,verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"index = 56\npath= test_generator.filenames[index]\nplt.figure(figsize=(4, 4))\nimg=load_img('./test/'+path, target_size=(image_width,image_height))\nplt.imshow(img)\nif (predict[index,1]) >= 1.:\n    label='Dog'\nelse:\n    label='Cat'\nplt.title(\"Class: {}\".format(label))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"8-2zIHgq4D8o","trusted":true},"cell_type":"code","source":"submission = pd.DataFrame({\n    'id':pd.Series(test_generator.filenames),\n    'label':pd.Series(predict[:,1])\n    })\nsubmission['id'] = submission.id.str.extract('(\\d+)')\nsubmission['id']=pd.to_numeric(submission['id']).astype('int')\nsubmission['label']=pd.to_numeric(submission['label']).astype('int')\nsubmission.to_csv(\"submission_v9.csv\",index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.head(10)","execution_count":null,"outputs":[]},{"metadata":{"id":"-A3rIyTCBf_2","trusted":true},"cell_type":"code","source":"shutil.rmtree(\"./test\")\nshutil.rmtree(\"./train\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Referencies\n* https://lisaong.github.io/mldds-courseware/03_TextImage/transfer-learning.slides.html\n* https://mc.ai/image-classification-a-comparison-of-dnn-cnn-and-transfer-learning-approach/\n* https://data-flair.training/blogs/transfer-learning/\n* https://www.kaggle.com/rohit1277/cat-dog-classifier-using-vgg16-transfer-learning\n* https://www.kaggle.com/serkanpeldek/keras-cnn-transfer-learnings-on-cats-dogs-dataset\n* https://arxiv.org/abs/1512.00567\n* https://software.intel.com/content/www/us/en/develop/articles/inception-v3-deep-convolutional-architecture-for-classifying-acute-myeloidlymphoblastic.html\n* https://www.mathworks.com/help/deeplearning/ref/inceptionv3.html\n* https://medium.com/@sh.tsang/review-inception-v3-1st-runner-up-image-classification-in-ilsvrc-2015-17915421f77c\n* https://www.kaggle.com/overload10/transfer-learning-using-inception-on-full-data","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}