{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# TensorFlow Image Classification Guide\n\n**In this notebook, I will demonstrate how to create image classification models with Tensorflow.**\n\n --------------------------------------------------------------------------------------------------\n**Notebook Prerequisites:**\n- Python                               > https://www.kaggle.com/learn/python\n- Data Visualization with Matplotlib   > https://www.kaggle.com/learn/data-visualization\n- Basic Linear Algebra                 > https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab\n- Basic Knowledge of Machine Learning  > https://www.kaggle.com/learn/intro-to-machine-learning\n\n\n---------------------------------------------------------------------------------------------------\n\n\n**This notebook will cover the following topics:**\n- Loading datasets from the Tensorflow Datasets API\n- Creating an `ImageDataGenerator` to process images\n- Performing image augmentation in memory to reduce overfitting\n- Building deep convolutional neural networks from scratch\n- Evaluating a model's performance\n- Creating custom callbacks\n- Using premade models with transfer learning\n\n\n\n*Note: You should research anything in this notebook that you do not understand. Some links will be provided*","metadata":{}},{"cell_type":"markdown","source":"# Load Horses or Humans Dataset\n\nWe will load the `horses_or_humans` dataset from the Tensorflow Dataset API. Datasets loaded in this way are already processed in a way where they can be sent directly to a model. However, because images are rarely ever in such a nice format, we will save the images into folders to replicate real-world conditions.\n\nIn order to use the `ImageDataGenerator`, the images need to be organized in a specific folder organization (shown below).\n\n                      Data                      <-----------------Root Directory \n                       /\\\n                      /  \\\n                     /    \\\n                    /      \\\n                   /        \\\n               Train        Test                <-----------------Data Subset Directory (train/val/test)\n               /   \\         /  \\\n              /     \\       /    \\\n             /       \\     /      \\\n        Horses   Humans  Horses   Humans        <-----------------Class Directory (name of data classes)","metadata":{}},{"cell_type":"code","source":"#Import Libraries\nimport tensorflow as tf\nimport tensorflow_datasets as tfds #Dataset API\nimport numpy as np #Linear Algebra\nimport matplotlib.pyplot as plt #Data visualization\nimport os #Manipulate Files\nfrom PIL import Image #Manipulate Images\n\nimport warnings\nwarnings.filterwarnings('ignore') #ignores warnings\n\n#Make sure Tensorflow is version 2.0 or higher\nprint('Tensorflow Version:', tf.__version__)","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:45:10.623541Z","iopub.execute_input":"2022-04-22T10:45:10.624227Z","iopub.status.idle":"2022-04-22T10:45:10.630286Z","shell.execute_reply.started":"2022-04-22T10:45:10.624179Z","shell.execute_reply":"2022-04-22T10:45:10.629722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Makes Folders to store images\nos.makedirs('Data', exist_ok=True)\nos.makedirs('Data/Train/Horses', exist_ok=True)\nos.makedirs('Data/Train/Humans', exist_ok=True)\nos.makedirs('Data/Test/Horses', exist_ok=True)\nos.makedirs('Data/Test/Humans', exist_ok=True)\n\nbase_path = os.getcwd()\nhorse_counter = 0\nhuman_counter = 0\n#The below code will save the dataset images into the folders created above\n#Note: This step is not required when using Tensorflow datasets but will be required when \n# using datasets that are in the wild or possibly on Kaggle\n#see horse or humans doc here ->https://www.tensorflow.org/datasets/catalog/horses_or_humans\nfor i, dataset in enumerate(tfds.load('horses_or_humans', split=['train', 'test'])):\n    if i==0: #training set\n        set_path = os.path.join(base_path, 'Data/Train')\n    else: #test set\n        set_path = os.path.join(base_path, 'Data/Test')\n        \n    for row in list(dataset):\n        im = Image.fromarray(row['image'].numpy())\n        if row['label'] == 0: #0 is horse and 1 is human\n            class_path = os.path.join(set_path, 'Horses')\n            file_path = os.path.join(class_path, \"horse_{}.jpeg\".format(horse_counter))\n            horse_counter += 1\n        elif row['label'] == 1: #0 is horse and 1 is human\n            class_path = os.path.join(set_path, 'Humans')\n            file_path = os.path.join(class_path, \"human_{}.jpeg\".format(horse_counter))\n            human_counter += 1\n        im.save(file_path) #saves the image in the proper folder\n","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:45:12.673858Z","iopub.execute_input":"2022-04-22T10:45:12.674301Z","iopub.status.idle":"2022-04-22T10:45:20.090109Z","shell.execute_reply.started":"2022-04-22T10:45:12.67427Z","shell.execute_reply":"2022-04-22T10:45:20.08907Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Number of Horse Images in the Training Set:', len(os.listdir('Data/Train/Horses')))\nprint('Number of Human Images in the Training Set:', len(os.listdir('Data/Train/Humans')))\nprint('\\n')\nprint('Number of Horse Images in the Testing Set:', len(os.listdir('Data/Test/Horses')))\nprint('Number of Human Images in the Testing Set:', len(os.listdir('Data/Test/Humans')))","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:46:29.474657Z","iopub.execute_input":"2022-04-22T10:46:29.474972Z","iopub.status.idle":"2022-04-22T10:46:29.483168Z","shell.execute_reply.started":"2022-04-22T10:46:29.474928Z","shell.execute_reply":"2022-04-22T10:46:29.48256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Print Sample Images\nhorse_imgs = []\nhuman_imgs = []\n\nfor i in range(5):\n    horse_im = Image.open(os.path.join('Data/Train/Horses', os.listdir('Data/Train/Horses')[i]))\n    human_im = Image.open(os.path.join('Data/Train/Humans', os.listdir('Data/Train/Humans')[i]))\n    horse_imgs.append(horse_im)\n    human_imgs.append(human_im)\n    \n\nplt.rcParams[\"figure.figsize\"] = (20,5)\nfig, axs = plt.subplots(2, 5)\nfor i in range(2):\n    for j in range(5):\n        if i == 0:\n            axs[i, j].imshow(horse_imgs[j])\n        else:\n            axs[i, j].imshow(human_imgs[j])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:46:31.440081Z","iopub.execute_input":"2022-04-22T10:46:31.440528Z","iopub.status.idle":"2022-04-22T10:46:32.845364Z","shell.execute_reply.started":"2022-04-22T10:46:31.440495Z","shell.execute_reply":"2022-04-22T10:46:32.844406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# `ImageDataGenerator` that Loads Images and Performs Image Augmentation\n\nThe `ImageDataGenerator` will be able automatically detect the different classes in our dataset from the folder structure that was setup in the previous section. The `ImageDataGenerator` will then take each of these images and apply transformations (such as rotations) as to augment our image dataset. After this, the data will be ready to be fed to a machine learning model.","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n\n#Here we initialize an image generator that will conduct in-memory image augmentation\n#see here for docs -> https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator\nprint('Training Set:')\ntrain_gen = ImageDataGenerator(\n    rescale=(1./255), #Rescales pixel values (originally 0-256) to 0-1\n    rotation_range=0.4, #Rotates the image up to 40 degrees in either direction\n    shear_range=0.2, #shears the image up to 20 degrees\n    width_shift_range=0.2, #shifts the width by up to 20 %\n    height_shift_range=0.2, #shifts the height by up to 20 %\n    horizontal_flip=True, #flips the image along the horizontal axis\n    fill_mode='nearest' #fills pixels lost during transformations with its nearest pixel\n    )\n\ntrain_generator = train_gen.flow_from_directory(\n    'Data/Train',\n    target_size=(150,150),\n    batch_size=32,\n    class_mode='binary'\n)\n\n#Now we do the same thing for the test set but do not include any augmentations except for pixel rescaling\nprint('Testing Set:')\ntest_gen = ImageDataGenerator(rescale=(1./255))\n\ntest_generator = test_gen.flow_from_directory(\n    'Data/Test',\n    target_size=(150,150),\n    batch_size=32,\n    class_mode='binary'\n)","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2022-04-22T10:46:43.801081Z","iopub.execute_input":"2022-04-22T10:46:43.801839Z","iopub.status.idle":"2022-04-22T10:46:45.050179Z","shell.execute_reply.started":"2022-04-22T10:46:43.801786Z","shell.execute_reply":"2022-04-22T10:46:45.048976Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Custom Callback\n\nCallbacks are used in Tensorflow to allow user intervention during model training. A callback can be executed at a number of specific intances during model training. \nFor example: \n- `on_batch_begin`/`end`\n- `on_epoch_begin`/`end`\n- `on_predict_batch_begin`/`end`\n- `on_predict_begin`/`end`\n- `on_test_batch_begin`/`end`\n- `on_test_begin`/`end`\n- `on_train_batch_begin`/`end`\n- `on_train_begin`/`end`\n\nWe will create `CustomCallback` which will stop the model from training once the model reaches 95% acccuracy on the training set.\n\nLink: https://keras.io/api/callbacks/","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.callbacks import Callback\n\n#creates a custom callback class\nclass CustomCallback(Callback):\n    \"\"\"\n    This callback will stop the model from training once the model reaches 95% accuracy on the training data\n    \"\"\"\n    def on_epoch_end(self, epoch, logs={}):\n        if logs.get('accuracy') > 0.95:\n            print('Accuracy above 95% -- Stopping Training')\n            self.model.stop_training = True #stops model training\n\nmy_callback = CustomCallback()","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:46:55.090959Z","iopub.execute_input":"2022-04-22T10:46:55.091514Z","iopub.status.idle":"2022-04-22T10:46:55.099231Z","shell.execute_reply.started":"2022-04-22T10:46:55.091475Z","shell.execute_reply":"2022-04-22T10:46:55.098219Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Predefined Callback - `LearningRateScheduler`\n\nThere are also a number of predefined callbacks. We will use the `LearningRateScheduler` to dynamically update the learning rate of our optimizer.","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.callbacks import LearningRateScheduler\n\n#creates a function that updates the learning rate based on the epoch number\ndef lr_update(epoch, lr):\n    \"\"\"\n    For the first 5 epochs the learning rate will be 0.005.\n    From epoch 6 and on, the learning rate will be reduced 1% per epoch\n    \"\"\"\n    if epoch <= 5:\n        return 0.005\n    else:\n        return lr * 0.99\n    \nlr_scheduler = LearningRateScheduler(lr_update)","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:46:57.474775Z","iopub.execute_input":"2022-04-22T10:46:57.475591Z","iopub.status.idle":"2022-04-22T10:46:57.481291Z","shell.execute_reply.started":"2022-04-22T10:46:57.475534Z","shell.execute_reply":"2022-04-22T10:46:57.480426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Image Classifier Model from Scratch\n\nWe will create a basic convolution neural network to classifiy the images. Convolution neural networks typically follow the following pattern:\n\n\n**Convolution Layer -> Pooling Layer :: Repeated a number of times followed by -> Flatten -> Dense**\n\nWe will create a model with this architecture and train it on the training data.\nFor more information on what these layers are visit -> https://www.youtube.com/watch?v=YRhxdVk_sIs","metadata":{}},{"cell_type":"code","source":"#Creates a model with the architecture mentioned above\n\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Conv2D, MaxPool2D, Flatten, Dropout\n\n\nmodel = Sequential([\n    Conv2D(16, (3,3), activation='relu', input_shape=(150,150,3)),\n    MaxPool2D((2,2)),\n    Conv2D(32, (3,3), activation='relu'),\n    MaxPool2D((2,2)),\n    Conv2D(64, (3,3), activation='relu'),\n    MaxPool2D((2,2)),\n    Flatten(),\n    Dense(512, activation='relu'),\n    Dropout(0.3),\n    Dense(1, activation='sigmoid')\n])\n\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:47:00.319423Z","iopub.execute_input":"2022-04-22T10:47:00.320037Z","iopub.status.idle":"2022-04-22T10:47:00.485868Z","shell.execute_reply.started":"2022-04-22T10:47:00.319985Z","shell.execute_reply":"2022-04-22T10:47:00.484866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Compiles and trains the model\n\nfrom tensorflow.keras.optimizers import Adam\n\nmodel.compile(loss='binary_crossentropy',\n              optimizer=Adam(),\n              metrics=['accuracy'])\n\nhistory = model.fit(\n    train_generator,\n    epochs=20,\n    callbacks=[my_callback, lr_scheduler]\n)","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:47:03.781599Z","iopub.execute_input":"2022-04-22T10:47:03.781935Z","iopub.status.idle":"2022-04-22T10:50:23.665424Z","shell.execute_reply.started":"2022-04-22T10:47:03.781898Z","shell.execute_reply":"2022-04-22T10:50:23.664712Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Plots model training history\n\nfig, axs = plt.subplots(1, 2)\n\naxs[0].plot(history.history['loss'])\naxs[0].set_xlabel('Epoch')\naxs[0].set_ylabel('Loss')\naxs[0].set_title('Training Loss')\n\naxs[1].plot(history.history['accuracy'])\naxs[1].set_xlabel('Epoch')\naxs[1].set_ylabel('Accuracy')\naxs[1].set_title('Training Accuracy')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:53:58.221225Z","iopub.execute_input":"2022-04-22T10:53:58.221522Z","iopub.status.idle":"2022-04-22T10:53:58.547354Z","shell.execute_reply.started":"2022-04-22T10:53:58.221493Z","shell.execute_reply":"2022-04-22T10:53:58.546276Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Evaluate Custom CNN on Test Data\n#We will now see how well our model performs on test data. \n#It is likely that the model is overfitting the training data.\ntest_acc = model.evaluate(test_generator, verbose=0)[1]\nprint('Model Accuracy on Test Data:', round(test_acc,3))","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:54:12.94657Z","iopub.execute_input":"2022-04-22T10:54:12.946948Z","iopub.status.idle":"2022-04-22T10:54:13.892102Z","shell.execute_reply.started":"2022-04-22T10:54:12.946905Z","shell.execute_reply":"2022-04-22T10:54:13.891427Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Transfer Learning to Increase Accuracy\n\nThe accuracy of our basic CNN was not very high. Luckily, we are able to use models created and trained by others on our classification problem. \n\nThe steps to conduct transfer learning are as follows:\n1. Download the base model and weights\n2. Make the base model untrainable (lock the weights)\n3. Add a few layers to the end of the model \n4. Train the new model","metadata":{}},{"cell_type":"code","source":"#1. Download the base model which we will use for transfer learning 'MobileNetV2'\nfrom tensorflow.keras.applications import MobileNetV2\n\nbase_model = MobileNetV2(\n            input_shape=(150,150,3),\n            include_top=False,\n            weights='imagenet')\n\n#2. Locks all the base model's weights\nfor layer in base_model.layers:\n    layer.trainable = False\n\n#3. Adds a few layers to the end of the model\nx = Flatten()(base_model.output)\nx = Dense(512, activation='relu')(x)\nx = Dropout(0.3)(x)\nx = Dense(1, activation='sigmoid')(x)\n\n\nnew_model = tf.keras.Model(base_model.input, x)\n\nnew_model.compile(loss='binary_crossentropy',\n              optimizer=Adam(),\n              metrics=['accuracy'])\n\n#4. Trains the new model\nhistory = new_model.fit(\n    train_generator,\n    epochs=20,\n    callbacks=[my_callback, lr_scheduler]\n)","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:54:19.395673Z","iopub.execute_input":"2022-04-22T10:54:19.396508Z","iopub.status.idle":"2022-04-22T10:54:43.834817Z","shell.execute_reply.started":"2022-04-22T10:54:19.396467Z","shell.execute_reply":"2022-04-22T10:54:43.833619Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Plots training history of the new model\nfig, axs = plt.subplots(1, 2)\n\naxs[0].plot(history.history['loss'])\naxs[0].set_xlabel('Epoch')\naxs[0].set_ylabel('Loss')\naxs[0].set_title('Transfer Model Training Loss')\n\naxs[1].plot(history.history['accuracy'])\naxs[1].set_xlabel('Epoch')\naxs[1].set_ylabel('Accuracy')\naxs[1].set_title('Transfer Model Training Accuracy')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:54:47.952693Z","iopub.execute_input":"2022-04-22T10:54:47.952994Z","iopub.status.idle":"2022-04-22T10:54:48.279963Z","shell.execute_reply.started":"2022-04-22T10:54:47.952965Z","shell.execute_reply":"2022-04-22T10:54:48.279321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Evaluation of the new model\ntest_acc = new_model.evaluate(test_generator, verbose=0)[1]\nprint('Model Accuracy on Test Data:', round(test_acc,3))","metadata":{"execution":{"iopub.status.busy":"2022-04-22T10:54:51.85124Z","iopub.execute_input":"2022-04-22T10:54:51.851936Z","iopub.status.idle":"2022-04-22T10:54:54.540964Z","shell.execute_reply.started":"2022-04-22T10:54:51.851891Z","shell.execute_reply":"2022-04-22T10:54:54.539949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Wow!! The transfer learning model performed extremely well with just a few epochs!**","metadata":{}},{"cell_type":"markdown","source":"### Now try out these methods for yourself!\n\n### Similar Notebooks\n**TensorFlow Natural Language Processing Guide**: https://www.kaggle.com/code/calebreigada/tensorflow-natural-language-processing-guide\n\n**TensorFlow Time Series Forecasting Guide**:\nhttps://www.kaggle.com/code/calebreigada/tensorflow-time-series-forecasting-guide","metadata":{}}]}