{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#End-to-End Multi-class Dog Breed Classification\n\nThis notebook builds an end-to-end multi-class image classifier using Tensorflow 2.0 and TensorFlow Hub\n\n##1. Problem\n\nIdentifying the breed of a dog given an image of a dog.\n\n##2. Data\n\nThe data we're using is from Kaggle's dog breed identification competition.\n\nhttps://www.kaggle.com/c/dog-breed-identification/data\n\n##3. Evalution\n\nThe evalution is a file with prediction probabilities for each dog breed of each test image.\n\nhttps://www.kaggle.com/c/dog-breed-identification/overview/evaluation\n\n##4. Features\n\nSome information about data:\n* We're dealing with images(Unstructured data) so it's probably best we use deep learning/transfer learning\n* There are 120 breeds of dog (this means therer are 120 diffrent classes).\n* There are aroung 10,000+ images in training set (have labels)\n* There are around 10,000+ images in test set(no labels)","metadata":{"id":"NLYI0lUdyVc_"}},{"cell_type":"code","source":"","metadata":{"id":"Ik3Jb9chX8Sb"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Unzip the uploaded data into Google Drive\n#!unzip \"/content/drive/MyDrive/Dog-Vision/dog-breed-identification.zip\" -d \"/content/drive/MyDrive/Dog-Vision/\"","metadata":{"id":"4Fy-gSpvaTD_"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### **Get our workspace ready**\n* Import TensorFlow\n* Import Tesnsorflow Hub\n* Make sure we're using GPU","metadata":{"id":"Xnrt7pqx1d2J"}},{"cell_type":"code","source":"#Import necessary tools\n\nimport tensorflow as tf\nimport tensorflow_hub as hub\nprint(\"TF version : \",tf.__version__)\nprint(\"TF hub version : \",hub.__version__)\n\n#Check for GPU availability\nprint(\"GPU\", \"available (YESSSSS!!!!)\" if tf.config.list_physical_devices(\"GPU\") else \"not available :(\")","metadata":{"id":"yQ0vyIhWqI68","outputId":"abd3d229-06b3-4c2b-932e-cba7ce0968c1"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Getting our data ready (turning into Tensors)\n\nWith  all machine learning models, our data has to be in numerical format.so that's whar we'll be doing first. Turning out images into Tensors.\n\nLet's start by accessing out data and check out the labels.","metadata":{"id":"m8cA1NM315pI"}},{"cell_type":"code","source":"import pandas as pd\nlabels_csv=pd.read_csv(\"/content/drive/MyDrive/Dog-Vision/labels.csv\")\nprint(labels_csv.describe())","metadata":{"id":"yZHwv5wQRBmA","outputId":"991aa62b-3216-4c00-fead-1390b63c203f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_csv.head()","metadata":{"id":"j1iPDag8R9TX","outputId":"56df776e-8f21-42dc-ccf5-8264b79ef162"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#how many images are therre of each breed?\n\nlabels_csv[\"breed\"].value_counts().plot(kind=\"bar\", figsize=(20,10))","metadata":{"id":"rMfNk47-SPEU","outputId":"de0ee3b8-1931-4d67-8107-75fcbc858f80"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_csv[\"breed\"].value_counts().median()","metadata":{"id":"7q0ak-UfSuuE","outputId":"e146ea43-36ed-4571-b70b-d2a64aa60b04"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Let's view an image\nfrom IPython.display import Image\nImage(\"/content/drive/MyDrive/Dog-Vision/train/000bec180eb18c7604dcecc8fe0dba07.jpg\")","metadata":{"id":"G6430GbgTQja","outputId":"820ece9c-5841-4780-b117-c6020e7aaaea"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Getting images and their labels\n\nLet's get alist of all of our images file pathnames.","metadata":{"id":"04UcSJfIVbPX"}},{"cell_type":"code","source":"#Creat pathnames from image ID's\nfilenames = [\"/content/drive/MyDrive/Dog-Vision/train/\" + fname + \".jpg\" for fname in labels_csv[\"id\"]]","metadata":{"id":"qrxqDhCpV_dE"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Check the first 10\nfilenames[:10]","metadata":{"id":"aqcvpVlRWi0I","outputId":"bae32e10-d6db-42ed-e6f3-748363b3cab1"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Check wheather number of filenames match number of actual image files\nimport os\n\nif len(os.listdir(\"/content/drive/MyDrive/Dog-Vision/train\")) == len(filenames):\n  print(\"Filenames Match actual amount of files..\")\nelse:\n  print(\"Filename do not match actual amount of files, check the target directory\") ","metadata":{"id":"fQs7VLMlWmz-","outputId":"9c4bfff7-50da-4989-fd15-305da5d63880"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#One more check \nImage(filenames[9000])\n#print(labels_csv[\"breed\"][9000])","metadata":{"id":"aERlZWRtYubq","outputId":"750a0d2c-c112-4321-c108-5f3986e51ced"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since we've now got out training filepaths in a listr, let's prepare our label","metadata":{"id":"6G2_ohMKcSi7"}},{"cell_type":"code","source":"","metadata":{"id":"AOqX34w7RUwB"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nlabels = labels_csv[\"breed\"]\nlabels = np.array(labels)\nlabels","metadata":{"id":"o990XsIDrcqH","outputId":"6c9b4819-b0e2-4fa9-f317-000580cdd773"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(labels)","metadata":{"id":"rdP4ZDdCsGdi","outputId":"e9d89685-c13d-4ff8-9044-236e9c737a69"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#See if number of labewls matches the number of filenames\nif len(labels)==len(filenames):\n  print(\"Matches\")\nelse:\n  print(\"Does'nt match\")","metadata":{"id":"DDvmft07sJ_B","outputId":"e69f2c9f-8cd5-4198-8112-d6d6915a3f54"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Find the unique label values\nUnique_breeds = np.unique(labels)","metadata":{"id":"PZfgRsWlsfdo"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(Unique_breeds)","metadata":{"id":"iyordIRJtHS3","outputId":"fa9af7b4-7a9c-400a-ec0b-0fe4c7b8b6f1"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Turn every label into a boolean array\nboolean_labels = [label==Unique_breeds for label in labels]\nboolean_labels","metadata":{"id":"5mvhgTbftSBo","outputId":"97bef161-32f5-42c4-bfda-e763fb4074e6"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Example : Turing boolean array into integers\nprint(labels[0])# original label\nprint(np.where(Unique_breeds== labels[0]))# index where label occurs\nprint(boolean_labels[0].argmax())#index where label occurs in boolean array\nprint(boolean_labels[0].astype(int))# there will be a 1 where the sample label occurs","metadata":{"id":"ZRl_pDU0uKOh","outputId":"0cc482b6-74d8-419b-905c-c7ecc1183594"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"type(boolean_labels)","metadata":{"id":"WVoc1CervoPa","outputId":"988a1915-e890-4d0c-ebe2-24fac5a910ea"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##Creating our own validation set\nSince the dataset from Kaggle doesn't come with a validation set, we're going to create our own.","metadata":{"id":"EDNa23AuwzjM"}},{"cell_type":"code","source":"#Setup X & y variables\nX=filenames\ny=boolean_labels","metadata":{"id":"IJwvDtuOyPGb"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We're going to start off experimenting with ~1000 images and increase as needed","metadata":{"id":"S_AMWJj4gfPR"}},{"cell_type":"code","source":"#Set number of images to use for experimenting\nNUM_IMAGES = 1000 #@param {type:\"slider\", min:1000, max:10000, step:1000}","metadata":{"id":"UqNlqW2oyYXj"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# splite our data into train and validation sets\nfrom sklearn.model_selection import train_test_split\n\n# split them into training and validation of total size NUM_IMAGES\nX_train, X_valid, y_train, y_valid = train_test_split(X[:NUM_IMAGES], y[:NUM_IMAGES], test_size=0.2, random_state=42)\n\nlen(X_train), len(y_train), len(X_valid), len(y_valid)","metadata":{"id":"lbrfw91MhcMX","outputId":"7f49518b-4347-4685-d5a6-ace8aebf431a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Let's have geez at the training data\nX_train[:2], y_train[:2]","metadata":{"id":"1RO6FATvjD5s","outputId":"5db80c52-48c9-476d-b6e4-4bbc79fac868"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Preprocessing Images (turing images into Tensors)\n \nTo preprocess our images into Tensors we're goin gto write a function which does a few things:\n\n1. Take an image filepath as input\n2. Use Tensorflow to read the file and save it to a variabel `image`\n3. Turn our `image` (a jpg) into Tensors\n4. Normalize our image (convert color channel values from 0-255 to 0-1)\n4. Resize the `image` to be a shape of (224,224)\n5. Return the modified `image`","metadata":{"id":"FWoMxuVPlvoI"}},{"cell_type":"markdown","source":"Before we do, Let's see what importing an image looks like...","metadata":{"id":"6LhRqZLon-DT"}},{"cell_type":"code","source":"# COnvert image to Numpy array\nfrom matplotlib.pyplot import imread\nimage= imread(filenames[2000])\nimage.shape","metadata":{"id":"Vd38asZkpKxx","outputId":"1a2ccb11-ebf3-4a29-f7c0-0336465cc067"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image.min(), image.max()","metadata":{"id":"mfgmTFgQpmq5","outputId":"d0db0a9b-40d3-416a-839c-ddb9015eb9d1"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image[:1]","metadata":{"id":"AcKDtvA_q8C3","outputId":"9d6faa8e-2ed0-4cfa-c1bb-848fe07d1a7a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#turn image into Tensor\ntf.constant(image[:1])","metadata":{"id":"nyAiDdK_rSgy","outputId":"1bccc715-fe1f-4699-86e0-b06189cbafc6"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we've seen what an image looks like Tensor, let's make function to preprocess them .","metadata":{"id":"1B2hmLldrrni"}},{"cell_type":"code","source":"#Define image size\nIMG_SIZE = 224\n\n#Create a function for preprocessing images\ndef process_image(image_path, img_size=IMG_SIZE):\n  \"\"\"\n  Take an image file path, image size and turns the image into a Tensor.\n  \"\"\"\n\n  #Read in an image file\n  image = tf.io.read_file(image_path)\n  \n  #Turn the jpeg image into numerical Tensor with 3 color channels (Red, Green and Blue)\n  image = tf.image.decode_jpeg(image, channels=3)\n\n  #Convert the color channel value from 0-255 to 0-1 values\n  image = tf.image.convert_image_dtype(image, tf.float32)\n\n  #Resize the image to our desired value (224,224)\n  image = tf.image.resize(image, size=[img_size, img_size])\n\n  return image","metadata":{"id":"FjXvk6Z6sJHc"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Turning data into batches\n\nWhy turn our data into batches?\n\nLet's say you're trying to process 10,000+ images in one go....\nthey all might not fit into memory\n\nSo that's why we do about 32 (this is batch size) images at a time (you can manualy adjust the batch size if need to be).\n\nIn order to use tensorflow effectively, we need our data in the form of Tensor tuple which look like this:\n`(image,label)`","metadata":{"id":"vYASvZqvz4nf"}},{"cell_type":"code","source":"#Create a simple function to return a tuple (image, label)\ndef get_image_label(image_path, label):\n\n  \"\"\"\n  Takes an image file path name and the assoscaited label,\n  processes the image and returns a tuple of(image, label).\n  \"\"\"\n\n  image= process_image(image_path)\n  return image, label","metadata":{"id":"447e5zAlX6kL"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we've got a way to turn our data into tuples of Tensors in the form: `(image,label)`, let's make a function to turn all of our data (X & y) into batches!","metadata":{"id":"atE0exFYZgo5"}},{"cell_type":"code","source":"# Define the batch size, 32 is good start\nBATCH_SIZE = 32\n\n#Create a function to turn data into batches\ndef create_data_batches(X, y=None, batch_size=BATCH_SIZE, valid_data=False, test_data=False):\n  \"\"\"\n  Create batches of data out of images (X) and label (y) pairs.\n  Shuffles the data if it's training data but doesn't shuffle if it's validation data.\n  Also accept test data as input (no label).\n  \"\"\"\n  # If the data is test dataset, we probably don't have labels\n  if test_data:\n    print(\"Creating test data batcehs...\")\n    data=tf.data.Dataset.from_tensor_slices((tf.constant(X))) #only filepaths (no labels)\n    data_batch = data.map(process_image).batch(BATCH_SIZE)\n    return data_batch\n\n  elif valid_data:\n    print(\"Creating Valid data batches...\")\n    data=tf.data.Dataset.from_tensor_slices((tf.constant(X), # filepaths\n                                            tf.constant(y))) # labels\n    data_batch = data.map(get_image_label).batch(BATCH_SIZE)\n\n    return data_batch\n\n  else:\n    print(\"Creatin training data batches...\")\n    #Turn filpaths and labels into Tensors\n    data=tf.data.Dataset.from_tensor_slices((tf.constant(X),\n                                             tf.constant(y)))\n    #Shuffling pathnames and labels before mapping image preocessor function is faster that than shuffling image\n    data = data.shuffle(buffer_size=len(X))\n  \n    #Create (image, label) tuple (this also turn the image path into a preprocessing image )\n    data_batch = data.map(get_image_label).batch(BATCH_SIZE)\n    return data_batch","metadata":{"id":"DzplUjeyauNd"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Create training and validation data batches\n\ntrain_data = create_data_batches(X_train, y_train)\nvalid_data = create_data_batches(X_valid, y_valid ,valid_data=True)","metadata":{"id":"1UumDJCT5eCi","outputId":"e1370909-9df6-437c-e15b-86accee1d42d"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data.element_spec,valid_data.element_spec","metadata":{"id":"xvKv31PV9NUL","outputId":"abf9c9fa-3f00-40a0-e03a-dbc05a0efa48"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualizing Data Batches\n\nOur data is now in batches, however, these can be a little hard to understand/comprehend, let's visualize them!\n","metadata":{"id":"L7XUcENRCW71"}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n#Create a function for viewing images in a data batch\ndef show_25_images(images, labels):\n  \"\"\"\n  Displays a plot of 25 images and their labels from data batch.\n  \"\"\"\n  fig=plt.figure(figsize=(10,10))\n  #loop through 25 (for display 25 images)\n  for i in range(25):\n    ax=plt.subplot(5,5,i+1)\n    ax.imshow(images[i])\n    #Add the image label as the title\n    ax.set_title(Unique_breeds[labels[i].argmax()])\n\n    #Turn the grid lines off\n    plt.axis(\"off\")","metadata":{"id":"fhI2spb97Nvb"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_images, train_labels = next(train_data.as_numpy_iterator())\ntrain_images, train_labels","metadata":{"id":"BWrifmzJ8D3Q","outputId":"1fa8dace-9364-4937-a6f2-9ae6c8050e73"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(train_images), len(train_labels)","metadata":{"id":"9gFb8zvCBRcO","outputId":"0b06cf68-5ee3-4533-ef35-3f28ba70a713"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Now let's visualize the data in a training batch\nshow_25_images(train_images, train_labels)","metadata":{"id":"t5QyAGgpBnyJ","outputId":"e305b075-d206-4662-fd85-36cec0d75d3e"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Now let's visualize our validation data\nvalid_images, valid_labels = next(train_data.as_numpy_iterator())\nshow_25_images(valid_images,valid_labels)","metadata":{"id":"KEvrCVom0av8","outputId":"012890a0-cba2-4091-cb8e-25a560868245"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##Buliding a model\n\nBefore we building a model, there are a few things we nee to define:\n\n* The input shape (our images shape, in the form of Tensors) to our model.\n* The output shape (image labels, in the form of Tensors) of our model.\n* The URL of the model we want to use.\n  from Tensorflow Hub : https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/4","metadata":{"id":"IWgmR5ae15Ap"}},{"cell_type":"code","source":"#Set input shape to the model\nINPUT_SHAPE = [None, IMG_SIZE, IMG_SIZE, 3] # batch ,height, width, color channels\n\n#Setup output shape of model\nOUTPUT_SHAPE = len(Unique_breeds)\n\n#Setup model URL from Tensorflow Hub\nMODEL_URL = \"https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/4\"\n","metadata":{"id":"KYoIIy-z3GKd"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we've got our inputs, outputs, and model ready to go.\nLet's put them togather into a keras deep learning model!\n\nKnowing this, let's create a functio nwhich:\n* Take the input shape, output shape and the model we've chosen as parameters.\n* Define the layer in keras model in sequential fastion (do this first, then this, then that).\n* Compiles the model (says it should be evaluated and improved).\n* Building the model (tells the model the inputs shape it'll be getting).\n* Return the model","metadata":{"id":"7EK4W3Yf-Skq"}},{"cell_type":"code","source":"#Create a function which builds a Keras model\ndef create_model(input_shape=INPUT_SHAPE, output_shape=OUTPUT_SHAPE, model_url=MODEL_URL):\n  print(\"Building model with : \",MODEL_URL)\n\n  #Setup the model layers\n  model =tf.keras.Sequential([\n                              hub.KerasLayer(MODEL_URL), # Layer 1 (Input layer)\n                              tf.keras.layers.Dense(units=OUTPUT_SHAPE, activation=\"softmax\") # Layer 2 (output layer)\n  ])\n\n  # Compile the model\n  model.compile(loss=tf.keras.losses.CategoricalCrossentropy(),\n                optimizer= tf.keras.optimizers.Adam(),\n                metrics=[\"accuracy\"]\n                )\n  \n  #Build the model\n  model.build(INPUT_SHAPE)\n\n  return model","metadata":{"id":"b5Tkew2cAFFl"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = create_model()\nmodel.summary()","metadata":{"id":"T1AQ82WZmeur","outputId":"27cef65a-ec67-40d3-ed71-061814565443"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Creating callbacks\n \n Callbacks are helper function a model can use during trainging to do such things as save it progress, check it progress or stop training if a model stops improving\n\n We'll create two callbacks, one for TensorBoard which helps track our model progress and another for early-stopping which prevents our model from training for too long.\n\n ### TensorBoard Callback\n To setup TensorBoard callback, we need to do 3 things:\n 1. Load the TensorBoard norebook extension\n 2. Create a TensorBoard callback which is able to save logs\n to directory and pass it to our model's `fit()` function\n 3. Visualize our model training logs with the `%tensorboard` magic function (we'll do this after model training).","metadata":{"id":"0gfJxtNjRxqL"}},{"cell_type":"code","source":"#Load TensorBoard notebook extension\n%load_ext tensorboard ","metadata":{"id":"Tkypf5PVRxsh"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import datetime\n\n# Craete a function to build TensorBoard callback\ndef create_tensorboard_callback():\n   # create a log directory for storing TensorBoard logs\n   logdir= os.path.join(\"/content/drive/MyDrive/Dog-Vision/logs\",\n                        #Make it so the  log get tracked whenever we run an experiment\n                        datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n                        )\n   return tf.keras.callbacks.TensorBoard(logdir)","metadata":{"id":"T2ItYYR6A_DG"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"###Early stopping callback\n\nEarly stopping helps stop our model from overfitting by stopping  training if a certain evalution metric stops improving","metadata":{"id":"bckldy-uZy2X"}},{"cell_type":"code","source":"#create early stoping call back\nearly_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"val_accuracy\",\n                                                  patience=3)","metadata":{"id":"tmP3xTcE4fbf"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training a model (on subset of data)\n\nOur first model is only going to train on 1000 images, to make sure everything is working.","metadata":{"id":"CyBiQkv8RW4n"}},{"cell_type":"code","source":"NUM_EPOCHS = 100 #@param {type:\"slider\", min:10 , max:100 , step:10}","metadata":{"id":"9L0-n5KXR4N1"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Cheak to make sur we are running on GPU\nprint(\"GPU\" , \"available\" if tf.config.list_physical_devices(\"GPU\") else \" not availble\")","metadata":{"id":"YfCwCYb0Sfs3"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's create a function which train a model\n\n* Create a model using `create_model()`\n* Setup a TensorBoard callbacks using 'create_tensorboard_callback()'\n* Call the `fit()` function on our model passing it the training data, validation data, number of epochs to train for (`NUM_EPOCHS`) and callbacks  we'd like to use\n* Return the model","metadata":{"id":"jBM_7P8xTGDN"}},{"cell_type":"code","source":"#Build a function to train and return a trained model\n\ndef train_model():\n  \"\"\"\n  Train a given model returns the trained version.\n  \"\"\"\n\n  #Create a model\n  model = create_model()\n\n  #Create new TensorBoard seesion everytime we train a model\n  tensorboard = create_tensorboard_callback()\n\n  #Fit the model to the data passing it the callbacks we created\n  model.fit(x=train_data,\n            epochs=NUM_EPOCHS,\n            validation_data=valid_data,\n            validation_freq=1,\n            callbacks=[tensorboard, early_stopping])\n  \n  #Return the fitted model\n  return model","metadata":{"id":"cmqCal3BUBbg"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Fit the moodel to the data\nmodel=train_model()","metadata":{"id":"TKCibEbxVHzw"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"###Checkin the TensorBoard logs\n\nThe TensorBoard fuction (%tensorboard) will access the logs directory we create earlier and visualize its content.","metadata":{"id":"Un3L8NreVWMt"}},{"cell_type":"code","source":"%tensorboard --logdir /content/drive/MyDrive/Dog-Vision/logs","metadata":{"id":"TWwQvhuYcNNF"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##Makeing and evaluting prediction using a train model","metadata":{"id":"Fpgi6KKicY9b"}},{"cell_type":"code","source":"#Make prediction on validation data (not used to train on)\npredictions= model.predict(valid_data, verbose=1)\npredictions","metadata":{"id":"zpPzueFPe8Fn"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predictions.shape","metadata":{"id":"iEnh-DG1fWOG"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#First prediction\nindex= 42\nprint(predictions[index])\nprint(f\"Max value (probability of prediction) : {np.max(predictions[index])}\")\nprint(f\"Sum : {np.sum(predictions[index])}\")\nprint(f\"Index number : {predictions[index].argmax()}\")\nprint(f\"Predicted label : {Unique_breeds[predictions[index].argmax()]}\")","metadata":{"id":"3N18wg7-fYlc"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Having the above functionality is great but we want to be able to do it at scale.\nAnd it would be even better if we could see the image the prediction is being made on!\n\n**Note :** Prediction probabilities are also known as confidence levels.","metadata":{"id":"kpl7vL_UiE0M"}},{"cell_type":"code","source":"#Turn prediction probabiltites into their respective label (easier to understand)\n\ndef get_pred_label(prediction_probabilities):\n  \"\"\"\n  Turns an array of predictio nprobabilities into a label.\n  \"\"\"\n  return Unique_breeds[np.argmax(prediction_probabilities)]\n\n# Get a prediction label based on an array of prediction probabilities\npred_label = get_pred_label(predictions[25])\npred_label","metadata":{"id":"jaKsemq5lMwP"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now since our validation data is still in batch dataset,\nwe'll have to unbatchify it to make prediction on the validation images and then compare those prediction to the validation laebls (truth labels).","metadata":{"id":"MoUr1UOtnAMa"}},{"cell_type":"code","source":"valid_data","metadata":{"id":"qLt0CspHmFDA"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Create a function to unbatch a batch dataset\ndef unbatchify(data):\n  \"\"\"\n  Takes a batched dataset of (image, label) Tensors and returns separate arrays\n  of the images and labels.\n  \"\"\"\n  images = []\n  labels =[]\n\n  #Loop throug unbatch data\n  for image,label in valid_data.unbatch().as_numpy_iterator():\n    images.append(image)\n    labels.append(Unique_breeds[np.argmax(label)])\n  return images, labels\n\n#Unbatchify validation data\n\nval_images, val_labels = unbatchify(valid_data)","metadata":{"id":"Hrxxm52zm3WU"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"val_images[0] , val_labels[0]","metadata":{"id":"ueEngzvNoCDe"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"get_pred_label(val_labels[34])","metadata":{"id":"ASNiHLvAoa_C"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we've got ways to get:\n* Predictino labels\n* Validation labels(truth labels)\n* Validation images\n\nlet's make some function to make these all a bit more\nvisaulize.\n\nwe'll create a function which:\n* Take an array of prediction probabilities, an array of truth labels and an array of images and integers.\n* Convert the prediction probabilities to a predicted label.\n* Plot the predicted label, its predicted probability, the truth label and the target image on a single plot.","metadata":{"id":"dKvUrzFHq98e"}},{"cell_type":"code","source":"def plot_pred(prediction_probabilities, labels, images, n=1):\n  \"\"\"\n  View the prediction, ground truth and image for sample n\n  \"\"\"\n\n  pred_prob, true_label, image =prediction_probabilities[n], labels[n], images[n]\n\n  #Get the pred label \n  pred_label = get_pred_label(pred_prob)\n\n  #Plot images & remove ticks\n  plt.imshow(image)\n  plt.xticks([])\n  plt.yticks([])\n  \n  #Change the color of the title depending on if the prediction is right or wrong\n\n  if pred_label == true_label:\n    color=\"green\"\n  else:\n    color=\"red\"\n\n  #Change plot title to be predicted, probability of prediction and truth label\n  plt.title(\"prediction : {} \\n pred_probability : {:.2f}% \\n True_label : {}\".format(pred_label, np.max(pred_prob)*100, true_label), color=color)\n  ","metadata":{"id":"9SBWUnpCsaoY"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_pred(prediction_probabilities=predictions,\n          labels=val_labels,\n          images=val_images,\n          n=74)","metadata":{"id":"ZAbH_RLHu7Kk"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now've got one function to visualize our model top prediction, let's make another to view our model top 10 predictions\n\nThis function will:\n* Take an input of prediction probabilities array and a ground thruth array and an integer\n* Find the top 10:\n  * Prediction probabilities indexes\n  * Prediction probabilities values\n  * Prediction labels\n* Plot top 10 prediction probability values and labels, coloring the true label green","metadata":{"id":"ZZvqM0vFvrix"}},{"cell_type":"code","source":"def plot_pred_conf(prediction_probabilities, labels, n=1):\n  \"\"\"\n  Plus the top 10 highest prediction confidence along with the truth label for sample n.\n  \"\"\"\n\n  pred_prob, true_label =prediction_probabilities[n], labels[n]\n\n  #Get the predicted label\n  pred_label  = get_pred_label(pred_prob)\n\n  #Find the top 10 prediction confidence indexes\n  top_10_pred_indexes = pred_prob.argsort()[-10:][::-1]\n\n  #Find the top 10 prediction confidence values\n  top_10_pred_values = pred_prob[top_10_pred_indexes]\n\n  #Find top_10_pred_labels\n  top_10_pred_labels = Unique_breeds[top_10_pred_indexes]\n\n  #Setup plot\n\n  top_plot = plt.bar(np.arange(len(top_10_pred_labels)),\n                     top_10_pred_values,\n                     color=\"grey\")\n  \n  plt.xticks(np.arange(len(top_10_pred_labels)),\n             labels= top_10_pred_labels,\n             rotation=\"vertical\")\n \n\n  \n\n  #Change color of true label\n  if np.isin(true_label, top_10_pred_labels):\n    top_plot[np.argmax(top_10_pred_labels == true_label)].set_color(\"green\")\n  else:\n    pass","metadata":{"id":"8B06KpUU0l3k"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_pred_conf(prediction_probabilities=predictions,\n               labels=val_labels,\n               n=9)","metadata":{"id":"OFNzTmTc0vmL"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we've got some function to help  us viusalize our predictino and evalute our model, Let's check out a few.","metadata":{"id":"QK9GoGH8BVIR"}},{"cell_type":"code","source":"#Let's check out few predictions and their different values\n\ni_multiplier = 10\nnum_rows = 3\nnum_cols= 2\nnum_images = num_rows*num_cols\nplt.figure(figsize=(10*num_cols , 5*num_rows))\n\nfor i in range(num_images):\n  plt.subplot(num_rows, 2*num_cols, 2*i+1)\n  plot_pred(prediction_probabilities=predictions,\n            labels=val_labels,\n            images=val_images,\n            n=i+i_multiplier)\n  plt.subplot(num_rows, 2*num_cols, 2*i+2)\n  plot_pred_conf(prediction_probabilities=predictions,\n                labels=val_labels,\n                n=i+i_multiplier)\nplt.tight_layout()\n\n","metadata":{"id":"d_f3CSVIBViR"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##Saving and reloading a trained Model\n","metadata":{"id":"Ey8QbLxHHawj"}},{"cell_type":"code","source":"#Create a function to save model\ndef save_model(model, suffix=None):\n  \"\"\"\n  save a given model in a models directory and appends a suffix (string).\n  \"\"\"\n  #create a model directory pathname with current time\n  modeldir=os.path.join(\"/content/drive/MyDrive/Dog-Vision/models\",datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n\n  model_path =modeldir + \"-\" + suffix + \".h5\" #save format of model\n\n  print(f\"Saving model to : {model_path}\")\n  model.save(model_path)\n  return model_path\n  ","metadata":{"id":"n90GInD0Oq5m"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Create a function to a train model\ndef load_model(model_path):\n  \"\"\"\n  Loads a saved model from specified path.\n  \"\"\"\n  print(f\"Loading saved moodel from : {model_path}\")\n  model=tf.keras.models.load_model(model_path, custom_objects={\"KerasLayer\":hub.KerasLayer})\n\n  return model","metadata":{"id":"e134hTDqQolX"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we've got fuction to save and load a trained model, let's make sure thry work!","metadata":{"id":"EaCpGpvZTRN4"}},{"cell_type":"code","source":"#Save our model trained on 1000 images\nsave_model(model, suffix=\"1000-images-mobilenetv2-Adam\")","metadata":{"id":"CkxGjVuXTn9G"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#load a trained  model \nloaded_1000_image_model=load_model(\"/content/drive/MyDrive/Dog-Vision/models/20210320-110517-1000-images-mobilenetv2-Adam.h5\")\n","metadata":{"id":"ARCjZaD9T40V"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Evaluate the pre_saved model\nmodel.evaluate(valid_data)\n","metadata":{"id":"Csdc21DhUVMd"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n#Evaluate the loaded model\nloaded_1000_image_model.evaluate(valid_data)","metadata":{"id":"ikFhW5C3U5wm"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##Traing a model on the full data","metadata":{"id":"lYYMpldoVFhW"}},{"cell_type":"code","source":"len(X), len(y)","metadata":{"id":"G_imNg4tVbYF"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Create a data batch with full data set\nfull_data = create_data_batches(X,y)","metadata":{"id":"mdwQIGCDVede"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"full_data","metadata":{"id":"Ktam67QNV8aV"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Create a model for full model\nfull_model = create_model()","metadata":{"id":"sAPywjrPV_4m"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Create full model callbacks\nfull_model_tensorboard = create_tensorboard_callback()\n\nfull_model_early_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"accuracy\",\n                                                             patience=3)","metadata":{"id":"LZBLpg5QWU8k"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note :** Running the cell below will take little while\n(Maybe up to 30 minutes for the first epoch) because the GPU we're using the runtime has to load all the images into memory.","metadata":{"id":"_aM2MsYTYA46"}},{"cell_type":"code","source":"# fit the full model to the full data\nfull_model.fit(x=full_data,\n               epochs=NUM_EPOCHS,\n               callbacks=[full_model_tensorboard, full_model_early_stopping])","metadata":{"id":"igwzz-vrXI_l"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"save_model(full_model,suffix=\"full_data_trained_model_mobilenetv2_Adam\")","metadata":{"id":"GHB_rwiDYrtl"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load in the full model\nloaded_full_model = load_model(\"/content/drive/MyDrive/Dog-Vision/models/20210320-141158-full_data_trained_model_mobilenetv2_Adam.h5\")","metadata":{"id":"v3vncNc9ZnTd","outputId":"5c003b5c-e4e6-45d3-ce2b-01c9152f13f9"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Making prediction on test data\n\nSince our model has been trained on images in tghe form of Tensor batches, to make predictions on the test data, we'll have to get it into same format.\n\nLuckily we created `create_data_batches()` earlier which can take a list of filenames as input and conver them into Tensor batches.\n\nTo make predictions on the test data, we'll:\n* Get the test images filenames\n* convert the filenames into test data batches using  `create_data_batches()` and setting the `test_data` parameter to `True` (since the test data doestn't have labels).\n* Make prediction array by passing the test batches to the `predict()` method called on our model","metadata":{"id":"HM8Ko15IgzeZ"}},{"cell_type":"code","source":"#Load test images filenames\n\ntest_path = \"/content/drive/MyDrive/Dog-Vision/test/\"\ntest_filenames = [test_path + images for images in os.listdir(test_path)]","metadata":{"id":"SWIfeu0ehgAy"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_filenames[:10]","metadata":{"id":"5cEOpi3pmV05","outputId":"fde188a9-0190-4df7-c91c-a32ca0b5af94"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(test_filenames)","metadata":{"id":"0TwgcQLymZhe","outputId":"c753bd33-3b3f-41fc-a6d0-f40f3780e213"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Create test data batch\ntest_data = create_data_batches(test_filenames, test_data=True)","metadata":{"id":"kUMb9ATYm8mW","outputId":"b6f9625d-355f-4104-d625-6869e119f2a0"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_data","metadata":{"id":"tF1nRfxnqBcu","outputId":"f9621276-b2c2-41a2-a545-e4252a95ea05"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note :** calling `predict()` on ourfull model and passing it the test data batch will take a long time to run (about ~ 1 hour)","metadata":{"id":"iGNjgI9ttCDD"}},{"cell_type":"code","source":"#Make prediction on test data batch using loaded full model\ntest_predictions = loaded_full_model.predict(test_data,verbose=1)","metadata":{"id":"OdhLj62Aspa2","outputId":"449fc7eb-37f9-4d0a-b3ef-7eb699ce00bf"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_predictions","metadata":{"id":"6W39-wi6tR2c","outputId":"78e1cb77-ea3e-492b-f087-93e5140748f0"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Save prediction (numpy array) to csv file (for acces later)\nnp.savetxt(\"/content/drive/MyDrive/Dog-Vision/preds_array.csv\", test_predictions, delimiter=\",\")","metadata":{"id":"P720dIkO8_TI"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Load prediction (NumPy array) from csv file\ntest_predictions = np.loadtxt(\"/content/drive/MyDrive/Dog-Vision/preds_array.csv\",delimiter=\",\")","metadata":{"id":"1JrDfHsC9lsD"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_predictions","metadata":{"id":"Sq3wPD84-BMl","outputId":"f0d2271c-1841-4811-b530-2acd5c1cd3a5"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_predictions.shape","metadata":{"id":"1ZYXXaKE-MqN","outputId":"7c8f2062-4023-4869-e907-e8dbfbb454f9"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Preparing test dataset prediction for Kaggle\n\nLooking at the Kaggle sample submission, we find that it wants our prediction probability outputs in a DataFrame with ID and a column for each diffrent dog breed.\nhttps://www.kaggle.com/c/dog-breed-identification/overview/evaluation","metadata":{"id":"W0p4airD-R8T"}},{"cell_type":"code","source":"#Create a pandas DataFrame with empty columns\npred_df=pd.DataFrame(columns=[\"id\"] + list(Unique_breeds))","metadata":{"id":"yDQwRSCZ_cOd","outputId":"12355c92-c7a1-4788-b073-f32da10b50d9"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"os.path.splitext() : https://www.geeksforgeeks.org/python-os-path-splitext-method/","metadata":{"id":"f4sCLNQOCDMz"}},{"cell_type":"code","source":"#Append test image ID's to predictions DataFrame\ntest_ids = [os.path.splitext(fname)[0] for fname in os.listdir(test_path)]\ntest_ids","metadata":{"id":"1nledEbX_27B","outputId":"f6ec6fd7-34d5-4c53-e323-287eacabe5d5"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_df[\"id\"] = test_ids","metadata":{"id":"d_9cjrXyAoDj"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_df","metadata":{"id":"d3TuaIoPCV8_","outputId":"5e1df182-90aa-4861-d64d-355e93594a14"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_df[list(Unique_breeds)] = test_predictions","metadata":{"id":"lDqtnX6iDlr5"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_df.head()","metadata":{"id":"VmmLMdj1Elhq","outputId":"76aab801-39aa-4a1d-d22a-d32b13ab0cc6"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Save our prediction dataframe to CSV for submission to kaggle\npred_df.to_csv(\"/content/drive/MyDrive/Dog-Vision/full_model_prediction.csv\",index=False)","metadata":{"id":"BgtudyC9FAYy"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"id":"cT3AYzGPFamM","outputId":"cdfbddbb-30d7-4fdd-e4c5-44a9fdacc098"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Make predictions on custom images\nmy_dog_path = \"/content/drive/MyDrive/Dog-Vision/My dog (saint_bernard)/\"\nmy_dog_image = [my_dog_path + fname for fname in os.listdir(my_dog_path)]","metadata":{"id":"7HqdhWWWIkD9"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Turn image into batch datasets\nmy_dog_data = create_data_batches(my_dog_image, test_data =True)","metadata":{"id":"O1EExr6lKFd1","outputId":"055f796e-478f-42db-e587-2f2c018ab4f9"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#make prediction on my dog data\nmy_dog_pred  = loaded_full_model.predict(my_dog_data)","metadata":{"id":"WCI4TB0EKYWo"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#prediction labe\nUnique_breeds[np.argmax(my_dog_pred)]","metadata":{"id":"taLGXjqBKkfy","outputId":"bd2a8958-fecc-4392-d076-ba8a0c1b0ac3"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for image in my_dog_data.unbatch().as_numpy_iterator():\n  plt.imshow(image)","metadata":{"id":"M47tIwaWK3ia","outputId":"fc65d79d-1dfe-455e-a449-2954f68c6014"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"id":"H1WpJKADMOTh"},"execution_count":null,"outputs":[]}]}