{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Dog Breed Classifier\n\n**Name:** Dog Breed Classifier\n\n**Author:** Sharome Burton\n\n**Date:** 07/20/2021\n\n**Description:** Machine learning model used to determine the breed of a dog from a given image.\n\n**Kaggle:** https://www.kaggle.com/sharomeethan/disaster-tweet-classifier\n\n**Colab:** https://colab.research.google.com/drive/1wLBBuwKx4a9w3jTectO0nzCzU9d3w_BB?usp=sharing\n\n<img src=\"https://raw.githubusercontent.com/koulkoudakis/dog-breed-classifier/main/dog-breed-classifier.png\"\n     alt=\"dog-breed-classifier\"\n     style=\"float: left; margin-right: 10px;\" />\n\n## 1. Problem definition\n> How well can we identify the breed of a dog from a given image?\n\n## 2. Data\nWe are provided with a training set and a test set of images of dogs. Each image has a filename that is its unique `id`. The dataset comprises 120 breeds of dogs.\n   \n* `train.zip` - the training set, we are provided the breed for these dogs\n* `test.zip` - the test set, we must predict the probability of each breed for each image\n* `sample_submission.csv` - a sample submission file in the correct format\n* `labels.csv` - the breeds for the images in the train set\n\nThere are 10,000+ labeled images in each set.\n    \nsource: https://www.kaggle.com/c/dog-breed-identification/data\n\n## 3. Features\n\n   * `id` - a unique identifier for each image\n   * `breed` - the breed of the dog, eg. \n    * affenpinscher\n    * afghan_hound\n    * african_hunting_dog\n    * airedale\n    * american_staffordshire_terrier \n   \n## 4. Evaluation \n\n> **Goal:** Determine the breed of a dog in a given image with >75% accuracy.\n\nThe evaluation is a file with prediction probabilities for each dog breed of each test image. Submissions are evaluated on Multi Class Log Loss between the predicted probability and the observed target.\n\nsource: https://www.kaggle.com/c/dog-breed-identification/overview/evaluation\n\n\n","metadata":{"id":"L9L9ht6GHcOc"}},{"cell_type":"markdown","source":"## Loading Dataset","metadata":{"id":"X5foUDW1xGpa"}},{"cell_type":"code","source":"# Unzipping dataset\n# !unzip \"drive/MyDrive/ML Projects/dog-breed-identification.zip\" -d \"drive/MyDrive/ML Projects/\"","metadata":{"id":"ECSRi0UVezcb"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Getting tools ready\n\n* Import TensorFlow 2.x \n* Import TensorFlow Hub\n* Ensure access to GPU\n","metadata":{"id":"wgb4voD4xU_F"}},{"cell_type":"code","source":"# TensorFlow\nimport tensorflow as tf\nprint(\"Tf version:\", tf.__version__)\n# Tensorflow Hub\nimport tensorflow_hub as hub\nprint(\"TF Hub version:\", hub.__version__)\n\n\n# Check GPU availability\nprint(\"GPU: \", \"available\" if tf.config.list_physical_devices else \"not available\")","metadata":{"id":"6jtTnGlyr0QF","outputId":"d11be9f3-81bd-45ed-9ee0-f6dc011a189a"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Getting data ready\nWith all machine learning models, data has to be in numerical format. Here we must convert our images into tensors.\n","metadata":{"id":"QpOtmW6ksA5s"}},{"cell_type":"code","source":"# Display labels\nimport pandas as pd\n\nlabels_csv = pd.read_csv(\"/content/drive/MyDrive/ML Projects/labels.csv\")\n\n\nprint(labels_csv.describe())\nlabels_csv.describe()\nlabels_csv\n","metadata":{"id":"_004S67Fx4wc","outputId":"dcaede28-6a4c-40e2-8d67-f9194ffbfbd9"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_csv[\"breed\"].value_counts()","metadata":{"id":"o97BqrQXzT2G","outputId":"8c58b1bc-c091-49d6-ca45-a200ca419369"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_csv[\"breed\"].value_counts().plot.bar(figsize=(30,10))","metadata":{"id":"fjQN73Hg0Lhr","outputId":"e669ef9e-83a9-4c26-c679-a6d5bf42e141"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_csv[\"breed\"].value_counts().median()","metadata":{"id":"cbHYVptU0Vur","outputId":"d9ffaf7f-904f-4537-c87b-9214ada4b346"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# View image sample\nfrom IPython.display import Image\nImage(\"/content/drive/MyDrive/ML Projects/train/ffe5f6d8e2bff356e9482a80a6e29aac.jpg\")","metadata":{"id":"jpMRQPbN3TU8","outputId":"3fea00a0-3590-4138-b7bd-bc017ac5de6f"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Fetching images and labels\n\n","metadata":{"id":"vw-sPFLu5Zvr"}},{"cell_type":"code","source":"labels_csv.tail()","metadata":{"id":"twxv7rS25270","outputId":"42fa2040-7f68-436a-fa5c-700ae03f8b9a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create pathnames from image ID's\nfilenames = [\"drive/MyDrive/ML Projects/train/\" + fname + \".jpg\" for fname in labels_csv[\"id\"]]\n\n# Check first 10\nfilenames[:10]","metadata":{"id":"2NAojJN95wxC","outputId":"95c6053c-8c58-447d-e35d-19b74df84bf4"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check whether number of filenames matches number of images\nimport os\nif len(os.listdir(\"drive/MyDrive/ML Projects/train/\")) != len(filenames):\n  print(\"Mismatched number of filenames and files, check target directory.\")\nelse:\n  print(\"Identical number of filenames and files.\")","metadata":{"id":"I24RYFv252S8","outputId":"8de43eff-ed3a-48b5-899c-e2903ca1f913"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Image(filenames[9000])","metadata":{"id":"yNdtGJK47L4D","outputId":"cdd8f9dd-779b-4a81-b2c9-d9b685935abf"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_csv[\"breed\"][9000]","metadata":{"id":"E9xYUPQX7zvL","outputId":"60977d18-833f-4a48-fa47-4bbb22984960"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since we have all training filepaths in a list, let's prepare labels.","metadata":{"id":"iRTFwdgJ7-pb"}},{"cell_type":"code","source":"import numpy as np\nlabels = labels_csv[\"breed\"]\nlabels","metadata":{"id":"1V0OZrWubrn3","outputId":"d0feedd0-10e6-40ac-a708-32de7cf1871e"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(labels)","metadata":{"id":"2mrqNel1bw1G","outputId":"850e80c5-b1ee-490a-b1d5-1d8c791a8b73"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check for missing files\nif len(labels) == len(filenames):\n  print(\"Number of labels matches number of filenames\")\nelse:\n  print(\"Number of labels does not match numbe of filenames\")","metadata":{"id":"VlpZOPSBb5mE","outputId":"00b5cc78-820b-4339-a2a7-b4d2b1ba5740"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Find unique label values\nunique_breeds = np.unique(labels)\nunique_breeds[:10]","metadata":{"id":"fADERsGoeOxd","outputId":"df297312-3e55-40de-e441-849ed8108efb"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(unique_breeds)","metadata":{"id":"_xqofPokeXY0","outputId":"084e02e2-f03b-4cf6-f6c9-715ef4ba4145"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# turn single label into array of booleans\nlabels[0] == unique_breeds","metadata":{"id":"FUafkthXegRb","outputId":"38fefdcf-6588-4180-a376-5da70a7d0dc3"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Convert every label into boolean array\nboolean_labels = [label == unique_breeds for label in labels]\nboolean_labels[:2]\n","metadata":{"id":"2K873NCNep8z","outputId":"03e5ee46-c0cc-4496-a98f-f52b5f102f6a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(boolean_labels)","metadata":{"id":"aOr-n5HMe68j","outputId":"e63debe4-0b14-4833-98fe-137b28ea6f2b"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Converting boolean array into integers\nprint(labels[0]) # original label\nprint(np.where(unique_breeds == labels[0])) # index where label occurs\nprint(boolean_labels[0].argmax()) # index where label occurs in boolean array\nprint(boolean_labels[0].astype(int)) # value 1 where sample label occurs","metadata":{"id":"uXbtvncje9DD","outputId":"4afb7a6e-dff7-4d34-f698-f14c25eb4315"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Creating validation set\nSince the dataset from Kaggle has no validations set, we will create our own.","metadata":{"id":"ZFgPux4Sfq_L"}},{"cell_type":"code","source":"# Set up X & y variables\nX = filenames\ny = boolean_labels","metadata":{"id":"Ug7oixJQgYcL"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Set number of images to use for experimentation\nNUM_IMAGES = 1000 #@param {type:\"slider\", min:1000, max:10000, step:1}","metadata":{"id":"n27otm72gt6s"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Splitting data","metadata":{"id":"JlfV3y2KhmD0"}},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\nX_train, X_val, y_train, y_val = train_test_split(X[:NUM_IMAGES],\n                                                  y[:NUM_IMAGES],\n                                                   test_size=0.2,\n                                                   random_state=18)\n\nlen(X_train), len(y_train), len(X_val), len(y_val)","metadata":{"id":"Vf4MLUDNhudT","outputId":"6047bf97-4807-42a8-a5a6-e00c6e7cabb0"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train[:2], y_train[:2]","metadata":{"id":"qyE_MrBmisND","outputId":"76736f40-be4c-47a6-b758-472d257b0e61"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Preprocessing Images (turning images into Tensors)\n\nTo preprocess our images into Tensors, we will write a function which does these things:\n1. Take an image filepath as input\n2. Use TensorFlow to read the file and save it to a variable, `image`\n3. Turn our `image` (.jpg) into Tensors\n4. Normalize our image (convert color channel values from 0-255 to 0-1)\n5. Resize the `image` to be a shape of (224,224)\n6. Return the modified `image`\n\nLet's see what importing an image looks like:","metadata":{"id":"-8iHjfrPjRUD"}},{"cell_type":"code","source":"# Convert image to NumPy array\nfrom matplotlib.pyplot import imread\nimage = imread(filenames[0])\nimage.shape","metadata":{"id":"QXLuQqDGppys","outputId":"3c2b10d8-fa76-4bfc-f925-4eef1c4d2c86"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image.max(), image.min()","metadata":{"id":"x28V_68rqTrs","outputId":"b4f89fcf-5917-4192-fa12-0da732b7d27f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"tf.constant(image)[0]","metadata":{"id":"BM1pRG2vqrJD","outputId":"c71e6aae-744e-4ae0-c9b3-742b5496d85f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Define image size\nIMG_SIZE = 224\n\n# Create function for preprocessing images\ndef process_image(image_path, img_size=IMG_SIZE):\n  \"\"\"\n  Takes an image file path and turns image into a tensor\n  \"\"\"\n  # Read in image file\n  image = tf.io.read_file(image_path)\n  # Turn .jpg image into numerical tensor with 3 color channels\n  image = tf.image.decode_jpeg(image, channels=3)\n  # Convert color channel values from 0-255 to 0-1 values\n  image = tf.image.convert_image_dtype(image, tf.float32)\n  # Resize image to our desired value(224,224)\n  image = tf.image.resize(image, size=[img_size, img_size])\n\n  return image","metadata":{"id":"iUkpDxhlq5VT"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Turning our data into batches\n\nIn order to use TensorFlow effectively, we need dat in the form of tensor tuples: `(image, label)`\n\n","metadata":{"id":"TnG5Zche5MTJ"}},{"cell_type":"code","source":"# Create a simple function to return a tuple (image, label)\ndef get_image_label(image_path, label):\n  \"\"\"\n  Takes an image file path name and associated label, processes\n  the image and returns a tuple of (image, label).\n  \"\"\"\n  image = process_image(image_path)\n  return image, label\n\n# Demo of above function\n(process_image(X[18]), tf.constant(y[18]))\n","metadata":{"id":"LAc2LaWp6qMB","outputId":"b91dd92b-4118-4459-a682-50f8833c07f6"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Define the batch size\nBATCH_SIZE = 32\n\n# Create function to convert data to batches\ndef create_data_batches(X, y=None, batch_size=BATCH_SIZE, valid_data=False, test_data=False):\n  \"\"\"\n  Creates batches of data out of image (X) and label (y) pairs.\n  Shuffles the data if it is training data but does not shuffle\n  validations data. \n  Also accepts test data as input (no labels).\n  \"\"\"\n  # If test data, we don't have labels\n  if test_data:\n    print(\"Creating test data batches...\")\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(X))) # only filepaths (no labels)\n    data_batch = data.map(process_image).batch(BATCH_SIZE)\n    print(\"Test data batches created\")\n    return data_batch\n\n  # If data is valid dataset, we don't need to shuffle it\n  elif valid_data:\n    print(\"Creating validation data batches...\")\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(X), # filepaths\n                                               tf.constant(y)))\n    data_batch = data.map(get_image_label).batch(BATCH_SIZE)\n    print(\"Validation data batches created\")\n    return data_batch\n\n  else:\n    print(\"Creating training data batches...\")\n    # Turn filepaths and labels into tensors\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(X),\n                                               tf.constant(y)))\n    # Shuffling pathnames and lables before mapping image processor function\n    # is faster than shuffling images\n    data = data.shuffle(buffer_size=len(X))\n\n    # Create (image, label) tuples (also turns image path into preprocessed image)\n    data = data.map(get_image_label)\n\n    # Turn training data into batches\n    data_batch = data.batch(BATCH_SIZE)\n\n    print(\"Training data batches created\")\n\n  return data_batch\n","metadata":{"id":"GmlmxZ9R8vgD"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check attributes of data batches\ntrain_data = create_data_batches(X_train, y_train)\nval_data = create_data_batches(X_val, y_val, valid_data=True)","metadata":{"id":"158L_3rDB8mJ","outputId":"913fe4ca-b918-4b02-c137-c1534dbc9425"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check attributes of data batches\ntrain_data.element_spec, val_data.element_spec","metadata":{"id":"JF_xBvxWDWlQ","outputId":"3870e8aa-b268-44f1-bb25-d54b13422503"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(train_data)","metadata":{"id":"PyJqIwDBDcaC","outputId":"3db4848f-1b78-4831-9785-f72069cfaba3"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualizing data batches","metadata":{"id":"IIcr4UVMDm9B"}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport math\n\n# Create a function for viewing images in data batch\ndef show_images(images, labels, size=25):\n  \"\"\"\n  Displays a plot of up to 32 images and their labels from a data batch\n  \"\"\"\n  # Setup figure\n  plt.figure(figsize=(10,10))\n  # Loop through size\n  for i in range(size):\n    # Create subplots (dimension*dimension grid)\n    dimension = int(math.sqrt(size-1))+1\n    ax = plt.subplot(dimension,dimension,i+1)\n    # Display an image\n    plt.imshow(images[i])\n    # Add image label as title\n    plt.title(unique_breeds[labels[i].argmax()])\n    # Turn grid lines off\n    plt.axis(\"off\")\n\n\n","metadata":{"id":"1400jRMnELnY"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_images, train_labels = next(train_data.as_numpy_iterator())\n# train_images, train_labels","metadata":{"id":"fqQcW2cAEhrB"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(train_images), len(train_labels)","metadata":{"id":"7JTnZDutGr2S","outputId":"44314053-97ff-4a56-b14e-a9250e3a2366"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Now let's visualize data in training batch\nshow_images(train_images, train_labels, size=16)","metadata":{"id":"lEh2ofn-G6KQ","outputId":"64667485-f850-4579-a675-c544d36a6b16"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualize validation set\nval_images, val_labels = next(val_data.as_numpy_iterator())\nshow_images(val_images, val_labels)","metadata":{"id":"UCsUXWRlHE0o","outputId":"110d8798-f6e1-400e-da18-de95169189bf"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Building a model\nBefore building a model, there are a few things to define:\n* The input shape (image shape, in the form of tensors) to our model.\n* The output shape(image labels, in form of tensors) of our model.\n* (optional) the URL of the model we want to use from TensorFlow Hub - https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/5","metadata":{"id":"H3jN489-I8Mf"}},{"cell_type":"code","source":"# Setup input shape to the model\nINPUT_SHAPE = [None, IMG_SIZE, IMG_SIZE, 3] # batch, height, width, color channels\n\n# Setup output shape\nOUTPUT_SHAPE = len(unique_breeds)\n\n# Setup model URL from TensorFlow Hub\nMODEL_URL = \"https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/5\"","metadata":{"id":"D0Cpsq_CKaFX"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's create a function which:\n* Takes input shape, output shape and model we've chosen as parameters\n* Defines the layers in a Keras model in sequential fashion (do this first, then this, then that).\n* Compiles model (says it should be evaluated and improved upon)\n* Builds model (tells the model the input shape to expect)\n* Returns the model\n\nSteps may be found here: https://www.tensorflow.org/guide/keras/sequential_model","metadata":{"id":"Sbv_VC_hK-HA"}},{"cell_type":"code","source":"# Create a function which builds a Keras model\ndef create_model(input_shape=INPUT_SHAPE, output_shape=OUTPUT_SHAPE, model_url=MODEL_URL):\n  \"\"\"\n  Creates of specified input shape, output shape, with specified\n  URL from TensorFlow Hub\n  \"\"\"\n  print(\"Building model with:\", model_url)\n\n  # Setup model layers\n  model = tf.keras.Sequential([\n                              hub.KerasLayer(model_url), # Layer 1 (input layer)\n                              tf.keras.layers.Dense(units=output_shape,\n                              activation=\"softmax\") # Layer 2 (output layer)\n                              ])\n  # Compile model\n  model.compile(\n      loss=tf.keras.losses.CategoricalCrossentropy(),\n      optimizer=tf.keras.optimizers.Adam(),\n      metrics=[\"accuracy\"]\n  )\n\n  # Build model\n  model.build(INPUT_SHAPE)\n\n  return model\n","metadata":{"id":"BIFeMAldQmzX"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = create_model()\nmodel.summary()","metadata":{"id":"o5bdX6XzR_Kv","outputId":"9050ee39-e1ba-4fee-8f42-9bebcb86a9f7"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Creating model callbacks\nCallbacks are helper functions a model can use during training to do such things as save its progress, check its progress or stop training early if a model stops improving.\n\nWe will create two callbacks:\n* One for TensorBoard which helps track model's progress,\n* Another for stopping training early to prevent overfitting\n\n### TensorBoard Callback\n\nTo setup a TensorBoard callback, we need to do 3 things:\n1. Load the TensorBoard notebook extension\n2. Create a TensorBoard callback which is able to save logs to a directory and pass it to our model's training logs with the `%tensorboard` magic function ","metadata":{"id":"-k64_rKETkQo"}},{"cell_type":"code","source":"# Load TensorBoard notebook extension\n%load_ext tensorboard","metadata":{"id":"o9VwPIJzmJiM"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import datetime\n\n# Create a function to build a TensorBoard callback\ndef create_tensorboard_callback():\n  # Create a log directory for storing TensorBoard logs\n  logdir = os.path.join(\"drive/MyDrive/ML Projects/logs\",\n                        # Make it so that the lgos get tracked when\n                        # we run an experiment\n                        datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n  return tf.keras.callbacks.TensorBoard(logdir)","metadata":{"id":"aR4ZS85nmxjt"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Early Stopping Callback\n\nEarly stopping helps stop our model from overfitting by stopping training if a certain training evaluation metric is met.\n\nLink: https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/EarlyStopping","metadata":{"id":"z9LQkf3boNOV"}},{"cell_type":"code","source":"# Create early stopping callback\nearly_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"val_accuracy\",\n                                                  patience=3)","metadata":{"id":"XKSx384Bopsr"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training model (on subset of data)\n\nOur first model will train on 1000 images in order to make sure everything is working.","metadata":{"id":"w3qiCH7nqo3v"}},{"cell_type":"code","source":"NUM_EPOCHS = 100 #@param {type:\"slider\", min:\"10\", max:\"100\"}","metadata":{"id":"pihLc5uSq__j"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check GPU availability\nprint(\"GPU: \", \"available\" if tf.config.list_physical_devices else \"not available\")","metadata":{"id":"KfZTv8LJsGic","outputId":"05c084fa-b0c8-442b-81e8-9f3e39c64b75"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We will:\n* Create a model using `create_model()`\n* Setup a TensorBoard callback using `create_tensorboard_callback()`\n* Call the `fit()` function on our model, passing it the training data (`train_data`), validation data (`val_data`), number of epochs to train for (`NUM_EPOCHS`) and the callbacks we would like to use\n* Return the model","metadata":{"id":"X64pxA_dsNhj"}},{"cell_type":"code","source":"# Build a function to train and return a trained model\ndef train_model():\n  \"\"\"\n  Trains a given model and returns the trained version.\n  \"\"\"\n  # Create model\n  model = create_model()\n\n  # Create new TensorBoard session each time we train a model\n  tensorboard = create_tensorboard_callback()\n\n  # Fit model to data \n  model.fit(x=train_data,\n            epochs=NUM_EPOCHS,\n            validation_data=val_data,\n            validation_freq=1,\n            callbacks=[tensorboard, early_stopping])\n  # Return fitted model\n  return model","metadata":{"id":"ibkW0Efws1qj"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Fit model to data\nmodel = train_model()","metadata":{"id":"P3p9d1v9u5r0","outputId":"0249ccb3-277a-4a67-a2d1-dc783a17a8ab"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It looks like our model is over-fitting because its performance on the training set far exceeds the performance of the validation set.\n\n**Note**: Overfitting at the beginning is good; it means our model is learning.","metadata":{"id":"lBe9eMEXu9aj"}},{"cell_type":"markdown","source":"### Checking TensorBoard logs\n\nThe TensorBoard magic function (`%tensorboard`) will access the logs directory and visualize its contents","metadata":{"id":"NUqyKZR2w9Gb"}},{"cell_type":"code","source":"%tensorboard --logdir drive/My\\ Drive/ML\\ Projects/logs","metadata":{"id":"3cDh6dzrxcnS","outputId":"75c49b40-28dc-4154-88b8-c7ca4b14fd62"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Making and evaluating predictions using trained model","metadata":{"id":"_BwEkj6gx35D"}},{"cell_type":"code","source":"# Make predictions on validation data\npredictions = model.predict(val_data, verbose=1)\npredictions","metadata":{"id":"hlxTElcUzLw0","outputId":"3d05c317-ebae-4f3b-e829-7af64ad44461"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predictions.shape","metadata":{"id":"sZg_5Bmjzlgy","outputId":"72feb6a9-80f5-4d6b-f752-551598101cf0"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predictions[0]","metadata":{"id":"Y218tQekzoSz","outputId":"346ab6b6-c8e5-43ab-c57e-552c68a35b82"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"np.sum(predictions[0])","metadata":{"id":"DP-kHjcsz0kD","outputId":"ccf261d0-88c4-4429-ed53-e1a1d58e2a63"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predictions[0][predictions[0].argmax()]","metadata":{"id":"1Vrcv1NVz-GK","outputId":"4f92a336-bfee-4043-80f8-c352330f6bb0"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# First prediction\n\nindex = 2\nprint(predictions[index])\nprint(f'Max value (probability of prediction): {np.max(predictions[index])}')\nprint(f'Sum: {np.sum(predictions[index])}')\nprint(f'Max index: {np.argmax(predictions[index])}')\nprint(f'Predicted label: {unique_breeds[np.argmax(predictions[index])]}')","metadata":{"id":"Ke7eWVPq0DTN","outputId":"b89babdc-33e4-4be7-b911-708d9bfdbc6a"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note:** Predictions probabilities are also known as confidence intervals","metadata":{"id":"C16PZ1X_0lD7"}},{"cell_type":"code","source":"# Turn prediction probabilities into their respective labels (easier to understand)\ndef get_pred_label(prediction_probabilities):\n  \"\"\"\n  Turns an array of prediction probabilities into a label\n  \"\"\"\n  return unique_breeds[np.argmax(prediction_probabilities)]\n\n# Get predicted label based on the array of predictions probabilities\npred_label = get_pred_label(predictions[18])\npred_label","metadata":{"id":"rH9GUu3l4GyK","outputId":"f2dda1d5-deba-4f04-826e-3c677c9eaa46"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"val_data","metadata":{"id":"52ElKduN4ltr","outputId":"a8c8905b-f8e4-4068-b5f0-fbd1bfa6f9f8"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"SInce our validation data is still in a batch dataset, we must unbatch the data to make predictions on the validation images and then compare those predictions to the validation labels (truth labels)","metadata":{"id":"ZBA9lxQo4sjb"}},{"cell_type":"code","source":"# Create a function to unbatch a batch dataset\ndef unbatch_data(data):\n  \"\"\"\n  Takes a batched dataset of (image,label) tensors and returns separate arrays\n  of images and labels.\n  \"\"\"\n  images = []\n  labels = []\n  # Loop through unbatched data\n  for image, label in data.unbatch().as_numpy_iterator():\n    images.append(image)\n    labels.append(unique_breeds[np.argmax(label)])\n  return images, labels\n\n# Unbatch the validation data\nval_images, val_labels = unbatch_data(val_data)\nval_images[3], val_labels[3]","metadata":{"id":"Ny1b9NOj5FgS","outputId":"f40c28af-6c75-4c40-c829-e8fdbcc9ec4d"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# images_ = []\n# labels_ = []\n\n# # Loop through unbatched data\n# for image, label in val_data.unbatch().as_numpy_iterator():\n#   images_.append(image)\n#   labels_.append(label)\n\n# images_[0], labels_[0]","metadata":{"id":"x3FyZ5MG5Y-C"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"get_pred_label(val_labels[3])","metadata":{"id":"aRVK2BKb5w9z","outputId":"500a9ff6-f5e2-45cf-9ee6-913a5b3db071"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"get_pred_label(predictions[3])","metadata":{"id":"OiCuNZ3A6Rh7","outputId":"2d4a3f0b-42cf-48ea-fc8e-2c07dd8866cf"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We now have ways to obtain:\n* Prediction labels\n* Validation labels (truth labels)\n* Validation images\n\nWe will now make some functions to visualize the results.\n\nWe will make a function which:\n* Takes an array of prediction probabilities, an array of truth labels and an array of images and integers.\n* Convert the prediction probabilities to a predicted label.\n* Plot the predicted label, its predicted probability, the truth label and the target image on a single plot.","metadata":{"id":"UeW5HPne6WUi"}},{"cell_type":"code","source":"def plot_pred(prediction_probabilities, labels, images, n=1):\n  \"\"\"\n  View the prediction, ground truth and image for sample n\n  \"\"\"\n  pred_prob, true_label, image = prediction_probabilities[n], labels[n], images[n]\n\n  # Get pred label\n  pred_label = get_pred_label(pred_prob)\n\n  # Plot image and remove ticks\n  plt.imshow(image)\n  plt.xticks([])\n  plt.yticks([])\n\n  # Change color of the title depending on if prediction is right or wrong\n  if pred_label == true_label:\n    color = \"green\"\n  else:\n    color = \"red\"\n\n  # change plot title to be predicted, probability of prediction and truth label\n  plt.title(\"{} {:2.0f}% {}\".format(pred_label,\n                                    np.max(pred_prob)*100,\n                                    true_label),\n                                    color=color\n                                    )\n                                    ","metadata":{"id":"kGY6hu2R8qoi"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_pred(prediction_probabilities=predictions,\n          labels=val_labels,\n          images=val_images,\n          n=18)","metadata":{"id":"Jyg-Boq_9uZS","outputId":"2fa823e5-6bd5-41a7-f985-b77c5148e95f"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we have one function to visualize our models top prediction, we will make another to view our model's top 10 predictions.\n\nThis function will:\n* Take an input of prediction probabilities array and a ground truth array and an integer\n* Find the prediction using `get_pred_label()`\n* FInd the top 10:\n  * Prediction probabilities indices\n  * Prediction probabilities values\n  * Prediction labels\n* Plot the top 10 prediciton probabilities, values and labels, coloring the true label green","metadata":{"id":"VG74Swhd-3py"}},{"cell_type":"code","source":"def plot_pred_conf(prediction_probabilities, labels, n=1):\n  \"\"\"\n  Plot the top 10 highest predictions confidences along\n  with the truth label for sample n.\n  \"\"\"\n  pred_prob, true_label = prediction_probabilities[n], labels[n]\n\n  # Get predicted label\n  pred_label = get_pred_label(pred_prob)\n\n  # Find top 10 prediction confidence indices\n  top_10_pred_indices = pred_prob.argsort()[-10:][::-1]\n  # Find top 10 prediction confidence values\n  top_10_pred_values = pred_prob[top_10_pred_indices]\n  # Find top 10 prediction labels\n  top_10_pred_labels = unique_breeds[top_10_pred_indices]\n\n  # Setup plot\n  top_plot = plt.bar(np.arange(len(top_10_pred_labels)),\n                     top_10_pred_values,\n                     color=\"gray\")\n  plt.xticks(np.arange(len(top_10_pred_labels)),\n             labels=top_10_pred_labels,\n             rotation=\"vertical\")\n  \n  # Change color of true label\n  if np.isin(true_label, top_10_pred_labels):\n    top_plot[np.argmax(top_10_pred_labels == true_label)].set_color(\"green\")\n  else:\n    pass","metadata":{"id":"RQ7CFW8QFONC"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_pred_conf(prediction_probabilities=predictions,\n               labels = val_labels,\n               n=25)","metadata":{"id":"VhMCoP5YGCnj","outputId":"8ea11f34-c5f3-4205-f93d-2c2b4104b398"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_pred(prediction_probabilities=predictions,\n          labels=val_labels,\n          images=val_images,\n          n=25)","metadata":{"id":"3Uk5jqTGH4za","outputId":"ae399acc-4dc7-4692-cc22-8abd68cd5e52"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we have some functions to help us visualize our predictions and evaluate our model, let's check out a few.","metadata":{"id":"_5i5G8QNIbM5"}},{"cell_type":"code","source":"i_multiplier = 18\nnum_rows = 2\nnum_cols = 2\nnum_images = num_rows*num_cols\nplt.figure(figsize=(10*num_cols, 5*num_rows))\nfor i in range(num_images):\n  plt.subplot(num_rows, 2*num_cols, 2*i+1)\n  plot_pred(prediction_probabilities=predictions,\n            labels=val_labels,\n            images=val_images,\n            n=i+i_multiplier)\n  plt.subplot(num_rows, 2*num_cols, 2*i+2)\n  plot_pred_conf(prediction_probabilities=predictions,\n                 labels=val_labels,\n                 n=i+i_multiplier)\n\nplt.tight_layout(h_pad=1.0)\nplt.show()","metadata":{"id":"VqnC28aiK6mJ","outputId":"16db566d-0b52-499f-b97d-0740070528cc"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Saving and loading a trained model","metadata":{"id":"ertWTCMML2dp"}},{"cell_type":"code","source":"# Create a function to save model\ndef save_model(model, suffix=None):\n  \"\"\"\n  Saves a given model in a models directory and appends a suffix (string).\n  \"\"\"\n  # Create a model directory pathname with current time\n  modeldir = os.path.join(\"drive/MyDrive/ML Projects/models\", \n                          datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n  model_path=modeldir + \"-\" + suffix + \".h5\" # Save format of model\n  print(f\"Saving model to: {model_path}...\")\n  model.save(model_path)\n  return model_path","metadata":{"id":"JnwH9biuWphy"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create a function to save a model\ndef load_model(model_path):\n  \"\"\"\n  Loads a saved model from a specified path.\n  \"\"\"\n  print(f\"Loading saved model from: {model_path}\")\n  model = tf.keras.models.load_model(model_path,\n                                     custom_objects={\"KerasLayer\":hub.KerasLayer})\n  return model","metadata":{"id":"JJ_KruBzXsgS"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"save_model(model, suffix=\"1000-images-mobilenetv2-Adam\")","metadata":{"id":"0qeqbCyCZGBw","outputId":"d950a505-c8ba-4923-dcd5-cdbf08a3aebd"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load a trained model\nloaded_1000_image_model = load_model(\"drive/MyDrive/ML Projects/models/20210724-164447-1000-images-mobilenetv2-Adam.h5\")","metadata":{"id":"YvOKij_MZi7Q","outputId":"564025fc-5ade-4c4c-e654-a89d242f4322"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Evaluate the pre-saved model\nmodel.evaluate(val_data)","metadata":{"id":"sEQtn883aFgo","outputId":"82a435dd-dc1c-48a7-c455-248b5634b190"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Evaluate saved model\nloaded_1000_image_model.evaluate(val_data)","metadata":{"id":"c9VeFQEYafVp","outputId":"c8d39559-67aa-4e82-9e13-94b8080a7a0f"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"##Training model on full dataset","metadata":{"id":"8omb9rnCarxg"}},{"cell_type":"code","source":"len(X), len(y)","metadata":{"id":"8ofN8Svbb3uI","outputId":"7f516ea3-25a3-4ec8-ceb5-2ac5b690161c"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create data batches with full dataset\nfull_data = create_data_batches(X,y)","metadata":{"id":"EzfOsLL4b6wI","outputId":"9b990801-01e8-4936-889d-02ffda2b992b"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"full_data","metadata":{"id":"A66qR8KecTw4","outputId":"6f576cb4-fb8f-4c1f-c5bc-4b207b81e348"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create a model for full dataset\nfull_model = create_model()","metadata":{"id":"hQ7mcogZcUzI","outputId":"12ea9559-a3c9-4cb1-9e15-290c6e5a231a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create full model callbacks\nfull_model_tensorboard = create_tensorboard_callback()\n# No validation set when training on all data, so we cannot monitor validation accuracy\nfull_model_early_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"accuracy\",\n                                                             patience=3)","metadata":{"id":"f4qy0wIieEsI"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note** Running cell below will take longer on first epoch because GPU has to load all images into memory","metadata":{"id":"5jqjBYKreq3o"}},{"cell_type":"code","source":"# Fit full model to full dataset\nfull_model.fit(x=full_data,\n               epochs=NUM_EPOCHS,\n               callbacks=[full_model_tensorboard, full_model_early_stopping])","metadata":{"id":"JSSqd5Onec3Y","outputId":"a0ad4a4e-e34d-41f3-94cc-e41adce89e0e"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Save trained full model\nsave_model(full_model, suffix=\"full-imageset-mobilenetv2-Adam\")","metadata":{"id":"KXABNKRzeniA","outputId":"8a08fa01-f815-421f-839a-4ea84d724b21"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load in full model\nloaded_full_model = load_model(\"drive/MyDrive/ML Projects/models/20210724-173146-full-imageset-mobilenetv2-Adam.h5\")","metadata":{"id":"4d7YcSW6jUkE","outputId":"d021b81d-a150-42a2-ef3c-466c4f7fd31c"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Making predictions on test dataset\n\nSince our model has been trained on images in the form of tensor batches, to make predictions on the test data, we will have to get it into the same format\n\nWe created `create_data_batches` earlier which can take a list of filenames as input and convert them into tensor batches\n\nTo make predictions on the test data, we will:\n* Get the test image filenames\n* Convert the filenames into test data batches using `create_data_batches` and setting the `test_data` parameter to `True` (the test data does not have labels)\n* Make predictions array by passing the test batches to `predict()` method called on our model","metadata":{"id":"nY8lqhOwl8Us"}},{"cell_type":"code","source":"# Load test image filenames\ntest_path = \"drive/MyDrive/ML Projects/test/\"\ntest_filenames = [test_path +  fname for fname in os.listdir(test_path)]\ntest_filenames[:10]","metadata":{"id":"0asOmOK_mf0Q","outputId":"64ce6d90-60cf-41f1-adb2-28d32fa5d534"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(test_filenames)","metadata":{"id":"EE3hE8ctoNYH","outputId":"432e4798-2b72-4119-c494-744419caf7e3"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create test data batch from filenames\ntest_data = create_data_batches(test_filenames, test_data=True)\n","metadata":{"id":"FEChbgsQoZn_","outputId":"5a0a2b84-cff8-4b34-ad72-4497dc6bfe3d"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_data","metadata":{"id":"M1pYBxdPoqfp","outputId":"6ba3be95-76e1-43d6-8e49-fe202f27583c"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note:** Calling `predict()` on our full model and passing it to test data batch will take a long time to run","metadata":{"id":"uCWPmrixpHoX"}},{"cell_type":"code","source":"# Make predictions on test data batch using loaded full model\ntest_predictions = loaded_full_model.predict(test_data, verbose=1)","metadata":{"id":"br5PQKeyouLU","outputId":"a2e0dcce-34c0-4339-9e83-a95b64a33f37"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Save predictions to csv file for later access\nnp.savetxt(\"drive/MyDrive/ML Projects/predictions/pred_array.csv\",\n           test_predictions,\n           delimiter=\",\")","metadata":{"id":"KY7jugSnp1YQ"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load predictions from csv file\ntest_predictions = np.loadtxt(\"drive/MyDrive/ML Projects/predictions/pred_array.csv\",\n                              delimiter=\",\")","metadata":{"id":"Ec7IT7XWq6Fa"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"id":"R10thN9GrIJ_"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_predictions.shape","metadata":{"id":"4xMYYheKrEnI","outputId":"8fd61818-f84d-4145-a129-460ea0604998"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Preparing test dataset predictions for Kaggle\n\nFrom the Kaggle sample submission, we find that the model prediction probability must be output in a DataFrame with an ID and a column for each dog breed. \n\nlink: https://www.kaggle.com/c/dog-breed-identification/overview/evaluation\n\nTo get data in this format, we will:\n* Create a pandas DataFrame with an ID column as well as a column for each dog breed\n* Add data to the ID column by extracting test image IDs from their filepaths\n* Add data (prediction probabilities) to each of the dog breed columns\n* Export DataFrame as a CSV to submit to Kaggle\n","metadata":{"id":"xGpPJ-rLrHSv"}},{"cell_type":"code","source":"# Create a pandas DataFrame with empty columns\npreds_df = pd.DataFrame(columns=[\"id\"] + list(unique_breeds))\npreds_df.head()","metadata":{"id":"4TWVKgCjsHRw","outputId":"055d3580-bc87-4db8-e945-24fbd5d6de3f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Append test image ID's to predictions DataFrame\ntest_ids = [os.path.splitext(path)[0] for path in os.listdir(test_path)]\ntest_ids[:10]","metadata":{"id":"wZf9ymtRwktv","outputId":"e5e07d52-bc9c-420b-aa04-676b7ebe5644"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preds_df[\"id\"] = test_ids","metadata":{"id":"CUMo2dGayCrH"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.path.splitext(test_filenames[0])","metadata":{"id":"Y7ibg-MMxpnv","outputId":"6d78f0e9-0686-409a-87c6-e153e01ad856"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Add prediction probabilities to each dog breed column\npreds_df[list(unique_breeds)] = test_predictions\npreds_df.head()\n","metadata":{"id":"m_xH0YZpx4C0","outputId":"73037bbf-f3e5-44f9-ea82-2154ef63a51d"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Export to .csv for submission to Kaggle\npreds_df.to_csv(\"drive/MyDrive/ML Projects/predictions/full_model_predictions_1_mobilenetV2.csv\",\n                index=False)","metadata":{"id":"6uAcvB4nyte_"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Making predictions on custom images\n\nTo make predictions on custom images, we must:\n* Get the filepaths of our own images\n* Turn the filepaths into data batches using `create_data_batches()`. \nSince our custom images don't have labels, we set the `test_data` parameter to `True`.\n* Pass the custom image data batch to our model's `predict()` method.\n* Convert prediction output probabilities to prediction labels.\n* Compare predicted labels to custom images","metadata":{"id":"AL7PHLwyzJvo"}},{"cell_type":"code","source":"# Get custom image filepaths\ncustom_path = \"drive/MyDrive/ML Projects/custom images/\"\ncustom_image_paths = [custom_path + fname for fname in os.listdir(custom_path)]","metadata":{"id":"j7OWZQZU2aKw"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"custom_image_paths","metadata":{"id":"DuutSUf05qmA","outputId":"f6c54a21-3ac7-486e-ff74-f7949becc75f"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Turn custom images into batch datasets\ncustom_data = create_data_batches(custom_image_paths, test_data=True)\ncustom_data","metadata":{"id":"H23acOKX6LSn","outputId":"a29435cf-647f-4d47-9e3e-4ed99eb2fc06"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Make predictions on custom data\ncustom_preds = loaded_full_model.predict(custom_data)\ncustom_preds.shape\n","metadata":{"id":"KJBg64lm728W","outputId":"6e607d8c-66d6-4dd0-97d1-30e47af12b62"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get custom image predictionn labels\ncustom_pred_labels = [get_pred_label(custom_preds[i]) for i in range(len(custom_preds))]\ncustom_pred_labels","metadata":{"id":"r2lDFk_M8COm","outputId":"dfac18da-ee4e-4d3f-b8c5-db54113f6112"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get custom images (unbatch_data() function will not work since there are no labels)\ncustom_images = []\n# Loop through unbatched data\nfor image in custom_data.unbatch().as_numpy_iterator():\n  custom_images.append(image)","metadata":{"id":"cV461lh28RqI"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check custom image predictions\nn = len(custom_image_paths)\n\nplt.figure(figsize=(10,10))\nfor i, image in enumerate(custom_images):\n  plt.subplot(int(math.sqrt(n))+1,int(math.sqrt(n))+1, i+1)\n  plt.xticks([])\n  plt.yticks([])\n  plt.title(custom_pred_labels[i])\n  plt.imshow(image)\n","metadata":{"id":"sDsYlqhm8ih-","outputId":"3885a9d5-99a6-46d9-9815-eaa9e4a67061"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"id":"fzJPlZUa9Pk-"},"execution_count":null,"outputs":[]}]}