{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":7327,"databundleVersionId":861871,"sourceType":"competition"},{"sourceId":944368,"sourceType":"datasetVersion","datasetId":511869},{"sourceId":5186836,"sourceType":"datasetVersion","datasetId":3015645},{"sourceId":7011972,"sourceType":"datasetVersion","datasetId":3997099},{"sourceId":7015282,"sourceType":"datasetVersion","datasetId":4033458}],"dockerImageVersionId":29853,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Preface:\n\nThis is my first Kernel (I hope I'm doing it right)! I'd love to do more like this. So if you have any feedback or advice, let me know in the comments or on [Twitter](https://dbourke.link/twitter).\n\n---\n\n# 🐶 Using Transfer Learning and TensorFlow 2.x to Classify Different Dog Breeds\n\nWho's that doggy in the window?\n\nDogs are incredible. But have you ever been sitting at a cafe, seen a dog and not known what breed it is? I have. And then someone says, \"it's an English Terrier\" and you think, how did they know that?\n\nIn this project we're going to be using machine learning to help us identify different breeds of dogs.\n\nTo do this, we'll be using data from the [Kaggle dog breed identification competition](https://www.kaggle.com/c/dog-breed-identification/overview). It consists of a collection of 10,000+ labelled images of 120 different dog breeds.\n\nThis kind of problem is called multi-class image classification. It's multi-class because we're trying to classify mutliple different breeds of dog. If we were only trying to classify dogs versus cats, it would be called binary classification.\n\nMulti-class image classification is an important problem because it's the same kind of technology Tesla uses in their self-driving cars or Airbnb uses in atuomatically adding information to their listings.\n\nSince the most important step in a deep learng problem is getting the data ready (turning it into numbers), that's what we're going to start with.\n\nWe're going to go through the following TensorFlow/Deep Learning workflow:\n![TesnorFlow WorkFlow Example](https://raw.githubusercontent.com/mrdbourke/zero-to-mastery-ml/master/images/unstructured-data-tensorflow-workflow.png)\n1. Get data ready (download from Kaggle, store, import).\n2. Prepare the data (preprocessing, the 3 sets, X & y).\n3. Choose and fit/train a model ([TensorFlow Hub](https://www.tensorflow.org/hub), `tf.keras.applications`, [TensorBoard](https://www.tensorflow.org/tensorboard), [EarlyStopping](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/EarlyStopping)).\n4. Evaluating a model (making predictions, comparing them with the ground truth labels).\n5. Improve the model through experimentation (start with 1000 images, make sure it works, increase the number of images).\n6. Save, sharing and reloading your model (once you're happy with the results).\n\nFor preprocessing our data, we're going to use TensorFlow 2.x. The whole premise here is to get our data into Tensors (arrays of numbers which can be run on GPUs) and then allow a machine learning model to find patterns between them.\n\nFor our machine learning model, we're going to be using a pretrained deep learning model from TensorFlow Hub. \n\nThe process of using a pretrained model and adapting it to your own problem is called **transfer learning**. We do this because rather than train our own model from scratch (could be timely and expensive), we leverage the patterns of another model which has been trained to classify images.\n\n## Getting our workspace ready\n\nBefore we get started, since we'll be using TensorFlow 2.x and TensorFlow Hub (TensorFlow Hub), let's import them.\n\nAnd we'll also check if we're using a GPU.","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0"}},{"cell_type":"code","source":"# THIS Kaggle is Top on from the Daniel Brouke my idol in the role of self-learn data scientist","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:20:26.83367Z","iopub.execute_input":"2023-11-21T04:20:26.833971Z","iopub.status.idle":"2023-11-21T04:20:26.837369Z","shell.execute_reply.started":"2023-11-21T04:20:26.833927Z","shell.execute_reply":"2023-11-21T04:20:26.836586Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tensorflow as tf\nimport tensorflow_hub as hub\n\nprint(\"TF version:\", tf.__version__)\nprint(\"Hub version:\", hub.__version__)\n\n# Check for GPU\nprint(\"GPU\", \"available (YESS!!!!)\" if tf.config.list_physical_devices(\"GPU\") else \"not available :(\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:20:30.138726Z","iopub.execute_input":"2023-11-21T04:20:30.13902Z","iopub.status.idle":"2023-11-21T04:20:30.14501Z","shell.execute_reply.started":"2023-11-21T04:20:30.138976Z","shell.execute_reply":"2023-11-21T04:20:30.144242Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"You might be wondering what a GPU is or why we need one. The short story is, a GPU is a computer chip which is faster at doing numerical computing. And since machine learning is all about finding patterns in numbers, that's what we're after. ","metadata":{}},{"cell_type":"markdown","source":"# Getting data ready\n\nSince much of machine learning is getting your data ready to be used with a machine learning model, we'll take extra care getting it setup.\n\nTo run this notebook, I've connected the Kaggle Dog Breed Identification Dataset (you can do this by clicking the \"+ Add Data\" button and searching for the right dataset). ","metadata":{}},{"cell_type":"markdown","source":"## Accessing the data\n\nNow the data files we're working with are available  we can start to check them out.\n\nLet's start with `labels.csv` which contains all of the image ID's and their assosciated dog breed (our data and labels).","metadata":{}},{"cell_type":"code","source":"# Checkout the labels of our data\nimport pandas as pd\nlabels_csv = pd.read_csv(\"../input/dog-breed-identification/labels.csv\")\nprint(labels_csv.describe())\nprint(labels_csv.head())","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:39:37.828131Z","iopub.execute_input":"2023-11-21T03:39:37.828444Z","iopub.status.idle":"2023-11-21T03:39:37.883492Z","shell.execute_reply.started":"2023-11-21T03:39:37.828388Z","shell.execute_reply":"2023-11-21T03:39:37.882671Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Looking at this, we can see there are 10222 different ID's (meaning 10222 different images) and 120 different breeds.\n\nLet's figure out how many images there are of each breed.","metadata":{}},{"cell_type":"code","source":"# How many images are there of each breed?\nlabels_csv[\"breed\"].value_counts().plot.bar(figsize=(20, 10));","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:39:45.669077Z","iopub.execute_input":"2023-11-21T03:39:45.669405Z","iopub.status.idle":"2023-11-21T03:39:47.520822Z","shell.execute_reply.started":"2023-11-21T03:39:45.669346Z","shell.execute_reply":"2023-11-21T03:39:47.519192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Okay sweet. If we were to roughly draw a line across the middle of the graph, we'd see there's about 60+ images for each dog breed.\n\nThis is a good amount as for some of their vision products [Google recommends a minimum of 10 images per class to get started](https://cloud.google.com/vision/automl/object-detection/docs/prepare). And as you might imagine, the more images per class available, the more chance a model has to figure out patterns between them.\n\nLet's check out one of the images.\n\n**Note:** Loading an image file for the first time may take a while as it gets loaded into the runtime memory.","metadata":{}},{"cell_type":"code","source":"from IPython.display import display, Image\nImage(\"../input/dog-breed-identification/train/000bec180eb18c7604dcecc8fe0dba07.jpg\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:04:59.574765Z","iopub.execute_input":"2023-11-21T03:04:59.575086Z","iopub.status.idle":"2023-11-21T03:04:59.582381Z","shell.execute_reply.started":"2023-11-21T03:04:59.575026Z","shell.execute_reply":"2023-11-21T03:04:59.581677Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Getting images and their labels\n\nSince we've got the image ID's and their labels in a DataFrame (`labels_csv`), we'll use it to create:\n* A list a filepaths to training images\n* An array of all labels\n* An array of all unique labels\n\nWe'll only create a list of filepaths to images rather than importing them all to begin with. This is because working with filepaths (strings) is much efficient than working with images.\n\nBecause we'll use it a few times, we'll save our training files pathname to a variable `train_path`.","metadata":{}},{"cell_type":"code","source":"# Define our training file path for ease of use\ntrain_path = \"../input/dog-breed-identification/train/\"","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:40:47.844272Z","iopub.execute_input":"2023-11-21T03:40:47.844567Z","iopub.status.idle":"2023-11-21T03:40:47.848638Z","shell.execute_reply.started":"2023-11-21T03:40:47.844522Z","shell.execute_reply":"2023-11-21T03:40:47.847641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create pathnames from image ID's\nfilenames = [train_path + fname + \".jpg\" for fname in labels_csv[\"id\"]]\n\n# Check the first 10 filenames\nfilenames[:10]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:40:51.221252Z","iopub.execute_input":"2023-11-21T03:40:51.221532Z","iopub.status.idle":"2023-11-21T03:40:51.232155Z","shell.execute_reply.started":"2023-11-21T03:40:51.22149Z","shell.execute_reply":"2023-11-21T03:40:51.231405Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we've got a list of all the filenames from the ID column of `labels_csv`, we can compare it to the number of files in our training data directory to see if they line up.\n\nIf they do, great. If not, there may be an issue with the code we've written.","metadata":{}},{"cell_type":"code","source":"# Check whether number of filenames matches number of actual image files\nimport os\nif len(os.listdir(train_path)) == len(filenames):\n    print(\"Filenames match actual amount of files!\")\nelse:\n    print(\"Filenames do not match actual amount of files, check the target directory.\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:41:26.084509Z","iopub.execute_input":"2023-11-21T03:41:26.084833Z","iopub.status.idle":"2023-11-21T03:41:26.095946Z","shell.execute_reply.started":"2023-11-21T03:41:26.08478Z","shell.execute_reply":"2023-11-21T03:41:26.095107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If everything worked, we should see a match up.\n\nLet's do one more check. Visualizing directly from a filepath.","metadata":{}},{"cell_type":"code","source":"# Check an image directly from a filepath\nImage(filenames[9000])","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:41:27.80407Z","iopub.execute_input":"2023-11-21T03:41:27.804356Z","iopub.status.idle":"2023-11-21T03:41:27.812234Z","shell.execute_reply.started":"2023-11-21T03:41:27.804312Z","shell.execute_reply":"2023-11-21T03:41:27.811315Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Woah! What a beast!\n\nNow we've got our image filepaths together, let's get the labels.\n\nWe'll take them from `labels_csv` and turn them into a NumPy array.","metadata":{}},{"cell_type":"code","source":"import numpy as np\nlabels = labels_csv[\"breed\"].to_numpy() # convert labels column to NumPy array\nlabels[:10]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:41:34.279026Z","iopub.execute_input":"2023-11-21T03:41:34.279309Z","iopub.status.idle":"2023-11-21T03:41:34.285255Z","shell.execute_reply.started":"2023-11-21T03:41:34.279265Z","shell.execute_reply":"2023-11-21T03:41:34.284536Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Wonderful, now lets do the same thing as before, compare the amount of labels to number of filenames.","metadata":{}},{"cell_type":"code","source":"# See if number of labels matches the number of filenames\nif len(labels) == len(filenames):\n    print(\"Number of labels matches number of filenames!\")\nelse:\n    print(\"Number of labels does not match number of filenames, check data directories.\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:41:43.009388Z","iopub.execute_input":"2023-11-21T03:41:43.009735Z","iopub.status.idle":"2023-11-21T03:41:43.014401Z","shell.execute_reply.started":"2023-11-21T03:41:43.009663Z","shell.execute_reply":"2023-11-21T03:41:43.013672Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If it all worked, we should have the same amount of images and labels.\n\nFinally, since a machine learning model can't take strings as input (what `labels` currently is), we'll have to convert our labels to numbers. \n\nTo begin with, we'll find all of the unique dog breed names.\n\nThen we'll go through the list of `labels` and compare them to unique breeds and create a list of booleans indicating which one is the real label (`True`) and which ones aren't (`False`).","metadata":{}},{"cell_type":"code","source":"# Find the unique label values\nunique_breeds = np.unique(labels)\nlen(unique_breeds)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:41:48.030583Z","iopub.execute_input":"2023-11-21T03:41:48.030914Z","iopub.status.idle":"2023-11-21T03:41:48.045973Z","shell.execute_reply.started":"2023-11-21T03:41:48.030853Z","shell.execute_reply":"2023-11-21T03:41:48.045203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The length of `unique_breeds` should be 120, meaning we're working with images of 120 different breeds of dogs.\n\nNow use `unique_breeds` to help turn our `labels` array into an array of booleans.","metadata":{}},{"cell_type":"code","source":"# Example: Turn one label into an array of booleans\nprint(labels[0])\nlabels[0] == unique_breeds # use comparison operator to create boolean array","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:41:49.585427Z","iopub.execute_input":"2023-11-21T03:41:49.585772Z","iopub.status.idle":"2023-11-21T03:41:49.591935Z","shell.execute_reply.started":"2023-11-21T03:41:49.585713Z","shell.execute_reply":"2023-11-21T03:41:49.591158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"That's for one example, let's do the whole thing.","metadata":{}},{"cell_type":"code","source":"# Turn every label into a boolean array\nboolean_labels = [label == np.array(unique_breeds) for label in labels]\nboolean_labels[:2]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:41:53.108179Z","iopub.execute_input":"2023-11-21T03:41:53.108479Z","iopub.status.idle":"2023-11-21T03:41:53.224046Z","shell.execute_reply.started":"2023-11-21T03:41:53.108427Z","shell.execute_reply":"2023-11-21T03:41:53.223275Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Why do it like this?\n\nRemember, an important concept in machine learning is converting your data to numbers before passing it to a machine learning model.\n\nIn this case, we've transformed a single dog breed name such as `boston_bull` into a one-hot array. \n\nLet's see an example.","metadata":{}},{"cell_type":"code","source":"# Example: Turning a boolean array into integers\nprint(labels[0]) # original label\nprint(np.where(unique_breeds == labels[0])[0][0]) # index where label occurs\nprint(boolean_labels[0].argmax()) # index where label occurs in boolean array\nprint(boolean_labels[0].astype(int)) # there will be a 1 where the sample label occurs","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:41:55.905111Z","iopub.execute_input":"2023-11-21T03:41:55.905389Z","iopub.status.idle":"2023-11-21T03:41:55.91178Z","shell.execute_reply.started":"2023-11-21T03:41:55.905345Z","shell.execute_reply":"2023-11-21T03:41:55.911064Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Wonderful! Now we've got our labels in a numeric format and our image filepaths easily accessible (they aren't numeric yet), let's split our data up.","metadata":{}},{"cell_type":"markdown","source":"# Creating our own validation set\n\nSince the dataset from Kaggle doesn't come with a validation set (a split of the data we can test our model on before making final predicitons on the test set), let's make one.\n\nWe could use Scikit-Learn's [train_test_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html) function or we could simply make manual splits of the data.\n\nFor accessibility later, let's save our filenames variable to `X` (data) and our labels to `y`.","metadata":{}},{"cell_type":"code","source":"# Setup X & y variables\nX = filenames\ny = boolean_labels\n\nprint(f\"Number of training images: {len(X)}\")\nprint(f\"Number of labels: {len(y)}\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:42:00.987004Z","iopub.execute_input":"2023-11-21T03:42:00.987281Z","iopub.status.idle":"2023-11-21T03:42:00.996578Z","shell.execute_reply.started":"2023-11-21T03:42:00.987238Z","shell.execute_reply":"2023-11-21T03:42:00.995857Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since we're working with 10,000+ images, it's a good idea to work with a portion of them to make sure things are working before training on them all.\n\nThis is because computing with 10,000+ images could take a fairly long time. And our goal when working through machine learning projects is to reduce the time between experiments.\n\nLet's start experimenting with 1000 and increase it as we need.","metadata":{}},{"cell_type":"code","source":"# Set number of images to use for experimenting\nNUM_IMAGES = 1000\nNUM_IMAGES","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:42:08.282179Z","iopub.execute_input":"2023-11-21T03:42:08.282465Z","iopub.status.idle":"2023-11-21T03:42:08.287674Z","shell.execute_reply.started":"2023-11-21T03:42:08.282423Z","shell.execute_reply":"2023-11-21T03:42:08.286793Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now let's split our data into training and validation sets. We'll use and 80/20 split (80% training data, 20% validation data).","metadata":{}},{"cell_type":"code","source":"# Import train_test_split from Scikit-Learn\nfrom sklearn.model_selection import train_test_split\n\n# Split them into training and validation using NUM_IMAGES \nX_train, X_val, y_train, y_val = train_test_split(X[:NUM_IMAGES],\n                                                  y[:NUM_IMAGES], \n                                                  test_size=0.2,\n                                                  random_state=42)\n\nlen(X_train), len(y_train), len(X_val), len(y_val)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:42:13.967158Z","iopub.execute_input":"2023-11-21T03:42:13.967442Z","iopub.status.idle":"2023-11-21T03:42:13.977266Z","shell.execute_reply.started":"2023-11-21T03:42:13.967399Z","shell.execute_reply":"2023-11-21T03:42:13.976354Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check out the training data (image file paths and labels)\nX_train[:5], y_train[:2]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:42:14.360593Z","iopub.execute_input":"2023-11-21T03:42:14.360888Z","iopub.status.idle":"2023-11-21T03:42:14.368035Z","shell.execute_reply.started":"2023-11-21T03:42:14.360843Z","shell.execute_reply":"2023-11-21T03:42:14.367236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preprocessing images (turning images into Tensors)\n\nOur labels are in numeric format but our images are still just file paths.\n\nSince we're using TensorFlow, our data has to be in the form of Tensors.\n\nA Tensor is a way to represent information in numbers. If you're familar with NumPy arrays (you should be), a Tensor can be thought of as a combination of NumPy arrays, except with the special ability to be used on a GPU.\n\nBecause of how TensorFlow stores information (in Tensors), it allows machine learning and deep learning models to be run on GPUs (generally faster at numerical computing).\n\nTo preprocess our images into Tensors we're going to write a function which does a few things:\n1. Takes an image filename as input.\n2. Uses TensorFlow to read the file and save it to a variable, `image`.\n3. Turn our `image` (a jpeg file) into Tensors.\n4. Resize the `image` to be of shape (224, 224).\n5. Return the modified `image`.\n\nA good place to read about this type of function is the [TensorFlow documentation on loading images](https://www.tensorflow.org/tutorials/load_data/images). \n\nYou might be wondering why (224, 224), which is (heigh, width). It's because this is the size of input our model (we'll see this soon) takes, an image which is (224, 224, 3).\n\nWhat? Where's the 3 from? We're getting ahead of ourselves but that's the number of colour channels per pixel, red, green and blue.\n\nLet's make this a little more concrete.","metadata":{}},{"cell_type":"code","source":"# Convert image to NumPy array\nfrom matplotlib.pyplot import imread\nimage = imread(filenames[42]) # read in an image\nimage.shape","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:42:20.054579Z","iopub.execute_input":"2023-11-21T03:42:20.055057Z","iopub.status.idle":"2023-11-21T03:42:20.066541Z","shell.execute_reply.started":"2023-11-21T03:42:20.054855Z","shell.execute_reply":"2023-11-21T03:42:20.065676Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Notice the shape of `image`. It's (257, 350, 3). This is height, width, colour channel value. \n\nAnd you can easily convert it to a Tensor using [`tf.constant()`](https://www.tensorflow.org/api_docs/python/tf/constant).","metadata":{}},{"cell_type":"code","source":"tf.constant(image)[:2]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:42:28.089705Z","iopub.execute_input":"2023-11-21T03:42:28.090023Z","iopub.status.idle":"2023-11-21T03:42:28.096939Z","shell.execute_reply.started":"2023-11-21T03:42:28.089976Z","shell.execute_reply":"2023-11-21T03:42:28.096183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Ok, now let's build that function we were talking about.","metadata":{}},{"cell_type":"code","source":"# Define image size\nIMG_SIZE = 224\n\ndef process_image(image_path):\n    \"\"\"\n      Takes an image file path and turns it into a Tensor.\n    \"\"\"\n    # Read in image file\n    image = tf.io.read_file(image_path)\n    # Turn the jpeg image into numerical Tensor with 3 colour channels (Red, Green, Blue)\n    image = tf.image.decode_jpeg(image, channels=3)\n    # Convert the colour channel values from 0-225 values to 0-1 values\n    image = tf.image.convert_image_dtype(image, tf.float32)\n    # Resize the image to our desired size (224, 244)\n    image = tf.image.resize(image, size=[IMG_SIZE, IMG_SIZE])\n    return image","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:43:06.291401Z","iopub.execute_input":"2023-11-21T03:43:06.291849Z","iopub.status.idle":"2023-11-21T03:43:06.298052Z","shell.execute_reply.started":"2023-11-21T03:43:06.291652Z","shell.execute_reply":"2023-11-21T03:43:06.297298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Creating data batches\n\nWonderful. Now we've got a function to convert our images into Tensors, we'll now build one to turn our data into batches (more specifically, a TensorFlow [`BatchDataset`](https://www.tensorflow.org/guide/data#batching_dataset_elements)).\n\nWhat's a batch?\n\nA batch (also called mini-batch) is a small portion of your data, say 32 (32 is generally the default batch size) images and their labels. In deep learning, instead of finding patterns in an entire dataset at the same time, you often find them one batch at a time.\n\nLet's say you're dealing with 10,000+ images (which we are). Together, these files may take up more memory than your GPU has. Trying to compute on them all would result in an error.\n\nInstead, it's more efficient to create smaller batches of your data and compute on one batch at a time.\n\nTensorFlow is very efficient when your data is in batches of (image, label) Tensors. So we'll build a function to do create those first. We'll take advantage of of `process_image` function at the same time.","metadata":{}},{"cell_type":"code","source":"# Create a simple function to return a tuple (image, label)\ndef get_image_label(image_path, label):\n    \"\"\"\n      Takes an image file path name and the associated label,\n      processes the image and returns a tuple of (image, label).\n    \"\"\"\n    image = process_image(image_path)\n    return image, label","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:43:28.43298Z","iopub.execute_input":"2023-11-21T03:43:28.433306Z","iopub.status.idle":"2023-11-21T03:43:28.438097Z","shell.execute_reply.started":"2023-11-21T03:43:28.433248Z","shell.execute_reply":"2023-11-21T03:43:28.437136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we've got a simple function to turn our image file path names and their associated labels into tuples (we can turn these into Tensors next), we'll create a function to make data batches.\n\nBecause we'll be dealing with 3 different sets of data (training, validation and test), we'll make sure the function can accomodate for each set.\n\nWe'll set a default batch size of 32 because [according to Yann Lecun](https://twitter.com/ylecun/status/989610208497360896?s=20) (one of the OG's of deep learning), friends don't let friends train with batch sizes over 32.","metadata":{}},{"cell_type":"code","source":"# Define the batch size, 32 is a good default\nBATCH_SIZE = 32\n\n# Create a function to turn data into batches\ndef create_data_batches(x, y=None, batch_size=BATCH_SIZE, valid_data=False, test_data=False):\n  \"\"\"\n  Creates batches of data out of image (x) and label (y) pairs.\n  Shuffles the data if it's training data but doesn't shuffle it if it's validation data.\n  Also accepts test data as input (no labels).\n  \"\"\"\n  # If the data is a test dataset, we probably don't have labels\n  if test_data:\n    print(\"Creating test data batches...\")\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(x))) # only filepaths\n    data_batch = data.map(process_image).batch(BATCH_SIZE)\n    return data_batch\n  \n  # If the data if a valid dataset, we don't need to shuffle it\n  elif valid_data:\n    print(\"Creating validation data batches...\")\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(x), # filepaths\n                                               tf.constant(y))) # labels\n    data_batch = data.map(get_image_label).batch(BATCH_SIZE)\n    return data_batch\n\n  else:\n    # If the data is a training dataset, we shuffle it\n    print(\"Creating training data batches...\")\n    # Turn filepaths and labels into Tensors\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(x), # filepaths\n                                              tf.constant(y))) # labels\n    \n    # Shuffling pathnames and labels before mapping image processor function is faster than shuffling images\n    data = data.shuffle(buffer_size=len(x))\n\n    # Create (image, label) tuples (this also turns the image path into a preprocessed image)\n    data = data.map(get_image_label)\n\n    # Turn the data into batches\n    data_batch = data.batch(BATCH_SIZE)\n  return data_batch","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:43:30.224642Z","iopub.execute_input":"2023-11-21T03:43:30.224951Z","iopub.status.idle":"2023-11-21T03:43:30.235622Z","shell.execute_reply.started":"2023-11-21T03:43:30.224899Z","shell.execute_reply":"2023-11-21T03:43:30.234657Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create training and validation data batches\ntrain_data = create_data_batches(X_train, y_train)\nval_data = create_data_batches(X_val, y_val, valid_data=True)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:43:35.775019Z","iopub.execute_input":"2023-11-21T03:43:35.775329Z","iopub.status.idle":"2023-11-21T03:43:36.03256Z","shell.execute_reply.started":"2023-11-21T03:43:35.775282Z","shell.execute_reply":"2023-11-21T03:43:36.031773Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check out the different attributes of our data batches\ntrain_data.element_spec, val_data.element_spec","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:13.600603Z","iopub.execute_input":"2023-11-21T03:48:13.600955Z","iopub.status.idle":"2023-11-21T03:48:13.606643Z","shell.execute_reply.started":"2023-11-21T03:48:13.600896Z","shell.execute_reply":"2023-11-21T03:48:13.605844Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Look at that! We've got our data in batches, more specifically, they're in Tensor pairs of (images, labels) ready for use on a GPU.\n\nBut having our data in batches can be a bit of a hard concept to understand.\n\n# Visualizing data batches\n\nLet's build a function which helps us visualize what's going on under the hood.","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# Create a function for viewing images in a data batch\ndef show_25_images(images, labels):\n  \"\"\"\n  Displays 25 images from a data batch.\n  \"\"\"\n  # Setup the figure\n  plt.figure(figsize=(15, 10))\n  # Loop through 25 (for displaying 25 images)\n  for i in range(25):\n    # Create subplots (5 rows, 5 columns)\n    ax = plt.subplot(5, 5, i+1)\n    # Display an image\n    plt.imshow(images[i])\n    # Add the image label as the title\n    plt.title(unique_breeds[labels[i].argmax()])\n    # Turn gird lines off\n    plt.axis(\"off\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:15.415737Z","iopub.execute_input":"2023-11-21T03:48:15.41603Z","iopub.status.idle":"2023-11-21T03:48:15.423123Z","shell.execute_reply.started":"2023-11-21T03:48:15.415986Z","shell.execute_reply":"2023-11-21T03:48:15.422019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To make computation efficient, a batch is a tighly wound collection of Tensors.\n\nSo to view data in a batch, we've got to unwind it.\n\nWe can do so by calling the [`as_numpy_iterator()`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#as_numpy_iterator) method on a data batch.\n\nThis will turn our a data batch into something which can be iterated over.\n\nPassing an iterable to [`next()`](https://docs.python.org/3/library/functions.html#next) will return the next item in the iterator.\n\nIn our case, next will return a batch of 32 images and label pairs.\n\n**Note:** Running the cell below and loading images may take a little while.","metadata":{}},{"cell_type":"code","source":"# Visualize training images from the training data batch\ntrain_images, train_labels = next(train_data.as_numpy_iterator())\nshow_25_images(train_images, train_labels)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:22.126821Z","iopub.execute_input":"2023-11-21T03:48:22.127159Z","iopub.status.idle":"2023-11-21T03:48:23.881022Z","shell.execute_reply.started":"2023-11-21T03:48:22.127098Z","shell.execute_reply":"2023-11-21T03:48:23.880299Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Look at all those beautiful dogs!\n\n**Question:** Rerun the cell above, why do you think a different set of images is displayed each time you run it?","metadata":{}},{"cell_type":"code","source":"# Visualize validation images from the validation data batch\nval_images, val_labels = next(val_data.as_numpy_iterator())\nshow_25_images(val_images, val_labels)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:28.13506Z","iopub.execute_input":"2023-11-21T03:48:28.1354Z","iopub.status.idle":"2023-11-21T03:48:29.854005Z","shell.execute_reply.started":"2023-11-21T03:48:28.135339Z","shell.execute_reply":"2023-11-21T03:48:29.853011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Even more dogs!\n\n**Question:** Why does running the cell above and viewing validation images return the same dogs each time?","metadata":{}},{"cell_type":"markdown","source":"# Creating and training a model\n\nNow our data is ready, let's prepare it modelling. We'll use an existing model from [TensorFlow Hub](https://tfhub.dev/).\n\nTensorFlow Hub is a resource where you can find pretrained machine learning models for the problem you're working on.\n\nUsing a pretrained machine learning model is often referred to as **transfer learning**.\n\n## Why use a pretrained model?\n\nBuilding a machine learning model and training it on lots from scratch can be expensive and time consuming.\n\nTransfer learning helps eliviate some of these by taking what another model has learned and using that information with your own problem.\n\n## How do we choose a model?\n\nSince we know our problem is image classification (classifying different dog breeds), we can navigate the [TensorFlow Hub page by our problem domain (image)](https://tfhub.dev/s?module-type=image-augmentation,image-classification,image-feature-vector,image-generator,image-object-detection,image-others,image-style-transfer,image-rnn-agent). \n\nWe start by choosing the image problem domain, and then can filter it down by subdomains, in our case, [image classification](https://tfhub.dev/s?module-type=image-classification).\n\nDoing this gives a list of different pretrained models we can apply to our task.\n\nClicking on one gives us information about the model as well as instructions for using it.\n\nFor example, clicking on the [mobilenet_v2_130_224](https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/4) model, tells us this model takes an input of images in the shape 224, 224. It also says the model has been trained in the domain of image classification.\n\nLet's try it out.\n\n# Building a model\nBefore we build a model, there are a few things we need to define:\n* The input shape (images, in the form of Tensors) to our model.\n* The output shape (image labels, in the form of Tensors) of our model.\n* The URL of the model we want to use.\n\nThese things will be standard practice with whatever machine learning model you use. And because we're using TensorFlow, everything will be in the form of Tensors.\n\n**Note:** To use TensorFlow Hub URLs on Kaggle, you will need to switch \"internet\" to *on* in the kernel settings.","metadata":{}},{"cell_type":"code","source":"# Setup input shape to the model\nINPUT_SHAPE = [None, IMG_SIZE, IMG_SIZE, 3] # batch, height, width, colour channels\n\n# Setup output shape of the model\nOUTPUT_SHAPE = len(unique_breeds) # number of unique labels\n\n# Setup model URL from TensorFlow Hub\nMODEL_URL = \"https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/4\"","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:30.138391Z","iopub.execute_input":"2023-11-21T03:48:30.138704Z","iopub.status.idle":"2023-11-21T03:48:30.143509Z","shell.execute_reply.started":"2023-11-21T03:48:30.138641Z","shell.execute_reply":"2023-11-21T03:48:30.142594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we've got the inputs, outputs and model we're using ready to go. We can start to put them together\n\nThere are many ways of building a model in TensorFlow but one of the best ways to get started is to [use the Keras API](https://www.tensorflow.org/guide/keras/overview).\n\nDefining a deep learning model in Keras can be as straightforward as saying, \"here are the layers of the model, the input shape and the output shape, let's go!\"\n\nKnowing this, let's create a function which:\n* Takes the input shape, output shape and the model we've chosen's URL as parameters.\n* Defines the layers in a Keras model in a sequential fashion (do this first, then this, then that).\n* Compiles the model (says how it should be evaluated and improved).\n* Builds the model (tells it what kind of input shape it'll be getting).\n* Returns the model.\n\nWe'll take a look at the code first, then dicuss each part.","metadata":{}},{"cell_type":"code","source":"# Create a function which builds a Keras model\ndef create_model(input_shape=INPUT_SHAPE, output_shape=OUTPUT_SHAPE, model_url=MODEL_URL):\n  print(\"Building model with:\", MODEL_URL)\n\n  # Setup the model layers\n  model = tf.keras.Sequential([\n    hub.KerasLayer(MODEL_URL), # Layer 1 (input layer), TensorFlow Hub layer, requires Kaggle internet setting turned on\n    tf.keras.layers.Dense(units=OUTPUT_SHAPE, \n                          activation=\"softmax\") # Layer 2 (output layer)\n  ])\n\n  # Compile the model\n  model.compile(\n      loss=tf.keras.losses.CategoricalCrossentropy(), # Our model wants to reduce this (how wrong its guesses are)\n      optimizer=tf.keras.optimizers.Adam(), # A friend telling our model how to improve its guesses\n      metrics=[\"accuracy\"] # We'd like this to go up\n  )\n\n  # Build the model\n  model.build(INPUT_SHAPE) # Let the model know what kind of inputs it'll be getting\n  \n  return model","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:36.211602Z","iopub.execute_input":"2023-11-21T03:48:36.211943Z","iopub.status.idle":"2023-11-21T03:48:36.219254Z","shell.execute_reply.started":"2023-11-21T03:48:36.211882Z","shell.execute_reply":"2023-11-21T03:48:36.218466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"What's happening here?\n\n## Setting up the model layers\n\nThere are two ways to do this in Keras, the [functional](https://www.tensorflow.org/guide/keras/functional) and [sequential API](https://www.tensorflow.org/guide/keras/overview#build_a_simple_model). We've used the sequential.\n\nWhich one should you use?\n\nThe Keras documentation states the functional API is the way to go for defining complex models but the sequential API (a linear stack of layers) is perfectly fine for getting started, which is what we're doing.\n\nThe first layer we use is the model from TensorFlow Hub (`hub.KerasLayer(MODEL_URL)`. So our first layer is actually an entire model (many more layers). This **input layer** takes in our images and finds patterns in them based on the patterns [`mobilenet_v2_130_224`](https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/4) has found.\n\nThe next layer (`tf.keras.layers.Dense()`) is the **output layer** of our model. It brings all of the information discovered in the input layer together and outputs it in the shape we're after, 120 (the number of unique labels we have).\n\nThe `activation=\"softmax\"` parameter tells the output layer, we'd like to assign a probability value to each of the 120 labels [somewhere between 0 & 1](https://en.wikipedia.org/wiki/Softmax_function). The higher the value, the more the model believes the input image should have that label. If we were working on a binary classification problem, we'd use `activation=\"sigmoid\"`.\n\nFor more on which activation function to use, see the article [Which Loss and Activation Functions Should I Use](https://towardsdatascience.com/deep-learning-which-loss-and-activation-functions-should-i-use-ac02f1c56aa8)?\n\n## Compiling the model\n\nThis one is best explained with a story.\n\nLet's say you're at the international hill descending championships. Where your start standing on top of a hill and your goal is to get to the bottom of the hill. The catch is you're blindfolded.\n\nLuckily, your friend Adam is standing at the bottom of the hill shouting instructions on how to get down.\n\nAt the bottom of the hill there's a judge evaluating how you're doing. They know where you need to end up so they compare how you're doing to where you're supposed to be. Their comparison is how you get scored.\n\nTransferring this to `model.compile()` terminology:\n* `loss` - The height of the hill is the loss function, the models goal is to minimize this, getting to 0 (the bottom of the hill) means the model is learning perfectly.\n* `optimizer` - Your friend Adam is the optimizer, he's the one telling you how to navigate the hill (lower the loss function) based on what you've done so far. His name is Adam because the [Adam optimizer](https://machinelearningmastery.com/adam-optimization-algorithm-for-deep-learning/) is a great general which performs well on most models. Other optimizers include [RMSprop](https://ruder.io/optimizing-gradient-descent/index.html#rmsprop) and [Stochastic Gradient Descent](https://en.wikipedia.org/wiki/Stochastic_gradient_descent).\n* `metrics` - This is the onlooker at the bottom of the hill rating how well your perfomance is. Or in our case, giving the accuracy of how well our model is predicting the correct image label.\n\n### Building the model\n\nWe use `model.build()` whenever we're using a layer from TensorFlow Hub to tell our model what input shape it can expect.\n\nIn this case, the input shape is `[None, IMG_SIZE, IMG_SIZE, 3]` or `[None, 224, 224, 3]` or `[batch_size, img_height, img_width, color_channels]`.\n\nBatch size is left as `None` as this is inferred from the data we pass the model. In our case, it'll be 32 since that's what we've set up our data batches as.\n\nNow we've gone through each section of the function, let's use it to create a model.\n\nWe can call `summary()` on our model to get idea of what our model looks like.","metadata":{}},{"cell_type":"code","source":"# Create a model and check its details\nmodel = create_model()\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:41.32854Z","iopub.execute_input":"2023-11-21T03:48:41.328862Z","iopub.status.idle":"2023-11-21T03:48:43.773459Z","shell.execute_reply.started":"2023-11-21T03:48:41.328804Z","shell.execute_reply":"2023-11-21T03:48:43.772647Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The non-trainable parameters are the patterns learned by `mobilenet_v2_130_224` and the trainable parameters are the ones in the dense layer we added.\n\nThis means the main bulk of the information in our model has already been learned and we're going to take that and adapt it to our own problem.","metadata":{}},{"cell_type":"markdown","source":"# Creating callbacks (things to help our model)\n\nWe've got a model ready to go but before we train it we'll make some callbacks.\n\nCallbacks are helper functions a model can use during training to do things such as save a models progress, check a models progress or stop training early if a model stops improving.\n\nThe two callbacks we're going to add are a TensorBoard callback and an Early Stopping callback.\n\n## TensorBoard Callback\n\n[TensorBoard](https://www.tensorflow.org/tensorboard/get_started) helps provide a visual way to monitor the progress of your model during and after training.\n\nIt can be used [directly in a notebook](https://www.tensorflow.org/tensorboard/tensorboard_in_notebooks) to track the performance measures of a model such as loss and accuracy.\n\nTo set up a TensorBoard callback and view TensorBoard in a notebook, we need to do three things:\n1. Load the TensorBoard notebook extension.\n2. Create a TensorBoard callback which is able to save logs to a directory and pass it to our model's `fit()` function.\n3. Visualize the our models training logs using the `%tensorboard` magic function (we'll do this later on).","metadata":{}},{"cell_type":"code","source":"# Load the TensorBoard notebook extension\n%load_ext tensorboard","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:43.775476Z","iopub.execute_input":"2023-11-21T03:48:43.775729Z","iopub.status.idle":"2023-11-21T03:48:43.781194Z","shell.execute_reply.started":"2023-11-21T03:48:43.775661Z","shell.execute_reply":"2023-11-21T03:48:43.780144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import datetime\n\n# Create a function to build a TensorBoard callback\ndef create_tensorboard_callback():\n  # Create a log directory for storing TensorBoard logs\n  logdir = os.path.join(\"logs\",\n                        # Make it so the logs get tracked whenever we run an experiment\n                        datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n  return tf.keras.callbacks.TensorBoard(logdir)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:49.067645Z","iopub.execute_input":"2023-11-21T03:48:49.067976Z","iopub.status.idle":"2023-11-21T03:48:49.073402Z","shell.execute_reply.started":"2023-11-21T03:48:49.067927Z","shell.execute_reply":"2023-11-21T03:48:49.072348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Early Stopping Callback\n\n[Early stopping](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/EarlyStopping) helps prevent overfitting by stopping a model when a certain evaluation metric stops improving. If a model trains for too long, it can do so well at finding patterns in a certain dataset that it's not able to use those patterns on another dataset it hasn't seen before (doesn't generalize).\n\nIt's basically like saying to our model, \"keep finding patterns until the quality of those patterns starts to go down.\"","metadata":{}},{"cell_type":"code","source":"# Create early stopping (once our model stops improving, stop training)\nearly_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"val_accuracy\",\n                                                  patience=3) # stops after 3 rounds of no improvements","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:49.481078Z","iopub.execute_input":"2023-11-21T03:48:49.481405Z","iopub.status.idle":"2023-11-21T03:48:49.485657Z","shell.execute_reply.started":"2023-11-21T03:48:49.481346Z","shell.execute_reply":"2023-11-21T03:48:49.484588Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training a model (on a subset of data)\nOur first model is only going to be trained on 1000 images. Or trained on 800 images and then validated on 200 images, meaning 1000 images total or about 10% of the total data.\n\nWe do this to make sure everything is working. And if it is, we can step it up later and train on the entire training dataset.\n\nThe final parameter we'll define before training is `NUM_EPOCHS` (also known as **number of epochs**).\n\n`NUM_EPOCHS` defines how many passes of the data we'd like our model to do. A pass is equivalent to our model trying to find patterns in each dog image and see which patterns relate to each label.\n\nIf `NUM_EPOCHS=1`, the model will only look at the data once and will probably score badly because it hasn't a chance to correct itself. It would be like you competing in the international hill descent championships and your friend Adam only being able to give you 1 single instruction to get down the hill.\n\nWhat's a good value for `NUM_EPOCHS`?\n\nThis one is hard to say. 10 could be a good start but so could 100. This is one of the reasons we created an early stopping callback. Having early stopping setup means if we set `NUM_EPOCHS` to 100 but our model stops improving after 22 epochs, it'll stop training.\n\nAlong with this, let's quickly check if we're still using a GPU.","metadata":{}},{"cell_type":"code","source":"# Check again if GPU is available (otherwise computing will take a looooonnnnggggg time)\nprint(\"GPU\", \"available (YESS!!!!)\" if tf.config.list_physical_devices(\"GPU\") else \"not available :(\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:50.809049Z","iopub.execute_input":"2023-11-21T03:48:50.809365Z","iopub.status.idle":"2023-11-21T03:48:50.813692Z","shell.execute_reply.started":"2023-11-21T03:48:50.80931Z","shell.execute_reply":"2023-11-21T03:48:50.812995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# How many rounds should we get the model to look through the data?\nNUM_EPOCHS = 100","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:51.227739Z","iopub.execute_input":"2023-11-21T03:48:51.228064Z","iopub.status.idle":"2023-11-21T03:48:51.23191Z","shell.execute_reply.started":"2023-11-21T03:48:51.228004Z","shell.execute_reply":"2023-11-21T03:48:51.230996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Boom! We've got a GPU running and `NUM_EPOCHS` setup. Let's create a simple function which trains a model. The function will:\n* Create a model using `create_model()`.\n* Setup a TensorBoard callback using `create_tensorboard_callback()` (we do this here so it creates a log directory of the current date and time).\n* Call the `fit()` function on our model passing it the training data, validatation data, number of epochs to train for and the callbacks we'd like to use.\n* Return the fitted model.","metadata":{}},{"cell_type":"code","source":"# Build a function to train and return a trained model\ndef train_model():\n  \"\"\"\n  Trains a given model and returns the trained version.\n  \"\"\"\n  # Create a model\n  model = create_model()\n\n  # Create new TensorBoard session everytime we train a model\n  tensorboard = create_tensorboard_callback()\n\n  # Fit the model to the data passing it the callbacks we created\n  model.fit(x=train_data,\n            epochs=NUM_EPOCHS,\n            validation_data=val_data,\n            validation_freq=1, # check validation metrics every epoch\n            callbacks=[tensorboard, early_stopping])\n  \n  return model","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:51.468812Z","iopub.execute_input":"2023-11-21T03:48:51.46914Z","iopub.status.idle":"2023-11-21T03:48:51.475017Z","shell.execute_reply.started":"2023-11-21T03:48:51.46908Z","shell.execute_reply":"2023-11-21T03:48:51.474263Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note:** When training a model for the first time, the first epoch will take a while to load compared to the rest. This is because the model is getting ready and the data is being initialised. Using more data will generally take longer, which is why we've started with ~1000 images. After the first epoch, subsequent epochs should take a few seconds.","metadata":{}},{"cell_type":"code","source":"# Fit the model to the data\nmodel = train_model()","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:48:54.334714Z","iopub.execute_input":"2023-11-21T03:48:54.335085Z","iopub.status.idle":"2023-11-21T03:49:35.687679Z","shell.execute_reply.started":"2023-11-21T03:48:54.335028Z","shell.execute_reply":"2023-11-21T03:49:35.687047Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Question:** It looks like our model might be overfitting (getting far better results on the training set than the validation set), what are some ways to prevent model overfitting? Hint: this may involve searching something like \"ways to prevent overfitting in a deep learning model?\".\n\n**Note:** Overfitting to begin with is a good thing. It means our model is learning something.","metadata":{}},{"cell_type":"markdown","source":"## Checking the TensorBoard logs\nNow our model has been trained, we can make its performance visual by checking the TensorBoard logs.\n\nThe TensorBoard magic function (`%tensorboard`) will access the logs directory we created earlier and viualize its contents.","metadata":{}},{"cell_type":"code","source":"%tensorboard --logdir logs","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:49:35.69146Z","iopub.execute_input":"2023-11-21T03:49:35.691809Z","iopub.status.idle":"2023-11-21T03:49:39.776342Z","shell.execute_reply.started":"2023-11-21T03:49:35.691657Z","shell.execute_reply":"2023-11-21T03:49:39.775514Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Thanks to our `early_stopping` callback, the model stopped training after 26 or so epochs (in my case, yours might be slightly different). This is because the validation accuracy failed to improve for 3 epochs.\n\nBut the good new is, we can definitely see our model is learning something. The validation accuracy got to 65% in only a few minutes.\n\nThis means, if we were to scale up the number of images, hopefully we'd see the accuracy increase.","metadata":{}},{"cell_type":"markdown","source":"# Making and evaluating predictions using a trained model\n\nBefore we scale up and train on more data, let's see some other ways we can evaluate our model. Because although accuracy is a pretty good indicator of how our model is doing, it would be even better if we could could see it in action.\n\nMaking predictions with a trained model is as calling `predict()` on it and passing it data in the same format the model was trained on.","metadata":{}},{"cell_type":"code","source":"# Make predictions on the validation data (not used to train on)\npredictions = model.predict(val_data, verbose=1) # verbose shows us how long there is to go\npredictions","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:49:39.777923Z","iopub.execute_input":"2023-11-21T03:49:39.778189Z","iopub.status.idle":"2023-11-21T03:49:40.841647Z","shell.execute_reply.started":"2023-11-21T03:49:39.778143Z","shell.execute_reply":"2023-11-21T03:49:40.840962Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check the shape of predictions\npredictions.shape","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:11:51.257531Z","iopub.execute_input":"2023-11-21T03:11:51.257831Z","iopub.status.idle":"2023-11-21T03:11:51.263039Z","shell.execute_reply.started":"2023-11-21T03:11:51.257788Z","shell.execute_reply":"2023-11-21T03:11:51.262283Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Making predictions with our model returns an array with a different value for each label.\n\nIn this case, making predictions on the validation data (200 images) returns an array (`predictions`) of arrays, each containing 120 different values (one for each unique dog breed).\n\nThese different values are the probabilities or the likelihood the model has predicted a certain image being a certain breed of dog. The higher the value, the more likely the model thinks a given image is a specific breed of dog.\n\nLet's see how we'd convert an array of probabilities into an actual label.","metadata":{}},{"cell_type":"code","source":"# First prediction\nprint(predictions[0])\nprint(f\"Max value (probability of prediction): {np.max(predictions[0])}\") # the max probability value predicted by the model\nprint(f\"Sum: {np.sum(predictions[0])}\") # because we used softmax activation in our model, this will be close to 1\nprint(f\"Max index: {np.argmax(predictions[0])}\") # the index of where the max value in predictions[0] occurs\nprint(f\"Predicted label: {unique_breeds[np.argmax(predictions[0])]}\") # the predicted label","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:11:53.611296Z","iopub.execute_input":"2023-11-21T03:11:53.611624Z","iopub.status.idle":"2023-11-21T03:11:53.620446Z","shell.execute_reply.started":"2023-11-21T03:11:53.611565Z","shell.execute_reply":"2023-11-21T03:11:53.619372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Having this information is great but it would be even better if we could compare a prediction to its true label and original image.\n\nTo help us, let's first build a little function to convert prediction probabilities into predicted labels.\n\n**Note:** Prediction probabilities are also known as confidence levels.","metadata":{}},{"cell_type":"code","source":"# Turn prediction probabilities into their respective label (easier to understand)\ndef get_pred_label(prediction_probabilities):\n  \"\"\"\n  Turns an array of prediction probabilities into a label.\n  \"\"\"\n  return unique_breeds[np.argmax(prediction_probabilities)]\n\n# Get a predicted label based on an array of prediction probabilities\npred_label = get_pred_label(predictions[0])\npred_label","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:12:04.747376Z","iopub.execute_input":"2023-11-21T03:12:04.74766Z","iopub.status.idle":"2023-11-21T03:12:04.754382Z","shell.execute_reply.started":"2023-11-21T03:12:04.747617Z","shell.execute_reply":"2023-11-21T03:12:04.753585Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Wonderful! Now we've got a list of all different predictions our model has made, we'll do the same for the validation images and validation labels.\n\nRemember, the model hasn't trained on the validation data, during the `fit()` function, it only used the validation data to evaluate itself. So we can use the validation images to visually compare our models predictions with the validation labels.\n\nSince our validation data (`val_data`) is in batch form, to get a list of validation images and labels, we'll have to unbatch it (using [`unbatch()`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#unbatch)) and then turn it into an iterator using [`as_numpy_iterator()`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#as_numpy_iterator).\n\nLet's make a small function to do so.","metadata":{}},{"cell_type":"code","source":"# Create a function to unbatch a batched dataset\ndef unbatchify(data):\n  \"\"\"\n  Takes a batched dataset of (image, label) Tensors and returns separate arrays\n  of images and labels.\n  \"\"\"\n  images = []\n  labels = []\n  # Loop through unbatched data\n  for image, label in data.unbatch().as_numpy_iterator():\n    images.append(image)\n    labels.append(unique_breeds[np.argmax(label)])\n  return images, labels\n\n# Unbatchify the validation data\nval_images, val_labels = unbatchify(val_data)\nval_images[0], val_labels[0]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:12:06.988672Z","iopub.execute_input":"2023-11-21T03:12:06.989028Z","iopub.status.idle":"2023-11-21T03:12:07.580997Z","shell.execute_reply.started":"2023-11-21T03:12:06.98896Z","shell.execute_reply":"2023-11-21T03:12:07.580267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Nailed it!\n\nNow we've got ways to get:\n* Prediction labels\n* Validation labels (truth labels)\n* Validation images\n\nLet's make some functions to make these all a bit more visualize.\n\nMore specifically, we want to be able to view an image, its predicted label and its actual label (true label).\n\nThe first function we'll create will:\n* Take an array of prediction probabilities, an array of truth labels, an array of images and an integer.\n* Convert the prediction probabilities to a predicted label.\n* Plot the predicted label, its predicted probability, the truth label and target image on a single plot.","metadata":{}},{"cell_type":"code","source":"def plot_pred(prediction_probabilities, labels, images, n=1):\n  \"\"\"\n  View the prediction, ground truth label and image for sample n.\n  \"\"\"\n  pred_prob, true_label, image = prediction_probabilities[n], labels[n], images[n]\n  \n  # Get the pred label\n  pred_label = get_pred_label(pred_prob)\n  \n  # Plot image & remove ticks\n  plt.imshow(image)\n  plt.xticks([])\n  plt.yticks([])\n\n  # Change the color of the title depending on if the prediction is right or wrong\n  if pred_label == true_label:\n    color = \"green\"\n  else:\n    color = \"red\"\n\n  plt.title(\"{} {:2.0f}% ({})\".format(pred_label,\n                                      np.max(pred_prob)*100,\n                                      true_label),\n                                      color=color)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:12:13.850465Z","iopub.execute_input":"2023-11-21T03:12:13.850761Z","iopub.status.idle":"2023-11-21T03:12:13.858567Z","shell.execute_reply.started":"2023-11-21T03:12:13.850717Z","shell.execute_reply":"2023-11-21T03:12:13.857588Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# View an example prediction, original image and truth label\nplot_pred(prediction_probabilities=predictions,\n          labels=val_labels,\n          images=val_images)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:12:14.577845Z","iopub.execute_input":"2023-11-21T03:12:14.578195Z","iopub.status.idle":"2023-11-21T03:12:14.736301Z","shell.execute_reply.started":"2023-11-21T03:12:14.578145Z","shell.execute_reply":"2023-11-21T03:12:14.735224Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Nice! Making functions to help visual your models results are really helpful in understanding how your model is doing.\n\nSince we're working with a multi-class problem (120 different dog breeds), it would also be good to see what other guesses our model is making. More specifically, if our model predicts a certain label with 24% probability, what else did it predict?\n\nLet's build a function to demonstrate. The function will:\n* Take an input of a prediction probabilities array, a ground truth labels array and an integer.\n* Find the predicted label using `get_pred_label()`.\n* Find the top 10:\n  * Prediction probabilities indexes\n  * Prediction probabilities values\n  * Prediction labels\n* Plot the top 10 prediction probability values and labels, coloring the true label green.","metadata":{}},{"cell_type":"code","source":"def plot_pred_conf(prediction_probabilities, labels, n=1):\n  \"\"\"\n  Plots the top 10 highest prediction confidences along with\n  the truth label for sample n.\n  \"\"\"\n  pred_prob, true_label = prediction_probabilities[n], labels[n]\n\n  # Get the predicted label\n  pred_label = get_pred_label(pred_prob)\n\n  # Find the top 10 prediction confidence indexes\n  top_10_pred_indexes = pred_prob.argsort()[-10:][::-1]\n  # Find the top 10 prediction confidence values\n  top_10_pred_values = pred_prob[top_10_pred_indexes]\n  # Find the top 10 prediction labels\n  top_10_pred_labels = unique_breeds[top_10_pred_indexes]\n\n  # Setup plot\n  top_plot = plt.bar(np.arange(len(top_10_pred_labels)), \n                     top_10_pred_values, \n                     color=\"grey\")\n  plt.xticks(np.arange(len(top_10_pred_labels)),\n             labels=top_10_pred_labels,\n             rotation=\"vertical\")\n\n  # Change color of true label\n  if np.isin(true_label, top_10_pred_labels):\n    top_plot[np.argmax(top_10_pred_labels == true_label)].set_color(\"green\")\n  else:\n    pass","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:12:24.895318Z","iopub.execute_input":"2023-11-21T03:12:24.895604Z","iopub.status.idle":"2023-11-21T03:12:24.90465Z","shell.execute_reply.started":"2023-11-21T03:12:24.895561Z","shell.execute_reply":"2023-11-21T03:12:24.903808Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_pred_conf(prediction_probabilities=predictions,\n               labels=val_labels,\n               n=9)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:12:25.609507Z","iopub.execute_input":"2023-11-21T03:12:25.609801Z","iopub.status.idle":"2023-11-21T03:12:25.79193Z","shell.execute_reply.started":"2023-11-21T03:12:25.609756Z","shell.execute_reply":"2023-11-21T03:12:25.79125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Wonderful! Now we've got some functions to help us visualize our predictions and evaluate our model, let's check out a few.","metadata":{}},{"cell_type":"code","source":"# Let's check a few predictions and their different values\ni_multiplier = 0\nnum_rows = 3\nnum_cols = 2\nnum_images = num_rows*num_cols\nplt.figure(figsize=(5*2*num_cols, 5*num_rows))\nfor i in range(num_images):\n  plt.subplot(num_rows, 2*num_cols, 2*i+1)\n  plot_pred(prediction_probabilities=predictions,\n            labels=val_labels,\n            images=val_images,\n            n=i+i_multiplier)\n  plt.subplot(num_rows, 2*num_cols, 2*i+2)\n  plot_pred_conf(prediction_probabilities=predictions,\n                labels=val_labels,\n                n=i+i_multiplier)\nplt.tight_layout(h_pad=1.0)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:12:53.59514Z","iopub.execute_input":"2023-11-21T03:12:53.595455Z","iopub.status.idle":"2023-11-21T03:12:56.002722Z","shell.execute_reply.started":"2023-11-21T03:12:53.595398Z","shell.execute_reply":"2023-11-21T03:12:56.001905Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Saving and reloading a model\n**Note:** The following functions are only examples of how you can save and load a model, they might not work very well on Kaggle but should work in your own Jupyter Notebooks.\n\nAfter training a model, it's a good idea to save it. Saving it means you can share it with colleagues, put it in an application and more importantly, won't have to go through the potentially expensive step of retraining it.\n\nThe format of an [entire saved Keras model is h5](https://www.tensorflow.org/tutorials/keras/save_and_load). So we'll make a function which can take a model as input and utilise the [`save()`](https://www.tensorflow.org/tutorials/keras/save_and_load#savedmodel_format) method to save it as a h5 file to a specified directory.","metadata":{}},{"cell_type":"code","source":"%mkdir models","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def save_model(model, suffix=None):\n    \"\"\"\n      Saves a given model in a models directory and appends a suffix (str)\n      for clarity and reuse.\n    \"\"\"\n    # Create model directory with current time\n    modeldir = os.path.join(\"./models\",\n                          datetime.datetime.now().strftime(\"%Y%m%d-%H%M%s\"))\n    model_path = modeldir + \"-\" + suffix + \".h5\" # save format of model\n    \n    print(f\"Saving model to: {model_path}...\")\n    tf.keras.models.save_model(model, model_path)\n    return model_path\n\n","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:26:02.451788Z","iopub.execute_input":"2023-11-21T03:26:02.452081Z","iopub.status.idle":"2023-11-21T03:26:02.458128Z","shell.execute_reply.started":"2023-11-21T03:26:02.452037Z","shell.execute_reply":"2023-11-21T03:26:02.457313Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If we've got a saved model, we'd like to load it, let's create a function which can take a model path and use the [`tf.keras.models.load_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/models/load_model) function to load it into the notebook.\n\nBecause we're using a component from TensorFlow Hub (`hub.KerasLayer`) we'll have to pass this as a parameter to the `custom_objects` parameter.","metadata":{}},{"cell_type":"code","source":"def load_model(model_path):\n    \"\"\"\n    Loads a saved model from a specified path.\n    \"\"\"\n    print(f\"Loading saved model from: {model_path}\")\n    model = tf.keras.models.load_model(model_path,\n                                     custom_objects={\"KerasLayer\":hub.KerasLayer})\n    return model","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:26:06.040914Z","iopub.execute_input":"2023-11-21T03:26:06.041245Z","iopub.status.idle":"2023-11-21T03:26:06.046208Z","shell.execute_reply.started":"2023-11-21T03:26:06.041187Z","shell.execute_reply":"2023-11-21T03:26:06.045286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Save our model trained on 1000 images\nsaved_1000_image_model_path = save_model(model, suffix=\"1000-images-Adam\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:26:06.213193Z","iopub.execute_input":"2023-11-21T03:26:06.213503Z","iopub.status.idle":"2023-11-21T03:26:06.435318Z","shell.execute_reply.started":"2023-11-21T03:26:06.21344Z","shell.execute_reply":"2023-11-21T03:26:06.434689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load our model trained on 1000 images\nloaded_model_1000_images = load_model(saved_1000_image_model_path)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:26:09.77216Z","iopub.execute_input":"2023-11-21T03:26:09.772455Z","iopub.status.idle":"2023-11-21T03:26:12.246659Z","shell.execute_reply.started":"2023-11-21T03:26:09.772404Z","shell.execute_reply":"2023-11-21T03:26:12.246036Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Compare the two models (the original one and loaded one). We can do so easily using the `evaluate()` method.","metadata":{}},{"cell_type":"code","source":"# Evaluate the pre-saved model\nmodel.evaluate(val_data)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:26:35.227017Z","iopub.execute_input":"2023-11-21T03:26:35.227301Z","iopub.status.idle":"2023-11-21T03:26:35.838455Z","shell.execute_reply.started":"2023-11-21T03:26:35.227259Z","shell.execute_reply":"2023-11-21T03:26:35.83756Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Evaluate the loaded model\nloaded_model_1000_images.evaluate(val_data)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:26:36.967787Z","iopub.execute_input":"2023-11-21T03:26:36.968065Z","iopub.status.idle":"2023-11-21T03:26:38.420463Z","shell.execute_reply.started":"2023-11-21T03:26:36.968023Z","shell.execute_reply":"2023-11-21T03:26:38.419649Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training a model (on the full data)\n\nNow we know our model works on a subset of the data, we can start to move forward with training one on the full data.\n\nAbove, we saved all of the training filepaths to `X` and all of the training labels to `y`. Let's check them out.","metadata":{}},{"cell_type":"code","source":"# Remind ourselves of the size of the full dataset\nlen(X), len(y)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:15:42.270266Z","iopub.execute_input":"2023-11-21T03:15:42.270564Z","iopub.status.idle":"2023-11-21T03:15:42.27617Z","shell.execute_reply.started":"2023-11-21T03:15:42.27052Z","shell.execute_reply":"2023-11-21T03:15:42.275306Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There we go! We've got over 10,000 images and labels in our training set.\n\nBefore we can train a model on these, we'll have to turn them into a data batch.\n\nThe beautiful thing is, we can use our `create_data_batches()` function from above which also preprocesses our images for us (thank you past us for writing a helpful function).","metadata":{}},{"cell_type":"code","source":"# Turn full training data in a data batch\nfull_data = create_data_batches(X, y)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:15:43.661196Z","iopub.execute_input":"2023-11-21T03:15:43.661483Z","iopub.status.idle":"2023-11-21T03:15:43.794605Z","shell.execute_reply.started":"2023-11-21T03:15:43.661439Z","shell.execute_reply":"2023-11-21T03:15:43.79384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Our data is in a data batch, all we need now is a model.\n\nAnd surprise, we've got a function for that too! Let's use `create_model()` to instantiate another model. ","metadata":{}},{"cell_type":"code","source":"# Instantiate a new model for training on the full dataset\nfull_model = create_model()","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:15:47.413575Z","iopub.execute_input":"2023-11-21T03:15:47.413925Z","iopub.status.idle":"2023-11-21T03:15:49.866212Z","shell.execute_reply.started":"2023-11-21T03:15:47.413868Z","shell.execute_reply":"2023-11-21T03:15:49.865452Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since we've made a new model instance, `full_model`, we'll need some callbacks too.","metadata":{}},{"cell_type":"code","source":"# Create full model callbacks\n\n# TensorBoard callback\nfull_model_tensorboard = create_tensorboard_callback()\n\n# Early stopping callback\n# Note: No validation set when training on all the data, therefore can't monitor validation accruacy\nfull_model_early_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"accuracy\",\n                                                             patience=3)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:15:54.083893Z","iopub.execute_input":"2023-11-21T03:15:54.084196Z","iopub.status.idle":"2023-11-21T03:15:54.088659Z","shell.execute_reply.started":"2023-11-21T03:15:54.084153Z","shell.execute_reply":"2023-11-21T03:15:54.087916Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%tensorboard --logdir logs","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:15:55.457379Z","iopub.execute_input":"2023-11-21T03:15:55.457701Z","iopub.status.idle":"2023-11-21T03:15:55.467553Z","shell.execute_reply.started":"2023-11-21T03:15:55.457642Z","shell.execute_reply":"2023-11-21T03:15:55.466622Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note:** Since running the cell below will cause the model to train on all of the data (10,000+) images, it may take a fairly long time to get started and finish. However, thanks to our `full_model_early_stopping` callback, it'll stop before it starts going too long.\n\nRemember, the first epoch is always the longest as data gets loaded into memory. After it's there, it'll speed up.","metadata":{}},{"cell_type":"code","source":"# Fit the full model to the full training data\nhistory = full_model.fit(x=full_data,\n               epochs=2,\n               callbacks=[full_model_tensorboard, \n                          full_model_early_stopping])","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:27:07.694323Z","iopub.execute_input":"2023-11-21T03:27:07.694609Z","iopub.status.idle":"2023-11-21T03:28:01.526314Z","shell.execute_reply.started":"2023-11-21T03:27:07.694565Z","shell.execute_reply":"2023-11-21T03:28:01.525558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"With results like this (close to 100% accuracy), it looks like our model could be overfitting (learning the training data too well). This is not a bad thing. We'll come back to this towards the end of the notebook.","metadata":{}},{"cell_type":"markdown","source":"# History Performance","metadata":{}},{"cell_type":"code","source":"history.history","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:28:09.022521Z","iopub.execute_input":"2023-11-21T03:28:09.022818Z","iopub.status.idle":"2023-11-21T03:28:09.028067Z","shell.execute_reply.started":"2023-11-21T03:28:09.022773Z","shell.execute_reply":"2023-11-21T03:28:09.027345Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['accuracy'])\nplt.title('model accuracy')\nplt.ylabel('accuracy')\nplt.xlabel('epoch')\nplt.legend(['Train'],\n           loc='upper left')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:28:42.821548Z","iopub.execute_input":"2023-11-21T03:28:42.821842Z","iopub.status.idle":"2023-11-21T03:28:43.098282Z","shell.execute_reply.started":"2023-11-21T03:28:42.821798Z","shell.execute_reply":"2023-11-21T03:28:43.096871Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.plot(history.history['loss'])\nplt.title('model loss')\nplt.ylabel('loss')\nplt.xlabel('epoch')\nplt.legend(['Train'],\n           loc='upper left')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:28:51.525675Z","iopub.execute_input":"2023-11-21T03:28:51.525985Z","iopub.status.idle":"2023-11-21T03:28:51.781717Z","shell.execute_reply.started":"2023-11-21T03:28:51.525932Z","shell.execute_reply":"2023-11-21T03:28:51.780616Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Saving and reloading the full model\n\nEven on a GPU, our full model took a while to train. So it's a good idea to save it.\n\nWe can do so using our `save_model()` function.\n\n**Challenge:** It may be a good idea to incorporate the `save_model()` function into a `train_model()` function. Or look into setting up a [checkpoint callback](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/ModelCheckpoint).","metadata":{}},{"cell_type":"code","source":"# Save model to file\nsaved_full_model_path = save_model(full_model, suffix=\"all-images-Adam\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:29:03.064892Z","iopub.execute_input":"2023-11-21T03:29:03.065173Z","iopub.status.idle":"2023-11-21T03:29:03.269014Z","shell.execute_reply.started":"2023-11-21T03:29:03.065132Z","shell.execute_reply":"2023-11-21T03:29:03.26827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load in the full model\nloaded_full_model = load_model(saved_full_model_path)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:29:06.046527Z","iopub.execute_input":"2023-11-21T03:29:06.046829Z","iopub.status.idle":"2023-11-21T03:29:08.310962Z","shell.execute_reply.started":"2023-11-21T03:29:06.046783Z","shell.execute_reply":"2023-11-21T03:29:08.31019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Making predictions on the test dataset\n\nSince our model has been trained on images in the form of Tensor batches, to make predictions on the test data, we'll have to get it into the same format.\n\nLuckily we created `create_data_batches()` earlier which can take a list of filenames as input and convert them into Tensor batches.\n\nTo make predictions on the test data, we'll:\n* Get the test image filenames.\n* Convert the filenames into test data batches using `create_data_batches()` and setting the `test_data` parameter to `True` (since there are no labels with the test images).\n* Make a predictions array by passing the test data batches to the `predict()` function.","metadata":{}},{"cell_type":"code","source":"# Load test image filenames (since we're using os.listdir(), these already have .jpg)\ntest_path = \"../input/dog-breed-identification/test/\"\ntest_filenames = [test_path + fname for fname in os.listdir(test_path)]\n\ntest_filenames[:10]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:29:11.097569Z","iopub.execute_input":"2023-11-21T03:29:11.097867Z","iopub.status.idle":"2023-11-21T03:29:11.574636Z","shell.execute_reply.started":"2023-11-21T03:29:11.097824Z","shell.execute_reply":"2023-11-21T03:29:11.573795Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# How many test images are there?\nlen(test_filenames)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:29:13.840994Z","iopub.execute_input":"2023-11-21T03:29:13.841347Z","iopub.status.idle":"2023-11-21T03:29:13.84706Z","shell.execute_reply.started":"2023-11-21T03:29:13.841281Z","shell.execute_reply":"2023-11-21T03:29:13.846191Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create test data batch\ntest_data = create_data_batches(test_filenames, test_data=True)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:29:15.625312Z","iopub.execute_input":"2023-11-21T03:29:15.625655Z","iopub.status.idle":"2023-11-21T03:29:15.644756Z","shell.execute_reply.started":"2023-11-21T03:29:15.625596Z","shell.execute_reply":"2023-11-21T03:29:15.643815Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Note:** Since there are 10,000+ test images, making predictions could take a while, even on a GPU. So beware running the cell below may take up to an hour.","metadata":{}},{"cell_type":"code","source":"# Make predictions on test data batch using the loaded full model\ntest_predictions = full_model.predict(test_data,\n                                      verbose=1)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:29:19.278997Z","iopub.execute_input":"2023-11-21T03:29:19.27931Z","iopub.status.idle":"2023-11-21T03:31:10.963609Z","shell.execute_reply.started":"2023-11-21T03:29:19.279255Z","shell.execute_reply":"2023-11-21T03:31:10.962939Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check out the test predictions\ntest_predictions[:10]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:31:12.060157Z","iopub.execute_input":"2023-11-21T03:31:12.060438Z","iopub.status.idle":"2023-11-21T03:31:12.067038Z","shell.execute_reply.started":"2023-11-21T03:31:12.060396Z","shell.execute_reply":"2023-11-21T03:31:12.066144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create a function to unbatch a batched dataset\ndef test_unbatchify(data):\n    \"\"\"\n    Takes a batched dataset of (image, label) Tensors and returns separate arrays\n    of images and labels.\n    \"\"\"\n    images = []\n    # Loop through unbatched data\n    for image in data.unbatch().as_numpy_iterator():\n        images.append(image)\n    return images","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:59:26.89792Z","iopub.execute_input":"2023-11-21T03:59:26.898297Z","iopub.status.idle":"2023-11-21T03:59:26.903556Z","shell.execute_reply.started":"2023-11-21T03:59:26.898234Z","shell.execute_reply":"2023-11-21T03:59:26.902736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Unbatchify the validation data\ntest_images = test_unbatchify(test_data)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:59:28.450391Z","iopub.execute_input":"2023-11-21T03:59:28.450676Z","iopub.status.idle":"2023-11-21T04:00:01.572712Z","shell.execute_reply.started":"2023-11-21T03:59:28.450632Z","shell.execute_reply":"2023-11-21T04:00:01.571994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_test_pred(prediction_probabilities, images, n=1):\n    \"\"\"\n    View the prediction, ground truth label and image for sample n.\n    \"\"\"\n    pred_prob, image = prediction_probabilities[n], images[n]\n    # Get the pred label\n    pred_label = get_pred_label(pred_prob)\n    \n    print(pred_label)\n    # Plot image & remove ticks\n    plt.imshow(image)\n    plt.xticks([])\n    plt.yticks([])\n    plt.title(f\"{pred_label} {np.max(pred_prob)*100:2.2f}%\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:06:39.671591Z","iopub.execute_input":"2023-11-21T04:06:39.671896Z","iopub.status.idle":"2023-11-21T04:06:39.67784Z","shell.execute_reply.started":"2023-11-21T04:06:39.671849Z","shell.execute_reply":"2023-11-21T04:06:39.676915Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# View an example prediction, original image and truth label\nplot_test_pred(prediction_probabilities=test_predictions,\n          images=test_images,\n          n=2)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:06:39.851498Z","iopub.execute_input":"2023-11-21T04:06:39.851819Z","iopub.status.idle":"2023-11-21T04:06:40.031959Z","shell.execute_reply.started":"2023-11-21T04:06:39.851769Z","shell.execute_reply":"2023-11-21T04:06:40.030772Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_test_pred_conf(prediction_probabilities, n=1):\n  \"\"\"\n  Plots the top 10 highest prediction confidences along with\n  the truth label for sample n.\n  \"\"\"\n  pred_prob = prediction_probabilities[n]\n\n  # Get the predicted label\n  pred_label = get_pred_label(pred_prob)\n\n  # Find the top 10 prediction confidence indexes\n  top_10_pred_indexes = pred_prob.argsort()[-10:][::-1]\n  # Find the top 10 prediction confidence values\n  top_10_pred_values = pred_prob[top_10_pred_indexes]\n  # Find the top 10 prediction labels\n  top_10_pred_labels = unique_breeds[top_10_pred_indexes]\n\n  # Setup plot\n  top_plot = plt.bar(np.arange(len(top_10_pred_labels)), \n                     top_10_pred_values, \n                     color=\"grey\")\n  plt.xticks(np.arange(len(top_10_pred_labels)),\n             labels=top_10_pred_labels,\n             rotation=\"vertical\")","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:19:01.755501Z","iopub.execute_input":"2023-11-21T04:19:01.756049Z","iopub.status.idle":"2023-11-21T04:19:01.764069Z","shell.execute_reply.started":"2023-11-21T04:19:01.755769Z","shell.execute_reply":"2023-11-21T04:19:01.763175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_test_pred_conf(prediction_probabilities=test_predictions,\n               n=2)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:19:02.655006Z","iopub.execute_input":"2023-11-21T04:19:02.655325Z","iopub.status.idle":"2023-11-21T04:19:02.931753Z","shell.execute_reply.started":"2023-11-21T04:19:02.655266Z","shell.execute_reply":"2023-11-21T04:19:02.930658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Custom Prediction","metadata":{}},{"cell_type":"code","source":"# Load test image filenames (since we're using os.listdir(), these already have .jpg)\ncustom_path = \"../input/demo-dog-breed-prediction/\"\ncustom_filenames = [custom_path + fname for fname in os.listdir(custom_path)]\n\ncustom_filenames[:10]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:19:04.61186Z","iopub.execute_input":"2023-11-21T04:19:04.612153Z","iopub.status.idle":"2023-11-21T04:19:04.618863Z","shell.execute_reply.started":"2023-11-21T04:19:04.612109Z","shell.execute_reply":"2023-11-21T04:19:04.618128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# How many test images are there?\nlen(custom_filenames)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:14:23.597338Z","iopub.execute_input":"2023-11-21T04:14:23.597624Z","iopub.status.idle":"2023-11-21T04:14:23.602903Z","shell.execute_reply.started":"2023-11-21T04:14:23.59758Z","shell.execute_reply":"2023-11-21T04:14:23.602046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create test data batch\ncustom_data = create_data_batches(custom_filenames, test_data=True)\n\ncustom_predictions = full_model.predict(custom_data,\n                                      verbose=1)\n\n# Check out the test predictions\ncustom_predictions[:10]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:14:39.291107Z","iopub.execute_input":"2023-11-21T04:14:39.29139Z","iopub.status.idle":"2023-11-21T04:14:39.385523Z","shell.execute_reply.started":"2023-11-21T04:14:39.291348Z","shell.execute_reply":"2023-11-21T04:14:39.384641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Unbatchify the validation data\ncustom_images = test_unbatchify(custom_data)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:14:47.018353Z","iopub.execute_input":"2023-11-21T04:14:47.018652Z","iopub.status.idle":"2023-11-21T04:14:47.083162Z","shell.execute_reply.started":"2023-11-21T04:14:47.018607Z","shell.execute_reply":"2023-11-21T04:14:47.08226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# View an example prediction, original image and truth label\nplot_test_pred(prediction_probabilities=custom_predictions,\n          images=custom_images,\n          n=3)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:16:34.265885Z","iopub.execute_input":"2023-11-21T04:16:34.266169Z","iopub.status.idle":"2023-11-21T04:16:34.418481Z","shell.execute_reply.started":"2023-11-21T04:16:34.266127Z","shell.execute_reply":"2023-11-21T04:16:34.416809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_test_pred_conf(prediction_probabilities=custom_predictions,\n               n=3)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T04:16:45.755413Z","iopub.execute_input":"2023-11-21T04:16:45.7557Z","iopub.status.idle":"2023-11-21T04:16:45.924523Z","shell.execute_reply.started":"2023-11-21T04:16:45.755649Z","shell.execute_reply":"2023-11-21T04:16:45.923443Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preparing test dataset predictions for Kaggle\n\nLooking at the [Kaggle sample submission](https://www.kaggle.com/c/dog-breed-identification/overview/evaluation), it looks like they want the models output probabilities each for label along with the image ID's.\n\nTo get the data in this format, we'll:\n*   Create a pandas DataFrame with an ID column as well as a column for each dog breed.\n*   Add data to the ID column by extracting the test image ID's from their filepaths.\n* Add data (the prediction probabilities) to each of the dog breed columns using the `unique_breeds` list and the `test_predictions` list.\n* Export the DataFrame as a CSV to submit it to Kaggle.\n","metadata":{}},{"cell_type":"code","source":"# Create pandas DataFrame with empty columns\npreds_df = pd.DataFrame(columns=[\"id\"] + list(unique_breeds))\npreds_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:31:14.585985Z","iopub.execute_input":"2023-11-21T03:31:14.586278Z","iopub.status.idle":"2023-11-21T03:31:14.620949Z","shell.execute_reply.started":"2023-11-21T03:31:14.586232Z","shell.execute_reply":"2023-11-21T03:31:14.619923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Append test image ID's to predictions DataFrame\npreds_df[\"id\"] = [os.path.splitext(path)[0] for path in os.listdir(test_path)]\npreds_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:31:16.359947Z","iopub.execute_input":"2023-11-21T03:31:16.360225Z","iopub.status.idle":"2023-11-21T03:31:16.433186Z","shell.execute_reply.started":"2023-11-21T03:31:16.360184Z","shell.execute_reply":"2023-11-21T03:31:16.432228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Add the prediction probabilities to each dog breed column\npreds_df[list(unique_breeds)] = test_predictions\npreds_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:31:21.025557Z","iopub.execute_input":"2023-11-21T03:31:21.025853Z","iopub.status.idle":"2023-11-21T03:31:21.199057Z","shell.execute_reply.started":"2023-11-21T03:31:21.025808Z","shell.execute_reply":"2023-11-21T03:31:21.198173Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Boom! Let's now export our predictions DataFrame to CSV so we can submit it to Kaggle.","metadata":{}},{"cell_type":"code","source":"preds_df.to_csv(\"submission_with_mobilienetV2.csv\",\n                 index=False)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:31:28.500925Z","iopub.execute_input":"2023-11-21T03:31:28.501226Z","iopub.status.idle":"2023-11-21T03:31:31.365063Z","shell.execute_reply.started":"2023-11-21T03:31:28.501183Z","shell.execute_reply":"2023-11-21T03:31:31.364203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Making predictions on custom images\n\nIt's great being able to make predictions on a test dataset already provided for us.\n\nBut how could we use our model on our own images?\n\nThe premise remains, if we want to make predictions on our own custom images, we have to pass them to the model in the same format the model was trained on.\n\nTo do so, we'll:\n* Get the filepaths of our own images.\n* Turn the filepaths into data batches using `create_data_batches()`. And since our custom images won't have labels, we set the `test_data` parameter to `True`.\n* Pass the custom image data batch to our model's `predict()` method.\n* Convert the prediction output probabilities to prediction labels.\n* Compare the predicted labels to the custom images.\n\n**Note:** To make predictions on custom images, I've uploaded pictures of my own to a directory located at `..input/dogvisioncustomimages/` (as seen in the cell below) or at https://www.kaggle.com/mrdbourke/dogvisioncustomimages. In order to make predictions on your own images, you can add your own data by clicking \"Add data\" and then navigating to upload.","metadata":{}},{"cell_type":"code","source":"# Get custom image filepaths\ncustom_path = \"../input/dogs-breed-dataset/dog_v1/french_bulldog\"\ncustom_image_paths = [custom_path + fname for fname in os.listdir(custom_path)]","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:37:34.197627Z","iopub.execute_input":"2023-11-21T03:37:34.197967Z","iopub.status.idle":"2023-11-21T03:37:34.222541Z","shell.execute_reply.started":"2023-11-21T03:37:34.197896Z","shell.execute_reply":"2023-11-21T03:37:34.22193Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Turn custom image into batch (set to test data because there are no labels)\ncustom_data = create_data_batches(custom_image_paths, test_data=True)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:37:36.666901Z","iopub.execute_input":"2023-11-21T03:37:36.667236Z","iopub.status.idle":"2023-11-21T03:37:36.683911Z","shell.execute_reply.started":"2023-11-21T03:37:36.667174Z","shell.execute_reply":"2023-11-21T03:37:36.683094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"custom_data","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:37:37.554859Z","iopub.execute_input":"2023-11-21T03:37:37.555187Z","iopub.status.idle":"2023-11-21T03:37:37.560386Z","shell.execute_reply.started":"2023-11-21T03:37:37.555129Z","shell.execute_reply":"2023-11-21T03:37:37.55948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Make predictions on the custom data\ncustom_preds = full_model.predict(custom_data)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:37:39.991874Z","iopub.execute_input":"2023-11-21T03:37:39.992203Z","iopub.status.idle":"2023-11-21T03:37:40.062588Z","shell.execute_reply.started":"2023-11-21T03:37:39.992145Z","shell.execute_reply":"2023-11-21T03:37:40.061174Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we've got some predictions arrays, let's convert them to labels and compare them with each image.","metadata":{}},{"cell_type":"code","source":"# Get custom image prediction labels\ncustom_pred_labels = [get_pred_label(custom_preds[i]) for i in range(len(custom_preds))]\ncustom_pred_labels","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:37:45.917169Z","iopub.execute_input":"2023-11-21T03:37:45.91745Z","iopub.status.idle":"2023-11-21T03:37:45.942376Z","shell.execute_reply.started":"2023-11-21T03:37:45.917406Z","shell.execute_reply":"2023-11-21T03:37:45.941317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get custom images (our unbatchify() function won't work since there aren't labels)\ncustom_images = []\n# Loop through unbatched data\nfor image in custom_data.unbatch().as_numpy_iterator():\n  custom_images.append(image)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:37:50.186904Z","iopub.execute_input":"2023-11-21T03:37:50.187206Z","iopub.status.idle":"2023-11-21T03:37:50.2753Z","shell.execute_reply.started":"2023-11-21T03:37:50.187162Z","shell.execute_reply":"2023-11-21T03:37:50.274027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check custom image predictions\nplt.figure(figsize=(10, 10))\nfor i, image in enumerate(custom_images):\n  plt.subplot(1, 3, i+1)\n  plt.xticks([])\n  plt.yticks([])\n  plt.title(custom_pred_labels[i])\n  plt.imshow(image)","metadata":{"execution":{"iopub.status.busy":"2023-11-21T03:37:50.487701Z","iopub.execute_input":"2023-11-21T03:37:50.488004Z","iopub.status.idle":"2023-11-21T03:37:50.496526Z","shell.execute_reply.started":"2023-11-21T03:37:50.487959Z","shell.execute_reply":"2023-11-21T03:37:50.495858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## What's next?\n\nWoah! What an effort. If you've made it this far, you've just gone end-to-end on a multi-class image classification problem.\n\nThis is the same style of problem self-driving cars have, except with different data.\n\nIf you're looking on where to go next, you've got plenty of options.\n\nYou could try to improve the full model we trained in this notebook in a few ways (there are a fair few options). Since our early experiment (using only 1000 images) hinted at our model overfitting (the results on the training set far outperformed the results on the validation set), one goal going forward would be to try and prevent it.\n\n1. [Trying another model from TensorFlow Hub](https://tfhub.dev/) - Perhaps a different model would perform better on our dataset. One option would be to experiment with a different pretrained model from TensorFlow Hub or look into the [`tf.keras.applications`](https://www.tensorflow.org/api_docs/python/tf/keras/applications) module.\n2. [Data augmentation](https://bair.berkeley.edu/blog/2019/06/07/data_aug/) - Take the training images and manipulate (crop, resize) or distort them (flip, rotate) to create even more training data for the model to learn from. Check out the [TensorFlow images](https://www.tensorflow.org/api_docs/python/tf/image) documentation for a whole bunch of functions you can use on images. A great idea would be to try and replicate the techniques in [this example cat vs. dog image classification notebook](https://github.com/google/eng-edu/blob/master/ml/pc/exercises/image_classification_part2.ipynb) for our dog breeds problem.\n2. [Fine-tuning](https://www.tensorflow.org/hub/tf2_saved_model#fine-tuning) - The model we used in this notebook was directly from TensorFlow Hub, we took what it had already learned from another dataset (ImageNet) and applied it to our own. Another option is to use what the model already knows and fine-tune this knowledge to our own dataset (pictures of dogs). This would mean all of the patterns within the model would be updated to be more specific to pictures of dogs rather than general images.\n\nIf you're ever after more, one of the best ways to find out something is to search for something like:\n- \"How to improve a TensorFlow 2.x image classification model?\"\n- \"TensorFlow 2.x image classification best practices\"\n- \"Transfer learning for image classification with TensorFlow 2.x\"\n\nAnd when you see an example you think might be beyond your reach (because it looks too complicated), remember, if in doubt, run the code. Try and reproduce what you see. This is the best way to get hands-on and build your own knowledge.\n\nNo one starts out knowing how to do everything single thing. They just get better are knowing what to look for.","metadata":{}}]}