{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# IdentiBreed - The dog breed identification AI\n\nIn this notebook I will go through an entire workflow of my first Neural Network/Deep Learning model. The ultimate goal of this project is creating an answer (a late submission which will allow me to measure the actual performance of the model) to the \"Dog Breed Identification\" competition available on Kaggle under <a href=\"https://www.kaggle.com/competitions/dog-breed-identification\">this link</a>. \n\nInitially this was a project from a Udemy course I was taking. Since then I expanded on this topic, made some improvements and further experimented with the model. At this stage there's only one model built - it's a transfer learning model using an import from TensorFlow Hub. At later stage I would like to attempt to create my own model from ground up.\n\nIf you've seen any of my previous projects on my <a href=\"https://github.com/Ale-Cie\">GitHub</a> you can expect the typical framework - The notebook will follow loosely these few steps:\n - Defining the problem - Kaggle did that actually for me. We can read on the competition's page that this is a multi-class classification problem\n - Exploring the data\n - Create initial model(s) and tune the hyperparameters - to that end I will use TensorFlow and TensorFlow Hub\n - Evaluate the model(s) \n - Experiment untill satisfied \n\n\nThis notebook was originally created, edited and trained on Google Colaboratory. However due to the limited GPU usage, session timeouts and overall better performance of my computer I decided on migrating it onto my machine and training it locally. I have no clue how long it will take to finish running here on Kaggle. \n\nI have some plans on expanding this project further, but all future updates will be most likely only available on my GitHub. You can visit the repository by clicking this <a href=\"https://github.com/Ale-Cie/IdentiBreed\">link</a>.\n\nLet's commence...\n","metadata":{"id":"LBwXuytd7EMb"}},{"cell_type":"markdown","source":"## Getting the workspace ready\n\nLet's start by importing the necessary tools and launching a TensorBoard session","metadata":{"id":"KMJdyrloBpNc"}},{"cell_type":"code","source":"\nimport tensorflow as tf\nimport tensorflow_hub as hub\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport datetime\nimport os\n\nfrom IPython.display import Image\nfrom sklearn.model_selection import train_test_split\n\n%load_ext tensorboard\n\n","metadata":{"id":"xJpG3bgj6-TC","execution":{"iopub.status.busy":"2023-05-15T15:58:46.081232Z","iopub.execute_input":"2023-05-15T15:58:46.081724Z","iopub.status.idle":"2023-05-15T15:58:46.088757Z","shell.execute_reply.started":"2023-05-15T15:58:46.081703Z","shell.execute_reply":"2023-05-15T15:58:46.087692Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Getting the dataset ready\n\nSince this notebook will be posted on Kaggle as well, I have to future-proof it by creating a `try` and `expect` for loading in the dataset. At this stage I will also created three variables:\n - `unique_lables` - this will be a list of all the dog breeds available in the data set\n - `bool_labels` - this will be a list of lists. Each list within this variable will correspond to an 'id' and which of the indexes of `unique_labels` represents its breed (this will help us with training the model)\n - `train_file_paths` - this will be a list containing all paths to training images\n\nAfter declaring these variables we will take a look at the value counts of instances of each breed within this dataset.","metadata":{"id":"htWpX0_ab-uG"}},{"cell_type":"code","source":"try:\n    labels_csv = pd.read_csv(\"./data/labels.csv\")\nexcept:\n    labels_csv = pd.read_csv(\"/kaggle/input/dog-breed-identification/labels.csv\")\n\nlabels_csv","metadata":{"id":"l4bCt9XhcG36","outputId":"301fb2f2-8923-4171-9eb5-683041c916c9","execution":{"iopub.status.busy":"2023-05-15T15:58:52.883768Z","iopub.execute_input":"2023-05-15T15:58:52.884799Z","iopub.status.idle":"2023-05-15T15:58:52.944895Z","shell.execute_reply.started":"2023-05-15T15:58:52.884748Z","shell.execute_reply":"2023-05-15T15:58:52.943994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(figsize=(12, 30))\nplt.barh(\n    y=np.unique(labels_csv[\"breed\"]),\n    width=labels_csv[\"breed\"].value_counts(),\n    color= \"tab:orange\")\nplt.margins(y=0)\nfor container in ax.containers:\n    ax.bar_label(container)\nax.set_title(\"Unique dog breeds and their representation in the Kaggle dataset\", loc=\"left\", fontdict={\"fontsize\": 16});\n\n","metadata":{"id":"KEw_iMYAdguN","outputId":"2f02582d-9e87-4aa3-a0b2-d08df729245b","execution":{"iopub.status.busy":"2023-05-15T15:58:56.071374Z","iopub.execute_input":"2023-05-15T15:58:56.071855Z","iopub.status.idle":"2023-05-15T15:58:57.873961Z","shell.execute_reply.started":"2023-05-15T15:58:56.071819Z","shell.execute_reply":"2023-05-15T15:58:57.873045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Here we create a list of 120 unique breeds in the dataset\nunique_labels = np.unique(np.array(labels_csv[\"breed\"]))\n\n# Now we go through the whole dataset and create a boolean representation of each training image's breed, I will later show what I mean\nbool_labels = [label == unique_labels for label in labels_csv[\"breed\"]]\n\n# Now let's create a variable which contains all paths to image files. We have to Kaggle-proof it as well\n# try:\n#     train_file_paths = [f\"./data/train/{id}.jpg\" for id in labels_csv[\"id\"]]\n# except:\ntrain_file_paths = [f\"/kaggle/input/dog-breed-identification/train/{id}.jpg\" for id in labels_csv[\"id\"]]","metadata":{"execution":{"iopub.status.busy":"2023-05-15T16:00:32.014308Z","iopub.execute_input":"2023-05-15T16:00:32.014647Z","iopub.status.idle":"2023-05-15T16:00:32.074486Z","shell.execute_reply.started":"2023-05-15T16:00:32.014622Z","shell.execute_reply":"2023-05-15T16:00:32.073728Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In order to explain what's the purpose of the `bool_labels` variable let's look at the example below. \n\n1. I start by printing the first item in the list `bool_labels` (which corresponds to the first item in the `labels_csv`) DataFrame - it's a list consisting of 120 boolean values. This amount is not accidental, since it's the exactly the same number as the the length of the `unique_labels` variable\n2. Next I will locate which index has the `True` value. In the same cell I will check what breed is saved in the `unique_labels` variable under this index and if it matches the true label of this particular image (we can check it in the 0 index row of `labels_csv`)","metadata":{}},{"cell_type":"code","source":"# 1.\nprint(bool_labels[0]), print(f\"The length of 'unique_labels' is {len(unique_labels)}\")","metadata":{"id":"wmUfKDADqvea","outputId":"5ac028e9-259e-4259-f670-3cba6ae086de","execution":{"iopub.status.busy":"2023-05-15T16:00:36.027635Z","iopub.execute_input":"2023-05-15T16:00:36.028006Z","iopub.status.idle":"2023-05-15T16:00:36.035526Z","shell.execute_reply.started":"2023-05-15T16:00:36.027976Z","shell.execute_reply":"2023-05-15T16:00:36.034227Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 2.\nidx = np.argwhere(bool_labels[0]==True)\n\nprint(f\"The True label is at index {idx}\") \nprint(f\"In 'unique_labels' under index {idx} is a breed - {unique_labels[idx]}\")\nprint(f\"The actual label is {labels_csv['breed'][0]}\")","metadata":{"execution":{"iopub.status.busy":"2023-05-15T16:00:36.529106Z","iopub.execute_input":"2023-05-15T16:00:36.529488Z","iopub.status.idle":"2023-05-15T16:00:36.536474Z","shell.execute_reply.started":"2023-05-15T16:00:36.529465Z","shell.execute_reply":"2023-05-15T16:00:36.535394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Preparing the data for modelling\n\nKaggle in this competition provides us with train and test folders - images in the test folder being unlabeled. These pictures will have to be labeled by our trained model and creating a late submission will give us a final verdict on the models performance. \n\nWe have to split the train images into a train set and validation set with `train_test_split()`, setting the `test_size` parameter to 0.2 and `random_state` to a number, which will make this notebook reproducable.\n\nInitial training and experimenting will be done on a fraction of the dataset. Let's create a parameter that we can change as we need","metadata":{"id":"nvzkRBn6pLIh"}},{"cell_type":"code","source":"# On Colab this @param decorator (?) created a  slider that could increase/decrease the amount of images used in following cells\n# Let's ignore it and just leave it at a 1000 for now\n\nIMG_NUM = 1000 #@param {type:\"slider\", min:1000, max:10222}","metadata":{"id":"2SKzh91QriY8","execution":{"iopub.status.busy":"2023-05-15T16:00:39.092889Z","iopub.execute_input":"2023-05-15T16:00:39.093297Z","iopub.status.idle":"2023-05-15T16:00:39.098118Z","shell.execute_reply.started":"2023-05-15T16:00:39.093267Z","shell.execute_reply":"2023-05-15T16:00:39.097150Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# We start by creating X and y variables which correspond to previouly declared ones \nX = train_file_paths\ny = bool_labels\n\n# Now it's time to split the data into train and validation sets\n# In total the amount of images in these will add up to the IMG_NUM\nX_train, X_validation, y_train, y_validation = train_test_split(X[:IMG_NUM], y[:IMG_NUM], test_size=0.2, random_state=7821)\n\nlen(X_train), len(y_train), len(X_validation), len(y_validation)\n","metadata":{"id":"04vMP5vqwLK0","outputId":"3c2635dd-f062-4463-a6f1-af15fb43d334","execution":{"iopub.status.busy":"2023-05-15T16:00:39.792370Z","iopub.execute_input":"2023-05-15T16:00:39.792712Z","iopub.status.idle":"2023-05-15T16:00:39.803263Z","shell.execute_reply.started":"2023-05-15T16:00:39.792686Z","shell.execute_reply":"2023-05-15T16:00:39.802158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Turning images into Tensors\n\nIn this notebook I will define functions as I go. I might attempt to make an abridged version of this with all functions moved to an external .py file, but that's a job for a later date.\n\nWe start by creating a function that will load each image from the path provided and turn it into a tensor. \n\nThe second function defined will take a tensor and a label corresponding to it and put them in a pair. That will help us with creating batches and training the model.","metadata":{"id":"r4fwzqYttcmQ"}},{"cell_type":"code","source":"# This is yet another variable that can be changed depending on preferences and (in this case) model used\nIMG_SIZE = 224\n\ndef turn_to_tensor(img_path, img_size=IMG_SIZE):\n  \"\"\"\n  This function takes an image path and uses it to load in an image and turn it into a Tensor.\n  Then it converts the dtype and resizes the image to img_size x img_size\n\n  Returns a normalized Tensor\n  \"\"\"\n\n  # Load in the image file and turn it into a numerical Tensor\n  img = tf.io.read_file(img_path)\n  img = tf.image.decode_jpeg(img, channels=3)\n\n  # Normalizing the image and resizing it to a constant (244x244)\n  img = tf.image.convert_image_dtype(img, tf.float32)\n  img = tf.image.resize(img, size=(img_size, img_size))\n\n  return img\n\ndef create_tensor_img_tuple(img_path, label):\n  \"\"\"\n  This function takes an image path and its corresponding label. \n  Then it turns the image from path provided into a Tensor. \n\n  Returns a tuple - (Tensor, label)\n  \"\"\"\n\n  image = turn_to_tensor(img_path)\n  \n  return image, label\n","metadata":{"id":"Fia2HxqQtBEi","execution":{"iopub.status.busy":"2023-05-15T16:01:09.569274Z","iopub.execute_input":"2023-05-15T16:01:09.569655Z","iopub.status.idle":"2023-05-15T16:01:09.576134Z","shell.execute_reply.started":"2023-05-15T16:01:09.569628Z","shell.execute_reply":"2023-05-15T16:01:09.575358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Dividing data into minibatches\n\nUsing the two functions from above we can go ahead and start splitting the images into minibatches that we will later feed to the model. \n\nWe will start by defining yet another function - `minibatch_maker`. We will be able to use it at every stage of experimentation since it will distinguish between training, validation and testing sets. \n\nFirst however we declare another variable `BATCH_SIZE` with value of 32. As Yann LeCun says:\n    \n    \"[...] Friends dont let friends use minibatches larger than 32\"","metadata":{"id":"jIvZ8Z2Nte97"}},{"cell_type":"code","source":"# We could change that, but I'm leaving it as is\nBATCH_SIZE = 32\n\ndef minibatch_maker(X, y=None, batch_size=BATCH_SIZE, valid_data=False, test_data=False):\n  \"\"\"\n  Creates data batches out of image and label pairs (X and y pairs).\n  Shuffles the data if it's training data.\n  Doesn't shuffle if it's validation data.\n  Accepts the Kaggle test data.\n\n  Returns a data batch\n  \"\"\"\n\n  if test_data:\n    # I commented out the line below because it was really annoying when I started tossing single images at the model to test it IRL\n    #print(\"Splitting test data into batches\")\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(X)))\n    data_batch = data.map(turn_to_tensor).batch(batch_size)\n    return data_batch\n  elif valid_data:\n    print(\"Splitting validation data into batches\")\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(X), tf.constant(y)))\n    data_batch = data.map(create_tensor_img_tuple).batch(batch_size)\n    return data_batch\n  else:\n    # This one we want to shuffle\n    np.random.seed(7821)\n    print(\"Splitting training data into batches\")\n    data = tf.data.Dataset.from_tensor_slices((tf.constant(X), tf.constant(y)))\n    data = data.shuffle(buffer_size=len(X))\n    data_batch = data.map(create_tensor_img_tuple).batch(batch_size)\n    return data_batch\n","metadata":{"id":"i3tQEd28swNm","execution":{"iopub.status.busy":"2023-05-15T16:01:12.047267Z","iopub.execute_input":"2023-05-15T16:01:12.047808Z","iopub.status.idle":"2023-05-15T16:01:12.054614Z","shell.execute_reply.started":"2023-05-15T16:01:12.047779Z","shell.execute_reply":"2023-05-15T16:01:12.053789Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Let's create train and validation batches now\n\ntrain_data = minibatch_maker(X_train, y_train)\nvalidation_data = minibatch_maker(X_validation, y_validation, valid_data=True)","metadata":{"id":"Q6BGbB2A1OE-","outputId":"fb48bdae-4a7c-44e3-f6ad-a1e63380af40","execution":{"iopub.status.busy":"2023-05-15T16:01:13.775808Z","iopub.execute_input":"2023-05-15T16:01:13.776288Z","iopub.status.idle":"2023-05-15T16:01:14.004048Z","shell.execute_reply.started":"2023-05-15T16:01:13.776244Z","shell.execute_reply":"2023-05-15T16:01:14.003133Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Building a model\n\nWe will start by defining variables that will later come in handy when creating a model.\n\nRight now we have to declare the input shape that will be fed into the first layer of our Neural Network and the output shape we are expecting to get. \n\n - `INPUT_SHAPE` will be equal to the dimensions of each image and will contain the information about the colour channels\n - `OUTPUT_SHAPE` will be simply an array of 120 values\n\nThe final variable that we will declare is the `MODEL_URL` which is a link to a MobileNet V2 model from the TensorFlow Hub\n","metadata":{"id":"9kl3hOjHx7V3"}},{"cell_type":"code","source":"# Input shape\nINPUT_SHAPE = [None, IMG_SIZE, IMG_SIZE, 3] # batch, height, width, colour channels\n\n# Output shape\nOUTPUT_SHAPE = len(unique_labels)\n\n# TensorFlow model URL\nMODEL_URL = \"https://tfhub.dev/google/imagenet/mobilenet_v2_130_224/classification/5\"","metadata":{"id":"MGsHQANT2b6U","execution":{"iopub.status.busy":"2023-05-15T16:01:20.667398Z","iopub.execute_input":"2023-05-15T16:01:20.667775Z","iopub.status.idle":"2023-05-15T16:01:20.673675Z","shell.execute_reply.started":"2023-05-15T16:01:20.667747Z","shell.execute_reply":"2023-05-15T16:01:20.672093Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now that we now what our inputs and outputs look like it's time to define a function that will create model with the settings we want to use.","metadata":{}},{"cell_type":"code","source":"# Creating a model instantiating function\n\ndef build_network(input_shape= INPUT_SHAPE, output_shape= OUTPUT_SHAPE, model_url= MODEL_URL):\n  \"\"\"\n  This function can take an input_shape, output_shape and a model_url in order to create a Neural Network.\n  Network created by this function has '2' layers:\n   - Input Layer\n   - Output Layer, which outputs data in the shape of output_shape\n  Any other layers present are hidden within the imported model.\n\n  Returns a network\n  \"\"\"\n  model = tf.keras.Sequential([\n      hub.KerasLayer(\n        model_url, \n        name=\"InputLayer\"\n      ), # Layer 1 - Input layer\n      tf.keras.layers.Dense(\n        units= output_shape, \n        activation= \"softmax\",\n        name= \"OutputLayer\"\n      ) # Layer 2 - Output layer \n\n  ])\n\n  model.compile(\n      loss= tf.keras.losses.CategoricalCrossentropy(),\n      optimizer= tf.keras.optimizers.Adam(),\n      metrics= [\"accuracy\"]\n  )\n\n  model.build(input_shape)\n\n  return model","metadata":{"id":"c0W5GjPx0-Ve","execution":{"iopub.status.busy":"2023-05-15T16:01:24.482758Z","iopub.execute_input":"2023-05-15T16:01:24.483147Z","iopub.status.idle":"2023-05-15T16:01:24.490428Z","shell.execute_reply.started":"2023-05-15T16:01:24.483120Z","shell.execute_reply":"2023-05-15T16:01:24.489246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = build_network()\nmodel.summary()","metadata":{"id":"nKRUNGznB2d0","outputId":"6760c076-b7ae-4349-84f5-57dbe406c1dc","execution":{"iopub.status.busy":"2023-05-15T16:01:26.949543Z","iopub.execute_input":"2023-05-15T16:01:26.949884Z","iopub.status.idle":"2023-05-15T16:01:29.732974Z","shell.execute_reply.started":"2023-05-15T16:01:26.949860Z","shell.execute_reply":"2023-05-15T16:01:29.731727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Creating callbacks\n\nNext step to take is creating callbacks - we will create only two. One of them will allow us to make checkopints during training, which will save us if anything goes terribly wrong and the whole procedure fails. The second callback will (hopefully) prevent the model from overfitting by checking if the accuracy is not improving for three consecutive epochs. ","metadata":{"id":"FzP33lJiLrk1"}},{"cell_type":"code","source":"def tensorboard_callback():\n  \"\"\"\n  Creates checkpoints and saves them in a /logs folder under the date of creation.\n\n  Returns a TensorBoard callback\n  \"\"\"\n  try:\n    logdir = os.path.join(\"./logs\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n  except:\n    logdir = os.path.join(\"/kaggle/working/\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n\n  return tf.keras.callbacks.TensorBoard(logdir)\n\nearly_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"val_accuracy\", patience= 3)","metadata":{"id":"KeKsN3n4LpKL","execution":{"iopub.status.busy":"2023-05-15T16:01:31.770232Z","iopub.execute_input":"2023-05-15T16:01:31.770599Z","iopub.status.idle":"2023-05-15T16:01:31.777812Z","shell.execute_reply.started":"2023-05-15T16:01:31.770572Z","shell.execute_reply":"2023-05-15T16:01:31.776639Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training the model\n\nWe are starting slow with the number of images previously declared as `IMG_NUM` variable (1000). Before training anything however I will declare yet another variable - `NUM_EPOCHS` which as the name suggests will decide on the number of training epochs that will take place during fitting.","metadata":{"id":"8GlFra03RrKq"}},{"cell_type":"code","source":"# It used to be a slider on Colab, now it's just a shadow of its former glory\nNUM_EPOCHS = 100 #@param {type: \"slider\", min: 10, max: 150, step: 10 }","metadata":{"id":"WK4q5UVOOOnk","execution":{"iopub.status.busy":"2023-05-15T16:01:34.563343Z","iopub.execute_input":"2023-05-15T16:01:34.563665Z","iopub.status.idle":"2023-05-15T16:01:34.568674Z","shell.execute_reply.started":"2023-05-15T16:01:34.563641Z","shell.execute_reply":"2023-05-15T16:01:34.567595Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def train_model():\n  \"\"\"\n  This function creates an instance of the Neural Netowrk, assigns the TensorBoard callback and fits the newly created network.\n\n  Returns a trained network\n  \"\"\"\n  model = build_network() # I'm not passing anything into the function since all the default values are set previously in the notebook\n  tensorboard = tensorboard_callback()\n\n  model.fit(\n    x= train_data, # Here we give the batches of image tensors and their labels\n    epochs= NUM_EPOCHS,\n    validation_data= validation_data,\n    validation_freq=1,\n    callbacks=[tensorboard, early_stopping])\n  \n  return model","metadata":{"id":"-WOKZxYfSlj0","execution":{"iopub.status.busy":"2023-05-15T16:01:40.949752Z","iopub.execute_input":"2023-05-15T16:01:40.950128Z","iopub.status.idle":"2023-05-15T16:01:40.956671Z","shell.execute_reply.started":"2023-05-15T16:01:40.950101Z","shell.execute_reply":"2023-05-15T16:01:40.955279Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = train_model()","metadata":{"id":"R2vB3TM8V9nz","outputId":"ab9aa14e-21f8-41c1-d87e-e7640bf02b89","execution":{"iopub.status.busy":"2023-05-15T16:01:44.973496Z","iopub.execute_input":"2023-05-15T16:01:44.973887Z","iopub.status.idle":"2023-05-15T16:04:35.588867Z","shell.execute_reply.started":"2023-05-15T16:01:44.973860Z","shell.execute_reply":"2023-05-15T16:04:35.586678Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Making predictions on the validation set\n\nNow that we have a trained neural network we can make predictions on the validation set and see how well it does. We will have to however put in a little elbow grease (or digit grease rather) to turn the prediction probabilities into actual dog breed names.","metadata":{"id":"5a77eFmlZ6zy"}},{"cell_type":"code","source":"# We start by making predictions which is fairly straight forward\npred_probabilities = model.predict(validation_data, verbose=1)\n\npred_probabilities","metadata":{"id":"cc6-207wBXYG","outputId":"48c2ae14-4655-4842-ccf7-fea0b72bae7a"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def name_predicted_label(probabilities):\n  \"\"\"\n  This function takes a list of 120 probabilities, finds the index of the most probable one and searches 'unique_labels' for the corresponding name\n\n  Returns the breed name\n  \"\"\"\n\n  return unique_labels[np.argmax(probabilities)]\n\ndef unbatch_data(batched_data):\n  \"\"\"\n  This function takes a data batch and unbatches it into an ordered list of images, and another ordered list of corresponding true labels\n\n  Returns images list and labels list\n  \"\"\"\n\n  images = []\n  labels = []\n\n  for image, label in batched_data.unbatch().as_numpy_iterator():\n    images.append(image)\n    labels.append(unique_labels[np.argmax(label)])\n\n  return images, labels","metadata":{"id":"dAt0gIgfEhC7"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"val_images, val_labels = unbatch_data(validation_data)","metadata":{"id":"8SFU7CJwLY0v"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display_prediction(prediction_probabilities, labels, images, n=0):\n  \"\"\"\n  This function takes a list of prediction probabilities, list of labels, list of images and an index. \n  It plots the result of models prediction - it displays the image of the dog with predicted breed, the probability and an actual label.\n  If the prediction was right, the text will appear in green. Else it will be red\n  \"\"\"\n\n  n = 0 if n > len(prediction_probabilities) else n\n  probabilities, true_label, image = prediction_probabilities[n], labels[n], images[n]\n\n  pred_label = name_predicted_label(probabilities)\n\n  if pred_label == true_label:\n    t_color = \"tab:green\"\n  else:\n    t_color = \"tab:red\"\n\n  plt.imshow(image)\n  plt.title(\"{} {:0.3f}% {}\".format(\n      pred_label,\n      np.max(probabilities)*100,\n      true_label),\n      color= t_color)\n  plt.xticks([])\n  plt.yticks([])\n","metadata":{"id":"J7qx4eU6MVkU"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_prediction(\n    prediction_probabilities=pred_probabilities,\n    labels= val_labels,\n    images= val_images\n)","metadata":{"id":"2j41EfATSKjk","outputId":"fcfe2c32-89bf-4da0-993a-33ae4f3c59dc"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_k_confidences(prediction_probabilities, labels, k=5, n=0):\n  \"\"\"\n  This function takes list of prediction probabilites, list of dog breed labels.\n  Optionally it takes:\n   - 'k' - a variable specifying how many highest probabilites to plot\n   - 'n' - a specific index of an image to check for prediction and an actual label\n  \"\"\"\n  n = 0 if n > len(prediction_probabilities) else n\n  pred_probabilities, true_label = prediction_probabilities[n], labels[n]\n\n  # pred_label = name_predicted_label(pred_probabilities)\n\n  top_k_idx = pred_probabilities.argsort()[-k:][::-1]\n  top_k_confidences = pred_probabilities[top_k_idx]\n  top_k_labels = unique_labels[top_k_idx]\n\n  top_k_plot = plt.bar(np.arange(len(top_k_labels)),\n                     top_k_confidences,\n                     color=\"grey\")\n  plt.xticks(np.arange(len(top_k_labels)),\n             labels=top_k_labels,\n             rotation=\"vertical\")\n  \n  if np.isin(true_label, top_k_labels):\n    top_k_plot[np.argmax(top_k_labels == true_label)].set_color(\"tab:green\")\n  else:\n    pass\n","metadata":{"id":"uqIPaiXvSVsA"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_k_confidences(\n    prediction_probabilities=pred_probabilities,\n    labels= val_labels\n    )","metadata":{"id":"vDvGNLI7axRt","outputId":"bf09232f-09b9-4bbe-d988-84993130bcbc"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_k_predictions(prediction_probabilities, labels, images, i_multiplier=0, n_rows= 4, n_cols= 2, k=5):\n  \"\"\"\n  This function creates a figure showing different predictions and the top probabilities. \n  It takes following arguments:\n   - prediction_probabilities - an ordered list of probabilities created by the model\n   - labels - an ordered list of actual labels for the dataset on which predictions were made\n   - images - an ordered list of tensors, on which predictions were made\n  \n  Optional arguments:\n   - i_multiplier - int that makes the function show next images\n   - n_rows - int specifying how many rows of predictions to describe\n   - n_cols - int specifying how many columns of predictions to describe\n   - k - int specifying how many top prediction values to plot\n  \"\"\"\n\n  n_images = n_rows * n_cols\n\n  plt.figure(figsize=(10*n_cols, 5*n_rows))\n  for i in range(n_images):\n    plt.subplot(n_rows, 2*n_cols, 2*i+1)\n    display_prediction(\n        prediction_probabilities= prediction_probabilities,\n        labels = labels,\n        images = images,\n        n = i+i_multiplier\n    )\n    plt.subplot(n_rows, 2*n_cols, 2*i+2)\n    plot_k_confidences(\n        prediction_probabilities= prediction_probabilities,\n        labels = labels,\n        k= k,\n        n= i+i_multiplier,\n    )\n  plt.tight_layout(h_pad=1.0)\n  plt.show()\n\nplot_k_predictions(\n  prediction_probabilities=pred_probabilities, \n  labels=val_labels, \n  images=val_images\n  )\n\n","metadata":{"id":"Yksf4GTZbE3d","outputId":"95ba872b-0c5b-4799-f7d3-afef518e0ba7"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Saving the model","metadata":{"id":"nFvcJ838gOAJ"}},{"cell_type":"code","source":"def save_model(model, suffix=None):\n  \"\"\"\n  Saves the model in the ./models directory. It takes two arguments:\n   - model - the actual model to be saved\n   - suffix (optional) - any user defined string that describes the model and will be attached to the file name for identification purposes\n\n  Returns path to the saved .h5 file\n  \"\"\"\n  try:\n    modeldir = os.path.join(\"./models\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%s\"))\n  except:  \n    modeldir = os.path.join(\"/kaggle/working/\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%s\")) \n  model_path = modeldir + \"-\" + suffix + \".h5\"\n\n  print(f\"Saving model to: {model_path}...\")\n  model.save(model_path)\n  \n  return model_path\n\ndef load_model(model_path):\n  \"\"\"\n  Loads the model from the .h5 file under 'model_path'\n\n  Returns model\n  \"\"\"\n  print(f\"Loading saved model from: {model_path}...\")\n  model = tf.keras.models.load_model(model_path,\n                                     custom_objects={\"KerasLayer\":hub.KerasLayer})\n  \n  return model","metadata":{"id":"M1EUJjofjoHA"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"save_model(\n    model= model, \n    suffix=\"1000-imgs-MobileNetV2-Adam\"\n    )","metadata":{"id":"aDTxueh1kyuA","outputId":"884a738b-b549-4efd-e0dc-b035d623af68"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training model on the full data set\n\nNow that we have experimented a bit on the model with just the 1000 images, we can go on and train a new instance of the model using the whole set of images. \n\nTo begin we have to create a new data batch using all of the images and the corresponding labels and then feed it into the model.\n\nBefore we fit the model however we have to create a new `EarlyStopping` callback, because we don't want to use a validation set this time around. ","metadata":{"id":"ZHQR91mO3FFM"}},{"cell_type":"code","source":"full_data = minibatch_maker(\n    X= X,\n    y= y,\n)","metadata":{"id":"P-abVJrYvK1G","outputId":"ed31d793-ffd7-4d4f-8594-27a7724aafc3"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"full_model = build_network()","metadata":{"id":"lXtuJp_0vnpM"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"full_model_tensorboard = tensorboard_callback()\n\n# This time around we will only monitor the accuracy of the model\n# However we will still stop training once the model stops improving for three epochs\nfull_model_early_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"accuracy\", patience=3)","metadata":{"id":"MkwUrUtnxSpr"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"full_model.fit(\n    x= full_data,\n    epochs= NUM_EPOCHS,\n    callbacks = [full_model_tensorboard, full_model_early_stopping]\n)","metadata":{"id":"XcZV-IunxpXt","outputId":"2409bac9-7a5b-4a1b-f57a-41741674bca9"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now that we have finally (nearly 15 minuts on my MacBook) trained the model on the entire dataset we can save it for posterity. \n\nThe model claims it's accuracy is about 98% (97.94% allegedly) but we can't really tell how good it performs until we test it in the real world scenario. For that we will use the test dataset provided, translate the predictions into labels and put them in the format that Kaggle specified on the competition page. Posting a late submission will tell us how good this model performed. I'll update this notebook with the score below.","metadata":{}},{"cell_type":"code","source":"save_model(\n    model= full_model, \n    suffix= \"full-image-set-MobileNetV2-Adam\"\n    )","metadata":{"id":"RCyTBmEry6Cg","outputId":"83ae60f8-1a75-4ab3-8864-ad13b4fecbd3"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Making predictions on test data\n\nIn order to make the predictions we have to start all over again - we have to create a list of file paths to the test images, create batches of data and feed them into the model.\n\nMaking batches differs this time however - we have to make sure to set the `test_data` argument in the `minibatch_maker()` function to `True`","metadata":{"id":"U0-KIHUDijKk"}},{"cell_type":"code","source":"try:\n    test_files = os.listdir(\"./data/test\")\n    try:\n        test_files.remove(\"test.txt\")\n    except:\n        print(\"No 'test.txt' file located. Creating X_test list\")\n    X_test = [f\"./data/test/{file}\" for file in test_files]\nexcept:\n    test_files = os.listdir(\"/kaggle/input/dog-breed-identification/test\")\n    try:\n        test_files.remove(\"test.txt\")\n    except:\n        print(\"No 'test.txt' file located. Creating X_test list\")\n    X_test = [f\"/kaggle/input/dog-breed-identification/test/{file}\" for file in test_files]\n\nlen(X_test)\n","metadata":{"id":"IoEIvmDbSrp6","outputId":"c91cfd3c-cf0d-4258-bb24-61bee9bb68d6"},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_data = minibatch_maker(\n    X= X_test,\n    test_data= True\n)","metadata":{"id":"Z_FPbEu4Tygu","outputId":"751dd38b-2056-4d1a-c5e0-a2dd7b3266f7"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now that we have the data ready for predicting it's time to sit down, feed the model and wait for it to finish.","metadata":{}},{"cell_type":"code","source":"test_predictions = full_model.predict(test_data, verbose=1)\n","metadata":{"id":"mV7Y7lvPXTs5","outputId":"0ada38cd-bde9-4215-e5be-bdf5f81f9200"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"After a little hiccough I prepared for myself when making the project GitHub ready we now have predictions, that we should save and then go on preparing for evaluation on Kaggle.","metadata":{}},{"cell_type":"code","source":"try:\n    np.savetxt(\"./predictions_array.csv\", test_predictions, delimiter=\",\")\nexcept:\n    np.savetxt(\"/kaggle/working/predictions_array.csv\", test_predictions, delimiter=\",\")","metadata":{"id":"kXRzxLo3Yn6F"},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Preparing the predictions for Kaggle publication\n\nNow all that is left for us to do is create a submission DataFrame that follows the guidelines presented by Kaggle on the competition <a href=\"https://www.kaggle.com/competitions/dog-breed-identification/overview/evaluation\">page</a>.","metadata":{}},{"cell_type":"code","source":"def submission_maker(predictions):\n    \"\"\"This function takes predictions array and along with the breed names and image names puts them into a Kaggle ready DataFrame \"\"\"\n    # We start by creating a list of all the test images names, without the extension which starts at index -4\n    global unique_labels\n    try:\n        index_names = [name[:-4] for name in os.listdir(\"./data/test/\")]\n    except:\n        index_names = [name[:-4] for name in os.listdir(\"/kaggle/input/dog-breed-identification/test\")]\n    try:\n        index_names.remove(\"test\")\n    except:\n        print(\"No 'test.txt' located, preparing index_names...\")\n\n    submission = pd.DataFrame(predictions, columns=unique_labels, index=index_names)\n\n    submission.index.name = \"id\"\n    return submission\n\nsubmission_df = submission_maker(test_predictions)\nsubmission_df","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Now after all this work we can go on and export our submission into a .csv file\ntry:\n    submission_df.to_csv(\"./submissions/full_model_preds_v2.csv\")\nexcept:\n    submission_df.to_csv(\"/kaggle/working/full_model_preds_v2.csv\")","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Model did quite good since it scored 0.92286 on Kaggle. Next step would be to get down and create an entirely original network, without relying on the TensorFlow Hub. It definitely is on my to-do list, but for now I feel like it's time to stop experimenting, end this notebook here and get on going with further exploration of this model in real-life scenarios.\n\nIn some time I'm going to update the github with what I hope will be a streamlined version of this notebook. I will move all the functions into a separate .py file and probably work in another .ipynb file on creating functions for the 'app' version of IdentiBreed.\n\nIf you've followed along big thanks to you, as for now I am signing out of this notebook. ","metadata":{}}]}