{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## A Friendly ☺️ Introduction to CNNs with Keras\n\nThis is a basic introduction to anyone getting started with CNNs. This is practitioner's guide, so there's not a lot of theory.\n\n### Computer Vision\n\nIn a nutshell, CNNs are special deep learning architectures that have revolutionized the field of computer vision. Computer Vision is a field that is concerned about deriving information from images using computers. Some examples of computer vision include- Identifying whether a given image contains an item, like identifying a pedestrian in traffic, identifying cracks in an industrial machine, identifying if an X-Ray is abnormal. Self-Driving cars rely on computer vision algorithms, which are often CNNs. Filters on Instagram, face recognition systems, all use deep learning under the hood.\n\nBut some general types of computer vision problems are:\n\n- Image Classification (Is this image of a cat?)\n- Object Detection (Is there a cat in this image, and WHERE exactly is it?)\n- Segmentation (give me the exact outline of the cat in this image, if it exists)\n\n![ExampleImage](https://miro.medium.com/max/788/1*OCsh4qf4lLoRAY-rlSZmJw.png)\n\n### Difference from Traditional Modeling\n\nIf you're here, it probably means you're familiar with basic machine learning concepts- like training data, predictions, feature engineering, etc. on tabular data. However image data is different from tabular data due to having a different structure, hence traditional algorithms like random forest cannot be used for classifying images.\n\n### The CNN Breakthrough\n\nThis is where CNNs made the breakthrough, and achieved tremendous results on image data. Before CNNs, image analytics required a lot of feature engineering and pre-processing (tons of hand-made filters). CNNs outperformed all the traditional methods without requiring such feature engineering. CNNs learnt the features and filters by itself. All you had to do was feed a lot of data to the model.\n\n### Deep Learning and Neural Networks\n\nCNN, that is, Convolutional Neural Networks are a subfield of neural networks, a family of algorithms. A neural network is a collection of nodes or neurons, where each neuron has a weight*. These weights are learnt during the training process such that the model is able to predict the output when input is given. When a lot of such neurons are stacked together, we get a neural network. A neural network with a lot of layers would be called deep neural network, a phenomenon which has driven majority of the AI success in the last decade.\n\nIn CNNs, the neurons are arranged and stacked in a manner suitable for images.\n\n### Convolutional Neural Networks\n\nIn CNN, we have filters (which are tiny 3x3 matrices) which \"convolve\" an image to get a transformed matrix. We won't worry about the theory and filters here. All you need to know, that filters transform the image to a new matrix. This matrix is made smaller by a method called Pooling. These two operations create one Convolution Layer, and several such Layers create a CNN. This order isn't mandatory, as we'll see later.\n\nThis is a [nice animation](https://www.youtube.com/watch?v=YRhxdVk_sIs) that showcases the convolution operation.\n\n
\n\n### Transfer Learning\n\nThere's a neat trick in deep learning called transfer learning- which is covered at the end of the notebook in case you make it.\n\nThat's quite a lot of theory, on to the problem at hand.\n\n## Problem At Hand\n\nThe task at hand is an image classification task. You're given a ton of images that are either a cat image or a dog image. Now, if you give a new image, you should be able to predict if it's of a dog or a cat.\n\nWe are going to train a CNN to do this. Using the keras library.","metadata":{}},{"cell_type":"markdown","source":"## Typical Workflow\n\nTypically when you work on a CNN task, this is how your notebook flow will look like:\nWhatever time you spend with CNNs, it will be in one of these sections.\n\n1. **Get the Images** \n\n(collecting images itself can be either a herculean task or sometimes ready-made data is available, time and effort varies with dataset)\n\nDifficulty Level: Varies\nTime Needed: Varies\n\n2. **Look at the Images and the Targets** \n\n(see how the images actually look like, what are the classes, how many of them.)\n\n3. **Create a Data Loader**\n\n(in most libraries you need a guy that reads the images and feeds to the model, and does the intermediate work- batching, augmentation, split, multiprocessing, etc. configuring this step will be a good chunk of your time\n)\n\n4. **Define Your Model**\n\n(how many CNN layers? How many filters, the optimizer, the loss function? this could be as easy as downloading/pasting an existing model in ten minutes, or the experiments could go on forever)\n\n5. **Train the Model**\n\n(now throw the dataloader function on the model and let it train. sit back and sip coffee.)\n\n6. **Get The Predictions**\n\n(here you actually use the model. for some task, or just to check if it's doing good. evaluating whether the model is giving good predictions can also be challenging in some use cases.)\n\n0. **Debugging**\n\nIf you keep it simple, load pre-built modules, the model will work. But there could be many possible problems that might arise in the task. These will be covered at the end.","metadata":{}},{"cell_type":"markdown","source":"Importing the libraries","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd \nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img\nfrom tensorflow.keras.utils import plot_model\nfrom tensorflow.keras.utils import to_categorical\nfrom keras.models import Sequential\nfrom keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten, Dense, Activation, BatchNormalization\nfrom sklearn.model_selection import train_test_split\nimport matplotlib.pyplot as plt\nimport random\nimport os\nfrom PIL import Image\nimport glob","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-09-30T06:26:23.097109Z","iopub.execute_input":"2023-09-30T06:26:23.097457Z","iopub.status.idle":"2023-09-30T06:26:23.103636Z","shell.execute_reply.started":"2023-09-30T06:26:23.097430Z","shell.execute_reply":"2023-09-30T06:26:23.102588Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!ls ../input/dogs-vs-cats/\n\n# the images are in a zip file in this folder","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:23:59.851943Z","iopub.execute_input":"2023-09-30T06:23:59.853068Z","iopub.status.idle":"2023-09-30T06:24:00.848921Z","shell.execute_reply.started":"2023-09-30T06:23:59.853031Z","shell.execute_reply":"2023-09-30T06:24:00.847787Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1. Get the Images\n\nExtract the images from the zip file. Now there are two folders: train/ and test1/","metadata":{}},{"cell_type":"code","source":"import zipfile\n\nwith zipfile.ZipFile(\"../input/dogs-vs-cats/train.zip\",\"r\") as z:\n z.extractall(\".\")\n \nwith zipfile.ZipFile(\"../input/dogs-vs-cats/test1.zip\",\"r\") as z:\n z.extractall(\".\")\n!ls /kaggle/working/","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:24:00.851250Z","iopub.execute_input":"2023-09-30T06:24:00.852229Z","iopub.status.idle":"2023-09-30T06:24:17.610264Z","shell.execute_reply.started":"2023-09-30T06:24:00.852181Z","shell.execute_reply":"2023-09-30T06:24:17.609009Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# let's look at any five files in the train folder\nglob.glob(\"/kaggle/working/train/*.jpg\")[:5]","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:24:17.612862Z","iopub.execute_input":"2023-09-30T06:24:17.613239Z","iopub.status.idle":"2023-09-30T06:24:17.752797Z","shell.execute_reply.started":"2023-09-30T06:24:17.613201Z","shell.execute_reply":"2023-09-30T06:24:17.751956Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"You now notice that the label is encoded in the filename itself- cat or dog! We need to extract that to be able to train the model.\n\nWe are collecting each filename, and the corresponding label in a pandas dataframe (needed later)","metadata":{}},{"cell_type":"code","source":"train_files = glob.glob(\"/kaggle/working/train/*.jpg\")\ntrain_labels = [i.strip('/kaggle/working/train/')[:3] for i in train_files]\ntrain_df = pd.DataFrame({'filename': train_files, 'class': train_labels})\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:24:19.692405Z","iopub.execute_input":"2023-09-30T06:24:19.692818Z","iopub.status.idle":"2023-09-30T06:24:19.784014Z","shell.execute_reply.started":"2023-09-30T06:24:19.692788Z","shell.execute_reply":"2023-09-30T06:24:19.783021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"First step is done- we have the images, and we have the labels. Let's move to the second step.","metadata":{"execution":{"iopub.status.busy":"2022-12-15T17:51:39.209994Z","iopub.execute_input":"2022-12-15T17:51:39.210383Z","iopub.status.idle":"2022-12-15T17:51:39.218492Z","shell.execute_reply.started":"2022-12-15T17:51:39.210350Z","shell.execute_reply":"2022-12-15T17:51:39.217576Z"}}},{"cell_type":"markdown","source":"## 2. Look at Them!\n\nWe now observe what the images look like.\nWe'll look at four random images from the data.","metadata":{"execution":{"iopub.status.busy":"2022-12-15T17:51:39.824303Z","iopub.execute_input":"2022-12-15T17:51:39.825266Z","iopub.status.idle":"2022-12-15T17:51:39.835381Z","shell.execute_reply.started":"2022-12-15T17:51:39.825228Z","shell.execute_reply":"2022-12-15T17:51:39.834084Z"}}},{"cell_type":"code","source":"fig, axs = plt.subplots(2, 2, figsize=(10,10))\naxs = axs.ravel()\nfor i in range(0,4):\n idx = random.choice(train_df.index)\n axs[i].imshow(Image.open(train_df['filename'][idx]))\n axs[i].set_title(train_df['class'][idx])","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:24:26.197490Z","iopub.execute_input":"2023-09-30T06:24:26.197823Z","iopub.status.idle":"2023-09-30T06:24:26.877016Z","shell.execute_reply.started":"2023-09-30T06:24:26.197796Z","shell.execute_reply":"2023-09-30T06:24:26.876068Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This is a pretty clean dataset- that's good. The subjects of the image are in center, occupy a majority of the image, no blurriness or anything. (Ideally you'd look at more images than a dozen tho,:) )\n\nYou might've noticed that the dimensions of the images are not constant, they vary a lot. This will be a problem for the CNN- it expects images of a fixed size! How do we take care of it? We don't, our data loader guy will do it for us.\n\n(Well, sometimes we might want to do it ourselves, if we think the resizing needs extra attention- but in this case, we'll let it be automated.)","metadata":{}},{"cell_type":"markdown","source":"## 3. Release the Loader!\n\nDataloaders are the unsung heroes of the CNN world- they take care of a lot of nasty work that would be a nightmare if we had to do it by hand. What they essentially do is simple- read the data from the disk, and feed to the model. But under the hood they take care of many things, like...\n\n1. Resizing, as we discussed. Ensuring that each image is of a fixed size.\n2. Batching. Feeding images one-by-one to the model is tedious, would take a lot of time. It's better to feed a large number of them at once (as much your computer will allow)\n3. Label Encoding. Computers don't understand string like 'cat' or 'dog', you have to convert them to numbers like 0 or 1. \n4. Data Augmentation. Create more images by slightly modifying an image (flipping it, rotating it teeny bit, adding some spots, etc.)\n5. Validation Split. Keras now has support for validation splitting.\n6. Sometimes the data is too large to fit into memory (10+ GB, say), then loaders can iterate through the dataset on disk chunk-by-chunk instead of loading everything at once.\n\nWe are using keras's `ImageDataGenerator` to create our training data loader.\n\nTwo steps:\n\n1. Define a `ImageDataGenerator` instance, and specify the augmentation strategies.\n2. Create a generator from this instance by specifying the image file paths and labels.\nPass this generator to the model for training.\n\nIn pytorch there's `torch.utils.data.Dataset` and`torch.utils.data.DataLoader`. Sometimes you may need to define a custom dataloader, but the default is good enough for most use cases.","metadata":{}},{"cell_type":"code","source":"train_datagen = ImageDataGenerator(\n rotation_range=5,\n rescale=1./255,\n horizontal_flip=True,\n shear_range=0.2,\n zoom_range=0.2,\n validation_split=0.2)\n\n# create a image data generator object. \n# all these are data augmentation parameters.\n\n# now let's specify the image size to which each image will be resized to\nimg_height, img_width = 224, 224\nbatch_size = 64","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:24:41.716882Z","iopub.execute_input":"2023-09-30T06:24:41.717574Z","iopub.status.idle":"2023-09-30T06:24:41.722944Z","shell.execute_reply.started":"2023-09-30T06:24:41.717540Z","shell.execute_reply":"2023-09-30T06:24:41.721999Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are two ways to create data generators/loaders from above instance. I recommend going through the [ImageDataGenerator API page](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator), but the two methods are:\n\n1. Flow from Dataframe (here, you can contain the filenames and labels in a pandas dataframe, and pass the dataframe), we are using this, remember the dataframe we created earlier?\n2. Flow from Directory (here, you can pass the path of a directory. This directory should contain subfolders corresponding to each class. You will have to rearrange your directory so that it looks like this.\n\n```\nTrain/\n---| Dog/\n ---| Dog1.jpg\n ---| Dog1.jpg\n---| Cat/\n ---| Cat1.jpg\n ---| Cat2.jpg\nVal/\n---| Dog/\n ---| Dog5.jpg\n ---| Dog6.jpg\n---| Cat/\n ---| Cat7.jpg\n ---| Cat8.jpg\n```\nSince we are using method1, we will not be rearranging the folders.","metadata":{}},{"cell_type":"code","source":"train_generator = train_datagen.flow_from_dataframe(\n train_df,\n target_size=(img_height, img_width),\n batch_size=batch_size,\n class_mode='categorical',\n subset='training') # set as training data\n\n# remember we put 0.2 validation split while defining ImageDataGenerator?\nvalidation_generator = train_datagen.flow_from_dataframe(\n train_df, # same directory as training data\n target_size=(img_height, img_width),\n batch_size=batch_size,\n class_mode='categorical',\n subset='validation') # set as validation data\n\nprint(\"Class Indices:\", train_generator.class_indices)","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:25:30.761940Z","iopub.execute_input":"2023-09-30T06:25:30.762891Z","iopub.status.idle":"2023-09-30T06:25:31.114981Z","shell.execute_reply.started":"2023-09-30T06:25:30.762856Z","shell.execute_reply":"2023-09-30T06:25:31.113913Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4. Define A Model\n\nNow we come to the meaty part- defining the CNN network, the engine of our application. As we discussed earlier, we are going to define a series of Convolution Layers, and each convolution layer consists of a convolution operation followed by a max pool layer.\n\nThis order is not mandatory- you can have two Convolution operations followed by one max pool, or three, or an average pool- that's the neat part of neural networks, they're so adaptable and malleable, and the best configurations are often found out by trial and error. In this case, we are going with the wisdom of our elders, and go by this order.\n\n### Head of A CNN\n\nAs we apply more Conv Layers, you will get a transformed matrix of somesize x somesize. But what good is a matrix to us? We need a simple answer- 0 or 1! In order to get this answer, we \"flatten\" the final matrix to a single vector of size somesize-times-2 x 1. Then we pass it through more neural network neurons to get a single neuron at the end. This neuron's output is constrained between 0 and 1. This is our final probability! If it's greater than 0.5, the prediction is 1, if not, it's 0.\n\nIf you have more than 2 classes, like predicting a digit. In this case, there would be ten neurons at the end. Each of their output would be the probability of that class.\n\n### Model Hyperparameters\n\nApart from number of Conv layers, there are other design choices while designing a CNN- they include selecting the optimizer, the learning rate, the loss function, the number of filters. For an introductory notebook, discussion on those is not necessary.\n\nWhat's important to note: passing the input shape to the first layer. Ensuring that the last layer corresponds to the number of classes. \n\nTry tinkering with this configuration to see how the results change. Try using only one Conv Layer, reducing number of filters, increasing number of filters, etc.","metadata":{}},{"cell_type":"code","source":"model = Sequential()\n\nmodel.add(Conv2D(32, (3, 3), activation='relu', \n input_shape=(img_width, img_height, 3)))\n# 32 here means this layer will contain 32 filters of size 3x3 being learnt\nmodel.add(BatchNormalization())\n# batchnorm is a useful layer that helps in convergence\nmodel.add(MaxPooling2D(pool_size=(2, 2)))\n# maxpooling will reduce the size of the image\nmodel.add(Dropout(0.25))\n# dropout is used for regularization, ensuring that model doesn't overfit\n\nmodel.add(Conv2D(64, (3, 3), activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(MaxPooling2D(pool_size=(2, 2)))\nmodel.add(Dropout(0.25))\n\nmodel.add(Conv2D(128, (3, 3), activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(MaxPooling2D(pool_size=(2, 2)))\nmodel.add(Dropout(0.25))\n\n# convolutional block is complete. now on to defining the \"head\"\n\n# first flatten the matrix to get a single array\nmodel.add(Flatten())\n# adding a dense hidden layer of neurons\nmodel.add(Dense(512, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.5))\n# finally the output layer with neurons=number of classes and softmax activation\nmodel.add(Dense(2, activation='softmax')) # 2 because we have cat and dog classes","metadata":{"execution":{"iopub.status.busy":"2023-01-07T09:23:59.962603Z","iopub.execute_input":"2023-01-07T09:23:59.962988Z","iopub.status.idle":"2023-01-07T09:24:03.643146Z","shell.execute_reply.started":"2023-01-07T09:23:59.962953Z","shell.execute_reply":"2023-01-07T09:24:03.642101Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's visualize how our network looks like, and what the shapes of input and output at each layer.\n\nThe shapes of input and output can be useful for debugging. If there's a mismatch between output of one layer and input of next, model will throw up error.","metadata":{}},{"cell_type":"code","source":"plot_model(model, show_shapes=True, show_layer_names=False, dpi=60)","metadata":{"execution":{"iopub.status.busy":"2022-12-18T15:12:05.061699Z","iopub.execute_input":"2022-12-18T15:12:05.062156Z","iopub.status.idle":"2022-12-18T15:12:05.977001Z","shell.execute_reply.started":"2022-12-18T15:12:05.062119Z","shell.execute_reply":"2022-12-18T15:12:05.975850Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# compile the model while defining a loss, optimizer, and metrics to track, \n# and add callbacks if necessary\nmodel.compile(loss='categorical_crossentropy', \n optimizer='adam', \n metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2022-12-18T15:12:23.625078Z","iopub.execute_input":"2022-12-18T15:12:23.625477Z","iopub.status.idle":"2022-12-18T15:12:23.643979Z","shell.execute_reply.started":"2022-12-18T15:12:23.625433Z","shell.execute_reply":"2022-12-18T15:12:23.642599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"epochs=10\nhistory = model.fit(\n train_generator, \n epochs=epochs,\n validation_data=validation_generator,\n)","metadata":{"execution":{"iopub.status.busy":"2022-12-18T15:13:35.505438Z","iopub.execute_input":"2022-12-18T15:13:35.506356Z","iopub.status.idle":"2022-12-18T15:19:06.240770Z","shell.execute_reply.started":"2022-12-18T15:13:35.506307Z","shell.execute_reply":"2022-12-18T15:19:06.239750Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_loss(history):\n fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(6, 6))\n ax1.plot(history.history['loss'], color='b', label=\"Training loss\")\n ax1.plot(history.history['val_loss'], color='r', label=\"validation loss\")\n ax1.set_xticks(np.arange(1, epochs, 1))\n\n ax2.plot(history.history['accuracy'], color='b', label=\"Training accuracy\")\n ax2.plot(history.history['val_accuracy'], color='r',label=\"Validation accuracy\")\n ax2.set_xticks(np.arange(1, epochs, 1))\n\n legend = plt.legend(loc='best', shadow=True)\n plt.tight_layout()\n plt.show()\nplot_loss(history)","metadata":{"execution":{"iopub.status.busy":"2022-12-18T15:19:06.242904Z","iopub.execute_input":"2022-12-18T15:19:06.243484Z","iopub.status.idle":"2022-12-18T15:19:06.507317Z","shell.execute_reply.started":"2022-12-18T15:19:06.243434Z","shell.execute_reply":"2022-12-18T15:19:06.506423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 6. Get the Predictions\n\nNow that the model is trained, let's check if the model is giving us good predictions, by trying it out on the test data.","metadata":{}},{"cell_type":"code","source":"test_files = glob.glob('/kaggle/working/test1/*.jpg')\ntest_df = pd.DataFrame({'filename': test_files})\ntest_gen = ImageDataGenerator(rescale=1./255)\ntest_generator = test_gen.flow_from_dataframe(\n test_df, \n x_col='filename',\n y_col=None,\n class_mode=None,\n target_size=(img_height, img_width),\n batch_size=batch_size,\n shuffle=False)","metadata":{"execution":{"iopub.status.busy":"2023-01-07T11:35:14.590960Z","iopub.execute_input":"2023-01-07T11:35:14.591410Z","iopub.status.idle":"2023-01-07T11:35:14.779079Z","shell.execute_reply.started":"2023-01-07T11:35:14.591366Z","shell.execute_reply":"2023-01-07T11:35:14.777367Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def visualize_predictions(test_generator, model):\n plt.figure(figsize=(12, 12))\n for i in range(0, 15):\n plt.subplot(5, 3, i+1)\n for X_batch in test_generator:\n prediction = model.predict(X_batch)[0]\n image = X_batch[0]\n plt.imshow(image)\n plt.title('cat' if np.argmax(prediction)==0 else \"dog\")\n break\n plt.tight_layout()\n plt.show()\nvisualize_predictions(test_generator, model)","metadata":{"execution":{"iopub.status.busy":"2023-01-07T11:35:02.278947Z","iopub.execute_input":"2023-01-07T11:35:02.279793Z","iopub.status.idle":"2023-01-07T11:35:02.305831Z","shell.execute_reply.started":"2023-01-07T11:35:02.279751Z","shell.execute_reply":"2023-01-07T11:35:02.304438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Transfer Learning\n\nIf you made it here, we'll talk about deep learning's most important tricks- transfer learning!\n\nNeural networks are notoriously data hungry- they can eat millions of images and digest them to be able to generalize upon their features. In this case, what if you don't have millions of images?\n\nIn this case, you use a model that has been trained on millions of images. And take it as your starting point. And train your model from them. Those massive datasets don't necessarily have to be related to your image classes.\n\nThere are many publicly available models like resnet, xception, convnext (particular architectures of CNNs) trained on ImageNet dataset (a very large image dataset with 100+ different classes). You can simply download them, and use it for your task (classifying dogs), and it will work much better than defining a model from new.\n\nWe'll implement a model using transfer learning below.","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.applications import ResNet50, Xception\nfrom tensorflow.keras import Input\nfrom tensorflow.keras.models import Model\n\n# ResNet50 is our \"backbone\" of sorts, a CNN architecture \n# pretrained on the imagenet dataset\n# we are only taking the CNN portion of it (include_top = False)\n# and dropping the dense layer, we'll initialize a dense network of our own\n\nbasemodel = Xception(include_top = False, \n weights = 'imagenet',\n input_shape=(img_height, img_width, 3))\n\nbasemodel.trainable = False\n\n# use the output of the baseModel to create a \"head\"\nheadModel = basemodel.output\nheadModel = MaxPooling2D(pool_size=(5, 5))(headModel)\nheadModel = Flatten(name=\"flatten\")(headModel)\nheadModel = Dense(128, activation=\"relu\")(headModel)\nheadModel = Dropout(0.8)(headModel)\n# headModel = Dense(32, activation=\"relu\")(headModel)\n# headModel = Dropout(0.5)(headModel)\nheadModel = Dense(2, activation=\"softmax\")(headModel)\n# at the end, we'll have two neurons, for two of the classes\n\n# we're \"disabling\" the backbone, and only training the head for this task\n# we're assuming that the backbone is already sufficiently trained to generate\n# features from images like ours.\n# we can also \"disable\" all CNN layers except last 4\n\n# create a model object\nmodel = Model(inputs=basemodel.input, outputs=headModel)","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:26:35.911979Z","iopub.execute_input":"2023-09-30T06:26:35.912341Z","iopub.status.idle":"2023-09-30T06:26:36.950063Z","shell.execute_reply.started":"2023-09-30T06:26:35.912313Z","shell.execute_reply":"2023-09-30T06:26:36.949105Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot_model(model, show_shapes=True, show_layer_names=False, dpi=60)","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:26:40.977701Z","iopub.execute_input":"2023-09-30T06:26:40.978037Z","iopub.status.idle":"2023-09-30T06:26:40.982836Z","shell.execute_reply.started":"2023-09-30T06:26:40.978008Z","shell.execute_reply":"2023-09-30T06:26:40.981916Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Experiment Tracking\n\nIn earlier case, we plotted our losses and metrics after the training was done, manually using matplotlib. But there are tools available using which we can observe real-time how our training is progressing. They also log system metrics like GPU usage, and can keep track of multiple experiments, hyperparameters etc. One such tool is wandb.ai, using which you can track your model even on phone as it's running in background.\n\nUsing it is very simple, signup on [wandb.ai](wandb.ai), and add only few lines of code. Get the API token, and go through [this discussion](https://www.kaggle.com/general/209530) on how to add it as a kaggle secret.\n","metadata":{}},{"cell_type":"code","source":"use_wandb = False # set to false if you don't want to use wandb for tracking\nif use_wandb:\n from kaggle_secrets import UserSecretsClient\n import wandb\n from wandb.keras import WandbCallback\n user_secrets = UserSecretsClient()\n wandb_api = user_secrets.get_secret(\"wandb_api\")\n wandb.login(key=wandb_api)","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:26:40.985076Z","iopub.execute_input":"2023-09-30T06:26:40.985638Z","iopub.status.idle":"2023-09-30T06:26:41.002700Z","shell.execute_reply.started":"2023-09-30T06:26:40.985604Z","shell.execute_reply":"2023-09-30T06:26:41.001766Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if use_wandb:\n wandb.init(project=\"keras_cats_and_dogs\", config={\"batch_size\": batch_size})","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:26:41.004192Z","iopub.execute_input":"2023-09-30T06:26:41.004570Z","iopub.status.idle":"2023-09-30T06:26:41.014346Z","shell.execute_reply.started":"2023-09-30T06:26:41.004538Z","shell.execute_reply":"2023-09-30T06:26:41.013443Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.callbacks import EarlyStopping\nopt = Adam(learning_rate=0.001)\nmodel.compile(loss='categorical_crossentropy', \n optimizer=opt, \n metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:26:41.326887Z","iopub.execute_input":"2023-09-30T06:26:41.327216Z","iopub.status.idle":"2023-09-30T06:26:41.342602Z","shell.execute_reply.started":"2023-09-30T06:26:41.327189Z","shell.execute_reply":"2023-09-30T06:26:41.341676Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"epochs=5\ncallbacks=[EarlyStopping(monitor='loss', patience=2), ]\nif use_wandb: callbacks.append(WandbCallback())\n\nhistory = model.fit_generator(\n train_generator,\n epochs=epochs,\n validation_data=validation_generator,\n callbacks=callbacks,\n)","metadata":{"execution":{"iopub.status.busy":"2023-09-30T06:26:45.966900Z","iopub.execute_input":"2023-09-30T06:26:45.967265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Massive Improvement Using Transfer Learning\n\nWhen we defined a custom model earlier, the best validation accuracy we got after 10 epochs was **85%**.\n\nHere, by using a pre-trained model, our validation accuracy after 1 epoch is as high as **98%**!\n\nAs we can see, using a pre-trained model can really boost our performance with minimal training efforts. ","metadata":{}},{"cell_type":"code","source":"plot_loss(history)","metadata":{"execution":{"iopub.status.busy":"2023-01-07T09:20:50.734641Z","iopub.status.idle":"2023-01-07T09:20:50.736773Z","shell.execute_reply.started":"2023-01-07T09:20:50.736542Z","shell.execute_reply":"2023-01-07T09:20:50.736566Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"visualize_predictions(test_generator, model)","metadata":{"execution":{"iopub.status.busy":"2023-01-07T11:35:18.100638Z","iopub.execute_input":"2023-01-07T11:35:18.101788Z","iopub.status.idle":"2023-01-07T11:35:28.491808Z","shell.execute_reply.started":"2023-01-07T11:35:18.101718Z","shell.execute_reply":"2023-01-07T11:35:28.487710Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save(\"cats_and_dogs.h5\")","metadata":{},"execution_count":null,"outputs":[]}]}