{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

Dogs & Cats Images

\n\n#

Classifier Dogs๐Ÿถ & Cats๐Ÿ˜บ (val_acc=+98.5%)

\n\n

Thanks for visiting my notebook ๐ŸŒน

\n\n***\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12053762%2F3c94ab343f0e44a43b6f3563a7116373%2FXUGEPVQJJMI6LBP3WAQRGYECEI.jpg?generation=1695786557088668&alt=media)\n\n\n

Hello everyone!

\n\n
\n\n

๐Ÿ“Œ Feel free to fork or edit the notebook for your own convenience. If you liked the notebook, consider upvoting. It helps other people discover the notebook as well. Your support inspires me to produce more of these kernel.๐Ÿ˜Š\n\n

\n
\n\n\n\n\n\n\n\n
\n ๐Ÿ“Œ Don't forget to upvote if you liked the notebook ๐Ÿ˜Š๐Ÿ‘๐Ÿ”ผ
\n
\n
\n","metadata":{}},{"cell_type":"markdown","source":"
\n

Table of Contents

\n \n
\n","metadata":{}},{"cell_type":"markdown","source":"\n

\n Overview\n

\n\n# 1๏ธโƒฃ. Overview\n\n\n\n\nIn this notebook we will make a **classifier** for **dogs-&-cats** datasets according to the following steps:\n\n- We will pre-process the data using **Generator** way, and we will explain why we used it\n- Then we will explain both ideas **Early_stopping** and **Reduction_learning_rate** and why we used them\n- After that, we explained in detail the importance of the idea of **Exponential Decay** and how to implement it\n* Compare between**TL Models** and choose the **pest one** \n- Then we chose **Xception** to train the model on it \n- Finally, we built the neural network, trained it, and checked its accuracy","metadata":{}},{"cell_type":"markdown","source":"\n

\n Import Libraries\n

\n\n# 2๏ธโƒฃ. Import Libraries","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport os\nimport cv2\nimport glob as gb\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.image as mpimg\nfrom tqdm import tqdm\nfrom PIL import Image\n\nimport tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\n\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.callbacks import ReduceLROnPlateau\nfrom keras.callbacks import EarlyStopping\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import classification_report, confusion_matrix\nfrom keras.optimizers import Adam\n\nfrom keras.applications import DenseNet121\nfrom keras.applications import ResNet50V2\nfrom keras.applications import ResNet152V2\nfrom keras.applications import InceptionV3\nfrom keras.applications import Xception","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-11-13T09:41:28.524782Z","iopub.execute_input":"2023-11-13T09:41:28.525186Z","iopub.status.idle":"2023-11-13T09:41:28.536147Z","shell.execute_reply.started":"2023-11-13T09:41:28.525155Z","shell.execute_reply":"2023-11-13T09:41:28.534979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

\n Preprocessing\n

\n\n# 3๏ธโƒฃ.Preprocessing","metadata":{}},{"cell_type":"markdown","source":"### 3.1 Get the pathes","metadata":{}},{"cell_type":"code","source":"train_path = '../input/dogs-cats-images/dog vs cat/dataset/training_set'\ntest_path = '../input/dogs-cats-images/dog vs cat/dataset/test_set'","metadata":{"execution":{"iopub.status.busy":"2023-11-13T08:34:44.110758Z","iopub.execute_input":"2023-11-13T08:34:44.111165Z","iopub.status.idle":"2023-11-13T08:34:44.115935Z","shell.execute_reply.started":"2023-11-13T08:34:44.111137Z","shell.execute_reply":"2023-11-13T08:34:44.114743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 3.2 Load the Images with a generator and Data Augmentation\n","metadata":{}},{"cell_type":"markdown","source":" ### Why you should use (flow_from_directory) for Visualize Images ?\n\n\n Using `flow_from_directory` in conjunction with `ImageDataGenerator` offers several advantages when working with image datasets in deep learning tasks. Here are some reasons why you might want to use `flow_from_directory`:\n\n1. **Directory Structure and Labels**: If your data is organized in a directory structure where each subdirectory corresponds to a class, `flow_from_directory` can automatically assign labels to images based on the directory names. This eliminates the need for you to manually assign labels to each image.\n\n2. **Data Augmentation**: `ImageDataGenerator` allows you to apply data augmentation techniques such as rotation, flipping, and scaling to your images on-the-fly while they are being loaded. This helps improve the generalization of your model and prevents overfitting.\n\n3. **Batch Loading**: Loading images in batches helps manage memory efficiently, **especially when working with large datasets that may not fit into memory**. `flow_from_directory`loads images in batches, allowing you to train your model with minimal memory usage.\n\n4. **Preprocessing**: `ImageDataGenerator` can apply preprocessing steps such as rescaling pixel values and standardizing input data on-the-fly. This ensures that your input data is properly preprocessed before being fed into the model.\n\n5. **Easy Integration with Keras Models**: The data generator returned by `flow_from_directory` can be directly used as input for Keras' `fit` and `evaluate` methods. This streamlines the training and evaluation process, making it easier to manage data loading and preprocessing.\n\n6. **Shuffling**: By setting the `shuffle` parameter to `True` in `flow_from_directory`, you can shuffle the order of images within each epoch, preventing the model from memorizing the order of training examples.\n\n7. **Efficient Coding**: Using `flow_from_directory` reduces the amount of manual coding required to load and preprocess images, freeing you to focus more on building and training your models.","metadata":{}},{"cell_type":"code","source":"# Data augmentation for training\ntrain_datagen = ImageDataGenerator(\n rescale=1.0/255, # Rescale pixel values to [0, 1]\n rotation_range=20, # Random rotation within 20 degrees\n width_shift_range=0.2, # Random horizontal shift by 20% of image width\n height_shift_range=0.2, # Random vertical shift by 20% of image height\n horizontal_flip=True, # Random horizontal flipping\n fill_mode='nearest' # Fill mode for new pixels after shifts/rotations\n)","metadata":{"execution":{"iopub.status.busy":"2023-11-13T08:34:48.318187Z","iopub.execute_input":"2023-11-13T08:34:48.318572Z","iopub.status.idle":"2023-11-13T08:34:48.323493Z","shell.execute_reply.started":"2023-11-13T08:34:48.318542Z","shell.execute_reply":"2023-11-13T08:34:48.322633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"images_size = 200\nbatch_size = 32","metadata":{"execution":{"iopub.status.busy":"2023-11-13T08:37:48.892183Z","iopub.execute_input":"2023-11-13T08:37:48.892687Z","iopub.status.idle":"2023-11-13T08:37:48.898603Z","shell.execute_reply.started":"2023-11-13T08:37:48.892653Z","shell.execute_reply":"2023-11-13T08:37:48.897448Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_generator = train_datagen.flow_from_directory(\n train_path, # Path to the training data\n target_size=(images_size, images_size), # Resize images to this size\n batch_size=batch_size, # Number of images in each batch\n seed=32, # Optional: Set a random seed for shuffling\n shuffle=True, # Shuffle the data during training\n class_mode='categorical' # Mode for class labels (categorical for one-hot encoding)\n)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T08:37:52.037861Z","iopub.execute_input":"2023-11-13T08:37:52.038281Z","iopub.status.idle":"2023-11-13T08:37:52.428406Z","shell.execute_reply.started":"2023-11-13T08:37:52.038250Z","shell.execute_reply":"2023-11-13T08:37:52.427443Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get the class labels\nclass_labels = list(train_generator.class_indices.keys())\n\n# Calculate the count of images per class\nclass_counts = {label: 0 for label in class_labels}\n\nfor i in range(len(train_generator)):\n batch_data, batch_labels = train_generator[i]\n for j in range(len(batch_data)):\n class_idx = int(batch_labels[j].argmax())\n class_label = class_labels[class_idx]\n class_counts[class_label] += 1\n\n# Define unique colors for each class\nclass_colors = plt.cm.tab20(np.linspace(0, 1, len(class_labels)))\n\n# Create a bar chart with different colors for each class\nplt.figure(figsize=(8, 4))\nbars = plt.bar(class_counts.keys(), class_counts.values(), color=class_colors)\nplt.xlabel('Class')\nplt.ylabel('Count')\nplt.title('Number of Images per Class')\nplt.xticks(rotation=45)\n\n# Add a legend for class colors\nlegend_labels = [plt.Line2D([0], [0], color=class_colors[i], lw=4, label=class_labels[i]) for i in range(len(class_labels))]\nplt.legend(handles=legend_labels, title=\"Classes\")\n\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T08:41:07.789868Z","iopub.execute_input":"2023-11-13T08:41:07.790512Z","iopub.status.idle":"2023-11-13T08:42:43.859079Z","shell.execute_reply.started":"2023-11-13T08:41:07.790479Z","shell.execute_reply":"2023-11-13T08:42:43.858107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# data augmentation for testing\ntest_datagen = ImageDataGenerator(rescale=1.0/255) # Rescale pixel values to [0, 1]","metadata":{"execution":{"iopub.status.busy":"2023-11-13T08:40:32.213747Z","iopub.execute_input":"2023-11-13T08:40:32.214704Z","iopub.status.idle":"2023-11-13T08:40:32.219199Z","shell.execute_reply.started":"2023-11-13T08:40:32.214672Z","shell.execute_reply":"2023-11-13T08:40:32.218183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create a generator for testing data\ntest_generator = test_datagen.flow_from_directory(\n test_path,\n target_size=(images_size, images_size),\n batch_size = batch_size,\n class_mode='categorical')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T08:40:39.508600Z","iopub.execute_input":"2023-11-13T08:40:39.508979Z","iopub.status.idle":"2023-11-13T08:40:39.636726Z","shell.execute_reply.started":"2023-11-13T08:40:39.508950Z","shell.execute_reply":"2023-11-13T08:40:39.635869Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get the class labels\nclass_labels = list(test_generator.class_indices.keys())\n\n# Calculate the count of images per class\nclass_counts = {label: 0 for label in class_labels}\n\nfor i in range(len(test_generator)):\n batch_data, batch_labels = test_generator[i]\n for j in range(len(batch_data)):\n class_idx = int(batch_labels[j].argmax())\n class_label = class_labels[class_idx]\n class_counts[class_label] += 1\n\n# Define unique colors for each class\nclass_colors = plt.cm.tab20(np.linspace(0, 1, len(class_labels)))\n\n# Create a bar chart with different colors for each class\nplt.figure(figsize=(8, 4))\nbars = plt.bar(class_counts.keys(), class_counts.values(), color=class_colors)\nplt.xlabel('Class')\nplt.ylabel('Count')\nplt.title('Number of Images per Class')\nplt.xticks(rotation=45)\n\n# Add a legend for class colors\nlegend_labels = [plt.Line2D([0], [0], color=class_colors[i], lw=4, label=class_labels[i]) for i in range(len(class_labels))]\nplt.legend(handles=legend_labels, title=\"Classes\")\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-09-21T05:24:25.669279Z","iopub.execute_input":"2023-09-21T05:24:25.669582Z","iopub.status.idle":"2023-09-21T05:24:31.271426Z","shell.execute_reply.started":"2023-09-21T05:24:25.669557Z","shell.execute_reply":"2023-09-21T05:24:31.270445Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 3.3 Display Images","metadata":{}},{"cell_type":"code","source":"def Show_Images(target_gen):\n # Get a batch of images and labels\n batch_images, batch_labels = next(target_gen)\n\n # Get class labels\n class_labels = list(target_gen.class_indices.keys())\n\n # Display images with labels\n plt.figure(figsize=(20, 20))\n for n , i in enumerate(list(np.random.randint(0,len(batch_images),16))):\n plt.subplot(4, 4, n + 1)\n plt.imshow(batch_images[i])\n plt.title(class_labels[np.argmax(batch_labels[i])]) # Display the class label\n plt.axis('off')\n plt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T08:45:02.427587Z","iopub.execute_input":"2023-11-13T08:45:02.427976Z","iopub.status.idle":"2023-11-13T08:45:02.435330Z","shell.execute_reply.started":"2023-11-13T08:45:02.427948Z","shell.execute_reply":"2023-11-13T08:45:02.434332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Show_Images(train_generator)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T08:45:05.698751Z","iopub.execute_input":"2023-11-13T08:45:05.699503Z","iopub.status.idle":"2023-11-13T08:45:08.437547Z","shell.execute_reply.started":"2023-11-13T08:45:05.699465Z","shell.execute_reply":"2023-11-13T08:45:08.436244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Show_Images(test_generator)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 3.4 EarlyStopping\n***Early stopping*** works by monitoring the performance of the model on a validation set during training. The validation set is a set of data that is held out from the training data and is not used to train the model. The model's performance on the validation set is **used to determine if the model is overfitting.**\n\nIf the model's performance on the validation set starts to plateau or decrease, then the training is stopped early. **This prevents the model from continuing to fit the training data too closely**, which can lead to overfitting.\n\n๐Ÿ’ก Here is a simple analogy to help you understand early stopping:\n\nImagine you are trying to learn how to play a new game. You start by playing the game against a computer opponent. As you play, you get better and better at the game. However, at some point, you start to plateau. You are not getting any better at the game, even though you are still playing against the same computer opponent.\n\n\n","metadata":{}},{"cell_type":"code","source":"early_stopping = EarlyStopping(\n min_delta=0.001, # minimium amount of change to count as an improvement\n patience=5, # how many epochs to wait before stopping\n restore_best_weights=True,\n)","metadata":{"execution":{"iopub.status.busy":"2023-11-13T08:45:45.454903Z","iopub.execute_input":"2023-11-13T08:45:45.455902Z","iopub.status.idle":"2023-11-13T08:45:45.460254Z","shell.execute_reply.started":"2023-11-13T08:45:45.455867Z","shell.execute_reply":"2023-11-13T08:45:45.459273Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 3.5 Reduction learning_rate\n\nReduction of the learning rate means decreasing the step size (learning rate) during training to help a machine learning model converge more effectively and avoid convergence issues.","metadata":{}},{"cell_type":"code","source":"# Configure the learning rate reduction callback\nlearning_rate_reduce = ReduceLROnPlateau(\n monitor='val_acc', # Metric to monitor for changes (usually validation accuracy)\n patience=5, # Number of epochs with no improvement after which learning rate will be reduced\n verbose=1, # Verbosity mode (0: silent, 1: update messages)\n factor=0.5, # Factor by which the learning rate will be reduced (e.g., 0.5 means halving)\n min_lr=0.00001 # Lower bound for the learning rate (it won't go below this value)\n)","metadata":{"execution":{"iopub.status.busy":"2023-11-13T08:45:48.308757Z","iopub.execute_input":"2023-11-13T08:45:48.309445Z","iopub.status.idle":"2023-11-13T08:45:48.314316Z","shell.execute_reply.started":"2023-11-13T08:45:48.309412Z","shell.execute_reply":"2023-11-13T08:45:48.313429Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

\n Exponential Decay\n

\n\n# 4๏ธโƒฃ.Exponential Decay\n\n๐Ÿ“• ***ExponentialDecay*** is a learning rate schedule that reduces the learning rate exponentially over time. This is done by multiplying the learning rate by a factor of less than 1, such as 0.5 or 0.9, after a certain number of epochs or training steps.\n\n- The formula for exponential decay can be expressed as:\n\n## ***New Value = Initial Value ร— Decay Rate ^(Step Count)***\n\n\n For example, if you start with an initial learning rate of `0.1` and use an exponential decay rate of `0.9`, the learning rate might evolve as follows:\n\n- After 1 step: \\(0.1 \\times 0.9^1 = 0.09\\)\n- After 2 steps: \\(0.1 \\times 0.9^2 = 0.081\\)\n- After 3 steps: \\(0.1 \\times 0.9^3 = 0.0729\\)\n- ...\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12053762%2F33e63b666e59d83dc3f11e46f4d04430%2F640px-Plot-exponential-decay.svg.png?generation=1695445033268258&alt=media)\n\n๐Ÿ“˜ `ExponentialDecay` is a common and effective learning rate schedule **for image classification**. It is often used in conjunction with other techniques, such as momentum and weight decay, **to improve the performance of deep learning models.**\n\nโœ”๏ธ Here are some of the **benefits** of using `ExponentialDecay`:\n\n* It can help to **prevent overfitting** by reducing the learning rate over time.\n* It can help the model to converge **more quickly and smoothly.**\n* It can help to **improve the fine-tuning** performance of the model.\n\nโŒ However, there are also some **potential drawbacks** to using` ExponentialDecay`:\n\n* It can be **difficult to choose the right initial learning rate and decay rate.**\n* It can be **slow to converge in the early stages of training.**\n\n๐Ÿ’ก ***Overall***, `ExponentialDecay` is a powerful learning rate schedule that **can be used to improve the performance of deep learning models**. However, **it is important to carefully consider the benefits and drawbacks before using it.**","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras import optimizers\n\n# Create a learning rate schedule using Exponential Decay\nlearning_rate_schedule = keras.optimizers.schedules.ExponentialDecay(\n initial_learning_rate=0.01, # Initial learning rate for training\n decay_steps=1000, # Number of steps before decaying the learning rate\n decay_rate=0.5, # Rate at which the learning rate decreases\n)\n\n# learning_rate = initial_learning_rate * decay_rate ^ (global_step / decay_steps)\n","metadata":{"execution":{"iopub.status.busy":"2023-11-13T08:45:53.211001Z","iopub.execute_input":"2023-11-13T08:45:53.211639Z","iopub.status.idle":"2023-11-13T08:45:53.218121Z","shell.execute_reply.started":"2023-11-13T08:45:53.211606Z","shell.execute_reply":"2023-11-13T08:45:53.217284Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from keras.callbacks import LearningRateScheduler\n\n# Create a Learning Rate Scheduler callback using a pre-defined schedule\nlr_callback = LearningRateScheduler(learning_rate_schedule)\ncallback=[ lr_callback , learning_rate_reduce ,early_stopping ]","metadata":{"execution":{"iopub.status.busy":"2023-11-13T08:45:55.522411Z","iopub.execute_input":"2023-11-13T08:45:55.522810Z","iopub.status.idle":"2023-11-13T08:45:55.527343Z","shell.execute_reply.started":"2023-11-13T08:45:55.522783Z","shell.execute_reply":"2023-11-13T08:45:55.526446Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

\n Compare between TL Models\n

\n\n# 5๏ธโƒฃ Compare between TL Models\n
๐Ÿ”ด In this section, we aim to compare different TL Models architectures to determine which one performs best on our dataset.\n we will first collect a small subset of the data and train a basic model architecture on it. \n then we will analyze the performance of these TL Models and choose the best one โœ….\n
","metadata":{}},{"cell_type":"code","source":"# Collect all TL models\nTL_Models =[\n ResNet50V2(input_shape=(images_size, images_size, 3), weights='imagenet', include_top=False),\n ResNet152V2(input_shape=(images_size, images_size, 3), weights='imagenet', include_top=False),\n InceptionV3(input_shape=(images_size, images_size, 3), weights='imagenet', include_top=False),\n Xception(input_shape=(images_size, images_size, 3), weights='imagenet', include_top=False),\n DenseNet121( input_shape=(images_size, images_size, 3) ,weights='imagenet', include_top=False),\n]\n\n# Define all the TL models names. This will be later used during visualization\nTL_Models_NAMES = [\n 'ResNet50V2',\n 'ResNet152V2',\n 'InceptionV3',\n 'Xception',\n 'DenseNet121',\n]\n\n# Freeze the weights of all the TL models\nfor tl_model in TL_Models:\n tl_model.trainable = False","metadata":{"execution":{"iopub.status.busy":"2023-11-13T06:53:22.247077Z","iopub.execute_input":"2023-11-13T06:53:22.247450Z","iopub.status.idle":"2023-11-13T06:53:45.896036Z","shell.execute_reply.started":"2023-11-13T06:53:22.247420Z","shell.execute_reply":"2023-11-13T06:53:45.894991Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 5.1 Get subset from training data","metadata":{}},{"cell_type":"code","source":"# Create variables to collect images and labels\nsubset_images = []\nsubset_labels = []\n\n# Set the desired number of images you want in the subset\ndesired_count = 1000\n\n# Loop through the generator until you have collected the desired number of images\nfor i in range(desired_count):\n # Generate a batch of data (images and labels)\n batch = train_generator.next()\n images, labels = batch\n subset_images.extend(images)\n subset_labels.extend(labels)\n\n # Check if you've collected enough images\n if len(subset_images) >= desired_count:\n break\n\n# Convert the collected images and labels to numpy arrays\nX_sub = np.array(subset_images)\ny_sub = np.array(subset_labels)","metadata":{"execution":{"iopub.status.busy":"2023-11-13T07:15:20.511028Z","iopub.execute_input":"2023-11-13T07:15:20.511702Z","iopub.status.idle":"2023-11-13T07:15:31.910038Z","shell.execute_reply.started":"2023-11-13T07:15:20.511668Z","shell.execute_reply":"2023-11-13T07:15:31.909080Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 5.2 Training the Models and get Histories ","metadata":{}},{"cell_type":"code","source":"# Initialize an empty list to hold the histories of each TL_models architecture.\nHISTORIES = []\n\n# Loop over every backbone in the BACKBONES list.\nfor tl_model in tqdm(TL_Models, desc=\"Training Tl Models\"):\n \n # Create the simplest model architecture using the current backbone.\n model = keras.Sequential([\n tl_model,\n layers.GlobalAveragePooling2D(),\n layers.Dropout(0.5),\n layers.Dense(2, activation='softmax')\n ])\n \n # Compile the model with the specified loss function, optimizer, and metrics.\n model.compile(\n loss='categorical_crossentropy',\n optimizer=Adam(learning_rate = learning_rate_schedule),\n metrics='accuracy'\n )\n \n # Train the model on a subset of the training data.\n history = model.fit(\n X_sub, y_sub,\n epochs=10,\n validation_split=0.2,\n batch_size=batch_size\n )\n \n # Store the history of the trained model.\n HISTORIES.append(history.history)","metadata":{"execution":{"iopub.status.busy":"2023-11-13T07:15:38.461167Z","iopub.execute_input":"2023-11-13T07:15:38.462070Z","iopub.status.idle":"2023-11-13T07:19:49.514873Z","shell.execute_reply.started":"2023-11-13T07:15:38.462033Z","shell.execute_reply":"2023-11-13T07:19:49.513835Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 5.3 Analyze the performance of TL Models","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# Define the metrics to plot (e.g., 'accuracy' and 'loss')\nmetrics_to_plot = ['accuracy', 'loss']\n\n# Create subplots for each metric\nfig, axes = plt.subplots(len(metrics_to_plot), 1, figsize=(10, 6))\n\nfor i, metric in enumerate(metrics_to_plot):\n plt.subplot(2, 1, i+1) # Create subplots for each metric\n for idx, history in enumerate(HISTORIES):\n plt.plot(history[metric], label=TL_Models_NAMES[idx])\n plt.title(f'Training and Validation {metric}')\n plt.xlabel('Epochs')\n plt.ylabel(metric)\n plt.legend()\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-11-13T07:20:34.111751Z","iopub.execute_input":"2023-11-13T07:20:34.112670Z","iopub.status.idle":"2023-11-13T07:20:34.841455Z","shell.execute_reply.started":"2023-11-13T07:20:34.112634Z","shell.execute_reply":"2023-11-13T07:20:34.840472Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Note โ—\n
\nAll training models are similar in performance, but there is a red line that is better than the rest, and it points to Xception. \nSo we will select Xception for traning the model\n
","metadata":{}},{"cell_type":"markdown","source":"\n

\n Transfer Learning\n

","metadata":{}},{"cell_type":"markdown","source":"# 6๏ธโƒฃ Transfer Learning","metadata":{}},{"cell_type":"markdown","source":"### 6.1 Get the weights from Xception","metadata":{}},{"cell_type":"code","source":"base_model = Xception(weights='imagenet', include_top=False, input_shape=(images_size, images_size, 3))","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-11-13T09:10:45.008357Z","iopub.execute_input":"2023-11-13T09:10:45.008867Z","iopub.status.idle":"2023-11-13T09:10:47.032844Z","shell.execute_reply.started":"2023-11-13T09:10:45.008832Z","shell.execute_reply":"2023-11-13T09:10:47.032028Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 6.2 Freeze the layers of Xception","metadata":{}},{"cell_type":"code","source":"# Freeze the layers of the base model\nfor layer in base_model.layers:\n layer.trainable = False","metadata":{"execution":{"iopub.status.busy":"2023-11-13T09:10:53.121533Z","iopub.execute_input":"2023-11-13T09:10:53.122202Z","iopub.status.idle":"2023-11-13T09:10:53.132163Z","shell.execute_reply.started":"2023-11-13T09:10:53.122170Z","shell.execute_reply":"2023-11-13T09:10:53.131351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

\n Build the model\n

\n\n# 7๏ธโƒฃ.Build the model","metadata":{}},{"cell_type":"code","source":"model = tf.keras.models.Sequential([\n base_model,\n \n layers.Flatten(),\n \n layers.Dense(256,activation='relu'),\n layers.Dropout(0.5),\n layers.Dense(2,activation='softmax'),\n])\n\nmodel.summary()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T09:11:06.337818Z","iopub.execute_input":"2023-11-13T09:11:06.338181Z","iopub.status.idle":"2023-11-13T09:11:06.842185Z","shell.execute_reply.started":"2023-11-13T09:11:06.338152Z","shell.execute_reply":"2023-11-13T09:11:06.841280Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 7.1 Set the optimizer","metadata":{}},{"cell_type":"code","source":"optimizer = optimizers.Adam(learning_rate=learning_rate_schedule)","metadata":{"execution":{"iopub.status.busy":"2023-11-13T09:11:36.211761Z","iopub.execute_input":"2023-11-13T09:11:36.212121Z","iopub.status.idle":"2023-11-13T09:11:36.219865Z","shell.execute_reply.started":"2023-11-13T09:11:36.212089Z","shell.execute_reply":"2023-11-13T09:11:36.218994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.compile(optimizer=optimizer,\n loss=\"categorical_crossentropy\",\n metrics=['accuracy']\n )","metadata":{"execution":{"iopub.status.busy":"2023-11-13T09:11:36.221722Z","iopub.execute_input":"2023-11-13T09:11:36.222059Z","iopub.status.idle":"2023-11-13T09:11:36.239176Z","shell.execute_reply.started":"2023-11-13T09:11:36.222028Z","shell.execute_reply":"2023-11-13T09:11:36.238301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 7.2 Training the model","metadata":{}},{"cell_type":"code","source":"history = model.fit(\n train_generator,\n steps_per_epoch=train_generator.samples // batch_size,\n epochs=20,\n validation_data=test_generator,\n validation_steps=test_generator.samples // batch_size,\n callbacks=[callback]\n)","metadata":{"execution":{"iopub.status.busy":"2023-11-13T09:11:36.240189Z","iopub.execute_input":"2023-11-13T09:11:36.240507Z","iopub.status.idle":"2023-11-13T09:32:41.110237Z","shell.execute_reply.started":"2023-11-13T09:11:36.240482Z","shell.execute_reply":"2023-11-13T09:32:41.109424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

\n Evaluate the model\n

\n\n# 8๏ธโƒฃ. Evaluate the model","metadata":{}},{"cell_type":"markdown","source":"### 8.1 Evaluate the validation","metadata":{}},{"cell_type":"code","source":"# Evaluate on test dataset\nscore = model.evaluate(test_generator, verbose=False)\nprint('Test loss:', score[0])\nprint('Test accuracy:', score[1])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T09:34:37.750476Z","iopub.execute_input":"2023-11-13T09:34:37.751210Z","iopub.status.idle":"2023-11-13T09:34:46.079866Z","shell.execute_reply.started":"2023-11-13T09:34:37.751177Z","shell.execute_reply":"2023-11-13T09:34:46.078937Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 8.2 Accuracy graph ","metadata":{}},{"cell_type":"code","source":"# Accuracy\nacc = history.history['accuracy']\nval_acc = history.history['val_accuracy']\nepochs = range(len(acc))\n\nplt.plot(epochs, acc, 'b', label='Training Accuracy')\nplt.plot(epochs, val_acc, 'r', label='Validation Accuracy')\nplt.title('Accuracy Graph')\nplt.legend()\nplt.figure()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T09:35:09.952046Z","iopub.execute_input":"2023-11-13T09:35:09.952493Z","iopub.status.idle":"2023-11-13T09:35:10.345175Z","shell.execute_reply.started":"2023-11-13T09:35:09.952458Z","shell.execute_reply":"2023-11-13T09:35:10.344119Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 8.3 Loss graph ","metadata":{}},{"cell_type":"code","source":"# Loss\nloss = history.history['loss']\nval_loss = history.history['val_loss']\nplt.plot(epochs, loss, 'b', label='Training Loss')\nplt.plot(epochs, val_loss, 'r', label='Validation Loss')\nplt.title('Loss Graph')\nplt.legend()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T09:35:10.347033Z","iopub.execute_input":"2023-11-13T09:35:10.347436Z","iopub.status.idle":"2023-11-13T09:35:10.633908Z","shell.execute_reply.started":"2023-11-13T09:35:10.347399Z","shell.execute_reply":"2023-11-13T09:35:10.632795Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 8.4 Confusion Matrix","metadata":{}},{"cell_type":"code","source":"# Confusion Matrix\nimport itertools\n\n# Make predictions on the test data\npredictions = model.predict(test_generator)\n\n# Convert probabilities to class labels\npredicted_labels = np.argmax(predictions, axis=1)\n\n# Get true labels from the test_generator\ntrue_labels = test_generator.classes\n\n# Calculate confusion matrix\ncm = confusion_matrix(true_labels, predicted_labels)\n\n# Define class names (assuming you have class_names defined)\nclass_names = test_generator.class_indices.keys()\n\n# Plot confusion matrix\ndef plot_confusion_matrix(cm, classes, normalize=False, title=\"Confusion Matrix\", cmap=plt.cm.Blues):\n plt.imshow(cm, interpolation=\"nearest\", cmap=cmap)\n plt.title(title)\n plt.colorbar()\n tick_marks = np.arange(len(classes))\n plt.xticks(tick_marks, classes, rotation=45)\n plt.yticks(tick_marks, classes)\n\n if normalize:\n cm = cm.astype(\"float\") / cm.sum(axis=1)[:, np.newaxis]\n\n thresh = cm.max() / 2.0\n for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n plt.text(j, i, cm[i, j], horizontalalignment=\"center\", color=\"white\" if cm[i, j] > thresh else \"black\")\n\n plt.tight_layout()\n plt.ylabel(\"True label\")\n plt.xlabel(\"Predicted label\")\n\n# Plot non-normalized confusion matrix\nplt.figure(figsize=(8, 6))\nplot_confusion_matrix(cm, classes=class_names, title=\"Confusion Matrix\")\nplt.show()\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T09:35:31.469751Z","iopub.execute_input":"2023-11-13T09:35:31.470620Z","iopub.status.idle":"2023-11-13T09:35:42.844290Z","shell.execute_reply.started":"2023-11-13T09:35:31.470585Z","shell.execute_reply":"2023-11-13T09:35:42.843277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 8.5 Generate predictions for the test data","metadata":{}},{"cell_type":"code","source":"# Collect true labels and model predictions\ntrue_labels = []\npredicted_labels = []\nclass_names = test_generator.class_indices.keys() \nclass_names = list(class_names)\nnum_batches = len(test_generator)\nfor i in range(10):\n x_batch, y_batch = test_generator[i]\n predictions = model.predict(x_batch)\n true_labels.extend(np.argmax(y_batch, axis=1)) # Convert one-hot encoded labels to class indices\n predicted_labels.extend(np.argmax(predictions, axis=1))\n\n\ntrue_class_labels = [class_names[i] for i in true_labels]\npredicted_class_labels = [class_names[i] for i in predicted_labels]\n\n# Plot true labels and predicted labels\nplt.figure(figsize=(10, 5))\nnum_samples_to_display = min(10, len(x_batch)) # Display up to 10 samples or less if available\nfor i in range(num_samples_to_display):\n plt.subplot(2, 5, i + 1)\n plt.imshow(x_batch[i])\n plt.title(f'True: {true_class_labels[i]}\\nPredicted: {predicted_class_labels[i]}', fontsize=8)\n plt.axis('off')\n\nplt.tight_layout()\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-13T09:38:41.755035Z","iopub.execute_input":"2023-11-13T09:38:41.755911Z","iopub.status.idle":"2023-11-13T09:38:46.364891Z","shell.execute_reply.started":"2023-11-13T09:38:41.755878Z","shell.execute_reply":"2023-11-13T09:38:46.363964Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

\n Conclusion\n

\n\n\n# ๐Ÿ˜Š. Conclusion\n#### I hope this code can help you, if you liked this notebook please press up votes, thanks โค\n
\n

\n Thanks for viewing my work. If you like it, consider sharing it to others or give feedback to improve the notebook. Have a beautiful day my friend โค.\n

\n
","metadata":{}}]}