\n \n \n* [Step 1 | Setup and Initialization](#Initialization)\n* [Step 2 | Dataset Analysis](#Analysis)\n* [Step 3 | Dataset Preparation](#Preparation)\n - [Step 3.1 | Centralizing Dataset Information](#Centralize_Dataset)\n - [Step 3.2 | Data Splitting into Training & Validation Sets](#Data_Splitting)\n - [Step 3.3 | Weather Simulation](#Weather_Simulation)\n - [Step 3.3.1 | Rain Effect Integration](#rain)\n - [Step 3.3.2 | Snow Effect Integration](#snow)\n - [Step 3.3.3 | Fog Effect Integration](#fog)\n - [Step 3.3.4 | Applying Random Weather Effects to Dataset](#random_effect)\n - [Step 3.4 | Data Augmentation & Preprocessing](#Augmentation)\n* [Step 4 | Model Architecture Development](#Model_Architecture)\n - [Step 4.1 | ResNet50V2 Architecture](#ResNet50V2_Architecture)\n - [Step 4.2 | InceptionV3 Architecture](#InceptionV3_Architecture)\n - [Step 4.3 | MobileNetV2 Architecture](#MobileNetV2_Architecture)\n - [Step 4.3.1 | MobileNetV2 with Neural Network Classifier](#NN_Classifier)\n - [Step 4.3.2 | MobileNetV2 with SVM Classifier](#SVM_Classifier)\n* [Step 5 | Classifier Training](#Classifier_Training)\n - [Step 5.1 | Training the ResNet50V2 Classifier](#ResNet50V2_Classifier)\n - [Step 5.2 | Training the InceptionV3 Classifier](#InceptionV3_Classifier)\n - [Step 5.3 | Training the Neural Network Classifier for MobileNetV2](#MobileNetV2_NN_Classifier)\n - [Step 5.4 | Training the SVM Classifier for MobileNetV2](#MobileNetV2_SVM_Classifier)\n - [Step 5.4.1 | Data Preparation](#5_4_1)\n - [Step 5.4.2 | Feature extraction](#5_4_2)\n - [Step 5.4.3 | Dimensionality reduction](#5_4_3)\n - [Step 5.4.4 | Train the SVM model](#5_4_4)\n* [Step 6 | Model Performance Assessment](#Model_Performance)\n - [Step 6.1 | Learning Curve Analysis](#Learning_Curve)\n - [Step 6.2 | Accuracy Assessment](#Accuracy_Assessment)\n - [Step 6.3 | Inference Time Comparison](#Inference_Time)\n - [Step 6.4 | Model Size and Deployment Considerations](#Model_Size)\n* [Step 7 | Final Model Selection and Justification](#Model_Selection)","metadata":{}},{"cell_type":"markdown","source":"
\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nFirst, I'll import the necessary libraries for our project:","metadata":{}},{"cell_type":"code","source":"# Install albumentations, a library to add weather effects and other transformations to images\n!pip install albumentations","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:21:42.189468Z","iopub.execute_input":"2024-02-14T07:21:42.190177Z","iopub.status.idle":"2024-02-14T07:21:54.393116Z","shell.execute_reply.started":"2024-02-14T07:21:42.190149Z","shell.execute_reply":"2024-02-14T07:21:54.392109Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Disable warnings in the notebook to maintain clean output cells\nimport warnings # Import the warnings module to handle warnings\nwarnings.filterwarnings('ignore') # Filter and ignore warnings to maintain clean output\n\n# Import necessary libraries\nimport numpy as np # Import numpy library for numerical operations\nimport pandas as pd # Import pandas library for data manipulation and analysis\nimport matplotlib.pyplot as plt # Import matplotlib library for data visualization\nimport seaborn as sns # Import seaborn library for statistical data visualization\nimport os # Import os module for interacting with the operating system\nimport cv2 # Import OpenCV library for computer vision tasks\nimport zipfile # Import zipfile module for working with zip files\nimport random # Import random module for generating random numbers and sequences\nimport albumentations # Import albumentations library for image augmentation\nimport subprocess # Import subprocess module for running shell commands\nfrom sklearn.model_selection import train_test_split # Import train_test_split function from sklearn for splitting dataset\nfrom keras.preprocessing.image import ImageDataGenerator # Import ImageDataGenerator from keras for image data augmentation\nfrom tensorflow.keras.applications import ResNet50V2 # Import ResNet50V2 model from keras for transfer learning\nfrom tensorflow.keras.applications.inception_v3 import InceptionV3 # Import InceptionV3 model from keras for transfer learning\nfrom tensorflow.keras.applications import MobileNetV2 # Import MobileNetV2 model from keras for transfer learning\nfrom tensorflow.keras.models import Model # Import Model class from keras for defining custom neural network architectures\nfrom tensorflow.keras.layers import Dense, Dropout, GlobalAveragePooling2D, Input # Import layers from keras for building neural networks\nfrom tensorflow.keras.optimizers import Adam # Import Adam optimizer from keras for model optimization\nfrom keras.utils import plot_model # Import plot_model function from keras for visualizing model architectures\nfrom tensorflow.keras.callbacks import ReduceLROnPlateau, EarlyStopping # Import callbacks from keras for adjusting learning rate and early stopping\nfrom tensorflow.keras.applications.resnet_v2 import preprocess_input as resnet_preprocess_input # Import preprocess_input function from keras for ResNetV2 models\nfrom tensorflow.keras.applications.inception_v3 import preprocess_input as inceptionv3_preprocess_input # Import preprocess_input function from keras for InceptionV3 models\nfrom tensorflow.keras.applications.mobilenet_v2 import preprocess_input as mobilenetv2_preprocess_input # Import preprocess_input function from keras for MobileNetV2 models\nfrom tensorflow.keras.models import load_model # Import load_model function from keras for loading pre-trained models\nfrom sklearn.svm import SVC # Import SVC (Support Vector Classifier) from sklearn for machine learning classification\nfrom sklearn.decomposition import PCA # Import PCA (Principal Component Analysis) from sklearn for dimensionality reduction\nfrom sklearn.metrics import accuracy_score # Import accuracy_score function from sklearn for evaluating classification accuracy\nfrom joblib import dump # Import dump function from joblib for saving trained models\nfrom IPython.display import FileLink, display # Import FileLink and display functions from IPython.display for displaying files\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:21:54.395202Z","iopub.execute_input":"2024-02-14T07:21:54.395526Z","iopub.status.idle":"2024-02-14T07:21:54.407113Z","shell.execute_reply.started":"2024-02-14T07:21:54.395498Z","shell.execute_reply":"2024-02-14T07:21:54.406192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Configure the visual appearance of Seaborn plots\nsns.set(rc={'axes.facecolor': 'gainsboro'}, style='darkgrid')","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:21:54.408293Z","iopub.execute_input":"2024-02-14T07:21:54.408646Z","iopub.status.idle":"2024-02-14T07:21:54.423515Z","shell.execute_reply.started":"2024-02-14T07:21:54.408612Z","shell.execute_reply":"2024-02-14T07:21:54.422721Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n#
Step 2 | Dataset Analysis
\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n \nFirst of all, I am going to do initial analysis to examine the distribution of image classes within the dataset:","metadata":{}},{"cell_type":"code","source":"# Specify the path to the zip file\nzip_path = '/kaggle/input/dogs-vs-cats/train.zip'\n\n# Specify the directory where we want to extract the contents of the zip file\nextract_path = '/kaggle/working/'\n\n# Extract the zip file\nwith zipfile.ZipFile(zip_path, 'r') as zip_ref:\n zip_ref.extractall(extract_path)\n\n# After extracting, specify the directory of the actual images\nactual_extracted_path = os.path.join(extract_path, 'train')\n\n# List all filenames in the actual extracted directory\nfilenames = os.listdir(actual_extracted_path)\n\n# Separate filenames into cats and dogs\ncat_images = [filename for filename in filenames if \"cat\" in filename]\ndog_images = [filename for filename in filenames if \"dog\" in filename]\n\n# Get the count of each class\ncat_count = len(cat_images)\ndog_count = len(dog_images)\n\n# Calculate the percentages\ntotal_images = cat_count + dog_count\ncat_percentage = (cat_count / total_images) * 100\ndog_percentage = (dog_count / total_images) * 100\n\n# Plotting\nlabels = ['Cats', 'Dogs']\ncounts = [cat_count, dog_count]\npercentages = [cat_percentage, dog_percentage]\n\n# Set the figure size\nplt.figure(figsize=(15, 4))\n\n# Create a horizontal bar plot\nax = sns.barplot(y=labels, x=counts, orient='h', color='#33312b')\n\n# Set x-axis interval\nax.set_xticks([0, 15000])\n\n# Annotate each bar with the count and percentage\nfor i, p in enumerate(ax.patches):\n width = p.get_width()\n ax.text(width + 100, p.get_y() + p.get_height()/2., \n '{:1.2f}% ({})'.format(percentages[i], counts[i]),\n va=\"center\", fontsize=15)\n \n# Set the x-label for the plot\nplt.xlabel('Number of Images', fontsize=14)\n\n# Set the title and show the plot\nplt.title(\"Number of images per class\", fontsize=18)\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:21:54.425561Z","iopub.execute_input":"2024-02-14T07:21:54.425876Z","iopub.status.idle":"2024-02-14T07:22:02.275822Z","shell.execute_reply.started":"2024-02-14T07:21:54.425847Z","shell.execute_reply":"2024-02-14T07:22:02.274901Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
🔍 Inference on Image Class Distributions
\n\nFrom the visual representation, we can observe that our dataset is perfectly balanced. Both cats and dogs have an equal number of images, specifically 12,500 each, constituting 50% of the dataset for each class. \n\nHaving a balanced dataset means we don't need to worry about one class being overrepresented, which could lead to biases in our machine learning model. This sets a solid foundation for training our models!","metadata":{}},{"cell_type":"markdown","source":"
\n \nNext, I'll examine the dimensions of the images:","metadata":{}},{"cell_type":"code","source":"# Lists to store heights and widths of all images\nheights = []\nwidths = []\n\n# Initialize a set to store unique dimensions and channels\nunique_dims = set()\nunique_channels = set()\n\n# Loop over filenames and check dimensions\nfor filename in filenames:\n img = cv2.imread(os.path.join(actual_extracted_path, filename))\n if img is not None:\n # Add the dimensions (height, width, channels) to the set\n unique_dims.add((img.shape[0], img.shape[1]))\n \n # Add the channels to the set\n unique_channels.add(img.shape[2])\n \n # Append heights and widths for statistical calculations\n heights.append(img.shape[0])\n widths.append(img.shape[1])\n\n# Check if all images have the same dimension\nif len(unique_dims) == 1:\n print(f\"All images have the same dimensions: {list(unique_dims)[0]}\")\nelse:\n print(f\"There are {len(unique_dims)} different image dimensions in the dataset.\")\n print(f\"Min height: {min(heights)}, Max height: {max(heights)}, Mean height: {np.mean(heights):.2f}\")\n print(f\"Min width: {min(widths)}, Max width: {max(widths)}, Mean width: {np.mean(widths):.2f}\")\n\n# Check if all images have the same number of channels\nif len(unique_channels) == 1:\n channel = list(unique_channels)[0]\n if channel == 3:\n print(\"All images are color images.\")\n else:\n print(\"All images have the same number of channels, but they are not color images.\")\nelse:\n print(\"Images have different numbers of channels.\")\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:22:02.277212Z","iopub.execute_input":"2024-02-14T07:22:02.277871Z","iopub.status.idle":"2024-02-14T07:22:45.264727Z","shell.execute_reply.started":"2024-02-14T07:22:02.277837Z","shell.execute_reply":"2024-02-14T07:22:45.263836Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
🔍 Inference on Image Dimensions
\n \nFrom the dataset, we observed that there are a variety of image sizes, with heights ranging from 32 to 768 pixels and widths from 42 to 1050 pixels. The average dimensions are roughly (360, 404). Thankfully, all images are color images.\n\nGiven the diversity in image sizes, we'll need to consider resizing images to a fixed size later on. A common choice for many standard CNN architectures is (224, 224, 3). This will ensure consistency and compatibility with our chosen model.\n","metadata":{}},{"cell_type":"markdown","source":"
\nLet's dive into a visual exploration of images from both classes:","metadata":{}},{"cell_type":"code","source":"# Setting the random seed for reproducibility\nnp.random.seed(42) # Set the random seed to ensure reproducibility of results\n\n# Randomly select 6 images from each category\nrandom_cat_images = np.random.choice(cat_images, 6) # Randomly select 6 cat images\nrandom_dog_images = np.random.choice(dog_images, 6) # Randomly select 6 dog images\n\n# Function to plot images\ndef plot_images(images, title):\n plt.figure(figsize=(14, 3)) # Create a new figure with a specified size\n for i, img_name in enumerate(images): # Iterate over the selected images\n plt.subplot(1, 6, i+1) # Create subplots in a row\n img = cv2.imread(os.path.join(actual_extracted_path, img_name)) # Read the image\n # Convert the BGR image (default in OpenCV) to RGB\n img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) \n plt.imshow(img) # Display the image\n plt.axis('off') # Turn off axis labels\n plt.suptitle(title, fontsize=20) # Add a centered title to the plot\n plt.show() # Show the plot\n\n# Plot the images\nplot_images(random_cat_images, \"Randomly Selected Cat Images\") # Plot randomly selected cat images\nplot_images(random_dog_images, \"Randomly Selected Dog Images\") # Plot randomly selected dog images\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:22:45.267162Z","iopub.execute_input":"2024-02-14T07:22:45.267874Z","iopub.status.idle":"2024-02-14T07:22:46.605368Z","shell.execute_reply.started":"2024-02-14T07:22:45.267848Z","shell.execute_reply":"2024-02-14T07:22:46.604512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
🔍 Inference on Image Scale
\n \nThe dataset images __vary in scale__, with some capturing subjects closely and others at a more distant perspective.","metadata":{}},{"cell_type":"code","source":"# Deleting unnecessary variables to free up memory\ndel filenames, heights, widths, unique_dims, unique_channels\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:22:46.606666Z","iopub.execute_input":"2024-02-14T07:22:46.607087Z","iopub.status.idle":"2024-02-14T07:22:46.613686Z","shell.execute_reply.started":"2024-02-14T07:22:46.607056Z","shell.execute_reply":"2024-02-14T07:22:46.612837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n#
Step 3 | Dataset Preparation
\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 3.1 | Centralizing Dataset Information\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nIn this step,I am going to centralize the dataset's information into a DataFrame. This makes it easier to manage and process the data in the subsequent steps:","metadata":{}},{"cell_type":"code","source":"# Initialize an empty list to store image file paths and their respective labels\ndata = []\n\n# Append the cat image file paths with label \"cat\" to the data list\ndata.extend([(os.path.join(actual_extracted_path, filename), \"cat\") for filename in cat_images])\n\n# Append the dog image file paths with label \"dog\" to the data list\ndata.extend([(os.path.join(actual_extracted_path, filename), \"dog\") for filename in dog_images])\n\n# Convert the collected data into a DataFrame\ndf = pd.DataFrame(data, columns=['filepath', 'label'])\n\n# Display the first few entries of the DataFrame\ndf.head()\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:22:46.614721Z","iopub.execute_input":"2024-02-14T07:22:46.614965Z","iopub.status.idle":"2024-02-14T07:22:46.684765Z","shell.execute_reply.started":"2024-02-14T07:22:46.614943Z","shell.execute_reply":"2024-02-14T07:22:46.683919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Deleting unnecessary variables to free up memory\ndel data, cat_images, dog_images","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:22:46.685979Z","iopub.execute_input":"2024-02-14T07:22:46.686252Z","iopub.status.idle":"2024-02-14T07:22:46.691518Z","shell.execute_reply.started":"2024-02-14T07:22:46.686229Z","shell.execute_reply":"2024-02-14T07:22:46.690531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n#
Step 3.2 | Data Splitting into Training & Validation Sets\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nThen, I am going to split our dataset into train and validation sets, ensuring a balanced class distribution with stratification and mixing the images for randomness using shuffling:","metadata":{}},{"cell_type":"code","source":"# Split the data into training and validation sets\ntrain_df, val_df = train_test_split(df, test_size=0.2, stratify=df['label'], random_state=42)\n\n# Display the shape of the training and validation sets\nprint(\"Training data shape:\", train_df.shape)\nprint(\"Validation data shape:\", val_df.shape)\n\n# Deleting the original DataFrame to free up memory\ndel df","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:23:50.603781Z","iopub.execute_input":"2024-02-14T07:23:50.604717Z","iopub.status.idle":"2024-02-14T07:23:50.659794Z","shell.execute_reply.started":"2024-02-14T07:23:50.604678Z","shell.execute_reply":"2024-02-14T07:23:50.658741Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Display the first few rows of the train DataFrame\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:23:59.484688Z","iopub.execute_input":"2024-02-14T07:23:59.485391Z","iopub.status.idle":"2024-02-14T07:23:59.494212Z","shell.execute_reply.started":"2024-02-14T07:23:59.485351Z","shell.execute_reply":"2024-02-14T07:23:59.493207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n#
Step 3.3 | Weather Simulation\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n \nOur goal is to build a weather-proof model, and as observed, our dataset images are all under normal conditions. To address this, I will define three separate functions to introduce __rain__, __snow__, and __fog__ effects to the images. The `albumentations` library will be instrumental in achieving this:","metadata":{}},{"cell_type":"markdown","source":"
\n##
Step 3.3.1 | Rain Effect Integration\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nIn this part, I am going to use `RandomRain` method of `albumentations` library to define a function for adding rain effect to the input image. The definition of this method's arguments are as follows:\n\n- **slant_lower and slant_upper**: These determine the angle range at which the rain falls. For instance, a value of `-10` and `10` means rain can slant between -10 to 10 degrees.\n \n \n- **drop_length**: The length of individual raindrops. A higher value results in longer raindrops.\n \n \n- **drop_width**: The width of individual raindrops. A higher value makes thicker raindrops.\n \n \n- **drop_color**: Color of the raindrops, given in RGB format. For example, `(200, 200, 200)` represents a light gray color.\n \n \n- **blur_value**: How blurry the rain appears. Lower values mean sharper raindrops, while higher values give a foggy appearance to the rain.\n \n \n- **brightness_coefficient**: Controls the brightness of the image after adding the rain. A value less than 1 darkens the image, and greater than 1 brightens it.\n \n \n- **rain_type**: Can be set to `'drizzle'`, `'light'`, `'heavy'`, or `None`. This sets the pre-defined styles of rain. If set to `None`, the method uses the provided arguments to generate the rain effect.\n \n \n- **p**: The probability of applying the rain augmentation to an image. A value of `1` ensures that the rain effect is always applied.\n\n \nRemember, adjusting these parameters carefully can help in achieving the desired rain effect in the image.","metadata":{}},{"cell_type":"code","source":"def add_rain(image, \n slant_lower=-10, # Lower bound for rain slant\n slant_upper=10, # Upper bound for rain slant\n drop_length=20, # Length of the raindrops\n drop_width=1, # Width of the raindrops\n drop_color=(200, 200, 200), # Color of the raindrops\n blur_value=3, # Amount of blur added due to rain\n brightness_coefficient=0.9, # Adjust brightness of image \n rain_type=None): # Type of rain (drizzle or None for regular)\n \n \"\"\"Add rain effect to the input image using specified parameters.\"\"\"\n \n # Create the rain augmentation using albumentations\n rain_aug = albumentations.RandomRain(\n slant_lower=slant_lower, \n slant_upper=slant_upper, \n drop_length=drop_length, \n drop_width=drop_width, \n drop_color=drop_color, \n blur_value=blur_value, \n brightness_coefficient=brightness_coefficient, \n rain_type=rain_type, \n p=1 # Probability of augmentation. Set to 1 for always applying\n )\n \n # Apply the rain augmentation to the image\n augmented = rain_aug(image=image)\n\n return augmented['image']\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:25:44.824796Z","iopub.execute_input":"2024-02-14T07:25:44.825687Z","iopub.status.idle":"2024-02-14T07:25:44.833147Z","shell.execute_reply.started":"2024-02-14T07:25:44.825657Z","shell.execute_reply":"2024-02-14T07:25:44.832128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n##
Step 3.3.2 | Snow Effect Integration\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nIn this part, I utilize the `RandomSnow` method from the `albumentations` library to define a function for adding a snow effect to the input image. The definitions of this method's arguments are as follows:\n\n- **snow_point_lower and snow_point_upper**: These determine the intensity range of the snow effect. For example, values of `0.1` and `0.3` mean that the snow can cover between `10%` to `30%` of the image.\n \n \n- **brightness_coeff**: Controls the brightness of the image after adding the snow. A value greater than `1` will brighten the image, reflecting the nature of snowy landscapes which often appear brighter.\n \n\n- **p**: The probability of applying the snow augmentation to an image. A value of `1` ensures that the snow effect is always applied.\n \n**Note**: Just like the rain effect, adjusting these parameters can lead to varying intensities and appearances of snow in the image, making it crucial to fine-tune them based on the desired outcome.","metadata":{}},{"cell_type":"code","source":"def add_snow(image, \n snow_point_lower=0.1, # Minimum intensity of snow \n snow_point_upper=0.2, # Maximum intensity of snow \n brightness_coeff=2.0): # Brightness coefficient for the image post snow effect\n \n \"\"\"Add snow effect to the input image using specified parameters.\"\"\"\n \n # Define the snow augmentation from albumentations with provided parameters\n snow_aug = albumentations.RandomSnow(snow_point_lower=snow_point_lower, \n snow_point_upper=snow_point_upper, \n brightness_coeff=brightness_coeff, \n p=1 # Probability of augmentation. Set to 1 for always applying\n )\n \n # Apply the snow augmentation to the image\n augmented = snow_aug(image=image)\n \n return augmented['image']\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:26:48.158899Z","iopub.execute_input":"2024-02-14T07:26:48.159244Z","iopub.status.idle":"2024-02-14T07:26:48.165102Z","shell.execute_reply.started":"2024-02-14T07:26:48.159219Z","shell.execute_reply":"2024-02-14T07:26:48.164191Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n##
Step 3.3.3 | Fog Effect Integration\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nIn this part, I am going to use the `RandomFog` method of the `albumentations` library to define a function for adding a fog effect to the input image. The definition of this method's arguments are as follows:\n\n- **fog_coef_lower**: Represents the minimum intensity of fog. For example, a value of `0.3` means that at least `30%` of the image will have a fog effect.\n \n\n- **fog_coef_upper**: Represents the maximum intensity of fog. A value of `1` would mean that up to `100%` of the image can be covered in fog.\n \n\n- **alpha_coef**: Defines the transparency of the fog with a range between `0` and `1`. A value closer to `0` makes the fog more transparent, while a value closer to `1` makes it denser and more opaque.\n \n \n- **p**: The probability of applying the fog augmentation to an image. A value of `1` ensures that the fog effect is always applied.\n\n \nBy using these parameters, we can simulate different foggy conditions on our images, enhancing the model's robustness against such weather conditions.","metadata":{}},{"cell_type":"code","source":"def add_fog(image, \n fog_coef_lower=0.3, # Minimum intensity of fog\n fog_coef_upper=1, # Maximum intensity of fog\n alpha_coef=0.08): # Transparency of the fog\n \n \"\"\"Add fog effect to the input image using specified parameters.\"\"\"\n \n # Create the fog augmentation using albumentations\n fog_aug = albumentations.RandomFog(fog_coef_lower=fog_coef_lower, \n fog_coef_upper=fog_coef_upper, \n alpha_coef=alpha_coef, \n p=1 # Probability of augmentation. Set to 1 for always applying\n )\n \n # Apply the fog augmentation to the image\n augmented = fog_aug(image=image)\n \n return augmented['image']\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:27:42.387135Z","iopub.execute_input":"2024-02-14T07:27:42.387822Z","iopub.status.idle":"2024-02-14T07:27:42.393500Z","shell.execute_reply.started":"2024-02-14T07:27:42.387793Z","shell.execute_reply":"2024-02-14T07:27:42.392460Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nHaving defined the three augmentations above, let's visualize their impact on 6 randomly selected images from the training dataset:","metadata":{}},{"cell_type":"code","source":"# Setting the random seed for reproducibility\nnp.random.seed(42) # Set the random seed to ensure reproducibility of results\n\n# Select 6 random images from the train_df\nrandom_image_paths = random.sample(train_df['filepath'].tolist(), 6) # Randomly select 6 image paths from the DataFrame\n\n# For each image, display original and the ones with effects\nfor image_path in random_image_paths: # Iterate over the selected image paths\n # Read the image\n image = cv2.imread(image_path) # Read the image using OpenCV\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # Convert the image from BGR to RGB color space\n \n # Apply augmentations\n image_rain = add_rain(image) # Add rain effect to the image\n image_snow = add_snow(image) # Add snow effect to the image\n image_fog = add_fog(image) # Add fog effect to the image\n \n # Display images\n fig, axes = plt.subplots(1, 4, figsize=(15, 5)) # Create a figure with subplots\n \n axes[0].imshow(image) # Display the original image\n axes[0].set_title(\"Original Image\", fontsize=15) # Set the title for the subplot\n axes[0].axis(\"off\") # Turn off axis labels\n \n axes[1].imshow(image_rain) # Display the image with rain effect\n axes[1].set_title(\"With Rain\", fontsize=15) # Set the title for the subplot\n axes[1].axis(\"off\") # Turn off axis labels\n \n axes[2].imshow(image_snow) # Display the image with snow effect\n axes[2].set_title(\"With Snow\", fontsize=15) # Set the title for the subplot\n axes[2].axis(\"off\") # Turn off axis labels\n \n axes[3].imshow(image_fog) # Display the image with fog effect\n axes[3].set_title(\"With Fog\", fontsize=15) # Set the title for the subplot\n axes[3].axis(\"off\") # Turn off axis labels\n \n plt.tight_layout() # Adjust the layout of the subplots\n plt.show() # Show the plot\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:29:41.615750Z","iopub.execute_input":"2024-02-14T07:29:41.616615Z","iopub.status.idle":"2024-02-14T07:29:47.986400Z","shell.execute_reply.started":"2024-02-14T07:29:41.616573Z","shell.execute_reply":"2024-02-14T07:29:47.985517Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n##
Step 3.3.4 | Applying Random Weather Effects to Dataset\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n
🌦️ Random Weather Effect Explanation
\n\nI am going to define the `random_weather_effect` function which is designed to apply a random weather effect to a given image. It selects among the three weather effects we've defined earlier - snow, rain, and fog. Additionally, there's a chance it might decide not to apply any effect at all. So, every time an image is processed through this function, it could receive a touch of rain, a sprinkle of snow, a mist of fog, or remain completely unchanged:","metadata":{}},{"cell_type":"code","source":"def random_weather_effect(img):\n \"\"\"\n Apply a random weather effect (rain, snow, fog) or no effect to an image.\n \n Parameters:\n - img: Input image to be augmented.\n\n Returns:\n - Augmented image with a random weather effect or the original image.\n \"\"\"\n \n # List of possible weather effects including no effect (lambda x: x)\n effects = [add_snow, add_rain, add_fog, lambda x: x]\n \n # Randomly choose one effect from the list\n effect = random.choice(effects)\n \n # Apply the chosen effect and return the result\n return effect(img)\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:31:28.239423Z","iopub.execute_input":"2024-02-14T07:31:28.240107Z","iopub.status.idle":"2024-02-14T07:31:28.245287Z","shell.execute_reply.started":"2024-02-14T07:31:28.240075Z","shell.execute_reply":"2024-02-14T07:31:28.244316Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nIn order to make my model more robust to different environmental conditions, I am going to apply the above function to all images in the `train_df` DataFrame and overwrite the original images with the augmented ones:","metadata":{}},{"cell_type":"code","source":"def apply_effects_and_save(image_path):\n \"\"\"\n Apply a random weather effect to an image at a given path and overwrite the original image.\n\n This function reads an image from a specified file path using OpenCV, applies one of several\n weather effects (snow, rain, fog, or none), and saves the altered image back to the same path,\n effectively overwriting the original image.\n\n Parameters:\n - image_path (str): The file path to the image that will be augmented with a weather effect.\n\n Raises:\n - ValueError: If the image at the given path cannot be opened or processed.\n \"\"\"\n \n # Read the image file using OpenCV\n img = cv2.imread(image_path)\n \n # Check if the image was correctly opened\n if img is None:\n raise ValueError(f\"Image at path {image_path} could not be opened.\")\n \n # Convert from BGR to RGB since OpenCV uses BGR by default\n img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n \n # Apply random weather effect\n augmented_img = random_weather_effect(img)\n \n # Convert back from RGB to BGR before saving with OpenCV\n augmented_img = cv2.cvtColor(augmented_img, cv2.COLOR_RGB2BGR)\n \n # Save the augmented image back to the original path\n cv2.imwrite(image_path, augmented_img)\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:32:17.808852Z","iopub.execute_input":"2024-02-14T07:32:17.809480Z","iopub.status.idle":"2024-02-14T07:32:17.815873Z","shell.execute_reply.started":"2024-02-14T07:32:17.809450Z","shell.execute_reply":"2024-02-14T07:32:17.814816Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Iterate over the images in the train set DataFrame\nfor index, row in train_df.iterrows():\n image_path = row['filepath']\n # Check if the image file exists\n if os.path.exists(image_path):\n try:\n apply_effects_and_save(image_path) # Apply random weather effect and save the image\n except ValueError as e:\n print(e) # Print any errors encountered during the process\n else:\n print(f\"Image not found at path: {image_path}\") # Print a message if the image file is not found\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:32:33.614913Z","iopub.execute_input":"2024-02-14T07:32:33.615378Z","iopub.status.idle":"2024-02-14T07:35:44.664856Z","shell.execute_reply.started":"2024-02-14T07:32:33.615327Z","shell.execute_reply":"2024-02-14T07:35:44.663841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nLet's display a selection of images randomely from the training set to verify the correct application of weather effects:","metadata":{}},{"cell_type":"code","source":"def display_random_images(image_paths, labels, num_images=16):\n \"\"\"\n Display a random set of images in a grid.\n \n Parameters:\n - image_paths (list): List of filepaths to images.\n - labels (list): Corresponding labels for the images.\n - num_images (int): Total number of images to display.\n \"\"\"\n # Select a random subset of image paths and their labels\n random_indices = np.random.choice(len(image_paths), num_images, replace=False)\n selected_image_paths = [image_paths[i] for i in random_indices]\n selected_labels = [labels[i] for i in random_indices]\n \n # Calculate number of rows and columns for the subplot grid\n num_rows = num_images // 4\n num_cols = 4\n \n # Create the subplots\n fig, axes = plt.subplots(num_rows, num_cols, figsize=(15, 15))\n axes = axes.ravel()\n \n # Read and display each image\n for i, (path, label, ax) in enumerate(zip(selected_image_paths, selected_labels, axes)):\n # Read the image using OpenCV\n img = cv2.imread(path)\n # Convert BGR image to RGB\n img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n ax.imshow(img)\n ax.set_title(label, fontsize=15)\n ax.axis('off')\n \n plt.tight_layout()\n plt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:38:08.442328Z","iopub.execute_input":"2024-02-14T07:38:08.442985Z","iopub.status.idle":"2024-02-14T07:38:08.451006Z","shell.execute_reply.started":"2024-02-14T07:38:08.442960Z","shell.execute_reply":"2024-02-14T07:38:08.450046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Display the images\ndisplay_random_images(train_df['filepath'].tolist(), train_df['label'].tolist())","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:38:49.575207Z","iopub.execute_input":"2024-02-14T07:38:49.575874Z","iopub.status.idle":"2024-02-14T07:38:53.366873Z","shell.execute_reply.started":"2024-02-14T07:38:49.575842Z","shell.execute_reply":"2024-02-14T07:38:53.365613Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nThe displayed images confirm that weather effects have been successfully and randomly implemented across the training set.","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 3.4 | Data Augmentation & Preprocessing\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n \n###
🔍 Concerns:\n \n1️⃣ To ensure our model works well in various situations, we want to change our images in different ways, such as rotating or shifting them. This also helps in preventing overfitting when fine-tuning our predefined models.\n \n2️⃣ With a large dataset of 20,000 images, storing all augmented versions in memory isn't feasible. So, we should apply these changes in real-time, or \"__on-the-fly__\", as we train.\n \n3️⃣ Each pre-trained model we plan to use has its unique way of processing images. To get the best results, we need to ensure we're preparing our images in the right way for each model.\n","metadata":{}},{"cell_type":"markdown","source":"
\n \n###
🛠️ Strategy:\n\n- Keras offers a powerful utility called __`ImageDataGenerator`__, which enables real-time image augmentation during training. This approach is both memory-efficient, as it eliminates the need to store the augmented images, and dynamic, allowing us to introduce a variety of transformations such as rotation and shifts to enhance model generalization.\n\n \n- By leveraging the __`flow_from_dataframe`__ method, we can directly link our image transformations to our DataFrame source. This ensures a seamless integration of __on-the-fly__ augmentation with our training process.\n\n \n- To cater to the distinct preprocessing protocols of different pre-trained models, we'll encapsulate our augmentation logic within a function. This function will intelligently apply the specific preprocessing steps (`preprocessing_function`) needed for each model, ensuring optimal compatibility and performance during fine-tuning.\n","metadata":{}},{"cell_type":"code","source":"def create_data_generators(preprocessing_function=None, batch_size=32, image_dimensions=(224, 224)):\n \"\"\"\n Creates and returns training and validation data generators with optional weather effects and preprocessing.\n \n Parameters:\n - preprocessing_function (function, optional): Preprocessing function specific to a model. Defaults to None.\n - batch_size (int, optional): Number of images per batch for the generators. Defaults to 32.\n - image_dimensions (tuple, optional): Dimensions to which the images will be resized (height, width). Defaults to (224, 224).\n \n Returns:\n - train_generator (ImageDataGenerator): Generator for training data with augmentations.\n - val_generator (ImageDataGenerator): Generator for validation data without augmentations.\n \n Notes:\n - The training generator uses augmentations.\n - The validation generator does not use any augmentations.\n - If provided, the preprocessing function is applied to both generators.\n \"\"\"\n\n # Define our training data generator with specific augmentations\n train_datagen = ImageDataGenerator(\n rotation_range=15, # Randomly rotate the images by up to 15 degrees\n width_shift_range=0.15, # Randomly shift images horizontally by up to 15% of the width\n height_shift_range=0.15, # Randomly shift images vertically by up to 15% of the height\n zoom_range=0.15, # Randomly zoom in or out by up to 15%\n horizontal_flip=True, # Randomly flip images horizontally\n vertical_flip=False, # Do not flip images vertically as it doesn't make sense in our context\n shear_range=0.02, # Apply slight shear transformations\n preprocessing_function=preprocessing_function # Apply preprocessing function if provided\n )\n\n # Define our validation data generator without any augmentations but with the preprocessing function if provided\n val_datagen = ImageDataGenerator(\n preprocessing_function=preprocessing_function # Apply preprocessing function if provided\n )\n\n # Create an iterable generator for training data\n train_generator = train_datagen.flow_from_dataframe(\n dataframe=train_df, # DataFrame containing training data\n x_col=\"filepath\", # Column with paths to image files\n y_col=\"label\", # Column with image labels\n target_size=image_dimensions, # Resize all images to size of 224x224 \n batch_size=batch_size, # Number of images per batch\n class_mode='binary', # Specify binary classification task\n seed=42, # Seed for random number generator to ensure reproducibility\n shuffle=True # Shuffle the data to ensure the model gets a randomized batch during training\n )\n\n # Create an iterable generator for validation data\n val_generator = val_datagen.flow_from_dataframe(\n dataframe=val_df, # DataFrame containing validation data\n x_col=\"filepath\", \n y_col=\"label\",\n target_size=image_dimensions,\n batch_size=batch_size,\n class_mode='binary',\n seed=42,\n shuffle=False # Shuffling not necessary for validation data\n )\n \n # Return the training and validation generators\n return train_generator, val_generator","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:43:32.663611Z","iopub.execute_input":"2024-02-14T07:43:32.663957Z","iopub.status.idle":"2024-02-14T07:43:32.673728Z","shell.execute_reply.started":"2024-02-14T07:43:32.663929Z","shell.execute_reply":"2024-02-14T07:43:32.672614Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n \n###
📝 Note on Validation Data Augmentation:\n\nI applied neither the weather effects nor the data augmentation techniques on the validation images. The reason for this is twofold:\n \n1️⃣ **Validation Set Should Reflect Real-World Conditions**: The purpose of the validation set is to evaluate how well the model might perform on unseen real-world data. By applying augmentation techniques, especially artificial ones, we risk __distorting__ this real-world representation. We want to ensure that the model's performance metrics on the validation set are as realistic as possible.\n \n \n2️⃣ **Measure the Impact of Augmentation**: By keeping the validation set free from such augmentations, we can measure how effective our training-time augmentations (like the weather effects) are. If the model performs well on the __unaugmented__ validation set, it indicates that the augmentations during training have improved the model's generalization capabilities.","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 4 | Model Architecture Development
\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n
🏗️ Leveraging Transfer Learning for Robust Performance
\n \nIn our project, we're tackling a classic binary image classification problem: distinguishing between **cats** and **dogs**. The **ImageNet** dataset, known for its extensive variety of images including numerous cat and dog breeds, serves as the foundational training ground for many state-of-the-art **pre-trained** neural networks. These networks have developed robust feature maps for cats and dogs, which we can leverage for our specific task through **transfer learning**.\n\nTransfer learning allows us to take a pre-trained model and repurpose it for our **cat/dog classification** by utilizing its powerful feature-extracting capabilities. To adapt the model to our needs, we remove its original classification head – the part trained to identify 1,000 different ImageNet classes – and replace it with a new classifier tailored to our **binary** task.\n\nDuring this process, we **freeze** the feature-extraction layers of the pre-trained network. This means we keep the weights, which have been fine-tuned through extensive ImageNet training, intact. **We only train the newly added classifier from scratch.** This approach effectively turns the pre-trained model into a feature extractor for our dataset, allowing us to utilize the complex feature maps already learned without having to train these layers all over again. By focusing on training just the classifier, we efficiently re-target the model's learned patterns to discern between cats and dogs in our images.","metadata":{}},{"cell_type":"markdown","source":"
\n\nBy employing transfer learning with a pre-trained model for our cat/dog classification, we enjoy several advantages:\n\n1️⃣ **Efficiency in Training**: We drastically reduce the training time since the complex feature maps have already been learned.\n \n2️⃣ **Lower Data Requirement**: There's less need for a large labeled dataset, as the pre-trained layers already understand general features.\n \n3️⃣ **Improved Performance**: The pre-trained network brings a level of sophistication and learning that is challenging to achieve from scratch, especially with limited data.\n \n4️⃣ **Resource Optimization**: It allows for the use of less computational power, making it a cost-effective solution.\n","metadata":{}},{"cell_type":"markdown","source":"
\n
💡 Model Selection Criteria for Real-Time Analysis
\n\nWhen selecting the optimal models for real-time CCTV footage analysis for a desktop application, we aim for a sweet spot between accuracy and computational efficiency. Our choices are informed by [Keras pre-trained models benchmark](https://keras.io/api/applications/):\n\n### 1️⃣ ResNet50V2: A Balance of Depth and Efficiency\n- **Efficiency**: At 98 MB, ResNet50V2 is relatively compact, suitable for desktop deployments.\n- **Accuracy**: Offers a Top-1 accuracy of 76.0%, providing a solid performance for our needs.\n- **Speed**: Reasonable inference time on CPU and GPU, crucial for real-time applications.\n\n### 2️⃣ InceptionV3: Mastering Multi-Scale Patterns\n- **Multi-Scale Patterns**: Its design is adept at managing images with varying complexities, crucial for our diverse dataset.\n- **Trade-off**: A well-measured balance between model size (92 MB), computational demand, and accuracy (Top-1 accuracy of 77.9%).\n\n### 3️⃣ MobileNetV2: The Speed Demon\n- **Lightweight**: Exceptionally small at 14 MB, it's perfect for resource-limited desktop applications.\n- **Speed**: Boasts fast inference times, essential for immediate processing.\n- **Accuracy vs. Size**: While the accuracy (71.3% Top-1) is marginally lower, the efficiency and speed benefits are particularly valuable for our use case.","metadata":{}},{"cell_type":"markdown","source":"
\n\nConsidering the strengths of each model, our goal is to find a balance between accuracy, processing speed, and ease of deployment for our real-time analysis project. The final selection will be the model that excels in both accuracy and speed, aligning with the requirements of our desktop application.","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 4.1 | ResNet50V2 Architecture\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"code","source":"# Load the ResNet50V2 model pre-trained on ImageNet data, excluding the top classifier\nresnet50v2_base = ResNet50V2(weights='imagenet', include_top=False, input_shape=(224, 224, 3))\nresnet50v2_base.summary()","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:47:29.529041Z","iopub.execute_input":"2024-02-14T07:47:29.529422Z","iopub.status.idle":"2024-02-14T07:47:33.294570Z","shell.execute_reply.started":"2024-02-14T07:47:29.529392Z","shell.execute_reply":"2024-02-14T07:47:33.293656Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n \nThe feature extractor portion of the **ResNet50V2** model comprises approximately **23.5 million** trainable parameters, all of which will be frozen:","metadata":{}},{"cell_type":"code","source":"# Freezes all layers in the ResNet50V2 base model\nresnet50v2_base.trainable = False","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:48:00.056567Z","iopub.execute_input":"2024-02-14T07:48:00.056929Z","iopub.status.idle":"2024-02-14T07:48:00.069096Z","shell.execute_reply.started":"2024-02-14T07:48:00.056901Z","shell.execute_reply":"2024-02-14T07:48:00.068111Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nThen, I am going to add a custom fully connected neural network on top of the pre-trained **ResNet50V2** base model to serve as a binary classifier for our task:","metadata":{}},{"cell_type":"code","source":"x = resnet50v2_base.output\n# Add a global spatial average pooling layer to reduce the dimensions of the feature maps\nx = GlobalAveragePooling2D()(x)\n# Add a fully-connected layer to learn more complex representations specific to our task\nx = Dense(1024, activation='relu')(x) \n# Add a dropout layer for regularization\nx = Dropout(0.5)(x) \n# Add a logistic layer for binary classification\npredictions = Dense(1, activation='sigmoid')(x) \n\n# This is the model we will train\nresnet50v2_model = Model(inputs=resnet50v2_base.input, outputs=predictions)\n\n# Compile the model after setting layers to non-trainable\nresnet50v2_model.compile(optimizer=Adam(learning_rate=0.0001), loss='binary_crossentropy', metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:48:26.131510Z","iopub.execute_input":"2024-02-14T07:48:26.131858Z","iopub.status.idle":"2024-02-14T07:48:26.204857Z","shell.execute_reply.started":"2024-02-14T07:48:26.131832Z","shell.execute_reply":"2024-02-14T07:48:26.204113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n \nNow, let's visualize the architecture of our final ResNet50V2 model:","metadata":{}},{"cell_type":"code","source":"plot_model(\n resnet50v2_model, # The ResNet50V2 model object to visualize\n show_shapes=True, # Display the shapes of the layers\n show_layer_names=False, # Hide the names of the layers\n dpi=150 # Set the resolution of the generated plot\n)\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:49:35.409031Z","iopub.execute_input":"2024-02-14T07:49:35.410067Z","iopub.status.idle":"2024-02-14T07:49:40.229584Z","shell.execute_reply.started":"2024-02-14T07:49:35.410028Z","shell.execute_reply":"2024-02-14T07:49:40.228626Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n \nWe can also examine our final **ResNet50V2** model summary to understand the number of parameters involved:","metadata":{}},{"cell_type":"code","source":"# Model summary\nresnet50v2_model.summary()","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:53:12.464323Z","iopub.execute_input":"2024-02-14T07:53:12.464698Z","iopub.status.idle":"2024-02-14T07:53:12.977502Z","shell.execute_reply.started":"2024-02-14T07:53:12.464670Z","shell.execute_reply":"2024-02-14T07:53:12.976332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n \n###
📉 Trainable Parameter Reduction in ResNet50V2\n\nBy freezing the feature extraction layers of the ResNet50V2 model, we have significantly decreased the trainable parameters to around **2 million** from the original **23.5 million**. This reduction strikes a balance, mitigating overfitting and leveraging the model's pre-trained intelligence.","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 4.2 | InceptionV3 Architecture\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n \nSame as the previous step, I am going to initialize the **InceptionV3** base model with weights pre-trained on ImageNet, keeping the architecture's default input size of `(299, 299, 3)`. The layers of the base model are frozen to maintain the learned features, and a custom classifier is appended for the binary classification task:","metadata":{}},{"cell_type":"code","source":"# Load the InceptionV3 model pre-trained on ImageNet data, excluding the top classifier\ninceptionv3_base = InceptionV3(weights='imagenet', include_top=False, input_shape=(299, 299, 3))\ninceptionv3_base.summary()\n\n# Freezes all layers in the InceptionV3 base model\ninceptionv3_base.trainable = False\n\n# Following the same pattern to add new layers\nx = inceptionv3_base.output\nx = GlobalAveragePooling2D()(x) \nx = Dense(1024, activation='relu')(x) \nx = Dropout(0.5)(x) \npredictions = Dense(1, activation='sigmoid')(x)\n\n# This is the model we will train\ninceptionv3_model = Model(inputs=inceptionv3_base.input, outputs=predictions)\n\n# Compile the model after setting layers to non-trainable\ninceptionv3_model.compile(optimizer=Adam(learning_rate=0.0001), loss='binary_crossentropy', metrics=['accuracy'])\n\n# InceptionV3 model summary\ninceptionv3_model.summary()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Load the InceptionV3 model pre-trained on ImageNet data, excluding the top classifier\ninceptionv3_base = InceptionV3(weights='imagenet', include_top=False, input_shape=(299, 299, 3))\ninceptionv3_base.summary()\n\n# Freezes all layers in the InceptionV3 base model\ninceptionv3_base.trainable = False\n\n# Following the same pattern to add new layers\nx = inceptionv3_base.output\nx = GlobalAveragePooling2D()(x) \nx = Dense(1024, activation='relu')(x) \nx = Dropout(0.5)(x) \npredictions = Dense(1, activation='sigmoid')(x)\n\n# This is the model we will train\ninceptionv3_model = Model(inputs=inceptionv3_base.input, outputs=predictions)\n\n# Compile the model after setting layers to non-trainable\ninceptionv3_model.compile(optimizer=Adam(learning_rate=0.0001), loss='binary_crossentropy', metrics=['accuracy'])\n\n# InceptionV3 model summary\ninceptionv3_model.summary()\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T07:55:03.815165Z","iopub.execute_input":"2024-02-14T07:55:03.815536Z","iopub.status.idle":"2024-02-14T07:55:08.636448Z","shell.execute_reply.started":"2024-02-14T07:55:03.815508Z","shell.execute_reply":"2024-02-14T07:55:08.635558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nAs anticipated, we observe that **InceptionV3** has a similar count of **2 million** trainable parameters to ResNet50V2, attributed to the identical classifiers and the completely frozen feature extractor layers in both models. Let's take a look at the architecture of our final **InceptionV3** model:","metadata":{}},{"cell_type":"code","source":"plot_model(\n inceptionv3_model, # The InceptionV3 model object to visualize\n show_shapes=True, # Display the shapes of the layers\n show_layer_names=False, # Hide the names of the layers\n dpi=150 # Set the resolution of the generated plot\n)\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T08:01:02.402777Z","iopub.execute_input":"2024-02-14T08:01:02.403165Z","iopub.status.idle":"2024-02-14T08:01:14.844602Z","shell.execute_reply.started":"2024-02-14T08:01:02.403134Z","shell.execute_reply":"2024-02-14T08:01:14.842917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n#
Step 4.3 | MobileNetV2 Architecture\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nIn this step, my aim is to harness the **MobileNetV2** architecture and explore two distinct classifier strategies:\n \n- Employing a Neural Network Classifier in conjunction with MobileNetV2\n- Integrating a Support Vector Machine (SVM) Classifier with MobileNetV2\n","metadata":{}},{"cell_type":"markdown","source":"
\n \nAt this stage, I will adopt a consistent method for the feature extraction part of **MobileNetV2** as we did with ResNet50V2, utilizing an input shape of **(224, 224, 3)**. The **MobileNetV2** model is initialized with ImageNet pre-trained weights, and the `include_top=False` parameter is set to remove the original classification layers. This prepares the model for the addition of a custom classifier tailored to our binary classification task:\n","metadata":{}},{"cell_type":"code","source":"# Load the MobileNetV2 model pre-trained on ImageNet data, excluding the top classifier\nmobilenetv2_base = MobileNetV2(weights='imagenet', include_top=False, input_shape=(224, 224, 3))\nmobilenetv2_base.summary()\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T08:04:50.553667Z","iopub.execute_input":"2024-02-14T08:04:50.554674Z","iopub.status.idle":"2024-02-14T08:04:52.644381Z","shell.execute_reply.started":"2024-02-14T08:04:50.554638Z","shell.execute_reply":"2024-02-14T08:04:52.643413Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nThe **MobileNetV2** feature extractor comprises approximately **2.2 million** parameters, indicative of a lightweight model, which we will freeze to leverage its pre-trained weights:","metadata":{}},{"cell_type":"code","source":"# Freeze all layers in the MobileNetV2 base model\nmobilenetv2_base.trainable = False","metadata":{"execution":{"iopub.status.busy":"2024-02-14T08:05:50.761079Z","iopub.execute_input":"2024-02-14T08:05:50.761847Z","iopub.status.idle":"2024-02-14T08:05:50.772063Z","shell.execute_reply.started":"2024-02-14T08:05:50.761812Z","shell.execute_reply":"2024-02-14T08:05:50.771153Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n##
Step 4.3.1 | MobileNetV2 with Neural Network Classifier\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nFor the first approach, I am going to implement a fully connected neural network as a classifier following the pattern used for **InceptionV3** and **ResNet50V2**:","metadata":{}},{"cell_type":"code","source":"# Following the same pattern to add new layers to MobileNetV2 base\nx = mobilenetv2_base.output\nx = GlobalAveragePooling2D()(x) \nx = Dense(128, activation='relu')(x) # Reduced number of neurons!\nx = Dropout(0.5)(x) \npredictions = Dense(1, activation='sigmoid')(x)\n\n# This is the model we will train\nmobilenetv2_nn_model = Model(inputs=mobilenetv2_base.input, outputs=predictions)\n\n# Compile the model after setting layers to non-trainable\nmobilenetv2_nn_model.compile(optimizer=Adam(learning_rate=0.0001), loss='binary_crossentropy', metrics=['accuracy'])\n\n# MobileNetV2 model summary\nmobilenetv2_nn_model.summary()\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T08:08:20.421788Z","iopub.execute_input":"2024-02-14T08:08:20.422637Z","iopub.status.idle":"2024-02-14T08:08:20.851078Z","shell.execute_reply.started":"2024-02-14T08:08:20.422605Z","shell.execute_reply":"2024-02-14T08:08:20.850191Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nThe tailored **MobileNetV2** architecture comprises approximately **164,000** trainable parameters, a reduction achieved by opting for a denser layer with fewer neurons in the classifier (**128 neurons**).","metadata":{}},{"cell_type":"markdown","source":"
\n##
Step 4.3.2 | MobileNetV2 with SVM Classifier\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nIn this approach, I plan to employ a **Support Vector Machine (SVM)** as the classifier, stepping away from the neural network-based methods. The rationale is that an SVM may offer superior performance for our specific dataset, following feature extraction via a CNN. This potential improvement can be attributed to the SVM's effectiveness at determining an optimal decision boundary within the feature space created by the CNN.\n \nThis technique is often referred to as \"**feature extraction**\" where the deep learning model is used as a means to convert raw image data into a rich, more abstract representation that captures the essential aspects of the visual content. These representations, or feature vectors, can then be used as inputs to traditional classifiers. To construct our feature extractor for this method, we will be reemploying the `mobilenetv2_base`, which is the MobileNetV2 architecture pretrained on the ImageNet dataset:","metadata":{}},{"cell_type":"code","source":"# Add a GlobalAveragePooling layer to reduce the feature maps to a single vector per map\ngap = GlobalAveragePooling2D()(mobilenetv2_base.output)\n\n# Construct the feature extractor model\nmobilenetv2_feature_extractor = Model(inputs=mobilenetv2_base.input, outputs=gap)","metadata":{"execution":{"iopub.status.busy":"2024-02-14T08:21:11.671407Z","iopub.execute_input":"2024-02-14T08:21:11.671809Z","iopub.status.idle":"2024-02-14T08:21:11.694808Z","shell.execute_reply.started":"2024-02-14T08:21:11.671779Z","shell.execute_reply":"2024-02-14T08:21:11.694094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Investigate the output shape of feature extractor\noutput_shape = mobilenetv2_feature_extractor.output_shape\nprint(\"Output shape of feature extractor model:\", output_shape)","metadata":{"execution":{"iopub.status.busy":"2024-02-14T08:21:25.863422Z","iopub.execute_input":"2024-02-14T08:21:25.863752Z","iopub.status.idle":"2024-02-14T08:21:25.869399Z","shell.execute_reply.started":"2024-02-14T08:21:25.863729Z","shell.execute_reply":"2024-02-14T08:21:25.868376Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nThe **`mobilenetv2_feature_extractor`** transforms each input image into a feature vector with **1280** elements. The `None` in the output shape represents the batch size, which is flexible and depends on the number of images processed in a single batch.","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 5 | Classifier Training
\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 5.1 | Training the ResNet50V2 Classifier\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n \nI am going to define a versatile function to train various models, leveraging the consistency in our training process and data handling, allowing us to efficiently swap different architectures with minimal code changes:","metadata":{}},{"cell_type":"code","source":"def train_model(model, preprocessing_function, image_dimensions=(224, 224), batch_size=32, num_epochs=20):\n \"\"\"\n Trains the given model and returns the trained model, its history, and the validation generator.\n \n Parameters:\n - model: Model, a compiled instance of a Keras model to be trained.\n - preprocessing_function: function, preprocessing function to be applied to input data.\n - image_dimensions: tuple, dimensions of the images (width, height).\n - batch_size: int, number of samples per batch of computation.\n - num_epochs: int, number of epochs to train the model.\n \n Returns:\n - model: The trained model instance.\n - history: A History object containing the training history.\n - val_generator: The validation data generator.\n \"\"\"\n\n # Create data generators\n train_generator, val_generator = create_data_generators(preprocessing_function, batch_size, image_dimensions)\n\n # Define the callbacks\n reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=3, min_lr=0.00001)\n early_stopping = EarlyStopping(monitor='val_loss', mode='min', patience=10, restore_best_weights=True, verbose=1)\n\n # Train the model\n history = model.fit(\n train_generator,\n steps_per_epoch=len(train_generator),\n epochs=num_epochs,\n validation_data=val_generator,\n validation_steps=len(val_generator),\n callbacks=[reduce_lr, early_stopping]\n )\n \n return model, history, val_generator\n","metadata":{"execution":{"iopub.status.busy":"2024-02-14T08:23:42.914684Z","iopub.execute_input":"2024-02-14T08:23:42.915055Z","iopub.status.idle":"2024-02-14T08:23:42.922923Z","shell.execute_reply.started":"2024-02-14T08:23:42.915018Z","shell.execute_reply":"2024-02-14T08:23:42.921929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n \nNow, let's call the above function to train our **ResNet50V2** model classifier:","metadata":{}},{"cell_type":"code","source":"# Train the ResNet50V2 model classifier\nresnet50v2_model, resnet50v2_history, resnet50v2_val_generator = train_model(\n resnet50v2_model,\n preprocessing_function=resnet_preprocess_input, # The preprocessing function for ResNet50V2\n image_dimensions=(224, 224), # Adjusting to the expected input size for ResNet50V2\n batch_size=32,\n num_epochs=10\n)","metadata":{"execution":{"iopub.status.busy":"2024-02-14T08:24:28.034793Z","iopub.execute_input":"2024-02-14T08:24:28.035154Z","iopub.status.idle":"2024-02-14T09:30:42.888887Z","shell.execute_reply.started":"2024-02-14T08:24:28.035125Z","shell.execute_reply":"2024-02-14T09:30:42.887908Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n#
Step 5.2 | Training the InceptionV3 Classifier\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n \nNow, let's call the above function to train our **ResNet50V2** model classifier:","metadata":{}},{"cell_type":"code","source":"# Train the InceptionV3 model classifier\ninceptionv3_model, inceptionv3_history, inceptionv3_val_generator = train_model(\n inceptionv3_model, \n preprocessing_function=inceptionv3_preprocess_input, # The preprocessing function for InceptionV3\n image_dimensions=(299, 299), # Adjusting to the expected input size for InceptionV3\n batch_size=32,\n num_epochs=10\n)","metadata":{"execution":{"iopub.status.busy":"2024-02-14T09:49:36.642787Z","iopub.execute_input":"2024-02-14T09:49:36.643184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n#
Step 5.3 | Training the Neural Network Classifier for MobileNetV2\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nNext, let's use the previously defined function to train our **MobileNetV2** neural network classifier:","metadata":{}},{"cell_type":"code","source":"# Train the MobileNetV2 neural network classifier\nmobilenetv2_nn_model, mobilenetv2_nn_history, mobilenetv2_nn_val_generator = train_model(\n mobilenetv2_nn_model, \n preprocessing_function=mobilenetv2_preprocess_input, # The preprocessing function for MobileNetV2\n image_dimensions=(224, 224), # Input size for MobileNetV2\n batch_size=32,\n num_epochs=0\n)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n#
Step 5.4 | Training the SVM Classifier for MobileNetV2\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nTo train the **SVM classifier** using our **MobileNetV2** deep learning model, we'll employ a technique commonly known as \"**feature extraction**\". This involves using the deep learning model to transform raw image data into a rich, abstract representation that encapsulates the core elements of the visual content. These representations, or \"**feature vectors**\", will then serve as the input for traditional machine learning classifiers. Here's a detailed step-by-step guide on how we'll proceed:","metadata":{}},{"cell_type":"markdown","source":"
\n##
Step 5.4.1 | Data Preparation\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nWhen training an SVM with features extracted by a pre-trained MobileNetV2, data augmentation may not be required because:\n\n1️⃣ The extracted features are already stable and robust, capturing the essential information for classification tasks.\n\n2️⃣ MobileNetV2 is trained on a comprehensive dataset like ImageNet, which means it's inherently designed to handle variations in images that data augmentation would introduce.","metadata":{}},{"cell_type":"markdown","source":"
\n\nTherefore, I am going to define a function that creates a non-augmented data generator, tailored for feature extraction. This generator will be essential for later stages of our pipeline:","metadata":{}},{"cell_type":"code","source":"def create_feature_extraction_generator(preprocessing_function, dataframe, x_col, y_col, batch_size=32, image_dimensions=(224, 224)):\n \"\"\"\n Creates a data generator without augmentation for feature extraction purposes.\n \n Parameters:\n - preprocessing_function: function, preprocessing function to be applied to input data.\n - dataframe: pandas.DataFrame, the dataframe containing the file paths and labels.\n - x_col: str, the name of the dataframe column containing the file paths.\n - y_col: str, the name of the dataframe column containing the labels.\n - batch_size: int, number of samples per batch of computation.\n - image_dimensions: tuple, dimensions of the images (width, height).\n \n Returns:\n - generator: An iterable Keras generator that outputs batches of preprocessed images.\n \"\"\"\n \n # Initialize the ImageDataGenerator with the preprocessing function\n datagen = ImageDataGenerator(preprocessing_function=preprocessing_function)\n \n # Create the generator to read images from the dataframe\n generator = datagen.flow_from_dataframe(\n dataframe=dataframe, # The dataframe containing paths and labels\n x_col=x_col, # Column in dataframe that contains the paths to the images\n y_col=y_col, # Column in dataframe that contains the labels\n target_size=image_dimensions, # The dimensions to which all images found will be resized\n batch_size=batch_size, # Number of images to be yielded from the generator per batch\n class_mode='binary', # Class mode for binary classification\n shuffle=False # No shuffling. Important to keep data in order!\n )\n \n return generator","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n##
Step 5.4.2 | Feature extraction\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n \nNext, we'll create a function to extract features with the specified model and the data provided by our generator. This step is crucial for the subsequent phases of our processing pipeline:","metadata":{}},{"cell_type":"code","source":"def extract_features(model, generator):\n \"\"\"\n Extract features using the given model and data from the generator.\n \n Parameters:\n - model: The pre-trained model used for feature extraction.\n - generator: The generator that yields batches of input data.\n \n Returns:\n - features: A numpy array of extracted features.\n - labels: A numpy array of corresponding labels.\n \"\"\"\n # Use the model's predict method to extract features\n features = model.predict(generator, steps=len(generator), verbose=1)\n \n # Retrieve the labels from the generator's classes attribute\n labels = generator.classes\n \n return features, labels","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n##
Step 5.4.3 | Dimensionality reduction\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nIn our next steps, I aim to enhance the efficiency of SVM training by reducing the feature dimensions using a technique such as **PCA**. Currently, the 1280 features may be excessive, potentially impacting the SVM model's accuracy and resulting in an unnecessarily large model size — a significant issue for deployment. To address this, I will implement a function to apply dimensionality reduction as follows:","metadata":{}},{"cell_type":"code","source":"def reduce_dimensionality(features, n_components=0.90, fit_pca=None):\n \"\"\"\n Reduce the dimensionality of the given feature set using PCA.\n\n Parameters:\n - features: array-like, feature set to reduce.\n - n_components: int, float, or None, the number of principal components to keep.\n If 0 < n_components < 1, select the number of components such that\n the amount of variance that needs to be explained is greater than the\n percentage specified by n_components.\n - fit_pca: PCA object from sklearn, pre-fitted PCA to use for transforming features.\n If None, fit a new PCA on the provided features.\n\n Returns:\n - reduced_features: array-like, feature set transformed into reduced dimensionality space.\n - pca: PCA object, the PCA used to transform the features. Only returned if fit_pca is None.\n \"\"\"\n # If no pre-fitted PCA is provided, fit PCA on the features\n if fit_pca is None:\n pca = PCA(n_components=n_components)\n reduced_features = pca.fit_transform(features)\n # Print the cumulative explained variance ratio\n print(f'Cumulative explained variance by {pca.n_components_} principal components: {np.sum(pca.explained_variance_ratio_):.2f}')\n return reduced_features, pca\n else:\n # If a pre-fitted PCA is provided, transform the features using it\n reduced_features = fit_pca.transform(features)\n return reduced_features","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n##
Step 5.4.4 | Train the SVM model\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"code","source":"def train_svm(features, labels):\n \"\"\"\n Trains an SVM classifier using the provided features and labels.\n\n Parameters:\n - features: array-like, shape (n_samples, n_features)\n The feature vectors extracted from the images.\n - labels: array-like, shape (n_samples,)\n The target labels corresponding to each feature vector.\n\n Returns:\n - svm: object\n The trained SVM classifier.\n \"\"\"\n\n # Initialize the SVM classifier with a linear kernel.\n # Linear is chosen for its efficiency and effectiveness for high dimensional datasets especially image data.\n svm = SVC(kernel='linear') \n\n # Train the SVM classifier on the provided features and labels.\n svm.fit(features, labels)\n\n # The trained model is returned for further use, such as prediction on test data.\n return svm","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nFinally, It's time to combine everything and **train the SVM with MobileNetV2 features**:","metadata":{}},{"cell_type":"code","source":"# Create data generators for training and validation sets\ntrain_generator = create_feature_extraction_generator(mobilenetv2_preprocess_input, train_df, \"filepath\", \"label\", 32, (224, 224))\nval_generator = create_feature_extraction_generator(mobilenetv2_preprocess_input, val_df, \"filepath\", \"label\", 32, (224, 224))\n \n# Extract features from the generators using the feature extractor model\ntrain_features, train_labels = extract_features(mobilenetv2_feature_extractor, train_generator)\nval_features, val_labels = extract_features(mobilenetv2_feature_extractor, val_generator)\n \n# Reduce the dimensionality of the training features and get the fitted PCA\ntrain_features_reduced, pca = reduce_dimensionality(train_features, n_components=0.90)\n# Use the fitted PCA to transform the validation features\nval_features_reduced = reduce_dimensionality(val_features, fit_pca=pca)\n \n# Train the SVM model on the reduced training features\nsvm_model = train_svm(train_features_reduced, train_labels)\n \n# Predict on the reduced validation features\nval_predictions = svm_model.predict(val_features_reduced)\n \n# Calculate and print the SVM classifier accuracy on the validation set\nsvm_validation_accuracy = accuracy_score(val_labels, val_predictions)\nprint(f\"SVM Classifier Validation Accuracy: {svm_validation_accuracy * 100:.2f}%\")","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n \nThe high accuracy achieved on the validation set indicates that our MobileNetV2 model, equipped with an SVM classifier, has undergone successful training.","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 6 | Model Performance Assessment
\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nAt this step, I am going to evaluate the performance of the models in several critical aspects:\n \n- **Learning curves** will show us how well the models are learning from the training data.\n- **Accuracy** will give us a direct metric for comparison to see which model performs best on our data.\n- **Inference time** is crucial, especially in production environments.\n- **Model size** is important for deployment, particularly in environments with limited storage or memory.","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 6.1 | Learning Curve Analysis\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nSince an **SVM** does not train over epochs but rather solves an optimization problem in one go, it doesn't have a learning curve that shows progress over time. Therefore, for the SVM classifier using the MobileNetV2 feature extractor, we **would not** have a learning curve to analyze. So, I will focus on monitoring the learning curves of our neural network models, which will enable us to observe how the error evolves as these models progressively learn from the training data across successive epochs:","metadata":{}},{"cell_type":"markdown","source":"
\n \nI am going to define a function for plotting the learning curves of the models:","metadata":{}},{"cell_type":"code","source":"def plot_learning_curves(history, title='Model Learning Curves'):\n # Convert the history.history dict to a pandas DataFrame for easy plotting\n df = pd.DataFrame(history.history)\n\n # Set the style of seaborn for better visualization\n sns.set(rc={'axes.facecolor': 'gainsboro'}, style='darkgrid')\n\n # Plotting the learning curves\n plt.figure(figsize=(15,6))\n plt.suptitle(title, fontsize=16)\n\n # Plotting the training and validation loss\n plt.subplot(1, 2, 1)\n sns.lineplot(x=df.index, y=df['loss'], color='royalblue', label='Train Loss', marker='o', linestyle='--')\n sns.lineplot(x=df.index, y=df['val_loss'], color='orangered', label='Validation Loss', marker='o', linestyle='--')\n plt.title('Loss Evolution')\n plt.xlabel('Epochs')\n plt.ylabel('Loss')\n\n # Plotting the training and validation accuracy\n plt.subplot(1, 2, 2)\n sns.lineplot(x=df.index, y=df['accuracy'], color='royalblue', label='Train Accuracy', marker='o', linestyle='--')\n sns.lineplot(x=df.index, y=df['val_accuracy'], color='orangered', label='Validation Accuracy', marker='o', linestyle='--')\n plt.title('Accuracy Evolution')\n plt.xlabel('Epochs')\n plt.ylabel('Accuracy')\n\n plt.show()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_learning_curves(resnet50v2_history, title='ResNet50V2 Learning Curves')\nplot_learning_curves(inceptionv3_history, title='InceptionV3 Learning Curves')\nplot_learning_curves(mobilenetv2_nn_history, title='MobileNetV2 with NN Classifier Learning Curves')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
🧠 Insights from Model Training on the Cat/Dog Dataset
\n\nAll of our deep learning models demonstrate a smooth and consistent decrease in training loss, indicating effective learning progression. The validation loss also decreases steadily, closely mirroring the training loss, which suggests that the models are generalizing well and not overfitting. Regarding accuracy, both training and validation metrics improve with each epoch and plateau at high levels, signaling that the models have confidently learned the task. Overall, all models have successfully trained and achieved high validation accuracies, indicating effective performance in this classification task.","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 6.2 | Accuracy Assessment\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n \nTo assess the accuracy of the **ResNet50V2**, **InceptionV3**, and **MobileNetV2** (with a neural network classifier) models, I am going to define a function first:","metadata":{}},{"cell_type":"code","source":"def calculate_accuracy(model, val_generator):\n # Evaluates the model on the validation set\n results = model.evaluate(val_generator, steps=len(val_generator))\n # The 'results' list contains loss as the first element and accuracy as the second\n accuracy = results[1]\n return accuracy","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nThe, I am going to call this function for each of the neural network models to get their accuracies:","metadata":{}},{"cell_type":"code","source":"resnet50v2_accuracy = calculate_accuracy(resnet50v2_model, resnet50v2_val_generator)\ninceptionv3_accuracy = calculate_accuracy(inceptionv3_model, inceptionv3_val_generator)\nmobilenetv2_nn_accuracy = calculate_accuracy(mobilenetv2_nn_model, mobilenetv2_nn_val_generator)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nNow that I have gathered all the model accuracies, including the SVM classifier's accuracy from `step 5.4.4`, I am ready to proceed with comparing these accuracies on a horizontal bar plot for a visual and straightforward comparison:","metadata":{}},{"cell_type":"code","source":"# A dictionary to hold our model names and their corresponding accuracies\naccuracies = {\n 'MobileNetV2 with SVM': svm_validation_accuracy,\n 'ResNet50V2': resnet50v2_accuracy,\n 'InceptionV3': inceptionv3_accuracy,\n 'MobileNetV2 with NN': mobilenetv2_nn_accuracy\n}\n\n# Convert the dictionary into a pandas DataFrame for easy plotting\naccuracy_df = pd.DataFrame(list(accuracies.items()), columns=['Model', 'Accuracy'])\n\n# Sort the DataFrame by the 'Accuracy' column to ensure the plot is ordered\naccuracy_df.sort_values('Accuracy', ascending=True, inplace=True)\n\n# Create a horizontal bar plot\nplt.figure(figsize=(15, 6))\nbar_plot = sns.barplot(x='Accuracy', y='Model', data=accuracy_df, color='#33312b', orient='h')\n\n# Add the accuracy values on the bars\nfor p in bar_plot.patches:\n bar_plot.annotate(format(p.get_width(), '.2f'), \n (p.get_width()+0.02, p.get_y() + p.get_height() / 2),\n ha='center', va='center',\n xytext=(5, 0), textcoords='offset points', fontsize=15)\n\nplt.xlabel('Accuracy')\nplt.title('Accuracy Assessment of Models')\nplt.xlim(0, 1.1) \nplt.tight_layout()\nplt.show()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
🔍 Accuracy Assessment Across Models
\n\nThe plot provides an accuracy comparison across our models. **InceptionV3** stands out with top performance, while **ResNet50V2** and **MobileNetV2** using a Neural Network approach exhibit closely trailing high accuracy. The MobileNetV2 with a Support Vector Machine (SVM) shows a slightly reduced accuracy. Enhancements to the SVM variant could include SVM hyperparameter optimization. With high accuracies being almost universal, additional factors such as model size and inference time become pivotal in model selection.","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 6.3 | Inference Time Comparison\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nDue to the uncertainty of the end-user environment for our desktop app deployment and the potential variability in whether the end-user system will have GPU capabilities, we are unable to conduct a fair inference time comparison at this moment. Our current setup on Kaggle uses a **P100 GPU**, which would give an advantage to neural network models in terms of speed and not accurately reflect their performance on systems without GPU acceleration. As such, to avoid drawing misleading conclusions, we will not be comparing inference times until we can test on a CPU-only environment that more closely resembles our potential deployment scenarios.","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 6.4 | Model Size and Deployment Considerations\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n\nTo evaluate the storage footprint of our models, which is typically indicative of their complexity and parameter count, we can save each model to a file and examine the respective file sizes. Given that ResNet50V2 and InceptionV3 are substantially larger in size compared to MobileNetV2 models, we will focus our comparison on the MobileNetV2 variants:","metadata":{}},{"cell_type":"markdown","source":"
\n \nSaving **MobileNetV2** model with **SVM classifier** Components:","metadata":{}},{"cell_type":"code","source":"# Save the MobileNetV2 feature extractor, which is used to extract features from images before passing them to the SVM\nmobilenetv2_feature_extractor.save('mobilenetv2_feature_extractor.h5')\n\n# Save the PCA transformer, which is used to reduce the dimensionality of the features from the MobileNetV2 feature extractor\ndump(pca, 'pca_transformer.joblib')\n\n# Save the SVM classifier, which is the final classification model\ndump(svm_model, 'svm_classifier.joblib')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n \nSaving **MobileNetV2 model with Neural Network classifier**, which includes both the feature extraction and classification layers in one package:","metadata":{}},{"cell_type":"code","source":"# Save the entire MobileNetV2 neural network model\nmobilenetv2_nn_model.save('mobilenetv2_nn_model.h5')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n \nNow, I am going to retrieve their sizes:","metadata":{}},{"cell_type":"code","source":"# Function to get file size in MB\ndef get_file_size(file_path):\n return os.path.getsize(file_path) / (1024 * 1024) # Convert bytes to MB","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get the file sizes using the above function\nsize_feature_extractor = get_file_size('mobilenetv2_feature_extractor.h5')\nsize_pca = get_file_size('pca_transformer.joblib')\nsize_svm = get_file_size('svm_classifier.joblib')\nsize_mobilenetv2_nn = get_file_size('mobilenetv2_nn_model.h5')\n\n# Sum the sizes for the SVM-based model\ntotal_size_svm = size_feature_extractor + size_pca + size_svm\n\n# Create a dictionary with the total model sizes\ntotal_model_sizes = {\n 'MobileNetV2_SVM_Total': total_size_svm,\n 'MobileNetV2_NN': size_mobilenetv2_nn\n}\n\n# Prepare the data for plotting\ntotal_model_size_df = pd.DataFrame(list(total_model_sizes.items()), columns=['Model', 'Size'])","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plot\nplt.figure(figsize=(10, 4))\nbar_plot = sns.barplot(x='Size', y='Model', data=total_model_size_df, color='#33312b', orient='h')\n\n# Add the size values on the bars\nfor p in bar_plot.patches:\n bar_plot.annotate(format(p.get_width(), '.2f'), \n (p.get_width()+0.7, p.get_y() + p.get_height() / 2), \n ha='center', va='center', \n xytext=(5, 0), textcoords='offset points')\n\nplt.xlabel('Size in MB')\nplt.title('Model Size Comparison between MobileNetV2 Variants')\nplt.xlim(0, 24) \nplt.tight_layout()\nplt.show()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
📉 Model Size Comparison of MobileNetV2 Variants
\n\nThe horizontal bar plot demonstrates that the **MobileNetV2 with an SVM classifier** has a larger total model size compared to its **NN counterpart**. This total size for of the SVM variant encompasses the MobileNetV2 feature extractor, the SVM classifier, and the PCA transformer for dimensionality reduction. Understanding that InceptionV3 and ResNet50V2 are not included due to their much larger sizes, the choice between these MobileNetV2 models may be influenced by deployment constraints where model size is a critical factor.\n","metadata":{}},{"cell_type":"markdown","source":"
\n#
Step 7 | Final Model Selection and Justification
\n⬆️ [Tabel of Contents](#contents_tabel)","metadata":{}},{"cell_type":"markdown","source":"
\n
🏆 Final Model Selection and Rationale
\n\nGiven the near-perfect accuracy across all models and our deployment considerations, the model size has become the decisive factor. Our choice is the **MobileNetV2 with NN classifier**. This model not only offers high accuracy but also benefits from the smallest size among the competitors. This advantage is likely to translate into faster inference speeds, which is particularly beneficial for our desktop application deployment. Thus, the **MobileNetV2 with NN classifier** stands out as the optimal choice for our requirements.","metadata":{}},{"cell_type":"markdown","source":"
\n \nFinally, let's download our trained model for deployment:","metadata":{}},{"cell_type":"code","source":"def download_all_models(files, download_file_name):\n os.chdir('/kaggle/working/')\n zip_name = f\"/kaggle/working/{download_file_name}.zip\"\n \n # Create a command to zip all files\n command = f\"zip {zip_name} \" + \" \".join(files)\n \n result = subprocess.run(command, shell=True, capture_output=True, text=True)\n if result.returncode != 0:\n print(\"Unable to run zip command!\")\n print(result.stderr)\n return\n \n display(FileLink(f'{download_file_name}.zip'))","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# List of model files to download\nmodel_files = ['mobilenetv2_nn_model.h5']\n\n# Call the function with the list of files (Uncomment the following line, if you want to save your model)\n# download_all_models(model_files, 'models_archive')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n\n
🎯 For more details and to explore the App code, kindly check out the project's GitHub repository 🎯\n
","metadata":{}},{"cell_type":"markdown","source":"
Love you all and keep supporting
","metadata":{}}]}
![](\"https://github.com/Engr-Umer/Prodigy-InfoTech-/blob/3980e179216f3cfbbed8cf6d10259198565b8f2a/Dog-Cat.jpeg?raw=true\")
![\"Banner\"](\"/kaggle/input/photo-of-dogcat/OIP.jpg\")
\n","metadata":{}},{"cell_type":"markdown","source":"