{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.11","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":5441,"databundleVersionId":38425,"sourceType":"competition"}],"dockerImageVersionId":31011,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Predictive Analytics : Homework 4\n## Ujjwal Khanna\n\n#### This is an Image classification exercise. We will play with an expired Kaggle competition, please follow the rule of Kaggle and check the following page for more information https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition/overview. After submit you results, use screenshot to show you score (see image below). They don’t provide ranking for this competition anymore. Thus, no need to show your ranking at Public/Private Leaderboard. (Name formation of your submission: MSBA.SessionX.Yourname, please replace X and Yourname with the correct string) using the screenshot on the Private Leaderboard, along with the description of your solutions and the codes (ipynb formation is highly encouraged) as the final submission.","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-04-22T21:52:37.349462Z","iopub.execute_input":"2025-04-22T21:52:37.349713Z","iopub.status.idle":"2025-04-22T21:52:37.355488Z","shell.execute_reply.started":"2025-04-22T21:52:37.349693Z","shell.execute_reply":"2025-04-22T21:52:37.354721Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Unzip training and test data with overwrite\n!unzip -o -q /kaggle/input/dogs-vs-cats-redux-kernels-edition/train.zip -d /kaggle/working/train\n!unzip -o -q /kaggle/input/dogs-vs-cats-redux-kernels-edition/test.zip -d /kaggle/working/test","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-22T21:52:49.127264Z","iopub.execute_input":"2025-04-22T21:52:49.127559Z","iopub.status.idle":"2025-04-22T21:52:59.424598Z","shell.execute_reply.started":"2025-04-22T21:52:49.127537Z","shell.execute_reply":"2025-04-22T21:52:59.423611Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os, shutil\nfrom tqdm import tqdm\n\n# Correct source path (after unzip)\nsource_folder = '/kaggle/working/train/train'\ntarget_folder = '/kaggle/working/train_split'\n\n# Create directories\nos.makedirs(f'{target_folder}/cat', exist_ok=True)\nos.makedirs(f'{target_folder}/dog', exist_ok=True)\n\n# Move images into respective folders\nfor fname in tqdm(os.listdir(source_folder)):\n    src_path = os.path.join(source_folder, fname)\n    if fname.startswith('cat'):\n        shutil.move(src_path, os.path.join(target_folder, 'cat', fname))\n    elif fname.startswith('dog'):\n        shutil.move(src_path, os.path.join(target_folder, 'dog', fname))\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-22T21:53:03.599423Z","iopub.execute_input":"2025-04-22T21:53:03.599701Z","iopub.status.idle":"2025-04-22T21:53:04.355503Z","shell.execute_reply.started":"2025-04-22T21:53:03.599677Z","shell.execute_reply":"2025-04-22T21:53:04.354791Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Confirm folders exist and contain .jpg files\nprint(\"\\nFolders inside /kaggle/working/train_split:\", os.listdir('/kaggle/working/train_split'))\n\nprint(\"\\nSample images:\")\nprint(os.listdir('/kaggle/working/train_split/')[:5])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-22T21:53:07.783674Z","iopub.execute_input":"2025-04-22T21:53:07.783937Z","iopub.status.idle":"2025-04-22T21:53:07.789170Z","shell.execute_reply.started":"2025-04-22T21:53:07.783914Z","shell.execute_reply":"2025-04-22T21:53:07.788615Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(\"Moved cat images:\", len(os.listdir('/kaggle/working/train_split/cat')))\nprint(\"Moved dog images:\", len(os.listdir('/kaggle/working/train_split/dog')))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-22T21:53:09.723084Z","iopub.execute_input":"2025-04-22T21:53:09.723661Z","iopub.status.idle":"2025-04-22T21:53:09.742171Z","shell.execute_reply.started":"2025-04-22T21:53:09.723638Z","shell.execute_reply":"2025-04-22T21:53:09.741448Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(os.listdir('/kaggle/working/test/test')[:5])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-22T21:53:11.665640Z","iopub.execute_input":"2025-04-22T21:53:11.665908Z","iopub.status.idle":"2025-04-22T21:53:11.676652Z","shell.execute_reply.started":"2025-04-22T21:53:11.665889Z","shell.execute_reply":"2025-04-22T21:53:11.675961Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Project Overview\n\nIn this image classification task, we aim to distinguish between **dog** and **cat** images using supervised learning techniques. The dataset comes from the popular Kaggle competition [Dogs vs. Cats Redux](https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition/overview), which provides labeled training images and unlabeled test images.\n\nThis notebook explores two modeling approaches:\n1. A **custom CNN** built from scratch as a strong baseline\n2. A **ResNet50-based transfer learning model** leveraging pretrained ImageNet weights\n\nEach model is evaluated based on validation accuracy and loss, and final predictions are generated for submission. Results are reported via a screenshot from the Private Leaderboard, in accordance with the assignment guidelines.\n","metadata":{}},{"cell_type":"markdown","source":"### CNN for Dogs vs Cats","metadata":{}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\nimport matplotlib.pyplot as plt\n\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array\nfrom tensorflow.keras import models, layers, optimizers\nfrom tensorflow.keras.callbacks import EarlyStopping\n\n# Configuration\nimg_size = (150, 150)\nbatch_size = 32\ntrain_path = '/kaggle/working/train_split'\ntest_path = '/kaggle/working/test/test'\n\n# Data Augmentation\ntrain_datagen = ImageDataGenerator(\n    rescale=1./255,\n    validation_split=0.2,\n    rotation_range=20,\n    width_shift_range=0.2,\n    height_shift_range=0.2,\n    horizontal_flip=True\n)\n\ntrain_generator = train_datagen.flow_from_directory(\n    train_path,\n    target_size=img_size,\n    batch_size=batch_size,\n    class_mode='binary',\n    subset='training'\n)\n\nval_generator = train_datagen.flow_from_directory(\n    train_path,\n    target_size=img_size,\n    batch_size=batch_size,\n    class_mode='binary',\n    subset='validation'\n)\n\n# CNN Model Architecture\nmodel = models.Sequential([\n    layers.Conv2D(32, (3, 3), activation='relu', input_shape=(150, 150, 3)),\n    layers.MaxPooling2D(2, 2),\n\n    layers.Conv2D(64, (3, 3), activation='relu'),\n    layers.MaxPooling2D(2, 2),\n\n    layers.Conv2D(128, (3, 3), activation='relu'),\n    layers.MaxPooling2D(2, 2),\n\n    layers.Flatten(),\n    layers.Dense(512, activation='relu'),\n    layers.Dropout(0.5),\n    layers.Dense(1, activation='sigmoid')\n])\n\nmodel.compile(\n    loss='binary_crossentropy',\n    optimizer=optimizers.Adam(learning_rate=1e-4),\n    metrics=['accuracy']\n)\n\n# Early stopping\nearly_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)\n\n# Training\nhistory = model.fit(\n    train_generator,\n    steps_per_epoch=train_generator.samples // batch_size,\n    validation_data=val_generator,\n    validation_steps=val_generator.samples // batch_size,\n    epochs=10,\n    callbacks=[early_stop]\n)\n\n# Plot Training Performance\ndef plot_history(hist):\n    plt.figure(figsize=(12, 4))\n    for i, metric in enumerate(['accuracy', 'loss']):\n        plt.subplot(1, 2, i+1)\n        plt.plot(hist.history[metric], label='Train')\n        plt.plot(hist.history[f'val_{metric}'], label='Val')\n        plt.title(f'Model {metric.title()}')\n        plt.xlabel('Epoch')\n        plt.ylabel(metric.title())\n        plt.legend()\n    plt.tight_layout()\n    plt.show()\n\nplot_history(history)\n\n# Generate Predictions\nsubmission = []\nfor fname in tqdm(sorted(os.listdir(test_path))):\n    if fname.endswith('.jpg'):\n        img_path = os.path.join(test_path, fname)\n        img = load_img(img_path, target_size=img_size)\n        img_array = img_to_array(img) / 255.0\n        img_array = np.expand_dims(img_array, axis=0)\n        pred = model.predict(img_array)[0][0]\n        submission.append((int(fname.split('.')[0]), pred))\n\n# Save Submission\nsubmission_df = pd.DataFrame(submission, columns=['id', 'label']).sort_values('id')\nsubmission_df.to_csv('MSBA.Session002.UjjwalKhanna_CNN.csv', index=False)  # Rename accordingly","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-22T23:55:00.578777Z","iopub.execute_input":"2025-04-22T23:55:00.579214Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Summary \n## Model 1: Custom Convolutional Neural Network (CNN)\n\nThis model is built from scratch and trained end-to-end. It serves as a lightweight yet surprisingly strong baseline for the binary classification task (dogs vs. cats).\n\n### Architecture\n- **Input**: 150×150 RGB images\n- **Layers**:\n  - `Conv2D (32) → MaxPooling`\n  - `Conv2D (64) → MaxPooling`\n  - `Conv2D (128) → MaxPooling`\n  - `Flatten → Dense(512) → Dropout(0.5) → Dense(1, sigmoid)`\n- **Optimizer**: Adam (`lr=1e-4`)\n- **Loss Function**: Binary Crossentropy\n- **Data Augmentation**: Rotation, shift, flip, zoom via `ImageDataGenerator`\n\n### Training Performance\n- Trained for **10 epochs** with `EarlyStopping`\n- Final **training accuracy**: 74.6%\n- Final **validation accuracy**: 75.0%\n- Best **validation accuracy** reached: **100%** (Epochs 6 & 8)\n- **Validation loss** improved from **0.5972 → 0.2670**\n- Some instability in training (accuracy drops every alternate epoch), suggesting variance or augmentation sensitivity\n","metadata":{}},{"cell_type":"markdown","source":"### ResNet50 for Dogs vs Cats","metadata":{}},{"cell_type":"code","source":"# ResNet50 for Dogs vs Cats - Clean 5 Epoch Version\n\nimport os\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom tqdm import tqdm\n\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input\nfrom tensorflow.keras import layers, models, optimizers, Input, Model\nfrom tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint\n\n# Configurations\nimg_size = (224, 224)\nbatch_size = 64\ntrain_path = '/kaggle/working/train_split'\ntest_path = '/kaggle/working/test/test'\n\n# Data Augmentation & Preprocessing\ntrain_datagen = ImageDataGenerator(\n    preprocessing_function=preprocess_input,\n    validation_split=0.2,\n    rotation_range=15,\n    width_shift_range=0.1,\n    height_shift_range=0.1,\n    horizontal_flip=True,\n    zoom_range=0.1\n)\n\ntrain_generator = train_datagen.flow_from_directory(\n    train_path,\n    target_size=img_size,\n    batch_size=batch_size,\n    class_mode='binary',\n    subset='training'\n)\n\nval_generator = train_datagen.flow_from_directory(\n    train_path,\n    target_size=img_size,\n    batch_size=batch_size,\n    class_mode='binary',\n    subset='validation'\n)\n\n# ResNet50 Architecture with Functional API\ninput_tensor = Input(shape=(224, 224, 3))\nbase_model = ResNet50(weights='imagenet', include_top=False, input_tensor=input_tensor)\nbase_model.trainable = False\n\nx = base_model.output\nx = layers.GlobalAveragePooling2D()(x)\nx = layers.Dense(512, activation='relu')(x)\nx = layers.Dropout(0.5)(x)\nx = layers.Dense(256, activation='relu')(x)\nx = layers.Dropout(0.3)(x)\nout = layers.Dense(1, activation='sigmoid')(x)\n\nmodel = Model(inputs=input_tensor, outputs=out)\nmodel.compile(\n    loss='binary_crossentropy',\n    optimizer=optimizers.Adam(learning_rate=0.0005),\n    metrics=['accuracy']\n)\n\n# Callbacks\nearly_stop = EarlyStopping(monitor='val_loss', patience=3, restore_best_weights=True)\ncheckpoint = ModelCheckpoint('best_model_resnet.keras', monitor='val_loss', save_best_only=True, verbose=1)\n\n# Train for 5 Epochs (Initial + Fine-Tuning Combined)\nmodel.fit(\n    train_generator,\n    steps_per_epoch=train_generator.samples // batch_size,\n    validation_data=val_generator,\n    validation_steps=val_generator.samples // batch_size,\n    epochs=5,\n    callbacks=[early_stop, checkpoint]\n)\n\n# Plot History (if needed)\ndef plot_history(hist):\n    plt.figure(figsize=(12, 4))\n    for i, metric in enumerate(['accuracy', 'loss']):\n        plt.subplot(1, 2, i+1)\n        plt.plot(hist.history[metric], label='Train')\n        plt.plot(hist.history[f'val_{metric}'], label='Val')\n        plt.title(f'Model {metric.title()}')\n        plt.xlabel('Epoch')\n        plt.ylabel(metric.title())\n        plt.legend()\n    plt.tight_layout()\n    plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-22T22:53:08.962435Z","iopub.execute_input":"2025-04-22T22:53:08.962704Z","iopub.status.idle":"2025-04-22T23:06:59.842974Z","shell.execute_reply.started":"2025-04-22T22:53:08.962685Z","shell.execute_reply":"2025-04-22T23:06:59.842027Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array\n# Inference on Test Set\nsubmission = []\nfor fname in tqdm(sorted(os.listdir(test_path))):\n    if fname.endswith('.jpg'):\n        img_path = os.path.join(test_path, fname)\n        img = load_img(img_path, target_size=img_size)\n        img_array = img_to_array(img)\n        img_array = preprocess_input(img_array)\n        img_array = np.expand_dims(img_array, axis=0)\n        pred = model.predict(img_array)[0][0]\n        submission.append((int(fname.split('.')[0]), pred))\n\n# Save Submission\nsubmission_df = pd.DataFrame(submission, columns=['id', 'label']).sort_values('id')\nsubmission_df.to_csv('MSBA.Session002.UjjwalKhanna_ResNet.csv', index=False)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-22T23:09:17.225358Z","iopub.execute_input":"2025-04-22T23:09:17.225601Z","iopub.status.idle":"2025-04-22T23:23:40.756375Z","shell.execute_reply.started":"2025-04-22T23:09:17.225585Z","shell.execute_reply":"2025-04-22T23:23:40.755622Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Summary \n## Model 2: Transfer Learning with ResNet50\n\nThis model uses a pretrained ResNet50 (trained on ImageNet) as a feature extractor. Only the custom dense layers on top are trainable, leveraging transfer learning to boost generalization.\n\n### Architecture\n- **Base**: ResNet50 (`include_top=False`, `weights='imagenet'`)\n- **Top Layers**:\n  - `GlobalAvgPooling → Dense(512) → Dropout(0.5) → Dense(256) → Dropout(0.3) → Dense(1, sigmoid)`\n- **Input**: 224×224 RGB images\n- **Normalization**: `preprocess_input()` (ResNet-specific)\n- **Optimizer**: Adam (`lr=0.0005`)\n- **Loss Function**: Binary Crossentropy\n\n### Training Performance\n- Trained for **5 epochs** with `EarlyStopping` and `ModelCheckpoint`\n- Final **training accuracy**: 98.5%\n- Final **validation accuracy**: 98.5%\n- Best **validation accuracy**: **100%** (Epochs 2 & 4)\n- **Validation loss** dropped significantly from **0.0476 → 0.0437**\n- Stable performance with little variance across epochs — a strong sign of effective transfer learning\n","metadata":{}},{"cell_type":"markdown","source":"## Conclusion\n\nBoth models demonstrated strong performance, but with distinct trade-offs:\n\n- The **custom CNN**, despite being simple and built from scratch, achieved high validation accuracy (~100% at times). However, its performance showed more fluctuation across epochs, suggesting a higher sensitivity to data augmentation and batch variance.\n\n- The **ResNet50 transfer learning model** offered **more consistent and reliable results**. With minimal training and fewer epochs, it quickly converged to high accuracy and extremely low validation loss, showcasing the value of pretrained features on limited datasets.\n\n### Key Takeaways:\n- Transfer learning (ResNet50) is highly effective when working with limited training time or data.\n- Custom CNNs can still perform competitively and serve as a great benchmark or lightweight deployment option.\n- Data augmentation, early stopping, and careful model checkpointing were critical for both approaches.\n\nFor deployment or leaderboard submission, **ResNet50** is the preferred choice due to its **stability, generalization, and speed of convergence**.\n","metadata":{}}]}