{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#### <center> AbdelRahman Rahal's</center>\n# <center style='margin: 0; padding: 0;'>Hackathon CNN Model</center>\nThis notebook shall serve as a submission and a guide for my CNN (Convolutional Neural Network) model designed to classify images of breast tumours into 3 categories: normal, benign, and malignant.\n\nThis model uses VGG-16 as its base, and a couple elegant dense layers as its head, making it lightweight and reliable. It's trained with optimization and quality in mind. As such, it can handle small and large datasets smoothly.\n\nLet's get started.","metadata":{}},{"cell_type":"markdown","source":"## Importing the libraries and modules needed 💿\n\n","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport os\nimport pandas as pd\nimport time\n\nimport tensorflow as tf\nfrom tensorflow.keras import Sequential\nfrom tensorflow.keras.applications.vgg16 import VGG16 # The base model\nfrom tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint\nfrom tensorflow.keras.layers import Flatten, Dense, Dropout\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.preprocessing import image_dataset_from_directory\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img, img_to_array","metadata":{"execution":{"iopub.status.busy":"2023-08-31T17:58:26.132083Z","iopub.execute_input":"2023-08-31T17:58:26.132469Z","iopub.status.idle":"2023-08-31T17:58:26.139214Z","shell.execute_reply.started":"2023-08-31T17:58:26.132438Z","shell.execute_reply":"2023-08-31T17:58:26.138059Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Setting up data augmentation 🎞️\nThis model uses a combination of carefully picked data augmentation techniques and strategies to boost its accuracy.","metadata":{}},{"cell_type":"code","source":"datagen = ImageDataGenerator(\n    rotation_range = 20,\n    zoom_range = 0.15,\n    width_shift_range = 0.2,\n    height_shift_range = 0.2,\n    shear_range = 0.15,\n    horizontal_flip = True,\n    fill_mode = \"nearest\",\n    rescale = 1./255,\n    validation_split = 0.2\n)\n\ndatagen_full = ImageDataGenerator(\n    rotation_range = 20,\n    zoom_range = 0.15,\n    width_shift_range = 0.2,\n    height_shift_range = 0.2,\n    shear_range = 0.15,\n    horizontal_flip = True,\n    fill_mode = \"nearest\",\n    rescale = 1./255\n)","metadata":{"execution":{"iopub.status.busy":"2023-08-31T17:37:34.133122Z","iopub.execute_input":"2023-08-31T17:37:34.133484Z","iopub.status.idle":"2023-08-31T17:37:34.139671Z","shell.execute_reply.started":"2023-08-31T17:37:34.133453Z","shell.execute_reply":"2023-08-31T17:37:34.138696Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Loading the datasets 📀\nBy utilising `ImageDataGenerator`'s `flow_from_directory()` method, we are able to efficiently load the datasets, while performing preprocessing and data augmentation on them. The enchanced dataset is now ready to fed to the model.","metadata":{}},{"cell_type":"code","source":"train_gen = datagen.flow_from_directory(\n    '../input/bioverse-hackathon-a/train/',\n    target_size = (256, 256),\n    batch_size = 32,\n    class_mode = 'categorical',\n    subset = 'training',\n    shuffle = True,\n    seed = 1\n)\n\nvalid_gen = datagen.flow_from_directory(\n    '../input/bioverse-hackathon-a/train/',\n    target_size = (256, 256),\n    batch_size = 64,    # A larger batch size helps assess the accuracy better\n    class_mode = 'categorical',\n    subset = 'validation',\n    seed = 1\n)\n\nfull_train_gen = datagen_full.flow_from_directory(\n    '../input/bioverse-hackathon-a/train/',\n    target_size = (256, 256),\n    batch_size = 32,\n    class_mode = 'categorical',\n    shuffle = True,\n    seed = 1\n)\n\ntest_images_dir = '../input/bioverse-hackathon-a/test/'\nimage_size = (256, 256)\nbatch_size = 64\n\ntest_dataset = tf.keras.preprocessing.image_dataset_from_directory(\n    test_images_dir,\n    label_mode = None,\n    image_size = image_size,\n    batch_size = batch_size\n)","metadata":{"execution":{"iopub.status.busy":"2023-08-31T17:37:35.526868Z","iopub.execute_input":"2023-08-31T17:37:35.527268Z","iopub.status.idle":"2023-08-31T17:37:35.809578Z","shell.execute_reply.started":"2023-08-31T17:37:35.527238Z","shell.execute_reply":"2023-08-31T17:37:35.808642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Creating the Model 🤖\nThe model creation is wrapped inside a function. This allows for the creation of multiple models—with same or different settings—that can work in parallel, though that is not covered in this notebook. It also allows for reusability, and code neatness and minimality.","metadata":{}},{"cell_type":"code","source":"def create_model(input_shape, num_classes):\n    base_model = VGG16(\n        include_top = False,\n        weights = 'imagenet',\n        input_shape = input_shape\n    )\n    base_model.trainable = False\n    \n    # Model definition\n    model = Sequential([\n        base_model,\n        Flatten(),\n        Dense(64, activation = 'relu'),\n        Dense(num_classes, activation = 'softmax')\n    ])\n    \n    # Model compilation\n    model.compile(\n        optimizer = Adam(),\n        loss = tf.keras.losses.CategoricalCrossentropy(),\n        metrics = ['accuracy']\n    )\n    \n    return model","metadata":{"execution":{"iopub.status.busy":"2023-08-31T17:37:38.343544Z","iopub.execute_input":"2023-08-31T17:37:38.343929Z","iopub.status.idle":"2023-08-31T17:37:38.351625Z","shell.execute_reply.started":"2023-08-31T17:37:38.343898Z","shell.execute_reply":"2023-08-31T17:37:38.350643Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Initialising the Model and Training It 🚀\nThe model is fed the desired parameters (the `input_shape`, and the `num_classes`) for the dataset and initialised.\n\nThe training for the model uses advanced strategies to ensure its speed, and quality, and minimising the training time as much as possible. That, combined with the now-enhanced dataset, it is sure to give reliable results.","metadata":{}},{"cell_type":"code","source":"# Model initialisation\nmodel = create_model((256, 256, 3), 3)\n\n# Callback definition\nearly_stopping = EarlyStopping(\n    monitor = 'val_loss',\n    patience = 5,\n    restore_best_weights = True\n)\n\n# Model training\nhistory = model.fit(\n    train_gen,\n    validation_data = valid_gen,\n    epochs = 30,\n    callbacks = [early_stopping],\n#     verbose = 0    # Mutes the training progress log. Comment or omit to see the progress\n)\n\n# Model evaluation\nloss, accuracy = model.evaluate(valid_gen)\nprint('Test accuracy :', accuracy)","metadata":{"execution":{"iopub.status.busy":"2023-08-31T17:39:56.470458Z","iopub.execute_input":"2023-08-31T17:39:56.470879Z","iopub.status.idle":"2023-08-31T17:39:56.880054Z","shell.execute_reply.started":"2023-08-31T17:39:56.470839Z","shell.execute_reply":"2023-08-31T17:39:56.879017Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualising the Magic 📉\nThe cell below should give you a good idea on the model training in the form of neat graphs.","metadata":{}},{"cell_type":"code","source":"history_frame = pd.DataFrame(history.history)\n\nhistory_frame.loc[:, ['loss', 'val_loss']].plot()\nhistory_frame.loc[:, ['accuracy', 'val_accuracy']].plot()","metadata":{"execution":{"iopub.status.busy":"2023-08-31T17:21:18.508725Z","iopub.execute_input":"2023-08-31T17:21:18.509010Z","iopub.status.idle":"2023-08-31T17:21:19.133470Z","shell.execute_reply.started":"2023-08-31T17:21:18.508984Z","shell.execute_reply":"2023-08-31T17:21:19.132524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Training the Model on the Full Dataset 💥\nNow that we are sure that the model is working properly, we can train it on the entire training set to make proper predictions.","metadata":{}},{"cell_type":"code","source":"model.fit(\n    full_train_gen,\n    epochs = 30,\n    callbacks = [early_stopping],\n#     verbose = 0    # Mutes the training progress log. Comment or omit to see the progress\n)","metadata":{"execution":{"iopub.status.busy":"2023-08-31T17:40:07.388595Z","iopub.execute_input":"2023-08-31T17:40:07.389300Z","iopub.status.idle":"2023-08-31T17:56:16.965909Z","shell.execute_reply.started":"2023-08-31T17:40:07.389259Z","shell.execute_reply":"2023-08-31T17:56:16.964878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Saving Predictions File 🗃️","metadata":{}},{"cell_type":"code","source":"predictions = model.predict(\n    test_dataset,\n    steps = len(test_dataset),\n    verbose = 1\n)\n# Convert predictions to class labels\npredicted_class_indices = np.argmax(predictions, axis = 1)\n\nlabels = (train_gen.class_indices)\nlabels = dict((v,k) for k,v in labels.items())\npredictions = [labels[k] for k in predicted_class_indices]\n\n# Get the filenames of the test dataset\nfilenames = [os.path.basename(file_path) for file_path in test_dataset.file_paths]\n\n# Use the filenames to create the results DataFrame\nresults = pd.DataFrame({\n    \"Filename\": filenames,\n    \"Predictions\": predictions\n})\n\nresults.to_csv(\"results.csv\", index=False)\n\n# Saving the model\nmodel.save('final_model.h5')","metadata":{"execution":{"iopub.status.busy":"2023-08-31T17:58:38.928284Z","iopub.execute_input":"2023-08-31T17:58:38.928704Z","iopub.status.idle":"2023-08-31T17:58:45.858583Z","shell.execute_reply.started":"2023-08-31T17:58:38.928667Z","shell.execute_reply":"2023-08-31T17:58:45.857567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Inference Time ⏲️\nWe can get the inference time by getting the prediction of an image from the test dataset and seeing how long it takes to make that prediction.","metadata":{}},{"cell_type":"code","source":"img = load_img(\n    '../input/bioverse-hackathon-a/test/BAHEYA_114876720430193_10000029.png',\n    target_size = (256, 256)\n)\nimg_array = img_to_array(img)\nimg_array = tf.expand_dims(img_array, 0)  # Create batch axis\nimg_array /= 255.  # Rescale the image\n\n\nstart_time = time.time()\npredictions = model.predict(img_array)\nend_time = time.time()\n\ninference_time = end_time - start_time\nprint('Inference time for one image: {:.3f} seconds'.format(inference_time))","metadata":{"execution":{"iopub.status.busy":"2023-08-31T17:58:52.447068Z","iopub.execute_input":"2023-08-31T17:58:52.447419Z","iopub.status.idle":"2023-08-31T17:58:53.199352Z","shell.execute_reply.started":"2023-08-31T17:58:52.447390Z","shell.execute_reply":"2023-08-31T17:58:53.198150Z"},"trusted":true},"execution_count":null,"outputs":[]}],"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}}