{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":66610,"databundleVersionId":7498878,"sourceType":"competition"}],"dockerImageVersionId":30636,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Settings","metadata":{"id":"vsUbKR0lQ3vd"}},{"cell_type":"markdown","source":"In this notebook we work with the problem of multi-class classification, that is, the correct assignment of an object to one of the classes. Our goal is to achieve high accuracy of assignment to the correct category. We will use the widely used transfer learning method.\n","metadata":{"id":"9u-iEr9kQ7FU"}},{"cell_type":"markdown","source":"Transfer learning is a machine learning technique that uses a pre-trained model. In this approach, a model trained on one data is used or retrained for use with another set of data. By using transfer learning, you can achieve significantly higher classification accuracy, especially when the available data set is limited.","metadata":{}},{"cell_type":"markdown","source":"Before running the code, we need to make sure that we are using the GPU. We plan to work with a convolutional neural network, and using a GPU allows us to reduce training time by several times.","metadata":{}},{"cell_type":"code","source":"# display information about available GPU devices\n!nvidia-smi","metadata":{"id":"oVSPUzXQ2acG","outputId":"b34dfb32-bc8e-4c11-8364-5c3841493e34","execution":{"iopub.status.busy":"2024-01-15T17:58:36.350407Z","iopub.execute_input":"2024-01-15T17:58:36.350721Z","iopub.status.idle":"2024-01-15T17:58:37.420246Z","shell.execute_reply.started":"2024-01-15T17:58:36.350673Z","shell.execute_reply":"2024-01-15T17:58:37.419113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Import libraries","metadata":{"id":"IbIAPBJODDAz"}},{"cell_type":"code","source":"!pip install split-folders\n!pip install optuna","metadata":{"execution":{"iopub.status.busy":"2024-01-15T17:58:37.422646Z","iopub.execute_input":"2024-01-15T17:58:37.423070Z","iopub.status.idle":"2024-01-15T17:59:02.864960Z","shell.execute_reply.started":"2024-01-15T17:58:37.423034Z","shell.execute_reply":"2024-01-15T17:59:02.863708Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport shutil\nimport splitfolders\nfrom pathlib import Path\nimport imghdr\nimport numpy as np\nfrom tensorboardX import SummaryWriter\nimport matplotlib.pyplot as plt \nimport random\nimport pandas as pd\nimport seaborn as sns\nimport tensorflow as tf\nfrom tensorflow import keras\nfrom sklearn.metrics import classification_report, confusion_matrix, roc_curve, auc, roc_auc_score","metadata":{"execution":{"iopub.status.busy":"2024-01-15T17:59:02.866479Z","iopub.execute_input":"2024-01-15T17:59:02.866815Z","iopub.status.idle":"2024-01-15T17:59:21.708937Z","shell.execute_reply.started":"2024-01-15T17:59:02.866787Z","shell.execute_reply":"2024-01-15T17:59:21.708125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Description of the data set","metadata":{}},{"cell_type":"markdown","source":"The set used has about 14,000 images of various types of waste, divided into 10 different categories - metal, glass, biological waste, etc.","metadata":{}},{"cell_type":"markdown","source":"\n# Data Preprcessing","metadata":{"id":"eEHoivmkDuxP"}},{"cell_type":"code","source":"data_dir = '/kaggle/input/garbage-guru-challenge'","metadata":{"execution":{"iopub.status.busy":"2024-01-15T17:59:21.710969Z","iopub.execute_input":"2024-01-15T17:59:21.711528Z","iopub.status.idle":"2024-01-15T17:59:21.715796Z","shell.execute_reply.started":"2024-01-15T17:59:21.711501Z","shell.execute_reply":"2024-01-15T17:59:21.714889Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Cleaning the working directory if there are any files in it.","metadata":{}},{"cell_type":"code","source":"def remove_folder_contents(folder):\n    for the_file in os.listdir(folder):\n        file_path = os.path.join(folder, the_file)\n        try:\n            if os.path.isfile(file_path):\n                os.unlink(file_path)\n            elif os.path.isdir(file_path):\n                remove_folder_contents(file_path)\n                os.rmdir(file_path)\n        except Exception as e:\n            print(e)\n\nfolder_path = '/kaggle/working'\nremove_folder_contents(folder_path)\n# os.rmdir(folder_path)","metadata":{"execution":{"iopub.status.busy":"2024-01-15T17:59:21.716868Z","iopub.execute_input":"2024-01-15T17:59:21.717153Z","iopub.status.idle":"2024-01-15T17:59:21.741987Z","shell.execute_reply.started":"2024-01-15T17:59:21.717129Z","shell.execute_reply":"2024-01-15T17:59:21.741025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Copying the dataset to the working directory to allow data cleaning","metadata":{}},{"cell_type":"code","source":"shutil.copytree(data_dir, '/kaggle/working/dataset')","metadata":{"execution":{"iopub.status.busy":"2024-01-15T17:59:21.743607Z","iopub.execute_input":"2024-01-15T17:59:21.744005Z","iopub.status.idle":"2024-01-15T17:59:58.432698Z","shell.execute_reply.started":"2024-01-15T17:59:21.743955Z","shell.execute_reply":"2024-01-15T17:59:58.431618Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_dir  = '/kaggle/working/dataset'","metadata":{"execution":{"iopub.status.busy":"2024-01-15T17:59:58.433982Z","iopub.execute_input":"2024-01-15T17:59:58.434317Z","iopub.status.idle":"2024-01-15T17:59:58.438877Z","shell.execute_reply.started":"2024-01-15T17:59:58.434289Z","shell.execute_reply":"2024-01-15T17:59:58.437708Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.chdir('/kaggle/working/')","metadata":{"execution":{"iopub.status.busy":"2024-01-15T17:59:58.440025Z","iopub.execute_input":"2024-01-15T17:59:58.440298Z","iopub.status.idle":"2024-01-15T17:59:58.448608Z","shell.execute_reply.started":"2024-01-15T17:59:58.440275Z","shell.execute_reply":"2024-01-15T17:59:58.447674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.getcwd()","metadata":{"execution":{"iopub.status.busy":"2024-01-15T17:59:58.449808Z","iopub.execute_input":"2024-01-15T17:59:58.450182Z","iopub.status.idle":"2024-01-15T17:59:58.459178Z","shell.execute_reply.started":"2024-01-15T17:59:58.450148Z","shell.execute_reply":"2024-01-15T17:59:58.458032Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's set a random value for reproducibility of the result. In particular, when splitting the dataset randomly (as in the following block of code), a fixed SEED value ensures that we get exactly the same split of data next time.","metadata":{}},{"cell_type":"code","source":"SEED = 42","metadata":{"id":"OnPP3rwT2Ved","execution":{"iopub.status.busy":"2024-01-15T17:59:58.463090Z","iopub.execute_input":"2024-01-15T17:59:58.463387Z","iopub.status.idle":"2024-01-15T17:59:58.467922Z","shell.execute_reply.started":"2024-01-15T17:59:58.463362Z","shell.execute_reply":"2024-01-15T17:59:58.466950Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Setting paths to training and test data","metadata":{}},{"cell_type":"code","source":"train_path = '/kaggle/working/dataset/train'\nval_path = '/kaggle/working/dataset/val'\ntest_path = '/kaggle/working/dataset/test'","metadata":{"id":"us5IR8qATvos","execution":{"iopub.status.busy":"2024-01-15T17:59:58.469115Z","iopub.execute_input":"2024-01-15T17:59:58.469636Z","iopub.status.idle":"2024-01-15T17:59:58.475104Z","shell.execute_reply.started":"2024-01-15T17:59:58.469601Z","shell.execute_reply":"2024-01-15T17:59:58.474188Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Visualization of data from a dataset","metadata":{"id":"O38W9BwME3Cs"}},{"cell_type":"code","source":"fig, axes = plt.subplots(2, 2, figsize = (5,5)) #выведем 4 изображения\naxes = axes.ravel() \n\nfor i in np.arange(0, 4): \n\n    category = random.choice(os.listdir(train_path)) #случайный выбор класса\n    class_dir = os.path.join(train_path, category)\n\n    image = random.choice(os.listdir(class_dir)) #cлучайный выбор изображения из класса\n \n    img = plt.imread(os.path.join(class_dir,image))\n    axes[i].imshow( img )\n    axes[i].set_title(category) \n    axes[i].axis('off')","metadata":{"id":"u6Jj3OTKU0qS","outputId":"45f10593-76da-4aed-8f01-b044eb6c02a2","execution":{"iopub.status.busy":"2024-01-15T17:59:58.476185Z","iopub.execute_input":"2024-01-15T17:59:58.476435Z","iopub.status.idle":"2024-01-15T17:59:59.010570Z","shell.execute_reply.started":"2024-01-15T17:59:58.476414Z","shell.execute_reply":"2024-01-15T17:59:59.009524Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's look at the distribution of data between classes.","metadata":{}},{"cell_type":"code","source":"total = 0\nfor category in os.listdir(train_path):\n    count= 0\n    for image in os.listdir(train_path + \"/\" + category):\n        count += 1\n        total +=1\n    print(str(category).title() + \": \" + str(count))  \nprint(f\"\\nTotal number of train images: {total}\")","metadata":{"id":"ULC2mFLyT7bh","outputId":"3b4cb6ef-9241-4bce-dc6a-cd5ef3a8619d","execution":{"iopub.status.busy":"2024-01-15T17:59:59.012215Z","iopub.execute_input":"2024-01-15T17:59:59.012575Z","iopub.status.idle":"2024-01-15T17:59:59.029200Z","shell.execute_reply.started":"2024-01-15T17:59:59.012539Z","shell.execute_reply":"2024-01-15T17:59:59.028282Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It is noticeable that in some classes the number of images differs several times. Let's visualize the distribution of classes for clarity.","metadata":{}},{"cell_type":"code","source":"# class names\nclass_names = sorted(os.listdir(train_path))\nclass_names","metadata":{"id":"D_QCfavht2A7","outputId":"e198fe3b-3f32-4700-ef5d-80b94abf98f6","execution":{"iopub.status.busy":"2024-01-15T17:59:59.030294Z","iopub.execute_input":"2024-01-15T17:59:59.030578Z","iopub.status.idle":"2024-01-15T17:59:59.037257Z","shell.execute_reply.started":"2024-01-15T17:59:59.030553Z","shell.execute_reply":"2024-01-15T17:59:59.036246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class_dis = [len(os.listdir(train_path + f\"/{name}\")) for name in class_names]\nclass_dis","metadata":{"id":"OZEESOk2tpJT","outputId":"e959ebe6-e840-4092-e0bb-87de21a813f2","execution":{"iopub.status.busy":"2024-01-15T17:59:59.038769Z","iopub.execute_input":"2024-01-15T17:59:59.039377Z","iopub.status.idle":"2024-01-15T17:59:59.052372Z","shell.execute_reply.started":"2024-01-15T17:59:59.039342Z","shell.execute_reply":"2024-01-15T17:59:59.051522Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"DF = pd.DataFrame(columns=['Class names','Count'])\nDF['Class names']=class_names\nDF['Count']=class_dis\nplt.figure(figsize=(7,4))\nax=sns.barplot(x='Class names', y='Count', data=DF)\nax.bar_label(ax.containers[0])\nax.set_xticklabels(ax.get_xticklabels(), rotation=90)\nplt.tight_layout()","metadata":{"id":"bA1spKdkukCw","outputId":"eee70baa-7b17-4b63-cc4a-f5a0ad5b3b5f","execution":{"iopub.status.busy":"2024-01-15T17:59:59.053426Z","iopub.execute_input":"2024-01-15T17:59:59.053718Z","iopub.status.idle":"2024-01-15T17:59:59.495146Z","shell.execute_reply.started":"2024-01-15T17:59:59.053692Z","shell.execute_reply":"2024-01-15T17:59:59.494153Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"A situation where one or more classes contain a significantly larger number of images is called class imbalance. In our situation, it is natural and determined by the specifics of the data. Thus, there are many more varieties of clothing or glass objects than batteries. However, this may negatively affect the training results of the model and cause it to be biased towards the largest class (that is, it will be much worse at classifying smaller classes). There are different approaches to solving this problem, the applicability of which depends on the problem being solved. We will try to solve this problem using augmentation methods that we integrate into the model.","metadata":{}},{"cell_type":"code","source":"data_path = '/kaggle/working/dataset'","metadata":{"execution":{"iopub.status.busy":"2024-01-15T17:59:59.496539Z","iopub.execute_input":"2024-01-15T17:59:59.497026Z","iopub.status.idle":"2024-01-15T17:59:59.501448Z","shell.execute_reply.started":"2024-01-15T17:59:59.496964Z","shell.execute_reply":"2024-01-15T17:59:59.500528Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To train a neural network, it is necessary to ensure unification of image sizes. To do this, let’s find the average width and height of images in the entire dataset.","metadata":{}},{"cell_type":"code","source":"width = []\nheight = []\nfrom PIL import Image\nimport numpy as np\n\nfor dirname in os.listdir(data_path):\n    for category in os.listdir(data_path + '/' + dirname):\n        for image in os.listdir(data_path + '/' + dirname + '/' + category):\n            img = Image.open(data_path + '/' + dirname + '/' + category + '/' + image)\n            width.append(img.width)\n            height.append(img.height)\n\nprint('Mean width: {:.4f}'.format(np.mean(width)))\nprint('Mean Height: {:.4f}'.format(np.mean(height)))","metadata":{"id":"yL3at6Tobf4t","outputId":"ec307c58-1fc4-44ba-c657-67033607b2dd","execution":{"iopub.status.busy":"2024-01-15T17:59:59.502649Z","iopub.execute_input":"2024-01-15T17:59:59.502997Z","iopub.status.idle":"2024-01-15T18:00:01.086075Z","shell.execute_reply.started":"2024-01-15T17:59:59.502962Z","shell.execute_reply":"2024-01-15T18:00:01.085141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's set the size of the input data (image size) taking into account the found average values","metadata":{}},{"cell_type":"code","source":"IMG_SIZE = (400, 400)","metadata":{"id":"YETLHhbPRPoL","execution":{"iopub.status.busy":"2024-01-15T18:00:01.087248Z","iopub.execute_input":"2024-01-15T18:00:01.087537Z","iopub.status.idle":"2024-01-15T18:00:01.091941Z","shell.execute_reply.started":"2024-01-15T18:00:01.087513Z","shell.execute_reply":"2024-01-15T18:00:01.090976Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's set the size of the data package for loading images into the model in parts","metadata":{}},{"cell_type":"code","source":"BATCH_SIZE = 32\n","metadata":{"id":"viL4vjPx3S1z","execution":{"iopub.status.busy":"2024-01-15T18:00:01.093129Z","iopub.execute_input":"2024-01-15T18:00:01.093411Z","iopub.status.idle":"2024-01-15T18:00:01.102773Z","shell.execute_reply.started":"2024-01-15T18:00:01.093388Z","shell.execute_reply":"2024-01-15T18:00:01.101897Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The data is divided  into three sets: train, test and val. Now we need to prepare them for working with the model - distribute them into packages and add labels to them. Data packets generated from the train folder, constituting 80% of the total number of images, will be used to train the model, from the val(10%) folder - for checking during training and adjusting parameters, and from the test folder - to evaluate the accuracy of the model on new data not used during training.\n","metadata":{"id":"ILLDQ8JGRw73"}},{"cell_type":"code","source":"train_data = tf.keras.utils.image_dataset_from_directory(train_path,\n                                                         image_size=IMG_SIZE,\n                                                         label_mode='categorical',\n                                                         batch_size=BATCH_SIZE,\n                                                         shuffle=True,\n                                                         seed=SEED)","metadata":{"id":"q9FfKNZt2IyR","outputId":"b62d7602-ea0c-480c-9333-022d75b9b3ff","execution":{"iopub.status.busy":"2024-01-15T18:00:01.103954Z","iopub.execute_input":"2024-01-15T18:00:01.104277Z","iopub.status.idle":"2024-01-15T18:00:02.578321Z","shell.execute_reply.started":"2024-01-15T18:00:01.104254Z","shell.execute_reply":"2024-01-15T18:00:02.577276Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_data = tf.keras.utils.image_dataset_from_directory(test_path,\n                                                        image_size=IMG_SIZE,\n                                                        label_mode='categorical',\n                                                        batch_size=BATCH_SIZE,\n                                                        shuffle=False)","metadata":{"id":"x7j2GWtsnQVc","outputId":"aac04b41-a5a6-4ec4-a205-25dceab66889","execution":{"iopub.status.busy":"2024-01-15T18:00:02.580009Z","iopub.execute_input":"2024-01-15T18:00:02.580425Z","iopub.status.idle":"2024-01-15T18:00:02.781920Z","shell.execute_reply.started":"2024-01-15T18:00:02.580388Z","shell.execute_reply":"2024-01-15T18:00:02.780905Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"validation_data = tf.keras.utils.image_dataset_from_directory(val_path,\n                                                              image_size=IMG_SIZE,\n                                                              label_mode='categorical',\n                                                              batch_size=BATCH_SIZE,\n                                                              shuffle=True,\n                                                              seed=SEED)","metadata":{"id":"8pOrlURenWTU","outputId":"c413bc4d-a62c-496d-dac0-a74851d5c9e3","execution":{"iopub.status.busy":"2024-01-15T18:00:02.783498Z","iopub.execute_input":"2024-01-15T18:00:02.783897Z","iopub.status.idle":"2024-01-15T18:00:03.405820Z","shell.execute_reply.started":"2024-01-15T18:00:02.783868Z","shell.execute_reply":"2024-01-15T18:00:03.404843Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Building the model","metadata":{"id":"I5aE_6F2GzbB"}},{"cell_type":"markdown","source":"When selecting a model, we examined the classification models available in Keras with pre-trained weights on the public ImageNet dataset, which includes about 1000 categories of various images (https://keras.io/api/applications/).\nFor our task, the most important parameter is the classification accuracy. Based on the accuracy values presented in the table (acc1, percentage of correct answers), the EfficientNetV2S model, which has 88 layers, was selected. Although the EfficientNetV2M and EfficientNetV2L models have higher accuracy, they have significantly higher weights.\n","metadata":{"id":"tCo5FeboGGYp"}},{"cell_type":"markdown","source":"Loading the model","metadata":{}},{"cell_type":"code","source":"base_model = tf.keras.applications.EfficientNetV2S(include_top=False,\n                                                   weights='imagenet', \n                                                   input_shape=(IMG_SIZE[0], IMG_SIZE[1], 3))","metadata":{"id":"yu4rU-XK3_GO","outputId":"9154f97a-04a4-41c3-a64d-f2f215c1fa4a","execution":{"iopub.status.busy":"2024-01-15T18:00:03.407178Z","iopub.execute_input":"2024-01-15T18:00:03.407901Z","iopub.status.idle":"2024-01-15T18:00:08.966241Z","shell.execute_reply.started":"2024-01-15T18:00:03.407862Z","shell.execute_reply":"2024-01-15T18:00:08.965286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's see what the architecture of the pretrained model looks like","metadata":{}},{"cell_type":"code","source":"base_model.summary()","metadata":{"id":"dZwuSdeL3_va","outputId":"aeebf4f6-1ba8-42f6-f34b-eb2e0fd73fa4","execution":{"iopub.status.busy":"2024-01-15T18:00:08.967525Z","iopub.execute_input":"2024-01-15T18:00:08.968032Z","iopub.status.idle":"2024-01-15T18:00:10.288759Z","shell.execute_reply.started":"2024-01-15T18:00:08.967994Z","shell.execute_reply":"2024-01-15T18:00:10.287779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The architecture of the model we are using already has a Rescale layer, so we may not include an additional image normalization layer in our model.ь.","metadata":{}},{"cell_type":"markdown","source":"Let's freeze all layers of the pretrained model so that its parameters do not change during training.","metadata":{}},{"cell_type":"code","source":"base_model.trainable = False","metadata":{"id":"dp1SuwIE4MST","execution":{"iopub.status.busy":"2024-01-15T18:00:10.290119Z","iopub.execute_input":"2024-01-15T18:00:10.290414Z","iopub.status.idle":"2024-01-15T18:00:10.314921Z","shell.execute_reply.started":"2024-01-15T18:00:10.290388Z","shell.execute_reply":"2024-01-15T18:00:10.313830Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's create a data augmentation layer to improve classification accuracy. This layer is active only during model training.","metadata":{"id":"IGC7aXMfS1AO"}},{"cell_type":"code","source":"data_augmentation = tf.keras.Sequential([tf.keras.layers.RandomFlip(\"horizontal\"),\n                                         tf.keras.layers.RandomRotation(0.2), \n                                         tf.keras.layers.RandomZoom(0.2),\n                                         tf.keras.layers.RandomHeight(0.2),\n                                         tf.keras.layers.RandomWidth(0.2),],\n                                         name =\"data_augmentation\")","metadata":{"id":"5szOnYzudMLZ","execution":{"iopub.status.busy":"2024-01-15T18:00:10.316306Z","iopub.execute_input":"2024-01-15T18:00:10.316771Z","iopub.status.idle":"2024-01-15T18:00:10.339683Z","shell.execute_reply.started":"2024-01-15T18:00:10.316718Z","shell.execute_reply":"2024-01-15T18:00:10.338384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's see how this layer works with images from the training set.","metadata":{}},{"cell_type":"code","source":"for image, _ in train_data.take(1):  \n    plt.figure(figsize=(5, 5))\n    first_image = image[0]\n    for i in range(9):\n        ax = plt.subplot(3, 3, i + 1)\n        augmented_image = data_augmentation(tf.expand_dims(first_image, 0))\n        plt.imshow(augmented_image[0] / 255)\n        plt.axis('off')","metadata":{"id":"bCeyDOQ-etzq","outputId":"2f74cbc2-b2fa-47e1-cf1b-20914430a91a","execution":{"iopub.status.busy":"2024-01-15T18:00:10.349834Z","iopub.execute_input":"2024-01-15T18:00:10.350562Z","iopub.status.idle":"2024-01-15T18:00:12.484545Z","shell.execute_reply.started":"2024-01-15T18:00:10.350523Z","shell.execute_reply":"2024-01-15T18:00:12.483641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's combine the model with the augmentation layer, and also add several layers:\n\n* GlobalAveragePooling2D layer, which calculates the arithmetic average over all channels to combine feature maps\n* A fully connected layer with 128 neurons and a Dropout layer, which excludes some neurons from the previous fully connected layer with a given probability, thereby reducing the possible effect of overfitting.\n* Output fully connected layer with the number of outputs corresponding to the number of classes of our data.","metadata":{}},{"cell_type":"code","source":"name=\"EfficientNetV2S\"\n\nEfficientNetV2S_model=tf.keras.Sequential([tf.keras.Input(shape=(None, None, 3), name=\"input_layer\"),\n                    data_augmentation,\n                    base_model,\n                    tf.keras.layers.GlobalAveragePooling2D(),\n                    tf.keras.layers.Dense(128, activation='relu'),\n                    tf.keras.layers.Dropout(0.2),\n                    tf.keras.layers.Dense(len(class_names), activation='softmax')\n                    ], name=name)","metadata":{"id":"J6Kqa_yWA3BC","execution":{"iopub.status.busy":"2024-01-15T18:00:12.485709Z","iopub.execute_input":"2024-01-15T18:00:12.486063Z","iopub.status.idle":"2024-01-15T18:00:15.122384Z","shell.execute_reply.started":"2024-01-15T18:00:12.486035Z","shell.execute_reply":"2024-01-15T18:00:15.121332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"For training, we will define a loss function, an optimizer, and a tracked accuracy metric","metadata":{}},{"cell_type":"code","source":"EfficientNetV2S_model.compile(loss='categorical_crossentropy',\n                              optimizer=tf.keras.optimizers.Adam(learning_rate = 0.001),\n                              metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:00:15.123713Z","iopub.execute_input":"2024-01-15T18:00:15.124087Z","iopub.status.idle":"2024-01-15T18:00:15.154817Z","shell.execute_reply.started":"2024-01-15T18:00:15.124057Z","shell.execute_reply":"2024-01-15T18:00:15.153971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's look at the structure of our model","metadata":{}},{"cell_type":"code","source":"EfficientNetV2S_model.summary()","metadata":{"id":"eK8W5Phq2fR6","outputId":"0cb37142-ca26-45b3-adfd-0a4359107625","execution":{"iopub.status.busy":"2024-01-15T18:00:15.156369Z","iopub.execute_input":"2024-01-15T18:00:15.156790Z","iopub.status.idle":"2024-01-15T18:00:15.262700Z","shell.execute_reply.started":"2024-01-15T18:00:15.156748Z","shell.execute_reply":"2024-01-15T18:00:15.261577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The summary already shows the advantage of using a pre-trained model - out of more than 20 million parameters, only 165 thousand are trainable, which will significantly reduce training time.","metadata":{}},{"cell_type":"markdown","source":"While training the models, we will use a callback function to stop training when the metrics being tracked stop improving. After this, we will save the trained model and weights for further use.","metadata":{}},{"cell_type":"code","source":"# Creating a Callback Function\ndef create_callback(model_name):\n    \n    # termination of training when accuracy does not improve within 3 epochs\n    early_stop = tf.keras.callbacks.EarlyStopping(patience=3,\n                                                monitor=\"val_loss\",\n                                                mode=\"min\",\n                                                verbose=1)\n\n    # decrease in learning rate when the error rate does not decrease over 2 epochs\n    reduce_learning_rate = tf.keras.callbacks.ReduceLROnPlateau(monitor=\"val_loss\",\n                                                              factor=0.2,\n                                                              patience=2,\n                                                              verbose=1,\n                                                              min_lr=0.00001)\n    # saving model\n    check_model = tf.keras.callbacks.ModelCheckpoint(model_name + \".h5\",\n                                                   monitor=\"val_accuracy\",\n                                                   mode=\"max\",\n                                                   save_best_only=True)\n    callback = [early_stop, reduce_learning_rate, check_model]\n    return callback","metadata":{"id":"ERnRJA_87r2t","execution":{"iopub.status.busy":"2024-01-15T18:00:15.264204Z","iopub.execute_input":"2024-01-15T18:00:15.264947Z","iopub.status.idle":"2024-01-15T18:00:15.272575Z","shell.execute_reply.started":"2024-01-15T18:00:15.264873Z","shell.execute_reply":"2024-01-15T18:00:15.271492Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"EfficientNetV2S_callback = create_callback(name)","metadata":{"id":"YqRNxjQh5EUx","execution":{"iopub.status.busy":"2024-01-15T18:00:15.273957Z","iopub.execute_input":"2024-01-15T18:00:15.274332Z","iopub.status.idle":"2024-01-15T18:00:15.281896Z","shell.execute_reply.started":"2024-01-15T18:00:15.274303Z","shell.execute_reply":"2024-01-15T18:00:15.280917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's set the number of training epochs","metadata":{}},{"cell_type":"code","source":"EPOCH=20","metadata":{"id":"JZgRKxTchBdT","execution":{"iopub.status.busy":"2024-01-15T18:00:15.283624Z","iopub.execute_input":"2024-01-15T18:00:15.284042Z","iopub.status.idle":"2024-01-15T18:00:15.292722Z","shell.execute_reply.started":"2024-01-15T18:00:15.284006Z","shell.execute_reply":"2024-01-15T18:00:15.291845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's train the model","metadata":{}},{"cell_type":"code","source":"import time\nstart_time = time.time()\nEfficientNetV2S_history = EfficientNetV2S_model.fit(train_data, \n                                                    epochs=EPOCH, \n                                                    steps_per_epoch=len(train_data),\n                                                    validation_data=validation_data, \n                                                    validation_steps=len(validation_data),\n                                                    callbacks=EfficientNetV2S_callback)","metadata":{"id":"5qIdQpt74x0d","outputId":"75f306a2-5fb9-49e0-8db9-091a2fba9de9","execution":{"iopub.status.busy":"2024-01-15T18:00:15.293898Z","iopub.execute_input":"2024-01-15T18:00:15.294196Z","iopub.status.idle":"2024-01-15T18:17:08.576998Z","shell.execute_reply.started":"2024-01-15T18:00:15.294170Z","shell.execute_reply":"2024-01-15T18:17:08.576132Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Based on the training history, you can evaluate the effectiveness of using our callback function - after reducing the learning speed, the errors also decreased.","metadata":{}},{"cell_type":"markdown","source":"Let's look at the model training schedules.","metadata":{}},{"cell_type":"code","source":"end_time = time.time()\ntraining_time = end_time - start_time\nprint(\"Total training time: {:.2f} seconds\".format(training_time))\nEfficientNetV2S_model.save(\"/kaggle/working/garbage-classification.h5\")","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:17:08.578650Z","iopub.execute_input":"2024-01-15T18:17:08.578978Z","iopub.status.idle":"2024-01-15T18:17:09.354986Z","shell.execute_reply.started":"2024-01-15T18:17:08.578951Z","shell.execute_reply":"2024-01-15T18:17:09.354162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"EfficientNetV2S_history.history.keys()","metadata":{"id":"ltAkXMJl_v0W","outputId":"83284b23-6363-49d5-9972-f5a9fcefbd47","execution":{"iopub.status.busy":"2024-01-15T18:17:09.356369Z","iopub.execute_input":"2024-01-15T18:17:09.356738Z","iopub.status.idle":"2024-01-15T18:17:09.362636Z","shell.execute_reply.started":"2024-01-15T18:17:09.356691Z","shell.execute_reply":"2024-01-15T18:17:09.361775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Function for displaying training graphs\ndef plot_loss_curves(history):\n\n    loss = history.history[\"loss\"]\n    val_loss = history.history[\"val_loss\"]\n    \n    accuracy = history.history[\"accuracy\"]\n    val_accuracy = history.history[\"val_accuracy\"]\n    \n    epochs = range(len(history.history[\"loss\"]))\n    \n    plt.plot(epochs, loss, label=\"training_loss\")\n    plt.plot(epochs, val_loss, label=\"val_loss\")\n    plt.title(\"Loss\")\n    plt.xlabel(\"Epochs\")\n    plt.legend()\n    \n    plt.figure()\n    plt.plot(epochs, accuracy, label=\"training_accuracy\")\n    plt.plot(epochs, val_accuracy, label=\"val_accuracy\")\n    plt.title(\"Accuracy\")\n    plt.xlabel(\"Epochs\")\n    plt.legend()","metadata":{"id":"hEigRh5ZCBaQ","execution":{"iopub.status.busy":"2024-01-15T18:17:09.363604Z","iopub.execute_input":"2024-01-15T18:17:09.363880Z","iopub.status.idle":"2024-01-15T18:17:09.372565Z","shell.execute_reply.started":"2024-01-15T18:17:09.363857Z","shell.execute_reply":"2024-01-15T18:17:09.371594Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_loss_curves(EfficientNetV2S_history)","metadata":{"id":"McOrDCmXARjd","execution":{"iopub.status.busy":"2024-01-15T18:17:09.373836Z","iopub.execute_input":"2024-01-15T18:17:09.374400Z","iopub.status.idle":"2024-01-15T18:17:10.047791Z","shell.execute_reply.started":"2024-01-15T18:17:09.374373Z","shell.execute_reply":"2024-01-15T18:17:10.046795Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The learning curves look a little jumpy, but the dynamics look pretty good, there is a decrease in learning losses and an increase in testing accuracy.","metadata":{}},{"cell_type":"markdown","source":"# Model Evaluation","metadata":{}},{"cell_type":"markdown","source":"We use a set of test data to evaluate the accuracy of the model on new data","metadata":{}},{"cell_type":"code","source":"test_loss, test_accuracy = EfficientNetV2S_model.evaluate(test_data, verbose=0)","metadata":{"id":"ysyA_N5-HB1l","execution":{"iopub.status.busy":"2024-01-15T18:17:10.048965Z","iopub.execute_input":"2024-01-15T18:17:10.049225Z","iopub.status.idle":"2024-01-15T18:17:39.369282Z","shell.execute_reply.started":"2024-01-15T18:17:10.049202Z","shell.execute_reply":"2024-01-15T18:17:39.368450Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Test Loss: {:.5f}\".format(test_loss))\nprint(\"Test Accuracy: {:.2f}%\".format(test_accuracy * 100))","metadata":{"id":"h9LbofxnHNlA","execution":{"iopub.status.busy":"2024-01-15T18:17:39.370676Z","iopub.execute_input":"2024-01-15T18:17:39.371358Z","iopub.status.idle":"2024-01-15T18:17:39.376750Z","shell.execute_reply.started":"2024-01-15T18:17:39.371322Z","shell.execute_reply":"2024-01-15T18:17:39.375771Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Given the limited data set, we obtained a good accuracy of 97%.","metadata":{}},{"cell_type":"markdown","source":"Let's look at the main classification metrics","metadata":{}},{"cell_type":"code","source":"pred_probs = EfficientNetV2S_model.predict(test_data, verbose=1)","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:17:39.378050Z","iopub.execute_input":"2024-01-15T18:17:39.378344Z","iopub.status.idle":"2024-01-15T18:18:09.718812Z","shell.execute_reply.started":"2024-01-15T18:17:39.378319Z","shell.execute_reply":"2024-01-15T18:18:09.717958Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_classes = pred_probs.argmax(axis=1)\npred_classes[:10]","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:18:09.720171Z","iopub.execute_input":"2024-01-15T18:18:09.720475Z","iopub.status.idle":"2024-01-15T18:18:09.728063Z","shell.execute_reply.started":"2024-01-15T18:18:09.720449Z","shell.execute_reply":"2024-01-15T18:18:09.727033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_labels = []\nfor images, labels in test_data.unbatch(): \n    y_labels.append(labels.numpy().argmax()) \ny_labels[:10] ","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:18:09.729285Z","iopub.execute_input":"2024-01-15T18:18:09.729626Z","iopub.status.idle":"2024-01-15T18:18:14.882371Z","shell.execute_reply.started":"2024-01-15T18:18:09.729598Z","shell.execute_reply":"2024-01-15T18:18:14.881288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since the images in the test dataset were not shuffled, all the output labels correspond to the same class.","metadata":{}},{"cell_type":"code","source":"print('Classification Report \\n')\ntarget_names = class_names\nprint(classification_report(y_labels, pred_classes, target_names=target_names))","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:18:14.884381Z","iopub.execute_input":"2024-01-15T18:18:14.884697Z","iopub.status.idle":"2024-01-15T18:18:14.904452Z","shell.execute_reply.started":"2024-01-15T18:18:14.884670Z","shell.execute_reply":"2024-01-15T18:18:14.903319Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* The “Precision” column shows the percentage of correct predictions among all predictions for a particular class.\n* The \"Recall\" column shows the percentage of images of a particular class that were predicted by the model to belong to that class.\n* Column “F1” shows what percentage of model predictions were correct.\n* The \"Support\" column shows how many images of each class were in the test dataset.","metadata":{}},{"cell_type":"markdown","source":"From the summary of metrics it is clear that the model classifies plastic worst of all, and best of all - things and various biological waste. However, despite the small data set, the model shows excellent performance in terms of Precision, Recall and F1 scores.","metadata":{}},{"cell_type":"markdown","source":"For submission create a csv file that calculates the predicted value","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import accuracy_score\nimport csv\n\n\n# Calculate scores\naccuracy = accuracy_score(y_labels, pred_classes)\n\n# Save predictions and scores in a CSV file without 'predicted_label'\nwith open('solutions.csv', 'w', newline='') as csvfile:\n    csv_writer = csv.writer(csvfile)\n    csv_writer.writerow(['id', 'label'])\n    \n    for i, (true_label, pred_label) in enumerate(zip(y_labels, pred_classes)):\n        csv_writer.writerow([i, pred_label])\n    ","metadata":{"execution":{"iopub.status.busy":"2024-01-15T19:10:04.886911Z","iopub.execute_input":"2024-01-15T19:10:04.887308Z","iopub.status.idle":"2024-01-15T19:10:04.904102Z","shell.execute_reply.started":"2024-01-15T19:10:04.887281Z","shell.execute_reply":"2024-01-15T19:10:04.903114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We visualize the error matrix to clearly see the correctness of class definitions","metadata":{}},{"cell_type":"code","source":"import itertools","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:18:14.906187Z","iopub.execute_input":"2024-01-15T18:18:14.906521Z","iopub.status.idle":"2024-01-15T18:18:14.910956Z","shell.execute_reply.started":"2024-01-15T18:18:14.906492Z","shell.execute_reply":"2024-01-15T18:18:14.909926Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_confusion_matrix(cm, classes):\n    plt.figure(figsize=(7,7))\n    plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)\n    plt.title('Confusion matrix')\n    plt.colorbar()\n    tick_marks = np.arange(len(classes))\n    plt.xticks(tick_marks, classes, rotation=45)\n    plt.yticks(tick_marks, classes)\n    cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n    cm = np.around(cm, decimals=2)\n    cm[np.isnan(cm)] = 0.0\n    thresh = cm.max() / 2.\n    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n        plt.text(j, i, cm[i, j],\n                 horizontalalignment=\"center\",\n                 color=\"white\" if cm[i, j] > thresh else \"black\")\n    plt.tight_layout()\n    plt.ylabel('True label')\n    plt.xlabel('Predicted label')","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:18:14.912270Z","iopub.execute_input":"2024-01-15T18:18:14.912702Z","iopub.status.idle":"2024-01-15T18:18:14.922312Z","shell.execute_reply.started":"2024-01-15T18:18:14.912668Z","shell.execute_reply":"2024-01-15T18:18:14.921438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cm = confusion_matrix(y_labels, pred_classes)\nplot_confusion_matrix(cm, class_names)","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:18:14.923645Z","iopub.execute_input":"2024-01-15T18:18:14.924073Z","iopub.status.idle":"2024-01-15T18:18:15.707958Z","shell.execute_reply.started":"2024-01-15T18:18:14.924027Z","shell.execute_reply":"2024-01-15T18:18:15.706933Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The rows of the matrix are responsible for the class predicted by the model, and the columns are for the actual class. From the error matrix, it is noticeable that in general the model’s errors are very small, although it sometimes incorrectly predicts categories, for example, it confuses plastic and glass.","metadata":{}},{"cell_type":"markdown","source":"Visualizing several model predictions","metadata":{}},{"cell_type":"code","source":"def pred_random_images(model_name, folder_path, class_names):\n    plt.figure(figsize=(17, 10))\n    for i in range(3):\n        class_name = random.choice(class_names)\n        filename = random.choice(os.listdir(folder_path + \"/\" + class_name))\n        filepath = folder_path + \"/\" + class_name + \"/\" + filename\n    \n        img = tf.io.read_file(filepath)\n        img = tf.image.decode_jpeg(img)\n        img = tf.image.resize(img, [IMG_SIZE[0], IMG_SIZE[1]])\n     #  img = img/255.\n        pred_prob = model_name.predict(tf.expand_dims(img, axis=0), verbose=0) \n        pred_class = class_names[pred_prob.argmax()] \n\n  # Plot the image(s)\n        plt.subplot(1, 3, i+1)\n        plt.imshow(img/255.)\n        if class_name == pred_class: \n            title_color = \"g\"\n        else:\n            title_color = \"r\"\n        plt.title(f\"Class: {class_name},\\n Pred: {pred_class},\\n Prob: {pred_prob.max():.2f}\", c=title_color)\n        plt.axis(False);","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:18:15.709250Z","iopub.execute_input":"2024-01-15T18:18:15.709556Z","iopub.status.idle":"2024-01-15T18:18:15.718934Z","shell.execute_reply.started":"2024-01-15T18:18:15.709530Z","shell.execute_reply":"2024-01-15T18:18:15.717604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_random_images(EfficientNetV2S_model, test_path, class_names)","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:18:15.720272Z","iopub.execute_input":"2024-01-15T18:18:15.720589Z","iopub.status.idle":"2024-01-15T18:18:20.830420Z","shell.execute_reply.started":"2024-01-15T18:18:15.720563Z","shell.execute_reply":"2024-01-15T18:18:20.829448Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pickle\npickle.dump(EfficientNetV2S_model,open(\"EfficientNetV2S_model.h5\",\"wb\"))\n# EfficientNetV2S_model.save('EfficientNetV2S_model.h5')","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:18:20.831877Z","iopub.execute_input":"2024-01-15T18:18:20.832351Z","iopub.status.idle":"2024-01-15T18:18:23.373355Z","shell.execute_reply.started":"2024-01-15T18:18:20.832317Z","shell.execute_reply":"2024-01-15T18:18:23.372520Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ls ../working","metadata":{"execution":{"iopub.status.busy":"2024-01-15T18:18:23.374561Z","iopub.execute_input":"2024-01-15T18:18:23.375189Z","iopub.status.idle":"2024-01-15T18:18:24.453063Z","shell.execute_reply.started":"2024-01-15T18:18:23.375152Z","shell.execute_reply":"2024-01-15T18:18:24.452039Z"},"trusted":true},"execution_count":null,"outputs":[]}]}