{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":3649,"databundleVersionId":46718,"sourceType":"competition"}],"dockerImageVersionId":30698,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **Image Classification (CIFAR-10)**\n---","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2024-05-28T20:51:07.558972Z","iopub.execute_input":"2024-05-28T20:51:07.559390Z","iopub.status.idle":"2024-05-28T20:51:08.487412Z","shell.execute_reply.started":"2024-05-28T20:51:07.559358Z","shell.execute_reply":"2024-05-28T20:51:08.486302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Useful Information\n---\nMake sure to check the competition [link](https://www.kaggle.com/c/cifar-10) for full details. Also make sure to check out this useful [tutorial](https://d2l.ai/chapter_computer-vision/kaggle-cifar10.html) for a good starting point.","metadata":{}},{"cell_type":"code","source":"!pip install d2l==1.0.3","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:57:47.402176Z","iopub.execute_input":"2024-05-28T20:57:47.403011Z","iopub.status.idle":"2024-05-28T20:58:31.181684Z","shell.execute_reply.started":"2024-05-28T20:57:47.402978Z","shell.execute_reply":"2024-05-28T20:58:31.180432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install py7zr","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:51:13.234701Z","iopub.execute_input":"2024-05-28T20:51:13.234979Z","iopub.status.idle":"2024-05-28T20:51:38.738817Z","shell.execute_reply.started":"2024-05-28T20:51:13.234956Z","shell.execute_reply":"2024-05-28T20:51:38.737904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import collections\nimport math\nimport os\n\nimport shutil\nimport py7zr\nfrom py7zr import unpack_7zarchive\n\n\nimport torch\nimport torchvision\nfrom torch import nn\nfrom d2l import torch as d2l\n","metadata":{"execution":{"iopub.status.busy":"2024-05-28T21:01:52.218695Z","iopub.execute_input":"2024-05-28T21:01:52.219099Z","iopub.status.idle":"2024-05-28T21:01:52.224354Z","shell.execute_reply.started":"2024-05-28T21:01:52.219059Z","shell.execute_reply":"2024-05-28T21:01:52.223317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n<a id=\"1\"></a>\n# **Dataset Retrieval & Organization**\n\nThe competition dataset is divided into\na training set and a test set,\nwhich contain 50000 and 300000 images, respectively.\nIn the test set,\n10000 images will be used for evaluation,\nwhile the remaining 290000 images will not\nbe evaluated:\nthey are included just\nto make it hard\nto cheat with\n*manually* labeled results of the test set.\nThe images in this dataset\nare all png color (RGB channels) image files,\nwhose height and width are both 32 pixels.\nThe images cover a total of 10 categories, namely airplanes, cars, birds, cats, deer, dogs, frogs, horses, boats, and trucks.","metadata":{}},{"cell_type":"markdown","source":"---\n<a id=\"101\"></a>\n## Download the Dataset\n\nAfter logging in to Kaggle, we can click the \"Data\" tab on the CIFAR-10 image classification competition webpage shown in :numref:`fig_kaggle_cifar10` and download the dataset by clicking the \"Download All\" button.\nAfter unzipping the downloaded file in `../data`, and unzipping `train.7z` and `test.7z` inside it, you will find the entire dataset in the following paths:\n\n* `../data/cifar-10/train/[1-50000].png`\n* `../data/cifar-10/test/[1-300000].png`\n* `../data/cifar-10/trainLabels.csv`\n* `../data/cifar-10/sampleSubmission.csv`\n\nwhere the `train` and `test` directories contain the training and testing images, respectively, `trainLabels.csv` provides labels for the training images, and `sample_submission.csv` is a sample submission file.\n\nTo make it easier to get started, **we provide a small-scale sample of the dataset that\ncontains the first 1000 training images and 5 random testing images.**\nTo use the full dataset of the Kaggle competition, you need to set the following `demo` variable to `False`.\n","metadata":{}},{"cell_type":"code","source":"#@save\n# d2l.DATA_HUB['cifar10_tiny'] = (d2l.DATA_URL + 'kaggle_cifar10_tiny.zip',\n#                                 '2068874e4b9a9f0fb07ebe0ad2b29754449ccacd')\n\n# If you use the full dataset downloaded for the Kaggle competition, set\n# `demo` to False\ndemo = False\n\nif demo:\n    data_dir = d2l.download_extract('cifar10_tiny')\nelse:\n    data_dir = '/kaggle/working/'\n","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:52:05.278880Z","iopub.execute_input":"2024-05-28T20:52:05.279597Z","iopub.status.idle":"2024-05-28T20:52:05.283999Z","shell.execute_reply.started":"2024-05-28T20:52:05.279567Z","shell.execute_reply":"2024-05-28T20:52:05.283098Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"shutil.register_unpack_format('7zip',['.7z'],unpack_7zarchive)\nshutil.unpack_archive('/kaggle/input/cifar-10/train.7z', '/kaggle/working/')","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:52:13.468532Z","iopub.execute_input":"2024-05-28T20:52:13.469389Z","iopub.status.idle":"2024-05-28T20:52:38.817613Z","shell.execute_reply.started":"2024-05-28T20:52:13.469356Z","shell.execute_reply":"2024-05-28T20:52:38.816796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# with py7zr.SevenZipFile('/kaggle/input/cifar-10/train.7z', mode='r') as z:\n#     z.extractall()","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:52:38.819381Z","iopub.execute_input":"2024-05-28T20:52:38.819813Z","iopub.status.idle":"2024-05-28T20:52:38.823836Z","shell.execute_reply.started":"2024-05-28T20:52:38.819774Z","shell.execute_reply":"2024-05-28T20:52:38.822979Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"shutil.unpack_archive('/kaggle/input/cifar-10/test.7z','/kaggle/working/')","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:52:38.825033Z","iopub.execute_input":"2024-05-28T20:52:38.825310Z","iopub.status.idle":"2024-05-28T20:55:37.145103Z","shell.execute_reply.started":"2024-05-28T20:52:38.825287Z","shell.execute_reply":"2024-05-28T20:55:37.144307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# with py7zr.SevenZipFile('/kaggle/input/cifar-10/test.7z', mode='r') as z:\n#     z.extractall()","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:55:37.147031Z","iopub.execute_input":"2024-05-28T20:55:37.147494Z","iopub.status.idle":"2024-05-28T20:55:37.151582Z","shell.execute_reply.started":"2024-05-28T20:55:37.147461Z","shell.execute_reply":"2024-05-28T20:55:37.150703Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n<a id=\"102\"></a>\n## Organize the Dataset\n\nWe need to organize datasets to facilitate model training and testing. Let's first read the labels from the csv file. The following function returns a dictionary that maps the non-extension part of the filename to its label..\n","metadata":{}},{"cell_type":"code","source":"#@save\ndef read_csv_labels(fname):\n    \"\"\"Read `fname` to return a filename to label dictionary.\"\"\"\n    with open(fname, 'r') as f:\n        # Skip the file header line (column name)\n        lines = f.readlines()[1:]\n    tokens = [l.rstrip().split(',') for l in lines]\n    return dict(((name, label) for name, label in tokens))\n\n#labels = read_csv_labels(os.path.join(data_dir, 'trainLabels.csv'))\nlabels = read_csv_labels('/kaggle/input/cifar-10/trainLabels.csv')\nprint('# training examples:', len(labels))\nprint('# classes:', len(set(labels.values())))","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:55:40.671218Z","iopub.execute_input":"2024-05-28T20:55:40.671601Z","iopub.status.idle":"2024-05-28T20:55:40.751936Z","shell.execute_reply.started":"2024-05-28T20:55:40.671571Z","shell.execute_reply":"2024-05-28T20:55:40.750824Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next, we define the `reorg_train_valid` function to **split the validation set out of the original training set.**\nThe argument `valid_ratio` in this function is the ratio of the number of examples in the validation set to the number of examples in the original training set.\nMore concretely,\nlet $n$ be the number of images of the class with the least examples, and $r$ be the ratio.\nThe validation set will split out\n$\\max(\\lfloor nr\\rfloor,1)$ images for each class.\nLet's use `valid_ratio=0.1` as an example. Since the original training set has 50000 images,\nthere will be 45000 images used for training in the path `train_valid_test/train`,\nwhile the other 5000 images will be split out\nas validation set in the path `train_valid_test/valid`. After organizing the dataset, images of the same class will be placed under the same folder.\n","metadata":{}},{"cell_type":"code","source":"#@save\ndef copyfile(filename, target_dir):\n    \"\"\"Copy a file into a target directory.\"\"\"\n    os.makedirs(target_dir, exist_ok=True)\n    shutil.copy(filename, target_dir)\n\n#@save\ndef reorg_train_valid(data_dir, labels, valid_ratio):\n    \"\"\"Split the validation set out of the original training set.\"\"\"\n    # The number of examples of the class that has the fewest examples in the\n    # training dataset\n    n = collections.Counter(labels.values()).most_common()[-1][1]\n    # The number of examples per class for the validation set\n    n_valid_per_label = max(1, math.floor(n * valid_ratio))\n    label_count = {}\n    for train_file in os.listdir(os.path.join(data_dir, 'train')):\n        label = labels[train_file.split('.')[0]]\n        fname = os.path.join(data_dir, 'train', train_file)\n        copyfile(fname, os.path.join(data_dir, 'train_valid_test',\n                                     'train_valid', label))\n        if label not in label_count or label_count[label] < n_valid_per_label:\n            copyfile(fname, os.path.join(data_dir, 'train_valid_test',\n                                         'valid', label))\n            label_count[label] = label_count.get(label, 0) + 1\n        else:\n            copyfile(fname, os.path.join(data_dir, 'train_valid_test',\n                                         'train', label))\n    return n_valid_per_label","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:55:41.784043Z","iopub.execute_input":"2024-05-28T20:55:41.784382Z","iopub.status.idle":"2024-05-28T20:55:41.793360Z","shell.execute_reply.started":"2024-05-28T20:55:41.784356Z","shell.execute_reply":"2024-05-28T20:55:41.792449Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The `reorg_test` function below **organizes the testing set for data loading during prediction.**\n","metadata":{}},{"cell_type":"code","source":"#@save\ndef reorg_test(data_dir):\n    \"\"\"Organize the testing set for data loading during prediction.\"\"\"\n    for test_file in os.listdir(os.path.join(data_dir, 'test')):\n        copyfile(os.path.join(data_dir, 'test', test_file),\n                 os.path.join(data_dir, 'train_valid_test', 'test',\n                              'unknown'))","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:55:42.970921Z","iopub.execute_input":"2024-05-28T20:55:42.971302Z","iopub.status.idle":"2024-05-28T20:55:42.976611Z","shell.execute_reply.started":"2024-05-28T20:55:42.971265Z","shell.execute_reply":"2024-05-28T20:55:42.975711Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Finally, we use a function to **invoke** the `read_csv_labels`, `reorg_train_valid`, and `reorg_test` (**functions defined above.**)","metadata":{}},{"cell_type":"code","source":"def reorg_cifar10_data(data_dir, valid_ratio):\n    #labels = read_csv_labels(os.path.join(data_dir, 'trainLabels.csv'))\n    labels = read_csv_labels('/kaggle/input/cifar-10/trainLabels.csv')\n    reorg_train_valid(data_dir, labels, valid_ratio)\n    reorg_test(data_dir)","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:55:43.920228Z","iopub.execute_input":"2024-05-28T20:55:43.920618Z","iopub.status.idle":"2024-05-28T20:55:43.925401Z","shell.execute_reply.started":"2024-05-28T20:55:43.920589Z","shell.execute_reply":"2024-05-28T20:55:43.924446Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here we only set the batch size to 32 for the small-scale sample of the dataset.\nWhen training and testing\nthe complete dataset of the Kaggle competition,\n`batch_size` should be set to a larger integer, such as 128.\nWe split out 10% of the training examples as the validation set for tuning hyperparameters.\n","metadata":{}},{"cell_type":"code","source":"\n__train_dir = \"/kaggle/working/train\"\n__num_files = len(os.listdir(__train_dir))\nprint(f\"There are {__num_files} files in the {__train_dir} directory.\")\n__test_dir = \"/kaggle/working/test\"\n__num_files2 = len(os.listdir(__test_dir))\nprint(f\"There are {__num_files2} files in the {__test_dir} directory.\")","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:55:46.176940Z","iopub.execute_input":"2024-05-28T20:55:46.177431Z","iopub.status.idle":"2024-05-28T20:55:46.421763Z","shell.execute_reply.started":"2024-05-28T20:55:46.177394Z","shell.execute_reply":"2024-05-28T20:55:46.420562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def dir_file_type(dir_):\n    file_types = set()\n    for filename in os.listdir(dir_):\n        file_path = os.path.join(dir_, filename)\n        if os.path.isfile(file_path):\n            file_extension = os.path.splitext(filename)[1][1:].lower()\n            file_types.add(file_extension)\n\n    print(f\"File types in {dir_}:\")\n    for file_type in file_types:\n        print(f\"- {file_type}\")\n        \n        \ndir_file_type(__train_dir)\ndir_file_type(__test_dir)","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:55:48.041717Z","iopub.execute_input":"2024-05-28T20:55:48.042386Z","iopub.status.idle":"2024-05-28T20:55:51.401541Z","shell.execute_reply.started":"2024-05-28T20:55:48.042351Z","shell.execute_reply":"2024-05-28T20:55:51.400633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"batch_size = 32 if demo else 128\nvalid_ratio = 0.1\nreorg_cifar10_data(data_dir, valid_ratio)\n","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:55:51.403088Z","iopub.execute_input":"2024-05-28T20:55:51.403393Z","iopub.status.idle":"2024-05-28T20:56:52.270204Z","shell.execute_reply.started":"2024-05-28T20:55:51.403369Z","shell.execute_reply":"2024-05-28T20:56:52.269010Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n<a id=\"2\"></a>\n# **Image Augmentation**\n\nWe use **image augmentation to address overfitting.**\nFor example, images can be flipped horizontally at random during training.\nWe can also perform standardization for the three RGB channels of color images. Below lists some of these operations that you can tweak.\n","metadata":{}},{"cell_type":"code","source":"transform_train = torchvision.transforms.Compose([\n    # Scale the image up to a square of 40 pixels in both height and width\n    torchvision.transforms.Resize(40),\n    # Randomly crop a square image of 40 pixels in both height and width to\n    # produce a small square of 0.64 to 1 times the area of the original\n    # image, and then scale it to a square of 32 pixels in both height and\n    # width\n    torchvision.transforms.RandomResizedCrop(32, scale=(0.64, 1.0),\n                                                   ratio=(1.0, 1.0)),\n    torchvision.transforms.RandomHorizontalFlip(),\n    torchvision.transforms.ToTensor(),\n    # Standardize each channel of the image\n    torchvision.transforms.Normalize([0.4914, 0.4822, 0.4465],\n                                     [0.2023, 0.1994, 0.2010])])","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:57:05.931115Z","iopub.execute_input":"2024-05-28T20:57:05.931928Z","iopub.status.idle":"2024-05-28T20:57:05.938219Z","shell.execute_reply.started":"2024-05-28T20:57:05.931893Z","shell.execute_reply":"2024-05-28T20:57:05.937281Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"During testing, we only perform standardization on images so as to remove randomness in the evaluation results.\n","metadata":{}},{"cell_type":"code","source":"transform_test = torchvision.transforms.Compose([\n    torchvision.transforms.ToTensor(),\n    torchvision.transforms.Normalize([0.4914, 0.4822, 0.4465],\n                                     [0.2023, 0.1994, 0.2010])])","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:57:07.472567Z","iopub.execute_input":"2024-05-28T20:57:07.472928Z","iopub.status.idle":"2024-05-28T20:57:07.478029Z","shell.execute_reply.started":"2024-05-28T20:57:07.472893Z","shell.execute_reply":"2024-05-28T20:57:07.477122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n<a id=\"3\"></a>\n# Reading the Dataset\n\nNext, we **read the organized dataset consisting of raw image files**. Each example includes an image and a label.\n","metadata":{}},{"cell_type":"code","source":"train_ds, train_valid_ds = [torchvision.datasets.ImageFolder(\n    os.path.join(data_dir, 'train_valid_test', folder),\n    transform=transform_train) for folder in ['train', 'train_valid']]\n\nvalid_ds, test_ds = [torchvision.datasets.ImageFolder(\n    os.path.join(data_dir, 'train_valid_test', folder),\n    transform=transform_test) for folder in ['valid', 'test']]","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:57:09.071610Z","iopub.execute_input":"2024-05-28T20:57:09.072306Z","iopub.status.idle":"2024-05-28T20:57:10.960260Z","shell.execute_reply.started":"2024-05-28T20:57:09.072269Z","shell.execute_reply":"2024-05-28T20:57:10.959489Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"During training, we need to **specify all the image augmentation operations defined above**. When the validation set is used for model evaluation during hyperparameter tuning, no randomness from image augmentation should be introduced. Before final prediction,we train the model on the combined training set and validation set to make full use of all the labeled data.\n","metadata":{}},{"cell_type":"code","source":"train_iter, train_valid_iter = [torch.utils.data.DataLoader(\n    dataset, batch_size, shuffle=True, drop_last=True)\n    for dataset in (train_ds, train_valid_ds)]\n\nvalid_iter = torch.utils.data.DataLoader(valid_ds, batch_size, shuffle=False,\n                                         drop_last=True)\n\ntest_iter = torch.utils.data.DataLoader(test_ds, batch_size, shuffle=False,\n                                        drop_last=False)","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:57:11.716952Z","iopub.execute_input":"2024-05-28T20:57:11.717315Z","iopub.status.idle":"2024-05-28T20:57:11.724515Z","shell.execute_reply.started":"2024-05-28T20:57:11.717288Z","shell.execute_reply":"2024-05-28T20:57:11.723572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n<a id=\"4\"></a>\n# Defining the **Model**\nWe define the ResNet-18 model.\n","metadata":{}},{"cell_type":"code","source":"def get_net():\n    num_classes = 10\n    net = d2l.resnet18(num_classes, 3)\n    return net\n\nloss = nn.CrossEntropyLoss(reduction=\"none\")","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:57:16.061158Z","iopub.execute_input":"2024-05-28T20:57:16.062020Z","iopub.status.idle":"2024-05-28T20:57:16.066417Z","shell.execute_reply.started":"2024-05-28T20:57:16.061987Z","shell.execute_reply":"2024-05-28T20:57:16.065424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n<a id=\"5\"></a>\n# Defining the **Training Function**\n\nWe will select models and tune hyperparameters according to the model's performance on the validation set.\nIn the following, we define the model training function `train`.\n","metadata":{}},{"cell_type":"code","source":"def train(net, train_iter, valid_iter, num_epochs, lr, wd, devices, lr_period,\n          lr_decay):\n    trainer = torch.optim.SGD(net.parameters(), lr=lr, momentum=0.9,\n                              weight_decay=wd)\n    scheduler = torch.optim.lr_scheduler.StepLR(trainer, lr_period, lr_decay)\n    num_batches, timer = len(train_iter), d2l.Timer()\n    legend = ['train loss', 'train acc']\n    if valid_iter is not None:\n        legend.append('valid acc')\n    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n                            legend=legend)\n    net = nn.DataParallel(net, device_ids=devices).to(devices[0])\n    for epoch in range(num_epochs):\n        net.train()\n        metric = d2l.Accumulator(3)\n        for i, (features, labels) in enumerate(train_iter):\n            timer.start()\n            l, acc = d2l.train_batch_ch13(net, features, labels,\n                                          loss, trainer, devices)\n            metric.add(l, acc, labels.shape[0])\n            timer.stop()\n            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n                animator.add(epoch + (i + 1) / num_batches,\n                             (metric[0] / metric[2], metric[1] / metric[2],\n                              None))\n        if valid_iter is not None:\n            valid_acc = d2l.evaluate_accuracy_gpu(net, valid_iter)\n            animator.add(epoch + 1, (None, None, valid_acc))\n        scheduler.step()\n    measures = (f'train loss {metric[0] / metric[2]:.3f}, '\n                f'train acc {metric[1] / metric[2]:.3f}')\n    if valid_iter is not None:\n        measures += f', valid acc {valid_acc:.3f}'\n    print(measures + f'\\n{metric[2] * num_epochs / timer.sum():.1f}'\n          f' examples/sec on {str(devices)}')","metadata":{"execution":{"iopub.status.busy":"2024-05-28T20:57:17.324764Z","iopub.execute_input":"2024-05-28T20:57:17.325486Z","iopub.status.idle":"2024-05-28T20:57:17.338418Z","shell.execute_reply.started":"2024-05-28T20:57:17.325443Z","shell.execute_reply":"2024-05-28T20:57:17.337417Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"----\n<a id=\"6\"></a>\n# **Training and Validating the Model**\n\nNow, we can train and validate the model.\nAll the following hyperparameters can be tuned.\nFor example, we can increase the number of epochs.\nWhen `lr_period` and `lr_decay` are set to 4 and 0.9, respectively, the learning rate of the optimization algorithm will be multiplied by 0.9 after every 4 epochs. Just for ease of demonstration,\nwe only train 20 epochs here.","metadata":{}},{"cell_type":"code","source":"print(torch.cuda.is_available())\nprint(torch.cuda.device_count())\nlen(d2l.try_all_gpus())","metadata":{"execution":{"iopub.status.busy":"2024-05-28T21:02:00.229310Z","iopub.execute_input":"2024-05-28T21:02:00.230176Z","iopub.status.idle":"2024-05-28T21:02:00.237896Z","shell.execute_reply.started":"2024-05-28T21:02:00.230142Z","shell.execute_reply":"2024-05-28T21:02:00.236919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"devices, num_epochs, lr, wd = d2l.try_all_gpus(), 20, 2e-4, 5e-4\nlr_period, lr_decay = 4, 0.9\n\nif len(devices) == 0:\n    devices = [torch.device('cpu')]\n    \nnet = get_net()\nlr_period, lr_decay, net = 4, 0.9, get_net()\nnet(next(iter(train_iter))[0])\ntrain(net, train_iter, valid_iter, num_epochs, lr, wd, devices, lr_period,\n      lr_decay)","metadata":{"execution":{"iopub.status.busy":"2024-05-28T21:02:01.316580Z","iopub.execute_input":"2024-05-28T21:02:01.317396Z","iopub.status.idle":"2024-05-28T21:21:49.511319Z","shell.execute_reply.started":"2024-05-28T21:02:01.317363Z","shell.execute_reply":"2024-05-28T21:21:49.509955Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"----\n<a id=\"7\"></a>\n# Classifying the Testing Set and Submitting Results on Kaggle\n\nAfter obtaining a promising model with hyperparameters,\nwe use all the labeled data (including the validation set) to retrain the model and classify the testing set.","metadata":{}},{"cell_type":"code","source":"net, preds = get_net(), []\nnet(next(iter(train_valid_iter))[0])\ntrain(net, train_valid_iter, None, num_epochs, lr, wd, devices, lr_period,\n      lr_decay)\n\nfor X, _ in test_iter:\n    y_hat = net(X.to(devices[0]))\n    preds.extend(y_hat.argmax(dim=1).type(torch.int32).cpu().numpy())","metadata":{"execution":{"iopub.status.busy":"2024-05-28T21:33:29.675107Z","iopub.execute_input":"2024-05-28T21:33:29.675508Z","iopub.status.idle":"2024-05-28T21:57:36.235261Z","shell.execute_reply.started":"2024-05-28T21:33:29.675477Z","shell.execute_reply":"2024-05-28T21:57:36.233920Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sorted_ids = list(range(1, len(test_ds) + 1))\nsorted_ids.sort(key=str)\nlabels = [train_valid_ds.classes[x] for x in preds]\nnew_series = pd.Series(np.array(labels), index=sorted_ids)\nsubmission_df = new_series.to_frame('label')\nsubmission_df\n#submission_df.to_csv('submission.csv')","metadata":{"execution":{"iopub.status.busy":"2024-05-28T22:03:15.917950Z","iopub.execute_input":"2024-05-28T22:03:15.918615Z","iopub.status.idle":"2024-05-28T22:03:16.291813Z","shell.execute_reply.started":"2024-05-28T22:03:15.918583Z","shell.execute_reply":"2024-05-28T22:03:16.290939Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create DataFrame\n# sorted_ids = list(range(1, len(test_ds) + 1))\n# sorted_ids.sort(key=lambda x: str(x))\n# df = pd.DataFrame({'id': sorted_ids, 'label': preds})\n# # Convert label indices to class names (assuming you have access to the class names)\n# df['label'] = df['label'].apply(lambda x: train_valid_ds.classes[x])\n# df['label'] = df['label'].astype(str)  # Convert labels to strings\n# df.to_csv('submission.csv', index=False)  # Save DataFrame to CSV\n\n# sorted_ids = list(range(1, len(test_ds) + 1))\n# sorted_ids.sort(key=lambda x: str(x))\n# df = pd.DataFrame({'id': sorted_ids, 'label': preds})\n# df['label'] = df['label'].apply(lambda x: train_valid_ds.classes[x])\n# #df.to_csv('submission.csv', index=False, header=['id', 'label'])\n# #df.reset_index().to_csv('submission.csv', index=False)\n# df.to_csv('submission.csv', index=True, index_label=['id', 'label'])\n# with open('submission.csv', 'w') as f:\n#     f.write(df.to_string(index=False))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# with open('submission.csv', 'w') as f:\n#     f.write('id,label\\n')  # Write the header\n#     f.write(submission_df.reset_index(name='label').to_string(index=True, index_label='id', header=False).replace(' ', ','))  # Write the DataFrame\n    \n# with open('submission.csv', 'w') as f:\n#     f.write('id,label\\n')  # Write the header\nprint('{},{}'.format('id', 'label'))\nfor index, row in submission_df.loc[[1,2,3,4,5,6], :].iterrows():\n    print('{},{}'.format(index, row['label']))","metadata":{"execution":{"iopub.status.busy":"2024-05-28T22:06:40.424391Z","iopub.execute_input":"2024-05-28T22:06:40.424747Z","iopub.status.idle":"2024-05-28T22:06:40.432137Z","shell.execute_reply.started":"2024-05-28T22:06:40.424719Z","shell.execute_reply":"2024-05-28T22:06:40.431230Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with open('submission.csv', 'w') as f:\n    f.write('id,label\\n')  # Write the header\n    for index, row in submission_df.iterrows():\n        f.write('{},{}\\n'.format(index, row['label']))","metadata":{"execution":{"iopub.status.busy":"2024-05-28T22:06:48.490286Z","iopub.execute_input":"2024-05-28T22:06:48.491088Z","iopub.status.idle":"2024-05-28T22:07:04.311745Z","shell.execute_reply.started":"2024-05-28T22:06:48.491060Z","shell.execute_reply":"2024-05-28T22:07:04.310964Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":" pd.read_csv('/kaggle/input/cifar-10/sampleSubmission.csv').head(3)","metadata":{"execution":{"iopub.status.busy":"2024-05-28T22:18:03.055923Z","iopub.execute_input":"2024-05-28T22:18:03.056589Z","iopub.status.idle":"2024-05-28T22:18:03.198224Z","shell.execute_reply.started":"2024-05-28T22:18:03.056558Z","shell.execute_reply":"2024-05-28T22:18:03.197286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pd.read_csv('/kaggle/working/submission.csv').head(3)","metadata":{"execution":{"iopub.status.busy":"2024-05-28T22:18:06.013478Z","iopub.execute_input":"2024-05-28T22:18:06.013811Z","iopub.status.idle":"2024-05-28T22:18:06.098516Z","shell.execute_reply.started":"2024-05-28T22:18:06.013787Z","shell.execute_reply":"2024-05-28T22:18:06.097605Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n----\n<a id=\"8\"></a>\n# Summary\n\n* We can read datasets containing raw image files after organizing them into the required format.\n* We can use convolutional neural networks and image augmentation in an image classification competition.\n\n","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}