{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":10931879,"sourceType":"datasetVersion","datasetId":6797489}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Dog Breed Identification (ImageNet Dogs) on Kaggle\n\nIn this section, we will practice\nthe dog breed identification problem on\nKaggle. (**The web address of this competition is https://www.kaggle.com/c/dog-breed-identification**)\n\nIn this competition,\n120 different breeds of dogs will be recognized.\nIn fact,\nthe dataset for this competition is\na subset of the ImageNet dataset.\nUnlike the images in the CIFAR-10 dataset in :numref:`sec_kaggle_cifar10`,\nthe images in the ImageNet dataset are both higher and wider in varying dimensions.\n:numref:`fig_kaggle_dog` shows the information on the competition's webpage. You need a Kaggle account\nto submit your results.\n\n\n![The dog breed identification competition website. The competition dataset can be obtained by clicking the \"Data\" tab.](../img/kaggle-dog.jpg)\n:width:`400px`\n:label:`fig_kaggle_dog`\n","metadata":{"origin_pos":0}},{"cell_type":"code","source":"!pip install d2l==1.0.3","metadata":{"execution":{"iopub.status.busy":"2025-03-05T15:43:34.891033Z","iopub.execute_input":"2025-03-05T15:43:34.891321Z","iopub.status.idle":"2025-03-05T15:43:38.439281Z","shell.execute_reply.started":"2025-03-05T15:43:34.8913Z","shell.execute_reply":"2025-03-05T15:43:38.438292Z"},"trusted":true},"outputs":[{"name":"stdout","text":"Requirement already satisfied: d2l==1.0.3 in /usr/local/lib/python3.10/dist-packages (1.0.3)\nRequirement already satisfied: jupyter==1.0.0 in /usr/local/lib/python3.10/dist-packages (from d2l==1.0.3) (1.0.0)\nRequirement already satisfied: numpy==1.23.5 in /usr/local/lib/python3.10/dist-packages (from d2l==1.0.3) (1.23.5)\nRequirement already satisfied: matplotlib==3.7.2 in /usr/local/lib/python3.10/dist-packages (from d2l==1.0.3) (3.7.2)\nRequirement already satisfied: matplotlib-inline==0.1.6 in /usr/local/lib/python3.10/dist-packages (from d2l==1.0.3) (0.1.6)\nRequirement already satisfied: requests==2.31.0 in /usr/local/lib/python3.10/dist-packages (from d2l==1.0.3) (2.31.0)\nRequirement already satisfied: pandas==2.0.3 in /usr/local/lib/python3.10/dist-packages (from d2l==1.0.3) (2.0.3)\nRequirement already satisfied: scipy==1.10.1 in /usr/local/lib/python3.10/dist-packages (from d2l==1.0.3) (1.10.1)\nRequirement already satisfied: notebook in /usr/local/lib/python3.10/dist-packages (from jupyter==1.0.0->d2l==1.0.3) (6.5.4)\nRequirement already satisfied: qtconsole in /usr/local/lib/python3.10/dist-packages (from jupyter==1.0.0->d2l==1.0.3) (5.6.1)\nRequirement already satisfied: jupyter-console in /usr/local/lib/python3.10/dist-packages (from jupyter==1.0.0->d2l==1.0.3) (6.1.0)\nRequirement already satisfied: nbconvert in /usr/local/lib/python3.10/dist-packages (from jupyter==1.0.0->d2l==1.0.3) (6.4.5)\nRequirement already satisfied: ipykernel in /usr/local/lib/python3.10/dist-packages (from jupyter==1.0.0->d2l==1.0.3) (5.5.6)\nRequirement already satisfied: ipywidgets in /usr/local/lib/python3.10/dist-packages (from jupyter==1.0.0->d2l==1.0.3) (8.1.5)\nRequirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib==3.7.2->d2l==1.0.3) (1.3.1)\nRequirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib==3.7.2->d2l==1.0.3) (0.12.1)\nRequirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib==3.7.2->d2l==1.0.3) (4.55.3)\nRequirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib==3.7.2->d2l==1.0.3) (1.4.7)\nRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib==3.7.2->d2l==1.0.3) (24.2)\nRequirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib==3.7.2->d2l==1.0.3) (11.0.0)\nRequirement already satisfied: pyparsing<3.1,>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib==3.7.2->d2l==1.0.3) (3.0.9)\nRequirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib==3.7.2->d2l==1.0.3) (2.9.0.post0)\nRequirement already satisfied: traitlets in /usr/local/lib/python3.10/dist-packages (from matplotlib-inline==0.1.6->d2l==1.0.3) (5.7.1)\nRequirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas==2.0.3->d2l==1.0.3) (2025.1)\nRequirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas==2.0.3->d2l==1.0.3) (2025.1)\nRequirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests==2.31.0->d2l==1.0.3) (3.4.1)\nRequirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests==2.31.0->d2l==1.0.3) (3.10)\nRequirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests==2.31.0->d2l==1.0.3) (2.3.0)\nRequirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests==2.31.0->d2l==1.0.3) (2025.1.31)\nRequirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib==3.7.2->d2l==1.0.3) (1.17.0)\nRequirement already satisfied: ipython-genutils in /usr/local/lib/python3.10/dist-packages (from ipykernel->jupyter==1.0.0->d2l==1.0.3) (0.2.0)\nRequirement already satisfied: ipython>=5.0.0 in /usr/local/lib/python3.10/dist-packages (from ipykernel->jupyter==1.0.0->d2l==1.0.3) (7.34.0)\nRequirement already satisfied: jupyter-client in /usr/local/lib/python3.10/dist-packages (from ipykernel->jupyter==1.0.0->d2l==1.0.3) (8.6.3)\nRequirement already satisfied: tornado>=4.2 in /usr/local/lib/python3.10/dist-packages (from ipykernel->jupyter==1.0.0->d2l==1.0.3) (6.3.3)\nRequirement already satisfied: comm>=0.1.3 in /usr/local/lib/python3.10/dist-packages (from ipywidgets->jupyter==1.0.0->d2l==1.0.3) (0.2.2)\nRequirement already satisfied: widgetsnbextension~=4.0.12 in /usr/local/lib/python3.10/dist-packages (from ipywidgets->jupyter==1.0.0->d2l==1.0.3) (4.0.13)\nRequirement already satisfied: jupyterlab-widgets~=3.0.12 in /usr/local/lib/python3.10/dist-packages (from ipywidgets->jupyter==1.0.0->d2l==1.0.3) (3.0.13)\nRequirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from jupyter-console->jupyter==1.0.0->d2l==1.0.3) (3.0.48)\nRequirement already satisfied: pygments in /usr/local/lib/python3.10/dist-packages (from jupyter-console->jupyter==1.0.0->d2l==1.0.3) (2.19.1)\nRequirement already satisfied: mistune<2,>=0.8.1 in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (0.8.4)\nRequirement already satisfied: jinja2>=2.4 in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (3.1.4)\nRequirement already satisfied: jupyterlab-pygments in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (0.3.0)\nRequirement already satisfied: jupyter-core in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (5.7.2)\nRequirement already satisfied: nbformat>=4.4 in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (5.10.4)\nRequirement already satisfied: entrypoints>=0.2.2 in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (0.4)\nRequirement already satisfied: bleach in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (6.2.0)\nRequirement already satisfied: pandocfilters>=1.4.1 in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (1.5.1)\nRequirement already satisfied: testpath in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (0.6.0)\nRequirement already satisfied: defusedxml in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (0.7.1)\nRequirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (4.12.3)\nRequirement already satisfied: nbclient<0.6.0,>=0.5.0 in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (0.5.13)\nRequirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from nbconvert->jupyter==1.0.0->d2l==1.0.3) (3.0.2)\nRequirement already satisfied: pyzmq>=17 in /usr/local/lib/python3.10/dist-packages (from notebook->jupyter==1.0.0->d2l==1.0.3) (24.0.1)\nRequirement already satisfied: argon2-cffi in /usr/local/lib/python3.10/dist-packages (from notebook->jupyter==1.0.0->d2l==1.0.3) (23.1.0)\nRequirement already satisfied: nest-asyncio>=1.5 in /usr/local/lib/python3.10/dist-packages (from notebook->jupyter==1.0.0->d2l==1.0.3) (1.6.0)\nRequirement already satisfied: Send2Trash>=1.8.0 in /usr/local/lib/python3.10/dist-packages (from notebook->jupyter==1.0.0->d2l==1.0.3) (1.8.3)\nRequirement already satisfied: terminado>=0.8.3 in /usr/local/lib/python3.10/dist-packages (from notebook->jupyter==1.0.0->d2l==1.0.3) (0.18.1)\nRequirement already satisfied: prometheus-client in /usr/local/lib/python3.10/dist-packages (from notebook->jupyter==1.0.0->d2l==1.0.3) (0.21.1)\nRequirement already satisfied: nbclassic>=0.4.7 in /usr/local/lib/python3.10/dist-packages (from notebook->jupyter==1.0.0->d2l==1.0.3) (1.1.0)\nRequirement already satisfied: qtpy>=2.4.0 in /usr/local/lib/python3.10/dist-packages (from qtconsole->jupyter==1.0.0->d2l==1.0.3) (2.4.3)\nRequirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.10/dist-packages (from ipython>=5.0.0->ipykernel->jupyter==1.0.0->d2l==1.0.3) (75.1.0)\nRequirement already satisfied: jedi>=0.16 in /usr/local/lib/python3.10/dist-packages (from ipython>=5.0.0->ipykernel->jupyter==1.0.0->d2l==1.0.3) (0.19.2)\nRequirement already satisfied: decorator in /usr/local/lib/python3.10/dist-packages (from ipython>=5.0.0->ipykernel->jupyter==1.0.0->d2l==1.0.3) (4.4.2)\nRequirement already satisfied: pickleshare in /usr/local/lib/python3.10/dist-packages (from ipython>=5.0.0->ipykernel->jupyter==1.0.0->d2l==1.0.3) (0.7.5)\nRequirement already satisfied: backcall in /usr/local/lib/python3.10/dist-packages (from ipython>=5.0.0->ipykernel->jupyter==1.0.0->d2l==1.0.3) (0.2.0)\nRequirement already satisfied: pexpect>4.3 in /usr/local/lib/python3.10/dist-packages (from ipython>=5.0.0->ipykernel->jupyter==1.0.0->d2l==1.0.3) (4.9.0)\nRequirement already satisfied: platformdirs>=2.5 in /usr/local/lib/python3.10/dist-packages (from jupyter-core->nbconvert->jupyter==1.0.0->d2l==1.0.3) (4.3.6)\nRequirement already satisfied: notebook-shim>=0.2.3 in /usr/local/lib/python3.10/dist-packages (from nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (0.2.4)\nRequirement already satisfied: fastjsonschema>=2.15 in /usr/local/lib/python3.10/dist-packages (from nbformat>=4.4->nbconvert->jupyter==1.0.0->d2l==1.0.3) (2.21.1)\nRequirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.10/dist-packages (from nbformat>=4.4->nbconvert->jupyter==1.0.0->d2l==1.0.3) (4.23.0)\nRequirement already satisfied: wcwidth in /usr/local/lib/python3.10/dist-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->jupyter-console->jupyter==1.0.0->d2l==1.0.3) (0.2.13)\nRequirement already satisfied: ptyprocess in /usr/local/lib/python3.10/dist-packages (from terminado>=0.8.3->notebook->jupyter==1.0.0->d2l==1.0.3) (0.7.0)\nRequirement already satisfied: argon2-cffi-bindings in /usr/local/lib/python3.10/dist-packages (from argon2-cffi->notebook->jupyter==1.0.0->d2l==1.0.3) (21.2.0)\nRequirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->nbconvert->jupyter==1.0.0->d2l==1.0.3) (2.6)\nRequirement already satisfied: webencodings in /usr/local/lib/python3.10/dist-packages (from bleach->nbconvert->jupyter==1.0.0->d2l==1.0.3) (0.5.1)\nRequirement already satisfied: parso<0.9.0,>=0.8.4 in /usr/local/lib/python3.10/dist-packages (from jedi>=0.16->ipython>=5.0.0->ipykernel->jupyter==1.0.0->d2l==1.0.3) (0.8.4)\nRequirement already satisfied: attrs>=22.2.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=2.6->nbformat>=4.4->nbconvert->jupyter==1.0.0->d2l==1.0.3) (25.1.0)\nRequirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=2.6->nbformat>=4.4->nbconvert->jupyter==1.0.0->d2l==1.0.3) (2024.10.1)\nRequirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=2.6->nbformat>=4.4->nbconvert->jupyter==1.0.0->d2l==1.0.3) (0.35.1)\nRequirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=2.6->nbformat>=4.4->nbconvert->jupyter==1.0.0->d2l==1.0.3) (0.22.3)\nRequirement already satisfied: jupyter-server<3,>=1.8 in /usr/local/lib/python3.10/dist-packages (from notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (2.12.5)\nRequirement already satisfied: cffi>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from argon2-cffi-bindings->argon2-cffi->notebook->jupyter==1.0.0->d2l==1.0.3) (1.17.1)\nRequirement already satisfied: pycparser in /usr/local/lib/python3.10/dist-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->notebook->jupyter==1.0.0->d2l==1.0.3) (2.22)\nRequirement already satisfied: anyio>=3.1.0 in /usr/local/lib/python3.10/dist-packages (from jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (3.7.1)\nRequirement already satisfied: jupyter-events>=0.9.0 in /usr/local/lib/python3.10/dist-packages (from jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (0.12.0)\nRequirement already satisfied: jupyter-server-terminals in /usr/local/lib/python3.10/dist-packages (from jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (0.5.3)\nRequirement already satisfied: overrides in /usr/local/lib/python3.10/dist-packages (from jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (7.7.0)\nRequirement already satisfied: websocket-client in /usr/local/lib/python3.10/dist-packages (from jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (1.8.0)\nRequirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio>=3.1.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (1.3.1)\nRequirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio>=3.1.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (1.2.2)\nRequirement already satisfied: python-json-logger>=2.0.4 in /usr/local/lib/python3.10/dist-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (3.2.1)\nRequirement already satisfied: pyyaml>=5.3 in /usr/local/lib/python3.10/dist-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (6.0.2)\nRequirement already satisfied: rfc3339-validator in /usr/local/lib/python3.10/dist-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (0.1.4)\nRequirement already satisfied: rfc3986-validator>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (0.1.1)\nRequirement already satisfied: fqdn in /usr/local/lib/python3.10/dist-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (1.5.1)\nRequirement already satisfied: isoduration in /usr/local/lib/python3.10/dist-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (20.11.0)\nRequirement already satisfied: jsonpointer>1.13 in /usr/local/lib/python3.10/dist-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (3.0.0)\nRequirement already satisfied: uri-template in /usr/local/lib/python3.10/dist-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (1.3.0)\nRequirement already satisfied: webcolors>=24.6.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (24.11.1)\nRequirement already satisfied: arrow>=0.15.0 in /usr/local/lib/python3.10/dist-packages (from isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (1.3.0)\nRequirement already satisfied: types-python-dateutil>=2.8.10 in /usr/local/lib/python3.10/dist-packages (from arrow>=0.15.0->isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=1.8->notebook-shim>=0.2.3->nbclassic>=0.4.7->notebook->jupyter==1.0.0->d2l==1.0.3) (2.9.0.20241206)\n","output_type":"stream"}],"execution_count":17},{"cell_type":"code","source":"import os\nimport torch\nimport torchvision\nfrom torch import nn\nfrom d2l import torch as d2l","metadata":{"origin_pos":2,"tab":["pytorch"],"trusted":true,"execution":{"iopub.status.busy":"2025-03-05T12:43:36.129905Z","iopub.execute_input":"2025-03-05T12:43:36.130245Z","iopub.status.idle":"2025-03-05T12:43:36.134729Z","shell.execute_reply.started":"2025-03-05T12:43:36.13021Z","shell.execute_reply":"2025-03-05T12:43:36.133691Z"}},"outputs":[],"execution_count":18},{"cell_type":"markdown","source":"## Obtaining and Organizing the Dataset\n\nThe competition dataset is divided into a training set and a test set, which contain 10222 and 10357 JPEG images\nof three RGB (color) channels, respectively.\nAmong the training dataset,\nthere are 120 breeds of dogs\nsuch as Labradors, Poodles, Dachshunds, Samoyeds, Huskies, Chihuahuas, and Yorkshire Terriers.\n\n\n### Downloading the Dataset\n\nAfter logging into Kaggle,\nyou can click on the \"Data\" tab on the\ncompetition webpage shown in :numref:`fig_kaggle_dog` and download the dataset by clicking the \"Download All\" button.\nAfter unzipping the downloaded file in `../data`, you will find the entire dataset in the following paths:\n\n* ../data/dog-breed-identification/labels.csv\n* ../data/dog-breed-identification/sample_submission.csv\n* ../data/dog-breed-identification/train\n* ../data/dog-breed-identification/test\n\nYou may have noticed that the above structure is\nsimilar to that of the CIFAR-10 competition in :numref:`sec_kaggle_cifar10`, where folders `train/` and `test/` contain training and testing dog images, respectively, and `labels.csv` contains\nthe labels for the training images.\n","metadata":{"origin_pos":3}},{"cell_type":"markdown","source":"The data is in the input folder and is read-only. To manipulate the data, we first copy it to a working folder.","metadata":{}},{"cell_type":"code","source":"!cp -r '/kaggle/input/dog-breed-identification1/msba-7028-2025-hw-2' '/kaggle/working/dog-breed-identification'","metadata":{"execution":{"iopub.status.busy":"2025-03-05T15:44:21.11054Z","iopub.execute_input":"2025-03-05T15:44:21.110924Z","iopub.status.idle":"2025-03-05T15:44:30.111394Z","shell.execute_reply.started":"2025-03-05T15:44:21.110893Z","shell.execute_reply":"2025-03-05T15:44:30.110297Z"},"trusted":true},"outputs":[],"execution_count":19},{"cell_type":"markdown","source":"From now on we set the directory as the working directory.","metadata":{}},{"cell_type":"code","source":"data_dir='/kaggle/working/dog-breed-identification/msba-7028-2025-hw-2'","metadata":{"execution":{"iopub.status.busy":"2025-03-05T15:44:33.180459Z","iopub.execute_input":"2025-03-05T15:44:33.180813Z","iopub.status.idle":"2025-03-05T15:44:33.184749Z","shell.execute_reply.started":"2025-03-05T15:44:33.180784Z","shell.execute_reply":"2025-03-05T15:44:33.183824Z"},"trusted":true},"outputs":[],"execution_count":20},{"cell_type":"markdown","source":"### [**Organizing the Dataset**]\n\nWe can organize the dataset similarly to what we did in :numref:`sec_kaggle_cifar10`, namely splitting out\na validation set from the original training set, and moving images into subfolders grouped by labels.\n\nThe `reorg_dog_data` function below reads\nthe training data labels, splits out the validation set, and organizes the training set.\n","metadata":{"origin_pos":5}},{"cell_type":"code","source":"def reorg_dog_data(data_dir, valid_ratio):\n    labels = d2l.read_csv_labels(os.path.join(data_dir, 'labels.csv'))\n    d2l.reorg_train_valid(data_dir, labels, valid_ratio)\n    d2l.reorg_test(data_dir)\n\n\nbatch_size = 128\nvalid_ratio = 0.1\nreorg_dog_data(data_dir, valid_ratio)","metadata":{"origin_pos":6,"tab":["pytorch"],"execution":{"iopub.status.busy":"2025-03-05T15:44:36.14798Z","iopub.execute_input":"2025-03-05T15:44:36.148314Z","iopub.status.idle":"2025-03-05T15:44:38.589442Z","shell.execute_reply.started":"2025-03-05T15:44:36.148286Z","shell.execute_reply":"2025-03-05T15:44:38.588643Z"},"trusted":true},"outputs":[],"execution_count":21},{"cell_type":"markdown","source":"## [**Image Augmentation**]\n\nRecall that this dog breed dataset\nis a subset of the ImageNet dataset,\nwhose images\nare larger than those of the CIFAR-10 dataset\nin :numref:`sec_kaggle_cifar10`.\nThe following\nlists a few image augmentation operations\nthat might be useful for relatively larger images.\n","metadata":{"origin_pos":7}},{"cell_type":"code","source":"transform_train = torchvision.transforms.Compose([\n    # Randomly crop the image to obtain an image with an area of 0.08 to 1 of\n    # the original area and height-to-width ratio between 3/4 and 4/3. Then,\n    # scale the image to create a new 224 x 224 image\n    torchvision.transforms.RandomResizedCrop(224, scale=(0.08, 1.0),\n                                             ratio=(3.0/4.0, 4.0/3.0)),\n    torchvision.transforms.RandomHorizontalFlip(),\n    # Randomly change the brightness, contrast, and saturation\n    torchvision.transforms.ColorJitter(brightness=0.4,\n                                       contrast=0.4,\n                                       saturation=0.4),\n    # Add random noise\n    torchvision.transforms.ToTensor(),\n    # Standardize each channel of the image\n    torchvision.transforms.Normalize([0.485, 0.456, 0.406],\n                                     [0.229, 0.224, 0.225])])","metadata":{"origin_pos":9,"tab":["pytorch"],"execution":{"iopub.status.busy":"2025-03-05T15:44:43.493878Z","iopub.execute_input":"2025-03-05T15:44:43.494178Z","iopub.status.idle":"2025-03-05T15:44:43.499632Z","shell.execute_reply.started":"2025-03-05T15:44:43.494154Z","shell.execute_reply":"2025-03-05T15:44:43.498567Z"},"trusted":true},"outputs":[],"execution_count":22},{"cell_type":"markdown","source":"During prediction,\nwe only use image preprocessing operations\nwithout randomness.\n","metadata":{"origin_pos":10}},{"cell_type":"code","source":"transform_test = torchvision.transforms.Compose([\n    torchvision.transforms.Resize(256),\n    # Crop a 224 x 224 square area from the center of the image\n    torchvision.transforms.CenterCrop(224),\n    torchvision.transforms.ToTensor(),\n    torchvision.transforms.Normalize([0.485, 0.456, 0.406],\n                                     [0.229, 0.224, 0.225])])","metadata":{"origin_pos":12,"tab":["pytorch"],"execution":{"iopub.status.busy":"2025-03-05T15:44:50.362017Z","iopub.execute_input":"2025-03-05T15:44:50.362322Z","iopub.status.idle":"2025-03-05T15:44:50.366972Z","shell.execute_reply.started":"2025-03-05T15:44:50.3623Z","shell.execute_reply":"2025-03-05T15:44:50.36609Z"},"trusted":true},"outputs":[],"execution_count":23},{"cell_type":"markdown","source":"## [**Reading the Dataset**]\n\nAs in :numref:`sec_kaggle_cifar10`,\nwe can read the organized dataset\nconsisting of raw image files.\n","metadata":{"origin_pos":13}},{"cell_type":"code","source":"train_ds, train_valid_ds = [torchvision.datasets.ImageFolder(\n    os.path.join(data_dir, 'train_valid_test', folder),\n    transform=transform_train) for folder in ['train', 'train_valid']]\n\nvalid_ds, test_ds = [torchvision.datasets.ImageFolder(\n    os.path.join(data_dir, 'train_valid_test', folder),\n    transform=transform_test) for folder in ['valid', 'test']]","metadata":{"origin_pos":15,"tab":["pytorch"],"execution":{"iopub.status.busy":"2025-03-05T15:44:53.421369Z","iopub.execute_input":"2025-03-05T15:44:53.421708Z","iopub.status.idle":"2025-03-05T15:44:53.477911Z","shell.execute_reply.started":"2025-03-05T15:44:53.421681Z","shell.execute_reply":"2025-03-05T15:44:53.47703Z"},"trusted":true},"outputs":[],"execution_count":24},{"cell_type":"markdown","source":"Below we create data iterator instances\nthe same way\nas in :numref:`sec_kaggle_cifar10`.\n","metadata":{"origin_pos":16}},{"cell_type":"code","source":"train_iter, train_valid_iter = [torch.utils.data.DataLoader(\n    dataset, batch_size, shuffle=True, drop_last=True)\n    for dataset in (train_ds, train_valid_ds)]\n\nvalid_iter = torch.utils.data.DataLoader(valid_ds, batch_size, shuffle=False,\n                                         drop_last=True)\n\ntest_iter = torch.utils.data.DataLoader(test_ds, batch_size, shuffle=False,\n                                        drop_last=False)","metadata":{"origin_pos":18,"tab":["pytorch"],"execution":{"iopub.status.busy":"2025-03-05T15:44:56.781469Z","iopub.execute_input":"2025-03-05T15:44:56.781794Z","iopub.status.idle":"2025-03-05T15:44:56.787056Z","shell.execute_reply.started":"2025-03-05T15:44:56.78177Z","shell.execute_reply":"2025-03-05T15:44:56.78612Z"},"trusted":true},"outputs":[],"execution_count":25},{"cell_type":"markdown","source":"## [**Fine-Tuning a Pretrained Model**]\n\nAgain,\nthe dataset for this competition is a subset of the ImageNet dataset.\nTherefore, we can use the approach discussed in\n:numref:`sec_fine_tuning`\nto select a model pretrained on the\nfull ImageNet dataset and use it to extract image features to be fed into a\ncustom small-scale output network.\nHigh-level APIs of deep learning frameworks\nprovide a wide range of models\npretrained on the ImageNet dataset.\nHere, we choose\na pretrained ResNet-34 model,\nwhere we simply reuse\nthe input of this model's output layer\n(i.e., the extracted\nfeatures).\nThen we can replace the original output layer with a small custom\noutput network that can be trained,\nsuch as stacking two\nfully connected layers.\nDifferent from the experiment in\n:numref:`sec_fine_tuning`,\nthe following does\nnot retrain the pretrained model used for feature\nextraction. This reduces training time and\nmemory for storing gradients.\n\nRecall that we\nstandardized images using\nthe means and standard deviations of the three RGB channels for the full ImageNet dataset.\nIn fact,\nthis is also consistent with the standardization operation\nby the pretrained model on ImageNet.\n","metadata":{"origin_pos":19}},{"cell_type":"code","source":"def get_net(devices):\n    finetune_net = nn.Sequential()\n    finetune_net.features = torchvision.models.resnet50(pretrained=True)\n    # Define a new output network (there are 120 output categories)\n    finetune_net.output_new = nn.Sequential(nn.Linear(1000, 256),\n                                            nn.ReLU(),\n                                            nn.Linear(256, 120))\n    # Move the model to devices\n    finetune_net = finetune_net.to(devices[0])\n    # Freeze parameters of feature layers\n    for param in finetune_net.features.parameters():\n        param.requires_grad = False\n    return finetune_net","metadata":{"origin_pos":21,"tab":["pytorch"],"execution":{"iopub.status.busy":"2025-03-05T15:45:02.942371Z","iopub.execute_input":"2025-03-05T15:45:02.942871Z","iopub.status.idle":"2025-03-05T15:45:02.950071Z","shell.execute_reply.started":"2025-03-05T15:45:02.942821Z","shell.execute_reply":"2025-03-05T15:45:02.949031Z"},"trusted":true},"outputs":[],"execution_count":26},{"cell_type":"markdown","source":"Before [**calculating the loss**],\nwe first obtain the input of the pretrained model's output layer, i.e., the extracted feature.\nThen we use this feature as input for our small custom output network to calculate the loss.\n","metadata":{"origin_pos":22}},{"cell_type":"code","source":"loss = nn.CrossEntropyLoss(reduction='none')\n\ndef evaluate_loss(data_iter, net, devices):\n    l_sum, n = 0.0, 0\n    for features, labels in data_iter:\n        features, labels = features.to(devices[0]), labels.to(devices[0])\n        outputs = net(features)\n        l = loss(outputs, labels)\n        l_sum += l.sum()\n        n += labels.numel()\n    return l_sum / n","metadata":{"origin_pos":24,"tab":["pytorch"],"execution":{"iopub.status.busy":"2025-03-05T15:45:11.715237Z","iopub.execute_input":"2025-03-05T15:45:11.715566Z","iopub.status.idle":"2025-03-05T15:45:11.720493Z","shell.execute_reply.started":"2025-03-05T15:45:11.715538Z","shell.execute_reply":"2025-03-05T15:45:11.719535Z"},"trusted":true},"outputs":[],"execution_count":27},{"cell_type":"markdown","source":"## Defining [**the Training Function**]\n\nWe will select the model and tune hyperparameters according to the model's performance on the validation set. The model training function `train` only\niterates parameters of the small custom output network.\n","metadata":{"origin_pos":25}},{"cell_type":"code","source":"def train(net, train_iter, valid_iter, num_epochs, lr, wd, devices, lr_period,\n          lr_decay):\n    # Only train the small custom output network\n    net = nn.DataParallel(net, device_ids=devices).to(devices[0])\n    trainer = torch.optim.Adam((param for param in net.parameters() if param.requires_grad), \n                            lr=lr, betas=(0.9, 0.999), \n                            weight_decay=wd)\n    scheduler = torch.optim.lr_scheduler.StepLR(trainer, lr_period, lr_decay)\n    num_batches, timer = len(train_iter), d2l.Timer()\n    legend = ['train loss']\n    if valid_iter is not None:\n        legend.append('valid loss')\n    animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs],\n                            legend=legend)\n    for epoch in range(num_epochs):\n        metric = d2l.Accumulator(2)\n        for i, (features, labels) in enumerate(train_iter):\n            timer.start()\n            features, labels = features.to(devices[0]), labels.to(devices[0])\n            trainer.zero_grad()\n            output = net(features)\n            l = loss(output, labels).sum()\n            l.backward()\n            trainer.step()\n            metric.add(l, labels.shape[0])\n            timer.stop()\n            if (i + 1) % (num_batches // 5) == 0 or i == num_batches - 1:\n                animator.add(epoch + (i + 1) / num_batches,\n                             (metric[0] / metric[1], None))\n        measures = f'train loss {metric[0] / metric[1]:.3f}'\n        if valid_iter is not None:\n            valid_loss = evaluate_loss(valid_iter, net, devices)\n            animator.add(epoch + 1, (None, valid_loss.detach().cpu()))\n        scheduler.step()\n    if valid_iter is not None:\n        measures += f', valid loss {valid_loss:.3f}'\n    print(measures + f'\\n{metric[1] * num_epochs / timer.sum():.1f}'\n          f' examples/sec on {str(devices)}')","metadata":{"origin_pos":27,"tab":["pytorch"],"execution":{"iopub.status.busy":"2025-03-05T15:45:15.718409Z","iopub.execute_input":"2025-03-05T15:45:15.718784Z","iopub.status.idle":"2025-03-05T15:45:15.726963Z","shell.execute_reply.started":"2025-03-05T15:45:15.718751Z","shell.execute_reply":"2025-03-05T15:45:15.725958Z"},"trusted":true},"outputs":[],"execution_count":28},{"cell_type":"markdown","source":"## [**Training and Validating the Model**]\n\nNow we can train and validate the model.\nThe following hyperparameters are all tunable.\nFor example, the number of epochs can be increased. Because `lr_period` and `lr_decay` are set to 2 and 0.9, respectively, the learning rate of the optimization algorithm will be multiplied by 0.9 after every 2 epochs.\n","metadata":{"origin_pos":28}},{"cell_type":"code","source":"devices, num_epochs, lr, wd = d2l.try_all_gpus(), 20, 1e-4, 1e-4\nlr_period, lr_decay, net = 2, 0.9, get_net(devices)\ntrain(net, train_iter, valid_iter, num_epochs, lr, wd, devices, lr_period,\n      lr_decay)","metadata":{"origin_pos":30,"tab":["pytorch"],"execution":{"iopub.status.busy":"2025-03-05T15:45:22.457678Z","iopub.execute_input":"2025-03-05T15:45:22.458006Z","iopub.status.idle":"2025-03-05T16:02:44.793394Z","shell.execute_reply.started":"2025-03-05T15:45:22.457978Z","shell.execute_reply":"2025-03-05T16:02:44.792265Z"},"trusted":true},"outputs":[{"name":"stdout","text":"train loss 0.806, valid loss 0.539\n616.8 examples/sec on [device(type='cuda', index=0)]\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"<Figure size 350x250 with 1 Axes>","image/svg+xml":"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"229.425pt\" height=\"183.35625pt\" viewBox=\"0 0 229.425 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n <metadata>\n  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n   <cc:Work>\n    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n    <dc:date>2025-03-05T16:02:44.759776</dc:date>\n    <dc:format>image/svg+xml</dc:format>\n    <dc:creator>\n     <cc:Agent>\n      <dc:title>Matplotlib v3.7.5, https://matplotlib.org/</dc:title>\n     </cc:Agent>\n    </dc:creator>\n   </cc:Work>\n  </rdf:RDF>\n </metadata>\n <defs>\n  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 183.35625 \nL 229.425 183.35625 \nL 229.425 0 \nL 0 0 \nz\n\" style=\"fill: #ffffff\"/>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"patch_2\">\n    <path d=\"M 20.5625 145.8 \nL 215.8625 145.8 \nL 215.8625 7.2 \nL 20.5625 7.2 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g id=\"matplotlib.axis_1\">\n    <g id=\"xtick_1\">\n     <g id=\"line2d_1\">\n      <path d=\"M 61.678289 145.8 \nL 61.678289 7.2 \n\" clip-path=\"url(#p4941e19c63)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_2\">\n      <defs>\n       <path id=\"m9f6427740d\" d=\"M 0 0 \nL 0 3.5 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#m9f6427740d\" x=\"61.678289\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_1\">\n      <!-- 5 -->\n      <g transform=\"translate(58.497039 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \nL 3169 4666 \nL 3169 4134 \nL 1269 4134 \nL 1269 2991 \nQ 1406 3038 1543 3061 \nQ 1681 3084 1819 3084 \nQ 2600 3084 3056 2656 \nQ 3513 2228 3513 1497 \nQ 3513 744 3044 326 \nQ 2575 -91 1722 -91 \nQ 1428 -91 1123 -41 \nQ 819 9 494 109 \nL 494 744 \nQ 775 591 1075 516 \nQ 1375 441 1709 441 \nQ 2250 441 2565 725 \nQ 2881 1009 2881 1497 \nQ 2881 1984 2565 2268 \nQ 2250 2553 1709 2553 \nQ 1456 2553 1204 2497 \nQ 953 2441 691 2322 \nL 691 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_2\">\n     <g id=\"line2d_3\">\n      <path d=\"M 113.073026 145.8 \nL 113.073026 7.2 \n\" clip-path=\"url(#p4941e19c63)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_4\">\n      <g>\n       <use xlink:href=\"#m9f6427740d\" x=\"113.073026\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_2\">\n      <!-- 10 -->\n      <g transform=\"translate(106.710526 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-31\" d=\"M 794 531 \nL 1825 531 \nL 1825 4091 \nL 703 3866 \nL 703 4441 \nL 1819 4666 \nL 2450 4666 \nL 2450 531 \nL 3481 531 \nL 3481 0 \nL 794 0 \nL 794 531 \nz\n\" transform=\"scale(0.015625)\"/>\n        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \nQ 1547 4250 1301 3770 \nQ 1056 3291 1056 2328 \nQ 1056 1369 1301 889 \nQ 1547 409 2034 409 \nQ 2525 409 2770 889 \nQ 3016 1369 3016 2328 \nQ 3016 3291 2770 3770 \nQ 2525 4250 2034 4250 \nz\nM 2034 4750 \nQ 2819 4750 3233 4129 \nQ 3647 3509 3647 2328 \nQ 3647 1150 3233 529 \nQ 2819 -91 2034 -91 \nQ 1250 -91 836 529 \nQ 422 1150 422 2328 \nQ 422 3509 836 4129 \nQ 1250 4750 2034 4750 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_3\">\n     <g id=\"line2d_5\">\n      <path d=\"M 164.467763 145.8 \nL 164.467763 7.2 \n\" clip-path=\"url(#p4941e19c63)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_6\">\n      <g>\n       <use xlink:href=\"#m9f6427740d\" x=\"164.467763\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_3\">\n      <!-- 15 -->\n      <g transform=\"translate(158.105263 160.398438) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-35\" x=\"63.623047\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_4\">\n     <g id=\"line2d_7\">\n      <path d=\"M 215.8625 145.8 \nL 215.8625 7.2 \n\" clip-path=\"url(#p4941e19c63)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_8\">\n      <g>\n       <use xlink:href=\"#m9f6427740d\" x=\"215.8625\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_4\">\n      <!-- 20 -->\n      <g transform=\"translate(209.5 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \nL 3431 531 \nL 3431 0 \nL 469 0 \nL 469 531 \nQ 828 903 1448 1529 \nQ 2069 2156 2228 2338 \nQ 2531 2678 2651 2914 \nQ 2772 3150 2772 3378 \nQ 2772 3750 2511 3984 \nQ 2250 4219 1831 4219 \nQ 1534 4219 1204 4116 \nQ 875 4013 500 3803 \nL 500 4441 \nQ 881 4594 1212 4672 \nQ 1544 4750 1819 4750 \nQ 2544 4750 2975 4387 \nQ 3406 4025 3406 3419 \nQ 3406 3131 3298 2873 \nQ 3191 2616 2906 2266 \nQ 2828 2175 2409 1742 \nQ 1991 1309 1228 531 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-32\"/>\n       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_5\">\n     <!-- epoch -->\n     <g transform=\"translate(102.984375 174.076563) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \nL 3597 1613 \nL 953 1613 \nQ 991 1019 1311 708 \nQ 1631 397 2203 397 \nQ 2534 397 2845 478 \nQ 3156 559 3463 722 \nL 3463 178 \nQ 3153 47 2828 -22 \nQ 2503 -91 2169 -91 \nQ 1331 -91 842 396 \nQ 353 884 353 1716 \nQ 353 2575 817 3079 \nQ 1281 3584 2069 3584 \nQ 2775 3584 3186 3129 \nQ 3597 2675 3597 1894 \nz\nM 3022 2063 \nQ 3016 2534 2758 2815 \nQ 2500 3097 2075 3097 \nQ 1594 3097 1305 2825 \nQ 1016 2553 972 2059 \nL 3022 2063 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \nL 1159 -1331 \nL 581 -1331 \nL 581 3500 \nL 1159 3500 \nL 1159 2969 \nQ 1341 3281 1617 3432 \nQ 1894 3584 2278 3584 \nQ 2916 3584 3314 3078 \nQ 3713 2572 3713 1747 \nQ 3713 922 3314 415 \nQ 2916 -91 2278 -91 \nQ 1894 -91 1617 61 \nQ 1341 213 1159 525 \nz\nM 3116 1747 \nQ 3116 2381 2855 2742 \nQ 2594 3103 2138 3103 \nQ 1681 3103 1420 2742 \nQ 1159 2381 1159 1747 \nQ 1159 1113 1420 752 \nQ 1681 391 2138 391 \nQ 2594 391 2855 752 \nQ 3116 1113 3116 1747 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \nQ 1497 3097 1228 2736 \nQ 959 2375 959 1747 \nQ 959 1119 1226 758 \nQ 1494 397 1959 397 \nQ 2419 397 2687 759 \nQ 2956 1122 2956 1747 \nQ 2956 2369 2687 2733 \nQ 2419 3097 1959 3097 \nz\nM 1959 3584 \nQ 2709 3584 3137 3096 \nQ 3566 2609 3566 1747 \nQ 3566 888 3137 398 \nQ 2709 -91 1959 -91 \nQ 1206 -91 779 398 \nQ 353 888 353 1747 \nQ 353 2609 779 3096 \nQ 1206 3584 1959 3584 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \nL 3122 2828 \nQ 2878 2963 2633 3030 \nQ 2388 3097 2138 3097 \nQ 1578 3097 1268 2742 \nQ 959 2388 959 1747 \nQ 959 1106 1268 751 \nQ 1578 397 2138 397 \nQ 2388 397 2633 464 \nQ 2878 531 3122 666 \nL 3122 134 \nQ 2881 22 2623 -34 \nQ 2366 -91 2075 -91 \nQ 1284 -91 818 406 \nQ 353 903 353 1747 \nQ 353 2603 823 3093 \nQ 1294 3584 2113 3584 \nQ 2378 3584 2631 3529 \nQ 2884 3475 3122 3366 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 4863 \nL 1159 4863 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-65\"/>\n      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_2\">\n    <g id=\"ytick_1\">\n     <g id=\"line2d_9\">\n      <path d=\"M 20.5625 125.370713 \nL 215.8625 125.370713 \n\" clip-path=\"url(#p4941e19c63)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_10\">\n      <defs>\n       <path id=\"mb63037ab22\" d=\"M 0 0 \nL -3.5 0 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#mb63037ab22\" x=\"20.5625\" y=\"125.370713\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_6\">\n      <!-- 1 -->\n      <g transform=\"translate(7.2 129.169932) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_2\">\n     <g id=\"line2d_11\">\n      <path d=\"M 20.5625 95.169805 \nL 215.8625 95.169805 \n\" clip-path=\"url(#p4941e19c63)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_12\">\n      <g>\n       <use xlink:href=\"#mb63037ab22\" x=\"20.5625\" y=\"95.169805\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_7\">\n      <!-- 2 -->\n      <g transform=\"translate(7.2 98.969023) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-32\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_3\">\n     <g id=\"line2d_13\">\n      <path d=\"M 20.5625 64.968896 \nL 215.8625 64.968896 \n\" clip-path=\"url(#p4941e19c63)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_14\">\n      <g>\n       <use xlink:href=\"#mb63037ab22\" x=\"20.5625\" y=\"64.968896\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_8\">\n      <!-- 3 -->\n      <g transform=\"translate(7.2 68.768115) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \nQ 3050 2419 3304 2112 \nQ 3559 1806 3559 1356 \nQ 3559 666 3084 287 \nQ 2609 -91 1734 -91 \nQ 1441 -91 1130 -33 \nQ 819 25 488 141 \nL 488 750 \nQ 750 597 1062 519 \nQ 1375 441 1716 441 \nQ 2309 441 2620 675 \nQ 2931 909 2931 1356 \nQ 2931 1769 2642 2001 \nQ 2353 2234 1838 2234 \nL 1294 2234 \nL 1294 2753 \nL 1863 2753 \nQ 2328 2753 2575 2939 \nQ 2822 3125 2822 3475 \nQ 2822 3834 2567 4026 \nQ 2313 4219 1838 4219 \nQ 1578 4219 1281 4162 \nQ 984 4106 628 3988 \nL 628 4550 \nQ 988 4650 1302 4700 \nQ 1616 4750 1894 4750 \nQ 2613 4750 3031 4423 \nQ 3450 4097 3450 3541 \nQ 3450 3153 3228 2886 \nQ 3006 2619 2597 2516 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-33\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_4\">\n     <g id=\"line2d_15\">\n      <path d=\"M 20.5625 34.767988 \nL 215.8625 34.767988 \n\" clip-path=\"url(#p4941e19c63)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_16\">\n      <g>\n       <use xlink:href=\"#mb63037ab22\" x=\"20.5625\" y=\"34.767988\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_9\">\n      <!-- 4 -->\n      <g transform=\"translate(7.2 38.567207) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \nL 825 1625 \nL 2419 1625 \nL 2419 4116 \nz\nM 2253 4666 \nL 3047 4666 \nL 3047 1625 \nL 3713 1625 \nL 3713 1100 \nL 3047 1100 \nL 3047 0 \nL 2419 0 \nL 2419 1100 \nL 313 1100 \nL 313 1709 \nL 2253 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-34\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"line2d_17\">\n    <path d=\"M 12.241447 13.5 \nL 14.199342 18.995002 \nL 16.157237 24.407503 \nL 18.115132 29.681697 \nL 20.073026 35.172603 \nL 20.5625 36.627451 \nL 22.520395 70.828269 \nL 24.478289 75.612028 \nL 26.436184 79.560473 \nL 28.394079 82.754329 \nL 30.351974 85.29936 \nL 30.841447 86.067694 \nL 32.799342 102.837225 \nL 34.757237 104.143348 \nL 36.715132 105.692752 \nL 38.673026 106.917405 \nL 40.630921 107.880709 \nL 41.120395 108.109812 \nL 43.078289 115.219356 \nL 45.036184 114.394394 \nL 46.994079 115.274579 \nL 48.951974 115.478531 \nL 50.909868 115.911959 \nL 51.399342 116.092489 \nL 53.357237 118.116932 \nL 55.315132 118.93813 \nL 57.273026 119.466277 \nL 59.230921 119.739635 \nL 61.188816 120.045017 \nL 61.678289 120.140455 \nL 63.636184 120.833277 \nL 65.594079 120.728444 \nL 67.551974 121.965248 \nL 69.509868 121.749274 \nL 71.467763 122.080034 \nL 71.957237 122.244757 \nL 73.915132 124.92332 \nL 75.873026 124.710981 \nL 77.830921 124.722338 \nL 79.788816 124.74164 \nL 81.746711 124.830701 \nL 82.236184 124.966083 \nL 84.194079 125.637995 \nL 86.151974 126.322039 \nL 88.109868 126.620738 \nL 90.067763 126.63226 \nL 92.025658 126.48327 \nL 92.515132 126.482442 \nL 94.473026 124.746492 \nL 96.430921 125.844769 \nL 98.388816 126.048212 \nL 100.346711 126.213352 \nL 102.304605 126.352833 \nL 102.794079 126.39109 \nL 104.751974 126.691571 \nL 106.709868 127.153551 \nL 108.667763 127.277577 \nL 110.625658 127.340904 \nL 112.583553 127.558081 \nL 113.073026 127.588665 \nL 115.030921 127.610243 \nL 116.988816 128.336359 \nL 118.946711 128.189343 \nL 120.904605 128.083051 \nL 122.8625 127.834249 \nL 123.351974 127.869058 \nL 125.309868 128.312073 \nL 127.267763 128.446927 \nL 129.225658 128.717253 \nL 131.183553 128.496944 \nL 133.141447 128.697988 \nL 133.630921 128.628914 \nL 135.588816 129.397126 \nL 137.546711 128.529113 \nL 139.504605 129.226537 \nL 141.4625 129.441423 \nL 143.420395 129.228295 \nL 143.909868 129.015077 \nL 145.867763 128.659178 \nL 147.825658 129.345174 \nL 149.783553 129.442983 \nL 151.741447 129.436749 \nL 153.699342 129.483989 \nL 154.188816 129.3661 \nL 156.146711 129.107445 \nL 158.104605 130.234384 \nL 160.0625 129.689309 \nL 162.020395 129.846299 \nL 163.978289 130.122394 \nL 164.467763 130.103828 \nL 166.425658 129.027391 \nL 168.383553 128.481137 \nL 170.341447 128.511763 \nL 172.299342 129.31842 \nL 174.257237 129.449784 \nL 174.746711 129.612039 \nL 176.704605 131.862502 \nL 178.6625 131.606285 \nL 180.620395 131.314208 \nL 182.578289 130.988812 \nL 184.536184 130.958792 \nL 185.025658 131.000655 \nL 186.983553 130.80125 \nL 188.941447 130.595385 \nL 190.899342 130.270676 \nL 192.857237 130.293732 \nL 194.815132 130.35337 \nL 195.304605 130.294984 \nL 197.2625 131.009391 \nL 199.220395 131.390514 \nL 201.178289 131.484022 \nL 203.136184 131.322553 \nL 205.094079 131.389353 \nL 205.583553 131.435903 \nL 207.541447 131.611625 \nL 209.499342 131.398028 \nL 211.457237 131.67988 \nL 213.415132 131.782168 \nL 215.373026 131.30993 \nL 215.8625 131.215996 \n\" clip-path=\"url(#p4941e19c63)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n   </g>\n   <g id=\"line2d_18\">\n    <path d=\"M 20.5625 71.917799 \nL 30.841447 111.806203 \nL 41.120395 124.555783 \nL 51.399342 129.75824 \nL 61.678289 132.65893 \nL 71.957237 134.011566 \nL 82.236184 135.524518 \nL 92.515132 136.435869 \nL 102.794079 136.933766 \nL 113.073026 137.341452 \nL 123.351974 137.799461 \nL 133.630921 138.167301 \nL 143.909868 138.387306 \nL 154.188816 138.452645 \nL 164.467763 138.658456 \nL 174.746711 138.817964 \nL 185.025658 139.086426 \nL 195.304605 139.158372 \nL 205.583553 139.5 \nL 215.8625 139.279502 \n\" clip-path=\"url(#p4941e19c63)\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n   </g>\n   <g id=\"patch_3\">\n    <path d=\"M 20.5625 145.8 \nL 20.5625 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_4\">\n    <path d=\"M 215.8625 145.8 \nL 215.8625 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_5\">\n    <path d=\"M 20.5625 145.8 \nL 215.8625 145.8 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_6\">\n    <path d=\"M 20.5625 7.2 \nL 215.8625 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"legend_1\">\n    <g id=\"patch_7\">\n     <path d=\"M 130.417188 44.55625 \nL 208.8625 44.55625 \nQ 210.8625 44.55625 210.8625 42.55625 \nL 210.8625 14.2 \nQ 210.8625 12.2 208.8625 12.2 \nL 130.417188 12.2 \nQ 128.417188 12.2 128.417188 14.2 \nL 128.417188 42.55625 \nQ 128.417188 44.55625 130.417188 44.55625 \nz\n\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n    </g>\n    <g id=\"line2d_19\">\n     <path d=\"M 132.417188 20.298438 \nL 142.417188 20.298438 \nL 152.417188 20.298438 \n\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n    </g>\n    <g id=\"text_10\">\n     <!-- train loss -->\n     <g transform=\"translate(160.417188 23.798438) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \nL 1172 3500 \nL 2356 3500 \nL 2356 3053 \nL 1172 3053 \nL 1172 1153 \nQ 1172 725 1289 603 \nQ 1406 481 1766 481 \nL 2356 481 \nL 2356 0 \nL 1766 0 \nQ 1100 0 847 248 \nQ 594 497 594 1153 \nL 594 3053 \nL 172 3053 \nL 172 3500 \nL 594 3500 \nL 594 4494 \nL 1172 4494 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \nQ 2534 3019 2420 3045 \nQ 2306 3072 2169 3072 \nQ 1681 3072 1420 2755 \nQ 1159 2438 1159 1844 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1341 3275 1631 3429 \nQ 1922 3584 2338 3584 \nQ 2397 3584 2469 3576 \nQ 2541 3569 2628 3553 \nL 2631 2963 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \nQ 1497 1759 1228 1600 \nQ 959 1441 959 1056 \nQ 959 750 1161 570 \nQ 1363 391 1709 391 \nQ 2188 391 2477 730 \nQ 2766 1069 2766 1631 \nL 2766 1759 \nL 2194 1759 \nz\nM 3341 1997 \nL 3341 0 \nL 2766 0 \nL 2766 531 \nQ 2569 213 2275 61 \nQ 1981 -91 1556 -91 \nQ 1019 -91 701 211 \nQ 384 513 384 1019 \nQ 384 1609 779 1909 \nQ 1175 2209 1959 2209 \nL 2766 2209 \nL 2766 2266 \nQ 2766 2663 2505 2880 \nQ 2244 3097 1772 3097 \nQ 1472 3097 1187 3025 \nQ 903 2953 641 2809 \nL 641 3341 \nQ 956 3463 1253 3523 \nQ 1550 3584 1831 3584 \nQ 2591 3584 2966 3190 \nQ 3341 2797 3341 1997 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \nL 1178 3500 \nL 1178 0 \nL 603 0 \nL 603 3500 \nz\nM 603 4863 \nL 1178 4863 \nL 1178 4134 \nL 603 4134 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \nL 1178 4863 \nL 1178 0 \nL 603 0 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \nL 2834 2853 \nQ 2591 2978 2328 3040 \nQ 2066 3103 1784 3103 \nQ 1356 3103 1142 2972 \nQ 928 2841 928 2578 \nQ 928 2378 1081 2264 \nQ 1234 2150 1697 2047 \nL 1894 2003 \nQ 2506 1872 2764 1633 \nQ 3022 1394 3022 966 \nQ 3022 478 2636 193 \nQ 2250 -91 1575 -91 \nQ 1294 -91 989 -36 \nQ 684 19 347 128 \nL 347 722 \nQ 666 556 975 473 \nQ 1284 391 1588 391 \nQ 1994 391 2212 530 \nQ 2431 669 2431 922 \nQ 2431 1156 2273 1281 \nQ 2116 1406 1581 1522 \nL 1381 1569 \nQ 847 1681 609 1914 \nQ 372 2147 372 2553 \nQ 372 3047 722 3315 \nQ 1072 3584 1716 3584 \nQ 2034 3584 2315 3537 \nQ 2597 3491 2834 3397 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-74\"/>\n      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n     </g>\n    </g>\n    <g id=\"line2d_20\">\n     <path d=\"M 132.417188 34.976562 \nL 142.417188 34.976562 \nL 152.417188 34.976562 \n\" style=\"fill: none; stroke-dasharray: 5.55,2.4; stroke-dashoffset: 0; stroke: #bf00bf; stroke-width: 1.5\"/>\n    </g>\n    <g id=\"text_11\">\n     <!-- valid loss -->\n     <g transform=\"translate(160.417188 38.476562) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-76\" d=\"M 191 3500 \nL 800 3500 \nL 1894 563 \nL 2988 3500 \nL 3597 3500 \nL 2284 0 \nL 1503 0 \nL 191 3500 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-64\" d=\"M 2906 2969 \nL 2906 4863 \nL 3481 4863 \nL 3481 0 \nL 2906 0 \nL 2906 525 \nQ 2725 213 2448 61 \nQ 2172 -91 1784 -91 \nQ 1150 -91 751 415 \nQ 353 922 353 1747 \nQ 353 2572 751 3078 \nQ 1150 3584 1784 3584 \nQ 2172 3584 2448 3432 \nQ 2725 3281 2906 2969 \nz\nM 947 1747 \nQ 947 1113 1208 752 \nQ 1469 391 1925 391 \nQ 2381 391 2643 752 \nQ 2906 1113 2906 1747 \nQ 2906 2381 2643 2742 \nQ 2381 3103 1925 3103 \nQ 1469 3103 1208 2742 \nQ 947 2381 947 1747 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-76\"/>\n      <use xlink:href=\"#DejaVuSans-61\" x=\"59.179688\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" x=\"120.458984\"/>\n      <use xlink:href=\"#DejaVuSans-69\" x=\"148.242188\"/>\n      <use xlink:href=\"#DejaVuSans-64\" x=\"176.025391\"/>\n      <use xlink:href=\"#DejaVuSans-20\" x=\"239.501953\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" x=\"271.289062\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" x=\"299.072266\"/>\n      <use xlink:href=\"#DejaVuSans-73\" x=\"360.253906\"/>\n      <use xlink:href=\"#DejaVuSans-73\" x=\"412.353516\"/>\n     </g>\n    </g>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"p4941e19c63\">\n   <rect x=\"20.5625\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n  </clipPath>\n </defs>\n</svg>\n"},"metadata":{}}],"execution_count":29},{"cell_type":"markdown","source":"## [**Classifying the Testing Set**] and Submitting Results on Kaggle\n\n\nSimilar to the final step in :numref:`sec_kaggle_cifar10`,\nin the end all the labeled data (including the validation set) are used for training the model and classifying the testing set.\nWe will use the trained custom output network\nfor classification.\n","metadata":{"origin_pos":31}},{"cell_type":"code","source":"net = get_net(devices)\ntrain(net, train_valid_iter, None, num_epochs, lr, wd, devices, lr_period,\n      lr_decay)\n\npreds = []\nfor data, label in test_iter:\n    output = torch.nn.functional.softmax(net(data.to(devices[0])), dim=1)\n    preds.extend(output.cpu().detach().numpy())\nids = sorted(os.listdir(\n    os.path.join(data_dir, 'train_valid_test', 'test', 'unknown')))\nwith open('submission.csv', 'w') as f:\n    f.write('id,' + ','.join(train_valid_ds.classes) + '\\n')\n    for i, output in zip(ids, preds):\n        f.write(i.split('.')[0] + ',' + ','.join(\n            [str(num) for num in output]) + '\\n')","metadata":{"origin_pos":33,"tab":["pytorch"],"execution":{"iopub.status.busy":"2025-03-05T16:02:44.794563Z","iopub.execute_input":"2025-03-05T16:02:44.7948Z","iopub.status.idle":"2025-03-05T16:19:52.274234Z","shell.execute_reply.started":"2025-03-05T16:02:44.79478Z","shell.execute_reply":"2025-03-05T16:19:52.273182Z"},"trusted":true},"outputs":[{"name":"stdout","text":"train loss 0.830\n618.5 examples/sec on [device(type='cuda', index=0)]\n","output_type":"stream"},{"output_type":"display_data","data":{"text/plain":"<Figure size 350x250 with 1 Axes>","image/svg+xml":"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n<svg xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"229.425pt\" height=\"183.35625pt\" viewBox=\"0 0 229.425 183.35625\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.1\">\n <metadata>\n  <rdf:RDF xmlns:dc=\"http://purl.org/dc/elements/1.1/\" xmlns:cc=\"http://creativecommons.org/ns#\" xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\">\n   <cc:Work>\n    <dc:type rdf:resource=\"http://purl.org/dc/dcmitype/StillImage\"/>\n    <dc:date>2025-03-05T16:19:52.243816</dc:date>\n    <dc:format>image/svg+xml</dc:format>\n    <dc:creator>\n     <cc:Agent>\n      <dc:title>Matplotlib v3.7.5, https://matplotlib.org/</dc:title>\n     </cc:Agent>\n    </dc:creator>\n   </cc:Work>\n  </rdf:RDF>\n </metadata>\n <defs>\n  <style type=\"text/css\">*{stroke-linejoin: round; stroke-linecap: butt}</style>\n </defs>\n <g id=\"figure_1\">\n  <g id=\"patch_1\">\n   <path d=\"M 0 183.35625 \nL 229.425 183.35625 \nL 229.425 0 \nL 0 0 \nz\n\" style=\"fill: #ffffff\"/>\n  </g>\n  <g id=\"axes_1\">\n   <g id=\"patch_2\">\n    <path d=\"M 20.5625 145.8 \nL 215.8625 145.8 \nL 215.8625 7.2 \nL 20.5625 7.2 \nz\n\" style=\"fill: #ffffff\"/>\n   </g>\n   <g id=\"matplotlib.axis_1\">\n    <g id=\"xtick_1\">\n     <g id=\"line2d_1\">\n      <path d=\"M 61.678289 145.8 \nL 61.678289 7.2 \n\" clip-path=\"url(#p5b9d64c025)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_2\">\n      <defs>\n       <path id=\"mc26acaeb56\" d=\"M 0 0 \nL 0 3.5 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#mc26acaeb56\" x=\"61.678289\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_1\">\n      <!-- 5 -->\n      <g transform=\"translate(58.497039 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-35\" d=\"M 691 4666 \nL 3169 4666 \nL 3169 4134 \nL 1269 4134 \nL 1269 2991 \nQ 1406 3038 1543 3061 \nQ 1681 3084 1819 3084 \nQ 2600 3084 3056 2656 \nQ 3513 2228 3513 1497 \nQ 3513 744 3044 326 \nQ 2575 -91 1722 -91 \nQ 1428 -91 1123 -41 \nQ 819 9 494 109 \nL 494 744 \nQ 775 591 1075 516 \nQ 1375 441 1709 441 \nQ 2250 441 2565 725 \nQ 2881 1009 2881 1497 \nQ 2881 1984 2565 2268 \nQ 2250 2553 1709 2553 \nQ 1456 2553 1204 2497 \nQ 953 2441 691 2322 \nL 691 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-35\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_2\">\n     <g id=\"line2d_3\">\n      <path d=\"M 113.073026 145.8 \nL 113.073026 7.2 \n\" clip-path=\"url(#p5b9d64c025)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_4\">\n      <g>\n       <use xlink:href=\"#mc26acaeb56\" x=\"113.073026\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_2\">\n      <!-- 10 -->\n      <g transform=\"translate(106.710526 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-31\" d=\"M 794 531 \nL 1825 531 \nL 1825 4091 \nL 703 3866 \nL 703 4441 \nL 1819 4666 \nL 2450 4666 \nL 2450 531 \nL 3481 531 \nL 3481 0 \nL 794 0 \nL 794 531 \nz\n\" transform=\"scale(0.015625)\"/>\n        <path id=\"DejaVuSans-30\" d=\"M 2034 4250 \nQ 1547 4250 1301 3770 \nQ 1056 3291 1056 2328 \nQ 1056 1369 1301 889 \nQ 1547 409 2034 409 \nQ 2525 409 2770 889 \nQ 3016 1369 3016 2328 \nQ 3016 3291 2770 3770 \nQ 2525 4250 2034 4250 \nz\nM 2034 4750 \nQ 2819 4750 3233 4129 \nQ 3647 3509 3647 2328 \nQ 3647 1150 3233 529 \nQ 2819 -91 2034 -91 \nQ 1250 -91 836 529 \nQ 422 1150 422 2328 \nQ 422 3509 836 4129 \nQ 1250 4750 2034 4750 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_3\">\n     <g id=\"line2d_5\">\n      <path d=\"M 164.467763 145.8 \nL 164.467763 7.2 \n\" clip-path=\"url(#p5b9d64c025)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_6\">\n      <g>\n       <use xlink:href=\"#mc26acaeb56\" x=\"164.467763\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_3\">\n      <!-- 15 -->\n      <g transform=\"translate(158.105263 160.398438) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n       <use xlink:href=\"#DejaVuSans-35\" x=\"63.623047\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"xtick_4\">\n     <g id=\"line2d_7\">\n      <path d=\"M 215.8625 145.8 \nL 215.8625 7.2 \n\" clip-path=\"url(#p5b9d64c025)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_8\">\n      <g>\n       <use xlink:href=\"#mc26acaeb56\" x=\"215.8625\" y=\"145.8\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_4\">\n      <!-- 20 -->\n      <g transform=\"translate(209.5 160.398438) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-32\" d=\"M 1228 531 \nL 3431 531 \nL 3431 0 \nL 469 0 \nL 469 531 \nQ 828 903 1448 1529 \nQ 2069 2156 2228 2338 \nQ 2531 2678 2651 2914 \nQ 2772 3150 2772 3378 \nQ 2772 3750 2511 3984 \nQ 2250 4219 1831 4219 \nQ 1534 4219 1204 4116 \nQ 875 4013 500 3803 \nL 500 4441 \nQ 881 4594 1212 4672 \nQ 1544 4750 1819 4750 \nQ 2544 4750 2975 4387 \nQ 3406 4025 3406 3419 \nQ 3406 3131 3298 2873 \nQ 3191 2616 2906 2266 \nQ 2828 2175 2409 1742 \nQ 1991 1309 1228 531 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-32\"/>\n       <use xlink:href=\"#DejaVuSans-30\" x=\"63.623047\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"text_5\">\n     <!-- epoch -->\n     <g transform=\"translate(102.984375 174.076563) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-65\" d=\"M 3597 1894 \nL 3597 1613 \nL 953 1613 \nQ 991 1019 1311 708 \nQ 1631 397 2203 397 \nQ 2534 397 2845 478 \nQ 3156 559 3463 722 \nL 3463 178 \nQ 3153 47 2828 -22 \nQ 2503 -91 2169 -91 \nQ 1331 -91 842 396 \nQ 353 884 353 1716 \nQ 353 2575 817 3079 \nQ 1281 3584 2069 3584 \nQ 2775 3584 3186 3129 \nQ 3597 2675 3597 1894 \nz\nM 3022 2063 \nQ 3016 2534 2758 2815 \nQ 2500 3097 2075 3097 \nQ 1594 3097 1305 2825 \nQ 1016 2553 972 2059 \nL 3022 2063 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-70\" d=\"M 1159 525 \nL 1159 -1331 \nL 581 -1331 \nL 581 3500 \nL 1159 3500 \nL 1159 2969 \nQ 1341 3281 1617 3432 \nQ 1894 3584 2278 3584 \nQ 2916 3584 3314 3078 \nQ 3713 2572 3713 1747 \nQ 3713 922 3314 415 \nQ 2916 -91 2278 -91 \nQ 1894 -91 1617 61 \nQ 1341 213 1159 525 \nz\nM 3116 1747 \nQ 3116 2381 2855 2742 \nQ 2594 3103 2138 3103 \nQ 1681 3103 1420 2742 \nQ 1159 2381 1159 1747 \nQ 1159 1113 1420 752 \nQ 1681 391 2138 391 \nQ 2594 391 2855 752 \nQ 3116 1113 3116 1747 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6f\" d=\"M 1959 3097 \nQ 1497 3097 1228 2736 \nQ 959 2375 959 1747 \nQ 959 1119 1226 758 \nQ 1494 397 1959 397 \nQ 2419 397 2687 759 \nQ 2956 1122 2956 1747 \nQ 2956 2369 2687 2733 \nQ 2419 3097 1959 3097 \nz\nM 1959 3584 \nQ 2709 3584 3137 3096 \nQ 3566 2609 3566 1747 \nQ 3566 888 3137 398 \nQ 2709 -91 1959 -91 \nQ 1206 -91 779 398 \nQ 353 888 353 1747 \nQ 353 2609 779 3096 \nQ 1206 3584 1959 3584 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-63\" d=\"M 3122 3366 \nL 3122 2828 \nQ 2878 2963 2633 3030 \nQ 2388 3097 2138 3097 \nQ 1578 3097 1268 2742 \nQ 959 2388 959 1747 \nQ 959 1106 1268 751 \nQ 1578 397 2138 397 \nQ 2388 397 2633 464 \nQ 2878 531 3122 666 \nL 3122 134 \nQ 2881 22 2623 -34 \nQ 2366 -91 2075 -91 \nQ 1284 -91 818 406 \nQ 353 903 353 1747 \nQ 353 2603 823 3093 \nQ 1294 3584 2113 3584 \nQ 2378 3584 2631 3529 \nQ 2884 3475 3122 3366 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-68\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 4863 \nL 1159 4863 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-65\"/>\n      <use xlink:href=\"#DejaVuSans-70\" x=\"61.523438\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" x=\"125\"/>\n      <use xlink:href=\"#DejaVuSans-63\" x=\"186.181641\"/>\n      <use xlink:href=\"#DejaVuSans-68\" x=\"241.162109\"/>\n     </g>\n    </g>\n   </g>\n   <g id=\"matplotlib.axis_2\">\n    <g id=\"ytick_1\">\n     <g id=\"line2d_9\">\n      <path d=\"M 20.5625 132.421512 \nL 215.8625 132.421512 \n\" clip-path=\"url(#p5b9d64c025)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_10\">\n      <defs>\n       <path id=\"m36147dbac1\" d=\"M 0 0 \nL -3.5 0 \n\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </defs>\n      <g>\n       <use xlink:href=\"#m36147dbac1\" x=\"20.5625\" y=\"132.421512\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_6\">\n      <!-- 1 -->\n      <g transform=\"translate(7.2 136.220731) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-31\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_2\">\n     <g id=\"line2d_11\">\n      <path d=\"M 20.5625 100.341483 \nL 215.8625 100.341483 \n\" clip-path=\"url(#p5b9d64c025)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_12\">\n      <g>\n       <use xlink:href=\"#m36147dbac1\" x=\"20.5625\" y=\"100.341483\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_7\">\n      <!-- 2 -->\n      <g transform=\"translate(7.2 104.140702) scale(0.1 -0.1)\">\n       <use xlink:href=\"#DejaVuSans-32\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_3\">\n     <g id=\"line2d_13\">\n      <path d=\"M 20.5625 68.261454 \nL 215.8625 68.261454 \n\" clip-path=\"url(#p5b9d64c025)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_14\">\n      <g>\n       <use xlink:href=\"#m36147dbac1\" x=\"20.5625\" y=\"68.261454\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_8\">\n      <!-- 3 -->\n      <g transform=\"translate(7.2 72.060672) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-33\" d=\"M 2597 2516 \nQ 3050 2419 3304 2112 \nQ 3559 1806 3559 1356 \nQ 3559 666 3084 287 \nQ 2609 -91 1734 -91 \nQ 1441 -91 1130 -33 \nQ 819 25 488 141 \nL 488 750 \nQ 750 597 1062 519 \nQ 1375 441 1716 441 \nQ 2309 441 2620 675 \nQ 2931 909 2931 1356 \nQ 2931 1769 2642 2001 \nQ 2353 2234 1838 2234 \nL 1294 2234 \nL 1294 2753 \nL 1863 2753 \nQ 2328 2753 2575 2939 \nQ 2822 3125 2822 3475 \nQ 2822 3834 2567 4026 \nQ 2313 4219 1838 4219 \nQ 1578 4219 1281 4162 \nQ 984 4106 628 3988 \nL 628 4550 \nQ 988 4650 1302 4700 \nQ 1616 4750 1894 4750 \nQ 2613 4750 3031 4423 \nQ 3450 4097 3450 3541 \nQ 3450 3153 3228 2886 \nQ 3006 2619 2597 2516 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-33\"/>\n      </g>\n     </g>\n    </g>\n    <g id=\"ytick_4\">\n     <g id=\"line2d_15\">\n      <path d=\"M 20.5625 36.181424 \nL 215.8625 36.181424 \n\" clip-path=\"url(#p5b9d64c025)\" style=\"fill: none; stroke: #b0b0b0; stroke-width: 0.8; stroke-linecap: square\"/>\n     </g>\n     <g id=\"line2d_16\">\n      <g>\n       <use xlink:href=\"#m36147dbac1\" x=\"20.5625\" y=\"36.181424\" style=\"stroke: #000000; stroke-width: 0.8\"/>\n      </g>\n     </g>\n     <g id=\"text_9\">\n      <!-- 4 -->\n      <g transform=\"translate(7.2 39.980643) scale(0.1 -0.1)\">\n       <defs>\n        <path id=\"DejaVuSans-34\" d=\"M 2419 4116 \nL 825 1625 \nL 2419 1625 \nL 2419 4116 \nz\nM 2253 4666 \nL 3047 4666 \nL 3047 1625 \nL 3713 1625 \nL 3713 1100 \nL 3047 1100 \nL 3047 0 \nL 2419 0 \nL 2419 1100 \nL 313 1100 \nL 313 1709 \nL 2253 4666 \nz\n\" transform=\"scale(0.015625)\"/>\n       </defs>\n       <use xlink:href=\"#DejaVuSans-34\"/>\n      </g>\n     </g>\n    </g>\n   </g>\n   <g id=\"line2d_17\">\n    <path d=\"M 12.277975 13.5 \nL 14.272398 19.385367 \nL 16.266821 25.676537 \nL 18.261243 32.175477 \nL 20.255666 38.468148 \nL 20.5625 39.346489 \nL 22.556923 77.321663 \nL 24.551345 82.43872 \nL 26.545768 86.365082 \nL 28.54019 89.802205 \nL 30.534613 93.112847 \nL 30.841447 93.558414 \nL 32.83587 110.712378 \nL 34.830293 112.015643 \nL 36.824715 112.904702 \nL 38.819138 114.378598 \nL 40.81356 115.442679 \nL 41.120395 115.558025 \nL 43.114817 119.601988 \nL 45.10924 120.988655 \nL 47.103663 121.126088 \nL 49.098085 122.115137 \nL 51.092508 122.974797 \nL 51.399342 122.984666 \nL 53.393765 127.084547 \nL 55.388187 127.211044 \nL 57.38261 127.628034 \nL 59.377033 127.572723 \nL 61.371455 127.722722 \nL 61.678289 127.600913 \nL 63.672712 130.261638 \nL 65.667135 130.030143 \nL 67.661557 130.4941 \nL 69.65598 130.370541 \nL 71.650403 130.765764 \nL 71.957237 130.777794 \nL 73.951659 132.005196 \nL 75.946082 131.384453 \nL 77.940505 131.678137 \nL 79.934927 131.788866 \nL 81.92935 131.563568 \nL 82.236184 131.615418 \nL 84.230607 133.842123 \nL 86.225029 133.019538 \nL 88.219452 133.115997 \nL 90.213875 133.341156 \nL 92.208297 133.334421 \nL 92.515132 133.340949 \nL 94.509554 135.142929 \nL 96.503977 134.637688 \nL 98.498399 134.612019 \nL 100.492822 134.61859 \nL 102.487245 134.758095 \nL 102.794079 134.827001 \nL 104.788502 135.766655 \nL 106.782924 134.108938 \nL 108.777347 134.197813 \nL 110.771769 134.805294 \nL 112.766192 134.84496 \nL 113.073026 134.841292 \nL 115.067449 135.080227 \nL 117.061872 135.181967 \nL 119.056294 135.265367 \nL 121.050717 135.418859 \nL 123.045139 135.634409 \nL 123.351974 135.636649 \nL 125.346396 135.937466 \nL 127.340819 136.122295 \nL 129.335242 136.268773 \nL 131.329664 136.113967 \nL 133.324087 136.078104 \nL 133.630921 136.057967 \nL 135.625344 136.472796 \nL 137.619766 136.991175 \nL 139.614189 136.730589 \nL 141.608612 136.879469 \nL 143.603034 136.602212 \nL 143.909868 136.632557 \nL 145.904291 138.396264 \nL 147.898714 138.009632 \nL 149.893136 137.615015 \nL 151.887559 137.247398 \nL 153.881982 137.178378 \nL 154.188816 137.231632 \nL 156.183238 137.790062 \nL 158.177661 136.897745 \nL 160.172084 137.140306 \nL 162.166506 137.372889 \nL 164.160929 137.197408 \nL 164.467763 137.23151 \nL 166.462186 137.846694 \nL 168.456608 138.179341 \nL 170.451031 138.632105 \nL 172.445454 137.967465 \nL 174.439876 137.908598 \nL 174.746711 137.90717 \nL 176.741133 138.540751 \nL 178.735556 137.940415 \nL 180.729978 137.853537 \nL 182.724401 137.90367 \nL 184.718824 137.966947 \nL 185.025658 137.912615 \nL 187.020081 137.85756 \nL 189.014503 138.021551 \nL 191.008926 138.441641 \nL 193.003348 138.483873 \nL 194.997771 138.239867 \nL 195.304605 138.102394 \nL 197.299028 139.5 \nL 199.293451 138.202775 \nL 201.287873 138.004096 \nL 203.282296 138.200494 \nL 205.276718 138.27378 \nL 205.583553 138.231872 \nL 207.577975 136.679594 \nL 209.572398 136.890158 \nL 211.566821 137.310149 \nL 213.561243 137.616652 \nL 215.555666 137.860757 \nL 215.8625 137.876055 \n\" clip-path=\"url(#p5b9d64c025)\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n   </g>\n   <g id=\"line2d_18\"/>\n   <g id=\"patch_3\">\n    <path d=\"M 20.5625 145.8 \nL 20.5625 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_4\">\n    <path d=\"M 215.8625 145.8 \nL 215.8625 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_5\">\n    <path d=\"M 20.5625 145.8 \nL 215.8625 145.8 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"patch_6\">\n    <path d=\"M 20.5625 7.2 \nL 215.8625 7.2 \n\" style=\"fill: none; stroke: #000000; stroke-width: 0.8; stroke-linejoin: miter; stroke-linecap: square\"/>\n   </g>\n   <g id=\"legend_1\">\n    <g id=\"patch_7\">\n     <path d=\"M 131.09375 29.878125 \nL 208.8625 29.878125 \nQ 210.8625 29.878125 210.8625 27.878125 \nL 210.8625 14.2 \nQ 210.8625 12.2 208.8625 12.2 \nL 131.09375 12.2 \nQ 129.09375 12.2 129.09375 14.2 \nL 129.09375 27.878125 \nQ 129.09375 29.878125 131.09375 29.878125 \nz\n\" style=\"fill: #ffffff; opacity: 0.8; stroke: #cccccc; stroke-linejoin: miter\"/>\n    </g>\n    <g id=\"line2d_19\">\n     <path d=\"M 133.09375 20.298438 \nL 143.09375 20.298438 \nL 153.09375 20.298438 \n\" style=\"fill: none; stroke: #1f77b4; stroke-width: 1.5; stroke-linecap: square\"/>\n    </g>\n    <g id=\"text_10\">\n     <!-- train loss -->\n     <g transform=\"translate(161.09375 23.798438) scale(0.1 -0.1)\">\n      <defs>\n       <path id=\"DejaVuSans-74\" d=\"M 1172 4494 \nL 1172 3500 \nL 2356 3500 \nL 2356 3053 \nL 1172 3053 \nL 1172 1153 \nQ 1172 725 1289 603 \nQ 1406 481 1766 481 \nL 2356 481 \nL 2356 0 \nL 1766 0 \nQ 1100 0 847 248 \nQ 594 497 594 1153 \nL 594 3053 \nL 172 3053 \nL 172 3500 \nL 594 3500 \nL 594 4494 \nL 1172 4494 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-72\" d=\"M 2631 2963 \nQ 2534 3019 2420 3045 \nQ 2306 3072 2169 3072 \nQ 1681 3072 1420 2755 \nQ 1159 2438 1159 1844 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1341 3275 1631 3429 \nQ 1922 3584 2338 3584 \nQ 2397 3584 2469 3576 \nQ 2541 3569 2628 3553 \nL 2631 2963 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-61\" d=\"M 2194 1759 \nQ 1497 1759 1228 1600 \nQ 959 1441 959 1056 \nQ 959 750 1161 570 \nQ 1363 391 1709 391 \nQ 2188 391 2477 730 \nQ 2766 1069 2766 1631 \nL 2766 1759 \nL 2194 1759 \nz\nM 3341 1997 \nL 3341 0 \nL 2766 0 \nL 2766 531 \nQ 2569 213 2275 61 \nQ 1981 -91 1556 -91 \nQ 1019 -91 701 211 \nQ 384 513 384 1019 \nQ 384 1609 779 1909 \nQ 1175 2209 1959 2209 \nL 2766 2209 \nL 2766 2266 \nQ 2766 2663 2505 2880 \nQ 2244 3097 1772 3097 \nQ 1472 3097 1187 3025 \nQ 903 2953 641 2809 \nL 641 3341 \nQ 956 3463 1253 3523 \nQ 1550 3584 1831 3584 \nQ 2591 3584 2966 3190 \nQ 3341 2797 3341 1997 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-69\" d=\"M 603 3500 \nL 1178 3500 \nL 1178 0 \nL 603 0 \nL 603 3500 \nz\nM 603 4863 \nL 1178 4863 \nL 1178 4134 \nL 603 4134 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6e\" d=\"M 3513 2113 \nL 3513 0 \nL 2938 0 \nL 2938 2094 \nQ 2938 2591 2744 2837 \nQ 2550 3084 2163 3084 \nQ 1697 3084 1428 2787 \nQ 1159 2491 1159 1978 \nL 1159 0 \nL 581 0 \nL 581 3500 \nL 1159 3500 \nL 1159 2956 \nQ 1366 3272 1645 3428 \nQ 1925 3584 2291 3584 \nQ 2894 3584 3203 3211 \nQ 3513 2838 3513 2113 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-20\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-6c\" d=\"M 603 4863 \nL 1178 4863 \nL 1178 0 \nL 603 0 \nL 603 4863 \nz\n\" transform=\"scale(0.015625)\"/>\n       <path id=\"DejaVuSans-73\" d=\"M 2834 3397 \nL 2834 2853 \nQ 2591 2978 2328 3040 \nQ 2066 3103 1784 3103 \nQ 1356 3103 1142 2972 \nQ 928 2841 928 2578 \nQ 928 2378 1081 2264 \nQ 1234 2150 1697 2047 \nL 1894 2003 \nQ 2506 1872 2764 1633 \nQ 3022 1394 3022 966 \nQ 3022 478 2636 193 \nQ 2250 -91 1575 -91 \nQ 1294 -91 989 -36 \nQ 684 19 347 128 \nL 347 722 \nQ 666 556 975 473 \nQ 1284 391 1588 391 \nQ 1994 391 2212 530 \nQ 2431 669 2431 922 \nQ 2431 1156 2273 1281 \nQ 2116 1406 1581 1522 \nL 1381 1569 \nQ 847 1681 609 1914 \nQ 372 2147 372 2553 \nQ 372 3047 722 3315 \nQ 1072 3584 1716 3584 \nQ 2034 3584 2315 3537 \nQ 2597 3491 2834 3397 \nz\n\" transform=\"scale(0.015625)\"/>\n      </defs>\n      <use xlink:href=\"#DejaVuSans-74\"/>\n      <use xlink:href=\"#DejaVuSans-72\" x=\"39.208984\"/>\n      <use xlink:href=\"#DejaVuSans-61\" x=\"80.322266\"/>\n      <use xlink:href=\"#DejaVuSans-69\" x=\"141.601562\"/>\n      <use xlink:href=\"#DejaVuSans-6e\" x=\"169.384766\"/>\n      <use xlink:href=\"#DejaVuSans-20\" x=\"232.763672\"/>\n      <use xlink:href=\"#DejaVuSans-6c\" x=\"264.550781\"/>\n      <use xlink:href=\"#DejaVuSans-6f\" x=\"292.333984\"/>\n      <use xlink:href=\"#DejaVuSans-73\" x=\"353.515625\"/>\n      <use xlink:href=\"#DejaVuSans-73\" x=\"405.615234\"/>\n     </g>\n    </g>\n   </g>\n  </g>\n </g>\n <defs>\n  <clipPath id=\"p5b9d64c025\">\n   <rect x=\"20.5625\" y=\"7.2\" width=\"195.3\" height=\"138.6\"/>\n  </clipPath>\n </defs>\n</svg>\n"},"metadata":{}}],"execution_count":30},{"cell_type":"markdown","source":"The above code\nwill generate a `submission.csv` file\nto be submitted\nto Kaggle in the same way described in :numref:`sec_kaggle_house`.\n\n\n## Summary\n\n\n* Images in the ImageNet dataset are larger (with varying dimensions) than CIFAR-10 images. We may modify image augmentation operations for tasks on a different dataset.\n* To classify a subset of the ImageNet dataset, we can leverage pre-trained models on the full ImageNet dataset to extract features and only train a custom small-scale output network. This will lead to less computational time and memory cost.\n\n","metadata":{"origin_pos":34}},{"cell_type":"markdown","source":"[Discussions](https://discuss.d2l.ai/t/1481)\n","metadata":{"origin_pos":36,"tab":["pytorch"]}}]}