{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":38284,"databundleVersionId":4155799,"sourceType":"competition"}],"dockerImageVersionId":30733,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"},"papermill":{"default_parameters":{},"duration":992.930314,"end_time":"2024-06-09T14:38:00.392662","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-06-09T14:21:27.462348","version":"2.5.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<a href=\"https://www.kaggle.com/code/tunkhanhbi/4-animals?scriptVersionId=182400815\" target=\"_blank\"><img align=\"left\" alt=\"Kaggle\" title=\"Open in Kaggle\" src=\"https://kaggle.com/static/images/open-in-kaggle.svg\"></a>","metadata":{}},{"cell_type":"markdown","source":"# Image classification using 4 animals Kaggle dataset\n\nView dataset description and leaderboard [here](https://www.kaggle.com/competitions/4-animal-classification/)","metadata":{"papermill":{"duration":0.005311,"end_time":"2024-06-09T14:21:30.165388","exception":false,"start_time":"2024-06-09T14:21:30.160077","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## 1. Import modules","metadata":{"papermill":{"duration":0.004445,"end_time":"2024-06-09T14:21:30.17486","exception":false,"start_time":"2024-06-09T14:21:30.170415","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import csv\nimport torch\nfrom torchvision import transforms, models\nfrom torch import nn\nfrom tqdm import tqdm\nfrom glob import glob\nfrom PIL import Image\nimport numpy as np\nfrom sklearn.model_selection import train_test_split\nfrom torch.utils.data import DataLoader, Dataset","metadata":{"papermill":{"duration":5.670814,"end_time":"2024-06-09T14:21:35.850264","exception":false,"start_time":"2024-06-09T14:21:30.17945","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-10T10:51:16.106821Z","iopub.execute_input":"2024-06-10T10:51:16.107656Z","iopub.status.idle":"2024-06-10T10:51:19.354886Z","shell.execute_reply.started":"2024-06-10T10:51:16.107626Z","shell.execute_reply":"2024-06-10T10:51:19.354113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check GPU compatibility\nif torch.cuda.is_available():\n    print(\"GPU is available\")\n    print(\"GPU device:\", torch.cuda.get_device_name(0))\n    print(\"GPU memory:\", round(torch.cuda.get_device_properties(0).total_memory/1024**3),\"GB\")\nelse:\n    print(\"GPU is not available. Using CPU\")\n    \nCUDA = torch.cuda.is_available()\ndevice = \"cuda\" if CUDA else \"cpu\"\n","metadata":{"papermill":{"duration":0.082774,"end_time":"2024-06-09T14:21:35.938317","exception":false,"start_time":"2024-06-09T14:21:35.855543","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-10T10:51:19.356667Z","iopub.execute_input":"2024-06-10T10:51:19.357080Z","iopub.status.idle":"2024-06-10T10:51:19.410974Z","shell.execute_reply.started":"2024-06-10T10:51:19.357055Z","shell.execute_reply":"2024-06-10T10:51:19.410075Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. Process data","metadata":{"papermill":{"duration":0.004517,"end_time":"2024-06-09T14:21:35.947844","exception":false,"start_time":"2024-06-09T14:21:35.943327","status":"completed"},"tags":[]}},{"cell_type":"code","source":"\n# load data\ntest_data = []\ntest_id = []\ntrain_data_x = []\ntrain_data_y = []\n\ntransform = transforms.Compose([\n    transforms.PILToTensor(),\n    transforms.Lambda(lambda x: x.to(device)/255),\n    transforms.Resize((256,256),antialias=True),\n    transforms.CenterCrop((224,224)),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])\n])\n\nfor file in glob(\"/kaggle/input/4-animal-classification/test/test/*\"):\n    # Process image\n    img = Image.open(file)\n    img = transform(img).to(\"cpu\")\n    test_data.append(img)\n    \n    # Process ID\n    id = file.split(\"/\")[-1].split(\".\")[0]\n    test_id.append(int(id))\n    \n    \n\nprint(f\"Processed {len(test_data)} image.\")\n\nanimals = ['cat','deer','dog','horse']\n\nfor label, animal in enumerate(animals):\n    for file in glob(f\"/kaggle/input/4-animal-classification/train/{animal}/*\"):\n        img = Image.open(file)\n        img = transform(img).to(\"cpu\")\n        train_data_x.append(img)\n        train_data_y.append(label)\n\nprint(f\"Processed {len(train_data_x)} image.\")","metadata":{"papermill":{"duration":33.686322,"end_time":"2024-06-09T14:22:09.638943","exception":false,"start_time":"2024-06-09T14:21:35.952621","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-10T10:51:19.411959Z","iopub.execute_input":"2024-06-10T10:51:19.412218Z","iopub.status.idle":"2024-06-10T10:51:37.531133Z","shell.execute_reply.started":"2024-06-10T10:51:19.412195Z","shell.execute_reply":"2024-06-10T10:51:37.530210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert to tensor\ntest_data = torch.tensor(np.array(test_data),dtype=torch.float32)\ntest_id = torch.tensor(np.array(test_id),dtype=torch.long)\ntrain_data_x = torch.tensor(np.array(train_data_x),dtype=torch.float32)\ntrain_data_y = torch.tensor(np.array(train_data_y),dtype=torch.long)\n","metadata":{"papermill":{"duration":1.375528,"end_time":"2024-06-09T14:22:11.029655","exception":false,"start_time":"2024-06-09T14:22:09.654127","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-10T10:51:37.532398Z","iopub.execute_input":"2024-06-10T10:51:37.533688Z","iopub.status.idle":"2024-06-10T10:51:38.946434Z","shell.execute_reply.started":"2024-06-10T10:51:37.533653Z","shell.execute_reply":"2024-06-10T10:51:38.945517Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Train data shape:\",train_data_x.shape)\nprint(\"Train label shape:\",train_data_y.shape)\nprint(\"Test data shape:\",test_data.shape)\nprint(\"Test id shape: \",test_id.shape)","metadata":{"papermill":{"duration":0.01291,"end_time":"2024-06-09T14:22:11.048075","exception":false,"start_time":"2024-06-09T14:22:11.035165","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-10T10:51:38.949441Z","iopub.execute_input":"2024-06-10T10:51:38.950093Z","iopub.status.idle":"2024-06-10T10:51:38.955111Z","shell.execute_reply.started":"2024-06-10T10:51:38.950058Z","shell.execute_reply":"2024-06-10T10:51:38.954229Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# save data \ntorch.save(train_data_x, \"/kaggle/working/train_data_x.pt\")\ntorch.save(train_data_y, \"/kaggle/working/train_data_y.pt\")\ntorch.save(test_data, \"/kaggle/working/test_data.pt\")\ntorch.save(test_id,\"/kaggle/working/test_id.pt\")\n","metadata":{"papermill":{"duration":2.249878,"end_time":"2024-06-09T14:22:13.303162","exception":false,"start_time":"2024-06-09T14:22:11.053284","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-10T10:51:38.956340Z","iopub.execute_input":"2024-06-10T10:51:38.956613Z","iopub.status.idle":"2024-06-10T10:51:42.667758Z","shell.execute_reply.started":"2024-06-10T10:51:38.956586Z","shell.execute_reply":"2024-06-10T10:51:42.666744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 3. Define architectures","metadata":{"papermill":{"duration":0.005121,"end_time":"2024-06-09T14:22:13.31398","exception":false,"start_time":"2024-06-09T14:22:13.308859","status":"completed"},"tags":[]}},{"cell_type":"code","source":"model = models.vgg16(weights='DEFAULT')\nmodel.classifier[6] = nn.Linear(in_features=4096,out_features=4)\nprint(model.forward)","metadata":{"execution":{"iopub.status.busy":"2024-06-10T10:51:42.669142Z","iopub.execute_input":"2024-06-10T10:51:42.669528Z","iopub.status.idle":"2024-06-10T10:51:47.828731Z","shell.execute_reply.started":"2024-06-10T10:51:42.669496Z","shell.execute_reply":"2024-06-10T10:51:47.827758Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4. Train models","metadata":{"papermill":{"duration":0.004971,"end_time":"2024-06-09T14:22:13.35487","exception":false,"start_time":"2024-06-09T14:22:13.349899","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# load train data\ntrain_data_x = torch.load(\"/kaggle/working/train_data_x.pt\").to(device)\ntrain_data_y = torch.load(\"/kaggle/working/train_data_y.pt\").to(device)\n","metadata":{"papermill":{"duration":1.437343,"end_time":"2024-06-09T14:22:14.7974","exception":false,"start_time":"2024-06-09T14:22:13.360057","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-10T10:51:47.829979Z","iopub.execute_input":"2024-06-10T10:51:47.830352Z","iopub.status.idle":"2024-06-10T10:51:49.326862Z","shell.execute_reply.started":"2024-06-10T10:51:47.830320Z","shell.execute_reply":"2024-06-10T10:51:49.326019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# define data loader\nbatch_size = 32\n\ntrain_x, val_x, train_y, val_y = train_test_split(train_data_x,train_data_y, test_size=0.15, random_state=42)\n\nclass Data(Dataset):\n    def __init__(self,data,label):\n        self.data = data\n        self.label = label\n    def __len__(self):\n        return self.data.shape[0]\n    def __getitem__(self,idx):\n        return self.data[idx], self.label[idx]\n\n\ntrain_data = Data(train_x,train_y)\nval_data = Data(val_x,val_y)\n\ntrain_loader = DataLoader(train_data,batch_size=batch_size)\nval_loader = DataLoader(val_data,batch_size=batch_size)\n\n","metadata":{"papermill":{"duration":0.054794,"end_time":"2024-06-09T14:22:14.85824","exception":false,"start_time":"2024-06-09T14:22:14.803446","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-10T10:51:49.328147Z","iopub.execute_input":"2024-06-10T10:51:49.328561Z","iopub.status.idle":"2024-06-10T10:51:52.238591Z","shell.execute_reply.started":"2024-06-10T10:51:49.328535Z","shell.execute_reply":"2024-06-10T10:51:52.237783Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"criterion = nn.CrossEntropyLoss()\noptimizer = torch.optim.Adam(model.parameters(),lr=0.00005)\n\nmodel = model.to(device)\ncriterion = criterion.to(device)","metadata":{"papermill":{"duration":2.564734,"end_time":"2024-06-09T14:22:17.428209","exception":false,"start_time":"2024-06-09T14:22:14.863475","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-10T10:51:52.239744Z","iopub.execute_input":"2024-06-10T10:51:52.240071Z","iopub.status.idle":"2024-06-10T10:51:54.177341Z","shell.execute_reply.started":"2024-06-10T10:51:52.240046Z","shell.execute_reply":"2024-06-10T10:51:54.176133Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"min_val_loss=10000000\nfor epoch in range(20):\n    total_loss_train=0\n    total_acc_train=0\n    \n    model.train()\n    for x, y in train_loader:\n        output = model(x.float())\n\n        batch_loss = criterion(output, y)\n        total_loss_train += batch_loss.item()\n\n        acc = (output.argmax(dim=1)==y).sum().item()\n        total_acc_train += acc\n\n        optimizer.zero_grad()\n        batch_loss.backward()\n        optimizer.step()\n\n    total_loss_val=0\n    total_acc_val=0\n    \n    model.eval()\n    with torch.no_grad():\n        for x, y in val_loader:\n            output = model(x.float())\n            batch_loss = criterion(output, y)\n            total_loss_val += batch_loss.item()\n            \n            acc = (output.argmax(dim=1)==y).sum().item()\n            total_acc_val += acc\n\n    print(\n        f'Epochs: {epoch+1} | Train Loss: {total_loss_train / len(train_x):.3f}\\\n        | Train Accuracy: {total_acc_train/len(train_x):.3f}\\\n        | Val Loss: {total_loss_val/len(val_x):.3f}\\\n        | Val Accuracy:{total_acc_val/len(val_x):.3f}'\n    )\n\n    if min_val_loss>total_loss_val/len(val_x):\n        min_val_loss = total_loss_val/len(val_x)\n        torch.save(model.state_dict(), \"/kaggle/working/model.pt\")\n        print(f\"Save model because val loss improve loss {min_val_loss:.3f}\")\n    \n    print(\"-\"*50)","metadata":{"papermill":{"duration":941.204517,"end_time":"2024-06-09T14:37:58.638346","exception":false,"start_time":"2024-06-09T14:22:17.433829","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-06-10T10:51:54.181206Z","iopub.execute_input":"2024-06-10T10:51:54.181582Z","iopub.status.idle":"2024-06-10T10:55:24.461458Z","shell.execute_reply.started":"2024-06-10T10:51:54.181556Z","shell.execute_reply":"2024-06-10T10:55:24.460477Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 5. Evaluate test","metadata":{}},{"cell_type":"code","source":"# load test data\ntest_data = torch.load(\"/kaggle/working/test_data.pt\").to(device)\ntest_id = torch.load(\"/kaggle/working/test_id.pt\")","metadata":{"execution":{"iopub.status.busy":"2024-06-10T10:55:24.462759Z","iopub.execute_input":"2024-06-10T10:55:24.463173Z","iopub.status.idle":"2024-06-10T10:55:24.846248Z","shell.execute_reply.started":"2024-06-10T10:55:24.463138Z","shell.execute_reply":"2024-06-10T10:55:24.845460Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# create data loader\ntest_loader = DataLoader(Data(test_id,test_data),batch_size=batch_size)","metadata":{"execution":{"iopub.status.busy":"2024-06-10T10:55:24.847347Z","iopub.execute_input":"2024-06-10T10:55:24.847662Z","iopub.status.idle":"2024-06-10T10:55:24.852172Z","shell.execute_reply.started":"2024-06-10T10:55:24.847637Z","shell.execute_reply":"2024-06-10T10:55:24.851200Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# load model best state dict\nmodel.load_state_dict(torch.load(\"/kaggle/working/model.pt\"))\nmodel = model.to(device)","metadata":{"execution":{"iopub.status.busy":"2024-06-10T10:55:24.855793Z","iopub.execute_input":"2024-06-10T10:55:24.856119Z","iopub.status.idle":"2024-06-10T10:55:25.259908Z","shell.execute_reply.started":"2024-06-10T10:55:24.856097Z","shell.execute_reply":"2024-06-10T10:55:25.259081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# evaluate and write to csv file\nwith open(\"/kaggle/working/submission.csv\",\"w\") as f:\n    csvwriter = csv.writer(f)\n    csvwriter.writerow([\"Id\",\"Label\"])\n    model.eval()\n    with torch.no_grad():\n        for id_list, img in test_loader:\n            output = model(img)\n            label_list = output.argmax(dim=1)\n            for id, label in zip(id_list,label_list):\n                csvwriter.writerow([id.item(),label.item()])","metadata":{"execution":{"iopub.status.busy":"2024-06-10T10:55:25.261078Z","iopub.execute_input":"2024-06-10T10:55:25.261540Z","iopub.status.idle":"2024-06-10T10:55:27.352136Z","shell.execute_reply.started":"2024-06-10T10:55:25.261503Z","shell.execute_reply":"2024-06-10T10:55:27.351078Z"},"trusted":true},"execution_count":null,"outputs":[]}]}