{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.10.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"23719cb1-73d0-4781-8239-9de8075b94fe","cell_type":"markdown","source":"# Lab 6 — Classic CNN Architectures + ResNet18 Fine-Tuning (Complete Solution)\n\n> **How to run**: import this notebook on Kaggle, attach the datasets you used for notebooks 6-1/3/5/7 (`flowers-recognition`, `images`, `bengaliai-cv19`, and `trained400` for Part 4's second half), plus `lab4-fruit` (thomaschxu) for Part 5. Turn on GPU and run all cells. Every figure title carries the personal watermark `2330026139`, so the screenshots can be used in the report directly.\n\n- Part 1 — LeNet-5 on MNIST (PyTorch)\n- Part 2 — AlexNet (pretrained) vs custom CNN on Flowers\n- Part 3 — VGG-style network on a food subset\n- Part 4 — ResNet18/34 on BengaliAI (Kaggle competition)\n- Part 5 — ResNet18 fine-tuning on 36 Fruit/Veggie (Intermediate & Advanced)","metadata":{}},{"id":"a89c9fe8-53b5-46e5-9652-655e699ad80f","cell_type":"markdown","source":"\n---\n\n# Part 1 — LeNet-5 on MNIST (PyTorch)","metadata":{}},{"id":"13fea8f6-7883-4e5a-a21e-1339339748ae","cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport os\n# (optional) list all input files\n# for dirname, _, filenames in os.walk('/kaggle/input'):\n#     for filename in filenames:\n#         print(os.path.join(dirname, filename))","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2025-10-23T04:13:46.390526Z","iopub.execute_input":"2025-10-23T04:13:46.390759Z","iopub.status.idle":"2025-10-23T04:13:46.414075Z","shell.execute_reply.started":"2025-10-23T04:13:46.390696Z","shell.execute_reply":"2025-10-23T04:13:46.413313Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"0f483675-4f25-42f3-8d66-eda2ea8e90fe","cell_type":"markdown","source":"# About data","metadata":{}},{"id":"8c704202-bb21-4654-b92b-982d49ee2a5a","cell_type":"markdown","source":"The data files train.csv and test.csv contain gray-scale images of hand-drawn digits, from zero through nine.\n\nEach image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.\n\nThe training data set, (train.csv), has 785 columns. The first column, called \"label\", is the digit that was drawn by the user. The rest of the columns contain the pixel-values of the associated image.\n\nEach pixel column in the training set has a name like pixelx, where x is an integer between 0 and 783, inclusive. To locate this pixel on the image, suppose that we have decomposed x as x = i * 28 + j, where i and j are integers between 0 and 27, inclusive. Then pixelx is located on row i and column j of a 28 x 28 matrix, (indexing by zero).\n\nFor example, pixel31 indicates the pixel that is in the fourth column from the left, and the second row from the top, as in the ascii-diagram below.","metadata":{}},{"id":"32627c16-f2fc-4b6d-b031-f6dd3c08102a","cell_type":"markdown","source":"## What is Lenet5?\nLenet-5 is one of the earliest pre-trained models proposed by Yann LeCun and others in the year 1998, in the research paper Gradient-Based Learning Applied to Document Recognition. They used this architecture for recognizing the handwritten and machine-printed characters.","metadata":{}},{"id":"e622f034-816d-405e-abed-fce0284af48b","cell_type":"markdown","source":"The first layer is the input layer with feature map size 32X32X1.\n\nThen we have the first convolution layer with 6 filters of size 5X5 and stride is 1. The activation function used at his layer is tanh. The output feature map is  28X28X6.\n\nNext, we have an average pooling layer with filter size 2X2 and stride 1. The resulting feature map is 14X14X6. Since the pooling layer doesn’t affect the number of channels.\n\nAfter this comes the second convolution layer with 16 filters of 5X5 and stride 1. Also, the activation function is tanh. Now the output size is 10X10X16.\n\nAgain comes the other average pooling layer of 2X2 with stride 2. As a result, the size of the feature map reduced to 5X5X16.\n\nThe final pooling layer has 120 filters of 5X5  with stride 1 and activation function tanh. Now the output size is 120.\n\nThe next is a fully connected layer with 84 neurons that result in the output to 84 values and the activation function used here is again tanh.\n\nThe last layer is the output layer with 10 neurons and  Softmax function. The Softmax gives the probability that a data point belongs to a particular class. The highest value is then predicted.\n\nThis is the entire architecture of the Lenet-5 model. The number of trainable parameters of this architecture is around sixty thousand.","metadata":{}},{"id":"9ee06d07-082c-4c58-8d93-ae6e84e416ea","cell_type":"markdown","source":"If you like this kindly Consider Upvoting!! Happy learning!!","metadata":{}},{"id":"96ed9431-935d-4b47-8b7c-b21ec53933f8","cell_type":"markdown","source":"# Important Library Loading & Data Loading","metadata":{}},{"id":"13c9e9ca-e5c9-4ec5-83f2-e6a42d8a377c","cell_type":"code","source":"# Pytorch in python can be accessed by Torch library\nimport torch","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:46.41551Z","iopub.execute_input":"2025-10-23T04:13:46.415696Z","iopub.status.idle":"2025-10-23T04:13:47.976784Z","shell.execute_reply.started":"2025-10-23T04:13:46.415674Z","shell.execute_reply":"2025-10-23T04:13:47.976205Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"6399b839-f3fb-41a4-a2d9-9438f7e5481b","cell_type":"code","source":"# Check current torch version\ntorch.__version__","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:47.9777Z","iopub.execute_input":"2025-10-23T04:13:47.977899Z","iopub.status.idle":"2025-10-23T04:13:47.983938Z","shell.execute_reply.started":"2025-10-23T04:13:47.977871Z","shell.execute_reply":"2025-10-23T04:13:47.983297Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"d5a8d168-90a1-47f0-a39d-53d376d1b694","cell_type":"code","source":"# In case Nvidea Cuda available, this will give True as result else False\ntorch.cuda.is_available()","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:47.984815Z","iopub.execute_input":"2025-10-23T04:13:47.985014Z","iopub.status.idle":"2025-10-23T04:13:48.040913Z","shell.execute_reply.started":"2025-10-23T04:13:47.984964Z","shell.execute_reply":"2025-10-23T04:13:48.040238Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"acc2162f-ca22-4e95-afe0-ae1b9ba2e89b","cell_type":"markdown","source":"Torch Vision is library to work with Images in Pytorch","metadata":{}},{"id":"e047221d-e86c-4da6-8319-5862448669d7","cell_type":"code","source":"import torchvision\n# transform is used to convert data into Tensor form with transformations\nimport torchvision.transforms as transforms","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:48.041887Z","iopub.execute_input":"2025-10-23T04:13:48.042201Z","iopub.status.idle":"2025-10-23T04:13:48.304169Z","shell.execute_reply.started":"2025-10-23T04:13:48.042168Z","shell.execute_reply":"2025-10-23T04:13:48.303406Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"92285a15-ece5-4b9b-a6ba-061162283c55","cell_type":"markdown","source":"Torchvision has inbuilt Dataset where Mnist dataset is available, Lets import data\n","metadata":{}},{"id":"7a72950c-eaf3-43a4-9f57-d2dba99ac5f6","cell_type":"code","source":"import torchvision\nimport torchvision.transforms as transforms","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:48.305159Z","iopub.execute_input":"2025-10-23T04:13:48.305346Z","iopub.status.idle":"2025-10-23T04:13:48.309015Z","shell.execute_reply.started":"2025-10-23T04:13:48.305322Z","shell.execute_reply":"2025-10-23T04:13:48.308229Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"9f1c417b-7b7c-4bca-92cc-ef3172af3ed7","cell_type":"code","source":"train_set = torchvision.datasets.MNIST(\nroot = './data',\ntrain = True,\ndownload = True,\ntransform = transforms.Compose([transforms.ToTensor()])\n)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:48.311825Z","iopub.execute_input":"2025-10-23T04:13:48.312342Z","iopub.status.idle":"2025-10-23T04:13:57.995865Z","shell.execute_reply.started":"2025-10-23T04:13:48.312314Z","shell.execute_reply":"2025-10-23T04:13:57.995143Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"51f2fbd4-1c05-4ebc-8df8-96f01b9b48b1","cell_type":"code","source":"# Length of train set\nlen(train_set)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:57.99699Z","iopub.execute_input":"2025-10-23T04:13:57.997318Z","iopub.status.idle":"2025-10-23T04:13:58.002297Z","shell.execute_reply.started":"2025-10-23T04:13:57.997281Z","shell.execute_reply":"2025-10-23T04:13:58.001647Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"3e64a300-8a35-4d50-96a6-9c5e4c3af120","cell_type":"markdown","source":"Let's define a function which can help us to plot an Image and hence reproduceable\n","metadata":{}},{"id":"ea28dc81-f17f-4631-b20a-9f56914bb3f1","cell_type":"markdown","source":"# Data Study","metadata":{}},{"id":"faf94bf4-4c8c-406b-b8fd-a3fef5b94ee1","cell_type":"code","source":"# Lets study first image\n\nimg, label = train_set[0]\nimg.shape, label","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.003246Z","iopub.execute_input":"2025-10-23T04:13:58.003452Z","iopub.status.idle":"2025-10-23T04:13:58.031347Z","shell.execute_reply.started":"2025-10-23T04:13:58.003429Z","shell.execute_reply":"2025-10-23T04:13:58.030708Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"defb665f-4d16-45b9-b8e6-983e5735362f","cell_type":"markdown","source":"So each Image is 28*28 pixel and Its Gray scale image, as color channel is 1","metadata":{}},{"id":"63f6310e-c54e-45ab-8c6e-27a13234d8e4","cell_type":"code","source":"def show_img(img, label):\n    print('Label: ', label)\n    plt.imshow(img.permute(1,2,0), cmap = 'gray')","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.032313Z","iopub.execute_input":"2025-10-23T04:13:58.03276Z","iopub.status.idle":"2025-10-23T04:13:58.03676Z","shell.execute_reply.started":"2025-10-23T04:13:58.032724Z","shell.execute_reply":"2025-10-23T04:13:58.036022Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"77b7b2f9-2094-4a57-be35-e50cd4ae838a","cell_type":"markdown","source":"As torch has image representation as [color channel, Height, Width] but Maplotlib accepts [height, width, color channel], hence above .permute() function does that.","metadata":{}},{"id":"0a2e4953-de99-4b33-9106-9054d314c1d7","cell_type":"code","source":"# Without these libraries, we cant think of running any Data analysis related python program\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.03781Z","iopub.execute_input":"2025-10-23T04:13:58.038382Z","iopub.status.idle":"2025-10-23T04:13:58.047091Z","shell.execute_reply.started":"2025-10-23T04:13:58.038354Z","shell.execute_reply":"2025-10-23T04:13:58.046481Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"d4a71947-8e9d-4b72-b69f-c79ebf3d8672","cell_type":"code","source":"# lets observe some images\nshow_img(*train_set[1])","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.047988Z","iopub.execute_input":"2025-10-23T04:13:58.048224Z","iopub.status.idle":"2025-10-23T04:13:58.253624Z","shell.execute_reply.started":"2025-10-23T04:13:58.048193Z","shell.execute_reply":"2025-10-23T04:13:58.252987Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"679c66de-0dcc-4edc-800a-30c50d4a0370","cell_type":"code","source":"show_img(*train_set[19992])","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.254412Z","iopub.execute_input":"2025-10-23T04:13:58.254615Z","iopub.status.idle":"2025-10-23T04:13:58.438144Z","shell.execute_reply.started":"2025-10-23T04:13:58.254585Z","shell.execute_reply":"2025-10-23T04:13:58.437437Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"98e32c66-7800-463e-9f14-6a4b74c15f86","cell_type":"markdown","source":"In Neural Network we have to perform Data Normalization to reduce possibility of Over Fitting, also CNN algorithms use Gradient boosting, So its must to perform Normalization else some variable may be given higher weightage.","metadata":{}},{"id":"4907e9d3-557d-40db-971b-54f3a1c3bbd0","cell_type":"markdown","source":"# Data Normalization","metadata":{}},{"id":"cf8626e3-737f-4105-98b7-ae2d4c87bf65","cell_type":"code","source":"trans = transforms.Compose([\n    # To resize image\n    transforms.Resize((32,32)),\n    transforms.ToTensor(),\n    # To normalize image\n    transforms.Normalize((0.5,), (0.5,))\n])","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.439242Z","iopub.execute_input":"2025-10-23T04:13:58.43952Z","iopub.status.idle":"2025-10-23T04:13:58.44389Z","shell.execute_reply.started":"2025-10-23T04:13:58.439488Z","shell.execute_reply":"2025-10-23T04:13:58.443256Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"1344ce58-f661-4d73-8d5e-1622bc598e00","cell_type":"code","source":"train_set = torchvision.datasets.MNIST(\nroot = './data',\ntrain = True,\ndownload = True,\ntransform = trans\n)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.444986Z","iopub.execute_input":"2025-10-23T04:13:58.445223Z","iopub.status.idle":"2025-10-23T04:13:58.490079Z","shell.execute_reply.started":"2025-10-23T04:13:58.445193Z","shell.execute_reply":"2025-10-23T04:13:58.489541Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"9e5596ac-7bd2-4ce4-9bb7-e9577f5af96e","cell_type":"code","source":"test_set = torchvision.datasets.MNIST(\nroot = './data',\ntrain = False,\ndownload = True,\ntransform = trans\n)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.491037Z","iopub.execute_input":"2025-10-23T04:13:58.491638Z","iopub.status.idle":"2025-10-23T04:13:58.50148Z","shell.execute_reply.started":"2025-10-23T04:13:58.491602Z","shell.execute_reply":"2025-10-23T04:13:58.500891Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"dbd143c0-be8e-4d70-807a-396166562039","cell_type":"code","source":"len(train_set), len(test_set)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.502418Z","iopub.execute_input":"2025-10-23T04:13:58.502648Z","iopub.status.idle":"2025-10-23T04:13:58.508441Z","shell.execute_reply.started":"2025-10-23T04:13:58.502615Z","shell.execute_reply":"2025-10-23T04:13:58.507784Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"5375df1f-8414-4ecb-a171-e192f5a991a4","cell_type":"markdown","source":"Lets observe some changes","metadata":{}},{"id":"bdfa93fc-e3f7-4819-8370-d379032b2f66","cell_type":"code","source":"img, label = train_set[0]\nimg.shape, label","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.50944Z","iopub.execute_input":"2025-10-23T04:13:58.509672Z","iopub.status.idle":"2025-10-23T04:13:58.526862Z","shell.execute_reply.started":"2025-10-23T04:13:58.509639Z","shell.execute_reply":"2025-10-23T04:13:58.526314Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"fbb0c0e7-e0fc-4b1e-8b14-c234f626e3de","cell_type":"markdown","source":"So now we have images with 32*32 pixels","metadata":{}},{"id":"c30774f1-e243-47eb-88f8-d902fb663007","cell_type":"code","source":"show_img(*train_set[0])","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.527615Z","iopub.execute_input":"2025-10-23T04:13:58.527799Z","iopub.status.idle":"2025-10-23T04:13:58.716894Z","shell.execute_reply.started":"2025-10-23T04:13:58.527778Z","shell.execute_reply":"2025-10-23T04:13:58.716294Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"e0776f79-83ec-446c-95a1-e6978c25757b","cell_type":"code","source":"show_img(*train_set[9999])","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.717614Z","iopub.execute_input":"2025-10-23T04:13:58.717771Z","iopub.status.idle":"2025-10-23T04:13:58.911369Z","shell.execute_reply.started":"2025-10-23T04:13:58.717752Z","shell.execute_reply":"2025-10-23T04:13:58.910679Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"83aab983-286d-4cc3-abdb-5a618190a2f2","cell_type":"code","source":"show_img(*test_set[5999])","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:58.912204Z","iopub.execute_input":"2025-10-23T04:13:58.912408Z","iopub.status.idle":"2025-10-23T04:13:59.028895Z","shell.execute_reply.started":"2025-10-23T04:13:58.912375Z","shell.execute_reply":"2025-10-23T04:13:59.028136Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"87a5ebeb-e6f7-47d6-be20-8df51b433896","cell_type":"markdown","source":"# Train & Validation data Split","metadata":{}},{"id":"27041349-6e0c-42b0-8cd9-8ef7acabf2b6","cell_type":"code","source":"# this function will generate random indexes between 0 and 59999\ndef split_indices(n, val_per, seed = 0):\n    n_val = int(n * val_per)\n    np.random.seed(seed)\n    idx = np.random.permutation(n)\n    return idx[n_val : ], idx[: n_val]","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.033279Z","iopub.execute_input":"2025-10-23T04:13:59.033473Z","iopub.status.idle":"2025-10-23T04:13:59.037714Z","shell.execute_reply.started":"2025-10-23T04:13:59.033446Z","shell.execute_reply":"2025-10-23T04:13:59.036994Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"c9993e90-266e-4172-8847-aa83bee625e0","cell_type":"code","source":"val_per = 0.2\nrand_seed = 42\n\ntrain_indices, val_indices = split_indices(len(train_set), val_per, rand_seed)\n\nprint(len(train_indices), len(val_indices))","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.038541Z","iopub.execute_input":"2025-10-23T04:13:59.038771Z","iopub.status.idle":"2025-10-23T04:13:59.052571Z","shell.execute_reply.started":"2025-10-23T04:13:59.03874Z","shell.execute_reply":"2025-10-23T04:13:59.051646Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"d0bb46fd-a1ee-4500-8afb-b52c3bfc9c23","cell_type":"code","source":"# Lets plot some indexes\n\nprint(\"Validation Indices: \", val_indices[:20])\nprint(\"Training Indices: \", train_indices[:20])","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.053583Z","iopub.execute_input":"2025-10-23T04:13:59.053814Z","iopub.status.idle":"2025-10-23T04:13:59.064298Z","shell.execute_reply.started":"2025-10-23T04:13:59.053781Z","shell.execute_reply":"2025-10-23T04:13:59.063577Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"0fdc6d39-fff3-4738-bcda-db3aba9e3722","cell_type":"markdown","source":"It seems indexes are distributed across range as provided","metadata":{}},{"id":"4a6699ab-a7f1-4b20-aed8-e1a1ec5d5326","cell_type":"markdown","source":"Let's Import libraries to generate Random Subset & dataLoader to feed Batch of data to model, as feeding whole dataset may lead to System failure or Hang\n","metadata":{}},{"id":"63f1afd6-c976-4074-a569-ff2fbc28625f","cell_type":"code","source":"from torch.utils.data.sampler import SubsetRandomSampler\nfrom torch.utils.data.dataloader import DataLoader","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.065341Z","iopub.execute_input":"2025-10-23T04:13:59.065597Z","iopub.status.idle":"2025-10-23T04:13:59.07396Z","shell.execute_reply.started":"2025-10-23T04:13:59.065558Z","shell.execute_reply":"2025-10-23T04:13:59.073238Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"875cd47f-3aab-44c6-970f-953a842b3005","cell_type":"code","source":"# this is one of Hyper parameter, but let's select given below value\nbatch_size = 512","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.074917Z","iopub.execute_input":"2025-10-23T04:13:59.075127Z","iopub.status.idle":"2025-10-23T04:13:59.088719Z","shell.execute_reply.started":"2025-10-23T04:13:59.075105Z","shell.execute_reply":"2025-10-23T04:13:59.088161Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"3ca4872a-5751-48a1-8cf4-7cad493f7a4c","cell_type":"code","source":"# training data loader\ntrain_sampler = SubsetRandomSampler(train_indices)\ntrain_dl = DataLoader(train_set, batch_size, sampler = train_sampler)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.089467Z","iopub.execute_input":"2025-10-23T04:13:59.089633Z","iopub.status.idle":"2025-10-23T04:13:59.10098Z","shell.execute_reply.started":"2025-10-23T04:13:59.089611Z","shell.execute_reply":"2025-10-23T04:13:59.100422Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"b6509ea9-b36e-40f4-a3d8-15100353b865","cell_type":"code","source":"# validation dataloader\nval_sampler = SubsetRandomSampler(val_indices)\nval_dl = DataLoader(train_set, batch_size, sampler = val_sampler)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.101811Z","iopub.execute_input":"2025-10-23T04:13:59.10208Z","iopub.status.idle":"2025-10-23T04:13:59.110932Z","shell.execute_reply.started":"2025-10-23T04:13:59.102042Z","shell.execute_reply":"2025-10-23T04:13:59.110333Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"0e256aca-c45a-49ed-927d-9b91f598ec84","cell_type":"markdown","source":"Let's verify where we have all class from output columns in both train and validation set\n","metadata":{}},{"id":"1e03b843-e598-43d2-a3f9-91e649156286","cell_type":"code","source":"from torchvision.utils import make_grid\n# this will help us to create Grid of images","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.111673Z","iopub.execute_input":"2025-10-23T04:13:59.111819Z","iopub.status.idle":"2025-10-23T04:13:59.120491Z","shell.execute_reply.started":"2025-10-23T04:13:59.1118Z","shell.execute_reply":"2025-10-23T04:13:59.119854Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"324a51fb-071f-4c1a-a488-d09750fda183","cell_type":"code","source":"# We will select first 110 image from first batch of size = 512\ndef show_batch(dl):\n    for img, label in dl:\n        fig, ax = plt.subplots(figsize = (12,8))\n        ax.imshow(make_grid(img[:110], 10).permute(1,2,0))\n        break","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.121292Z","iopub.execute_input":"2025-10-23T04:13:59.121475Z","iopub.status.idle":"2025-10-23T04:13:59.130627Z","shell.execute_reply.started":"2025-10-23T04:13:59.121454Z","shell.execute_reply":"2025-10-23T04:13:59.129949Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"eb7b42f4-f266-4ac4-8e6b-38227317c4fb","cell_type":"code","source":"show_batch(val_dl)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.131315Z","iopub.execute_input":"2025-10-23T04:13:59.131495Z","iopub.status.idle":"2025-10-23T04:13:59.623429Z","shell.execute_reply.started":"2025-10-23T04:13:59.131469Z","shell.execute_reply":"2025-10-23T04:13:59.622765Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"f11770f5-93ab-4dc0-9b40-d3d4c83959a5","cell_type":"code","source":"show_batch(train_dl)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:13:59.624665Z","iopub.execute_input":"2025-10-23T04:13:59.624924Z","iopub.status.idle":"2025-10-23T04:14:00.017314Z","shell.execute_reply.started":"2025-10-23T04:13:59.624887Z","shell.execute_reply":"2025-10-23T04:14:00.016685Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"4f8b9577-b9d1-4b25-8fee-b268402c7cfb","cell_type":"markdown","source":"Images seems equally Distributed\n","metadata":{}},{"id":"1f50c394-8f21-4aca-b8d7-b5d487c41a6c","cell_type":"markdown","source":"# Model Building using Torch.nn","metadata":{}},{"id":"c7bb472d-a43d-4f67-8fc5-c9d0536f9c49","cell_type":"code","source":"import torch.nn as nn\nimport torch.nn.functional as F","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.01851Z","iopub.execute_input":"2025-10-23T04:14:00.018781Z","iopub.status.idle":"2025-10-23T04:14:00.022876Z","shell.execute_reply.started":"2025-10-23T04:14:00.018745Z","shell.execute_reply":"2025-10-23T04:14:00.022096Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"fcbd9339-6fbf-4874-9beb-d522f87e37df","cell_type":"markdown","source":"This is how Yen LeKun has decribed and built First Neural network in 1988, that was probably first working Neural model","metadata":{}},{"id":"6c900ab3-6e5f-488c-a6b9-cc4ed6f95647","cell_type":"code","source":"class LeNet5(nn.Module):\n    \n    def __init__(self, num_classes):\n        \n        super().__init__()\n        \n        self.num_classes = num_classes\n        \n        self.features = nn.Sequential(\n            nn.Conv2d(1, 6, kernel_size = 5),\n            nn.Tanh(),\n            nn.MaxPool2d(kernel_size = 2),\n            nn.Conv2d(6, 16, kernel_size = 5),\n            nn.Tanh(),\n            nn.MaxPool2d(kernel_size = 2)\n        )\n        \n        self.classifier = nn.Sequential(\n            nn.Linear(16*5*5, 120),\n            nn.Tanh(),\n            nn.Linear(120, 84),\n            nn.Tanh(),\n            nn.Linear(84, num_classes)  \n        )\n        \n        \n        \n    def forward(self, x):\n        x = self.features(x)\n        x = torch.flatten(x, 1)\n        logit = self.classifier(x)\n        return logit","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.023939Z","iopub.execute_input":"2025-10-23T04:14:00.024192Z","iopub.status.idle":"2025-10-23T04:14:00.034771Z","shell.execute_reply.started":"2025-10-23T04:14:00.024158Z","shell.execute_reply":"2025-10-23T04:14:00.034161Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"27645176-d2b6-4a7d-93a0-0ad28fd7fa54","cell_type":"code","source":"# Load Model\nmodel = LeNet5(num_classes = 10)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.035533Z","iopub.execute_input":"2025-10-23T04:14:00.035722Z","iopub.status.idle":"2025-10-23T04:14:00.052377Z","shell.execute_reply.started":"2025-10-23T04:14:00.03569Z","shell.execute_reply":"2025-10-23T04:14:00.051806Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"179873fb-7ae2-474d-a6bb-09929bf78f0f","cell_type":"code","source":"model","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.053308Z","iopub.execute_input":"2025-10-23T04:14:00.053727Z","iopub.status.idle":"2025-10-23T04:14:00.062374Z","shell.execute_reply.started":"2025-10-23T04:14:00.053692Z","shell.execute_reply":"2025-10-23T04:14:00.061651Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"0ccdc8e1-c850-4741-8359-2bd0cfb518bc","cell_type":"markdown","source":"Let's test model based on initial Guesses by Pytorch\n","metadata":{}},{"id":"2126d357-178e-43f9-b267-d7ce64b3a70f","cell_type":"code","source":"sample = next(iter(train_set))\nimg = sample[0]\nimg.shape","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.063616Z","iopub.execute_input":"2025-10-23T04:14:00.064094Z","iopub.status.idle":"2025-10-23T04:14:00.072923Z","shell.execute_reply.started":"2025-10-23T04:14:00.064056Z","shell.execute_reply":"2025-10-23T04:14:00.072272Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"a46839e7-9872-4a19-889b-e32f04f59d17","cell_type":"markdown","source":"We need to import one extra dimension as Model need shape as [batch_size COlor Channel heigh * Width]\n\n.unsqueeze() from torch help to add extra dimension","metadata":{}},{"id":"b72e6d00-05bb-458d-90b4-cf51ccaf8293","cell_type":"code","source":"img.unsqueeze(0).shape\n# Now we have [1, 1, 32, 32] shape of image","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.073752Z","iopub.execute_input":"2025-10-23T04:14:00.073989Z","iopub.status.idle":"2025-10-23T04:14:00.082433Z","shell.execute_reply.started":"2025-10-23T04:14:00.073941Z","shell.execute_reply":"2025-10-23T04:14:00.081737Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"c05bee0d-1f76-44e5-b20e-c2a0b16a73df","cell_type":"markdown","source":"Lets see initial guesses from model","metadata":{}},{"id":"5ca26493-2e4c-4cab-a2c9-7762076cebc9","cell_type":"code","source":"out = model(img.unsqueeze(0))\nout","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.083309Z","iopub.execute_input":"2025-10-23T04:14:00.083848Z","iopub.status.idle":"2025-10-23T04:14:00.194651Z","shell.execute_reply.started":"2025-10-23T04:14:00.083815Z","shell.execute_reply":"2025-10-23T04:14:00.194035Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"afd33677-2030-44a9-ad5f-a10a279cff3f","cell_type":"code","source":"# Output In terms of Probability\nF.softmax(out)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.195422Z","iopub.execute_input":"2025-10-23T04:14:00.195611Z","iopub.status.idle":"2025-10-23T04:14:00.20272Z","shell.execute_reply.started":"2025-10-23T04:14:00.195584Z","shell.execute_reply":"2025-10-23T04:14:00.201908Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"76f6365e-87f8-4408-8afb-1a9641fe325f","cell_type":"markdown","source":"It seems model predict index 7, i.e. letter 7 as output, its initial guess !!","metadata":{}},{"id":"2a5d545f-9add-48fb-a9a2-86e7a60cdb40","cell_type":"code","source":" # Demo function to test result\n\nfor images, labels in train_dl:\n    print('Image Shape', images.shape)\n    out = model(images)\n    print('output shape', out.shape)\n    print('out[0]', out[0])\n    break","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.203663Z","iopub.execute_input":"2025-10-23T04:14:00.203836Z","iopub.status.idle":"2025-10-23T04:14:00.485784Z","shell.execute_reply.started":"2025-10-23T04:14:00.203815Z","shell.execute_reply":"2025-10-23T04:14:00.485046Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"6035b177-46e4-4904-86b2-2f0ae10445ca","cell_type":"code","source":"probs = F.softmax(out[0], dim = 0)\nprobs","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.486921Z","iopub.execute_input":"2025-10-23T04:14:00.487199Z","iopub.status.idle":"2025-10-23T04:14:00.493655Z","shell.execute_reply.started":"2025-10-23T04:14:00.487162Z","shell.execute_reply":"2025-10-23T04:14:00.492925Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"24a1749f-c049-4dbc-8de1-7fa6c7c8284e","cell_type":"markdown","source":"Initial guess from model suggest, probability of each class as 10%, which is kind of 1st prediction from model\n","metadata":{}},{"id":"bbfaad11-9396-4363-b131-f594549c29df","cell_type":"code","source":"m = torch.argmax(probs)\nm","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.494763Z","iopub.execute_input":"2025-10-23T04:14:00.495108Z","iopub.status.idle":"2025-10-23T04:14:00.506063Z","shell.execute_reply.started":"2025-10-23T04:14:00.495073Z","shell.execute_reply":"2025-10-23T04:14:00.505236Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"54e5b75b-35a6-4797-9db1-e4a5464c59e7","cell_type":"code","source":"# Lets plot\nplt.imshow(img.permute(1,2,0))","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.507069Z","iopub.execute_input":"2025-10-23T04:14:00.507561Z","iopub.status.idle":"2025-10-23T04:14:00.704537Z","shell.execute_reply.started":"2025-10-23T04:14:00.507526Z","shell.execute_reply":"2025-10-23T04:14:00.703854Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"a0666c6f-890f-4151-9588-2e6c206187be","cell_type":"markdown","source":"Well, it was wrong guess from Model :(","metadata":{}},{"id":"8f6dd44a-804e-43c4-a683-99ea9ad4079e","cell_type":"markdown","source":"# Device Selection","metadata":{}},{"id":"a99f522c-cf30-473b-91cd-aa357fd0aa04","cell_type":"code","source":"def get_default_device():\n    \"\"\"Pick GPU if available, else CPU\"\"\"\n    if torch.cuda.is_available():\n        return torch.device('cuda')\n    else:\n        return torch.device('cpu')\n    \ndef to_device(data, device):\n    \"\"\"Move tensor(s) to chosen device\"\"\"\n    if isinstance(data, (list,tuple)):\n        return [to_device(x, device) for x in data]\n    return data.to(device, non_blocking=True)\n\nclass DeviceDataLoader():\n    \"\"\"Wrap a dataloader to move data to a device\"\"\"\n    def __init__(self, dl, device):\n        self.dl = dl\n        self.device = device\n        \n    def __iter__(self):\n        \"\"\"Yield a batch of data after moving it to device\"\"\"\n        for b in self.dl: \n            yield to_device(b, self.device)\n\n    def __len__(self):\n        \"\"\"Number of batches\"\"\"\n        return len(self.dl)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.705387Z","iopub.execute_input":"2025-10-23T04:14:00.705556Z","iopub.status.idle":"2025-10-23T04:14:00.711886Z","shell.execute_reply.started":"2025-10-23T04:14:00.705534Z","shell.execute_reply":"2025-10-23T04:14:00.711259Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"5f104e46-0422-4a58-be12-a9af6e82297a","cell_type":"markdown","source":"As we are using GPU, so below code will help us to access GPU at different stage of Processing\n","metadata":{}},{"id":"1a3a5a36-bf4b-4005-9d68-5f80e5f0abf0","cell_type":"code","source":"device = get_default_device()\ndevice","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.712825Z","iopub.execute_input":"2025-10-23T04:14:00.713024Z","iopub.status.idle":"2025-10-23T04:14:00.724789Z","shell.execute_reply.started":"2025-10-23T04:14:00.712997Z","shell.execute_reply":"2025-10-23T04:14:00.724193Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"32aae505-9cd7-4b26-8b2d-6f02f5605726","cell_type":"code","source":"train_dl = DeviceDataLoader(train_dl, device)\nval_dl = DeviceDataLoader(val_dl, device)\nto_device(model, device)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:00.725464Z","iopub.execute_input":"2025-10-23T04:14:00.725656Z","iopub.status.idle":"2025-10-23T04:14:03.695643Z","shell.execute_reply.started":"2025-10-23T04:14:00.725634Z","shell.execute_reply":"2025-10-23T04:14:03.694853Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"db42cb2a-aa74-4d77-8b24-32ade524ba85","cell_type":"markdown","source":"# Define Helper Fit Model","metadata":{}},{"id":"b7bd0b19-39d2-4de4-be81-f58d3082b978","cell_type":"code","source":"def loss_batch(model, loss_func, x, y, opt = None, metric = None):\n    \n    pred = model(x)\n    \n    loss = loss_func(pred, y)\n    \n    if opt is not None:\n        \n        loss.backward()\n        opt.step()\n        opt.zero_grad()\n        \n    metric_result = None\n    \n    if metric is not None:\n        \n        metric_result = metric(pred, y)\n        \n    return loss.item(), len(x), metric_result","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:03.696557Z","iopub.execute_input":"2025-10-23T04:14:03.696762Z","iopub.status.idle":"2025-10-23T04:14:03.70165Z","shell.execute_reply.started":"2025-10-23T04:14:03.696737Z","shell.execute_reply":"2025-10-23T04:14:03.701046Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"2fe20708-722f-44e2-9069-2fd9b00e459f","cell_type":"code","source":"def evaluate(model, loss_fn, val_dl, metric = None):\n    \n    with torch.no_grad():\n        \n        results = [loss_batch(model, loss_fn, x, y, metric = metric) for x, y in val_dl]\n        \n        losses, nums, metrics = zip(*results)\n        \n        total = np.sum(nums)\n        \n        avg_loss = np.sum(np.multiply(losses, nums)) / total\n        \n        avg_metric = None\n        \n        if metric is not None:\n            avg_metric = np.sum(np.multiply(metrics, nums)) / total\n            \n    return avg_loss, total, avg_metric","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:03.702771Z","iopub.execute_input":"2025-10-23T04:14:03.703297Z","iopub.status.idle":"2025-10-23T04:14:03.714499Z","shell.execute_reply.started":"2025-10-23T04:14:03.703258Z","shell.execute_reply":"2025-10-23T04:14:03.713863Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"024f911d-2155-4b7c-b7ef-8f147e2bef76","cell_type":"code","source":"def fit(epochs, model, loss_fn, train_dl, val_dl, opt_fn = None, metric = None, scheduler = None, scheduler_on = 'val_metric'):\n    \n    train_losses, val_losses, val_metrics, train_metrics = [], [], [], []\n    \n    \n    for epoch in range(epochs):\n        \n        model.train()\n        for x, y in train_dl:\n            train_loss, _, train_metric = loss_batch(model, loss_fn, x, y, opt_fn, metric)\n            \n        model.eval()\n        result = evaluate(model, loss_fn, val_dl, metric)\n        val_loss, total, val_metric = result\n        \n        train_losses.append(train_loss)\n        val_losses.append(val_loss)\n        val_metrics.append(val_metric)\n        train_metrics.append(train_metric)\n        \n        if metric is None:\n            print('Epoch{}/{}, train_loss: {:.4f}, val_loss: {:.4f}' \n                 .format(epoch+1, epochs, train_loss, val_loss))\n            \n        else:\n            print('Epoch {}/{}, train_loss: {:.4f}, val_loss: {:.4f}, val_{}: {:.4f}, train_{}: {:.4f}'\n                 .format(epoch+1, epochs, train_loss, val_loss, metric.__name__, val_metric, metric.__name__, train_metric))\n            \n        if scheduler is not None:\n            if scheduler_on == 'val_metric':\n                scheduler.step(val_metrics[-1])\n        \n            \n    return train_losses, val_losses, val_metrics, train_metrics","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:03.715368Z","iopub.execute_input":"2025-10-23T04:14:03.715609Z","iopub.status.idle":"2025-10-23T04:14:03.725586Z","shell.execute_reply.started":"2025-10-23T04:14:03.71557Z","shell.execute_reply":"2025-10-23T04:14:03.724978Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"0aca79eb-970d-4381-a564-8a14b259bcc4","cell_type":"markdown","source":"Define Accuracy function","metadata":{}},{"id":"ca3efe36-9ae4-4240-a62d-77ac6a6543d2","cell_type":"code","source":"def accuracy(output, labels):\n    _, preds = torch.max(output, dim = 1)\n    \n    return torch.sum(preds == labels).item() / len(preds)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:03.726762Z","iopub.execute_input":"2025-10-23T04:14:03.727081Z","iopub.status.idle":"2025-10-23T04:14:03.74047Z","shell.execute_reply.started":"2025-10-23T04:14:03.727049Z","shell.execute_reply":"2025-10-23T04:14:03.739783Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"b6fd44f9-e18e-4e1b-84eb-83b8015cb4a1","cell_type":"markdown","source":"Initial guess on Validation data","metadata":{}},{"id":"ce1edcd8-fae7-4c69-adcd-dd39a52988d8","cell_type":"code","source":"val_loss, _, val_acc = evaluate(model, F.cross_entropy, val_dl, metric = accuracy)\n\nprint(val_loss, val_acc)","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:03.74127Z","iopub.execute_input":"2025-10-23T04:14:03.741464Z","iopub.status.idle":"2025-10-23T04:14:11.449599Z","shell.execute_reply.started":"2025-10-23T04:14:03.741435Z","shell.execute_reply":"2025-10-23T04:14:11.44894Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"24e1a1af-f659-4c59-8ed0-e591a5fd30a1","cell_type":"code","source":"num_epochs = 5\n\noptimizer = torch.optim.SGD(model.parameters(), lr = 0.1, momentum = 0.9)\n# PyTorch >= 2.2 removed ReduceLROnPlateau's `verbose` arg\nscheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(optimizer, factor = 0.1, mode = 'max')","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:11.450945Z","iopub.execute_input":"2025-10-23T04:14:11.451491Z","iopub.status.idle":"2025-10-23T04:14:11.455929Z","shell.execute_reply.started":"2025-10-23T04:14:11.451445Z","shell.execute_reply":"2025-10-23T04:14:11.455156Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"7a7e0156-de02-4fc9-8577-af566a2afd1e","cell_type":"code","source":"history = fit(num_epochs, model, F.cross_entropy, train_dl, val_dl, optimizer, accuracy, scheduler, 'val_metric')","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:11.456875Z","iopub.execute_input":"2025-10-23T04:14:11.457294Z","iopub.status.idle":"2025-10-23T04:14:59.692503Z","shell.execute_reply.started":"2025-10-23T04:14:11.457268Z","shell.execute_reply":"2025-10-23T04:14:59.69171Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"ead49e85-9bee-49d2-9dde-9ae269c9023c","cell_type":"markdown","source":"Well, this is amazing right ?? 100% Train Accuracy and 98.86% Validation accuracy!!","metadata":{}},{"id":"b237d5a4-a97e-4729-b1e2-12e30546c5a0","cell_type":"markdown","source":"May Be over fitting ?? Well, its fairly simplest dataset that anyone ever get, so It was not probably over fitting :(","metadata":{}},{"id":"7ef53205-ca22-485f-9ca0-a44999dcde89","cell_type":"markdown","source":"# Plotting result","metadata":{}},{"id":"b90126f7-a94b-4efe-b2af-b00537d21bff","cell_type":"markdown","source":"Plot between Training Loss vs Epochs","metadata":{}},{"id":"288ba660-3467-4f88-ad30-a83440b8be81","cell_type":"code","source":"plt.figure(figsize = (8,8))\nplt.plot(history[0], '-x')\nplt.xlabel('Epochs')\nplt.ylabel('Training Loss')\nplt.title('Plot between Training Loss vs Epochs - 2330026139')","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:59.693431Z","iopub.execute_input":"2025-10-23T04:14:59.693619Z","iopub.status.idle":"2025-10-23T04:14:59.913622Z","shell.execute_reply.started":"2025-10-23T04:14:59.693595Z","shell.execute_reply":"2025-10-23T04:14:59.912894Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"29d111e2-fc87-4a75-89b3-89fdf646f7cb","cell_type":"markdown","source":"Too Much of noise because of Batch Stochastic gradient","metadata":{}},{"id":"990363a7-2b8c-4977-8ba6-750e32ecd852","cell_type":"code","source":"plt.figure(figsize = (8,8))\nplt.plot(history[1], '-x')\nplt.xlabel('Epochs')\nplt.ylabel('Validation Loss')\nplt.title('Plot between Validation Loss vs Epochs - 2330026139')","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:14:59.914536Z","iopub.execute_input":"2025-10-23T04:14:59.914739Z","iopub.status.idle":"2025-10-23T04:15:00.137899Z","shell.execute_reply.started":"2025-10-23T04:14:59.914715Z","shell.execute_reply":"2025-10-23T04:15:00.137178Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"80ec2305-5a8e-4a81-82db-74fdc2ae93f6","cell_type":"code","source":"plt.figure(figsize = (8,8))\nplt.plot(history[0], '-go')\nplt.plot(history[1], '-yx')\nplt.xlabel('Epochs')\nplt.ylabel('Validation Loss')\nplt.title('Plot between Validation Loss & training Loss vs Epochs - 2330026139')\nplt.legend(['Train Loss', 'Validation Loss'], loc = 'upper right')","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:15:00.138999Z","iopub.execute_input":"2025-10-23T04:15:00.139255Z","iopub.status.idle":"2025-10-23T04:15:00.304835Z","shell.execute_reply.started":"2025-10-23T04:15:00.139218Z","shell.execute_reply":"2025-10-23T04:15:00.304146Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"5f26f795-e815-4233-9533-9d37d00eb856","cell_type":"code","source":"plt.figure(figsize = (8,8))\nplt.plot(history[3], '-go')\nplt.plot(history[2], '-yx')\nplt.xlabel('Epochs')\nplt.ylabel('Accuracy')\nplt.title('Plot between Training Accuracy & Validation vs Epochs - 2330026139')\nplt.legend(['Train Accuracy', 'Validation Accuracy'], loc = 'lower right')","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:15:00.305807Z","iopub.execute_input":"2025-10-23T04:15:00.306008Z","iopub.status.idle":"2025-10-23T04:15:00.541925Z","shell.execute_reply.started":"2025-10-23T04:15:00.305957Z","shell.execute_reply":"2025-10-23T04:15:00.541172Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"885e41c8-b029-4c7c-9c2b-ff50e7e1e562","cell_type":"markdown","source":"# Test data & Model Evaluation","metadata":{}},{"id":"9867b6be-7224-4541-856b-e29a0bfd5e85","cell_type":"code","source":"def predict_image(img, model):\n    xb = to_device(img.unsqueeze(0), device)\n    yb = model(xb)\n    _, preds  = torch.max(yb, dim=1)\n    return preds[0].item()","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:15:00.543188Z","iopub.execute_input":"2025-10-23T04:15:00.543885Z","iopub.status.idle":"2025-10-23T04:15:00.548683Z","shell.execute_reply.started":"2025-10-23T04:15:00.543838Z","shell.execute_reply":"2025-10-23T04:15:00.548031Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"b8648911-181b-4e26-82e4-01e9902d2cc5","cell_type":"code","source":"img, label = test_set[0]\nplt.imshow(img[0], cmap='gray')\nprint('Label:', label, ', Predicted:', predict_image(img, model))","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:15:00.549598Z","iopub.execute_input":"2025-10-23T04:15:00.549784Z","iopub.status.idle":"2025-10-23T04:15:00.682481Z","shell.execute_reply.started":"2025-10-23T04:15:00.549762Z","shell.execute_reply":"2025-10-23T04:15:00.681788Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"7f0aa10a-295d-4aa1-90e0-9ea98176495c","cell_type":"markdown","source":"Yippy!! That's Correct\n","metadata":{}},{"id":"f2f8c763-9303-41d7-8cd7-e99c948f0572","cell_type":"code","source":"img, label = test_set[1839]\nplt.imshow(img[0], cmap='gray')\nprint('Label:', label, ', Predicted:', predict_image(img, model))","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:15:00.683419Z","iopub.execute_input":"2025-10-23T04:15:00.683678Z","iopub.status.idle":"2025-10-23T04:15:00.809212Z","shell.execute_reply.started":"2025-10-23T04:15:00.683639Z","shell.execute_reply":"2025-10-23T04:15:00.80857Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"ac244320-6376-425b-832a-42924ec61240","cell_type":"code","source":"img, label = test_set[193]\nplt.imshow(img[0], cmap='gray')\nprint('Label:', label, ', Predicted:', predict_image(img, model))","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:15:00.81051Z","iopub.execute_input":"2025-10-23T04:15:00.810744Z","iopub.status.idle":"2025-10-23T04:15:00.934417Z","shell.execute_reply.started":"2025-10-23T04:15:00.81071Z","shell.execute_reply":"2025-10-23T04:15:00.933703Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"8bd7576c-3792-40f5-9d57-60196b102f05","cell_type":"markdown","source":"This was tough one, but model predicted coreectly","metadata":{}},{"id":"7ebf4fd6-29c0-422e-b312-ef4954411be3","cell_type":"code","source":"img, label = test_set[1000]\nplt.imshow(img[0], cmap='gray')\nprint('Label:', label, ', Predicted:', predict_image(img, model))","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:15:00.935343Z","iopub.execute_input":"2025-10-23T04:15:00.935615Z","iopub.status.idle":"2025-10-23T04:15:01.057688Z","shell.execute_reply.started":"2025-10-23T04:15:00.935574Z","shell.execute_reply":"2025-10-23T04:15:01.057018Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"dc74b45c-57dc-4444-93b0-f2f39f32bec5","cell_type":"markdown","source":"As a final step, let's also look at the overall loss and accuracy of the model on the test set.\n","metadata":{}},{"id":"2836e487-a01d-491f-874d-866e82313302","cell_type":"code","source":"test_loader = DeviceDataLoader(DataLoader(test_set, batch_size=256), device)\nresult = evaluate(model, F.cross_entropy, test_loader, metric = accuracy)\nresult\nAccuracy = result[2] * 100\nAccuracy\nloss = result[0]\nprint(\"Total Losses: {}, Accuracy: {}\".format(loss, Accuracy))","metadata":{"execution":{"iopub.status.busy":"2025-10-23T04:15:01.058719Z","iopub.execute_input":"2025-10-23T04:15:01.058965Z","iopub.status.idle":"2025-10-23T04:15:02.616565Z","shell.execute_reply.started":"2025-10-23T04:15:01.058931Z","shell.execute_reply":"2025-10-23T04:15:02.615652Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"e13f5152-90de-484a-b39d-0bb63a2f7656","cell_type":"markdown","source":"Wow, 98.87% Accuracy, I can just believe inventor of this model, Their Hard Work and Their dedication, hats Off to those guys","metadata":{}},{"id":"d1355cf5-7e02-4394-8d7b-c8d6f24ad8d1","cell_type":"markdown","source":"Thats it for this Notebook...\n\nIf you like this kindly Consider Upvoting!! Happy learning!!","metadata":{}},{"id":"55fc8bce-6416-49fd-a72e-21a050465662","cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"d463e5db-939e-41c8-90cc-8052d2afc24f","cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"d6cbb1be-1c52-4179-8f54-5668fefda490","cell_type":"markdown","source":"\n---\n\n# Part 2 — AlexNet on Flowers (PyTorch)","metadata":{}},{"id":"3a954a49-b796-4ef1-a629-7a13395dacfd","cell_type":"markdown","source":"# Flowers Classify Torch AlexNet","metadata":{}},{"id":"b45dc6d4-e4b4-4467-807b-6bcc49cf123d","cell_type":"markdown","source":"---\n- **torch**\n - nn\n   - functional\n     - relu\n     - max_pool2d\n   - Conv2d\n   - Linear\n - utils\n   - data\n     - DataLoader\n---\n- **torchvision**\n - datasets\n   - ImageFolder\n - transforms\n   - Compose\n   - Resize\n   - ToTensor\n   - Normalize\n - models\n - utils\n   - make_grid\n---","metadata":{}},{"id":"834ca9c3-a326-4535-b431-e578ea4c9df8","cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import DataLoader\nfrom torchvision import datasets, transforms, models \nfrom torchvision.utils import make_grid\nimport os\nimport random\nfrom sklearn.model_selection import train_test_split\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"papermill":{"duration":2.226365,"end_time":"2021-10-16T13:11:35.594724","exception":false,"start_time":"2021-10-16T13:11:33.368359","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:43.695821Z","iopub.execute_input":"2025-10-23T04:22:43.696185Z","iopub.status.idle":"2025-10-23T04:22:46.302544Z","shell.execute_reply.started":"2025-10-23T04:22:43.696076Z","shell.execute_reply":"2025-10-23T04:22:46.301723Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"d426a030-4a0b-40b6-8931-4dae00f22f61","cell_type":"code","source":"train_transform=transforms.Compose([\n        transforms.RandomRotation(10),      # rotate +/- 10 degrees\n        transforms.RandomHorizontalFlip(),  # reverse 50% of images\n        transforms.Resize(224),             # resize shortest side to 224 pixels\n        transforms.CenterCrop(224),         # crop longest side to 224 pixels at center\n        transforms.ToTensor(),\n        transforms.Normalize([0.485, 0.456, 0.406],\n                             [0.229, 0.224, 0.225])\n])","metadata":{"papermill":{"duration":0.035423,"end_time":"2021-10-16T13:11:35.650718","exception":false,"start_time":"2021-10-16T13:11:35.615295","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:46.304908Z","iopub.execute_input":"2025-10-23T04:22:46.305229Z","iopub.status.idle":"2025-10-23T04:22:46.311391Z","shell.execute_reply.started":"2025-10-23T04:22:46.305188Z","shell.execute_reply":"2025-10-23T04:22:46.310358Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"b6a4a6ec-d5fb-4b6d-918a-1e956513e475","cell_type":"markdown","source":"# Set Train and Test","metadata":{"papermill":{"duration":0.019619,"end_time":"2021-10-16T13:11:35.74543","exception":false,"start_time":"2021-10-16T13:11:35.725811","status":"completed"},"tags":[]}},{"id":"fd003225-a6f7-4c4e-9f68-b6e55c619530","cell_type":"code","source":"import os as _os\n_flower_root_cands = [\"/kaggle/input/datasets/alxmamaev/flowers-recognition/flowers\",\n                      \"/kaggle/input/datasets/alxmamaev/flowers-recognition\",\n                      \"../input/flowers-recognition/flowers\",\n                      \"../input/flowers-recognition\",\n                      \"../input/flowers-recognition/data/flowers\"]\n_flower_root = next((_p for _p in _flower_root_cands if _os.path.isdir(_p)), _flower_root_cands[0])\nprint('flower data root:', _flower_root)\ndataset=datasets.ImageFolder(root=_flower_root,transform=train_transform)\ndataset","metadata":{"papermill":{"duration":6.427069,"end_time":"2021-10-16T13:11:42.192371","exception":false,"start_time":"2021-10-16T13:11:35.765302","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:46.3126Z","iopub.execute_input":"2025-10-23T04:22:46.312843Z","iopub.status.idle":"2025-10-23T04:22:54.690972Z","shell.execute_reply.started":"2025-10-23T04:22:46.312814Z","shell.execute_reply":"2025-10-23T04:22:54.690036Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"d2debeef-4474-47a6-b03e-76ca05492ded","cell_type":"code","source":"class_names=dataset.classes\nprint(class_names)\nprint(len(class_names))","metadata":{"papermill":{"duration":0.02844,"end_time":"2021-10-16T13:11:42.24176","exception":false,"start_time":"2021-10-16T13:11:42.21332","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:54.692091Z","iopub.execute_input":"2025-10-23T04:22:54.692331Z","iopub.status.idle":"2025-10-23T04:22:54.698369Z","shell.execute_reply.started":"2025-10-23T04:22:54.692303Z","shell.execute_reply":"2025-10-23T04:22:54.697238Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"6aa6138a-3e5f-4ceb-8d78-76f1040400db","cell_type":"code","source":"train_indices, test_indices = train_test_split(list(range(len(dataset.targets))), test_size=0.2, stratify=dataset.targets)\ntrain_data = torch.utils.data.Subset(dataset, train_indices)\ntest_data = torch.utils.data.Subset(dataset, test_indices)","metadata":{"papermill":{"duration":0.043166,"end_time":"2021-10-16T13:11:42.306129","exception":false,"start_time":"2021-10-16T13:11:42.262963","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:54.699593Z","iopub.execute_input":"2025-10-23T04:22:54.699955Z","iopub.status.idle":"2025-10-23T04:22:54.717716Z","shell.execute_reply.started":"2025-10-23T04:22:54.699917Z","shell.execute_reply":"2025-10-23T04:22:54.71669Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"f6d6c45f-e0c6-4f13-8b9b-ba4c7bdcee00","cell_type":"code","source":"train_loader=DataLoader(train_data,batch_size=10,shuffle=True)\ntest_loader=DataLoader(test_data,batch_size=10)","metadata":{"papermill":{"duration":0.027156,"end_time":"2021-10-16T13:11:42.353859","exception":false,"start_time":"2021-10-16T13:11:42.326703","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:54.7188Z","iopub.execute_input":"2025-10-23T04:22:54.719097Z","iopub.status.idle":"2025-10-23T04:22:54.724399Z","shell.execute_reply.started":"2025-10-23T04:22:54.719064Z","shell.execute_reply":"2025-10-23T04:22:54.723291Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"436a1ea0-45c5-4cf8-9235-9297cf63677f","cell_type":"code","source":"print(len(train_data))\nprint(len(test_data))","metadata":{"papermill":{"duration":0.028998,"end_time":"2021-10-16T13:11:42.403685","exception":false,"start_time":"2021-10-16T13:11:42.374687","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:54.726791Z","iopub.execute_input":"2025-10-23T04:22:54.727207Z","iopub.status.idle":"2025-10-23T04:22:54.736812Z","shell.execute_reply.started":"2025-10-23T04:22:54.727171Z","shell.execute_reply":"2025-10-23T04:22:54.736042Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"8d9bb74b-7a80-46d9-9693-258d861ed01d","cell_type":"markdown","source":"# Images and Labels","metadata":{"papermill":{"duration":0.020799,"end_time":"2021-10-16T13:11:42.445709","exception":false,"start_time":"2021-10-16T13:11:42.42491","status":"completed"},"tags":[]}},{"id":"c62571fc-813b-4e12-822c-492fe59cee6d","cell_type":"code","source":"for images, labels in train_loader:\n    break\n#print the labels\nprint('Label:', labels.numpy())\nprint('Class:', *np.array([class_names[i] for i in labels]))\n\nim=make_grid(images,nrow=5)","metadata":{"papermill":{"duration":0.193229,"end_time":"2021-10-16T13:11:42.659927","exception":false,"start_time":"2021-10-16T13:11:42.466698","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:54.738277Z","iopub.execute_input":"2025-10-23T04:22:54.738821Z","iopub.status.idle":"2025-10-23T04:22:55.011739Z","shell.execute_reply.started":"2025-10-23T04:22:54.738777Z","shell.execute_reply":"2025-10-23T04:22:55.010656Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"d5a91680-c74c-497b-a104-30864e0c1f08","cell_type":"code","source":"plt.figure(figsize=(10,10))\nplt.imshow(np.transpose(im.numpy(),(1,2,0)))\nplt.title('Normalized training sample images - 2330026139')","metadata":{"papermill":{"duration":0.358689,"end_time":"2021-10-16T13:11:43.039926","exception":false,"start_time":"2021-10-16T13:11:42.681237","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:55.01284Z","iopub.execute_input":"2025-10-23T04:22:55.013087Z","iopub.status.idle":"2025-10-23T04:22:55.496997Z","shell.execute_reply.started":"2025-10-23T04:22:55.013057Z","shell.execute_reply":"2025-10-23T04:22:55.496022Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"74fcd47d-8184-4e61-ba03-d5b6c8a75130","cell_type":"code","source":"inv_normalize=transforms.Normalize(mean=[-0.485/0.229,-0.456/0.224,-0.406/0.225],\n                                     std=[1/0.229,1/0.224,1/0.225])\nim=inv_normalize(im)","metadata":{"papermill":{"duration":0.035273,"end_time":"2021-10-16T13:11:43.098376","exception":false,"start_time":"2021-10-16T13:11:43.063103","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:55.498744Z","iopub.execute_input":"2025-10-23T04:22:55.499039Z","iopub.status.idle":"2025-10-23T04:22:55.509513Z","shell.execute_reply.started":"2025-10-23T04:22:55.499003Z","shell.execute_reply":"2025-10-23T04:22:55.508635Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"e0c9a92b-cdd2-4496-9da6-9f1c8d796ee5","cell_type":"code","source":"plt.figure(figsize=(10,10))\nplt.imshow(np.transpose(im.numpy(),(1,2,0)))\nplt.title('De-normalized training sample images - 2330026139')","metadata":{"papermill":{"duration":0.326051,"end_time":"2021-10-16T13:11:43.447612","exception":false,"start_time":"2021-10-16T13:11:43.121561","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:55.510879Z","iopub.execute_input":"2025-10-23T04:22:55.511153Z","iopub.status.idle":"2025-10-23T04:22:56.004339Z","shell.execute_reply.started":"2025-10-23T04:22:55.511113Z","shell.execute_reply":"2025-10-23T04:22:56.00337Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"4dbb9592-7795-4e7c-8573-45986248b1be","cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"b089dbce-20e3-4156-8720-6038de6cef66","cell_type":"markdown","source":"# CNN Model","metadata":{"papermill":{"duration":0.02458,"end_time":"2021-10-16T13:11:43.49676","exception":false,"start_time":"2021-10-16T13:11:43.47218","status":"completed"},"tags":[]}},{"id":"0083264a-fe16-47ed-9d05-06fff1628577","cell_type":"code","source":"class ConvolutionalNetwork(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv1=nn.Conv2d(3,6,3,1)\n        self.conv2=nn.Conv2d(6,16,3,1)\n        self.fc1=nn.Linear(16*54*54,120) \n        self.fc2=nn.Linear(120,84)\n        self.fc3=nn.Linear(84,20)\n        self.fc4=nn.Linear(20,5)\n    def forward(self,X):\n        X=F.relu(self.conv1(X))\n        X=F.max_pool2d(X,2,2)\n        X=F.relu(self.conv2(X))\n        X=F.max_pool2d(X,2,2)\n        X=X.view(-1,16*54*54)\n        X=F.relu(self.fc1(X))\n        X=F.relu(self.fc2(X))\n        X=F.relu(self.fc3(X))\n        X=self.fc4(X)\n        \n        return F.log_softmax(X, dim=1)","metadata":{"papermill":{"duration":0.03552,"end_time":"2021-10-16T13:11:43.557155","exception":false,"start_time":"2021-10-16T13:11:43.521635","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:56.006017Z","iopub.execute_input":"2025-10-23T04:22:56.00633Z","iopub.status.idle":"2025-10-23T04:22:56.019212Z","shell.execute_reply.started":"2025-10-23T04:22:56.006292Z","shell.execute_reply":"2025-10-23T04:22:56.018105Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"f36017e1-11ec-4217-a7f0-13d36f97d72f","cell_type":"code","source":"CNNmodel=ConvolutionalNetwork()\ncriterion=nn.CrossEntropyLoss()\noptimizer=torch.optim.Adam(CNNmodel.parameters(),lr=0.001)","metadata":{"papermill":{"duration":0.085758,"end_time":"2021-10-16T13:11:43.667346","exception":false,"start_time":"2021-10-16T13:11:43.581588","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:56.020743Z","iopub.execute_input":"2025-10-23T04:22:56.021079Z","iopub.status.idle":"2025-10-23T04:22:56.098223Z","shell.execute_reply.started":"2025-10-23T04:22:56.021043Z","shell.execute_reply":"2025-10-23T04:22:56.097384Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"195298c5-ad58-438f-ba66-50fc7cea7b75","cell_type":"code","source":"CNNmodel","metadata":{"papermill":{"duration":0.03286,"end_time":"2021-10-16T13:11:43.724559","exception":false,"start_time":"2021-10-16T13:11:43.691699","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:56.099565Z","iopub.execute_input":"2025-10-23T04:22:56.099911Z","iopub.status.idle":"2025-10-23T04:22:56.106549Z","shell.execute_reply.started":"2025-10-23T04:22:56.099869Z","shell.execute_reply":"2025-10-23T04:22:56.105575Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"654c88b0-4e54-493a-912c-0222f6b00941","cell_type":"code","source":"def count_parameters(model):\n    params = [p.numel() for p in model.parameters() if p.requires_grad]\n    for item in params:\n        print(f'{item:>8}')\n    print(f'________\\n{sum(params):>8}')\ncount_parameters(CNNmodel)","metadata":{"papermill":{"duration":0.034303,"end_time":"2021-10-16T13:11:43.783616","exception":false,"start_time":"2021-10-16T13:11:43.749313","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:56.108204Z","iopub.execute_input":"2025-10-23T04:22:56.108546Z","iopub.status.idle":"2025-10-23T04:22:56.119559Z","shell.execute_reply.started":"2025-10-23T04:22:56.1085Z","shell.execute_reply":"2025-10-23T04:22:56.118683Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"1d41da9d-2e75-47f1-b2f1-8650eb9f0d11","cell_type":"code","source":"import time\nstart_time=time.time()\ntrain_losses=[]\ntest_losses=[]\ntrain_correct=[]\ntest_correct=[]\nepochs=5\n\nfor i in range(epochs):\n    trn_corr=0\n    tst_corr=0\n    for b, (X_train,y_train) in enumerate(train_loader):\n        b+=1                                            \n        y_pred=CNNmodel(X_train)\n        loss=criterion(y_pred,y_train)\n\n        predicted=torch.max(y_pred.data,1)[1]\n        batch_corr=(predicted==y_train).sum()\n        trn_corr+=batch_corr\n        \n        optimizer.zero_grad()\n        loss.backward()\n        optimizer.step()\n        \n        if b%200==0:\n            print(f\"epoch: {i} loss: {loss.item} batch: {b} accuracy: {trn_corr.item()*100/(10*b):7.3f}%\")\n    loss=loss.detach().numpy()\n    train_losses.append(loss)\n    train_correct.append(trn_corr)\n    \n    \n    with torch.no_grad():\n        for b, (X_test,y_test) in enumerate(test_loader):\n            y_val=CNNmodel(X_test)\n            loss=criterion(y_val,y_test)\n            \n            predicted=torch.max(y_val.data,1)[1]\n            btach_corr=(predicted==y_test).sum()\n            tst_corr+=btach_corr\n            \n        loss=loss.detach().numpy()\n        test_losses.append(loss)\n        test_correct.append(tst_corr)\n        \nprint(f'\\nDuration: {time.time() - start_time:.0f} seconds')    ","metadata":{"papermill":{"duration":996.035843,"end_time":"2021-10-16T13:28:19.844672","exception":false,"start_time":"2021-10-16T13:11:43.808829","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:22:56.120973Z","iopub.execute_input":"2025-10-23T04:22:56.121293Z","iopub.status.idle":"2025-10-23T04:29:02.593276Z","shell.execute_reply.started":"2025-10-23T04:22:56.121248Z","shell.execute_reply":"2025-10-23T04:29:02.592359Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"807abce2-338e-4446-a4a8-7938ae40c60d","cell_type":"code","source":"plt.plot(train_losses,label=\"train_losses\")\nplt.plot(test_losses,label=\"test_losses\")\nplt.legend()\nplt.title('Training and Test Loss - 2330026139')","metadata":{"papermill":{"duration":0.2335,"end_time":"2021-10-16T13:28:20.109018","exception":false,"start_time":"2021-10-16T13:28:19.875518","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:29:02.594566Z","iopub.execute_input":"2025-10-23T04:29:02.594841Z","iopub.status.idle":"2025-10-23T04:29:02.851371Z","shell.execute_reply.started":"2025-10-23T04:29:02.594809Z","shell.execute_reply":"2025-10-23T04:29:02.850476Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"c30fa5b5-920d-4ac9-b57d-b53df4a20787","cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"3ff009ce-f72e-4d26-98a2-57644ea93082","cell_type":"markdown","source":"# AlexNetModel","metadata":{"papermill":{"duration":0.03174,"end_time":"2021-10-16T13:28:20.172488","exception":false,"start_time":"2021-10-16T13:28:20.140748","status":"completed"},"tags":[]}},{"id":"05bdedfe-95f4-49a9-b862-184e887b289d","cell_type":"code","source":"alexnetmodel= models.alexnet(pretrained=True)","metadata":{"papermill":{"duration":24.762605,"end_time":"2021-10-16T13:28:44.967412","exception":false,"start_time":"2021-10-16T13:28:20.204807","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:29:02.852786Z","iopub.execute_input":"2025-10-23T04:29:02.853098Z","iopub.status.idle":"2025-10-23T04:29:04.832184Z","shell.execute_reply.started":"2025-10-23T04:29:02.853057Z","shell.execute_reply":"2025-10-23T04:29:04.831196Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"a5f1d94b-8582-4395-8b51-e904702be4b8","cell_type":"code","source":"alexnetmodel","metadata":{"papermill":{"duration":0.041313,"end_time":"2021-10-16T13:28:45.042602","exception":false,"start_time":"2021-10-16T13:28:45.001289","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:29:04.833201Z","iopub.execute_input":"2025-10-23T04:29:04.833422Z","iopub.status.idle":"2025-10-23T04:29:04.839975Z","shell.execute_reply.started":"2025-10-23T04:29:04.833395Z","shell.execute_reply":"2025-10-23T04:29:04.839018Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"a0d200c1-7c44-4691-bf77-f1a9ffee8bc5","cell_type":"code","source":"for param in alexnetmodel.parameters():\n    param.requires_grad=False","metadata":{"papermill":{"duration":0.040425,"end_time":"2021-10-16T13:28:45.115606","exception":false,"start_time":"2021-10-16T13:28:45.075181","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:29:04.840955Z","iopub.execute_input":"2025-10-23T04:29:04.841226Z","iopub.status.idle":"2025-10-23T04:29:04.856409Z","shell.execute_reply.started":"2025-10-23T04:29:04.841197Z","shell.execute_reply":"2025-10-23T04:29:04.855455Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"59a335a4-fb60-48c6-ab1e-94cff6853c29","cell_type":"code","source":"torch.manual_seed(42)\n\nalexnetmodel.classifier=nn.Sequential(nn.Linear(9216,1024),\n                                     nn.ReLU(),\n                                     nn.Dropout(p=0.5),\n                                     nn.Linear(1024,5),\n                                     nn.LogSoftmax(dim=1))\nalexnetmodel","metadata":{"papermill":{"duration":0.139974,"end_time":"2021-10-16T13:28:45.288586","exception":false,"start_time":"2021-10-16T13:28:45.148612","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:29:04.857556Z","iopub.execute_input":"2025-10-23T04:29:04.857809Z","iopub.status.idle":"2025-10-23T04:29:04.989141Z","shell.execute_reply.started":"2025-10-23T04:29:04.85778Z","shell.execute_reply":"2025-10-23T04:29:04.988183Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"88528a4c-fc2f-47ee-82c9-66de09e2a5b5","cell_type":"code","source":"count_parameters(alexnetmodel)","metadata":{"papermill":{"duration":0.04141,"end_time":"2021-10-16T13:28:45.363448","exception":false,"start_time":"2021-10-16T13:28:45.322038","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:29:04.992324Z","iopub.execute_input":"2025-10-23T04:29:04.992583Z","iopub.status.idle":"2025-10-23T04:29:04.998446Z","shell.execute_reply.started":"2025-10-23T04:29:04.992548Z","shell.execute_reply":"2025-10-23T04:29:04.997426Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"6f6cba44-4ff1-497b-ad0e-25d182424c1d","cell_type":"code","source":"criterion=nn.CrossEntropyLoss()\noptimizer=torch.optim.Adam(alexnetmodel.classifier.parameters(),lr=0.001)","metadata":{"papermill":{"duration":0.054567,"end_time":"2021-10-16T13:28:45.459263","exception":false,"start_time":"2021-10-16T13:28:45.404696","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:29:04.999799Z","iopub.execute_input":"2025-10-23T04:29:05.000128Z","iopub.status.idle":"2025-10-23T04:29:05.019923Z","shell.execute_reply.started":"2025-10-23T04:29:05.000088Z","shell.execute_reply":"2025-10-23T04:29:05.018722Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"86819295-bef9-4353-aa53-669437646373","cell_type":"code","source":"import time\nstart_time=time.time()\ntrain_losses=[]\ntest_losses=[]\ntrn_correct=[]\ntst_correct=[]\nepochs=5\n\nfor i in range(epochs):\n    trn_corr=0\n    tst_corr=0\n    for b, (X_train,y_train) in enumerate(train_loader):\n        b+=1\n        \n        y_pred=alexnetmodel(X_train)\n        loss=criterion(y_pred,y_train)       \n\n        predicted=torch.max(y_pred.data,1)[1]\n        batch_corr=(predicted==y_train).sum()\n        trn_corr+= batch_corr\n        \n        optimizer.zero_grad()\n        loss.backward()\n        optimizer.step()\n        \n        if b%200==0:\n            print(f'epoch: {i:2}  batch: {b:4} [{10*b:6}/8000]  loss: {loss.item():10.8f}  \\\naccuracy: {trn_corr.item()*100/(10*b):7.3f}%')\n    \n    loss=loss.detach().numpy()\n    train_losses.append(loss)\n    train_correct.append(trn_corr)\n                  \n    with torch.no_grad():\n        for b, (X_test,y_test) in enumerate(test_loader):\n            b+=1\n                  \n            y_val=alexnetmodel(X_test)\n            predicted=torch.max(y_val.data,1)[1]\n            btach_corr=(predicted==y_test).sum()\n            tst_corr+=btach_corr\n\n    loss=criterion(y_val,y_test)\n    loss=loss.detach().numpy()\n    test_losses.append(loss)\n    test_correct.append(tst_corr)\n                  \nprint(f'\\nDuration: {time.time() - start_time:.0f} seconds')        ","metadata":{"papermill":{"duration":2073.100892,"end_time":"2021-10-16T14:03:18.606678","exception":false,"start_time":"2021-10-16T13:28:45.505786","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:29:05.021276Z","iopub.execute_input":"2025-10-23T04:29:05.02159Z","iopub.status.idle":"2025-10-23T04:40:20.985072Z","shell.execute_reply.started":"2025-10-23T04:29:05.021558Z","shell.execute_reply":"2025-10-23T04:40:20.983961Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"933a46d1-7c2f-4002-9d5b-c863afa4027b","cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"3b0421f6-a1a9-4673-9954-476f16dfe8b9","cell_type":"markdown","source":"# Prediction","metadata":{"papermill":{"duration":0.046523,"end_time":"2021-10-16T14:03:18.692607","exception":false,"start_time":"2021-10-16T14:03:18.646084","status":"completed"},"tags":[]}},{"id":"634c6590-8c29-4d8e-a50f-7b0164a9abda","cell_type":"code","source":"x=100\nim = inv_normalize(test_data[x][0])\nplt.imshow(np.transpose(im.numpy(),(1,2,0)))\nplt.title('Test sample image (de-normalized) - 2330026139')","metadata":{"papermill":{"duration":0.243427,"end_time":"2021-10-16T14:03:18.986681","exception":false,"start_time":"2021-10-16T14:03:18.743254","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:40:20.986467Z","iopub.execute_input":"2025-10-23T04:40:20.98675Z","iopub.status.idle":"2025-10-23T04:40:21.264917Z","shell.execute_reply.started":"2025-10-23T04:40:20.986687Z","shell.execute_reply":"2025-10-23T04:40:21.264007Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"1437504e-4ac9-4f92-9c97-26633044224a","cell_type":"code","source":"test_data[x][0].shape","metadata":{"papermill":{"duration":0.053906,"end_time":"2021-10-16T14:03:19.095284","exception":false,"start_time":"2021-10-16T14:03:19.041378","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:40:21.266104Z","iopub.execute_input":"2025-10-23T04:40:21.266335Z","iopub.status.idle":"2025-10-23T04:40:21.278329Z","shell.execute_reply.started":"2025-10-23T04:40:21.266307Z","shell.execute_reply":"2025-10-23T04:40:21.277364Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"9132f57d-fac5-41a2-bd4a-98e881d99e12","cell_type":"code","source":"CNNmodel.eval()\nwith torch.no_grad():\n    new_pred=CNNmodel(test_data[x][0].view(1,3,224,224)).argmax()\nprint(f'Predicted value: {new_pred.item()} {class_names[new_pred.item()]}')","metadata":{"papermill":{"duration":0.083533,"end_time":"2021-10-16T14:03:19.220779","exception":false,"start_time":"2021-10-16T14:03:19.137246","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:40:21.279885Z","iopub.execute_input":"2025-10-23T04:40:21.280172Z","iopub.status.idle":"2025-10-23T04:40:21.300946Z","shell.execute_reply.started":"2025-10-23T04:40:21.280123Z","shell.execute_reply":"2025-10-23T04:40:21.299954Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"bfdd2118-fc31-4d7d-8684-e905284ef140","cell_type":"code","source":"alexnetmodel.eval()\nwith torch.no_grad():\n    new_pred_1=alexnetmodel(test_data[x][0].view(1,3,224,224)).argmax()\nprint(f'Predicted value: {new_pred.item()} {class_names[new_pred.item()]}')","metadata":{"papermill":{"duration":0.088101,"end_time":"2021-10-16T14:03:19.351156","exception":false,"start_time":"2021-10-16T14:03:19.263055","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2025-10-23T04:40:21.302511Z","iopub.execute_input":"2025-10-23T04:40:21.302852Z","iopub.status.idle":"2025-10-23T04:40:21.341359Z","shell.execute_reply.started":"2025-10-23T04:40:21.302815Z","shell.execute_reply":"2025-10-23T04:40:21.340357Z"},"trusted":true},"outputs":[],"execution_count":null},{"id":"a59b9267-12ae-460f-935c-097340512d2e","cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"338c66d4-583d-4589-bff2-760e9148c6e9","cell_type":"code","source":"","metadata":{"papermill":{"duration":0.040346,"end_time":"2021-10-16T14:03:19.432209","exception":false,"start_time":"2021-10-16T14:03:19.391863","status":"completed"},"tags":[],"trusted":true},"outputs":[],"execution_count":null},{"id":"9abb3c7d-b3c7-490c-a6e1-8dccbc3b3bf7","cell_type":"markdown","source":"\n---\n\n# Part 3 — VGG on Food Subset (PyTorch)","metadata":{}},{"id":"425957d2-18be-4453-a391-25b318051cbd","cell_type":"code","source":"%matplotlib inline\nimport os\nimport sys\nimport cv2\nimport numpy as np\nimport torch\nimport torch.utils.data as data\nfrom torch.autograd import Variable\nimport torch.nn as nn\nfrom torchvision import datasets, transforms\nfrom glob import glob\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\nuse_cuda = True\ndevice = torch.device(\"cuda\" if use_cuda else \"cpu\")\ntorch.manual_seed(42) # try and make the results more reproducible\nBASE_PATH = os.path.join('..', 'input')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-10-23T04:44:28.85634Z","iopub.execute_input":"2025-10-23T04:44:28.856563Z","iopub.status.idle":"2025-10-23T04:44:29.925544Z","shell.execute_reply.started":"2025-10-23T04:44:28.856518Z","shell.execute_reply":"2025-10-23T04:44:29.924971Z"}},"outputs":[],"execution_count":null},{"id":"fa0f4687-7531-4b7d-a02d-0483c346dd9b","cell_type":"markdown","source":"## Parameters\nTry to keep the model parameters and configurations all in one cell to make it easier to keep track of (and automate later)","metadata":{"_uuid":"cc35ad75fffbed0d6e0533180dbf9b660acf2b97"}},{"id":"74bd014c-fa74-490d-9ee0-88fa377ff8b4","cell_type":"code","source":"TEST_SPLIT = 0.25\nepochs = 5\nbatch_size = 64\nMICRO_DATA = True # very small subset (just 3 groups)\nSAMPLE_TRAINING = False # make train set smaller for faster iteration\nIMG_SIZE = (224, 244)","metadata":{"trusted":true,"_uuid":"282dfd9a5c4388b3e71b344f7e3485b5dbe88f95","execution":{"iopub.status.busy":"2025-10-23T04:44:29.92664Z","iopub.execute_input":"2025-10-23T04:44:29.926815Z","iopub.status.idle":"2025-10-23T04:44:29.929977Z","shell.execute_reply.started":"2025-10-23T04:44:29.926782Z","shell.execute_reply":"2025-10-23T04:44:29.929356Z"}},"outputs":[],"execution_count":null},{"id":"de628db7-989e-411a-8f56-21e3934a7df8","cell_type":"markdown","source":"## Create Dataset\nHere we find all of the files and organize them into categories","metadata":{"_uuid":"ea20e1944fdc24a8dd36034871e39acdf80b9a14"}},{"id":"23b2237b-30b0-439d-84db-3014074303da","cell_type":"code","source":"_food_img_cands = [\"/kaggle/input/datasets/kmader/food41/images\",\n                    os.path.join(BASE_PATH, 'images'),\n                    os.path.join(BASE_PATH, 'food41', 'images'),\n                    os.path.join(BASE_PATH, 'food-images-food-101', 'images')]\nFOOD_IMAGES_DIR = next((_p for _p in _food_img_cands if os.path.isdir(_p)), _food_img_cands[0])\nprint('food images dir:', FOOD_IMAGES_DIR)\nall_img_df = pd.DataFrame({'path':\n                           glob(os.path.join(FOOD_IMAGES_DIR, '*', '*.jpg'))})\nall_img_df['category'] = all_img_df['path'].map(lambda x:\n                                                os.path.split(os.path.dirname(x))[-1].replace('_', ' '))\nif MICRO_DATA:\n    all_img_df = all_img_df[all_img_df['category'].isin(['samosa', 'gnocchi', 'hot dog'])]","metadata":{"trusted":true,"_uuid":"9f3851dc02fc8fed1de7c3e11c7201b259da3388","execution":{"iopub.status.busy":"2025-10-23T04:44:29.930943Z","iopub.execute_input":"2025-10-23T04:44:29.93115Z","iopub.status.idle":"2025-10-23T04:44:33.102272Z","shell.execute_reply.started":"2025-10-23T04:44:29.931106Z","shell.execute_reply":"2025-10-23T04:44:33.101454Z"}},"outputs":[],"execution_count":null},{"id":"267e3566-cf26-4d9b-8ee1-2affae513091","cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\ncat_enc = LabelEncoder()\nall_img_df['cat_idx'] = cat_enc.fit_transform(all_img_df['category'])\nN_CLASSES = len(cat_enc.classes_)\nprint(N_CLASSES, 'classes')\nall_img_df.sample(5)","metadata":{"trusted":true,"_uuid":"736b65c893c058daeb9c0a8176869513a3519346","execution":{"iopub.status.busy":"2025-10-23T04:44:33.103511Z","iopub.execute_input":"2025-10-23T04:44:33.103806Z","iopub.status.idle":"2025-10-23T04:44:33.994392Z","shell.execute_reply.started":"2025-10-23T04:44:33.103753Z","shell.execute_reply":"2025-10-23T04:44:33.99376Z"}},"outputs":[],"execution_count":null},{"id":"fbf2a322-7357-4c26-aa85-86a8233cdfc7","cell_type":"markdown","source":"# Data Loading Classes\nHere are the classes to load the data","metadata":{"_uuid":"06d328ce95cecc83390e1c5e83915a6a315bb74a"}},{"id":"de334216-a98c-4cdd-9250-84bfb1de0231","cell_type":"code","source":"class DataWrapper(data.Dataset):\n    ''' Data wrapper for pytorch's data loader function '''\n    def __init__(self, image_df, resize):\n        self.dataset = image_df\n        self.resize = resize\n\n    def __getitem__(self, index):\n        c_row = self.dataset.iloc[index]\n        image_path, target = c_row['path'], c_row['cat_idx']  #image and target\n        #read as rgb image, resize and convert to range 0 to 1\n        image = cv2.imread(image_path, 1)\n        if self.resize:\n            image = cv2.resize(image, IMG_SIZE)/255.0 \n        else:\n            image = image/255.0\n        image = (torch.from_numpy(image.transpose(2,0,1))).float() #NxCxHxW\n        return image, int(target)\n\n    def __len__(self):\n        return self.dataset.shape[0] ","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"execution":{"iopub.status.busy":"2025-10-23T04:44:33.995733Z","iopub.execute_input":"2025-10-23T04:44:33.995962Z","iopub.status.idle":"2025-10-23T04:44:34.001372Z","shell.execute_reply.started":"2025-10-23T04:44:33.995913Z","shell.execute_reply":"2025-10-23T04:44:34.000527Z"}},"outputs":[],"execution_count":null},{"id":"c5e4dc2a-c81f-4a59-aa4c-b9b1a65f3931","cell_type":"markdown","source":"## Split data into a train/test group\nWe can also shrink the size of the training group to make it quicker","metadata":{"_uuid":"1cfe0c93051156559a7057446c6333babec61a69"}},{"id":"adf33bf1-a8d3-453f-ac68-52fd0d014e8e","cell_type":"code","source":"from sklearn.model_selection import train_test_split\ntrain_df, test_df = train_test_split(all_img_df, \n                                     test_size=TEST_SPLIT, \n                                     random_state=42,\n                                     stratify=all_img_df['category'])\n\nif SAMPLE_TRAINING: # make train smaller for faster testing\n    train_df = train_df.\\\n        groupby('category').\\\n        apply(lambda x: x.sample(50)).\\\n        reset_index(drop=True).\\\n        sample(frac=1).\\\n        reset_index(drop=True)\nprint('train', train_df.shape[0], 'test', test_df.shape[0])","metadata":{"trusted":true,"_uuid":"b3e564bc66df41989a99dd6b6fa7719c59e76eb9","execution":{"iopub.status.busy":"2025-10-23T04:44:34.002408Z","iopub.execute_input":"2025-10-23T04:44:34.002627Z","iopub.status.idle":"2025-10-23T04:44:34.051686Z","shell.execute_reply.started":"2025-10-23T04:44:34.002576Z","shell.execute_reply":"2025-10-23T04:44:34.05108Z"}},"outputs":[],"execution_count":null},{"id":"a80dd998-a7d2-4f4a-86e2-903da0c02157","cell_type":"code","source":"train_dataset = DataWrapper(train_df, True)\ntrain_loader = torch.utils.data.DataLoader(train_dataset,shuffle=True, \n            batch_size=batch_size, pin_memory=False)#, num_workers=4)\n\ntest_dataset = DataWrapper(test_df, True)\ntest_loader = torch.utils.data.DataLoader(test_dataset,shuffle=True, \n            batch_size=batch_size, pin_memory=False) #num_workers=1)","metadata":{"trusted":true,"_uuid":"6207f36100d6de3a9dc57b8da1e645729dcd4e3e","execution":{"iopub.status.busy":"2025-10-23T04:44:34.052872Z","iopub.execute_input":"2025-10-23T04:44:34.053088Z","iopub.status.idle":"2025-10-23T04:44:34.057056Z","shell.execute_reply.started":"2025-10-23T04:44:34.053042Z","shell.execute_reply":"2025-10-23T04:44:34.056152Z"}},"outputs":[],"execution_count":null},{"id":"1a58f163-ec60-445b-8788-275033e4febb","cell_type":"markdown","source":"# Build the VGG Model","metadata":{"_uuid":"a4eee13da070cb3f6f13607d008c4e493232e580"}},{"id":"ab442c4b-fbdf-42ea-9218-a17535014daa","cell_type":"code","source":"class ConvBnReLu(nn.Module):\n    '''Repetitive block of conv->batch norm->relu'''\n    def __init__(self,in_planes,out_planes,drop=0.0,kernel=3,padding=1,stride=1):\n        super(ConvBnReLu, self).__init__()\n        self.conv = nn.Conv2d(in_planes, out_planes, kernel_size=3, \n                                          stride=stride, padding=1)\n        self.bn = nn.BatchNorm2d(out_planes)\n        self.relu = nn.ReLU()\n        self.drop = nn.Dropout(p=drop)\n\n    def forward(self,x):\n        x = self.conv(x)\n        x = self.bn(x)\n        return self.drop(self.relu(x))\n    \n\ndef conv3x3(in_planes, out_planes, stride=1):\n    '''Simple 3x3 convolution'''\n    return nn.Conv2d(in_planes, out_planes, kernel_size=3, \n                              stride=stride, padding=1)\n\nclass VGGNet(nn.Module):\n    def __init__(self,num_feat=16,num_class=N_CLASSES):\n        \"\"\"Define the components of a VGG 11 model\"\"\"\n        super(VGGNet, self).__init__()     \n        self.num_feat = num_feat\n        self.conv_1 = conv3x3(3,num_feat)\n        self.conv_2 = conv3x3(num_feat,2*num_feat)\n        self.conv_3_1 = conv3x3(2*num_feat,4*num_feat)\n        self.conv_3_2 = conv3x3(4*num_feat,4*num_feat)\n        self.conv_4_1 = conv3x3(4*num_feat,8*num_feat)\n        self.conv_4_2 = conv3x3(8*num_feat,8*num_feat)\n        self.conv_5_1 = conv3x3(8*num_feat,8*num_feat)\n        self.conv_5_2 = conv3x3(8*num_feat,2*num_feat)\n\n        self.fc1 = nn.Linear((2*num_feat)*7*7,512)\n        self.fc2 = nn.Linear(512,256)\n        self.pred = nn.Linear(256,num_class)\n\n        self.relu = nn.ReLU(inplace=True)\n        self.max_pool = nn.MaxPool2d(kernel_size=2, stride=2)\n\n    def forward(self,x):\n        \"\"\"Input x is expected to be a 4d tensor (N x C X H x W)\n           N - Number of images in minibatch\n           C - Number of channels\n           H,W  - Height and Width of image, respectively \"\"\"\n\n        conv_1 = self.conv_1(x)\n        relu_1 = self.relu(conv_1)\n        pool_1 = self.max_pool(relu_1)\n\n        conv_2 = self.conv_2(pool_1)\n        relu_2 = self.relu(conv_2)\n        pool_2 = self.max_pool(relu_2)\n\n        conv_3_1 = self.conv_3_1(pool_2)\n        relu_3_1 = self.relu(conv_3_1)\n        conv_3_2 = self.conv_3_2(relu_3_1)\n        relu_3_2 = self.relu(conv_3_2)\n        pool_3 = self.max_pool(relu_3_2)\n\n        conv_4_1 = self.conv_4_1(pool_3)\n        relu_4_1 = self.relu(conv_4_1)\n        conv_4_2 = self.conv_4_2(relu_4_1)\n        relu_4_2 = self.relu(conv_4_2)\n        pool_4 = self.max_pool(relu_4_2)\n\n        conv_5_1 = self.conv_5_1(pool_4)\n        relu_5_1 = self.relu(conv_5_1)\n        conv_5_2 = self.conv_5_2(relu_5_1)\n        relu_5_2 = self.relu(conv_5_2)\n        pool_5 = self.max_pool(relu_5_2)\n\n        fc1 = self.relu(self.fc1(pool_5.view(-1,(2*self.num_feat)*7*7)))\n        fc2 = self.relu(self.fc2(fc1))\n\n        return self.pred(fc2)","metadata":{"trusted":true,"_uuid":"78c9e6728ab27d22d62d69c994418aad0088e90f","execution":{"iopub.status.busy":"2025-10-23T04:44:34.058237Z","iopub.execute_input":"2025-10-23T04:44:34.058501Z","iopub.status.idle":"2025-10-23T04:44:34.071514Z","shell.execute_reply.started":"2025-10-23T04:44:34.058444Z","shell.execute_reply":"2025-10-23T04:44:34.070769Z"}},"outputs":[],"execution_count":null},{"id":"9a4b3b8c-568c-4d92-9183-776f29123909","cell_type":"markdown","source":"# Build and Train","metadata":{"_uuid":"babc75ea2ff1661c9628b15a3e6933e0b47b07cf"}},{"id":"a0728726-b01f-476a-9f47-0ebb67265482","cell_type":"code","source":"model = VGGNet().to(device) #VGG style model\ncriterion = nn.CrossEntropyLoss() #Use cross entropy loss","metadata":{"trusted":true,"_uuid":"db1ca24dab1501309366f0a9cef0b6c2c326d4e4","execution":{"iopub.status.busy":"2025-10-23T04:44:34.072835Z","iopub.execute_input":"2025-10-23T04:44:34.073115Z","iopub.status.idle":"2025-10-23T04:44:38.474339Z","shell.execute_reply.started":"2025-10-23T04:44:34.073061Z","shell.execute_reply":"2025-10-23T04:44:38.473488Z"}},"outputs":[],"execution_count":null},{"id":"5e7476ec-98ed-4cb0-9a80-a215c23e541e","cell_type":"code","source":"optimizer = torch.optim.Adam(model.parameters(),lr=1e-3)","metadata":{"trusted":true,"_uuid":"76e06bae374b903cbada471ba59154eea04bba0d","execution":{"iopub.status.busy":"2025-10-23T04:44:38.475673Z","iopub.execute_input":"2025-10-23T04:44:38.475961Z","iopub.status.idle":"2025-10-23T04:44:38.479972Z","shell.execute_reply.started":"2025-10-23T04:44:38.475907Z","shell.execute_reply":"2025-10-23T04:44:38.479234Z"}},"outputs":[],"execution_count":null},{"id":"fc70315f-ee4e-4558-96e5-c8716a7d1d88","cell_type":"code","source":"use_gpu = True","metadata":{"trusted":true,"_uuid":"a6cf06c89da09d74dc94e3d02ce94a0d2e3c0c64","execution":{"iopub.status.busy":"2025-10-23T04:44:38.481305Z","iopub.execute_input":"2025-10-23T04:44:38.481599Z","iopub.status.idle":"2025-10-23T04:44:38.490405Z","shell.execute_reply.started":"2025-10-23T04:44:38.481541Z","shell.execute_reply":"2025-10-23T04:44:38.489654Z"}},"outputs":[],"execution_count":null},{"id":"89d4fd07-706a-4d87-9bd6-4976647332ca","cell_type":"code","source":"# nice wait bars\nfrom tqdm.auto import tqdm, trange as tnrange","metadata":{"trusted":true,"_uuid":"425fd7a7c6559dafd81ee577729c15115603cdf1","execution":{"iopub.status.busy":"2025-10-23T04:44:38.491512Z","iopub.execute_input":"2025-10-23T04:44:38.49176Z","iopub.status.idle":"2025-10-23T04:44:38.500036Z","shell.execute_reply.started":"2025-10-23T04:44:38.491685Z","shell.execute_reply":"2025-10-23T04:44:38.499498Z"}},"outputs":[],"execution_count":null},{"id":"2afec1fd-112b-433a-937a-06421958756c","cell_type":"markdown","source":"# Train for given number of epochs","metadata":{"_uuid":"7da72c9389c9d84b29219e7a6dfba8679e9b3a51"}},{"id":"f78c1595-0010-4163-9f14-e7d3ab69caa1","cell_type":"code","source":"from collections import defaultdict\ntrain_results = defaultdict(list)\ntrain_iter, test_iter, best_acc = 0,0,0\nfig, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2, figsize = (10, 10))\nax1.set_title('Train Loss - 2330026139')\nax2.set_title('Train Accuracy - 2330026139')\nax3.set_title('Test Loss - 2330026139')\nax4.set_title('Test Accuracy - 2330026139')\n\nfor i in tnrange(epochs, desc='Epochs'):\n    print(\"Epoch \",i)\n    ## Train Phase\n    #Model switches to train phase\n    model.train() \n    \n    # Running through all mini batches in the dataset\n    count, loss_val, correct, total = train_iter, 0, 0, 0\n    for data, target in tqdm(train_loader, desc='Training'):    \n        if use_gpu: #Using GPU & Cuda\n            data, target = data.to(device), target.to(device)\n\n        output = model(data) #FWD prop\n        loss = criterion(output, target) #Cross entropy loss\n        c_loss = loss.data.item()\n        ax1.plot(count, c_loss, 'r.')\n        loss_val += c_loss\n\n        optimizer.zero_grad() #Zero out any cached gradients\n        loss.backward() #Backward pass\n        optimizer.step() #Update the weights\n\n        #Compute accuracy\n        predicted = output.data.max(1)[1] #get index of max\n        total += target.size(0) #total samples in mini batch\n        c_acc = (predicted == target).sum().item()\n        ax2.plot(count, c_acc/target.size(0), 'r.')\n        correct += c_acc\n        count +=1\n    train_loss_val, train_iter, train_acc = loss_val/len(train_loader.dataset), count, correct/float(total)\n    \n    print(\"Training loss: \", train_loss_val, \" train acc: \",train_acc)    \n    ## Test Phase\n    \n    #Model switches to test phase\n    model.eval()\n\n    #Running through all mini batches in the dataset\n    count, correct, total, lost_val = test_iter, 0, 0, 0\n    for data, target in tqdm(test_loader, desc='Testing'):\n        if use_gpu: #Using GPU & Cuda\n            data, target = data.to(device), target.to(device)\n        output = model(data)\n        loss = criterion(output, target) #Cross entropy loss\n        c_loss = loss.data.item()\n        ax3.plot(count, c_loss, 'b.')\n        loss_val += c_loss\n        #Compute accuracy\n        predicted = output.data.max(1)[1] #get index of max\n        total += target.size(0) #total samples in mini batch\n        c_acc = (predicted == target).sum().item()\n        ax4.plot(count, c_acc/target.size(0), 'b.')\n        correct += c_acc\n        count += 1\n\n    #Accuracy over entire dataset\n    test_acc, test_iter, test_loss_val = correct/float(total), count, loss_val/len(test_loader.dataset)\n    print(\"Epoch: \",i,\" test set accuracy: \",test_acc)\n    \n    train_results['epoch'].append(i)\n    train_results['train_loss'].append(train_loss_val)\n    train_results['train_acc'].append(train_acc)\n    train_results['train_iter'].append(train_iter)\n    \n    train_results['test_loss'].append(test_loss_val)\n    train_results['test_acc'].append(test_acc)\n    train_results['test_iter'].append(test_iter)\n    \n    #Save model with best accuracy\n    if test_acc > best_acc:\n        best_acc = test_acc\n        torch.save(model.state_dict(), 'best_model.pth') \nfig.savefig('train_curves.png')","metadata":{"trusted":true,"_uuid":"302fd53900f9e7efe66e77efc6d8864ff9bd56ba","execution":{"iopub.status.busy":"2025-10-23T04:44:38.50124Z","iopub.execute_input":"2025-10-23T04:44:38.501459Z","iopub.status.idle":"2025-10-23T04:46:49.891299Z","shell.execute_reply.started":"2025-10-23T04:44:38.501421Z","shell.execute_reply":"2025-10-23T04:46:49.890485Z"}},"outputs":[],"execution_count":null},{"id":"7a528ce1-51e3-4771-897a-94a9ebc3c4ec","cell_type":"code","source":"train_results_df = pd.DataFrame(train_results)\nfig, (ax1, ax2) = plt.subplots(1, 2, figsize = (10, 5))\ntrain_results_df.plot('epoch', ['train_loss', 'test_loss'], ax=ax1)\nax1.set_title('Loss - 2330026139')\ntrain_results_df.plot('epoch', ['train_acc', 'test_acc'], ax=ax2)\nax2.set_title('Accuracy - 2330026139')\nfig.savefig('out_curves.png')","metadata":{"trusted":true,"_uuid":"afd4d321ef0ad9c016467e94b17ac421e6db714d","execution":{"iopub.status.busy":"2025-10-23T04:46:49.892714Z","iopub.execute_input":"2025-10-23T04:46:49.893002Z","iopub.status.idle":"2025-10-23T04:46:50.549808Z","shell.execute_reply.started":"2025-10-23T04:46:49.892945Z","shell.execute_reply":"2025-10-23T04:46:50.548499Z"}},"outputs":[],"execution_count":null},{"id":"3c7794b9-256a-4fd3-a9db-450de3a4b0f7","cell_type":"code","source":"","metadata":{"trusted":true,"_uuid":"6e32eb51f8c38c0bd4828cab7f2d1fa45422e50d"},"outputs":[],"execution_count":null},{"id":"4fe0813a-13d2-4f21-a703-9b6a64551063","cell_type":"markdown","source":"\n---\n\n# Part 4 — ResNet18/34 on BengaliAI (PyTorch)","metadata":{}},{"id":"1ca584d5-78c9-48f8-a75e-b0b15c5f4ee0","cell_type":"markdown","source":"# ResNet18/34 PyTorch\n\n- References (ResNet):\n  - https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py\n  - https://arxiv.org/pdf/1512.03385.pdf\n  \n  \n- Acknowledgements:\n  - Original kernels: https://www.kaggle.com/hanjoonchoe/grapheme-resnet-18-n-l-inference-lb-0-8566 and https://www.kaggle.com/hanjoonchoe/grapheme-resnet-18-naive-learning-3","metadata":{}},{"id":"4460e037-51e9-43e9-81e0-ad552a63a4a5","cell_type":"markdown","source":"# Part 1","metadata":{}},{"id":"2c337568-0841-4a89-ab73-a77102073dd9","cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport os\n'''\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n'''\n\n# Any results you write to the current directory are saved as output.\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import Dataset,DataLoader\nfrom torchvision import transforms,models\nfrom tqdm.auto import tqdm","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:45.966637Z","iopub.execute_input":"2025-10-23T05:39:45.967059Z","iopub.status.idle":"2025-10-23T05:39:45.973127Z","shell.execute_reply.started":"2025-10-23T05:39:45.966991Z","shell.execute_reply":"2025-10-23T05:39:45.972244Z"}},"outputs":[],"execution_count":null},{"id":"06fa86f4-ccca-40df-8fb4-5fe594be5fd3","cell_type":"code","source":"import os as _os\n_BENGALI_CANDS = [\"/kaggle/input/competitions/bengaliai-cv19\",\n                  \"/kaggle/input/bengaliai-cv19\"]\nBENGALI_BASE = next((_p for _p in _BENGALI_CANDS if _os.path.isdir(_p)), _BENGALI_CANDS[0])\nprint('bengali data root:', BENGALI_BASE)\ntrain = pd.read_csv(os.path.join(BENGALI_BASE, 'train.csv'))\n# The feather files come from another notebook's output and may not be attached.\n# If missing, fall back to resizing the raw parquet images (137x236 -> 64x64).\n_feather_dir = '/kaggle/usr/lib/resize-and-load-with-feather-format-much-faster'\nif all(_os.path.exists(os.path.join(_feather_dir, f'train_data_{_i}.feather')) for _i in range(4)):\n    print('loading preprocessed feather data')\n    data0 = pd.read_feather(os.path.join(_feather_dir, 'train_data_0.feather'))\n    data1 = pd.read_feather(os.path.join(_feather_dir, 'train_data_1.feather'))\n    data2 = pd.read_feather(os.path.join(_feather_dir, 'train_data_2.feather'))\n    data3 = pd.read_feather(os.path.join(_feather_dir, 'train_data_3.feather'))\nelse:\n    print('feather notebook output not attached; resizing raw parquet to 64x64 (takes a few minutes)')\n    import cv2 as _cv2\n    import numpy as _np\n    def _resize_parquet(_fname, _size=64):\n        _df = pd.read_parquet(os.path.join(BENGALI_BASE, _fname))\n        _pix = _df.drop('image_id', axis=1).values.reshape(-1, 137, 236)\n        _small = _cv2.resize(_pix.reshape(-1, 236), (_size, _size * len(_pix))).reshape(-1, _size * _size)\n        _out = pd.DataFrame(_small)\n        _out.insert(0, 'image_id', _df['image_id'].values)\n        return _out\n    data0 = _resize_parquet('train_image_data_0.parquet')\n    data1 = _resize_parquet('train_image_data_1.parquet')\n    data2 = _resize_parquet('train_image_data_2.parquet')\n    data3 = _resize_parquet('train_image_data_3.parquet')","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:45.974919Z","iopub.execute_input":"2025-10-23T05:39:45.975197Z","iopub.status.idle":"2025-10-23T05:39:57.612787Z","shell.execute_reply.started":"2025-10-23T05:39:45.975142Z","shell.execute_reply":"2025-10-23T05:39:57.611904Z"}},"outputs":[],"execution_count":null},{"id":"b3dfb0af-688b-4dab-928d-dfc81dff5d84","cell_type":"code","source":"# legacy shell listing of the optional feather notebook output, replaced by a python check\nprint('feather notebook output available:', all(\n    _os.path.exists(os.path.join(_feather_dir, f'train_data_{_i}.feather')) for _i in range(4)))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:57.614348Z","iopub.execute_input":"2025-10-23T05:39:57.614684Z","iopub.status.idle":"2025-10-23T05:39:58.636114Z","shell.execute_reply.started":"2025-10-23T05:39:57.614622Z","shell.execute_reply":"2025-10-23T05:39:58.635199Z"}},"outputs":[],"execution_count":null},{"id":"ba8861af-df18-4bd1-b530-f3808559cb0f","cell_type":"code","source":"data_full = pd.concat([data0,data1,data2,data3],ignore_index=True)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:58.637522Z","iopub.execute_input":"2025-10-23T05:39:58.637759Z","iopub.status.idle":"2025-10-23T05:39:58.948581Z","shell.execute_reply.started":"2025-10-23T05:39:58.637707Z","shell.execute_reply":"2025-10-23T05:39:58.94795Z"}},"outputs":[],"execution_count":null},{"id":"8a492cf2-8e11-47b1-bd22-c10fe081e565","cell_type":"code","source":"class GraphemeDataset(Dataset):\n    def __init__(self,df,label,_type='train'):\n        self.df = df\n        self.label = label\n    def __len__(self):\n        return len(self.df)\n    def __getitem__(self,idx):\n        label1 = self.label.vowel_diacritic.values[idx]\n        label2 = self.label.grapheme_root.values[idx]\n        label3 = self.label.consonant_diacritic.values[idx]\n        image = self.df.iloc[idx][1:].values.reshape(64,64).astype(np.float)\n        return image,label1,label2,label3","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:58.951054Z","iopub.execute_input":"2025-10-23T05:39:58.951387Z","iopub.status.idle":"2025-10-23T05:39:58.957142Z","shell.execute_reply.started":"2025-10-23T05:39:58.951321Z","shell.execute_reply":"2025-10-23T05:39:58.956555Z"}},"outputs":[],"execution_count":null},{"id":"b430fa6b-6153-43ae-acc3-923fb496254d","cell_type":"code","source":"class ResidualBlock(nn.Module):\n    def __init__(self,in_channels,out_channels,stride=1,kernel_size=3,padding=1,bias=False):\n        super(ResidualBlock,self).__init__()\n        self.cnn1 =nn.Sequential(\n            nn.Conv2d(in_channels,out_channels,kernel_size,stride,padding,bias=False),\n            nn.BatchNorm2d(out_channels),\n            nn.ReLU(True)\n        )\n        self.cnn2 = nn.Sequential(\n            nn.Conv2d(out_channels,out_channels,kernel_size,1,padding,bias=False),\n            nn.BatchNorm2d(out_channels)\n        )\n        if stride != 1 or in_channels != out_channels:\n            self.shortcut = nn.Sequential(\n                nn.Conv2d(in_channels,out_channels,kernel_size=1,stride=stride,bias=False),\n                nn.BatchNorm2d(out_channels)\n            )\n        else:\n            self.shortcut = nn.Sequential()\n            \n    def forward(self,x):\n        residual = x\n        x = self.cnn1(x)\n        x = self.cnn2(x)\n        x += self.shortcut(residual)\n        x = nn.ReLU(True)(x)\n        return x","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:58.959183Z","iopub.execute_input":"2025-10-23T05:39:58.959444Z","iopub.status.idle":"2025-10-23T05:39:58.968461Z","shell.execute_reply.started":"2025-10-23T05:39:58.959387Z","shell.execute_reply":"2025-10-23T05:39:58.967707Z"}},"outputs":[],"execution_count":null},{"id":"ac5a95ed-9231-4940-be8a-fff5e171fc18","cell_type":"markdown","source":"## ResNet-18 Model","metadata":{}},{"id":"868fad44-9517-4264-9cba-b18c60447a84","cell_type":"code","source":"class ResNet18(nn.Module):\n    def __init__(self):\n        super(ResNet18,self).__init__()\n        \n        self.block1 = nn.Sequential(\n            nn.Conv2d(1,64,kernel_size=2,stride=2,padding=3,bias=False),\n            nn.BatchNorm2d(64),\n            nn.ReLU(True)\n        )\n        \n        self.block2 = nn.Sequential(\n            nn.MaxPool2d(1,1),\n            ResidualBlock(64,64),\n            ResidualBlock(64,64,2)\n        )\n        \n        self.block3 = nn.Sequential(\n            ResidualBlock(64,128),\n            ResidualBlock(128,128,2)\n        )\n        \n        self.block4 = nn.Sequential(\n            ResidualBlock(128,256),\n            ResidualBlock(256,256,2)\n        )\n        self.block5 = nn.Sequential(\n            ResidualBlock(256,512),\n            ResidualBlock(512,512,2)\n        )\n        \n        self.avgpool = nn.AvgPool2d(2)\n        # vowel_diacritic\n        self.fc1 = nn.Linear(512,11)\n        # grapheme_root\n        self.fc2 = nn.Linear(512,168)\n        # consonant_diacritic\n        self.fc3 = nn.Linear(512,7)\n        \n    def forward(self,x):\n        x = self.block1(x)\n        x = self.block2(x)\n        x = self.block3(x)\n        x = self.block4(x)\n        x = self.block5(x)\n        x = self.avgpool(x)\n        x = x.view(x.size(0),-1)\n        x1 = self.fc1(x)\n        x2 = self.fc2(x)\n        x3 = self.fc3(x)\n        return x1,x2,x3","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:58.969397Z","iopub.execute_input":"2025-10-23T05:39:58.969613Z","iopub.status.idle":"2025-10-23T05:39:58.979811Z","shell.execute_reply.started":"2025-10-23T05:39:58.969577Z","shell.execute_reply":"2025-10-23T05:39:58.979183Z"}},"outputs":[],"execution_count":null},{"id":"5dd3628d-0e90-46d3-abce-fb601fa65cbb","cell_type":"markdown","source":"## ResNet34 Model","metadata":{}},{"id":"c00ddf17-d5a8-40c4-9ccd-0401cc5912a9","cell_type":"code","source":"class ResNet34(nn.Module):\n    def __init__(self):\n        super(ResNet34,self).__init__()\n        \n        self.block1 = nn.Sequential(\n            nn.Conv2d(1,64,kernel_size=2,stride=2,padding=3,bias=False),\n            nn.BatchNorm2d(64),\n            nn.ReLU(True)\n        )\n        \n        self.block2 = nn.Sequential(\n            nn.MaxPool2d(1,1),\n            ResidualBlock(64,64),\n            ResidualBlock(64,64,2)\n        )\n        \n        self.block3 = nn.Sequential(\n            ResidualBlock(64,128),\n            ResidualBlock(128,128,2)\n        )\n        \n        self.block4 = nn.Sequential(\n            ResidualBlock(128,256),\n            ResidualBlock(256,256,2)\n        )\n        self.block5 = nn.Sequential(\n            ResidualBlock(256,512),\n            ResidualBlock(512,512,2)\n        )\n        \n        self.avgpool = nn.AvgPool2d(2)\n        # vowel_diacritic\n        self.fc1 = nn.Linear(512,11)\n        # grapheme_root\n        self.fc2 = nn.Linear(512,168)\n        # consonant_diacritic\n        self.fc3 = nn.Linear(512,7)\n        \n    def forward(self,x):\n        x = self.block1(x)\n        x = self.block2(x)\n        x = self.block3(x)\n        x = self.block4(x)\n        x = self.block5(x)\n        x = self.avgpool(x)\n        x = x.view(x.size(0),-1)\n        x1 = self.fc1(x)\n        x2 = self.fc2(x)\n        x3 = self.fc3(x)\n        return x1,x2,x3","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:58.980749Z","iopub.execute_input":"2025-10-23T05:39:58.980974Z","iopub.status.idle":"2025-10-23T05:39:58.994189Z","shell.execute_reply.started":"2025-10-23T05:39:58.98093Z","shell.execute_reply":"2025-10-23T05:39:58.99341Z"}},"outputs":[],"execution_count":null},{"id":"9c9d9ff4-21c8-4def-bddb-8594019b54b8","cell_type":"markdown","source":"## Remarks\n\n- See previous version for torch summary as submission must not have internet enabled","metadata":{}},{"id":"26b862e6-5f44-4576-8786-a82451166390","cell_type":"code","source":"#!pip install torchsummary\n#from torchsummary import summary","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:58.9952Z","iopub.execute_input":"2025-10-23T05:39:58.995426Z","iopub.status.idle":"2025-10-23T05:39:59.008004Z","shell.execute_reply.started":"2025-10-23T05:39:58.995377Z","shell.execute_reply":"2025-10-23T05:39:59.007184Z"}},"outputs":[],"execution_count":null},{"id":"a18861bd-3423-4ec5-9bfd-9d512989111d","cell_type":"code","source":"device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:59.009035Z","iopub.execute_input":"2025-10-23T05:39:59.009305Z","iopub.status.idle":"2025-10-23T05:39:59.09472Z","shell.execute_reply.started":"2025-10-23T05:39:59.009258Z","shell.execute_reply":"2025-10-23T05:39:59.093845Z"}},"outputs":[],"execution_count":null},{"id":"eb2c3682-7364-49c3-81da-6eaad4d1d159","cell_type":"code","source":"#model_resnet34 = ResNet34().to(device)\n#summary(model_resnet34, (1, 64, 64))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:59.095824Z","iopub.execute_input":"2025-10-23T05:39:59.096014Z","iopub.status.idle":"2025-10-23T05:39:59.109113Z","shell.execute_reply.started":"2025-10-23T05:39:59.095982Z","shell.execute_reply":"2025-10-23T05:39:59.10834Z"}},"outputs":[],"execution_count":null},{"id":"f544d737-1094-42d7-9f2b-180d3dc22d65","cell_type":"markdown","source":"## Training Model\n\n- We use ResNet34 and train for 50 epochs","metadata":{}},{"id":"e1fc7d0a-796a-4404-841e-056667bdc8d7","cell_type":"code","source":"model = ResNet34().to(device)\noptimizer = optimizer = torch.optim.Adam(model.parameters(), lr=4e-4)\n#scheduler = torch.optim.lr_scheduler.CyclicLR(optimizer, base_lr=1e-4, max_lr=0.05)\ncriterion = nn.CrossEntropyLoss()\nbatch_size=32","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:39:59.110335Z","iopub.execute_input":"2025-10-23T05:39:59.110653Z","iopub.status.idle":"2025-10-23T05:40:03.780414Z","shell.execute_reply.started":"2025-10-23T05:39:59.110597Z","shell.execute_reply":"2025-10-23T05:40:03.779777Z"}},"outputs":[],"execution_count":null},{"id":"1cb35de7-b5d6-4959-bdcc-ea62634d3f50","cell_type":"code","source":"epochs = 5\nmodel.train()\nlosses = []\naccs = []\nfor epoch in range(epochs):\n    reduced_index =train.groupby(['grapheme_root', 'vowel_diacritic', 'consonant_diacritic']).apply(lambda x: x.sample(5)).image_id.values\n    reduced_train = train.loc[train.image_id.isin(reduced_index)]\n    reduced_data = data_full.loc[data_full.image_id.isin(reduced_index)]\n    train_image = GraphemeDataset(reduced_data,reduced_train)\n    train_loader = torch.utils.data.DataLoader(train_image,batch_size=batch_size,shuffle=True)\n    \n    print('epochs {}/{} '.format(epoch+1,epochs))\n    running_loss = 0.0\n    running_acc = 0.0\n    for idx, (inputs,labels1,labels2,labels3) in tqdm(enumerate(train_loader),total=len(train_loader)):\n        inputs = inputs.to(device)\n        labels1 = labels1.to(device)\n        labels2 = labels2.to(device)\n        labels3 = labels3.to(device)\n        \n        optimizer.zero_grad()\n        outputs1,outputs2,outputs3 = model(inputs.unsqueeze(1).float())\n        loss1 = criterion(outputs1,labels1)\n        loss2 = criterion(outputs2,labels2)\n        loss3 = criterion(outputs3,labels3)\n        running_loss += loss1+loss2+loss3\n        running_acc += (outputs1.argmax(1)==labels1).float().mean()\n        running_acc += (outputs2.argmax(1)==labels2).float().mean()\n        running_acc += (outputs3.argmax(1)==labels3).float().mean()\n        (loss1+loss2+loss3).backward()\n        optimizer.step()\n    #scheduler.step()\n    losses.append(running_loss/len(train_loader))\n    accs.append(running_acc/(len(train_loader)*3))\n    print('acc : {:.2f}%'.format(running_acc/(len(train_loader)*3)))\n    print('loss : {:.4f}'.format(running_loss/len(train_loader)))\ntorch.save(model.state_dict(), 'resnet34_50epochs_saved_weights.pth')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:40:03.781869Z","iopub.execute_input":"2025-10-23T05:40:03.782155Z","iopub.status.idle":"2025-10-23T05:42:05.737584Z","shell.execute_reply.started":"2025-10-23T05:40:03.782101Z","shell.execute_reply":"2025-10-23T05:42:05.736839Z"}},"outputs":[],"execution_count":null},{"id":"c1540413-a817-4639-a8f3-e510f201385f","cell_type":"code","source":"import matplotlib.pyplot as plt\nfig,ax = plt.subplots(1,2,figsize=(15,5))\nax[0].plot(losses)\nax[0].set_title('loss - 2330026139')\nax[1].plot(accs)\nax[1].set_title('acc - 2330026139')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:42:05.738872Z","iopub.execute_input":"2025-10-23T05:42:05.739161Z","iopub.status.idle":"2025-10-23T05:42:06.266481Z","shell.execute_reply.started":"2025-10-23T05:42:05.739098Z","shell.execute_reply":"2025-10-23T05:42:06.265476Z"}},"outputs":[],"execution_count":null},{"id":"ef17f23e-887a-4dcb-a956-8ab3ccbec7c9","cell_type":"markdown","source":"# Part 2","metadata":{}},{"id":"1dc08ffb-bb36-42f1-b351-b620318de6c2","cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\n# (optional) list all input files\n# for dirname, _, filenames in os.walk('/kaggle/input'):\n#     for filename in filenames:\n#         print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.\nimport cv2\nimport torch\nimport torch.nn as nn\nfrom torch.utils.data import Dataset,DataLoader\nfrom torchvision import transforms,models\nfrom tqdm.auto import tqdm","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:42:06.268121Z","iopub.execute_input":"2025-10-23T05:42:06.268694Z","iopub.status.idle":"2025-10-23T05:42:06.542867Z","shell.execute_reply.started":"2025-10-23T05:42:06.268631Z","shell.execute_reply":"2025-10-23T05:42:06.542293Z"}},"outputs":[],"execution_count":null},{"id":"b9057295-4f3e-4f33-afea-a244b6db4b60","cell_type":"code","source":"test = pd.read_csv(os.path.join(BENGALI_BASE, 'test.csv'))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:42:06.543866Z","iopub.execute_input":"2025-10-23T05:42:06.544058Z","iopub.status.idle":"2025-10-23T05:42:06.553261Z","shell.execute_reply.started":"2025-10-23T05:42:06.544024Z","shell.execute_reply":"2025-10-23T05:42:06.552425Z"}},"outputs":[],"execution_count":null},{"id":"83fcfe77-5a16-4d70-bccb-ef071b01c778","cell_type":"code","source":"class GraphemeDataset(Dataset):\n    def __init__(self,df,_type='train'):\n        self.df = df\n    def __len__(self):\n        return len(self.df)\n    def __getitem__(self,idx):\n        image = self.df.iloc[idx][1:].values.reshape(64,64).astype(float)\n        return image","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:42:06.554305Z","iopub.execute_input":"2025-10-23T05:42:06.554575Z","iopub.status.idle":"2025-10-23T05:42:06.56015Z","shell.execute_reply.started":"2025-10-23T05:42:06.554526Z","shell.execute_reply":"2025-10-23T05:42:06.559466Z"}},"outputs":[],"execution_count":null},{"id":"d18e48e8-7d0d-466a-912e-b62bf41e3d32","cell_type":"code","source":"device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\nmodel = ResNet34().to(device)\n_weights_path = '/kaggle/input/trained400/resnet34_400epochs_saved_weights.pth'\nif not os.path.exists(_weights_path):\n    print('trained400 weights not found; using locally trained resnet34_50epochs_saved_weights.pth')\n    _weights_path = 'resnet34_50epochs_saved_weights.pth'\nmodel.load_state_dict(torch.load(_weights_path))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:42:06.561252Z","iopub.execute_input":"2025-10-23T05:42:06.561518Z","iopub.status.idle":"2025-10-23T05:42:07.047433Z","shell.execute_reply.started":"2025-10-23T05:42:06.56145Z","shell.execute_reply":"2025-10-23T05:42:07.046734Z"}},"outputs":[],"execution_count":null},{"id":"50d23b07-06c4-4fa5-9e36-6398761e0f27","cell_type":"code","source":"def Resize(df,size=64):\n    resized = {} \n    df = df.set_index('image_id')\n    for i in tqdm(range(df.shape[0])):\n        image = cv2.resize(df.loc[df.index[i]].values.reshape(137,236),(size,size))\n        resized[df.index[i]] = image.reshape(-1)\n    resized = pd.DataFrame(resized).T.reset_index()\n    resized.columns = resized.columns.astype(str)\n    resized.rename(columns={'index':'image_id'},inplace=True)\n    return resized","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:42:07.049023Z","iopub.execute_input":"2025-10-23T05:42:07.049405Z","iopub.status.idle":"2025-10-23T05:42:07.055285Z","shell.execute_reply.started":"2025-10-23T05:42:07.049324Z","shell.execute_reply":"2025-10-23T05:42:07.054599Z"}},"outputs":[],"execution_count":null},{"id":"c18d69dd-ba64-4d47-ae04-e951752986c7","cell_type":"code","source":"model.eval()\ntest_data = ['test_image_data_0.parquet','test_image_data_1.parquet','test_image_data_2.parquet','test_image_data_3.parquet']\npredictions = []\nbatch_size=1\nfor fname in test_data:\n    data = pd.read_parquet(os.path.join(BENGALI_BASE, fname))\n    data = Resize(data)\n    test_image = GraphemeDataset(data)\n    test_loader = torch.utils.data.DataLoader(test_image,batch_size=1,shuffle=False)\n    with torch.no_grad():\n        for idx, (inputs) in tqdm(enumerate(test_loader),total=len(test_loader)):\n            inputs.to(device)\n            \n            outputs1,outputs2,outputs3 = model(inputs.unsqueeze(1).float().cuda())\n            predictions.append(outputs3.argmax(1).cpu().detach().numpy())\n            predictions.append(outputs2.argmax(1).cpu().detach().numpy())\n            predictions.append(outputs1.argmax(1).cpu().detach().numpy())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:42:07.056703Z","iopub.execute_input":"2025-10-23T05:42:07.057009Z","iopub.status.idle":"2025-10-23T05:44:51.212578Z","shell.execute_reply.started":"2025-10-23T05:42:07.056939Z","shell.execute_reply":"2025-10-23T05:44:51.211844Z"}},"outputs":[],"execution_count":null},{"id":"2867065e-d0c2-4720-9147-aab8defdf83f","cell_type":"markdown","source":"# Save Results","metadata":{}},{"id":"1710abd7-f5ae-4379-8801-6c14d32679fe","cell_type":"code","source":"submission = pd.read_csv(os.path.join(BENGALI_BASE, 'sample_submission.csv'))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:44:51.213614Z","iopub.execute_input":"2025-10-23T05:44:51.213814Z","iopub.status.idle":"2025-10-23T05:44:51.223924Z","shell.execute_reply.started":"2025-10-23T05:44:51.21378Z","shell.execute_reply":"2025-10-23T05:44:51.223384Z"}},"outputs":[],"execution_count":null},{"id":"9f8419d0-8ddd-48b5-a482-0f207151d938","cell_type":"code","source":"submission.target = np.hstack(predictions)\nsubmission.head(10)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:44:51.22525Z","iopub.execute_input":"2025-10-23T05:44:51.225566Z","iopub.status.idle":"2025-10-23T05:44:51.240352Z","shell.execute_reply.started":"2025-10-23T05:44:51.225509Z","shell.execute_reply":"2025-10-23T05:44:51.239705Z"}},"outputs":[],"execution_count":null},{"id":"f69a1569-3a82-4908-badf-02af8d4a7a7f","cell_type":"code","source":"submission.to_csv('submission.csv',index=False)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-23T05:44:51.241502Z","iopub.execute_input":"2025-10-23T05:44:51.241744Z","iopub.status.idle":"2025-10-23T05:44:51.557277Z","shell.execute_reply.started":"2025-10-23T05:44:51.241697Z","shell.execute_reply":"2025-10-23T05:44:51.556564Z"}},"outputs":[],"execution_count":null},{"id":"0dc4bbe7-ff6a-44b9-b8d5-6397fb6f77a4","cell_type":"markdown","source":"\n---\n\n# Part 5 [Intermediate & Advanced] — ResNet18 Fine-Tuning on 36 Fruit/Veggie\n\nUsing one of the models covered in the lecture (ResNet18), we fine-tune the ImageNet pretrained network on the 36 Fruit/Veggie dataset, then evaluate and predict on the held-out test set.","metadata":{}},{"id":"fd5f3e0a-5dce-46f6-b709-efd388d6c00a","cell_type":"code","source":"import os, copy, time, random\nimport numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torchvision import datasets, models, transforms\nfrom torch.utils.data import DataLoader\nimport matplotlib.pyplot as plt\nfrom sklearn.metrics import classification_report, confusion_matrix\nimport seaborn as sns\n\nFRUIT36_BASE = '/kaggle/input/datasets/thomaschxu/lab4-fruit/36FruitVeggie/36FruitVeggie'\nif not os.path.exists(FRUIT36_BASE):\n    FRUIT36_BASE = '/kaggle/input/lab4-fruit/36FruitVeggie/36FruitVeggie'\nTRAIN_DIR = os.path.join(FRUIT36_BASE, 'train')\nVAL_DIR   = os.path.join(FRUIT36_BASE, 'validation')\nTEST_DIR  = os.path.join(FRUIT36_BASE, 'test')\nprint('data root:', FRUIT36_BASE)\n\ndevice = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\nprint('device:', device)\n\nIMG_MEAN, IMG_STD = [0.485, 0.456, 0.406], [0.229, 0.224, 0.225]\ntrain_tf = transforms.Compose([\n    transforms.RandomResizedCrop(224, scale=(0.7, 1.0)),\n    transforms.RandomHorizontalFlip(),\n    transforms.ToTensor(),\n    transforms.Normalize(IMG_MEAN, IMG_STD),\n])\neval_tf = transforms.Compose([\n    transforms.Resize(256),\n    transforms.CenterCrop(224),\n    transforms.ToTensor(),\n    transforms.Normalize(IMG_MEAN, IMG_STD),\n])\n\ntrain_ds = datasets.ImageFolder(TRAIN_DIR, train_tf)\nval_ds   = datasets.ImageFolder(VAL_DIR, eval_tf)\ntest_ds  = datasets.ImageFolder(TEST_DIR, eval_tf)\nclass_names = train_ds.classes\nnum_classes = len(class_names)\nprint(f'classes: {num_classes} | train: {len(train_ds)} | val: {len(val_ds)} | test: {len(test_ds)}')\n\ntrain_loader = DataLoader(train_ds, batch_size=32, shuffle=True, num_workers=2)\nval_loader   = DataLoader(val_ds, batch_size=64, shuffle=False, num_workers=2)\ntest_loader  = DataLoader(test_ds, batch_size=64, shuffle=False, num_workers=2)","metadata":{},"outputs":[],"execution_count":null},{"id":"b1798387-5dfa-4295-9ce2-35a9b204c457","cell_type":"code","source":"model = models.resnet18(weights=models.ResNet18_Weights.IMAGENET1K_V1)\nmodel.fc = nn.Linear(model.fc.in_features, num_classes)\nmodel = model.to(device)\ntotal_params = sum(p.numel() for p in model.parameters())\ntrainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)\nprint(f'ResNet18 | total params: {total_params:,} | trainable: {trainable_params:,}')\n\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.Adam(model.parameters(), lr=3e-4)\nscheduler = optim.lr_scheduler.StepLR(optimizer, step_size=5, gamma=0.1)","metadata":{},"outputs":[],"execution_count":null},{"id":"67ed1877-061a-4dd2-b512-bbd1f28a5621","cell_type":"code","source":"def run_epoch(model, loader, criterion, optimizer=None, train=True):\n    model.train() if train else model.eval()\n    total = correct = 0\n    running_loss = 0.0\n    with torch.set_grad_enabled(train):\n        for x, y in loader:\n            x, y = x.to(device), y.to(device)\n            out = model(x)\n            loss = criterion(out, y)\n            if train:\n                optimizer.zero_grad(); loss.backward(); optimizer.step()\n            running_loss += loss.item() * x.size(0)\n            _, pred = out.max(1)\n            correct += (pred == y).sum().item()\n            total += y.size(0)\n    return running_loss / total, correct / total\n\nEPOCHS = 15\nhist = {'train_loss': [], 'val_loss': [], 'train_acc': [], 'val_acc': []}\nfor ep in range(1, EPOCHS + 1):\n    tl, ta = run_epoch(model, train_loader, criterion, optimizer, train=True)\n    vl, va = run_epoch(model, val_loader, criterion, None, train=False)\n    scheduler.step()\n    hist['train_loss'].append(tl); hist['val_loss'].append(vl)\n    hist['train_acc'].append(ta); hist['val_acc'].append(va)\n    print(f'Epoch {ep:2d}/{EPOCHS}: train_loss {tl:.4f} train_acc {ta:.4f} | '\n          f'val_loss {vl:.4f} val_acc {va:.4f}')","metadata":{},"outputs":[],"execution_count":null},{"id":"912fa6e6-e833-49c6-b5e6-5f1b5e95f3be","cell_type":"code","source":"plt.figure(figsize=(7, 5))\nplt.plot(range(1, EPOCHS + 1), hist['train_acc'], 'o-', label='train')\nplt.plot(range(1, EPOCHS + 1), hist['val_acc'], 's-', label='val')\nplt.xlabel('Epoch'); plt.ylabel('Accuracy'); plt.legend()\nplt.title('ResNet18 fine-tune: train/val accuracy - 2330026139')\nplt.grid(alpha=0.3); plt.tight_layout(); plt.show()\n\nplt.figure(figsize=(7, 5))\nplt.plot(range(1, EPOCHS + 1), hist['train_loss'], 'o-', label='train')\nplt.plot(range(1, EPOCHS + 1), hist['val_loss'], 's-', label='val')\nplt.xlabel('Epoch'); plt.ylabel('Loss'); plt.legend()\nplt.title('ResNet18 fine-tune: train/val loss - 2330026139')\nplt.grid(alpha=0.3); plt.tight_layout(); plt.show()","metadata":{},"outputs":[],"execution_count":null},{"id":"7c1b847e-75f0-4fab-b56d-9ffa123a9bce","cell_type":"code","source":"model.eval()\ny_true, y_pred = [], []\ntest_loss = correct = total = 0\nwith torch.no_grad():\n    for x, y in test_loader:\n        x, y = x.to(device), y.to(device)\n        out = model(x)\n        test_loss += criterion(out, y).item() * x.size(0)\n        _, pred = out.max(1)\n        correct += (pred == y).sum().item(); total += y.size(0)\n        y_true += y.cpu().tolist(); y_pred += pred.cpu().tolist()\nprint(f'Test accuracy: {correct / total:.4f} ({correct}/{total}) | Test loss: {test_loss / total:.4f}')\nprint(classification_report(y_true, y_pred, target_names=class_names, digits=4))","metadata":{},"outputs":[],"execution_count":null},{"id":"8040c7ba-24c1-4299-9c38-00aa234b1c6c","cell_type":"code","source":"cm = confusion_matrix(y_true, y_pred)\nplt.figure(figsize=(14, 12))\nsns.heatmap(cm, annot=True, fmt='d', cmap='Blues',\n            xticklabels=class_names, yticklabels=class_names)\nplt.xlabel('Predicted'); plt.ylabel('True')\nplt.title('ResNet18 confusion matrix on 36 Fruit/Veggie - 2330026139')\nplt.tight_layout(); plt.show()\n\ntorch.save(model.state_dict(), 'fruit36_resnet18.pth')\nprint('model saved: fruit36_resnet18.pth')","metadata":{},"outputs":[],"execution_count":null},{"id":"25162559-7974-426f-a792-35feae3a79fd","cell_type":"code","source":"inv = transforms.Normalize(\n    mean=[-m / s for m, s in zip(IMG_MEAN, IMG_STD)],\n    std=[1 / s for s in IMG_STD])\nmodel.eval()\nidxs = random.sample(range(len(test_ds)), 6)\nfig, axes = plt.subplots(2, 3, figsize=(12, 8))\nfor ax, idx in zip(axes.flat, idxs):\n    img, label = test_ds[idx]\n    with torch.no_grad():\n        out = model(img.unsqueeze(0).to(device))\n        prob = torch.softmax(out, 1)[0]\n        conf, pred = prob.max(0)\n    ax.imshow(np.transpose(inv(img).numpy(), (1, 2, 0)).clip(0, 1))\n    ax.set_title(f'True: {class_names[label]} | Pred: {class_names[pred.item()]} ({conf.item():.3f}) - 2330026139')\n    ax.axis('off')\nfig.suptitle('ResNet18 predictions on 36 Fruit/Veggie - 2330026139')\nplt.tight_layout(); plt.show()","metadata":{},"outputs":[],"execution_count":null}]}