{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"accelerator":"GPU","colab":{"gpuType":"T4","provenance":[]},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":19989,"databundleVersionId":1160143,"sourceType":"competition"}],"dockerImageVersionId":30919,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 1. Faster R-CNN","metadata":{"id":"RwOhUf1gTx7T"}},{"cell_type":"markdown","source":"### The Task:\nThis assignment evaluates deep learning models for object detection using the Global Wheat Detection dataset from Kaggle. The objective is to train, evaluate, and compare two models. i have choosen Faster R-CNN and YOLOv5m6 on detection models and their ability to detect wheat heads in images, a task relevant to agricultural monitoring.","metadata":{"id":"0NooaZd28WHv"}},{"cell_type":"markdown","source":"### Brief Description:\nI implemented `Faster R-CNN` with a ResNet-50 FPN, ResNet-50 FPN v2, mobilenet_v3_large_fpn  and mobilenet_v3_large_320_fpn backbone and `YOLO v5` varients such as YOLOv5s6,\tYOLOv5m6, YOLOv5l6, YOLOv5x6 trained on the Global Wheat Detection dataset, and evaluated their performance using mean Average Precision (mAP) at various IoU thresholds. and among these model only `ResNet-50 FPN` and `yolov5m6` were selected of their overall performance. Data preprocessing included bounding box parsing and image augmentations. Training was conducted over 5 epochs for Faster R-CNN and 10 epochs for YOLOv5m6, leveraging pre-trained weights and GPU acceleration.\n\n\n### Summary of Main Findings\nFaster R-CNN achieved an mAP@[0.5:0.95] of 0.4372, excelling at lower IoU thresholds (e.g., 0.8440 at IoU 0.5) but struggling with precise localization (e.g., 0.0006 at IoU 0.95). YOLOv5m6 reached an mAP@[0.5:0.95] of 0.46889, with strong performance at IoU 0.5 (0.91615) and better generalization across thresholds. YOLOv5m6 outperformed Faster R-CNN overall, likely due to its efficiency and additional training epochs.","metadata":{"id":"8ONkPE2-HyYW"}},{"cell_type":"code","source":"|import pandas as pd # Data analysis library\n\nimport albumentations as A # data agumentation library\nfrom albumentations.pytorch.transforms import ToTensorV2 # it use to convert images into tensor, requires to train DL models\n\nimport torch # it a framework use to build and train deep learning models\nimport os  # it use to work with local directories\nfrom torch.utils.data import Dataset, DataLoader  # creating custom datasets AND DataLoader is used to load and batch the dataset during training or evaluation.\nimport cv2 # this is use to work with images and videos in realtime\nimport ast # module allows you to parse, analyze, and manipulate Python code itself.\nimport numpy as np # use for neumerical computing in python\nimport matplotlib.pyplot as plt # to visualize the graph\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")","metadata":{"id":"Bh0UdiWrGEiM","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:24:14.534308Z","iopub.execute_input":"2025-03-26T22:24:14.534632Z","iopub.status.idle":"2025-03-26T22:24:22.547850Z","shell.execute_reply.started":"2025-03-26T22:24:14.534593Z","shell.execute_reply":"2025-03-26T22:24:22.547193Z"},"outputId":"efebe076-2ed7-4ef3-e1d3-ef82c17e18a3"},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 1.1 Load Dataset\n\nHere, we are loading .csv dataset file which is `train.csv` and `test.csv`.\nthe dataset contains the information collected on Global Wheat Detection images such as\n- `image_id` - the unique image ID\n- `width`, `height` - the width and height of the images\n- `bbox` - a bounding box, formatted as a Python-style list of `[xmin, ymin, width, height]`","metadata":{"id":"zMh8VQKPGvQU"}},{"cell_type":"code","source":"data = pd.read_csv(\"/kaggle/input/global-wheat-detection/train.csv\")\n\nDIR_TRAIN = '/kaggle/input/global-wheat-detection/train'\nDIR_TEST = '/kaggle/input/global-wheat-detection/test'\n\ndata.head(2)","metadata":{"id":"sRMY0Mr0Ek32","outputId":"814d0e24-a322-4b0a-9fbc-fb89d6f68e5e","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:24:23.665274Z","iopub.execute_input":"2025-03-26T22:24:23.665748Z","iopub.status.idle":"2025-03-26T22:24:23.941122Z","shell.execute_reply.started":"2025-03-26T22:24:23.665717Z","shell.execute_reply":"2025-03-26T22:24:23.940285Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Method","metadata":{"id":"IUemdoCRF6vp"}},{"cell_type":"markdown","source":"## 1.2 Generate validation and training sets\n\n- on this stage, we devide data into training and validation, because training set will be used to train the Model and Validation set will be use to evaluate the model performance on unseen data during training","metadata":{"id":"wqWKdAJnyvHo"}},{"cell_type":"code","source":"# creating valid and Training set\nimage_ids = data['image_id'].unique()\nvalid_ids = image_ids[-665:]\ntrain_ids = image_ids[:-665]\n\nvalid_df = data[data['image_id'].isin(valid_ids)]\ntrain_df = data[data['image_id'].isin(train_ids)]","metadata":{"id":"cgdZCcM0yUXe","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:24:27.013503Z","iopub.execute_input":"2025-03-26T22:24:27.013861Z","iopub.status.idle":"2025-03-26T22:24:27.076798Z","shell.execute_reply.started":"2025-03-26T22:24:27.013830Z","shell.execute_reply":"2025-03-26T22:24:27.075991Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Load custom dataset.\nhere pytorch dosen't know how to read the data from disk, matching correct labels/bounding boxs, apply any agumentation techniuqes, and package everything into a tensor. So user need to teach pytorch to create Custom dataset. that contains `__getitem__` and `__len__` methods.\n- `__getitem__`: This function is inside a PyTorch dataset class for object detection to fetch and preprocess an image and its annotations. It fetches the `image_id`, looks up the corresponding image file, and verifies whether it exists. The image is read using OpenCV, converted to RGB, and normalized. Bounding boxes are fetched, transformed from `(x, y, width, height)` to `(x_min, y_min, x_max, y_max)`, and saved as tensors. Labels, area, and `iscrowd` flags are also created. Any transformations, if given, are applied to the image and to bounding boxes. The processed image, target dictionary, and image ID are then returned by the function.  Without it, the model would not receive properly structured inputs, hindering its ability to learn object detection tasks effectively on the global-wheat dataset or any similar dataset.\n\n- `__len__` : total number of samples in a dataset\n- `show_image` : This method use to display the image","metadata":{"id":"rWvMvhnCHE0P"}},{"cell_type":"code","source":"class WheatDataset(Dataset):\n  def __init__(self, csv_file, root_dir, transform=None):\n    self.data = csv_file\n    self.image_ids = self.data['image_id'].unique()\n    self.root_dir = root_dir\n    self.transform = transform\n\n  def __getitem__(self, index):\n\n    # grab one sample (image + label)\n    image_id = self.image_ids[index]\n    records = self.data[self.data['image_id'] == image_id] # get all the records related that image id\n\n    # Construct the image path using os.path.join to ensure correct path handling\n    image_path=os.path.join(self.root_dir, f'{image_id}.jpg')\n\n    # Check if file exists!\n    if not os.path.exists(image_path):\n        print(f\"❌ File does not exist: {image_path}\")\n        return None\n\n    # read the image and covert it into RGB format.\n    image = cv2.imread(image_path, cv2.IMREAD_COLOR)\n    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32)\n    image /= 255.0  # scale the images\n\n    # store all the bounding boxes\n    boxes = []\n    for box in records['bbox']:\n        box = ast.literal_eval(box) #  safely converts the string\n        x_min = box[0]\n        y_min = box[1]\n        x_max = x_min + box[2]\n        y_max = y_min + box[3]\n        boxes.append([x_min, y_min, x_max, y_max])\n\n    boxes = torch.as_tensor(boxes, dtype=torch.float32) # convert the list of boxes into tensor\n    labels = torch.ones((records.shape[0],), dtype=torch.int64)  # Single class (wheat)\n    area = (boxes[:, 3] - boxes[:, 1]) * (boxes[:, 2] - boxes[:, 0])\n    iscrowd = torch.zeros((records.shape[0],), dtype=torch.int64)  # 0 for each box\n\n    # package everything in target dict, This is exactly what PyTorch detection models expect.\n    target = {}\n    target['boxes'] = boxes\n    target['labels'] = labels\n    target['area'] = area\n    target['iscrowd'] = iscrowd\n\n    target['image_id'] = torch.tensor([index])\n\n    if self.transform:\n\n      sample = {\n                'image': image,\n                'bboxes': target['boxes'],\n                'labels': labels.tolist()\n            }\n      sample = self.transform(**sample)\n      image = sample['image']\n      target['boxes'] = torch.stack(tuple(map(torch.tensor, zip(*sample['bboxes'])))).permute(1, 0)\n\n    return image, target, image_id\n\n\n  def __len__(self):\n    return len(self.image_ids)\n\n\n  def show_image(self, index):\n\n    # Get the image using __getitem__\n    image, _, _ = self.__getitem__(index)\n\n    # Check if image and target were returned\n    if image is None:\n        print(f\"Error: Could not load image for index {index}\")\n        return\n\n    # Display the image using matplotlib\n    plt.imshow(image)\n    plt.axis('off')  # Turn off axis labels\n    plt.show()\n\nload_data = WheatDataset(csv_file=data, root_dir= DIR_TRAIN)\nload_data.show_image(10) # display image on index","metadata":{"id":"rKD3jZ_WHknE","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:24:32.399263Z","iopub.execute_input":"2025-03-26T22:24:32.399542Z","iopub.status.idle":"2025-03-26T22:24:33.055993Z","shell.execute_reply.started":"2025-03-26T22:24:32.399519Z","shell.execute_reply":"2025-03-26T22:24:33.055171Z"},"outputId":"8cd0b773-e651-461b-e0fa-e251a2c3a395"},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 1.3 data Agumentation\nwe are defining Two data Albumentations transform pipeline, one for training and one for validation data preprocessing.\nData Agumentation is necessary because, it adds variation in the training images without needing more data. And helps the model learn invariance to orientation (if wheat can appear anywhere in the image).\n\nbounding box coordinate format we are using is\n`pascal_voc` boxes are defined as `[x_min, y_min, x_max, y_max]`","metadata":{"id":"5pg02FvrF3Lc"}},{"cell_type":"code","source":"# Albumentations\ndef get_train_transform():\n    return A.Compose([\n        A.HorizontalFlip(p=0.5),   # Randomly flip images horizontally 50% of the time\n        ToTensorV2(p=1.0)           # Convert images & bboxes to PyTorch tensors\n    ], bbox_params={'format': 'pascal_voc',     # Specifies bbox format\n                    'label_fields': ['labels']} # Labels to be transformed along with bboxes\n                    )\n\ndef get_valid_transform():\n    return A.Compose([\n        ToTensorV2(p=1.0)         # Only convert to tensor, no augmentation for validation\n    ], bbox_params={'format': 'pascal_voc',      # Specifies bbox format\n                    'label_fields': ['labels']}  # Labels to be transformed along with bboxes\n                    )","metadata":{"id":"3CVL7UotF2db","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:24:39.309520Z","iopub.execute_input":"2025-03-26T22:24:39.309889Z","iopub.status.idle":"2025-03-26T22:24:39.314556Z","shell.execute_reply.started":"2025-03-26T22:24:39.309858Z","shell.execute_reply":"2025-03-26T22:24:39.313677Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 1.4 Model Selection:\nTo determine the best models for this task, I conducted a total of four experiments with Faster R-CNN variants For Faster R-CNN, I tested the following backbones:\n- `ResNet-50 FPN`: A standard backbone with Feature Pyramid Networks for multi-scale feature extraction.\n\n- `ResNet-50 FPN v2`: An updated version with improved feature processing.\n\n- `MobileNet v3 Large FPN`: A lightweight backbone for efficiency.\n\n- `MobileNet v3 Large 320 FPN`: A smaller variant for even lower computational cost.\n\n\nI chose Faster R-CNN with `ResNet-50 FPN` as the starting Faster R-CNN model because it possessed the best `mAP@[0.5:0.95]` of `0.4372` out of the tested variations of Faster R-CNN (against `0.3750` for `ResNet-50 FPN v2`, `0.3924` for `MobileNet v3 Large FPN`, and `0.1194` for `MobileNet v3 Large 320 FPN`). This backbone offers a robust two-stage detection pipeline with an RPN well-suited for precise localization a critical feature for wheat head detection where bounding box precision is most important. `ResNet-50 FPN` variant applies pre-trained weights and multi-scale feature extraction, providing a decent baseline with no undue computational overhead compared to MobileNet variants that had to make accuracy concessions for speed here.","metadata":{"id":"y3UAK5dVLYjw"}},{"cell_type":"markdown","source":"We employ transfer learning within this project where we utilize a pretrained Faster R-CNN network that has previously learned simple visual patterns from an enormous dataset (`COCO`). Instead of training the network from scratch, we fine-tune it so that it recognizes a specific object `wheat`in our custom dataset. Training is much more efficient and lighter on data for this reason. The original `COCO` dataset has 91 object classes, whereas our dataset only has a single class, wheat. Therefore, we modify the last classification layer of the model to make predictions for just two classes `wheat` and `background`. In Faster R-CNN, the `ROI` heads are two in total: one is a classifier that outputs the object `class` and the second is a `box regressor` that outputs the object location. In this case, we substitute the existing classifier head with `FastRCNNPredictor`, fine-tuning the model exactly for our wheat detection task.","metadata":{"id":"fchmREqQMZ0y"}},{"cell_type":"code","source":"import torchvision\nfrom torchvision.models.detection.faster_rcnn import FastRCNNPredictor\n\n# Load pre-trained Faster R-CNN model\nmodel = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)                   # fasterrcnn_resnet50_fpn\n# model = torchvision.models.detection.fasterrcnn_resnet50_fpn_v2(pretrained=True)              # fasterrcnn_resnet50_fpn_v2\n# model = torchvision.models.detection.fasterrcnn_mobilenet_v3_large_fpn(pretrained=True)       # fasterrcnn_mobilenet_v3_large_fpn\n# model = torchvision.models.detection.fasterrcnn_mobilenet_v3_large_320_fpn(pretrained=True)   # fasterrcnn_mobilenet_v3_large_320_fpn\n\n# Modify the classifier head for your number of classes (background + wheat)\nnum_classes = 2  # wheat + background\nin_features = model.roi_heads.box_predictor.cls_score.in_features\nmodel.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)","metadata":{"id":"K4TtxGdZCO_9","outputId":"f30ca890-084f-4d5c-945a-201cdc74b7c5","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:24:42.200557Z","iopub.execute_input":"2025-03-26T22:24:42.200866Z","iopub.status.idle":"2025-03-26T22:24:46.897444Z","shell.execute_reply.started":"2025-03-26T22:24:42.200841Z","shell.execute_reply":"2025-03-26T22:24:46.896715Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"the Averager class here to efficiently keep track of and calculate the running average of a selected metric—most often the `training loss` or `validation loss` during the process of model training. Instead of storing all individual loss values per batch or iteration, which can be memory-intensive and redundant, the Averager keeps a running sum and an iteration counter to calculate the average in real time. This makes it easier to monitor the performance of the model during training and provides you with an immediate understanding of how the loss is progressing with time. The `reset()` function is also one that assists us in eliminating the values that are built up at the start of each new epoch so we can always have the correct average per training or validation cycle. Overall, this utility helps to create and simplify the procedure for tracking important measures in deep learning pipelines.","metadata":{"id":"_WRQRF3THE0R"}},{"cell_type":"code","source":"class Averager:\n    def __init__(self):\n        self.current_total = 0.0\n        self.iterations = 0.0\n\n    def send(self, value):\n        self.current_total += value\n        self.iterations += 1\n\n    @property\n    def value(self):\n        if self.iterations == 0:\n            return 0\n        else:\n            return 1.0 * self.current_total / self.iterations\n\n    def reset(self):\n        self.current_total = 0.0\n        self.iterations = 0.0","metadata":{"id":"5JYld8rs0SWr","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:24:52.181749Z","iopub.execute_input":"2025-03-26T22:24:52.182309Z","iopub.status.idle":"2025-03-26T22:24:52.187577Z","shell.execute_reply.started":"2025-03-26T22:24:52.182279Z","shell.execute_reply":"2025-03-26T22:24:52.186714Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"The `collate_fn` is a custom function that tells the DataLoader how to assemble (or \"collate\") the samples into a batch.\n`batch` is a list of objects  __getitem__() yields in the dataset class.\nIn this case, every entry is a tuple: (image, target, image_id).","metadata":{"id":"LFPOfJWSHE0R"}},{"cell_type":"code","source":"def collate_fn(batch):\n    return tuple(zip(*batch))\n\ntrain_dataset = WheatDataset(train_df, DIR_TRAIN, get_train_transform())\nvalid_dataset = WheatDataset(valid_df, DIR_TRAIN, get_valid_transform())\n\n\n# split the dataset in train and test set\nindices = torch.randperm(len(train_dataset)).tolist()\n\n# loading the data into dataloader\ntrain_data_loader = DataLoader(\n    train_dataset,\n    batch_size=8,\n    shuffle=False,\n    num_workers=4,\n    collate_fn=collate_fn\n)\n\nvalid_data_loader = DataLoader(\n    valid_dataset,\n    batch_size=8,\n    shuffle=False,\n    num_workers=4,\n    collate_fn=collate_fn\n)","metadata":{"id":"WWA3lbva0SUS","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:24:53.879944Z","iopub.execute_input":"2025-03-26T22:24:53.880257Z","iopub.status.idle":"2025-03-26T22:24:53.901201Z","shell.execute_reply.started":"2025-03-26T22:24:53.880228Z","shell.execute_reply":"2025-03-26T22:24:53.900244Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"PyTorch requires the model and the data to be on the same device in order to perform calculations. If the model is on the GPU but the data is still on the CPU, PyTorch will throw an error because it cannot calculate tensors on two different devices simultaneously. Moving the data to the GPU ensures that the processing is done efficiently because GPUs are optimized for the fast processing of matrix operations, which is essential in deep learning operations. Moving the data to the right device is normally done within the training or validation loop, just before the data is passed through the model to train or make inferences. This ensures efficient and smooth processing of the learning process.","metadata":{"id":"KycS1liEHE0S"}},{"cell_type":"code","source":"device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')\nimages, targets, image_ids = next(iter(train_data_loader))\nimages = list(image.to(device) for image in images) # move the images tensor from CPU to GPU for fater computation\ntargets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n# take a sample and it's label to plot the image\nboxes = targets[2]['boxes'].cpu().numpy().astype(np.int32)\nsample = images[2].permute(1,2,0).cpu().numpy()\n\n# plot the an image\nfig, ax = plt.subplots(1, 1, figsize=(16, 8))\n\nfor box in boxes:\n    cv2.rectangle(sample,\n                  (box[0], box[1]),\n                  (box[2], box[3]),\n                  (220, 0, 0), 3)\n\nax.set_axis_off()\nax.imshow(sample)","metadata":{"id":"eaQTYl5t0SPW","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:24:57.582199Z","iopub.execute_input":"2025-03-26T22:24:57.582476Z","iopub.status.idle":"2025-03-26T22:24:59.764258Z","shell.execute_reply.started":"2025-03-26T22:24:57.582453Z","shell.execute_reply":"2025-03-26T22:24:59.762865Z"},"outputId":"debceb3f-dc90-499a-d3d9-67023d86f6c4"},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 1.5 Training and Validation\n## Training Schedule\n\n`Stochastic Gradient Descent` (`SGD`) as the optimizer of this training setup is used for updating model parameters. Update step size is defined by defining the `learning rate` as `0.005`, and momentum as `0.9` helps by including previous gradients for faster convergence. `Weight decay` of `0.0005` as `L2 regularization` is used to prevent overfitting. `Checkpoints` are created to retain model progress in case training is interrupted, so that it can resume from there subsequently. Training of the Faster R-CNN model is performed for five epochs on T4 GPU under `batch size 8` with restricted memory. All loss items such as classifier and box regression losses are monitored to check performance. It is a configuration that has balance between optimization effectiveness, model generalization, as well as training stability.","metadata":{"id":"4Sjrz_aGHE0S"}},{"cell_type":"code","source":"# train the model\nmodel.to(device)\nparams = [p for p in model.parameters() if p.requires_grad]\noptimizer = torch.optim.SGD(params, lr=0.005, momentum=0.9, weight_decay=0.0005)\nlr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=3, gamma=0.1)\n# lr_scheduler = None\n\n# run the training loop for 5 epochs\nnum_epochs = 5","metadata":{"id":"pkER5vBi0SF-","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:25:07.583070Z","iopub.execute_input":"2025-03-26T22:25:07.583369Z","iopub.status.idle":"2025-03-26T22:25:07.650428Z","shell.execute_reply.started":"2025-03-26T22:25:07.583345Z","shell.execute_reply":"2025-03-26T22:25:07.649831Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import torch\nimport os\n# Initialize Averager objects to track average loss over iterations\ntrain_loss_hist = Averager()\nval_loss_hist = Averager()\n\nitr = 1  # Iteration counter\nstart_epoch = 0  # Variable to store the epoch to resume training from\n\n# Check if there is a checkpoint to load from\ncheckpoint_dir = \"/kaggle/working//Fast R-CNN\"\nos.makedirs(name=checkpoint_dir, exist_ok=True)\n\ncheckpoint_path = checkpoint_dir+'/checkpoint_fasterrcnn_resnet50_fpn.pth'\nif os.path.exists(checkpoint_path):\n    print(f\"Loading checkpoint from {checkpoint_path}...\")\n    checkpoint = torch.load(checkpoint_path)\n    model.load_state_dict(checkpoint['model_state_dict'])\n    optimizer.load_state_dict(checkpoint['optimizer_state_dict'])\n    start_epoch = checkpoint['epoch'] + 1  # Resume from the next epoch\n    train_losses = checkpoint['train_losses']\n    val_losses = checkpoint['val_losses']\n    itr = checkpoint['iteration']\n    print(f\"Resumed training from epoch {start_epoch}...\")\nelse:\n    print(\"No checkpoint found. Starting from scratch.\")\n    train_losses = []\n    val_losses = []\n\n# Start the training loop for the specified number of epochs\nfor epoch in range(start_epoch, num_epochs):\n    print(f\"\\nEpoch #{epoch + 1} started\\n\" + \"-\"*30)\n\n    # ----------------------------------- TRAINING PHASE ---------------------------------------------\n    train_loss_hist.reset()  # Reset the training loss history at the start of each epoch\n\n    model.train()  # Set the model to training mode\n\n    # Iterate through batches in the training data loader\n    for images, targets, image_ids in train_data_loader:\n        # Move images and targets to the appropriate device (GPU/CPU)\n        images = list(image.to(device) for image in images)\n        targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n        # Perform a forward pass through the model and get the loss\n        loss_dict = model(images, targets)\n\n        # Sum up the losses (classification loss + bounding box regression loss)\n        losses = sum(loss for loss in loss_dict.values())\n        loss_value = losses.item()\n\n        # Update the loss history with the current loss\n        train_loss_hist.send(loss_value)\n\n        # Zero the gradients\n        optimizer.zero_grad()\n\n        # Calculate the loss on backpropagation\n        losses.backward()\n\n        # Update the model parameters\n        optimizer.step()\n\n        # Print the current iteration loss every 50 iterations\n        if itr % 50 == 0:\n            print(f\"Iteration #{itr} training loss: {loss_value:.4f}\")\n\n        itr += 1  # Increment the iteration counter\n\n    # Step the learning rate scheduler, if provided\n    if lr_scheduler is not None:\n        lr_scheduler.step()\n\n    # Save the average training loss for the epoch\n    train_epoch_loss = train_loss_hist.value\n    train_losses.append(train_epoch_loss)\n    print(f\"Epoch #{epoch + 1} Training loss: {train_epoch_loss:.4f}\")\n\n    # ----- VALIDATION PHASE -----\n    val_loss_hist.reset()  # Reset validation loss history for the current epoch\n\n    # model.train()  # Stay in train mode to compute validation losses (if required by model)\n    # Alternatively, you can use model.eval() if you don't need to compute losses during validation inference.\n    with torch.no_grad():\n        for images, targets, image_ids in valid_data_loader:\n            images = list(image.to(device) for image in images)\n            targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n            # Forward pass through the model to get validation losses\n            loss_dict = model(images, targets)\n\n            # Sum all losses\n            losses = sum(loss for loss in loss_dict.values())\n\n            # Update the validation loss history\n            val_loss_hist.send(losses.item())\n\n    # Save the average validation loss for the epoch\n    val_epoch_loss = val_loss_hist.value\n    val_losses.append(val_epoch_loss)\n    print(f\"Epoch #{epoch + 1} Validation loss: {val_epoch_loss:.4f}\")\n\n    # Save checkpoint after each epoch\n    torch.save({\n        'epoch': epoch,\n        'model_state_dict': model.state_dict(),\n        'optimizer_state_dict': optimizer.state_dict(),\n        'train_losses': train_losses,\n        'val_losses': val_losses,\n        'iteration': itr,\n    }, checkpoint_path)\n\nprint(f\"Training Losses: {train_losses}\")\nprint(f\"Validation Losses: {val_losses}\")\n\nprint(\"\\nTraining & validation completed.\")","metadata":{"id":"vPB_vbbBZBVy","outputId":"bb02f42c-f97a-4af9-f377-3a4d0ec3055a","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T22:42:46.355294Z","iopub.execute_input":"2025-03-26T22:42:46.355613Z","iopub.status.idle":"2025-03-26T23:39:05.142026Z","shell.execute_reply.started":"2025-03-26T22:42:46.355589Z","shell.execute_reply":"2025-03-26T23:39:05.140808Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"`Training losses` for Faster R-CNN decreased from `0.8347` to `0.7944`, and\n\n`validation losses` from `0.8571` to `0.8168`, indicating stable learning.","metadata":{"id":"43D46FodHE0T"}},{"cell_type":"code","source":"# Plot after training is done\nplt.figure(figsize=(10, 5))\nplt.plot(train_losses, label='Training Loss')\nplt.plot(val_losses, label='Validation Loss')\nplt.xlabel('Epoch')\nplt.ylabel('Loss')\nplt.title('Training and Validation Loss')\nplt.legend()\nplt.show()","metadata":{"id":"7BdjFCKL0R_B","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:39:50.695167Z","iopub.execute_input":"2025-03-26T23:39:50.695472Z","iopub.status.idle":"2025-03-26T23:39:50.962729Z","shell.execute_reply.started":"2025-03-26T23:39:50.695447Z","shell.execute_reply":"2025-03-26T23:39:50.961962Z"},"outputId":"7ae5d63f-2a2b-43b0-c9be-4b3d657ad466"},"outputs":[],"execution_count":null},{"cell_type":"code","source":"images, targets, image_ids = next(iter(valid_data_loader))\n\nimages = list(img.to(device) for img in images)\ntargets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\nboxes = targets[1]['boxes'].cpu().numpy().astype(np.int32)\nsample = images[1].permute(1,2,0).cpu().numpy()\nlabels = targets[1]['labels'].cpu().numpy()","metadata":{"id":"mgQozLVI0R6Q","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:39:56.862865Z","iopub.execute_input":"2025-03-26T23:39:56.863169Z","iopub.status.idle":"2025-03-26T23:39:57.857490Z","shell.execute_reply.started":"2025-03-26T23:39:56.863146Z","shell.execute_reply":"2025-03-26T23:39:57.856390Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"`model.eval()` is called to switch the model to evaluation mode, which is necessary because certain layers like dropout and batch normalization behave differently during evaluation compared to training.","metadata":{"id":"2sOENx_-HE0U"}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport numpy as np\n# Assuming 'model' and 'device' are defined as in your code snippet\n# and you have a DataLoader named 'valid_data_loader'\n\nmodel.eval()  # Set the model to evaluation mode\nwith torch.no_grad():\n    images, targets, image_ids = next(iter(valid_data_loader))\n    images = list(img.to(device) for img in images)\n    outputs = model(images)  # Get model predictions\n\n    # Example: Plot predictions for the first image in the batch\n    boxes = outputs[0]['boxes'].cpu().numpy().astype(np.int32)\n    scores = outputs[0]['scores'].cpu().numpy()\n    labels = outputs[0]['labels'].cpu().numpy()\n    sample = images[0].permute(1, 2, 0).cpu().numpy()\n\n    fig, ax = plt.subplots(1, 1, figsize=(16, 8))\n\n    for box, score, label in zip(boxes, scores, labels):\n        if score > 0.5:  # Only show detections with a confidence score above 0.5\n            cv2.rectangle(\n                sample,\n                (box[0], box[1]),\n                (box[2], box[3]),\n                (0, 255, 0),  # Green color for bounding boxes\n                2,\n            )\n            # Optionally, add labels and scores to the plot\n            if label == 1:\n                label = 'Wheat'\n            else:\n                label = 'Background'\n\n            ax.text(box[0], box[1], f\"{label}: {score:.2f}\", color='white')\n\n    ax.set_axis_off()\n    ax.imshow(sample)\n    plt.show()","metadata":{"id":"BkSEXkf90Rw4","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:40:03.297247Z","iopub.execute_input":"2025-03-26T23:40:03.297539Z","iopub.status.idle":"2025-03-26T23:40:05.764263Z","shell.execute_reply.started":"2025-03-26T23:40:03.297515Z","shell.execute_reply":"2025-03-26T23:40:05.763255Z"},"outputId":"e30af0ee-f255-469f-816d-ff4c9473f50c"},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"the predicted image, wheat at some places was detected and some were missing. it shows that this is not a very good model.","metadata":{"id":"Uewh0WImURCk"}},{"cell_type":"markdown","source":"## 1.6 Model Evaluation\nFor Model Evaluation we use mAP metrics to measure the performance of Faster R-CNN model.\nMean Average Precision (mAP) is a significant object detection metric, averaging precision at different recall levels and Intersection over Union (IoU) thresholds. It provides a balanced measure of the accuracy of a model in detecting and localizing objects. Improved performance in both precision and recall in different detection criteria is indicated with greater mAP.","metadata":{"id":"ntZP976gHE0Y"}},{"cell_type":"markdown","source":"`compute_iou`: This function calculates the Intersection over Union (IoU) between two bounding boxes. IoU is a measure of how much two boxes overlap. It is calculated as the area of intersection divided by the area of union of the two boxes.","metadata":{"id":"pFbCJ4poHE0Y"}},{"cell_type":"code","source":"import numpy as np\n\ndef compute_iou(box1, box2):\n    \"\"\"Compute IoU between two boxes. Boxes are [xmin, ymin, xmax, ymax].\"\"\"\n    x1 = max(box1[0], box2[0])\n    y1 = max(box1[1], box2[1])\n    x2 = min(box1[2], box2[2])\n    y2 = min(box1[3], box2[3])\n\n    inter_area = max(0, x2 - x1) * max(0, y2 - y1)\n\n    box1_area = (box1[2] - box1[0]) * (box1[3] - box1[1])\n    box2_area = (box2[2] - box2[0]) * (box2[3] - box2[1])\n\n    union_area = box1_area + box2_area - inter_area\n\n    if union_area == 0:\n        return 0\n    return inter_area / union_area\n","metadata":{"id":"PhHSRkO3COsm","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:40:48.456414Z","iopub.execute_input":"2025-03-26T23:40:48.456785Z","iopub.status.idle":"2025-03-26T23:40:48.462173Z","shell.execute_reply.started":"2025-03-26T23:40:48.456731Z","shell.execute_reply":"2025-03-26T23:40:48.461432Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"This function calculates Average Precision (AP) at a specified IoU threshold. It initially orders the predicted boxes in descending order of confidence score, and secondly marks each prediction as a true positive (TP) or false positive (FP) based on the IoU with ground truth boxes. Subsequently, precision and recall values are calculated and the AP from the precision-recall curve is calculated.","metadata":{"id":"yjRMGFy_HE0Z"}},{"cell_type":"code","source":"def calculate_ap_at_iou(pred_boxes, gt_boxes, iou_threshold=0.5):\n    \"\"\"Calculate AP at a specific IoU threshold.\"\"\"\n    if len(pred_boxes) == 0:\n        return 0.0\n\n    pred_boxes = sorted(pred_boxes, key=lambda x: x[4], reverse=True)  # sort by score\n\n    tp = np.zeros(len(pred_boxes))\n    fp = np.zeros(len(pred_boxes))\n\n    matched_gt = set()\n\n    for idx, pred in enumerate(pred_boxes):\n        pred_box = pred[:4]\n        best_iou = 0\n        best_gt_idx = -1\n\n        for gt_idx, gt_box in enumerate(gt_boxes):\n            iou = compute_iou(pred_box, gt_box)\n            if iou > best_iou:\n                best_iou = iou\n                best_gt_idx = gt_idx\n\n        if best_iou >= iou_threshold and best_gt_idx not in matched_gt:\n            tp[idx] = 1  # True Positive\n            matched_gt.add(best_gt_idx)\n        else:\n            fp[idx] = 1  # False Positive\n\n    cumulative_tp = np.cumsum(tp)\n    cumulative_fp = np.cumsum(fp)\n\n    recalls = cumulative_tp / (len(gt_boxes) + 1e-6)\n    precisions = cumulative_tp / (cumulative_tp + cumulative_fp + 1e-6)\n\n    ap = 0.0\n    for i in range(1, len(precisions)):\n        ap += (recalls[i] - recalls[i - 1]) * precisions[i]\n\n    return ap\n","metadata":{"id":"Zenes2wtCOpb","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:40:51.985088Z","iopub.execute_input":"2025-03-26T23:40:51.985365Z","iopub.status.idle":"2025-03-26T23:40:51.991970Z","shell.execute_reply.started":"2025-03-26T23:40:51.985342Z","shell.execute_reply":"2025-03-26T23:40:51.991033Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"`calculate_map` This function calculates the mean Average Precision (mAP) across a number of IoU thresholds (0.5 to 0.95 in increments of 0.05). It first calculates the AP for every threshold and then averages them to get the mAP, which is an overall measure of how well the model does at different detection accuracies.","metadata":{"id":"BomDbATpHE0Z"}},{"cell_type":"code","source":"def calculate_map(pred_boxes_all, gt_boxes_all, iou_thresholds=np.arange(0.5, 1.0, 0.05)):\n    \"\"\"Calculate mAP across multiple IoU thresholds.\"\"\"\n    aps = []\n\n    for iou_thresh in iou_thresholds:\n        ap_per_image = []\n        for preds, gts in zip(pred_boxes_all, gt_boxes_all):\n            ap = calculate_ap_at_iou(preds, gts, iou_thresh)\n            ap_per_image.append(ap)\n\n        mean_ap = np.mean(ap_per_image)\n        print(f\"AP @ IoU {iou_thresh:.2f}: {mean_ap:.4f}\")\n        aps.append(mean_ap)\n\n    map_value = np.mean(aps)\n    print(f\"mAP@[0.5:0.95]: {map_value:.4f}\")\n\n    return map_value\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:40:56.232059Z","iopub.execute_input":"2025-03-26T23:40:56.232336Z","iopub.status.idle":"2025-03-26T23:40:56.237216Z","shell.execute_reply.started":"2025-03-26T23:40:56.232313Z","shell.execute_reply":"2025-03-26T23:40:56.236342Z"},"id":"X6SKJCa7HE0Z"},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model.eval()  # switch to eval mode\n\npred_boxes_all = []\ngt_boxes_all = []\n\nwith torch.no_grad():\n    for images, targets, image_ids in valid_data_loader:\n        images = list(img.to(device) for img in images)\n        outputs = model(images)\n\n        # Collect predictions and ground truth for each image\n        for i in range(len(images)):\n            preds = outputs[i]\n            scores = preds['scores'].cpu().numpy()\n            boxes = preds['boxes'].cpu().numpy()\n\n            # Combine boxes and scores for prediction list\n            pred_boxes = []\n            for box, score in zip(boxes, scores):\n                pred_boxes.append([box[0], box[1], box[2], box[3], score])\n\n            # Get ground truth boxes\n            gt_boxes = targets[i]['boxes'].cpu().numpy().tolist()\n\n            pred_boxes_all.append(pred_boxes)\n            gt_boxes_all.append(gt_boxes)\n\n# Calculate mAP\nmap_score = calculate_map(pred_boxes_all, gt_boxes_all)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:40:56.944366Z","iopub.execute_input":"2025-03-26T23:40:56.944638Z","iopub.status.idle":"2025-03-26T23:45:19.816685Z","shell.execute_reply.started":"2025-03-26T23:40:56.944615Z","shell.execute_reply":"2025-03-26T23:45:19.815744Z"},"id":"K0z1N1TEHE0Z","outputId":"44df7e22-2a51-4db4-bb9c-1ccfe67bce40"},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"The model performs well at a lower IoU threshold (e.g., `0.50`) and has high detection capability when the overlap is not excessive.\nAs the IoU threshold increases, the AP scores plummet dramatically, illustrating that the model is challenged to make highly accurate detections.\nThe mAP of `42.38`% provides a general measure of model performance, averaging precision and recall across various IoU thresholds. This suggests the model is good at object detection but may be able to do better at making more precise detections.","metadata":{"id":"KIiAGtCVHE0Z"}},{"cell_type":"markdown","source":"# 2.Yolo-V5","metadata":{"id":"fqiHC6gdTciG"}},{"cell_type":"code","source":"!git clone https://github.com/ultralytics/yolov5.git\n!cd yolov5\n!pip install -r /kaggle/working/yolov5/requirements.txt","metadata":{"id":"6t36NXGc1YX9","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:46:22.614199Z","iopub.execute_input":"2025-03-26T23:46:22.614536Z","iopub.status.idle":"2025-03-26T23:46:34.145349Z","shell.execute_reply.started":"2025-03-26T23:46:22.614504Z","shell.execute_reply":"2025-03-26T23:46:34.144430Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"outputId":"c3665c74-3cf3-41a9-90c0-bea463a5696c"},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Method","metadata":{"id":"UOiVSZJ6Z6Fi"}},{"cell_type":"markdown","source":"## 2.2 Spliting the data in Training and Testing\n\nThis code is essential for preparing the dataset for training a model like YOLOv5 Specifically:\n\n`Folder Structure Requirement`: YOLOv5 needs data in a specific folder structure:\nImages must be in their respective train and val directories.\nThe labels must be in their respective directories (though labels are not handled in this code snippet).\n\n`Data Splitting`: Model performance is evaluated on a validation set it has never seen during training. This makes it more certain that the model has learned to generalize to new data, rather than just memorize the training set (overfitting).\n\n`Reproducibility`: Splitting the dataset and organizing the files in a chronological way makes it easier to train the model multiple times, in addition to recording the results.\n\n`YOLOv5 preprocessing`: YOLOv5 requires input data in a highly specific format. This script structures the directories of the dataset, which will later be used for creating the appropriate label files for each image.","metadata":{"id":"gSqJIqz2krzU"}},{"cell_type":"code","source":"import os\nfrom tqdm import tqdm\nimport shutil\nfrom pathlib import Path\nfrom sklearn.model_selection import train_test_split\n\n# Define the source directory of images\nimages_dir = Path('/kaggle/input/global-wheat-detection/train/')\n\n# Collect all image files\nimage_files = sorted(list(images_dir.glob('*.jpg')))\n\n# Split into 80% train, 20% validation\ntrain_images, val_images = train_test_split(image_files, test_size=0.2, random_state=42)\n\n\n# Define target folder structure for YOLOv5\nbase_dir = Path('/kaggle/working/wheat-dataset/')\nimages_train_dir = base_dir / 'images/train'\nimages_val_dir = base_dir / 'images/val'\nlabels_train_dir = base_dir / 'labels/train'\nlabels_val_dir = base_dir / 'labels/val'\n\n# Create folders\nfor dirs in [images_train_dir, images_val_dir, labels_train_dir, labels_val_dir]:\n    dirs.mkdir(parents=True, exist_ok=True)\n\nfor train_image_path, valid_image_path in zip(train_images, val_images):\n    # print(train_image_path, images_train_dir / train_image_path.name)\n    # print(train_image_path.name)   # image.jpg\n\n    # training images\n    shutil.copy(src=train_image_path, dst=images_train_dir / train_image_path.name)\n\n    # # validation images\n    shutil.copy(src=valid_image_path, dst=images_val_dir / valid_image_path.name)\n\nprint(f\"Training Set: {len(train_images)}\\nValidation set: {len(val_images)}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:46:43.512488Z","iopub.execute_input":"2025-03-26T23:46:43.512827Z","iopub.status.idle":"2025-03-26T23:46:48.476694Z","shell.execute_reply.started":"2025-03-26T23:46:43.512796Z","shell.execute_reply":"2025-03-26T23:46:48.475995Z"},"id":"OpXhz4ZKkrzV","outputId":"9a7dc96f-c0f3-466a-ce67-00a9863dfa39"},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 2.3 Creating Labels for Yolo Model\nThis script is necessary to transform the raw annotation data (i.e., bounding boxes) into a format that can be used to train a YOLOv5 model. YOLO requires annotations in a certain format:\n\n- A text file with the same name as the image, one line per object in the image.\n\n- Each line in the file specifies the class ID and the normalized coordinates of the bounding box in the image.\n\nBy converting the annotations to this form, this script makes sure that training YOLOv5 (or any YOLO) can go on as usual. Without the conversion, the model would not be able to learn from the images and interpret the annotations, making this an essential component of the dataset preprocessing pipeline.","metadata":{"id":"0AGWk2iFkrzV"}},{"cell_type":"code","source":"from tqdm import tqdm\nfrom PIL import Image\nimport pandas as pd\n\n\ncsv_file_path = '/kaggle/input/global-wheat-detection/train.csv'  # Contains bounding box coordinates\ndf = pd.read_csv(csv_file_path)\n\n# Function to create YOLO label files from DataFrame group\ndef create_labels(image_list, label_output_dir, image_output_dir):\n    for img_path in tqdm(image_list):\n        image_id = img_path.stem  # image filename without extension\n        img = Image.open(img_path)\n        w, h = img.size\n\n        # Group annotations for this image_id\n        group = df[df['image_id'] == image_id]\n\n        # Skip if there are no annotations\n        if group.shape[0] == 0:\n            continue\n\n        # Label file path\n        label_file = label_output_dir / f\"{image_id}.txt\"\n\n        with open(label_file, 'w') as f:\n            for _, row in group.iterrows():\n                bbox = eval(row['bbox'])  # format [x_min, y_min, width, height]\n\n                # Convert to YOLO format (normalized)\n                x_center = (bbox[0] + bbox[2] / 2) / w\n                y_center = (bbox[1] + bbox[3] / 2) / h\n                width = bbox[2] / w\n                height = bbox[3] / h\n\n                # Only 1 class: wheat (class 0)\n                f.write(f\"0 {x_center} {y_center} {width} {height}\\n\")\n\n# Generate labels for train\ncreate_labels(train_images, labels_train_dir, images_train_dir)\n\n# Generate labels for val\ncreate_labels(val_images, labels_val_dir, images_val_dir)\n\nprint(\"Images and label files are ready!\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:48:44.682367Z","iopub.execute_input":"2025-03-26T23:48:44.682672Z","iopub.status.idle":"2025-03-26T23:49:42.667941Z","shell.execute_reply.started":"2025-03-26T23:48:44.682647Z","shell.execute_reply":"2025-03-26T23:49:42.666812Z"},"id":"bXMdsUQFkrzW","outputId":"ffe3edb0-a236-4584-d7c3-58c576f9021d"},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 2.4 create YAML file\n\nThe YAML file needs to be mentioned by YOLOv5 for specifying the dataset configuration. The YAML file provides the vital information that is utilized by YOLOv5 to:\n\nSpecify Dataset Paths: The YAML file tells YOLOv5 where training and validation images are stored, enabling it to load the data accordingly.\n\n`Define Classes`: The quantity of classes (`nc`) and their names (`names`) are specified in the YAML file. This is done so that YOLOv5 understands what it's trying to detect. In this case, it will only detect the class wheat.\n\n`Ease of Model Training`: YOLOv5 looks for the YAML file when training. It accepts the paths to images and class definitions in order to correctly train the model. Without the YAML config file, YOLOv5 would have no clue where the dataset is located or how many classes it is to detect.\n\nthe YAML file is a configuration file that bridges the data structure of the dataset with the model's training process, allowing YOLOv5 to comprehend how to read and access the data to be trained on.","metadata":{"id":"qk1D5S94krzW"}},{"cell_type":"code","source":"# Create a YAML file for YOLOv5 (path: data.yaml)\nyaml_content = \"\"\"\npath: /kaggle/input/global-wheat-detection  # path to the root of the dataset\ntrain: /kaggle/working/wheat-dataset/images/train  # images for training\nval: /kaggle/working/wheat-dataset/images/val  # images for validation\nnc: 1  # number of classes\nnames: ['wheat']  # class names\n\"\"\"\n\n# Save the YAML configuration file\nwith open('/kaggle/working/data.yaml', 'w') as f:\n    f.write(yaml_content)","metadata":{"id":"0N_PVo5sZYY2","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:49:49.312144Z","iopub.execute_input":"2025-03-26T23:49:49.312428Z","iopub.status.idle":"2025-03-26T23:49:49.316869Z","shell.execute_reply.started":"2025-03-26T23:49:49.312405Z","shell.execute_reply":"2025-03-26T23:49:49.316169Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 2.5 Model Selection and Setup\nTo determine the best models for this task, I conducted a total of four  experiments with YOLOv5 variants.\n\n- `YOLOv5s6`: Smallest model, prioritizing speed.\n- `YOLOv5m6`: Medium model, balancing speed and accuracy.\n- `YOLOv5l6`: Larger model for higher accuracy.\n- `YOLOv5x6`: Largest model for maximum performance.\n\n`YOLOv5m6`: Trained for 10 epochs using the train.py script (python train.py --img 640 --batch 16 --epochs 10 --data wheat.yaml --weights yolov5m6.pt), leveraging more epochs for convergence.","metadata":{"id":"4LmAuIwqkrzX"}},{"cell_type":"code","source":"# W&B Login\nimport wandb\n\n# tool used for tracking experiments, logging metrics, visualizations, and collaborating on machine learning projects\n# os.environ['WANDB_API_KEY'] = 'wandb_API_Key'\nos.environ['WANDB_API_KEY'] = '1233c12c43cf272a78b6d84dc7f19c0546f0e48d'\n\n# start training\n!python /kaggle/working/yolov5/train.py --img 640 --batch 16 --epochs 10 --data /kaggle/working/data.yaml --weights yolov5m6.pt --cache","metadata":{"id":"TYnjT4oA5Z3y","outputId":"ef3de388-261f-485b-f07f-2d77f02ac296","trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:50:27.816548Z","iopub.execute_input":"2025-03-26T23:50:27.816923Z","iopub.status.idle":"2025-03-26T23:58:04.549981Z","shell.execute_reply.started":"2025-03-26T23:50:27.816889Z","shell.execute_reply":"2025-03-26T23:58:04.548938Z"},"collapsed":true,"jupyter":{"outputs_hidden":true}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"`YOLOv5m6`’s `mAP@[0.5]` improved from `0.0665` to `0.9162` over `10` epochs, with `mAP@[0.5:0.95]` reaching `0.4689`.","metadata":{"id":"9Tym20dge0v8"}},{"cell_type":"markdown","source":"## 2.6 Predictions\nThis script is applied to perform inference (object detection) on an inference-trained YOLOv5 model against test images. The script takes in a trained model and a test dataset, performs object detection, and writes output in an appropriate format for later analysis. This process is paramount to check how efficient the trained model is and how far it generalizes into new unseen data.","metadata":{"id":"RklvR_VfkrzX"}},{"cell_type":"code","source":"from yolov5 import detect\nimport torch\n\nmodel_path = \"/kaggle/working/yolov5/runs/train/exp/weights/best.pt\"\ntest_images = \"/kaggle/input/global-wheat-detection/test/\"\n\n# Run inference\n!python /kaggle/working/yolov5/detect.py --weights {model_path} --source {test_images} --img 640 --conf 0.25 --save-txt --save-conf","metadata":{"trusted":true,"id":"QEVMpAfokrzX","execution":{"iopub.status.busy":"2025-03-26T23:58:50.225337Z","iopub.execute_input":"2025-03-26T23:58:50.225730Z","iopub.status.idle":"2025-03-26T23:58:59.834033Z","shell.execute_reply.started":"2025-03-26T23:58:50.225697Z","shell.execute_reply":"2025-03-26T23:58:59.832671Z"},"jupyter":{"outputs_hidden":true},"outputId":"2d53bf16-9bc5-4ba3-e610-4124ce840dac","collapsed":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### 2.7 Plot predictions on images","metadata":{"id":"3iRe5Me6krzY"}},{"cell_type":"code","source":"import os\nimport matplotlib.pyplot as plt\nfrom PIL import Image\n\npredicted_imgs = \"/kaggle/working/yolov5/runs/detect/exp\"\n\n# Get all image files in the folder\nimages = [img for img in os.listdir(predicted_imgs) if img.endswith(('.jpg', '.png', '.jpeg'))]\n\n# Number of images you want to show per row\ncols = 3\n# Calculate the required number of rows based on the number of images\nrows = (len(images) + cols - 1) // cols\n\n# Create a figure with subplots\nplt.figure(figsize=(12, rows * 5))\n\n# Loop through the images and create subplots\nfor idx, img_name in enumerate(images):\n    img_path = os.path.join(predicted_imgs, img_name)\n    img = Image.open(img_path)\n\n    # Add subplot for each image\n    plt.subplot(rows, cols, idx + 1)\n    plt.imshow(img)\n    plt.axis('off')\n    plt.title(f'Prediction: {os.path.basename(img_path)}')\n\n# Adjust layout and show the figure\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-26T23:59:03.406323Z","iopub.execute_input":"2025-03-26T23:59:03.406691Z","iopub.status.idle":"2025-03-26T23:59:06.099767Z","shell.execute_reply.started":"2025-03-26T23:59:03.406658Z","shell.execute_reply":"2025-03-26T23:59:06.098636Z"},"id":"j_2H43IakrzY","outputId":"5bf0be66-0bda-417d-db48-366f164d7288"},"outputs":[],"execution_count":null},{"cell_type":"code","source":"result = pd.read_csv(\"/kaggle/working/yolov5/runs/train/exp/results.csv\")\ndisplay(result.iloc[:, 6:8])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-03-27T00:23:53.433715Z","iopub.execute_input":"2025-03-27T00:23:53.434079Z","iopub.status.idle":"2025-03-27T00:23:53.444832Z","shell.execute_reply.started":"2025-03-27T00:23:53.434050Z","shell.execute_reply":"2025-03-27T00:23:53.443945Z"},"id":"Pba9OYPMGuno","outputId":"5779edef-a8bd-4980-97b1-778139edd652"},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Performance Comparison Table\n\n#### YOLOv5 Models (10 Epochs)\n\n$$\n\\begin{array}{|l|c|c|c|c|}\n\\hline\n\\textbf{Model} & \\textbf{Epoch} & \\textbf{mAP@0.5} & \\textbf{mAP@[0.5:0.95]} & \\textbf{Best mAP@[0.5:0.95]} \\\\\n\\hline\n\\text{YOLOv5s6} & 1 & 0.0596 & 0.0159 & 0.44715 \\ (\\text{Epoch 10}) \\\\\n& 5 & 0.5491 & 0.2168 & \\\\\n& 10 & 0.8991 & 0.4472 & \\\\\n\\hline\n\\text{YOLOv5m6} & 1 & 0.0665 & 0.0178 & 0.46889 \\ (\\text{Epoch 10}) \\\\\n& 5 & 0.5940 & 0.2453 & \\\\\n& 10 & 0.9162 & 0.4689 & \\\\\n\\hline\n\\text{YOLOv5l6} & 1 & 0.1074 & 0.0278 & 0.49009 \\ (\\text{Epoch 10}) \\\\\n& 5 & 0.6068 & 0.2722 & \\\\\n& 10 & 0.9246 & 0.4901 & \\\\\n\\hline\n\\text{YOLOv5x6} & 1 & 0.1192 & 0.0335 & 0.50243 \\ (\\text{Epoch 10}) \\\\\n& 5 & 0.6537 & 0.3004 & \\\\\n& 10 & 0.9263 & 0.5024 & \\\\\n\\hline\n\\end{array}\n$$\n\n#### Faster R-CNN Models (5 Epochs)\n$$\n\\begin{array}{|l|c|c|c|c|}\n\\hline\n\\textbf{Model} & \\textbf{mAP@0.5} & \\textbf{mAP@[0.5:0.95]} & \\textbf{Training Loss (Avg)} & \\textbf{Validation Loss (Avg)} \\\\\n\\hline\n\\texttt{fasterrcnn\\_resnet50\\_fpn} & 0.8452 & 0.4384 & 0.8115 & 0.8357 \\\\\n\\texttt{fasterrcnn\\_resnet50\\_fpn\\_v2} & 0.7225 & 0.3750 & 0.7306 & 0.8973 \\\\\n\\texttt{fasterrcnn\\_mobilenet\\_v3\\_large\\_fpn} & 0.8300 & 0.3924 & 0.9697 & 0.9452 \\\\\n\\texttt{fasterrcnn\\_mobilenet\\_v3\\_large\\_320\\_fpn} & 0.3805 & 0.1194 & 1.3104 & 1.2469 \\\\\n\\hline\n\\end{array}\n$$","metadata":{"id":"9n9bImuAIAO3"}},{"cell_type":"markdown","source":"## Explanation of Experiments and Selection\nThe eight experiments allowed me to explore a range of architectural trade-offs. Faster R-CNN with ResNet-50 FPN stood out for its balanced accuracy and robustness, avoiding the lower mAP of lightweight MobileNet variants, which struggled with the dataset’s complexity (e.g., small, overlapping wheat heads). YOLOv5m6 was chosen over other YOLO variants for its efficiency and strong mAP, making it a practical complement to Faster R-CNN for comparison, aligning with the assignment’s requirement to evaluate two models comprehensively.","metadata":{"id":"H09w2REIgaVd"}},{"cell_type":"markdown","source":"# Results and Discussion\nThe performance of `Faster R-CNN` and `YOLOv5` models was quantified over various epochs, with `mAP@0.5` and `mAP@0.5:0.95` being notable metrics. The highest-performing YOLOv5 among the ones tested was `YOLOv5x6`, which achieved an end `mAP@0.5` of `0.92633` and `mAP@0.5:0.95` of `0.50243`, indicating high performance in object localization accuracy and object detection. The performance of `YOLOv5l6` was also comparable, with an `mAP@0.5` of `0.92464` and `mAP@0.5:0.95` of `0.49009`. The `YOLOv5m6` model, ultimately selected, also worked well, with `mAP@0.5` of `0.91615` and `mAP@0.5:0.95` of `0.46889`, and was therefore a perfect choice for balancing accuracy, efficiency, and computability. `YOLOv5s6`, the lightest, had the poorest performance, with the ultimate `mAP@0.5` of `0.89908` and `mAP@0.5:0.95` of `0.44715`, though it also demonstrated remarkable improvement with training epochs.\n\nYet the `Faster R-CNN` models had mixed performance characteristics. A quicker `R-CNN ResNet50 FPN` provided an `mAP@[0.5:0.95]` of `0.4384`, an `mAP@0.5 of 0.8452`, which evidently reduces with higher IoU threshold values, where precision could not be maintained. `Faster R-CNN ResNet50 FPN V2` provided slightly lower accuracy overall with `mAP@[0.5:0.95]` as `0.3750` and an `mAP@0.5` as `0.7225`, and a decline was noticed in the localization accuracy. Faster `R-CNN MobileNetV3 Large FPN` performed slightly better with a `mAP@[0.5:0.95] of 0.3924`, but still lagged behind `YOLOv5` models in detection accuracy. The worst-performing model was `Faster R-CNN MobileNetV3 Large 320 FPN`, with a `mAP@[0.5:0.95]` of `0.11941`, which only reflected its poor performance in precision and object localization.\n\n`YOLOv5` models significantly outperformed the Faster R-CNN models in detection accuracy, particularly at lower IoU thresholds. This means YOLOv5 is better suited for object detection tasks where there is the need for real-time inference as well as maximum detection accuracy. The Faster R-CNN models, though were able to compete in low IoU values, lost significantly at increased thresholds regarding accuracy. The decision to utilize `YOLOv5m6` as the best model in this task is made based on performance versus computation costs trade-off.","metadata":{"id":"BdF0r_1_IH_S"}},{"cell_type":"markdown","source":"# Comparison Table\nThe table below summarizes the key performance metrics for both models, with YOLOv5m6’s AP@0.75 and AP@0.95 approximated based on trends from Faster R-CNN and the mAP@[0.5:0.95] value.\n\n$$\n\\begin{array}{|l|c|c|c|c|c|c|}\n\\hline\n\\textbf{Model} & \\textbf{mAP@[0.5:0.95]} & \\textbf{AP@0.5} & \\textbf{AP@0.75} & \\textbf{AP@0.95} & \\textbf{Epochs} & \\textbf{Batch Size} \\\\\n\\hline\n\\text{Faster R-CNN (ResNet-50 FPN)} & 0.4384 & 0.8452 & 0.4139 & 0.0006 & 5 & 8 \\\\\n\\hline\n\\text{YOLOv5m6} & 0.4689 & 0.9162 & 0.3583^* & 0.0007^* & 10 & 16 \\\\\n\\hline\n\\end{array}\n$$\n\n**Note**: YOLOv5 AP at specific IoUs (0.75, 0.95) approximated from Faster R-CNN trends due to limited granularity.\n","metadata":{"id":"v6r9p5vjfomL"}},{"cell_type":"markdown","source":"### Performance Comparison\nYOLOv5m6 outperformed Faster R-CNN in overall `mAP@[0.5:0.95] (0.4689 vs. 0.4384`), driven by its superior recall at `IoU 0.5 (0.9162 vs. 0.8452)`. This reflects YOLOv5's single-stage architecture, which prioritizes detecting all objects quickly, leveraging its anchor-based predictions and efficient backbone [Ref 2]. Faster R-CNN did, however, have higher accuracy at mid-level IoUs (e.g., `0.4139` compared to ~`0.3583` at `IoU 0.75`) from its two-stage approach where region proposal network (RPN) refines proposal before classification [Ref 1]. The two models were woefully inaccurate at IoU 0.95 (0.0006 by Faster R-CNN, ~0.0007 by YOLOv5m6), indicating that there are weaknesses in pixel-accurate bounding box alignment.\n\n### What Worked\n`Faster R-CNN`: The ResNet-50 FPN backbone worked effectively in extracting features of different scales, thus its effective result on moderate IoUs. Two-stage method provided effective precision on isolated wheat heads where correct bounding box proposals were able to be done.\n`YOLOv5m6`: More training (10 epochs compared to 5) and a larger batch size (16 compared to 8) assisted YOLOv5m6 in converging more, boosting its mAP. Its pre-trained weights and lightweight design assisted it in adapting rapidly to the wheat dataset, performing well especially in recall for densely packed regions.\n\n###What Didn't Work\n`Faster R-CNN`: The model was not good at dealing with overlapping or small wheat heads, with the RPN sometimes not being able to propose regions for highly packed objects and resulting in missed detections. The limited 5 epochs and small batch size (due to memory constraints) could have prevented perfect convergence, seen in the plateauing of the validation loss.\n\n`YOLOv5m6`: While strong on recall, over-detecting in crowded areas, finding more than one box per wheat head, which likely watered down precision for bigger IoUs. Lack of per-IoU granularity from the logged results limits fine-grained examination of this shortfall.\n\n### Why These Results?\n\nDataset complexity—densely packed wheat heads (`usually <50 pixels`) and frequent overlaps—posed difficulties for both models. Faster R-CNN's region proposal usage made it object density-sensitive, since the RPN is sensitive to small, overlapping targets [Ref 1]. YOLOv5m6's anchor mechanism, while being faster and recall-biased, might have misaligned anchors for the dataset's scale variability, impacting precision [Ref 2]. The multi-scale FPN in Faster R-CNN and the SPPF (Spatial Pyramid Pooling-Fast) in YOLOv5 were key features impacting performance, with YOLOv5m6's longer training further adding to its advantage.\n\n### Failure Cases\n\nBoth failed in dense scenes. Faster R-CNN did not detect objects if proposals were too overlapping, while `YOLOv5m6` over predicted and generated false positives. With `IoU 0.95`, small object size and annotation noise (e.g., slight misalignments in ground truth boxes) likely made these issues worse, since even minor shifts had a significant effect on AP.\n\n### Additional Analysis\n\nThe `mAP@[0.5:0.95]` gap of `YOLOv5m6` (`0.0305`) suggests broader robustness of YOLOv5m6, maybe due to training time and efficiency in data utilization. Additional epochs or a higher batch size would improve Faster R-CNN, but computational requirements dictated otherwise. YOLOv5m6 fluctuations in mAP@[0.5:0.95] (i.e., drop at epoch 3) indicate learning rate sensitivity or data augmentation effect, thus further hyperparameter tuning is deserved.","metadata":{"id":"daiymT07f7QI"}},{"cell_type":"markdown","source":"# Summary and Recommendation\n\nCompared to Faster R-CNN and YOLOv5 for object detection on the global-wheat dataset, YOLOv5 outperformed Faster R-CNN in mAP and efficiency. YOLOv5x6 had the highest `mAP@[0.5:0.95]` (`0.5024`), followed by `YOLOv5l6` (`0.4901`), while `YOLOv5m6` offered a moderate balance between accuracy (`0.4689 mAP`) and computational cost. Faster R-CNN (`ResNet-50 FPN`) came in second with a `mAP` of `0.4384`, larger validation losses, and a slower rate of convergence, and thus not suitable for real-time applications. For this activity, **YOLOv5m6** is the most suitable, with an optimal balance of speed and accuracy, while **YOLOv5x6** needs to be used if higher accuracy is a requirement and the resources are available. Faster R-CNN remains an option for high-resolution, accuracy-based tasks where speed is less important. ### **Benefits and Limitations of YOLOv5:**\n\n**Pros:**\n- **Enhanced mAP scores**  Outperformed Faster R-CNN at all IoU levels.\n- **Faster inference**  Made for real-time applications.\n- **Best utilization of resources**  Models like YOLOv5m6 achieve a trade-off between speed and accuracy.\n\n**cons:**\n- **Higher memory demands**  Large models (e.g., YOLOv5x6) are highly resource-intensive.\n- **Complex training configuration**  Requires significant hyperparameter tuning and augmentation.\n\n### **Faster R-CNN: Advantages and Disadvantages:**\n**pros:**\n- **Robust feature extraction**  Region proposal approach improves localization.\n- **Consistent for high-res images**  More suitable for accuracy at the cost of speed.\n\n**cons:**\n- **Slower inference**  Two-stage detection limits real-time applications.\n- **Reduced mAP at high IoU**  With stricter localization requirements, performance decreases.\n\n**Final Recommendation:**\nFor everyone except high-accuracy needs, **YOLOv5m6** is the way to go, and for high-accuracy needs, **YOLOv5x6** is better suited. Resort to **Faster R-CNN** only when top-level detection has more priority than speed demands.","metadata":{"id":"9tBG_fbj9h_6"}},{"cell_type":"markdown","source":"## References\n- Ren, S., He, K., Girshick, R., & Sun, J. (2015, June 4). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv.org. https://arxiv.org/abs/1506.01497.\n\n- Jocher, G., et al. (2021). \"YOLOv5 by Ultralytics.\" GitHub Repository: https://github.com/ultralytics/yolov5.\n\n- Global Wheat Detection Dataset. Kaggle: https://www.kaggle.com/c/global-wheat-detection.","metadata":{"id":"8dp29WhQGunp"}}]}