{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Object Detection with PyTorch and DETR\n\n**Author:** Shingo Nakazawa ([@shnakazawa](https://twitter.com/shnakazawa))\n\n**Objective:** In this notebook, we will create a object detection model using [**Detection Transformer (DETR) architecture**](https://arxiv.org/abs/2005.12872) with [PyTorch](https://pytorch.org/). ([A reference article (in Japanese)](https://zenn.dev/aidemy/articles/30f10cf2c48e6a))\n\nDETR is a deep learning model based on the Transformer architecture used for object detection tasks. We will **download a pretrained model provided by the research team using `torch.hub.load()`**.\n\nPlease see also\n- [Image Classification with PyTorch and EfficientNetV2](https://www.kaggle.com/code/shnakazawa/image-classification-with-pytorch-and-efficientnet)\n- [Semantic Segmentation with PyTorch and U-Net](https://www.kaggle.com/code/shnakazawa/semantic-segmentation-with-pytorch-and-u-net)\n\n# Import Modules","metadata":{"papermill":{"duration":0.009476,"end_time":"2023-03-07T06:17:01.946165","exception":false,"start_time":"2023-03-07T06:17:01.936689","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport math\nimport time\nimport random\nimport gc\nfrom pathlib import Path\nimport cv2\nfrom tqdm.notebook import tqdm\nfrom sklearn.model_selection import KFold\n\n# Image augmentation\nimport albumentations as A\nfrom albumentations.pytorch import ToTensorV2\n\n# Modeling\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.data import Dataset, DataLoader\n\n# Visualization\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\n\nprint(f'PyTorch version {torch.__version__}')\nprint(f'Albumentations version {A.__version__}')","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:17:01.964203Z","iopub.status.busy":"2023-03-07T06:17:01.963511Z","iopub.status.idle":"2023-03-07T06:17:06.297585Z","shell.execute_reply":"2023-03-07T06:17:06.296425Z"},"papermill":{"duration":4.345744,"end_time":"2023-03-07T06:17:06.300153","exception":false,"start_time":"2023-03-07T06:17:01.954409","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Set Configs\n\nSeparatelly setting configs such as runnning objectives, file paths, hyperparameters, etc. is helpful in maintain a clear and organized workflow.","metadata":{"papermill":{"duration":0.007878,"end_time":"2023-03-07T06:17:06.316457","exception":false,"start_time":"2023-03-07T06:17:06.308579","status":"completed"},"tags":[]}},{"cell_type":"code","source":"RUN_EDA = True\nRUN_TRAINING = True\nTRAIN_ALL = False # If true, train with all data and output a single model. If False, run cross-validation and output multiple models.\nFOLD_NUM = 5 # For cross-validation\nEPOCHS = 3\nRUN_INFERENCE = False\n\n# Directory setting\nDETR_DIR = '/kaggle/working/detr'\nDATA_DIR = '/kaggle/input/global-wheat-detection/'\nMODEL_DIR = '/kaggle/working/'\nIMG_SAVE_DIR = '/kaggle/working/'\n\n# PyTorch variables\nSEED = 42\nNUM_CLASSES = 2 # Wheat + background = 2 classes\n# NUM_QUERIES = 100 # How many bboxes shall the model output per image. In this notebook, it is defined based on the num of bboxes in training data\nNUM_WORKERS = 2\nBATCH_SIZE = 16\nLR = 2e-5\n\nNULL_CLASS_COEF = 0.5\nGAMMA = 0.1\nTHRESHOLD = 0.3 # Threshold for bbox prediction score","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:17:06.335280Z","iopub.status.busy":"2023-03-07T06:17:06.333574Z","iopub.status.idle":"2023-03-07T06:17:06.341942Z","shell.execute_reply":"2023-03-07T06:17:06.340987Z"},"papermill":{"duration":0.019511,"end_time":"2023-03-07T06:17:06.344008","exception":false,"start_time":"2023-03-07T06:17:06.324497","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Import DETR Modules\n\nTo use DETR model and its unique loss function, close its repository. ","metadata":{"papermill":{"duration":0.008088,"end_time":"2023-03-07T06:17:06.360741","exception":false,"start_time":"2023-03-07T06:17:06.352653","status":"completed"},"tags":[]}},{"cell_type":"code","source":"if os.path.exists(DETR_DIR) == False:\n    !git clone https://github.com/facebookresearch/detr.git\n\nimport sys\nsys.path.append(DETR_DIR)\n\nfrom detr.models.matcher import HungarianMatcher\nfrom detr.models.detr import SetCriterion","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:17:06.378708Z","iopub.status.busy":"2023-03-07T06:17:06.377746Z","iopub.status.idle":"2023-03-07T06:17:09.541309Z","shell.execute_reply":"2023-03-07T06:17:09.540144Z"},"papermill":{"duration":3.175422,"end_time":"2023-03-07T06:17:09.544143","exception":false,"start_time":"2023-03-07T06:17:06.368721","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Define Helper Functions\n\nDefining reused functions at the top of a Jupyter Notebook can lead to cleaner, more organized, and more efficient code, which can be especially helpful for larger and more complex projects.","metadata":{"papermill":{"duration":0.008316,"end_time":"2023-03-07T06:17:09.561495","exception":false,"start_time":"2023-03-07T06:17:09.553179","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = True\n\n# Set seed\nseed_everything(SEED)\n\n\ndef show_gpu_memory(device):\n    print(f\"Allocated GPU memory: {torch.cuda.memory_allocated(device) / 1024 / 1024:.2f} MB\")\n    print(f\"Cached GPU memory: {torch.cuda.memory_cached(device) / 1024 / 1024:.2f} MB\")    \n\n    \ndef load_img(path):\n    img_bgr = cv2.imread(path)\n    img_rgb = img_bgr[:, :, ::-1]\n    return img_rgb\n\n\ndef group_bboxes(df):\n    df_ = df.copy()\n    df_['bbox_count'] = 1\n    df_ = df_.groupby(['image_id', 'source']).count().reset_index()\n    return_df = df_[['image_id', 'source', 'bbox_count']]\n    return return_df\n\n\ndef create_gallery(array, ncols=3):\n    \"\"\"Display multiple images in a gallery style.\n    Source: https://www.amazon.co.jp/Data-Analysis-Machine-Learning-Kaggle-ebook/dp/B09F3STL34/\n    \n    Args:\n        array (numpy.ndarray): array of images.\n        ncols (int, optional): Num of columns. Defaults to 3.\n\n    Returns:\n        numpy.ndarray: One concatenated image.\n    \"\"\"    \n    nindex, height, width, intensity = array.shape\n    nrows = nindex//ncols\n    assert nindex == nrows * ncols\n    result = (array.reshape(nrows, ncols, height, width, intensity)\n        .swapaxes(1,2)\n        .reshape(height*nrows, width*ncols, intensity))\n    return result\n\n\ndef normalize_bbox(bboxes, rows, cols):\n    \"\"\"Normalize coordinates of a bounding box. Divide x-coordinates by image width and y-coordinates\n    by image height.\n    Source: https://vfdev-5-albumentations.readthedocs.io/en/docs_pytorch_fix/_modules/albumentations/augmentations/bbox_utils.html\n    with minor modification\n    \"\"\"\n    normalized_bboxes = []\n    for bbox in bboxes:\n        x_min, y_min, x_max, y_max = bbox\n        normalized_bbox = (x_min / cols, y_min / rows, x_max / cols, y_max / rows)\n        normalized_bboxes.append(normalized_bbox)\n    return normalized_bboxes\n\n\ndef denormalize_bbox(bboxes, rows, cols):\n    \"\"\"Denormalize coordinates of a bounding box. Multiply x-coordinates by image width and y-coordinates\n    by image height. This is an inverse operation for :func:`~albumentations.augmentations.bbox.normalize_bbox`.\n    Source: https://vfdev-5-albumentations.readthedocs.io/en/docs_pytorch_fix/_modules/albumentations/augmentations/bbox_utils.html\n    with minor modification\n    \"\"\"\n    denormalized_bboxes = []\n    for bbox in bboxes:\n        x_min, y_min, x_max, y_max = bbox\n        denormalized_bbox = (x_min * cols, y_min * rows, x_max * cols, y_max * rows)\n        denormalized_bboxes.append(denormalized_bbox)\n    return denormalized_bboxes\n\n\ndef show_validation_score(train_loss_list, valid_loss_list, save=False, save_dir=IMG_SAVE_DIR, save_name='objectdetection_validation_score.png'):\n    fig = plt.figure(figsize=(10,10))\n    for i in range(FOLD_NUM):\n        train_loss = train_loss_list[i]\n        valid_loss = valid_loss_list[i]\n        \n        ax = fig.add_subplot(math.ceil(np.sqrt(FOLD_NUM)), math.ceil(np.sqrt(FOLD_NUM)), i+1, title=f'Fold {i+1}')\n        ax.plot(range(EPOCHS), train_loss, c='orange', label='train')\n        ax.plot(range(EPOCHS), valid_loss, c='blue', label='valid')\n        ax.set_xlabel('epoch')\n        ax.set_ylabel('loss')\n        ax.legend()\n    \n    plt.tight_layout()\n    if save:\n        os.makedirs(save_dir, exist_ok=True)\n        plt.savefig(save_dir+save_name)\n    else:\n        plt.show()","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:17:09.580787Z","iopub.status.busy":"2023-03-07T06:17:09.580469Z","iopub.status.idle":"2023-03-07T06:17:09.599628Z","shell.execute_reply":"2023-03-07T06:17:09.598626Z"},"papermill":{"duration":0.031525,"end_time":"2023-03-07T06:17:09.601779","exception":false,"start_time":"2023-03-07T06:17:09.570254","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load and Reshape a Table\n\nDifferent formats are used to annotate bounding boxes for object detection tasks, depending on the model. The most common formats are `coco`, `voc-pascal`, and `yolo`, which encode the coordinates in the following ways:\n\n- `coco`: `[x, y, width, height]`, where `x` and `y` represent the upper-left coordinate of the rectangle.\n- `voc-pascal`: `[x1, y1, x2, y2]`, where `x1` and `y1` represent the upper-left coordinate of the rectangle, and `x2` and `y2` represent the lower-right coordinate.\n- `yolo`: `[x, y, width, height]`, where `x` and `y` represent the center of the rectangle.\n\nIn the present dataset, bounding box coordinates are given in the coco format, which is used by the DETR model. To prepare the data for training, we load and reshape the table to match the expected format of the model.","metadata":{"papermill":{"duration":0.008444,"end_time":"2023-03-07T06:17:09.619044","exception":false,"start_time":"2023-03-07T06:17:09.610600","status":"completed"},"tags":[]}},{"cell_type":"code","source":"df = pd.read_csv(DATA_DIR + 'train.csv')\ndf.head()","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:17:09.637886Z","iopub.status.busy":"2023-03-07T06:17:09.637583Z","iopub.status.idle":"2023-03-07T06:17:09.894271Z","shell.execute_reply":"2023-03-07T06:17:09.893126Z"},"papermill":{"duration":0.26884,"end_time":"2023-03-07T06:17:09.896549","exception":false,"start_time":"2023-03-07T06:17:09.627709","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As you can see, the current dataset has **one bounding box per row** and x, y, width, and height of the bounding box are represented in the single column using list-style formatting.\n\nTo better work with the data, we need to **split the values of the bounding box into separate columns**.","metadata":{"papermill":{"duration":0.008777,"end_time":"2023-03-07T06:17:09.914978","exception":false,"start_time":"2023-03-07T06:17:09.906201","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# ref: https://www.kaggle.com/code/tanulsingh077/end-to-end-object-detection-with-transformers-detr#Wheat-Detection-Competition-With-DETR\nmarking = df.copy()\nbboxs = np.stack(marking['bbox'].apply(lambda x: np.fromstring(x[1:-1], sep=',')))\nfor i, column in enumerate(['x', 'y', 'w', 'h']):\n    marking[column] = bboxs[:,i]\nmarking.drop(columns=['bbox'], inplace=True)\nmarking.head()","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:17:09.934207Z","iopub.status.busy":"2023-03-07T06:17:09.933879Z","iopub.status.idle":"2023-03-07T06:17:10.370136Z","shell.execute_reply":"2023-03-07T06:17:10.368874Z"},"papermill":{"duration":0.448874,"end_time":"2023-03-07T06:17:10.372786","exception":false,"start_time":"2023-03-07T06:17:09.923912","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next, let's restructure the data to have **one image per row**, with the number of bounding boxes in each image. This will make it easier to work with the data and feed it into the model.","metadata":{"papermill":{"duration":0.009017,"end_time":"2023-03-07T06:17:10.392369","exception":false,"start_time":"2023-03-07T06:17:10.383352","status":"completed"},"tags":[]}},{"cell_type":"code","source":"grouped_df = group_bboxes(marking)\nNUM_QUERIES = max(grouped_df['bbox_count']) # How many bboxes shall the model output per image.\ngrouped_df.head()","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:17:10.412551Z","iopub.status.busy":"2023-03-07T06:17:10.411555Z","iopub.status.idle":"2023-03-07T06:17:10.471476Z","shell.execute_reply":"2023-03-07T06:17:10.470149Z"},"papermill":{"duration":0.072438,"end_time":"2023-03-07T06:17:10.473883","exception":false,"start_time":"2023-03-07T06:17:10.401445","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Exploratory Data Analysis (EDA)\n\nEDA for images is typically simpler than for tabular data.\n\n**Please set `RUN_EDA = True` in the `Set Config` section.**\n\n## Inspect Representative Images\n\nVisually inspecting images is crucial. If the images are noisy or weird, preprocessing may be required before training a model.","metadata":{"papermill":{"duration":0.009348,"end_time":"2023-03-07T06:17:10.493822","exception":false,"start_time":"2023-03-07T06:17:10.484474","status":"completed"},"tags":[]}},{"cell_type":"code","source":"if RUN_EDA:\n    img_names = Path(DATA_DIR+'train/').glob('*.jpg')\n    img_list = []\n    for i, img_name in enumerate(img_names):\n        img_list.append(load_img(img_name.as_posix()))\n        print(img_name.name)\n        if i == 5: \n            break\n    plt.figure(figsize=(10,10))\n    plt.imshow(create_gallery(np.array(img_list), ncols=3))\nelse:\n    print('RUN_EDA is False')","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:17:10.514564Z","iopub.status.busy":"2023-03-07T06:17:10.514252Z","iopub.status.idle":"2023-03-07T06:17:12.641485Z","shell.execute_reply":"2023-03-07T06:17:12.640162Z"},"papermill":{"duration":2.155239,"end_time":"2023-03-07T06:17:12.658867","exception":false,"start_time":"2023-03-07T06:17:10.503628","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Check Image Shape\n\nIt is also important to check the size and color of the images in the dataset.\n\nCreating histograms can also be helpful.","metadata":{"papermill":{"duration":0.027014,"end_time":"2023-03-07T06:17:12.713888","exception":false,"start_time":"2023-03-07T06:17:12.686874","status":"completed"},"tags":[]}},{"cell_type":"code","source":"if RUN_EDA:\n    img_shape = set()\n    img_ext = set()\n    img_names = Path(DATA_DIR+'train/').glob('*')\n    pbar = tqdm(img_names, total=len(grouped_df))\n    for img_name in pbar:\n        img = load_img(img_name.as_posix())\n        img_shape.add(img.shape)\n        img_ext.add(img_name.suffix)\n    print(f'Image shapes are {img_shape}.')\n    print(f'Image extensions are {img_ext}.')","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:17:12.767324Z","iopub.status.busy":"2023-03-07T06:17:12.766245Z","iopub.status.idle":"2023-03-07T06:18:27.833415Z","shell.execute_reply":"2023-03-07T06:18:27.832284Z"},"papermill":{"duration":75.096806,"end_time":"2023-03-07T06:18:27.835770","exception":false,"start_time":"2023-03-07T06:17:12.738964","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We are confident that all images are 1024 x 1024 RGB jpg.","metadata":{"papermill":{"duration":0.024631,"end_time":"2023-03-07T06:18:27.885557","exception":false,"start_time":"2023-03-07T06:18:27.860926","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## Plot histogram of pixel values\n\nCreating a histogram of pixel values can help identify any outlier images (such as those with all zeros).","metadata":{"papermill":{"duration":0.023949,"end_time":"2023-03-07T06:18:27.934228","exception":false,"start_time":"2023-03-07T06:18:27.910279","status":"completed"},"tags":[]}},{"cell_type":"code","source":"if RUN_EDA:\n    img_names = Path(DATA_DIR+'train/').glob('*')\n    plt.figure(figsize=(10,10))\n    pbar = tqdm(img_names, total=len(df))\n    for img_name in pbar:\n        img = load_img(img_name.as_posix())\n        hist = cv2.calcHist([img],[0],None,[256],[0,256])\n        plt.plot(hist)\n    plt.show()\nelse:\n    print('RUN_EDA is False')","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:18:27.985251Z","iopub.status.busy":"2023-03-07T06:18:27.984900Z","iopub.status.idle":"2023-03-07T06:20:03.661772Z","shell.execute_reply":"2023-03-07T06:20:03.660884Z"},"papermill":{"duration":95.709108,"end_time":"2023-03-07T06:20:03.667773","exception":false,"start_time":"2023-03-07T06:18:27.958665","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The image signals in many images appear dark, while in some images they are saturated.\n\nIn this notebook, for sake of simplicity, we will proceed to the next section without any preprocessing.","metadata":{"papermill":{"duration":0.029962,"end_time":"2023-03-07T06:20:03.729605","exception":false,"start_time":"2023-03-07T06:20:03.699643","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# Define Model Components\n\nBefore training a model using PyTorch, the following steps need to be completed:\n\n1. Define Image Transformation and Augmentation\n2. Define the Dataset\n3. Define the DataLoader\n4. Define the Model\n\n## Define Image Transformation and Augmentation\n\nBefore creating the dataset, let's define the transformations and augmentations that will be applied to the images.\n\n**For object detection tasks, the bounding boxes also need to undergo transformations and augmentations.**\n\nTo transform the bounding boxes, you can refer to the following:\n- [GitHub/albumentations-team/albumentations#spatial-level-transforms](https://github.com/albumentations-team/albumentations#spatial-level-transforms)\n- [Albumentations Documentation/Bounding boxes augmentation for object detection](https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/#bounding-boxes-augmentation)\n    - How `min_area` and `min_visibility` work.","metadata":{"papermill":{"duration":0.028604,"end_time":"2023-03-07T06:20:03.787294","exception":false,"start_time":"2023-03-07T06:20:03.758690","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Image Transformation & Augmentation\ndef transform_train():\n    transforms = [\n        #A.RandomResizedCrop(512,512,p=1),\n        A.Resize(512, 512, p=1),\n        A.OneOf([\n            A.HueSaturationValue(hue_shift_limit=0.2, sat_shift_limit= 0.2, val_shift_limit=0.2, p=0.9),\n            A.RandomBrightnessContrast(brightness_limit=0.2, contrast_limit=0.2, p=0.9)\n        ], p=0.9),\n        A.ToGray(p=0.01),\n        A.HorizontalFlip(p=0.5),\n        A.VerticalFlip(p=0.5),\n        A.Cutout(num_holes=8, max_h_size=64, max_w_size=64, fill_value=0, p=0.5),   \n        ToTensorV2(p=1)\n    ]\n    bbox_params = A.BboxParams(format='coco', min_area=0, min_visibility=0, label_fields=['labels'])\n    return A.Compose(transforms, bbox_params, p=1)\n\n\n# Validation images undergo only resizing.\ndef transform_valid():\n    transforms = [\n        A.Resize(512,512,p=1),\n        ToTensorV2(p=1)\n    ]\n    bbox_params = A.BboxParams(format='coco', min_area=0, min_visibility=0,label_fields=['labels'])\n    return A.Compose(transforms, bbox_params, p=1)","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:20:03.846820Z","iopub.status.busy":"2023-03-07T06:20:03.846443Z","iopub.status.idle":"2023-03-07T06:20:03.855245Z","shell.execute_reply":"2023-03-07T06:20:03.854137Z"},"papermill":{"duration":0.04187,"end_time":"2023-03-07T06:20:03.857678","exception":false,"start_time":"2023-03-07T06:20:03.815808","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Define the Dataset","metadata":{"papermill":{"duration":0.028865,"end_time":"2023-03-07T06:20:03.915410","exception":false,"start_time":"2023-03-07T06:20:03.886545","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# Dataset\nclass WheatDataset(Dataset):\n    def __init__(self, image_ids, dataframe, data_root, transforms=None):\n        super().__init__()\n        self.image_ids = image_ids\n        self.dataframe = dataframe\n        self.data_root = data_root\n        self.transforms = transforms\n\n    def __len__(self):\n        return self.image_ids.shape[0]\n    \n    def __getitem__(self, index):\n        image_id = self.image_ids[index]\n        records = self.dataframe[self.dataframe['image_id'] == image_id]\n        # Load images\n        image  = load_img(f'{self.data_root}{image_id}.jpg').astype(np.float32)\n        image /= 255.0 # normalization\n\n        # bbox and area\n        boxes = records[['x', 'y', 'w', 'h']].to_numpy()\n        area = boxes[:,2] * boxes[:,3]\n        area = torch.as_tensor(area, dtype=torch.float32)\n\n        labels = np.zeros(len(boxes), dtype=np.int32)\n\n        # Transform images\n        if self.transforms:\n            sample = {\n                'image': image,\n                'bboxes': boxes,\n                'labels': labels\n            }\n            sample = self.transforms(**sample)\n            image = sample['image']\n            boxes = sample['bboxes']\n            labels = sample['labels']\n        \n        # Normalizing bboxes\n        _,h,w = image.shape\n        boxes = normalize_bbox(sample['bboxes'],rows=h,cols=w)\n        \n        target = {}\n        target['boxes'] = torch.as_tensor(boxes, dtype=torch.float32)\n        target['labels'] = torch.as_tensor(labels, dtype=torch.long)\n        target['image_id'] = torch.tensor([index])\n        target['area'] = area\n\n        return image, target, image_id","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:20:03.975766Z","iopub.status.busy":"2023-03-07T06:20:03.975415Z","iopub.status.idle":"2023-03-07T06:20:03.985982Z","shell.execute_reply":"2023-03-07T06:20:03.984995Z"},"papermill":{"duration":0.043272,"end_time":"2023-03-07T06:20:03.988008","exception":false,"start_time":"2023-03-07T06:20:03.944736","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Define the DataLoader","metadata":{"papermill":{"duration":0.029284,"end_time":"2023-03-07T06:20:04.045842","exception":false,"start_time":"2023-03-07T06:20:04.016558","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def collate_fn(batch):\n    return tuple(zip(*batch))\n\n\ndef create_dataloader(df, marking, trn_idx, val_idx):\n    train_ = df.loc[trn_idx,:].reset_index(drop=True)\n    valid_ = df.loc[val_idx,:].reset_index(drop=True)\n\n    # Dataset\n    train_datasets = WheatDataset(train_['image_id'].to_numpy(), marking, DATA_DIR+'train/', transforms=transform_train())\n    valid_datasets = WheatDataset(valid_['image_id'].to_numpy(), marking, DATA_DIR+'train/', transforms=transform_valid())\n\n    # DataLoader\n    train_loader = DataLoader(train_datasets, batch_size=BATCH_SIZE, collate_fn=collate_fn, num_workers=NUM_WORKERS, shuffle=True, multiprocessing_context='fork')\n    valid_loader = DataLoader(valid_datasets, batch_size=BATCH_SIZE, collate_fn=collate_fn, num_workers=NUM_WORKERS, shuffle=False, multiprocessing_context='fork')\n\n    return train_loader, valid_loader","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:20:04.108276Z","iopub.status.busy":"2023-03-07T06:20:04.107206Z","iopub.status.idle":"2023-03-07T06:20:04.115508Z","shell.execute_reply":"2023-03-07T06:20:04.114510Z"},"papermill":{"duration":0.042483,"end_time":"2023-03-07T06:20:04.117713","exception":false,"start_time":"2023-03-07T06:20:04.075230","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Define Model\n\nIn this notebook, we will use a pre-trained DETR model downloaded using `torch.hub.load()`.\n\n- Initial DETR model has 91 (+ 1 background) classes\n- Initial DETR model takes in 100 queries = outputs 100 bboxes for every image\n\nThus, we need to modify these parameters (`NUM_CLASSES`, `NUM_QUERIES`) to match **the number of classes and bounding boxes** in the target dataset.","metadata":{"papermill":{"duration":0.030903,"end_time":"2023-03-07T06:20:04.178378","exception":false,"start_time":"2023-03-07T06:20:04.147475","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class DETRModel(nn.Module):\n    def __init__(self):\n        super(DETRModel,self).__init__()\n        self.num_classes = NUM_CLASSES\n        self.num_queries = NUM_QUERIES\n        \n        # Donwload pre-trained model\n        self.model = torch.hub.load('facebookresearch/detr', 'detr_resnet50', pretrained=True)\n        self.in_features = self.model.class_embed.in_features\n        \n        self.model.class_embed = nn.Linear(in_features=self.in_features, out_features=self.num_classes)\n        self.model.num_queries = self.num_queries\n        \n    def forward(self,imgs):\n        return self.model(imgs)","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:20:04.241943Z","iopub.status.busy":"2023-03-07T06:20:04.241348Z","iopub.status.idle":"2023-03-07T06:20:04.248119Z","shell.execute_reply":"2023-03-07T06:20:04.247070Z"},"papermill":{"duration":0.041236,"end_time":"2023-03-07T06:20:04.250268","exception":false,"start_time":"2023-03-07T06:20:04.209032","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Define a class for averaging loss,metric,etc over epochs\n\nSource: https://www.kaggle.com/code/tanulsingh077/end-to-end-object-detection-with-transformers-detr#Wheat-Detection-Competition-With-DETR\n","metadata":{"papermill":{"duration":0.030273,"end_time":"2023-03-07T06:20:04.309245","exception":false,"start_time":"2023-03-07T06:20:04.278972","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class AverageMeter(object):\n    \"\"\"Computes and stores the average and current value\"\"\"\n    def __init__(self):\n        self.reset()\n\n    def reset(self):\n        self.val = 0\n        self.avg = 0\n        self.sum = 0\n        self.count = 0\n\n    def update(self, val, n=1):\n        self.val = val\n        self.sum += val * n\n        self.count += n\n        self.avg = self.sum / self.count","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:20:04.377826Z","iopub.status.busy":"2023-03-07T06:20:04.377090Z","iopub.status.idle":"2023-03-07T06:20:04.384116Z","shell.execute_reply":"2023-03-07T06:20:04.382954Z"},"papermill":{"duration":0.044829,"end_time":"2023-03-07T06:20:04.386470","exception":false,"start_time":"2023-03-07T06:20:04.341641","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Run Training\n\nTo create an effective deep learning model, we need to determine the optimal architecture and hyperparameters. \n\nOne approach for this is to use **cross-validation**, where data is divided into subsets to train and validate the model. Using MLOps tools like [**MLFlow**](https://mlflow.org/) can greatly assist in tracking the performance of the model.\n\n**Once we identify the optimal combination of architecture and hyperparameters**, we can **train the model on all available data**.\n\nIt's debatable whether to retrain a single model or use an ensemble of models created during cross-validation. \n\n- [StackExchange - How to choose a predictive model after k-fold cross-validation?](https://stats.stackexchange.com/questions/52274/how-to-choose-a-predictive-model-after-k-fold-cross-validation)\n- [StackOverflow - k-fold cross validation model selection method](https://stackoverflow.com/questions/46860325/k-fold-cross-validation-model-selection-method)\n\n## Cross-Validation\n\n**Before running the following cell, please set `RUN_TRAINING = True` and `TRAIN_ALL = False` in the `Set Config` section.**","metadata":{"papermill":{"duration":0.032825,"end_time":"2023-03-07T06:20:04.451472","exception":false,"start_time":"2023-03-07T06:20:04.418647","status":"completed"},"tags":[]}},{"cell_type":"code","source":"if RUN_TRAINING and (TRAIN_ALL == False):\n    # Set device\n    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') \n    print(f'Using {device} device')\n\n    # Cross-validation\n    folds = KFold(n_splits=FOLD_NUM, shuffle=True, random_state=SEED)\\\n            .split(np.arange(grouped_df.shape[0]), grouped_df['image_id'].to_numpy())\n    \n    # For Visualization\n    train_loss_list = []\n    valid_loss_list = []\n\n    for fold, (trn_idx, val_idx) in enumerate(folds):     \n        print(f'==========Cross-Validation Fold {fold+1}==========')   \n        # Define matcher, weight, and loss. \n        # See https://github.com/facebookresearch/detr/blob/3af9fa878e73b6894ce3596450a8d9b89d918ca9/models/detr.py#L304\n        matcher = HungarianMatcher()\n        weight_dict = weight_dict = {'loss_ce': 1, 'loss_bbox': 1 , 'loss_giou': 1}\n        losses = ['labels', 'boxes', 'cardinality']\n\n        # Load Data\n        train_loader, valid_loader = create_dataloader(grouped_df, marking, trn_idx, val_idx)\n\n        # Load model, loss function, and optimizing algorithm\n        model = DETRModel().to(device)\n        criterion = SetCriterion(NUM_CLASSES-1, matcher, weight_dict, eos_coef = NULL_CLASS_COEF, losses=losses).to(device) # eos_coef is used in the output layer to affect the output corresponding to the absence of an object.\n        optimizer = optim.Adam(model.parameters(), lr=LR)\n                \n        # For Visualization\n        train_losses = []\n        valid_losses = []\n\n        # Start training\n        best_loss = 10**5\n        for epoch in range(EPOCHS):\n            time_start = time.time()\n            print(f'==========Epoch {epoch+1} Start Training==========')\n            model.train()\n            # criterion.train()\n            train_loss = AverageMeter()\n            pbar = tqdm(enumerate(train_loader), total=len(train_loader))\n            for step, (imgs, targets, image_ids) in pbar:\n                # print('.', end='') # Sometimes progress bars do not emerge on the notebook. In in the case, remove hash\n                img_list = list(img.to(device) for img in imgs)\n                targets = [{k: v.to(device) for k, v in target.items()} for target in targets]\n                \n                output = model(img_list)\n                \n                loss_dict = criterion(output, targets)\n                weight_dict = criterion.weight_dict\n\n                losses = sum(loss_dict[k] * weight_dict[k] for k in loss_dict.keys() if k in weight_dict)\n\n                optimizer.zero_grad()\n                losses.backward()\n                optimizer.step()\n\n                train_loss.update(losses.item(), BATCH_SIZE)         \n\n            print(f'==========Epoch {epoch+1} Start Validation==========')\n            \n            with torch.no_grad():\n                valid_loss = AverageMeter() \n                preds = []\n                pbar = tqdm(enumerate(valid_loader), total=len(valid_loader))\n                for step, (imgs, targets, image_ids) in pbar:\n                    img_list = list(img.to(device) for img in imgs)\n                    targets = [{k: v.to(device) for k, v in target.items()} for target in targets]\n\n                    output = model(img_list)\n\n                    loss_dict = criterion(output, targets)\n                    weight_dict = criterion.weight_dict\n\n                    val_losses = sum(loss_dict[k] * weight_dict[k] for k in loss_dict.keys() if k in weight_dict)\n                    valid_loss.update(val_losses.item(), BATCH_SIZE)\n\n            # print results from this epoch\n            exec_t = int((time.time() - time_start)/60)\n            print(\n                f'Epoch : {epoch+1} - loss : {train_loss.avg:.4f} - val_loss : {valid_loss.avg:.4f} / Exec time {exec_t} min\\n'\n            )\n\n            # For visualization\n            train_losses.append(train_loss.avg)\n            valid_losses.append(valid_loss.avg)\n        \n        train_loss_list.append(train_losses)\n        valid_loss_list.append(valid_losses)\n        del model, optimizer, train_loader, valid_loader, train_losses, valid_losses\n        gc.collect()\n        torch.cuda.empty_cache()\n    \n    show_validation_score(train_loss_list, valid_loss_list)\n","metadata":{"execution":{"iopub.execute_input":"2023-03-07T06:20:04.520186Z","iopub.status.busy":"2023-03-07T06:20:04.519778Z","iopub.status.idle":"2023-03-07T07:10:36.792993Z","shell.execute_reply":"2023-03-07T07:10:36.791917Z"},"papermill":{"duration":3032.310869,"end_time":"2023-03-07T07:10:36.795484","exception":false,"start_time":"2023-03-07T06:20:04.484615","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Train on All Data\n\nAfter finding **the best number of epochs and hyperparameters** through cross-validation, the final step is **to train the model using all available data**.\n\nTo simplify the process and avoid errors, **combining the code for cross-validation and training on all data into a single script can be helpful** ([ref](https://www.kaggle.com/code/shnakazawa/semantic-segmentation-with-pytorch-and-u-net)). This approach streamlines the workflow and makes it easier to transition from cross-validation to training with all data.\n\n**Before running the following cell, please set `RUN_TRAINING = True` and `TRAIN_ALL = True` in the `Set Config` section.**","metadata":{"papermill":{"duration":0.036339,"end_time":"2023-03-07T07:10:36.871281","exception":false,"start_time":"2023-03-07T07:10:36.834942","status":"completed"},"tags":[]}},{"cell_type":"code","source":"if RUN_TRAINING and TRAIN_ALL:\n    # Set device\n    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') \n    print(f'Using {device} device')\n    \n    matcher = HungarianMatcher()\n    weight_dict = weight_dict = {'loss_ce': 1, 'loss_bbox': 1 , 'loss_giou': 1}\n    losses = ['labels', 'boxes', 'cardinality']\n        \n    # Load Data\n    train_datasets = WheatDataset(grouped_df['image_id'].to_numpy(), marking, DATA_DIR+'train/', transforms=transform_train())\n    train_loader = DataLoader(train_datasets, batch_size=BATCH_SIZE, collate_fn=collate_fn, num_workers=NUM_WORKERS, shuffle=True, multiprocessing_context='fork')\n\n    # Load model, loss function, and optimizing algorithm\n    model = DETRModel().to(device)\n    criterion = SetCriterion(NUM_CLASSES-1, matcher, weight_dict, eos_coef = NULL_CLASS_COEF, losses=losses).to(device)\n    optimizer = optim.Adam(model.parameters(), lr=LR)\n            \n    # Start training\n    best_loss = 10**5\n    for epoch in range(EPOCHS):\n        time_start = time.time()\n        print(f'==========Epoch {epoch+1} Start Training==========')\n        model.train()\n        criterion.train()\n        train_loss = AverageMeter()\n        pbar = tqdm(enumerate(train_loader), total=len(train_loader))\n        for step, (imgs, targets, image_ids) in pbar:\n            # print('.', end='') # Sometimes progress bars do not emerge on the notebook. In in the case, remove hash\n            img_list = list(img.to(device) for img in imgs)\n            targets = [{k: v.to(device) for k, v in target.items()} for target in targets]\n            \n            output = model(img_list)\n            \n            loss_dict = criterion(output, targets)\n            weight_dict = criterion.weight_dict\n\n            losses = sum(loss_dict[k] * weight_dict[k] for k in loss_dict.keys() if k in weight_dict)\n\n            optimizer.zero_grad()\n            losses.backward()\n            optimizer.step()  \n\n            train_loss.update(losses.item(), BATCH_SIZE)      \n\n        # print result from this epoch\n        exec_t = int((time.time() - time_start)/60)\n        print(\n            f'Epoch : {epoch+1} - loss : {train_loss.avg:.4f} / Exec time {exec_t} min\\n'\n        )    \n\n    print(f'Save model trained with all data')\n    os.makedirs(MODEL_DIR, exist_ok=True)\n    torch.save(model.state_dict(), MODEL_DIR+'objectdetection.pth')           \n\n    del model, optimizer, train_loader\n    gc.collect()\n    torch.cuda.empty_cache()","metadata":{"execution":{"iopub.execute_input":"2023-03-07T07:10:36.948803Z","iopub.status.busy":"2023-03-07T07:10:36.948431Z","iopub.status.idle":"2023-03-07T07:10:36.963247Z","shell.execute_reply":"2023-03-07T07:10:36.962267Z"},"papermill":{"duration":0.056935,"end_time":"2023-03-07T07:10:36.965523","exception":false,"start_time":"2023-03-07T07:10:36.908588","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Creating a final model using the parameters obtained from cross-validation could increase our confidence that the deep learning model is optimized for the specific task at hand. \n\nRemember that **deep learning models may require adjustments or retraining when new data becomes available or if the task requirements change**. This is especially important to keep in mind when working on your own projects.","metadata":{"papermill":{"duration":0.036153,"end_time":"2023-03-07T07:10:37.038752","exception":false,"start_time":"2023-03-07T07:10:37.002599","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# Run Inference of Test Data\n\nThe input data structure for testing differs from that of training, therefore the Dataset class needs to be re-defined.\n\n**Before running the following cells, please set `RUN_INFERENCE = True` in the `Set Config` section.**","metadata":{"papermill":{"duration":0.036753,"end_time":"2023-03-07T07:10:37.112550","exception":false,"start_time":"2023-03-07T07:10:37.075797","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def transform_test():\n    transforms = [\n        A.Resize(512,512,p=1),\n        ToTensorV2(p=1)\n    ]\n    return A.Compose(transforms)\n\n\nclass WheatTestDataset(Dataset):\n    def __init__(self, image_ids, data_root, transforms=None):\n        super().__init__()\n        self.image_ids = image_ids\n        self.data_root = data_root\n        self.transforms = transforms\n\n    def __len__(self):\n        return self.image_ids.shape[0]\n    \n    def __getitem__(self, index):\n        image_id = self.image_ids[index][:-4]\n        # Load images\n        image  = load_img(f'{self.data_root}{image_id}.jpg').astype(np.float32)\n        image /= 255.0 # normalization\n\n        # Transform images\n        if self.transforms:\n            # image = self.transforms(image)['image']\n            sample = {'image': image}\n            sample = self.transforms(**sample)\n            image = sample['image']\n\n        return image, image_id","metadata":{"execution":{"iopub.execute_input":"2023-03-07T07:10:37.190340Z","iopub.status.busy":"2023-03-07T07:10:37.189960Z","iopub.status.idle":"2023-03-07T07:10:37.198302Z","shell.execute_reply":"2023-03-07T07:10:37.197259Z"},"papermill":{"duration":0.049802,"end_time":"2023-03-07T07:10:37.200485","exception":false,"start_time":"2023-03-07T07:10:37.150683","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Then, run inference.","metadata":{"papermill":{"duration":0.038182,"end_time":"2023-03-07T07:10:37.276301","exception":false,"start_time":"2023-03-07T07:10:37.238119","status":"completed"},"tags":[]}},{"cell_type":"code","source":"if RUN_INFERENCE:\n    submission_df = pd.DataFrame()   \n    image_id_list = list(os.listdir(DATA_DIR+'test/'))\n    submission_df['image_file'] = image_id_list\n    submission_df['image_id'] = [image[:-4] for image in image_id_list]\n    submission_df['PredictionString'] = ''\n    \n    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') \n    print(f'Using {device} device')\n\n    # Load Data\n    test_datasets = WheatTestDataset(submission_df['image_file'].to_numpy(), DATA_DIR+'test/', transforms=transform_test())\n\n    # Data Loader\n    test_loader = DataLoader(test_datasets, batch_size=BATCH_SIZE, collate_fn=collate_fn, num_workers=NUM_WORKERS, shuffle=False, multiprocessing_context='fork')\n\n    # Load model\n    model = DETRModel().to(device)\n    model.load_state_dict(torch.load(MODEL_DIR+'objectdetection.pth'))\n\n    # Start Inference\n    print(f'==========Start Inference==========')\n    with torch.no_grad():\n        pbar = tqdm(enumerate(test_loader), total=len(test_loader))\n        for step, (imgs, image_ids) in pbar:\n            img_list = list(img.to(device) for img in imgs)\n            output = model(img_list)\n            for i, image_id in enumerate(image_ids):\n                pred_str = ''\n                prediction_scores = output['pred_logits'][i].softmax(1).detach().cpu().numpy()[:,0]\n                predicted_boxes = output['pred_boxes'][i].detach().cpu().numpy()\n\n                # _, h, w = imgs[0].shape\n                h,w,_ = load_img(f'{DATA_DIR}test/{image_id}.jpg').shape # height, width, color of origianl image\n                denormalized_boxes = denormalize_bbox(predicted_boxes, rows=h, cols=w)\n\n                for box, p in zip(denormalized_boxes, prediction_scores):\n                    if p > THRESHOLD:\n                        score = p\n                        pred_str += f'{score} {int(box[0])} {int(box[1])} {int(box[2])} {int(box[3])} '\n                submission_df.loc[submission_df['image_id'] == image_id, 'PredictionString'] = pred_str\n    print(submission_df.head())\nelse:\n    print('RUN_INFERENCE is False')","metadata":{"execution":{"iopub.execute_input":"2023-03-07T07:10:37.352381Z","iopub.status.busy":"2023-03-07T07:10:37.351315Z","iopub.status.idle":"2023-03-07T07:10:37.365141Z","shell.execute_reply":"2023-03-07T07:10:37.363965Z"},"papermill":{"duration":0.054991,"end_time":"2023-03-07T07:10:37.367893","exception":false,"start_time":"2023-03-07T07:10:37.312902","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Save inference result","metadata":{"papermill":{"duration":0.037318,"end_time":"2023-03-07T07:10:37.442380","exception":false,"start_time":"2023-03-07T07:10:37.405062","status":"completed"},"tags":[]}},{"cell_type":"code","source":"if RUN_INFERENCE:\n    submission_df.to_csv('submission.csv', index=False)\n\nelse:\n    print('RUN_INFERENCE is False')","metadata":{"execution":{"iopub.execute_input":"2023-03-07T07:10:37.518956Z","iopub.status.busy":"2023-03-07T07:10:37.517953Z","iopub.status.idle":"2023-03-07T07:10:37.525633Z","shell.execute_reply":"2023-03-07T07:10:37.524153Z"},"papermill":{"duration":0.048268,"end_time":"2023-03-07T07:10:37.527816","exception":false,"start_time":"2023-03-07T07:10:37.479548","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualize One Prediction Example\n\nLet's visual one prediction example.","metadata":{"papermill":{"duration":0.036404,"end_time":"2023-03-07T07:10:37.600667","exception":false,"start_time":"2023-03-07T07:10:37.564263","status":"completed"},"tags":[]}},{"cell_type":"code","source":"if RUN_INFERENCE:\n    target = submission_df.iloc[0]\n    img = load_img(DATA_DIR+'test/'+target['image_file']).copy()\n    prediction_results = target['PredictionString'].split(' ')\n    num_bbox = len(prediction_results)//5\n\n    for i in range(num_bbox):\n        x = int(prediction_results[i*5+1])\n        y = int(prediction_results[i*5+2])\n        w = int(prediction_results[i*5+3])\n        h = int(prediction_results[i*5+4])\n        cv2.rectangle(img,\n                    (x, y),\n                    (x+w, y+h),\n                    [255,0,0], 3)\n    plt.imshow(img)\n\nelse:\n    print('RUN_INFERENCE is False')","metadata":{"execution":{"iopub.execute_input":"2023-03-07T07:10:37.678124Z","iopub.status.busy":"2023-03-07T07:10:37.677102Z","iopub.status.idle":"2023-03-07T07:10:37.687546Z","shell.execute_reply":"2023-03-07T07:10:37.686546Z"},"papermill":{"duration":0.051165,"end_time":"2023-03-07T07:10:37.689503","exception":false,"start_time":"2023-03-07T07:10:37.638338","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# To Improve the Model\n\nIt seems that our object detection model works fairly well.\n\nThere are various techniques that can be employed to further enhance its performance, such as:\n\n- **Incorporating a wider range of data augmentation techniques**\n- **Implementing a more complex model architecture**\n- **Performing thorough hyperparameter tuning**\n- **Using ensembling methods**\n- and more.\n\nIt is important to bear in mind that the effectiveness of these techniques is highly dependent on the specifics of the object detection task at hand, including the available data. A combination of multiple techniques may be necessary to achieve the desired level of performance.\n\nI hope that this notebook has been informative and helpful in building an object detection model. I wish you the best of luck in all of your future modeling endeavors!\n\n# Acknowledgement\n\nI thank to [Matchan from YouTube Channel - Engawa AI Research Institute](https://www.youtube.com/channel/UCRwO-ewBHhNiC4qBEppi_JQ) and Mr. Masaaki Aiba for valuable discussion.\n\n\n# References\n\n- [End-to-End Object Detection with Transformers](https://arxiv.org/abs/2005.12872) ... Original paper of DETR\n- [PyTorch](https://pytorch.org/)\n\t- [PyTorch - SAVING AND LOADING MODELS](https://pytorch.org/tutorials/beginner/saving_loading_models.html)\n- [PyTorch Hub](https://pytorch.org/hub/)\n    - [PyTorch/Docs/torch.hub](https://pytorch.org/docs/stable/hub.html)\n- [Albumentations](https://albumentations.ai/)\n- [GitHub/facebookresearch/detr](https://github.com/facebookresearch/detr)\n- [End to End Object Detection with Transformers:DETR](https://www.kaggle.com/code/tanulsingh077/end-to-end-object-detection-with-transformers-detr#Wheat-Detection-Competition-With-DETR)\n- [GitHub/albumentations-team/albumentations#spatial-level-transforms](https://github.com/albumentations-team/albumentations#spatial-level-transforms)\n- [Albumentations Documentation/Bounding boxes augmentation for object detection](https://albumentations.ai/docs/getting_started/bounding_boxes_augmentation/#bounding-boxes-augmentation)","metadata":{"papermill":{"duration":0.036795,"end_time":"2023-03-07T07:10:37.764128","exception":false,"start_time":"2023-03-07T07:10:37.727333","status":"completed"},"tags":[]}}]}