{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":43873,"databundleVersionId":5282756,"sourceType":"competition"},{"sourceId":1154184,"sourceType":"datasetVersion","datasetId":652413},{"sourceId":1164617,"sourceType":"datasetVersion","datasetId":659784},{"sourceId":5264647,"sourceType":"datasetVersion","datasetId":3064176},{"sourceId":5271916,"sourceType":"datasetVersion","datasetId":3067635},{"sourceId":123745954,"sourceType":"kernelVersion"}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"- ### Inference code: [Benetech | EfficientDet [Inference]](https://www.kaggle.com/alejopaullier/benetech-efficientdet-inference)\n\n# | Benetech: EfficientDet [Train] \n\n***\n\n\n### Table of Contents \n
\n
  • Introduction
    • \n
  • Install libraries
    • \n
  • Import Libraries
    • \n
  • Configuration
    • \n
  • Utils
    • \n
  • Weights and Biases
    • \n
  • Load Data
    • \n
  • Validation
    • \n
  • Data Augmentation
    • \n
  • Dataset
    • \n
  • Data Loader
    • \n
  • Scheduler
    • \n
  • Model
    • \n
  • Loss Functions
    • \n
  • Train Function
    • \n
  • Validation Function
    • \n
  • Train Loop
    • \n
  • Training
    • \n
  • References
    • \n
    ","metadata":{}},{"cell_type":"markdown","source":"# | Introduction [↑](#top) \n\n***\n### Goal \n\n1. You need this notebook to detect the bounding boxes for the x-axis and y-axis ticks and labels. \n2. Knowing these bounding boxes you can later interpolate the data series with cross multiplication. \n3. By knowing the precise location of a chart element you can pass it to an OCR network to detect the numbers and text in the bounding box.\n\nThis notebook follows the [How to tackle this competition](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/396773) discussion. Here, we will address Step 3:\n\n
    \n\n### EfficientDet\n\nIn this notebook we will train the EfficientDet Object Detection network. This network was implemented in PyTorch by Ross Wightman, you can find the GitHub repo [here](https://github.com/rwightman/efficientdet-pytorch). \n\nEfficientDet is a family of state-of-the-art object detection models developed by researchers at Google, based on the EfficientNet architecture. These models are designed to achieve high accuracy while being computationally efficient, making them suitable for real-time applications.\n\nThe EfficientDet models are based on a compound scaling method, which involves scaling up the depth, width, and resolution of the network simultaneously. This allows for better feature extraction and higher accuracy without adding too much computational complexity.\n\nThese models have achieved state-of-the-art performance on several object detection benchmarks, including COCO, PASCAL VOC, and Open Images. These models are also used in various applications, including autonomous driving, robotics, and surveillance.\n\nThere are several variations with different model sizes, including EfficientDet-D0, D1, D2, D3, D4, and D5. Each variation has different parameters that are optimized for specific use cases. For example, EfficientDet-D0 is a small model suitable for mobile devices, while EfficientDet-D5 is a larger model designed for high-performance computing.\n\n
    \n\n","metadata":{}},{"cell_type":"markdown","source":"# | Install Libraries     [↑](#top) \n\n***","metadata":{}},{"cell_type":"code","source":"!pip install pycocotools","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2023-03-31T15:23:38.661475Z","iopub.execute_input":"2023-03-31T15:23:38.661838Z","iopub.status.idle":"2023-03-31T15:24:11.169322Z","shell.execute_reply.started":"2023-03-31T15:23:38.661801Z","shell.execute_reply":"2023-03-31T15:24:11.168057Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Import Libraries [↑](#top) \n\n***\n\nImport all the required libraries for this notebook.","metadata":{}},{"cell_type":"code","source":"import sys\nsys.path.insert(0, \"../input/timm-efficientdet-pytorch\")\nsys.path.insert(0, \"../input/omegaconf\")\nsys.path.insert(0, \"../input/albumentations\")\n\nimport albumentations as A\nimport copy\nimport cv2\nimport gc\nimport matplotlib.pyplot as plt\nimport multiprocessing\nimport numpy as np\nimport os\nimport pandas as pd\nimport PIL\nimport random\nimport time\nimport timm\nimport torch\nimport torch.nn as nn\n\n\nfrom albumentations.pytorch.transforms import ToTensorV2\nfrom colorama import Fore, Back, Style\nfrom datetime import datetime, timedelta\nfrom glob import glob\nfrom pprint import pprint\nfrom sklearn.model_selection import KFold, GroupKFold, StratifiedKFold\nfrom torch.utils.data import Dataset, DataLoader\nfrom torch.utils.data.sampler import SequentialSampler, RandomSampler\nfrom torch.optim import lr_scheduler\nfrom tqdm import tqdm\n\n\nc_ = Fore.GREEN\nsr_ = Style.RESET_ALL\nprint(f\"There are {multiprocessing.cpu_count()} CPUs available\")\nprint()\n!mkdir logs\n!mkdir saved_models","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-03-31T15:24:11.172651Z","iopub.execute_input":"2023-03-31T15:24:11.174274Z","iopub.status.idle":"2023-03-31T15:24:18.401901Z","shell.execute_reply.started":"2023-03-31T15:24:11.174229Z","shell.execute_reply":"2023-03-31T15:24:18.400464Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Configuration [↑](#top) \n\n***\n\nCentral repository for this notebook's hyperparameters.","metadata":{}},{"cell_type":"code","source":"class config:\n BATCH_SIZE_TRAIN = 4\n BATCH_SIZE_VALID = 2\n DEBUG = False\n DEVICE = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n EPOCHS = 5\n FOLDS = 5\n LR = 2e-4\n MIN_LR = 1e-6\n NUM_WORKERS = multiprocessing.cpu_count()\n RESOLUTION = 512\n SAMPLE = 30_000\n SEED = 42\n SCHEDULER = 'CosineAnnealingLR'\n T_0 = 25\n T_MAX = int(30_000/BATCH_SIZE_TRAIN*EPOCHS)+50\n WARMUP_EPOCHS = 0\n WEIGHT_DECAY = 1e-6\n \n \nclass paths:\n TRAIN_ANNOTATIONS_FOLDER = \"/kaggle/input/benetech-making-graphs-accessible/train/annotations/\"\n TRAIN_IMAGES_FOLDER = \"/kaggle/input/benetech-making-graphs-accessible/train/images/\"","metadata":{"execution":{"iopub.status.busy":"2023-03-31T15:26:02.072606Z","iopub.execute_input":"2023-03-31T15:26:02.073769Z","iopub.status.idle":"2023-03-31T15:26:02.229313Z","shell.execute_reply.started":"2023-03-31T15:26:02.073713Z","shell.execute_reply":"2023-03-31T15:26:02.228063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Utils [↑](#top) \n\n***\n\nUtility functions used throughout the notebook.","metadata":{}},{"cell_type":"code","source":"def get_stoi(df):\n \"\"\"Get String to Index dictionary\"\"\"\n stoi = {}\n for idx, string in enumerate(df.label.unique()):\n stoi[string] = idx + 1\n itos = {item[1]: item[0] for item in stoi.items()}\n df = df.replace({\"label\": stoi})\n return stoi\n\n\ndef seed_everything(seed):\n random.seed(seed)\n os.environ['PYTHONHASHSEED'] = str(seed)\n np.random.seed(seed)\n torch.manual_seed(seed)\n torch.cuda.manual_seed(seed)\n torch.backends.cudnn.deterministic = True\n torch.backends.cudnn.benchmark = True\n\nseed_everything(config.SEED)","metadata":{"execution":{"iopub.status.busy":"2023-03-31T15:26:03.012081Z","iopub.execute_input":"2023-03-31T15:26:03.012780Z","iopub.status.idle":"2023-03-31T15:26:03.023765Z","shell.execute_reply.started":"2023-03-31T15:26:03.012742Z","shell.execute_reply":"2023-03-31T15:26:03.022785Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Load Data [↑](#top) \n\n***\n\n- `image_id`: unique identifier for each image.\n- `source`: whether the chart was extracted or generated.\n- `chart`: the type of chart.\n- `x0`: x-axis coordinate in pixels of top left corner.\n- `y0`: y-axis coordinate in pixels of top left corner.\n- `w`: width in pixels. Same as x-axis difference between top left corner and bottom right corner.\n- `h`: height in pixels. Same as y-axis difference between top left corner and bottom right corner.\n- `label`: label of the bounding box.","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(\"/kaggle/input/benetech-create-bounding-box-dataframe/train.csv\")\nprint(f\"Dataframe shape is: {df.shape}\")\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-31T15:27:45.769172Z","iopub.execute_input":"2023-03-31T15:27:45.769615Z","iopub.status.idle":"2023-03-31T15:27:47.444885Z","shell.execute_reply.started":"2023-03-31T15:27:45.769559Z","shell.execute_reply":"2023-03-31T15:27:47.443786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Data Pre-procesing","metadata":{}},{"cell_type":"code","source":"image_ids = df[\"image_id\"].unique().tolist()\nimage_ids = random.sample(image_ids, config.SAMPLE)\nfilter = df[(df[\"x0\"]<0) | (df[\"y0\"]<0) | (df[\"h\"]<=0) | (df[\"w\"]<=0)].index\ndf = df[~df.index.isin(filter)]\ndf = df[df[\"image_id\"].isin(image_ids)]\nstoi = get_stoi(df)\npprint(stoi)\nconfig.NUM_CLASSES = len(stoi)\ndf = df[df[\"label\"].isin(list(stoi.keys()))]\ndf = df.replace({\"label\": stoi})\ndf.reset_index(inplace=True)\nprint(f\"Dataframe shape is: {df.shape}\")\nprint(f\"Number of classes: {config.NUM_CLASSES}\")\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2023-03-31T15:27:48.285605Z","iopub.execute_input":"2023-03-31T15:27:48.286228Z","iopub.status.idle":"2023-03-31T15:27:49.546053Z","shell.execute_reply.started":"2023-03-31T15:27:48.286187Z","shell.execute_reply":"2023-03-31T15:27:49.545006Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Validation [↑](#top) \n\n***\n\n\nSplit the dataset into train and validation sets.","metadata":{}},{"cell_type":"code","source":"gkf = GroupKFold(n_splits=config.FOLDS) # Seed for reproducibility\nX = df.loc[:, df.columns != \"label\"]\ny = df.loc[:, df.columns == \"label\"]\ngroups = df.loc[:, df.columns == \"image_id\"]\n\nfor fold, (train_index, val_index) in enumerate(gkf.split(X, y, groups)):\n df.loc[val_index, 'fold'] = int(fold) # Assign to each row its Fold ID\ndisplay(df.groupby(['fold','chart'])[\"index\"].count())","metadata":{"execution":{"iopub.status.busy":"2023-03-31T12:51:15.913535Z","iopub.execute_input":"2023-03-31T12:51:15.914212Z","iopub.status.idle":"2023-03-31T12:51:16.478701Z","shell.execute_reply.started":"2023-03-31T12:51:15.914176Z","shell.execute_reply":"2023-03-31T12:51:16.477591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Data Augmentation [↑](#top) \n\n***\n\nAlbumentations transformations.","metadata":{}},{"cell_type":"code","source":"def get_train_transforms():\n return A.Compose(\n [\n A.Resize(height=config.RESOLUTION, width=config.RESOLUTION, p=1),\n A.Normalize(p=1),\n ToTensorV2(p=1.0),\n ], \n p=1.0, \n bbox_params=A.BboxParams(\n format='pascal_voc', min_area=0, min_visibility=0, label_fields=['labels']\n )\n )\n\ndef get_valid_transforms():\n return A.Compose(\n [\n A.Resize(height=config.RESOLUTION, width=config.RESOLUTION, p=1.0),\n A.Normalize(p=1),\n ToTensorV2(p=1.0),\n ], \n p=1.0, \n bbox_params=A.BboxParams(\n format='pascal_voc', min_area=0, min_visibility=0, label_fields=['labels']\n )\n )","metadata":{"execution":{"iopub.status.busy":"2023-03-31T12:51:16.481054Z","iopub.execute_input":"2023-03-31T12:51:16.481708Z","iopub.status.idle":"2023-03-31T12:51:16.489916Z","shell.execute_reply.started":"2023-03-31T12:51:16.481668Z","shell.execute_reply":"2023-03-31T12:51:16.488763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Dataset [↑](#top) \n\n***\n\nCreate a custom PyTorch dataset.","metadata":{}},{"cell_type":"code","source":"class CustomDataset(Dataset):\n\n def __init__(self, df, transforms=None):\n super().__init__()\n self.df = df # pandas dataframe\n self.transforms = transforms # albumentations transformations\n self.image_ids = self.df[\"image_id\"].unique().tolist() # list with unique image IDs\n\n def __getitem__(self, index: int):\n \"\"\"\n :return image: an augmented image as a numpy array.\n :return target: a dictionary containing a tensor with the bboxes for the image (torch.Tensor),\n a list of strings containing the bboxes labels and a tensor containing the image index.\n :return image_id: the image ID. A unique identifier for each image. \n \n \"\"\"\n image_id = self.image_ids[index] # select one image\n image, boxes = self.load_image_and_boxes(index) # load the image and its associated bounding boxes\n labels = self.get_labels(index)\n target = {\n 'boxes' : boxes, \n 'labels' : labels, \n 'index' : torch.tensor([index])\n }\n transformed_image = self.transforms(**{\n 'image': image,\n 'bboxes': target['boxes'],\n 'labels': target[\"labels\"]\n })\n image = transformed_image['image']\n _, new_h, new_w = image.shape\n \n # This creates a torch tensor of size (number_bboxes, 4) where each row is a bounding box:\n target['boxes'] = torch.stack(tuple(map(torch.tensor, zip(*transformed_image['bboxes'])))).permute(1, 0)\n target['boxes'][:,[0,1,2,3]] = target['boxes'][:,[1,0,3,2]] # Required: change order to: (y, x, y, x)\n target[\"img_size\"] = (new_h, new_w)\n target[\"img_scale\"] = torch.tensor([1.0])\n return image, target, image_id\n\n def __len__(self) -> int:\n return len(self.image_ids)\n\n def load_image_and_boxes(self, index):\n \"\"\"\n :return image: the image as a numpy array. The array is scaled to the [0,1] interval. Numpy array.\n :return boxes: an array containing bounding boxes rows = [x0, y0, x0 + height, y0 + width]. List of lists.\n \"\"\"\n image_id = self.image_ids[index] # select image\n image = cv2.imread(f'{paths.TRAIN_IMAGES_FOLDER}{image_id}.jpg', cv2.IMREAD_COLOR) # read image from path\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32) # convert to RGB \n records = self.df[self.df['image_id'] == image_id] # select all rows corresponding to the image\n boxes = records[['x0', 'y0', 'w', 'h']].values # get bounding box information\n boxes[:, 2] = boxes[:, 0] + boxes[:, 2] # x0 + Δx (or also x0 + height), pascal_voc format\n boxes[:, 3] = boxes[:, 1] + boxes[:, 3] # y0 + Δy (or also y0 + width), pascal_voc format\n boxes = boxes.tolist() # convert to list\n return image, boxes\n \n def get_labels(self, index):\n image_id = self.image_ids[index]\n labels = self.df[self.df['image_id'] == image_id][\"label\"].values.tolist()\n labels = torch.tensor(labels)\n return labels","metadata":{"execution":{"iopub.status.busy":"2023-03-31T12:51:16.819744Z","iopub.execute_input":"2023-03-31T12:51:16.820517Z","iopub.status.idle":"2023-03-31T12:51:16.835181Z","shell.execute_reply.started":"2023-03-31T12:51:16.820473Z","shell.execute_reply":"2023-03-31T12:51:16.833955Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Plot one image\n \n \nCheck everything is all right.","metadata":{}},{"cell_type":"code","source":"train_dataset = CustomDataset(\n df=df,\n transforms=get_train_transforms()\n)\nindex = random.choice(range(0,1000)) # select random image index\nimage, target, image_id = train_dataset[index] # get one sample from dataset\nboxes = target['boxes'].cpu().numpy().astype(np.int32) # get bounding boxes from torch tensor\nnumpy_image = image.permute(1,2,0).cpu().numpy() # get image\nprint(f\"Image ID: {image_id}\")\noriginal_image = cv2.imread(paths.TRAIN_IMAGES_FOLDER + image_id + \".jpg\")\nfig, ax = plt.subplots(1, 1, figsize=(12, 10)) # create figure\n# === Plot bounding boxes ====\nfor box in boxes:\n cv2.rectangle(numpy_image, (box[1], box[0]), (box[3], box[2]), (0, 1, 0), 2)\n \nax.set_axis_off()\nax.imshow(numpy_image)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-03-31T12:51:17.278685Z","iopub.execute_input":"2023-03-31T12:51:17.279358Z","iopub.status.idle":"2023-03-31T12:51:17.868742Z","shell.execute_reply.started":"2023-03-31T12:51:17.279321Z","shell.execute_reply":"2023-03-31T12:51:17.867655Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | DataLoaders [↑](#top) \n\n***\n\nPrepare dataloaders.","metadata":{}},{"cell_type":"code","source":"def collate_fn(batch):\n return tuple(zip(*batch))\n\n\ndef prepare_loaders(fold, df):\n \"\"\"\n Splits data into Train and Validation sets depending on the current fold. Creates PyTorch\n datasets from the splits. Creates DataLoaders from Datasets.\n :param fold: current fold (int).\n :param df: train dataframe (pandas dataframe).\n :return train_loader, valid_loader: dataloaders for each stage.\n \"\"\"\n # === Select data for Train and Validation ===\n train_df = df.query(\"fold!=@fold\").reset_index(drop=True) # Select all rows not from validation fold\n valid_df = df.query(\"fold==@fold\").reset_index(drop=True) # Select all rows from validation fold\n \n # === Mini sample for debugging purposes ===\n if config.DEBUG:\n train_df = train_df.head(32*5).query(\"empty==0\")\n valid_df = valid_df.head(32*3).query(\"empty==0\")\n \n # === Build Datasets ===\n train_dataset = CustomDataset(train_df, transforms=get_train_transforms())\n valid_dataset = CustomDataset(valid_df, transforms=get_valid_transforms())\n \n # === Create DataLoaders for Train and Validation ===\n train_loader = DataLoader(train_dataset, \n batch_size=config.BATCH_SIZE_TRAIN if not config.DEBUG else 20, \n num_workers=config.NUM_WORKERS,\n sampler=RandomSampler(train_dataset),\n pin_memory=False, drop_last=False, collate_fn=collate_fn)\n valid_loader = DataLoader(valid_dataset,\n batch_size=config.BATCH_SIZE_VALID if not config.DEBUG else 20, \n num_workers=config.NUM_WORKERS,\n sampler=SequentialSampler(valid_dataset),\n shuffle=False, pin_memory=True, collate_fn=collate_fn)\n \n return train_loader, valid_loader\n\n\ntrain_loader, valid_loader = prepare_loaders(fold=0, df=df)","metadata":{"execution":{"iopub.status.busy":"2023-03-31T12:51:21.352518Z","iopub.execute_input":"2023-03-31T12:51:21.353195Z","iopub.status.idle":"2023-03-31T12:51:21.498364Z","shell.execute_reply.started":"2023-03-31T12:51:21.353158Z","shell.execute_reply":"2023-03-31T12:51:21.497211Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Scheduler [↑](#top) \n\n***","metadata":{}},{"cell_type":"code","source":"def fetch_scheduler(optimizer):\n if config.SCHEDULER == 'CosineAnnealingLR':\n scheduler = lr_scheduler.CosineAnnealingLR(optimizer,T_max=config.T_MAX, \n eta_min=config.MIN_LR)\n elif config.SCHEDULER == 'CosineAnnealingWarmRestarts':\n scheduler = lr_scheduler.CosineAnnealingWarmRestarts(optimizer,T_0=config.T_0, \n eta_min=config.MIN_LR)\n elif config.SCHEDULER == 'ReduceLROnPlateau':\n scheduler = lr_scheduler.ReduceLROnPlateau(optimizer,\n mode='min',\n factor=0.1,\n patience=7,\n threshold=0.0001,\n min_lr=config.MIN_LR,)\n elif config.SCHEDULER == 'ExponentialLR':\n scheduler = lr_scheduler.ExponentialLR(optimizer, gamma=0.85)\n elif config.SCHEDULER == None:\n return None\n \n return scheduler","metadata":{"execution":{"iopub.status.busy":"2023-03-31T12:51:21.969598Z","iopub.execute_input":"2023-03-31T12:51:21.970329Z","iopub.status.idle":"2023-03-31T12:51:21.979736Z","shell.execute_reply.started":"2023-03-31T12:51:21.970294Z","shell.execute_reply":"2023-03-31T12:51:21.978251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Model [↑](#top) \n\n***\n\nCreate the training Model.","metadata":{}},{"cell_type":"code","source":"from effdet.config.model_config import efficientdet_model_param_dict\nfrom effdet import get_efficientdet_config, EfficientDet, DetBenchTrain, DetBenchPredict\nfrom effdet.efficientdet import HeadNet\nfrom effdet.config.model_config import efficientdet_model_param_dict\n\ndef create_model(num_classes=config.NUM_CLASSES, image_size=512,\n architecture=\"tf_efficientnetv2_s\", verbose=False):\n \n efficientdet_model_param_dict['tf_efficientnetv2_s'] = dict(\n name='tf_efficientnetv2_s',\n backbone_name='tf_efficientnetv2_s',\n backbone_args=dict(drop_path_rate=0.2),\n num_classes=num_classes,\n url='')\n \n cfg = get_efficientdet_config(architecture)\n cfg.update({'num_classes': num_classes})\n cfg.update({'image_size': (image_size, image_size)})\n \n if verbose:\n pprint(cfg)\n\n net = EfficientDet(cfg, pretrained_backbone=True)\n net.class_net = HeadNet(\n cfg,\n num_outputs=cfg.num_classes,\n )\n return DetBenchTrain(net, cfg)\n\n\ndef load_model(model_weights_path, model):\n \"\"\"\n Load model weights.\n \"\"\"\n model.load_state_dict(torch.load(model_weights_path))\n model.eval()\n return model","metadata":{"execution":{"iopub.status.busy":"2023-03-31T12:51:22.755583Z","iopub.execute_input":"2023-03-31T12:51:22.756670Z","iopub.status.idle":"2023-03-31T12:51:23.322003Z","shell.execute_reply.started":"2023-03-31T12:51:22.756597Z","shell.execute_reply":"2023-03-31T12:51:23.320977Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Average Meter [↑](#top) \n\n***\n\nTrack metrics and loss.","metadata":{}},{"cell_type":"code","source":"class AverageMeter(object):\n \"\"\"Computes and stores the average and current value\"\"\"\n def __init__(self):\n \"\"\"Initializes an instance by reseting its values\"\"\"\n self.reset()\n\n def reset(self):\n \"\"\"Resets all values to zero\"\"\"\n self.val = 0\n self.avg = 0\n self.sum = 0\n self.count = 0\n\n def update(self, val, n=1):\n \"\"\"\n Tracks values, count, sum and average.\n :param val: usually the loss function value.\n :param n: usually the number of samples.\n \"\"\"\n self.val = val\n self.sum += val * n\n self.count += n\n self.avg = self.sum / self.count","metadata":{"execution":{"iopub.status.busy":"2023-03-31T12:51:24.005470Z","iopub.execute_input":"2023-03-31T12:51:24.006173Z","iopub.status.idle":"2023-03-31T12:51:24.013078Z","shell.execute_reply.started":"2023-03-31T12:51:24.006133Z","shell.execute_reply":"2023-03-31T12:51:24.011689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Train Function [↑](#top) \n\n***\n\nTrain one epoch.","metadata":{}},{"cell_type":"code","source":"def train_one_epoch(model, optimizer, scheduler, dataloader, device, epoch):\n model.train() # Set model in training mode\n loss_meter = AverageMeter() # Create instance\n pbar = tqdm(enumerate(dataloader), total=len(dataloader), desc='Train ') # Progress bar\n for step, (images, targets, image_ids) in pbar:\n # === Collate ===\n images = torch.stack(images).to(device).float() # Get images (batch_size, 3, RESOLUTION, RESOLUTION)\n batch_size = images.shape[0] # Get batch size\n boxes = [target['boxes'].to(device).float() for target in targets] # Get bounding boxes\n labels = [target['labels'].to(device).float() for target in targets] # Get labels (tuple with strings)\n img_size = torch.tensor([target[\"img_size\"] for target in targets]).to(device).float()\n img_scale = torch.tensor([target[\"img_scale\"] for target in targets]).to(device).float()\n annotations = {\n \"bbox\": boxes,\n \"cls\": labels,\n \"img_size\": img_size,\n \"img_scale\": img_scale\n }\n optimizer.zero_grad() # Zero out gradients\n loss = model(images, annotations) # Forward pass\n loss = loss[\"loss\"]\n loss.backward() # Back propagation\n # Since the reduction type of the loss is \"mean\" we multiply by batch_size\n loss_meter.update(loss.detach().item(), batch_size) # Update loss\n optimizer.step() # Update params\n scheduler.step() # Update learning rate\n \n # === Evaluate model ===\n \n mem = torch.cuda.memory_reserved() / 1e9 if torch.cuda.is_available() else 0 # Track memory\n current_lr = optimizer.param_groups[0]['lr'] # Get current Learning Rate\n pbar.set_postfix(train_loss=f'{loss_meter.avg:0.4f}',\n lr=f'{current_lr:0.5f}',\n gpu_mem=f'{mem:0.2f} GB')\n # === Release memory ===\n torch.cuda.empty_cache()\n gc.collect()\n \n return loss_meter","metadata":{"execution":{"iopub.status.busy":"2023-03-31T12:51:25.141085Z","iopub.execute_input":"2023-03-31T12:51:25.142051Z","iopub.status.idle":"2023-03-31T12:51:25.154031Z","shell.execute_reply.started":"2023-03-31T12:51:25.141994Z","shell.execute_reply":"2023-03-31T12:51:25.152697Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Validation Function [↑](#top) \n\n***\n\n\nValidate one epoch.","metadata":{}},{"cell_type":"code","source":"@torch.no_grad()\ndef valid_one_epoch(model, dataloader, device, epoch):\n model.eval() # Set model in evaluation mode\n loss_meter = AverageMeter() # Create instance\n pbar = tqdm(enumerate(dataloader), total=len(dataloader), desc='Valid ') # Progress bar\n for step, (images, targets, image_ids) in pbar: \n # === Collate ===\n images = torch.stack(images).to(device).float() # Get images\n batch_size = images.shape[0] # Get batch size\n boxes = [target['boxes'].to(device).float() for target in targets] # Get boxes\n labels = [target['labels'].to(device).float() for target in targets] # Get labels\n img_size = torch.tensor([target[\"img_size\"] for target in targets]).to(device).float()\n img_scale = torch.tensor([target[\"img_scale\"] for target in targets]).to(device).float()\n \n annotations = {\n \"bbox\": boxes,\n \"cls\": labels,\n \"img_size\": img_size,\n \"img_scale\": img_scale\n }\n loss = model(images, annotations) # Forward pass\n loss = loss[\"loss\"]\n loss_meter.update(loss.detach().item(), batch_size) # Update loss\n # === Evaluate model ===\n \n mem = torch.cuda.memory_reserved() / 1e9 if torch.cuda.is_available() else 0 # Track memory\n current_lr = optimizer.param_groups[0]['lr'] # Get current learning rate\n pbar.set_postfix(valid_loss=f'{loss_meter.avg:0.4f}',\n lr=f'{current_lr:0.5f}',\n gpu_memory=f'{mem:0.2f} GB')\n # === Release memory ===\n torch.cuda.empty_cache()\n gc.collect()\n \n return loss_meter","metadata":{"execution":{"iopub.status.busy":"2023-03-31T12:51:26.035161Z","iopub.execute_input":"2023-03-31T12:51:26.036140Z","iopub.status.idle":"2023-03-31T12:51:26.047388Z","shell.execute_reply.started":"2023-03-31T12:51:26.036091Z","shell.execute_reply":"2023-03-31T12:51:26.046192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Train Loop [↑](#top)\n\n***","metadata":{}},{"cell_type":"code","source":"def train_loop(model, optimizer, scheduler, device, num_epochs):\n f = open(f\"/kaggle/working/logs/logs.txt\", \"w+\") # Create log file\n \n if torch.cuda.is_available(): # Check if GPU is available\n print(\"Cuda: {}\\n\".format(torch.cuda.get_device_name()))\n \n start = time.time() # Track execution time\n best_model_weights = copy.deepcopy(model.state_dict())\n epochs = config.EPOCHS\n best_loss = 1e10\n for epoch in range(1, epochs + 1):\n print(f'Epoch {epoch}/{num_epochs}', end='')\n loss_meter_train = train_one_epoch(model, optimizer, scheduler, \n dataloader=train_loader, \n device=config.DEVICE, epoch=epoch)\n \n loss_meter_valid = valid_one_epoch(model, valid_loader, \n device=config.DEVICE, \n epoch=epoch)\n \n duration = str(timedelta(seconds=time.time() - start))[:7]\n # === Print to log file ===\n with open(f\"logs/logs.txt\", 'a+') as f:\n print('{} | Epoch: {}/{} | Train Loss: {:.4} '. \\\n format(duration, epoch + 1, epochs, loss_meter_train.avg), file=f)\n print('{} | Epoch: {}/{} | Valid Loss: {:.4}'. \\\n format(duration, epoch + 1, epochs, loss_meter_valid.avg), file=f)\n print(\"\\n\" + \"-\"*100 + \"\\n\", file=f)\n \n # === Save model if there is an improvement ===\n if loss_meter_valid.avg < best_loss:\n best_loss = loss_meter_valid.avg\n best_epoch = epoch\n best_model_weights = copy.deepcopy(model.state_dict())\n PATH = f\"/kaggle/working/saved_models/best_epoch-{fold:02d}.bin\"\n torch.save(model.state_dict(), PATH)\n print(f\"Model Saved | Best Epoch {best_epoch} | Best Loss {round(best_loss,2)} {sr_}\")\n \n last_model_wts = copy.deepcopy(model.state_dict())\n PATH = f\"last_epoch-{fold:02d}.bin\"\n torch.save(model.state_dict(), PATH)\n print(); print()\n \n end = time.time()\n time_elapsed = end - start\n print('Training complete in {:.0f}h {:.0f}m {:.0f}s'.format(\n time_elapsed // 3600, (time_elapsed % 3600) // 60, (time_elapsed % 3600) % 60))\n \n return model","metadata":{"execution":{"iopub.status.busy":"2023-03-31T12:51:27.035571Z","iopub.execute_input":"2023-03-31T12:51:27.036094Z","iopub.status.idle":"2023-03-31T12:51:27.063745Z","shell.execute_reply.started":"2023-03-31T12:51:27.036040Z","shell.execute_reply":"2023-03-31T12:51:27.061331Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# | Training [↑](#top) \n\n***","metadata":{}},{"cell_type":"code","source":"for fold in range(1):\n print(f'================= Fold: {1} =================')\n train_loader, valid_loader = prepare_loaders(fold=fold, df=df)\n model = create_model()\n model.to(config.DEVICE)\n optimizer = torch.optim.AdamW(model.parameters(), lr=config.LR, weight_decay=config.WEIGHT_DECAY)\n scheduler = fetch_scheduler(optimizer)\n model = train_loop(model, optimizer, scheduler,\n device=config.DEVICE,\n num_epochs=config.EPOCHS)","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2023-03-31T12:51:28.844255Z","iopub.execute_input":"2023-03-31T12:51:28.844862Z","iopub.status.idle":"2023-03-31T12:54:34.793373Z","shell.execute_reply.started":"2023-03-31T12:51:28.844821Z","shell.execute_reply":"2023-03-31T12:54:34.791710Z"},"trusted":true},"execution_count":null,"outputs":[]}]}