In this competition, you’ll detect wheat heads from outdoor images of wheat plants, including wheat datasets from around the globe. Using worldwide data, you will focus on a generalized solution to estimate the number and size of wheat heads. To better gauge the performance for unseen genotypes, environments, and observational conditions, the training dataset covers multiple regions. You will use more than 3,000 images from Europe (France, UK, Switzerland) and North America (Canada). The test data includes about 1,000 images from Australia, Japan, and China.
\n\n\n***\nThis competition wants us to predict bounding boxes around whaet heads from images of wheat plants, to solve this problem we have a sample of image, and csv file containing the image_id (the unique image ID), the width and height of the images, and bounding box, formatted as a Python-style list of [xmin, ymin, width, height]
\n\n***\nIn this notebook I want to share how to visualize image data with boundary box and some augmentation, then I am going to train data with FasterRCNN resnet50 to solve this problem and compare with difference configuration, if you want to see my other notebook with a different model, [Detectron 2 compare models](http://www.kaggle.com/dhiiyaur/detectron-2-compare-models)\n","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"### Change Log\n\n#### V4\n\n> - add new model\n> - add cutout augmentation\n\n#### V3\n\n> - add some information\n\n\n#### V2\n\n> - add new model (LB score 0.663 --> 0.673)\n\n#### V1\n\n> - add inference\n> - add another model to compare\n> - change code for train and test","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Import necessary libraries","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nimport time\nimport shutil\nimport random\n\nimport numpy as np\nimport pandas as pd\n\nimport torch\nimport torchvision\nfrom torchvision import transforms\nfrom torch.utils.data import DataLoader, Dataset\nfrom torchvision.models.detection.faster_rcnn import FastRCNNPredictor, FasterRCNN\nfrom torchvision.models.detection.backbone_utils import resnet_fpn_backbone\nfrom tqdm.notebook import tqdm\n\nimport albumentations\nfrom albumentations import *\nfrom albumentations.pytorch.transforms import ToTensorV2\nimport cv2\nimport matplotlib.pyplot as plt\nfrom matplotlib import patches\nimport seaborn as sns\n%matplotlib inline\nprint('Ready...')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Load Data and Simple EDA","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"DATA_DIR = '../input/global-wheat-detection/train/'\nTEST_DIR = '../input/global-wheat-detection/test/'\ntrain_df_path = '../input/global-wheat-detection/train.csv'\ntest_df_path = '../input/global-wheat-detection/sample_submission.csv'\nList_Data_dir = os.listdir(DATA_DIR)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"raw = pd.read_csv(train_df_path)\nraw","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"let's check if there is empty data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"raw.isnull().any().any()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"raw.info()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Based on the data in the table, it can be seen that there is no empty data, and we can see the data types (Dtype) from each column. let's dig a little deeper\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"raw.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Based on the data in the table, we can see there are only two tables (width and height), Why? because from the data we have there are only two columns of type number (Dtype --> number (int/float). From this table, it is known that all our images are 1024 x 1024. Ok, let's see how many that pictures we have","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f'Total number of train images: {raw.image_id.nunique()}')\nprint(f'Total number of test images: {len(os.listdir(TEST_DIR))}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"visualize wheat distribution from source colomn","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(15,8))\nplt.title('Wheat Distribution', fontsize= 20)\nsns.countplot(x=\"source\", data=raw)\n\n# based on the chart, there are seven types of wheat from images data, with the most types 'ethz_1' and the least is 'inrae_1'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's visualize some image, that we have","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(1, 2, figsize = (24, 24))\nax = ax.flatten()\nimg_path = os.path.join(DATA_DIR, 'b6ab77fd7' + '.jpg')\nimg_path2 = os.path.join(DATA_DIR, '5e0747034' + '.jpg')\n\nimage = cv2.imread(img_path, cv2.IMREAD_COLOR)\nimage = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32)\nimage /= 255.0\n \nax[0].set_title('Image')\nax[0].imshow(image)\n \nimage2 = cv2.imread(img_path2, cv2.IMREAD_COLOR)\nimage2 = cv2.cvtColor(image2, cv2.COLOR_BGR2RGB).astype(np.float32)\nimage2 /= 255.0\n\nax[1].set_title('Image')\nax[1].imshow(image2)\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Extract bbox column to xmin, ymin, width, height, then create xmax, ymax, and area columns\n\nraw[['xmin','ymin','w','h']] = pd.DataFrame(raw.bbox.str.strip('[]').str.split(',').tolist()).astype(float)\nraw['xmax'], raw['ymax'], raw['area'] = raw['xmin'] + raw['w'], raw['ymin'] + raw['h'], raw['w'] * raw['h']\nraw.drop(['bbox'], axis=1, inplace= True)\nraw","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In object detection, we usually use a boundary box to describe the target location. The boundary box is a rectangular box that can be determined by the x and y axis coordinates in the upper-left corner and the x and y axis coordinates in the lower-right corner of the rectangle.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**Let's look at some random images with boundary boxes**","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def show_image(image_id):\n \n fig, ax = plt.subplots(1, 2, figsize = (24, 24))\n ax = ax.flatten()\n \n bbox = raw[raw['image_id'] == image_id ]\n img_path = os.path.join(DATA_DIR, image_id + '.jpg')\n \n image = cv2.imread(img_path, cv2.IMREAD_COLOR)\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32)\n image /= 255.0\n image2 = image\n \n ax[0].set_title('Original Image')\n ax[0].imshow(image)\n \n for idx, row in bbox.iterrows():\n x1 = row['xmin']\n y1 = row['ymin']\n x2 = row['xmax']\n y2 = row['ymax']\n label = row['source']\n \n cv2.rectangle(image2, (int(x1),int(y1)), (int(x2),int(y2)), (255,255,255), 2)\n font = cv2.FONT_HERSHEY_SIMPLEX\n cv2.putText(image2, label, (int(x1),int(y1-10)), font, 1, (255,255,255), 2)\n \n ax[1].set_title('Image with Bondary Box')\n ax[1].imshow(image2)\n\n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"show_image(raw.image_id.unique()[91])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"show_image(raw.image_id.unique()[1231])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"show_image(raw.image_id.unique()[3121])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Augmentations\n\n\nData augmentation is a strategy that enables practitioners to significantly increase the diversity of data available for training models, without actually collecting new data. \nIn image data, augmentation can range from basic image manipulation like color variation, Fliping, resize, or rotate image, and data augmentation can also reduce overfitting.
","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_bboxes(bboxes, col, bbox_format = 'pascal_voc', color='white'):\n for i in range(len(bboxes)):\n x_min = bboxes[i][0]\n y_min = bboxes[i][1]\n x_max = bboxes[i][2]\n y_max = bboxes[i][3]\n width = x_max - x_min\n height = y_max - y_min\n rect = patches.Rectangle((x_min, y_min), \n width, height, \n linewidth=2, \n edgecolor=color, \n facecolor='none')\n col.add_patch(rect)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def augmented_images(image, augment):\n \n fig, ax = plt.subplots(1, 2, figsize = (24, 24))\n ax = ax.flatten()\n \n image_data = raw[raw['image_id'] == image]\n bbox = image_data[['xmin', 'ymin', 'xmax', 'ymax']].astype(np.int32).values\n labels = np.ones((len(bbox), ))\n\n image = cv2.imread(os.path.join(DATA_DIR + '/{}.jpg').format(image), cv2.IMREAD_COLOR)\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32)\n image /= 255.0\n \n get_bboxes(bbox, ax[0], color='white')\n \n ax[0].set_title('Original Image with Bounding Boxes')\n ax[0].imshow(image)\n \n aug = albumentations.Compose([augment], \n bbox_params={'format': 'pascal_voc', 'label_fields':['labels']})\n\n \n aug_result = aug(image=image, bboxes=bbox, labels=labels)\n\n aug_image = aug_result['image']\n get_bboxes(aug_result['bboxes'], ax[1], color='red')\n \n ax[1].set_title('Augmented Image with Bounding Boxes')\n ax[1].imshow(aug_image)\n \n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# HorizontalFlip Augmentation\naugmented_images(raw.image_id.unique()[1230], albumentations.HorizontalFlip(p=1))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# VerticalFlip Augmentation\naugmented_images(raw.image_id.unique()[2110], albumentations.VerticalFlip(p=1))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Change Color to gray\naugmented_images(raw.image_id.unique()[1212], albumentations.ToGray(p=1))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Random Change Brightness Contrast\naugmented_images(raw.image_id.unique()[1230], albumentations.RandomBrightnessContrast(p=1))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Random Cutout Image\naugmented_images(raw.image_id.unique()[230], albumentations.Cutout(num_holes= random.randint(10, 20), max_h_size= 64, max_w_size =64, p = 1))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Data Preprocessing (Train Data)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class wheatdataset_train(Dataset):\n \n def __init__(self, dataframe, data_dir, transforms=None):\n super().__init__()\n self.df = dataframe \n self.image_list = list(self.df['image_id'].unique())\n self.image_dir = data_dir\n self.transforms = transforms\n \n def __len__(self):\n return len(self.image_list)\n \n def __getitem__(self, idx):\n \n image_id = self.image_list[idx]\n image_data = self.df.loc[self.df['image_id'] == image_id]\n boxes = torch.as_tensor(np.array(image_data[['xmin','ymin','xmax','ymax']]), \n dtype=torch.float32)\n area = torch.tensor(np.array(image_data['area']), dtype=torch.int64) \n labels = torch.ones((image_data.shape[0],), dtype=torch.int64)\n iscrowd = torch.zeros((image_data.shape[0],), dtype=torch.uint8)\n \n target = {}\n target['boxes'] = boxes\n target['area'] = area\n target['labels'] = labels\n target['iscrowd'] = iscrowd\n \n image = cv2.imread((self.image_dir + '/' + image_id + '.jpg'), cv2.IMREAD_COLOR)\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32)\n image /= 255.0\n \n if self.transforms:\n \n image_transforms = {\n 'image': image,\n 'bboxes': target['boxes'],\n 'labels': labels\n }\n \n image_transforms = self.transforms(**image_transforms)\n image = image_transforms['image']\n \n target['boxes'] = torch.as_tensor(image_transforms['bboxes'], dtype=torch.float32)\n \n return image, target","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Albumentations\n\ndef get_train_transform():\n return albumentations.Compose([\n #albumentations.Resize(p=1, height=512, width=512),\n albumentations.ToGray(p=0.5),\n albumentations.Flip(p=0.5),\n albumentations.RandomBrightnessContrast(p=0.5),\n albumentations.Cutout(num_holes= random.randint(10, 20), max_h_size= 64, max_w_size =64, p = 0.40),\n ToTensorV2(p=1.0)\n ], bbox_params={'format': 'pascal_voc', 'label_fields': ['labels']})\n\n\ndef get_test_transform():\n return albumentations.Compose([\n ToTensorV2(p=1.0)\n ])\n\n\ndef collate_fn(batch):\n return tuple(zip(*batch))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data = wheatdataset_train(raw, DATA_DIR, get_train_transform())\ntrain_dataloader = DataLoader(train_data, batch_size=16,shuffle=True, num_workers=4,collate_fn=collate_fn)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(train_data)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Create a model and training","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')\ntorch.cuda.empty_cache()\nprint(device)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def train_model():\n \n model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=True)\n num_classes = 2\n in_features = model.roi_heads.box_predictor.cls_score.in_features\n model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)\n\n return model\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"def train(data_loader, epoch):\n \n model = train_model()\n model.train()\n model.to(device)\n params = [p for p in model.parameters() if p.requires_grad]\n optimizer = torch.optim.SGD(params, lr=0.005, momentum=0.9, weight_decay=0.0005)\n model.parameters\n\n\n total_train_loss = []\n itr = 1\n\n for epoch in tqdm(range(epoch)):\n \n print(f'Epoch :{epoch + 1}')\n start_time = time.time()\n train_loss = []\n \n for images, targets in tqdm(data_loader):\n \n images = list(image.to(device) for image in images)\n targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n loss_dict = model(images, targets)\n\n losses = sum(loss for loss in loss_dict.values())\n \n loss_value = losses.item()\n \n train_loss.append(losses.item())\n optimizer.zero_grad()\n losses.backward()\n optimizer.step()\n \n if itr % 50 == 0:\n print(f\"Iteration #{itr} loss: {loss_value:.4f}\")\n\n itr += 1\n \n \n epoch_train_loss = np.mean(train_loss)\n total_train_loss.append(epoch_train_loss)\n print(f'Epoch train loss is {epoch_train_loss:.4f}')\n time_elapsed = time.time() - start_time\n print('{:.0f}m {:.0f}s'.format(time_elapsed // 60, time_elapsed % 60))\n\n torch.save(model.state_dict(), 'resnet50_GWD.pth')\n\n #visualize\n plt.figure(figsize=(12,6))\n plt.title('Train Loss', fontsize= 20)\n plt.plot(total_train_loss)\n plt.xlabel('epochs')\n plt.ylabel('loss')\n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#train(train_dataloader, num_epochs)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Inference and Compare Model","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Load test data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"test_df = pd.read_csv(test_df_path)\ntest_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"