{"cells":[{"metadata":{"_uuid":"a201cfe2-f1dd-48d2-a896-627cdcb207b1","_cell_guid":"1d427f1f-a097-49f4-be58-3f9d39366a77","trusted":true},"cell_type":"markdown","source":"# Your first submission on Kaggle for the Global Wheat Detection challenge !\n\n> If it is your first Kaggle competition, it is the right place to start your journey toward counting wheat head like a jedi !\n\nYou can find way better notebook to get an insight to the data. An important part is that the origin is available is the train.csv. In the hidden test set, the images comes from different origins. It still wheat head but the data are \"out of distribution\" (o.o.d), when usually the test set is usually independantly and identically distributed. (a very nice introduction on the difference between the two concepts are described here: https://arxiv.org/abs/2004.07780\n\nThe code for training the baseline Faster-RCNN comes from https://www.kaggle.com/pestipeti/pytorch-starter-fasterrcnn-train \n\nFirst difficulty: for the moment, access to internet is disabled so you need to save your model in a dataset and let your submission notebook access it. Dataset can be private. You just have to click on \"File\" then \"Add Data\". To run this tutorial, don't forget to add: https://www.kaggle.com/bendvd/torchvisionfasterrcnn\n\nThe submission file needs a prediction string. When there is more than one box to predict, just join the different box string with a space (\"confidence_1 x_min_1 y_min_1 x_max_1 y_max_1 confidence_2 x_min_2 y_min_2 x_max_2 y_max_2 etc...\")\n\nI hope it's clear ! Don't hesitate to ask more question below !\n\n(If you think it was easy for me to create the tutorial... I took me 12 attempts to get it right ! )","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"\n\n![alt text](http://www.global-wheat.com/wp-content/uploads/2020/04/ILLU_01_EN-1024x982.jpg \"Welcome to the dark side of the force !\")\n","execution_count":null},{"metadata":{"_uuid":"ddaffa4a-8183-4c8c-8313-7f6f0aeac8cb","_cell_guid":"28e5669f-7419-4aad-b560-97e77ea71763","trusted":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport cv2\nimport os\nimport re\n\nfrom PIL import Image\n\nimport albumentations as A\nfrom albumentations.pytorch.transforms import ToTensorV2\n\nimport torch\nimport torchvision\n\nfrom torchvision.models.detection.faster_rcnn import FastRCNNPredictor\nfrom torchvision.models.detection import FasterRCNN\nfrom torchvision.models.detection.rpn import AnchorGenerator\n\nfrom torch.utils.data import DataLoader, Dataset\nfrom torch.utils.data.sampler import SequentialSampler\n\nfrom matplotlib import pyplot as plt\n\n\nDIR_INPUT = '../input/global-wheat-detection'\nDIR_TRAIN = f'{DIR_INPUT}/train'\nDIR_TEST = f'{DIR_INPUT}/test'\n\n\ndef expand_bbox(x):\n    r = np.array(re.findall(\"([0-9]+[.]?[0-9]*)\", x))\n    if len(r) == 0:\n        r = [-1, -1, -1, -1]\n    return r\n\nclass WheatDataset(Dataset):\n\n    def __init__(self, dataframe, image_dir, transforms=None):\n        super().__init__()\n\n        self.image_ids = dataframe['image_id'].unique()\n        self.df = dataframe\n        self.image_dir = image_dir\n        self.transforms = transforms\n\n    def __getitem__(self, index: int):\n\n        image_id = self.image_ids[index]\n        records = self.df[self.df['image_id'] == image_id]\n\n        image = cv2.imread(f'{self.image_dir}/{image_id}.jpg', cv2.IMREAD_COLOR)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32)\n        image /= 255.0\n\n        boxes = records[['x', 'y', 'w', 'h']].values\n        boxes[:, 2] = boxes[:, 0] + boxes[:, 2]\n        boxes[:, 3] = boxes[:, 1] + boxes[:, 3]\n        \n        area = (boxes[:, 3] - boxes[:, 1]) * (boxes[:, 2] - boxes[:, 0])\n        area = torch.as_tensor(area, dtype=torch.float32)\n\n        # there is only one class\n        labels = torch.ones((records.shape[0],), dtype=torch.int64)\n        \n        # suppose all instances are not crowd\n        iscrowd = torch.zeros((records.shape[0],), dtype=torch.int64)\n        \n        target = {}\n        target['boxes'] = boxes\n        target['labels'] = labels\n        # target['masks'] = None\n        target['image_id'] = torch.tensor([index])\n        target['area'] = area\n        target['iscrowd'] = iscrowd\n\n        if self.transforms:\n            sample = {\n                'image': image,\n                'bboxes': target['boxes'],\n                'labels': labels\n            }\n            sample = self.transforms(**sample)\n            image = sample['image']\n            \n            target['boxes'] = torch.stack(tuple(map(torch.tensor, zip(*sample['bboxes'])))).permute(1, 0)\n\n        return image, target, image_id\n\n    def __len__(self) -> int:\n        return self.image_ids.shape[0]\n\nclass Averager:\n    def __init__(self):\n        self.current_total = 0.0\n        self.iterations = 0.0\n\n    def send(self, value):\n        self.current_total += value\n        self.iterations += 1\n\n    @property\n    def value(self):\n        if self.iterations == 0:\n            return 0\n        else:\n            return 1.0 * self.current_total / self.iterations\n\n    def reset(self):\n        self.current_total = 0.0\n        self.iterations = 0.0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_img_number = -1 # I set here a very low number of image for faster training\n\ntrain_df = pd.read_csv(f'{DIR_INPUT}/train.csv')\ntrain_df.shape\ntrain_df['x'] = -1\ntrain_df['y'] = -1\ntrain_df['w'] = -1\ntrain_df['h'] = -1\n\ntrain_df[['x', 'y', 'w', 'h']] = np.stack(train_df['bbox'].apply(lambda x: expand_bbox(x)))\ntrain_df.drop(columns=['bbox'], inplace=True)\ntrain_df['x'] = train_df['x'].astype(np.float)\ntrain_df['y'] = train_df['y'].astype(np.float)\ntrain_df['w'] = train_df['w'].astype(np.float)\ntrain_df['h'] = train_df['h'].astype(np.float)\nimage_ids = train_df['image_id'].unique()\nvalid_ids = image_ids[train_img_number:]\ntrain_ids = image_ids[:train_img_number]\nvalid_df = train_df[train_df['image_id'].isin(valid_ids)]\ntrain_df = train_df[train_df['image_id'].isin(train_ids)]\nvalid_df.shape, train_df.shape\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"As you can notice, the train dataset contains very few annotations ! If you want to win, you will have to change that !","execution_count":null},{"metadata":{"_uuid":"c3d93a46-fe4a-4bbb-b0a0-423e2d8da7a3","_cell_guid":"929d6c6c-9ba3-4a63-99df-a5f7de06b5f1","trusted":true},"cell_type":"code","source":"# load a model; pre-trained on COCO\n# Internet is currently disabled so you have to create a dataset to save weight before submitting ! \n\nmodel = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained_backbone=False)\nmodel.load_state_dict(torch.load(\"../input/torchvisionfasterrcnn/fasterrcnn_resnet50_fpn.pth\"))\n#torch.save(model.state_dict(),\"fasterrcnn_resnet50_fpn.pth\")\n\n\nnum_classes = 2  # 1 class (wheat) + background\n# get number of input features for the classifier\nin_features = model.roi_heads.box_predictor.cls_score.in_features\n\n# replace the pre-trained head with a new one\nmodel.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1eb173f4-ea66-42c6-95f7-2d60341d3ebb","_cell_guid":"fa5cb5d7-7634-4676-82da-312c94db0937","trusted":true},"cell_type":"code","source":"# Albumentations\ndef get_train_transform():\n    return A.Compose([\n        A.Flip(0.5),\n        ToTensorV2(p=1.0)\n    ], bbox_params={'format': 'pascal_voc', 'label_fields': ['labels']})\n\ndef get_valid_transform():\n    return A.Compose([\n        ToTensorV2(p=1.0)\n    ], bbox_params={'format': 'pascal_voc', 'label_fields': ['labels']})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Train\n\nHere you have the code for training. In some papers, instead of putting the whole image, random patches of 512 px X 512 px are sampled from the original images. for prediction, one image is converted to a set of overlapping images and the prediction are merged afterwards","execution_count":null},{"metadata":{"_uuid":"8733d23d-14ea-4eed-ac75-f19d7d6f4abf","_cell_guid":"0c78fe71-284d-4379-83e8-082e11ff235d","trusted":true},"cell_type":"code","source":"def collate_fn(batch):\n    return tuple(zip(*batch))\n\ntrain_dataset = WheatDataset(train_df, DIR_TRAIN, get_train_transform())\nvalid_dataset = WheatDataset(valid_df, DIR_TRAIN, get_valid_transform())\n\n\n# split the dataset in train and test set\nindices = torch.randperm(len(train_dataset)).tolist()\n\ntrain_data_loader = DataLoader(\n    train_dataset,\n    batch_size=16,\n    shuffle=False,\n    num_workers=4,\n    collate_fn=collate_fn\n)\n\nvalid_data_loader = DataLoader(\n    valid_dataset,\n    batch_size=8,\n    shuffle=False,\n    num_workers=4,\n    collate_fn=collate_fn\n)\n\ndevice = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')\n\nmodel.to(device)\nparams = [p for p in model.parameters() if p.requires_grad]\noptimizer = torch.optim.SGD(params, lr=0.005, momentum=0.9, weight_decay=0.0005)\n# lr_scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=3, gamma=0.1)\nlr_scheduler = None\n\nnum_epochs = 10\n\nloss_hist = Averager()\nitr = 1000\n\nfor epoch in range(num_epochs):\n    loss_hist.reset()\n    \n    for images, targets, image_ids in train_data_loader:\n        \n        images = list(image.to(device) for image in images)\n        targets = [{k: v.to(device) for k, v in t.items()} for t in targets]\n\n        loss_dict = model(images, targets)\n\n        losses = sum(loss for loss in loss_dict.values())\n        loss_value = losses.item()\n\n        loss_hist.send(loss_value)\n\n        optimizer.zero_grad()\n        losses.backward()\n        optimizer.step()\n\n        if itr % 25 == 0:\n            print(f\"Iteration #{itr} loss: {loss_value}\")\n\n        itr += 1\n    \n    # update the learning rate\n    if lr_scheduler is not None:\n        lr_scheduler.step()\n\n    print(f\"Epoch #{epoch} loss: {loss_hist.value}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 4- Make your submission","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"i like to have a function that describes well how the prediction is done. It would be way faster if done per batch. Test Time Augmentation would be a very nice idea to test !","execution_count":null},{"metadata":{"_uuid":"433cc43a-6fa8-47c6-95ee-59f65690fb10","_cell_guid":"203e871c-1c2f-4f6c-8068-94301f36b0df","trusted":true},"cell_type":"code","source":"\ndef model_prediction(image_path,model,device):\n    model.eval()\n    image = cv2.imread(image_path, cv2.IMREAD_COLOR)\n    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.float32)\n    image /= 255.0\n    images = torch.from_numpy(image).float().permute(2,0,1).unsqueeze(0).to(device)\n    outputs = model(images)\n\n\n    outputs = [{k: v.detach().cpu().numpy() for k, v in t.items()} for t in outputs]\n    boxes = outputs[0][\"boxes\"]\n    scores = outputs[0][\"scores\"]\n    valid_boxes = boxes[scores > 0.5]\n    valid_scores = scores[scores > 0.5]\n    return valid_boxes, valid_scores","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0b9cf11d-a6b9-47ca-816c-4952c8b801e0","_cell_guid":"ad8472f3-83d3-4a39-93ba-fb1b005363d6","trusted":true},"cell_type":"code","source":"submission = pd.read_csv(\"/kaggle/input/global-wheat-detection/sample_submission.csv\")\nsubmission.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The expected output is then a two columns csv. The first column contains the image_id related to the prediction. PredictionString follow the format \"{confidence} {x} {y} {h} {w}\" where confidence is a float, and x,y,h and w are integer.","execution_count":null},{"metadata":{"_uuid":"9fb694ca-b0f5-4627-968f-572d12c217d8","_cell_guid":"a07c23cf-affe-4dfe-ac9f-c51425ecd968","trusted":true},"cell_type":"code","source":"from tqdm import tqdm\nfrom pathlib import Path\ndata_dir = '../input/global-wheat-detection/test'\n\nsubmission = pd.read_csv(f'{DIR_INPUT}/sample_submission.csv')\n\n\n\nroot_image = Path(\"../input/global-wheat-detection/test\")\ntest_images = [root_image / f\"{img}.jpg\" for img in submission.image_id]\n\n\nsubmission = []\nmodel.eval()\n\nfor image in tqdm(test_images):\n    boxes, scores = model_prediction(str(image),model,device)\n    prediction_string = []\n    for (x_min,y_min,x_max,y_max),s in zip(boxes,scores):\n        x = round(x_min)\n        y = round(y_min)\n        h = round(x_max-x_min)\n        w = round(y_max-y_min)\n        prediction_string.append(f\"{s} {x} {y} {h} {w}\")\n    prediction_string = \" \".join(prediction_string)\n    \n    submission.append([image.name[:-4],prediction_string])\n\nsample_submission = pd.DataFrame(submission, columns=[\"image_id\",\"PredictionString\"])\nsample_submission.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"1- When your notebook or script is done, please commit it\n\n2- Then go on https://www.kaggle.com/your-name/your-notebook (remove edit/run)\n\n3- There is an output tab, you can click on it after committing\n\n4- You will find a \"submit to competition\" button :) \n\n5- Grab a coffee...\n\n6- You're on the leaderboard ! ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}