{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Importing Libraries","metadata":{}},{"cell_type":"code","source":"import os\nimport cv2\nimport csv\nimport glob\nimport pandas as pd\nimport numpy as np\nimport random\nimport itertools\nfrom collections import Counter\nfrom math import ceil\nimport matplotlib.pyplot as plt\nfrom tqdm.notebook import tqdm\nfrom IPython.core.interactiveshell import InteractiveShell\nInteractiveShell.ast_node_interactivity = \"all\"\n%matplotlib inline","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:23:13.735111Z","start_time":"2020-05-10T15:23:13.139469Z"},"execution":{"iopub.status.busy":"2022-03-21T06:06:08.080969Z","iopub.execute_input":"2022-03-21T06:06:08.081387Z","iopub.status.idle":"2022-03-21T06:06:08.354448Z","shell.execute_reply.started":"2022-03-21T06:06:08.081315Z","shell.execute_reply":"2022-03-21T06:06:08.353447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In this kernel, I present some utility functions to do some sanity check on images, as well as some functions that you can reuse for future projects when you want to plot multiple images in a grid. A sneak peek of how a multiple bounding box plot is as such:","metadata":{}},{"cell_type":"markdown","source":"![sample](https://i.ibb.co/9GXMpWT/img.png)","metadata":{}},{"cell_type":"markdown","source":"**References**\n\n- [Paperspace DataAugmentationForObjectDetection](https://github.com/Paperspace/DataAugmentationForObjectDetection)\n\n- [ateplyuk's gwd starter](https://www.kaggle.com/ateplyuk/gwd-starter-efficientdet-train)\n\n- [Shonenkov's awesome code](https://www.kaggle.com/shonenkov/training-efficientdet)","metadata":{}},{"cell_type":"markdown","source":"# Utilities","metadata":{}},{"cell_type":"markdown","source":"Utility functions are stored here, they are useful and feel free to add these into your arsenal.","metadata":{}},{"cell_type":"code","source":"def skip_csv_header(file):\n    has_header = csv.Sniffer().has_header(file.read(1024))\n    file.seek(0)\n    if has_header:\n        next(file)\n\n\ndef total_image_list(image_folder_path):\n    total_img_list = [os.path.basename(img_path_name) for img_path_name in glob.glob(os.path.join(image_folder_path, \"*.jpg\"))]\n    return total_img_list\n\ndef draw_rect(img, bboxes, color=None):\n    img = img.copy()\n    bboxes = bboxes[:, :4]\n    bboxes = bboxes.reshape(-1, 4)\n    for bbox in bboxes:\n        pt1, pt2 = (bbox[0], bbox[1]), (bbox[2], bbox[3])\n        pt1 = int(pt1[0]), int(pt1[1])\n        pt2 = int(pt2[0]), int(pt2[1])\n        img = cv2.rectangle(img.copy(), pt1, pt2, color, int(max(img.shape[:2]) / 200))\n    return img\n\ndef plot_multiple_img(img_matrix_list, title_list, ncols, main_title=\"\"):\n    fig, myaxes = plt.subplots(figsize=(20, 15), nrows=ceil(len(img_matrix_list) / ncols), ncols=ncols, squeeze=False)\n    fig.suptitle(main_title, fontsize = 30)\n    fig.subplots_adjust(wspace=0.3)\n    fig.subplots_adjust(hspace=0.3)\n    for i, (img, title) in enumerate(zip(img_matrix_list, title_list)):\n        myaxes[i // ncols][i % ncols].imshow(img)\n        myaxes[i // ncols][i % ncols].set_title(title, fontsize=15)\n    plt.show()","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:27:14.796817Z","start_time":"2020-05-10T15:27:14.785846Z"},"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-03-21T06:06:32.731127Z","iopub.execute_input":"2022-03-21T06:06:32.731496Z","iopub.status.idle":"2022-03-21T06:06:32.750008Z","shell.execute_reply.started":"2022-03-21T06:06:32.731446Z","shell.execute_reply":"2022-03-21T06:06:32.748902Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Reading and Loading the Dataset","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv(\"/kaggle/input/global-wheat-detection/train.csv\")  \nimage_folder_path = \"/kaggle/input/global-wheat-detection/train/\"","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:23:18.556841Z","start_time":"2020-05-10T15:23:18.425088Z"},"execution":{"iopub.status.busy":"2022-03-21T06:06:46.861719Z","iopub.execute_input":"2022-03-21T06:06:46.862174Z","iopub.status.idle":"2022-03-21T06:06:47.151123Z","shell.execute_reply.started":"2022-03-21T06:06:46.862112Z","shell.execute_reply":"2022-03-21T06:06:47.150125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train","metadata":{"execution":{"iopub.status.busy":"2022-03-21T06:07:01.632495Z","iopub.execute_input":"2022-03-21T06:07:01.632902Z","iopub.status.idle":"2022-03-21T06:07:01.657542Z","shell.execute_reply.started":"2022-03-21T06:07:01.632849Z","shell.execute_reply":"2022-03-21T06:07:01.656296Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I personally like to expand the bounding box coordinates into the form of **x_min, y_min, x_max, y_max**, but currently they are stored in a list of **[x_min,y_min, width of bbox, height of bbox]**. So the next portion will help to expand them out. **This is a personal preference, in actual fact you do not need to do this, it is easier for me to normalize the bboxes**.","metadata":{}},{"cell_type":"code","source":"# train['bbox'] = train['bbox'].apply(lambda x: x[1:-1].split(\",\"))\n# train['x_min'] = train['bbox'].apply(lambda x: x[0]).astype('float32')\n# train['y_min'] = train['bbox'].apply(lambda x: x[1]).astype('float32')\n# train['width'] = train['bbox'].apply(lambda x: x[2]).astype('float32')\n# train['height'] = train['bbox'].apply(lambda x: x[3]).astype('float32')\n# train = train[['image_id','x_min', 'y_min', 'width', 'height']]\n# train[\"x_max\"] = train.apply(lambda col: col.x_min + col.width, axis=1)\n# train[\"y_max\"] = train.apply(lambda col: col.y_min + col.height, axis = 1)\n# train.head()","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:23:28.582189Z","start_time":"2020-05-10T15:23:20.780108Z"},"execution":{"iopub.status.busy":"2022-03-21T06:07:08.045993Z","iopub.execute_input":"2022-03-21T06:07:08.046417Z","iopub.status.idle":"2022-03-21T06:07:08.050922Z","shell.execute_reply.started":"2022-03-21T06:07:08.046362Z","shell.execute_reply":"2022-03-21T06:07:08.049960Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I have since used [Shonenkov's awesome code](https://www.kaggle.com/shonenkov/training-efficientdet) to make the code above more compact.","metadata":{}},{"cell_type":"code","source":"bboxes = np.stack(train['bbox'].apply(lambda x: np.fromstring(x[1:-1], sep=',')))\nfor i, column in enumerate(['x_min', 'y_min', 'width', 'height']):\n    train[column] = bboxes[:,i]\n    \ntrain[\"x_max\"] = train.apply(lambda col: col.x_min + col.width, axis=1)\ntrain[\"y_max\"] = train.apply(lambda col: col.y_min + col.height, axis = 1)\ntrain.drop(columns=['bbox'], inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-03-21T06:07:08.899869Z","iopub.execute_input":"2022-03-21T06:07:08.900396Z","iopub.status.idle":"2022-03-21T06:07:23.250785Z","shell.execute_reply.started":"2022-03-21T06:07:08.900352Z","shell.execute_reply":"2022-03-21T06:07:23.249828Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-21T06:07:23.252344Z","iopub.execute_input":"2022-03-21T06:07:23.252791Z","iopub.status.idle":"2022-03-21T06:07:23.268963Z","shell.execute_reply.started":"2022-03-21T06:07:23.252747Z","shell.execute_reply":"2022-03-21T06:07:23.267974Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Range Checking on Bounding Box Coordinates","metadata":{}},{"cell_type":"markdown","source":"Furthermore, due to python's internal floating problems, there may be weird values like negative or values that adds up to be more than 1024 in `x_max, y_max`. We need to be careful here. \n\n**This is a serious problem that one can run into when you Normalize the bounding box, it may exceed 1 and this will cause an error especially if you decide to augment the images as well.**","metadata":{}},{"cell_type":"code","source":"train[train[\"x_max\"] > 1024]\ntrain[train[\"y_max\"] > 1024]\ntrain[train[\"x_min\"] < 0]\ntrain[train[\"y_min\"] < 0]","metadata":{"execution":{"iopub.status.busy":"2022-03-21T06:07:32.976471Z","iopub.execute_input":"2022-03-21T06:07:32.976852Z","iopub.status.idle":"2022-03-21T06:07:33.013391Z","shell.execute_reply.started":"2022-03-21T06:07:32.976791Z","shell.execute_reply":"2022-03-21T06:07:33.012666Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The sole reason that for eg row 31785 has `x_max` more than 1024 is because of the original dataset's labelling. Let's look at the respective problematic rows. For example, in row 31785, the `x_min` provided is 873.200012, and when you add that to the width being 150.800003, it gives you 1024.000015, which exceeds the image size already. So you have to round down. And as far as I feel, bounding boxes, when de-normalized, should necessary be in integer. But this is just my opinion. Let's change these problematic values to 1024","metadata":{}},{"cell_type":"code","source":"x_max = np.array(train[\"x_max\"].values.tolist())\ny_max = np.array(train[\"y_max\"].values.tolist())\ntrain[\"x_max\"] = np.where(x_max > 1024, 1024, x_max).tolist()\ntrain[\"y_max\"] = np.where(y_max > 1024, 1024, y_max).tolist()","metadata":{"execution":{"iopub.status.busy":"2022-03-21T06:07:45.527771Z","iopub.execute_input":"2022-03-21T06:07:45.528546Z","iopub.status.idle":"2022-03-21T06:07:45.679915Z","shell.execute_reply.started":"2022-03-21T06:07:45.528471Z","shell.execute_reply":"2022-03-21T06:07:45.678970Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can delete width and height columns because we do not need them, it can be easily pulled out from the images itself.","metadata":{}},{"cell_type":"code","source":"del train[\"width\"]\ndel train[\"height\"]\ndel train[\"source\"]\ntrain.head()","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:23:51.619723Z","start_time":"2020-05-10T15:23:51.608205Z"},"execution":{"iopub.status.busy":"2022-03-21T06:08:13.909278Z","iopub.execute_input":"2022-03-21T06:08:13.909941Z","iopub.status.idle":"2022-03-21T06:08:13.931490Z","shell.execute_reply.started":"2022-03-21T06:08:13.909882Z","shell.execute_reply":"2022-03-21T06:08:13.930646Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I assign a class \"1\" which is the label wheat. It may be useful later on should we wish to add in images with no wheat inside the image.","metadata":{}},{"cell_type":"code","source":"train[\"class\"] = \"1\"","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:23:55.105905Z","start_time":"2020-05-10T15:23:55.101893Z"},"execution":{"iopub.status.busy":"2022-03-21T06:08:18.534237Z","iopub.execute_input":"2022-03-21T06:08:18.534615Z","iopub.status.idle":"2022-03-21T06:08:18.541777Z","shell.execute_reply.started":"2022-03-21T06:08:18.534564Z","shell.execute_reply":"2022-03-21T06:08:18.540497Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Check if image extensions are all jpg","metadata":{}},{"cell_type":"markdown","source":"First, we check if all images in the train folder are all in **.jpg** format. It is better to check because if there are a mixture of image type, we may face troubles later on.","metadata":{}},{"cell_type":"code","source":"def check_file_type(image_folder_path):\n    extension_type = []\n    file_list = os.listdir(image_folder_path)\n    for file in file_list:\n        extension_type.append(file.rsplit(\".\", 1)[1].lower())\n    print(Counter(extension_type).keys())\n    print(Counter(extension_type).values())\n    \ncheck_file_type(image_folder_path)","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:24:04.563766Z","start_time":"2020-05-10T15:24:04.552795Z"},"execution":{"iopub.status.busy":"2022-03-21T06:08:21.551641Z","iopub.execute_input":"2022-03-21T06:08:21.552092Z","iopub.status.idle":"2022-03-21T06:08:21.806941Z","shell.execute_reply.started":"2022-03-21T06:08:21.552006Z","shell.execute_reply":"2022-03-21T06:08:21.805801Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Good, seems like all our images in the folder are of **.jpg** format. Next, it is better to append **.jpg** behind all the **image_id** in the dataframe. This will make us manipulate the data easier later.","metadata":{}},{"cell_type":"code","source":"## replace image_id with .jpg behind the image_id\n# image_id_list = train[\"image_id\"].tolist()\n# image_id_append_jpg = []\n# for image_id in image_id_list:\n#     image_id_append_jpg.append(image_id + \".jpg\")\n# train[\"image_id\"] = image_id_append_jpg\n# train.head()\n\n\n## Alternatively like Rohit suggested, an one liner will do the trick\n\ntrain[\"image_id\"] = train[\"image_id\"].apply(lambda x: str(x) + \".jpg\")\ntrain.head()","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:25:04.020873Z","start_time":"2020-05-10T15:25:03.951036Z"},"execution":{"iopub.status.busy":"2022-03-21T06:08:26.194119Z","iopub.execute_input":"2022-03-21T06:08:26.194706Z","iopub.status.idle":"2022-03-21T06:08:26.278438Z","shell.execute_reply.started":"2022-03-21T06:08:26.194632Z","shell.execute_reply":"2022-03-21T06:08:26.277305Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train[\"image_id\"] = train[\"image_id\"].astype(\"str\")\n","metadata":{"ExecuteTime":{"end_time":"2020-05-10T11:45:33.459017Z","start_time":"2020-05-10T11:45:32.877548Z"},"execution":{"iopub.status.busy":"2022-03-21T06:08:36.867225Z","iopub.execute_input":"2022-03-21T06:08:36.867945Z","iopub.status.idle":"2022-03-21T06:08:36.903502Z","shell.execute_reply.started":"2022-03-21T06:08:36.867875Z","shell.execute_reply":"2022-03-21T06:08:36.902623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.to_csv(\"wheat.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2022-03-21T06:08:37.596589Z","iopub.execute_input":"2022-03-21T06:08:37.597165Z","iopub.status.idle":"2022-03-21T06:08:38.967731Z","shell.execute_reply.started":"2022-03-21T06:08:37.597111Z","shell.execute_reply":"2022-03-21T06:08:38.966630Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Check if there are corrupted images and all images are 1024 by 1024","metadata":{}},{"cell_type":"markdown","source":"Most people will use `df['width'].unique() == df['height'].unique() == [1024]` to check if all images are of 1024x1024 resolution; But we will not be 100% sure if its true in the training folder. So we won't use the same way here.","metadata":{}},{"cell_type":"code","source":"def check_image_size(image_folder_path):\n    total_img_list = glob.glob(os.path.join(image_folder_path,\"*\"))\n    counter = 0\n    for image in tqdm(total_img_list, desc = \"Checking in progress\"):\n        try:\n            img = cv2.imread(image)\n            height, width = img.shape[1], img.shape[0]\n            if not (height == 1024 and width == 1024):\n                counter = counter + 1\n        except:\n            print(\"This {} is problematic.\".format(image))\n    return counter \n        \n        ","metadata":{"ExecuteTime":{"end_time":"2020-05-10T16:03:48.307185Z","start_time":"2020-05-10T16:03:48.302222Z"},"execution":{"iopub.status.busy":"2022-03-21T06:08:41.824643Z","iopub.execute_input":"2022-03-21T06:08:41.825044Z","iopub.status.idle":"2022-03-21T06:08:41.833366Z","shell.execute_reply.started":"2022-03-21T06:08:41.824970Z","shell.execute_reply":"2022-03-21T06:08:41.832007Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"check_image_size(image_folder_path)","metadata":{"ExecuteTime":{"end_time":"2020-05-10T16:04:30.587206Z","start_time":"2020-05-10T16:03:48.465394Z"},"execution":{"iopub.status.busy":"2022-03-21T06:08:43.274725Z","iopub.execute_input":"2022-03-21T06:08:43.275434Z","iopub.status.idle":"2022-03-21T06:10:08.313104Z","shell.execute_reply.started":"2022-03-21T06:08:43.275379Z","shell.execute_reply":"2022-03-21T06:10:08.312171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Great, indeed all our images are of 1024x1024 in size. And the good thing is, this code also helps us to check for corrupted images as well, so if there is a corrupted image, it will definitely show up that the counter is non zero. And from there you can further check which image is the one causing problem.","metadata":{}},{"cell_type":"markdown","source":"# Sanity Check between train csv and train images","metadata":{}},{"cell_type":"markdown","source":"We will write a function to check if the number of **unique image_ids** match the number of unique **images** in the folder.","metadata":{}},{"cell_type":"code","source":"## our new dataset\nwheat = pd.read_csv(\"wheat.csv\") \nimage_folder_path = \"/kaggle/input/global-wheat-detection/train/\"\nimage_annotation_file = \"wheat.csv\"","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:26:25.634413Z","start_time":"2020-05-10T15:26:25.548639Z"},"execution":{"iopub.status.busy":"2022-03-21T06:10:31.632179Z","iopub.execute_input":"2022-03-21T06:10:31.632787Z","iopub.status.idle":"2022-03-21T06:10:31.754315Z","shell.execute_reply.started":"2022-03-21T06:10:31.632599Z","shell.execute_reply":"2022-03-21T06:10:31.753080Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"wheat.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-21T06:10:38.434046Z","iopub.execute_input":"2022-03-21T06:10:38.434741Z","iopub.status.idle":"2022-03-21T06:10:38.449754Z","shell.execute_reply.started":"2022-03-21T06:10:38.434688Z","shell.execute_reply":"2022-03-21T06:10:38.448644Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def sanity_tally(image_folder_path, image_annotation_file):\n    img_dict = {}\n    with open(image_annotation_file, \"r\") as file:\n        skip_csv_header(file)\n        for row in file:\n            try:\n                image_name, x_min, y_min, x_max, y_max, class_idx = row.split(\",\")\n                if image_name not in img_dict:\n                    img_dict[image_name] = list()\n                img_dict[image_name].append(\n                    [float(x_min), float(y_min), float(x_max), float(y_max), int(class_idx)]\n                )\n            except ValueError:\n                print(\"Could not convert float to string, likely that your data has empty values.\")\n        \n    img_annotation_list = [*img_dict]\n    total_img_list = total_image_list(image_folder_path)\n    if set(img_annotation_list) == set(total_img_list):\n        print(\"Sanity Check Status: True\")\n    else:\n        print(\"Sanity Check Status: Failed. \\nThe elements in wheat/train.csv but not in the train image folder is {}. \\nThe elements in train image folder but not in wheat/train.csv is {}\".format(\n                set(img_annotation_list) - set(total_img_list), set(total_img_list) - set(img_annotation_list)))\n        return list(set(img_annotation_list) - set(total_img_list)), list(set(total_img_list) - set(img_annotation_list))","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:26:26.690066Z","start_time":"2020-05-10T15:26:26.684082Z"},"execution":{"iopub.status.busy":"2022-03-21T06:11:05.189100Z","iopub.execute_input":"2022-03-21T06:11:05.189486Z","iopub.status.idle":"2022-03-21T06:11:05.205822Z","shell.execute_reply.started":"2022-03-21T06:11:05.189435Z","shell.execute_reply":"2022-03-21T06:11:05.204362Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"set_diff1, set_diff2 = sanity_tally(image_folder_path, image_annotation_file = image_annotation_file)\n\nprint(\"There are {} images without annotations in the train/wheat.csv\".format(len(set_diff2)))","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:27:19.168619Z","start_time":"2020-05-10T15:27:18.826034Z"},"execution":{"iopub.status.busy":"2022-03-21T06:11:07.929702Z","iopub.execute_input":"2022-03-21T06:11:07.930282Z","iopub.status.idle":"2022-03-21T06:11:08.455309Z","shell.execute_reply.started":"2022-03-21T06:11:07.930207Z","shell.execute_reply":"2022-03-21T06:11:08.454171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As we can see from the above, there are 49 images without bounding box annotations because they do not have wheats in the image, and hence did not appear in the **train.csv**. It might be an idea that we can put these 49 images inside the train.csv and label them as 0.","metadata":{}},{"cell_type":"markdown","source":"# Plotting Multiple Images","metadata":{}},{"cell_type":"markdown","source":"Here we define a nice function that is useful not only for this competition, but for similar project as well. Note that we used our utility function here to plot them. One can tune the parameters accordingly.","metadata":{}},{"cell_type":"code","source":"def plot_random_images(image_folder_path, image_annotation_file, num = 12):\n    img_dict = {}\n    with open(image_annotation_file, \"r\") as file:\n        skip_csv_header(file)\n        for row in file:\n            try:\n                image_name, x_min, y_min, x_max, y_max, class_idx = row.split(\",\")\n                if image_name not in img_dict:\n                    img_dict[image_name] = list()\n                img_dict[image_name].append(\n                    [float(x_min), float(y_min), float(x_max), float(y_max), int(class_idx)]\n                )\n            except ValueError:\n                print(\"Could not convert float to string, likely that your data has empty values.\")\n\n    # randomly choose 12 images to plot\n    img_files_list = np.random.choice(list(img_dict.keys()), num)\n    print(\"The images' names are {}\".format(img_files_list))\n    img_matrix_list = []\n    \n    for img_file in img_files_list:\n        image_file_path = os.path.join(image_folder_path, img_file)\n        img = cv2.imread(image_file_path)[:,:,::-1]  \n        img_matrix_list.append(img)\n\n    \n    return plot_multiple_img(img_matrix_list, title_list = img_files_list, ncols = 4, main_title=\"Wheat Images\")","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:04:54.304789Z","start_time":"2020-05-10T15:04:54.297782Z"},"execution":{"iopub.status.busy":"2022-03-21T06:11:27.637791Z","iopub.execute_input":"2022-03-21T06:11:27.638215Z","iopub.status.idle":"2022-03-21T06:11:27.651362Z","shell.execute_reply.started":"2022-03-21T06:11:27.638161Z","shell.execute_reply":"2022-03-21T06:11:27.650131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here we see a nice grid of 12 images plotted.","metadata":{}},{"cell_type":"code","source":"plot_random_images(image_folder_path, image_annotation_file, num = 12)","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:04:56.958859Z","start_time":"2020-05-10T15:04:54.560035Z"},"execution":{"iopub.status.busy":"2022-03-21T06:11:48.190348Z","iopub.execute_input":"2022-03-21T06:11:48.190714Z","iopub.status.idle":"2022-03-21T06:11:51.812959Z","shell.execute_reply.started":"2022-03-21T06:11:48.190665Z","shell.execute_reply":"2022-03-21T06:11:51.809780Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Plotting Multiples Images with Bounding Boxes","metadata":{}},{"cell_type":"markdown","source":"In object detection with bounding boxes, it is always a good idea to randomly plot some images with their bounding boxes to check for any awry bounding box coordinates. Although I have to say that in this particular competition, there are quite a lot of images with many bounding boxes and hence you have to scrutinize clearly.","metadata":{}},{"cell_type":"code","source":"def random_bbox_check(image_folder_path, image_annotation_file, num = 12):\n    img_dict = {}\n    labels = [\"wheat\", \"no wheat\"]\n    with open(image_annotation_file, \"r\") as file:\n        skip_csv_header(file)\n        for row in file:\n            try:\n                image_name, x_min, y_min, x_max, y_max, class_idx = row.split(\",\")\n                if image_name not in img_dict:\n                    img_dict[image_name] = list()\n                img_dict[image_name].append(\n                    [float(x_min), float(y_min), float(x_max), float(y_max), int(class_idx)]\n                )\n            except ValueError:\n                print(\"Could not convert float to string, likely that your data has empty values.\")\n\n    # randomly choose 12 image.\n    img_files_list = np.random.choice(list(img_dict.keys()), num)\n    print(\"The images' names are {}\".format(img_files_list))\n    image_file_path_list = []\n\n    bbox_list = []\n    img_matrix_list = []\n    random_image_matrix_list = []\n    \n    for img_file in img_files_list:\n        image_file_path = os.path.join(image_folder_path, img_file)\n        img = cv2.imread(image_file_path)[:,:,::-1]  \n        height, width, channels = img.shape\n        bbox_list.append(img_dict[img_file])\n        img_matrix_list.append(img)\n\n    \n    final_bbox_list = []\n    for bboxes, img in zip(bbox_list, img_matrix_list):\n        final_bbox_array = np.array([])\n        #bboxes is a 2d array [[...], [...]]\n        for bbox in bboxes:\n            bbox = np.array(bbox).reshape(1,5)\n            final_bbox_array = np.append(final_bbox_array, bbox)\n        final_bbox_array = final_bbox_array.reshape(-1,5)\n        random_image = draw_rect(img.copy(), final_bbox_array.copy(), color = (255,0,0))\n        random_image_matrix_list.append(random_image)\n    plot_multiple_img(random_image_matrix_list, title_list = img_files_list, ncols = 4, main_title=\"Bounding Box Wheat Images\")    \n    \n\n","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:05:03.610126Z","start_time":"2020-05-10T15:05:03.599127Z"},"execution":{"iopub.status.busy":"2022-03-21T06:11:52.827541Z","iopub.execute_input":"2022-03-21T06:11:52.827954Z","iopub.status.idle":"2022-03-21T06:11:52.845397Z","shell.execute_reply.started":"2022-03-21T06:11:52.827892Z","shell.execute_reply":"2022-03-21T06:11:52.844381Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"random_bbox_check(image_folder_path, image_annotation_file)","metadata":{"ExecuteTime":{"end_time":"2020-05-10T15:05:06.915322Z","start_time":"2020-05-10T15:05:03.72127Z"},"execution":{"iopub.status.busy":"2022-03-21T06:11:53.623781Z","iopub.execute_input":"2022-03-21T06:11:53.624238Z","iopub.status.idle":"2022-03-21T06:11:57.403721Z","shell.execute_reply.started":"2022-03-21T06:11:53.624173Z","shell.execute_reply":"2022-03-21T06:11:57.400770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Augmentations","metadata":{}},{"cell_type":"markdown","source":"Augmentation is an important technique to artifically boost your data size. In particular, when the dataset is small, augmentation prior to training the model will help the network to learn better.","metadata":{}},{"cell_type":"code","source":"# Albumentations\nimport albumentations as A","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_folder_path = \"/kaggle/input/global-wheat-detection/train/\"\nchosen_image = cv2.imread(os.path.join(image_folder_path, \"1ee6b9669.jpg\"))[:,:,::-1]\nplt.imshow(chosen_image)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And if you are interested in just visualizing one certain image's bounding box plot, you can first extract the chosen image's dataframe, and convert the bounding box of the image into a **2d-array**. Then apply the `draw_rect` function to plot.","metadata":{}},{"cell_type":"code","source":"chosen_image_dataframe = wheat.loc[wheat[\"image_id\"]==\"1ee6b9669.jpg\",[\"x_min\",\"y_min\",\"x_max\",\"y_max\",\"class\"]]\nbbox_array_of_chosen_image = np.array(chosen_image_dataframe.values.tolist())\nbbox_array_of_chosen_image.shape\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"draw_chosen_image = draw_rect(chosen_image.copy(), bbox_array_of_chosen_image.copy(), color = (255,0,0))\nplt.imshow(draw_chosen_image)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Below are some snippets of augmentation types you can use, interestingly, Albumentation offers `RandomSunFlare`,`RandomFog` and `RandomSnow`; although all the images seem to be taken in a very good lighting, but it might not be that bad an idea since in the real world, images of wheat may taken in ***different weather conditions.***","metadata":{}},{"cell_type":"code","source":"albumentation_list = [A.RandomSunFlare(p=1), A.RandomFog(p=1), A.RandomBrightness(p=1),\n                      A.RandomCrop(p=1,height = 512, width = 512), A.Rotate(p=1, limit=90),\n                      A.RGBShift(p=1), A.RandomSnow(p=1),\n                      A.HorizontalFlip(p=1), A.VerticalFlip(p=1), A.RandomContrast(limit = 0.5,p = 1),\n                      A.HueSaturationValue(p=1,hue_shift_limit=20, sat_shift_limit=30, val_shift_limit=50)]\n\nimg_matrix_list = []\nbboxes_list = []\nfor aug_type in albumentation_list:\n    img = aug_type(image = chosen_image)['image']\n    img_matrix_list.append(img)\n\nimg_matrix_list.insert(0,chosen_image)    \n\ntitles_list = [\"Original\",\"RandomSunFlare\",\"RandomFog\",\"RandomBrightness\",\n               \"RandomCrop\",\"Rotate\", \"RGBShift\", \"RandomSnow\",\"HorizontalFlip\", \"VerticalFlip\", \"RandomContrast\",\"HSV\"]\n\n##reminder of helper function\ndef plot_multiple_img(img_matrix_list, title_list, ncols, main_title=\"\"):\n    fig, myaxes = plt.subplots(figsize=(20, 15), nrows=3, ncols=ncols, squeeze=False)\n    fig.suptitle(main_title, fontsize = 30)\n    fig.subplots_adjust(wspace=0.3)\n    fig.subplots_adjust(hspace=0.3)\n    for i, (img, title) in enumerate(zip(img_matrix_list, title_list)):\n        myaxes[i // ncols][i % ncols].imshow(img)\n        myaxes[i // ncols][i % ncols].set_title(title, fontsize=15)\n    plt.show()\n    \nplot_multiple_img(img_matrix_list, titles_list, ncols = 4,main_title=\"Different Types of Augmentations\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Bounding Boxes with Albumentations","metadata":{}},{"cell_type":"markdown","source":"Recall we are using our **chosen image** as example, for convenience, I will remind you of the chosen images image matrix and its bounding boxes coordinates below. But there is a caveat here, my bounding boxes array is of shape [N,5], where the last element is the labels. But when you want to use Albumentations to plot bounding boxes, it takes in bboxes in the format of `pascal_voc` which is **[x_min, y_min, x_max, y_max]**; it also takes in `label_fields` which are the labels for each bounding box. So we still need to do some simple preprocessing below.","metadata":{}},{"cell_type":"code","source":"chosen_image = cv2.imread(os.path.join(image_folder_path, \"1ee6b9669.jpg\"))[:,:,::-1]\nchosen_image_dataframe = wheat.loc[wheat[\"image_id\"]==\"1ee6b9669.jpg\",[\"x_min\",\"y_min\",\"x_max\",\"y_max\"]]\nbbox_array_of_chosen_image = np.array(chosen_image_dataframe.values.tolist())\nlabels_of_chosen_image = np.ones((len(bbox_array_of_chosen_image),))\n\n# A caution here, this competition has all labels to be 1, so a neat trick is just to use np.ones to populate the label fields.\n# However, when dealing with multiple classes, you should not do this and instead just take the labels from the dataframe accordingly.\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**References:** [Albumentation Documentations](https://github.com/albumentations-team/albumentations_examples/tree/master/notebooks)","metadata":{}},{"cell_type":"code","source":"def draw_rect_with_labels(img, bboxes,class_id, class_dict, color=None):\n    img = img.copy()\n    bboxes = bboxes[:, :4]\n    bboxes = bboxes.reshape(-1, 4)\n    for bbox, label in zip(bboxes, class_id):\n        pt1, pt2 = (bbox[0], bbox[1]), (bbox[2], bbox[3])\n        pt1 = int(pt1[0]), int(pt1[1])\n        pt2 = int(pt2[0]), int(pt2[1])\n        class_name = class_dict[label]\n        ((text_width, text_height), _) = cv2.getTextSize(class_name, cv2.FONT_HERSHEY_SIMPLEX, 0.35, 1) \n        img = cv2.rectangle(img.copy(), pt1, pt2, color, int(max(img.shape[:2]) / 200))\n        img = cv2.putText(img.copy(), class_name, (int(bbox[0]), int(bbox[1]) - int(0.3 * text_height)), cv2.FONT_HERSHEY_SIMPLEX,fontScale=1,color = (255,255,255), lineType=cv2.LINE_AA)\n    return img","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Vertical Flip","metadata":{}},{"cell_type":"code","source":"ver_flip = A.Compose([\n        A.VerticalFlip(p=1),\n    ], bbox_params={'format': 'pascal_voc', 'label_fields': ['labels']})\n\n\nver_flip_annotations = ver_flip(image=chosen_image, bboxes=bbox_array_of_chosen_image, labels=labels_of_chosen_image)\nver_flip_annotations['bboxes'] = [list(bbox) for bbox in ver_flip_annotations['bboxes']]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ver_flip_img = draw_rect_with_labels(img = ver_flip_annotations['image'], bboxes = np.array(ver_flip_annotations['bboxes']),\n                          class_id = ver_flip_annotations['labels'], class_dict = {0: \"background\",1: \"wheat\"}, color=(255,0,0))\n\n## using my good old plotting functions\ndef plot_multiple_img(img_matrix_list, title_list, ncols, main_title=\"\"):\n    fig, myaxes = plt.subplots(figsize=(20, 15), nrows=1, ncols=ncols, squeeze=False)\n    fig.suptitle(main_title, fontsize = 30)\n    fig.subplots_adjust(wspace=0.3)\n    fig.subplots_adjust(hspace=0.3)\n    for i, (img, title) in enumerate(zip(img_matrix_list, title_list)):\n        myaxes[i // ncols][i % ncols].imshow(img)\n        myaxes[i // ncols][i % ncols].set_title(title, fontsize=15)\n    plt.show()\n    \nimg_matrix_list = [draw_chosen_image, ver_flip_img]\ntitles_list = [\"Original\", \"VerticalFlipped\"]\n\nplot_multiple_img(img_matrix_list, titles_list, ncols = 2,main_title=\"Vertical Flip\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Horizontal Flip","metadata":{}},{"cell_type":"code","source":"hor_flip = A.Compose([\n        A.HorizontalFlip(p=1),\n    ], bbox_params={'format': 'pascal_voc', 'label_fields': ['labels']})\n\n\nhor_flip_annotations = hor_flip(image=chosen_image, bboxes=bbox_array_of_chosen_image, labels=labels_of_chosen_image)\nhor_flip_annotations['bboxes'] = [list(bbox) for bbox in hor_flip_annotations['bboxes']]\n\n\nhor_flip_img = draw_rect_with_labels(img = hor_flip_annotations['image'], bboxes = np.array(hor_flip_annotations['bboxes']),\n                          class_id = hor_flip_annotations['labels'], class_dict = {0: \"background\",1: \"wheat\"}, color=(255,0,0))\n    \nimg_matrix_list = [draw_chosen_image, hor_flip_img]\ntitles_list = [\"Original\", \"HorizontalFlipped\"]\n\nplot_multiple_img(img_matrix_list, titles_list, ncols = 2,main_title=\"Horizontal Flip\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# An updated tutorial on Albumentations","metadata":{}},{"cell_type":"markdown","source":"Recently, Albumentations updated their website and has more thorough walkthrough on applying their augmentations. \n\nThe link is [here](https://albumentations.ai/docs/).","metadata":{}},{"cell_type":"markdown","source":"## Image Augmentation for classification","metadata":{}},{"cell_type":"markdown","source":"The below are purely for my documentation and learning experience so most of the content are copied verbatim from [Albumentation's website](https://albumentations.ai/docs/getting_started/image_augmentation/).","metadata":{}},{"cell_type":"markdown","source":"**Step 1: Import the required libraries**\n\n    import cv2\n    import albumentations as A\n\n**Step 2: Define an augmentation pipeline.**\n\nTo define an augmentation pipeline, you need to create an instance of the `Compose` class. As an argument to the `Compose` class, you need to pass a list of augmentations you want to apply. A call to `Compose` will return a transform function that will perform image augmentation.\n\nLet's look at an example:\n\n    transform = A.Compose([\n        A.RandomCrop(width=256, height=256),\n        A.HorizontalFlip(p=0.5),\n        A.RandomBrightnessContrast(p=0.2),\n    ])\n\nIn the example, `Compose` receives a list with three augmentations: `A.RandomCrop`, `A.HorizontalFlip`, and `A.RandomBrighntessContrast`.\n\nTo create an augmentation, you create an instance of the required augmentation class and pass augmentation parameters to it. `A.RandomCrop` receives two parameters, height and width. `A.RandomCrop(width=256, height=256)` means that `A.RandomCrop` will take an input image, extract a random patch with size 256 by 256 pixels from it and then pass the result to the next augmentation in the pipeline (in this case to `A.HorizontalFlip`).\n\n`A.HorizontalFlip` in this example has one parameter named `p`. `p` is a special parameter that is supported by almost all augmentations. It controls the probability of applying the augmentation. `p=0.5` means that with a probability of 50%, the transform will flip the image horizontally, and with a probability of 50%, the transform won't modify the input image.\n\n`A.RandomBrighntessContrast` in the example also has one parameter, `p`. With a probability of 20%, this augmentation will change the brightness and contrast of the image received from `A.HorizontalFlip`. And with a probability of 80%, it will keep the received image unchanged.\n\nThe following picture depicts the `Compose` idea wonderfully.\n\nALSO THIS [WEBSITE IS AMAZING FOR YOU TO TUNE AND VISUALIZE YOUR AUGMENTATIONS](https://albumentations-demo.herokuapp.com/).","metadata":{}},{"cell_type":"markdown","source":"![augmentation-pipeline](https://albumentations.ai/docs/images/getting_started/augmenting_images/augmentation_pipeline_visualized.jpg)","metadata":{}},{"cell_type":"markdown","source":"Now we will use the above context on our wheat example.","metadata":{}},{"cell_type":"code","source":"transform = A.Compose([\n    A.CoarseDropout(max_height=100, max_width=100, p = 1),\n    A.RandomBrightnessContrast(p=0.9),\n    A.HueSaturationValue(\n                        hue_shift_limit=0.2,\n                        sat_shift_limit=0.2,\n                        val_shift_limit=0.2,\n                        p=0.9,\n                        )\n])\nchosen_image = cv2.imread(os.path.join(image_folder_path, \"1ee6b9669.jpg\"))[:,:,::-1]\naugmented_image = transform(image=chosen_image)['image']\nplt.imshow(augmented_image)","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}