{ "id": 164140, "title": "Some insights", "url": "/competitions/global-wheat-detection/discussion/164140", "author_name": "", "post_date": "2020-07-04T21:33:58.953427700Z", "votes": 65, "comment_count": 7, "views": 0, "content": "

A new competition, a new insights post! Here are some:

\n\n

I hope this is useful, I will keep adding new things along the way. Enjoy the competition!

", "messages": [ { "id": "915574", "postDate": "07/04/2020 21:33:58", "content": "

A new competition, a new insights post! Here are some:

\n\n

I hope this is useful, I will keep adding new things along the way. Enjoy the competition!

", "rawMarkdown": "A new competition, a new insights post! Here are some: \n\n\n\n* Train images: \n\n * There is a toal of **3422** unique train images. The code to get this number:\n ```\n from pathlib import Path\n train_paths = Path(\"/path/to/train\")\n len(list(train_paths.glob(\"*\")))\n ```\n * There is a total of **3373** unique images with masks\n * Thus some images don't have any masks\n * There is a toal of **147793** masks\n * Thus, on average, there are **43.8** masks per image\n * The image with the most masks contains **116**. It is the image with id `35b935b6c`.\n * All the train images have the same size: 1024 x 1024. \n * There are 3 channels: R, G, B.\n * The train images come from **7** different sources. Here is the distribution: \n\n ``` \n {'arvalis_1': 1055,\n 'arvalis_2': 204,\n 'arvalis_3': 559,\n 'ethz_1': 747,\n 'inrae_1': 176,\n 'rres_1': 432,\n 'usask_1': 200}\n ``` \n * **3** of these sources are probably similar: *arvalis_1*, *arvalis_2*, and *arvalis_3*. Most likely, the breakdown is by the competition sponsors/organisers?\n * Bounding boxes where added using this [tool](https://github.com/jsbroks/coco-annotator).\n * You can find resized datasets [here](https://www.kaggle.com/c/global-wheat-detection/discussion/160408).\n * There are some data quality issues: masks [too big](https://www.kaggle.com/c/global-wheat-detection/discussion/159578), or [missing masks](https://www.kaggle.com/c/global-wheat-detection/discussion/163263). The organizers are working on a fix for the private test dataset (as far as I know).\n * The average bounding box area is **6843**, the smallest is only **2**, and the largest is **529788**! Here attached the two images with the largest (upper right including many smaller) and the smallest (it is the in the lower right corner, at the edge) bounding boxes: \n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2F51306c59989506fbc085778805afaea0%2F42e6efaaa_with_bbox.png?generation=1594240530216553&alt=media)\n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2Fb4859bccc9fe7de7d7a19385479ab34f%2Fd0912a485_with_bbox.png?generation=1594240509985483&alt=media)\n\n\n * The bbox area histogram with the top 1000 values truncated and the bottom 100 as well:\n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2F6397d02cd4ada74403ae0eaa955394ce%2Fhist_truncated.png?generation=1594242665230885&alt=media)\n \n\n * For more details about the dataset creation, check this [paper](https://arxiv.org/pdf/2005.02162.pdf). It contains a lot of statistics and data acquisition methodolgy descritption, so make sure to check it. Here are some graphs from the paper: \n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2F063dea7e597defdd1b44fdd3bab843be%2Fwheat_dataset_table.png?generation=1593898190270563&alt=media)\n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2F518f78637d9026e7ee76380745749a93%2Foriginal_wheat_images.png?generation=1593898211077093&alt=media)\n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2F6dd629a68c99e0f840a8c446a8bc1092%2Fwheat_table_statistics.png?generation=1593898390536691&alt=media)\n\n\n\n* Test images:\n\n * Only **10** images have been made public. The code to get this: \n ```\n from pathlib import Path\n test_paths = Path(\"/path/to/test\")\n len(list(test_paths.glob(\"*\")))\n ```\n * Don't worry though, there is a \"real\" test dataset that runs when you submit your inference kernel.\n * The test dataset has been collected from different sources than the train. This is an important aspect of the competition for sure.\n * There are few images in the test dataset that aren't 1024 x 1024. These will be fixed in the private part as far as I know.\n * More insights about the test dataset could be found in [this](https://www.kaggle.com/c/global-wheat-detection/discussion/149770) discussion. \n\n* Short code snippets\n\n\n * Load an image and overlay the corresponding mask(s):\n\n ```\n from pathlib import Path\n\n img_id = \"ID_OF_IMAGE\"\n img_path = Path(f\"/path/to/train/{img_id}.jpg\")\n img = Image.open(img_path)\n draw = ImageDraw.Draw(img)\n bboxes = train_labels_df.loc[lambda df: df[\"image_id\"] == img_id, \"bbox\"].tolist()\n # The box contains the upper left corner (x, y) coordinates then width and height.\n # So we need to change these to (x1, y1) and (x2, y2) where they are the upper \n # left and lower right corners\n for bbox in bboxes:\n x, y, w, h = eval(bbox)\n transformed_bbox = [x, y, x + w, y + h]\n draw.rectangle(transformed_bbox, outline=\"black\", width=3)\n img.save(\"/path/to/output/{img_id}.png\")\n img\n ```\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2Fe56c560fc347f24e44a9840c3afbf61c%2Fimag_with_bbox.png?generation=1593939578542921&alt=media)\n\n * Extracting bounding boxes coordinates, width, height, and area: \n ```\n import pandas as pd\n train_labels_df = pd.read_csv(\"path/to/train/labels.csv\")\n \n x = []\n y = []\n w = []\n h = []\n for bbox in train_labels_df['bbox']:\n bbox = eval(bbox)\n x.append(bbox[0])\n y.append(bbox[1])\n w.append(bbox[2])\n h.append(bbox[3])\n\n processed_train_labels_df = train_labels_df.copy()\n processed_train_labels_df[\"x\"] = x\n processed_train_labels_df[\"y\"] = y\n processed_train_labels_df[\"w\"] = w\n processed_train_labels_df[\"h\"] = h\n\n processed_train_labels_df[\"area\"] = processed_train_labels_df[\"w\"] * processed_train_labels_df[\"h\"]\n processed_train_labels_df[\"x2\"] = processed_train_labels_df[\"x\"] + processed_train_labels_df[\"w\"]\n processed_train_labels_df[\"y2\"] = processed_train_labels_df[\"y\"] + processed_train_labels_df[\"h\"]\n ```\n\n\n\n* Metric\n\n * There are many components to this metric: first, we compute Jaccard/[IoU](https://www.pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/) scores, then for some threshold `t`, the wheat head is a TP if the IoU is above it, finally we compute the [precision](https://en.wikipedia.org/wiki/Precision_and_recall) for this threshold. Then, average over the thresholds (given the AP) and over images (given mAP, mean average prediction).\n * Thus, there are two levels of averaging: over the thresholds and then over the images. \n * Notice that there are some edge-cases where the precision will be 0: no mask is predicted and there is at least one, a mask is predicted but there aren't any.\n\n* Some backbone models that work well for object detection: \n\n * [YOLOv5](https://github.com/ultralytics/yolov5)\n * [Detectron2](https://github.com/facebookresearch/detectron2): this isn't a single model but rather a collection of object detection models. It contains Faster R-CNN, RetinaNet and many more. \n\n* Previous similar competitions:\n\n * [TGS Salt Identification](https://www.kaggle.com/c/tgs-salt-identification-challenge/overview). This is an object detection competition with exactly the same metric. Here is a link to the [first place](https://www.kaggle.com/c/tgs-salt-identification-challenge/discussion/69291) solution, the code [repo](https://github.com/ybabakhin/kaggle_salt_bes_phalanx), and a [paper](https://arxiv.org/pdf/1904.04445.pdf) published.\n * [Severstal: Steel Defect Detection](https://www.kaggle.com/c/severstal-steel-defect-detection/). This isn't exactly an object detection competition but rather an **semantic segmentation** one. Here is a link the [first place](https://www.kaggle.com/c/severstal-steel-defect-detection/discussion/114254) solution. \n * [Airbus Ship Detection](https://www.kaggle.com/c/airbus-ship-detection). Another instance segmentation competition. Here is the 4th place solution. The metric here is close to the current competition but instead of AP, it uses [F2](https://www.quora.com/What-is-the-F2-score-in-machine-learning) score. \n * [Data Science Bowl 2018](https://www.kaggle.com/c/data-science-bowl-2018/overview/evaluation). Yet another semantic segmentation competition. The evaluation metric is exactly the same. Here is the [top solution](https://www.kaggle.com/c/data-science-bowl-2018/discussion/54741). \n \n* Some interesting ideas:\n\n * Multi-rounds pseudo-labelling (graph below from [here](https://arxiv.org/pdf/1904.04445.pdf))\n * Train a classifier first to find images without wheat heads\n * Randomly cropping the train images\n * Using augmentation. [Albumentations](https://github.com/albumentations-team/albumentations) is a great library for that.\n * Use BCE (binary cross entropy) + Jaccard as a loss (given some weight to each) or [Lovasz](https://arxiv.org/pdf/1705.08790.pdf)\n * Train using one loss for some epochs then use another one\n * Average over the last n snapshots for more generalization\n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2Fb8105b2f464ab58a47bf068debbe24c9%2Fkround_pseudo_labeling.png?generation=1593898259088583&alt=media)\n\n* Some additional links: \n\n * A [paper](https://raw.githubusercontent.com/rafaelpadilla/Object-Detection-Metrics/master/paper_survey_on_performance_metrics_for_object_detection_algorithms.pdf) detailing metrics for object detection\n\nI hope this is useful, I will keep adding new things along the way. Enjoy the competition!", "votes": null }, { "id": "915644", "postDate": "07/05/2020 01:09:14", "content": "

Good stuff! Thanks.

\n\n

Minor correction: \"All the train images have the same size: 1024 x 1024.\"

", "rawMarkdown": "Good stuff! Thanks. \n\nMinor correction: \"All the train images have the same size: **1024 x 1024**.\"", "votes": null }, { "id": "915792", "postDate": "07/05/2020 05:18:51", "content": "

Good catch. I will correct it. Thanks!

", "rawMarkdown": "Good catch. I will correct it. Thanks!", "votes": null }, { "id": "920825", "postDate": "07/08/2020 20:21:28", "content": "

[EDIT 8-7-2020] Added a short code snippet to extract the bounding boxes in a (x1, y1, x2, y2) format for the train labels + some observations about the bbox areas and their distribution.

", "rawMarkdown": "[EDIT 8-7-2020] Added a short code snippet to extract the bounding boxes in a (x1, y1, x2, y2) format for the train labels + some observations about the bbox areas and their distribution.", "votes": null }, { "id": "922275", "postDate": "07/10/2020 01:37:35", "content": "

I met object detection for the first time in this competition. Great explanation and it would be very helpful for me! Thanks!

", "rawMarkdown": "I met object detection for the first time in this competition. Great explanation and it would be very helpful for me! Thanks!", "votes": null }, { "id": "923071", "postDate": "07/10/2020 14:10:48", "content": "

Awesome! I am glad this helped. Stay tuned for an upcoming notebook. 😎

", "rawMarkdown": "Awesome! I am glad this helped. Stay tuned for an upcoming notebook. 😎", "votes": null }, { "id": "926706", "postDate": "07/12/2020 22:55:59", "content": "

Hi, do you know how to evaluate those pictures without any bounding box?

", "rawMarkdown": "Hi, do you know how to evaluate those pictures without any bounding box?", "votes": null }, { "id": "926947", "postDate": "07/13/2020 05:46:26", "content": "

That's a good question. In previous competitions (for example Airbus Ship Detection), competitions added a classification step before the object detection one. Check this notebook for an example. Let me know if this answered your question. Cheers!

", "rawMarkdown": "That's a good question. In previous competitions (for example Airbus Ship Detection), competitions added a classification step before the object detection one. Check this [notebook](https://www.kaggle.com/hmendonca/classification-and-segmentation-fp) for an example. Let me know if this answered your question. Cheers!", "votes": null } ], "comments": [ { "id": 915644, "author_name": "msjose", "author_url": "", "post_date": "07/05/2020 01:09:14", "content": "

Good stuff! Thanks.

\n\n

Minor correction: \"All the train images have the same size: 1024 x 1024.\"

", "votes": null, "replies": [ { "id": 915792, "author_name": "yassinealouini", "author_url": "", "post_date": "07/05/2020 05:18:51", "content": "

Good catch. I will correct it. Thanks!

", "votes": null, "replies": [] } ] }, { "id": 920825, "author_name": "yassinealouini", "author_url": "", "post_date": "07/08/2020 20:21:28", "content": "

[EDIT 8-7-2020] Added a short code snippet to extract the bounding boxes in a (x1, y1, x2, y2) format for the train labels + some observations about the bbox areas and their distribution.

", "votes": null, "replies": [] }, { "id": 922275, "author_name": "yummycocopalm", "author_url": "", "post_date": "07/10/2020 01:37:35", "content": "

I met object detection for the first time in this competition. Great explanation and it would be very helpful for me! Thanks!

", "votes": null, "replies": [ { "id": 923071, "author_name": "yassinealouini", "author_url": "", "post_date": "07/10/2020 14:10:48", "content": "

Awesome! I am glad this helped. Stay tuned for an upcoming notebook. 😎

", "votes": null, "replies": [] } ] }, { "id": 926706, "author_name": "haoanwang", "author_url": "", "post_date": "07/12/2020 22:55:59", "content": "

Hi, do you know how to evaluate those pictures without any bounding box?

", "votes": null, "replies": [ { "id": 926947, "author_name": "yassinealouini", "author_url": "", "post_date": "07/13/2020 05:46:26", "content": "

That's a good question. In previous competitions (for example Airbus Ship Detection), competitions added a classification step before the object detection one. Check this notebook for an example. Let me know if this answered your question. Cheers!

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "915574": "A new competition, a new insights post! Here are some: \n\n\n\n* Train images: \n\n * There is a toal of **3422** unique train images. The code to get this number:\n ```\n from pathlib import Path\n train_paths = Path(\"/path/to/train\")\n len(list(train_paths.glob(\"*\")))\n ```\n * There is a total of **3373** unique images with masks\n * Thus some images don't have any masks\n * There is a toal of **147793** masks\n * Thus, on average, there are **43.8** masks per image\n * The image with the most masks contains **116**. It is the image with id `35b935b6c`.\n * All the train images have the same size: 1024 x 1024. \n * There are 3 channels: R, G, B.\n * The train images come from **7** different sources. Here is the distribution: \n\n ``` \n {'arvalis_1': 1055,\n 'arvalis_2': 204,\n 'arvalis_3': 559,\n 'ethz_1': 747,\n 'inrae_1': 176,\n 'rres_1': 432,\n 'usask_1': 200}\n ``` \n * **3** of these sources are probably similar: *arvalis_1*, *arvalis_2*, and *arvalis_3*. Most likely, the breakdown is by the competition sponsors/organisers?\n * Bounding boxes where added using this [tool](https://github.com/jsbroks/coco-annotator).\n * You can find resized datasets [here](https://www.kaggle.com/c/global-wheat-detection/discussion/160408).\n * There are some data quality issues: masks [too big](https://www.kaggle.com/c/global-wheat-detection/discussion/159578), or [missing masks](https://www.kaggle.com/c/global-wheat-detection/discussion/163263). The organizers are working on a fix for the private test dataset (as far as I know).\n * The average bounding box area is **6843**, the smallest is only **2**, and the largest is **529788**! Here attached the two images with the largest (upper right including many smaller) and the smallest (it is the in the lower right corner, at the edge) bounding boxes: \n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2F51306c59989506fbc085778805afaea0%2F42e6efaaa_with_bbox.png?generation=1594240530216553&alt=media)\n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2Fb4859bccc9fe7de7d7a19385479ab34f%2Fd0912a485_with_bbox.png?generation=1594240509985483&alt=media)\n\n\n * The bbox area histogram with the top 1000 values truncated and the bottom 100 as well:\n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2F6397d02cd4ada74403ae0eaa955394ce%2Fhist_truncated.png?generation=1594242665230885&alt=media)\n \n\n * For more details about the dataset creation, check this [paper](https://arxiv.org/pdf/2005.02162.pdf). It contains a lot of statistics and data acquisition methodolgy descritption, so make sure to check it. Here are some graphs from the paper: \n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2F063dea7e597defdd1b44fdd3bab843be%2Fwheat_dataset_table.png?generation=1593898190270563&alt=media)\n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2F518f78637d9026e7ee76380745749a93%2Foriginal_wheat_images.png?generation=1593898211077093&alt=media)\n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2F6dd629a68c99e0f840a8c446a8bc1092%2Fwheat_table_statistics.png?generation=1593898390536691&alt=media)\n\n\n\n* Test images:\n\n * Only **10** images have been made public. The code to get this: \n ```\n from pathlib import Path\n test_paths = Path(\"/path/to/test\")\n len(list(test_paths.glob(\"*\")))\n ```\n * Don't worry though, there is a \"real\" test dataset that runs when you submit your inference kernel.\n * The test dataset has been collected from different sources than the train. This is an important aspect of the competition for sure.\n * There are few images in the test dataset that aren't 1024 x 1024. These will be fixed in the private part as far as I know.\n * More insights about the test dataset could be found in [this](https://www.kaggle.com/c/global-wheat-detection/discussion/149770) discussion. \n\n* Short code snippets\n\n\n * Load an image and overlay the corresponding mask(s):\n\n ```\n from pathlib import Path\n\n img_id = \"ID_OF_IMAGE\"\n img_path = Path(f\"/path/to/train/{img_id}.jpg\")\n img = Image.open(img_path)\n draw = ImageDraw.Draw(img)\n bboxes = train_labels_df.loc[lambda df: df[\"image_id\"] == img_id, \"bbox\"].tolist()\n # The box contains the upper left corner (x, y) coordinates then width and height.\n # So we need to change these to (x1, y1) and (x2, y2) where they are the upper \n # left and lower right corners\n for bbox in bboxes:\n x, y, w, h = eval(bbox)\n transformed_bbox = [x, y, x + w, y + h]\n draw.rectangle(transformed_bbox, outline=\"black\", width=3)\n img.save(\"/path/to/output/{img_id}.png\")\n img\n ```\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2Fe56c560fc347f24e44a9840c3afbf61c%2Fimag_with_bbox.png?generation=1593939578542921&alt=media)\n\n * Extracting bounding boxes coordinates, width, height, and area: \n ```\n import pandas as pd\n train_labels_df = pd.read_csv(\"path/to/train/labels.csv\")\n \n x = []\n y = []\n w = []\n h = []\n for bbox in train_labels_df['bbox']:\n bbox = eval(bbox)\n x.append(bbox[0])\n y.append(bbox[1])\n w.append(bbox[2])\n h.append(bbox[3])\n\n processed_train_labels_df = train_labels_df.copy()\n processed_train_labels_df[\"x\"] = x\n processed_train_labels_df[\"y\"] = y\n processed_train_labels_df[\"w\"] = w\n processed_train_labels_df[\"h\"] = h\n\n processed_train_labels_df[\"area\"] = processed_train_labels_df[\"w\"] * processed_train_labels_df[\"h\"]\n processed_train_labels_df[\"x2\"] = processed_train_labels_df[\"x\"] + processed_train_labels_df[\"w\"]\n processed_train_labels_df[\"y2\"] = processed_train_labels_df[\"y\"] + processed_train_labels_df[\"h\"]\n ```\n\n\n\n* Metric\n\n * There are many components to this metric: first, we compute Jaccard/[IoU](https://www.pyimagesearch.com/2016/11/07/intersection-over-union-iou-for-object-detection/) scores, then for some threshold `t`, the wheat head is a TP if the IoU is above it, finally we compute the [precision](https://en.wikipedia.org/wiki/Precision_and_recall) for this threshold. Then, average over the thresholds (given the AP) and over images (given mAP, mean average prediction).\n * Thus, there are two levels of averaging: over the thresholds and then over the images. \n * Notice that there are some edge-cases where the precision will be 0: no mask is predicted and there is at least one, a mask is predicted but there aren't any.\n\n* Some backbone models that work well for object detection: \n\n * [YOLOv5](https://github.com/ultralytics/yolov5)\n * [Detectron2](https://github.com/facebookresearch/detectron2): this isn't a single model but rather a collection of object detection models. It contains Faster R-CNN, RetinaNet and many more. \n\n* Previous similar competitions:\n\n * [TGS Salt Identification](https://www.kaggle.com/c/tgs-salt-identification-challenge/overview). This is an object detection competition with exactly the same metric. Here is a link to the [first place](https://www.kaggle.com/c/tgs-salt-identification-challenge/discussion/69291) solution, the code [repo](https://github.com/ybabakhin/kaggle_salt_bes_phalanx), and a [paper](https://arxiv.org/pdf/1904.04445.pdf) published.\n * [Severstal: Steel Defect Detection](https://www.kaggle.com/c/severstal-steel-defect-detection/). This isn't exactly an object detection competition but rather an **semantic segmentation** one. Here is a link the [first place](https://www.kaggle.com/c/severstal-steel-defect-detection/discussion/114254) solution. \n * [Airbus Ship Detection](https://www.kaggle.com/c/airbus-ship-detection). Another instance segmentation competition. Here is the 4th place solution. The metric here is close to the current competition but instead of AP, it uses [F2](https://www.quora.com/What-is-the-F2-score-in-machine-learning) score. \n * [Data Science Bowl 2018](https://www.kaggle.com/c/data-science-bowl-2018/overview/evaluation). Yet another semantic segmentation competition. The evaluation metric is exactly the same. Here is the [top solution](https://www.kaggle.com/c/data-science-bowl-2018/discussion/54741). \n \n* Some interesting ideas:\n\n * Multi-rounds pseudo-labelling (graph below from [here](https://arxiv.org/pdf/1904.04445.pdf))\n * Train a classifier first to find images without wheat heads\n * Randomly cropping the train images\n * Using augmentation. [Albumentations](https://github.com/albumentations-team/albumentations) is a great library for that.\n * Use BCE (binary cross entropy) + Jaccard as a loss (given some weight to each) or [Lovasz](https://arxiv.org/pdf/1705.08790.pdf)\n * Train using one loss for some epochs then use another one\n * Average over the last n snapshots for more generalization\n\n ![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F172860%2Fb8105b2f464ab58a47bf068debbe24c9%2Fkround_pseudo_labeling.png?generation=1593898259088583&alt=media)\n\n* Some additional links: \n\n * A [paper](https://raw.githubusercontent.com/rafaelpadilla/Object-Detection-Metrics/master/paper_survey_on_performance_metrics_for_object_detection_algorithms.pdf) detailing metrics for object detection\n\nI hope this is useful, I will keep adding new things along the way. Enjoy the competition!", "915644": "Good stuff! Thanks. \n\nMinor correction: \"All the train images have the same size: **1024 x 1024**.\"", "915792": "Good catch. I will correct it. Thanks!", "920825": "[EDIT 8-7-2020] Added a short code snippet to extract the bounding boxes in a (x1, y1, x2, y2) format for the train labels + some observations about the bbox areas and their distribution.", "922275": "I met object detection for the first time in this competition. Great explanation and it would be very helpful for me! Thanks!", "923071": "Awesome! I am glad this helped. Stay tuned for an upcoming notebook. 😎", "926706": "Hi, do you know how to evaluate those pictures without any bounding box?", "926947": "That's a good question. In previous competitions (for example Airbus Ship Detection), competitions added a classification step before the object detection one. Check this [notebook](https://www.kaggle.com/hmendonca/classification-and-segmentation-fp) for an example. Let me know if this answered your question. Cheers!" }, "source": "meta" }