{ "id": 187120, "title": "Advanced Topics", "url": "/competitions/global-wheat-detection/discussion/187120", "author_name": "", "post_date": "2020-09-27T15:39:33.916539800Z", "votes": 16, "comment_count": 4, "views": 0, "content": "

Some advanced computer vision topics that I have learned (and I am still learning) during this competition:

\n

Mosaic Augmentation

\n

This technique consists in stitching together different images to form a bigger one.
\nThe more diverse the images, the better it is.
\nIf you are curious, you can check an example of how to use it
\nhere from the winning competition solution. The following example is extracted from the same post, thanks again to DungNB for the write-up!

\n

\n

Here is a simple explanation using cat images:

\n

\n

TTA

\n

TTA, short for test-time augmentation, is a popular technique for getting better performance for computer vision tasks and you will often encounter it when doing Kaggle computer vision competitions.

\n

How does it work?

\n

The basic idea is quite simple:

\n
    \n
  1. Create additional test images from the provided ones using augmentation techniques.
    2. \n
  2. Predict using the trained model: you will have the prediction for the original image plus the augmented ones.
    3. \n
  3. Blend the predictions: more on this in the next section
    4. \n
\n

For the augmentation step, horizontal and vertical flips are quite common strategies for TTA. Notice that since this step happens during inference time and since code competitions have time limits, you can't use a lot of augmentation.
\n2 or 3 additional images per original test one is more than enough.
\nFinally, for object detection, since there is a bounding box to predict, the augmentation step should preserve the shape of the bounding box. Thus some augmentations aren't allowed (for instance 90 degrees rotations).

\n

To finish this section, here is an illustration (again, using cats :p)

\n

\n

WBF Blending

\n

If you are accustomed to object detection tasks and challenges, you may have heard about NMS (short for non-maximum suppression). Here is a short description of its pseudo-code:

\n

\n

Check this post for more details (the figure above is from it).

\n

This technique is used to merge many boxes (called proposals) from one model and can be used to merge the results of many models as well.

\n

Here, we will go over another technique called weighted-boxes fusion or (WBF in short).

\n

To get started, you can check the Github implementation here, the paper or this great notebook again from Alex Shonenkov, so thanks!

\n

So how does it work and how is it different? Let's have a look at the following schema (from the WBF paper)

\n

\n

If you may have noticed, the WBF algorithm \"made\" a bounding box that isn't any of the proposed ones whereas NMS came up with one from the proposals. Indeed, that's one of the big differences between the two: how different proposals are merged. For the WBF case, the main objective is to compute a new confidence score C from existing confidence scores and then use it to compute the blended bounding box coordinates and confidence as a weighted mean (or other functions).

\n

Finally, if you want even more blending techniques, you can explore these techniques:

\n\n

\n", "messages": [ { "id": "1029244", "postDate": "09/27/2020 15:39:33", "content": "

Some advanced computer vision topics that I have learned (and I am still learning) during this competition:

\n

Mosaic Augmentation

\n

This technique consists in stitching together different images to form a bigger one.
\nThe more diverse the images, the better it is.
\nIf you are curious, you can check an example of how to use it
\nhere from the winning competition solution. The following example is extracted from the same post, thanks again to DungNB for the write-up!

\n

\n

Here is a simple explanation using cat images:

\n

\n

TTA

\n

TTA, short for test-time augmentation, is a popular technique for getting better performance for computer vision tasks and you will often encounter it when doing Kaggle computer vision competitions.

\n

How does it work?

\n

The basic idea is quite simple:

\n
    \n
  1. Create additional test images from the provided ones using augmentation techniques.
    2. \n
  2. Predict using the trained model: you will have the prediction for the original image plus the augmented ones.
    3. \n
  3. Blend the predictions: more on this in the next section
    4. \n
\n

For the augmentation step, horizontal and vertical flips are quite common strategies for TTA. Notice that since this step happens during inference time and since code competitions have time limits, you can't use a lot of augmentation.
\n2 or 3 additional images per original test one is more than enough.
\nFinally, for object detection, since there is a bounding box to predict, the augmentation step should preserve the shape of the bounding box. Thus some augmentations aren't allowed (for instance 90 degrees rotations).

\n

To finish this section, here is an illustration (again, using cats :p)

\n

\n

WBF Blending

\n

If you are accustomed to object detection tasks and challenges, you may have heard about NMS (short for non-maximum suppression). Here is a short description of its pseudo-code:

\n

\n

Check this post for more details (the figure above is from it).

\n

This technique is used to merge many boxes (called proposals) from one model and can be used to merge the results of many models as well.

\n

Here, we will go over another technique called weighted-boxes fusion or (WBF in short).

\n

To get started, you can check the Github implementation here, the paper or this great notebook again from Alex Shonenkov, so thanks!

\n

So how does it work and how is it different? Let's have a look at the following schema (from the WBF paper)

\n

\n

If you may have noticed, the WBF algorithm \"made\" a bounding box that isn't any of the proposed ones whereas NMS came up with one from the proposals. Indeed, that's one of the big differences between the two: how different proposals are merged. For the WBF case, the main objective is to compute a new confidence score C from existing confidence scores and then use it to compute the blended bounding box coordinates and confidence as a weighted mean (or other functions).

\n

Finally, if you want even more blending techniques, you can explore these techniques:

\n\n

\n", "rawMarkdown": "Some advanced computer vision topics that I have learned (and I am still learning) during this competition: \n\n\n## Mosaic Augmentation\n\nThis technique consists in stitching together different images to form a bigger one. \nThe more diverse the images, the better it is. \nIf you are curious, you can check an example of how to use it\n[here](https://www.kaggle.com/c/global-wheat-detection/discussion/172418) from the winning competition solution. The following example is extracted from the same post, thanks again to [DungNB](https://www.kaggle.com/nguyenbadung) for the write-up!\n\n\n\n\n\n\n\n\n\n\n\n\nHere is a simple explanation using cat images: \n\n\n\n\n\n\n\n## TTA\n\n\nTTA, short for **test-time augmentation**, is a popular technique for getting better performance for computer vision tasks and you will often encounter it when doing Kaggle computer vision competitions.\n\nHow does it work?\n\nThe basic idea is quite simple: \n\n1. Create additional test images from the provided ones using **augmentation** techniques.\n2. Predict using the trained model: you will have the prediction for the original image plus the augmented ones.\n3. Blend the predictions: more on this in the next section \n\nFor the augmentation step, horizontal and vertical flips are quite common strategies for TTA. Notice that since this step happens during inference time and since code competitions have time limits, you can't use a lot of augmentation. \n2 or 3 additional images per original test one is more than enough.\nFinally, for object detection, since there is a bounding box to predict, the augmentation step should preserve the shape of the bounding box. Thus some augmentations aren't allowed (for instance 90 degrees rotations). \n\nTo finish this section, here is an illustration (again, using cats :p)\n\n\n\n\n\n\n\n\n## WBF Blending\n\n\nIf you are accustomed to **object detection** tasks and challenges, you may have heard about **NMS** (short for non-maximum suppression). Here is a short description of its pseudo-code:\n\n\n\n\n\n\n\nCheck this [post](https://towardsdatascience.com/non-maximum-suppression-nms-93ce178e177c) for more details (the figure above is from it).\n\n\nThis technique is used to merge many boxes (called proposals) from one model and can be used to merge the results of many models as well. \n\nHere, we will go over another technique called **weighted-boxes fusion** or (**WBF** in short). \n\nTo get started, you can check the Github implementation [here](https://github.com/ZFTurbo/Weighted-Boxes-Fusion), the [paper](https://arxiv.org/pdf/1910.13302.pdf) or this great [notebook](https://www.kaggle.com/shonenkov/wbf-approach-for-ensemble) again from [Alex Shonenkov](https://www.kaggle.com/shonenkov), so thanks!\n\nSo how does it work and how is it different? Let's have a look at the following schema (from the **WBF** paper)\n\n\n\n\n\nIf you may have noticed, the **WBF** algorithm \"made\" a bounding box that isn't any of the proposed ones whereas NMS came up with one from the proposals. Indeed, that's one of the big differences between the two: how different proposals are merged. For the WBF case, the main objective is to compute a new confidence score C from existing confidence scores and then use it to compute the blended bounding box coordinates and confidence as a weighted mean (or other functions).\n\n\nFinally, if you want even more blending techniques, you can explore these techniques:\n\n- Soft-NMS: similar to NMS (non-maximum suppression) but with a soft selection criterion\n\n\n\n\n- Non-maximum weighted (NMW): this is the same as the NMS but computing a weighted a bounding box instead\nof computing only one.", "votes": null }, { "id": "1031453", "postDate": "09/29/2020 13:05:15", "content": "

Thanks for sharing @yassinealouini

", "rawMarkdown": "Thanks for sharing @yassinealouini", "votes": null }, { "id": "1034783", "postDate": "10/02/2020 07:03:30", "content": "

You are welcome. Let me know if you need more details.

", "rawMarkdown": "You are welcome. Let me know if you need more details.", "votes": null }, { "id": "1040873", "postDate": "10/07/2020 12:14:34", "content": "

Thanks for your sharing! And I also want to know that how to ensemble models, if I got k folds?🙏

", "rawMarkdown": "Thanks for your sharing! And I also want to know that how to ensemble models, if I got k folds?🙏", "votes": null }, { "id": "1368593", "postDate": "06/28/2021 17:46:45", "content": "

You are welcome, I am glad that it helps.

\n

Regarding ensembling, there are at least two things I can think of:

\n\n

The topic of ensembling is quite complex and tricky, so don't hesitate to ask more questions.

", "rawMarkdown": "You are welcome, I am glad that it helps.\n\nRegarding ensembling, there are at least two things I can think of: \n\n- if you have only one model per fold and you want to ensemble these models, then you can take the average across these k folds or you can take a weighted average using the scores you get during training (so the best model gets more weights etc)\n- if you have more than one model per fold and you want to train another model on top, you can do it by training the model on the correct out-of-fold predictions for each model. So if you have two models and 5 folds, you will have 2 features for each fold. At the end, you can take the average of the 5 folds model.\n\nThe topic of **ensembling** is quite **complex** and **tricky**, so don't hesitate to ask more questions.", "votes": null } ], "comments": [ { "id": 1031453, "author_name": "aditi81k", "author_url": "", "post_date": "09/29/2020 13:05:15", "content": "

Thanks for sharing @yassinealouini

", "votes": null, "replies": [ { "id": 1034783, "author_name": "yassinealouini", "author_url": "", "post_date": "10/02/2020 07:03:30", "content": "

You are welcome. Let me know if you need more details.

", "votes": null, "replies": [] } ] }, { "id": 1040873, "author_name": "wanyanbin", "author_url": "", "post_date": "10/07/2020 12:14:34", "content": "

Thanks for your sharing! And I also want to know that how to ensemble models, if I got k folds?🙏

", "votes": null, "replies": [ { "id": 1368593, "author_name": "yassinealouini", "author_url": "", "post_date": "06/28/2021 17:46:45", "content": "

You are welcome, I am glad that it helps.

\n

Regarding ensembling, there are at least two things I can think of:

\n\n

The topic of ensembling is quite complex and tricky, so don't hesitate to ask more questions.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1029244": "Some advanced computer vision topics that I have learned (and I am still learning) during this competition: \n\n\n## Mosaic Augmentation\n\nThis technique consists in stitching together different images to form a bigger one. \nThe more diverse the images, the better it is. \nIf you are curious, you can check an example of how to use it\n[here](https://www.kaggle.com/c/global-wheat-detection/discussion/172418) from the winning competition solution. The following example is extracted from the same post, thanks again to [DungNB](https://www.kaggle.com/nguyenbadung) for the write-up!\n\n\n\n\n\n\n\n\n\n\n\n\nHere is a simple explanation using cat images: \n\n\n\n\n\n\n\n## TTA\n\n\nTTA, short for **test-time augmentation**, is a popular technique for getting better performance for computer vision tasks and you will often encounter it when doing Kaggle computer vision competitions.\n\nHow does it work?\n\nThe basic idea is quite simple: \n\n1. Create additional test images from the provided ones using **augmentation** techniques.\n2. Predict using the trained model: you will have the prediction for the original image plus the augmented ones.\n3. Blend the predictions: more on this in the next section \n\nFor the augmentation step, horizontal and vertical flips are quite common strategies for TTA. Notice that since this step happens during inference time and since code competitions have time limits, you can't use a lot of augmentation. \n2 or 3 additional images per original test one is more than enough.\nFinally, for object detection, since there is a bounding box to predict, the augmentation step should preserve the shape of the bounding box. Thus some augmentations aren't allowed (for instance 90 degrees rotations). \n\nTo finish this section, here is an illustration (again, using cats :p)\n\n\n\n\n\n\n\n\n## WBF Blending\n\n\nIf you are accustomed to **object detection** tasks and challenges, you may have heard about **NMS** (short for non-maximum suppression). Here is a short description of its pseudo-code:\n\n\n\n\n\n\n\nCheck this [post](https://towardsdatascience.com/non-maximum-suppression-nms-93ce178e177c) for more details (the figure above is from it).\n\n\nThis technique is used to merge many boxes (called proposals) from one model and can be used to merge the results of many models as well. \n\nHere, we will go over another technique called **weighted-boxes fusion** or (**WBF** in short). \n\nTo get started, you can check the Github implementation [here](https://github.com/ZFTurbo/Weighted-Boxes-Fusion), the [paper](https://arxiv.org/pdf/1910.13302.pdf) or this great [notebook](https://www.kaggle.com/shonenkov/wbf-approach-for-ensemble) again from [Alex Shonenkov](https://www.kaggle.com/shonenkov), so thanks!\n\nSo how does it work and how is it different? Let's have a look at the following schema (from the **WBF** paper)\n\n\n\n\n\nIf you may have noticed, the **WBF** algorithm \"made\" a bounding box that isn't any of the proposed ones whereas NMS came up with one from the proposals. Indeed, that's one of the big differences between the two: how different proposals are merged. For the WBF case, the main objective is to compute a new confidence score C from existing confidence scores and then use it to compute the blended bounding box coordinates and confidence as a weighted mean (or other functions).\n\n\nFinally, if you want even more blending techniques, you can explore these techniques:\n\n- Soft-NMS: similar to NMS (non-maximum suppression) but with a soft selection criterion\n\n\n\n\n- Non-maximum weighted (NMW): this is the same as the NMS but computing a weighted a bounding box instead\nof computing only one.", "1031453": "Thanks for sharing @yassinealouini", "1034783": "You are welcome. Let me know if you need more details.", "1040873": "Thanks for your sharing! And I also want to know that how to ensemble models, if I got k folds?🙏", "1368593": "You are welcome, I am glad that it helps.\n\nRegarding ensembling, there are at least two things I can think of: \n\n- if you have only one model per fold and you want to ensemble these models, then you can take the average across these k folds or you can take a weighted average using the scores you get during training (so the best model gets more weights etc)\n- if you have more than one model per fold and you want to train another model on top, you can do it by training the model on the correct out-of-fold predictions for each model. So if you have two models and 5 folds, you will have 2 features for each fold. At the end, you can take the average of the 5 folds model.\n\nThe topic of **ensembling** is quite **complex** and **tricky**, so don't hesitate to ask more questions." }, "source": "meta" }