Hi, All.
\nIf the bounding box of the training dataset is slightly shifted, or the box size is subtly reduced or increased, will there be an augmentation effect to improve the generalization performance of object detection?
\nI wonder …
\nThanks, in advance.
\nBest,
\n@bemoregt.
Hi, All.
\nIf the bounding box of the training dataset is slightly shifted, or the box size is subtly reduced or increased, will there be an augmentation effect to improve the generalization performance of object detection?
\nI wonder …
\nThanks, in advance.
\nBest,
\n@bemoregt.
Hi,
\nBriefly said :
\nIn any case, it shouldn't improve the network's ability to generalize.
\nTo me, a data augmentation policy makes sense if it modifies the content of the original image that is fed into the network. To improve generalization, we want our network to see as much image & annotations as possible that could fit the targeted distribution. All the commonly used augmentation policies modify the original image to create content that is likely to fit the target distribution.
\nBy modifying bboxes just a little (shifting, rescaling), we do not modify the original image, neither the area of interest we ask the network to extract content from ; At least if the targeted object has a low resolution, where a small shift dramatically changes the pixels distribution of the image contained by the bbox. When the resolution is low, it's always another story. To my knowledge it is still an active field of research…
\nBut if the resolution of your targeted objects is high enough, applying small changes to the bboxes without modifying the pixels values should result in adding noise to the localization part of the network, without helping the filters to extract content from the image that would help generalization. Intuitively, we could see it as saying to the network : \"This exact same combination of pixels could result in slightly different bbox position.\" If the network has a high learning rate, it might overfit to this part of the image. If the learning rate is low, the network might not be able to extract features that really matter to accuratly distinguish the targeted object from the background.
\nOn the contrary, commonly used data augmentation modifies the pixel values while preserving the quality of content inside the bboxes - a flipped wheat head is still a wheat head - providing additional high quality resources which could be used to \"interpolate\" wheat heads.
\nI never saw slight bbox shifting/rescaling improving the performances of a DL model. If someone has a different view, I would be interested to know !
", "rawMarkdown": "Hi, \n\n\nBriefly said :\n- Small bbox modifications relatively to the targeted object's size => training on the same image multiple times\n- Large bbox modifications relatively to the targeted object's size => add noise to the localization part of the network\n\nIn any case, it shouldn't improve the network's ability to generalize.\n\n---\n\nTo me, a data augmentation policy makes sense if it modifies the content of the original image that is fed into the network. To improve generalization, we want our network to see as much image & annotations as possible that could fit the targeted distribution. All the commonly used augmentation policies modify the original image to create content that is likely to fit the target distribution.\n\nBy modifying bboxes just a little (shifting, rescaling), we do not modify the original image, neither the area of interest we ask the network to extract content from ; At least if the targeted object has a low resolution, where a small shift dramatically changes the pixels distribution of the image contained by the bbox. When the resolution is low, it's always another story. To my knowledge it is still an active field of research... \n\nBut if the resolution of your targeted objects is high enough, applying small changes to the bboxes without modifying the pixels values should result in adding noise to the localization part of the network, without helping the filters to extract content from the image that would help generalization. Intuitively, we could see it as saying to the network : \"This exact same combination of pixels could result in slightly different bbox position.\" If the network has a high learning rate, it might overfit to this part of the image. If the learning rate is low, the network might not be able to extract features that really matter to accuratly distinguish the targeted object from the background. \n\nOn the contrary, commonly used data augmentation modifies the pixel values while preserving the quality of content inside the bboxes - a flipped wheat head is still a wheat head - providing additional high quality resources which could be used to \"interpolate\" wheat heads.\n\nI never saw slight bbox shifting/rescaling improving the performances of a DL model. If someone has a different view, I would be interested to know !", "votes": null }, { "id": "1122984", "postDate": "12/22/2020 20:39:12", "content": "Hi, KT-karl.
\nOK, I see…
\nif so,
\nWith the bounding box fixed
\nWhat about inverting the image color?
Or what if you rotate the image?
\nOf course, you have to rotate the bounding box as well..
Will there be an augmentation effect even in that case?
\nI wonder.
Color modification & image rotation are commonly used augmentation for object detection. Both are great to improve generalization, but it always depend on your task : does it make the training dataset closer to the (unknown) target distribution ?
\nSometimes it doesn't make sense to apply photometric variations, or rotation. Vertical flipping is not likely to be relevant on images which are meant to describe scenes submitted to gravity…
\nI would suggest you to take a look at this repo and that one if you want to build a good intuition about data augmentation. There are also plenty of great resources about it on the internet.
", "rawMarkdown": "Color modification & image rotation are commonly used augmentation for object detection. Both are great to improve generalization, but it always depend on your task : does it make the training dataset closer to the (unknown) target distribution ?\n\nSometimes it doesn't make sense to apply photometric variations, or rotation. Vertical flipping is not likely to be relevant on images which are meant to describe scenes submitted to gravity...\n\nI would suggest you to take a look at [this repo](https://github.com/aleju/imgaug) and [that one](https://github.com/albumentations-team/albumentations) if you want to build a good intuition about data augmentation. There are also plenty of great resources about it on the internet.", "votes": null }, { "id": "1123141", "postDate": "12/23/2020 00:27:40", "content": "Thank you for your suggestion sites.
\nI'm interested in a kind of special augmentation using like Fourier transforms.
\n", "rawMarkdown": "Thank you for your suggestion sites.\n\nI'm interested in a kind of special augmentation using like Fourier transforms.\n\n@bemoregt.", "votes": null } ], "comments": [ { "id": 1122690, "author_name": "johannhuber", "author_url": "", "post_date": "12/22/2020 16:15:50", "content": "Hi,
\nBriefly said :
\nIn any case, it shouldn't improve the network's ability to generalize.
\nTo me, a data augmentation policy makes sense if it modifies the content of the original image that is fed into the network. To improve generalization, we want our network to see as much image & annotations as possible that could fit the targeted distribution. All the commonly used augmentation policies modify the original image to create content that is likely to fit the target distribution.
\nBy modifying bboxes just a little (shifting, rescaling), we do not modify the original image, neither the area of interest we ask the network to extract content from ; At least if the targeted object has a low resolution, where a small shift dramatically changes the pixels distribution of the image contained by the bbox. When the resolution is low, it's always another story. To my knowledge it is still an active field of research…
\nBut if the resolution of your targeted objects is high enough, applying small changes to the bboxes without modifying the pixels values should result in adding noise to the localization part of the network, without helping the filters to extract content from the image that would help generalization. Intuitively, we could see it as saying to the network : \"This exact same combination of pixels could result in slightly different bbox position.\" If the network has a high learning rate, it might overfit to this part of the image. If the learning rate is low, the network might not be able to extract features that really matter to accuratly distinguish the targeted object from the background.
\nOn the contrary, commonly used data augmentation modifies the pixel values while preserving the quality of content inside the bboxes - a flipped wheat head is still a wheat head - providing additional high quality resources which could be used to \"interpolate\" wheat heads.
\nI never saw slight bbox shifting/rescaling improving the performances of a DL model. If someone has a different view, I would be interested to know !
", "votes": null, "replies": [ { "id": 1122984, "author_name": "bemorekgg", "author_url": "", "post_date": "12/22/2020 20:39:12", "content": "Hi, KT-karl.
\nOK, I see…
\nif so,
\nWith the bounding box fixed
\nWhat about inverting the image color?
Or what if you rotate the image?
\nOf course, you have to rotate the bounding box as well..
Will there be an augmentation effect even in that case?
\nI wonder.
Color modification & image rotation are commonly used augmentation for object detection. Both are great to improve generalization, but it always depend on your task : does it make the training dataset closer to the (unknown) target distribution ?
\nSometimes it doesn't make sense to apply photometric variations, or rotation. Vertical flipping is not likely to be relevant on images which are meant to describe scenes submitted to gravity…
\nI would suggest you to take a look at this repo and that one if you want to build a good intuition about data augmentation. There are also plenty of great resources about it on the internet.
", "votes": null, "replies": [] }, { "id": 1123141, "author_name": "bemorekgg", "author_url": "", "post_date": "12/23/2020 00:27:40", "content": "Thank you for your suggestion sites.
\nI'm interested in a kind of special augmentation using like Fourier transforms.
\n", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1122115": "Hi, All.\n\nIf the bounding box of the training dataset is slightly shifted, or the box size is subtly reduced or increased, will there be an augmentation effect to improve the generalization performance of object detection?\n\nI wonder …\n\nThanks, in advance.\nBest,\n@bemoregt.", "1122690": "Hi, \n\n\nBriefly said :\n- Small bbox modifications relatively to the targeted object's size => training on the same image multiple times\n- Large bbox modifications relatively to the targeted object's size => add noise to the localization part of the network\n\nIn any case, it shouldn't improve the network's ability to generalize.\n\n---\n\nTo me, a data augmentation policy makes sense if it modifies the content of the original image that is fed into the network. To improve generalization, we want our network to see as much image & annotations as possible that could fit the targeted distribution. All the commonly used augmentation policies modify the original image to create content that is likely to fit the target distribution.\n\nBy modifying bboxes just a little (shifting, rescaling), we do not modify the original image, neither the area of interest we ask the network to extract content from ; At least if the targeted object has a low resolution, where a small shift dramatically changes the pixels distribution of the image contained by the bbox. When the resolution is low, it's always another story. To my knowledge it is still an active field of research... \n\nBut if the resolution of your targeted objects is high enough, applying small changes to the bboxes without modifying the pixels values should result in adding noise to the localization part of the network, without helping the filters to extract content from the image that would help generalization. Intuitively, we could see it as saying to the network : \"This exact same combination of pixels could result in slightly different bbox position.\" If the network has a high learning rate, it might overfit to this part of the image. If the learning rate is low, the network might not be able to extract features that really matter to accuratly distinguish the targeted object from the background. \n\nOn the contrary, commonly used data augmentation modifies the pixel values while preserving the quality of content inside the bboxes - a flipped wheat head is still a wheat head - providing additional high quality resources which could be used to \"interpolate\" wheat heads.\n\nI never saw slight bbox shifting/rescaling improving the performances of a DL model. If someone has a different view, I would be interested to know !", "1122984": "Hi, KT-karl.\n\nOK, I see...\n\nif so,\nWith the bounding box fixed\nWhat about inverting the image color?\n\nOr what if you rotate the image?\nOf course, you have to rotate the bounding box as well..\n\nWill there be an augmentation effect even in that case?\nI wonder.\n\n@bemoregt.", "1123136": "Color modification & image rotation are commonly used augmentation for object detection. Both are great to improve generalization, but it always depend on your task : does it make the training dataset closer to the (unknown) target distribution ?\n\nSometimes it doesn't make sense to apply photometric variations, or rotation. Vertical flipping is not likely to be relevant on images which are meant to describe scenes submitted to gravity...\n\nI would suggest you to take a look at [this repo](https://github.com/aleju/imgaug) and [that one](https://github.com/albumentations-team/albumentations) if you want to build a good intuition about data augmentation. There are also plenty of great resources about it on the internet.", "1123141": "Thank you for your suggestion sites.\n\nI'm interested in a kind of special augmentation using like Fourier transforms.\n\n@bemoregt." }, "source": "meta" }