{ "id": 160408, "title": "Speeding up the training process with a simple trick", "url": "/competitions/global-wheat-detection/discussion/160408", "author_name": "Hieu Phung", "post_date": "2020-06-21T06:09:48.825000", "votes": 70, "comment_count": 10, "views": 0, "content": "

I've examined many awesome public kernels and was impressed by the ways that the contributors built their training pipelines incorporating various techniques to improve the models' performance. Yet, it seems that most of the solutions are frequently using resize-transform with a fixed parameter such as 512, 224, etc.

\n\n

Transforming images, bounding boxes, and other objects on-the-fly is a good way to experiment with new configurations and data preprocessing techniques, but in some cases where the transformation could be applied to the data prior to the training process such as resizing, we can further improve the training speed by applying these transformations only 1 time to the dataset before using it.

\n\n

To do the trick, I've created a few datasets which contain resized images and annotations from the original one which can be found here:\n- Global Wheat Detection 768x768\n- Global Wheat Detection 512x512\n- Global Wheat Detection 384x384\n- Global Wheat Detection 256x256\n- Global Wheat Detection 224x224

\n\n

Want to create your own versions, this short & simple NOTEBOOK could help you 😉 . It demonstrates the process of resizing images with coresponding bounding boxes and packing them into a new dataset which could be used later.

\n\n

Happy Kaggling!!!

", "messages": [ { "id": 895141, "postDate": "2020-06-21T06:09:48.827Z", "content": "

I've examined many awesome public kernels and was impressed by the ways that the contributors built their training pipelines incorporating various techniques to improve the models' performance. Yet, it seems that most of the solutions are frequently using resize-transform with a fixed parameter such as 512, 224, etc.

\n\n

Transforming images, bounding boxes, and other objects on-the-fly is a good way to experiment with new configurations and data preprocessing techniques, but in some cases where the transformation could be applied to the data prior to the training process such as resizing, we can further improve the training speed by applying these transformations only 1 time to the dataset before using it.

\n\n

To do the trick, I've created a few datasets which contain resized images and annotations from the original one which can be found here:\n- Global Wheat Detection 768x768\n- Global Wheat Detection 512x512\n- Global Wheat Detection 384x384\n- Global Wheat Detection 256x256\n- Global Wheat Detection 224x224

\n\n

Want to create your own versions, this short & simple NOTEBOOK could help you 😉 . It demonstrates the process of resizing images with coresponding bounding boxes and packing them into a new dataset which could be used later.

\n\n

Happy Kaggling!!!

", "rawMarkdown": "I've examined many awesome public kernels and was impressed by the ways that the contributors built their training pipelines incorporating various techniques to improve the models' performance. Yet, it seems that most of the solutions are frequently using resize-transform with a fixed parameter such as 512, 224, etc.\n\nTransforming images, bounding boxes, and other objects on-the-fly is a good way to experiment with new configurations and data preprocessing techniques, but in some cases where the transformation could be applied to the data prior to the training process such as resizing, we can further improve the training speed by applying these transformations only 1 time to the dataset before using it.\n\nTo do the trick, I've created a few datasets which contain resized images and annotations from the original [**one**](https://www.kaggle.com/c/global-wheat-detection/data) which can be found here:\n- [*Global Wheat Detection 768x768*](https://www.kaggle.com/phunghieu/global-wheat-detection-768x768)\n- [*Global Wheat Detection 512x512*](https://www.kaggle.com/phunghieu/global-wheat-detection-512x512)\n- [*Global Wheat Detection 384x384*](https://www.kaggle.com/phunghieu/global-wheat-detection-384x384)\n- [*Global Wheat Detection 256x256*](https://www.kaggle.com/phunghieu/global-wheat-detection-256x256)\n- [*Global Wheat Detection 224x224*](https://www.kaggle.com/phunghieu/global-wheat-detection-224x224)\n\nWant to create your own versions, this short & simple [**NOTEBOOK**](https://www.kaggle.com/phunghieu/gwd-resize-images-bboxes) could help you 😉 . It demonstrates the process of resizing images with coresponding bounding boxes and packing them into a new dataset which could be used later.\n\nHappy Kaggling!!!", "votes": 67 }, { "id": 904434, "postDate": "2020-06-27T16:13:02.317Z", "content": "

Great job with the datasets and notebook. Thanks for sharing!

", "rawMarkdown": "Great job with the datasets and notebook. Thanks for sharing!", "votes": 1 }, { "id": 898955, "postDate": "2020-06-23T21:26:09.617Z", "content": "

Thanks for sharing! That's really helpful :)

", "rawMarkdown": "Thanks for sharing! That's really helpful :)", "votes": 1 }, { "id": 898111, "postDate": "2020-06-23T10:07:25.950Z", "content": "

I have limited computational resources. This really helped me a lot. Thanks for sharing :D

", "rawMarkdown": "I have limited computational resources. This really helped me a lot. Thanks for sharing :D", "votes": 1 }, { "id": 896184, "postDate": "2020-06-22T00:05:46.037Z", "content": "

Good initiative! Optimizing energy consumption, you're saving the planet 👌

", "rawMarkdown": "Good initiative! Optimizing energy consumption, you're saving the planet 👌 ", "votes": 1 }, { "id": 895163, "postDate": "2020-06-21T06:39:53.393Z", "content": "

Nice! I've found that using the full image size is usually the best but hopefully this will be good for users with limited computational resources. Thanks for sharing

", "rawMarkdown": "Nice! I've found that using the full image size is usually the best but hopefully this will be good for users with limited computational resources. Thanks for sharing", "votes": 1 }, { "id": 906137, "postDate": "2020-06-29T04:48:47.740Z", "content": "

That's really helpful. Thanks for sharing!

", "rawMarkdown": "That's really helpful. Thanks for sharing!", "votes": 2 }, { "id": 906844, "postDate": "2020-06-29T15:04:04.437Z", "rawMarkdown": "", "isDeleted": true }, { "id": 946662, "postDate": "2020-07-26T17:59:44.920Z", "content": "

thanks for sharing.

", "rawMarkdown": "thanks for sharing.", "votes": 1 }, { "id": 897848, "postDate": "2020-06-23T06:20:04.813Z", "content": "

Thanks for sharing. Very helpful!!!

", "rawMarkdown": "Thanks for sharing. Very helpful!!!", "votes": 1 }, { "id": 906846, "postDate": "2020-06-29T15:04:59.357Z", "content": "

That's good. Thanks for sharing!

", "rawMarkdown": "That's good. Thanks for sharing!", "votes": 2 } ], "comments": [ { "id": 904434, "author_name": "Eisa", "author_url": "", "post_date": "2020-06-27T16:13:02.317000", "content": "

Great job with the datasets and notebook. Thanks for sharing!

", "votes": 1, "replies": [] }, { "id": 898955, "author_name": "Marie", "author_url": "", "post_date": "2020-06-23T21:26:09.617000", "content": "

Thanks for sharing! That's really helpful :)

", "votes": 1, "replies": [] }, { "id": 898111, "author_name": "Rifat Rashid", "author_url": "", "post_date": "2020-06-23T10:07:25.950000", "content": "

I have limited computational resources. This really helped me a lot. Thanks for sharing :D

", "votes": 1, "replies": [] }, { "id": 896184, "author_name": "Fares FOURATI", "author_url": "", "post_date": "2020-06-22T00:05:46.037000", "content": "

Good initiative! Optimizing energy consumption, you're saving the planet 👌

", "votes": 1, "replies": [] }, { "id": 895163, "author_name": "Matt", "author_url": "", "post_date": "2020-06-21T06:39:53.393000", "content": "

Nice! I've found that using the full image size is usually the best but hopefully this will be good for users with limited computational resources. Thanks for sharing

", "votes": 1, "replies": [] }, { "id": 906137, "author_name": "Abu Noman Md. Sakib", "author_url": "", "post_date": "2020-06-29T04:48:47.740000", "content": "

That's really helpful. Thanks for sharing!

", "votes": 2, "replies": [] }, { "id": 906844, "author_name": "", "author_url": "", "post_date": "2020-06-29T15:04:04.437000", "content": "", "votes": 0, "replies": [] }, { "id": 946662, "author_name": "Prachi", "author_url": "", "post_date": "2020-07-26T17:59:44.920000", "content": "

thanks for sharing.

", "votes": 1, "replies": [] }, { "id": 897848, "author_name": "ai1776", "author_url": "", "post_date": "2020-06-23T06:20:04.813000", "content": "

Thanks for sharing. Very helpful!!!

", "votes": 1, "replies": [] }, { "id": 906846, "author_name": "Qingyuan Wang", "author_url": "", "post_date": "2020-06-29T15:04:59.357000", "content": "

That's good. Thanks for sharing!

", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "895141": "I've examined many awesome public kernels and was impressed by the ways that the contributors built their training pipelines incorporating various techniques to improve the models' performance. Yet, it seems that most of the solutions are frequently using resize-transform with a fixed parameter such as 512, 224, etc.\n\nTransforming images, bounding boxes, and other objects on-the-fly is a good way to experiment with new configurations and data preprocessing techniques, but in some cases where the transformation could be applied to the data prior to the training process such as resizing, we can further improve the training speed by applying these transformations only 1 time to the dataset before using it.\n\nTo do the trick, I've created a few datasets which contain resized images and annotations from the original [**one**](https://www.kaggle.com/c/global-wheat-detection/data) which can be found here:\n- [*Global Wheat Detection 768x768*](https://www.kaggle.com/phunghieu/global-wheat-detection-768x768)\n- [*Global Wheat Detection 512x512*](https://www.kaggle.com/phunghieu/global-wheat-detection-512x512)\n- [*Global Wheat Detection 384x384*](https://www.kaggle.com/phunghieu/global-wheat-detection-384x384)\n- [*Global Wheat Detection 256x256*](https://www.kaggle.com/phunghieu/global-wheat-detection-256x256)\n- [*Global Wheat Detection 224x224*](https://www.kaggle.com/phunghieu/global-wheat-detection-224x224)\n\nWant to create your own versions, this short & simple [**NOTEBOOK**](https://www.kaggle.com/phunghieu/gwd-resize-images-bboxes) could help you 😉 . It demonstrates the process of resizing images with coresponding bounding boxes and packing them into a new dataset which could be used later.\n\nHappy Kaggling!!!", "904434": "Great job with the datasets and notebook. Thanks for sharing!", "898955": "Thanks for sharing! That's really helpful :)", "898111": "I have limited computational resources. This really helped me a lot. Thanks for sharing :D", "896184": "Good initiative! Optimizing energy consumption, you're saving the planet 👌 ", "895163": "Nice! I've found that using the full image size is usually the best but hopefully this will be good for users with limited computational resources. Thanks for sharing", "906137": "That's really helpful. Thanks for sharing!", "906844": "", "946662": "thanks for sharing.", "897848": "Thanks for sharing. Very helpful!!!", "906846": "That's good. Thanks for sharing!" } }