I'm trying to understand how everybody's Pytorch system working.(especially https://www.kaggle.com/pestipeti/pytorch-starter-fasterrcnn-train).\nThen I wanna clean my head,so I posted.
\n\n1.\n Pytorch needs class including(getitem,_len_).\nNext,substitute train and valid for the cxlass,make DataLoader each of train and valid.\nLast,throw DataLoader to model(FasterCNN),train it,save the model
\n\nQ)\nIs there any wrong?Do I understand elementary Pytorch's flow?
\n\n2.\nI think this method is very very powerfully,but I wonder if there are little room to improve score...\nAfter searching, I find [Augmentation].\nI read some notebook about Augmentation.And written how to edit image and bounding-box ,but no explain how let new image into model.
\n\nQ)\nCould you tell me how to use new image?\nDo I need save new image into train File, and using it?\nIs there any easy way using new image?
\n\nThank you for giving me your attention,and sorry for my stupid question and bad English.
", "messages": [ { "id": "941727", "postDate": "07/23/2020 11:29:03", "content": "I'm trying to understand how everybody's Pytorch system working.(especially https://www.kaggle.com/pestipeti/pytorch-starter-fasterrcnn-train).\nThen I wanna clean my head,so I posted.
\n\n1.\n Pytorch needs class including(getitem,_len_).\nNext,substitute train and valid for the cxlass,make DataLoader each of train and valid.\nLast,throw DataLoader to model(FasterCNN),train it,save the model
\n\nQ)\nIs there any wrong?Do I understand elementary Pytorch's flow?
\n\n2.\nI think this method is very very powerfully,but I wonder if there are little room to improve score...\nAfter searching, I find [Augmentation].\nI read some notebook about Augmentation.And written how to edit image and bounding-box ,but no explain how let new image into model.
\n\nQ)\nCould you tell me how to use new image?\nDo I need save new image into train File, and using it?\nIs there any easy way using new image?
\n\nThank you for giving me your attention,and sorry for my stupid question and bad English.
", "rawMarkdown": "I'm trying to understand how everybody's Pytorch system working.(especially https://www.kaggle.com/pestipeti/pytorch-starter-fasterrcnn-train).\nThen I wanna clean my head,so I posted.\n\n1.\n Pytorch needs class including(__getitem__,__len__).\nNext,substitute train and valid for the cxlass,make DataLoader each of train and valid.\nLast,throw DataLoader to model(FasterCNN),train it,save the model\n\nQ)\nIs there any wrong?Do I understand elementary Pytorch's flow?\n\n2.\nI think this method is very very powerfully,but I wonder if there are little room to improve score...\nAfter searching, I find [Augmentation].\nI read some notebook about Augmentation.And written how to edit image and bounding-box ,but no explain how let new image into model. \n\nQ)\nCould you tell me how to use new image?\nDo I need save new image into train File, and using it?\nIs there any easy way using new image?\n\nThank you for giving me your attention,and sorry for my stupid question and bad English.", "votes": null }, { "id": "941827", "postDate": "07/23/2020 12:39:40", "content": "Yes, thats how it work. make a dataset class, create its object, then prepare a dataloader giving the batch size you prefer. input from loader to model to predict and train(both train and val loaders).
For augumentations use Aulbumentions it has almost everything you'll need and also is easy to implement pytorch.\nyou don't need to do much, just pass that function to dataset class(make input and get output from it then return it) you can find notebooks from the community. No need to save or anything it happens one the fly. once you get it you'll know its easy enough. If you want more easy stick to tensorflow instead, its syntax are minimal.
Thanks!!!\n.\nDepending on URL I wrote previously,maybe using transform in First class.\nAnd I think apply the transform to all of image in there.\n.\nDoes this really improve quantity of data?\nIt seems like just editing image...\n.\nIs my understanding wrong?\nI understood Augumentations is improving quantity of image by adding edited image...\n.\nDoes Augumentations mean just editing image,not add new image that edited to Dataset?\n.\nIf you have a time, could you teach me how Augmentations working...?
", "rawMarkdown": "Thanks!!!\n.\nDepending on URL I wrote previously,maybe using transform in First class.\nAnd I think apply the transform to all of image in there.\n.\nDoes this really improve quantity of data?\nIt seems like just editing image...\n.\nIs my understanding wrong?\nI understood Augumentations is improving quantity of image by adding edited image...\n.\nDoes Augumentations mean just editing image,not add new image that edited to Dataset?\n.\nIf you have a time, could you teach me how Augmentations working...?", "votes": null }, { "id": "942860", "postDate": "07/24/2020 03:32:46", "content": "You're right in that it 'edits' the image and doesn't directly increase the dataset size. However, it does 'artificially' increase the dataset size by providing the network with more variation in the data. Check out this Quora page for a little more in-depth answer to your questions. It might be helpful to google about \"data augmentation importance\" to consult more resources on this topic.
", "rawMarkdown": "You're right in that it 'edits' the image and doesn't directly increase the dataset size. However, it does 'artificially' increase the dataset size by providing the network with more variation in the data. Check out [this Quora page](https://www.quora.com/What-is-data-augmentation-in-machine-learning) for a little more in-depth answer to your questions. It might be helpful to google about \"data augmentation importance\" to consult more resources on this topic.", "votes": null }, { "id": "942879", "postDate": "07/24/2020 03:42:08", "content": "The transform in torch return some auguments, some normals as per as given probablities(so not changing the size of dataset). Say you want to flip images you gave flip(p=0.5) so the dataloader will 50% of time will return fliped images and 50% of them normal unfliped. so new images are not added but replaced by transformed one, on the fly. Here is some info link
Thank you so much!!!\n.\nI finally understood!!!\n.\nThen,If I apply Flip(p=0.5), how the bounding box change?\nBounding box should change with image, but I think there are no code about changing box.\nOr our Pytorch did that in black box?
", "rawMarkdown": "Thank you so much!!!\n.\nI finally understood!!!\n.\nThen,If I apply Flip(p=0.5), how the bounding box change?\nBounding box should change with image, but I think there are no code about changing box.\nOr our Pytorch did that in black box?", "votes": null }, { "id": "944532", "postDate": "07/25/2020 07:11:24", "content": "Flip is just example for normal image(with no boxes). Albumentions handles that for you, check out their examples here
", "rawMarkdown": "Flip is just example for normal image(with no boxes). Albumentions handles that for you, check out their examples [here](https://github.com/albumentations-team/albumentations_examples/blob/master/notebooks/example_bboxes.ipynb)", "votes": null }, { "id": "944552", "postDate": "07/25/2020 07:26:07", "content": "thank you!!!\nI didnt have enough knowledge,so embarrassing.\nYou are my best teacher!!! Thank you!
", "rawMarkdown": "thank you!!!\nI didnt have enough knowledge,so embarrassing.\nYou are my best teacher!!! Thank you!", "votes": null }, { "id": "944591", "postDate": "07/25/2020 07:46:59", "content": "Haha, dont be and I am a learner too.. Keep Kaggling friend!!
", "rawMarkdown": "Haha, dont be and I am a learner too.. Keep Kaggling friend!!", "votes": null } ], "comments": [ { "id": 941827, "author_name": "anku5hk", "author_url": "", "post_date": "07/23/2020 12:39:40", "content": "Yes, thats how it work. make a dataset class, create its object, then prepare a dataloader giving the batch size you prefer. input from loader to model to predict and train(both train and val loaders).
For augumentations use Aulbumentions it has almost everything you'll need and also is easy to implement pytorch.\nyou don't need to do much, just pass that function to dataset class(make input and get output from it then return it) you can find notebooks from the community. No need to save or anything it happens one the fly. once you get it you'll know its easy enough. If you want more easy stick to tensorflow instead, its syntax are minimal.
Thanks!!!\n.\nDepending on URL I wrote previously,maybe using transform in First class.\nAnd I think apply the transform to all of image in there.\n.\nDoes this really improve quantity of data?\nIt seems like just editing image...\n.\nIs my understanding wrong?\nI understood Augumentations is improving quantity of image by adding edited image...\n.\nDoes Augumentations mean just editing image,not add new image that edited to Dataset?\n.\nIf you have a time, could you teach me how Augmentations working...?
", "votes": null, "replies": [] }, { "id": 942860, "author_name": "matthewmasters", "author_url": "", "post_date": "07/24/2020 03:32:46", "content": "You're right in that it 'edits' the image and doesn't directly increase the dataset size. However, it does 'artificially' increase the dataset size by providing the network with more variation in the data. Check out this Quora page for a little more in-depth answer to your questions. It might be helpful to google about \"data augmentation importance\" to consult more resources on this topic.
", "votes": null, "replies": [] }, { "id": 942879, "author_name": "anku5hk", "author_url": "", "post_date": "07/24/2020 03:42:08", "content": "The transform in torch return some auguments, some normals as per as given probablities(so not changing the size of dataset). Say you want to flip images you gave flip(p=0.5) so the dataloader will 50% of time will return fliped images and 50% of them normal unfliped. so new images are not added but replaced by transformed one, on the fly. Here is some info link
Thank you so much!!!\n.\nI finally understood!!!\n.\nThen,If I apply Flip(p=0.5), how the bounding box change?\nBounding box should change with image, but I think there are no code about changing box.\nOr our Pytorch did that in black box?
", "votes": null, "replies": [] }, { "id": 944532, "author_name": "anku5hk", "author_url": "", "post_date": "07/25/2020 07:11:24", "content": "Flip is just example for normal image(with no boxes). Albumentions handles that for you, check out their examples here
", "votes": null, "replies": [] }, { "id": 944552, "author_name": "okamotos", "author_url": "", "post_date": "07/25/2020 07:26:07", "content": "thank you!!!\nI didnt have enough knowledge,so embarrassing.\nYou are my best teacher!!! Thank you!
", "votes": null, "replies": [] }, { "id": 944591, "author_name": "anku5hk", "author_url": "", "post_date": "07/25/2020 07:46:59", "content": "Haha, dont be and I am a learner too.. Keep Kaggling friend!!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "941727": "I'm trying to understand how everybody's Pytorch system working.(especially https://www.kaggle.com/pestipeti/pytorch-starter-fasterrcnn-train).\nThen I wanna clean my head,so I posted.\n\n1.\n Pytorch needs class including(__getitem__,__len__).\nNext,substitute train and valid for the cxlass,make DataLoader each of train and valid.\nLast,throw DataLoader to model(FasterCNN),train it,save the model\n\nQ)\nIs there any wrong?Do I understand elementary Pytorch's flow?\n\n2.\nI think this method is very very powerfully,but I wonder if there are little room to improve score...\nAfter searching, I find [Augmentation].\nI read some notebook about Augmentation.And written how to edit image and bounding-box ,but no explain how let new image into model. \n\nQ)\nCould you tell me how to use new image?\nDo I need save new image into train File, and using it?\nIs there any easy way using new image?\n\nThank you for giving me your attention,and sorry for my stupid question and bad English.", "941827": "1. Yes, thats how it work. make a dataset class, create its object, then prepare a dataloader giving the batch size you prefer. input from loader to model to predict and train(both train and val loaders).\n\n2. For augumentations use Aulbumentions it has almost everything you'll need and also is easy to implement pytorch.\nyou don't need to do much, just pass that function to dataset class(make input and get output from it then return it) you can find notebooks from the community. No need to save or anything it happens one the fly. once you get it you'll know its easy enough. If you want more easy stick to tensorflow instead, its syntax are minimal.", "941887": "Thanks!!!\n.\nDepending on URL I wrote previously,maybe using transform in First class.\nAnd I think apply the transform to all of image in there.\n.\nDoes this really improve quantity of data?\nIt seems like just editing image...\n.\nIs my understanding wrong?\nI understood Augumentations is improving quantity of image by adding edited image...\n.\nDoes Augumentations mean just editing image,not add new image that edited to Dataset?\n.\nIf you have a time, could you teach me how Augmentations working...?", "942860": "You're right in that it 'edits' the image and doesn't directly increase the dataset size. However, it does 'artificially' increase the dataset size by providing the network with more variation in the data. Check out [this Quora page](https://www.quora.com/What-is-data-augmentation-in-machine-learning) for a little more in-depth answer to your questions. It might be helpful to google about \"data augmentation importance\" to consult more resources on this topic.", "942879": "The transform in torch return some auguments, some normals as per as given probablities(so not changing the size of dataset). Say you want to flip images you gave `flip(p=0.5)` so the dataloader will 50% of time will return fliped images and 50% of them normal unfliped. so new images are not added but replaced by transformed one, on the fly. Here is some info [link](https://stackoverflow.com/questions/51677788/data-augmentation-in-pytorch)", "944495": "Thank you so much!!!\n.\nI finally understood!!!\n.\nThen,If I apply Flip(p=0.5), how the bounding box change?\nBounding box should change with image, but I think there are no code about changing box.\nOr our Pytorch did that in black box?", "944532": "Flip is just example for normal image(with no boxes). Albumentions handles that for you, check out their examples [here](https://github.com/albumentations-team/albumentations_examples/blob/master/notebooks/example_bboxes.ipynb)", "944552": "thank you!!!\nI didnt have enough knowledge,so embarrassing.\nYou are my best teacher!!! Thank you!", "944591": "Haha, dont be and I am a learner too.. Keep Kaggling friend!!" }, "source": "meta" }