I am a noob on kaggle and this is my first competition. I tried various data augmentation and pytorch,albumentation transforma but I have been unable to achieve a lb score>0.63.(I am using faster rcnn with resnet 50)\nIf it isn't against the competition rules ,can somebody help me out or provide me with some suggestion or tell me what I could possibly be doing wrong?
", "messages": [ { "id": "934547", "postDate": "07/18/2020 14:21:01", "content": "I am a noob on kaggle and this is my first competition. I tried various data augmentation and pytorch,albumentation transforma but I have been unable to achieve a lb score>0.63.(I am using faster rcnn with resnet 50)\nIf it isn't against the competition rules ,can somebody help me out or provide me with some suggestion or tell me what I could possibly be doing wrong?
", "rawMarkdown": "I am a noob on kaggle and this is my first competition. I tried various data augmentation and pytorch,albumentation transforma but I have been unable to achieve a lb score>0.63.(I am using faster rcnn with resnet 50)\nIf it isn't against the competition rules ,can somebody help me out or provide me with some suggestion or tell me what I could possibly be doing wrong?", "votes": null }, { "id": "934782", "postDate": "07/18/2020 18:12:58", "content": "How are you selecting the best weights? Do you use validation set?
", "rawMarkdown": "How are you selecting the best weights? Do you use validation set?", "votes": null }, { "id": "934788", "postDate": "07/18/2020 18:19:22", "content": "Well i am pretty new to pytorch and it took me long enough to make one submission ,so up until now I'e trained only one model.\nSo I haven't tried anything like that.\nBut even after I added the image augmentations and pytorch transforms I see almost no different in the visualiations
", "rawMarkdown": "Well i am pretty new to pytorch and it took me long enough to make one submission ,so up until now I'e trained only one model.\nSo I haven't tried anything like that.\nBut even after I added the image augmentations and pytorch transforms I see almost no different in the visualiations", "votes": null }, { "id": "934792", "postDate": "07/18/2020 18:27:26", "content": "It would definitely help to have a validation set. You can use it to select the best weights and make sure you're not over/under-fitting to your training set.
\n\nThere's other tricks that people have shown here to increase LB score such as pseudolabeling and TTA but I would worry about the basics first. Using a validation set or cross-validation is important for almost all data science projects.
", "rawMarkdown": "It would definitely help to have a validation set. You can use it to select the best weights and make sure you're not over/under-fitting to your training set.\n\nThere's other tricks that people have shown here to increase LB score such as pseudolabeling and TTA but I would worry about the basics first. Using a validation set or cross-validation is important for almost all data science projects.", "votes": null }, { "id": "934806", "postDate": "07/18/2020 18:41:48", "content": "Okay I guess I need to give it a try.\nIf you don't mind,could you help me out with some resource for these basic things with pytorch?\nLike some notebook,pdf , article etc.
", "rawMarkdown": "Okay I guess I need to give it a try.\nIf you don't mind,could you help me out with some resource for these basic things with pytorch?\nLike some notebook,pdf , article etc.", "votes": null }, { "id": "934811", "postDate": "07/18/2020 18:49:54", "content": "The public notebooks here should be pretty helpful. Most of them have all the code for training and inference. You can see this post here about how to prepare a stratified validation split.
\n\nIf you're looking to learn the basics of PyTorch, there is a book titled \"Deep learning with PyTorch\" that they are giving out for free for a limited time here. so maybe starting with some of the projects in there would be good practice.
", "rawMarkdown": "The public notebooks here should be pretty helpful. Most of them have all the code for training and inference. You can see [this post here](https://www.kaggle.com/c/global-wheat-detection/discussion/152569) about how to prepare a stratified validation split.\n\nIf you're looking to learn the basics of PyTorch, there is a book titled \"Deep learning with PyTorch\" that they are giving out for free for a limited time [here](https://pytorch.org/deep-learning-with-pytorch). so maybe starting with some of the projects in there would be good practice.", "votes": null }, { "id": "934816", "postDate": "07/18/2020 19:03:19", "content": "Well thanks😅\nI actually have started this book\nAnd thanks for the post\nJust a follow up question:will setting up a validation set(which I actually had set up ,I just hadn't any strategy & code to select the best weights) really improve the LB score from 0.639.
", "rawMarkdown": "Well thanks😅\nI actually have started this book\nAnd thanks for the post\nJust a follow up question:will setting up a validation set(which I actually had set up ,I just hadn't any strategy & code to select the best weights) really improve the LB score from 0.639.", "votes": null }, { "id": "934830", "postDate": "07/18/2020 19:20:22", "content": "I imagine it would. Right now you are basically selecting the best epoch at random. That could be an issue because those weights could be under/over-fit to the training data. If it is underfitting, you need to train longer. If it is overfitting, the best weights will be from an earlier epoch. Additionally, if you are using a small batch size the accuracy likely fluctuates a bit epoch from epoch and you might be selecting one of those bad epochs.
", "rawMarkdown": "I imagine it would. Right now you are basically selecting the best epoch at random. That could be an issue because those weights could be under/over-fit to the training data. If it is underfitting, you need to train longer. If it is overfitting, the best weights will be from an earlier epoch. Additionally, if you are using a small batch size the accuracy likely fluctuates a bit epoch from epoch and you might be selecting one of those bad epochs.", "votes": null }, { "id": "934861", "postDate": "07/18/2020 20:16:37", "content": "I did think that fluctuation might be an issue but I am not able to increase the batch size more than 8 (I am using kaggle GPU)
", "rawMarkdown": "I did think that fluctuation might be an issue but I am not able to increase the batch size more than 8 (I am using kaggle GPU)", "votes": null }, { "id": "934864", "postDate": "07/18/2020 20:22:51", "content": "You might be interested in \"gradient accumulation\". It allows you to effectively increase your batch size and I think is pretty easy to implement in pytorch
", "rawMarkdown": "You might be interested in \"gradient accumulation\". It allows you to effectively increase your batch size and I think is pretty easy to implement in pytorch", "votes": null } ], "comments": [ { "id": 934782, "author_name": "matthewmasters", "author_url": "", "post_date": "07/18/2020 18:12:58", "content": "How are you selecting the best weights? Do you use validation set?
", "votes": null, "replies": [ { "id": 934788, "author_name": "bhavesjain", "author_url": "", "post_date": "07/18/2020 18:19:22", "content": "Well i am pretty new to pytorch and it took me long enough to make one submission ,so up until now I'e trained only one model.\nSo I haven't tried anything like that.\nBut even after I added the image augmentations and pytorch transforms I see almost no different in the visualiations
", "votes": null, "replies": [] }, { "id": 934792, "author_name": "matthewmasters", "author_url": "", "post_date": "07/18/2020 18:27:26", "content": "It would definitely help to have a validation set. You can use it to select the best weights and make sure you're not over/under-fitting to your training set.
\n\nThere's other tricks that people have shown here to increase LB score such as pseudolabeling and TTA but I would worry about the basics first. Using a validation set or cross-validation is important for almost all data science projects.
", "votes": null, "replies": [] }, { "id": 934806, "author_name": "bhavesjain", "author_url": "", "post_date": "07/18/2020 18:41:48", "content": "Okay I guess I need to give it a try.\nIf you don't mind,could you help me out with some resource for these basic things with pytorch?\nLike some notebook,pdf , article etc.
", "votes": null, "replies": [] }, { "id": 934811, "author_name": "matthewmasters", "author_url": "", "post_date": "07/18/2020 18:49:54", "content": "The public notebooks here should be pretty helpful. Most of them have all the code for training and inference. You can see this post here about how to prepare a stratified validation split.
\n\nIf you're looking to learn the basics of PyTorch, there is a book titled \"Deep learning with PyTorch\" that they are giving out for free for a limited time here. so maybe starting with some of the projects in there would be good practice.
", "votes": null, "replies": [] }, { "id": 934816, "author_name": "bhavesjain", "author_url": "", "post_date": "07/18/2020 19:03:19", "content": "Well thanks😅\nI actually have started this book\nAnd thanks for the post\nJust a follow up question:will setting up a validation set(which I actually had set up ,I just hadn't any strategy & code to select the best weights) really improve the LB score from 0.639.
", "votes": null, "replies": [] }, { "id": 934830, "author_name": "matthewmasters", "author_url": "", "post_date": "07/18/2020 19:20:22", "content": "I imagine it would. Right now you are basically selecting the best epoch at random. That could be an issue because those weights could be under/over-fit to the training data. If it is underfitting, you need to train longer. If it is overfitting, the best weights will be from an earlier epoch. Additionally, if you are using a small batch size the accuracy likely fluctuates a bit epoch from epoch and you might be selecting one of those bad epochs.
", "votes": null, "replies": [] }, { "id": 934861, "author_name": "bhavesjain", "author_url": "", "post_date": "07/18/2020 20:16:37", "content": "I did think that fluctuation might be an issue but I am not able to increase the batch size more than 8 (I am using kaggle GPU)
", "votes": null, "replies": [] }, { "id": 934864, "author_name": "matthewmasters", "author_url": "", "post_date": "07/18/2020 20:22:51", "content": "You might be interested in \"gradient accumulation\". It allows you to effectively increase your batch size and I think is pretty easy to implement in pytorch
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "934547": "I am a noob on kaggle and this is my first competition. I tried various data augmentation and pytorch,albumentation transforma but I have been unable to achieve a lb score>0.63.(I am using faster rcnn with resnet 50)\nIf it isn't against the competition rules ,can somebody help me out or provide me with some suggestion or tell me what I could possibly be doing wrong?", "934782": "How are you selecting the best weights? Do you use validation set?", "934788": "Well i am pretty new to pytorch and it took me long enough to make one submission ,so up until now I'e trained only one model.\nSo I haven't tried anything like that.\nBut even after I added the image augmentations and pytorch transforms I see almost no different in the visualiations", "934792": "It would definitely help to have a validation set. You can use it to select the best weights and make sure you're not over/under-fitting to your training set.\n\nThere's other tricks that people have shown here to increase LB score such as pseudolabeling and TTA but I would worry about the basics first. Using a validation set or cross-validation is important for almost all data science projects.", "934806": "Okay I guess I need to give it a try.\nIf you don't mind,could you help me out with some resource for these basic things with pytorch?\nLike some notebook,pdf , article etc.", "934811": "The public notebooks here should be pretty helpful. Most of them have all the code for training and inference. You can see [this post here](https://www.kaggle.com/c/global-wheat-detection/discussion/152569) about how to prepare a stratified validation split.\n\nIf you're looking to learn the basics of PyTorch, there is a book titled \"Deep learning with PyTorch\" that they are giving out for free for a limited time [here](https://pytorch.org/deep-learning-with-pytorch). so maybe starting with some of the projects in there would be good practice.", "934816": "Well thanks😅\nI actually have started this book\nAnd thanks for the post\nJust a follow up question:will setting up a validation set(which I actually had set up ,I just hadn't any strategy & code to select the best weights) really improve the LB score from 0.639.", "934830": "I imagine it would. Right now you are basically selecting the best epoch at random. That could be an issue because those weights could be under/over-fit to the training data. If it is underfitting, you need to train longer. If it is overfitting, the best weights will be from an earlier epoch. Additionally, if you are using a small batch size the accuracy likely fluctuates a bit epoch from epoch and you might be selecting one of those bad epochs.", "934861": "I did think that fluctuation might be an issue but I am not able to increase the batch size more than 8 (I am using kaggle GPU)", "934864": "You might be interested in \"gradient accumulation\". It allows you to effectively increase your batch size and I think is pretty easy to implement in pytorch" }, "source": "meta" }