I'm curious if anyone has spent any time customizing the model parameters or just stuck with one of the four variations of yaml model configs. I've looked at the data on speed and accuracy of those four stock models -- just wondering if it's a better use of time leaving those as they are and using one and finding other areas to squeeze out better accuracy like with augmentations etc.?
", "messages": [ { "id": "912972", "postDate": "07/02/2020 21:21:40", "content": "I'm curious if anyone has spent any time customizing the model parameters or just stuck with one of the four variations of yaml model configs. I've looked at the data on speed and accuracy of those four stock models -- just wondering if it's a better use of time leaving those as they are and using one and finding other areas to squeeze out better accuracy like with augmentations etc.?
", "rawMarkdown": "I'm curious if anyone has spent any time customizing the model parameters or just stuck with one of the four variations of yaml model configs. I've looked at the data on speed and accuracy of those four stock models -- just wondering if it's a better use of time leaving those as they are and using one and finding other areas to squeeze out better accuracy like with augmentations etc.?", "votes": null }, { "id": "912975", "postDate": "07/02/2020 21:25:54", "content": "don't know if it counts but I did try changing the batch size. also, doesn't the github implementation of yolov5 algorithm apply augmentations on its own already?
", "rawMarkdown": "don't know if it counts but I did try changing the batch size. also, doesn't the github implementation of yolov5 algorithm apply augmentations on its own already?", "votes": null }, { "id": "912983", "postDate": "07/02/2020 21:36:57", "content": "I would recommend not to tweak anything before getting baseline results. Once baseline results are achieved, you can start to explore parameters tweaking.
\n\nHowever, I am not really sure that you will see a crucial difference of results: the point of focus of this competition being the use of WBF and TTA (I'd advise the library from one of the best Kaggle member ZFTurbo: https://github.com/ZFTurbo/Weighted-Boxes-Fusion ).
\n\nAlso, as explained by @jonykarki , there is already very fine augmentation performed when calling the \"train.py\" file of the library.\nThe augmentation details can be found in this file: https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py
", "rawMarkdown": "I would recommend not to tweak anything before getting baseline results. Once baseline results are achieved, you can start to explore parameters tweaking. \n\nHowever, I am not really sure that you will see a crucial difference of results: the point of focus of this competition being the use of WBF and TTA (I'd advise the library from one of the best Kaggle member ZFTurbo: [https://github.com/ZFTurbo/Weighted-Boxes-Fusion](https://github.com/ZFTurbo/Weighted-Boxes-Fusion) ).\n\nAlso, as explained by @jonykarki , there is already very fine augmentation performed when calling the \"train.py\" file of the library.\nThe augmentation details can be found in this file: [https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py](https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py)", "votes": null }, { "id": "912990", "postDate": "07/02/2020 21:42:15", "content": "yes, it's doing a mosaic augmentation
", "rawMarkdown": "yes, it's doing a mosaic augmentation", "votes": null }, { "id": "913000", "postDate": "07/02/2020 21:57:55", "content": "@dimartinot I saw those kernals. I'm definitely working off baselines and getting my data from the various four yaml files. I was just curious before getting into the WBF and TTA attempts just wanted to baseline and was curious if any custom changes had improvements.
\n\n@jonykarki batch = 2 definitely was my best baseline. And I did see that there are augmentations already built-in but I've read on a few posts for TTA that they are turning it off and doing their own augmentations?
", "rawMarkdown": "dimartinot I saw those kernals. I'm definitely working off baselines and getting my data from the various four yaml files. I was just curious before getting into the WBF and TTA attempts just wanted to baseline and was curious if any custom changes had improvements. \n\n@jonykarki batch = 2 definitely was my best baseline. And I did see that there are augmentations already built-in but I've read on a few posts for TTA that they are turning it off and doing their own augmentations?", "votes": null }, { "id": "913542", "postDate": "07/03/2020 09:13:45", "content": "Some interesting observations:
\n\nbatch shouldn't matter as long as its < 64, because if you take a closer look at train.py, they are doing gradient accumulation anyways; also from the group normalization paper they report great performance loss with single-GPU batch size = 2, but strangely so far for me batch size = 2 gives the best PB.
I visualized model predictions on the 10-image \"test\" set and a selection of the validation set. YOLOv5x seems to give out poorer predictions when compared to models in other pipeline that I developed, given similar performence (in the 0.005 range)
so, one of the two takeaways may be derived:
\n\n1) don't trust public lb when the labeling is very noisy (especially when they relabeled the private lb)\n2) deep learning is magic, and intuition may not always work
", "rawMarkdown": "Some interesting observations:\n\n1. batch shouldn't matter as long as its < 64, because if you take a closer look at train.py, they are doing gradient accumulation anyways; also from the group normalization paper they report great performance loss with single-GPU batch size = 2, but strangely so far for me batch size = 2 gives the best PB.\n\n2. I visualized model predictions on the 10-image \"test\" set and a selection of the validation set. YOLOv5x seems to give out poorer predictions when compared to models in other pipeline that I developed, given similar performence (in the 0.005 range)\n\nso, one of the two takeaways may be derived:\n\n1) don't trust public lb when the labeling is very noisy (especially when they relabeled the private lb)\n2) deep learning is magic, and intuition may not always work", "votes": null }, { "id": "916467", "postDate": "07/05/2020 17:27:28", "content": "I was experimenting with 1/3 of the training data and I tried including/excluding few extra augmentations.\nIt did affect my validation scores though. But I am yet training the whole model based on experimented augmentations....so not sure if it will give a boost on LB or not.
", "rawMarkdown": "I was experimenting with 1/3 of the training data and I tried including/excluding few extra augmentations.\nIt did affect my validation scores though. But I am yet training the whole model based on experimented augmentations....so not sure if it will give a boost on LB or not.", "votes": null }, { "id": "916616", "postDate": "07/05/2020 21:00:37", "content": "That's what I'm doing right now. Just takes forever to run enough epochs to get a sense for whether those augmentations are good or not.
", "rawMarkdown": "That's what I'm doing right now. Just takes forever to run enough epochs to get a sense for whether those augmentations are good or not.", "votes": null }, { "id": "937226", "postDate": "07/20/2020 20:46:04", "content": "Yes, the accumulation logic should help different batch sizes achieve similar performance, but I think lower batch sizes may also provide worse statistics for the batchnorm layers, so at some point a very small batch size will probably show worse performance, though this may be negated to some degree by adjusting the batchnorm momentum.
\n\nTODO: perhaps investigate if lower batch sizes can/should be paired with lower batchnorm2d momentum?
", "rawMarkdown": "Yes, the accumulation logic should help different batch sizes achieve similar performance, but I think lower batch sizes may also provide worse statistics for the batchnorm layers, so at some point a very small batch size will probably show worse performance, though this may be negated to some degree by adjusting the batchnorm momentum.\n\nTODO: perhaps investigate if lower batch sizes can/should be paired with lower batchnorm2d momentum?", "votes": null } ], "comments": [ { "id": 912975, "author_name": "jonykarki", "author_url": "", "post_date": "07/02/2020 21:25:54", "content": "don't know if it counts but I did try changing the batch size. also, doesn't the github implementation of yolov5 algorithm apply augmentations on its own already?
", "votes": null, "replies": [ { "id": 912990, "author_name": "ilu000", "author_url": "", "post_date": "07/02/2020 21:42:15", "content": "yes, it's doing a mosaic augmentation
", "votes": null, "replies": [] } ] }, { "id": 912983, "author_name": "dimartinot", "author_url": "", "post_date": "07/02/2020 21:36:57", "content": "I would recommend not to tweak anything before getting baseline results. Once baseline results are achieved, you can start to explore parameters tweaking.
\n\nHowever, I am not really sure that you will see a crucial difference of results: the point of focus of this competition being the use of WBF and TTA (I'd advise the library from one of the best Kaggle member ZFTurbo: https://github.com/ZFTurbo/Weighted-Boxes-Fusion ).
\n\nAlso, as explained by @jonykarki , there is already very fine augmentation performed when calling the \"train.py\" file of the library.\nThe augmentation details can be found in this file: https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py
", "votes": null, "replies": [] }, { "id": 913000, "author_name": "crained", "author_url": "", "post_date": "07/02/2020 21:57:55", "content": "@dimartinot I saw those kernals. I'm definitely working off baselines and getting my data from the various four yaml files. I was just curious before getting into the WBF and TTA attempts just wanted to baseline and was curious if any custom changes had improvements.
\n\n@jonykarki batch = 2 definitely was my best baseline. And I did see that there are augmentations already built-in but I've read on a few posts for TTA that they are turning it off and doing their own augmentations?
", "votes": null, "replies": [] }, { "id": 913542, "author_name": "alexanderliao", "author_url": "", "post_date": "07/03/2020 09:13:45", "content": "Some interesting observations:
\n\nbatch shouldn't matter as long as its < 64, because if you take a closer look at train.py, they are doing gradient accumulation anyways; also from the group normalization paper they report great performance loss with single-GPU batch size = 2, but strangely so far for me batch size = 2 gives the best PB.
I visualized model predictions on the 10-image \"test\" set and a selection of the validation set. YOLOv5x seems to give out poorer predictions when compared to models in other pipeline that I developed, given similar performence (in the 0.005 range)
so, one of the two takeaways may be derived:
\n\n1) don't trust public lb when the labeling is very noisy (especially when they relabeled the private lb)\n2) deep learning is magic, and intuition may not always work
", "votes": null, "replies": [ { "id": 937226, "author_name": "glennjocherultralytics", "author_url": "", "post_date": "07/20/2020 20:46:04", "content": "Yes, the accumulation logic should help different batch sizes achieve similar performance, but I think lower batch sizes may also provide worse statistics for the batchnorm layers, so at some point a very small batch size will probably show worse performance, though this may be negated to some degree by adjusting the batchnorm momentum.
\n\nTODO: perhaps investigate if lower batch sizes can/should be paired with lower batchnorm2d momentum?
", "votes": null, "replies": [] } ] }, { "id": 916467, "author_name": "zaber666", "author_url": "", "post_date": "07/05/2020 17:27:28", "content": "I was experimenting with 1/3 of the training data and I tried including/excluding few extra augmentations.\nIt did affect my validation scores though. But I am yet training the whole model based on experimented augmentations....so not sure if it will give a boost on LB or not.
", "votes": null, "replies": [ { "id": 916616, "author_name": "crained", "author_url": "", "post_date": "07/05/2020 21:00:37", "content": "That's what I'm doing right now. Just takes forever to run enough epochs to get a sense for whether those augmentations are good or not.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "912972": "I'm curious if anyone has spent any time customizing the model parameters or just stuck with one of the four variations of yaml model configs. I've looked at the data on speed and accuracy of those four stock models -- just wondering if it's a better use of time leaving those as they are and using one and finding other areas to squeeze out better accuracy like with augmentations etc.?", "912975": "don't know if it counts but I did try changing the batch size. also, doesn't the github implementation of yolov5 algorithm apply augmentations on its own already?", "912983": "I would recommend not to tweak anything before getting baseline results. Once baseline results are achieved, you can start to explore parameters tweaking. \n\nHowever, I am not really sure that you will see a crucial difference of results: the point of focus of this competition being the use of WBF and TTA (I'd advise the library from one of the best Kaggle member ZFTurbo: [https://github.com/ZFTurbo/Weighted-Boxes-Fusion](https://github.com/ZFTurbo/Weighted-Boxes-Fusion) ).\n\nAlso, as explained by @jonykarki , there is already very fine augmentation performed when calling the \"train.py\" file of the library.\nThe augmentation details can be found in this file: [https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py](https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py)", "912990": "yes, it's doing a mosaic augmentation", "913000": "dimartinot I saw those kernals. I'm definitely working off baselines and getting my data from the various four yaml files. I was just curious before getting into the WBF and TTA attempts just wanted to baseline and was curious if any custom changes had improvements. \n\n@jonykarki batch = 2 definitely was my best baseline. And I did see that there are augmentations already built-in but I've read on a few posts for TTA that they are turning it off and doing their own augmentations?", "913542": "Some interesting observations:\n\n1. batch shouldn't matter as long as its < 64, because if you take a closer look at train.py, they are doing gradient accumulation anyways; also from the group normalization paper they report great performance loss with single-GPU batch size = 2, but strangely so far for me batch size = 2 gives the best PB.\n\n2. I visualized model predictions on the 10-image \"test\" set and a selection of the validation set. YOLOv5x seems to give out poorer predictions when compared to models in other pipeline that I developed, given similar performence (in the 0.005 range)\n\nso, one of the two takeaways may be derived:\n\n1) don't trust public lb when the labeling is very noisy (especially when they relabeled the private lb)\n2) deep learning is magic, and intuition may not always work", "916467": "I was experimenting with 1/3 of the training data and I tried including/excluding few extra augmentations.\nIt did affect my validation scores though. But I am yet training the whole model based on experimented augmentations....so not sure if it will give a boost on LB or not.", "916616": "That's what I'm doing right now. Just takes forever to run enough epochs to get a sense for whether those augmentations are good or not.", "937226": "Yes, the accumulation logic should help different batch sizes achieve similar performance, but I think lower batch sizes may also provide worse statistics for the batchnorm layers, so at some point a very small batch size will probably show worse performance, though this may be negated to some degree by adjusting the batchnorm momentum.\n\nTODO: perhaps investigate if lower batch sizes can/should be paired with lower batchnorm2d momentum?" }, "source": "meta" }