![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F390015%2F0651635164af67f5f07333f9e5821925%2FScreen%20Shot%202020-07-11%20at%2021.35.51.png?generation=1594474578515313&alt=media\")
What do you think about mixup in this competition?
\n\nI have updated kernel about evaluation with 3 implementation of mixup for this competition.
\n\nAt the moment I have used only Mixup#1, things that I have changed in baseline for getting new score:
\n\nAfter this I have got new best score:
\n\nOOF-score 0.6859 --> 0.6906\nLB score 0.7294 --> 0.7318
\n\nI hope it can help you.
", "messages": [ { "id": "858877", "postDate": "05/23/2020 22:16:23", "content": "What do you think about mixup in this competition?
\n\nI have updated kernel about evaluation with 3 implementation of mixup for this competition.
\n\nAt the moment I have used only Mixup#1, things that I have changed in baseline for getting new score:
\n\nAfter this I have got new best score:
\n\nOOF-score 0.6859 --> 0.6906\nLB score 0.7294 --> 0.7318
\n\nI hope it can help you.
", "rawMarkdown": "What do you think about mixup in this competition? \n\nI have updated [kernel about evaluation](https://www.kaggle.com/shonenkov/oof-evaluation-mixup-efficientdet) with 3 implementation of mixup for this competition. \n\nAt the moment I have used only Mixup#1, things that I have changed in [baseline](https://www.kaggle.com/shonenkov/training-efficientdet) for getting new score:\n\n- add mixup#1\n- equal proba mixup/cutmix\n- epoch 40 --> 100\n- patience 1 --> 2\n\nAfter this I have got new best score:\n\nOOF-score 0.6859 --> 0.6906\nLB score 0.7294 --> 0.7318\n\n\nI hope it can help you.", "votes": null }, { "id": "859072", "postDate": "05/24/2020 05:26:39", "content": "I haven't tried mixup #2 and #3, but I think #1 can be improved a bit:
\n\n+1 I did not test this, but I think we can get even better results if we use images (for the 2nd image, with a 2nd DataLoader) with a maximized number of boxes. There are some cases (especially in ethz_1 source) where we have too many boxes. Without this, the results could be \"too crowded\". (Method #2 and #3 solve this issue.)
", "rawMarkdown": "I haven't tried mixup #2 and #3, but I think #1 can be improved a bit:\n\n- Instead of an average, generate a lambda value (similarly, that the original mixup) using the beta distribution.\n- For the alpha param, use a high value (2, 5, or 10 maybe). With this higher param, we will get lambda near to 50% with a higher probability.\n- Calculate losses separately for the boxes from image1 and image2 and weight the losses by lambda. (I am not sure how to weight false positives; 0.5 maybe)\n\n+1 I did not test this, but I think we can get even better results if we use images (for the 2nd image, with a 2nd DataLoader) with a maximized number of boxes. There are some cases (especially in ethz_1 source) where we have too many boxes. Without this, the results could be \"too crowded\". (Method #2 and #3 solve this issue.)", "votes": null }, { "id": "859193", "postDate": "05/24/2020 08:13:21", "content": "Thank you, Peter! Idea for mixing losses is good, I have used it in another competition in classification task. But now I understand that detection task can improved also, thank you for this!
\n\nI think we don’t need in the 2nd loader, we can return image and 2 groups of boxes from one loader. And I will upgrade loss calculation and create new version of notebook with these improvements
", "rawMarkdown": "Thank you, Peter! Idea for mixing losses is good, I have used it in another competition in classification task. But now I understand that detection task can improved also, thank you for this!\n\nI think we don’t need in the 2nd loader, we can return image and 2 groups of boxes from one loader. And I will upgrade loss calculation and create new version of notebook with these improvements", "votes": null }, { "id": "871562", "postDate": "06/02/2020 13:11:39", "content": "@shonenkov How do mean by equal proba 1 or 2:\n 1. if random>0.5:\n cutmix\n if random>0.5:\n mixup\n else:\n others\n 2. if random>0.5:\n others\n else:\n if random>0.5:\n cutmix\n else:\n mixup
\n\nAnd what is update for mixup#2 and #3\nThanks
", "rawMarkdown": "shonenkov How do mean by equal proba 1 or 2:\n 1. if random>0.5:\n cutmix\n if random>0.5:\n mixup\n else:\n others\n 2. if random>0.5:\n others\n else:\n if random>0.5:\n cutmix\n else:\n mixup\n\nAnd what is update for mixup#2 and #3\nThanks", "votes": null }, { "id": "885480", "postDate": "06/14/2020 08:55:51", "content": "Hi @shonenkov have you try mixup with loss? Does it give some boost?😃
", "rawMarkdown": "Hi @shonenkov have you try mixup with loss? Does it give some boost?😃", "votes": null }, { "id": "885508", "postDate": "06/14/2020 09:19:27", "content": "@cswwp347724 I didn't try mixup with modification loss, not enough time :(
", "rawMarkdown": "cswwp347724 I didn't try mixup with modification loss, not enough time :(", "votes": null }, { "id": "886439", "postDate": "06/15/2020 03:39:28", "content": "Ok, your notebook is very impressive😃
", "rawMarkdown": "Ok, your notebook is very impressive😃", "votes": null }, { "id": "888674", "postDate": "06/16/2020 13:49:31", "content": "@pestipeti @shonenkov \nI like your idea about mixing losses but I want to be sure that I've understood correctly. Assuming a batch size equal to 1, the dataloader will return 2 images:\n- Does the model consider these two images as a batch and backprograte the resulting loss ?\n- Or do we need to run twice the model (once for img1 and once for img2) ? then backpropagating the sum of the 2 losses\nThanks
", "rawMarkdown": "pestipeti @shonenkov \nI like your idea about mixing losses but I want to be sure that I've understood correctly. Assuming a batch size equal to 1, the dataloader will return 2 images:\n- Does the model consider these two images as a batch and backprograte the resulting loss ?\n- Or do we need to run twice the model (once for img1 and once for img2) ? then backpropagating the sum of the 2 losses\nThanks", "votes": null }, { "id": "924521", "postDate": "07/11/2020 13:38:11", "content": "Hi, was wondering if folks are getting crazily fluctuating validation loss curve like this when mixup is applied:
\n\n
Are you applying mixup to the validation set too? (you shouldnt be)
", "rawMarkdown": "Are you applying mixup to the validation set too? (you shouldnt be)", "votes": null }, { "id": "924767", "postDate": "07/11/2020 16:12:14", "content": "nope, no mixup for the validation set, though i wonder if this means that if I apply mixup to it, its curve will be smooth like the train set's.
", "rawMarkdown": "nope, no mixup for the validation set, though i wonder if this means that if I apply mixup to it, its curve will be smooth like the train set's.", "votes": null } ], "comments": [ { "id": 859072, "author_name": "pestipeti", "author_url": "", "post_date": "05/24/2020 05:26:39", "content": "I haven't tried mixup #2 and #3, but I think #1 can be improved a bit:
\n\n+1 I did not test this, but I think we can get even better results if we use images (for the 2nd image, with a 2nd DataLoader) with a maximized number of boxes. There are some cases (especially in ethz_1 source) where we have too many boxes. Without this, the results could be \"too crowded\". (Method #2 and #3 solve this issue.)
", "votes": null, "replies": [ { "id": 859193, "author_name": "shonenkov", "author_url": "", "post_date": "05/24/2020 08:13:21", "content": "Thank you, Peter! Idea for mixing losses is good, I have used it in another competition in classification task. But now I understand that detection task can improved also, thank you for this!
\n\nI think we don’t need in the 2nd loader, we can return image and 2 groups of boxes from one loader. And I will upgrade loss calculation and create new version of notebook with these improvements
", "votes": null, "replies": [] }, { "id": 885480, "author_name": "cswwp347724", "author_url": "", "post_date": "06/14/2020 08:55:51", "content": "Hi @shonenkov have you try mixup with loss? Does it give some boost?😃
", "votes": null, "replies": [] }, { "id": 885508, "author_name": "shonenkov", "author_url": "", "post_date": "06/14/2020 09:19:27", "content": "@cswwp347724 I didn't try mixup with modification loss, not enough time :(
", "votes": null, "replies": [] }, { "id": 886439, "author_name": "cswwp347724", "author_url": "", "post_date": "06/15/2020 03:39:28", "content": "Ok, your notebook is very impressive😃
", "votes": null, "replies": [] }, { "id": 888674, "author_name": "alexj21", "author_url": "", "post_date": "06/16/2020 13:49:31", "content": "@pestipeti @shonenkov \nI like your idea about mixing losses but I want to be sure that I've understood correctly. Assuming a batch size equal to 1, the dataloader will return 2 images:\n- Does the model consider these two images as a batch and backprograte the resulting loss ?\n- Or do we need to run twice the model (once for img1 and once for img2) ? then backpropagating the sum of the 2 losses\nThanks
", "votes": null, "replies": [] } ] }, { "id": 871562, "author_name": "gyanendradas", "author_url": "", "post_date": "06/02/2020 13:11:39", "content": "@shonenkov How do mean by equal proba 1 or 2:\n 1. if random>0.5:\n cutmix\n if random>0.5:\n mixup\n else:\n others\n 2. if random>0.5:\n others\n else:\n if random>0.5:\n cutmix\n else:\n mixup
\n\nAnd what is update for mixup#2 and #3\nThanks
", "votes": null, "replies": [] }, { "id": 924521, "author_name": "jackchungchiehyu", "author_url": "", "post_date": "07/11/2020 13:38:11", "content": "Hi, was wondering if folks are getting crazily fluctuating validation loss curve like this when mixup is applied:
\n\n
Are you applying mixup to the validation set too? (you shouldnt be)
", "votes": null, "replies": [] }, { "id": 924767, "author_name": "jackchungchiehyu", "author_url": "", "post_date": "07/11/2020 16:12:14", "content": "nope, no mixup for the validation set, though i wonder if this means that if I apply mixup to it, its curve will be smooth like the train set's.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "858877": "What do you think about mixup in this competition? \n\nI have updated [kernel about evaluation](https://www.kaggle.com/shonenkov/oof-evaluation-mixup-efficientdet) with 3 implementation of mixup for this competition. \n\nAt the moment I have used only Mixup#1, things that I have changed in [baseline](https://www.kaggle.com/shonenkov/training-efficientdet) for getting new score:\n\n- add mixup#1\n- equal proba mixup/cutmix\n- epoch 40 --> 100\n- patience 1 --> 2\n\nAfter this I have got new best score:\n\nOOF-score 0.6859 --> 0.6906\nLB score 0.7294 --> 0.7318\n\n\nI hope it can help you.", "859072": "I haven't tried mixup #2 and #3, but I think #1 can be improved a bit:\n\n- Instead of an average, generate a lambda value (similarly, that the original mixup) using the beta distribution.\n- For the alpha param, use a high value (2, 5, or 10 maybe). With this higher param, we will get lambda near to 50% with a higher probability.\n- Calculate losses separately for the boxes from image1 and image2 and weight the losses by lambda. (I am not sure how to weight false positives; 0.5 maybe)\n\n+1 I did not test this, but I think we can get even better results if we use images (for the 2nd image, with a 2nd DataLoader) with a maximized number of boxes. There are some cases (especially in ethz_1 source) where we have too many boxes. Without this, the results could be \"too crowded\". (Method #2 and #3 solve this issue.)", "859193": "Thank you, Peter! Idea for mixing losses is good, I have used it in another competition in classification task. But now I understand that detection task can improved also, thank you for this!\n\nI think we don’t need in the 2nd loader, we can return image and 2 groups of boxes from one loader. And I will upgrade loss calculation and create new version of notebook with these improvements", "871562": "shonenkov How do mean by equal proba 1 or 2:\n 1. if random>0.5:\n cutmix\n if random>0.5:\n mixup\n else:\n others\n 2. if random>0.5:\n others\n else:\n if random>0.5:\n cutmix\n else:\n mixup\n\nAnd what is update for mixup#2 and #3\nThanks", "885480": "Hi @shonenkov have you try mixup with loss? Does it give some boost?😃", "885508": "cswwp347724 I didn't try mixup with modification loss, not enough time :(", "886439": "Ok, your notebook is very impressive😃", "888674": "pestipeti @shonenkov \nI like your idea about mixing losses but I want to be sure that I've understood correctly. Assuming a batch size equal to 1, the dataloader will return 2 images:\n- Does the model consider these two images as a batch and backprograte the resulting loss ?\n- Or do we need to run twice the model (once for img1 and once for img2) ? then backpropagating the sum of the 2 losses\nThanks", "924521": "Hi, was wondering if folks are getting crazily fluctuating validation loss curve like this when mixup is applied:\n\n", "924579": "Are you applying mixup to the validation set too? (you shouldnt be)", "924767": "nope, no mixup for the validation set, though i wonder if this means that if I apply mixup to it, its curve will be smooth like the train set's." }, "source": "meta" }