Here are a few insights that I gained while training models for this challenge. You may see this as incentives to move away from Yolov5 (since it's no longer allowed, and choosing them as final submissions will be considered as violating the rules/cheating and will end up getting your positions removed). Since we are at the final weeks of this competition, I will refrain from sharing \"secrets.\"
\n\nWith a noisy dataset like this, finding a good validation strategy is nearly impossible. My take is that the best thing you can do is trust LB with a grain of salt, in the following ways:\n 1) I believe that it is accurate within 1% of range, e.g. a 0.77 model is comparable to a 0.76 one, but anything in between is sketchy.\n 2) I don't believe that any form of K-fold will work for the competition. First, the training/test sets are analogous but different distributions, which will break some of the underlying assumptions of Kfold. Additionally, the n-times overhead brought about by training multiple folds is too much for people running heavy models (unless you are doing weight-space averaging, which could also mess up your score). I suggest sticking to a pre-defined train/valid split. (I could also be horribly wrong on this one :( )\n3) Some images are more noisy than others, and some images look \"more similar\" to the test images than others.\n4) By the nature of a competition, the goal is never to find a \"well generalized\" model, but the model that performs the best on the private test set. This is partially why pseudo-labeling works so well, not just because it is generally a form of consistency regularization, but it also takes into account our desired distribution.
Proposal-suppression-based object detection is usually extremely sensitive wrt. to the choice of box suppression method and threshold, which has an interesting interaction with incremental development of tricks and models. Here is something I observed when probing the public LB. We already know that is pretty noisy, but naturally we may be able to \"incrementally\" test if some tricks works, like this:
\n\n1) train a naive model, obtain local performance l_0, and submit under a set of box suppression hyper-parameter (denoted A), we obtain a performance measure P\n2) retrain the same model with trick t_1 and get l_1, if l_1 is lower than l_0, evaluate under A and minor variations of it and obtain a set of P_1s\n3) Most of P_1 should be better than P_0, and if l_n is better than l_m, P_n should be better than P_m\n.... and so on, and you may wanna do combinations of tricks like t_12 and P_12 to figure out interactions between them.
But again, what we have is a generally unreliable validation and also a unreliable public leaderboard, so the above process does not always work. Empirically, here's a partial list of things that do hold under a fixed A:\n - hyper-parameters of an LR method\n - hyper-parameters of an augmentation\n - existence, or lack thereof, of an augmentation\n - loss function\n - data sources
\n\nAnd a partial list of what surprisingly don't seem to work:\n - input-altering techniques\n - choice of LR schedule\n - optimizing techniques
\n\n(Obviously changing the validation scheme will make the entire process invalid, so I didn't list them here.)
\n\nWhat's the significance of this? Let's show an example:
\n\nSay I train two YoloV5 models with exactly the same configurations till convergence (and assume that there isn't any optimization issues), except one of the include the --multi-scale option, and the other isn't. If you submit both of them to LB, some of you (depending on your exact setup) will find out that the multiscale model won't necessarily perform better (in terms of public LB) than your vanilla one. The reason is that the multiscale model may perform better under a different set of suppression hyperparameters A', which is also different for both public and private LB. Note that I did not attempt to predict which models will perform better under private LB, and how to choose a good A that performs the best under the private set.
Kaggle competitions sometimes have a very saturated leaderboard, in the sense of similarity between top solutions, or a poorly-designed dataset. In these cases, sometimes a model can win solely because the team stumbled upon a good seed that performs well on the private set without them knowing it. It is indeed frustrating when working with these competitions, but again there is still a way to get ahead of everyone: find the \"secret.\"
\n\nI will update more insights in this post in the following days.
", "messages": [ { "id": 927359, "postDate": "2020-07-13T10:44:53.577Z", "content": "Here are a few insights that I gained while training models for this challenge. You may see this as incentives to move away from Yolov5 (since it's no longer allowed, and choosing them as final submissions will be considered as violating the rules/cheating and will end up getting your positions removed). Since we are at the final weeks of this competition, I will refrain from sharing \"secrets.\"
\n\nWith a noisy dataset like this, finding a good validation strategy is nearly impossible. My take is that the best thing you can do is trust LB with a grain of salt, in the following ways:\n 1) I believe that it is accurate within 1% of range, e.g. a 0.77 model is comparable to a 0.76 one, but anything in between is sketchy.\n 2) I don't believe that any form of K-fold will work for the competition. First, the training/test sets are analogous but different distributions, which will break some of the underlying assumptions of Kfold. Additionally, the n-times overhead brought about by training multiple folds is too much for people running heavy models (unless you are doing weight-space averaging, which could also mess up your score). I suggest sticking to a pre-defined train/valid split. (I could also be horribly wrong on this one :( )\n3) Some images are more noisy than others, and some images look \"more similar\" to the test images than others.\n4) By the nature of a competition, the goal is never to find a \"well generalized\" model, but the model that performs the best on the private test set. This is partially why pseudo-labeling works so well, not just because it is generally a form of consistency regularization, but it also takes into account our desired distribution.
Proposal-suppression-based object detection is usually extremely sensitive wrt. to the choice of box suppression method and threshold, which has an interesting interaction with incremental development of tricks and models. Here is something I observed when probing the public LB. We already know that is pretty noisy, but naturally we may be able to \"incrementally\" test if some tricks works, like this:
\n\n1) train a naive model, obtain local performance l_0, and submit under a set of box suppression hyper-parameter (denoted A), we obtain a performance measure P\n2) retrain the same model with trick t_1 and get l_1, if l_1 is lower than l_0, evaluate under A and minor variations of it and obtain a set of P_1s\n3) Most of P_1 should be better than P_0, and if l_n is better than l_m, P_n should be better than P_m\n.... and so on, and you may wanna do combinations of tricks like t_12 and P_12 to figure out interactions between them.
But again, what we have is a generally unreliable validation and also a unreliable public leaderboard, so the above process does not always work. Empirically, here's a partial list of things that do hold under a fixed A:\n - hyper-parameters of an LR method\n - hyper-parameters of an augmentation\n - existence, or lack thereof, of an augmentation\n - loss function\n - data sources
\n\nAnd a partial list of what surprisingly don't seem to work:\n - input-altering techniques\n - choice of LR schedule\n - optimizing techniques
\n\n(Obviously changing the validation scheme will make the entire process invalid, so I didn't list them here.)
\n\nWhat's the significance of this? Let's show an example:
\n\nSay I train two YoloV5 models with exactly the same configurations till convergence (and assume that there isn't any optimization issues), except one of the include the --multi-scale option, and the other isn't. If you submit both of them to LB, some of you (depending on your exact setup) will find out that the multiscale model won't necessarily perform better (in terms of public LB) than your vanilla one. The reason is that the multiscale model may perform better under a different set of suppression hyperparameters A', which is also different for both public and private LB. Note that I did not attempt to predict which models will perform better under private LB, and how to choose a good A that performs the best under the private set.
Kaggle competitions sometimes have a very saturated leaderboard, in the sense of similarity between top solutions, or a poorly-designed dataset. In these cases, sometimes a model can win solely because the team stumbled upon a good seed that performs well on the private set without them knowing it. It is indeed frustrating when working with these competitions, but again there is still a way to get ahead of everyone: find the \"secret.\"
\n\nI will update more insights in this post in the following days.
", "rawMarkdown": "Here are a few insights that I gained while training models for this challenge. You may see this as incentives to **move away from Yolov5** (since it's no longer allowed, and choosing them as final submissions will be considered as violating the rules/cheating and will end up getting your positions removed). Since we are at the final weeks of this competition, I will refrain from sharing \"secrets.\"\n\n1. With a noisy dataset like this, finding a good validation strategy is nearly impossible. My take is that the best thing you can do is trust LB *with a grain of salt*, in the following ways:\n 1) I believe that it is accurate within 1% of range, e.g. a 0.77 model is comparable to a 0.76 one, but anything in between is sketchy.\n 2) I don't believe that any form of K-fold will work for the competition. First, the training/test sets are analogous but different distributions, which will break some of the underlying assumptions of Kfold. Additionally, the n-times overhead brought about by training multiple folds is too much for people running heavy models (unless you are doing weight-space averaging, which could also mess up your score). I suggest sticking to a pre-defined train/valid split. (I could also be horribly wrong on this one :( )\n 3) Some images are more noisy than others, and some images look \"more similar\" to the test images than others.\n 4) By the nature of a competition, the goal is never to find a \"well generalized\" model, but the model that performs the best on the private test set. This is partially why pseudo-labeling works so well, not just because it is generally a form of consistency regularization, but it also takes into account our desired distribution.\n\n2. Proposal-suppression-based object detection is usually **extremely sensitive** wrt. to the choice of box suppression method and threshold, which has an interesting interaction with incremental development of tricks and models. Here is something I observed when probing the public LB. We already know that is pretty noisy, but naturally we may be able to \"incrementally\" test if some tricks works, like this:\n\n 1) train a naive model, obtain local performance `l_0`, and submit under a set of box suppression hyper-parameter (denoted `A`), we obtain a performance measure `P`\n 2) retrain the same model with trick `t_1` and get `l_1`, if `l_1` is lower than `l_0`, evaluate under `A` and minor variations of it and obtain a set of `P_1`s\n 3) Most of `P_1` should be better than `P_0`, and if `l_n` is better than `l_m`, `P_n` should be better than `P_m`\n .... and so on, and you may wanna do combinations of tricks like `t_12` and `P_12` to figure out interactions between them.\n\nBut again, what we have is a generally unreliable validation and also a unreliable public leaderboard, so the above process does not always work. Empirically, here's a partial list of things that do hold under a fixed A:\n - hyper-parameters of an LR method\n - hyper-parameters of an augmentation\n - existence, or lack thereof, of an augmentation\n - loss function\n - data sources\n\nAnd a partial list of what surprisingly don't seem to work:\n - input-altering techniques\n - choice of LR schedule\n - optimizing techniques\n\n(Obviously changing the validation scheme will make the entire process invalid, so I didn't list them here.)\n\nWhat's the significance of this? Let's show an example:\n\nSay I train two YoloV5 models with exactly the same configurations till convergence (and assume that there isn't any optimization issues), except one of the include the `--multi-scale` option, and the other isn't. If you submit both of them to LB, some of you (depending on your exact setup) will find out that the multiscale model won't necessarily perform better (in terms of public LB) than your vanilla one. The reason is that the multiscale model may perform better under a different set of suppression hyperparameters A', which is also different for both public and private LB. Note that I did not attempt to predict which models will perform better under private LB, and how to choose a good A that performs the best under the private set.\n\nKaggle competitions sometimes have a very saturated leaderboard, in the sense of similarity between top solutions, or a poorly-designed dataset. In these cases, sometimes a model can win solely because the team stumbled upon a good seed that performs well on the private set without them knowing it. It is indeed frustrating when working with these competitions, but again there is still a way to get ahead of everyone: find the \"secret.\"\n\nI will update more insights in this post in the following days. \n \n", "votes": 47 }, { "id": 930885, "postDate": "2020-07-15T19:34:07.957Z", "content": "If I can add my 2 cents, I learnt over the years from the GMs:\n1. if it is not possible to find a stable CV, it is a lottery => GET OUT\n2. if there is a leak or change at the last minute, it usually put a mess in the ranking and it can become a real lottery => GET OUT
\n\nBut I am still here...
\n\nWhat worries me, as Liao mentionned in between the lines, is given that the test set is smaller than the train set (really important for pseudo labeling to work well) and that the hidden test set is extremely likely to have the same distribution than the LB (see paper prepared by sponsor), the pseudo labeling is likely to be essential and we are not really looking for a generalization of the model.
", "rawMarkdown": "If I can add my 2 cents, I learnt over the years from the GMs:\n1. if it is not possible to find a stable CV, it is a lottery => GET OUT\n2. if there is a leak or change at the last minute, it usually put a mess in the ranking and it can become a real lottery => GET OUT\n\nBut I am still here...\n\nWhat worries me, as Liao mentionned in between the lines, is given that the test set is smaller than the train set (really important for pseudo labeling to work well) and that the hidden test set is extremely likely to have the same distribution than the LB (see paper prepared by sponsor), the pseudo labeling is likely to be essential and we are not really looking for a generalization of the model.", "votes": 7, "replies": [ { "id": 931091, "postDate": "2020-07-16T00:55:35.337Z", "content": "Well said. My opinions on why pseudo-labels work:
\n\nIt addresses the domain shift between train and test set. From my few months of tuning an object detection model, I've realized that it is really sensitive w.r.t. the labeling quality. There is a paper (forgot the name) pointing out that 20% label noise in COCO (by randomly shifting bbox coords by +/- 5% or 10%) can bring about a huge performance drop in SOTA detectors.
It is easy to train. There are multiple consistency-based semi-supervised learning methods that outperform pseudo-labeling, such as VAT, Pi-Model, Mean-Teacher, etc, but they are hard to re-adapt under a 1- or 2-stage, anchor/anchor-free- object detection framework. I tried quite a few formulations but none of them can converge within the 6-hrs single GPU compute framework.
And, since YoloV5 is really out of the question (see here), I do think that there are non-trivial tricks to boost model performance besides the simple 5-fold average, hyper-parameter tuning and WBF threshold tuning.
\n\nI've trained around 15 novel models for this competition, and only a few turned out to work. I will write a comprehensive report after the end of the competition.
", "rawMarkdown": "Well said. My opinions on why pseudo-labels work:\n\n1. It addresses the domain shift between train and test set. From my few months of tuning an object detection model, I've realized that it is really sensitive w.r.t. the labeling quality. There is a paper (forgot the name) pointing out that 20% label noise in COCO (by randomly shifting bbox coords by +/- 5% or 10%) can bring about a huge performance drop in SOTA detectors.\n\n2. It is easy to train. There are multiple consistency-based semi-supervised learning methods that outperform pseudo-labeling, such as VAT, Pi-Model, Mean-Teacher, etc, but they are hard to re-adapt under a 1- or 2-stage, anchor/anchor-free- object detection framework. I tried quite a few formulations but none of them can converge within the 6-hrs single GPU compute framework.\n\nAnd, since YoloV5 is really out of the question (see [here](https://www.kaggle.com/c/global-wheat-detection/discussion/167140#930712)), I do think that there are non-trivial tricks to boost model performance besides the simple 5-fold average, hyper-parameter tuning and WBF threshold tuning.\n\nI've trained around 15 novel models for this competition, and only a few turned out to work. I will write a comprehensive report after the end of the competition.", "votes": 13 } ] }, { "id": 929272, "postDate": "2020-07-14T15:17:15.580Z", "content": "For those at the beginning of moving away from yolov5, can you provide some general advice on which models are useful to pursue/learn
", "rawMarkdown": "For those at the beginning of moving away from yolov5, can you provide some general advice on which models are useful to pursue/learn", "votes": 1, "replies": [ { "id": 929626, "postDate": "2020-07-14T19:56:30.700Z", "content": "Hi James, you can find a list of some popular frameworks and their licenses here. Many people seem to be moving to EfficientDet as it had good performance before YoloV5. Personally, I am still using FasterRCNN.
\n\nIf you plan on using a framework with Apache license, look here for advice from the creator of the pytorch EffDet implementation!
", "rawMarkdown": "Hi James, you can find a list of some popular frameworks and their licenses [here](https://www.kaggle.com/c/global-wheat-detection/discussion/165735#924609). Many people seem to be moving to EfficientDet as it had good performance before YoloV5. Personally, I am still using FasterRCNN.\n\nIf you plan on using a framework with Apache license, look [here](https://www.kaggle.com/c/global-wheat-detection/discussion/164845#923524) for advice from the creator of the pytorch EffDet implementation!", "votes": 1 }, { "id": 930610, "postDate": "2020-07-15T15:36:35.283Z", "content": "Thanks Matthew
", "rawMarkdown": "Thanks Matthew" } ] }, { "id": 927404, "postDate": "2020-07-13T11:05:15.203Z", "content": "Thanks for sharing.
\n\nAlthough yolov5 was not available, some of the parts mentioned were the same in yolov5.\n(It may vary depending on computing environment.)
\n\nI wonder what will take the place of yolov5.
", "rawMarkdown": "Thanks for sharing.\n\nAlthough yolov5 was not available, some of the parts mentioned were the same in yolov5.\n(It may vary depending on computing environment.)\n\nI wonder what will take the place of yolov5.", "votes": 1 }, { "id": 939117, "postDate": "2020-07-22T03:40:03.323Z", "content": "Thank you for your tips @alexanderliao, one question: Is effdet allowed?👀
", "rawMarkdown": "Thank you for your tips @alexanderliao, one question: Is effdet allowed?👀 " }, { "id": 938072, "postDate": "2020-07-21T10:27:02.077Z", "content": "I think some target should be droped.
", "rawMarkdown": "I think some target should be droped." }, { "id": 927635, "postDate": "2020-07-13T14:14:48.323Z", "rawMarkdown": "", "votes": -23, "isDeleted": true }, { "id": 930589, "postDate": "2020-07-15T15:15:38.253Z", "content": "Thanks for sharing!
", "rawMarkdown": "Thanks for sharing!" } ], "comments": [ { "id": 930885, "author_name": "eagle4", "author_url": "", "post_date": "2020-07-15T19:34:07.957000", "content": "If I can add my 2 cents, I learnt over the years from the GMs:\n1. if it is not possible to find a stable CV, it is a lottery => GET OUT\n2. if there is a leak or change at the last minute, it usually put a mess in the ranking and it can become a real lottery => GET OUT
\n\nBut I am still here...
\n\nWhat worries me, as Liao mentionned in between the lines, is given that the test set is smaller than the train set (really important for pseudo labeling to work well) and that the hidden test set is extremely likely to have the same distribution than the LB (see paper prepared by sponsor), the pseudo labeling is likely to be essential and we are not really looking for a generalization of the model.
", "votes": 7, "replies": [ { "id": 931091, "author_name": "Peiyuan Liao", "author_url": "", "post_date": "2020-07-16T00:55:35.337000", "content": "Well said. My opinions on why pseudo-labels work:
\n\nIt addresses the domain shift between train and test set. From my few months of tuning an object detection model, I've realized that it is really sensitive w.r.t. the labeling quality. There is a paper (forgot the name) pointing out that 20% label noise in COCO (by randomly shifting bbox coords by +/- 5% or 10%) can bring about a huge performance drop in SOTA detectors.
It is easy to train. There are multiple consistency-based semi-supervised learning methods that outperform pseudo-labeling, such as VAT, Pi-Model, Mean-Teacher, etc, but they are hard to re-adapt under a 1- or 2-stage, anchor/anchor-free- object detection framework. I tried quite a few formulations but none of them can converge within the 6-hrs single GPU compute framework.
And, since YoloV5 is really out of the question (see here), I do think that there are non-trivial tricks to boost model performance besides the simple 5-fold average, hyper-parameter tuning and WBF threshold tuning.
\n\nI've trained around 15 novel models for this competition, and only a few turned out to work. I will write a comprehensive report after the end of the competition.
", "votes": 13, "replies": [] } ] }, { "id": 929272, "author_name": "J Seb", "author_url": "", "post_date": "2020-07-14T15:17:15.580000", "content": "For those at the beginning of moving away from yolov5, can you provide some general advice on which models are useful to pursue/learn
", "votes": 1, "replies": [ { "id": 929626, "author_name": "Matt", "author_url": "", "post_date": "2020-07-14T19:56:30.700000", "content": "Hi James, you can find a list of some popular frameworks and their licenses here. Many people seem to be moving to EfficientDet as it had good performance before YoloV5. Personally, I am still using FasterRCNN.
\n\nIf you plan on using a framework with Apache license, look here for advice from the creator of the pytorch EffDet implementation!
", "votes": 1, "replies": [] }, { "id": 930610, "author_name": "J Seb", "author_url": "", "post_date": "2020-07-15T15:36:35.283000", "content": "Thanks Matthew
", "votes": 0, "replies": [] } ] }, { "id": 927404, "author_name": "Heroseo", "author_url": "", "post_date": "2020-07-13T11:05:15.203000", "content": "Thanks for sharing.
\n\nAlthough yolov5 was not available, some of the parts mentioned were the same in yolov5.\n(It may vary depending on computing environment.)
\n\nI wonder what will take the place of yolov5.
", "votes": 1, "replies": [] }, { "id": 939117, "author_name": "cswwp", "author_url": "", "post_date": "2020-07-22T03:40:03.323000", "content": "Thank you for your tips @alexanderliao, one question: Is effdet allowed?👀
", "votes": 0, "replies": [] }, { "id": 938072, "author_name": "HEEEEE", "author_url": "", "post_date": "2020-07-21T10:27:02.077000", "content": "I think some target should be droped.
", "votes": 0, "replies": [] }, { "id": 927635, "author_name": "", "author_url": "", "post_date": "2020-07-13T14:14:48.323000", "content": "", "votes": -23, "replies": [] }, { "id": 930589, "author_name": "Malyaban Bal", "author_url": "", "post_date": "2020-07-15T15:15:38.253000", "content": "Thanks for sharing!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "927359": "Here are a few insights that I gained while training models for this challenge. You may see this as incentives to **move away from Yolov5** (since it's no longer allowed, and choosing them as final submissions will be considered as violating the rules/cheating and will end up getting your positions removed). Since we are at the final weeks of this competition, I will refrain from sharing \"secrets.\"\n\n1. With a noisy dataset like this, finding a good validation strategy is nearly impossible. My take is that the best thing you can do is trust LB *with a grain of salt*, in the following ways:\n 1) I believe that it is accurate within 1% of range, e.g. a 0.77 model is comparable to a 0.76 one, but anything in between is sketchy.\n 2) I don't believe that any form of K-fold will work for the competition. First, the training/test sets are analogous but different distributions, which will break some of the underlying assumptions of Kfold. Additionally, the n-times overhead brought about by training multiple folds is too much for people running heavy models (unless you are doing weight-space averaging, which could also mess up your score). I suggest sticking to a pre-defined train/valid split. (I could also be horribly wrong on this one :( )\n 3) Some images are more noisy than others, and some images look \"more similar\" to the test images than others.\n 4) By the nature of a competition, the goal is never to find a \"well generalized\" model, but the model that performs the best on the private test set. This is partially why pseudo-labeling works so well, not just because it is generally a form of consistency regularization, but it also takes into account our desired distribution.\n\n2. Proposal-suppression-based object detection is usually **extremely sensitive** wrt. to the choice of box suppression method and threshold, which has an interesting interaction with incremental development of tricks and models. Here is something I observed when probing the public LB. We already know that is pretty noisy, but naturally we may be able to \"incrementally\" test if some tricks works, like this:\n\n 1) train a naive model, obtain local performance `l_0`, and submit under a set of box suppression hyper-parameter (denoted `A`), we obtain a performance measure `P`\n 2) retrain the same model with trick `t_1` and get `l_1`, if `l_1` is lower than `l_0`, evaluate under `A` and minor variations of it and obtain a set of `P_1`s\n 3) Most of `P_1` should be better than `P_0`, and if `l_n` is better than `l_m`, `P_n` should be better than `P_m`\n .... and so on, and you may wanna do combinations of tricks like `t_12` and `P_12` to figure out interactions between them.\n\nBut again, what we have is a generally unreliable validation and also a unreliable public leaderboard, so the above process does not always work. Empirically, here's a partial list of things that do hold under a fixed A:\n - hyper-parameters of an LR method\n - hyper-parameters of an augmentation\n - existence, or lack thereof, of an augmentation\n - loss function\n - data sources\n\nAnd a partial list of what surprisingly don't seem to work:\n - input-altering techniques\n - choice of LR schedule\n - optimizing techniques\n\n(Obviously changing the validation scheme will make the entire process invalid, so I didn't list them here.)\n\nWhat's the significance of this? Let's show an example:\n\nSay I train two YoloV5 models with exactly the same configurations till convergence (and assume that there isn't any optimization issues), except one of the include the `--multi-scale` option, and the other isn't. If you submit both of them to LB, some of you (depending on your exact setup) will find out that the multiscale model won't necessarily perform better (in terms of public LB) than your vanilla one. The reason is that the multiscale model may perform better under a different set of suppression hyperparameters A', which is also different for both public and private LB. Note that I did not attempt to predict which models will perform better under private LB, and how to choose a good A that performs the best under the private set.\n\nKaggle competitions sometimes have a very saturated leaderboard, in the sense of similarity between top solutions, or a poorly-designed dataset. In these cases, sometimes a model can win solely because the team stumbled upon a good seed that performs well on the private set without them knowing it. It is indeed frustrating when working with these competitions, but again there is still a way to get ahead of everyone: find the \"secret.\"\n\nI will update more insights in this post in the following days. \n \n", "930885": "If I can add my 2 cents, I learnt over the years from the GMs:\n1. if it is not possible to find a stable CV, it is a lottery => GET OUT\n2. if there is a leak or change at the last minute, it usually put a mess in the ranking and it can become a real lottery => GET OUT\n\nBut I am still here...\n\nWhat worries me, as Liao mentionned in between the lines, is given that the test set is smaller than the train set (really important for pseudo labeling to work well) and that the hidden test set is extremely likely to have the same distribution than the LB (see paper prepared by sponsor), the pseudo labeling is likely to be essential and we are not really looking for a generalization of the model.", "929272": "For those at the beginning of moving away from yolov5, can you provide some general advice on which models are useful to pursue/learn", "927404": "Thanks for sharing.\n\nAlthough yolov5 was not available, some of the parts mentioned were the same in yolov5.\n(It may vary depending on computing environment.)\n\nI wonder what will take the place of yolov5.", "939117": "Thank you for your tips @alexanderliao, one question: Is effdet allowed?👀 ", "938072": "I think some target should be droped.", "927635": "", "930589": "Thanks for sharing!" } }