I used EfficientNet and after a series of improvements, the LB finally reached 0.6. However, when I used the large model 'vit_large_patch14_dinov2.lvd142m' and Migrated all the previous strategies, it only reached 0.58. Does anyone have any recommended models? I think so far, the open-source training strategies seem to be quite different. I will continue to improve the current plan and observe which directions can bring improvements. After all, the dataset is too small, and many experiments must be done to possibly improve the LB.
", "messages": [ { "id": "3382240", "postDate": "12/27/2025 02:46:31", "content": "I used EfficientNet and after a series of improvements, the LB finally reached 0.6. However, when I used the large model 'vit_large_patch14_dinov2.lvd142m' and Migrated all the previous strategies, it only reached 0.58. Does anyone have any recommended models? I think so far, the open-source training strategies seem to be quite different. I will continue to improve the current plan and observe which directions can bring improvements. After all, the dataset is too small, and many experiments must be done to possibly improve the LB.
", "rawMarkdown": "I used EfficientNet and after a series of improvements, the LB finally reached 0.6. However, when I used the large model 'vit_large_patch14_dinov2.lvd142m' and Migrated all the previous strategies, it only reached 0.58. Does anyone have any recommended models? I think so far, the open-source training strategies seem to be quite different. I will continue to improve the current plan and observe which directions can bring improvements. After all, the dataset is too small, and many experiments must be done to possibly improve the LB.", "votes": null }, { "id": "3382241", "postDate": "12/27/2025 02:54:52", "content": "I have found that the \"best\" model backbone for this competition is highly dependent on the training strategy and image processing you decide to use. Some models seem to prefer certain pre-processing, resolutions, and tiling, while others are different. I haven't really found an approach where two fundamentally different backbones both performed well on the same strategy.
", "rawMarkdown": "I have found that the \"best\" model backbone for this competition is highly dependent on the training strategy and image processing you decide to use. Some models seem to prefer certain pre-processing, resolutions, and tiling, while others are different. I haven't really found an approach where two fundamentally different backbones both performed well on the same strategy.", "votes": null }, { "id": "3382270", "postDate": "12/27/2025 05:04:19", "content": "have you tried ConvNeXt or smaller ViT variants? Like you said, I think larger ViT typically need larger datasets to outperform other models.
", "rawMarkdown": "have you tried ConvNeXt or smaller ViT variants? Like you said, I think larger ViT typically need larger datasets to outperform other models.", "votes": null }, { "id": "3382274", "postDate": "12/27/2025 05:16:28", "content": "We need a bit more info - but did you set the image size correctly - v2 likes a size divisible by 14 - 224 for example - you might have used different size for efficientNet. You should be able to get to the 0.65 or so with v2 without any special magic besides the right parameters :)
\nPretty sure - the larger the model the harder to fine tune. And for sure tuning anything larger than a piano needs more than 357 images.
", "rawMarkdown": "We need a bit more info - but did you set the image size correctly - v2 likes a size divisible by 14 - 224 for example - you might have used different size for efficientNet. You should be able to get to the 0.65 or so with v2 without any special magic besides the right parameters :)\n\nPretty sure - the larger the model the harder to fine tune. And for sure tuning anything larger than a piano needs more than 357 images.", "votes": null }, { "id": "3382286", "postDate": "12/27/2025 06:03:04", "content": "My V2 uses 224 as input, and I employed hierarchical training to set different learning rates for the backbone and the head.
", "rawMarkdown": "My V2 uses 224 as input, and I employed hierarchical training to set different learning rates for the backbone and the head.", "votes": null }, { "id": "3382289", "postDate": "12/27/2025 06:06:42", "content": "I am planning to try the small version of the ViT model. Perhaps the small version of the ViT model has better generalization ability.
", "rawMarkdown": "I am planning to try the small version of the ViT model. Perhaps the small version of the ViT model has better generalization ability.", "votes": null }, { "id": "3382315", "postDate": "12/27/2025 07:35:11", "content": "How much are you freezing - that's were I saw a critical difference. I don't recall every freezing EfficentNet layers and seeing a huge difference, but do see it with Dino.
", "rawMarkdown": "How much are you freezing - that's were I saw a critical difference. I don't recall every freezing EfficentNet layers and seeing a huge difference, but do see it with Dino.", "votes": null }, { "id": "3382330", "postDate": "12/27/2025 08:12:45", "content": "I did not freeze the training; instead, it is a simplified hierarchical strategy.\n Config.backbone_lr = 5e-5\nConfig.head_lr = 2e-3
", "rawMarkdown": "I did not freeze the training; instead, it is a simplified hierarchical strategy.\n Config.backbone_lr = 5e-5\nConfig.head_lr = 2e-3", "votes": null }, { "id": "3382746", "postDate": "12/28/2025 12:50:26", "content": "What are the image sizes you are using with ViT. I was facing same problem where ConvNext and EfficientNet are yielding better result than ViT and Swin Transformer. My ConvNext and EfficientNet were using 512 x 512 sizes but my ViT was using 224 and Swin was using 384. My guess is loss of accuracy is due to the compression
", "rawMarkdown": "What are the image sizes you are using with ViT. I was facing same problem where ConvNext and EfficientNet are yielding better result than ViT and Swin Transformer. My ConvNext and EfficientNet were using 512 x 512 sizes but my ViT was using 224 and Swin was using 384. My guess is loss of accuracy is due to the compression", "votes": null }, { "id": "3382758", "postDate": "12/28/2025 13:20:45", "content": "I use 224, but I think this doesn't have a significant impact; the main thing is the training strategy.
", "rawMarkdown": "I use 224, but I think this doesn't have a significant impact; the main thing is the training strategy.", "votes": null } ], "comments": [ { "id": 3382241, "author_name": "chemdatafarmer", "author_url": "", "post_date": "12/27/2025 02:54:52", "content": "I have found that the \"best\" model backbone for this competition is highly dependent on the training strategy and image processing you decide to use. Some models seem to prefer certain pre-processing, resolutions, and tiling, while others are different. I haven't really found an approach where two fundamentally different backbones both performed well on the same strategy.
", "votes": null, "replies": [] }, { "id": 3382270, "author_name": "snoopydoo", "author_url": "", "post_date": "12/27/2025 05:04:19", "content": "have you tried ConvNeXt or smaller ViT variants? Like you said, I think larger ViT typically need larger datasets to outperform other models.
", "votes": null, "replies": [ { "id": 3382289, "author_name": "kagger3", "author_url": "", "post_date": "12/27/2025 06:06:42", "content": "I am planning to try the small version of the ViT model. Perhaps the small version of the ViT model has better generalization ability.
", "votes": null, "replies": [] } ] }, { "id": 3382274, "author_name": "pcjimmmy", "author_url": "", "post_date": "12/27/2025 05:16:28", "content": "We need a bit more info - but did you set the image size correctly - v2 likes a size divisible by 14 - 224 for example - you might have used different size for efficientNet. You should be able to get to the 0.65 or so with v2 without any special magic besides the right parameters :)
\nPretty sure - the larger the model the harder to fine tune. And for sure tuning anything larger than a piano needs more than 357 images.
", "votes": null, "replies": [ { "id": 3382286, "author_name": "kagger3", "author_url": "", "post_date": "12/27/2025 06:03:04", "content": "My V2 uses 224 as input, and I employed hierarchical training to set different learning rates for the backbone and the head.
", "votes": null, "replies": [ { "id": 3382315, "author_name": "pcjimmmy", "author_url": "", "post_date": "12/27/2025 07:35:11", "content": "How much are you freezing - that's were I saw a critical difference. I don't recall every freezing EfficentNet layers and seeing a huge difference, but do see it with Dino.
", "votes": null, "replies": [ { "id": 3382330, "author_name": "kagger3", "author_url": "", "post_date": "12/27/2025 08:12:45", "content": "I did not freeze the training; instead, it is a simplified hierarchical strategy.\n Config.backbone_lr = 5e-5\nConfig.head_lr = 2e-3
", "votes": null, "replies": [] } ] } ] } ] }, { "id": 3382746, "author_name": "vinodreddyg", "author_url": "", "post_date": "12/28/2025 12:50:26", "content": "What are the image sizes you are using with ViT. I was facing same problem where ConvNext and EfficientNet are yielding better result than ViT and Swin Transformer. My ConvNext and EfficientNet were using 512 x 512 sizes but my ViT was using 224 and Swin was using 384. My guess is loss of accuracy is due to the compression
", "votes": null, "replies": [ { "id": 3382758, "author_name": "kagger3", "author_url": "", "post_date": "12/28/2025 13:20:45", "content": "I use 224, but I think this doesn't have a significant impact; the main thing is the training strategy.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3382240": "I used EfficientNet and after a series of improvements, the LB finally reached 0.6. However, when I used the large model 'vit_large_patch14_dinov2.lvd142m' and Migrated all the previous strategies, it only reached 0.58. Does anyone have any recommended models? I think so far, the open-source training strategies seem to be quite different. I will continue to improve the current plan and observe which directions can bring improvements. After all, the dataset is too small, and many experiments must be done to possibly improve the LB.", "3382241": "I have found that the \"best\" model backbone for this competition is highly dependent on the training strategy and image processing you decide to use. Some models seem to prefer certain pre-processing, resolutions, and tiling, while others are different. I haven't really found an approach where two fundamentally different backbones both performed well on the same strategy.", "3382270": "have you tried ConvNeXt or smaller ViT variants? Like you said, I think larger ViT typically need larger datasets to outperform other models.", "3382274": "We need a bit more info - but did you set the image size correctly - v2 likes a size divisible by 14 - 224 for example - you might have used different size for efficientNet. You should be able to get to the 0.65 or so with v2 without any special magic besides the right parameters :)\n\nPretty sure - the larger the model the harder to fine tune. And for sure tuning anything larger than a piano needs more than 357 images.", "3382286": "My V2 uses 224 as input, and I employed hierarchical training to set different learning rates for the backbone and the head.", "3382289": "I am planning to try the small version of the ViT model. Perhaps the small version of the ViT model has better generalization ability.", "3382315": "How much are you freezing - that's were I saw a critical difference. I don't recall every freezing EfficentNet layers and seeing a huge difference, but do see it with Dino.", "3382330": "I did not freeze the training; instead, it is a simplified hierarchical strategy.\n Config.backbone_lr = 5e-5\nConfig.head_lr = 2e-3", "3382746": "What are the image sizes you are using with ViT. I was facing same problem where ConvNext and EfficientNet are yielding better result than ViT and Swin Transformer. My ConvNext and EfficientNet were using 512 x 512 sizes but my ViT was using 224 and Swin was using 384. My guess is loss of accuracy is due to the compression", "3382758": "I use 224, but I think this doesn't have a significant impact; the main thing is the training strategy." }, "source": "meta" }