{
  "id": 359798,
  "title": "39th place solution",
  "content": "<p>Thank you for competiton hosts and competitors!</p>\n<p>This is 39th place solution.</p>\n<h1>Model</h1>\n<p>Ensemble 2 heads using pytorch.</p>\n<ul>\n<li>Backbone : ViT-H-14 + LAION2B</li>\n<li>Head1 : Dropout(0.1) -&gt; Linear(64) -&gt; Normalize</li>\n<li>Head2 : Linear(64) -&gt; Normalize</li>\n<li>Output : Head1 + Head2 (add)</li>\n</ul>\n<p>ViT -&gt; Head1 / Head2 -&gt; Normalize -&gt; Ensemble</p>\n<p>In training, backbone was freezed and trained head module  using arcface.</p>\n<h1>Dataset</h1>\n<ul>\n<li>Head1 : GLR2021 + ImageNet1K + Products10K</li>\n<li>Head2 : GLR2021 + ImageNet1K + Products10K + AlibabaGoods + FoodRecognition2022 + MET</li>\n</ul>\n<h1>What did not work</h1>\n<p>some trial was by tensorflow implementation.</p>\n<ul>\n<li>ArcFace (large s + small margin)</li>\n<li>SubCenter ArcFace (Some K values)</li>\n<li>AdaCos</li>\n<li>Head : SelfAttention + Linear</li>\n<li>Head : Linear, ReLU, Dropout, Linear</li>\n<li>Head : Linear, ReLU, Dropout, Linear, BatchNormalization</li>\n<li>Contrastive Learning (SimCLR, SupCon)</li>\n<li>Focal Loss</li>\n<li>Single model or ensemble other CLIP(RN50, ViT-B, ViT-L (224px, 336px), ViT-g), CNN, other Head based arcface</li>\n<li>Ensemble method (concat + AveragePooling, weighted add)</li>\n<li>Using other dataset </li>\n</ul>\n<h1>Question</h1>\n<ul>\n<li>DataAugmentation is not good?</li>\n<li>Ensemble of concatenating channel axis looked good, but score was lower than simple 64 dim<br>\nmodel1 -&gt; 32dim, model2 -&gt; 32dim, ensemble (64 dim)</li>\n<li>Instance level label is not good?</li>\n<li>2 stage classification is good?  1 stage : genre classsification , 2 stage : embedding  (I had no time to try)</li>\n</ul>",
  "comments": [],
  "comment_count": 0,
  "messages": [
    {
      "id": 1985833,
      "postDate": "2022-10-13T16:06:54.857Z",
      "content": "<p>Thank you for competiton hosts and competitors!</p>\n<p>This is 39th place solution.</p>\n<h1>Model</h1>\n<p>Ensemble 2 heads using pytorch.</p>\n<ul>\n<li>Backbone : ViT-H-14 + LAION2B</li>\n<li>Head1 : Dropout(0.1) -&gt; Linear(64) -&gt; Normalize</li>\n<li>Head2 : Linear(64) -&gt; Normalize</li>\n<li>Output : Head1 + Head2 (add)</li>\n</ul>\n<p>ViT -&gt; Head1 / Head2 -&gt; Normalize -&gt; Ensemble</p>\n<p>In training, backbone was freezed and trained head module  using arcface.</p>\n<h1>Dataset</h1>\n<ul>\n<li>Head1 : GLR2021 + ImageNet1K + Products10K</li>\n<li>Head2 : GLR2021 + ImageNet1K + Products10K + AlibabaGoods + FoodRecognition2022 + MET</li>\n</ul>\n<h1>What did not work</h1>\n<p>some trial was by tensorflow implementation.</p>\n<ul>\n<li>ArcFace (large s + small margin)</li>\n<li>SubCenter ArcFace (Some K values)</li>\n<li>AdaCos</li>\n<li>Head : SelfAttention + Linear</li>\n<li>Head : Linear, ReLU, Dropout, Linear</li>\n<li>Head : Linear, ReLU, Dropout, Linear, BatchNormalization</li>\n<li>Contrastive Learning (SimCLR, SupCon)</li>\n<li>Focal Loss</li>\n<li>Single model or ensemble other CLIP(RN50, ViT-B, ViT-L (224px, 336px), ViT-g), CNN, other Head based arcface</li>\n<li>Ensemble method (concat + AveragePooling, weighted add)</li>\n<li>Using other dataset </li>\n</ul>\n<h1>Question</h1>\n<ul>\n<li>DataAugmentation is not good?</li>\n<li>Ensemble of concatenating channel axis looked good, but score was lower than simple 64 dim<br>\nmodel1 -&gt; 32dim, model2 -&gt; 32dim, ensemble (64 dim)</li>\n<li>Instance level label is not good?</li>\n<li>2 stage classification is good?  1 stage : genre classsification , 2 stage : embedding  (I had no time to try)</li>\n</ul>",
      "rawMarkdown": "Thank you for competiton hosts and competitors!\n\nThis is 39th place solution.\n\n# Model\n\nEnsemble 2 heads using pytorch.\n- Backbone : ViT-H-14 + LAION2B\n- Head1 : Dropout(0.1) -> Linear(64) -> Normalize\n- Head2 : Linear(64) -> Normalize\n- Output : Head1 + Head2 (add)\n\nViT -> Head1 / Head2 -> Normalize -> Ensemble\n\nIn training, backbone was freezed and trained head module  using arcface.\n\n# Dataset\n\n- Head1 : GLR2021 + ImageNet1K + Products10K\n- Head2 : GLR2021 + ImageNet1K + Products10K + AlibabaGoods + FoodRecognition2022 + MET\n\n# What did not work\n\nsome trial was by tensorflow implementation.\n\n- ArcFace (large s + small margin)\n- SubCenter ArcFace (Some K values)\n- AdaCos\n- Head : SelfAttention + Linear\n- Head : Linear, ReLU, Dropout, Linear\n- Head : Linear, ReLU, Dropout, Linear, BatchNormalization\n- Contrastive Learning (SimCLR, SupCon)\n- Focal Loss\n- Single model or ensemble other CLIP(RN50, ViT-B, ViT-L (224px, 336px), ViT-g), CNN, other Head based arcface\n- Ensemble method (concat + AveragePooling, weighted add)\n- Using other dataset \n\n# Question\n\n- DataAugmentation is not good?\n- Ensemble of concatenating channel axis looked good, but score was lower than simple 64 dim\n   model1 -> 32dim, model2 -> 32dim, ensemble (64 dim)\n- Instance level label is not good?\n- 2 stage classification is good?  1 stage : genre classsification , 2 stage : embedding  (I had no time to try)",
      "votes": 3
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-24T13:24:39Z",
  "competition_id": 36414,
  "competition_slug": "google-universal-image-embedding",
  "team_id": 8982791,
  "solution_url": "/competitions/google-universal-image-embedding/writeups/sukonbu-39th-place-solution"
}