{
  "id": 359198,
  "title": "Private 55th Solution and Lessons Learned",
  "content": "<p>Thanks to Kaggle and Google for this amazing competition, <a href=\"https://www.kaggle.com/evilpsycho42\" target=\"_blank\">@evilpsycho42</a> and I enjoyed our time a lot in this competition. It is a special competition in all the competitions I attended so far that need competitors to gather all domains of image datasets to get the best performing model possible.</p>\n<p>Thanks to <a href=\"https://www.kaggle.com/dschettler8845\" target=\"_blank\">@dschettler8845</a> for initiating the external data thread and providing a list of links to help data, which saved us a lot of effort finding the data ourselves.</p>\n<h2>Brief Solution Overview</h2>\n<h3>Data:</h3>\n<p>Datasets</p>\n<ul>\n<li>apparel: h&amp;m</li>\n<li>landmark: google landmark competition</li>\n<li>package goods: product 10k, rp2k</li>\n<li>furniture: amazon product dataset</li>\n<li>storefronts: storefront-146 </li>\n<li>artwork: met</li>\n<li>dishes: ifood</li>\n</ul>\n<p>Datasets sampling:</p>\n<ul>\n<li>Assign # of instances for each domain proportional to the ratio mentioned in this <a href=\"https://ai.googleblog.com/2022/08/introducing-google-universal-image.html\" target=\"_blank\">post</a></li>\n<li>Use different ids in the train and validation set, since we want to generate our models to unseen instances.</li>\n</ul>\n<h3>Modeling</h3>\n<ul>\n<li>backbone: clip 224/336 L + linear neck</li>\n<li>loss: ArcFace (scale = 10, margin = 0.3)</li>\n<li>training:<ul>\n<li>phase 1: freeze backbone + CosineAnnealing with warmup 0.1, lr=0.001, 100 epochs</li>\n<li>phase 2: unfreeze backbone + CosineAnnealing with warmup  0.1, lr=1e-5, 20 epochs</li></ul></li>\n<li>sampling strategy: id uniform sampling</li>\n<li>validation strategy: <ul>\n<li>use different instances different from the train set</li>\n<li>pick instance with &gt; 1 images as query images, keep the rest as indexing images</li></ul></li>\n</ul>\n<h3>Results</h3>\n<ul>\n<li>clip 224 L: cv 0.7949, public lb: 0.580, private lb: 0.598</li>\n<li>clip 336 L: cv 0.8074, public lb: 0.597, private lb: 0.614</li>\n</ul>\n<h3>What doesn't work</h3>\n<ul>\n<li>model fusion: <ul>\n<li>train swin large 384 + clip together</li>\n<li>train swin large 384 + clip separately, combine both backbone and train a new projection head</li></ul></li>\n<li>image preprocessing\\croppring with saliency segmentation</li>\n</ul>\n<h2>Lessons Learned</h2>\n<p>After seeing the amazing solution from the 6th place <a href=\"https://www.kaggle.com/competitions/google-universal-image-embedding/discussion/359161\" target=\"_blank\">post</a>, we realized what we could have done better to deliver our final solution:</p>\n<ol>\n<li><strong>We should have tried different backbones</strong>: OpenCLIP ViT-H/14 224x224</li>\n<li><strong>We should try to overfit our models first</strong>. We used smaller lr and much fewer epochs to train the projection head while keeping the backbone frozen. Even though it is important to keep iteration fast, it is important not to be underfitting in our final solution!</li>\n</ol>",
  "comments": [
    {
      "id": 1982078,
      "author_name": "KKY",
      "author_url": "",
      "post_date": "2022-10-11T08:44:27.913000",
      "content": "<p>Nice Summarize,  some extra information:</p>\n<ol>\n<li>cls-level dataset,  not work</li>\n<li>cls-instance-mix dataset , not work</li>\n<li>Additional cls head,  train isntance / cls at the same time ,  not work</li>\n<li>some mistake,  tried laion400m_e32_swish_64/laion400m_e31_swish_64/laion2B-s32B-b82K_swish_64 weight in error-dataset,  when fixed dataset error,  forget re-try laion-weight..</li>\n</ol>\n<p>Very special competition, enjoyed it,  Learned much from here:</p>\n<ol>\n<li>The multimodal pre-training model has strong zero shot performance, generalization ability, and abstraction ability.</li>\n<li>Learned a lot pretrain methods like :  CLIP,  COCA, MOCO , MAE etc..</li>\n<li>parallel training</li>\n<li>No longer being afraid of very larege dataset.</li>\n</ol>",
      "votes": 2,
      "replies": []
    }
  ],
  "comment_count": 1,
  "messages": [
    {
      "id": 1981971,
      "postDate": "2022-10-11T06:58:06.720Z",
      "content": "<p>Thanks to Kaggle and Google for this amazing competition, <a href=\"https://www.kaggle.com/evilpsycho42\" target=\"_blank\">@evilpsycho42</a> and I enjoyed our time a lot in this competition. It is a special competition in all the competitions I attended so far that need competitors to gather all domains of image datasets to get the best performing model possible.</p>\n<p>Thanks to <a href=\"https://www.kaggle.com/dschettler8845\" target=\"_blank\">@dschettler8845</a> for initiating the external data thread and providing a list of links to help data, which saved us a lot of effort finding the data ourselves.</p>\n<h2>Brief Solution Overview</h2>\n<h3>Data:</h3>\n<p>Datasets</p>\n<ul>\n<li>apparel: h&amp;m</li>\n<li>landmark: google landmark competition</li>\n<li>package goods: product 10k, rp2k</li>\n<li>furniture: amazon product dataset</li>\n<li>storefronts: storefront-146 </li>\n<li>artwork: met</li>\n<li>dishes: ifood</li>\n</ul>\n<p>Datasets sampling:</p>\n<ul>\n<li>Assign # of instances for each domain proportional to the ratio mentioned in this <a href=\"https://ai.googleblog.com/2022/08/introducing-google-universal-image.html\" target=\"_blank\">post</a></li>\n<li>Use different ids in the train and validation set, since we want to generate our models to unseen instances.</li>\n</ul>\n<h3>Modeling</h3>\n<ul>\n<li>backbone: clip 224/336 L + linear neck</li>\n<li>loss: ArcFace (scale = 10, margin = 0.3)</li>\n<li>training:<ul>\n<li>phase 1: freeze backbone + CosineAnnealing with warmup 0.1, lr=0.001, 100 epochs</li>\n<li>phase 2: unfreeze backbone + CosineAnnealing with warmup  0.1, lr=1e-5, 20 epochs</li></ul></li>\n<li>sampling strategy: id uniform sampling</li>\n<li>validation strategy: <ul>\n<li>use different instances different from the train set</li>\n<li>pick instance with &gt; 1 images as query images, keep the rest as indexing images</li></ul></li>\n</ul>\n<h3>Results</h3>\n<ul>\n<li>clip 224 L: cv 0.7949, public lb: 0.580, private lb: 0.598</li>\n<li>clip 336 L: cv 0.8074, public lb: 0.597, private lb: 0.614</li>\n</ul>\n<h3>What doesn't work</h3>\n<ul>\n<li>model fusion: <ul>\n<li>train swin large 384 + clip together</li>\n<li>train swin large 384 + clip separately, combine both backbone and train a new projection head</li></ul></li>\n<li>image preprocessing\\croppring with saliency segmentation</li>\n</ul>\n<h2>Lessons Learned</h2>\n<p>After seeing the amazing solution from the 6th place <a href=\"https://www.kaggle.com/competitions/google-universal-image-embedding/discussion/359161\" target=\"_blank\">post</a>, we realized what we could have done better to deliver our final solution:</p>\n<ol>\n<li><strong>We should have tried different backbones</strong>: OpenCLIP ViT-H/14 224x224</li>\n<li><strong>We should try to overfit our models first</strong>. We used smaller lr and much fewer epochs to train the projection head while keeping the backbone frozen. Even though it is important to keep iteration fast, it is important not to be underfitting in our final solution!</li>\n</ol>",
      "rawMarkdown": "Thanks to Kaggle and Google for this amazing competition, @evilpsycho42 and I enjoyed our time a lot in this competition. It is a special competition in all the competitions I attended so far that need competitors to gather all domains of image datasets to get the best performing model possible.\n\nThanks to @dschettler8845 for initiating the external data thread and providing a list of links to help data, which saved us a lot of effort finding the data ourselves.\n\n## Brief Solution Overview\n\n### Data:\nDatasets\n- apparel: h&m\n- landmark: google landmark competition\n- package goods: product 10k, rp2k\n- furniture: amazon product dataset\n- storefronts: storefront-146 \n- artwork: met\n- dishes: ifood\n\nDatasets sampling:\n- Assign # of instances for each domain proportional to the ratio mentioned in this [post](https://ai.googleblog.com/2022/08/introducing-google-universal-image.html)\n- Use different ids in the train and validation set, since we want to generate our models to unseen instances.\n\n\n### Modeling\n- backbone: clip 224/336 L + linear neck\n- loss: ArcFace (scale = 10, margin = 0.3)\n- training:\n    - phase 1: freeze backbone + CosineAnnealing with warmup 0.1, lr=0.001, 100 epochs\n    - phase 2: unfreeze backbone + CosineAnnealing with warmup  0.1, lr=1e-5, 20 epochs\n- sampling strategy: id uniform sampling\n- validation strategy: \n    - use different instances different from the train set\n    - pick instance with > 1 images as query images, keep the rest as indexing images\n\n### Results\n- clip 224 L: cv 0.7949, public lb: 0.580, private lb: 0.598\n- clip 336 L: cv 0.8074, public lb: 0.597, private lb: 0.614\n\n### What doesn't work\n- model fusion: \n    - train swin large 384 + clip together\n    - train swin large 384 + clip separately, combine both backbone and train a new projection head\n- image preprocessing\\croppring with saliency segmentation\n\n## Lessons Learned\nAfter seeing the amazing solution from the 6th place [post](https://www.kaggle.com/competitions/google-universal-image-embedding/discussion/359161), we realized what we could have done better to deliver our final solution:\n1. **We should have tried different backbones**: OpenCLIP ViT-H/14 224x224\n2. **We should try to overfit our models first**. We used smaller lr and much fewer epochs to train the projection head while keeping the backbone frozen. Even though it is important to keep iteration fast, it is important not to be underfitting in our final solution!",
      "votes": 7
    },
    {
      "id": 1982078,
      "postDate": "2022-10-11T08:44:27.913Z",
      "content": "<p>Nice Summarize,  some extra information:</p>\n<ol>\n<li>cls-level dataset,  not work</li>\n<li>cls-instance-mix dataset , not work</li>\n<li>Additional cls head,  train isntance / cls at the same time ,  not work</li>\n<li>some mistake,  tried laion400m_e32_swish_64/laion400m_e31_swish_64/laion2B-s32B-b82K_swish_64 weight in error-dataset,  when fixed dataset error,  forget re-try laion-weight..</li>\n</ol>\n<p>Very special competition, enjoyed it,  Learned much from here:</p>\n<ol>\n<li>The multimodal pre-training model has strong zero shot performance, generalization ability, and abstraction ability.</li>\n<li>Learned a lot pretrain methods like :  CLIP,  COCA, MOCO , MAE etc..</li>\n<li>parallel training</li>\n<li>No longer being afraid of very larege dataset.</li>\n</ol>",
      "rawMarkdown": "Nice Summarize,  some extra information:\n1. cls-level dataset,  not work\n2. cls-instance-mix dataset , not work\n3. Additional cls head,  train isntance / cls at the same time ,  not work\n4. some mistake,  tried laion400m_e32_swish_64/laion400m_e31_swish_64/laion2B-s32B-b82K_swish_64 weight in error-dataset,  when fixed dataset error,  forget re-try laion-weight..\n\nVery special competition, enjoyed it,  Learned much from here:\n1. The multimodal pre-training model has strong zero shot performance, generalization ability, and abstraction ability.\n2. Learned a lot pretrain methods like :  CLIP,  COCA, MOCO , MAE etc..\n3. parallel training\n4. No longer being afraid of very larege dataset.",
      "votes": 2
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-24T13:23:20Z",
  "competition_id": 36414,
  "competition_slug": "google-universal-image-embedding",
  "team_id": 8919404,
  "solution_url": "/competitions/google-universal-image-embedding/writeups/private-55th-solution-and-lessons-learned"
}