{
  "id": 359402,
  "title": "37-th solution - Self Supervised Approach",
  "content": "<p>I know that is not the best solution but I want to share this very simple approach to score 6.37 in private LB. This is the recipe:<br>\n1) Freezed backbone of openCLIP ViT-H-14 on LAION-2B</p>\n<p>2) Trainable linear head (no batch normalization, no dropout)</p>\n<p>3) Training in self supervised learning on 130k dataset:<br>\n<a href=\"https://www.kaggle.com/datasets/rhtsingh/130k-images-512x512-universal-image-embeddings\" target=\"_blank\">https://www.kaggle.com/datasets/rhtsingh/130k-images-512x512-universal-image-embeddings</a></p>\n<ul>\n<li><p>contrastive loss:<br>\n<a href=\"https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#contrastiveloss\" target=\"_blank\">https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#contrastiveloss</a><br>\n(NTXentLoss also produce similar results)</p></li>\n<li><p>image batch are simply created as:<br>\n[batch, augment(batch)] (simclr style)<br>\nwhere augment is composed by:<br>\nrandomcrop, randomrotation, randomblur, colojitter, cutout</p></li>\n<li><p>Total batch size of 512 and adam as optimizer</p></li>\n<li><p>30 epochs</p></li>\n</ul>\n<p>4) TTA using square padding (zero filled) and simple 224 resize</p>\n<p>To conclude: maybe with a larger dataset the score can be improved but I have no time to test it. Hope that this could be helpful.</p>",
  "comments": [],
  "comment_count": 0,
  "messages": [
    {
      "id": 1983255,
      "postDate": "2022-10-11T22:13:54.733Z",
      "content": "<p>I know that is not the best solution but I want to share this very simple approach to score 6.37 in private LB. This is the recipe:<br>\n1) Freezed backbone of openCLIP ViT-H-14 on LAION-2B</p>\n<p>2) Trainable linear head (no batch normalization, no dropout)</p>\n<p>3) Training in self supervised learning on 130k dataset:<br>\n<a href=\"https://www.kaggle.com/datasets/rhtsingh/130k-images-512x512-universal-image-embeddings\" target=\"_blank\">https://www.kaggle.com/datasets/rhtsingh/130k-images-512x512-universal-image-embeddings</a></p>\n<ul>\n<li><p>contrastive loss:<br>\n<a href=\"https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#contrastiveloss\" target=\"_blank\">https://kevinmusgrave.github.io/pytorch-metric-learning/losses/#contrastiveloss</a><br>\n(NTXentLoss also produce similar results)</p></li>\n<li><p>image batch are simply created as:<br>\n[batch, augment(batch)] (simclr style)<br>\nwhere augment is composed by:<br>\nrandomcrop, randomrotation, randomblur, colojitter, cutout</p></li>\n<li><p>Total batch size of 512 and adam as optimizer</p></li>\n<li><p>30 epochs</p></li>\n</ul>\n<p>4) TTA using square padding (zero filled) and simple 224 resize</p>\n<p>To conclude: maybe with a larger dataset the score can be improved but I have no time to test it. Hope that this could be helpful.</p>",
      "rawMarkdown": "I know that is not the best solution but I want to share this very simple approach to score 6.37 in private LB. This is the recipe:\n1) Freezed backbone of openCLIP ViT-H-14 on LAION-2B\n\n2) Trainable linear head (no batch normalization, no dropout)\n\n3) Training in self supervised learning on 130k dataset:\nhttps://www.kaggle.com/datasets/rhtsingh/130k-images-512x512-universal-image-embeddings\n\n- contrastive loss:\nhttps://kevinmusgrave.github.io/pytorch-metric-learning/losses/#contrastiveloss\n(NTXentLoss also produce similar results)\n\n- image batch are simply created as:\n[batch, augment(batch)] (simclr style)\nwhere augment is composed by:\nrandomcrop, randomrotation, randomblur, colojitter, cutout\n\n- Total batch size of 512 and adam as optimizer\n\n- 30 epochs\n\n4) TTA using square padding (zero filled) and simple 224 resize\n\nTo conclude: maybe with a larger dataset the score can be improved but I have no time to test it. Hope that this could be helpful.",
      "votes": 9
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-24T13:24:07Z",
  "competition_id": 36414,
  "competition_slug": "google-universal-image-embedding",
  "team_id": 8956637,
  "solution_url": "/competitions/google-universal-image-embedding/writeups/climbing-power-37-th-solution-self-supervised-appr"
}