{
  "id": 129835,
  "title": "silver solution (44th) ",
  "content": "<p>Congratulations to winners! And Thank you to Kaggle for hosting this. </p>\n\n<p>My Bert model had USE features + Crawl embeddings.  USE features improved my score quite a bit. Crawl not so much. I had answers and questions model separate, a little later I realized I could have saved lot of time by having QA branch in the model itself (too late).  My post processing was from public kernel. (All PyTorch) </p>\n\n<p>Roberta was surprisingly good pretty close to Bert on public LB, considering the amount of time I spent on Roberta vs Bert. This morning I realized that my Roberta model isn't using  USE Features - it was too late, had to let it go. </p>\n\n<p><strong>Preprocessing:</strong> None for Bert, For crawl I used the preprocessing from public kernel. \n<strong>Tokenization</strong> Head + Tail approach. \nCustom loss function</p>\n\n<p>I'll have my training code on GitHub and I'll make my inference kernel also public. </p>\n\n<p>Look forward to reading better solutions ! </p>\n\n<p>Edit: <a href=\"https://www.kaggle.com/rashmibanthia/googlequest-inference?scriptVersionId=28474231\">Inference Kernel</a></p>",
  "comments": [
    {
      "id": 742022,
      "author_name": "all2num",
      "author_url": "",
      "post_date": "2020-02-11T03:24:13.160000",
      "content": "<p>Congratulations and thank you for share !\nWhat is the meaning of USE features and how to do it ？</p>",
      "votes": 1,
      "replies": [
        {
          "id": 742028,
          "author_name": "RB",
          "author_url": "",
          "post_date": "2020-02-11T03:28:13.167000",
          "content": "<p>USE - <a href=\"https://tfhub.dev/google/universal-sentence-encoder/4\">https://tfhub.dev/google/universal-sentence-encoder/4</a> \nI did fine tune the Bert sequence classification class from Huggingface. Once you customize this class you can pass whatever features you like, I just had a simple linear layer in the customized class.  </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 741895,
      "author_name": "Ailurophile",
      "author_url": "",
      "post_date": "2020-02-11T01:56:15.430000",
      "content": "<p>Congratulations!! <a href=\"/rashmibanthia\">@rashmibanthia</a> </p>",
      "votes": 1,
      "replies": []
    }
  ],
  "comment_count": 3,
  "messages": [
    {
      "id": 741806,
      "postDate": "2020-02-11T00:39:10.853Z",
      "content": "<p>Congratulations to winners! And Thank you to Kaggle for hosting this. </p>\n\n<p>My Bert model had USE features + Crawl embeddings.  USE features improved my score quite a bit. Crawl not so much. I had answers and questions model separate, a little later I realized I could have saved lot of time by having QA branch in the model itself (too late).  My post processing was from public kernel. (All PyTorch) </p>\n\n<p>Roberta was surprisingly good pretty close to Bert on public LB, considering the amount of time I spent on Roberta vs Bert. This morning I realized that my Roberta model isn't using  USE Features - it was too late, had to let it go. </p>\n\n<p><strong>Preprocessing:</strong> None for Bert, For crawl I used the preprocessing from public kernel. \n<strong>Tokenization</strong> Head + Tail approach. \nCustom loss function</p>\n\n<p>I'll have my training code on GitHub and I'll make my inference kernel also public. </p>\n\n<p>Look forward to reading better solutions ! </p>\n\n<p>Edit: <a href=\"https://www.kaggle.com/rashmibanthia/googlequest-inference?scriptVersionId=28474231\">Inference Kernel</a></p>",
      "rawMarkdown": "Congratulations to winners! And Thank you to Kaggle for hosting this. \n\nMy Bert model had USE features + Crawl embeddings.  USE features improved my score quite a bit. Crawl not so much. I had answers and questions model separate, a little later I realized I could have saved lot of time by having QA branch in the model itself (too late).  My post processing was from public kernel. (All PyTorch) \n\nRoberta was surprisingly good pretty close to Bert on public LB, considering the amount of time I spent on Roberta vs Bert. This morning I realized that my Roberta model isn't using  USE Features - it was too late, had to let it go. \n\n**Preprocessing:** None for Bert, For crawl I used the preprocessing from public kernel. \n**Tokenization** Head + Tail approach. \nCustom loss function\n\nI'll have my training code on GitHub and I'll make my inference kernel also public. \n\nLook forward to reading better solutions ! \n\nEdit: [Inference Kernel](https://www.kaggle.com/rashmibanthia/googlequest-inference?scriptVersionId=28474231)",
      "votes": 9
    },
    {
      "id": 742022,
      "postDate": "2020-02-11T03:24:13.160Z",
      "content": "<p>Congratulations and thank you for share !\nWhat is the meaning of USE features and how to do it ？</p>",
      "rawMarkdown": "Congratulations and thank you for share !\nWhat is the meaning of USE features and how to do it ？\n\n",
      "votes": 1,
      "replies": [
        {
          "id": 742028,
          "postDate": "2020-02-11T03:28:13.167Z",
          "content": "<p>USE - <a href=\"https://tfhub.dev/google/universal-sentence-encoder/4\">https://tfhub.dev/google/universal-sentence-encoder/4</a> \nI did fine tune the Bert sequence classification class from Huggingface. Once you customize this class you can pass whatever features you like, I just had a simple linear layer in the customized class.  </p>",
          "rawMarkdown": "USE - https://tfhub.dev/google/universal-sentence-encoder/4 \nI did fine tune the Bert sequence classification class from Huggingface. Once you customize this class you can pass whatever features you like, I just had a simple linear layer in the customized class.  ",
          "votes": 1
        }
      ]
    },
    {
      "id": 741895,
      "postDate": "2020-02-11T01:56:15.430Z",
      "content": "<p>Congratulations!! <a href=\"/rashmibanthia\">@rashmibanthia</a> </p>",
      "rawMarkdown": "Congratulations!! @rashmibanthia ",
      "votes": 1
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T19:07:15Z",
  "competition_id": 7968,
  "competition_slug": "google-quest-challenge",
  "team_id": 4181915,
  "solution_url": "/competitions/google-quest-challenge/writeups/rb-silver-solution-44th"
}