{
  "id": 129904,
  "title": "23th place solution",
  "content": "<p>First, thanks to my teammate @agatan .</p>\n\n<h2>Pre-processing</h2>\n\n<ul>\n<li>apply html unescape to sentences</li>\n<li>sentances longer than maximum length was trimmed at the head and tail parts</li>\n</ul>\n\n<h2>Modeling</h2>\n\n<ul>\n<li>We trained Question Bert-base model and Question and Anser Bert-base model separetely on kernel</li>\n<li>Question model predicts question-targets and Q-A model predicts all targets</li>\n<li>3fold and 3epochs with GroupKFold</li>\n<li>BCE + margin ranking loss</li>\n</ul>\n\n<h2>Post-processing</h2>\n\n<ul>\n<li>used lightgbm as second-stage stacking model</li>\n<li>max_depth=1 and lr=0.1 was best from my experiments</li>\n<li>Input meta features such as text-length to lightgbm addition to predicted values</li>\n</ul>\n\n<h2>Best model</h2>\n\n<ul>\n<li>CV: 0.3851</li>\n<li>Public: 0.45979</li>\n<li>Private: 0.41440</li>\n</ul>\n\n<p>We shared our kernel, please see if you like.\n<a href=\"https://www.kaggle.com/shuheigoda/23th-place-solusion\">https://www.kaggle.com/shuheigoda/23th-place-solusion</a></p>",
  "comments": [
    {
      "id": 742410,
      "author_name": "liuze",
      "author_url": "",
      "post_date": "2020-02-11T09:30:59.937000",
      "content": "<p>cool, do you try RoBerta?, I tried bert-base and roberta-base, and roberta is better</p>",
      "votes": 0,
      "replies": [
        {
          "id": 742432,
          "author_name": "hakubishin3",
          "author_url": "",
          "post_date": "2020-02-11T09:41:53.523000",
          "content": "<p>Thank you for sharing! We have not tried RoBerta... </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 742361,
      "author_name": "Ailurophile",
      "author_url": "",
      "post_date": "2020-02-11T08:59:13.773000",
      "content": "<p>Congratulations!\nThanks for Sharing your Approach &amp; Insights!! <a href=\"/shuheigoda\">@shuheigoda</a> </p>",
      "votes": 0,
      "replies": [
        {
          "id": 742377,
          "author_name": "hakubishin3",
          "author_url": "",
          "post_date": "2020-02-11T09:13:03.333000",
          "content": "<p>Thank you !!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 742359,
      "author_name": "MaChaogong",
      "author_url": "",
      "post_date": "2020-02-11T08:56:56.627000",
      "content": "<p>really creative. Could you also share the custom loss? how much better is that than single bce</p>",
      "votes": 0,
      "replies": [
        {
          "id": 742375,
          "author_name": "hakubishin3",
          "author_url": "",
          "post_date": "2020-02-11T09:12:44.353000",
          "content": "<p>Thank you! Details of the loss function can be found in our kernel.\n<a href=\"https://www.kaggle.com/shuheigoda/23th-place-solusion\">https://www.kaggle.com/shuheigoda/23th-place-solusion</a></p>\n\n<p>Compared to single bce, boost +0.002 public score with custom loss.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 742514,
          "author_name": "MaChaogong",
          "author_url": "",
          "post_date": "2020-02-11T10:35:01.703000",
          "content": "<p>thanks!! </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 742326,
      "author_name": "Gursewak Dhiman",
      "author_url": "",
      "post_date": "2020-02-11T08:28:51.923000",
      "content": "<p>Nice approach, I had tried xgboost as post processing approach but didn't had much success. Will look into your kernel to learn. </p>",
      "votes": 0,
      "replies": [
        {
          "id": 742329,
          "author_name": "hakubishin3",
          "author_url": "",
          "post_date": "2020-02-11T08:36:10.637000",
          "content": "<p>Thank you!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 742312,
      "author_name": "Aditya Soni",
      "author_url": "",
      "post_date": "2020-02-11T08:20:17.577000",
      "content": "<p>The kernel is private :)</p>",
      "votes": 0,
      "replies": [
        {
          "id": 742317,
          "author_name": "hakubishin3",
          "author_url": "",
          "post_date": "2020-02-11T08:23:43.843000",
          "content": "<p>sorry, I changed my kernel to public</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "comment_count": 11,
  "messages": [
    {
      "id": 742302,
      "postDate": "2020-02-11T08:16:55.407Z",
      "content": "<p>First, thanks to my teammate @agatan .</p>\n\n<h2>Pre-processing</h2>\n\n<ul>\n<li>apply html unescape to sentences</li>\n<li>sentances longer than maximum length was trimmed at the head and tail parts</li>\n</ul>\n\n<h2>Modeling</h2>\n\n<ul>\n<li>We trained Question Bert-base model and Question and Anser Bert-base model separetely on kernel</li>\n<li>Question model predicts question-targets and Q-A model predicts all targets</li>\n<li>3fold and 3epochs with GroupKFold</li>\n<li>BCE + margin ranking loss</li>\n</ul>\n\n<h2>Post-processing</h2>\n\n<ul>\n<li>used lightgbm as second-stage stacking model</li>\n<li>max_depth=1 and lr=0.1 was best from my experiments</li>\n<li>Input meta features such as text-length to lightgbm addition to predicted values</li>\n</ul>\n\n<h2>Best model</h2>\n\n<ul>\n<li>CV: 0.3851</li>\n<li>Public: 0.45979</li>\n<li>Private: 0.41440</li>\n</ul>\n\n<p>We shared our kernel, please see if you like.\n<a href=\"https://www.kaggle.com/shuheigoda/23th-place-solusion\">https://www.kaggle.com/shuheigoda/23th-place-solusion</a></p>",
      "rawMarkdown": "First, thanks to my teammate @agatan .\n\n## Pre-processing\n- apply html unescape to sentences\n- sentances longer than maximum length was trimmed at the head and tail parts\n\n## Modeling\n- We trained Question Bert-base model and Question and Anser Bert-base model separetely on kernel\n- Question model predicts question-targets and Q-A model predicts all targets\n- 3fold and 3epochs with GroupKFold\n- BCE + margin ranking loss\n\n## Post-processing\n- used lightgbm as second-stage stacking model\n- max_depth=1 and lr=0.1 was best from my experiments\n- Input meta features such as text-length to lightgbm addition to predicted values\n\n## Best model\n- CV: 0.3851\n- Public: 0.45979\n- Private: 0.41440\n\nWe shared our kernel, please see if you like.\nhttps://www.kaggle.com/shuheigoda/23th-place-solusion",
      "votes": 22
    },
    {
      "id": 742410,
      "postDate": "2020-02-11T09:30:59.937Z",
      "content": "<p>cool, do you try RoBerta?, I tried bert-base and roberta-base, and roberta is better</p>",
      "rawMarkdown": "cool, do you try RoBerta?, I tried bert-base and roberta-base, and roberta is better",
      "replies": [
        {
          "id": 742432,
          "postDate": "2020-02-11T09:41:53.523Z",
          "content": "<p>Thank you for sharing! We have not tried RoBerta... </p>",
          "rawMarkdown": "Thank you for sharing! We have not tried RoBerta... "
        }
      ]
    },
    {
      "id": 742361,
      "postDate": "2020-02-11T08:59:13.773Z",
      "content": "<p>Congratulations!\nThanks for Sharing your Approach &amp; Insights!! <a href=\"/shuheigoda\">@shuheigoda</a> </p>",
      "rawMarkdown": "Congratulations!\nThanks for Sharing your Approach &amp; Insights!! @shuheigoda ",
      "replies": [
        {
          "id": 742377,
          "postDate": "2020-02-11T09:13:03.333Z",
          "content": "<p>Thank you !!</p>",
          "rawMarkdown": "Thank you !!"
        }
      ]
    },
    {
      "id": 742359,
      "postDate": "2020-02-11T08:56:56.627Z",
      "content": "<p>really creative. Could you also share the custom loss? how much better is that than single bce</p>",
      "rawMarkdown": "really creative. Could you also share the custom loss? how much better is that than single bce",
      "replies": [
        {
          "id": 742375,
          "postDate": "2020-02-11T09:12:44.353Z",
          "content": "<p>Thank you! Details of the loss function can be found in our kernel.\n<a href=\"https://www.kaggle.com/shuheigoda/23th-place-solusion\">https://www.kaggle.com/shuheigoda/23th-place-solusion</a></p>\n\n<p>Compared to single bce, boost +0.002 public score with custom loss.</p>",
          "rawMarkdown": "Thank you! Details of the loss function can be found in our kernel.\nhttps://www.kaggle.com/shuheigoda/23th-place-solusion\n\nCompared to single bce, boost +0.002 public score with custom loss.",
          "votes": 1
        },
        {
          "id": 742514,
          "postDate": "2020-02-11T10:35:01.703Z",
          "content": "<p>thanks!! </p>",
          "rawMarkdown": "thanks!! "
        }
      ]
    },
    {
      "id": 742326,
      "postDate": "2020-02-11T08:28:51.923Z",
      "content": "<p>Nice approach, I had tried xgboost as post processing approach but didn't had much success. Will look into your kernel to learn. </p>",
      "rawMarkdown": "Nice approach, I had tried xgboost as post processing approach but didn't had much success. Will look into your kernel to learn. ",
      "replies": [
        {
          "id": 742329,
          "postDate": "2020-02-11T08:36:10.637Z",
          "content": "<p>Thank you!</p>",
          "rawMarkdown": "Thank you!"
        }
      ]
    },
    {
      "id": 742312,
      "postDate": "2020-02-11T08:20:17.577Z",
      "content": "<p>The kernel is private :)</p>",
      "rawMarkdown": "The kernel is private :)",
      "replies": [
        {
          "id": 742317,
          "postDate": "2020-02-11T08:23:43.843Z",
          "content": "<p>sorry, I changed my kernel to public</p>",
          "rawMarkdown": "sorry, I changed my kernel to public"
        }
      ]
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T19:07:55Z",
  "competition_id": 7968,
  "competition_slug": "google-quest-challenge",
  "team_id": 4014991,
  "solution_url": "/competitions/google-quest-challenge/writeups/the-hand-23th-place-solution"
}