{
  "id": 106377,
  "title": "13th place solution",
  "url": "/competitions/champs-scalar-coupling/writeups/km-13th-place-solution",
  "author_name": "",
  "post_date": "2019-08-31T00:06:44.713Z",
  "votes": 30,
  "comment_count": 16,
  "views": 0,
  "content": "<p>First of all, I would like to thank the competition hosts. <br>\nHere I will try to summarize some of the main points of our solution.</p>\n\n<h2>1. Model architecture</h2>\n\n<p>We chose SchNet (CFConv) as a base neural network architecture.\nDetails are shown below.</p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2311404%2F77175b9a4b8e3a6f892a7ad421fe1851%2F2019-08-29%2018.31.19.png?generation=1567071105142394&amp;alt=media\" alt=\"\"></p>\n\n<ul>\n<li>Message passing part\n<ul><li>SchNet with Edge Update\n<ul><li>Might be similar to \"SchNet EU\" in <a href=\"https://storage.googleapis.com/kaggle-forum-message-attachments/610869/14042/csc_10th.pdf\">10th place</a></li>\n<li>Atom pair features which we used are explained in <code>2. Features</code></li></ul></li>\n<li>Prepare Edge Update Networks independently per graph distances (distance=1, 2, 3, and 4 or more)</li></ul></li>\n<li>Readout part\n<ul><li>Sum-up target atom embeddings updated by CFConv (message passing part)</li>\n<li>Concatenate Atom pair feature (same as used in message passing part)</li></ul></li>\n<li>Prediction part: MLP\n<ul><li>2 Layer MLPs for each coupling type (1JHC, 1JHN etc.)</li></ul></li>\n</ul>\n\n<h2>2. Features</h2>\n\n<ul>\n<li>Atom features\n<ul><li>Only use one-hot expression of chemical elements (C, H, etc.)</li></ul></li>\n<li>Atom pair features\n<ul><li>Gaussian expansion of 3D distance</li>\n<li>Gaussian expansion of dihedral angle features (cosθ, (cosθ)^2)</li>\n<li>Bond information (e.g. single-bond, double-bond, etc.)</li></ul></li>\n</ul>\n\n<h2>3. Training</h2>\n\n<ul>\n<li>1st-stage\n<ul><li>Training the model for all coupling types simultaneously.</li></ul></li>\n<li>2nd-stage\n<ul><li>Fine-tuning the model trained on the 1st-stage to each coupling types individually.</li></ul></li>\n<li>3rd-stage\n<ul><li>Perform the same training as the 1st stage, using data including examples pseudo-labeled on 2nd-stage.</li></ul></li>\n<li>4th-stage\n<ul><li>Fine-tuning the model trained on the 3rd-stage to each coupling types individually.</li></ul></li>\n</ul>",
  "messages": [
    {
      "id": "611489",
      "postDate": "08/29/2019 09:37:36",
      "content": "<p>First of all, I would like to thank the competition hosts. <br>\nHere I will try to summarize some of the main points of our solution.</p>\n\n<h2>1. Model architecture</h2>\n\n<p>We chose SchNet (CFConv) as a base neural network architecture.\nDetails are shown below.</p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2311404%2F77175b9a4b8e3a6f892a7ad421fe1851%2F2019-08-29%2018.31.19.png?generation=1567071105142394&amp;alt=media\" alt=\"\"></p>\n\n<ul>\n<li>Message passing part\n<ul><li>SchNet with Edge Update\n<ul><li>Might be similar to \"SchNet EU\" in <a href=\"https://storage.googleapis.com/kaggle-forum-message-attachments/610869/14042/csc_10th.pdf\">10th place</a></li>\n<li>Atom pair features which we used are explained in <code>2. Features</code></li></ul></li>\n<li>Prepare Edge Update Networks independently per graph distances (distance=1, 2, 3, and 4 or more)</li></ul></li>\n<li>Readout part\n<ul><li>Sum-up target atom embeddings updated by CFConv (message passing part)</li>\n<li>Concatenate Atom pair feature (same as used in message passing part)</li></ul></li>\n<li>Prediction part: MLP\n<ul><li>2 Layer MLPs for each coupling type (1JHC, 1JHN etc.)</li></ul></li>\n</ul>\n\n<h2>2. Features</h2>\n\n<ul>\n<li>Atom features\n<ul><li>Only use one-hot expression of chemical elements (C, H, etc.)</li></ul></li>\n<li>Atom pair features\n<ul><li>Gaussian expansion of 3D distance</li>\n<li>Gaussian expansion of dihedral angle features (cosθ, (cosθ)^2)</li>\n<li>Bond information (e.g. single-bond, double-bond, etc.)</li></ul></li>\n</ul>\n\n<h2>3. Training</h2>\n\n<ul>\n<li>1st-stage\n<ul><li>Training the model for all coupling types simultaneously.</li></ul></li>\n<li>2nd-stage\n<ul><li>Fine-tuning the model trained on the 1st-stage to each coupling types individually.</li></ul></li>\n<li>3rd-stage\n<ul><li>Perform the same training as the 1st stage, using data including examples pseudo-labeled on 2nd-stage.</li></ul></li>\n<li>4th-stage\n<ul><li>Fine-tuning the model trained on the 3rd-stage to each coupling types individually.</li></ul></li>\n</ul>",
      "rawMarkdown": "First of all, I would like to thank the competition hosts.  \nHere I will try to summarize some of the main points of our solution.\n\n## 1. Model architecture\n\nWe chose SchNet (CFConv) as a base neural network architecture.\nDetails are shown below.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2311404%2F77175b9a4b8e3a6f892a7ad421fe1851%2F2019-08-29%2018.31.19.png?generation=1567071105142394&amp;alt=media)\n\n- Message passing part\n    - SchNet with Edge Update\n        - Might be similar to \"SchNet EU\" in [10th place](https://storage.googleapis.com/kaggle-forum-message-attachments/610869/14042/csc_10th.pdf)\n        - Atom pair features which we used are explained in `2. Features`\n  - Prepare Edge Update Networks independently per graph distances (distance=1, 2, 3, and 4 or more)\n- Readout part\n  - Sum-up target atom embeddings updated by CFConv (message passing part)\n  - Concatenate Atom pair feature (same as used in message passing part)\n- Prediction part: MLP\n  - 2 Layer MLPs for each coupling type (1JHC, 1JHN etc.)\n\n\n## 2. Features\n\n- Atom features\n  - Only use one-hot expression of chemical elements (C, H, etc.)\n- Atom pair features\n  - Gaussian expansion of 3D distance\n  - Gaussian expansion of dihedral angle features (cosθ, (cosθ)^2)\n  - Bond information (e.g. single-bond, double-bond, etc.)\n\n## 3. Training\n\n- 1st-stage\n  - Training the model for all coupling types simultaneously.\n- 2nd-stage\n  - Fine-tuning the model trained on the 1st-stage to each coupling types individually.\n- 3rd-stage\n  - Perform the same training as the 1st stage, using data including examples pseudo-labeled on 2nd-stage.\n- 4th-stage\n  - Fine-tuning the model trained on the 3rd-stage to each coupling types individually.",
      "votes": null
    },
    {
      "id": "611502",
      "postDate": "08/29/2019 09:48:00",
      "content": "<p>Congrats.. Very Informative...!! Thanks for Sharing...!! <a href=\"/kfujikawa\">@kfujikawa</a> </p>",
      "rawMarkdown": "Congrats.. Very Informative...!! Thanks for Sharing...!! @kfujikawa",
      "votes": null
    },
    {
      "id": "611539",
      "postDate": "08/29/2019 10:07:24",
      "content": "<p>Wow, as I tought... SchNet really outperforms :) are you going to share the code?  How long was the training and what hardware did you use?\nThanks. </p>",
      "rawMarkdown": "Wow, as I tought... SchNet really outperforms :) are you going to share the code?  How long was the training and what hardware did you use?\nThanks.",
      "votes": null
    },
    {
      "id": "611576",
      "postDate": "08/29/2019 10:26:44",
      "content": "<p>おめでとうございます！スライド参考にさせていただきました。お互い金がとれたようでなによりですね。</p>",
      "rawMarkdown": "おめでとうございます！スライド参考にさせていただきました。お互い金がとれたようでなによりですね。",
      "votes": null
    },
    {
      "id": "611605",
      "postDate": "08/29/2019 10:45:53",
      "content": "<p>Congrats!!!</p>",
      "rawMarkdown": "Congrats!!!",
      "votes": null
    },
    {
      "id": "612025",
      "postDate": "08/29/2019 15:11:35",
      "content": "<p>Congrats on the result!  We tried pseudo labeling and it did not help much.  Interesting to see that it helped you.</p>",
      "rawMarkdown": "Congrats on the result!  We tried pseudo labeling and it did not help much.  Interesting to see that it helped you.",
      "votes": null
    },
    {
      "id": "612250",
      "postDate": "08/29/2019 17:16:33",
      "content": "<p>Congrats <a href=\"/kfujikawa\">@kfujikawa</a> , what software are you using to make the image of your model architecture? Very interesting read, thanks so much.</p>",
      "rawMarkdown": "Congrats @kfujikawa , what software are you using to make the image of your model architecture? Very interesting read, thanks so much.",
      "votes": null
    },
    {
      "id": "612251",
      "postDate": "08/29/2019 17:17:01",
      "content": "<p>I'd be very interested in having the answers to this too.</p>",
      "rawMarkdown": "I'd be very interested in having the answers to this too.",
      "votes": null
    },
    {
      "id": "613143",
      "postDate": "08/30/2019 08:57:53",
      "content": "<p>Thanks for your interest! I may publish my code, but need some time because of its complexity...\nI used Geforce 2080Ti x2, and p3.2xlarge or p3.8xlarge instances at the last week of the competition.\n1st stage (300epoch) and 2nd stage (150epoch) training take about 1 day per each.\nThe effect of 1st stage training on the overall performance was not significant, so I frozen the 1st stage model at an early timing.\n3rd stage (130epoch) training takes about 10h ~ 1 day for each coupling type. (1JHN has less data, so computational time is short: 10h)\nAs mentioned above, our model was not computationally efficient, so we could only complete 2 / 10 fold at the 4th stage.</p>",
      "rawMarkdown": "Thanks for your interest! I may publish my code, but need some time because of its complexity...\nI used Geforce 2080Ti x2, and p3.2xlarge or p3.8xlarge instances at the last week of the competition.\n1st stage (300epoch) and 2nd stage (150epoch) training take about 1 day per each.\nThe effect of 1st stage training on the overall performance was not significant, so I frozen the 1st stage model at an early timing.\n3rd stage (130epoch) training takes about 10h ~ 1 day for each coupling type. (1JHN has less data, so computational time is short: 10h)\nAs mentioned above, our model was not computationally efficient, so we could only complete 2 / 10 fold at the 4th stage.",
      "votes": null
    },
    {
      "id": "613146",
      "postDate": "08/30/2019 09:00:41",
      "content": "<p>ありがとうございます、 <a href=\"/akirasosa\">@akirasosa</a> さんもおめでとうございます！スライドがこんな所で活用されると思いませんでしたが、少しでも役立てて頂けたのなら幸いです。</p>",
      "rawMarkdown": "ありがとうございます、 @akirasosa さんもおめでとうございます！スライドがこんな所で活用されると思いませんでしたが、少しでも役立てて頂けたのなら幸いです。",
      "votes": null
    },
    {
      "id": "613147",
      "postDate": "08/30/2019 09:01:01",
      "content": "<p>Thanks!!!</p>",
      "rawMarkdown": "Thanks!!!",
      "votes": null
    },
    {
      "id": "613213",
      "postDate": "08/30/2019 10:01:55",
      "content": "<p>Congrats! We want to catch up your team but failed.</p>",
      "rawMarkdown": "Congrats! We want to catch up your team but failed.",
      "votes": null
    },
    {
      "id": "613950",
      "postDate": "08/31/2019 01:00:32",
      "content": "<p>Thanks, and congrats you too!\nI would show you group log MAE curve for the valid set at the 1st-stage and 3rd-stage training.</p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2311404%2Fb4f20cdba447cc569e536f554cde1b33%2F2019-08-31%209.58.29.png?generation=1567213143907405&amp;alt=media\" alt=\"\"></p>\n\n<p>Orange line (with pseudo label) is better than blue line (without pseudo label) in spite of the same NN architecture.\nI think one of the reasons why pseudo label worked is fine-tuning (2nd-stage)\n(I forgot to mention this procedure... Sorry. I revised the above document.)\nWe got 8 models of different parameters with the same architecture in 2nd-stage, which\ncould label with accuracy that could not be achieved by 1st-stage training.\nIt might be a kind of distillation, I think.</p>",
      "rawMarkdown": "Thanks, and congrats you too!\nI would show you group log MAE curve for the valid set at the 1st-stage and 3rd-stage training.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2311404%2Fb4f20cdba447cc569e536f554cde1b33%2F2019-08-31%209.58.29.png?generation=1567213143907405&amp;alt=media)\n\nOrange line (with pseudo label) is better than blue line (without pseudo label) in spite of the same NN architecture.\nI think one of the reasons why pseudo label worked is fine-tuning (2nd-stage)\n(I forgot to mention this procedure... Sorry. I revised the above document.)\nWe got 8 models of different parameters with the same architecture in 2nd-stage, which\ncould label with accuracy that could not be achieved by 1st-stage training.\nIt might be a kind of distillation, I think.",
      "votes": null
    },
    {
      "id": "614263",
      "postDate": "08/31/2019 09:38:38",
      "content": "<p>Thank you!\nI used chainer and chainer-chemistry, and mainly customized the following parts:\n<a href=\"https://github.com/pfnet-research/chainer-chemistry/blob/master/chainer_chemistry/links/update/schnet_update.py\">https://github.com/pfnet-research/chainer-chemistry/blob/master/chainer_chemistry/links/update/schnet_update.py</a></p>",
      "rawMarkdown": "Thank you!\nI used chainer and chainer-chemistry, and mainly customized the following parts:\nhttps://github.com/pfnet-research/chainer-chemistry/blob/master/chainer_chemistry/links/update/schnet_update.py",
      "votes": null
    },
    {
      "id": "614278",
      "postDate": "08/31/2019 09:53:54",
      "content": "<p>Thanks and congrats on the result and on becoming GM!\nI was afraid of your submissions that overtakes me every day in the last week.\nBut it helped to improve our score, I think.</p>",
      "rawMarkdown": "Thanks and congrats on the result and on becoming GM!\nI was afraid of your submissions that overtakes me every day in the last week.\nBut it helped to improve our score, I think.",
      "votes": null
    },
    {
      "id": "615268",
      "postDate": "09/01/2019 17:58:49",
      "content": "<p>Goldおめでとうございます．終盤の追い上げは見事でした．\n学習方法がとても工夫されていて，自分には無かった視点なので勉強になりました．</p>",
      "rawMarkdown": "Goldおめでとうございます．終盤の追い上げは見事でした．\n学習方法がとても工夫されていて，自分には無かった視点なので勉強になりました．",
      "votes": null
    },
    {
      "id": "615686",
      "postDate": "09/02/2019 08:50:01",
      "content": "<p>ありがとうございます！\n<a href=\"/toshik\">@toshik</a> さんもsoloでこれだけの結果が出せるのは流石です。またtriplet featureもユニークで大変勉強になりました。</p>",
      "rawMarkdown": "ありがとうございます！\n@toshik さんもsoloでこれだけの結果が出せるのは流石です。またtriplet featureもユニークで大変勉強になりました。",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 611502,
      "author_name": "veeralakrishna",
      "author_url": "",
      "post_date": "08/29/2019 09:48:00",
      "content": "<p>Congrats.. Very Informative...!! Thanks for Sharing...!! <a href=\"/kfujikawa\">@kfujikawa</a> </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 611539,
      "author_name": "jesucristo",
      "author_url": "",
      "post_date": "08/29/2019 10:07:24",
      "content": "<p>Wow, as I tought... SchNet really outperforms :) are you going to share the code?  How long was the training and what hardware did you use?\nThanks. </p>",
      "votes": null,
      "replies": [
        {
          "id": 612251,
          "author_name": "michelml",
          "author_url": "",
          "post_date": "08/29/2019 17:17:01",
          "content": "<p>I'd be very interested in having the answers to this too.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 613143,
          "author_name": "kfujikawa",
          "author_url": "",
          "post_date": "08/30/2019 08:57:53",
          "content": "<p>Thanks for your interest! I may publish my code, but need some time because of its complexity...\nI used Geforce 2080Ti x2, and p3.2xlarge or p3.8xlarge instances at the last week of the competition.\n1st stage (300epoch) and 2nd stage (150epoch) training take about 1 day per each.\nThe effect of 1st stage training on the overall performance was not significant, so I frozen the 1st stage model at an early timing.\n3rd stage (130epoch) training takes about 10h ~ 1 day for each coupling type. (1JHN has less data, so computational time is short: 10h)\nAs mentioned above, our model was not computationally efficient, so we could only complete 2 / 10 fold at the 4th stage.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 611576,
      "author_name": "akirasosa",
      "author_url": "",
      "post_date": "08/29/2019 10:26:44",
      "content": "<p>おめでとうございます！スライド参考にさせていただきました。お互い金がとれたようでなによりですね。</p>",
      "votes": null,
      "replies": [
        {
          "id": 613146,
          "author_name": "kfujikawa",
          "author_url": "",
          "post_date": "08/30/2019 09:00:41",
          "content": "<p>ありがとうございます、 <a href=\"/akirasosa\">@akirasosa</a> さんもおめでとうございます！スライドがこんな所で活用されると思いませんでしたが、少しでも役立てて頂けたのなら幸いです。</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 611605,
      "author_name": "kenmatsu4",
      "author_url": "",
      "post_date": "08/29/2019 10:45:53",
      "content": "<p>Congrats!!!</p>",
      "votes": null,
      "replies": [
        {
          "id": 613147,
          "author_name": "kfujikawa",
          "author_url": "",
          "post_date": "08/30/2019 09:01:01",
          "content": "<p>Thanks!!!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 612025,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "08/29/2019 15:11:35",
      "content": "<p>Congrats on the result!  We tried pseudo labeling and it did not help much.  Interesting to see that it helped you.</p>",
      "votes": null,
      "replies": [
        {
          "id": 613950,
          "author_name": "kfujikawa",
          "author_url": "",
          "post_date": "08/31/2019 01:00:32",
          "content": "<p>Thanks, and congrats you too!\nI would show you group log MAE curve for the valid set at the 1st-stage and 3rd-stage training.</p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2311404%2Fb4f20cdba447cc569e536f554cde1b33%2F2019-08-31%209.58.29.png?generation=1567213143907405&amp;alt=media\" alt=\"\"></p>\n\n<p>Orange line (with pseudo label) is better than blue line (without pseudo label) in spite of the same NN architecture.\nI think one of the reasons why pseudo label worked is fine-tuning (2nd-stage)\n(I forgot to mention this procedure... Sorry. I revised the above document.)\nWe got 8 models of different parameters with the same architecture in 2nd-stage, which\ncould label with accuracy that could not be achieved by 1st-stage training.\nIt might be a kind of distillation, I think.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 612250,
      "author_name": "michelml",
      "author_url": "",
      "post_date": "08/29/2019 17:16:33",
      "content": "<p>Congrats <a href=\"/kfujikawa\">@kfujikawa</a> , what software are you using to make the image of your model architecture? Very interesting read, thanks so much.</p>",
      "votes": null,
      "replies": [
        {
          "id": 614263,
          "author_name": "kfujikawa",
          "author_url": "",
          "post_date": "08/31/2019 09:38:38",
          "content": "<p>Thank you!\nI used chainer and chainer-chemistry, and mainly customized the following parts:\n<a href=\"https://github.com/pfnet-research/chainer-chemistry/blob/master/chainer_chemistry/links/update/schnet_update.py\">https://github.com/pfnet-research/chainer-chemistry/blob/master/chainer_chemistry/links/update/schnet_update.py</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 613213,
      "author_name": "senkin13",
      "author_url": "",
      "post_date": "08/30/2019 10:01:55",
      "content": "<p>Congrats! We want to catch up your team but failed.</p>",
      "votes": null,
      "replies": [
        {
          "id": 614278,
          "author_name": "kfujikawa",
          "author_url": "",
          "post_date": "08/31/2019 09:53:54",
          "content": "<p>Thanks and congrats on the result and on becoming GM!\nI was afraid of your submissions that overtakes me every day in the last week.\nBut it helped to improve our score, I think.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 615268,
      "author_name": "toshik",
      "author_url": "",
      "post_date": "09/01/2019 17:58:49",
      "content": "<p>Goldおめでとうございます．終盤の追い上げは見事でした．\n学習方法がとても工夫されていて，自分には無かった視点なので勉強になりました．</p>",
      "votes": null,
      "replies": [
        {
          "id": 615686,
          "author_name": "kfujikawa",
          "author_url": "",
          "post_date": "09/02/2019 08:50:01",
          "content": "<p>ありがとうございます！\n<a href=\"/toshik\">@toshik</a> さんもsoloでこれだけの結果が出せるのは流石です。またtriplet featureもユニークで大変勉強になりました。</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "611489": "First of all, I would like to thank the competition hosts.  \nHere I will try to summarize some of the main points of our solution.\n\n## 1. Model architecture\n\nWe chose SchNet (CFConv) as a base neural network architecture.\nDetails are shown below.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2311404%2F77175b9a4b8e3a6f892a7ad421fe1851%2F2019-08-29%2018.31.19.png?generation=1567071105142394&amp;alt=media)\n\n- Message passing part\n    - SchNet with Edge Update\n        - Might be similar to \"SchNet EU\" in [10th place](https://storage.googleapis.com/kaggle-forum-message-attachments/610869/14042/csc_10th.pdf)\n        - Atom pair features which we used are explained in `2. Features`\n  - Prepare Edge Update Networks independently per graph distances (distance=1, 2, 3, and 4 or more)\n- Readout part\n  - Sum-up target atom embeddings updated by CFConv (message passing part)\n  - Concatenate Atom pair feature (same as used in message passing part)\n- Prediction part: MLP\n  - 2 Layer MLPs for each coupling type (1JHC, 1JHN etc.)\n\n\n## 2. Features\n\n- Atom features\n  - Only use one-hot expression of chemical elements (C, H, etc.)\n- Atom pair features\n  - Gaussian expansion of 3D distance\n  - Gaussian expansion of dihedral angle features (cosθ, (cosθ)^2)\n  - Bond information (e.g. single-bond, double-bond, etc.)\n\n## 3. Training\n\n- 1st-stage\n  - Training the model for all coupling types simultaneously.\n- 2nd-stage\n  - Fine-tuning the model trained on the 1st-stage to each coupling types individually.\n- 3rd-stage\n  - Perform the same training as the 1st stage, using data including examples pseudo-labeled on 2nd-stage.\n- 4th-stage\n  - Fine-tuning the model trained on the 3rd-stage to each coupling types individually.",
    "611502": "Congrats.. Very Informative...!! Thanks for Sharing...!! @kfujikawa",
    "611539": "Wow, as I tought... SchNet really outperforms :) are you going to share the code?  How long was the training and what hardware did you use?\nThanks.",
    "611576": "おめでとうございます！スライド参考にさせていただきました。お互い金がとれたようでなによりですね。",
    "611605": "Congrats!!!",
    "612025": "Congrats on the result!  We tried pseudo labeling and it did not help much.  Interesting to see that it helped you.",
    "612250": "Congrats @kfujikawa , what software are you using to make the image of your model architecture? Very interesting read, thanks so much.",
    "612251": "I'd be very interested in having the answers to this too.",
    "613143": "Thanks for your interest! I may publish my code, but need some time because of its complexity...\nI used Geforce 2080Ti x2, and p3.2xlarge or p3.8xlarge instances at the last week of the competition.\n1st stage (300epoch) and 2nd stage (150epoch) training take about 1 day per each.\nThe effect of 1st stage training on the overall performance was not significant, so I frozen the 1st stage model at an early timing.\n3rd stage (130epoch) training takes about 10h ~ 1 day for each coupling type. (1JHN has less data, so computational time is short: 10h)\nAs mentioned above, our model was not computationally efficient, so we could only complete 2 / 10 fold at the 4th stage.",
    "613146": "ありがとうございます、 @akirasosa さんもおめでとうございます！スライドがこんな所で活用されると思いませんでしたが、少しでも役立てて頂けたのなら幸いです。",
    "613147": "Thanks!!!",
    "613213": "Congrats! We want to catch up your team but failed.",
    "613950": "Thanks, and congrats you too!\nI would show you group log MAE curve for the valid set at the 1st-stage and 3rd-stage training.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2311404%2Fb4f20cdba447cc569e536f554cde1b33%2F2019-08-31%209.58.29.png?generation=1567213143907405&amp;alt=media)\n\nOrange line (with pseudo label) is better than blue line (without pseudo label) in spite of the same NN architecture.\nI think one of the reasons why pseudo label worked is fine-tuning (2nd-stage)\n(I forgot to mention this procedure... Sorry. I revised the above document.)\nWe got 8 models of different parameters with the same architecture in 2nd-stage, which\ncould label with accuracy that could not be achieved by 1st-stage training.\nIt might be a kind of distillation, I think.",
    "614263": "Thank you!\nI used chainer and chainer-chemistry, and mainly customized the following parts:\nhttps://github.com/pfnet-research/chainer-chemistry/blob/master/chainer_chemistry/links/update/schnet_update.py",
    "614278": "Thanks and congrats on the result and on becoming GM!\nI was afraid of your submissions that overtakes me every day in the last week.\nBut it helped to improve our score, I think.",
    "615268": "Goldおめでとうございます．終盤の追い上げは見事でした．\n学習方法がとても工夫されていて，自分には無かった視点なので勉強になりました．",
    "615686": "ありがとうございます！\n@toshik さんもsoloでこれだけの結果が出せるのは流石です。またtriplet featureもユニークで大変勉強になりました。"
  },
  "source": "meta"
}