{
  "id": 324570,
  "title": " CodeBERT: A Pre-Trained Model for Programming and Natural Languages",
  "url": "/competitions/AI4Code/discussion/324570",
  "author_name": "",
  "post_date": "2022-05-12T10:05:39.480161Z",
  "votes": 46,
  "comment_count": 8,
  "views": 0,
  "content": "<p>Hi Guys! </p>\n<p>I just found CodeBERT! </p>\n<p><strong>CodeBERT is a pre-trained model for programming language</strong>, which is a multi-programming-lingual model pre-trained on NL-PL pairs in 6 programming languages (Python, Java, JavaScript, PHP, Ruby, Go).</p>\n<p>Available through Huggingface:</p>\n<pre><code>import torch\nfrom transformers import RobertaTokenizer, RobertaConfig, RobertaModel\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\ntokenizer = RobertaTokenizer.from_pretrained(\"microsoft/codebert-base\")\nmodel = RobertaModel.from_pretrained(\"microsoft/codebert-base\")\nmodel.to(device)\n</code></pre>\n<p>And the repository is <a href=\"https://github.com/microsoft/CodeBERT\" target=\"_blank\">here</a></p>\n<p>Have Fun! </p>",
  "messages": [
    {
      "id": "1785662",
      "postDate": "05/12/2022 10:05:39",
      "content": "<p>Hi Guys! </p>\n<p>I just found CodeBERT! </p>\n<p><strong>CodeBERT is a pre-trained model for programming language</strong>, which is a multi-programming-lingual model pre-trained on NL-PL pairs in 6 programming languages (Python, Java, JavaScript, PHP, Ruby, Go).</p>\n<p>Available through Huggingface:</p>\n<pre><code>import torch\nfrom transformers import RobertaTokenizer, RobertaConfig, RobertaModel\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\ntokenizer = RobertaTokenizer.from_pretrained(\"microsoft/codebert-base\")\nmodel = RobertaModel.from_pretrained(\"microsoft/codebert-base\")\nmodel.to(device)\n</code></pre>\n<p>And the repository is <a href=\"https://github.com/microsoft/CodeBERT\" target=\"_blank\">here</a></p>\n<p>Have Fun! </p>",
      "rawMarkdown": "Hi Guys! \n\nI just found CodeBERT! \n\n**CodeBERT is a pre-trained model for programming language**, which is a multi-programming-lingual model pre-trained on NL-PL pairs in 6 programming languages (Python, Java, JavaScript, PHP, Ruby, Go).\n\nAvailable through Huggingface:\n\n```\nimport torch\nfrom transformers import RobertaTokenizer, RobertaConfig, RobertaModel\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\ntokenizer = RobertaTokenizer.from_pretrained(\"microsoft/codebert-base\")\nmodel = RobertaModel.from_pretrained(\"microsoft/codebert-base\")\nmodel.to(device)\n```\n\nAnd the repository is [here](https://github.com/microsoft/CodeBERT)\n\nHave Fun!",
      "votes": null
    },
    {
      "id": "1786560",
      "postDate": "05/13/2022 02:39:46",
      "content": "<p>Thanks for sharing, i had never heard of this!</p>",
      "rawMarkdown": "Thanks for sharing, i had never heard of this!",
      "votes": null
    },
    {
      "id": "1790380",
      "postDate": "05/14/2022 20:50:53",
      "content": "<p><a href=\"https://www.kaggle.com/satoshidatamoto\" target=\"_blank\">@satoshidatamoto</a> This is a great pre-trained model that yields good results</p>",
      "rawMarkdown": "satoshidatamoto This is a great pre-trained model that yields good results",
      "votes": null
    },
    {
      "id": "1792597",
      "postDate": "05/17/2022 05:51:10",
      "content": "<p>Sound very promising. Thanks for sharing!</p>",
      "rawMarkdown": "Sound very promising. Thanks for sharing!",
      "votes": null
    },
    {
      "id": "1794400",
      "postDate": "05/18/2022 19:13:13",
      "content": "<p>Great ! thanks for sharing. Will definitely try this competition once I get enough practice and time.</p>",
      "rawMarkdown": "Great ! thanks for sharing. Will definitely try this competition once I get enough practice and time.",
      "votes": null
    },
    {
      "id": "1797675",
      "postDate": "05/22/2022 08:33:49",
      "content": "<p>This outperforms Roberta by 0.01 and DistillBert by 0.02 if anyone's wondering 😉</p>",
      "rawMarkdown": "This outperforms Roberta by 0.01 and DistillBert by 0.02 if anyone's wondering 😉",
      "votes": null
    },
    {
      "id": "1804654",
      "postDate": "05/29/2022 10:20:36",
      "content": "<p>Will GPT based models work better if I use?</p>",
      "rawMarkdown": "Will GPT based models work better if I use?",
      "votes": null
    },
    {
      "id": "1804666",
      "postDate": "05/29/2022 10:34:50",
      "content": "<p>Agree ! Gone through this paper. <a href=\"https://arxiv.org/pdf/2002.08155.pdf\" target=\"_blank\">https://arxiv.org/pdf/2002.08155.pdf</a> </p>",
      "rawMarkdown": "Agree ! Gone through this paper. https://arxiv.org/pdf/2002.08155.pdf",
      "votes": null
    },
    {
      "id": "1804763",
      "postDate": "05/29/2022 12:34:10",
      "content": "<p>Fascinating, thanks for sharing! 🙌</p>",
      "rawMarkdown": "Fascinating, thanks for sharing! 🙌",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1786560,
      "author_name": "bhavesh0124",
      "author_url": "",
      "post_date": "05/13/2022 02:39:46",
      "content": "<p>Thanks for sharing, i had never heard of this!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1790380,
      "author_name": "gshivansh",
      "author_url": "",
      "post_date": "05/14/2022 20:50:53",
      "content": "<p><a href=\"https://www.kaggle.com/satoshidatamoto\" target=\"_blank\">@satoshidatamoto</a> This is a great pre-trained model that yields good results</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1792597,
      "author_name": "erikbruin",
      "author_url": "",
      "post_date": "05/17/2022 05:51:10",
      "content": "<p>Sound very promising. Thanks for sharing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1794400,
      "author_name": "cid007",
      "author_url": "",
      "post_date": "05/18/2022 19:13:13",
      "content": "<p>Great ! thanks for sharing. Will definitely try this competition once I get enough practice and time.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1797675,
      "author_name": "suicaokhoailang",
      "author_url": "",
      "post_date": "05/22/2022 08:33:49",
      "content": "<p>This outperforms Roberta by 0.01 and DistillBert by 0.02 if anyone's wondering 😉</p>",
      "votes": null,
      "replies": [
        {
          "id": 1804666,
          "author_name": "paritoshmahto",
          "author_url": "",
          "post_date": "05/29/2022 10:34:50",
          "content": "<p>Agree ! Gone through this paper. <a href=\"https://arxiv.org/pdf/2002.08155.pdf\" target=\"_blank\">https://arxiv.org/pdf/2002.08155.pdf</a> </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1804654,
      "author_name": "paritoshmahto",
      "author_url": "",
      "post_date": "05/29/2022 10:20:36",
      "content": "<p>Will GPT based models work better if I use?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1804763,
      "author_name": "niekvanderzwaag",
      "author_url": "",
      "post_date": "05/29/2022 12:34:10",
      "content": "<p>Fascinating, thanks for sharing! 🙌</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1785662": "Hi Guys! \n\nI just found CodeBERT! \n\n**CodeBERT is a pre-trained model for programming language**, which is a multi-programming-lingual model pre-trained on NL-PL pairs in 6 programming languages (Python, Java, JavaScript, PHP, Ruby, Go).\n\nAvailable through Huggingface:\n\n```\nimport torch\nfrom transformers import RobertaTokenizer, RobertaConfig, RobertaModel\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\ntokenizer = RobertaTokenizer.from_pretrained(\"microsoft/codebert-base\")\nmodel = RobertaModel.from_pretrained(\"microsoft/codebert-base\")\nmodel.to(device)\n```\n\nAnd the repository is [here](https://github.com/microsoft/CodeBERT)\n\nHave Fun!",
    "1786560": "Thanks for sharing, i had never heard of this!",
    "1790380": "satoshidatamoto This is a great pre-trained model that yields good results",
    "1792597": "Sound very promising. Thanks for sharing!",
    "1794400": "Great ! thanks for sharing. Will definitely try this competition once I get enough practice and time.",
    "1797675": "This outperforms Roberta by 0.01 and DistillBert by 0.02 if anyone's wondering 😉",
    "1804654": "Will GPT based models work better if I use?",
    "1804666": "Agree ! Gone through this paper. https://arxiv.org/pdf/2002.08155.pdf",
    "1804763": "Fascinating, thanks for sharing! 🙌"
  },
  "source": "meta"
}