{ "id": 324470, "title": "Pretrained Python Models on HuggingFace 🤗", "url": "/competitions/AI4Code/discussion/324470", "author_name": "Sinan Calisir", "post_date": "2022-05-11T23:00:37.202000", "votes": 36, "comment_count": 10, "views": 0, "content": "

Hi everyone,

\n

Here is a list of pretrained Python (code) models on HuggingFace 🤗 that might be useful in this competition.

\n
    \n

  1. https://huggingface.co/shibing624/code-autocomplete-distilgpt2-python

    \n
    2. \n

  2. https://huggingface.co/tals/roberta_python

    \n
    3. \n

  3. https://huggingface.co/SEBIS/code_trans_t5_large_source_code_summarization_python_multitask_finetune

    \n
    4. \n

  4. https://huggingface.co/uclanlp/plbart-python-en_XX

    5. \n

  5. https://huggingface.co/SIC98/GPT2-python-code-generator

    6. \n

  6. https://huggingface.co/lvwerra/codeparrot (thanks @johnowhitaker for the reminder)

    \n
    7. \n
", "messages": [ { "id": 1785193, "postDate": "2022-05-11T23:00:37.203Z", "content": "

Hi everyone,

\n

Here is a list of pretrained Python (code) models on HuggingFace 🤗 that might be useful in this competition.

\n
    \n

  1. https://huggingface.co/shibing624/code-autocomplete-distilgpt2-python

    \n
    2. \n

  2. https://huggingface.co/tals/roberta_python

    \n
    3. \n

  3. https://huggingface.co/SEBIS/code_trans_t5_large_source_code_summarization_python_multitask_finetune

    \n
    4. \n

  4. https://huggingface.co/uclanlp/plbart-python-en_XX

    5. \n

  5. https://huggingface.co/SIC98/GPT2-python-code-generator

    6. \n

  6. https://huggingface.co/lvwerra/codeparrot (thanks @johnowhitaker for the reminder)

    \n
    7. \n
", "rawMarkdown": "Hi everyone,\n\nHere is a list of pretrained Python (code) models on HuggingFace 🤗 that might be useful in this competition.\n\n1. https://huggingface.co/shibing624/code-autocomplete-distilgpt2-python\n - **GPT2 for Code AutoComplete Model**\n - code-autocomplete, a code completion plugin for Python.\n - code-autocomplete can automatically complete the code of lines and blocks with GPT2.\n\n2. https://huggingface.co/tals/roberta_python\n - This is a roBERTa-base model trained on the python part of CodeSearchNet and reached a dev perplexity of 3.296\n - This model was used for the Programming Puzzles enumerative solver baseline detailed in [Programming Puzzles paper](https://arxiv.org/abs/2106.05784).\n - See also the [Python Programming Puzzles (P3) Repository](https://github.com/microsoft/PythonProgrammingPuzzles) for more details.\n\n3. https://huggingface.co/SEBIS/code_trans_t5_large_source_code_summarization_python_multitask_finetune\n - Pretrained model on programming language python using the t5 large model architecture. It was first released in [this repository](https://github.com/agemagician/CodeTrans). This model is trained on tokenized python code functions: it works best with tokenized python functions.\n - **Model description**\nThis CodeTrans model is based on the t5-large model. It has its own SentencePiece vocabulary model. It used multi-task training on 13 supervised tasks in the software development domain and 7 unsupervised datasets. It is then fine-tuned on the source code summarization task for the python code snippets.\n - **Intended uses & limitations**\nThe model could be used to generate the description for the python function or be fine-tuned on other python code tasks. It can be used on unparsed and untokenized python code. However, if the python code is tokenized, the performance should be better.\n\n4. https://huggingface.co/uclanlp/plbart-python-en_XX\n5. https://huggingface.co/SIC98/GPT2-python-code-generator\n6. https://huggingface.co/lvwerra/codeparrot (thanks @johnowhitaker for the reminder)\n - CodeParrot 🦜 is a GPT-2 model (1.5B parameters) trained to generate Python code.", "votes": 36 }, { "id": 1786903, "postDate": "2022-05-13T12:22:03.093Z", "content": "

Don't forget https://huggingface.co/lvwerra/codeparrot :)

", "rawMarkdown": "Don't forget https://huggingface.co/lvwerra/codeparrot :)", "votes": 1, "replies": [ { "id": 1793277, "postDate": "2022-05-17T18:37:43.330Z", "content": "

Oh, of course! How did I miss that :) Thank you for the addition!

", "rawMarkdown": "Oh, of course! How did I miss that :) Thank you for the addition! " } ] }, { "id": 1785390, "postDate": "2022-05-12T04:11:54.140Z", "content": "

Thanks a lot for sharing. However, by looking at markdowns, I believe multi-language models should be included.

", "rawMarkdown": "Thanks a lot for sharing. However, by looking at markdowns, I believe multi-language models should be included. ", "votes": 1, "replies": [ { "id": 1785564, "postDate": "2022-05-12T08:02:59.360Z", "content": "

Thank you for the feedback. Yes, definitely, that would be a good addition. But I wanted to focus on python models here since the other models are more common.

", "rawMarkdown": "Thank you for the feedback. Yes, definitely, that would be a good addition. But I wanted to focus on python models here since the other models are more common. " }, { "id": 1796316, "postDate": "2022-05-20T17:40:20.597Z", "content": "

If you mean multilingual on the Natural Language side, not the Programming Language side, then yes! This has been flagged in another thread, I think it was Russian, and I have no idea what proportion of notebooks have comments in a language other than English.

", "rawMarkdown": "If you mean multilingual on the Natural Language side, not the Programming Language side, then yes! This has been flagged in another thread, I think it was Russian, and I have no idea what proportion of notebooks have comments in a language other than English." } ] }, { "id": 1786561, "postDate": "2022-05-13T02:40:24.597Z", "content": "

Wow, i didn't know code models also exists, Thanks for sharing!

", "rawMarkdown": "Wow, i didn't know code models also exists, Thanks for sharing!", "votes": 2 }, { "id": 1793998, "postDate": "2022-05-18T12:20:06.390Z", "content": "

Hi, Thanks for sharing.

\n

I am quite new to Kaggle. How do I load the pre-trained models into the competition notebook? I can't find them in the \"Add Data\" section.

\n

Thanks for the help.

", "rawMarkdown": "Hi, Thanks for sharing.\n\nI am quite new to Kaggle. How do I load the pre-trained models into the competition notebook? I can't find them in the \"Add Data\" section.\n\nThanks for the help." }, { "id": 1786207, "postDate": "2022-05-12T17:17:35.103Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 1792557, "postDate": "2022-05-17T05:00:06.620Z", "content": "

Wonderful stuff, thanks for sharing 🙌

", "rawMarkdown": "Wonderful stuff, thanks for sharing 🙌", "votes": 1 }, { "id": 1799474, "postDate": "2022-05-24T01:34:41.400Z", "content": "

Thanks for sharing, it's very useful!

", "rawMarkdown": "Thanks for sharing, it's very useful!" } ], "comments": [ { "id": 1786903, "author_name": "JonathanWhitaker", "author_url": "", "post_date": "2022-05-13T12:22:03.093000", "content": "

Don't forget https://huggingface.co/lvwerra/codeparrot :)

", "votes": 1, "replies": [ { "id": 1793277, "author_name": "Sinan Calisir", "author_url": "", "post_date": "2022-05-17T18:37:43.330000", "content": "

Oh, of course! How did I miss that :) Thank you for the addition!

", "votes": 0, "replies": [] } ] }, { "id": 1785390, "author_name": "ericngo", "author_url": "", "post_date": "2022-05-12T04:11:54.140000", "content": "

Thanks a lot for sharing. However, by looking at markdowns, I believe multi-language models should be included.

", "votes": 1, "replies": [ { "id": 1785564, "author_name": "Sinan Calisir", "author_url": "", "post_date": "2022-05-12T08:02:59.360000", "content": "

Thank you for the feedback. Yes, definitely, that would be a good addition. But I wanted to focus on python models here since the other models are more common.

", "votes": 0, "replies": [] }, { "id": 1796316, "author_name": "benvens", "author_url": "", "post_date": "2022-05-20T17:40:20.597000", "content": "

If you mean multilingual on the Natural Language side, not the Programming Language side, then yes! This has been flagged in another thread, I think it was Russian, and I have no idea what proportion of notebooks have comments in a language other than English.

", "votes": 0, "replies": [] } ] }, { "id": 1786561, "author_name": "Bhavesh Bisht", "author_url": "", "post_date": "2022-05-13T02:40:24.597000", "content": "

Wow, i didn't know code models also exists, Thanks for sharing!

", "votes": 2, "replies": [] }, { "id": 1793998, "author_name": "Jing Kai", "author_url": "", "post_date": "2022-05-18T12:20:06.390000", "content": "

Hi, Thanks for sharing.

\n

I am quite new to Kaggle. How do I load the pre-trained models into the competition notebook? I can't find them in the \"Add Data\" section.

\n

Thanks for the help.

", "votes": 0, "replies": [] }, { "id": 1786207, "author_name": "", "author_url": "", "post_date": "2022-05-12T17:17:35.103000", "content": "", "votes": 1, "replies": [] }, { "id": 1792557, "author_name": "Niek van der Zwaag", "author_url": "", "post_date": "2022-05-17T05:00:06.620000", "content": "

Wonderful stuff, thanks for sharing 🙌

", "votes": 1, "replies": [] }, { "id": 1799474, "author_name": "Shin Katayama", "author_url": "", "post_date": "2022-05-24T01:34:41.400000", "content": "

Thanks for sharing, it's very useful!

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1785193": "Hi everyone,\n\nHere is a list of pretrained Python (code) models on HuggingFace 🤗 that might be useful in this competition.\n\n1. https://huggingface.co/shibing624/code-autocomplete-distilgpt2-python\n - **GPT2 for Code AutoComplete Model**\n - code-autocomplete, a code completion plugin for Python.\n - code-autocomplete can automatically complete the code of lines and blocks with GPT2.\n\n2. https://huggingface.co/tals/roberta_python\n - This is a roBERTa-base model trained on the python part of CodeSearchNet and reached a dev perplexity of 3.296\n - This model was used for the Programming Puzzles enumerative solver baseline detailed in [Programming Puzzles paper](https://arxiv.org/abs/2106.05784).\n - See also the [Python Programming Puzzles (P3) Repository](https://github.com/microsoft/PythonProgrammingPuzzles) for more details.\n\n3. https://huggingface.co/SEBIS/code_trans_t5_large_source_code_summarization_python_multitask_finetune\n - Pretrained model on programming language python using the t5 large model architecture. It was first released in [this repository](https://github.com/agemagician/CodeTrans). This model is trained on tokenized python code functions: it works best with tokenized python functions.\n - **Model description**\nThis CodeTrans model is based on the t5-large model. It has its own SentencePiece vocabulary model. It used multi-task training on 13 supervised tasks in the software development domain and 7 unsupervised datasets. It is then fine-tuned on the source code summarization task for the python code snippets.\n - **Intended uses & limitations**\nThe model could be used to generate the description for the python function or be fine-tuned on other python code tasks. It can be used on unparsed and untokenized python code. However, if the python code is tokenized, the performance should be better.\n\n4. https://huggingface.co/uclanlp/plbart-python-en_XX\n5. https://huggingface.co/SIC98/GPT2-python-code-generator\n6. https://huggingface.co/lvwerra/codeparrot (thanks @johnowhitaker for the reminder)\n - CodeParrot 🦜 is a GPT-2 model (1.5B parameters) trained to generate Python code.", "1786903": "Don't forget https://huggingface.co/lvwerra/codeparrot :)", "1785390": "Thanks a lot for sharing. However, by looking at markdowns, I believe multi-language models should be included. ", "1786561": "Wow, i didn't know code models also exists, Thanks for sharing!", "1793998": "Hi, Thanks for sharing.\n\nI am quite new to Kaggle. How do I load the pre-trained models into the competition notebook? I can't find them in the \"Add Data\" section.\n\nThanks for the help.", "1786207": "", "1792557": "Wonderful stuff, thanks for sharing 🙌", "1799474": "Thanks for sharing, it's very useful!" } }