{ "id": 402756, "title": "Protein Sequence Embedding Techniques (Resources)", "url": "/competitions/cafa-5-protein-function-prediction/discussion/402756", "author_name": "", "post_date": "2023-04-19T15:18:14.569589800Z", "votes": 16, "comment_count": 10, "views": 0, "content": "

When working with biological sequences, sequence embedding can play a crucial role in capturing important features of a protein's amino acid sequence that are relevant to their function. Here's a list of various embedding techniques used for protein sequences that have proved better than typical one-hot encoding or Tokenizers.
\nAs someone who has been working at the intersection of AI, genomics, and proteins, I guess, I can say they've seen to be more efficient.
\nMentioning a few techniques here. Hope, you find them useful! :)

\n
    \n
  1. ProtVec (word2vec) : Continuous distributed representation of biological sequences for deep proteomics and genomics. https://arxiv.org/abs/1503.05140
    2. \n
  2. SeqVec (ELMo) : Modeling aspects of the language of life through transfer-learning protein sequences. https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-3220-8
    3. \n
  3. ESM‐1b (Transformer) : TRANSFORMER PROTEIN LANGUAGE MODELS ARE
    \nUNSUPERVISED STRUCTURE LEARNERS. https://openreview.net/pdf?id=fylclEqgvgd
    4. \n
  4. ProtTrans (BERT) : Towards cracking the language of life’s code through self-supervised deep
    \nlearning and high performance computing. https://arxiv.org/abs/2007.06225
    5. \n
  5. UDSMProt (AWD‐LSTM) : Universal deep sequence models for protein classification. https://academic.oup.com/bioinformatics/article/36/8/2401/5698270
    6. \n
  6. UniRep (mLSTM) : Unified rational protein engineering with sequence-based deep representation learning. https://www.nature.com/articles/s41592-019-0598-1
    7. \n
", "messages": [ { "id": "2227218", "postDate": "04/19/2023 15:18:14", "content": "

When working with biological sequences, sequence embedding can play a crucial role in capturing important features of a protein's amino acid sequence that are relevant to their function. Here's a list of various embedding techniques used for protein sequences that have proved better than typical one-hot encoding or Tokenizers.
\nAs someone who has been working at the intersection of AI, genomics, and proteins, I guess, I can say they've seen to be more efficient.
\nMentioning a few techniques here. Hope, you find them useful! :)

\n
    \n
  1. ProtVec (word2vec) : Continuous distributed representation of biological sequences for deep proteomics and genomics. https://arxiv.org/abs/1503.05140
    2. \n
  2. SeqVec (ELMo) : Modeling aspects of the language of life through transfer-learning protein sequences. https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-3220-8
    3. \n
  3. ESM‐1b (Transformer) : TRANSFORMER PROTEIN LANGUAGE MODELS ARE
    \nUNSUPERVISED STRUCTURE LEARNERS. https://openreview.net/pdf?id=fylclEqgvgd
    4. \n
  4. ProtTrans (BERT) : Towards cracking the language of life’s code through self-supervised deep
    \nlearning and high performance computing. https://arxiv.org/abs/2007.06225
    5. \n
  5. UDSMProt (AWD‐LSTM) : Universal deep sequence models for protein classification. https://academic.oup.com/bioinformatics/article/36/8/2401/5698270
    6. \n
  6. UniRep (mLSTM) : Unified rational protein engineering with sequence-based deep representation learning. https://www.nature.com/articles/s41592-019-0598-1
    7. \n
", "rawMarkdown": "When working with biological sequences, sequence embedding can play a crucial role in capturing important features of a protein's amino acid sequence that are relevant to their function. Here's a list of various embedding techniques used for protein sequences that have proved better than typical one-hot encoding or Tokenizers. \nAs someone who has been working at the intersection of AI, genomics, and proteins, I guess, I can say they've seen to be more efficient. \nMentioning a few techniques here. Hope, you find them useful! :) \n1. ProtVec (word2vec) : Continuous distributed representation of biological sequences for deep proteomics and genomics. [https://arxiv.org/abs/1503.05140](url)\n2. SeqVec (ELMo) : Modeling aspects of the language of life through transfer-learning protein sequences. [https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-3220-8](url)\n3. ESM‐1b (Transformer) : TRANSFORMER PROTEIN LANGUAGE MODELS ARE\nUNSUPERVISED STRUCTURE LEARNERS. [ https://openreview.net/pdf?id=fylclEqgvgd](url)\n4. ProtTrans (BERT) : Towards cracking the language of life’s code through self-supervised deep\nlearning and high performance computing. [https://arxiv.org/abs/2007.06225](url)\n5. UDSMProt (AWD‐LSTM) : Universal deep sequence models for protein classification. [https://academic.oup.com/bioinformatics/article/36/8/2401/5698270](url)\n6. UniRep (mLSTM) : Unified rational protein engineering with sequence-based deep representation learning. [https://www.nature.com/articles/s41592-019-0598-1](url)", "votes": null }, { "id": "2227404", "postDate": "04/19/2023 18:31:49", "content": "

@ahsuna123 very helpful. Most of us need to study a lot of material to understand how to analyze the data and build models.

", "rawMarkdown": "ahsuna123 very helpful. Most of us need to study a lot of material to understand how to analyze the data and build models.", "votes": null }, { "id": "2227488", "postDate": "04/19/2023 19:39:03", "content": "

I am glad, you found it useful! :)
\nAgreed. It could take a little time to understand domain specific nuances but it's definitely worth it.

", "rawMarkdown": "I am glad, you found it useful! :)\nAgreed. It could take a little time to understand domain specific nuances but it's definitely worth it.", "votes": null }, { "id": "2227499", "postDate": "04/19/2023 19:49:32", "content": "

Cool ! Thanks for sharing. Also:

\n

Relevant Kaggle competition:
\nhttps://www.kaggle.com/competitions/novozymes-enzyme-stability-prediction

\n

And the notebook on ProtBert:
\nhttps://www.kaggle.com/code/cdeotte/prot-bert-finetune-lb-0-30

", "rawMarkdown": "Cool ! Thanks for sharing. Also:\n\nRelevant Kaggle competition: \nhttps://www.kaggle.com/competitions/novozymes-enzyme-stability-prediction\n\nAnd the notebook on ProtBert: \nhttps://www.kaggle.com/code/cdeotte/prot-bert-finetune-lb-0-30", "votes": null }, { "id": "2227507", "postDate": "04/19/2023 20:04:32", "content": "

Hey! Thanks for sharing additional links! :)

", "rawMarkdown": "Hey! Thanks for sharing additional links! :)", "votes": null }, { "id": "2228088", "postDate": "04/20/2023 09:01:57", "content": "

Shameless plug:

\n\n

Code & trained function prediction model:
\nhttps://github.com/nadavbra/protein_bert
\nPyTorch code: https://github.com/lucidrains/protein-bert-pytorch

", "rawMarkdown": "Shameless plug:\n- ProteinBERT: A universal deep-learning model of protein sequence and function. \nBioinformatics (2022). https://doi.org/10.1093/bioinformatics/btac020\n\nCode & trained function prediction model:\nhttps://github.com/nadavbra/protein_bert\nPyTorch code: https://github.com/lucidrains/protein-bert-pytorch", "votes": null }, { "id": "2228114", "postDate": "04/20/2023 09:25:00", "content": "

Hahah! Thanks for adding the link. Missed it! :)

", "rawMarkdown": "Hahah! Thanks for adding the link. Missed it! :)", "votes": null }, { "id": "2261999", "postDate": "05/16/2023 16:53:06", "content": "

Very Interesting. Great Read, Thank You!

", "rawMarkdown": "Very Interesting. Great Read, Thank You!", "votes": null }, { "id": "2262043", "postDate": "05/16/2023 17:35:47", "content": "

I'm glad you found it interesting! :)

", "rawMarkdown": "I'm glad you found it interesting! :)", "votes": null }, { "id": "2307092", "postDate": "06/17/2023 19:49:43", "content": "

you mentioned the best methods of machine learning that the researcher use to to predict function or structure of protein …well done

", "rawMarkdown": "you mentioned the best methods of machine learning that the researcher use to to predict function or structure of protein ...well done", "votes": null }, { "id": "2312687", "postDate": "06/22/2023 06:01:21", "content": "

Thank you so much. Means alot. Hope it helps everyone! :)

", "rawMarkdown": "Thank you so much. Means alot. Hope it helps everyone! :)", "votes": null } ], "comments": [ { "id": 2227404, "author_name": "serangu", "author_url": "", "post_date": "04/19/2023 18:31:49", "content": "

@ahsuna123 very helpful. Most of us need to study a lot of material to understand how to analyze the data and build models.

", "votes": null, "replies": [ { "id": 2227488, "author_name": "ahsuna123", "author_url": "", "post_date": "04/19/2023 19:39:03", "content": "

I am glad, you found it useful! :)
\nAgreed. It could take a little time to understand domain specific nuances but it's definitely worth it.

", "votes": null, "replies": [] } ] }, { "id": 2227499, "author_name": "alexandervc", "author_url": "", "post_date": "04/19/2023 19:49:32", "content": "

Cool ! Thanks for sharing. Also:

\n

Relevant Kaggle competition:
\nhttps://www.kaggle.com/competitions/novozymes-enzyme-stability-prediction

\n

And the notebook on ProtBert:
\nhttps://www.kaggle.com/code/cdeotte/prot-bert-finetune-lb-0-30

", "votes": null, "replies": [ { "id": 2227507, "author_name": "ahsuna123", "author_url": "", "post_date": "04/19/2023 20:04:32", "content": "

Hey! Thanks for sharing additional links! :)

", "votes": null, "replies": [] } ] }, { "id": 2228088, "author_name": "danofer", "author_url": "", "post_date": "04/20/2023 09:01:57", "content": "

Shameless plug:

\n\n

Code & trained function prediction model:
\nhttps://github.com/nadavbra/protein_bert
\nPyTorch code: https://github.com/lucidrains/protein-bert-pytorch

", "votes": null, "replies": [ { "id": 2228114, "author_name": "ahsuna123", "author_url": "", "post_date": "04/20/2023 09:25:00", "content": "

Hahah! Thanks for adding the link. Missed it! :)

", "votes": null, "replies": [] } ] }, { "id": 2261999, "author_name": "newdatasc", "author_url": "", "post_date": "05/16/2023 16:53:06", "content": "

Very Interesting. Great Read, Thank You!

", "votes": null, "replies": [ { "id": 2262043, "author_name": "ahsuna123", "author_url": "", "post_date": "05/16/2023 17:35:47", "content": "

I'm glad you found it interesting! :)

", "votes": null, "replies": [] } ] }, { "id": 2307092, "author_name": "belgmohamadi", "author_url": "", "post_date": "06/17/2023 19:49:43", "content": "

you mentioned the best methods of machine learning that the researcher use to to predict function or structure of protein …well done

", "votes": null, "replies": [ { "id": 2312687, "author_name": "ahsuna123", "author_url": "", "post_date": "06/22/2023 06:01:21", "content": "

Thank you so much. Means alot. Hope it helps everyone! :)

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2227218": "When working with biological sequences, sequence embedding can play a crucial role in capturing important features of a protein's amino acid sequence that are relevant to their function. Here's a list of various embedding techniques used for protein sequences that have proved better than typical one-hot encoding or Tokenizers. \nAs someone who has been working at the intersection of AI, genomics, and proteins, I guess, I can say they've seen to be more efficient. \nMentioning a few techniques here. Hope, you find them useful! :) \n1. ProtVec (word2vec) : Continuous distributed representation of biological sequences for deep proteomics and genomics. [https://arxiv.org/abs/1503.05140](url)\n2. SeqVec (ELMo) : Modeling aspects of the language of life through transfer-learning protein sequences. [https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-3220-8](url)\n3. ESM‐1b (Transformer) : TRANSFORMER PROTEIN LANGUAGE MODELS ARE\nUNSUPERVISED STRUCTURE LEARNERS. [ https://openreview.net/pdf?id=fylclEqgvgd](url)\n4. ProtTrans (BERT) : Towards cracking the language of life’s code through self-supervised deep\nlearning and high performance computing. [https://arxiv.org/abs/2007.06225](url)\n5. UDSMProt (AWD‐LSTM) : Universal deep sequence models for protein classification. [https://academic.oup.com/bioinformatics/article/36/8/2401/5698270](url)\n6. UniRep (mLSTM) : Unified rational protein engineering with sequence-based deep representation learning. [https://www.nature.com/articles/s41592-019-0598-1](url)", "2227404": "ahsuna123 very helpful. Most of us need to study a lot of material to understand how to analyze the data and build models.", "2227488": "I am glad, you found it useful! :)\nAgreed. It could take a little time to understand domain specific nuances but it's definitely worth it.", "2227499": "Cool ! Thanks for sharing. Also:\n\nRelevant Kaggle competition: \nhttps://www.kaggle.com/competitions/novozymes-enzyme-stability-prediction\n\nAnd the notebook on ProtBert: \nhttps://www.kaggle.com/code/cdeotte/prot-bert-finetune-lb-0-30", "2227507": "Hey! Thanks for sharing additional links! :)", "2228088": "Shameless plug:\n- ProteinBERT: A universal deep-learning model of protein sequence and function. \nBioinformatics (2022). https://doi.org/10.1093/bioinformatics/btac020\n\nCode & trained function prediction model:\nhttps://github.com/nadavbra/protein_bert\nPyTorch code: https://github.com/lucidrains/protein-bert-pytorch", "2228114": "Hahah! Thanks for adding the link. Missed it! :)", "2261999": "Very Interesting. Great Read, Thank You!", "2262043": "I'm glad you found it interesting! :)", "2307092": "you mentioned the best methods of machine learning that the researcher use to to predict function or structure of protein ...well done", "2312687": "Thank you so much. Means alot. Hope it helps everyone! :)" }, "source": "meta" }