{
  "id": 413594,
  "title": "How to preprocess data from beginning?",
  "url": "/competitions/cafa-5-protein-function-prediction/discussion/413594",
  "author_name": "",
  "post_date": "2023-05-29T11:38:42.394031600Z",
  "votes": 4,
  "comment_count": 2,
  "views": 0,
  "content": "<p>We already have the sequences whoseWe already have the sequences (train_sequences.fasta) whose dimension is 142246, and the target dimension (train_terms.tsv) is 5363863 rows × 3 columns. <br>\nHow can we prepare the sequences for training?<br>\nCan anyone help with preprocessing steps?</p>",
  "messages": [
    {
      "id": "2279473",
      "postDate": "05/29/2023 11:38:42",
      "content": "<p>We already have the sequences whoseWe already have the sequences (train_sequences.fasta) whose dimension is 142246, and the target dimension (train_terms.tsv) is 5363863 rows × 3 columns. <br>\nHow can we prepare the sequences for training?<br>\nCan anyone help with preprocessing steps?</p>",
      "rawMarkdown": "We already have the sequences whoseWe already have the sequences (train_sequences.fasta) whose dimension is 142246, and the target dimension (train_terms.tsv) is 5363863 rows × 3 columns. \nHow can we prepare the sequences for training?\nCan anyone help with preprocessing steps?",
      "votes": null
    },
    {
      "id": "2279720",
      "postDate": "05/29/2023 14:57:51",
      "content": "<p>Heyy! You dispose of N=142,246 sequences for each protein ID. At the end you need to submit ideally a prediction for each GO term represented in the dataset, a probability for them to appear in each protein. So for example if you have K GO term classes, you final prediction file will contains N x K rows. </p>\n<p>To process the sequences for training, you can start from pre trained embeddings from famous languages models trained on protein sequences (ProtBERT, EMS2, T5). You can also represent GO terms in each proteins as a sparse matrix and do clustering methods (KNN). </p>",
      "rawMarkdown": "Heyy! You dispose of N=142,246 sequences for each protein ID. At the end you need to submit ideally a prediction for each GO term represented in the dataset, a probability for them to appear in each protein. So for example if you have K GO term classes, you final prediction file will contains N x K rows. \n\nTo process the sequences for training, you can start from pre trained embeddings from famous languages models trained on protein sequences (ProtBERT, EMS2, T5). You can also represent GO terms in each proteins as a sparse matrix and do clustering methods (KNN).",
      "votes": null
    },
    {
      "id": "2280272",
      "postDate": "05/30/2023 02:37:54",
      "content": "<p>Thank you very much <a href=\"https://www.kaggle.com/henriupton\" target=\"_blank\">@henriupton</a>. Let me do the same.</p>",
      "rawMarkdown": "Thank you very much @henriupton. Let me do the same.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2279720,
      "author_name": "henriupton",
      "author_url": "",
      "post_date": "05/29/2023 14:57:51",
      "content": "<p>Heyy! You dispose of N=142,246 sequences for each protein ID. At the end you need to submit ideally a prediction for each GO term represented in the dataset, a probability for them to appear in each protein. So for example if you have K GO term classes, you final prediction file will contains N x K rows. </p>\n<p>To process the sequences for training, you can start from pre trained embeddings from famous languages models trained on protein sequences (ProtBERT, EMS2, T5). You can also represent GO terms in each proteins as a sparse matrix and do clustering methods (KNN). </p>",
      "votes": null,
      "replies": [
        {
          "id": 2280272,
          "author_name": "dplg007",
          "author_url": "",
          "post_date": "05/30/2023 02:37:54",
          "content": "<p>Thank you very much <a href=\"https://www.kaggle.com/henriupton\" target=\"_blank\">@henriupton</a>. Let me do the same.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2279473": "We already have the sequences whoseWe already have the sequences (train_sequences.fasta) whose dimension is 142246, and the target dimension (train_terms.tsv) is 5363863 rows × 3 columns. \nHow can we prepare the sequences for training?\nCan anyone help with preprocessing steps?",
    "2279720": "Heyy! You dispose of N=142,246 sequences for each protein ID. At the end you need to submit ideally a prediction for each GO term represented in the dataset, a probability for them to appear in each protein. So for example if you have K GO term classes, you final prediction file will contains N x K rows. \n\nTo process the sequences for training, you can start from pre trained embeddings from famous languages models trained on protein sequences (ProtBERT, EMS2, T5). You can also represent GO terms in each proteins as a sparse matrix and do clustering methods (KNN).",
    "2280272": "Thank you very much @henriupton. Let me do the same."
  },
  "source": "meta"
}