{
  "id": 419230,
  "title": "Naive baseline with LSTM",
  "url": "/competitions/cafa-5-protein-function-prediction/discussion/419230",
  "author_name": "thacrobatheskis",
  "post_date": "2023-06-24T20:54:18.042000",
  "votes": 6,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I'd like to share my baseline with a leaderboard score of 0.10694. I trained a simple 2-layer lstm with a linear projection from the last hidden state to the ~30,000 logits for each function. Here are the notebooks I used:</p>\n<h2><a href=\"https://www.kaggle.com/code/robbynevels/cafa5-baseline-dataset\" target=\"_blank\">Preparing dataset</a></h2>\n<ul>\n<li>Create single csv with all sequences and terms</li>\n</ul>\n<h2><a href=\"https://www.kaggle.com/code/robbynevels/cafa5-baseline-model\" target=\"_blank\">Training network</a></h2>\n<ul>\n<li>Only considered sequences &lt; 1000 amino acids long</li>\n<li>Dynamically padded sequences in order batch</li>\n<li>Loaded entire dataset into GPU before training</li>\n<li>80/20 train/validation split</li>\n<li>Each epoch took ~7 minutes</li>\n<li>Epoch 16 has the best validation loss, then the model started to overfit</li>\n</ul>\n<h2><a href=\"https://www.kaggle.com/code/robbynevels/cafa5-baseline-infer\" target=\"_blank\">Inferring on testset</a></h2>\n<ul>\n<li>Saved top 1500 most probable terms for each sequence that was &gt; 0.001 probability</li>\n<li>In total, 114,990,098 terms predicted (~811 terms/sequence)</li>\n<li>Took 35 minutes to run inference (with GPU) + 4 minutes to save to disk</li>\n</ul>",
  "messages": [
    {
      "id": 2316378,
      "postDate": "2023-06-24T20:54:18.043Z",
      "content": "<p>I'd like to share my baseline with a leaderboard score of 0.10694. I trained a simple 2-layer lstm with a linear projection from the last hidden state to the ~30,000 logits for each function. Here are the notebooks I used:</p>\n<h2><a href=\"https://www.kaggle.com/code/robbynevels/cafa5-baseline-dataset\" target=\"_blank\">Preparing dataset</a></h2>\n<ul>\n<li>Create single csv with all sequences and terms</li>\n</ul>\n<h2><a href=\"https://www.kaggle.com/code/robbynevels/cafa5-baseline-model\" target=\"_blank\">Training network</a></h2>\n<ul>\n<li>Only considered sequences &lt; 1000 amino acids long</li>\n<li>Dynamically padded sequences in order batch</li>\n<li>Loaded entire dataset into GPU before training</li>\n<li>80/20 train/validation split</li>\n<li>Each epoch took ~7 minutes</li>\n<li>Epoch 16 has the best validation loss, then the model started to overfit</li>\n</ul>\n<h2><a href=\"https://www.kaggle.com/code/robbynevels/cafa5-baseline-infer\" target=\"_blank\">Inferring on testset</a></h2>\n<ul>\n<li>Saved top 1500 most probable terms for each sequence that was &gt; 0.001 probability</li>\n<li>In total, 114,990,098 terms predicted (~811 terms/sequence)</li>\n<li>Took 35 minutes to run inference (with GPU) + 4 minutes to save to disk</li>\n</ul>",
      "rawMarkdown": "I'd like to share my baseline with a leaderboard score of 0.10694. I trained a simple 2-layer lstm with a linear projection from the last hidden state to the ~30,000 logits for each function. Here are the notebooks I used:\n\n## [Preparing dataset](https://www.kaggle.com/code/robbynevels/cafa5-baseline-dataset)\n- Create single csv with all sequences and terms\n\n## [Training network](https://www.kaggle.com/code/robbynevels/cafa5-baseline-model)\n- Only considered sequences < 1000 amino acids long\n- Dynamically padded sequences in order batch\n- Loaded entire dataset into GPU before training\n- 80/20 train/validation split\n- Each epoch took ~7 minutes\n- Epoch 16 has the best validation loss, then the model started to overfit\n\n## [Inferring on testset](https://www.kaggle.com/code/robbynevels/cafa5-baseline-infer)\n- Saved top 1500 most probable terms for each sequence that was > 0.001 probability\n- In total, 114,990,098 terms predicted (~811 terms/sequence)\n- Took 35 minutes to run inference (with GPU) + 4 minutes to save to disk",
      "votes": 6
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2316378": "I'd like to share my baseline with a leaderboard score of 0.10694. I trained a simple 2-layer lstm with a linear projection from the last hidden state to the ~30,000 logits for each function. Here are the notebooks I used:\n\n## [Preparing dataset](https://www.kaggle.com/code/robbynevels/cafa5-baseline-dataset)\n- Create single csv with all sequences and terms\n\n## [Training network](https://www.kaggle.com/code/robbynevels/cafa5-baseline-model)\n- Only considered sequences < 1000 amino acids long\n- Dynamically padded sequences in order batch\n- Loaded entire dataset into GPU before training\n- 80/20 train/validation split\n- Each epoch took ~7 minutes\n- Epoch 16 has the best validation loss, then the model started to overfit\n\n## [Inferring on testset](https://www.kaggle.com/code/robbynevels/cafa5-baseline-infer)\n- Saved top 1500 most probable terms for each sequence that was > 0.001 probability\n- In total, 114,990,098 terms predicted (~811 terms/sequence)\n- Took 35 minutes to run inference (with GPU) + 4 minutes to save to disk"
  }
}