{
  "id": 741453,
  "title": "11th Place Solution",
  "url": "/competitions/cafa-6-protein-function-prediction/discussion/741453",
  "author_name": "Antonina Dolgorukova",
  "post_date": "2026-09-15T11:45:36.091000",
  "votes": 4,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Thanks to the CAFA and Kaggle organizers for another great protein function prediction challenge!</p>\n<p>My CAFA6 solution built on two main ideas. From our previous <strong>CAFA5 solution</strong>, I kept pretrained protein embeddings, multilabel prediction, and Gene Ontology propagation. On top of this, I added a component-based learning-to-rank approach inspired by <strong>GOLabeler</strong> (You et al., 2018).</p>\n<p>The main additions I did were a a hierarchy-aware MLP, PyBoost, BLAST, explicit use of non-experimental evidence, and more extensive use of the GO structure throughout modeling and postprocessing.</p>\n<h2>Data and annotation update</h2>\n<p>I used a December 2025 UniProt-GOA snapshot to update the competition annotations. Experimental annotations were added to the training labels. Non-experimental annotations were used in the LTR stage both as a standalone component and as evidence-code features.</p>\n<h2>Main LTR component predictors</h2>\n<ol>\n<li><strong>hierarchy-aware multilabel MLP on ESM2-3B embeddings</strong>: parent GO terms were scored both directly from the protein representation and from the predictions of their modeled child terms, with the two sources combined inside the network.</li>\n<li><strong>PyBoost on ProtT5 embeddings</strong> with a multi-output PyBoost model.</li>\n<li><strong>a multilabel MLP trained on concatenated ProtT5 and ESM-1b embeddings</strong></li>\n<li><strong>BLAST-KNN</strong>: GO annotations were transferred from the strongest homologous training sequences, excluding self-hits and weighting neighbors by BLAST similarity.</li>\n</ol>\n<h2>Annotation-derived components</h2>\n<p>I also used two simple components derived from the annotation data. </p>\n<ol>\n<li><strong>GO-frequency prior</strong>: assigned each term its prevalence in the training annotations.</li>\n<li><strong>non-experimental annotation component</strong>: assigned a score of 1 to protein-GO pairs present among the non-experimental UniProt annotations.</li>\n</ol>\n<h2>Candidate generation and learning to rank</h2>\n<p>Each predictive component contributed its strongest terms for a protein, and their union formed the candidate set. Separate <strong>XGBoost learning-to-rank models</strong> were trained for BP, CC, and MF. For each candidate protein-GO pair, the LTR model used the component scores together with information accretion and non-experimental evidence-code features.</p>\n<h2>Validation</h2>\n<p>I did not try to construct a validation set that reproduced the CAFA test setting. I used ordinary out-of-fold validation on held-out labeled proteins, even though this evaluates recovery of existing annotations rather than prediction of future experimental annotations.</p>\n<p>All trainable components therefore produced OOF predictions. The LTR models were trained on those predictions, and BLAST OOF predictions were generated at the unique-sequence level to prevent leakage through identical sequences.</p>\n<p>Candidate generation was tuned together with the ranker. I compared different pruning settings and candidate-set sizes using the local CAFA metric, because the number and quality of terms passed to LTR had a noticeable effect on the validation score.</p>\n<h2>Postprocessing</h2>\n<p>After LTR, I applied a monotonic score transformation and propagated predictions to their GO ancestors. Terms already known for the test proteins were removed where appropriate, and the final output was restricted to the highest-ranked terms for each protein.</p>\n<h2>Result</h2>\n<p>The selected submission scored <strong>0.42114</strong> on the private leaderboard. After the private leaderboard was revealed, several nearby LTR variants scored slightly higher. The best reached <strong>0.42522</strong>. That variant differed from the selected solution in an interesting way: it removed the non-experimental evidence-code features and added richer BLAST pair-level features such as sequence identity, coverage, support from multiple homologs, and alignment statistics.</p>\n<p>The main lesson for me from CAFA6 was that the final ranking problem benefited more from combining genuinely different sources of evidence than from making any single predictor more complex. A cleaned and reproducible implementation of the final pipeline is in a public GitHub repo.</p>",
  "messages": [
    {
      "id": 3524600,
      "postDate": "2026-09-15T11:45:36.090Z",
      "content": "<p>Thanks to the CAFA and Kaggle organizers for another great protein function prediction challenge!</p>\n<p>My CAFA6 solution built on two main ideas. From our previous <strong>CAFA5 solution</strong>, I kept pretrained protein embeddings, multilabel prediction, and Gene Ontology propagation. On top of this, I added a component-based learning-to-rank approach inspired by <strong>GOLabeler</strong> (You et al., 2018).</p>\n<p>The main additions I did were a a hierarchy-aware MLP, PyBoost, BLAST, explicit use of non-experimental evidence, and more extensive use of the GO structure throughout modeling and postprocessing.</p>\n<h2>Data and annotation update</h2>\n<p>I used a December 2025 UniProt-GOA snapshot to update the competition annotations. Experimental annotations were added to the training labels. Non-experimental annotations were used in the LTR stage both as a standalone component and as evidence-code features.</p>\n<h2>Main LTR component predictors</h2>\n<ol>\n<li><strong>hierarchy-aware multilabel MLP on ESM2-3B embeddings</strong>: parent GO terms were scored both directly from the protein representation and from the predictions of their modeled child terms, with the two sources combined inside the network.</li>\n<li><strong>PyBoost on ProtT5 embeddings</strong> with a multi-output PyBoost model.</li>\n<li><strong>a multilabel MLP trained on concatenated ProtT5 and ESM-1b embeddings</strong></li>\n<li><strong>BLAST-KNN</strong>: GO annotations were transferred from the strongest homologous training sequences, excluding self-hits and weighting neighbors by BLAST similarity.</li>\n</ol>\n<h2>Annotation-derived components</h2>\n<p>I also used two simple components derived from the annotation data. </p>\n<ol>\n<li><strong>GO-frequency prior</strong>: assigned each term its prevalence in the training annotations.</li>\n<li><strong>non-experimental annotation component</strong>: assigned a score of 1 to protein-GO pairs present among the non-experimental UniProt annotations.</li>\n</ol>\n<h2>Candidate generation and learning to rank</h2>\n<p>Each predictive component contributed its strongest terms for a protein, and their union formed the candidate set. Separate <strong>XGBoost learning-to-rank models</strong> were trained for BP, CC, and MF. For each candidate protein-GO pair, the LTR model used the component scores together with information accretion and non-experimental evidence-code features.</p>\n<h2>Validation</h2>\n<p>I did not try to construct a validation set that reproduced the CAFA test setting. I used ordinary out-of-fold validation on held-out labeled proteins, even though this evaluates recovery of existing annotations rather than prediction of future experimental annotations.</p>\n<p>All trainable components therefore produced OOF predictions. The LTR models were trained on those predictions, and BLAST OOF predictions were generated at the unique-sequence level to prevent leakage through identical sequences.</p>\n<p>Candidate generation was tuned together with the ranker. I compared different pruning settings and candidate-set sizes using the local CAFA metric, because the number and quality of terms passed to LTR had a noticeable effect on the validation score.</p>\n<h2>Postprocessing</h2>\n<p>After LTR, I applied a monotonic score transformation and propagated predictions to their GO ancestors. Terms already known for the test proteins were removed where appropriate, and the final output was restricted to the highest-ranked terms for each protein.</p>\n<h2>Result</h2>\n<p>The selected submission scored <strong>0.42114</strong> on the private leaderboard. After the private leaderboard was revealed, several nearby LTR variants scored slightly higher. The best reached <strong>0.42522</strong>. That variant differed from the selected solution in an interesting way: it removed the non-experimental evidence-code features and added richer BLAST pair-level features such as sequence identity, coverage, support from multiple homologs, and alignment statistics.</p>\n<p>The main lesson for me from CAFA6 was that the final ranking problem benefited more from combining genuinely different sources of evidence than from making any single predictor more complex. A cleaned and reproducible implementation of the final pipeline is in a public GitHub repo.</p>",
      "rawMarkdown": "Thanks to the CAFA and Kaggle organizers for another great protein function prediction challenge!\n\nMy CAFA6 solution built on two main ideas. From our previous **CAFA5 solution**, I kept pretrained protein embeddings, multilabel prediction, and Gene Ontology propagation. On top of this, I added a component-based learning-to-rank approach inspired by **GOLabeler** (You et al., 2018).\n\nThe main additions I did were a a hierarchy-aware MLP, PyBoost, BLAST, explicit use of non-experimental evidence, and more extensive use of the GO structure throughout modeling and postprocessing.\n\n## Data and annotation update\n\nI used a December 2025 UniProt-GOA snapshot to update the competition annotations. Experimental annotations were added to the training labels. Non-experimental annotations were used in the LTR stage both as a standalone component and as evidence-code features.\n\n## Main LTR component predictors\n\n1. **hierarchy-aware multilabel MLP on ESM2-3B embeddings**: parent GO terms were scored both directly from the protein representation and from the predictions of their modeled child terms, with the two sources combined inside the network.\n2. **PyBoost on ProtT5 embeddings** with a multi-output PyBoost model.\n3. **a multilabel MLP trained on concatenated ProtT5 and ESM-1b embeddings**\n4. **BLAST-KNN**: GO annotations were transferred from the strongest homologous training sequences, excluding self-hits and weighting neighbors by BLAST similarity.\n\n## Annotation-derived components\n\nI also used two simple components derived from the annotation data. \n1. **GO-frequency prior**: assigned each term its prevalence in the training annotations.\n2. **non-experimental annotation component**: assigned a score of 1 to protein-GO pairs present among the non-experimental UniProt annotations.\n\n## Candidate generation and learning to rank\n\nEach predictive component contributed its strongest terms for a protein, and their union formed the candidate set. Separate **XGBoost learning-to-rank models** were trained for BP, CC, and MF. For each candidate protein-GO pair, the LTR model used the component scores together with information accretion and non-experimental evidence-code features.\n\n## Validation\n\nI did not try to construct a validation set that reproduced the CAFA test setting. I used ordinary out-of-fold validation on held-out labeled proteins, even though this evaluates recovery of existing annotations rather than prediction of future experimental annotations.\n\nAll trainable components therefore produced OOF predictions. The LTR models were trained on those predictions, and BLAST OOF predictions were generated at the unique-sequence level to prevent leakage through identical sequences.\n\nCandidate generation was tuned together with the ranker. I compared different pruning settings and candidate-set sizes using the local CAFA metric, because the number and quality of terms passed to LTR had a noticeable effect on the validation score.\n\n## Postprocessing\n\nAfter LTR, I applied a monotonic score transformation and propagated predictions to their GO ancestors. Terms already known for the test proteins were removed where appropriate, and the final output was restricted to the highest-ranked terms for each protein.\n\n## Result\n\nThe selected submission scored **0.42114** on the private leaderboard. After the private leaderboard was revealed, several nearby LTR variants scored slightly higher. The best reached **0.42522**. That variant differed from the selected solution in an interesting way: it removed the non-experimental evidence-code features and added richer BLAST pair-level features such as sequence identity, coverage, support from multiple homologs, and alignment statistics.\n\n\nThe main lesson for me from CAFA6 was that the final ranking problem benefited more from combining genuinely different sources of evidence than from making any single predictor more complex. A cleaned and reproducible implementation of the final pipeline is in a public GitHub repo.",
      "votes": 4
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "3524600": "Thanks to the CAFA and Kaggle organizers for another great protein function prediction challenge!\n\nMy CAFA6 solution built on two main ideas. From our previous **CAFA5 solution**, I kept pretrained protein embeddings, multilabel prediction, and Gene Ontology propagation. On top of this, I added a component-based learning-to-rank approach inspired by **GOLabeler** (You et al., 2018).\n\nThe main additions I did were a a hierarchy-aware MLP, PyBoost, BLAST, explicit use of non-experimental evidence, and more extensive use of the GO structure throughout modeling and postprocessing.\n\n## Data and annotation update\n\nI used a December 2025 UniProt-GOA snapshot to update the competition annotations. Experimental annotations were added to the training labels. Non-experimental annotations were used in the LTR stage both as a standalone component and as evidence-code features.\n\n## Main LTR component predictors\n\n1. **hierarchy-aware multilabel MLP on ESM2-3B embeddings**: parent GO terms were scored both directly from the protein representation and from the predictions of their modeled child terms, with the two sources combined inside the network.\n2. **PyBoost on ProtT5 embeddings** with a multi-output PyBoost model.\n3. **a multilabel MLP trained on concatenated ProtT5 and ESM-1b embeddings**\n4. **BLAST-KNN**: GO annotations were transferred from the strongest homologous training sequences, excluding self-hits and weighting neighbors by BLAST similarity.\n\n## Annotation-derived components\n\nI also used two simple components derived from the annotation data. \n1. **GO-frequency prior**: assigned each term its prevalence in the training annotations.\n2. **non-experimental annotation component**: assigned a score of 1 to protein-GO pairs present among the non-experimental UniProt annotations.\n\n## Candidate generation and learning to rank\n\nEach predictive component contributed its strongest terms for a protein, and their union formed the candidate set. Separate **XGBoost learning-to-rank models** were trained for BP, CC, and MF. For each candidate protein-GO pair, the LTR model used the component scores together with information accretion and non-experimental evidence-code features.\n\n## Validation\n\nI did not try to construct a validation set that reproduced the CAFA test setting. I used ordinary out-of-fold validation on held-out labeled proteins, even though this evaluates recovery of existing annotations rather than prediction of future experimental annotations.\n\nAll trainable components therefore produced OOF predictions. The LTR models were trained on those predictions, and BLAST OOF predictions were generated at the unique-sequence level to prevent leakage through identical sequences.\n\nCandidate generation was tuned together with the ranker. I compared different pruning settings and candidate-set sizes using the local CAFA metric, because the number and quality of terms passed to LTR had a noticeable effect on the validation score.\n\n## Postprocessing\n\nAfter LTR, I applied a monotonic score transformation and propagated predictions to their GO ancestors. Terms already known for the test proteins were removed where appropriate, and the final output was restricted to the highest-ranked terms for each protein.\n\n## Result\n\nThe selected submission scored **0.42114** on the private leaderboard. After the private leaderboard was revealed, several nearby LTR variants scored slightly higher. The best reached **0.42522**. That variant differed from the selected solution in an interesting way: it removed the non-experimental evidence-code features and added richer BLAST pair-level features such as sequence identity, coverage, support from multiple homologs, and alignment statistics.\n\n\nThe main lesson for me from CAFA6 was that the final ranking problem benefited more from combining genuinely different sources of evidence than from making any single predictor more complex. A cleaned and reproducible implementation of the final pipeline is in a public GitHub repo."
  }
}