![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F17742767%2F9d1a047c845e597e50c4e9cac68c0748%2FGOAlpha.png?generation=1781790391982739&alt=media\")
\nFigure 1. Overview of GOAlpha.Competition Name: CAFA6 Protein Function Prediction
\nPrivate Leaderboard Score: 0.52430
\nPrivate Leaderboard Place: 1
\n
\nName: Hancheng Liu
Location: Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
\nEmail: lhcheng0407@gmail.com
\n
\nName: Tianyang Huang
Location: Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
\nEmail: tyhuang22@m.fudan.edu.cn
\n
\nName: Huiying Yan
Location: Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
\nEmail: yanhuiyingwangyi@163.com
\nOur team consists of three PhD students from ZhuLab at Fudan University, which is led by Professor Shanfeng Zhu. Our research focuses on automated protein function prediction, and biomedical text mining. Our lab has actively participated in the CAFA challenges and achieved top performance in CAFA3, CAFA4, and CAFA5. For CAFA6, our team developed GOAlpha, a Learning-to-Rank framework that integrates diverse sources of biological evidence, including protein sequence, network, structure, and textual information, for large-scale protein function prediction. H. Liu was responsible for collecting data, building the framework, and training the model. T. Huang and H. Yan were mainly engaged in biomedical text mining for protein annotation and result analysis.
\nAs illustrated in Figure 1, our CAFA6 solution, GOAlpha, integrates multiple sources of biological evidence, including protein sequence, interaction network, structure, domain, and biomedical literature information. The prediction scores generated by individual component methods are subsequently combined through a Learning-to-Rank Intergration to produce the final protein function predictions.
\n\nFigure 1. Overview of GOAlpha.
There are seven major component methods in GOAlpha:
\nBLAST-KNN: A K-nearest-neighbor method based on sequence similarity. BLAST is used to identify homologous proteins in the annotated training set, and GO terms are transferred by weighted voting of the nearest homologs [1].
Net-KNN: Similar to BLAST-KNN, it predicts protein functions from neighboring proteins in the protein-protein interaction (PPI) network [3].
FoldSeek-KNN: Similar to BLAST-KNN, it transfers GO annotations from structurally similar proteins identified by FoldSeek.
SVM-ESM2: It predicts GO terms using protein representations generated by the ESM2 protein language model. For each GO term, an SVM classifier is trained to estimate the association between proteins and GO terms [6].
LR-InterPro: Logistic regression classifiers are trained on a 118,677-dimensional binary feature vector generated from protein families, domains, and motifs identified by InterProScan. Each dimension indicates the presence or absence of a specific InterPro feature in the target protein [1].
GOXML: It is a biomedical text mining framework for large-scale GO function prediction. It formulates GO annotation as an extreme multi-label classification problem over protein–literature evidence units.
GORetriever: It first retrieves candidate GO terms from proteins with similar descriptions and literature evidence, and subsequently reranks them according to the semantic relevance between GO concepts and protein-associated textual evidence [7].
In addition to the component methods described above, GOAlpha further incorporates GO term frequency, and a 21-dimensional species encoding. These features, together with the prediction scores from all component methods, are integrated using an XGBoost-based Learning-to-Rank framework [1] to generate the final predictions.
\nTo evaluate the contribution of different biological evidence sources, we first constructed a validation benchmark that mimics the species distribution of the CAFA6 test set. Specifically, after obtaining the species composition of the CAFA6 target proteins, we randomly sampled 1,000 experimentally annotated proteins according to the same species distribution. These proteins were excluded from model training and used exclusively for validation and model selection.
\nThe component methods of GOAlpha can be grouped into five categories according to the information source they utilize:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F17742767%2F1219e9b682258bef3e3b910f902f0222%2FGOAlpha_validation_component_bar.png?generation=1781786956537006&alt=media\")
\nFigure 2. Performance on the validation benchmark.
Figure 2 shows the performance of GOAlpha and its component methods on the validation benchmark. Among individual predictors, sequence-based methods achieved the strongest standalone performance, highlighting the importance of evolutionary and language-model-derived sequence representations. In particular, BLAST-KNN and SVM-ESM2 consistently ranked among the best-performing component methods. FoldSeek-KNN demonstrated that structural similarity provides valuable functional signals beyond sequence homology. Although its standalone performance was slightly lower than the best sequence-based methods, it contributed complementary information during ensemble learning. Although the literature-based methods GOXML and GORetriever achieved only moderate standalone performance, they provided unique biological evidence that was often unavailable from sequence, structure, or network information alone. Overall, no single information source dominated across all proteins and GO categories. The final GOAlpha model therefore combines heterogeneous biological evidence through Learning-to-Rank, allowing the strengths of different component methods to be effectively integrated.
\nSimilar to our previous studies [6], we collected functional annotation information from three databases, SwissProt, GOA, and GO, according to the CAFA6 evidence codes. For model selection, we constructed a validation benchmark consisting of 1,000 experimentally annotated proteins sampled according to the species distribution of the CAFA6 target proteins. Finally, the prediction scores from all component methods were integrated using a Learning-to-Rank framework to generate the final GOAlpha predictions.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F17742767%2Ff1b654993aeca131cb85cb0e944953d0%2FGOAlpha_public_private_bar.png?generation=1781786974640408&alt=media\")
\nFigure 3. Public and private leaderboard scores differ across components.
One interesting observation is that the relative performance of component methods differs between the public and private leaderboard. As shown in Figure 3, most component methods achieved higher scores on the private leaderboard than on the public leaderboard, although the magnitude of improvement varied considerably. In particular, BLAST-KNN, FoldSeek-KNN, and SVM-ESM2 showed noticeable gains on the private leaderboard, suggesting that sequence- and structure-based signals generalize well to unseen proteins. In contrast, the literature-based methods, GOXML and GORetriever achieved the strongest performance on the public leaderboard but experienced a decrease on the private leaderboard. These results indicate that leaderboard rankings may depend on the specific composition of the evaluation proteins. Consequently, strong public leaderboard performance does not necessarily guarantee the best final ranking. This was one of the main reasons why we relied on Learning-to-Rank to combine heterogeneous predictors instead of selecting a single best-performing component. More broadly, we believe that public leaderboard evaluation alone may not fully reflect a method's ability to generalize to future protein annotations. In practical applications, different evidence sources including protein domain/family databases, interaction networks, GO annotations databases, and literature resources are continuously updated over time. Therefore, when constructing evaluation benchmarks, it is important to ensure that all evidence used by a method is restricted to information available before the test protein experimental annotations were obtained [1,3,4,6].
\nAmong individual component methods, protein language model-based methods exhibit the most robust standalone performance. Nevertheless, integrating literature-based evidence and structural information consistently improves the final ensemble. We estimate that a combination of homology search and protein language models can recover a substantial fraction of the final performance, while additional information sources mainly contribute complementary signals for difficult proteins.
\nMost component methods can be trained independently and in parallel. Generating protein embeddings, InterPro annotations, structural similarity searches, and literature-derived features requires the majority of computational resources. Training all component methods typically takes less than one day on modern GPU servers. Inference for CAFA6-scale datasets can be completed within one day, while the final Learning-to-Rank integration requires only a few hours.
\nWe would like to thank Prof. Shanfeng Zhu and Dr. Shaojun Wang for their valuable technical support, insightful discussions, and helpful suggestions throughout this work.
\nYou R, Zhang Z, Xiong Y, et al. GOLabeler: improving sequence-based large-scale protein function prediction by learning to rank. Bioinformatics, Volume 34, Issue 14, 2018, 2465–2473.
You R, Huang X, Zhu S. DeepText2GO: Improving large-scale protein function prediction with deep semantic text representation. Methods, 2018, 145: 82-90.
You R, Yao S, Xiong Y, et al. NetGO: improving large-scale protein function prediction with massive network information. Nucleic Acids Research, Volume 47, Issue W1, 2019, W379–W387.
Yao S, You R, Wang S, et al. NetGO 2.0: improving large-scale protein function prediction with massive sequence, text, domain, family and network information. Nucleic Acids Research, Volume 49, Issue W1, 2021, W469-W475.
You R, Yao S, Mamitsuka H, et al. DeepGraphGO: graph neural network for large-scale, multispecies protein function prediction. Bioinformatics (ISMB 2021), Volume 37, Supplement_1, 2021, i262–i271.
Wang S, You R, Liu Y, et al. NetGO 3.0: Protein Language Model Improves Large-scale Functional Annotations. Genomics, Proteomics & Bioinformatics, Volume 21, Issue 2, 2023, 349-358.
Yan H, Wang S, Liu H, et al. GORetriever: reranking protein-description-based GO candidates by literature-driven deep information retrieval for protein function annotation. Bioinformatics (ECCB 2024), Volume 40, Issue Supplement_2, September 2024, Pages ii53–ii61.
Wang S, Liu H, Zhai W, Zhu S. DualGOFiller: A Dual-Channel Graph Neural Network with Contrastive Learning for Enhancing Function Prediction in Partially Annotated Proteins. International Conference on Research in Computational Molecular Biology (RECOMB 2025), 49-67.
Yan H, Liu H, Wang S, Zhu S. GOAnnotator: accurate protein function annotation using automatically retrieved literature. Bioinformatics (ISMB 2025), Volume 41, Issue Supplement_1, July 2025, Pages i410–i419.
Competition Name: CAFA6 Protein Function Prediction
\nPrivate Leaderboard Score: 0.52430
\nPrivate Leaderboard Place: 1
\n
\nName: Hancheng Liu
Location: Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
\nEmail: lhcheng0407@gmail.com
\n
\nName: Tianyang Huang
Location: Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
\nEmail: tyhuang22@m.fudan.edu.cn
\n
\nName: Huiying Yan
Location: Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China
\nEmail: yanhuiyingwangyi@163.com
\nOur team consists of three PhD students from ZhuLab at Fudan University, which is led by Professor Shanfeng Zhu. Our research focuses on automated protein function prediction, and biomedical text mining. Our lab has actively participated in the CAFA challenges and achieved top performance in CAFA3, CAFA4, and CAFA5. For CAFA6, our team developed GOAlpha, a Learning-to-Rank framework that integrates diverse sources of biological evidence, including protein sequence, network, structure, and textual information, for large-scale protein function prediction. H. Liu was responsible for collecting data, building the framework, and training the model. T. Huang and H. Yan were mainly engaged in biomedical text mining for protein annotation and result analysis.
\nAs illustrated in Figure 1, our CAFA6 solution, GOAlpha, integrates multiple sources of biological evidence, including protein sequence, interaction network, structure, domain, and biomedical literature information. The prediction scores generated by individual component methods are subsequently combined through a Learning-to-Rank Intergration to produce the final protein function predictions.
\n\nFigure 1. Overview of GOAlpha.
There are seven major component methods in GOAlpha:
\nBLAST-KNN: A K-nearest-neighbor method based on sequence similarity. BLAST is used to identify homologous proteins in the annotated training set, and GO terms are transferred by weighted voting of the nearest homologs [1].
Net-KNN: Similar to BLAST-KNN, it predicts protein functions from neighboring proteins in the protein-protein interaction (PPI) network [3].
FoldSeek-KNN: Similar to BLAST-KNN, it transfers GO annotations from structurally similar proteins identified by FoldSeek.
SVM-ESM2: It predicts GO terms using protein representations generated by the ESM2 protein language model. For each GO term, an SVM classifier is trained to estimate the association between proteins and GO terms [6].
LR-InterPro: Logistic regression classifiers are trained on a 118,677-dimensional binary feature vector generated from protein families, domains, and motifs identified by InterProScan. Each dimension indicates the presence or absence of a specific InterPro feature in the target protein [1].
GOXML: It is a biomedical text mining framework for large-scale GO function prediction. It formulates GO annotation as an extreme multi-label classification problem over protein–literature evidence units.
GORetriever: It first retrieves candidate GO terms from proteins with similar descriptions and literature evidence, and subsequently reranks them according to the semantic relevance between GO concepts and protein-associated textual evidence [7].
In addition to the component methods described above, GOAlpha further incorporates GO term frequency, and a 21-dimensional species encoding. These features, together with the prediction scores from all component methods, are integrated using an XGBoost-based Learning-to-Rank framework [1] to generate the final predictions.
\nTo evaluate the contribution of different biological evidence sources, we first constructed a validation benchmark that mimics the species distribution of the CAFA6 test set. Specifically, after obtaining the species composition of the CAFA6 target proteins, we randomly sampled 1,000 experimentally annotated proteins according to the same species distribution. These proteins were excluded from model training and used exclusively for validation and model selection.
\nThe component methods of GOAlpha can be grouped into five categories according to the information source they utilize:
\n\nFigure 2. Performance on the validation benchmark.
Figure 2 shows the performance of GOAlpha and its component methods on the validation benchmark. Among individual predictors, sequence-based methods achieved the strongest standalone performance, highlighting the importance of evolutionary and language-model-derived sequence representations. In particular, BLAST-KNN and SVM-ESM2 consistently ranked among the best-performing component methods. FoldSeek-KNN demonstrated that structural similarity provides valuable functional signals beyond sequence homology. Although its standalone performance was slightly lower than the best sequence-based methods, it contributed complementary information during ensemble learning. Although the literature-based methods GOXML and GORetriever achieved only moderate standalone performance, they provided unique biological evidence that was often unavailable from sequence, structure, or network information alone. Overall, no single information source dominated across all proteins and GO categories. The final GOAlpha model therefore combines heterogeneous biological evidence through Learning-to-Rank, allowing the strengths of different component methods to be effectively integrated.
\nSimilar to our previous studies [6], we collected functional annotation information from three databases, SwissProt, GOA, and GO, according to the CAFA6 evidence codes. For model selection, we constructed a validation benchmark consisting of 1,000 experimentally annotated proteins sampled according to the species distribution of the CAFA6 target proteins. Finally, the prediction scores from all component methods were integrated using a Learning-to-Rank framework to generate the final GOAlpha predictions.
\n\nFigure 3. Public and private leaderboard scores differ across components.
One interesting observation is that the relative performance of component methods differs between the public and private leaderboard. As shown in Figure 3, most component methods achieved higher scores on the private leaderboard than on the public leaderboard, although the magnitude of improvement varied considerably. In particular, BLAST-KNN, FoldSeek-KNN, and SVM-ESM2 showed noticeable gains on the private leaderboard, suggesting that sequence- and structure-based signals generalize well to unseen proteins. In contrast, the literature-based methods, GOXML and GORetriever achieved the strongest performance on the public leaderboard but experienced a decrease on the private leaderboard. These results indicate that leaderboard rankings may depend on the specific composition of the evaluation proteins. Consequently, strong public leaderboard performance does not necessarily guarantee the best final ranking. This was one of the main reasons why we relied on Learning-to-Rank to combine heterogeneous predictors instead of selecting a single best-performing component. More broadly, we believe that public leaderboard evaluation alone may not fully reflect a method's ability to generalize to future protein annotations. In practical applications, different evidence sources including protein domain/family databases, interaction networks, GO annotations databases, and literature resources are continuously updated over time. Therefore, when constructing evaluation benchmarks, it is important to ensure that all evidence used by a method is restricted to information available before the test protein experimental annotations were obtained [1,3,4,6].
\nAmong individual component methods, protein language model-based methods exhibit the most robust standalone performance. Nevertheless, integrating literature-based evidence and structural information consistently improves the final ensemble. We estimate that a combination of homology search and protein language models can recover a substantial fraction of the final performance, while additional information sources mainly contribute complementary signals for difficult proteins.
\nMost component methods can be trained independently and in parallel. Generating protein embeddings, InterPro annotations, structural similarity searches, and literature-derived features requires the majority of computational resources. Training all component methods typically takes less than one day on modern GPU servers. Inference for CAFA6-scale datasets can be completed within one day, while the final Learning-to-Rank integration requires only a few hours.
\nWe would like to thank Prof. Shanfeng Zhu and Dr. Shaojun Wang for their valuable technical support, insightful discussions, and helpful suggestions throughout this work.
\nYou R, Zhang Z, Xiong Y, et al. GOLabeler: improving sequence-based large-scale protein function prediction by learning to rank. Bioinformatics, Volume 34, Issue 14, 2018, 2465–2473.
You R, Huang X, Zhu S. DeepText2GO: Improving large-scale protein function prediction with deep semantic text representation. Methods, 2018, 145: 82-90.
You R, Yao S, Xiong Y, et al. NetGO: improving large-scale protein function prediction with massive network information. Nucleic Acids Research, Volume 47, Issue W1, 2019, W379–W387.
Yao S, You R, Wang S, et al. NetGO 2.0: improving large-scale protein function prediction with massive sequence, text, domain, family and network information. Nucleic Acids Research, Volume 49, Issue W1, 2021, W469-W475.
You R, Yao S, Mamitsuka H, et al. DeepGraphGO: graph neural network for large-scale, multispecies protein function prediction. Bioinformatics (ISMB 2021), Volume 37, Supplement_1, 2021, i262–i271.
Wang S, You R, Liu Y, et al. NetGO 3.0: Protein Language Model Improves Large-scale Functional Annotations. Genomics, Proteomics & Bioinformatics, Volume 21, Issue 2, 2023, 349-358.
Yan H, Wang S, Liu H, et al. GORetriever: reranking protein-description-based GO candidates by literature-driven deep information retrieval for protein function annotation. Bioinformatics (ECCB 2024), Volume 40, Issue Supplement_2, September 2024, Pages ii53–ii61.
Wang S, Liu H, Zhai W, Zhu S. DualGOFiller: A Dual-Channel Graph Neural Network with Contrastive Learning for Enhancing Function Prediction in Partially Annotated Proteins. International Conference on Research in Computational Molecular Biology (RECOMB 2025), 49-67.
Yan H, Liu H, Wang S, Zhu S. GOAnnotator: accurate protein function annotation using automatically retrieved literature. Bioinformatics (ISMB 2025), Volume 41, Issue Supplement_1, July 2025, Pages i410–i419.
congrats 👏
", "rawMarkdown": "congrats 👏", "votes": null }, { "id": "3481851", "postDate": "06/26/2026 14:06:36", "content": "Great work after doing this self. \nMay I ask how you get the literature for each protein without getting noise in this feature. \nAnd the huge site is how you get the training data for the rerank model? \nThese are some gap before our method and I failed to improve my method in these sides
", "rawMarkdown": "Great work after doing this self. \nMay I ask how you get the literature for each protein without getting noise in this feature. \nAnd the huge site is how you get the training data for the rerank model? \nThese are some gap before our method and I failed to improve my method in these sides", "votes": null }, { "id": "3484255", "postDate": "06/30/2026 06:59:57", "content": "Thanks! These are indeed two important points.
\nFor the literature feature, we mainly used high-quality manually curated literature associated with SwissProt proteins. This helps reduce noisy or weakly related papers. For more details, please refer to our GORetriever paper.
For the second question, if you mean the Learning-to-Rank (LTR) model, we split the annotated proteins into a training set and a validation set. The training set was used to train the component methods, and the validation set was used to train the LTR model. Specifically, we estimated the species distribution of the CAFA6 test proteins and sampled 1,000 experimentally annotated proteins with the same distribution as the validation set. The prediction scores generated by all component methods on this validation set were then used as features for the LTR model. More details about the Learning-to-Rank framework can be found in GOLabeler.
Hope this helps!
", "rawMarkdown": "Thanks! These are indeed two important points.\n\n1. For the literature feature, we mainly used high-quality manually curated literature associated with SwissProt proteins. This helps reduce noisy or weakly related papers. For more details, please refer to our **[GORetriever](https://doi.org/10.1093/bioinformatics/btae401)** paper.\n\n2. For the second question, if you mean the Learning-to-Rank (LTR) model, we split the annotated proteins into a training set and a validation set. The training set was used to train the component methods, and the validation set was used to train the LTR model. Specifically, we estimated the species distribution of the CAFA6 test proteins and sampled 1,000 experimentally annotated proteins with the same distribution as the validation set. The prediction scores generated by all component methods on this validation set were then used as features for the LTR model. More details about the Learning-to-Rank framework can be found in **[GOLabeler](https://doi.org/10.1093/bioinformatics/bty130)**.\n\nHope this helps!", "votes": null } ], "comments": [ { "id": 3476332, "author_name": "khalwaniev", "author_url": "", "post_date": "06/20/2026 13:26:19", "content": "congrats 👏
", "votes": null, "replies": [] }, { "id": 3481851, "author_name": "aizealotwu", "author_url": "", "post_date": "06/26/2026 14:06:36", "content": "Great work after doing this self. \nMay I ask how you get the literature for each protein without getting noise in this feature. \nAnd the huge site is how you get the training data for the rerank model? \nThese are some gap before our method and I failed to improve my method in these sides
", "votes": null, "replies": [ { "id": 3484255, "author_name": "liuhc98", "author_url": "", "post_date": "06/30/2026 06:59:57", "content": "Thanks! These are indeed two important points.
\nFor the literature feature, we mainly used high-quality manually curated literature associated with SwissProt proteins. This helps reduce noisy or weakly related papers. For more details, please refer to our GORetriever paper.
For the second question, if you mean the Learning-to-Rank (LTR) model, we split the annotated proteins into a training set and a validation set. The training set was used to train the component methods, and the validation set was used to train the LTR model. Specifically, we estimated the species distribution of the CAFA6 test proteins and sampled 1,000 experimentally annotated proteins with the same distribution as the validation set. The prediction scores generated by all component methods on this validation set were then used as features for the LTR model. More details about the Learning-to-Rank framework can be found in GOLabeler.
Hope this helps!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "3474662": "# A. MODEL SUMMARY\n\n## A.1 Background on you/your team\n\n**Competition Name:** CAFA6 Protein Function Prediction\n\n**Private Leaderboard Score:** 0.52430\n\n**Private Leaderboard Place:** 1\n\n