# 3rd Place Solution for the CAFA 5 Protein Function Prediction

Thanks to Kaggle and hosts for organising such an interesting competition.
This competition task:"Predict the function of this protein using any available data" was linked to the fundamental of biology, making it a very exciting experience for me.
Thanks also to the community, the discussions and notebooks were very helpful.
Overview
I used the following NN models.
![overview](assets/ed6db877700484fa2df8f4b6.png)
Protein Sequence
I have extracted embeddings using protein language models from a protein sequence.
The models I used include T5, ESM2t36, ESM2t48, and combinations of these.
Taxonomy Data
I one-hot encoded the taxonomy data and combined it with embeddings generated by protein language models.
During this process, only the 90 taxonomy IDs included in the test data were used.
Non-Experimental Annotation
The
[evidence codes](https://geneontology.org/docs/guide-go-evidence-codes/)
of UniProt GOA annotations consist of 13 experimental labels and 11 non-experimental labels.
Experimental labels are used as GT and non-experimental labels are not used in training data.
So I extracted non-experimental labels from UniProt GOA data and used it  as additional features.
The non-experimental labels were one-hot encoded and transformed into a tensor of dimensions (batch size, 11, number of GT labels).
Then this tensor is processed using a 1D-CNN which kernel_size is 1.
![](assets/67327014cd48d4cf4cdf1406.png)
Validation Data
The training data was created from experimental annotations in UniProtKB as of November 17, 2022, and the test data (private leaderboard) consists of data experimentally verified after August 22, 2023.
![](assets/2409cad9051429d21516d875.png)
Therefore, I approached the validation strategy as if it were time-series data.
Validation data was created by extracting experimental annotations from the latest UniProtKB data at the time of the competition, and any sub-ontology data which wrere annotated in the training data was excluded from the validation data.