# 68th Place Solution for the CAFA 6 Protein Function Prediction Competition

I am happy to share the solution that placed
68th of 2,259 teams
and received a
silver medal
in CAFA 6.
My final submission was not a new end-to-end protein model. It was a prediction-stage ensemble built from two complementary public notebook outputs. My contribution was the layer that made those outputs compatible and competition-ready: score harmonization, agreement-aware stacking, Gene Ontology processing, Information Accretion-aware filtering, memory-efficient inference, and strict submission validation.
Context
Business context:
[Competition overview](https://www.kaggle.com/competitions/cafa-6-protein-function-prediction/overview)
Data context:
[Competition data](https://www.kaggle.com/competitions/cafa-6-protein-function-prediction/data)
Reproducible notebook:
[Open the reproducible Kaggle notebook](https://www.kaggle.com/code/ismailcanoguz/68th-place-solution-for-the-cafa-6)
Overview of the Approach
Public prediction sources
The final stack used two public systems:
[CAFA ensemble 0.378](https://www.kaggle.com/code/antonoof/cafa-ensemble-0-378)
by
antonoof
This notebook averages three fold-level prediction files and propagates scores through the Gene Ontology graph.
[CAFA6 GOA + ProtT5 IC-Aware](https://www.kaggle.com/code/maryamkhaled67/cafa6-goa-prott5-ic-aware)
by
maryamkhaled67
This notebook combines GOA transfer with ProtT5/InterPro predictions, propagates GO ancestors, and applies Information Accretion-aware filtering.
These systems rely on different evidence. The first is a fold ensemble; the second combines annotation-transfer and sequence/domain signals. My objective was to preserve this complementarity without mixing incompatible score spaces or producing an excessively large union.
1. Score-space diagnosis and harmonization
The first important observation was that the two files were not numerically equivalent. One source could contain values above 1, while the other was probability-like.
A direct weighted average would therefore be misleading. I inspected the source distributions before loading the complete files and used:
if sample_max > 1.2:
    probability = sigmoid(raw_score / temperature)
else:
    probability = clip(raw_score, 0.001, 1.0)
For the unbounded source, I used a temperature of
8.0
to reduce sigmoid saturation. This is a monotonic score mapping rather than a claim of formal probabilistic calibration.
2. Memory-aware loading
The source files contained tens of millions of rows. Loading both into memory and joining them directly was unnecessary.
I processed each file in chunks, retained only official test targets and valid GO terms, canonicalized alternative GO IDs, merged duplicate
(protein, GO term)
pairs by maximum score, and periodically kept only the strongest terms per protein.
A final exact top-k pass ensured that every protein respected the intended loading limit.
3. Agreement-aware logit stacking
For each candidate GO term, I formed three signals:
pA
: harmonized score from Model A;
pB
: probability from Model B;
pAgreement
: a product-of-experts-style score when both systems predicted the term.
The final probability was calculated in logit space:
z = (
    wA * logit(pA)
    + wB * logit(pB)
    + wAgreement * logit(pAgreement)
) / blend_temperature

prediction = sigmoid(z)
The recorded final parameters were:
Model A weight:       0.0872988403
Model B weight:       0.7224695683
Agreement weight:     0.1902315915
Blend temperature:    0.8547518465
The agreement term acted as a consensus gate. Terms supported by both public systems were favored, while a source-specific term needed stronger evidence to survive.
4. Gene Ontology processing
The final stack was ontology-aware:
alternative GO IDs were mapped to primary IDs;
obsolete terms with valid replacements were handled;
both
is_a
and
part_of
parent relations were parsed;
each child score was propagated to all ancestors using hard maximum.
This prevented inconsistent output and duplicate semantic labels.
5. Information Accretion-aware pruning
CAFA evaluation gives more weight to informative, less generic GO terms. I therefore used Information Accretion in two ways:
weak low-IA terms were removed unless their confidence was high;
remaining terms were ranked using:
rank_score = probability * (information_accretion + epsilon)
I retained at most 650 terms per protein, below the official limit of 1,500.
6. Output shaping and validation
The final scores received a mild rank decay to separate tied values. The submission writer then enforced:
valid test target IDs;
valid canonical GO terms;
scores in
(0, 1]
;
up to three significant digits;
no duplicate
(protein, GO term)
pairs;
no more than 1,500 terms per protein;
tab-separated output without a header.
Validation Strategy
CAFA 6 is a prospective evaluation. The true test labels are annotations that appeared after the submission deadline, so ordinary cross-validation cannot reproduce the final leaderboard.
I used currently known annotations for target proteins that also appeared in
train_terms.tsv
as a proxy calibration set. The annotations were expanded with GO ancestors and split deterministically into calibration and holdout proteins.
The proxy metric used IA-weighted precision, recall, and F-max independently for Biological Process, Cellular Component, and Molecular Function.
This proxy was useful for checking whether the blend behaved sensibly, but it was not a reliable estimate of the final score. In particular, the GOA-derived source can be favored when current annotations are reused.
Details of the Submission
What was most important
Score compatibility before ensembling
The most important engineering decision was not to treat every public score as a probability. Diagnosing and harmonizing the score spaces made the later weighting meaningful.
Consensus rather than a simple union
Combining every term from both sources increased coverage but also introduced weak predictions. The explicit agreement signal provided a controlled way to reward shared evidence.
Metric-aware ontology processing
GO propagation and IA-aware ranking were not cosmetic post-processing. They aligned the output with the hierarchical label space and the competition metric.
Practicality on Kaggle
Chunked reading and per-protein pruning allowed the complete stack to run without loading both large submissions into a single dataframe.
What did not work well
A direct arithmetic average was rejected because the input score ranges differed.
A pure union retained too many low-confidence terms.
The original random search was not fully reproducible because NumPy was not seeded.
The current-annotation proxy was optimistic for GOA-derived predictions.
One global set of weights was used across BP, CC, and MF; ontology-specific calibration may be stronger.
Limitations and possible improvements
This is a stacking solution and depends on two public upstream outputs. It does not train a new protein language model.
Possible improvements include:
time-aware annotation splits;
separate temperatures and weights for BP, CC, and MF;
taxonomy-aware source reliability;
a learned missing-evidence policy for source-specific terms;
direct evaluation with the official CAFA evaluator;
systematic experiment logging for every submitted variant.
Reproduction
Attach the following inputs to the published notebook:
cafa-6-protein-function-prediction
output of
antonoof/cafa-ensemble-0-378
output of
maryamkhaled67/cafa6-goa-prott5-ic-aware
The notebook automatically checks that the two public sources resolve to different files. It then reports score diagnostics, source overlap, proxy results, output statistics, and final validation.
Sources
CAFA 6 competition
[https://www.kaggle.com/competitions/cafa-6-protein-function-prediction](https://www.kaggle.com/competitions/cafa-6-protein-function-prediction)
CAFA ensemble 0.378 — antonoof
[https://www.kaggle.com/code/antonoof/cafa-ensemble-0-378](https://www.kaggle.com/code/antonoof/cafa-ensemble-0-378)
CAFA6 GOA + ProtT5 IC-Aware — maryamkhaled67
[https://www.kaggle.com/code/maryamkhaled67/cafa6-goa-prott5-ic-aware](https://www.kaggle.com/code/maryamkhaled67/cafa6-goa-prott5-ic-aware)
Original competition notebook
[https://www.kaggle.com/code/ismailcanoguz/mybestscore-372?scriptVersionId=295439875](https://www.kaggle.com/code/ismailcanoguz/mybestscore-372?scriptVersionId=295439875)
CAFA-evaluator
[https://github.com/BioComputingUP/CAFA-evaluator](https://github.com/BioComputingUP/CAFA-evaluator)
Thank you to the public notebook authors, the competition organizers, and the CAFA community.