{"cells": [{"cell_type": "markdown", "metadata": {}, "source": "# CAFA-6 Protein Function Prediction: Complete Experiments Summary\n\nThis notebook documents all approaches tried, results obtained, and key insights from our extensive experimentation on the CAFA-6 Kaggle competition.\n\n**Competition Goal**: Predict Gene Ontology (GO) terms for 224,309 test proteins\n\n**Metric**: F-max (maximum protein-centric F-measure across thresholds)\n\n**Our Best Score**: 0.378 (ESM-2 Embedding approach)\n\n**Top Leaderboard**: 0.456 (Mixture of Experts team)"}, {"cell_type": "markdown", "metadata": {}, "source": "## Table of Contents\n\n1. [Executive Summary](#1.-Executive-Summary)\n2. [Approaches That Worked](#2.-Approaches-That-Worked)\n3. [Approaches That Failed](#3.-Approaches-That-Failed)\n4. [Complete Submission History](#4.-Complete-Submission-History)\n5. [Key Technical Insights](#5.-Key-Technical-Insights)\n6. [Gap Analysis vs Top Teams](#6.-Gap-Analysis-vs-Top-Teams)\n7. [Code Examples](#7.-Code-Examples)\n8. [Conclusions & Lessons Learned](#8.-Conclusions-&-Lessons-Learned)\n9. [Citations, Credits & References](#9.-Citations,-Credits-&-References)"}, {"cell_type": "markdown", "metadata": {}, "source": "---\n## 1. Executive Summary\n\n### Timeline Overview\n- **Start Date**: November 2025\n- **End Date**: January 2026\n- **Total Scripts Created**: 135\n- **Total Submissions**: 100+\n- **Best Score Achieved**: 0.378 F-max\n\n### Key Milestones\n\n| Date | Score | Approach | Significance |\n|------|-------|----------|-------------|\n| Nov 2025 | 0.340 | Multi-Model Blend V2 | Early best |\n| Dec 2025 | 0.336 | GOA Baseline | Realized GOA is strong |\n| Dec 2025 | 0.338 | ESM2 90/10 Blend | Found optimal blend ratio |\n| Jan 2026 | 0.374 | GOA 60% + ProtT5 40% | Major improvement |\n| Jan 2026 | 0.378 | ESM-2 Embedding Notebook | Current best |\n\n### Final Status\n- **Our Best**: 0.378\n- **Top Leaderboard**: 0.456\n- **Gap**: 0.078 (20.6% improvement needed)"}, {"cell_type": "markdown", "metadata": {}, "source": "---\n## 2. Approaches That Worked\n\n### 2.1 GOA Baseline (0.336)\n\n**What is GOA?**\n- UniProt Gene Ontology Annotations\n- Direct transfer of known annotations to test proteins\n- Pre-computed predictions available as Kaggle dataset\n\n**Why it works:**\n- Well-calibrated confidence scores (not all 1.0)\n- Comprehensive coverage (all 224,309 test proteins)\n- Based on evidence codes from UniProt\n\n**Key insight:** The GOA baseline is extremely hard to beat. Most ML approaches score worse.\n\n---\n\n### 2.2 ESM2 90/10 Weighted Blend (0.338)\n\n```python\n# Optimal blending formula\nfinal_score = 0.9 * GOA_score + 0.1 * ESM2_ML_score\n```\n\n**Why it works:**\n- 90% GOA preserves the strong baseline\n- 10% ESM2 provides calibration adjustment\n- Only blend where BOTH have predictions (don't add novel)\n\n**Experiments with different ratios:**\n\n| Ratio (GOA/ML) | Score | Result |\n|----------------|-------|--------|\n| 95/5 | 0.337 | Slightly worse |\n| **90/10** | **0.338** | **Optimal** |\n| 80/20 | 0.337 | Worse |\n| 70/30 | 0.335 | Worse |\n\n---\n\n### 2.3 GOA + ProtT5 Blend (0.374)\n\n**The breakthrough approach:**\n\n```python\nWEIGHT_GOA = 0.60\nWEIGHT_PROTT5 = 0.40\n\n# Blending rule: only average if BOTH have predictions\nif s_goa > 0 and s_prott5 > 0:\n    merged[t] = WEIGHT_GOA * s_goa + WEIGHT_PROTT5 * s_prott5\nelse:\n    merged[t] = s_goa if s_goa > 0 else s_prott5\n```\n\n**Key improvements over ESM2 blend:**\n- ProtT5 predictions have better coverage\n- 40% weight (vs 10% for ESM2) works better for ProtT5\n- ProtT5 provides complementary signal, not just calibration\n\n---\n\n### 2.4 ESM-2 Embedding Approach (0.378) - CURRENT BEST\n\nThis came from a Kaggle notebook using ESM-2 embeddings with a specific inference approach.\n\n**Why it beat our blending approaches:**\n- Direct embedding-based predictions\n- Better calibration than simple blending\n- Likely uses techniques we haven't fully replicated"}, {"cell_type": "markdown", "metadata": {}, "source": "---\n## 3. Approaches That Failed\n\n### 3.1 GO Hierarchy Propagation (0.005 - 0.263)\n\n**What we tried:**\n- Propagate predictions to parent GO terms\n- Depth-based decay scoring\n- Negative propagation\n\n**Why it failed:**\n- **CAFA evaluation handles propagation internally**\n- Our propagation creates duplicates\n- Depth decay destroys calibration\n\n**CRITICAL LESSON:** Never do GO propagation - the evaluation system does it.\n\n---\n\n### 3.2 Adding Novel Predictions (No improvement)\n\n**What we tried:**\n- Add ML predictions not in GOA\n- Use consensus (ESM2 + ProtT5 both agree)\n- Various confidence thresholds (0.35, 0.5, 0.8)\n\n**Results:**\n\n| Novel Predictions Added | Score | Result |\n|------------------------|-------|--------|\n| 16,502 (threshold 0.5) | 0.338 | No change |\n| 469,260 (threshold 0.35) | 0.332 | WORSE |\n| 0 (baseline blend) | 0.338 | Best |\n\n**Why it failed:**\n- F-max heavily penalizes false positives\n- Even high-confidence consensus predictions are mostly wrong\n- ML predictions not in GOA are usually false positives\n\n---\n\n### 3.3 GOA as Input Features (0.318)\n\n**The idea:**\nBased on CAFA-5 winner insight that GOA can be used as input features.\n\n```python\nclass GOAAwareModel(nn.Module):\n    # Input: ESM2 embeddings + GOA predictions\n    # Output: Refined predictions\n    # Learned goa_residual_weight (~0.67)\n```\n\n**Validation F-max: 0.5262** (amazing!)\n\n**Test F-max: 0.318** (WORSE than baseline!)\n\n**Why it failed:**\n- Model overfits to training GOA patterns\n- Test GOA distribution differs from training\n- Learned to \"copy and adjust\" GOA, which doesn't transfer\n- Circular dependency: using GOA to predict GO terms\n\n---\n\n### 3.4 GCN with GOA (ProtBoost approach) (0.197)\n\n**The idea:**\nFollowing CAFA-5 2nd place (ProtBoost), use GCN to refine predictions.\n\n**Our implementation:**\n- GCN stacker with GOA as input feature (1 of 3 features)\n- GO ontology graph structure\n- SAGEConv layers\n\n**Result: 0.197 (CATASTROPHIC - 41% worse than baseline)**\n\n**Why it failed:**\n- GOA dominated as 1/3 features (33% weight)\n- ProtBoost uses GOA as 1/29 features (3.4% weight)\n- Our GCN learned to copy GOA, not improve it\n- Graph convolutions spread predictions \u2192 false positives\n\n---\n\n### 3.5 Evidence Weighting (0.179 - 0.234)\n\n**The idea:**\nWeight GOA predictions by evidence code quality.\n\n**Evidence code hierarchy:**\n- EXP (experimental) = 1.0\n- IDA, IMP = 0.95\n- ISS (sequence similarity) = 0.8\n- IEA (electronic) = 0.7\n\n**Why it failed:**\n- GOA already has well-calibrated scores\n- Our weighting broke the calibration\n- Set high scores everywhere (mean 0.991 vs 0.308 in original)\n\n---\n\n### 3.6 Filtering Low-Confidence Predictions (0.323)\n\n**Hypothesis:** GOA over-predicts (20.3 GO terms/protein vs 6.5 in training)\n\n**What we tried:** Keep only predictions with score >= 0.2\n\n**Result:** WORSE (0.323 vs 0.336 baseline)\n\n**Why:** Low-confidence predictions provide valuable recall.\n\n---\n\n### 3.7 ML-Only Approaches\n\n| Approach | Score | Issue |\n|----------|-------|-------|\n| LightGBM multi-target | 0.132 | False positives |\n| Practical Model | 0.077 | Massive overfitting |\n| DIAMOND only | 0.027 | Homology insufficient |\n\n**Key insight:** Pure ML without GOA always fails on this task."}, {"cell_type": "markdown", "metadata": {}, "source": "---\n## 4. Complete Submission History\n\n### Score Distribution Visualization"}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": "import matplotlib.pyplot as plt\nimport numpy as np\n\n# Key submissions (selected for clarity)\nsubmissions = [\n    ('ESM-2 Embedding', 0.378),\n    ('GOA 60% + ProtT5 40%', 0.374),\n    ('Simple blend 60/40', 0.364),\n    ('Compact blend', 0.343),\n    ('ESM2 90/10 Blend', 0.338),\n    ('GOA Baseline', 0.336),\n    ('ProtBoost MLP', 0.330),\n    ('GOA filtered', 0.323),\n    ('GOA-aware model', 0.318),\n    ('Evidence weighted', 0.234),\n    ('GCN with GOA', 0.197),\n    ('LightGBM ML-only', 0.132),\n    ('Practical model', 0.077),\n    ('DIAMOND only', 0.027),\n    ('Bad propagation', 0.005),\n]\n\nnames, scores = zip(*submissions)\ncolors = ['green' if s >= 0.36 else 'blue' if s >= 0.33 else 'orange' if s >= 0.2 else 'red' for s in scores]\n\nfig, ax = plt.subplots(figsize=(12, 8))\nbars = ax.barh(names, scores, color=colors)\nax.set_xlabel('F-max Score')\nax.set_title('CAFA-6 Submission History: Key Approaches')\nax.axvline(x=0.336, color='gray', linestyle='--', label='GOA Baseline')\nax.axvline(x=0.456, color='purple', linestyle='--', label='Top Leaderboard')\nax.legend()\n\n# Add value labels\nfor bar, score in zip(bars, scores):\n    ax.text(score + 0.01, bar.get_y() + bar.get_height()/2, f'{score:.3f}', va='center')\n\nplt.tight_layout()\nplt.show()"}, {"cell_type": "markdown", "metadata": {}, "source": "### Chronological Summary Table\n\n| Date | Score | Method | Key Insight |\n|------|-------|--------|-------------|\n| 2026-01-25 | 0.364 | Simple blend (no power scaling) | Power scaling hurts |\n| 2026-01-25 | 0.343 | Compact blend (top 100, t=0.1) | Too aggressive filtering |\n| 2026-01-20 | **0.378** | **ESM-2 Embedding Notebook** | **CURRENT BEST** |\n| 2026-01-20 | **0.374** | **GOA 60% + ProtT5 40%** | **Major breakthrough** |\n| 2026-01-19 | 0.197 | GCN with GOA | CATASTROPHIC |\n| 2026-01-17 | 0.318 | GOA-aware model | Overfitting |\n| 2026-01-16 | 0.234 | Evidence weighted | Broke calibration |\n| 2026-01-09 | 0.338 | ESM2 90/10 blend | Optimal ratio |\n| 2026-01-08 | 0.336 | Py-Boost experiments | No improvement |\n| 2025-12-27 | 0.338 | ESM2 90/10 blend | First best |\n| 2025-12-17 | 0.336 | GOA baseline | Strong baseline |\n| 2025-12-04 | 0.005 | Bad propagation | WORST score |\n| 2025-11-14 | 0.340 | Multi-Model Blend V2 | Early best |"}, {"cell_type": "markdown", "metadata": {}, "source": "---\n## 5. Key Technical Insights\n\n### 5.1 The F-max Metric is Unforgiving\n\nF-max = max over thresholds of:\n$$F_1(t) = \\frac{2 \\cdot precision(t) \\cdot recall(t)}{precision(t) + recall(t)}$$\n\n**Key properties:**\n- False positives hurt precision dramatically\n- Adding predictions usually hurts more than helps\n- Calibration matters more than raw model performance\n\n### 5.2 Validation-Test Gap is Massive\n\n| Model | Validation F-max | Test F-max | Gap |\n|-------|-----------------|------------|-----|\n| GOA-aware | 0.5262 | 0.318 | -0.208 |\n| ProtBoost MLP | 0.4298 | 0.330 | -0.100 |\n| Practical model | 0.3486 | 0.077 | -0.272 |\n| ProtT5 | 0.33 | 0.165 | -0.165 |\n\n**Lesson:** Don't trust validation scores. Test distribution is fundamentally different.\n\n### 5.3 What the Top Teams Are Doing (0.456)\n\nBased on analysis of CAFA-5 winners and CAFA-6 top notebooks:\n\n1. **GOCurator (CAFA-5 1st)**: Text mining + literature retrieval\n2. **ProtBoost (CAFA-5 2nd)**: 29 features per GO term, GCN stacking\n3. **Top CAFA-6 notebooks**: Negative propagation, power scaling\n\n### 5.4 The GOA Paradox\n\n**Paradox:** GOA is both our best baseline AND impossible to improve.\n\nEvery attempt to use GOA in a learnable way failed:\n- GOA as input features: -5.4%\n- GCN with GOA: -41.4%\n- Evidence weighting: -46.7%\n- Pure GOA baseline: BEST\n\n**The only improvement came from blending with ProtT5 predictions.**"}, {"cell_type": "markdown", "metadata": {}, "source": "---\n## 6. Gap Analysis vs Top Teams\n\n### Current Leaderboard (Jan 2026)\n\n| Rank | Team | Score | Gap from Us |\n|------|------|-------|-------------|\n| 1 | Mixture of Experts | 0.456 | +0.078 (20.6%) |\n| 2 | WePredictProteins | 0.441 | +0.063 |\n| 3 | Guoliang&Born4 | 0.440 | +0.062 |\n| 4 | chya | 0.429 | +0.051 |\n| 5 | mirrandax | 0.423 | +0.045 |\n| ... | ... | ... | ... |\n| ~100+ | **Us** | **0.378** | - |\n\n### What We're Missing\n\n| Missing Component | Used By | Estimated Impact | Difficulty |\n|-------------------|---------|------------------|------------|\n| Literature/text mining | GOCurator | +0.05-0.08 | High |\n| Learning to Rank | GOCurator, NetGO | +0.03-0.05 | Medium |\n| Proper GCN (29 features) | ProtBoost | +0.02-0.04 | Medium |\n| ESM2 fine-tuning (LoRA) | Various | +0.01-0.03 | Medium |\n| AlphaFold structures | ESM-GNN | +0.01-0.02 | High |"}, {"cell_type": "markdown", "metadata": {}, "source": "---\n## 7. Code Examples\n\n### 7.1 Optimal Blending (0.374 approach)"}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": "# Simplified example of the 0.374 blending approach\ndef blend_goa_prott5(goa_preds, prott5_preds, test_proteins):\n    \"\"\"\n    Blend GOA and ProtT5 predictions.\n    \n    Parameters:\n    - goa_preds: dict of {protein_id: {go_term: score}}\n    - prott5_preds: dict of {protein_id: {go_term: score}}\n    - test_proteins: set of protein IDs\n    \n    Returns:\n    - blended: dict of {protein_id: {go_term: score}}\n    \"\"\"\n    GOA_WEIGHT = 0.60\n    PROTT5_WEIGHT = 0.40\n    \n    blended = {}\n    \n    for pid in test_proteins:\n        goa = goa_preds.get(pid, {})\n        prott5 = prott5_preds.get(pid, {})\n        \n        all_terms = set(goa.keys()) | set(prott5.keys())\n        \n        scores = {}\n        for go in all_terms:\n            g = goa.get(go, 0)\n            p = prott5.get(go, 0)\n            \n            if g > 0 and p > 0:\n                # Both have predictions - weighted blend\n                scores[go] = GOA_WEIGHT * g + PROTT5_WEIGHT * p\n            elif g > 0:\n                # GOA only - keep as is\n                scores[go] = g\n            elif p > 0:\n                # ProtT5 only - slight discount\n                scores[go] = p * 0.85\n        \n        if scores:\n            blended[pid] = scores\n    \n    return blended\n\nprint(\"Blending function defined. Key: only blend when BOTH have predictions.\")"}, {"cell_type": "markdown", "metadata": {}, "source": "### 7.2 Why Power Scaling Hurts"}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": "# Power scaling was tried but hurt performance\ndef apply_power_scaling(scores, power=0.8, max_cap=0.95):\n    \"\"\"\n    Power scaling: score^power (boosts lower scores)\n    \n    Problem: Distorts calibration that F-max relies on\n    \"\"\"\n    max_score = max(scores.values()) if scores else 0\n    if max_score <= 0:\n        return scores\n    \n    scaled = {}\n    for go, score in scores.items():\n        normalized = (score / max_score) ** power\n        scaled[go] = normalized * min(max_score, max_cap)\n    \n    return scaled\n\n# Example showing the distortion\nimport numpy as np\noriginal = np.array([0.1, 0.3, 0.5, 0.7, 0.9])\nscaled = original ** 0.8\n\nprint(\"Original scores:\", original)\nprint(\"After power=0.8:\", np.round(scaled, 3))\nprint(\"\\nNote: Low scores boosted disproportionately - breaks calibration\")"}, {"cell_type": "markdown", "metadata": {}, "source": "### 7.3 The Asymmetric Focal Loss"}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": "import torch\nimport torch.nn as nn\n\nclass AsymmetricFocalLoss(nn.Module):\n    \"\"\"\n    The key loss function for multi-label classification.\n    \n    Parameters:\n    - gamma_neg: Focus on hard negatives (default 4.0)\n    - gamma_pos: Focus on hard positives (default 0.0)\n    - clip: Minimum probability to avoid log(0)\n    \n    Why it works:\n    - Extreme class imbalance (~6 positives per 1500 labels)\n    - gamma_neg=4 down-weights easy negatives\n    - Allows model to focus on rare positive labels\n    \n    Reference: Ridnik et al. (2021) ICCV\n    \"\"\"\n    def __init__(self, gamma_neg=4.0, gamma_pos=0.0, clip=0.05):\n        super().__init__()\n        self.gamma_neg = gamma_neg\n        self.gamma_pos = gamma_pos\n        self.clip = clip\n\n    def forward(self, logits, targets):\n        # Sigmoid probabilities\n        probs = torch.sigmoid(logits)\n        \n        # Separate positive and negative\n        pos_mask = targets == 1\n        neg_mask = targets == 0\n        \n        # Positive loss (standard BCE)\n        pos_probs = probs[pos_mask].clamp(min=self.clip)\n        pos_loss = -torch.log(pos_probs)\n        if self.gamma_pos > 0:\n            pos_loss = pos_loss * ((1 - pos_probs) ** self.gamma_pos)\n        \n        # Negative loss (asymmetric focal)\n        neg_probs = probs[neg_mask].clamp(max=1-self.clip)\n        neg_loss = -torch.log(1 - neg_probs)\n        neg_loss = neg_loss * (neg_probs ** self.gamma_neg)  # Down-weight easy negatives\n        \n        return pos_loss.mean() + neg_loss.mean()\n\nprint(\"AsymmetricFocalLoss: gamma_neg=4 is critical for extreme class imbalance\")"}, {"cell_type": "markdown", "metadata": {}, "source": "---\n## 8. Conclusions & Lessons Learned\n\n### What We Learned\n\n1. **GOA is king**: The UniProt GOA baseline is extremely hard to beat\n2. **Don't do propagation**: CAFA handles it internally\n3. **F-max punishes false positives**: Be conservative with predictions\n4. **Validation doesn't transfer**: Expect 10-20% drops on test\n5. **Simple blending works**: 60/40 GOA/ProtT5 better than complex ML\n6. **More ML weight hurts**: 90/10 better than 80/20 for ESM2\n7. **Novel predictions don't help**: Even high-confidence consensus is mostly wrong\n\n### What We Would Do Differently\n\n1. **Start with GOA + ProtT5 blend** from day one\n2. **Skip propagation experiments** entirely\n3. **Focus on calibration** over raw model performance\n4. **Trust the validation-test gap** - expect ~0.1 drop\n5. **Study top notebooks first** before trying novel ideas\n\n### Remaining Gap to Top\n\nTo reach 0.456 from 0.378, we would need:\n- Text mining / literature features (like GOCurator)\n- Learning to Rank ensemble\n- Proper GCN stacking with 29+ features\n- ESM-2 fine-tuning with LoRA\n- Possibly structural features from AlphaFold\n\n### Final Takeaway\n\n**The CAFA protein function prediction task is fundamentally about annotation transfer, not ML.**\n\nThe best approaches leverage existing biological knowledge (GOA, ProtT5 pre-training, homology) rather than trying to learn from scratch. Pure ML approaches consistently fail because:\n- Extreme class imbalance (~6 positives per protein out of 40,000+ GO terms)\n- Test distribution differs from training\n- F-max metric heavily penalizes false positives\n\n**Our journey: From 0.005 (catastrophic) to 0.378 (respectable)**"}, {"cell_type": "markdown", "metadata": {}, "source": "---\n## 9. Citations, Credits & References\n\n### Competition & Data Sources\n\n1. **CAFA-6 Competition**: Kaggle CAFA 6 Protein Function Prediction\n   - https://www.kaggle.com/competitions/cafa-6-protein-function-prediction\n\n2. **GOA Predictions Dataset**: ymuroya47/cafa6-goa-predictions\n   - https://www.kaggle.com/datasets/ymuroya47/cafa6-goa-predictions\n   - Contains `goa_submission.tsv` and `prott5_interpro_predictions.tsv`\n\n3. **Gene Ontology Consortium**\n   - Ashburner, M., et al. (2000). Gene Ontology: tool for the unification of biology. *Nature Genetics*, 25(1), 25-29.\n   - http://geneontology.org/\n\n4. **UniProt-GOA Database**\n   - Huntley, R. P., et al. (2015). The GOA database: gene ontology annotation updates for 2015. *Nucleic Acids Research*, 43(D1), D1057-D1063.\n   - https://www.ebi.ac.uk/GOA\n\n### Protein Language Models\n\n5. **ESM-2** (Meta AI)\n   - Lin, Z., et al. (2023). Evolutionary-scale prediction of atomic-level protein structure with a language model. *Science*, 379(6637), 1123-1130.\n   - https://github.com/facebookresearch/esm\n\n6. **ProtT5** (Rostlab)\n   - Elnaggar, A., et al. (2022). ProtTrans: Toward understanding the language of life through self-supervised learning. *IEEE TPAMI*, 44(10), 7112-7127.\n   - https://github.com/agemagician/ProtTrans\n\n7. **Ankh** (ElnaggarLab)\n   - Elnaggar, A., et al. (2023). Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling.\n   - https://github.com/agemagician/Ankh\n\n### Key Methods & Papers\n\n8. **CAFA Evaluation**\n   - Radivojac, P., et al. (2013). A large-scale evaluation of computational protein function prediction. *Nature Methods*, 10(3), 221-227.\n\n9. **Asymmetric Focal Loss**\n   - Ridnik, T., et al. (2021). Asymmetric loss for multi-label classification. *ICCV 2021*.\n   - https://github.com/Alibaba-MIIL/ASL\n\n10. **DeepGOPlus**\n    - Kulmanov, M., & Hoehndorf, R. (2020). DeepGOPlus: improved protein function prediction from sequence. *Bioinformatics*, 36(2), 422-429.\n\n11. **DIAMOND** (Homology Search)\n    - Buchfink, B., Xie, C., & Huson, D. H. (2015). Fast and sensitive protein alignment using DIAMOND. *Nature Methods*, 12(1), 59-60.\n\n### CAFA-5 Winning Solutions (Referenced)\n\n12. **GOCurator** (1st Place, CAFA-5)\n    - Yuan, Q., et al. (2024). GORetriever: A Two-Stage Retrieval Framework for Gene Ontology Annotation.\n    - https://pmc.ncbi.nlm.nih.gov/articles/PMC11520413/\n\n13. **ProtBoost** (2nd Place, CAFA-5)\n    - Yuan, Q., et al. (2024). ProtBoost: Boosted protein function prediction.\n    - https://arxiv.org/abs/2412.04529\n\n### Top CAFA-6 Notebooks Referenced\n\n14. **GOA + ProtT5 Ensemble (0.375)**\n    - Ibrahim Qasimi\n    - https://www.kaggle.com/code/ibrahimqasimi/0-375-biological-function-of-a-protein\n\n15. **GOA + ProtT5 Ensemble (0.370)**\n    - Nikita Kuznetsov\n    - https://www.kaggle.com/code/nikitakuznetsof/cafa-6-goa-prott5-ensemble-0-370-f2fcb6\n\n16. **KTDK Final**\n    - Khoa Tran\n    - https://www.kaggle.com/code/khoatran512/ktdk-int3405e2-final\n\n### Tools & Libraries\n\n17. **PyTorch**: https://pytorch.org/\n18. **Hugging Face Transformers**: https://huggingface.co/\n19. **scikit-learn**: https://scikit-learn.org/\n20. **Py-Boost (SketchBoost)**: https://github.com/sb-ai-lab/Py-Boost\n\n### Acknowledgments\n\n- **Meta AI** for ESM-2 protein language models\n- **Rostlab** for ProtT5 models\n- **ElnaggarLab** for Ankh models\n- **Gene Ontology Consortium** for GO ontology\n- **UniProt-GOA** for annotation data"}, {"cell_type": "markdown", "metadata": {}, "source": "---\n## Appendix: Project Statistics\n\n| Metric | Value |\n|--------|-------|\n| Scripts created | 135 |\n| Output files | 119 |\n| Git commits | 50+ |\n| Submissions | 100+ |\n| Best score | 0.378 |\n| Worst score | 0.005 |\n| Time spent | ~3 months |\n| GOA baseline | 0.336 |\n| Improvement over baseline | +12.5% |"}], "metadata": {"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}, "language_info": {"name": "python", "version": "3.10.0"}}, "nbformat": 4, "nbformat_minor": 4}