{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":116062,"databundleVersionId":14084779,"sourceType":"competition"}],"dockerImageVersionId":31192,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🧬 CAFA 6: TF-IDF + Keras MLP [Simple Baseline]\n\n## 👋 Introduction\nWelcome to the **CAFA 6: Protein Function Prediction** competition! \n\nThis notebook serves as a **minimalist starter kit** (baseline) for beginners. Instead of using complex protein language models (like ESM or ProtBERT) which require heavy GPUs, we treat protein sequences as **text** and use standard Natural Language Processing (NLP) techniques.\n\n### 🚀 The Strategy\n1.  **Input:** Amino acid sequences (treated as characters).\n2.  **Features:** **TF-IDF** with character n-grams (3-5 chars) to capture small protein motifs (k-mers).\n3.  **Target:** We focus on the **Top 10 most frequent GO Terms** to create a super-fast, lightweight training loop.\n4.  **Model:** A simple **Deep Neural Network (MLP)** using TensorFlow/Keras.\n\n### 🌟 Why use this?\n* **Fast:** Trains in minutes.\n* **Low Memory:** Runs easily on CPU or low-tier GPU.\n* **Clean Code:** Easy to understand and extend.\n\n> **If you find this notebook helpful for starting your journey, please UPVOTE! 🔼 It keeps me motivated to share more.** Let's get started! 👇\n---","metadata":{}},{"cell_type":"code","source":"!pip install Bio","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-20T10:41:08.017861Z","iopub.execute_input":"2025-11-20T10:41:08.01824Z","iopub.status.idle":"2025-11-20T10:41:12.125599Z","shell.execute_reply.started":"2025-11-20T10:41:08.018211Z","shell.execute_reply":"2025-11-20T10:41:12.124372Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# Imports & Config","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport tensorflow as tf\nimport matplotlib.pyplot as plt\nfrom Bio import SeqIO\nfrom sklearn.preprocessing import MultiLabelBinarizer\nfrom sklearn.feature_extraction.text import TfidfVectorizer\nfrom sklearn.model_selection import train_test_split\n\n# --- CONFIGURATION ---\n# Adjust these paths based on the specific CAFA 6 directory structure\nTRAIN_SEQ_PATH = \"/kaggle/input/cafa-6-protein-function-prediction/Train/train_sequences.fasta\"\nTRAIN_TERMS_PATH = \"/kaggle/input/cafa-6-protein-function-prediction/Train/train_terms.tsv\"\nTEST_SEQ_PATH = \"/kaggle/input/cafa-6-protein-function-prediction/Test/testsuperset.fasta\" \n\n# Hyperparameters\nTOP_N_LABELS = 10         # STRICTLY TOP 10 for speed/baseline\nMAX_FEATURES = 5000       # Limit TF-IDF features for memory efficiency\nEPOCHS = 5\nBATCH_SIZE = 128\nRANDOM_SEED = 42\n\nprint(\"✅ Libraries loaded and configuration set.\")\nprint(f\"🎯 Target: Predicting the Top {TOP_N_LABELS} most common GO terms.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-20T10:41:12.127903Z","iopub.execute_input":"2025-11-20T10:41:12.128449Z","iopub.status.idle":"2025-11-20T10:41:12.136354Z","shell.execute_reply.started":"2025-11-20T10:41:12.128411Z","shell.execute_reply":"2025-11-20T10:41:12.135116Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# Helper Functions","metadata":{}},{"cell_type":"code","source":"def parse_fasta(path, is_train=True):\n    \"\"\"\n    Parses FASTA files efficiently using Biopython.\n    Standardizes the protein ID extraction.\n    \"\"\"\n    records = []\n    print(f\"📂 Parsing {path}...\")\n    \n    # Parse FASTA file\n    for rec in SeqIO.parse(path, \"fasta\"):\n        header = rec.description.strip()\n        parts = header.split()\n        \n        # Logic to extract Protein ID based on header format\n        # Case 1: Generic/Test format \">P12345 9606\"\n        protein_id = parts[0]\n        \n        # Case 2: SwissProt/Train format \"sp|P12345|...\" (Clean up if necessary)\n        if \"|\" in protein_id:\n            protein_id = protein_id.split(\"|\")[1]\n            \n        records.append({\n            \"protein_id\": protein_id,\n            \"sequence\": str(rec.seq)\n        })\n        \n    return pd.DataFrame(records)\n\nprint(\"✅ Helper functions defined.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-20T10:41:12.137335Z","iopub.execute_input":"2025-11-20T10:41:12.137631Z","iopub.status.idle":"2025-11-20T10:41:12.158174Z","shell.execute_reply.started":"2025-11-20T10:41:12.137601Z","shell.execute_reply":"2025-11-20T10:41:12.157084Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# Data Loading & Preprocessing","metadata":{}},{"cell_type":"code","source":"# 1. Load Sequences\ndf_train_seq = parse_fasta(TRAIN_SEQ_PATH, is_train=True)\ndf_test_seq = parse_fasta(TEST_SEQ_PATH, is_train=False)\n\nprint(f\"📊 Train Sequences: {df_train_seq.shape}\")\nprint(f\"📊 Test Sequences:  {df_test_seq.shape}\")\n\n# 2. Load Labels (GO Terms)\nprint(\"\\n📥 Loading Train Terms...\")\ntrain_terms = pd.read_csv(TRAIN_TERMS_PATH, sep=\"\\t\")\n\n# Rename columns to match our sequence dataframe\n# (Adjust column names if CAFA 6 uses different headers like 'protein_accession')\ntrain_terms.columns = [\"protein_id\", \"go_term\", \"aspect\"] if train_terms.shape[1] == 3 else [\"protein_id\", \"go_term\"]\n\n# 3. FILTER: Keep only Top 10 Labels\nprint(f\"\\n✂️ Filtering to Top {TOP_N_LABELS} GO Terms...\")\ntop_terms = train_terms[\"go_term\"].value_counts().head(TOP_N_LABELS).index.tolist()\ntrain_terms_filtered = train_terms[train_terms[\"go_term\"].isin(top_terms)]\n\n# 4. Merge Sequences with Labels\n# Group terms per protein: Protein A -> [GO:001, GO:002]\ndf_group = train_terms_filtered.groupby(\"protein_id\")[\"go_term\"].apply(list).reset_index()\ndf_merge = df_train_seq.merge(df_group, on=\"protein_id\", how=\"inner\")\n\nprint(f\"✅ Final Training Data Shape: {df_merge.shape}\")\ndisplay(df_merge.head())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-20T10:41:12.159323Z","iopub.execute_input":"2025-11-20T10:41:12.159624Z","iopub.status.idle":"2025-11-20T10:41:16.60608Z","shell.execute_reply.started":"2025-11-20T10:41:12.159601Z","shell.execute_reply":"2025-11-20T10:41:16.604855Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# Feature Engineering","metadata":{}},{"cell_type":"code","source":"# --- 1. Label Binarization (Y) ---\nmlb = MultiLabelBinarizer()\nY = mlb.fit_transform(df_merge[\"go_term\"])\n\nprint(f\"Y Shape: {Y.shape}\")\nprint(\"Classes (Top 10):\", mlb.classes_)\n\n# --- 2. TF-IDF Vectorization (X) ---\n# We use character-level n-grams.\n# Example: \"MKA...\" -> \"MKA\", \"KAL\", \"AL...\"\nprint(\"\\n🔠 Vectorizing Sequences (TF-IDF)...\")\n\ntfidf = TfidfVectorizer(\n    analyzer=\"char\",\n    ngram_range=(3, 5), # Look for patterns of length 3 to 5 amino acids\n    max_features=MAX_FEATURES,\n    dtype=np.float32\n)\n\nX = tfidf.fit_transform(df_merge[\"sequence\"])\nprint(f\"X Shape: {X.shape}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-20T10:41:16.608336Z","iopub.execute_input":"2025-11-20T10:41:16.608687Z","iopub.status.idle":"2025-11-20T10:43:09.692691Z","shell.execute_reply.started":"2025-11-20T10:41:16.608665Z","shell.execute_reply":"2025-11-20T10:43:09.69145Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# Model Building","metadata":{}},{"cell_type":"code","source":"def build_simple_mlp(input_dim, output_dim):\n    \"\"\"\n    Creates a simple Multi-Layer Perceptron.\n    \"\"\"\n    model = tf.keras.Sequential([\n        # Input Layer (Sparse handling)\n        tf.keras.layers.Input(shape=(input_dim,), sparse=True),\n        \n        # Hidden Layer 1\n        tf.keras.layers.Dense(512, activation=\"relu\"),\n        tf.keras.layers.Dropout(0.3), # Prevent overfitting\n        \n        # Hidden Layer 2\n        tf.keras.layers.Dense(256, activation=\"relu\"),\n        tf.keras.layers.Dropout(0.2),\n        \n        # Output Layer (Sigmoid for Multi-label classification)\n        tf.keras.layers.Dense(output_dim, activation=\"sigmoid\")\n    ])\n    \n    model.compile(\n        optimizer=\"adam\",\n        loss=\"binary_crossentropy\",\n        metrics=[\"binary_accuracy\", tf.keras.metrics.AUC(multi_label=True, name='auc')]\n    )\n    return model\n\nmodel = build_simple_mlp(X.shape[1], Y.shape[1])\nmodel.summary()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-20T10:43:09.693854Z","iopub.execute_input":"2025-11-20T10:43:09.694179Z","iopub.status.idle":"2025-11-20T10:43:09.974229Z","shell.execute_reply.started":"2025-11-20T10:43:09.694149Z","shell.execute_reply":"2025-11-20T10:43:09.972948Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n# Training","metadata":{}},{"cell_type":"code","source":"print(\"🏋️‍♂️ Starting Training...\")\n\nhistory = model.fit(\n    X, Y,\n    epochs=EPOCHS,\n    batch_size=BATCH_SIZE,\n    validation_split=0.15, # Keep 15% for validation\n    verbose=1\n)\n\n# Plotting Results\nplt.figure(figsize=(12, 5))\n\nplt.subplot(1, 2, 1)\nplt.plot(history.history['loss'], label='Train Loss')\nplt.plot(history.history['val_loss'], label='Val Loss')\nplt.title('Loss Curve')\nplt.legend()\n\nplt.subplot(1, 2, 2)\nplt.plot(history.history['auc'], label='Train AUC')\nplt.plot(history.history['val_auc'], label='Val AUC')\nplt.title('AUC Curve')\nplt.legend()\n\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-20T10:43:09.975215Z","iopub.execute_input":"2025-11-20T10:43:09.97557Z","iopub.status.idle":"2025-11-20T10:48:17.970864Z","shell.execute_reply.started":"2025-11-20T10:43:09.975549Z","shell.execute_reply":"2025-11-20T10:48:17.969577Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 📊 Analysis of Training Results\n\nLet's interpret the graphs above to understand our baseline model's performance:\n\n### 1. The Loss Curve (Left)\n* **Observation:** The **Training Loss (Blue)** decreases steadily, meaning the model is learning the training data well. However, the **Validation Loss (Orange)** drops initially but starts to **increase** after Epoch 1.\n* **Diagnosis:** This is a classic sign of **Overfitting**. The model is starting to \"memorize\" the training data instead of generalizing patterns. It performs well on data it has seen, but struggles with new, unseen sequences.\n\n### 2. The AUC Curve (Right)\n* **Observation:** There is a significant gap between **Training AUC** (reaching ~0.88) and **Validation AUC** (stagnating around ~0.73).\n* **Takeaway:** The model has the capacity to learn (high Train AUC), but the current architecture or feature set needs better regularization to generalize to the test set.\n\n### 💡 How to Improve (Next Steps)\nTo fix this overfitting and improve the score, you can try:\n1.  **Early Stopping:** Stop training at **Epoch 1 or 2** (where Validation Loss is lowest).\n2.  **More Dropout:** Increase the dropout rate in the Keras model layers (e.g., from 0.3 to 0.5).\n3.  **Reduce Complexity:** Reduce the number of neurons in the Dense layers or reduce `MAX_FEATURES` in TF-IDF.","metadata":{}},{"cell_type":"markdown","source":"---\n# Submission Generation\n","metadata":{}},{"cell_type":"code","source":"print(\"🔮 Generating Predictions...\")\n\n# 1. Vectorize Test Data\nX_test = tfidf.transform(df_test_seq[\"sequence\"])\n\n# 2. Predict Probabilities\npreds = model.predict(X_test, batch_size=BATCH_SIZE)\n\n# 3. Format for Submission\n# Structure: Protein_ID | GO_Term | Score\nsubmission_data = []\ntest_proteins = df_test_seq[\"protein_id\"].values\nclasses = mlb.classes_\n\n# Iterate efficiently\nprint(\"📝 Formatting CSV...\")\nfor i in range(len(test_proteins)):\n    protein = test_proteins[i]\n    scores = preds[i]\n    \n    # Since we only have 10 classes, we can save all of them\n    for j, score in enumerate(scores):\n        submission_data.append([protein, classes[j], f\"{score:.3f}\"])\n\n# Create DataFrame\ndf_sub = pd.DataFrame(submission_data, columns=[\"protein_id\", \"go_term\", \"prediction\"])\n\n# Save\ndf_sub.to_csv(\"submission.tsv\", sep=\"\\t\", index=False, header=False) \nprint(\"✅ 'submission.tsv' saved successfully!\")\n\ndisplay(df_sub.head())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-11-20T10:48:17.971902Z","iopub.execute_input":"2025-11-20T10:48:17.97225Z","iopub.status.idle":"2025-11-20T10:53:11.582495Z","shell.execute_reply.started":"2025-11-20T10:48:17.97222Z","shell.execute_reply":"2025-11-20T10:53:11.581633Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n## 🔚 Conclusion & Next Steps\n\nWe successfully created a baseline model using **TF-IDF** and a simple **Neural Network**, predicting only the **Top 10 GO Terms**.\n\n### 💡 Ideas to Improve (Try these!)\n1.  **Increase `TOP_N_LABELS`**: Change the variable from 10 to 100 or 500 to predict more functions.\n2.  **Tune `ngram_range`**: Try `(1, 3)` or `(2, 4)` in the TF-IDF vectorizer.\n3.  **Use Taxonomy**: Utilize the Tax ID to train separate models for bacteria vs. eukaryotes.\n4.  **Embeddings**: Try extracting features using `ESM2` (Facebook Research) for state-of-the-art results.\n\n**If this notebook helped you get started, please UPVOTE! 🔼 Happy Kaggling!**","metadata":{}}]}