{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":116062,"databundleVersionId":14084779,"sourceType":"competition"}],"dockerImageVersionId":31153,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"# **FASTA Protein Data: Length and Sequence**","metadata":{}},{"cell_type":"markdown","source":"---\n\n## 🚀 Brief Pipeline Explanation\n\nThis pipeline is designed to perform essential **Exploratory Data Analysis (EDA)** on a large FASTA file, transforming raw sequence data into a structured format suitable for machine learning or detailed statistical study.\n\n### 1. Data Parsing and Length Analysis (Visualization)\n\nThis initial stage uses the **Biopython** library (`SeqIO`) to efficiently parse the raw FASTA file containing over 82,000 protein sequences.\n* **Length Extraction:** The length of every sequence is computed and stored.\n* **Statistical Summary:** Basic statistics (`mean`, `min`, `max`, `quartiles`) are generated using Pandas `describe()` to quickly understand the size range of the proteins.\n* **Distribution Visualization:** **Histograms** are plotted to visualize the highly **right-skewed** distribution of sequence lengths (since most proteins are short, but a few are extremely long). A second, filtered plot focuses on the main body of the distribution (e.g., sequences under the 99th percentile) for clearer insight.\n\n### 2. Metadata Structuring and ID Parsing\n\nThe second stage structures the sequence metadata into a clean Pandas DataFrame, a process often critical for aligning protein data with annotation files.\n* **EntryID Splitting:** The pipeline iterates through all entries, taking the **Full EntryID** (e.g., `sp|A0A0C5B5G6|MOTSC_HUMAN`) and splitting it by the pipe (`|`) delimiter into three informative columns:\n    * **Accession\\_Type:** (e.g., `sp` for SwissProt/UniProtKB reviewed entry).\n    * **Accession\\_ID:** The unique identifier (e.g., `A0A0C5B5G6`).\n    * **Protein\\_Name:** A short descriptive name (e.g., `MOTSC_HUMAN`).\n* **Final DataFrame:** The output is a comprehensive DataFrame containing the parsed IDs, the sequence length, and a preview of the sequence, preparing the data for feature engineering or linkage with functional annotation datasets.\n\nWould you like to proceed with using the generated DataFrame (`df`) for further analysis, such as investigating the distribution of `Accession_Type` or grouping by species?","metadata":{}},{"cell_type":"code","source":"!pip install biopython","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-17T13:48:50.415507Z","iopub.execute_input":"2025-10-17T13:48:50.415831Z","iopub.status.idle":"2025-10-17T13:48:54.329861Z","shell.execute_reply.started":"2025-10-17T13:48:50.415808Z","shell.execute_reply":"2025-10-17T13:48:54.328826Z"},"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom Bio import SeqIO\nimport os\n\n# -----------------------------------------------------------\n# 1. Data Loading and Length Extraction\n# -----------------------------------------------------------\n\n# Specify the file path (assuming the CAFA 6 dataset structure)\nfile_path = '/kaggle/input/cafa-6-protein-function-prediction/Train/train_sequences.fasta'\n\n# Check if the file exists\nif not os.path.exists(file_path):\n    print(f\"Error: File not found at {file_path}\")\nelse:\n    # Use SeqIO to parse the FASTA file\n    sequences = list(SeqIO.parse(file_path, 'fasta'))\n    n_sequences = len(sequences)\n    print(f\"✅ Total number of sequences loaded: {n_sequences}\")\n\n    # Extract the length of all sequences into a list\n    lengths = [len(record.seq) for record in sequences]\n\n    # Create a Pandas DataFrame\n    df_lengths = pd.DataFrame(lengths, columns=['Length'])\n\n    # -----------------------------------------------------------\n    # 2. Display Statistical Summary\n    # -----------------------------------------------------------\n\n    print(\"\\n--- Statistical Summary (Sequence Length Statistics) ---\")\n    # Display stats formatted as integers for readability\n    print(df_lengths['Length'].describe().apply(lambda x: f'{x:,.0f}'))\n\n    # -----------------------------------------------------------\n    # 3. Visualization of Distribution\n    # -----------------------------------------------------------\n\n    # Since sequence length distributions are often heavily skewed by long outliers,\n    # we'll create two plots for better visualization.\n    \n    plt.style.use('ggplot')\n    \n    fig, axes = plt.subplots(2, 1, figsize=(12, 10))\n    \n    # === Plot 1: Histogram of Full Range ===\n    \n    # Check the overall shape and the presence of extreme outliers\n    sns.histplot(df_lengths['Length'], bins=100, kde=True, ax=axes[0], color='skyblue')\n    axes[0].set_title('① Distribution of All Sequence Lengths (Full Range)', fontsize=14)\n    axes[0].set_xlabel('Sequence Length (Amino Acids)', fontsize=12)\n    axes[0].set_ylabel('Frequency (Count)', fontsize=12)\n\n    # === Plot 2: Histogram Focusing on the Main Distribution ===\n    \n    # Focus on the most concentrated region (e.g., sequences shorter than the 99th percentile)\n    # The actual cutoff might need adjustment based on the data.\n    max_visible_length = df_lengths['Length'].quantile(0.99) \n    df_filtered = df_lengths[df_lengths['Length'] <= max_visible_length]\n\n    sns.histplot(df_filtered['Length'], bins=50, kde=True, ax=axes[1], color='coral')\n    axes[1].set_title(f'② Distribution of Sequence Lengths (Focus on Length <= {int(max_visible_length):,} AA)', fontsize=14)\n    axes[1].set_xlabel('Sequence Length (Amino Acids)', fontsize=12)\n    axes[1].set_ylabel('Frequency (Count)', fontsize=12)\n    \n    plt.tight_layout()\n    plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-17T13:48:55.318798Z","iopub.execute_input":"2025-10-17T13:48:55.319438Z","iopub.status.idle":"2025-10-17T13:48:57.724463Z","shell.execute_reply.started":"2025-10-17T13:48:55.319409Z","shell.execute_reply":"2025-10-17T13:48:57.723511Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\n\nif 'sequences' not in locals() or not sequences:\n    print(\"Error: The 'sequences' list is not found or is empty. Please ensure the FASTA file loading was successful.\")\nelse:\n    # 1. Data Extraction\n    data = []\n    for record in sequences:\n        # Split the EntryID by the '|' (pipe) delimiter\n        parts = record.id.split('|')\n        \n        # Expected format: ['sp', 'A0A0C5B5G6', 'MOTSC_HUMAN'] (3 elements)\n        if len(parts) == 3:\n            accession_type = parts[0]\n            accession_id = parts[1]\n            protein_name = parts[2]\n        else:\n            # Fallback for unexpected ID formats (e.g., store the full ID)\n            accession_type = 'N/A'\n            accession_id = record.id\n            protein_name = 'N/A'\n\n        # Append the row data to the list\n        data.append({\n            'Full_EntryID': record.id,\n            'Length': len(record.seq),\n            'Accession_Type': accession_type,\n            'Accession_ID': accession_id,\n            'Protein_Name': protein_name,\n            'Sequence': str(record.seq)[0:32] # The sequence itself is usually omitted due to its length\n        })\n\n    # 2. DataFrame Creation\n    df = pd.DataFrame(data)\n\n    # 3. Display Results\n    print(\"✅ DataFrame successfully created.\")\n    print(\"\\n--- First 5 rows of the DataFrame ---\")\n    print(df.head())\n    print(\"\\n--- DataFrame Information ---\")\n    print(df.info())\n\n    df.to_csv('fasta.csv',index=False)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-10-17T13:48:57.726188Z","iopub.execute_input":"2025-10-17T13:48:57.726459Z","iopub.status.idle":"2025-10-17T13:48:58.083806Z","shell.execute_reply.started":"2025-10-17T13:48:57.726440Z","shell.execute_reply":"2025-10-17T13:48:58.083022Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}