{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceType":"competition","sourceId":121150,"databundleVersionId":15833391},{"sourceType":"datasetVersion","sourceId":14808863,"datasetId":9469498,"databundleVersionId":15664641}],"dockerImageVersionId":31287,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Deep Past Baseline v2 (Readable Standard Notebook)\n\nObjective: keep a reliable end-to-end Kaggle pipeline while making every step easy to understand and audit.\n\nThis notebook now focuses on:\n- clear sectioning,\n- deterministic behavior,\n- explicit guardrails,\n- beginner-friendly explanations.\n","metadata":{}},{"cell_type":"markdown","source":"## Workflow Overview\n\n1. Setup and reproducibility (imports, seed, device).\n2. Centralized configuration (paths, model, training, inference).\n3. Data loading and schema detection with safety checks.\n4. Text preprocessing and test input preparation.\n5. Offline-safe model loading.\n6. Optional fine-tuning, local validation, and test inference.\n7. Submission and run-summary artifact generation.\n8. Optional visual diagnostics.\n","metadata":{}},{"cell_type":"markdown","source":"## 1) Import Core Dependencies\n\nThis cell imports the libraries used throughout the notebook.\n\nWhy this matters:\n- Keeping imports in one place makes dependency issues obvious.\n- It clarifies which packages are required for training/inference.\n","metadata":{}},{"cell_type":"code","source":"from __future__ import annotations\n\nimport json\nimport math\nimport os\nimport random\nimport re\nfrom datetime import datetime, timezone\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nimport torch\nfrom torch.utils.data import DataLoader, Dataset\nfrom transformers import AutoModelForSeq2SeqLM, AutoTokenizer, get_linear_schedule_with_warmup\nfrom tqdm.auto import tqdm\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:42:21.546239Z","iopub.execute_input":"2026-02-27T14:42:21.546443Z","iopub.status.idle":"2026-02-27T14:42:41.580473Z","shell.execute_reply.started":"2026-02-27T14:42:21.546423Z","shell.execute_reply":"2026-02-27T14:42:41.579680Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 2) Set Seed and Choose Device\n\nThis cell enforces deterministic randomness where possible and picks CPU/GPU.\n\nWhy this matters:\n- A fixed seed makes debugging and experiment comparison more reliable.\n- Explicit device selection avoids silent CPU/GPU mismatch.\n","metadata":{}},{"cell_type":"code","source":"# Fixed seed for reproducibility across Python, NumPy, and PyTorch.\nSEED = 42\nrandom.seed(SEED)\nnp.random.seed(SEED)\ntorch.manual_seed(SEED)\nif torch.cuda.is_available():\n    torch.cuda.manual_seed_all(SEED)\n\n# Use GPU when available for faster fine-tuning/inference.\nDEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nprint(\"Device:\", DEVICE)\nprint(\"Seed:\", SEED)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:43:42.862826Z","iopub.execute_input":"2026-02-27T14:43:42.863541Z","iopub.status.idle":"2026-02-27T14:43:42.874628Z","shell.execute_reply.started":"2026-02-27T14:43:42.863510Z","shell.execute_reply":"2026-02-27T14:43:42.874020Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 3) Configure Data and Model Locations\n\nThis cell defines where data and model files come from.\n\nWhy this matters:\n- Kaggle code competitions usually run without internet.\n- `MODEL_DIR` lets us load a locally attached model snapshot safely.\n","metadata":{}},{"cell_type":"code","source":"# Data and model source configuration.\nDATA_DIR = os.getenv(\"DPI_DATA_DIR\", \"/kaggle/input/deep-past-initiative-machine-translation\")\nMODEL_NAME = os.getenv(\"DPI_MODEL_NAME\", \"google/byt5-small\")\nMODEL_DIR = os.getenv(\"DPI_MODEL_DIR\", \"/kaggle/input/datasets/alexanterkapai/byt5-small\")  # Optional local snapshot.\nLOCAL_FILES_ONLY = os.getenv(\"DPI_LOCAL_FILES_ONLY\", \"1\") == \"1\"\n\n# Translation prompt prefix (task instruction to the seq2seq model).\nPREFIX = os.getenv(\"DPI_PREFIX\", \"translate Akkadian to English: \")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:43:46.430311Z","iopub.execute_input":"2026-02-27T14:43:46.430870Z","iopub.status.idle":"2026-02-27T14:43:46.434787Z","shell.execute_reply.started":"2026-02-27T14:43:46.430840Z","shell.execute_reply":"2026-02-27T14:43:46.434101Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 4) Configure Inference Behavior\n\nThese controls affect generation quality, speed, and output style.\n\nWhy this matters:\n- Beam size and decoding penalties can move score significantly.\n- Keeping them centralized makes ablations easier.\n","metadata":{}},{"cell_type":"code","source":"# Inference/generation controls.\nBATCH_SIZE = int(os.getenv(\"DPI_BATCH_SIZE\", \"8\"))\nMAX_SOURCE_LEN = int(os.getenv(\"DPI_MAX_SOURCE_LEN\", \"256\"))\nMAX_TARGET_LEN = int(os.getenv(\"DPI_MAX_TARGET_LEN\", \"256\"))\nMAX_NEW_TOKENS = int(os.getenv(\"DPI_MAX_NEW_TOKENS\", \"96\"))\nNUM_BEAMS = int(os.getenv(\"DPI_NUM_BEAMS\", \"6\"))\nNO_REPEAT_NGRAM_SIZE = int(os.getenv(\"DPI_NO_REPEAT_NGRAM_SIZE\", \"4\"))\nREPETITION_PENALTY = float(os.getenv(\"DPI_REPETITION_PENALTY\", \"1.2\"))\nLENGTH_PENALTY = float(os.getenv(\"DPI_LENGTH_PENALTY\", \"1.0\"))\nEARLY_STOPPING = os.getenv(\"DPI_EARLY_STOPPING\", \"1\") == \"1\"\nNUM_WORKERS = int(os.getenv(\"DPI_NUM_WORKERS\", \"0\"))\n\n# Quick local debug option: set >0 to run only first N test rows.\nLIMIT = int(os.getenv(\"DPI_LIMIT\", \"0\"))\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:43:52.270777Z","iopub.execute_input":"2026-02-27T14:43:52.271075Z","iopub.status.idle":"2026-02-27T14:43:52.276156Z","shell.execute_reply.started":"2026-02-27T14:43:52.271048Z","shell.execute_reply":"2026-02-27T14:43:52.275361Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 5) Configure Validation and Fine-Tuning\n\nThese controls decide whether we train, how we split validation, and optimization behavior.\n\nWhy this matters:\n- Deterministic holdout avoids leakage during local checks.\n- Training settings are explicit for reproducible experiments.\n","metadata":{}},{"cell_type":"code","source":"# Local validation controls.\nRUN_LOCAL_VAL = os.getenv(\"DPI_RUN_LOCAL_VAL\", \"1\") == \"1\"\nVAL_SAMPLES = int(os.getenv(\"DPI_VAL_SAMPLES\", \"300\"))\nVAL_SPLIT = float(os.getenv(\"DPI_VAL_SPLIT\", \"0.1\"))\n\n# Fine-tuning controls.\nDO_TRAIN = os.getenv(\"DPI_DO_TRAIN\", \"1\") == \"1\"\nTRAIN_EPOCHS = int(os.getenv(\"DPI_TRAIN_EPOCHS\", \"3\"))\nTRAIN_BATCH_SIZE = int(os.getenv(\"DPI_TRAIN_BATCH_SIZE\", \"4\"))\nTRAIN_GRAD_ACCUM = int(os.getenv(\"DPI_TRAIN_GRAD_ACCUM\", \"2\"))\nTRAIN_LR = float(os.getenv(\"DPI_TRAIN_LR\", \"1e-5\"))\nTRAIN_WEIGHT_DECAY = float(os.getenv(\"DPI_TRAIN_WEIGHT_DECAY\", \"0.01\"))\nTRAIN_WARMUP_RATIO = float(os.getenv(\"DPI_TRAIN_WARMUP_RATIO\", \"0.1\"))\nTRAIN_MAX_UPDATES = int(os.getenv(\"DPI_TRAIN_MAX_UPDATES\", \"0\"))  # 0 => no cap.\nTRAIN_USE_AMP = os.getenv(\"DPI_TRAIN_USE_AMP\", \"0\") == \"1\"  # Off by default for stability.\nTRAIN_SAMPLES = int(os.getenv(\"DPI_TRAIN_SAMPLES\", \"0\"))  # 0 => full fit split.\nTRAIN_SAVE_DIR = os.getenv(\"DPI_TRAIN_SAVE_DIR\", \"/kaggle/working/byt5_finetuned\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:44:13.004585Z","iopub.execute_input":"2026-02-27T14:44:13.005170Z","iopub.status.idle":"2026-02-27T14:44:13.010639Z","shell.execute_reply.started":"2026-02-27T14:44:13.005138Z","shell.execute_reply":"2026-02-27T14:44:13.010006Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 6) Configure Artifact Tracking and Print Final Config\n\nThis cell defines output metadata and prints the full run configuration.\n\nWhy this matters:\n- A printed config acts like a lightweight experiment log.\n- It helps diagnose accidental env-var mismatches.\n","metadata":{}},{"cell_type":"code","source":"# Experiment and artifact metadata.\nEXPERIMENT_TAG = os.getenv(\"DPI_EXPERIMENT_TAG\", \"ft_continue_e3_lr1e5_b6_nr4_rp12_lp10\")\nOUTPUT_DIR = os.getenv(\"DPI_OUTPUT_DIR\", \"/kaggle/working\")\n\ncfg_snapshot = {\n    \"DATA_DIR\": DATA_DIR,\n    \"MODEL_NAME\": MODEL_NAME,\n    \"MODEL_DIR\": MODEL_DIR or None,\n    \"LOCAL_FILES_ONLY\": LOCAL_FILES_ONLY,\n    \"PREFIX\": PREFIX,\n    \"DO_TRAIN\": DO_TRAIN,\n    \"TRAIN_EPOCHS\": TRAIN_EPOCHS,\n    \"TRAIN_BATCH_SIZE\": TRAIN_BATCH_SIZE,\n    \"TRAIN_GRAD_ACCUM\": TRAIN_GRAD_ACCUM,\n    \"TRAIN_LR\": TRAIN_LR,\n    \"TRAIN_WEIGHT_DECAY\": TRAIN_WEIGHT_DECAY,\n    \"TRAIN_WARMUP_RATIO\": TRAIN_WARMUP_RATIO,\n    \"TRAIN_MAX_UPDATES\": TRAIN_MAX_UPDATES,\n    \"TRAIN_USE_AMP\": TRAIN_USE_AMP,\n    \"TRAIN_SAMPLES\": TRAIN_SAMPLES,\n    \"TRAIN_SAVE_DIR\": TRAIN_SAVE_DIR,\n    \"BATCH_SIZE\": BATCH_SIZE,\n    \"MAX_SOURCE_LEN\": MAX_SOURCE_LEN,\n    \"MAX_TARGET_LEN\": MAX_TARGET_LEN,\n    \"MAX_NEW_TOKENS\": MAX_NEW_TOKENS,\n    \"NUM_BEAMS\": NUM_BEAMS,\n    \"NO_REPEAT_NGRAM_SIZE\": NO_REPEAT_NGRAM_SIZE,\n    \"REPETITION_PENALTY\": REPETITION_PENALTY,\n    \"LENGTH_PENALTY\": LENGTH_PENALTY,\n    \"EARLY_STOPPING\": EARLY_STOPPING,\n    \"NUM_WORKERS\": NUM_WORKERS,\n    \"LIMIT\": LIMIT,\n    \"RUN_LOCAL_VAL\": RUN_LOCAL_VAL,\n    \"VAL_SAMPLES\": VAL_SAMPLES,\n    \"VAL_SPLIT\": VAL_SPLIT,\n    \"EXPERIMENT_TAG\": EXPERIMENT_TAG,\n    \"OUTPUT_DIR\": OUTPUT_DIR,\n}\nprint(cfg_snapshot)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:44:27.298834Z","iopub.execute_input":"2026-02-27T14:44:27.299574Z","iopub.status.idle":"2026-02-27T14:44:27.305310Z","shell.execute_reply.started":"2026-02-27T14:44:27.299541Z","shell.execute_reply":"2026-02-27T14:44:27.304628Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 7) Resolve Dataset Paths and Load CSV Files\n\nThis cell reads the competition files and does immediate shape/column checks.\n\nWhy this matters:\n- Early file checks fail fast if dataset mounting is wrong.\n- Shape and column logs help catch schema drift.\n","metadata":{}},{"cell_type":"code","source":"# Resolve expected file paths under the configured data directory.\nDATA_DIR = Path(DATA_DIR)\ntrain_path = DATA_DIR / \"train.csv\"\ntest_path = DATA_DIR / \"test.csv\"\n\nif not test_path.exists():\n    raise FileNotFoundError(f\"test.csv not found at {test_path}\")\n\n# Train may be absent in some inference-only contexts.\ntrain_df = pd.read_csv(train_path) if train_path.exists() else None\ntest_df = pd.read_csv(test_path)\n\nprint(\"train_df:\", None if train_df is None else train_df.shape)\nprint(\"test_df:\", test_df.shape)\nprint(\"test columns:\", list(test_df.columns))\nif train_df is not None:\n    print(\"train columns:\", list(train_df.columns))\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:44:43.115606Z","iopub.execute_input":"2026-02-27T14:44:43.116224Z","iopub.status.idle":"2026-02-27T14:44:43.189566Z","shell.execute_reply.started":"2026-02-27T14:44:43.116194Z","shell.execute_reply":"2026-02-27T14:44:43.188969Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 8) Define Schema Detection Helpers\n\nThis cell creates a utility for robust column detection using candidate names.\n\nWhy this matters:\n- Competition datasets can rename columns (`id` vs `ID`, etc.).\n- We avoid hardcoding a single schema and failing silently.\n","metadata":{}},{"cell_type":"code","source":"def pick_first_existing(columns, candidates, field_name):\n    \"\"\"Return the first candidate column name that exists, or raise with context.\"\"\"\n    for c in candidates:\n        if c in columns:\n            return c\n    raise KeyError(\n        f\"Could not find {field_name}. Expected one of {candidates}. Found columns: {list(columns)}\"\n    )\n\n\n# Candidate column names observed across dataset variants.\nID_CANDIDATES = [\"id\", \"ID\", \"sample_id\"]\nSOURCE_CANDIDATES = [\"transliteration\", \"source\", \"text\", \"input\"]\nTARGET_CANDIDATES = [\"translation\", \"target\", \"label\", \"english\"]\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:45:45.699218Z","iopub.execute_input":"2026-02-27T14:45:45.699780Z","iopub.status.idle":"2026-02-27T14:45:45.704565Z","shell.execute_reply.started":"2026-02-27T14:45:45.699753Z","shell.execute_reply":"2026-02-27T14:45:45.703919Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 9) Detect Required Columns\n\nThis cell applies the helper to infer id/source/target columns for train and test.\n\nWhy this matters:\n- We validate schema before expensive model loading.\n- Explicit logs reduce debugging time during competition runs.\n","metadata":{}},{"cell_type":"code","source":"# Detect required columns for inference and optional training.\nid_col = pick_first_existing(test_df.columns, ID_CANDIDATES, \"id column\")\nsource_col_test = pick_first_existing(test_df.columns, SOURCE_CANDIDATES, \"source text column in test\")\n\nsource_col_train = None\ntarget_col_train = None\nif train_df is not None:\n    source_col_train = pick_first_existing(train_df.columns, SOURCE_CANDIDATES, \"source text column in train\")\n    target_col_train = pick_first_existing(train_df.columns, TARGET_CANDIDATES, \"target text column in train\")\n\nprint(\"Detected columns ->\")\nprint(\"  id_col:\", id_col)\nprint(\"  source_col_test:\", source_col_test)\nprint(\"  source_col_train:\", source_col_train)\nprint(\"  target_col_train:\", target_col_train)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:45:49.814495Z","iopub.execute_input":"2026-02-27T14:45:49.815034Z","iopub.status.idle":"2026-02-27T14:45:49.825198Z","shell.execute_reply.started":"2026-02-27T14:45:49.815004Z","shell.execute_reply":"2026-02-27T14:45:49.824335Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 10) Define Lightweight Text Cleaning\n\nThis cell standardizes text cleanup for both inputs and predictions.\n\nWhy this matters:\n- Normalized whitespace improves tokenizer consistency.\n- Minimal preprocessing avoids over-engineering and leakage risk.\n","metadata":{}},{"cell_type":"code","source":"WHITESPACE_RE = re.compile(r\"\\s+\")\n\n\ndef clean_text(x):\n    \"\"\"Normalize raw text by removing tab/newline noise and collapsing whitespace.\"\"\"\n    if pd.isna(x):\n        return \"\"\n    s = str(x)\n    s = s.replace(\"\\n\", \" \").replace(\"\\t\", \" \")\n    return WHITESPACE_RE.sub(\" \", s).strip()\n\n\ndef postprocess_prediction(x):\n    \"\"\"Apply the same whitespace normalization to generated predictions.\"\"\"\n    return WHITESPACE_RE.sub(\" \", str(x)).strip()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:45:53.994882Z","iopub.execute_input":"2026-02-27T14:45:53.995228Z","iopub.status.idle":"2026-02-27T14:45:53.999885Z","shell.execute_reply.started":"2026-02-27T14:45:53.995201Z","shell.execute_reply":"2026-02-27T14:45:53.999228Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 11) Build Test Inputs for Inference\n\nThis cell applies optional row limiting, builds prefixed model inputs, and stores ids.\n\nWhy this matters:\n- `LIMIT` enables fast smoke tests.\n- Prompt prefixing keeps train/inference behavior aligned.\n","metadata":{}},{"cell_type":"code","source":"# Optional truncation for quick debug runs.\nif LIMIT > 0:\n    test_df = test_df.head(LIMIT).copy()\n\n# Keep ids for submission, and build source strings for generation.\ntest_ids = test_df[id_col].tolist()\ntest_inputs = [PREFIX + clean_text(t) for t in test_df[source_col_test].tolist()]\n\nprint(\"Prepared test examples:\", len(test_inputs))\nprint(\"Example input:\", test_inputs[0][:250] if test_inputs else \"<empty>\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:46:00.890274Z","iopub.execute_input":"2026-02-27T14:46:00.890805Z","iopub.status.idle":"2026-02-27T14:46:00.895740Z","shell.execute_reply.started":"2026-02-27T14:46:00.890773Z","shell.execute_reply":"2026-02-27T14:46:00.894991Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 12) Choose Model Reference and Validate Local Path\n\nThis cell decides whether to load from `MODEL_DIR` or model hub name.\n\nWhy this matters:\n- In offline Kaggle runs, model files must exist locally.\n- Failing early here avoids confusing downstream errors.\n","metadata":{}},{"cell_type":"code","source":"# Prefer explicit local snapshot when provided.\nmodel_ref = MODEL_DIR if MODEL_DIR else MODEL_NAME\nif MODEL_DIR:\n    model_dir_path = Path(MODEL_DIR)\n    if not model_dir_path.exists():\n        raise FileNotFoundError(f\"DPI_MODEL_DIR does not exist: {MODEL_DIR}\")\n\nprint(\"Loading model from:\", model_ref)\nprint(\"local_files_only:\", LOCAL_FILES_ONLY)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:46:06.044572Z","iopub.execute_input":"2026-02-27T14:46:06.045197Z","iopub.status.idle":"2026-02-27T14:46:06.051490Z","shell.execute_reply.started":"2026-02-27T14:46:06.045165Z","shell.execute_reply":"2026-02-27T14:46:06.050748Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 13) Define Tokenizer Fallback Loader\n\nThis helper handles a known tokenizer config edge case seen in some exported ByT5 snapshots.\n\nWhy this matters:\n- Competition environments often reuse model snapshots from different tool versions.\n- This fallback avoids run failure from config schema mismatch.\n","metadata":{}},{"cell_type":"code","source":"def load_tokenizer_with_fallback(model_reference):\n    \"\"\"Load tokenizer and retry with a compatibility workaround when needed.\"\"\"\n    try:\n        return AutoTokenizer.from_pretrained(model_reference, local_files_only=LOCAL_FILES_ONLY)\n    except AttributeError as e:\n        if \"'list' object has no attribute 'keys'\" in str(e):\n            print(\n                \"Tokenizer config incompatibility detected (extra_special_tokens list). \"\n                \"Retrying with extra_special_tokens={}\"\n            )\n            return AutoTokenizer.from_pretrained(\n                model_reference,\n                local_files_only=LOCAL_FILES_ONLY,\n                extra_special_tokens={},\n            )\n        raise\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:46:13.163978Z","iopub.execute_input":"2026-02-27T14:46:13.164568Z","iopub.status.idle":"2026-02-27T14:46:13.169213Z","shell.execute_reply.started":"2026-02-27T14:46:13.164538Z","shell.execute_reply":"2026-02-27T14:46:13.168282Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 14) Load Tokenizer and Model, Then Move to Device\n\nThis cell performs actual model loading with a Kaggle-offline-friendly error message.\n\nWhy this matters:\n- If loading fails, the message tells exactly how to fix competition setup.\n- We move model to the selected device once to avoid repeated transfers.\n","metadata":{}},{"cell_type":"code","source":"try:\n    tokenizer = load_tokenizer_with_fallback(model_ref)\n    model = AutoModelForSeq2SeqLM.from_pretrained(model_ref, local_files_only=LOCAL_FILES_ONLY)\nexcept OSError as e:\n    raise RuntimeError(\n        \"Model load failed. In Kaggle code competitions, internet is disabled. \"\n        \"Set DPI_MODEL_DIR to a local model snapshot path added as a Kaggle Dataset, \"\n        \"or set DPI_LOCAL_FILES_ONLY=0 only when internet is allowed.\"\n    ) from e\n\nmodel = model.to(DEVICE)\nmodel.eval()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:46:17.929901Z","iopub.execute_input":"2026-02-27T14:46:17.930244Z","iopub.status.idle":"2026-02-27T14:46:27.136531Z","shell.execute_reply.started":"2026-02-27T14:46:17.930216Z","shell.execute_reply":"2026-02-27T14:46:27.135825Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 15) Define Dataset and Collation Utilities\n\nThis cell defines PyTorch datasets and collate functions for inference and supervised fine-tuning.\n\nWhy this matters:\n- Keeping tokenization in collators avoids duplicated preprocessing logic.\n- Label padding is converted to `-100` so ignored tokens do not affect loss.\n","metadata":{}},{"cell_type":"code","source":"class TextDataset(Dataset):\n    \"\"\"Simple dataset wrapper for a list of source texts used at inference time.\"\"\"\n\n    def __init__(self, texts):\n        self.texts = texts\n\n    def __len__(self):\n        return len(self.texts)\n\n    def __getitem__(self, idx):\n        return self.texts[idx]\n\n\nclass Seq2SeqTrainDataset(Dataset):\n    \"\"\"Dataset wrapper returning (source, target) pairs for supervised training.\"\"\"\n\n    def __init__(self, sources, targets):\n        self.sources = sources\n        self.targets = targets\n\n    def __len__(self):\n        return len(self.sources)\n\n    def __getitem__(self, idx):\n        return self.sources[idx], self.targets[idx]\n\n\ndef collate_fn(batch_texts):\n    \"\"\"Tokenize a batch of source texts for generation.\"\"\"\n    return tokenizer(\n        batch_texts,\n        max_length=MAX_SOURCE_LEN,\n        padding=True,\n        truncation=True,\n        return_tensors=\"pt\",\n    )\n\n\ndef collate_train_fn(batch_pairs):\n    \"\"\"Tokenize source/target pairs and build masked labels for seq2seq training.\"\"\"\n    src_texts = [x[0] for x in batch_pairs]\n    tgt_texts = [x[1] for x in batch_pairs]\n\n    model_inputs = tokenizer(\n        src_texts,\n        max_length=MAX_SOURCE_LEN,\n        padding=True,\n        truncation=True,\n        return_tensors=\"pt\",\n    )\n    labels = tokenizer(\n        text_target=tgt_texts,\n        max_length=MAX_TARGET_LEN,\n        padding=True,\n        truncation=True,\n        return_tensors=\"pt\",\n    )[\"input_ids\"]\n\n    # Ignore padded label positions in the loss.\n    labels[labels == tokenizer.pad_token_id] = -100\n    model_inputs[\"labels\"] = labels\n    return model_inputs\n\n\ndef move_batch_to_device(batch):\n    \"\"\"Move every tensor in a collated batch to the selected compute device.\"\"\"\n    return {k: v.to(DEVICE, non_blocking=True) for k, v in batch.items()}\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:46:40.581123Z","iopub.execute_input":"2026-02-27T14:46:40.581762Z","iopub.status.idle":"2026-02-27T14:46:40.589412Z","shell.execute_reply.started":"2026-02-27T14:46:40.581734Z","shell.execute_reply":"2026-02-27T14:46:40.588713Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 16) Define Pair Cleaning and Generation Helper\n\nThis cell creates reusable helpers for preparing train/val pairs and generating predictions.\n\nWhy this matters:\n- A single generation function ensures consistent decoding settings.\n- Centralizing postprocessing keeps outputs clean and comparable.\n","metadata":{}},{"cell_type":"code","source":"def build_clean_pairs(df, src_col, tgt_col):\n    \"\"\"Create non-empty cleaned (source, target) text pairs from a DataFrame.\"\"\"\n    clean_rows = []\n    for raw_src, raw_tgt in zip(df[src_col].tolist(), df[tgt_col].tolist()):\n        src = clean_text(raw_src)\n        tgt = clean_text(raw_tgt)\n        if src and tgt:\n            clean_rows.append((src, tgt))\n    return clean_rows\n\n\ndef generate_predictions(input_texts, desc=\"Inference\"):\n    \"\"\"Run batched generation and return postprocessed text predictions.\"\"\"\n    if len(input_texts) == 0:\n        return []\n\n    loader = DataLoader(\n        TextDataset(input_texts),\n        batch_size=BATCH_SIZE,\n        shuffle=False,\n        num_workers=NUM_WORKERS,\n        collate_fn=collate_fn,\n        pin_memory=(DEVICE == \"cuda\"),\n    )\n\n    preds = []\n    model.eval()\n    with torch.inference_mode():\n        for enc in tqdm(loader, total=len(loader), desc=desc):\n            enc = move_batch_to_device(enc)\n            generation_kwargs = {\n                \"input_ids\": enc[\"input_ids\"],\n                \"attention_mask\": enc[\"attention_mask\"],\n                \"max_new_tokens\": MAX_NEW_TOKENS,\n                \"num_beams\": NUM_BEAMS,\n                \"do_sample\": False,\n                \"repetition_penalty\": REPETITION_PENALTY,\n                \"length_penalty\": LENGTH_PENALTY,\n                \"early_stopping\": EARLY_STOPPING,\n            }\n            if NO_REPEAT_NGRAM_SIZE > 0:\n                generation_kwargs[\"no_repeat_ngram_size\"] = NO_REPEAT_NGRAM_SIZE\n\n            outputs = model.generate(**generation_kwargs)\n            decoded = tokenizer.batch_decode(outputs, skip_special_tokens=True)\n            preds.extend(postprocess_prediction(x) for x in decoded)\n\n    return preds\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:46:45.069838Z","iopub.execute_input":"2026-02-27T14:46:45.070470Z","iopub.status.idle":"2026-02-27T14:46:45.077471Z","shell.execute_reply.started":"2026-02-27T14:46:45.070440Z","shell.execute_reply":"2026-02-27T14:46:45.076844Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 17) Define Metric and Validation-Loss Helpers\n\nThis cell provides:\n- a competition metric proxy (geometric mean of BLEU and chrF++),\n- a clean validation-loss evaluator.\n\nWhy this matters:\n- Local score tracking gives quick feedback before submission.\n- Validation loss helps monitor overfitting during fine-tuning.\n","metadata":{}},{"cell_type":"code","source":"def compute_competition_metric(references, predictions):\n    \"\"\"Competition metric proxy: geometric mean of corpus BLEU and chrF++.\"\"\"\n    try:\n        import sacrebleu\n    except Exception:\n        print(\"sacrebleu is not installed; skipping local metric check.\")\n        return None\n\n    bleu = sacrebleu.corpus_bleu(predictions, [references]).score\n    chrfpp = sacrebleu.corpus_chrf(predictions, [references], word_order=2).score\n    geom = math.sqrt(max(bleu, 0.0) * max(chrfpp, 0.0))\n    return {\"bleu\": bleu, \"chrfpp\": chrfpp, \"geom_mean\": geom}\n\n\ndef evaluate_val_loss(data_loader):\n    \"\"\"Evaluate average validation loss while skipping non-finite batches.\"\"\"\n    model.eval()\n    total_loss = 0.0\n    total_items = 0\n    skipped_non_finite = 0\n\n    with torch.inference_mode():\n        for batch in data_loader:\n            batch = move_batch_to_device(batch)\n            loss = model(**batch).loss\n            if not torch.isfinite(loss):\n                skipped_non_finite += batch[\"input_ids\"].shape[0]\n                continue\n\n            bs = batch[\"input_ids\"].shape[0]\n            total_loss += loss.item() * bs\n            total_items += bs\n\n    if skipped_non_finite > 0:\n        print(f\"Validation skipped non-finite examples: {skipped_non_finite}\")\n\n    if total_items == 0:\n        return float(\"inf\")\n    return total_loss / total_items\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:46:49.256608Z","iopub.execute_input":"2026-02-27T14:46:49.257120Z","iopub.status.idle":"2026-02-27T14:46:49.263679Z","shell.execute_reply.started":"2026-02-27T14:46:49.257089Z","shell.execute_reply":"2026-02-27T14:46:49.263017Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 18) Create Deterministic Train/Validation Split\n\nThis cell builds a reproducible holdout split from train data and cleans text pairs.\n\nWhy this matters:\n- Deterministic split enables fair experiment comparisons.\n- Cleaning before training avoids empty/noisy rows causing instability.\n","metadata":{}},{"cell_type":"code","source":"# Deterministic split for no-leakage local training/eval.\nfit_pairs = None\nval_pairs = None\nif train_df is not None and source_col_train is not None and target_col_train is not None:\n    all_pairs = train_df[[source_col_train, target_col_train]].dropna().copy()\n    if len(all_pairs) > 1:\n        all_pairs = all_pairs.sample(frac=1.0, random_state=SEED).reset_index(drop=True)\n\n        # Clamp split to a safe range to avoid extreme holdout sizes.\n        split_n = max(1, int(len(all_pairs) * max(min(VAL_SPLIT, 0.5), 0.05)))\n        val_df = all_pairs.iloc[:split_n].copy()\n        fit_df = all_pairs.iloc[split_n:].copy()\n\n        if TRAIN_SAMPLES > 0:\n            fit_df = fit_df.head(min(TRAIN_SAMPLES, len(fit_df))).copy()\n        if VAL_SAMPLES > 0:\n            val_df = val_df.head(min(VAL_SAMPLES, len(val_df))).copy()\n\n        fit_pairs = build_clean_pairs(fit_df, source_col_train, target_col_train)\n        val_pairs = build_clean_pairs(val_df, source_col_train, target_col_train)\n\n        print({\n            \"fit_rows_raw\": len(fit_df),\n            \"val_rows_raw\": len(val_df),\n            \"fit_rows_clean\": len(fit_pairs),\n            \"val_rows_clean\": len(val_pairs),\n            \"val_split\": VAL_SPLIT,\n        })\n    else:\n        print(\"Not enough training rows after dropna to create fit/val split.\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:46:54.880969Z","iopub.execute_input":"2026-02-27T14:46:54.881265Z","iopub.status.idle":"2026-02-27T14:46:54.999754Z","shell.execute_reply.started":"2026-02-27T14:46:54.881241Z","shell.execute_reply":"2026-02-27T14:46:54.999002Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 19) Fine-Tuning Setup (DataLoaders + Optimizer + Scheduler)\n\nThis stage prepares everything needed before the epoch loop starts.\n\nWhat is prepared here:\n- train/validation dataloaders,\n- optimizer and LR scheduler,\n- AMP scaler settings,\n- checkpoint tracking variables.\n","metadata":{}},{"cell_type":"code","source":"training_info = None\ntrain_history = []\n\nif DO_TRAIN:\n    if fit_pairs is None or val_pairs is None or len(fit_pairs) == 0 or len(val_pairs) == 0:\n        raise RuntimeError(\"Training requested but fit/val split is unavailable after cleaning.\")\n\n    # Build prefixed training/validation source strings.\n    train_sources = [PREFIX + x[0] for x in fit_pairs]\n    train_targets = [x[1] for x in fit_pairs]\n    val_sources = [PREFIX + x[0] for x in val_pairs]\n    val_targets = [x[1] for x in val_pairs]\n\n    train_loader = DataLoader(\n        Seq2SeqTrainDataset(train_sources, train_targets),\n        batch_size=TRAIN_BATCH_SIZE,\n        shuffle=True,\n        num_workers=NUM_WORKERS,\n        collate_fn=collate_train_fn,\n        pin_memory=(DEVICE == \"cuda\"),\n    )\n    val_loader = DataLoader(\n        Seq2SeqTrainDataset(val_sources, val_targets),\n        batch_size=TRAIN_BATCH_SIZE,\n        shuffle=False,\n        num_workers=NUM_WORKERS,\n        collate_fn=collate_train_fn,\n        pin_memory=(DEVICE == \"cuda\"),\n    )\n\n    optimizer = torch.optim.AdamW(\n        model.parameters(),\n        lr=TRAIN_LR,\n        weight_decay=TRAIN_WEIGHT_DECAY,\n    )\n\n    # Number of optimizer updates after gradient accumulation.\n    updates_per_epoch = max(1, math.ceil(len(train_loader) / max(TRAIN_GRAD_ACCUM, 1)))\n    total_updates = updates_per_epoch * max(TRAIN_EPOCHS, 1)\n    if TRAIN_MAX_UPDATES > 0:\n        total_updates = min(total_updates, TRAIN_MAX_UPDATES)\n    warmup_steps = int(total_updates * max(TRAIN_WARMUP_RATIO, 0.0))\n\n    scheduler = get_linear_schedule_with_warmup(\n        optimizer,\n        num_warmup_steps=warmup_steps,\n        num_training_steps=total_updates,\n    )\n\n    amp_enabled = TRAIN_USE_AMP and DEVICE == \"cuda\"\n    scaler = torch.amp.GradScaler(\"cuda\", enabled=amp_enabled)\n\n    best_val_loss = float(\"inf\")\n    best_epoch = -1\n    best_dir = Path(TRAIN_SAVE_DIR)\n    best_dir.mkdir(parents=True, exist_ok=True)\n    update_count = 0\n    overflow_skips = 0\nelse:\n    print(\"Skipping fine-tuning because DPI_DO_TRAIN=0\")\n    training_info = {\"did_train\": False, \"history\": []}\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:47:05.234786Z","iopub.execute_input":"2026-02-27T14:47:05.235544Z","iopub.status.idle":"2026-02-27T14:47:05.246981Z","shell.execute_reply.started":"2026-02-27T14:47:05.235512Z","shell.execute_reply":"2026-02-27T14:47:05.246188Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 20) Fine-Tuning Epoch Loop\n\nThis stage performs gradient updates, validation loss evaluation, and best-checkpoint saving.\n\nWhy this matters:\n- handles NaN/Inf safety,\n- supports gradient accumulation,\n- logs per-epoch training diagnostics.\n","metadata":{}},{"cell_type":"code","source":"if DO_TRAIN:\n    for epoch in range(1, TRAIN_EPOCHS + 1):\n        model.train()\n        running_loss = 0.0\n        running_steps = 0\n        skipped_non_finite = 0\n        optimizer.zero_grad(set_to_none=True)\n\n        pbar = tqdm(train_loader, total=len(train_loader), desc=f\"Train epoch {epoch}/{TRAIN_EPOCHS}\")\n        for step, batch in enumerate(pbar, start=1):\n            batch = move_batch_to_device(batch)\n\n            with torch.amp.autocast(\"cuda\", enabled=amp_enabled):\n                loss = model(**batch).loss\n\n            # Skip batches that produce NaN/Inf loss to prevent optimizer corruption.\n            if not torch.isfinite(loss):\n                skipped_non_finite += batch[\"input_ids\"].shape[0]\n                optimizer.zero_grad(set_to_none=True)\n                continue\n\n            loss_scaled = loss / max(TRAIN_GRAD_ACCUM, 1)\n            scaler.scale(loss_scaled).backward()\n\n            running_loss += loss.item()\n            running_steps += 1\n            pbar.set_postfix({\"loss\": round(loss.item(), 4)})\n\n            do_update = (step % max(TRAIN_GRAD_ACCUM, 1) == 0) or (step == len(train_loader))\n            if do_update:\n                scaler.unscale_(optimizer)\n                torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n\n                if amp_enabled:\n                    prev_scale = scaler.get_scale()\n                    scaler.step(optimizer)\n                    scaler.update()\n                    new_scale = scaler.get_scale()\n                    did_update = new_scale >= prev_scale\n                    if not did_update:\n                        overflow_skips += 1\n                else:\n                    scaler.step(optimizer)\n                    scaler.update()\n                    did_update = True\n\n                optimizer.zero_grad(set_to_none=True)\n                if did_update:\n                    scheduler.step()\n                    update_count += 1\n\n                if TRAIN_MAX_UPDATES > 0 and update_count >= TRAIN_MAX_UPDATES:\n                    break\n\n        train_loss = running_loss / max(running_steps, 1)\n        val_loss = evaluate_val_loss(val_loader)\n        print({\n            \"epoch\": epoch,\n            \"train_loss\": round(train_loss, 4),\n            \"val_loss\": round(val_loss, 4),\n            \"train_steps_used\": running_steps,\n            \"skipped_non_finite_examples\": skipped_non_finite,\n            \"amp_overflow_skips\": overflow_skips,\n        })\n\n        train_history.append({\n            \"epoch\": int(epoch),\n            \"train_loss\": float(train_loss),\n            \"val_loss\": float(val_loss),\n            \"train_steps_used\": int(running_steps),\n            \"skipped_non_finite_examples\": int(skipped_non_finite),\n        })\n\n        # Save best checkpoint by validation loss.\n        if val_loss < best_val_loss:\n            best_val_loss = val_loss\n            best_epoch = epoch\n            model.save_pretrained(best_dir)\n            tokenizer.save_pretrained(best_dir)\n            print(f\"Saved new best checkpoint to {best_dir} at epoch {epoch}\")\n\n        if TRAIN_MAX_UPDATES > 0 and update_count >= TRAIN_MAX_UPDATES:\n            print(f\"Reached TRAIN_MAX_UPDATES={TRAIN_MAX_UPDATES}; stopping early.\")\n            break\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T14:47:12.385612Z","iopub.execute_input":"2026-02-27T14:47:12.386318Z","iopub.status.idle":"2026-02-27T15:01:22.695998Z","shell.execute_reply.started":"2026-02-27T14:47:12.386286Z","shell.execute_reply":"2026-02-27T15:01:22.695412Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 21) Finalize Fine-Tuning State for Inference\n\nThis stage reloads the best checkpoint and stores a compact training summary.\n\nWhy this matters:\n- guarantees inference uses best validation checkpoint,\n- preserves training history for diagnostics and logging.\n","metadata":{}},{"cell_type":"code","source":"if DO_TRAIN:\n    # Reload best checkpoint for evaluation/inference.\n    tokenizer = load_tokenizer_with_fallback(str(best_dir))\n    model = AutoModelForSeq2SeqLM.from_pretrained(best_dir, local_files_only=True)\n    model = model.to(DEVICE)\n    model.eval()\n    model_ref = str(best_dir)\n\n    training_info = {\n        \"did_train\": True,\n        \"epochs_requested\": TRAIN_EPOCHS,\n        \"epochs_best\": best_epoch,\n        \"best_val_loss\": best_val_loss,\n        \"train_rows\": len(fit_pairs),\n        \"val_rows\": len(val_pairs),\n        \"total_updates\": update_count,\n        \"amp_enabled\": amp_enabled,\n        \"amp_overflow_skips\": overflow_skips,\n        \"save_dir\": str(best_dir),\n        \"history\": train_history,\n    }\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T15:01:46.606175Z","iopub.execute_input":"2026-02-27T15:01:46.606774Z","iopub.status.idle":"2026-02-27T15:01:47.309331Z","shell.execute_reply.started":"2026-02-27T15:01:46.606747Z","shell.execute_reply":"2026-02-27T15:01:47.308398Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 22) Optional Local Metric Check on Holdout Split\n\nThis cell runs inference on the deterministic validation holdout and computes local proxy metrics.\n\nWhy this matters:\n- It gives directional feedback before costly Kaggle submissions.\n- It is a sanity check, not a leaderboard-equivalent score.\n","metadata":{}},{"cell_type":"code","source":"metrics = None\nval_size = 0\n\nif RUN_LOCAL_VAL and val_pairs is not None and len(val_pairs) > 0:\n    val_inputs = [PREFIX + x[0] for x in val_pairs]\n    val_refs = [x[1] for x in val_pairs]\n    val_size = len(val_inputs)\n\n    val_preds = generate_predictions(val_inputs, desc=\"Local metric inference\")\n    metrics = compute_competition_metric(val_refs, val_preds)\n    if metrics is not None:\n        print(\"Local metric sanity check on deterministic holdout (not leaderboard):\")\n        print({k: round(v, 4) for k, v in metrics.items()})\n        print({\"val_size\": val_size, \"val_split\": VAL_SPLIT})\nelse:\n    print(\"Skipping local metric check (missing val split or disabled).\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T15:01:55.770918Z","iopub.execute_input":"2026-02-27T15:01:55.771671Z","iopub.status.idle":"2026-02-27T15:04:24.570893Z","shell.execute_reply.started":"2026-02-27T15:01:55.771641Z","shell.execute_reply":"2026-02-27T15:04:24.570213Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 23) Run Test Inference for Submission\n\nThis cell generates predictions on the official test set.\n\nWhy this matters:\n- This is the exact output that will be packaged into Kaggle submission format.\n- A length check protects against silent batch/drop errors.\n","metadata":{}},{"cell_type":"code","source":"# In Kaggle code competitions, local test rows may be tiny.\n# Hidden rerun uses the full private test set.\ntest_preds = generate_predictions(test_inputs, desc=\"Test inference\")\n\nif len(test_preds) != len(test_ids):\n    raise RuntimeError(f\"Prediction length mismatch: {len(test_preds)} vs {len(test_ids)} ids\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T15:06:18.407509Z","iopub.execute_input":"2026-02-27T15:06:18.408286Z","iopub.status.idle":"2026-02-27T15:06:22.547972Z","shell.execute_reply.started":"2026-02-27T15:06:18.408252Z","shell.execute_reply":"2026-02-27T15:06:22.547269Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 24) Build and Save Submission File\n\nThis cell creates the required `id, translation` CSV and writes it to `OUTPUT_DIR`.\n\nWhy this matters:\n- Ensures strict submission schema compliance.\n- Stores file in a known artifact directory.\n","metadata":{}},{"cell_type":"code","source":"output_dir = Path(OUTPUT_DIR)\noutput_dir.mkdir(parents=True, exist_ok=True)\n\n# Build submission with trimmed text and empty-string fallback.\nsubmission = pd.DataFrame({\n    \"id\": test_ids,\n    \"translation\": [str(x).strip() for x in test_preds],\n})\nsubmission[\"translation\"] = submission[\"translation\"].fillna(\"\")\n\nsubmission_path = output_dir / \"submission.csv\"\nsubmission.to_csv(submission_path, index=False)\n\nprint(\"Saved submission:\", submission_path)\nprint(\"Submission shape:\", submission.shape)\nsubmission.head()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T15:07:06.975502Z","iopub.execute_input":"2026-02-27T15:07:06.976116Z","iopub.status.idle":"2026-02-27T15:07:07.002576Z","shell.execute_reply.started":"2026-02-27T15:07:06.976087Z","shell.execute_reply":"2026-02-27T15:07:07.002090Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 25) Save Run Summary JSON\n\nThis cell writes a structured run log containing config, metrics, and training history.\n\nWhy this matters:\n- Supports experiment tracking without external tools.\n- Makes later leaderboard analysis and rollback decisions easier.\n","metadata":{}},{"cell_type":"code","source":"run_summary = {\n    \"timestamp_utc\": datetime.now(timezone.utc).isoformat(),\n    \"experiment_tag\": EXPERIMENT_TAG,\n    \"model_ref\": model_ref,\n    \"seed\": SEED,\n    \"device\": DEVICE,\n    \"train_cfg\": {\n        \"do_train\": DO_TRAIN,\n        \"train_epochs\": TRAIN_EPOCHS,\n        \"train_batch_size\": TRAIN_BATCH_SIZE,\n        \"train_grad_accum\": TRAIN_GRAD_ACCUM,\n        \"train_lr\": TRAIN_LR,\n        \"train_weight_decay\": TRAIN_WEIGHT_DECAY,\n        \"train_warmup_ratio\": TRAIN_WARMUP_RATIO,\n        \"train_max_updates\": TRAIN_MAX_UPDATES,\n        \"train_samples\": TRAIN_SAMPLES,\n        \"train_save_dir\": TRAIN_SAVE_DIR,\n        \"train_use_amp\": TRAIN_USE_AMP,\n    },\n    \"inference_cfg\": {\n        \"batch_size\": BATCH_SIZE,\n        \"max_source_len\": MAX_SOURCE_LEN,\n        \"max_target_len\": MAX_TARGET_LEN,\n        \"max_new_tokens\": MAX_NEW_TOKENS,\n        \"num_beams\": NUM_BEAMS,\n        \"no_repeat_ngram_size\": NO_REPEAT_NGRAM_SIZE,\n        \"repetition_penalty\": REPETITION_PENALTY,\n        \"length_penalty\": LENGTH_PENALTY,\n        \"early_stopping\": EARLY_STOPPING,\n    },\n    \"limit\": LIMIT,\n    \"test_rows\": len(submission),\n    \"val_size\": val_size,\n    \"metrics\": metrics,\n    \"training_info\": training_info,\n}\n\nsummary_path = output_dir / f\"run_summary_{EXPERIMENT_TAG}.json\"\nsummary_path.write_text(json.dumps(run_summary, indent=2), encoding=\"utf-8\")\n\nprint(\"Saved run summary:\", summary_path)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T15:07:13.041563Z","iopub.execute_input":"2026-02-27T15:07:13.042122Z","iopub.status.idle":"2026-02-27T15:07:13.049024Z","shell.execute_reply.started":"2026-02-27T15:07:13.042090Z","shell.execute_reply":"2026-02-27T15:07:13.048274Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 26) Optional Visual Diagnostics\n\nThis cell plots:\n- training loss curves (if training history exists),\n- histogram of predicted output lengths.\n\nWhy this matters:\n- Loss curves quickly reveal underfitting/overfitting trends.\n- Length distribution helps detect degenerate decoding behavior.\n","metadata":{}},{"cell_type":"code","source":"try:\n    import matplotlib.pyplot as plt\nexcept Exception as e:\n    print(f\"matplotlib unavailable, skipping plots: {e}\")\nelse:\n    plots_made = 0\n\n    # Plot training/validation loss curve if fine-tuning history exists.\n    history = []\n    if isinstance(training_info, dict):\n        history = training_info.get(\"history\") or []\n\n    if history:\n        hist_df = pd.DataFrame(history)\n        plt.figure(figsize=(8, 4))\n        plt.plot(hist_df[\"epoch\"], hist_df[\"train_loss\"], marker=\"o\", label=\"train_loss\")\n        plt.plot(hist_df[\"epoch\"], hist_df[\"val_loss\"], marker=\"o\", label=\"val_loss\")\n        plt.title(\"Fine-tuning Loss Curves\")\n        plt.xlabel(\"Epoch\")\n        plt.ylabel(\"Loss\")\n        plt.grid(alpha=0.3)\n        plt.legend()\n        plt.show()\n        plots_made += 1\n    else:\n        print(\"No training history available for loss-curve plot.\")\n\n    # Plot prediction length distribution as a basic sanity check.\n    if \"test_preds\" in globals() and len(test_preds) > 0:\n        pred_lengths = [len(str(x).split()) for x in test_preds]\n        bins = min(20, max(5, len(set(pred_lengths))))\n\n        plt.figure(figsize=(8, 4))\n        plt.hist(pred_lengths, bins=bins, color=\"#4C78A8\", edgecolor=\"white\")\n        plt.title(\"Submission Prediction Length Distribution (words)\")\n        plt.xlabel(\"Predicted translation length\")\n        plt.ylabel(\"Count\")\n        plt.grid(axis=\"y\", alpha=0.3)\n        plt.show()\n\n        print({\n            \"prediction_count\": len(pred_lengths),\n            \"min_len\": int(min(pred_lengths)),\n            \"median_len\": float(np.median(pred_lengths)),\n            \"max_len\": int(max(pred_lengths)),\n        })\n        plots_made += 1\n    else:\n        print(\"No test predictions available for length histogram.\")\n\n    if plots_made == 0:\n        print(\"No plots were generated in this run.\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-27T15:07:23.098622Z","iopub.execute_input":"2026-02-27T15:07:23.099358Z","iopub.status.idle":"2026-02-27T15:07:23.404023Z","shell.execute_reply.started":"2026-02-27T15:07:23.099326Z","shell.execute_reply":"2026-02-27T15:07:23.402931Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Notes and Next Iteration Focus\n\n- This notebook is now the readable default pipeline: clear blocks, explicit configs, and safer checks.\n- Local metric is a sanity signal only; leaderboard remains the final truth.\n- Next performance gains usually come from: stronger normalization, improved fine-tuning schedule, and decoding/ensemble ablations.\n","metadata":{}}]}