{"cells":[{"cell_type":"markdown","metadata":{},"source":"# Bengali Speaker Embedding Fine-tuning\n\n**BUET DL Sprint 4.0 - Training Notebook (Problem 2)**\n\n### Novelty: Bengali-adapted speaker embeddings\n- ECAPA-TDNN pretrained on VoxCeleb (English/Western speakers)\n- Bengali speech has different phonetics, prosody, vocal characteristics\n- We fine-tune with contrastive learning on Bengali talkshow data\n- **Data sources**: Competition ground truth (10h) + auto-labeled subset from our 789h dataset\n\n### Pipeline:\n1. Load pyannote 3.1 + ECAPA-TDNN\n2. Stream ~30 files from our HuggingFace dataset → auto-label with pyannote\n3. Extract 3s speaker segments from competition data + auto-labeled data\n4. Fine-tune ECAPA with triplet loss (5 epochs)\n5. Evaluate pretrained vs fine-tuned on competition training set\n6. Upload fine-tuned model to HuggingFace\n\n### Resources:\n- **Dataset**: https://huggingface.co/datasets/nymtheescobar/bengali-talkshow-audio (789h, 1180 files)\n- **Model**: speechbrain/spkrec-ecapa-voxceleb → nymtheescobar/ecapa-bengali-finetuned\n- **Hardware**: Kaggle P100 GPU (16GB VRAM)"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 1: Install dependencies\n# ============================================================\n!NUMPY_VER=$(python -c \"import numpy; print(numpy.__version__)\") && \\\n  pip install -q huggingface_hub datasets \"numpy==$NUMPY_VER\" && \\\n  pip install -q pyannote.audio \"numpy==$NUMPY_VER\" && \\\n  pip install -q speechbrain \"numpy==$NUMPY_VER\" && \\\n  pip install -q silero-vad soundfile tqdm \"numpy==$NUMPY_VER\" && \\\n  echo \"All installs OK (numpy=$NUMPY_VER)\""},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 2: torch.load patch + imports\n# ============================================================\nimport torch\n# CRITICAL: PyTorch 2.6 defaults weights_only=True; lightning_fabric\n# EXPLICITLY passes weights_only=True. Must FORCE False.\n_original_torch_load = torch.load\ndef _patched_torch_load(*args, **kwargs):\n    kwargs['weights_only'] = False\n    return _original_torch_load(*args, **kwargs)\ntorch.load = _patched_torch_load\n\nimport json, random, time, warnings\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nimport soundfile as sf\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import Dataset, DataLoader\nfrom tqdm.auto import tqdm\nfrom sklearn.preprocessing import normalize\nfrom sklearn.cluster import AgglomerativeClustering\nfrom sklearn.metrics import silhouette_score\n\nwarnings.filterwarnings('ignore')\nDEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nprint(f\"torch.load patched | Device: {DEVICE}\")"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 3: Config + Auth\n# ============================================================\nHF_TOKEN = \"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\"\nfrom huggingface_hub import login\nlogin(token=HF_TOKEN)\n\n# Paths\nKAGGLE_ROOT = Path(\"/kaggle/input/dl-sprint-4-0-bengali-speaker-diarization-challenge/diarization/diarization\")\nLOCAL_ROOT = Path(\"data/diarization/diarization\")\nDATA_ROOT = KAGGLE_ROOT if KAGGLE_ROOT.exists() else LOCAL_ROOT\nTRAIN_AUDIO = DATA_ROOT / \"train\" / \"audio\"\nTRAIN_ANNOT = DATA_ROOT / \"train\" / \"annotation\"\n\n# HuggingFace dataset (789h Bengali talkshow audio)\nHF_DATASET = \"nymtheescobar/bengali-talkshow-audio\"\nHF_SUBSET_SIZE = 30  # Stream this many files for auto-labeling (~20h)\n\n# Fine-tuning config\nCFG = dict(\n    segment_sec  = 3.0,    # 3s chunks per speaker\n    min_segments = 3,      # Min segments to include a speaker\n    max_segments = 50,     # Cap per speaker for balance\n    batch_size   = 32,\n    lr           = 1e-4,\n    epochs       = 5,\n    margin       = 0.3,    # Triplet loss margin\n    sr           = 16000,\n)\n\nSAVE_DIR = Path(\"/kaggle/working/ecapa_bengali_finetuned\")\nSAVE_DIR.mkdir(parents=True, exist_ok=True)\n\nprint(f\"Competition data: {DATA_ROOT}\")\nprint(f\"HF dataset: {HF_DATASET} (streaming {HF_SUBSET_SIZE} files)\")\nprint(f\"Save dir: {SAVE_DIR}\")"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 4: Load models (pyannote for auto-labeling, ECAPA for fine-tuning)\n# ============================================================\nfrom pyannote.audio import Pipeline as PyannotePipeline\nfrom speechbrain.inference import EncoderClassifier\n\n# pyannote 3.1 for auto-labeling HF dataset\nprint(\"Loading pyannote 3.1...\")\ndiar_pipeline = PyannotePipeline.from_pretrained(\"pyannote/speaker-diarization-3.1\")\ndiar_pipeline.to(torch.device(DEVICE))\ndiar_pipeline.instantiate({\n    \"segmentation\": {\"min_duration_off\": 0.0},\n    \"clustering\": {\"method\": \"centroid\", \"min_cluster_size\": 15, \"threshold\": 0.40},\n})\nprint(\"  pyannote 3.1 ready\")\n\n# ECAPA-TDNN for fine-tuning\nprint(\"Loading ECAPA-TDNN...\")\nemb_model = EncoderClassifier.from_hparams(\n    source=\"speechbrain/spkrec-ecapa-voxceleb\",\n    run_opts={\"device\": DEVICE},\n    savedir=str(SAVE_DIR / \"pretrained\"),\n)\nencoder = emb_model.mods.embedding_model\nprint(f\"  ECAPA-TDNN ready ({sum(p.numel() for p in encoder.parameters()):,} params)\")"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 5: Extract speaker segments from audio + annotations\n# ============================================================\ndef _hhmmss(t):\n    p = str(t).strip().split(\":\")\n    if len(p)==3: return int(p[0])*3600+int(p[1])*60+float(p[2])\n    if len(p)==2: return int(p[0])*60+float(p[1])\n    return float(t)\n\ndef extract_segments_from_annotations(wav, annotations, seg_sec=3.0, sr=16000, prefix=\"\"):\n    \"\"\"Extract fixed-length speaker segments from diarization output.\n    annotations: list of (start_sec, end_sec, speaker_id)\n    Returns: dict[speaker_key] -> list of numpy arrays\n    \"\"\"\n    seg_samples = int(seg_sec * sr)\n    speakers = {}\n    for s, e, spk in annotations:\n        if e <= s: continue\n        key = f\"{prefix}_{spk}\" if prefix else spk\n        if key not in speakers: speakers[key] = []\n        pos = int(s * sr)\n        end_pos = int(e * sr)\n        while pos + seg_samples <= end_pos:\n            chunk = wav[pos:pos + seg_samples]\n            if len(chunk) == seg_samples and np.abs(chunk).max() > 0.01:\n                speakers[key].append(chunk)\n            pos += seg_samples\n    return speakers\n\ndef extract_from_competition_data():\n    \"\"\"Phase A: Competition training data (ground truth labels).\"\"\"\n    all_spk = {}\n    csvs = sorted(TRAIN_ANNOT.glob(\"*.csv\"))\n    for csv_path in tqdm(csvs, desc=\"Competition data\"):\n        stem = csv_path.stem\n        wav_path = TRAIN_AUDIO / f\"{stem}.wav\"\n        if not wav_path.exists(): continue\n        wav, _ = sf.read(str(wav_path))\n        if wav.ndim == 2: wav = wav.mean(axis=1)\n        wav = wav.astype(np.float32)\n        \n        df = pd.read_csv(csv_path)\n        df.columns = [c.lower() for c in df.columns]\n        df = df.dropna(subset=[\"start_time\", \"end_time\", \"speaker_id\"])\n        annots = [(_hhmmss(r[\"start_time\"]), _hhmmss(r[\"end_time\"]),\n                   f\"SPK{int(r['speaker_id'])}\") for _, r in df.iterrows()]\n        \n        spk = extract_segments_from_annotations(wav, annots, CFG[\"segment_sec\"], prefix=stem)\n        for k, v in spk.items():\n            if len(v) >= CFG[\"min_segments\"]:\n                if len(v) > CFG[\"max_segments\"]:\n                    v = random.sample(v, CFG[\"max_segments\"])\n                all_spk[k] = v\n        n_seg = sum(len(v) for v in spk.values())\n        print(f\"  {stem}: {len(spk)} speakers, {n_seg} segments\")\n    return all_spk\n\n# Run Phase A\ncomp_speakers = extract_from_competition_data()\nprint(f\"\\nCompetition: {len(comp_speakers)} speakers, \"\n      f\"{sum(len(v) for v in comp_speakers.values())} segments\")"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 6: Phase B - Stream HF dataset subset + auto-label with pyannote\n# ============================================================\nfrom datasets import load_dataset\nfrom huggingface_hub import hf_hub_download\n\ndef auto_label_hf_subset(n_files=30):\n    \"\"\"Download n_files from HF dataset, auto-label with pyannote 3.1.\"\"\"\n    print(f\"Loading metadata from {HF_DATASET}...\")\n    \n    # Load dataset metadata only (streaming) - audio column is just a string path\n    ds = load_dataset(HF_DATASET, split=\"train\", streaming=True)\n    \n    all_spk = {}\n    processed = 0\n    total_hours = 0\n    \n    for i, sample in enumerate(tqdm(ds, total=n_files, desc=\"Auto-labeling\")):\n        if processed >= n_files:\n            break\n        \n        try:\n            # sample[\"audio\"] is a string like \"data/bengali_0001_xyz.flac\"\n            audio_path_in_repo = sample[\"audio\"]\n            \n            # Skip short files using metadata (faster than downloading)\n            dur_min = sample.get(\"duration_min\", 0)\n            if dur_min and dur_min < 10:\n                continue\n            \n            # Download the actual FLAC file from HuggingFace\n            local_path = hf_hub_download(\n                repo_id=HF_DATASET,\n                filename=audio_path_in_repo,\n                repo_type=\"dataset\",\n            )\n            \n            # Decode with soundfile\n            wav, sr = sf.read(local_path)\n            if wav.ndim == 2:\n                wav = wav.mean(axis=1)\n            wav = wav.astype(np.float32)\n            \n            # Resample to 16kHz if needed\n            if sr != 16000:\n                import librosa\n                wav = librosa.resample(wav, orig_sr=sr, target_sr=16000)\n                sr = 16000\n            \n            # Skip very short files (< 10 min) - double check with actual audio\n            dur_min_actual = len(wav) / sr / 60\n            if dur_min_actual < 10:\n                continue\n            \n            # Limit to 30 min to save time\n            max_samples = 30 * 60 * sr\n            if len(wav) > max_samples:\n                wav = wav[:max_samples]\n            \n            dur_sec = len(wav) / sr\n            total_hours += dur_sec / 3600\n            vid = sample.get(\"video_id\", f\"hf_{i:04d}\")\n            \n            # Run pyannote diarization\n            wav_tensor = torch.from_numpy(wav).float().unsqueeze(0)\n            diarization = diar_pipeline(\n                {\"waveform\": wav_tensor, \"sample_rate\": sr},\n                min_speakers=2, max_speakers=25,\n            )\n            \n            # Extract annotations\n            annots = []\n            for turn, _, speaker in diarization.itertracks(yield_label=True):\n                annots.append((turn.start, turn.end, speaker))\n            \n            if len(annots) < 10:\n                continue\n            \n            # Extract speaker segments\n            spk = extract_segments_from_annotations(\n                wav, annots, CFG[\"segment_sec\"], sr, prefix=f\"hf_{vid}\"\n            )\n            for k, v in spk.items():\n                if len(v) >= CFG[\"min_segments\"]:\n                    if len(v) > CFG[\"max_segments\"]:\n                        v = random.sample(v, CFG[\"max_segments\"])\n                    all_spk[k] = v\n            \n            n_spk = len(set(a[2] for a in annots))\n            processed += 1\n            print(f\"  [{processed}/{n_files}] {vid}: {n_spk} speakers, \"\n                  f\"{len(annots)} turns, {dur_sec/60:.1f}min | \"\n                  f\"Total: {len(all_spk)} speakers, {total_hours:.1f}h\")\n        \n        except Exception as e:\n            print(f\"  Skip {i}: {type(e).__name__}: {e}\")\n            import traceback\n            traceback.print_exc()\n            continue\n    \n    print(f\"\\nAuto-labeled: {processed} files, {total_hours:.1f}h, \"\n          f\"{len(all_spk)} speakers, {sum(len(v) for v in all_spk.values())} segments\")\n    return all_spk\n\nhf_speakers = auto_label_hf_subset(HF_SUBSET_SIZE)\n\n# Combine competition + HF data\nall_speakers = {**comp_speakers, **hf_speakers}\nprint(f\"\\nCOMBINED: {len(all_speakers)} speakers, \"\n      f\"{sum(len(v) for v in all_speakers.values())} segments\")"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 7: Triplet dataset + loss\n# ============================================================\nclass SpeakerTripletDataset(Dataset):\n    def __init__(self, speakers_dict, n_triplets=20000):\n        self.speakers = speakers_dict\n        self.speaker_ids = list(speakers_dict.keys())\n        self.n_triplets = n_triplets\n    \n    def __len__(self):\n        return self.n_triplets\n    \n    def __getitem__(self, idx):\n        pos_spk = random.choice(self.speaker_ids)\n        pos_chunks = self.speakers[pos_spk]\n        a_idx, p_idx = random.sample(range(len(pos_chunks)), 2)\n        anchor = torch.from_numpy(pos_chunks[a_idx]).float()\n        positive = torch.from_numpy(pos_chunks[p_idx]).float()\n        neg_spk = random.choice([s for s in self.speaker_ids if s != pos_spk])\n        negative = torch.from_numpy(random.choice(self.speakers[neg_spk])).float()\n        return anchor, positive, negative\n\nclass TripletLoss(nn.Module):\n    def __init__(self, margin=0.3):\n        super().__init__()\n        self.margin = margin\n    \n    def forward(self, anchor, positive, negative):\n        pos_sim = F.cosine_similarity(anchor, positive)\n        neg_sim = F.cosine_similarity(anchor, negative)\n        return F.relu(neg_sim - pos_sim + self.margin).mean()\n\n# Build dataset (filter speakers with >=2 segments for triplets)\nvalid_speakers = {k: v for k, v in all_speakers.items() if len(v) >= 2}\nn_triplets = min(20000, len(valid_speakers) * 100)\ndataset = SpeakerTripletDataset(valid_speakers, n_triplets=n_triplets)\ndataloader = DataLoader(dataset, batch_size=CFG[\"batch_size\"],\n                       shuffle=True, num_workers=2, pin_memory=True)\n\nprint(f\"Valid speakers: {len(valid_speakers)}\")\nprint(f\"Triplets/epoch: {n_triplets}, Batches: {len(dataloader)}\")"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 8: Fine-tuning loop\n# ============================================================\n\n# CRITICAL: SpeechBrain's from_hparams() freezes all parameters for inference.\n# Must explicitly unfreeze before training.\nfor param in encoder.parameters():\n    param.requires_grad = True\n\n# Verify unfreezing worked\nn_trainable = sum(p.numel() for p in encoder.parameters() if p.requires_grad)\nn_total = sum(p.numel() for p in encoder.parameters())\nprint(f\"Unfrozen: {n_trainable:,}/{n_total:,} params trainable\")\n\nencoder.train()\ncriterion = TripletLoss(margin=CFG[\"margin\"])\noptimizer = torch.optim.Adam(encoder.parameters(), lr=CFG[\"lr\"])\nscheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=CFG[\"epochs\"])\n\nhistory = []\nfor epoch in range(CFG[\"epochs\"]):\n    epoch_loss, n_batches = 0, 0\n    pbar = tqdm(dataloader, desc=f\"Epoch {epoch+1}/{CFG['epochs']}\")\n    for anchor, positive, negative in pbar:\n        optimizer.zero_grad()\n        lens = torch.ones(anchor.shape[0], device=DEVICE)\n        with torch.enable_grad():\n            emb_a = emb_model.encode_batch(anchor.to(DEVICE), lens).squeeze(1)\n            emb_p = emb_model.encode_batch(positive.to(DEVICE), lens).squeeze(1)\n            emb_n = emb_model.encode_batch(negative.to(DEVICE), lens).squeeze(1)\n        loss = criterion(emb_a, emb_p, emb_n)\n        loss.backward()\n        torch.nn.utils.clip_grad_norm_(encoder.parameters(), 1.0)\n        optimizer.step()\n        epoch_loss += loss.item()\n        n_batches += 1\n        pbar.set_postfix(loss=f\"{loss.item():.4f}\")\n    \n    scheduler.step()\n    avg = epoch_loss / max(n_batches, 1)\n    history.append({\"epoch\": epoch+1, \"loss\": avg, \"lr\": scheduler.get_last_lr()[0]})\n    print(f\"  Epoch {epoch+1}: loss={avg:.4f}, lr={scheduler.get_last_lr()[0]:.6f}\")\n\nprint(\"\\nTraining complete!\")\npd.DataFrame(history)"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 9: Save fine-tuned model\n# ============================================================\n# Save just the embedding model weights (what nuclear notebook expects)\ntorch.save(encoder.state_dict(), SAVE_DIR / \"ecapa_bengali_finetuned.pt\")\n\n# Save full checkpoint with metadata\ntorch.save({\n    \"model_state_dict\": encoder.state_dict(),\n    \"config\": CFG,\n    \"n_speakers_comp\": len(comp_speakers),\n    \"n_speakers_hf\": len(hf_speakers),\n    \"n_speakers_total\": len(valid_speakers),\n    \"n_segments\": sum(len(v) for v in valid_speakers.values()),\n    \"history\": history,\n}, SAVE_DIR / \"ecapa_bengali_full.pt\")\nprint(f\"Saved to {SAVE_DIR}\")\nprint(f\"  ecapa_bengali_finetuned.pt = embedding weights only\")\nprint(f\"  ecapa_bengali_full.pt = weights + metadata\")"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 10: Evaluate pretrained vs fine-tuned on competition data\n# ============================================================\nfrom pyannote.core import Annotation, Segment\nfrom pyannote.metrics.diarization import DiarizationErrorRate\nfrom silero_vad import load_silero_vad, get_speech_timestamps\n\nvad_model = load_silero_vad(onnx=False)\nvad_model.eval()\n\ndef diarize_with_ecapa(audio_path):\n    \"\"\"Manual diarization pipeline using ECAPA embeddings + NME-SC.\"\"\"\n    wav, _ = sf.read(str(audio_path))\n    if wav.ndim == 2: wav = wav.mean(axis=1)\n    wav = wav.astype(np.float32)\n    sr = 16000\n    \n    # VAD\n    ts = get_speech_timestamps(torch.from_numpy(wav).float(), vad_model,\n                               sampling_rate=sr, threshold=0.45,\n                               speech_pad_ms=400, min_speech_duration_ms=300,\n                               min_silence_duration_ms=250)\n    segs = [(t[\"start\"]/sr, t[\"end\"]/sr) for t in ts]\n    if not segs: return []\n    \n    # Windows\n    win, hop = 3.0, 1.5\n    windows = []\n    for s, e in segs:\n        if e - s <= win: windows.append((s, e)); continue\n        t = s\n        while t + win <= e + 1e-6: windows.append((t, t + win)); t += hop\n    \n    # Embeddings\n    encoder.eval()\n    chunks, vw = [], []\n    for ws, we in windows:\n        c = wav[int(ws*sr):int(we*sr)]\n        if len(c) >= int(0.5*sr): chunks.append(c); vw.append((ws, we))\n    if not chunks: return []\n    \n    with torch.inference_mode():\n        all_embs = []\n        for i in range(0, len(chunks), 64):\n            batch = chunks[i:i+64]\n            mx = max(len(c) for c in batch)\n            pad = np.zeros((len(batch), mx), np.float32)\n            lens = []\n            for j, c in enumerate(batch):\n                pad[j,:len(c)] = c; lens.append(len(c))\n            x = torch.from_numpy(pad).to(DEVICE)\n            wl = torch.tensor([l/mx for l in lens], dtype=torch.float32).to(DEVICE)\n            e = emb_model.encode_batch(x, wl)\n            all_embs.append(e.squeeze(1).cpu().numpy())\n        embs = np.vstack(all_embs)\n    \n    # NME-SC speaker count\n    X = normalize(embs)\n    cos_sim = X @ X.T\n    affinity = (cos_sim + 1.0) / 2.0\n    np.fill_diagonal(affinity, 0)\n    n = affinity.shape[0]\n    max_k = min(25, n - 1)\n    if max_k < 2: return []\n    \n    degree = np.sum(affinity, axis=1)\n    degree[degree == 0] = 1e-10\n    D_inv = np.diag(1.0 / np.sqrt(degree))\n    L = np.eye(n) - D_inv @ affinity @ D_inv\n    evals = np.sort(np.maximum(np.real(np.linalg.eigvalsh(L)), 0))\n    best_k, best_gap = 2, 0\n    for k in range(1, min(max_k + 1, len(evals))):\n        if k < len(evals) and evals[k] > 1e-10:\n            gap = (evals[k] - evals[k-1]) / evals[k]\n            if gap > best_gap: best_gap, best_k = gap, k\n    # Silhouette refinement\n    lo, hi = max(2, best_k - 5), min(max_k, best_k + 5) + 1\n    best_s = -1.0\n    for k in range(lo, hi):\n        try:\n            la = AgglomerativeClustering(n_clusters=k, metric=\"cosine\", linkage=\"average\").fit_predict(X)\n            if len(set(la)) < 2: continue\n            sc = silhouette_score(X, la, metric=\"cosine\")\n            if sc > best_s: best_s, best_k = sc, k\n        except: continue\n    \n    labels = AgglomerativeClustering(n_clusters=best_k, metric=\"cosine\", linkage=\"average\").fit_predict(X)\n    \n    # Build turns\n    items = sorted(zip(vw, labels), key=lambda x: x[0][0])\n    cs, ce, cl = items[0][0][0], items[0][0][1], int(items[0][1])\n    turns = []\n    for (ws, we), lab in items[1:]:\n        lab = int(lab)\n        if lab == cl and ws <= ce + 1e-3: ce = max(ce, we)\n        else:\n            if ce - cs >= 0.3: turns.append((cs, ce, cl))\n            cs, ce, cl = ws, we, lab\n    if ce - cs >= 0.3: turns.append((cs, ce, cl))\n    \n    lmap, nxt, out = {}, 1, []\n    for s, e, l in turns:\n        if l not in lmap: lmap[l] = f\"SPEAKER_{nxt}\"; nxt += 1\n        out.append({\"start_time\": round(s,2), \"end_time\": round(e,2), \"speaker_id\": lmap[l]})\n    return out\n\n\n# Compare pretrained vs fine-tuned\n# Save pretrained weights BEFORE training overwrote them\n# (encoder already has fine-tuned weights from Cell 8)\nfinetuned_state = torch.load(SAVE_DIR / \"ecapa_bengali_finetuned.pt\", weights_only=False)\n\n# We need pretrained weights - reload from SpeechBrain\nprint(\"Reloading pretrained ECAPA for comparison...\")\npretrained_model = EncoderClassifier.from_hparams(\n    source=\"speechbrain/spkrec-ecapa-voxceleb\",\n    run_opts={\"device\": DEVICE},\n    savedir=str(SAVE_DIR / \"pretrained_eval\"),\n)\npretrained_state = {k: v.clone() for k, v in pretrained_model.mods.embedding_model.state_dict().items()}\ndel pretrained_model\n\nresults = {}\nfor mode_name, state in [(\"Pretrained\", pretrained_state), (\"Fine-tuned\", finetuned_state)]:\n    print(f\"\\n{'='*60}\")\n    print(f\"  {mode_name} ECAPA-TDNN\")\n    print(f\"{'='*60}\")\n    encoder.load_state_dict(state)\n    encoder.eval()\n    \n    total_err, total_ref = 0, 0\n    for csv_path in sorted(TRAIN_ANNOT.glob(\"*.csv\")):\n        stem = csv_path.stem\n        wav_path = TRAIN_AUDIO / f\"{stem}.wav\"\n        if not wav_path.exists(): continue\n        \n        pred = diarize_with_ecapa(wav_path)\n        df = pd.read_csv(csv_path)\n        df.columns = [c.lower() for c in df.columns]\n        ref = Annotation(uri=stem)\n        for _, r in df.dropna().iterrows():\n            s, e = _hhmmss(r[\"start_time\"]), _hhmmss(r[\"end_time\"])\n            if e > s: ref[Segment(s, e)] = f\"SPEAKER_{int(r['speaker_id'])}\"\n        hyp = Annotation(uri=stem)\n        for d in pred:\n            hyp[Segment(d[\"start_time\"], d[\"end_time\"])] = d[\"speaker_id\"]\n        \n        m = DiarizationErrorRate(collar=0.25)\n        der = m(ref, hyp)\n        c = m.accumulated_\n        total_err += c[\"false alarm\"] + c[\"missed detection\"] + c[\"confusion\"]\n        total_ref += c[\"total\"]\n        print(f\"  {stem}: DER={der:.4f}  spk {len(ref.labels())}>{len(hyp.labels())}\")\n    \n    agg = total_err / total_ref if total_ref else 1.0\n    results[mode_name] = agg\n    print(f\"\\n  AGGREGATE DER={agg:.4f}  Score={100*(1-agg):.1f}\")\n\n# Restore fine-tuned weights for upload\nencoder.load_state_dict(finetuned_state)\n\nimprovement = results.get(\"Pretrained\", 1) - results.get(\"Fine-tuned\", 1)\nprint(f\"\\n{'='*60}\")\nprint(f\"  DER IMPROVEMENT: {improvement:.4f} ({improvement*100:.1f} points)\")\nprint(f\"{'='*60}\")"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ============================================================\n# CELL 11: Upload fine-tuned model to HuggingFace\n# ============================================================\nfrom huggingface_hub import HfApi\n\nMODEL_REPO = \"nymtheescobar/ecapa-bengali-finetuned\"\napi = HfApi()\napi.create_repo(MODEL_REPO, exist_ok=True)\n\n# Upload embedding weights (this is what nuclear notebook downloads)\napi.upload_file(\n    path_or_fileobj=str(SAVE_DIR / \"ecapa_bengali_finetuned.pt\"),\n    path_in_repo=\"ecapa_bengali_finetuned.pt\",\n    repo_id=MODEL_REPO,\n)\n\n# Upload full checkpoint with metadata\napi.upload_file(\n    path_or_fileobj=str(SAVE_DIR / \"ecapa_bengali_full.pt\"),\n    path_in_repo=\"ecapa_bengali_full.pt\",\n    repo_id=MODEL_REPO,\n)\n\n# Upload model card\ncard = f\"\"\"# ECAPA-TDNN Fine-tuned for Bengali Speaker Diarization\n\n**BUET DL Sprint 4.0 - Problem 2**\n\n## Model Description\nECAPA-TDNN speaker embedding model fine-tuned on Bengali talkshow audio\nfor speaker diarization. Base model: `speechbrain/spkrec-ecapa-voxceleb`.\n\n## Usage\n```python\n# In the nuclear diarization notebook, this model is loaded automatically:\nfrom huggingface_hub import hf_hub_download\nckpt = hf_hub_download(\"nymtheescobar/ecapa-bengali-finetuned\", \"ecapa_bengali_finetuned.pt\")\nstate_dict = torch.load(ckpt)\necapa.mods.embedding_model.load_state_dict(state_dict, strict=False)\n```\n\n## Training Data\n- **Competition data**: 10 files, ~10h, ~170 speakers (ground truth)\n- **Bengali Talkshow Dataset**: {len(hf_speakers)} speakers from {HF_SUBSET_SIZE} files\n  auto-labeled with pyannote 3.1\n- **Total**: {len(valid_speakers)} speakers, {sum(len(v) for v in valid_speakers.values())} segments\n\n## Training\n- Loss: Triplet (margin={CFG['margin']})\n- Epochs: {CFG['epochs']}, LR: {CFG['lr']}\n- Segment length: {CFG['segment_sec']}s\n\n## Results\n- Pretrained DER: {results.get('Pretrained', 0):.4f}\n- Fine-tuned DER: {results.get('Fine-tuned', 0):.4f}\n- Improvement: {improvement:.4f}\n\n## Dataset\nhttps://huggingface.co/datasets/nymtheescobar/bengali-talkshow-audio\n\"\"\"\napi.upload_file(\n    path_or_fileobj=card.encode(),\n    path_in_repo=\"README.md\",\n    repo_id=MODEL_REPO,\n)\n\nprint(f\"Model uploaded: https://huggingface.co/{MODEL_REPO}\")\nprint(f\"  ecapa_bengali_finetuned.pt (embedding weights)\")\nprint(f\"  ecapa_bengali_full.pt (full checkpoint)\")\nprint(f\"  README.md (model card)\")"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.10.0"}},"nbformat":4,"nbformat_minor":4}