{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":129271,"databundleVersionId":15506307,"sourceType":"competition"}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"2997b40f","cell_type":"markdown","source":"# Bengali Long-form Speaker Diarization — Starter Notebook\n\n**Audio → ML-based Voice Activity Detection (Silero) → windows → ECAPA embeddings → clustering → speaker turns → CSV output**\n\n\n","metadata":{}},{"id":"75d7e06f","cell_type":"markdown","source":"## 1) Dependencies install","metadata":{}},{"id":"e5f1acdf","cell_type":"code","source":"\n!pip -q install soundfile torchaudio speechbrain scikit-learn tqdm silero-vad","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-01-30T13:28:26.793825Z","iopub.execute_input":"2026-01-30T13:28:26.794479Z"}},"outputs":[],"execution_count":null},{"id":"975bdae2","cell_type":"markdown","source":"## 2) Import setup","metadata":{}},{"id":"919b47c0","cell_type":"code","source":"import json\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nimport soundfile as sf\nimport torch\nimport torchaudio\nfrom tqdm.auto import tqdm\n\nfrom sklearn.cluster import AgglomerativeClustering\nfrom sklearn.metrics import silhouette_score\nfrom sklearn.preprocessing import normalize\n\nimport matplotlib.pyplot as plt","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"fd4f8ed9","cell_type":"markdown","source":"## 3) Paths + Device configuration","metadata":{}},{"id":"7f276d0b","cell_type":"code","source":"# === FILE PATH ===\nAUDIO_DIR = Path(\"/kaggle/input/dl-sprint-4-0-bengali-speaker-diarization-challenge/diarization/diarization/test/audio\")          \nOUT_PATH = Path(\"submission.csv\")  \n\n# === Device ===\nDEVICE = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nDEVICE","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"d809f9f5","cell_type":"markdown","source":"## 4) Collect files (and stems)","metadata":{}},{"id":"d7f43b37","cell_type":"code","source":"wav_paths = sorted(AUDIO_DIR.glob(\"*.wav\"))\nstems = [p.stem for p in wav_paths]\n\nlen(wav_paths), (wav_paths[0] if wav_paths else None)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"36129ce3","cell_type":"markdown","source":"## 5) Audio loader (mono float32 @ 16 kHz)\nReads an audio file, converts it to **mono** `float32`, and **resamples** to **16 kHz** (if needed) for consistent downstream VAD and embedding extraction.\n","metadata":{}},{"id":"746c420b","cell_type":"code","source":"def load_audio_mono_16k(path: str | Path, target_sr: int = 16000) -> np.ndarray:\n    \"\"\"Load audio file as mono float32 at target_sr.\"\"\"\n    wav, sr = sf.read(str(path))\n    if wav.ndim == 2:\n        wav = wav.mean(axis=1)\n    wav = wav.astype(np.float32)\n\n    if sr != target_sr:\n        x = torch.from_numpy(wav).unsqueeze(0)\n        resampler = torchaudio.transforms.Resample(orig_freq=sr, new_freq=target_sr)\n        wav = resampler(x).squeeze(0).cpu().numpy().astype(np.float32)\n    return wav","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"9846dc3e","cell_type":"markdown","source":"## 6) Load ML-based Voice Activity Detection(VAD) model (Silero)\nLoads the **Silero neural VAD** using the official `silero-vad` package (with an optional `torch.hub` fallback), supports **Torch/ONNX** variants, and places the VAD model on **CPU** for stable inference across environments.\n","metadata":{}},{"id":"c3f4f23e","cell_type":"code","source":"\nUSE_PIP = True   # set False to use torch.hub instead\nUSE_ONNX = False # set True to use ONNX model\nVAD_DEVICE=\"cpu\" # silero VAD is light-weight,okay with cpu \n\nif USE_PIP:\n    from silero_vad import (\n        load_silero_vad,\n        get_speech_timestamps,\n        read_audio,\n        save_audio,\n        VADIterator,\n        collect_chunks,\n    )\n    silero_model = load_silero_vad(onnx=USE_ONNX)\nelse:\n    silero_model, utils = torch.hub.load(\n        repo_or_dir=\"snakers4/silero-vad\",\n        model=\"silero_vad\",\n        trust_repo=True,\n        onnx=USE_ONNX,\n    )\n    (get_speech_timestamps, save_audio, read_audio, VADIterator, collect_chunks) = utils\n\nsilero_model.to(VAD_DEVICE)\nsilero_model.eval()\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"82fe7112","cell_type":"markdown","source":"## 7) Neural VAD → speech segments (Silero)\nRuns **Silero VAD** on the full **16 kHz mono waveform** to extract speech regions as `(start_sec, end_sec)`. The main knobs are `vad_threshold` (strictness), `padding_ms` (context around speech), and `min_speech_ms` / `min_silence_ms` (smoothing), with a final merge step to handle overlaps.\n","metadata":{}},{"id":"17f17967","cell_type":"code","source":"def silero_vad_speech_segments(\n    wav16k: np.ndarray,\n    sr: int = 16000,\n    vad_threshold: float = 0.5,\n    padding_ms: int = 300,\n    min_silence_ms: int = 200,\n    min_speech_ms: int = 300,\n):\n    \"\"\"\n    Returns list of (start_sec, end_sec) speech segments using Silero neural VAD.\n    Runs VAD on the full waveform (more robust than streaming chunk iteration).\n    \"\"\"\n    assert sr == 16000, \"Silero VAD expects 16kHz audio\"\n\n    x = torch.from_numpy(wav16k).float().to(VAD_DEVICE)  # Silero utils typically expect CPU tensor\n\n    ts = get_speech_timestamps(\n        x,\n        silero_model,\n        sampling_rate=sr,\n        threshold=vad_threshold,\n        min_speech_duration_ms=min_speech_ms,\n        min_silence_duration_ms=min_silence_ms,\n        speech_pad_ms=padding_ms,\n    )\n\n\n    segs = [(t[\"start\"] / sr, t[\"end\"] / sr) for t in ts]\n\n    # Defensive merge in case of overlaps\n    merged = []\n    for s, e in sorted(segs):\n        if not merged or s > merged[-1][1]:\n            merged.append([s, e])\n        else:\n            merged[-1][1] = max(merged[-1][1], e)\n\n    return [(float(s), float(e)) for s, e in merged]\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"17faacf3-e65d-443f-a7e8-028ef5967398","cell_type":"markdown","source":"### 7.1) Visual check: waveform + VAD speech segments\nOverlay detected speech regions on the waveform to quickly spot missed speech or false alarms.\n","metadata":{}},{"id":"577514ff-8a5f-4d30-a700-1e59617e61c2","cell_type":"code","source":"def plot_waveform_with_segments(wav, sr, segments, title=\"Waveform + VAD\"):\n    t = np.arange(len(wav)) / sr\n    plt.figure(figsize=(12, 3))\n    plt.plot(t, wav, linewidth=0.7)\n    for s, e in segments:\n        plt.axvspan(s, e, alpha=0.25)\n    plt.xlabel(\"Time (s)\")\n    plt.title(title)\n    plt.grid(True, axis=\"x\")\n    plt.show()\n\n# Example on one file\nif wav_paths:\n    wav = load_audio_mono_16k(wav_paths[0], 16000)\n    speech_segs = silero_vad_speech_segments(wav, sr=16000)\n    plot_waveform_with_segments(wav, 16000, speech_segs, title=f\"VAD: {wav_paths[0].name}\")\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"84d76125","cell_type":"markdown","source":"## 8) Sliding windows inside speech segments\nCreates overlapping windows `(ws, we)` to extract speaker embeddings.\n","metadata":{}},{"id":"2bd0369e","cell_type":"code","source":"def make_windows(segments, win_sec=1.5, hop_sec=0.75):\n    \"\"\"Create sliding windows (ws,we) inside each speech segment.\"\"\"\n    windows = []\n    for s, e in segments:\n        dur = e - s\n        if dur <= win_sec:\n            windows.append((s, e))\n            continue\n        t = s\n        while t + win_sec <= e + 1e-6:\n            windows.append((t, t + win_sec))\n            t += hop_sec\n        if windows and windows[-1][1] < e:\n            windows.append((max(s, e - win_sec), e))\n\n    # de-dup (rounding is only for identity keys)\n    out, seen = [], set()\n    for ws, we in windows:\n        key = (round(ws, 3), round(we, 3))\n        if key not in seen:\n            out.append((float(ws), float(we)))\n            seen.add(key)\n    return out","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"5148975f","cell_type":"markdown","source":"## 9) Load speaker embedding model (SpeechBrain ECAPA-TDNN)\nLoads a pretrained **ECAPA-TDNN speaker recognition model** (`spkrec-ecapa-voxceleb`) to convert speech audio into fixed-dimensional speaker embeddings for clustering.\n","metadata":{}},{"id":"46567787","cell_type":"code","source":"from speechbrain.pretrained import EncoderClassifier\n\nspk_model = EncoderClassifier.from_hparams(\n    source=\"speechbrain/spkrec-ecapa-voxceleb\",\n    run_opts={\"device\": DEVICE},\n)\n\"ECAPA model loaded\" ","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"3dcec382","cell_type":"markdown","source":"## 10) Speaker embedding extraction (ECAPA-TDNN)\nComputes a **fixed-dimensional speaker embedding** from a **mono 16 kHz** waveform using the pretrained ECAPA model in `torch.inference_mode()` (no gradients) and returns a `(D,)` `float32` NumPy vector for clustering.\n","metadata":{}},{"id":"0d259175","cell_type":"code","source":"@torch.inference_mode()\ndef ecapa_embedding(wav16k: np.ndarray) -> np.ndarray:\n    \"\"\"\n    wav16k: mono float32, 16kHz\n    returns: (D,) numpy embedding\n    \"\"\"\n    x = torch.from_numpy(wav16k).unsqueeze(0).to(DEVICE)\n    emb = spk_model.encode_batch(x)\n    return emb.squeeze().detach().cpu().numpy().astype(np.float32)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"3a6c23b3","cell_type":"markdown","source":"## 11) Estimate number of speakers (silhouette on cosine distance)\nTries `k = 2..max_k` using agglomerative clustering on **L2-normalized embeddings** and selects the `k` with the best **silhouette score** (cosine metric) as a rough speaker-count estimate.\n","metadata":{}},{"id":"9e143114","cell_type":"code","source":"def estimate_num_speakers(embs: np.ndarray, max_k: int = 6) -> int:\n    \"\"\"Rough speaker count estimation using silhouette score on cosine distance.\"\"\"\n    n = len(embs)\n    if n < 3:\n        return 1\n    max_k = min(max_k, n)\n    if max_k <= 1:\n        return 1\n\n    X = normalize(embs)\n\n    best_k = 1\n    best_score = -1.0\n    for k in range(2, max_k + 1):\n        try:\n            labels = AgglomerativeClustering(\n                n_clusters=k, metric=\"cosine\", linkage=\"average\"\n            ).fit_predict(X)\n            if len(set(labels)) < 2:\n                continue\n            score = silhouette_score(X, labels, metric=\"cosine\")\n            if score > best_score:\n                best_score = score\n                best_k = k\n        except Exception:\n            continue\n\n    return best_k","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"d4018cd9","cell_type":"markdown","source":"## 12) Cluster embeddings into speaker labels\nClusters the (normalized) window-level speaker embeddings into `n_speakers` groups using **agglomerative clustering** with **cosine distance** and **average linkage**, returning an integer label per window.\n","metadata":{}},{"id":"4b38d26b","cell_type":"code","source":"def cluster_embeddings(embs: np.ndarray, n_speakers: int) -> np.ndarray:\n    if len(embs) == 0:\n        return np.array([], dtype=int)\n    if n_speakers <= 1 or len(embs) < 2:\n        return np.zeros(len(embs), dtype=int)\n    X = normalize(embs)\n    labels = AgglomerativeClustering(\n        n_clusters=n_speakers, metric=\"cosine\", linkage=\"average\"\n    ).fit_predict(X)\n    return labels.astype(int)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"9d4af944-e662-4ad1-9c97-da5169cfe36c","cell_type":"markdown","source":"### 12.1) Visual check: embedding clusters (PCA to 2D)\nProjects speaker embeddings to 2D to see whether predicted speakers form separable clusters.\n","metadata":{}},{"id":"6c2c3e95-5812-4f50-bd02-85759fbaff57","cell_type":"code","source":"from sklearn.decomposition import PCA\n\ndef plot_embedding_pca(embs, labels, title=\"Embedding PCA\"):\n    if len(embs) < 2:\n        print(\"Not enough embeddings to plot.\")\n        return\n    X = normalize(embs)\n    Z = PCA(n_components=2).fit_transform(X)\n\n    plt.figure(figsize=(6, 5))\n    plt.scatter(Z[:, 0], Z[:, 1], c=labels, s=18)\n    plt.title(title)\n    plt.xlabel(\"PC1\")\n    plt.ylabel(\"PC2\")\n    plt.grid(True)\n    plt.show()\n\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"31d38ec8-551b-4a48-bacc-b1d8227b9e46","cell_type":"code","source":"# --- One-file debug run: compute embs + labels for visualization ---\ntest_path = wav_paths[0]\n\nwav = load_audio_mono_16k(test_path, 16000)\nspeech_segs = silero_vad_speech_segments(wav, sr=16000)\nwindows = make_windows(speech_segs, win_sec=1.5, hop_sec=0.75)\n\nembs = []\nvalid_windows = []\nfor ws, we in windows:\n    s = int(ws * 16000)\n    e = int(we * 16000)\n    chunk = wav[s:e]\n    if len(chunk) < int(0.4 * 16000):\n        continue\n    embs.append(ecapa_embedding(chunk))\n    valid_windows.append((ws, we))\n\nembs = np.vstack(embs) if len(embs) else np.empty((0, 192), dtype=np.float32)  # 192 is common for ECAPA\n\nn_spk = estimate_num_speakers(embs, max_k=6) if len(embs) else 1\nlabels = cluster_embeddings(embs, n_speakers=n_spk) if len(embs) else np.array([], dtype=int)\n\nlen(embs), len(labels), n_spk\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"b0dbdd33-f3f1-4f59-8847-f9ab1123fa47","cell_type":"code","source":"plot_embedding_pca(embs, labels, title=\"ECAPA embeddings clustered\")\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"b612934f","cell_type":"markdown","source":"## 13) Convert labeled windows into speaker turns\nMerges adjacent/overlapping windows that share the same speaker label into continuous **speaker turns** `(start_sec, end_sec, speaker_label)`, filtering very short turns and optionally bridging small same-speaker gaps.\n","metadata":{}},{"id":"b9eb3af2","cell_type":"code","source":"def windows_to_turns(windows, labels, min_turn_sec=0.2):\n    \"\"\"Merge adjacent/overlapping same-speaker windows into turns.\"\"\"\n    if not windows:\n        return []\n\n    items = sorted(zip(windows, labels), key=lambda x: x[0][0])\n    cur_s, cur_e = items[0][0]\n    cur_lab = int(items[0][1])\n\n    turns = []\n    for (ws, we), lab in items[1:]:\n        lab = int(lab)\n        if lab == cur_lab and ws <= cur_e + 1e-3:\n            cur_e = max(cur_e, we)\n        else:\n            if cur_e - cur_s >= min_turn_sec:\n                turns.append((cur_s, cur_e, cur_lab))\n            cur_s, cur_e, cur_lab = ws, we, lab\n\n    if cur_e - cur_s >= min_turn_sec:\n        turns.append((cur_s, cur_e, cur_lab))\n\n    # Merge small gaps for same speaker\n    merged = []\n    for s, e, lab in turns:\n        if not merged:\n            merged.append([s, e, lab])\n        else:\n            ps, pe, pl = merged[-1]\n            if lab == pl and s <= pe + 0.25:\n                merged[-1][1] = max(pe, e)\n            else:\n                merged.append([s, e, lab])\n\n    return [(float(s), float(e), int(lab)) for s, e, lab in merged]","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"f58b4843","cell_type":"markdown","source":"## 14) End-to-end diarization pipeline (ML VAD → embeddings → clustering → turns)\nRuns the full diarization workflow on one audio file: **load + resample to 16 kHz**, detect **speech segments** using Silero VAD, create **overlapping windows**, extract **ECAPA speaker embeddings**, estimate **speaker count**, **cluster** windows, merge them into **speaker turns**, and output a list of `{start_time, end_time, speaker_id}` labeled as `SPEAKER_1..N`.\n","metadata":{}},{"id":"4a6f4f1b","cell_type":"code","source":"def diarize_file(\n    audio_path: str | Path,\n    # --- ML VAD knobs ---\n    vad_threshold: float = 0.5,\n    padding_ms: int = 300,\n    min_silence_ms: int = 200,\n    min_speech_ms: int = 300,\n    # --- windowing knobs ---\n    win_sec: float = 1.5,\n    hop_sec: float = 0.75,\n    # --- clustering knobs ---\n    max_speakers: int = 6,\n):\n    \"\"\"\n    Returns list[dict] with keys: start_time, end_time, speaker_id\n    \"\"\"\n    wav = load_audio_mono_16k(audio_path, 16000)\n\n    speech_segs = silero_vad_speech_segments(\n        wav,\n        sr=16000,\n        vad_threshold=vad_threshold,\n        padding_ms=padding_ms,\n        min_silence_ms=min_silence_ms,\n        min_speech_ms=min_speech_ms,\n    )\n    if len(speech_segs) == 0:\n        return []\n\n    windows = make_windows(speech_segs, win_sec=win_sec, hop_sec=hop_sec)\n\n    # Embeddings per window\n    embs = []\n    valid_windows = []\n    for ws, we in windows:\n        s = int(ws * 16000)\n        e = int(we * 16000)\n        chunk = wav[s:e]\n        if len(chunk) < int(0.4 * 16000):  # drop too-short\n            continue\n        embs.append(ecapa_embedding(chunk))\n        valid_windows.append((ws, we))\n\n    if len(embs) == 0:\n        return []\n\n    embs = np.vstack(embs)\n\n    # Speaker count + clustering\n    n_spk = estimate_num_speakers(embs, max_k=max_speakers)\n    labels = cluster_embeddings(embs, n_speakers=n_spk)\n\n    turns = windows_to_turns(valid_windows, labels)\n\n    # Map cluster labels to SPEAKER_1.. in order of first appearance\n    label_map = {}\n    next_id = 1\n    out = []\n    for s, e, lab in turns:\n        if lab not in label_map:\n            label_map[lab] = f\"SPEAKER_{next_id}\"\n            next_id += 1\n        out.append({\n            \"start_time\": round(s, 2),\n            \"end_time\": round(e, 2),\n            \"speaker_id\": label_map[lab],\n        })\n\n    return out","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"44e7807e-5911-445e-8b1f-96558e595b59","cell_type":"markdown","source":"### 14.1) Visual check: diarization timeline (speaker turns)\nPlots the final speaker turns over time to verify speaker switches and boundary smoothness.","metadata":{}},{"id":"8fd40fa8-ce24-4d51-a809-bd2c6d58ee36","cell_type":"code","source":"def plot_diarization_turns(turns, title=\"Diarization timeline\"):\n    if not turns:\n        print(\"No turns to plot.\")\n        return\n    speakers = sorted({t[\"speaker_id\"] for t in turns})\n    spk_to_y = {s: i for i, s in enumerate(speakers)}\n\n    plt.figure(figsize=(12, 1 + 0.45 * len(speakers)))\n    for t in turns:\n        y = spk_to_y[t[\"speaker_id\"]]\n        plt.plot([t[\"start_time\"], t[\"end_time\"]], [y, y], linewidth=6)\n    plt.yticks(range(len(speakers)), speakers)\n    plt.xlabel(\"Time (s)\")\n    plt.title(title)\n    plt.grid(True, axis=\"x\")\n    plt.show()\n\nif wav_paths:\n    diar = diarize_file(wav_paths[0])\n    plot_diarization_turns(diar, title=f\"Diarization: {wav_paths[0].name}\")\n","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"ec4744bd","cell_type":"markdown","source":"## 15) Batch diarization + CSV export\nRuns `diarize_file()` over all audio files with a progress bar, serializes each diarization result as JSON per `filename`, and writes a submission-ready CSV to `OUT_PATH` (then previews the first rows).\n","metadata":{}},{"id":"83bc4b77","cell_type":"code","source":"pred_rows = []\nfor stem, wav_path in tqdm(list(zip(stems, wav_paths)), total=len(wav_paths), desc=\"Diarizing\"):\n    diar = diarize_file(wav_path)\n    pred_rows.append({\n        \"filename\": stem,\n        \"diarization\": json.dumps(diar, ensure_ascii=False),\n    })\n\nsub_df = pd.DataFrame(pred_rows)\nsub_df.to_csv(OUT_PATH, index=False)\n\nOUT_PATH, sub_df.head()","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}