{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.11","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":4477,"databundleVersionId":44220,"sourceType":"competition"}],"dockerImageVersionId":31041,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"***FUNDAMENTALS OF BIOMEDICAL SIGNAL PROCESSING FINAL PROJECT***\n\n\n","metadata":{}},{"cell_type":"markdown","source":"Mehmet Umut ASLAN - 2106102013\n\nHüseyin Onur TAŞTAN - 2006102015\n\nArjen GÖKDEMİR - 2006102031","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import Dataset, DataLoader\nfrom pathlib import Path\nfrom zipfile import ZipFile\nfrom tqdm import tqdm\nimport matplotlib.pyplot as plt\nimport warnings\nwarnings.filterwarnings('ignore')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T17:04:20.007452Z","iopub.execute_input":"2025-06-20T17:04:20.007648Z","iopub.status.idle":"2025-06-20T17:04:24.563935Z","shell.execute_reply.started":"2025-06-20T17:04:20.007631Z","shell.execute_reply":"2025-06-20T17:04:24.563261Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Data Loading and Preparation**\n\n> In this part, we start by loading and preparing the Grasp-and-Lift EEG dataset. We first extract the files, read the data, and then split it into training and validation sets. This way, we make sure our data is ready and reliable for the next steps of our analysis.","metadata":{}},{"cell_type":"code","source":"print(\"📂 Veri yükleniyor...\")\n\nlabels = ['HandStart', 'FirstDigitTouch', 'BothStartLoadPhase', 'LiftOff', 'Replace', 'BothReleased']\n\nzipdir = Path.cwd().parent / 'input' / 'grasp-and-lift-eeg-detection'\nfor zipfile in zipdir.glob('*.zip'):\n    with ZipFile(zipfile, 'r') as zf:\n        zf.extractall()\n\nxs, ys = [], []\ntraindir = Path('train')\n\nfor datapath in tqdm(sorted(traindir.glob('*_data.csv')), desc=\"Train data loading\"):\n    eventpath = datapath.parent / (datapath.stem[:-5] + '_events.csv')\n    \n    x = pd.read_csv(datapath).iloc[:, 1:].values.astype(np.float32)\n    y = pd.read_csv(eventpath).iloc[:, 1:].values.astype(np.float32)\n    \n    xs.append(x)\n    ys.append(y)\n\nxs_train, ys_train = xs[:-2], ys[:-2]\nxs_valid, ys_valid = xs[-2:], ys[-2:]\n\nprint(f\"✅ Train: {len(xs_train)} subjects, Valid: {len(xs_valid)} subjects\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T17:04:24.565403Z","iopub.execute_input":"2025-06-20T17:04:24.566086Z","iopub.status.idle":"2025-06-20T17:05:51.878095Z","shell.execute_reply.started":"2025-06-20T17:04:24.566065Z","shell.execute_reply":"2025-06-20T17:05:51.877235Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Preprocessing EEG Data**\n\n> Here, we preprocess our EEG signals to make them more suitable for the model. First, we normalize the data so that all channels have similar scales. Then, we add extra features: the difference between consecutive time steps (to capture signal changes) and a moving average (to smooth out the signal a bit).\nWe apply this process to both our training and validation sets so that everything is consistent.","metadata":{}},{"cell_type":"code","source":"def normalize_eeg(x):\n    \n    return (x - np.mean(x, axis=0, keepdims=True)) / (np.std(x, axis=0, keepdims=True) + 1e-8)\n\ndef add_features(x):\n    \n    diff = np.diff(x, axis=1)\n    diff = np.concatenate([diff, diff[:, -1:]], axis=1)  \n    \n    \n    window = 5\n    ma = np.convolve(x.flatten(), np.ones(window)/window, 'same').reshape(x.shape)\n    \n    return np.concatenate([x, diff, ma], axis=1)\n\n\nprint(\"🔧 Veri ön işleme...\")\nfor i in range(len(xs_train)):\n    xs_train[i] = add_features(normalize_eeg(xs_train[i]))\nfor i in range(len(xs_valid)):\n    xs_valid[i] = add_features(normalize_eeg(xs_valid[i]))\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T17:05:51.879142Z","iopub.execute_input":"2025-06-20T17:05:51.879447Z","iopub.status.idle":"2025-06-20T17:06:08.265002Z","shell.execute_reply.started":"2025-06-20T17:05:51.879419Z","shell.execute_reply":"2025-06-20T17:06:08.264428Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**EEG Dataset Class**\n\n\n> Here, we define a custom dataset class to prepare our EEG data for training and evaluation.We split the long EEG recordings into smaller overlapping windows, which helps the model learn better from shorter, manageable sequences. During training, we also apply simple data augmentation like adding noise or randomly shifting the signal to make the model more robust.This class handles both training and validation modes, and makes sure each batch of data is formatted correctly for the model.\n","metadata":{}},{"cell_type":"code","source":"class EEGDataset(Dataset):\n    def __init__(self, xs, ys=None, window_size=1000, stride=500, mode='train'):\n        self.xs = xs\n        self.ys = ys\n        self.window_size = window_size\n        self.stride = stride\n        self.mode = mode\n        \n        self.samples = []\n        for subj_idx, x in enumerate(xs):\n            seq_len = x.shape[0]\n            if mode == 'train':\n                for start in range(0, seq_len - window_size + 1, stride):\n                    self.samples.append((subj_idx, start, start + window_size))\n            else:\n                self.samples.append((subj_idx, 0, seq_len))\n    \n    def __len__(self):\n        return len(self.samples)\n    \n    def __getitem__(self, idx):\n        subj_idx, start, end = self.samples[idx]\n        x = self.xs[subj_idx][start:end]\n        \n        if self.mode == 'train':\n            if np.random.random() > 0.5:\n                x = x + np.random.normal(0, 0.01, x.shape)\n            if np.random.random() > 0.5:\n                shift = np.random.randint(-10, 11)\n                x = np.roll(x, shift, axis=0)\n            \n            y = self.ys[subj_idx][start:end]\n            return torch.tensor(x.T, dtype=torch.float32), torch.tensor(y.T, dtype=torch.float32)\n        \n        else:\n            if self.ys is not None:\n                y = self.ys[subj_idx]\n                return torch.tensor(x.T, dtype=torch.float32), torch.tensor(y.T, dtype=torch.float32)\n            else:\n                return torch.tensor(x.T, dtype=torch.float32)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T17:06:08.265844Z","iopub.execute_input":"2025-06-20T17:06:08.266165Z","iopub.status.idle":"2025-06-20T17:06:08.274890Z","shell.execute_reply.started":"2025-06-20T17:06:08.266137Z","shell.execute_reply":"2025-06-20T17:06:08.274228Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Model Design, Training & Evaluation**\n\nIn this section, we design, train, and evaluate our deep learning model.\n\n    \n> Model Architecture:    We build a custom EEG model combining convolutional layers, residual blocks, multi-scale attention, and a bidirectional LSTM. This setup helps us capture both local patterns and long-range dependencies in the EEG signals.\n>\n> Loss Function:    We use Focal Loss, which is helpful for handling class imbalance by focusing more on hard-to-classify examples.\n> \n>  Evaluation & Visualization:    Every few epochs, we validate our model and calculate AUC scores for each class. We also draw ROC curves and plot training loss & validation AUC to better understand the learning process. (This can be seen in Training Phase)\n> \n> Best Model Selection:    We keep track of the model with the highest validation AUC and return it at the end.\n","metadata":{}},{"cell_type":"code","source":"class ResidualBlock1D(nn.Module):\n    def __init__(self, in_channels, out_channels, kernel_size=3, stride=1):\n        super().__init__()\n        self.conv1 = nn.Conv1d(in_channels, out_channels, kernel_size, stride, padding=kernel_size//2)\n        self.bn1 = nn.BatchNorm1d(out_channels)\n        self.conv2 = nn.Conv1d(out_channels, out_channels, kernel_size, 1, padding=kernel_size//2)\n        self.bn2 = nn.BatchNorm1d(out_channels)\n        \n        if in_channels != out_channels or stride != 1:\n            self.skip = nn.Sequential(\n                nn.Conv1d(in_channels, out_channels, 1, stride),\n                nn.BatchNorm1d(out_channels)\n            )\n        else:\n            self.skip = nn.Identity()\n    \n    def forward(self, x):\n        residual = self.skip(x)\n        x = F.gelu(self.bn1(self.conv1(x)))\n        x = self.bn2(self.conv2(x))\n        return F.gelu(x + residual)\n\nclass MultiScaleAttention(nn.Module):\n    def __init__(self, in_channels, seq_len):\n        super().__init__()\n        self.scales = [1, 3] \n        self.attentions = nn.ModuleList([\n            nn.Conv1d(in_channels, in_channels//2, k, padding=k//2) \n            for k in self.scales\n        ])\n        self.combine = nn.Conv1d(in_channels, in_channels, 1)\n        \n    def forward(self, x):\n        features = [att(x) for att in self.attentions]\n        combined = torch.cat(features, dim=1)\n        \n        weights = torch.softmax(self.combine(combined), dim=2)\n        return x * weights\n\nclass EEGNet(nn.Module):\n    def __init__(self, n_channels=96, n_classes=6, seq_len=750):\n        super().__init__()\n        \n        self.features = nn.Sequential(\n            nn.Conv1d(n_channels, 32, kernel_size=7, padding=3),\n            nn.BatchNorm1d(32),\n            nn.GELU(),\n            \n            ResidualBlock1D(32, 64, kernel_size=5, stride=2),\n            ResidualBlock1D(64, 128, kernel_size=3, stride=2),\n        )\n        \n        self.attention = MultiScaleAttention(128, seq_len//4)\n        \n        self.lstm = nn.LSTM(128, 64, num_layers=1, batch_first=True, \n                           bidirectional=True, dropout=0.0)\n        \n        self.classifier = nn.Sequential(\n            nn.Conv1d(128, 64, kernel_size=3, padding=1),\n            nn.BatchNorm1d(64),\n            nn.GELU(),\n            nn.Conv1d(64, n_classes, kernel_size=1)\n        )\n        \n    def forward(self, x):\n        original_seq_len = x.shape[2]\n        \n        x = self.features(x)\n        \n        \n        x = self.attention(x)\n        x_lstm = x.transpose(1, 2)  \n        x_lstm, _ = self.lstm(x_lstm)\n        x_lstm = x_lstm.transpose(1, 2)  \n        \n       \n        output = self.classifier(x_lstm)\n        \n        \n        if output.shape[2] != original_seq_len:\n            output = F.interpolate(output, size=original_seq_len, mode='linear', align_corners=False)\n        \n        return output\n\n\n\nclass FocalLoss(nn.Module):\n    \n    def __init__(self, alpha=1, gamma=2):\n        super().__init__()\n        self.alpha = alpha\n        self.gamma = gamma\n    \n    def forward(self, inputs, targets):\n        bce_loss = F.binary_cross_entropy_with_logits(inputs, targets, reduction='none')\n        pt = torch.exp(-bce_loss)\n        focal_loss = self.alpha * (1-pt)**self.gamma * bce_loss\n        return focal_loss.mean()\n\n\n\ndef train_model(model, train_loader, valid_loader, device, epochs=20):\n    \n    optimizer = torch.optim.AdamW(model.parameters(), lr=3e-3, weight_decay=0.01)  \n    scheduler = torch.optim.lr_scheduler.OneCycleLR(\n        optimizer, max_lr=3e-3, epochs=epochs, \n        steps_per_epoch=len(train_loader), pct_start=0.1  \n    )\n    \n    criterion = FocalLoss(alpha=1, gamma=2)\n    scaler = torch.cuda.amp.GradScaler()\n    \n    best_score = 0\n    best_model = None\n    \n   \n    train_losses_history = []\n    val_aucs_history = []\n    epochs_history = []\n    \n    \n    for epoch in range(epochs):\n       \n        model.train()\n        train_losses = []\n        \n        for batch_idx, (x, y) in enumerate(tqdm(train_loader, desc=f\"Epoch {epoch+1}\")):\n            x, y = x.to(device), y.to(device)\n            \n            optimizer.zero_grad()\n            \n            with torch.cuda.amp.autocast():\n                outputs = model(x)\n                loss = criterion(outputs, y)\n            \n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n            scaler.step(optimizer)\n            scaler.update()\n            scheduler.step()\n            \n            train_losses.append(loss.item())\n        \n        \n        if (epoch + 1) % 5 == 0 or epoch == epochs - 1:\n            plot_roc = (epoch == epochs - 1)  \n            val_score = validate_model(model, valid_loader, device, plot_roc=plot_roc)\n            avg_train_loss = np.mean(train_losses)\n            \n            \n            train_losses_history.append(avg_train_loss)\n            val_aucs_history.append(val_score)\n            epochs_history.append(epoch + 1)\n            \n            print(f\"Epoch {epoch+1}: Train Loss: {avg_train_loss:.4f}, Val AUC: {val_score:.4f}\")\n            \n            if val_score > best_score:\n                best_score = val_score\n                best_model = model.state_dict().copy()\n                print(f\"💯 New best validation AUC: {best_score:.4f}\")\n    \n   \n    plot_training_history(epochs_history, train_losses_history, val_aucs_history)\n    \n    \n    if best_model is not None:\n        model.load_state_dict(best_model)\n    \n    return model, best_score\n\ndef plot_training_history(epochs, train_losses, val_aucs):\n    \n    plt.figure(figsize=(15, 5))\n    \n    \n    plt.subplot(1, 3, 1)\n    plt.plot(epochs, train_losses, 'b-o', linewidth=2, markersize=6)\n    plt.title('Training Loss Over Time', fontsize=14, fontweight='bold')\n    plt.xlabel('Epoch')\n    plt.ylabel('Loss')\n    plt.grid(True, alpha=0.3)\n    plt.yscale('log') \n    \n    \n    plt.subplot(1, 3, 2)\n    plt.plot(epochs, val_aucs, 'r-o', linewidth=2, markersize=6)\n    plt.title('Validation AUC Over Time', fontsize=14, fontweight='bold')\n    plt.xlabel('Epoch')\n    plt.ylabel('AUC')\n    plt.grid(True, alpha=0.3)\n    plt.ylim([0, 1])  \n    \n    \n    plt.subplot(1, 3, 3)\n    ax1 = plt.gca()\n    ax2 = ax1.twinx()\n    \n    line1 = ax1.plot(epochs, train_losses, 'b-o', linewidth=2, markersize=6, label='Train Loss')\n    line2 = ax2.plot(epochs, val_aucs, 'r-o', linewidth=2, markersize=6, label='Val AUC')\n    \n    ax1.set_xlabel('Epoch')\n    ax1.set_ylabel('Loss', color='b')\n    ax2.set_ylabel('AUC', color='r')\n    ax1.set_yscale('log')\n    ax2.set_ylim([0, 1])\n    \n    \n    lines = line1 + line2\n    labels = [l.get_label() for l in lines]\n    ax1.legend(lines, labels, loc='center right')\n    \n    plt.title('Training Progress', fontsize=14, fontweight='bold')\n    plt.grid(True, alpha=0.3)\n    \n    plt.tight_layout()\n    plt.show()\n    \n    \n    print(f\"\\n📈 Training Summary:\")\n    print(f\"   Initial Loss: {train_losses[0]:.4f}\")\n    print(f\"   Final Loss: {train_losses[-1]:.4f}\")\n    print(f\"   Loss Reduction: {((train_losses[0] - train_losses[-1]) / train_losses[0] * 100):.1f}%\")\n    print(f\"   Initial AUC: {val_aucs[0]:.4f}\")\n    print(f\"   Final AUC: {val_aucs[-1]:.4f}\")\n    print(f\"   Best AUC: {max(val_aucs):.4f}\")\n    print(f\"   AUC Improvement: {((max(val_aucs) - val_aucs[0]) / val_aucs[0] * 100 if val_aucs[0] > 0 else 0):.1f}%\")\n\ndef validate_model(model, valid_loader, device, plot_roc=True):\n    \n    from sklearn.metrics import roc_auc_score, roc_curve\n    import matplotlib.pyplot as plt\n    \n    model.eval()\n    all_preds, all_targets = [], []\n    \n    with torch.no_grad():\n        for x, y in valid_loader:\n            x = x.to(device)\n            \n            \n            seq_len = x.shape[2]\n            window_size = 1000\n            stride = 500\n            \n            if seq_len <= window_size:\n                with torch.cuda.amp.autocast():\n                    outputs = torch.sigmoid(model(x))\n                preds = outputs.cpu().numpy()[0]  \n                targets = y.numpy()[0]  \n            else:\n                \n                pred_np = np.zeros((6, seq_len))\n                counts = np.zeros(seq_len)\n                \n                for start in range(0, seq_len, stride):\n                    end = min(start + window_size, seq_len)\n                    actual_len = end - start\n                    x_chunk = x[:, :, start:end]\n                    \n                    if x_chunk.shape[2] < window_size:\n                        \n                        pad_size = window_size - x_chunk.shape[2]\n                        x_chunk = F.pad(x_chunk, (0, pad_size))\n                    \n                    with torch.cuda.amp.autocast():\n                        chunk_pred = torch.sigmoid(model(x_chunk))\n                    \n                    \n                    chunk_pred = chunk_pred[0, :, :actual_len].cpu().numpy()  # (n_classes, actual_len)\n                    \n                    \n                    pred_np[:, start:end] += chunk_pred\n                    counts[start:end] += 1\n                \n                \n                counts[counts == 0] = 1  \n                pred_np = pred_np / counts[np.newaxis, :]\n                preds = pred_np\n                targets = y.numpy()[0]  \n            \n            all_preds.append(preds)\n            all_targets.append(targets)\n    \n    all_preds = np.concatenate(all_preds, axis=1) \n    all_targets = np.concatenate(all_targets, axis=1)\n    \n    if all_preds.shape != all_targets.shape:\n        print(f\"❌ Shape mismatch: preds {all_preds.shape} vs targets {all_targets.shape}\")\n        return 0.0\n    \n    try:\n        auc_scores = []\n        \n        if plot_roc:\n            plt.figure(figsize=(15, 10))\n            \n        for i in range(6):\n            n_positives = np.sum(all_targets[i])\n            n_negatives = len(all_targets[i]) - n_positives\n            \n            if n_positives > 0 and n_negatives > 0:\n                pred_class = all_preds[i]\n                target_class = all_targets[i]\n                \n                if len(pred_class) != len(target_class):\n                    min_len = min(len(pred_class), len(target_class))\n                    pred_class = pred_class[:min_len]\n                    target_class = target_class[:min_len]\n                \n                auc = roc_auc_score(target_class, pred_class)\n                auc_scores.append(auc)\n                \n                if plot_roc:\n                    fpr, tpr, _ = roc_curve(target_class, pred_class)\n                    plt.subplot(2, 3, i+1)\n                    plt.plot(fpr, tpr, linewidth=2, label=f'ROC (AUC = {auc:.3f})')\n                    plt.plot([0, 1], [0, 1], 'k--', alpha=0.5)\n                    plt.xlim([0.0, 1.0])\n                    plt.ylim([0.0, 1.05])\n                    plt.xlabel('False Positive Rate')\n                    plt.ylabel('True Positive Rate')\n                    plt.title(f'{labels[i]}')\n                    plt.legend(loc=\"lower right\")\n                    plt.grid(True, alpha=0.3)\n            else:\n                auc_scores.append(0.0)\n        \n        if plot_roc and any(score > 0 for score in auc_scores):\n            plt.tight_layout()\n            plt.suptitle('ROC Curves for All Classes', fontsize=16, y=1.02)\n            plt.show()\n        \n        mean_auc = np.mean(auc_scores) if auc_scores else 0.0\n        print(f\"🏆 Mean AUC: {mean_auc:.4f}\")\n        print(f\"📊 Individual AUCs: {[f'{score:.4f}' for score in auc_scores]}\")\n        \n        \n        if plot_roc and any(score > 0 for score in auc_scores):\n            plot_auc_comparison(auc_scores, labels)\n        \n        return mean_auc\n    except Exception as e:\n        print(f\"❌ AUC calculation error: {e}\")\n        import traceback\n        traceback.print_exc()\n        return 0.0\n\ndef plot_auc_comparison(auc_scores, class_labels):\n    plt.figure(figsize=(12, 6))\n    \n   \n    colors = ['#FF6B6B', '#4ECDC4', '#45B7D1', '#96CEB4', '#FFEAA7', '#DDA0DD']\n    bars = plt.bar(range(len(auc_scores)), auc_scores, \n                   color=colors, alpha=0.8, edgecolor='black', linewidth=1)\n    \n    \n    for i, (bar, score) in enumerate(zip(bars, auc_scores)):\n        plt.text(bar.get_x() + bar.get_width()/2, bar.get_height() + 0.01,\n                f'{score:.3f}', ha='center', va='bottom', \n                fontweight='bold', fontsize=11)\n    \n    plt.axhline(y=0.5, color='red', linestyle='--', alpha=0.7, \n                label='Random Guess (0.5)')\n    \n    mean_auc = np.mean(auc_scores)\n    plt.axhline(y=mean_auc, color='green', linestyle='-', alpha=0.7, \n                label=f'Mean AUC ({mean_auc:.3f})')\n    plt.xlabel('EEG Event Classes', fontweight='bold')\n    plt.ylabel('AUC Score', fontweight='bold')\n    plt.title('AUC Scores by Event Class', fontsize=16, fontweight='bold')\n    plt.xticks(range(len(class_labels)), class_labels, rotation=45, ha='right')\n    plt.ylim(0, 1)\n    plt.grid(True, alpha=0.3, axis='y')\n    plt.legend()\n    \n    plt.tight_layout()\n    plt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T17:06:08.275709Z","iopub.execute_input":"2025-06-20T17:06:08.276158Z","iopub.status.idle":"2025-06-20T17:06:08.313232Z","shell.execute_reply.started":"2025-06-20T17:06:08.276139Z","shell.execute_reply":"2025-06-20T17:06:08.312621Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Training Phase**\n\n> In this section, we train our model using a sliding window approach. For each window size, we create the training and validation datasets, initialize the model, and move it to the appropriate device (GPU if available). If there's a previous model, we try to transfer its learned weights to the new one to improve training. Then, we train the model and store the best version based on validation performance.","metadata":{}},{"cell_type":"code","source":"device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nprint(f\"🚀 Device: {device}\")\n\n\nwindow_sizes = [750]  \nmodels = []\n\nfor i, window_size in enumerate(window_sizes):\n    print(f\"\\n🎯 Training Stage {i+1}: Window Size = {window_size}\")\n    \n    \n    train_dataset = EEGDataset(xs_train, ys_train, window_size=window_size, stride=window_size//2, mode='train')\n    valid_dataset = EEGDataset(xs_valid, ys_valid, window_size=window_size, mode='valid')\n    \n    train_loader = DataLoader(train_dataset, batch_size=96, shuffle=True, num_workers=1)  # Daha büyük batch\n    valid_loader = DataLoader(valid_dataset, batch_size=1, shuffle=False, num_workers=1)\n     \n    model = EEGNet(n_channels=96, n_classes=6, seq_len=window_size).to(device)\n    \n    \n    if i > 0 and models:\n        try:\n            \n            prev_state = models[-1].state_dict()\n            current_state = model.state_dict()\n            \n            for key in current_state:\n                if key in prev_state and current_state[key].shape == prev_state[key].shape:\n                    current_state[key] = prev_state[key]\n            model.load_state_dict(current_state)\n            print(\"📦 Transferred weights from previous stage\")\n        except:\n            print(\"⚠️ Could not transfer weights, training from scratch\")\n    \n    epochs = 20  \n    model, score = train_model(model, train_loader, valid_loader, device, epochs)\n    models.append(model)\n    \n    print(f\"✅ Stage {i+1} completed with AUC: {score:.4f}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T17:06:08.314687Z","iopub.execute_input":"2025-06-20T17:06:08.314875Z","iopub.status.idle":"2025-06-20T17:34:21.615863Z","shell.execute_reply.started":"2025-06-20T17:06:08.314861Z","shell.execute_reply":"2025-06-20T17:34:21.615026Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Test Phase**\n\n> In this part before final, we make predictions on the test data. First, we load and preprocess all the test files. Then, we pass the data through each trained model to generate predictions. If the EEG sequence is long, we use a sliding window to make partial predictions and then average them. Finally, we combine (ensemble) the outputs from all models by averaging them to get the final prediction results.","metadata":{}},{"cell_type":"code","source":"print(\"\\n🔮 Test predictions...\")\n\n\nxs_test = []\nlengths = {}\ntestdir = Path('test')\n\nfor subj in range(1, 13):\n    for series in [9, 10]:\n        datapath = testdir / f'subj{subj}_series{series}_data.csv'\n        x = pd.read_csv(datapath).iloc[:, 1:].values.astype(np.float32)\n        x = add_features(normalize_eeg(x))\n        xs_test.append(x)\n        lengths[f'{subj}_{series}'] = x.shape[0]\n\n\ntest_dataset = EEGDataset(xs_test, mode='test')\ntest_loader = DataLoader(test_dataset, batch_size=1, shuffle=False, num_workers=1)\n\n\nprint(\"🎪 Ensemble prediction...\")\nall_predictions = []\n\nfor model_idx, model in enumerate(models):\n    model.eval()\n    predictions = []\n    \n    with torch.no_grad():\n        for test_idx, x in enumerate(tqdm(test_loader, desc=f\"Model {model_idx+1} predicting\")):\n            x = x.to(device)\n            \n            seq_len = x.shape[2]\n            window_size = 1000\n            stride = 500\n            \n            if seq_len <= window_size:\n                with torch.cuda.amp.autocast():\n                    pred = torch.sigmoid(model(x))\n                pred_np = pred.cpu().numpy()[0].T  \n            else:\n                preds_list = []\n                positions = []\n                \n                for start in range(0, seq_len, stride):\n                    end = min(start + window_size, seq_len)\n                    actual_len = end - start\n                    \n                    x_chunk = x[:, :, start:end]\n                    \n                    if x_chunk.shape[2] < window_size:\n                        pad_size = window_size - x_chunk.shape[2]\n                        x_chunk = F.pad(x_chunk, (0, pad_size))\n                    \n                    with torch.cuda.amp.autocast():\n                        chunk_pred = torch.sigmoid(model(x_chunk))\n                    \n                    chunk_pred = chunk_pred[:, :, :actual_len]\n                    preds_list.append(chunk_pred.cpu().numpy()[0].T)  # (actual_len, 6)\n                    positions.append((start, end))\n                \n                pred_np = np.zeros((seq_len, 6))\n                counts = np.zeros(seq_len)\n                \n                for pred_chunk, (start, end) in zip(preds_list, positions):\n                    pred_np[start:end] += pred_chunk\n                    counts[start:end] += 1\n                \n                counts[counts == 0] = 1  # Division by zero'yu önle\n                pred_np = pred_np / counts[:, np.newaxis]\n            \n            predictions.append(pred_np)\n    \n    all_test_preds = np.concatenate(predictions, axis=0)\n    all_predictions.append(all_test_preds)\n    print(f\"Model {model_idx+1} predictions shape: {all_test_preds.shape}\")\n\nfinal_predictions = np.mean(all_predictions, axis=0)\nprint(f\"Final predictions shape: {final_predictions.shape}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T17:34:21.617047Z","iopub.execute_input":"2025-06-20T17:34:21.617519Z","iopub.status.idle":"2025-06-20T17:35:00.520210Z","shell.execute_reply.started":"2025-06-20T17:34:21.617483Z","shell.execute_reply":"2025-06-20T17:35:00.519192Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Submission Part**\n> In this final step, we prepare our submission file. We assign unique IDs to each prediction and save the results in the required CSV format. After that, we examine the prediction distributions for each class by printing summary statistics and plotting histograms. This helps us verify that the predictions look reasonable before submitting them.","metadata":{}},{"cell_type":"code","source":"print(\"📝 Creating submission...\")\n\nindices = []\nfor sbj in range(1, 13):\n    for series in [9, 10]:\n        for t in range(lengths[f'{sbj}_{series}']):\n            indices.append(f'subj{sbj}_series{series}_{t}')\n\nprint(f\"Number of indices: {len(indices)}\")\nprint(f\"Final predictions shape: {final_predictions.shape}\")\n\nsubmission = pd.DataFrame(\n    final_predictions,\n    index=indices,\n    columns=labels\n)\n\nsubmission.to_csv('submission.csv', index_label='id', float_format='%.6f')\n\nprint(\"✅ Submission saved!\")\nprint(f\"📊 Submission shape: {submission.shape}\")\nprint(f\"📈 Prediction range: [{submission.values.min():.4f}, {submission.values.max():.4f}]\")\n\nprint(\"\\n📊 Final Prediction Statistics per Class:\")\nfor i, label_name in enumerate(labels):\n    class_preds = submission.iloc[:, i].values\n    print(f\"{label_name}:\")\n    print(f\"  Mean: {np.mean(class_preds):.4f}\")\n    print(f\"  Std: {np.std(class_preds):.4f}\")\n    print(f\"  Min: {np.min(class_preds):.4f}\")\n    print(f\"  Max: {np.max(class_preds):.4f}\")\n    print(f\"  Median: {np.median(class_preds):.4f}\")\n\nplt.figure(figsize=(15, 10))\nfor i, label_name in enumerate(labels):\n    plt.subplot(2, 3, i+1)\n    plt.hist(submission.iloc[:, i].values, bins=50, alpha=0.7, density=True)\n    plt.title(f'{label_name} Prediction Distribution')\n    plt.xlabel('Prediction Value')\n    plt.ylabel('Density')\n    plt.grid(True, alpha=0.3)\n\nplt.tight_layout()\nplt.suptitle('Final Prediction Distributions', fontsize=16, y=1.02)\nplt.show()\n\n\nsubmission.head()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-06-20T17:35:00.522826Z","iopub.execute_input":"2025-06-20T17:35:00.523088Z","iopub.status.idle":"2025-06-20T17:35:30.253566Z","shell.execute_reply.started":"2025-06-20T17:35:00.523061Z","shell.execute_reply":"2025-06-20T17:35:30.252749Z"}},"outputs":[],"execution_count":null}]}