{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# TP3 - Deep Learning Models\n\nIn the previous notebooks, we used classic machine learning models based on features computed on the data. Now, we will use Deep Learning to perform sleep staging directly on the raw signals. \n\nIn this tutorial we will use pytorch: https://pytorch.org/","metadata":{"id":"UUv9Dfir60vr"}},{"cell_type":"code","source":"cd \"/kaggle/input/dreem-automated-sleep-staging/\"","metadata":{"execution":{"iopub.status.busy":"2022-10-04T16:16:45.587532Z","iopub.execute_input":"2022-10-04T16:16:45.588028Z","iopub.status.idle":"2022-10-04T16:16:45.598312Z","shell.execute_reply.started":"2022-10-04T16:16:45.587991Z","shell.execute_reply":"2022-10-04T16:16:45.596763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nimport os\nimport numpy as np\nimport random as rd","metadata":{"execution":{"iopub.status.busy":"2022-10-04T16:16:45.601118Z","iopub.execute_input":"2022-10-04T16:16:45.602124Z","iopub.status.idle":"2022-10-04T16:16:45.612143Z","shell.execute_reply.started":"2022-10-04T16:16:45.602077Z","shell.execute_reply":"2022-10-04T16:16:45.610460Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Loading the data\n\nFirst, we load the data from the first EEG channel for each record and the associated hypnogram.\n\nThen, we sample the training and validation records for the current run.","metadata":{}},{"cell_type":"code","source":"data_for_records = {}\nhypnogram_for_records = {}\nhypnograms = pd.read_csv('targets_train.csv')\nfor record in os.listdir(\"training_records\"):\n    record_number = int(record[-5])\n    x = np.load(f'training_records/{record}')\n    data_for_records[record] = x[:,1:250 * 30 + 1]\n    hypnogram_for_records[record] = list(hypnograms[hypnograms['record'] == record_number]['target'])\n    \n\n\n","metadata":{"id":"iG_vEjt_60vs","executionInfo":{"status":"ok","timestamp":1603111976603,"user_tz":-120,"elapsed":749,"user":{"displayName":"Antoine Guillot","photoUrl":"","userId":"13673594054995143914"}},"execution":{"iopub.status.busy":"2022-10-04T16:16:45.614460Z","iopub.execute_input":"2022-10-04T16:16:45.615986Z","iopub.status.idle":"2022-10-04T16:17:03.429473Z","shell.execute_reply.started":"2022-10-04T16:16:45.615912Z","shell.execute_reply":"2022-10-04T16:17:03.428115Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rd.seed(1234)\nrecords_list = list(data_for_records)\nrd.shuffle(records_list)\ntraining_record,test_records = records_list[:5],records_list[5:]\n\nprint('Training records: ',training_record)\nprint('Test records: ', test_records)","metadata":{"execution":{"iopub.status.busy":"2022-10-04T16:17:03.433606Z","iopub.execute_input":"2022-10-04T16:17:03.435159Z","iopub.status.idle":"2022-10-04T16:17:03.448124Z","shell.execute_reply.started":"2022-10-04T16:17:03.435110Z","shell.execute_reply":"2022-10-04T16:17:03.446011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def build_dataset(records, data_for_records,hypnogram_for_records):\n    X,y = [],[]\n    for record in records:\n        X.append(data_for_records[record])\n        y.extend(hypnogram_for_records[record])\n\n    return np.concatenate(X),y\n\n\nX_train,y_train = build_dataset(training_record,data_for_records,hypnogram_for_records)\nX_test,y_test = build_dataset(test_records,data_for_records,hypnogram_for_records)\n\n","metadata":{"execution":{"iopub.status.busy":"2022-10-04T16:17:03.451120Z","iopub.execute_input":"2022-10-04T16:17:03.452603Z","iopub.status.idle":"2022-10-04T16:17:03.668558Z","shell.execute_reply.started":"2022-10-04T16:17:03.452550Z","shell.execute_reply":"2022-10-04T16:17:03.666956Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Dataloader and dataset\n\nIn this TD, we will only work with one EEG channel.\nLet's create dataset functions that will be used for training and testing the model:\n\n*EegEpochDataset*: Eeg Class herited from pytorch Dataset to deal with our data\n\nThe dataloader are used by pytorch to randomly sampled elements from the datasets.","metadata":{"id":"dxnqHGci60v0"}},{"cell_type":"code","source":"\"\"\" Load project data\n    DataLoader and Dataset for single-channel EEG\n\n\"\"\"\n\nimport torch\nfrom torch.utils.data import Dataset, DataLoader\n\n\ndef normalize_data(eeg_array):\n    \"\"\"normalize signal between 0 and 1\"\"\"\n\n    normalized_array = np.clip(eeg_array, -250, 250)\n    normalized_array = normalized_array / 250\n\n    return normalized_array\n\n\nclass EegEpochDataset(Dataset):\n    \"\"\"EEG Epochs dataset.\"\"\"\n\n    def __init__(self, x_data, y_data, transform=None):\n        \"\"\"\n        Args:\n            x_data (numpy array): Numpy array of input data.\n            y_data (list of numpy array): Sleep Stages\n            transform (callable, optional): Optional transform to be applied\n                on a sample.\n        \"\"\"\n        self.y_data = y_data\n        self.x_data = x_data\n        self.transform = transform\n\n        self.x_data = normalize_data(x_data)\n\n    def __len__(self):\n        return len(self.y_data)\n\n    def __getitem__(self, idx):\n        if torch.is_tensor(idx):\n            idx = idx.tolist()\n\n        signal = np.expand_dims(self.x_data[idx], axis=0)\n        stage = self.y_data[idx]\n\n        if self.transform:\n            signal = self.transform(signal)\n\n        return signal, stage\n\n\ntraining_dataset = EegEpochDataset(X_train,y_train)\ntraining_dataloader = DataLoader(training_dataset,batch_size = 32)\nvalidation_dataset = EegEpochDataset(X_test,y_test)\nvalidation_dataloader = DataLoader(validation_dataset,batch_size = 32)\n","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","id":"CoIGS2qZ60v1","executionInfo":{"status":"error","timestamp":1603111979611,"user_tz":-120,"elapsed":3754,"user":{"displayName":"Antoine Guillot","photoUrl":"","userId":"13673594054995143914"}},"outputId":"f5ee4748-ceac-4ae8-8973-4738b0bd91fc","execution":{"iopub.status.busy":"2022-10-04T16:17:03.672591Z","iopub.execute_input":"2022-10-04T16:17:03.674322Z","iopub.status.idle":"2022-10-04T16:17:04.961427Z","shell.execute_reply.started":"2022-10-04T16:17:03.674275Z","shell.execute_reply":"2022-10-04T16:17:04.960085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## First CNN model\nThe model is rather simple, it's a succession of convolution layers with non-linearities.\n\nThen, we apply max-pooling followed by a classification layer.","metadata":{"id":"SEV2YgRT60v7"}},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\n\n\nclass SingleChannelConvNet(nn.Module):\n\n    def __init__(self):\n        super(SingleChannelConvNet, self).__init__()\n        self.conv_a = nn.Conv1d(1, 8, 25, stride=5)\n        self.conv_b = nn.Conv1d(8, 16, 10, stride=5)\n        self.conv_c = nn.Conv1d(16, 32, 10, stride=5)\n        self.conv_d = nn.Conv1d(32, 64, 10, stride=5)\n\n        self.relu = nn.ReLU()\n\n        self.fc1 = nn.Linear(64, 5)\n\n    def forward(self, x):\n\n        x = self.relu(self.conv_a(x))\n        x = self.relu(self.conv_b(x))\n        x = self.relu(self.conv_c(x))\n        x = self.relu(self.conv_d(x))\n        x = x.max(-1)[0]\n        x = self.fc1(x)\n\n        return x\n","metadata":{"id":"-Gkm01M_60v8","executionInfo":{"status":"aborted","timestamp":1603111979603,"user_tz":-120,"elapsed":3744,"user":{"displayName":"Antoine Guillot","photoUrl":"","userId":"13673594054995143914"}},"execution":{"iopub.status.busy":"2022-10-04T16:17:04.963283Z","iopub.execute_input":"2022-10-04T16:17:04.964267Z","iopub.status.idle":"2022-10-04T16:17:04.977762Z","shell.execute_reply.started":"2022-10-04T16:17:04.964206Z","shell.execute_reply":"2022-10-04T16:17:04.975090Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Question:** \n- What is the downsampling rate of the network ? \n- What does the following line do ?\n>x = x.max(-1)[0]\n- How many parameters are there in this network ? Is it a lot ?","metadata":{}},{"cell_type":"markdown","source":"\n## Training Loop\nWe iterate on the training dataloader and train the model.","metadata":{"id":"9P6xkBIq60wB"}},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.optim as optim\n\n# device: use GPU if available\ndevice = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n\n# parameters\nn_epoch = 30\n\n# neural network and co\nmy_net = SingleChannelConvNet()\nmy_net = my_net.to(device) # model into GPU\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.Adam(my_net.parameters())\nmy_net.train()\nprint('training...')\nfor epoch in range(n_epoch):  # loop over the dataset multiple times\n\n    running_loss = 0.0\n    for i, data in enumerate(training_dataloader, 0):\n        # get the inputs; data is a list of [inputs, labels]\n        inputs, labels = data\n        inputs, labels = inputs.to(device).float(), labels.to(device)\n\n        # zero the parameter gradients\n        optimizer.zero_grad()\n        # forward + backward + optimize\n        outputs = my_net.forward(inputs)\n        loss = criterion(outputs, labels)\n        loss.backward()\n        optimizer.step()\n\n        # print statistics\n        running_loss += loss.item()\n    print('epoch %d, %d samples, loss: %.3f' % (epoch + 1, (i+1)*training_dataloader.batch_size,running_loss / (i+1)))\n    running_loss = 0.0\n\nprint('Finished Training')\n","metadata":{"id":"lk5zPOOs60wC","executionInfo":{"status":"aborted","timestamp":1603111979606,"user_tz":-120,"elapsed":3742,"user":{"displayName":"Antoine Guillot","photoUrl":"","userId":"13673594054995143914"}},"execution":{"iopub.status.busy":"2022-10-04T16:17:04.980084Z","iopub.execute_input":"2022-10-04T16:17:04.981680Z","iopub.status.idle":"2022-10-04T16:17:37.458909Z","shell.execute_reply.started":"2022-10-04T16:17:04.981634Z","shell.execute_reply":"2022-10-04T16:17:37.457419Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Assessing the model performances","metadata":{}},{"cell_type":"markdown","source":"Now the training is complete, let's assess its performance on the validation data","metadata":{"id":"mrotrqI860wI"}},{"cell_type":"code","source":"from sklearn.metrics import balanced_accuracy_score, cohen_kappa_score, confusion_matrix\nfrom sklearn.metrics import plot_confusion_matrix, f1_score\n# params\nclasses = ['Wake', 'N1', 'N2', 'N3', 'REM']\n\nwith torch.no_grad():\n    prediction_list = torch.empty(0).to(device)\n    true_list = torch.empty(0).to(device)\n    for data in validation_dataloader:\n        inputs, labels = data\n        inputs, labels = inputs.to(device).float(), labels.to(device)\n        \n        outputs = my_net(inputs)\n        _, predicted = torch.max(outputs, 1)\n        prediction_list = torch.cat([prediction_list, predicted])\n        true_list = torch.cat([true_list, labels])\n\ntrue_list = true_list.cpu().numpy()\nprediction_list = prediction_list.cpu().numpy()\nscores = {'balanced_accuracy': balanced_accuracy_score(true_list, prediction_list),\n            'cohen_kappa_score': cohen_kappa_score(true_list, prediction_list),\n            'confusion_matrix': confusion_matrix(true_list, prediction_list)}\n\nfor elt in scores:\n    print(elt)\n    print(scores[elt])","metadata":{"id":"4OPeYIjs60wK","executionInfo":{"status":"aborted","timestamp":1603111979609,"user_tz":-120,"elapsed":3739,"user":{"displayName":"Antoine Guillot","photoUrl":"","userId":"13673594054995143914"}},"execution":{"iopub.status.busy":"2022-10-04T16:17:49.019769Z","iopub.execute_input":"2022-10-04T16:17:49.021038Z","iopub.status.idle":"2022-10-04T16:17:49.153620Z","shell.execute_reply.started":"2022-10-04T16:17:49.020971Z","shell.execute_reply":"2022-10-04T16:17:49.152110Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Question:**\n- How does the CNN ranks compared to model from the previous tutorial ? How could you improve it ?\n- Where do the mistakes happen ? Plot a few epoch to understand when the network is good or bad.","metadata":{}},{"cell_type":"markdown","source":"## Getting started with Deep Learning:\n    \n- Pytorch Blitz: https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html\n- Pytorch Documentation: https://pytorch.org/docs/stable/index.html","metadata":{}},{"cell_type":"markdown","source":"## Going further","metadata":{}},{"cell_type":"markdown","source":"**Question 1: The current model is trained without early stopping, split the records between the training set, validation set and test set and use early stopping to stop the training at the right moment.**\n\n**Question 2: The model parameters are not optimized, by building a k-folds validation optimize the model hyperparameters**\n\n**Question 3: As we have seen in the previous tutorial, spectrogram representation may be an efficient way to represent sleep. Build a model (CNN or RNN) which takes the spectrogram of the data as input. You can use the spectrogram from https://pytorch.org/audio/stable/transforms.html**\n\n**Question 4: The current model only use a single EEG channel, what is the impact of using all the EEG channels ?**\n","metadata":{}},{"cell_type":"markdown","source":"**Additional challenge:** \n\nIf you are done with sleep staging classification, you can try this other challenge on physiological data. The goal is to segment sleep apnea events based on PSG data.\nhttps://challengedata.ens.fr/challenges/45","metadata":{}}]}