{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":57930,"databundleVersionId":6465181,"sourceType":"competition"}],"dockerImageVersionId":30684,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"This exciting Notebook will develop artificial intelligence algorithms capable of accurately classifying different patterns of brain signals recorded through electroencephalography (EEG) to determine which words a person is imagining when pronouncing them.\n#  \nIn this competition, participants will have access to a dataset of EEG recordings from different individuals who were asked to think about certain commands, covering various experimental conditions. The goal is to use machine learning techniques and signal processing to develop models that can correctly discern in EEG signals whether a person is thinking about a vowel (/a/, /e/, /i/, /o/, /u/) or a direction. ","metadata":{}},{"cell_type":"code","source":"import os\nRANDOM_SEED = 24\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom scipy.stats import kurtosis, skew","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:33:50.484069Z","iopub.execute_input":"2024-04-11T07:33:50.484555Z","iopub.status.idle":"2024-04-11T07:33:50.490903Z","shell.execute_reply.started":"2024-04-11T07:33:50.484517Z","shell.execute_reply":"2024-04-11T07:33:50.489567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip3 install hiddenlayer","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:33:51.300416Z","iopub.execute_input":"2024-04-11T07:33:51.300896Z","iopub.status.idle":"2024-04-11T07:34:06.672130Z","shell.execute_reply.started":"2024-04-11T07:33:51.300859Z","shell.execute_reply":"2024-04-11T07:34:06.670870Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's load the dataset","metadata":{}},{"cell_type":"code","source":"data_path = \"/kaggle/input/habla-imaginada-clasificacion-senales-EEG/\"\n\ndf_train = pd.read_csv(f'{data_path}X_train/X_train')\ndf_test = pd.read_csv(f'{data_path}X_test/X_test')\ny_train = pd.read_csv(f'{data_path}y_train.csv', header=None, names=['is_letter'])\n","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:34:06.674951Z","iopub.execute_input":"2024-04-11T07:34:06.675368Z","iopub.status.idle":"2024-04-11T07:36:05.960096Z","shell.execute_reply.started":"2024-04-11T07:34:06.675332Z","shell.execute_reply":"2024-04-11T07:36:05.958910Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are 64 channels but we are interested in this 6 channels","metadata":{}},{"cell_type":"code","source":"channels = list(set([c[:2] for c in df_train.columns if len(c) < 5]))\nchannels.sort()","metadata":{"execution":{"iopub.status.busy":"2024-04-11T06:48:24.749594Z","iopub.execute_input":"2024-04-11T06:48:24.750953Z","iopub.status.idle":"2024-04-11T06:48:24.762985Z","shell.execute_reply.started":"2024-04-11T06:48:24.750906Z","shell.execute_reply":"2024-04-11T06:48:24.761827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(channels)","metadata":{"execution":{"iopub.status.busy":"2024-04-11T06:48:50.716780Z","iopub.execute_input":"2024-04-11T06:48:50.717210Z","iopub.status.idle":"2024-04-11T06:48:50.723431Z","shell.execute_reply.started":"2024-04-11T06:48:50.717177Z","shell.execute_reply":"2024-04-11T06:48:50.722131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nHere's a brief explanation of these electrode placements:\n\n- **C3 and C4**: These electrodes are located over the sensorimotor cortex, specifically over the left (C3) and right (C4) central sulcus. They are commonly used to capture brain activity associated with motor planning and execution, including speech production.\n\n- **F3 and F4**: These electrodes are located over the prefrontal cortex. The prefrontal cortex is involved in various cognitive functions, including language processing and speech production.\n\n- **P3 and P4**: These electrodes are located over the parietal cortex. The parietal cortex is involved in integrating sensory information and is also associated with language and speech processing.\n\nIn EEG studies on imagined speech, these electrode placements are used to capture brain activity associated with different aspects of speech production and language processing. For example, activity in the sensorimotor cortex (C3 and C4) might reflect motor planning and execution of speech, while activity in the prefrontal and parietal cortices (F3, F4, P3, and P4) might reflect higher-order cognitive processes involved in language and speech processing.\n","metadata":{}},{"cell_type":"code","source":"letter_sample = df_train[y_train.is_letter.values].head(1)\nletter_sample","metadata":{"execution":{"iopub.status.busy":"2024-04-11T06:55:12.050869Z","iopub.execute_input":"2024-04-11T06:55:12.051683Z","iopub.status.idle":"2024-04-11T06:55:12.361480Z","shell.execute_reply.started":"2024-04-11T06:55:12.051646Z","shell.execute_reply":"2024-04-11T06:55:12.360079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axes = plt.subplots(nrows=6, ncols=2, figsize=(10,10), sharey=True)\n\nletter_sample = df_train[y_train.is_letter.values].head(1)\ncommand_sample = df_train[~(y_train.is_letter).values].head(1)\n\nix_channel = 0\nfor ix_row in range(6):\n    for ix_col in range(2):\n        label = \"Letra\" if ix_col == 0 else \"Comando\"\n        channel_cols = [c for c in df_train.columns if channels[ix_channel] in c]\n        axes[ix_row, ix_col].set_title(f\"{label} (canal {channels[ix_channel]})\")\n        if label == \"Letra\":\n            axes[ix_row, ix_col].plot(letter_sample[channel_cols].values[0])\n        else:\n            axes[ix_row, ix_col].plot(command_sample[channel_cols].values[0])\n\n    ix_channel += 1\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2024-04-11T06:56:13.087276Z","iopub.execute_input":"2024-04-11T06:56:13.088585Z","iopub.status.idle":"2024-04-11T06:56:16.502279Z","shell.execute_reply.started":"2024-04-11T06:56:13.088543Z","shell.execute_reply":"2024-04-11T06:56:16.501312Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Feature Extraction**\nWe define the functions to process the EEG signals and extract the features that the classifier will see.\n\n**Features**\nThe temporal signal is processed to extract six statistics for each channel: mean, standard deviation, minimum, maximum, kurtosis, and the Fisher skewness coefficient.","metadata":{}},{"cell_type":"code","source":"X_train = pd.DataFrame()\nX_test = pd.DataFrame()","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:00:28.322060Z","iopub.execute_input":"2024-04-11T07:00:28.322640Z","iopub.status.idle":"2024-04-11T07:00:28.333409Z","shell.execute_reply.started":"2024-04-11T07:00:28.322596Z","shell.execute_reply":"2024-04-11T07:00:28.332289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"statistics = [np.mean, np.std, np.min, np.max, kurtosis, skew]\n\nfor channel in channels:\n    channel_columns = [c for c in df_train.columns if channel in c]\n    for stat in statistics:\n        col_name = f'{channel}_{stat.__name__}'\n        X_train[col_name] = stat(df_train[channel_columns], axis=1)\n        X_test[col_name] = stat(df_test[channel_columns], axis=1)","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:01:01.219342Z","iopub.execute_input":"2024-04-11T07:01:01.220625Z","iopub.status.idle":"2024-04-11T07:01:28.375538Z","shell.execute_reply.started":"2024-04-11T07:01:01.220579Z","shell.execute_reply":"2024-04-11T07:01:28.374298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train.head()","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:02:50.553828Z","iopub.execute_input":"2024-04-11T07:02:50.554319Z","iopub.status.idle":"2024-04-11T07:02:50.589250Z","shell.execute_reply.started":"2024-04-11T07:02:50.554285Z","shell.execute_reply":"2024-04-11T07:02:50.587951Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.preprocessing import StandardScaler\n# feature scaling\nscaler = StandardScaler()\nscaler.fit(X_train)\nX_train_scaled=scaler.transform(X_train)\nX_test_scaled=scaler.transform(X_test)","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:36:21.851798Z","iopub.execute_input":"2024-04-11T07:36:21.852899Z","iopub.status.idle":"2024-04-11T07:36:21.876240Z","shell.execute_reply.started":"2024-04-11T07:36:21.852856Z","shell.execute_reply":"2024-04-11T07:36:21.874992Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_train","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:36:22.917519Z","iopub.execute_input":"2024-04-11T07:36:22.917972Z","iopub.status.idle":"2024-04-11T07:36:22.932811Z","shell.execute_reply.started":"2024-04-11T07:36:22.917940Z","shell.execute_reply":"2024-04-11T07:36:22.931099Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unique_values = np.unique(y_train)\nprint(unique_values)\n\n","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:36:25.067329Z","iopub.execute_input":"2024-04-11T07:36:25.067817Z","iopub.status.idle":"2024-04-11T07:36:25.075773Z","shell.execute_reply.started":"2024-04-11T07:36:25.067776Z","shell.execute_reply":"2024-04-11T07:36:25.074494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n# Count the occurrences of each label using boolean indexing\nno_of_yes = np.sum(y_train == True)\nno_of_no = np.sum(y_train == False)\n\nprint(f'Number of \"True\" labels: {no_of_yes}')\nprint(f'Number of \"False\" labels: {no_of_no}')","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:36:25.972118Z","iopub.execute_input":"2024-04-11T07:36:25.972928Z","iopub.status.idle":"2024-04-11T07:36:25.983042Z","shell.execute_reply.started":"2024-04-11T07:36:25.972890Z","shell.execute_reply":"2024-04-11T07:36:25.981813Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nimport numpy as np\n\n\n# Convert y_train to numpy array\ny_train = np.array(y_train)\n\n# Initialize LabelEncoder\nlabel_encoder = LabelEncoder()\n\n# Fit and transform the labels\ny_encoded = label_encoder.fit_transform(y_train)\n\n# Count the occurrences of each encoded label\nno_of_1 = np.sum(y_encoded == 1)\nno_of_0 = np.sum(y_encoded == 0)\n\nprint(f'Number of \"1\" labels: {no_of_1}')\nprint(f'Number of \"0\" labels: {no_of_0}')\n\n","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:37:58.707528Z","iopub.execute_input":"2024-04-11T07:37:58.708027Z","iopub.status.idle":"2024-04-11T07:37:58.720264Z","shell.execute_reply.started":"2024-04-11T07:37:58.707994Z","shell.execute_reply":"2024-04-11T07:37:58.718964Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#importing Models\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.ensemble import RandomForestClassifier\nfrom xgboost import XGBClassifier\nfrom sklearn.naive_bayes import GaussianNB\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.svm import SVC\nfrom sklearn.pipeline import make_pipeline","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:38:05.079308Z","iopub.execute_input":"2024-04-11T07:38:05.079728Z","iopub.status.idle":"2024-04-11T07:38:05.086195Z","shell.execute_reply.started":"2024-04-11T07:38:05.079696Z","shell.execute_reply":"2024-04-11T07:38:05.085298Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"1-KNN Model","metadata":{}},{"cell_type":"code","source":"# Train KNN model on the scaled data \nKNN = KNeighborsClassifier()\n# Fit the model\nKNN.fit(X_train_scaled,y_train)\n#predict x_test_scaled\npred_1 = KNN.predict(X_train_scaled)","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:43:21.056657Z","iopub.execute_input":"2024-04-11T07:43:21.057113Z","iopub.status.idle":"2024-04-11T07:43:21.400810Z","shell.execute_reply.started":"2024-04-11T07:43:21.057079Z","shell.execute_reply":"2024-04-11T07:43:21.399533Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(classification_report(y_train, pred_1))","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:43:22.642814Z","iopub.execute_input":"2024-04-11T07:43:22.643287Z","iopub.status.idle":"2024-04-11T07:43:22.669119Z","shell.execute_reply.started":"2024-04-11T07:43:22.643250Z","shell.execute_reply":"2024-04-11T07:43:22.667980Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics import confusion_matrix, classification_report, f1_score, recall_score,precision_score, accuracy_score\nimport matplotlib.pyplot as plt\nclass_names =0,1\n# Compute confusion matrix\ncm = confusion_matrix(y_train, pred_1)\n\n# Plot confusion matrix\nplt.figure(figsize=(8, 6))\nplt.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)\nplt.title('Confusion Matrix')\nplt.colorbar()\ntick_marks = np.arange(len(class_names))\nplt.xticks(tick_marks, class_names, rotation=45)\nplt.yticks(tick_marks, class_names)\nplt.xlabel('Predicted Label')\nplt.ylabel('True Label')\n\n# Print the confusion matrix\nprint(\"Confusion Matrix:\")\nprint(cm)\n\n# Show the plot\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:46:10.316050Z","iopub.execute_input":"2024-04-11T07:46:10.317114Z","iopub.status.idle":"2024-04-11T07:46:10.632129Z","shell.execute_reply.started":"2024-04-11T07:46:10.317073Z","shell.execute_reply":"2024-04-11T07:46:10.630881Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"2-Decision Tree","metadata":{}},{"cell_type":"code","source":"# Train decision tree model on the scaled data\nDecision_Tree = DecisionTreeClassifier() \n# Fit the model\nDecision_Tree.fit(X_train_scaled,y_train)\n#predict x_test_scaled\npred_ = Decision_Tree.predict(X_train_scaled)","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:47:18.269951Z","iopub.execute_input":"2024-04-11T07:47:18.270504Z","iopub.status.idle":"2024-04-11T07:47:18.539669Z","shell.execute_reply.started":"2024-04-11T07:47:18.270462Z","shell.execute_reply":"2024-04-11T07:47:18.538462Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(classification_report(y_train, pred_))","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:48:26.099964Z","iopub.execute_input":"2024-04-11T07:48:26.100909Z","iopub.status.idle":"2024-04-11T07:48:26.123643Z","shell.execute_reply.started":"2024-04-11T07:48:26.100868Z","shell.execute_reply":"2024-04-11T07:48:26.122476Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#creating confusion matrix to know the errors\nconf = confusion_matrix(y_train, pred_ ,normalize=\"all\")\ndisp = ConfusionMatrixDisplay(conf).plot(cmap=plt.cm.PuBuGn)","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:48:03.733777Z","iopub.execute_input":"2024-04-11T07:48:03.734287Z","iopub.status.idle":"2024-04-11T07:48:04.058401Z","shell.execute_reply.started":"2024-04-11T07:48:03.734244Z","shell.execute_reply":"2024-04-11T07:48:04.057125Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"3-Random Forest","metadata":{}},{"cell_type":"code","source":"# Train Random forest model on the scaled data\nRandom_Forest= RandomForestClassifier()\n# Fit the model\nRandom_Forest.fit(X_train_scaled,y_train)\n#predict x_test_scaled\npred_rand = Random_Forest.predict(X_train_scaled)","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:49:40.029919Z","iopub.execute_input":"2024-04-11T07:49:40.031145Z","iopub.status.idle":"2024-04-11T07:49:43.425070Z","shell.execute_reply.started":"2024-04-11T07:49:40.031105Z","shell.execute_reply":"2024-04-11T07:49:43.423704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(classification_report(y_train, pred_rand))","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:50:13.400521Z","iopub.execute_input":"2024-04-11T07:50:13.401042Z","iopub.status.idle":"2024-04-11T07:50:13.425279Z","shell.execute_reply.started":"2024-04-11T07:50:13.401000Z","shell.execute_reply":"2024-04-11T07:50:13.424018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#creating confusion matrix to know the errors\nconf = confusion_matrix(y_train, pred_rand ,normalize=\"all\")\ndisp = ConfusionMatrixDisplay(conf).plot(cmap=plt.cm.PuBuGn)","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:50:29.110272Z","iopub.execute_input":"2024-04-11T07:50:29.110730Z","iopub.status.idle":"2024-04-11T07:50:29.444908Z","shell.execute_reply.started":"2024-04-11T07:50:29.110697Z","shell.execute_reply":"2024-04-11T07:50:29.443674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":" pred_rand = Random_Forest.predict(X_test_scaled)","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:51:36.457269Z","iopub.execute_input":"2024-04-11T07:51:36.458124Z","iopub.status.idle":"2024-04-11T07:51:36.532143Z","shell.execute_reply.started":"2024-04-11T07:51:36.458079Z","shell.execute_reply":"2024-04-11T07:51:36.530994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_test_predictions = pd.DataFrame({'is_letter' : pred_rand})\ndf_test_predictions.index = df_test.index + 1\ndf_test_predictions.index.names = ['ID']\n\ndf_test_predictions.to_csv(\"submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2024-04-11T07:51:59.062823Z","iopub.execute_input":"2024-04-11T07:51:59.063329Z","iopub.status.idle":"2024-04-11T07:51:59.078875Z","shell.execute_reply.started":"2024-04-11T07:51:59.063293Z","shell.execute_reply":"2024-04-11T07:51:59.077291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}