{"cells":[{"metadata":{},"cell_type":"markdown","source":"**GRASP-AND-LIFT EEG Detection Project**\n\nThis project aims to compare different machine learning models in terms of viability for detecting events from EEG signals."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np \nimport pandas as pd \nimport matplotlib.pyplot as plt\nimport matplotlib.patches as mpatches","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Introduction**"},{"metadata":{},"cell_type":"markdown","source":"Training data consists of EEG recordings of subjects performing grasp-and-lift trials.\n\nThere are 12 subjects in total, 10 series of trials for each subject.\n\nEach series recorded 32 EEG channels with sampling rate 500Hz.\n\nTraining data contains id columns, representing the subject, series, and time at which data point was recorded.\n"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"scrolled":false},"cell_type":"code","source":"train_set_signals = pd.read_csv(\"../input/train/subj1_series1_data.csv\")\ntrain_set_signals.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The aim is to predict whether a certain event is occuring at a given point.\n\nThere are 6 events. If an event occured at a given point is represented by 1, 0 otherwise.      \n"},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"e6fdf2790a2ac641decf5989897373021f3b6555"},"cell_type":"code","source":"train_set_labels = pd.read_csv(\"../input/train/subj1_series1_events.csv\")\ntrain_set_labels.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Setup for visualizations"},{"metadata":{"trusted":true,"_uuid":"5363122b47e28f5d0f7b8af774fcf8814cf71e57"},"cell_type":"code","source":"labels = train_set_labels.columns.drop('id')\nlabelNames = labels.values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"58286dcc970cdfdd6b548a8ea397a3c7cc200444"},"cell_type":"code","source":"train_set_complete = pd.concat([train_set_signals,train_set_labels], axis=1)\ntrain_set_complete.insert(0, \"order\", range(0, len(train_set_complete)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6cbf2d4f8971f71ffbeffc8b2c1c9b8814710a9f"},"cell_type":"code","source":"def highlight(indices,ax,color):\n    i=0\n    while i<len(indices):\n        ax.axvspan(indices[i]-0.5, indices[i]+0.5, facecolor=color, edgecolor='none', alpha=.4)\n        i+=1      \n    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def vizualize_predictions(signals, predictions, expected, labelName, limit=2000): \n    #0-31\n    signalIndex = 10\n    \n    #Relevant only for multilabel predictions, else is always 0\n    labelIndex = 0\n                \n    signals = pd.DataFrame(data=np.array(signals))\n    axis = signals[signals.columns[signalIndex]].iloc[0:limit].plot(figsize=(20,4))  \n        \n    expected = pd.DataFrame(data = expected)    \n    predictions = pd.DataFrame(data = np.around(predictions))\n    \n    expectedCropped = expected.iloc[0:limit,]\n    predictionsCropped = predictions.iloc[0:limit,]\n    \n    highlight(expectedCropped[expectedCropped.iloc[:,labelIndex]==1].index, axis, \"red\")\n    highlight(predictionsCropped[predictionsCropped.iloc[:,labelIndex]==1].index, axis, \"black\")\n    \n    red_patch = mpatches.Patch(color='red', label='Expected event')\n    black_patch = mpatches.Patch(color='black', label='Predicted event')\n    plt.legend(handles=[red_patch, black_patch])\n\n    plt.title(labelName)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Following graph shows five seconds of different recorded EEG channels with highlihted areas, which represent one of the events occuring. Events can overlap.\n\nOur task is to predict those areas for unlabeled set of signals."},{"metadata":{"_kg_hide-output":false,"_kg_hide-input":false,"trusted":true,"_uuid":"649de8bf9e30a6e6a8aad9628f01d1a45614d32f"},"cell_type":"code","source":"secondsToShow = 5\nchannelsToShow = 3\nlabelsToShow = 6\n\neeg_channels = train_set_signals.columns.drop('id')\nsample_set = train_set_complete[train_set_complete[\"order\"] < secondsToShow*500].drop(\"id\", axis=1).set_index(\"order\") #sample rate is 500hz \ncolors=[\"red\",\"blue\",\"yellow\",\"green\", \"purple\", \"black\"]\naxes = sample_set.plot(y=eeg_channels[:channelsToShow],subplots=True, figsize=(15,5))\n    \nfor axis in axes:    \n    colorindex = 0\n    for label in labels[:labelsToShow]:\n        highlight(sample_set[sample_set[label]==1].index, axis, colors[colorindex])        \n        colorindex = colorindex + 1\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Helper methods for loading data\n\nFeatures are standartized by removing the mean and scaling to unit variance\n\nFurther preprocessing can be done in *prepare_signals* function"},{"metadata":{"trusted":true},"cell_type":"code","source":"import math\nfrom sklearn.preprocessing import StandardScaler\n\ndef load_train_data(subject, series):\n    train_set_signals = pd.read_csv(f\"../input/train/subj{subject}_series{series}_data.csv\")\n    train_set_labels = pd.read_csv(f\"../input/train/subj{subject}_series{series}_events.csv\")\n    return train_set_signals, train_set_labels\n    \ndef load_all_test_data():\n    signals_dfs=[]\n    for i in range(1,13):\n        for j in range(9,11):            \n            signals = pd.read_csv(f\"../input/test/subj{i}_series{j}_data.csv\")\n            signals_dfs.append(signals)\n    return pd.concat(signals_dfs)       \n\ndef load_test_data(subject):\n    signals_dfs=[]    \n    for i in range(9,11):            \n        signals = pd.read_csv(f\"../input/test/subj{subject}_series{i}_data.csv\")\n        signals_dfs.append(signals)\n    return pd.concat(signals_dfs) \n\ndef load_test_data_series(subject, series):\n    return pd.read_csv(f\"../input/test/subj{subject}_series{series}_data.csv\") \n\ndef prepare_labels(data):    \n    return data.drop(\"id\", axis=1)\n    \ndef prepare_signals(data):\n    data = data.drop(\"id\", axis=1)\n    columns = data.columns      \n    \n    #Preprocessing    \n    scaler = StandardScaler() \n    data =np.asarray(data.astype(float))\n    data = scaler.fit_transform(data)\n    data = pd.DataFrame(data, columns=columns) \n    \n    return data\n    \ndef load_train_data_prepared(subject, series):    \n    signals, labels = load_train_data(subject,series)    \n    return prepare_signals(signals), prepare_labels(labels)   \n\ndef split_data(data_to_split, percent):\n    percenttotrainon = percent\n    spliton = math.floor(float(len(data_to_split))*(percenttotrainon/float(100)))\n    return data_to_split.iloc[:spliton], data_to_split[spliton:]   \n\ndef load_mass_data(fromsubj, tosubj): #we have 12 subjects\n    signals_dfs=[]\n    labels_dfs=[]\n    for i in range(1,9): #we have 8 series availible for each subject\n        for j in range(fromsubj, tosubj+1):\n            signals, labels = load_train_data_prepared(subject=j, series=i)\n            signals_dfs.append(signals)\n            labels_dfs.append(labels)        \n\n    signals_complete=pd.concat(signals_dfs)\n    labels_complete=pd.concat(labels_dfs)\n    return signals_complete, labels_complete \n\ndef load_mass_data_subject(subject, fromseries, toseries): \n    signals_dfs=[]\n    labels_dfs=[]\n    for i in range(fromseries,toseries+1):        \n        signals, labels = load_train_data_prepared(subject, series=i)\n        signals_dfs.append(signals)\n        labels_dfs.append(labels)        \n\n    signals_complete=pd.concat(signals_dfs)\n    labels_complete=pd.concat(labels_dfs)\n    return signals_complete, labels_complete ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Helper for printing success rates for given predictions and expected values"},{"metadata":{"trusted":true},"cell_type":"code","source":"def rd(x):\n        return round(x)\n\ndef printSucc(predictions, expected, dataLabel):\n    #success counters\n    succ = 0\n    onesTotal = 0\n    onesSucc = 0\n    \n    #Compute successes in data\n    for i in range(0, len(predictions)):    \n        if(np.array_equal(list(map(rd, predictions[i])),expected[i])):\n            succ+=1\n\n        if 1 in expected[i]:\n            onesTotal += 1           \n            if(np.array_equal(list(map(rd, predictions[i])),expected[i])):\n                onesSucc +=1           \n\n    print(dataLabel, \"success\", \"---\",\"TOTAL =\", \"{:0.4f}\".format(succ/len(predictions)), \"|||\",\"EVENT =\",\"{:0.4f}\".format(onesSucc/onesTotal) ,\"(\", onesSucc, \"/\",onesTotal, \")\")    \n    \n    return succ/len(predictions), onesSucc/onesTotal","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**LOGISTIC REGRESSION**\n\nLogistic regression model.\n\nThe model is trained for each label (event) separately. "},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"from sklearn.linear_model import LogisticRegression\n    \ndef logistic_regression_validation(model, subject, draw=False):\n    trainon_signals, trainon_labels = load_mass_data_subject(subject = subject,fromseries=1,toseries=7)\n    teston_signals, teston_labels = load_train_data_prepared(subject= subject,series=8)\n        \n    lr = model  \n    #success rates for different labels\n    successRates=[]  \n    onesSuccessRates=[]\n    #Train the model for each label separately\n    for label in labels:\n        #In sklearn framework, we can call fit method repeatedly, because it resets the model\n        lr.fit(trainon_signals, trainon_labels[[label]].values.ravel())        \n        \n        #Test success for all data\n        successRates.append(lr.score(teston_signals, teston_labels[[label]].values.ravel()))           \n                \n        #Test of success for 1's only\n        predictions = lr.predict(teston_signals)        \n        actual = teston_labels[[label]].values.ravel()        \n        \n        print(label,\"label model evaluation\")\n        printSucc(predictions.reshape(len(predictions),1), actual.reshape(len(actual),1), \"Testing\")\n        if(draw):\n            vizualize_predictions(teston_signals, predictions, actual, labelName=label,limit=5000)        \n        \n        total = 0\n        successes = 0 \n        for i in range(0, len(actual)):            \n            if(actual[i] == 1):               \n                total+=1\n                if(predictions[i] ==1):\n                    successes+=1       \n                    \n        onesSuccessRates.append(successes/total)\n                \n        \n\n    #Compute average success for all labels\n    total_success_rate = sum(successRates)/len(successRates)\n    total_success_rate_ones = sum(onesSuccessRates)/len(onesSuccessRates);  \n    \n    print(\"SUMMARY\")\n    print(\"TOTAL:\", \"{:1.4f}\".format(total_success_rate)) \n    print(\"EVENTS\",\"{:1.4f}\".format(total_success_rate_ones))   \n    \n    \n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Testing of the model for given subject \n\nEvaluating each label success rate separarely and averaging them. Random behaviour has 50% success rate.\n\nFor given subject, we use first 7 series as training data and 8'th series as test data. \n\nOnly a portion of data is visualized."},{"metadata":{"trusted":true},"cell_type":"code","source":"model = LogisticRegression(solver='lbfgs', max_iter = 1000)\nlogistic_regression_validation(model=model, subject= 1, draw=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since input data are sparse (there are a lot more 0's than 1's), balanced mode is tried.\n\nThe “balanced” mode uses the values of labels to automatically adjust weights inversely proportional to class frequencies in the input data."},{"metadata":{"trusted":true,"_kg_hide-output":true,"_kg_hide-input":false},"cell_type":"code","source":"model = LogisticRegression(solver='lbfgs', max_iter = 1000, class_weight=\"balanced\")\nlogistic_regression_validation(model=model, subject= 1, draw=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"CSV file generation for competition submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.linear_model import LogisticRegression\n\ndef logistic_regression_training(trainon_signals,trainon_labels,teston_signals):    \n    lr = LogisticRegression(solver='lbfgs', max_iter = 1000, class_weight=\"balanced\")\n    \n    ids = teston_signals[\"id\"]\n    teston_signals = prepare_signals(teston_signals)       \n    \n    result = np.empty((len(teston_signals),len(trainon_labels.columns)))    \n    \n    labelcounter = 0    \n    for label in labels:        \n        lr.fit(trainon_signals, trainon_labels[[label]].values.ravel())        \n        predictions = lr.predict_proba(teston_signals)[:,1]\n        result[:,labelcounter] = predictions \n        labelcounter+=1\n    return result, ids\n       \n\ndef logistic_regression_submission():\n    results=[]\n    results_ids=[]\n    for i in range(1,13):         \n        train_signals, train_labels = load_mass_data(i,i)\n        test_signals = load_test_data(i)\n        res, ids = logistic_regression_training(train_signals,train_labels,test_signals)\n        results.append(res)\n        results_ids.append(ids)\n    \n    \n    submission_name = \"balanced-logistic-submission.csv\"\n    submission = pd.DataFrame(columns=labelNames, data=np.concatenate(results), index=np.concatenate(results_ids))\n    submission.to_csv(submission_name,index_label=\"id\",float_format='%.3f')\n\n# Csv generation\n# logistic_regression_submission()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**RECURRENT NEURAL NETWORK**\n\nRNN with LSTM, dropout and activation layers\n\n* Adam optimizer\n* Binary crossentropy loss \n* Sigmoid activation layer"},{"metadata":{},"cell_type":"markdown","source":"Transform 2D dataset to 3D for LSTM layer - add floating window of *look_back* length"},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_sequences(dataset,labels, look_back=1):\n    dataX = []\n    dataY = labels[look_back:]\n    for i in range(len(dataset)-look_back):\n        dataX.append(dataset[i:(i+look_back), ])\n    return np.array(dataX), np.array(dataY)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Tests given rnn model on predicting label or more labels from testing data.\n\nProvides visualisations and success rates.\n\n\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"def evaluate_rnn(subject, model, config, LabelsRange, draw):\n    #Last (8th) series is used as testing data\n    test_signals, test_labels = load_train_data_prepared(subject=subject,series=8)\n    \n    #Creating sequences for lstm layer\n    X_test_signals, X_test_labels = create_sequences(\n        test_signals.values[::config.DOWNSAMPLING],\n        test_labels.values[::config.DOWNSAMPLING],\n        look_back=config.LOOK_BACK\n    )\n    #Selecting only desired labels\n    X_test_labels = X_test_labels[:,LabelsRange[0]:LabelsRange[1]] \n\n    #Last few data points that do not fit batch size are omitted \n    croppedSize = math.floor(len(X_test_signals)/config.BATCH_SIZE)*config.BATCH_SIZE    \n\n    #Prediction for testing data\n    predictions = model.predict(X_test_signals[0:croppedSize], batch_size=config.BATCH_SIZE)\n    expected = X_test_labels[0:croppedSize]\n    \n    #Selecting only desired labels   \n    labelsPredicted = len(predictions[0])\n    if(labelsPredicted == 1):\n        predictions = predictions[:,0:1]\n    else: #6 labels predicted    \n        predictions = predictions[:,LabelsRange[0]:LabelsRange[1]] \n    \n            \n    #Success rate printing    \n    totalPercent, onesPercent = printSucc(predictions,expected, dataLabel=\"Testing\")\n    \n    #Vizualization\n    if(draw):\n        vizualize_predictions(\n            test_signals.values[::config.DOWNSAMPLING][config.LOOK_BACK:croppedSize+config.LOOK_BACK:],\n            predictions,\n            expected,\n            labelName = labelNames[LabelsRange[0]],  \n            limit = 1000\n        )\n    \n    return totalPercent, onesPercent    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Training of a single model for a single or more subjects.\n\nUses series 1-7 (leaves series 8 for testing)"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.models import Sequential\nfrom keras.layers import Dense, Activation, LSTM, Dropout\n\ndef train_rnn(subjects, labelToTrain, trainSeparateLabels, config, modelGenerator, callbacks): \n    #Calculating output size of trained model\n    LabelsRange = [labelToTrain,labelToTrain+1]\n    if(not trainSeparateLabels):\n        LabelsRange = [0,6]\n    OutputSize = LabelsRange[1] - LabelsRange[0]     \n    #Model creation\n    model = modelGenerator(config, output_size = OutputSize) \n    \n    #Training\n    for i in range(config.ITERATIONS):\n        #For specified subjects\n        for subject in subjects:\n            #For series 1-7 \n            for j in range(1,8):\n                signals, labels = load_train_data_prepared(subject = subject,series = j) \n                #Create sequences\n                X_train_signals, X_train_labels = create_sequences(\n                    signals.iloc[::config.DOWNSAMPLING].values,\n                    labels.iloc[::config.DOWNSAMPLING].values,\n                    look_back=config.LOOK_BACK       \n                )        \n\n                X_train_labels = X_train_labels[:,LabelsRange[0]:LabelsRange[1]]\n                croppedSize = math.floor(len(X_train_signals)/config.BATCH_SIZE)*config.BATCH_SIZE        \n                #Train model on relevant label (calling fit repeatedly in keras doesnt reset the model)\n                model.fit(\n                    X_train_signals[0:croppedSize],\n                    X_train_labels[0:croppedSize],\n                    epochs=config.EPOCHS,\n                    batch_size=config.BATCH_SIZE,\n                    shuffle=config.SHUFFLE,\n                    verbose=config.VERBOSE,\n                    callbacks=callbacks\n                )            \n                if(config.STATEFUL):\n                    model.reset_states()\n    \n    \n    print(\"FITTING DONE\")\n    return model, LabelsRange","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Training of model for given subjects and evaluating it.\n\nEvaluating each label success rate separarely and averaging them. Random behaviour has 50% success rate.\n\nFor given subject, we use first 7 series as training data and 8'th series as test data. "},{"metadata":{"trusted":true},"cell_type":"code","source":"def rnn_validation(subjects, modelGenerator, trainSeparateLabels, callbacks, draw=False, subjectToEvaluateOn = 1):\n    config = Config()\n    totalPercentages = []\n    onesPercentages = []   \n    \n    model = None\n    labelsRange = None\n    \n    #One model for training on all labels combined\n    if(not trainSeparateLabels):\n        model, labelsRange = train_rnn(\n            subjects=subjects,\n            labelToTrain = -1, #doesnt matter\n            trainSeparateLabels = False,\n            config = config,\n            modelGenerator = modelGenerator,\n            callbacks=callbacks\n        )   \n    \n    for i in range (0,6):\n        #Separate models for training all labels separately\n        if(trainSeparateLabels):\n            model, labelsRange = train_rnn(  \n                subjects = subjects,\n                labelToTrain=i,\n                trainSeparateLabels=True,\n                config=config,\n                modelGenerator = modelGenerator,\n                callbacks=callbacks\n                )\n        print(labelNames[i],\"label model evaluation\")\n        \n        total, ones = evaluate_rnn(subjectToEvaluateOn, model, config, LabelsRange=[i,i+1], draw = draw)\n        totalPercentages.append(total)\n        onesPercentages.append(ones)        \n\n    print(\"SUMMARY\")\n    print(\"TOTAL :\", sum(totalPercentages)/len(totalPercentages))\n    print(\"EVENTS :\", sum(onesPercentages)/len(onesPercentages))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"RNN configuration"},{"metadata":{"trusted":true},"cell_type":"code","source":"def create_basic_rnn_model(config, output_size):      \n    model = Sequential()\n    model.add(LSTM(50,batch_input_shape=(config.BATCH_SIZE,config.LOOK_BACK,32),\n                   return_sequences=False, stateful=config.STATEFUL, dropout=0.5, activation=\"softsign\"))     \n    model.add(Dense(output_size, activation=\"sigmoid\"))\n    model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\n    return model \n\ndef create_stacked_rnn_model(config, output_size):      \n    model = Sequential()\n    model.add(LSTM(50,batch_input_shape=(config.BATCH_SIZE,config.LOOK_BACK,32), return_sequences=True, stateful=config.STATEFUL, dropout=0.5, activation=\"softsign\")) \n    model.add(LSTM(50, return_sequences=False, stateful=config.STATEFUL, dropout=0.5, activation=\"softsign\"))     \n    model.add(Dense(output_size, activation=\"sigmoid\"))\n    model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])\n    return model\n\nclass Config(object):\n    #data are downsampled, each 15th data point is taken\n    DOWNSAMPLING = 15\n    LOOK_BACK = 35\n    BATCH_SIZE = 512\n\n    #Iterations on stateful model, epochs on non-stateful\n    EPOCHS = 30\n    ITERATIONS = 1\n    STATEFUL = False\n    SHUFFLE = not STATEFUL\n    VERBOSE = 0    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.callbacks import EarlyStopping\n\ncallbacks=[EarlyStopping(monitor=\"acc\", verbose=0, patience=10, restore_best_weights=True)]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Basic model, training all labels separately for single subject"},{"metadata":{"trusted":true},"cell_type":"code","source":"rnn_validation(subjects=[1],\n               modelGenerator = create_basic_rnn_model,\n               trainSeparateLabels=True,\n               draw=True,\n               subjectToEvaluateOn=1,\n               callbacks=callbacks\n              )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Basic model, training all labels at once for single subject"},{"metadata":{"trusted":true},"cell_type":"code","source":"rnn_validation(subjects=[4],\n               modelGenerator = create_basic_rnn_model,\n               trainSeparateLabels=False,\n               draw=True,\n               subjectToEvaluateOn=4,\n               callbacks=callbacks\n              )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Stacked model, training all labels separately for single subject"},{"metadata":{"trusted":true},"cell_type":"code","source":"rnn_validation(subjects=[10],\n               modelGenerator = create_stacked_rnn_model,\n               trainSeparateLabels=True,\n               draw=True,\n               subjectToEvaluateOn=10,\n               callbacks=callbacks\n              )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Stacked model, training all labels at once for single subject"},{"metadata":{"trusted":true},"cell_type":"code","source":"rnn_validation(subjects=[9],\n               modelGenerator = create_stacked_rnn_model,\n               trainSeparateLabels=False,\n               draw=True,\n               subjectToEvaluateOn=9,\n               callbacks=callbacks\n              )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Basic model, training all labels at once for all subjects"},{"metadata":{"trusted":true},"cell_type":"code","source":"rnn_validation(subjects=list(range(1,13)),\n               modelGenerator = create_basic_rnn_model,\n               trainSeparateLabels=False,\n               draw=True,\n               subjectToEvaluateOn=7,\n               callbacks=callbacks\n              )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"CSV file generation for competition submission\n\nSome datapoints are not predicted (didnt fit batch size or are at the beginning of sequence).\nThose data points are predicted as vectors of 0's (no event)"},{"metadata":{"trusted":true},"cell_type":"code","source":"def rnn_model_submission(config, modelGenerator, trainAllLabels):   \n    results=[]\n    results_ids=[]\n    callbacks=[EarlyStopping(monitor=\"acc\", verbose=1, patience=20, restore_best_weights=True)]\n    #Train models for each subject\n    for i in range(1,13):\n        print(\"Training subject\", i)\n        #separate models for each subject           \n        model = modelGenerator(config, output_size = 6)\n        single_label_models = list() \n        for j in range(0,6):\n            single_label_models.append(modelGenerator(config, output_size = 1))\n        \n        #Train on all training data\n        for j in range(1,9):                   \n            train_signals, train_labels = load_train_data_prepared(subject = i,series = j)\n            train_signals, train_labels = create_sequences(\n                train_signals.iloc[::config.DOWNSAMPLING].values,\n                train_labels.iloc[::config.DOWNSAMPLING].values,\n                look_back = config.LOOK_BACK)            \n            \n            croppedSize = math.floor(len(train_signals)/config.BATCH_SIZE)*config.BATCH_SIZE \n            \n            #Training all labels at once (predicting vector of all labels)\n            if(trainAllLabels):                \n                for k in range(config.ITERATIONS):\n                    model.fit(\n                        train_signals[0:croppedSize],\n                        train_labels[0:croppedSize],\n                        epochs=config.EPOCHS,\n                        batch_size=config.BATCH_SIZE,\n                        shuffle=config.SHUFFLE,\n                        verbose=config.VERBOSE,\n                        callbacks=callbacks\n                    )\n                    if(config.STATEFUL):\n                        model.reset_states()\n            #Training all labels separately\n            else:                \n                for k in range(0,len(single_label_models)):\n                    labelModel = single_label_models[k]\n                    for l in range(config.ITERATIONS):\n                        labelModel.fit(\n                            train_signals[0:croppedSize],\n                            #training only for specific label\n                            train_labels[0:croppedSize,k:k+1],\n                            epochs=config.EPOCHS,\n                            batch_size=config.BATCH_SIZE,\n                            shuffle=config.SHUFFLE,\n                            verbose = config.VERBOSE,\n                            callbacks=callbacks\n                        )\n                        if(config.STATEFUL):\n                            labelModel.reset_states()\n        \n        #Generating results for testing data\n        for j in range(9,11):\n            \n            test_signals = load_test_data_series(subject = i, series = j)\n            ids = test_signals[\"id\"]\n            teston_signals = prepare_signals(test_signals)            \n                                  \n            omittedByDownsampling = len(teston_signals)%config.DOWNSAMPLING            \n            teston_signals, ignored_labels = create_sequences(teston_signals.iloc[::config.DOWNSAMPLING].values, [], look_back=config.LOOK_BACK)            \n            \n            croppedSize = math.floor(len(teston_signals)/config.BATCH_SIZE)*config.BATCH_SIZE \n            omitted = len(teston_signals)%config.BATCH_SIZE\n            \n            result = None\n            #Predicting vector of all labels\n            if(trainAllLabels):\n                result = model.predict(teston_signals[0:croppedSize], batch_size=config.BATCH_SIZE)\n            #Predicting each label separately and concat them into one vector\n            else:\n                result = list()\n                labelModelsPredictions = list()\n                for labelModel in single_label_models:\n                    labelPredictions = labelModel.predict(teston_signals[0:croppedSize], batch_size=config.BATCH_SIZE)\n                    labelModelsPredictions.append(labelPredictions)\n                for k in range(len(labelModelsPredictions[0])): \n                    resultVector = list()\n                    for prediction in labelModelsPredictions:\n                        resultVector.append(prediction[k,0])\n                    result.append(resultVector)\n                    \n            #No predictions at the beginning of the data (floating window start). Padding with vectors of 0's\n            for k in range(config.LOOK_BACK*config.DOWNSAMPLING):\n                results.append(np.array([[0,0,0,0,0,0]]))            \n            \n            result = np.array(result)\n            for element in result:                    \n                #Multiplying each result because of downsampling\n                for l in range(config.DOWNSAMPLING):                    \n                    results.append(np.array([element]))       \n            \n            #No predictions for data outside of batch size, padding with vectors of 0's\n            for k in range((omitted*config.DOWNSAMPLING)+omittedByDownsampling):\n                results.append(np.array([[0,0,0,0,0,0]]))\n                \n            results_ids.append(ids)\n            \n            resultslen = len(np.concatenate(results))\n            idslen = len(np.concatenate(results_ids))\n            \n            #Removing extra 0's at the end of predictions, now this is ugly\n            for k in range(0, resultslen-idslen):\n                results.pop(-1)          \n   \n    \n    print(\"submitting\")\n    submission_name = \"basic-rnn-submission.csv\"            \n    \n    submission = pd.DataFrame(columns=labelNames, data=np.concatenate(results), index=np.concatenate(results_ids))\n    submission.to_csv(submission_name,index_label=\"id\",float_format='%.3f')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class SubConfig(object):\n    DOWNSAMPLING = 15\n    LOOK_BACK = 35\n    BATCH_SIZE = 512\n    \n    EPOCHS = 50\n    ITERATIONS = 1\n    STATEFUL = False\n    SHUFFLE = not STATEFUL\n    VERBOSE = 0   \n\nconfig = SubConfig()\n# rnn_model_submission(config, modelGenerator=create_basic_rnn_model, trainAllLabels=True)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}