Imagine you would like to know how to control a cursor on a screen with just your mind! This project is a step in this direction. Specifically, the task is to take readings of brain activity (voltages) and learn a mapping from these readings to velocity vectors for a computer cursor. These readings were measured from the brain of a Rhesus monkey while it watched a researcher move a cursor with a joystick from a START state to a GOAL state. While watching it attempted to (with its mind) to direct the cursor from the START state to the GOAL state (this was encouraged by giving the monkey a reward in previous trials every time the cursor reached the GOAL state. The setup is depicted in the figure below:
![joystick task](http://mkusner.github.io/images/joystick_record.png)
Finding a mapping from brain activity to cursor velocity vectors is a non-trivial task. Currently, the state-of-the-art is to first perform spectral estimation on the voltage readings (this will give us a sense of the frequencies of the signal), and then to perform linear regression to map these frequencies to the true velocity vectors given by the joystick (see: example.m).
So the two tasks to improve upon the state-of-the-art are to:
Improve the spectral estimation (secondary task)
Improve regression from spectrum to velocity vectors (primary task)
The training set (train.mat) consists of the following variables:
signal_tr
: raw voltage readings at 2034.5 Hz. Each row is a new reading and each column is the reading from a particular electrode (32 in total). The voltage readings from the brain are collected via Electrocorticography (
[ECOG](https://en.wikipedia.org/wiki/Electrocorticography)
)
pos_x_tr
: x-component of the cursor's velocity vector at different time-steps (31,255 total time-steps). This one half the prediction label (the other is the y-component). These readings are at 25 Hz, so one of the things spectral estimation must do is down-sample the signal to 25 Hz as well (example.m does this already).
pos_y_tr
: y-component of the cursor's velocity vector at different time-steps. This is the same size as pos_x_tr and the second half of the label to be predicted.
timestamps_tr
: this gives the timestamps of the voltage samples (2034.5 Hz)
time_sample_tr
: the timestamps of the cursor at 25 Hz. Using this and the above data we can down-sample the signal to 25 Hz.
The testing set (test.mat) is very similar:
signal_te
: raw voltage readings that is a matrix of size (num. readings x electrodes).
timestamps_te
: the voltage timestamps
time_sample_te
: the cursor timestamps
For more information on the data please email: matt.kusner@gmail.com. For how to perform spectral estimation and regression please take a look at the example.m file. We can't wait to see what things you will come up with!
Acknowledgements
We thank NSF grant EFRI-1137211.