## Computational problem

The decoding problem consists of mapping a variable-length time series of neural activity to text with no ground-truth alignment between the two. We provide a [baseline algorithm](https://github.com/Neuroprosthetics-Lab/nejm-brain-to-text) that first decodes neural activity into phoneme probabilities and then uses an n-gram language model plus LLM rescoring to decode the phoneme probabilities into text (see the diagram below). Our baseline algorithm achieves a 6.70% word error rate on the held-out *test* set. We may provide additional example approaches as the competition progresses, based on our own ideas as well as the lessons learned from the [Brain-to-Text '24 challenge](https://eval.ai/web/challenges/challenge-page/2099).

![Speech neuroprosthesis overview](https://github.com/Neuroprosthetics-Lab/nejm-brain-to-text/raw/main/b2txt_methods_overview.png)

Some potential avenues for improvement beyond the baseline algorithm include:
- Data augmentation (e.g., temporal masking)
- Model architecture (e.g., transformer, end-to-end)
- Different loss function (e.g., transducer)
- Using different language models for sentences from [different corpora](https://github.com/Neuroprosthetics-Lab/nejm-brain-to-text/blob/main/data/t15_copyTaskData_description.csv) (e.g., perhaps a 1-gram language model would do better than a 5gram model on the "random word" sentences)
     - Keep in mind that, with the goal of generalization in mind, you ***cannot*** tell your model what corpus to use at test-time. If your model can automatically detect which corpus to use, that's fine. The intention is for models to generalize to all types of speech during real-world BCI usage. Submissions that violate this rule are not eligible for prize money.
- Different language modeling approach (e.g., neural LM instead of ngram)
- Model ensembling
- Test time augmentation
- Test time adaptation (i.e., continuous finetuning)
- Different tokenization strategies (e.g., syllables or BPE instead of phonemes)
- Transfer learning from other datasets (e.g., ASR datasets or the [Brain-to-Text '24](https://eval.ai/web/challenges/challenge-page/2099) T12 neural dataset)

## Dataset

The dataset used in this competition consists of 10,948 sentences spoken by a single research participant as described in [Card et al. “An Accurate and Rapidly Calibrating Speech Neuroprosthesis” (2024) *New England Journal of Medicine*](https://www.nejm.org/doi/full/10.1056/NEJMoa2314132). For each sentence, we provide the transcript of what the participant was attempting to say, along with the corresponding time series of neural spiking activity recorded from 256 microelectrodes in speech motor cortex. The dataset contains predefined “*train*”, “*val*”, and “*test*” partitions. The train and val partitions include the sentence labels, and you may repartition them if desired. The “*test*” partition does not include sentence labels and will be used for competition evaluation. The dataset can be downloaded either from this Kaggle competition page (see `Data` tab) or from [Dryad](https://doi.org/10.5061/dryad.dncjsxm85).

The dataset contains a mixture of speaking strategies and sentence corpuses (see table, below) on a block-by-block basis. A description of which blocks correspond to which corpuses and data splits can be found [here](https://github.com/Neuroprosthetics-Lab/nejm-brain-to-text/blob/main/data/t15_copyTaskData_description.csv). To make the competition more challenging and better approximate real-world use, we do not provide labels as to the speaking strategy for each block.

There are some notable differences between this dataset and the one used in the Brain-to-Text 2024 challenge:
| <img width=300/> | Brain-to-Text '24 | Brain-to-Text '25 |
| --- | --- | --- |
| **Participant** | 'T12' | 'T15' |
| **Neural recordings** | 128 intracortical electrodes in speech motor cortex, 128 intracortical electrodes in inferior frontal gyrus | 256 intracortical recording electrodes in speech motor cortex |
| **Dataset period** | 25 sessions spanning 4 months | 45 sessions spanning 20 months |
| **Number of Sentences** | 12,100 | 10,948 |
| **Sentence corpus** | Switchboard | 50-word vocabulary, Switchboard, Openwebtext2, Harvard sentences, custom high-frequency word sentences, random word sentences |
| **Speaking strategy** | Attempted vocalized | Attempted vocalized or attempted silent |
| **Speaking rate** | ~62 words per minute | ~30 words per minute (attempted vocalized) or ~50 words per minute (attempted silent) |

## Competition Timeline

Competitors may submit their submission `.csv` files until the competition ends on December 31st, 2025 at 11:59 pm (Pacific Standard Time). After the competition ends, entrants who would like to be considered for the cash prize will have until January 31st, 2026, to submit their one-page summary and code. Details on how to submit this information will be provided at a later date. Winners will be chosen and then announced in February 2026.