Solution submissions must be in the same file format as the disambiguated training data. Submission Reproducibility Requirements Solution submissions and model code will be accepted throughout the competition, but will be required by the release of the model evaluation set (see Timeline). The model code submission must contain all code, data, and parameter settings necessary to evaluate your models on validation set items, and include a README file with step-by-step instructions on how to do so. The purpose of this reproducibility requirement is to ensure that solutions do not use any manual annotation of product mentions and product candidates. The model submission is required to be eligible to win prize money (see Rules) .