The metric in this competition is the [word-level Jaccard score](https://en.wikipedia.org/wiki/Jaccard_index). A good description of Jaccard similarity for strings is [here](https://towardsdatascience.com/overview-of-text-similarity-metrics-3397c4601f50).

A Python implementation based on the links above, and matched with the output of the C# implementation on the back end, is provided below.

```
def jaccard(str1, str2): 
    a = set(str1.lower().split()) 
    b = set(str2.lower().split())
    c = a.intersection(b)
    return float(len(c)) / (len(a) + len(b) - len(c))
```
The formula for the overall metric, then, is:
$$
score = \frac{1}{n} \sum_{i=1}^n jaccard( gt\_i, dt\_i )
$$
where:
$$
n = \textrm{number of documents}
$$
$$
jaccard = \textrm{the function provided above}
$$
$$
gt\_i = \textrm{the ith ground truth}
$$
$$
dt\_i = \textrm{the ith prediction}
$$

## Submission File
For each ID in the test set, you must predict the string that best answers the provided `question` based on the `context`. Note that the selected text _needs_ to be **quoted** and **complete** to work correctly. Include punctuation, etc. - the above code splits ONLY on whitespace. The file should contain a header and have the following format:

    id,PredictionString
    8c8ee6504,"1"
    3163c22d0,"2 string"
    66aae423b,"4 word 6"
    722085a7b,"1"
    etc.