Evaluation Metric
The standard measures to report for Named Entity Recognition or NER are Recall, Precision, and F1 score evaluated at the **entity level**, not at the token level. 

If P and T are the sets of predicted and true named entity spans respectively, then

####
$$\text{Precision} = \frac{|\text{P}\;\cap\;\text{T}|}{|\text{P}|}$$ 
$$\text{Recall} = \frac{|\text{P}\;\cap\;\text{T}|}{|\text{T}|}$$ $$\text{F1} = \frac{2 * \text{Precision} * \text{Recall}}{\text{Precision} + \text{Recall}}$$
The F1 metric weights Recall and Precision equally, and a good retrieval algorithm will maximize both Precision and Recall simultaneously. Thus, moderately good performance on both will be favored over extremely good performance on one and poor performance on the other.

For each label, e.g. *LOC*, *MISC*, *ORG* and *PER*, we calculate the F1 score as shown above, and take the mean of all these F1 scores to get the **Entity Level Mean F1** score for the test set. If *N* is the total number of labels, then

####
$$\text{Entity Level Mean F1} = \frac{\sum_{i = 1}^{N} \text{F1} \text{ for label}\;i}{N}$$.
Submission Format
Your submission to Kaggle should be a CSV file consisting of five lines and two columns. The first line is a fixed header, and each of the rest four lines corresponds to one of the four types of named entities. The first column is the label identifier *Id*, i.e. one of LOC, MISC, ORG or PER and the second column *Predicted* is a list of entities, **separated by single space**, that you predict to be of that type. Each entity is specified by its starting and ending index, **concatenated by a hypen**, as given in the test corpus. 

NOTE: Ensure that there are **no** rows with *Id* = "O" in your Kaggle Submission.
The file should contain a header and have the following format:
Id,Expected
LOC,18-18 28-28
MISC,23-23
ORG,13-13
PER,15-16