{
  "fetched_at": "2026-10-19T21:39:15Z",
  "source_url": "https://www.kaggle.com/competitions/cs4740-fa21-p2/data",
  "pages": [
    {
      "name": "data-description",
      "content": "The dataset consists of two json files: *train.json* and *test.json*. These files are different from the original release of the CoNNL-2003 since we have already processed and tokenized them for you. Hence, the documents are represented as a list of strings. The dataset is **not** split into separate training and development/validation sets. You will need to do this yourself as needed using train set.\n\nThe train file contain the following 4 fields (each is a nested list): \n\n  - **'text'** - actual input tokens\n\n  - **'NER'** - the entity tag (LOC/MISC/O/ORG/PER)\n\n  - **'POS'** - the parts of speech tag (will be handy for feature engineering of the MEMM model)\n\n  - **'index'** - index of the token in the dataset\n\nThe test data only has 'text', 'POS' and 'index' fields. You will need to submit your prediction of 'NER' tag to the Kaggle.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Description",
      "content": "In this project, you will implement a model that named entities in text and tags it with the appropriate label. Particularly, the task of this project is **Named Entity Recognition**. A primer on this task is provided further on. The given dataset is a modified version of the CoNLL-2003 ([Sang et al](https://arxiv.org/pdf/cs/0306050v1.pdf)) dataset. Please use the datasets that we have released to you instead of versions found online as we have made simplifications to the dataset for your benifit. Your task is to develop NLP models to identify these named entities automatically. We will treat this as a **sequence-tagging task**: for each token in the input text, assign one of the following 5 labels: **ORG** (Organization), **PER** (Person), **LOC** (Location), **MISC** (Miscellaneous), and **O** (Not Named Entity). More information about the dataset is provided later\n\nFor this project, you will implement two sequence labeling approaches:\n- Model 1 : a Hidden Markov Model (HMM)\n- Model 2 : a Maximum Entropy Markov Model (MEMM), which is an adaptation of an HMM in which a Logistic Regression classifier (also known as a MaxEnt classifier) is used to obtain the lexical generation probabilities (i.e., the observation/emission probability matrix, so \"observations\" == \"emissions\" == \"lexical generations\"). Feature engineering is strongly suggested for this task!\n\nImplementation of the Viterbi algorithm (for finding the most likely tag sequence to assign to an input text) is required for both models above, so make sure that you understand it ASAP.\n\nYou will implement and train two sequence tagging models, generate your predictions for the provided test set, and submit them to **Kaggle**. Please enter all code in your Colab notebook and answer all the questions in the supporting document.\n\nTo refresh your memory on HMMs, MEMMs, and Viterbi you can refer to **Jurafsky & Martin Ch. 8.3–8.5** and the lecture slides which can be found on EdStem.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "Evaluation And Submission",
      "content": "<h2>Evaluation Metric</h2>\nThe standard measures to report for Named Entity Recognition or NER are Recall, Precision, and F1 score evaluated at the **entity level**, not at the token level. \n\nIf P and T are the sets of predicted and true named entity spans respectively, then\n\n####<center>$$\\text{Precision} = \\frac{|\\text{P}\\;\\cap\\;\\text{T}|}{|\\text{P}|}$$ \n$$\\text{Recall} = \\frac{|\\text{P}\\;\\cap\\;\\text{T}|}{|\\text{T}|}$$ $$\\text{F1} = \\frac{2 * \\text{Precision} * \\text{Recall}}{\\text{Precision} + \\text{Recall}}$$</center>\nThe F1 metric weights Recall and Precision equally, and a good retrieval algorithm will maximize both Precision and Recall simultaneously. Thus, moderately good performance on both will be favored over extremely good performance on one and poor performance on the other.\n\nFor each label, e.g. *LOC*, *MISC*, *ORG* and *PER*, we calculate the F1 score as shown above, and take the mean of all these F1 scores to get the **Entity Level Mean F1** score for the test set. If *N* is the total number of labels, then\n\n####<center>$$\\text{Entity Level Mean F1} = \\frac{\\sum_{i = 1}^{N} \\text{F1} \\text{ for label}\\;i}{N}$$. </center>\n<h2>Submission Format</h2>\n\nYour submission to Kaggle should be a CSV file consisting of five lines and two columns. The first line is a fixed header, and each of the rest four lines corresponds to one of the four types of named entities. The first column is the label identifier *Id*, i.e. one of LOC, MISC, ORG or PER and the second column *Predicted* is a list of entities, **separated by single space**, that you predict to be of that type. Each entity is specified by its starting and ending index, **concatenated by a hypen**, as given in the test corpus. \n\nNOTE: Ensure that there are **no** rows with *Id* = \"O\" in your Kaggle Submission.\n\n<p>The file should contain a header and have the following format:</p>\n<pre>Id,Expected<br>LOC,18-18 28-28<br>MISC,23-23<br>ORG,13-13<br>PER,15-16<br></pre>",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    },
    {
      "name": "rules",
      "content": "Work in teams of 2. Do not share code with anyone other than your partner.",
      "isPublished": false,
      "hasIsPublished": false,
      "postTitle": "",
      "hasPostTitle": false,
      "mimeType": "",
      "hasMimeType": false
    }
  ]
}