{
  "id": 230522,
  "title": "The imbalance of NER",
  "url": "/competitions/coleridgeinitiative-show-us-the-data/discussion/230522",
  "author_name": "",
  "post_date": "2021-04-04T08:07:57.587894600Z",
  "votes": 7,
  "comment_count": 7,
  "views": 0,
  "content": "<p>If I use NER method, I need to divide the long text into many sections(sliding windows). Some sections will contain several entities (positive cases), and some sections do not contain entities at all (negative cases). The ratio of positive and negative may reach up to 1:10. <br>\nFor better results, can I remove the negative examples when training, or balance them, or even ignore them?</p>",
  "messages": [
    {
      "id": "1262398",
      "postDate": "04/04/2021 08:07:57",
      "content": "<p>If I use NER method, I need to divide the long text into many sections(sliding windows). Some sections will contain several entities (positive cases), and some sections do not contain entities at all (negative cases). The ratio of positive and negative may reach up to 1:10. <br>\nFor better results, can I remove the negative examples when training, or balance them, or even ignore them?</p>",
      "rawMarkdown": "If I use NER method, I need to divide the long text into many sections(sliding windows). Some sections will contain several entities (positive cases), and some sections do not contain entities at all (negative cases). The ratio of positive and negative may reach up to 1:10. \nFor better results, can I remove the negative examples when training, or balance them, or even ignore them?",
      "votes": null
    },
    {
      "id": "1263139",
      "postDate": "04/05/2021 05:51:19",
      "content": "<p>result:<br>\nbalanced sampling: cv precision0.95 recall0.97 lb0.587<br>\nall data: cv precision0.93 recall0.97 lb0.591</p>",
      "rawMarkdown": "result:\nbalanced sampling: cv precision0.95 recall0.97 lb0.587\nall data: cv precision0.93 recall0.97 lb0.591",
      "votes": null
    },
    {
      "id": "1269981",
      "postDate": "04/11/2021 06:34:23",
      "content": "<p>Thanks for the results. What model do you use to achieve this score?</p>",
      "rawMarkdown": "Thanks for the results. What model do you use to achieve this score?",
      "votes": null
    },
    {
      "id": "1270008",
      "postDate": "04/11/2021 06:58:16",
      "content": "<p>bert base cased</p>",
      "rawMarkdown": "bert base cased",
      "votes": null
    },
    {
      "id": "1270020",
      "postDate": "04/11/2021 07:16:06",
      "content": "<p>Then your model combined with text matching may achieve more than LB 7.01. Because the single NER  model in <a href=\"url\" target=\"_blank\">https://www.kaggle.com/tungmphung/coleridge-matching-bert-ner</a>  only scored 0.527.</p>",
      "rawMarkdown": "Then your model combined with text matching may achieve more than LB 7.01. Because the single NER  model in [https://www.kaggle.com/tungmphung/coleridge-matching-bert-ner](url)  only scored 0.527.",
      "votes": null
    },
    {
      "id": "1270171",
      "postDate": "04/11/2021 11:32:13",
      "content": "<p>Thanks, I will try later😃</p>",
      "rawMarkdown": "Thanks, I will try later😃",
      "votes": null
    },
    {
      "id": "1273167",
      "postDate": "04/14/2021 06:22:33",
      "content": "<p>It will be better to increase weights of positive class (or copy positive examples several times). With this method, you use all training data, without loosing any data, so results should be better.</p>",
      "rawMarkdown": "It will be better to increase weights of positive class (or copy positive examples several times). With this method, you use all training data, without loosing any data, so results should be better.",
      "votes": null
    },
    {
      "id": "1273171",
      "postDate": "04/14/2021 06:25:43",
      "content": "<p>Another trick is use balanced dataset for first epochs of neural network training, and use original dataset for last epochs. With this method, network didn't ignore minor class during training, and model learns proper balance (no distribution shift, introduced by balancing)</p>",
      "rawMarkdown": "Another trick is use balanced dataset for first epochs of neural network training, and use original dataset for last epochs. With this method, network didn't ignore minor class during training, and model learns proper balance (no distribution shift, introduced by balancing)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1263139,
      "author_name": "zzy990106",
      "author_url": "",
      "post_date": "04/05/2021 05:51:19",
      "content": "<p>result:<br>\nbalanced sampling: cv precision0.95 recall0.97 lb0.587<br>\nall data: cv precision0.93 recall0.97 lb0.591</p>",
      "votes": null,
      "replies": [
        {
          "id": 1269981,
          "author_name": "a871435632",
          "author_url": "",
          "post_date": "04/11/2021 06:34:23",
          "content": "<p>Thanks for the results. What model do you use to achieve this score?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1270008,
          "author_name": "zzy990106",
          "author_url": "",
          "post_date": "04/11/2021 06:58:16",
          "content": "<p>bert base cased</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1270020,
          "author_name": "a871435632",
          "author_url": "",
          "post_date": "04/11/2021 07:16:06",
          "content": "<p>Then your model combined with text matching may achieve more than LB 7.01. Because the single NER  model in <a href=\"url\" target=\"_blank\">https://www.kaggle.com/tungmphung/coleridge-matching-bert-ner</a>  only scored 0.527.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1270171,
          "author_name": "zzy990106",
          "author_url": "",
          "post_date": "04/11/2021 11:32:13",
          "content": "<p>Thanks, I will try later😃</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1273167,
      "author_name": "zavodrobotov",
      "author_url": "",
      "post_date": "04/14/2021 06:22:33",
      "content": "<p>It will be better to increase weights of positive class (or copy positive examples several times). With this method, you use all training data, without loosing any data, so results should be better.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1273171,
      "author_name": "zavodrobotov",
      "author_url": "",
      "post_date": "04/14/2021 06:25:43",
      "content": "<p>Another trick is use balanced dataset for first epochs of neural network training, and use original dataset for last epochs. With this method, network didn't ignore minor class during training, and model learns proper balance (no distribution shift, introduced by balancing)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1262398": "If I use NER method, I need to divide the long text into many sections(sliding windows). Some sections will contain several entities (positive cases), and some sections do not contain entities at all (negative cases). The ratio of positive and negative may reach up to 1:10. \nFor better results, can I remove the negative examples when training, or balance them, or even ignore them?",
    "1263139": "result:\nbalanced sampling: cv precision0.95 recall0.97 lb0.587\nall data: cv precision0.93 recall0.97 lb0.591",
    "1269981": "Thanks for the results. What model do you use to achieve this score?",
    "1270008": "bert base cased",
    "1270020": "Then your model combined with text matching may achieve more than LB 7.01. Because the single NER  model in [https://www.kaggle.com/tungmphung/coleridge-matching-bert-ner](url)  only scored 0.527.",
    "1270171": "Thanks, I will try later😃",
    "1273167": "It will be better to increase weights of positive class (or copy positive examples several times). With this method, you use all training data, without loosing any data, so results should be better.",
    "1273171": "Another trick is use balanced dataset for first epochs of neural network training, and use original dataset for last epochs. With this method, network didn't ignore minor class during training, and model learns proper balance (no distribution shift, introduced by balancing)"
  },
  "source": "meta"
}