{
  "id": 230436,
  "title": "Labelling Question",
  "url": "/competitions/coleridgeinitiative-show-us-the-data/discussion/230436",
  "author_name": "",
  "post_date": "2021-04-03T16:59:34.487958600Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi!</p>\n<p>I'm poking around various ways of finding novel dataset mentions, and one method picked out \"the National Land Cover Dataset\". This does not seem to be one of the labels for the document (in fact, I don't see it in the list of possible datasets at all!), but the paper pretty clearly states:</p>\n<blockquote>\n  <p>We used bird abundance data from the NABBS, land cover data from the National Land Cover Dataset, and climate data from the National Climatic Data Center to model relative waterbird abundance as a function of environmental variables at three different spatial scales.</p>\n</blockquote>\n<p>NAABS is in the train labels, but not this land cover dataset. There's also no mention of the \"National Climatic Data Center\" data, but that might be a huge number of various sources. So I guess my question is about how the train dataset was labelled. In particular, are there labels <em>intentionally</em> left off the train set to prevent simple string matching, or is this one case just a matter of labeller error? </p>\n<p>The ID of this document is de53085a-e99c-41ef-9c02-5be28e6ce917.</p>",
  "messages": [
    {
      "id": "1261994",
      "postDate": "04/03/2021 16:59:34",
      "content": "<p>Hi!</p>\n<p>I'm poking around various ways of finding novel dataset mentions, and one method picked out \"the National Land Cover Dataset\". This does not seem to be one of the labels for the document (in fact, I don't see it in the list of possible datasets at all!), but the paper pretty clearly states:</p>\n<blockquote>\n  <p>We used bird abundance data from the NABBS, land cover data from the National Land Cover Dataset, and climate data from the National Climatic Data Center to model relative waterbird abundance as a function of environmental variables at three different spatial scales.</p>\n</blockquote>\n<p>NAABS is in the train labels, but not this land cover dataset. There's also no mention of the \"National Climatic Data Center\" data, but that might be a huge number of various sources. So I guess my question is about how the train dataset was labelled. In particular, are there labels <em>intentionally</em> left off the train set to prevent simple string matching, or is this one case just a matter of labeller error? </p>\n<p>The ID of this document is de53085a-e99c-41ef-9c02-5be28e6ce917.</p>",
      "rawMarkdown": "Hi!\n\nI'm poking around various ways of finding novel dataset mentions, and one method picked out \"the National Land Cover Dataset\". This does not seem to be one of the labels for the document (in fact, I don't see it in the list of possible datasets at all!), but the paper pretty clearly states:\n\n> We used bird abundance data from the NABBS, land cover data from the National Land Cover Dataset, and climate data from the National Climatic Data Center to model relative waterbird abundance as a function of environmental variables at three different spatial scales.\n\nNAABS is in the train labels, but not this land cover dataset. There's also no mention of the \"National Climatic Data Center\" data, but that might be a huge number of various sources. So I guess my question is about how the train dataset was labelled. In particular, are there labels *intentionally* left off the train set to prevent simple string matching, or is this one case just a matter of labeller error? \n\n\n The ID of this document is de53085a-e99c-41ef-9c02-5be28e6ce917.",
      "votes": null
    },
    {
      "id": "1262050",
      "postDate": "04/03/2021 18:04:51",
      "content": "<p>Indeed, there were many cases where my model was able to pick out a novel dataset mentioned in the text that was not originally labeled, resulting in a false positive. <br>\nI think that in the end the best model will be the best at mimicking the labelling process, whatever the kind of logic behind that is.</p>",
      "rawMarkdown": "Indeed, there were many cases where my model was able to pick out a novel dataset mentioned in the text that was not originally labeled, resulting in a false positive. \nI think that in the end the best model will be the best at mimicking the labelling process, whatever the kind of logic behind that is.",
      "votes": null
    },
    {
      "id": "1263557",
      "postDate": "04/05/2021 14:06:02",
      "content": "<p>Hi! Good question. Yes, in some training documents (namely, those that are also in the public test set) there are labels intentionally left out of train, as mentioned on the Data page.</p>",
      "rawMarkdown": "Hi! Good question. Yes, in some training documents (namely, those that are also in the public test set) there are labels intentionally left out of train, as mentioned on the Data page.",
      "votes": null
    },
    {
      "id": "1275079",
      "postDate": "04/15/2021 22:32:14",
      "content": "<p>Is it possible to reveal the identifier of at least one publication that is guaranteed to be fully labeled in the training file? Please see <a href=\"https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/232867\" target=\"_blank\">https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/232867</a> Thanks!</p>",
      "rawMarkdown": "Is it possible to reveal the identifier of at least one publication that is guaranteed to be fully labeled in the training file? Please see https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/232867 Thanks!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1262050,
      "author_name": "suicaokhoailang",
      "author_url": "",
      "post_date": "04/03/2021 18:04:51",
      "content": "<p>Indeed, there were many cases where my model was able to pick out a novel dataset mentioned in the text that was not originally labeled, resulting in a false positive. <br>\nI think that in the end the best model will be the best at mimicking the labelling process, whatever the kind of logic behind that is.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1263557,
      "author_name": "philculliton",
      "author_url": "",
      "post_date": "04/05/2021 14:06:02",
      "content": "<p>Hi! Good question. Yes, in some training documents (namely, those that are also in the public test set) there are labels intentionally left out of train, as mentioned on the Data page.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1275079,
          "author_name": "eguidotti",
          "author_url": "",
          "post_date": "04/15/2021 22:32:14",
          "content": "<p>Is it possible to reveal the identifier of at least one publication that is guaranteed to be fully labeled in the training file? Please see <a href=\"https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/232867\" target=\"_blank\">https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/232867</a> Thanks!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1261994": "Hi!\n\nI'm poking around various ways of finding novel dataset mentions, and one method picked out \"the National Land Cover Dataset\". This does not seem to be one of the labels for the document (in fact, I don't see it in the list of possible datasets at all!), but the paper pretty clearly states:\n\n> We used bird abundance data from the NABBS, land cover data from the National Land Cover Dataset, and climate data from the National Climatic Data Center to model relative waterbird abundance as a function of environmental variables at three different spatial scales.\n\nNAABS is in the train labels, but not this land cover dataset. There's also no mention of the \"National Climatic Data Center\" data, but that might be a huge number of various sources. So I guess my question is about how the train dataset was labelled. In particular, are there labels *intentionally* left off the train set to prevent simple string matching, or is this one case just a matter of labeller error? \n\n\n The ID of this document is de53085a-e99c-41ef-9c02-5be28e6ce917.",
    "1262050": "Indeed, there were many cases where my model was able to pick out a novel dataset mentioned in the text that was not originally labeled, resulting in a false positive. \nI think that in the end the best model will be the best at mimicking the labelling process, whatever the kind of logic behind that is.",
    "1263557": "Hi! Good question. Yes, in some training documents (namely, those that are also in the public test set) there are labels intentionally left out of train, as mentioned on the Data page.",
    "1275079": "Is it possible to reveal the identifier of at least one publication that is guaranteed to be fully labeled in the training file? Please see https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/232867 Thanks!"
  },
  "source": "meta"
}