{
  "id": 11251,
  "title": "Using external data",
  "url": "/competitions/datasciencebowl/discussion/11251",
  "author_name": "",
  "post_date": "2014-12-15T20:22:22.147Z",
  "votes": 1,
  "comment_count": 4,
  "views": 2829,
  "content": "<p>It seems that the goal of this challenge is of high importance. As stated in the description &quot;This is your chance to contribute to the health of the world&#8217;s oceans, one plankton at a time&quot;.&nbsp;</p>\n<p>That is why I don't understand the restrictive external data policy. Wouldn't it be preferable that the best possible achievable model will be utilized?&nbsp;(which may or may not require&nbsp;external data sources).</p>\n<p>Thank you for this challenge and for the opportunity!&nbsp;</p>",
  "messages": [
    {
      "id": "60273",
      "postDate": "12/15/2014 20:22:22",
      "content": "<p>It seems that the goal of this challenge is of high importance. As stated in the description &quot;This is your chance to contribute to the health of the world&#8217;s oceans, one plankton at a time&quot;.&nbsp;</p>\n<p>That is why I don't understand the restrictive external data policy. Wouldn't it be preferable that the best possible achievable model will be utilized?&nbsp;(which may or may not require&nbsp;external data sources).</p>\n<p>Thank you for this challenge and for the opportunity!&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60281",
      "postDate": "12/15/2014 21:01:49",
      "content": "<p>Hi Gideon,</p>\n<p>Great question. In this case, the rule is primarily intended to create a level playing field for participants. External data can&nbsp;present&nbsp;equality&nbsp;issues from a&nbsp;competition standpoint (e.g. via access to proprietary or restricted data, via paid data sources, or via payment to&nbsp;somebody else to label new training data). Even in the case where we approve public/free data sources, there remains&nbsp;a grey zone that can&nbsp;make&nbsp;interpretation ambiguous.</p>\n<p>Secondly, external data requires the host (in this case, Hatfield) to have ongoing access to the same type of external data. Why is this an issue? Let's say&nbsp;you find the <em>Encyclopedia of External Plankton Images</em> very helpful in improving your model. Now let's say Hatfield would like to add a new category to the classifier that is&nbsp;not in the <em>Encyclopedia... </em>now the model won't&nbsp;work&nbsp;as&nbsp;before because the associated external data does not&nbsp;exist. By training on their own data, in exactly the modality through which&nbsp;it is collected, they know that&nbsp;they will have continued access to a representative training set.</p>\n<p>This challenge asks you to make the most of what was collected and labeled here. The&nbsp;organizers believe that an accurate solution to the problem, as posed, will assist them greatly in their research.</p>\n<p>Thanks for participating!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60315",
      "postDate": "12/16/2014 02:55:39",
      "content": "<p>My question is kind of unrelated. I am concerned about people hand labelling the testing data. It wouldn't be too difficult for a team to hand label all the pictures that are difficult to classify by their algorithm and overfit their model to achieve a high score. Wouldn't it be better to mix in a few million dummy pictures (that get ignored during the LB calculation) with the testing data to make this more difficult?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60463",
      "postDate": "12/17/2014 11:10:32",
      "content": "<p>What about using outside DATA that is unrelated to the contest, such as the ImageNet Large Scale Visual Recognition Challenge?&nbsp;</p>\n<p>http://image-net.org/challenges/LSVRC/2012/index</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "60493",
      "postDate": "12/17/2014 15:32:33",
      "content": "<p>[quote=Gil Levi;60463]</p>\n<p>What about using outside DATA that is unrelated to the contest, such as the ImageNet Large Scale Visual Recognition Challenge?&nbsp;</p>\n<p>http://image-net.org/challenges/LSVRC/2012/index</p>\n<p>[/quote]</p>\n<p>This would still be external data, which is not allowed.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 60281,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "12/15/2014 21:01:49",
      "content": "<p>Hi Gideon,</p>\n<p>Great question. In this case, the rule is primarily intended to create a level playing field for participants. External data can&nbsp;present&nbsp;equality&nbsp;issues from a&nbsp;competition standpoint (e.g. via access to proprietary or restricted data, via paid data sources, or via payment to&nbsp;somebody else to label new training data). Even in the case where we approve public/free data sources, there remains&nbsp;a grey zone that can&nbsp;make&nbsp;interpretation ambiguous.</p>\n<p>Secondly, external data requires the host (in this case, Hatfield) to have ongoing access to the same type of external data. Why is this an issue? Let's say&nbsp;you find the <em>Encyclopedia of External Plankton Images</em> very helpful in improving your model. Now let's say Hatfield would like to add a new category to the classifier that is&nbsp;not in the <em>Encyclopedia... </em>now the model won't&nbsp;work&nbsp;as&nbsp;before because the associated external data does not&nbsp;exist. By training on their own data, in exactly the modality through which&nbsp;it is collected, they know that&nbsp;they will have continued access to a representative training set.</p>\n<p>This challenge asks you to make the most of what was collected and labeled here. The&nbsp;organizers believe that an accurate solution to the problem, as posed, will assist them greatly in their research.</p>\n<p>Thanks for participating!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60315,
      "author_name": "",
      "author_url": "",
      "post_date": "12/16/2014 02:55:39",
      "content": "<p>My question is kind of unrelated. I am concerned about people hand labelling the testing data. It wouldn't be too difficult for a team to hand label all the pictures that are difficult to classify by their algorithm and overfit their model to achieve a high score. Wouldn't it be better to mix in a few million dummy pictures (that get ignored during the LB calculation) with the testing data to make this more difficult?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60463,
      "author_name": "gillevi",
      "author_url": "",
      "post_date": "12/17/2014 11:10:32",
      "content": "<p>What about using outside DATA that is unrelated to the contest, such as the ImageNet Large Scale Visual Recognition Challenge?&nbsp;</p>\n<p>http://image-net.org/challenges/LSVRC/2012/index</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 60493,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "12/17/2014 15:32:33",
      "content": "<p>[quote=Gil Levi;60463]</p>\n<p>What about using outside DATA that is unrelated to the contest, such as the ImageNet Large Scale Visual Recognition Challenge?&nbsp;</p>\n<p>http://image-net.org/challenges/LSVRC/2012/index</p>\n<p>[/quote]</p>\n<p>This would still be external data, which is not allowed.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "60273": "",
    "60281": "",
    "60315": "",
    "60463": "",
    "60493": ""
  },
  "source": "meta"
}