{
  "id": 415793,
  "title": "Request for Opinion of Kaggle Team",
  "url": "/competitions/benetech-making-graphs-accessible/discussion/415793",
  "author_name": "",
  "post_date": "2023-06-08T05:40:56.638928300Z",
  "votes": 18,
  "comment_count": 1,
  "views": 0,
  "content": "<p><a href=\"https://www.kaggle.com/ryanholbrook\" target=\"_blank\">@ryanholbrook</a> <a href=\"https://www.kaggle.com/maggiemd\" target=\"_blank\">@maggiemd</a><br>\nThere is a great deal of confusion regarding the use of external data in this competition.<br>\nThe threads is scattered, so I'll organize the host's doings below.<br>\nI would like to hear Kaggle's side of the issue.</p>\n<h2>Background of the issue</h2>\n<p>First, from the start of the competition, the host recommended the use of the ICDAR dataset by stating the following in the Dataset Description Page: </p>\n<blockquote>\n  <p>Additional Resources<br>\n  This competition was inspired by the CHART-Info competition series. You may find resources (such as additional training data) from those competitions useful.</p>\n</blockquote>\n<p>Although the winner lisence is MIT, this dataset is under CC BY-NC-SA 4.0 and not available for commercial use. </p>\n<p>Just a month ago, I asked the host in <a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/407485\" target=\"_blank\">this thread</a>. In response, the host said, \"Benetech has no intention of disallowing the use of CHART-Info / ICDAR datasets for this competition. Your valuable time and effort working with this data will not go to waste\". As a result, we have been using this dataset to develop our model.</p>\n<p>The day before yesterday, seven days before the competition deadline, the hosts issued <a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275\" target=\"_blank\">a statement</a> prohibiting the use of that dataset. Furthermore, they began to say, \"It is the responsibility of each contestant to ensure that, given the possibility of remuneration, contestant understands any existing restrictions on the data set(s) employed\". At this time, there was not even any apology from the host. And now the host has remained silent to all protests in the thread.</p>\n<h2>What I want the Kaggle Team to understand</h2>\n<p>Considering that the official data set is about 1,000 non-synthetic images and the external data set is about 20,000 images, it should be clear how much the availability of this dataset will affect the strategy and pipeline. This incredible rule change will do irreparable damage to some participants. In fact, I have begun to create a pipeline that does not rely on external data, just in case, but there are already so many problems that I have no hope of replicating my original solution.</p>\n<p>Many participants, including myself, believe that this whole fiasco is entirely the host's fault. As our responsibility, we have properly understood the license for over a month now, inquired with the host about the license, and obtained permission to use it from the host. Meanwhile, the host changed an important rule near the end and remain silent. Do they not have a responsibility to run the competition fairly and smoothly? Of course, it is unfair to ban the dataset this close to the deadline or to remove users using the dataset from the LB. If anyone should be disadvantaged, it should be the host. Furthermore, many participants in <a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275\" target=\"_blank\">this thread</a> have made various suggestions to the host.</p>\n<p>I want to hear the opinion of the Kaggle Team. Is this series of host responses appropriate? Are our three months of time and effort, spent trusting the host's statements, wasted by the host's irresponsible rule changes? Thank you. </p>",
  "messages": [
    {
      "id": "2292180",
      "postDate": "06/08/2023 05:40:56",
      "content": "<p><a href=\"https://www.kaggle.com/ryanholbrook\" target=\"_blank\">@ryanholbrook</a> <a href=\"https://www.kaggle.com/maggiemd\" target=\"_blank\">@maggiemd</a><br>\nThere is a great deal of confusion regarding the use of external data in this competition.<br>\nThe threads is scattered, so I'll organize the host's doings below.<br>\nI would like to hear Kaggle's side of the issue.</p>\n<h2>Background of the issue</h2>\n<p>First, from the start of the competition, the host recommended the use of the ICDAR dataset by stating the following in the Dataset Description Page: </p>\n<blockquote>\n  <p>Additional Resources<br>\n  This competition was inspired by the CHART-Info competition series. You may find resources (such as additional training data) from those competitions useful.</p>\n</blockquote>\n<p>Although the winner lisence is MIT, this dataset is under CC BY-NC-SA 4.0 and not available for commercial use. </p>\n<p>Just a month ago, I asked the host in <a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/407485\" target=\"_blank\">this thread</a>. In response, the host said, \"Benetech has no intention of disallowing the use of CHART-Info / ICDAR datasets for this competition. Your valuable time and effort working with this data will not go to waste\". As a result, we have been using this dataset to develop our model.</p>\n<p>The day before yesterday, seven days before the competition deadline, the hosts issued <a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275\" target=\"_blank\">a statement</a> prohibiting the use of that dataset. Furthermore, they began to say, \"It is the responsibility of each contestant to ensure that, given the possibility of remuneration, contestant understands any existing restrictions on the data set(s) employed\". At this time, there was not even any apology from the host. And now the host has remained silent to all protests in the thread.</p>\n<h2>What I want the Kaggle Team to understand</h2>\n<p>Considering that the official data set is about 1,000 non-synthetic images and the external data set is about 20,000 images, it should be clear how much the availability of this dataset will affect the strategy and pipeline. This incredible rule change will do irreparable damage to some participants. In fact, I have begun to create a pipeline that does not rely on external data, just in case, but there are already so many problems that I have no hope of replicating my original solution.</p>\n<p>Many participants, including myself, believe that this whole fiasco is entirely the host's fault. As our responsibility, we have properly understood the license for over a month now, inquired with the host about the license, and obtained permission to use it from the host. Meanwhile, the host changed an important rule near the end and remain silent. Do they not have a responsibility to run the competition fairly and smoothly? Of course, it is unfair to ban the dataset this close to the deadline or to remove users using the dataset from the LB. If anyone should be disadvantaged, it should be the host. Furthermore, many participants in <a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275\" target=\"_blank\">this thread</a> have made various suggestions to the host.</p>\n<p>I want to hear the opinion of the Kaggle Team. Is this series of host responses appropriate? Are our three months of time and effort, spent trusting the host's statements, wasted by the host's irresponsible rule changes? Thank you. </p>",
      "rawMarkdown": "ryanholbrook @maggiemd\nThere is a great deal of confusion regarding the use of external data in this competition.\nThe threads is scattered, so I'll organize the host's doings below.\nI would like to hear Kaggle's side of the issue.\n\n## Background of the issue\nFirst, from the start of the competition, the host recommended the use of the ICDAR dataset by stating the following in the Dataset Description Page: \n> Additional Resources\nThis competition was inspired by the CHART-Info competition series. You may find resources (such as additional training data) from those competitions useful.\n\nAlthough the winner lisence is MIT, this dataset is under CC BY-NC-SA 4.0 and not available for commercial use. \n\nJust a month ago, I asked the host in [this thread](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/407485). In response, the host said, \"Benetech has no intention of disallowing the use of CHART-Info / ICDAR datasets for this competition. Your valuable time and effort working with this data will not go to waste\". As a result, we have been using this dataset to develop our model.\n\nThe day before yesterday, seven days before the competition deadline, the hosts issued [a statement](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275) prohibiting the use of that dataset. Furthermore, they began to say, \"It is the responsibility of each contestant to ensure that, given the possibility of remuneration, contestant understands any existing restrictions on the data set(s) employed\". At this time, there was not even any apology from the host. And now the host has remained silent to all protests in the thread.\n\n## What I want the Kaggle Team to understand\nConsidering that the official data set is about 1,000 non-synthetic images and the external data set is about 20,000 images, it should be clear how much the availability of this dataset will affect the strategy and pipeline. This incredible rule change will do irreparable damage to some participants. In fact, I have begun to create a pipeline that does not rely on external data, just in case, but there are already so many problems that I have no hope of replicating my original solution.\n\nMany participants, including myself, believe that this whole fiasco is entirely the host's fault. As our responsibility, we have properly understood the license for over a month now, inquired with the host about the license, and obtained permission to use it from the host. Meanwhile, the host changed an important rule near the end and remain silent. Do they not have a responsibility to run the competition fairly and smoothly? Of course, it is unfair to ban the dataset this close to the deadline or to remove users using the dataset from the LB. If anyone should be disadvantaged, it should be the host. Furthermore, many participants in [this thread](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275) have made various suggestions to the host.\n\n\nI want to hear the opinion of the Kaggle Team. Is this series of host responses appropriate? Are our three months of time and effort, spent trusting the host's statements, wasted by the host's irresponsible rule changes? Thank you.",
      "votes": null
    },
    {
      "id": "2292615",
      "postDate": "06/08/2023 13:55:06",
      "content": "<p>All great questions. I do not understand how it is acceptable to still not have any responses or clarifications. I would have thought after the massive response and the many discussions brought up that it would be handled. Here are a few examples of these discussions:</p>\n<p><a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415476\" target=\"_blank\">https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415476</a><br>\n<a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415474\" target=\"_blank\">https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415474</a><br>\n<a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275\" target=\"_blank\">https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275</a></p>",
      "rawMarkdown": "All great questions. I do not understand how it is acceptable to still not have any responses or clarifications. I would have thought after the massive response and the many discussions brought up that it would be handled. Here are a few examples of these discussions:\n\nhttps://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415476\nhttps://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415474\nhttps://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2292615,
      "author_name": "cody11null",
      "author_url": "",
      "post_date": "06/08/2023 13:55:06",
      "content": "<p>All great questions. I do not understand how it is acceptable to still not have any responses or clarifications. I would have thought after the massive response and the many discussions brought up that it would be handled. Here are a few examples of these discussions:</p>\n<p><a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415476\" target=\"_blank\">https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415476</a><br>\n<a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415474\" target=\"_blank\">https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415474</a><br>\n<a href=\"https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275\" target=\"_blank\">https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2292180": "ryanholbrook @maggiemd\nThere is a great deal of confusion regarding the use of external data in this competition.\nThe threads is scattered, so I'll organize the host's doings below.\nI would like to hear Kaggle's side of the issue.\n\n## Background of the issue\nFirst, from the start of the competition, the host recommended the use of the ICDAR dataset by stating the following in the Dataset Description Page: \n> Additional Resources\nThis competition was inspired by the CHART-Info competition series. You may find resources (such as additional training data) from those competitions useful.\n\nAlthough the winner lisence is MIT, this dataset is under CC BY-NC-SA 4.0 and not available for commercial use. \n\nJust a month ago, I asked the host in [this thread](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/407485). In response, the host said, \"Benetech has no intention of disallowing the use of CHART-Info / ICDAR datasets for this competition. Your valuable time and effort working with this data will not go to waste\". As a result, we have been using this dataset to develop our model.\n\nThe day before yesterday, seven days before the competition deadline, the hosts issued [a statement](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275) prohibiting the use of that dataset. Furthermore, they began to say, \"It is the responsibility of each contestant to ensure that, given the possibility of remuneration, contestant understands any existing restrictions on the data set(s) employed\". At this time, there was not even any apology from the host. And now the host has remained silent to all protests in the thread.\n\n## What I want the Kaggle Team to understand\nConsidering that the official data set is about 1,000 non-synthetic images and the external data set is about 20,000 images, it should be clear how much the availability of this dataset will affect the strategy and pipeline. This incredible rule change will do irreparable damage to some participants. In fact, I have begun to create a pipeline that does not rely on external data, just in case, but there are already so many problems that I have no hope of replicating my original solution.\n\nMany participants, including myself, believe that this whole fiasco is entirely the host's fault. As our responsibility, we have properly understood the license for over a month now, inquired with the host about the license, and obtained permission to use it from the host. Meanwhile, the host changed an important rule near the end and remain silent. Do they not have a responsibility to run the competition fairly and smoothly? Of course, it is unfair to ban the dataset this close to the deadline or to remove users using the dataset from the LB. If anyone should be disadvantaged, it should be the host. Furthermore, many participants in [this thread](https://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275) have made various suggestions to the host.\n\n\nI want to hear the opinion of the Kaggle Team. Is this series of host responses appropriate? Are our three months of time and effort, spent trusting the host's statements, wasted by the host's irresponsible rule changes? Thank you.",
    "2292615": "All great questions. I do not understand how it is acceptable to still not have any responses or clarifications. I would have thought after the massive response and the many discussions brought up that it would be handled. Here are a few examples of these discussions:\n\nhttps://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415476\nhttps://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415474\nhttps://www.kaggle.com/competitions/benetech-making-graphs-accessible/discussion/415275"
  },
  "source": "meta"
}