{
  "id": 343825,
  "title": "Is this competition evaluation subjective?",
  "url": "/competitions/big-data-derby-2022/discussion/343825",
  "author_name": "",
  "post_date": "2022-08-12T17:51:47.140390Z",
  "votes": 5,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Welcome to this competition 👋🏼</p>\n<p>Unlike most Kaggle competitions where there is a quantitative measurement of how good your notebook is through an evaluation metric, this competition lacks a Leaderboard. </p>\n<p>Notebook submissions are evaluated on four components <strong>(description says 5 however but there is no fifth)</strong>: <code>Innovation</code>, <code>Relevance</code>, <code>Competence</code> and <code>Presentation</code> where 0 is the low score and 100 is the high score. </p>\n<p>Since these components are some how relative to evaluate it would be nice to have some feedback from the competition's hosts, maybe a panel of judges or any feedback on the final notebooks. </p>\n<p>Also, the competition states:</p>\n<pre><code>All notebooks submitted must be made public on or before the submission deadline to be eligible. If submitting as a team, all team members must be listed as collaborators on all notebooks submitted.\n</code></pre>\n<p>One question that comes to my mind is: if evaluation is somehow subjective and notebooks need to be made public then some teams could copy good insights/original ideas from other teams and publish as theirs (unless notebooks are made public very close to the competitions end).</p>",
  "messages": [
    {
      "id": "1896264",
      "postDate": "08/12/2022 17:51:47",
      "content": "<p>Welcome to this competition 👋🏼</p>\n<p>Unlike most Kaggle competitions where there is a quantitative measurement of how good your notebook is through an evaluation metric, this competition lacks a Leaderboard. </p>\n<p>Notebook submissions are evaluated on four components <strong>(description says 5 however but there is no fifth)</strong>: <code>Innovation</code>, <code>Relevance</code>, <code>Competence</code> and <code>Presentation</code> where 0 is the low score and 100 is the high score. </p>\n<p>Since these components are some how relative to evaluate it would be nice to have some feedback from the competition's hosts, maybe a panel of judges or any feedback on the final notebooks. </p>\n<p>Also, the competition states:</p>\n<pre><code>All notebooks submitted must be made public on or before the submission deadline to be eligible. If submitting as a team, all team members must be listed as collaborators on all notebooks submitted.\n</code></pre>\n<p>One question that comes to my mind is: if evaluation is somehow subjective and notebooks need to be made public then some teams could copy good insights/original ideas from other teams and publish as theirs (unless notebooks are made public very close to the competitions end).</p>",
      "rawMarkdown": "Welcome to this competition 👋🏼\n\nUnlike most Kaggle competitions where there is a quantitative measurement of how good your notebook is through an evaluation metric, this competition lacks a Leaderboard. \n\nNotebook submissions are evaluated on four components **(description says 5 however but there is no fifth)**: `Innovation`, `Relevance`, `Competence` and `Presentation` where 0 is the low score and 100 is the high score. \n\nSince these components are some how relative to evaluate it would be nice to have some feedback from the competition's hosts, maybe a panel of judges or any feedback on the final notebooks. \n\nAlso, the competition states:\n\n```\nAll notebooks submitted must be made public on or before the submission deadline to be eligible. If submitting as a team, all team members must be listed as collaborators on all notebooks submitted.\n```\n\nOne question that comes to my mind is: if evaluation is somehow subjective and notebooks need to be made public then some teams could copy good insights/original ideas from other teams and publish as theirs (unless notebooks are made public very close to the competitions end).",
      "votes": null
    },
    {
      "id": "1896400",
      "postDate": "08/12/2022 19:40:56",
      "content": "<p>The competition will be judged by a panel of 5, likely announced in a few weeks. </p>\n<p>Because the contest is as much about understanding and explaining, rather than a fully finished solution, we don't think there's much to worry about w/r/t people copying off one another for a final submission. However by making a notebook public, we all understand that it is open-source by nature, and those ideas could be intentionally shared.</p>",
      "rawMarkdown": "The competition will be judged by a panel of 5, likely announced in a few weeks. \n\nBecause the contest is as much about understanding and explaining, rather than a fully finished solution, we don't think there's much to worry about w/r/t people copying off one another for a final submission. However by making a notebook public, we all understand that it is open-source by nature, and those ideas could be intentionally shared.",
      "votes": null
    },
    {
      "id": "1896447",
      "postDate": "08/12/2022 20:26:39",
      "content": "<p>Thank you for explaining! ✔️</p>",
      "rawMarkdown": "Thank you for explaining! ✔️",
      "votes": null
    },
    {
      "id": "1905281",
      "postDate": "08/18/2022 22:20:40",
      "content": "<p>I highly recommend waiting until the last hour or two to make your notebook public. It may or may not make a real difference although posting early can only increase your risk. Kagglers are an honorable group by and large. At the same time, this is a competition and a zero-sum game.</p>",
      "rawMarkdown": "I highly recommend waiting until the last hour or two to make your notebook public. It may or may not make a real difference although posting early can only increase your risk. Kagglers are an honorable group by and large. At the same time, this is a competition and a zero-sum game.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1896400,
      "author_name": "nythanyra",
      "author_url": "",
      "post_date": "08/12/2022 19:40:56",
      "content": "<p>The competition will be judged by a panel of 5, likely announced in a few weeks. </p>\n<p>Because the contest is as much about understanding and explaining, rather than a fully finished solution, we don't think there's much to worry about w/r/t people copying off one another for a final submission. However by making a notebook public, we all understand that it is open-source by nature, and those ideas could be intentionally shared.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1896447,
          "author_name": "bellapisani",
          "author_url": "",
          "post_date": "08/12/2022 20:26:39",
          "content": "<p>Thank you for explaining! ✔️</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1905281,
      "author_name": "jpmiller",
      "author_url": "",
      "post_date": "08/18/2022 22:20:40",
      "content": "<p>I highly recommend waiting until the last hour or two to make your notebook public. It may or may not make a real difference although posting early can only increase your risk. Kagglers are an honorable group by and large. At the same time, this is a competition and a zero-sum game.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1896264": "Welcome to this competition 👋🏼\n\nUnlike most Kaggle competitions where there is a quantitative measurement of how good your notebook is through an evaluation metric, this competition lacks a Leaderboard. \n\nNotebook submissions are evaluated on four components **(description says 5 however but there is no fifth)**: `Innovation`, `Relevance`, `Competence` and `Presentation` where 0 is the low score and 100 is the high score. \n\nSince these components are some how relative to evaluate it would be nice to have some feedback from the competition's hosts, maybe a panel of judges or any feedback on the final notebooks. \n\nAlso, the competition states:\n\n```\nAll notebooks submitted must be made public on or before the submission deadline to be eligible. If submitting as a team, all team members must be listed as collaborators on all notebooks submitted.\n```\n\nOne question that comes to my mind is: if evaluation is somehow subjective and notebooks need to be made public then some teams could copy good insights/original ideas from other teams and publish as theirs (unless notebooks are made public very close to the competitions end).",
    "1896400": "The competition will be judged by a panel of 5, likely announced in a few weeks. \n\nBecause the contest is as much about understanding and explaining, rather than a fully finished solution, we don't think there's much to worry about w/r/t people copying off one another for a final submission. However by making a notebook public, we all understand that it is open-source by nature, and those ideas could be intentionally shared.",
    "1896447": "Thank you for explaining! ✔️",
    "1905281": "I highly recommend waiting until the last hour or two to make your notebook public. It may or may not make a real difference although posting early can only increase your risk. Kagglers are an honorable group by and large. At the same time, this is a competition and a zero-sum game."
  },
  "source": "meta"
}