{
  "id": 409643,
  "title": "Question regarding evaluation",
  "url": "/competitions/2023-kaggle-ai-report/discussion/409643",
  "author_name": "",
  "post_date": "2023-05-12T01:35:24.250124Z",
  "votes": 4,
  "comment_count": 2,
  "views": 0,
  "content": "<p>First, I would like to thank the organizers for this interesting competition which I believe its the first of its kind so far. I am a bit puzzled by the Evaluation section still. I see that to make the competition as objective as possible there is a grading rubric and an eligibility checklist. </p>\n<p>My question is if, ultimately, the leaderboard is determined by the judges (human score). The leaderboard as described in the <a href=\"https://www.kaggle.com/competitions/2023-kaggle-ai-report/overview/submission-instructions\" target=\"_blank\">Submission instructions</a> section, is just for the purposes stated (team merging, etc).</p>\n<p>Also:</p>\n<blockquote>\n  <p>Participants will also take part in a peer feedback process that involves reviewing three peer essays, and your submission will be assessed by an expert grading panel of seven Kaggle Grandmasters.</p>\n</blockquote>\n<p>Does this mean that competitors must review other competitors essays? In which way does this affect other competitors score?</p>\n<p>Thanks in advance</p>",
  "messages": [
    {
      "id": "2255770",
      "postDate": "05/12/2023 01:35:24",
      "content": "<p>First, I would like to thank the organizers for this interesting competition which I believe its the first of its kind so far. I am a bit puzzled by the Evaluation section still. I see that to make the competition as objective as possible there is a grading rubric and an eligibility checklist. </p>\n<p>My question is if, ultimately, the leaderboard is determined by the judges (human score). The leaderboard as described in the <a href=\"https://www.kaggle.com/competitions/2023-kaggle-ai-report/overview/submission-instructions\" target=\"_blank\">Submission instructions</a> section, is just for the purposes stated (team merging, etc).</p>\n<p>Also:</p>\n<blockquote>\n  <p>Participants will also take part in a peer feedback process that involves reviewing three peer essays, and your submission will be assessed by an expert grading panel of seven Kaggle Grandmasters.</p>\n</blockquote>\n<p>Does this mean that competitors must review other competitors essays? In which way does this affect other competitors score?</p>\n<p>Thanks in advance</p>",
      "rawMarkdown": "First, I would like to thank the organizers for this interesting competition which I believe its the first of its kind so far. I am a bit puzzled by the Evaluation section still. I see that to make the competition as objective as possible there is a grading rubric and an eligibility checklist. \n\nMy question is if, ultimately, the leaderboard is determined by the judges (human score). The leaderboard as described in the [Submission instructions](https://www.kaggle.com/competitions/2023-kaggle-ai-report/overview/submission-instructions) section, is just for the purposes stated (team merging, etc).\n\nAlso:\n>Participants will also take part in a peer feedback process that involves reviewing three peer essays, and your submission will be assessed by an expert grading panel of seven Kaggle Grandmasters.\n\nDoes this mean that competitors must review other competitors essays? In which way does this affect other competitors score?\n\nThanks in advance",
      "votes": null
    },
    {
      "id": "2255794",
      "postDate": "05/12/2023 02:07:01",
      "content": "<p>Yes, this is indeed our first ever essay-writing competition! </p>\n<p>Q1: All submissions will <em>temporarily</em> be tied on the leaderboard until <em>after</em> the grading process has been completed, at which point the leaderboard positions will update to reflect your actual rank.</p>\n<p>Q2: Yes, in addition to writing your essay, you are also required to provide qualitative feedback on three essays that were written by your peers.</p>\n<p>Q3: When you provide feedback to your peer it does not directly impact either their score or their final leaderboard position. Final leaderboard positions are determined by the expert grading panel in a process that is separate from the peer feedback.</p>\n<p>Hopefully that helps, please feel free to keep asking questions if things are still unclear. This competition has a brand new format and we anticipate receiving lots of questions. If you have questions directed to the Kaggle Team, please know that there is a pinned <a href=\"https://www.kaggle.com/competitions/2023-kaggle-ai-report/discussion/400084\" target=\"_blank\">Question &amp; Answer thread</a>, and you should find that you'll have a much faster response time there. </p>\n<p>Best of luck, it would be really great if you end up submitting an essay!</p>",
      "rawMarkdown": "Yes, this is indeed our first ever essay-writing competition! \n\nQ1: All submissions will *temporarily* be tied on the leaderboard until *after* the grading process has been completed, at which point the leaderboard positions will update to reflect your actual rank.\n\nQ2: Yes, in addition to writing your essay, you are also required to provide qualitative feedback on three essays that were written by your peers.\n\nQ3: When you provide feedback to your peer it does not directly impact either their score or their final leaderboard position. Final leaderboard positions are determined by the expert grading panel in a process that is separate from the peer feedback.\n\nHopefully that helps, please feel free to keep asking questions if things are still unclear. This competition has a brand new format and we anticipate receiving lots of questions. If you have questions directed to the Kaggle Team, please know that there is a pinned [Question & Answer thread](https://www.kaggle.com/competitions/2023-kaggle-ai-report/discussion/400084), and you should find that you'll have a much faster response time there. \n\nBest of luck, it would be really great if you end up submitting an essay!",
      "votes": null
    },
    {
      "id": "2256331",
      "postDate": "05/12/2023 11:50:47",
      "content": "<p>Thanks for the clear explanation</p>",
      "rawMarkdown": "Thanks for the clear explanation",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2255794,
      "author_name": "paultimothymooney",
      "author_url": "",
      "post_date": "05/12/2023 02:07:01",
      "content": "<p>Yes, this is indeed our first ever essay-writing competition! </p>\n<p>Q1: All submissions will <em>temporarily</em> be tied on the leaderboard until <em>after</em> the grading process has been completed, at which point the leaderboard positions will update to reflect your actual rank.</p>\n<p>Q2: Yes, in addition to writing your essay, you are also required to provide qualitative feedback on three essays that were written by your peers.</p>\n<p>Q3: When you provide feedback to your peer it does not directly impact either their score or their final leaderboard position. Final leaderboard positions are determined by the expert grading panel in a process that is separate from the peer feedback.</p>\n<p>Hopefully that helps, please feel free to keep asking questions if things are still unclear. This competition has a brand new format and we anticipate receiving lots of questions. If you have questions directed to the Kaggle Team, please know that there is a pinned <a href=\"https://www.kaggle.com/competitions/2023-kaggle-ai-report/discussion/400084\" target=\"_blank\">Question &amp; Answer thread</a>, and you should find that you'll have a much faster response time there. </p>\n<p>Best of luck, it would be really great if you end up submitting an essay!</p>",
      "votes": null,
      "replies": [
        {
          "id": 2256331,
          "author_name": "alejopaullier",
          "author_url": "",
          "post_date": "05/12/2023 11:50:47",
          "content": "<p>Thanks for the clear explanation</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2255770": "First, I would like to thank the organizers for this interesting competition which I believe its the first of its kind so far. I am a bit puzzled by the Evaluation section still. I see that to make the competition as objective as possible there is a grading rubric and an eligibility checklist. \n\nMy question is if, ultimately, the leaderboard is determined by the judges (human score). The leaderboard as described in the [Submission instructions](https://www.kaggle.com/competitions/2023-kaggle-ai-report/overview/submission-instructions) section, is just for the purposes stated (team merging, etc).\n\nAlso:\n>Participants will also take part in a peer feedback process that involves reviewing three peer essays, and your submission will be assessed by an expert grading panel of seven Kaggle Grandmasters.\n\nDoes this mean that competitors must review other competitors essays? In which way does this affect other competitors score?\n\nThanks in advance",
    "2255794": "Yes, this is indeed our first ever essay-writing competition! \n\nQ1: All submissions will *temporarily* be tied on the leaderboard until *after* the grading process has been completed, at which point the leaderboard positions will update to reflect your actual rank.\n\nQ2: Yes, in addition to writing your essay, you are also required to provide qualitative feedback on three essays that were written by your peers.\n\nQ3: When you provide feedback to your peer it does not directly impact either their score or their final leaderboard position. Final leaderboard positions are determined by the expert grading panel in a process that is separate from the peer feedback.\n\nHopefully that helps, please feel free to keep asking questions if things are still unclear. This competition has a brand new format and we anticipate receiving lots of questions. If you have questions directed to the Kaggle Team, please know that there is a pinned [Question & Answer thread](https://www.kaggle.com/competitions/2023-kaggle-ai-report/discussion/400084), and you should find that you'll have a much faster response time there. \n\nBest of luck, it would be really great if you end up submitting an essay!",
    "2256331": "Thanks for the clear explanation"
  },
  "source": "meta"
}