{
  "id": 703907,
  "title": "Question about auto-selected final submission becoming Error after previous valid interim evaluations",
  "url": "/competitions/cafa-6-protein-function-prediction/discussion/703907",
  "author_name": "",
  "post_date": "2026-06-02T16:11:56.714796500Z",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hello,</p>\n<p>I have a question about my final submission selection.</p>\n<p>I used the auto-selection option for my submissions. During the previous interim evaluations, my higher-scoring submission files were valid and evaluated normally. However, after the final evaluation, some of these same submissions now show an “Error” status, and it looks like a lower-scoring submission was selected/used instead.</p>\n<p>Could the hosts or Kaggle team please clarify why submissions that were previously valid during interim evaluations became errors at the final stage? Is it possible to review whether the auto-selection used the intended valid higher-scoring submission?</p>\n<p>I understand that the private leaderboard is still preliminary while results are being verified, so I wanted to ask before the results are finalized.</p>\n<p>Thank you very much for your help.</p>",
  "messages": [
    {
      "id": "3465663",
      "postDate": "06/02/2026 16:11:56",
      "content": "<p>Hello,</p>\n<p>I have a question about my final submission selection.</p>\n<p>I used the auto-selection option for my submissions. During the previous interim evaluations, my higher-scoring submission files were valid and evaluated normally. However, after the final evaluation, some of these same submissions now show an “Error” status, and it looks like a lower-scoring submission was selected/used instead.</p>\n<p>Could the hosts or Kaggle team please clarify why submissions that were previously valid during interim evaluations became errors at the final stage? Is it possible to review whether the auto-selection used the intended valid higher-scoring submission?</p>\n<p>I understand that the private leaderboard is still preliminary while results are being verified, so I wanted to ask before the results are finalized.</p>\n<p>Thank you very much for your help.</p>",
      "rawMarkdown": "Hello,\n\nI have a question about my final submission selection.\n\nI used the auto-selection option for my submissions. During the previous interim evaluations, my higher-scoring submission files were valid and evaluated normally. However, after the final evaluation, some of these same submissions now show an “Error” status, and it looks like a lower-scoring submission was selected/used instead.\n\nCould the hosts or Kaggle team please clarify why submissions that were previously valid during interim evaluations became errors at the final stage? Is it possible to review whether the auto-selection used the intended valid higher-scoring submission?\n\nI understand that the private leaderboard is still preliminary while results are being verified, so I wanted to ask before the results are finalized.\n\nThank you very much for your help.",
      "votes": null
    },
    {
      "id": "3465672",
      "postDate": "06/02/2026 16:31:06",
      "content": "<blockquote>\n  <p>why submissions that were previously valid during interim evaluations became errors at the final stage?</p>\n</blockquote>\n<p>During interim evaluations, Kaggle rescored every submissions that you submitted, so they all showed up as valid. However, in the final stage, <strong>only two submissions are valid</strong>, and other submissions show up as invalid. These two submissions were the <strong>highest-scoring based on the February leaderboard</strong> (more details below), and were <strong>locked in</strong> right after submissions closed.</p>\n<ul>\n<li>The current leaderboard ranking is only based on the 2 locked in submissions, per Kaggle competition rules. If your ranking dropped between previous interim updates and current, that means you did have better scoring-submissions based on the previous leaderboard but these were not the highest-scoring in the February leaderboard, so they are not part of the final ranking.</li>\n<li>You will see errors in all of your submissions right now, except for the 2 submissions that were locked in. This is because we have invalidated the remaining submissions to exclude them from the final ranking. Now each team only has 2 entries, and the better one is shown on the leaderboard.</li>\n</ul>\n<p>The two currently selected submissions were chosen as follows:</p>\n<ul>\n<li>If your team hand-selected submissions, those 2 submissions remained your final submissions</li>\n<li>If you hand-selected 1 submission, the other submission with the highest score in the February leaderboard (when submissions closed) was chosen by the system and locked in</li>\n<li>If you hand-selected 0 submissions, two submissions with the highest scores in the February leaderboard were chosen by the system and locked in</li>\n</ul>",
      "rawMarkdown": ">why submissions that were previously valid during interim evaluations became errors at the final stage?\n\nDuring interim evaluations, Kaggle rescored every submissions that you submitted, so they all showed up as valid. However, in the final stage, **only two submissions are valid**, and other submissions show up as invalid. These two submissions were the **highest-scoring based on the February leaderboard** (more details below), and were **locked in** right after submissions closed.\n- The current leaderboard ranking is only based on the 2 locked in submissions, per Kaggle competition rules. If your ranking dropped between previous interim updates and current, that means you did have better scoring-submissions based on the previous leaderboard but these were not the highest-scoring in the February leaderboard, so they are not part of the final ranking.\n- You will see errors in all of your submissions right now, except for the 2 submissions that were locked in. This is because we have invalidated the remaining submissions to exclude them from the final ranking. Now each team only has 2 entries, and the better one is shown on the leaderboard.\n\nThe two currently selected submissions were chosen as follows:\n- If your team hand-selected submissions, those 2 submissions remained your final submissions\n- If you hand-selected 1 submission, the other submission with the highest score in the February leaderboard (when submissions closed) was chosen by the system and locked in\n- If you hand-selected 0 submissions, two submissions with the highest scores in the February leaderboard were chosen by the system and locked in",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3465672,
      "author_name": "ahphan",
      "author_url": "",
      "post_date": "06/02/2026 16:31:06",
      "content": "<blockquote>\n  <p>why submissions that were previously valid during interim evaluations became errors at the final stage?</p>\n</blockquote>\n<p>During interim evaluations, Kaggle rescored every submissions that you submitted, so they all showed up as valid. However, in the final stage, <strong>only two submissions are valid</strong>, and other submissions show up as invalid. These two submissions were the <strong>highest-scoring based on the February leaderboard</strong> (more details below), and were <strong>locked in</strong> right after submissions closed.</p>\n<ul>\n<li>The current leaderboard ranking is only based on the 2 locked in submissions, per Kaggle competition rules. If your ranking dropped between previous interim updates and current, that means you did have better scoring-submissions based on the previous leaderboard but these were not the highest-scoring in the February leaderboard, so they are not part of the final ranking.</li>\n<li>You will see errors in all of your submissions right now, except for the 2 submissions that were locked in. This is because we have invalidated the remaining submissions to exclude them from the final ranking. Now each team only has 2 entries, and the better one is shown on the leaderboard.</li>\n</ul>\n<p>The two currently selected submissions were chosen as follows:</p>\n<ul>\n<li>If your team hand-selected submissions, those 2 submissions remained your final submissions</li>\n<li>If you hand-selected 1 submission, the other submission with the highest score in the February leaderboard (when submissions closed) was chosen by the system and locked in</li>\n<li>If you hand-selected 0 submissions, two submissions with the highest scores in the February leaderboard were chosen by the system and locked in</li>\n</ul>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3465663": "Hello,\n\nI have a question about my final submission selection.\n\nI used the auto-selection option for my submissions. During the previous interim evaluations, my higher-scoring submission files were valid and evaluated normally. However, after the final evaluation, some of these same submissions now show an “Error” status, and it looks like a lower-scoring submission was selected/used instead.\n\nCould the hosts or Kaggle team please clarify why submissions that were previously valid during interim evaluations became errors at the final stage? Is it possible to review whether the auto-selection used the intended valid higher-scoring submission?\n\nI understand that the private leaderboard is still preliminary while results are being verified, so I wanted to ask before the results are finalized.\n\nThank you very much for your help.",
    "3465672": ">why submissions that were previously valid during interim evaluations became errors at the final stage?\n\nDuring interim evaluations, Kaggle rescored every submissions that you submitted, so they all showed up as valid. However, in the final stage, **only two submissions are valid**, and other submissions show up as invalid. These two submissions were the **highest-scoring based on the February leaderboard** (more details below), and were **locked in** right after submissions closed.\n- The current leaderboard ranking is only based on the 2 locked in submissions, per Kaggle competition rules. If your ranking dropped between previous interim updates and current, that means you did have better scoring-submissions based on the previous leaderboard but these were not the highest-scoring in the February leaderboard, so they are not part of the final ranking.\n- You will see errors in all of your submissions right now, except for the 2 submissions that were locked in. This is because we have invalidated the remaining submissions to exclude them from the final ranking. Now each team only has 2 entries, and the better one is shown on the leaderboard.\n\nThe two currently selected submissions were chosen as follows:\n- If your team hand-selected submissions, those 2 submissions remained your final submissions\n- If you hand-selected 1 submission, the other submission with the highest score in the February leaderboard (when submissions closed) was chosen by the system and locked in\n- If you hand-selected 0 submissions, two submissions with the highest scores in the February leaderboard were chosen by the system and locked in"
  },
  "source": "meta"
}