{
  "id": 699352,
  "title": "[UPDATE, Fri May 15th] Leaderboard Rescore - COMPLETE",
  "url": "/competitions/cafa-6-protein-function-prediction/discussion/699352",
  "author_name": "MartynaPlomecka",
  "post_date": "2026-05-13T17:54:31.713000",
  "votes": 7,
  "comment_count": 15,
  "views": 0,
  "content": "<p><strong>UPDATE: The rescore is complete.</strong></p>\n<p>Hi everyone,</p>\n<p>We are rescoring the leaderboard. \nDuring this time, the leaderboard may look unstable or partially updated. We expect things to be a bit fluctuating for the next couple of days, and we'll post another update once rescoring is complete.</p>\n<p>Thank you for your understanding!\nMartyna</p>",
  "messages": [
    {
      "id": 3457406,
      "postDate": "2026-05-13T17:54:31.713Z",
      "content": "<p><strong>UPDATE: The rescore is complete.</strong></p>\n<p>Hi everyone,</p>\n<p>We are rescoring the leaderboard. \nDuring this time, the leaderboard may look unstable or partially updated. We expect things to be a bit fluctuating for the next couple of days, and we'll post another update once rescoring is complete.</p>\n<p>Thank you for your understanding!\nMartyna</p>",
      "rawMarkdown": "**UPDATE: The rescore is complete.**\n\n\nHi everyone,\n\nWe are rescoring the leaderboard. \nDuring this time, the leaderboard may look unstable or partially updated. We expect things to be a bit fluctuating for the next couple of days, and we'll post another update once rescoring is complete.\n\nThank you for your understanding!\nMartyna\n",
      "votes": 7
    },
    {
      "id": 3457425,
      "postDate": "2026-05-13T18:21:28.233Z",
      "content": "<h1>Data versions:</h1>\n<p>We used the following data versions to create the ground truth for the May 2026 (final evaluation) update of the leaderboard (<strong>leaderboard rescore is still pending</strong> at the moment):</p>\n<ul>\n<li>Most updated GOA (<strong>t1</strong>): GOA UniProt file released on 2026-04-30, <a href=\"https://ftp.ebi.ac.uk/pub/databases/GO/goa/UNIPROT/goa_uniprot_all.gaf.gz\" target=\"_blank\">here</a></li>\n<li>Public knowledge cutoff (<strong>t0</strong>):  Any GO terms included in this public knowledge <strong>are not included</strong> in the ground truth. There are two sources:<ul>\n<li>GOA UniProt file released on 2025-12-04, <a href=\"https://ftp.ebi.ac.uk/pub/databases/GO/goa/old/UNIPROT/goa_uniprot_all.gaf.229.gz\" target=\"_blank\">here</a>. Annotations were propagated using graph in OBO file released on 2025-10-10, <a href=\"https://release.geneontology.org/2025-10-10/ontology/go-basic.obo\" target=\"_blank\">here</a></li>\n<li>UniProt SwissProt .dat file released on 2026-01-28, <a href=\"https://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.dat.gz\" target=\"_blank\">here</a>. Annotations were propagated using graph in OBO file released on 2026-01-23, <a href=\"https://release.geneontology.org/2026-01-23/ontology/go-basic.obo\" target=\"_blank\">here</a>. <em>In the March leaderboard update, these annotations were not included in the public knowledge cutoff, but are now included in the final evaluation.</em></li></ul></li>\n<li>UniProt SwissProt sequences: UniProt FASTA file released on 2026-01-28, <a href=\"https://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.fasta.gz\" target=\"_blank\">here</a>. This file is only used to compare current SwissProt protein sequences with the original <code>testsuperset.fasta</code> file (224,309 proteins originally).<ul>\n<li>Proteins with unchanged sequences: 224,121. Only these proteins will move on to ground truth creation.</li>\n<li>Proteins removed from SwissProt: 19</li>\n<li>Proteins with sequences changed: 169</li></ul></li>\n</ul>\n<h1>Ground truth creation:</h1>\n<p>New annotations (of proteins in the 224,121 unchanged sequences) are collected between t0 and t1 (and categorized into No Knowledge, Limited Knowledge, Partial Knowledge). Only GO terms that exist in both the original frozen graph (<code>go-basic.obo</code> in Data tab) and t0 graph are included in the ground truth creation (called terms-of-interest). </p>\n<h1>Evaluation:</h1>\n<p>The full configuration that was used to run <code>cafaevaluator</code> (example for PK evaluation): </p>\n<ul>\n<li><code>cafaeval \"go-basic.obo\" \"predictions/\" \"May2026_groundtruth_PK.tsv\" -ia \"IA.tsv\" -known \"May2026_groundtruth_PK_known.tsv\" -no_orphans -norm cafa -prop fill -toi \"May2026_groundtruth_terms_of_interest.txt\" -max_terms 500 -th_step 0.001</code></li>\n<li><code>\"go-basic.obo\"</code>: original frozen graph in Data tab. <code>\"IA.tsv\"</code>: original Information Accretion file in Data tab.</li>\n<li><code>\"May_2026_groundtruth_PK.tsv\"</code>: new annotations collected between t0 and t1. </li>\n<li><code>-known \"May2026_groundtruth_PK_known.tsv\"</code>: public annotations that existed up to t0 (see public knowledge cutoff above). Only PK evaluation uses <code>-known</code> argument</li>\n<li><code>-toi \"May2026_groundtruth_terms_of_interest.txt\"</code>: contains GO terms that exist in both the original frozen graph (<code>go-basic.obo</code> in Data tab) and t0 graph</li>\n</ul>\n<h1>Statistics:</h1>\n<ul>\n<li>Number of NK proteins: 749 proteins ('MFO': 578 proteins, 'BPO': 155 proteins, 'CCO': 112 proteins)</li>\n<li>Number of LK proteins: 485 proteins ('MFO': 193 proteins, 'BPO': 198 proteins, 'CCO': 138 proteins)</li>\n<li>Number of PK proteins: 6,951 proteins ('MFO': 1837 proteins, 'BPO': 3046 proteins, 'CCO': 2716 proteins)</li>\n<li>Total number of unique proteins: 8,023</li>\n<li><strong>NOTE</strong>: The final evaluation set does not include proteins in the hold-out set that were used for the live leaderboard evaluation during competition submission period (Oct 2025 to Feb 2026).</li>\n</ul>",
      "rawMarkdown": "# Data versions:\nWe used the following data versions to create the ground truth for the May 2026 (final evaluation) update of the leaderboard (**leaderboard rescore is still pending** at the moment):\n- Most updated GOA (**t1**): GOA UniProt file released on 2026-04-30, [here](https://ftp.ebi.ac.uk/pub/databases/GO/goa/UNIPROT/goa_uniprot_all.gaf.gz)\n- Public knowledge cutoff (**t0**):  Any GO terms included in this public knowledge **are not included** in the ground truth. There are two sources:\n   - GOA UniProt file released on 2025-12-04, [here](https://ftp.ebi.ac.uk/pub/databases/GO/goa/old/UNIPROT/goa_uniprot_all.gaf.229.gz). Annotations were propagated using graph in OBO file released on 2025-10-10, [here](https://release.geneontology.org/2025-10-10/ontology/go-basic.obo)\n   - UniProt SwissProt .dat file released on 2026-01-28, [here](https://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.dat.gz). Annotations were propagated using graph in OBO file released on 2026-01-23, [here](https://release.geneontology.org/2026-01-23/ontology/go-basic.obo). *In the March leaderboard update, these annotations were not included in the public knowledge cutoff, but are now included in the final evaluation.*\n- UniProt SwissProt sequences: UniProt FASTA file released on 2026-01-28, [here](https://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.fasta.gz). This file is only used to compare current SwissProt protein sequences with the original `testsuperset.fasta` file (224,309 proteins originally).\n   - Proteins with unchanged sequences: 224,121. Only these proteins will move on to ground truth creation.\n   - Proteins removed from SwissProt: 19\n   - Proteins with sequences changed: 169\n\n#Ground truth creation:\nNew annotations (of proteins in the 224,121 unchanged sequences) are collected between t0 and t1 (and categorized into No Knowledge, Limited Knowledge, Partial Knowledge). Only GO terms that exist in both the original frozen graph (`go-basic.obo` in Data tab) and t0 graph are included in the ground truth creation (called terms-of-interest). \n\n#Evaluation: \nThe full configuration that was used to run `cafaevaluator` (example for PK evaluation): \n\n- `cafaeval \"go-basic.obo\" \"predictions/\" \"May2026_groundtruth_PK.tsv\" -ia \"IA.tsv\" -known \"May2026_groundtruth_PK_known.tsv\" -no_orphans -norm cafa -prop fill -toi \"May2026_groundtruth_terms_of_interest.txt\" -max_terms 500 -th_step 0.001`\n- `\"go-basic.obo\"`: original frozen graph in Data tab. `\"IA.tsv\"`: original Information Accretion file in Data tab.\n- `\"May_2026_groundtruth_PK.tsv\"`: new annotations collected between t0 and t1. \n- `-known \"May2026_groundtruth_PK_known.tsv\"`: public annotations that existed up to t0 (see public knowledge cutoff above). Only PK evaluation uses `-known` argument\n- `-toi \"May2026_groundtruth_terms_of_interest.txt\"`: contains GO terms that exist in both the original frozen graph (`go-basic.obo` in Data tab) and t0 graph\n\n#Statistics:\n- Number of NK proteins: 749 proteins ('MFO': 578 proteins, 'BPO': 155 proteins, 'CCO': 112 proteins)\n- Number of LK proteins: 485 proteins ('MFO': 193 proteins, 'BPO': 198 proteins, 'CCO': 138 proteins)\n- Number of PK proteins: 6,951 proteins ('MFO': 1837 proteins, 'BPO': 3046 proteins, 'CCO': 2716 proteins)\n- Total number of unique proteins: 8,023\n- **NOTE**: The final evaluation set does not include proteins in the hold-out set that were used for the live leaderboard evaluation during competition submission period (Oct 2025 to Feb 2026).",
      "replies": [
        {
          "id": 3457664,
          "postDate": "2026-05-14T05:42:19.870Z",
          "content": "<p>does that mean competition finally ends in couple of days? <a href=\"https://www.kaggle.com/ahphan\" target=\"_blank\">@ahphan</a> <a href=\"https://www.kaggle.com/martynaplomecka\" target=\"_blank\">@martynaplomecka</a> </p>",
          "rawMarkdown": "does that mean competition finally ends in couple of days? @ahphan @martynaplomecka ",
          "votes": 1,
          "replies": [
            {
              "id": 3457872,
              "postDate": "2026-05-14T14:24:59.963Z",
              "content": "<p>Yes, when rescore completes the competition will end</p>",
              "rawMarkdown": "Yes, when rescore completes the competition will end",
              "votes": 6
            },
            {
              "id": 3457950,
              "postDate": "2026-05-14T17:16:59.753Z",
              "content": "<p>Thank you for the response!</p>",
              "rawMarkdown": "Thank you for the response!",
              "votes": 1
            },
            {
              "id": 3459057,
              "postDate": "2026-05-17T09:17:47.673Z",
              "content": "<p>Is the current ranking the final ranking? I've noticed that the leaderboard hasn't changed for two days.</p>",
              "rawMarkdown": "Is the current ranking the final ranking? I've noticed that the leaderboard hasn't changed for two days.",
              "votes": 1
            },
            {
              "id": 3459308,
              "postDate": "2026-05-17T16:19:28.713Z",
              "content": "<p><a href=\"https://www.kaggle.com/ahphan\" target=\"_blank\">@ahphan</a> ???</p>",
              "rawMarkdown": "@ahphan ???"
            },
            {
              "id": 3459423,
              "postDate": "2026-05-17T19:38:39.460Z",
              "content": "<p>Yes, the current leaderboard is the final ranking for this competition. We are still conducting some final validations</p>",
              "rawMarkdown": "Yes, the current leaderboard is the final ranking for this competition. We are still conducting some final validations",
              "votes": 2
            },
            {
              "id": 3462223,
              "postDate": "2026-05-22T14:49:30.617Z",
              "content": "<p>Hi <a href=\"https://www.kaggle.com/ahphan\" target=\"_blank\">@ahphan</a> ! is there any progress on conducting final validations?</p>",
              "rawMarkdown": "Hi @ahphan ! is there any progress on conducting final validations?\n"
            },
            {
              "id": 3463498,
              "postDate": "2026-05-27T03:13:32.917Z",
              "content": "<p>Hi organizers, is the current leaderboard just the public leaderboard ranking, and the final private leaderboard ranking will still be based on the last rescore results? And is it the top 2 submissions per team?</p>",
              "rawMarkdown": "Hi organizers, is the current leaderboard just the public leaderboard ranking, and the final private leaderboard ranking will still be based on the last rescore results? And is it the top 2 submissions per team?",
              "votes": 3
            },
            {
              "id": 3463533,
              "postDate": "2026-05-27T06:38:36.453Z",
              "content": "<p>hello, the public lb changed again, is it the final private lb? or?</p>",
              "rawMarkdown": "hello, the public lb changed again, is it the final private lb? or?",
              "votes": 2
            },
            {
              "id": 3463598,
              "postDate": "2026-05-27T11:54:06.257Z",
              "content": "<blockquote>\n  <p>Yes, the current leaderboard is the final ranking for this competition. We are still conducting some final validations</p>\n</blockquote>\n<p>Hello organizers,</p>\n<p>I’d like to ask whether the current leaderboard reflects the final ranking, or whether the private leaderboard will be released based on the previous rankings.</p>",
              "rawMarkdown": "> Yes, the current leaderboard is the final ranking for this competition. We are still conducting some final validations\n\nHello organizers,\n\nI’d like to ask whether the current leaderboard reflects the final ranking, or whether the private leaderboard will be released based on the previous rankings.",
              "votes": 1
            },
            {
              "id": 3463604,
              "postDate": "2026-05-27T12:04:15.820Z",
              "content": "<p>this is what i also want to know</p>",
              "rawMarkdown": "this is what i also want to know"
            },
            {
              "id": 3463606,
              "postDate": "2026-05-27T12:05:51.747Z",
              "content": "<p>The private leaderboard ranking is similar to the public leaderboard ranking, both leaderboards only include the 2 selected submissions, locked in based on February leaderboard (see details below).</p>\n<p>Final validations include checking the winners' solutions for reproducible code that they have used to generate the final submissions. This takes 2-3 weeks after competition ends.</p>\n<p>The two currently selected submissions were chosen as follows:</p>\n<ul>\n<li>If your team hand-selected submissions, those 2 submissions remained your final submissions</li>\n<li>If you hand-selected 1 submission, the other submission with the highest score in the February leaderboard (when submissions closed) was chosen by the system and locked in</li>\n<li>If you hand-selected 0 submissions, two submissions with the highest scores in the February leaderboard were chosen by the system and locked in\nThe most recent change is because of this:</li>\n</ul>\n<p>When we update the ground truth, Kaggle rescores everything of every team (i.e., all 41k submissions).</p>\n<ul>\n<li>The previous leaderboard available around mid-May was based on the ground truth I described at the top of this post. This remains our final ground truth data.</li>\n<li>The current leaderboard ranking is only based on the 2 locked in submissions as described above, per Kaggle competition rules. If your ranking dropped between mid-May and current, that means you did have better scoring-submissions based on the May leaderboard but these were not the highest-scoring in the February leaderboard, so they are not part of the final ranking.</li>\n<li>You will see errors in all of your submissions right now, except for the 2 submissions that were locked in. This is because we have invalidated the remaining submissions to exclude them from the final ranking. Now each team only has 2 entries, and the better one is shown on the public leaderboard.</li>\n</ul>",
              "rawMarkdown": "The private leaderboard ranking is similar to the public leaderboard ranking, both leaderboards only include the 2 selected submissions, locked in based on February leaderboard (see details below).\n\nFinal validations include checking the winners' solutions for reproducible code that they have used to generate the final submissions. This takes 2-3 weeks after competition ends.\n\n\nThe two currently selected submissions were chosen as follows:\n\n- If your team hand-selected submissions, those 2 submissions remained your final submissions\n- If you hand-selected 1 submission, the other submission with the highest score in the February leaderboard (when submissions closed) was chosen by the system and locked in\n- If you hand-selected 0 submissions, two submissions with the highest scores in the February leaderboard were chosen by the system and locked in\nThe most recent change is because of this:\n\nWhen we update the ground truth, Kaggle rescores everything of every team (i.e., all 41k submissions).\n- The previous leaderboard available around mid-May was based on the ground truth I described at the top of this post. This remains our final ground truth data.\n- The current leaderboard ranking is only based on the 2 locked in submissions as described above, per Kaggle competition rules. If your ranking dropped between mid-May and current, that means you did have better scoring-submissions based on the May leaderboard but these were not the highest-scoring in the February leaderboard, so they are not part of the final ranking.\n- You will see errors in all of your submissions right now, except for the 2 submissions that were locked in. This is because we have invalidated the remaining submissions to exclude them from the final ranking. Now each team only has 2 entries, and the better one is shown on the public leaderboard.",
              "votes": 1
            },
            {
              "id": 3463609,
              "postDate": "2026-05-27T12:32:44.283Z",
              "rawMarkdown": "",
              "isDeleted": true
            }
          ]
        }
      ]
    },
    {
      "id": 3462125,
      "postDate": "2026-05-22T08:24:44.040Z",
      "content": "<p>Is there actually a three-way tie for positions 4–6, or are only three decimal places being displayed while the underlying values differ enough for the ranking order to be correct?</p>",
      "rawMarkdown": "Is there actually a three-way tie for positions 4–6, or are only three decimal places being displayed while the underlying values differ enough for the ranking order to be correct?"
    }
  ],
  "comments": [
    {
      "id": 3457425,
      "author_name": "An Phan",
      "author_url": "",
      "post_date": "2026-05-13T18:21:28.233000",
      "content": "<h1>Data versions:</h1>\n<p>We used the following data versions to create the ground truth for the May 2026 (final evaluation) update of the leaderboard (<strong>leaderboard rescore is still pending</strong> at the moment):</p>\n<ul>\n<li>Most updated GOA (<strong>t1</strong>): GOA UniProt file released on 2026-04-30, <a href=\"https://ftp.ebi.ac.uk/pub/databases/GO/goa/UNIPROT/goa_uniprot_all.gaf.gz\" target=\"_blank\">here</a></li>\n<li>Public knowledge cutoff (<strong>t0</strong>):  Any GO terms included in this public knowledge <strong>are not included</strong> in the ground truth. There are two sources:<ul>\n<li>GOA UniProt file released on 2025-12-04, <a href=\"https://ftp.ebi.ac.uk/pub/databases/GO/goa/old/UNIPROT/goa_uniprot_all.gaf.229.gz\" target=\"_blank\">here</a>. Annotations were propagated using graph in OBO file released on 2025-10-10, <a href=\"https://release.geneontology.org/2025-10-10/ontology/go-basic.obo\" target=\"_blank\">here</a></li>\n<li>UniProt SwissProt .dat file released on 2026-01-28, <a href=\"https://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.dat.gz\" target=\"_blank\">here</a>. Annotations were propagated using graph in OBO file released on 2026-01-23, <a href=\"https://release.geneontology.org/2026-01-23/ontology/go-basic.obo\" target=\"_blank\">here</a>. <em>In the March leaderboard update, these annotations were not included in the public knowledge cutoff, but are now included in the final evaluation.</em></li></ul></li>\n<li>UniProt SwissProt sequences: UniProt FASTA file released on 2026-01-28, <a href=\"https://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.fasta.gz\" target=\"_blank\">here</a>. This file is only used to compare current SwissProt protein sequences with the original <code>testsuperset.fasta</code> file (224,309 proteins originally).<ul>\n<li>Proteins with unchanged sequences: 224,121. Only these proteins will move on to ground truth creation.</li>\n<li>Proteins removed from SwissProt: 19</li>\n<li>Proteins with sequences changed: 169</li></ul></li>\n</ul>\n<h1>Ground truth creation:</h1>\n<p>New annotations (of proteins in the 224,121 unchanged sequences) are collected between t0 and t1 (and categorized into No Knowledge, Limited Knowledge, Partial Knowledge). Only GO terms that exist in both the original frozen graph (<code>go-basic.obo</code> in Data tab) and t0 graph are included in the ground truth creation (called terms-of-interest). </p>\n<h1>Evaluation:</h1>\n<p>The full configuration that was used to run <code>cafaevaluator</code> (example for PK evaluation): </p>\n<ul>\n<li><code>cafaeval \"go-basic.obo\" \"predictions/\" \"May2026_groundtruth_PK.tsv\" -ia \"IA.tsv\" -known \"May2026_groundtruth_PK_known.tsv\" -no_orphans -norm cafa -prop fill -toi \"May2026_groundtruth_terms_of_interest.txt\" -max_terms 500 -th_step 0.001</code></li>\n<li><code>\"go-basic.obo\"</code>: original frozen graph in Data tab. <code>\"IA.tsv\"</code>: original Information Accretion file in Data tab.</li>\n<li><code>\"May_2026_groundtruth_PK.tsv\"</code>: new annotations collected between t0 and t1. </li>\n<li><code>-known \"May2026_groundtruth_PK_known.tsv\"</code>: public annotations that existed up to t0 (see public knowledge cutoff above). Only PK evaluation uses <code>-known</code> argument</li>\n<li><code>-toi \"May2026_groundtruth_terms_of_interest.txt\"</code>: contains GO terms that exist in both the original frozen graph (<code>go-basic.obo</code> in Data tab) and t0 graph</li>\n</ul>\n<h1>Statistics:</h1>\n<ul>\n<li>Number of NK proteins: 749 proteins ('MFO': 578 proteins, 'BPO': 155 proteins, 'CCO': 112 proteins)</li>\n<li>Number of LK proteins: 485 proteins ('MFO': 193 proteins, 'BPO': 198 proteins, 'CCO': 138 proteins)</li>\n<li>Number of PK proteins: 6,951 proteins ('MFO': 1837 proteins, 'BPO': 3046 proteins, 'CCO': 2716 proteins)</li>\n<li>Total number of unique proteins: 8,023</li>\n<li><strong>NOTE</strong>: The final evaluation set does not include proteins in the hold-out set that were used for the live leaderboard evaluation during competition submission period (Oct 2025 to Feb 2026).</li>\n</ul>",
      "votes": 0,
      "replies": [
        {
          "id": 3457664,
          "author_name": "Baidalin Adilzhan [dsmlkz]",
          "author_url": "",
          "post_date": "2026-05-14T05:42:19.870000",
          "content": "<p>does that mean competition finally ends in couple of days? <a href=\"https://www.kaggle.com/ahphan\" target=\"_blank\">@ahphan</a> <a href=\"https://www.kaggle.com/martynaplomecka\" target=\"_blank\">@martynaplomecka</a> </p>",
          "votes": 1,
          "replies": [
            {
              "id": 3457872,
              "author_name": "An Phan",
              "author_url": "",
              "post_date": "2026-05-14T14:24:59.963000",
              "content": "<p>Yes, when rescore completes the competition will end</p>",
              "votes": 6,
              "replies": []
            },
            {
              "id": 3457950,
              "author_name": "Baidalin Adilzhan [dsmlkz]",
              "author_url": "",
              "post_date": "2026-05-14T17:16:59.753000",
              "content": "<p>Thank you for the response!</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3459057,
              "author_name": "Boredom",
              "author_url": "",
              "post_date": "2026-05-17T09:17:47.673000",
              "content": "<p>Is the current ranking the final ranking? I've noticed that the leaderboard hasn't changed for two days.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3459308,
              "author_name": "Baidalin Adilzhan [dsmlkz]",
              "author_url": "",
              "post_date": "2026-05-17T16:19:28.713000",
              "content": "<p><a href=\"https://www.kaggle.com/ahphan\" target=\"_blank\">@ahphan</a> ???</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3459423,
              "author_name": "An Phan",
              "author_url": "",
              "post_date": "2026-05-17T19:38:39.460000",
              "content": "<p>Yes, the current leaderboard is the final ranking for this competition. We are still conducting some final validations</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 3462223,
              "author_name": "Baidalin Adilzhan [dsmlkz]",
              "author_url": "",
              "post_date": "2026-05-22T14:49:30.617000",
              "content": "<p>Hi <a href=\"https://www.kaggle.com/ahphan\" target=\"_blank\">@ahphan</a> ! is there any progress on conducting final validations?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3463498,
              "author_name": "HUG",
              "author_url": "",
              "post_date": "2026-05-27T03:13:32.917000",
              "content": "<p>Hi organizers, is the current leaderboard just the public leaderboard ranking, and the final private leaderboard ranking will still be based on the last rescore results? And is it the top 2 submissions per team?</p>",
              "votes": 3,
              "replies": []
            },
            {
              "id": 3463533,
              "author_name": "common-people",
              "author_url": "",
              "post_date": "2026-05-27T06:38:36.453000",
              "content": "<p>hello, the public lb changed again, is it the final private lb? or?</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 3463598,
              "author_name": "HUG",
              "author_url": "",
              "post_date": "2026-05-27T11:54:06.257000",
              "content": "<blockquote>\n  <p>Yes, the current leaderboard is the final ranking for this competition. We are still conducting some final validations</p>\n</blockquote>\n<p>Hello organizers,</p>\n<p>I’d like to ask whether the current leaderboard reflects the final ranking, or whether the private leaderboard will be released based on the previous rankings.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3463604,
              "author_name": "Born4",
              "author_url": "",
              "post_date": "2026-05-27T12:04:15.820000",
              "content": "<p>this is what i also want to know</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3463606,
              "author_name": "An Phan",
              "author_url": "",
              "post_date": "2026-05-27T12:05:51.747000",
              "content": "<p>The private leaderboard ranking is similar to the public leaderboard ranking, both leaderboards only include the 2 selected submissions, locked in based on February leaderboard (see details below).</p>\n<p>Final validations include checking the winners' solutions for reproducible code that they have used to generate the final submissions. This takes 2-3 weeks after competition ends.</p>\n<p>The two currently selected submissions were chosen as follows:</p>\n<ul>\n<li>If your team hand-selected submissions, those 2 submissions remained your final submissions</li>\n<li>If you hand-selected 1 submission, the other submission with the highest score in the February leaderboard (when submissions closed) was chosen by the system and locked in</li>\n<li>If you hand-selected 0 submissions, two submissions with the highest scores in the February leaderboard were chosen by the system and locked in\nThe most recent change is because of this:</li>\n</ul>\n<p>When we update the ground truth, Kaggle rescores everything of every team (i.e., all 41k submissions).</p>\n<ul>\n<li>The previous leaderboard available around mid-May was based on the ground truth I described at the top of this post. This remains our final ground truth data.</li>\n<li>The current leaderboard ranking is only based on the 2 locked in submissions as described above, per Kaggle competition rules. If your ranking dropped between mid-May and current, that means you did have better scoring-submissions based on the May leaderboard but these were not the highest-scoring in the February leaderboard, so they are not part of the final ranking.</li>\n<li>You will see errors in all of your submissions right now, except for the 2 submissions that were locked in. This is because we have invalidated the remaining submissions to exclude them from the final ranking. Now each team only has 2 entries, and the better one is shown on the public leaderboard.</li>\n</ul>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3463609,
              "author_name": "",
              "author_url": "",
              "post_date": "2026-05-27T12:32:44.283000",
              "content": "",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3462125,
      "author_name": "xaxipiruli",
      "author_url": "",
      "post_date": "2026-05-22T08:24:44.040000",
      "content": "<p>Is there actually a three-way tie for positions 4–6, or are only three decimal places being displayed while the underlying values differ enough for the ranking order to be correct?</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3457406": "**UPDATE: The rescore is complete.**\n\n\nHi everyone,\n\nWe are rescoring the leaderboard. \nDuring this time, the leaderboard may look unstable or partially updated. We expect things to be a bit fluctuating for the next couple of days, and we'll post another update once rescoring is complete.\n\nThank you for your understanding!\nMartyna\n",
    "3457425": "# Data versions:\nWe used the following data versions to create the ground truth for the May 2026 (final evaluation) update of the leaderboard (**leaderboard rescore is still pending** at the moment):\n- Most updated GOA (**t1**): GOA UniProt file released on 2026-04-30, [here](https://ftp.ebi.ac.uk/pub/databases/GO/goa/UNIPROT/goa_uniprot_all.gaf.gz)\n- Public knowledge cutoff (**t0**):  Any GO terms included in this public knowledge **are not included** in the ground truth. There are two sources:\n   - GOA UniProt file released on 2025-12-04, [here](https://ftp.ebi.ac.uk/pub/databases/GO/goa/old/UNIPROT/goa_uniprot_all.gaf.229.gz). Annotations were propagated using graph in OBO file released on 2025-10-10, [here](https://release.geneontology.org/2025-10-10/ontology/go-basic.obo)\n   - UniProt SwissProt .dat file released on 2026-01-28, [here](https://ftp.ebi.ac.uk/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.dat.gz). Annotations were propagated using graph in OBO file released on 2026-01-23, [here](https://release.geneontology.org/2026-01-23/ontology/go-basic.obo). *In the March leaderboard update, these annotations were not included in the public knowledge cutoff, but are now included in the final evaluation.*\n- UniProt SwissProt sequences: UniProt FASTA file released on 2026-01-28, [here](https://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.fasta.gz). This file is only used to compare current SwissProt protein sequences with the original `testsuperset.fasta` file (224,309 proteins originally).\n   - Proteins with unchanged sequences: 224,121. Only these proteins will move on to ground truth creation.\n   - Proteins removed from SwissProt: 19\n   - Proteins with sequences changed: 169\n\n#Ground truth creation:\nNew annotations (of proteins in the 224,121 unchanged sequences) are collected between t0 and t1 (and categorized into No Knowledge, Limited Knowledge, Partial Knowledge). Only GO terms that exist in both the original frozen graph (`go-basic.obo` in Data tab) and t0 graph are included in the ground truth creation (called terms-of-interest). \n\n#Evaluation: \nThe full configuration that was used to run `cafaevaluator` (example for PK evaluation): \n\n- `cafaeval \"go-basic.obo\" \"predictions/\" \"May2026_groundtruth_PK.tsv\" -ia \"IA.tsv\" -known \"May2026_groundtruth_PK_known.tsv\" -no_orphans -norm cafa -prop fill -toi \"May2026_groundtruth_terms_of_interest.txt\" -max_terms 500 -th_step 0.001`\n- `\"go-basic.obo\"`: original frozen graph in Data tab. `\"IA.tsv\"`: original Information Accretion file in Data tab.\n- `\"May_2026_groundtruth_PK.tsv\"`: new annotations collected between t0 and t1. \n- `-known \"May2026_groundtruth_PK_known.tsv\"`: public annotations that existed up to t0 (see public knowledge cutoff above). Only PK evaluation uses `-known` argument\n- `-toi \"May2026_groundtruth_terms_of_interest.txt\"`: contains GO terms that exist in both the original frozen graph (`go-basic.obo` in Data tab) and t0 graph\n\n#Statistics:\n- Number of NK proteins: 749 proteins ('MFO': 578 proteins, 'BPO': 155 proteins, 'CCO': 112 proteins)\n- Number of LK proteins: 485 proteins ('MFO': 193 proteins, 'BPO': 198 proteins, 'CCO': 138 proteins)\n- Number of PK proteins: 6,951 proteins ('MFO': 1837 proteins, 'BPO': 3046 proteins, 'CCO': 2716 proteins)\n- Total number of unique proteins: 8,023\n- **NOTE**: The final evaluation set does not include proteins in the hold-out set that were used for the live leaderboard evaluation during competition submission period (Oct 2025 to Feb 2026).",
    "3462125": "Is there actually a three-way tie for positions 4–6, or are only three decimal places being displayed while the underlying values differ enough for the ranking order to be correct?"
  }
}