{ "id": 612508, "title": "TSV submission only?", "url": "/competitions/cafa-6-protein-function-prediction/discussion/612508", "author_name": "sasa_leaf", "post_date": "2025-10-20T08:40:41.717000", "votes": 1, "comment_count": 8, "views": 0, "content": "

Quick question — this is a result-only competition, not a code competition, right? Just submission.tsv?

", "messages": [ { "id": 3304284, "postDate": "2025-10-20T08:40:41.717Z", "content": "

Quick question — this is a result-only competition, not a code competition, right? Just submission.tsv?

", "rawMarkdown": "Quick question — this is a result-only competition, not a code competition, right? Just `submission.tsv`?", "votes": 1 }, { "id": 3304295, "postDate": "2025-10-20T09:03:52.400Z", "content": "

Final test is unknown. I don't know if that means many go terms from this test are still unknown or if final test will be a different test. But at this moment yes, you can submit submission file only.

\n
\n

Important Note
\n This is a prospective (i.e., future) data competition. Many proteins in the Test data do not currently have any assigned functions. Proteins having one or more of their functions published by researchers during the curation phase of the competition will comprise the future test set. Final leaderboard scores will be calculated after the curation phase of the competition.

\n
\n

So I think in practice test will be a subset of well curated proteins from current test? And so we don't have to worry about execution times if we have a submission ready for the current test?

", "rawMarkdown": "Final test is unknown. I don't know if that means many go terms from this test are still unknown or if final test will be a different test. But at this moment yes, you can submit submission file only.\n\n>Important Note\nThis is a prospective (i.e., future) data competition. Many proteins in the Test data do not currently have any assigned functions. Proteins having one or more of their functions published by researchers during the curation phase of the competition will comprise the future test set. Final leaderboard scores will be calculated after the curation phase of the competition.\n\nSo I think in practice test will be a subset of well curated proteins from current test? And so we don't have to worry about execution times if we have a submission ready for the current test?", "votes": 2, "replies": [ { "id": 3304346, "postDate": "2025-10-20T10:56:15.797Z", "content": "

Thank you for the clarification!

\n

Since the final Test Set is selected from the Test Superset, it seems that preparing a prediction TSV for the entire Test Superset and submitting it as is should be sufficient for the Private Leaderboard. No further code execution would be necessary after that?

\n
\n

Test Superset

\n

The test superset is a set of protein sequences on which the participants are asked to predict GO terms and optionally a free-text paragraph describing the protein’s functions.

\n

Test Set

\n

The test set is unknown at the beginning of the competition. It will contain protein sequences (and their functions) from the test superset that gained experimental annotations between the submission deadline and the time of evaluation.

\n
", "rawMarkdown": "Thank you for the clarification!\n\nSince the final Test Set is selected from the Test Superset, it seems that preparing a prediction TSV for the entire Test Superset and submitting it as is should be sufficient for the Private Leaderboard. No further code execution would be necessary after that?\n\n>Test Superset\n>\n>The test superset is a set of protein sequences on which the participants are asked to predict GO terms and optionally a free-text paragraph describing the protein’s functions.\n>\n>\n>Test Set\n>\n>The test set is unknown at the beginning of the competition. It will contain protein sequences (and their functions) from the test superset that gained experimental annotations between the submission deadline and the time of evaluation.", "votes": 1, "replies": [ { "id": 3304365, "postDate": "2025-10-20T12:02:30.327Z", "content": "

Yes. If no more new sequences are added. Just go terms curated. Looks like is the case.

", "rawMarkdown": "Yes. If no more new sequences are added. Just go terms curated. Looks like is the case.", "votes": 1 }, { "id": 3304403, "postDate": "2025-10-20T13:25:04.673Z", "content": "

Yes, no more new sequences are added, the Test Set is entirely included in the Test Superset. No new GO terms will be added either, we will only evaluate new curated GO annotations for GO terms that are already in the go-basic.obo file.

", "rawMarkdown": "Yes, no more new sequences are added, the Test Set is entirely included in the Test Superset. No new GO terms will be added either, we will only evaluate new curated GO annotations for GO terms that are already in the go-basic.obo file.", "votes": 3 } ] } ] }, { "id": 3381361, "postDate": "2025-12-24T10:34:44.900Z", "content": "

I just submitted a submission.tsv and the scoring takes forever even the size of my file is smaller compared to previous submissions. Still waiting for the evaluation. Anyone has the same issue?

", "rawMarkdown": "I just submitted a submission.tsv and the scoring takes forever even the size of my file is smaller compared to previous submissions. Still waiting for the evaluation. Anyone has the same issue?", "replies": [ { "id": 3381430, "postDate": "2025-12-24T15:20:18.900Z", "content": "

Yes, this is a known issue that happens with a small percentage of submissions. Simply resubmit the same file and it should be scored successfully.

", "rawMarkdown": "Yes, this is a known issue that happens with a small percentage of submissions. Simply resubmit the same file and it should be scored successfully.", "replies": [ { "id": 3382195, "postDate": "2025-12-26T20:08:04.010Z", "content": "

I submitted the same file 2nd time and had the same issue. Turned out that the probability column contains values with less than 3 decimals. Converting these values to 3 decimals fixed the issue.

", "rawMarkdown": "I submitted the same file 2nd time and had the same issue. Turned out that the probability column contains values with less than 3 decimals. Converting these values to 3 decimals fixed the issue.", "votes": 1 } ] } ] }, { "id": 3304344, "postDate": "2025-10-20T10:55:44.227Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 3304295, "author_name": "Ángel Jacinto Sánchez Ruiz", "author_url": "", "post_date": "2025-10-20T09:03:52.400000", "content": "

Final test is unknown. I don't know if that means many go terms from this test are still unknown or if final test will be a different test. But at this moment yes, you can submit submission file only.

\n
\n

Important Note
\n This is a prospective (i.e., future) data competition. Many proteins in the Test data do not currently have any assigned functions. Proteins having one or more of their functions published by researchers during the curation phase of the competition will comprise the future test set. Final leaderboard scores will be calculated after the curation phase of the competition.

\n
\n

So I think in practice test will be a subset of well curated proteins from current test? And so we don't have to worry about execution times if we have a submission ready for the current test?

", "votes": 2, "replies": [ { "id": 3304346, "author_name": "sasa_leaf", "author_url": "", "post_date": "2025-10-20T10:56:15.797000", "content": "

Thank you for the clarification!

\n

Since the final Test Set is selected from the Test Superset, it seems that preparing a prediction TSV for the entire Test Superset and submitting it as is should be sufficient for the Private Leaderboard. No further code execution would be necessary after that?

\n
\n

Test Superset

\n

The test superset is a set of protein sequences on which the participants are asked to predict GO terms and optionally a free-text paragraph describing the protein’s functions.

\n

Test Set

\n

The test set is unknown at the beginning of the competition. It will contain protein sequences (and their functions) from the test superset that gained experimental annotations between the submission deadline and the time of evaluation.

\n
", "votes": 1, "replies": [ { "id": 3304365, "author_name": "Ángel Jacinto Sánchez Ruiz", "author_url": "", "post_date": "2025-10-20T12:02:30.327000", "content": "

Yes. If no more new sequences are added. Just go terms curated. Looks like is the case.

", "votes": 1, "replies": [] }, { "id": 3304403, "author_name": "An Phan", "author_url": "", "post_date": "2025-10-20T13:25:04.673000", "content": "

Yes, no more new sequences are added, the Test Set is entirely included in the Test Superset. No new GO terms will be added either, we will only evaluate new curated GO annotations for GO terms that are already in the go-basic.obo file.

", "votes": 3, "replies": [] } ] } ] }, { "id": 3381361, "author_name": "tdoh86", "author_url": "", "post_date": "2025-12-24T10:34:44.900000", "content": "

I just submitted a submission.tsv and the scoring takes forever even the size of my file is smaller compared to previous submissions. Still waiting for the evaluation. Anyone has the same issue?

", "votes": 0, "replies": [ { "id": 3381430, "author_name": "An Phan", "author_url": "", "post_date": "2025-12-24T15:20:18.900000", "content": "

Yes, this is a known issue that happens with a small percentage of submissions. Simply resubmit the same file and it should be scored successfully.

", "votes": 0, "replies": [ { "id": 3382195, "author_name": "tdoh86", "author_url": "", "post_date": "2025-12-26T20:08:04.010000", "content": "

I submitted the same file 2nd time and had the same issue. Turned out that the probability column contains values with less than 3 decimals. Converting these values to 3 decimals fixed the issue.

", "votes": 1, "replies": [] } ] } ] }, { "id": 3304344, "author_name": "", "author_url": "", "post_date": "2025-10-20T10:55:44.227000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3304284": "Quick question — this is a result-only competition, not a code competition, right? Just `submission.tsv`?", "3304295": "Final test is unknown. I don't know if that means many go terms from this test are still unknown or if final test will be a different test. But at this moment yes, you can submit submission file only.\n\n>Important Note\nThis is a prospective (i.e., future) data competition. Many proteins in the Test data do not currently have any assigned functions. Proteins having one or more of their functions published by researchers during the curation phase of the competition will comprise the future test set. Final leaderboard scores will be calculated after the curation phase of the competition.\n\nSo I think in practice test will be a subset of well curated proteins from current test? And so we don't have to worry about execution times if we have a submission ready for the current test?", "3381361": "I just submitted a submission.tsv and the scoring takes forever even the size of my file is smaller compared to previous submissions. Still waiting for the evaluation. Anyone has the same issue?", "3304344": "" } }