Hey, I was working on creating a baseline submission and noticed that I can't upload any files as it's notebook only. I looked through the rules and couldn't find anything saying it's notebook only. Is this a mistake or is the competition supposed to be notebook only?
\nIf so, is it okay to develop/submit files from a file (uploaded to notebook, then submitted) so long as the final submission is computed by a notebook?
\nThanks!
", "messages": [ { "id": "2236641", "postDate": "04/27/2023 02:48:36", "content": "Hey, I was working on creating a baseline submission and noticed that I can't upload any files as it's notebook only. I looked through the rules and couldn't find anything saying it's notebook only. Is this a mistake or is the competition supposed to be notebook only?
\nIf so, is it okay to develop/submit files from a file (uploaded to notebook, then submitted) so long as the final submission is computed by a notebook?
\nThanks!
", "rawMarkdown": "Hey, I was working on creating a baseline submission and noticed that I can't upload any files as it's notebook only. I looked through the rules and couldn't find anything saying it's notebook only. Is this a mistake or is the competition supposed to be notebook only?\n\nIf so, is it okay to develop/submit files from a file (uploaded to notebook, then submitted) _so long as the final submission is computed by a notebook_?\n\nThanks!", "votes": null }, { "id": "2236889", "postDate": "04/27/2023 07:15:19", "content": "The notebook is actually just a wrapper for the submission file. It does not have to compute anything.
\nhttps://www.kaggle.com/code/inversion/submitting-to-the-cafa-competition
The important part is that the submitted notebook comes with a submission.tsv file (in the specified format).
\nWhether this file is generated by the notebook or offline and then attached to it does not matter.
Thanks! Not sure if I missed something obvious or they should add this clarification somewhere visible.
", "rawMarkdown": "Thanks! Not sure if I missed something obvious or they should add this clarification somewhere visible.", "votes": null }, { "id": "2238672", "postDate": "04/28/2023 18:07:32", "content": "@inversion notebook is advertised as a pinned post at the top of code section and explains how to submit:
\nhttps://www.kaggle.com/competitions/cafa-5-protein-function-prediction/code
\nIt can be debated whether that is visible enough, but it is there.
", "rawMarkdown": "inversion notebook is advertised as a pinned post at the top of code section and explains how to submit:\n\nhttps://www.kaggle.com/competitions/cafa-5-protein-function-prediction/code\n\nIt can be debated whether that is visible enough, but it is there.", "votes": null }, { "id": "2238739", "postDate": "04/28/2023 19:01:58", "content": "Sorry for the confusion here. We've added a \"Code Requirements\" tab to clarify.
", "rawMarkdown": "Sorry for the confusion here. We've added a \"Code Requirements\" tab to clarify.", "votes": null }, { "id": "2238740", "postDate": "04/28/2023 19:02:27", "content": "Thanks. We've added a \"Code Requirements\" tab to clarify.
", "rawMarkdown": "Thanks. We've added a \"Code Requirements\" tab to clarify.", "votes": null }, { "id": "2238809", "postDate": "04/28/2023 20:14:00", "content": "I think this is misleading, because this isn't a code competition in the strict sense. For true code competitions all the calculations AND prediction submission must be made from the notebook. In this competition calculations can be done outside of the notebook, which would not be clear to me from \"Code Requirements.\" A couple of extra sentences on that page would go a long way in saving you some clarifications down the road.
", "rawMarkdown": "I think this is misleading, because this isn't a code competition in the strict sense. For true code competitions all the calculations AND prediction submission must be made from the notebook. In this competition calculations can be done outside of the notebook, which would not be clear to me from \"Code Requirements.\" A couple of extra sentences on that page would go a long way in saving you some clarifications down the road.", "votes": null }, { "id": "2239179", "postDate": "04/29/2023 08:29:05", "content": "The notebook simply serves as a container for the submission file and doesn't require any computations to be performed.
", "rawMarkdown": "The notebook simply serves as a container for the submission file and doesn't require any computations to be performed.", "votes": null }, { "id": "2239812", "postDate": "04/29/2023 22:19:56", "content": "I'm still confused. 'calculations can be done outside of the notebook' means the submission file will be used for the final score. But the data set seems to be generated on December. If the data set is being created this December, don't we even know at this point what protein to predict? I don't understand how this will give us the final score based solely on the submission file.
", "rawMarkdown": "I'm still confused. 'calculations can be done outside of the notebook' means the submission file will be used for the final score. But the data set seems to be generated on December. If the data set is being created this December, don't we even know at this point what protein to predict? I don't understand how this will give us the final score based solely on the submission file.", "votes": null }, { "id": "2239814", "postDate": "04/29/2023 22:25:06", "content": "The proteins used for the final scoring are already part of the superset we've been given. However, the labels (GO terms) that will be scored are determined after the deadline.
\nFor example, you might currently predict protein A to have GO term X. If this GO term X is currently not experimentally annotated for protein X but gets annotated before the creation of the final test set, then is would count as a true positive.
\nIn other words: The test set has ~142k proteins; each protein can have any number of the ~43k GO terms. Currently, we do know a few of those through experiments. However, we can assume that many of them we do not yet know, i.e., we have many missing positive labels. The final scoring will then focus on the proteins for which new positive labels (GO terms) have been verified (but not published, i.e., could not have been used for training).
", "rawMarkdown": "The proteins used for the final scoring are already part of the superset we've been given. However, the labels (GO terms) that will be scored are determined after the deadline.\n\nFor example, you might currently predict protein A to have GO term X. If this GO term X is currently not experimentally annotated for protein X but gets annotated before the creation of the final test set, then is would count as a true positive.\n\nIn other words: The test set has ~142k proteins; each protein can have any number of the ~43k GO terms. Currently, we do know a few of those through experiments. However, we can assume that many of them we do not yet know, i.e., we have many missing positive labels. The final scoring will then focus on the proteins for which new positive labels (GO terms) have been verified (but not published, i.e., could not have been used for training).", "votes": null }, { "id": "2239847", "postDate": "04/30/2023 00:08:11", "content": "Understood. Thank you so much!!
", "rawMarkdown": "Understood. Thank you so much!!", "votes": null } ], "comments": [ { "id": 2236889, "author_name": "norixmb", "author_url": "", "post_date": "04/27/2023 07:15:19", "content": "The notebook is actually just a wrapper for the submission file. It does not have to compute anything.
\nhttps://www.kaggle.com/code/inversion/submitting-to-the-cafa-competition
The important part is that the submitted notebook comes with a submission.tsv file (in the specified format).
\nWhether this file is generated by the notebook or offline and then attached to it does not matter.
Thanks! Not sure if I missed something obvious or they should add this clarification somewhere visible.
", "votes": null, "replies": [ { "id": 2238672, "author_name": "tilii7", "author_url": "", "post_date": "04/28/2023 18:07:32", "content": "@inversion notebook is advertised as a pinned post at the top of code section and explains how to submit:
\nhttps://www.kaggle.com/competitions/cafa-5-protein-function-prediction/code
\nIt can be debated whether that is visible enough, but it is there.
", "votes": null, "replies": [ { "id": 2238740, "author_name": "inversion", "author_url": "", "post_date": "04/28/2023 19:02:27", "content": "Thanks. We've added a \"Code Requirements\" tab to clarify.
", "votes": null, "replies": [ { "id": 2238809, "author_name": "tilii7", "author_url": "", "post_date": "04/28/2023 20:14:00", "content": "I think this is misleading, because this isn't a code competition in the strict sense. For true code competitions all the calculations AND prediction submission must be made from the notebook. In this competition calculations can be done outside of the notebook, which would not be clear to me from \"Code Requirements.\" A couple of extra sentences on that page would go a long way in saving you some clarifications down the road.
", "votes": null, "replies": [ { "id": 2239812, "author_name": "aesoptacit", "author_url": "", "post_date": "04/29/2023 22:19:56", "content": "I'm still confused. 'calculations can be done outside of the notebook' means the submission file will be used for the final score. But the data set seems to be generated on December. If the data set is being created this December, don't we even know at this point what protein to predict? I don't understand how this will give us the final score based solely on the submission file.
", "votes": null, "replies": [ { "id": 2239814, "author_name": "norixmb", "author_url": "", "post_date": "04/29/2023 22:25:06", "content": "The proteins used for the final scoring are already part of the superset we've been given. However, the labels (GO terms) that will be scored are determined after the deadline.
\nFor example, you might currently predict protein A to have GO term X. If this GO term X is currently not experimentally annotated for protein X but gets annotated before the creation of the final test set, then is would count as a true positive.
\nIn other words: The test set has ~142k proteins; each protein can have any number of the ~43k GO terms. Currently, we do know a few of those through experiments. However, we can assume that many of them we do not yet know, i.e., we have many missing positive labels. The final scoring will then focus on the proteins for which new positive labels (GO terms) have been verified (but not published, i.e., could not have been used for training).
", "votes": null, "replies": [ { "id": 2239847, "author_name": "aesoptacit", "author_url": "", "post_date": "04/30/2023 00:08:11", "content": "Understood. Thank you so much!!
", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] } ] }, { "id": 2238739, "author_name": "inversion", "author_url": "", "post_date": "04/28/2023 19:01:58", "content": "Sorry for the confusion here. We've added a \"Code Requirements\" tab to clarify.
", "votes": null, "replies": [] }, { "id": 2239179, "author_name": "ericka42", "author_url": "", "post_date": "04/29/2023 08:29:05", "content": "The notebook simply serves as a container for the submission file and doesn't require any computations to be performed.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2236641": "Hey, I was working on creating a baseline submission and noticed that I can't upload any files as it's notebook only. I looked through the rules and couldn't find anything saying it's notebook only. Is this a mistake or is the competition supposed to be notebook only?\n\nIf so, is it okay to develop/submit files from a file (uploaded to notebook, then submitted) _so long as the final submission is computed by a notebook_?\n\nThanks!", "2236889": "The notebook is actually just a wrapper for the submission file. It does not have to compute anything.\nhttps://www.kaggle.com/code/inversion/submitting-to-the-cafa-competition\n\nThe important part is that the submitted notebook comes with a `submission.tsv` file (in the specified format).\nWhether this file is generated by the notebook or offline and then attached to it does not matter.", "2237808": "Thanks! Not sure if I missed something obvious or they should add this clarification somewhere visible.", "2238672": "inversion notebook is advertised as a pinned post at the top of code section and explains how to submit:\n\nhttps://www.kaggle.com/competitions/cafa-5-protein-function-prediction/code\n\nIt can be debated whether that is visible enough, but it is there.", "2238739": "Sorry for the confusion here. We've added a \"Code Requirements\" tab to clarify.", "2238740": "Thanks. We've added a \"Code Requirements\" tab to clarify.", "2238809": "I think this is misleading, because this isn't a code competition in the strict sense. For true code competitions all the calculations AND prediction submission must be made from the notebook. In this competition calculations can be done outside of the notebook, which would not be clear to me from \"Code Requirements.\" A couple of extra sentences on that page would go a long way in saving you some clarifications down the road.", "2239179": "The notebook simply serves as a container for the submission file and doesn't require any computations to be performed.", "2239812": "I'm still confused. 'calculations can be done outside of the notebook' means the submission file will be used for the final score. But the data set seems to be generated on December. If the data set is being created this December, don't we even know at this point what protein to predict? I don't understand how this will give us the final score based solely on the submission file.", "2239814": "The proteins used for the final scoring are already part of the superset we've been given. However, the labels (GO terms) that will be scored are determined after the deadline.\n\nFor example, you might currently predict protein A to have GO term X. If this GO term X is currently not experimentally annotated for protein X but gets annotated before the creation of the final test set, then is would count as a true positive.\n\nIn other words: The test set has ~142k proteins; each protein can have any number of the ~43k GO terms. Currently, we do know a few of those through experiments. However, we can assume that many of them we do not yet know, i.e., we have many missing positive labels. The final scoring will then focus on the proteins for which new positive labels (GO terms) have been verified (but not published, i.e., could not have been used for training).", "2239847": "Understood. Thank you so much!!" }, "source": "meta" }