I'm trying to analyze the provided training data, but encountered the following issues that pose inconsistencies between the training data files and the description on Kaggle:
\nThe training set contains proteins from eukaryotes and 17 bacteria-of-interest. However, when checking the train_taxonomy.tsv file provided, I found 19 Bacteria and 1 Archaea (Taxon ID: 243232).selected-taxon-list.tsv mentioned in the description is not provided in the downloaded data files.train_terms.tsv, I cannot find them on the UniProt website, QuickGO website, or the GOA files on UniProt FTP site. For example, the annotations A0A1D6E0S8 GO:0009603 P and A0A1D6E0S8 GO:0005515 F are not on UniProt or QuickGO websites. And for annotation A0A1D6E0S8 GO:0045087 P, the evidence code on QuickGO (https://www.ebi.ac.uk/QuickGO/annotations?geneProductId=A0A1D6E0S8) is IBA, which is not experimental evidence.X1WGX5 GO:0008354 is not included in the train_terms.tsv. This protein X1WGX5 (Taxon ID: 7955) belongs to Eukaryota, and this annotation has an evidence code IMP, which is an experimental evidence code. This annotation should match all the descriptions in the Data section on Kaggle, but is not included in the training data.I would really appreciate it if the organizers could make some clarifications about the above inconsistency data issues.
", "messages": [ { "id": 3310344, "postDate": "2025-11-02T17:03:00.090Z", "content": "I'm trying to analyze the provided training data, but encountered the following issues that pose inconsistencies between the training data files and the description on Kaggle:
\nThe training set contains proteins from eukaryotes and 17 bacteria-of-interest. However, when checking the train_taxonomy.tsv file provided, I found 19 Bacteria and 1 Archaea (Taxon ID: 243232).selected-taxon-list.tsv mentioned in the description is not provided in the downloaded data files.train_terms.tsv, I cannot find them on the UniProt website, QuickGO website, or the GOA files on UniProt FTP site. For example, the annotations A0A1D6E0S8 GO:0009603 P and A0A1D6E0S8 GO:0005515 F are not on UniProt or QuickGO websites. And for annotation A0A1D6E0S8 GO:0045087 P, the evidence code on QuickGO (https://www.ebi.ac.uk/QuickGO/annotations?geneProductId=A0A1D6E0S8) is IBA, which is not experimental evidence.X1WGX5 GO:0008354 is not included in the train_terms.tsv. This protein X1WGX5 (Taxon ID: 7955) belongs to Eukaryota, and this annotation has an evidence code IMP, which is an experimental evidence code. This annotation should match all the descriptions in the Data section on Kaggle, but is not included in the training data.I would really appreciate it if the organizers could make some clarifications about the above inconsistency data issues.
", "rawMarkdown": "I'm trying to analyze the provided training data, but encountered the following issues that pose inconsistencies between the training data files and the description on Kaggle:\n1. The description says, `The training set contains proteins from eukaryotes and 17 bacteria-of-interest.` However, when checking the `train_taxonomy.tsv` file provided, I found 19 Bacteria and 1 Archaea (Taxon ID: 243232).\n2. The file `selected-taxon-list.tsv` mentioned in the description is not provided in the downloaded data files.\n3. For some annotations provided in `train_terms.tsv`, I cannot find them on the UniProt website, QuickGO website, or the GOA files on UniProt FTP site. For example, the annotations `A0A1D6E0S8\tGO:0009603\tP` and `A0A1D6E0S8\tGO:0005515\tF` are not on UniProt or QuickGO websites. And for annotation `A0A1D6E0S8\tGO:0045087\tP`, the evidence code on QuickGO (https://www.ebi.ac.uk/QuickGO/annotations?geneProductId=A0A1D6E0S8) is IBA, which is not experimental evidence.\n4. Some annotations in Swiss-Prot, for example, `X1WGX5 GO:0008354` is not included in the `train_terms.tsv`. This protein X1WGX5 (Taxon ID: 7955) belongs to Eukaryota, and this annotation has an evidence code IMP, which is an experimental evidence code. This annotation should match all the descriptions in the Data section on Kaggle, but is not included in the training data.\n\nI would really appreciate it if the organizers could make some clarifications about the above inconsistency data issues.", "votes": 5 }, { "id": 3310379, "postDate": "2025-11-02T18:24:13.790Z", "content": "(1) You are correct that the taxon ID:243232 is Archaea. We will modify the description to reflect this. About the number of bacteria-of-interest, a slight variation occurred due to some bacteria (like E.coli or Pseudomonas aeruginosa) having more than one taxon ID (for different strains) and could make the number of \"species\" slightly off. Sometimes the species level taxonID is used, and sometimes the strain level taxonID is used (in the FASTA file).\nExample:\n\"Escherichia coli (strain K12)\" has taxid:83333, and \"Escherichia coli (strain K12 / MC4100 / BW2952)\" has taxid:595496
\n(2) The description mentioned the filename selected-taxon-list.tsv, while the actual provided file was named testsuperset-taxon-list.tsv. Thank you for pointing this out, we will fix the description to match the name of the provided file.
(3) I double checked your examples, and A0A1D6E0S8 GO:0009603 P and A0A1D6E0S8 GO:0005515 F are included in the GOA file on UniProt FTP, and the annotation A0A1D6E0S8 GO:0045087 P has an evidence code of \"IMP\" (in addition to \"IBA\") in the same GOA file. Please refer to the GOA files on UniProt FTP site as the standard data source since this is where we obtained the released data and will obtain the evaluation data from.
(4) The annotation X1WGX5 GO:0008354 is not included in the goa_uniprot_all.gaf.226.gz file, so it was not included in the train_terms.tsv. It is just an inconsistency between different data sources.
Thank you so much for the quick response! It really helps!
", "rawMarkdown": "Thank you so much for the quick response! It really helps!", "votes": 1, "replies": [ { "id": 3310604, "postDate": "2025-11-03T09:01:09.970Z", "content": "Thanks for raising it up. It seems like QuickGO API is not reliable source of truth.
", "rawMarkdown": "Thanks for raising it up. It seems like QuickGO API is not reliable source of truth.", "votes": 1 } ] }, { "id": 3310693, "postDate": "2025-11-03T13:46:08.387Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3310737, "postDate": "2025-11-03T14:52:24.170Z", "content": "I have another question about the species in the training and test superset. The provided training set contains roughly 1381 species (unique taxon ids), while the test superset contains about 8453 species (unique taxon ids). Would the final time-based evaluation only select proteins from the species in the training set or from the species in the test superset? Thanks!
", "rawMarkdown": "I have another question about the species in the training and test superset. The provided training set contains roughly 1381 species (unique taxon ids), while the test superset contains about 8453 species (unique taxon ids). Would the final time-based evaluation only select proteins from the species in the training set or from the species in the test superset? Thanks!", "replies": [ { "id": 3310741, "postDate": "2025-11-03T15:03:19.393Z", "content": "Both training set and test superset were selected from the same list of taxon ids (training set has fewer species because a large number of species does not have any experimentally annotated proteins). The final evaluation set will be a subset of the test superset, so the proteins will all come from the provided list of taxon ids in testsuperset-taxon-list.tsv.
Thank you so much for the clarification!
", "rawMarkdown": "Thank you so much for the clarification!" } ] }, { "id": 3311164, "postDate": "2025-11-04T12:06:02.690Z", "content": "Hello, sir, did you update the training data and test ways these three day for the reason this author stated? I found we could not get the score with pre script, so werid
", "rawMarkdown": "Hello, sir, did you update the training data and test ways these three day for the reason this author stated? I found we could not get the score with pre script, so werid", "replies": [ { "id": 3311229, "postDate": "2025-11-04T14:34:57.537Z", "content": "No data files were updated recently, only the description in the Data tab has been updated to match the correct description of species (eukaryotes + 13 bacteria + 1 archaea). Also, the file name selected-taxon-list.tsv in the description did not match the actual file provided testsuperset-taxon-list.tsv, so that has been fixed.
(1) You are correct that the taxon ID:243232 is Archaea. We will modify the description to reflect this. About the number of bacteria-of-interest, a slight variation occurred due to some bacteria (like E.coli or Pseudomonas aeruginosa) having more than one taxon ID (for different strains) and could make the number of \"species\" slightly off. Sometimes the species level taxonID is used, and sometimes the strain level taxonID is used (in the FASTA file).\nExample:\n\"Escherichia coli (strain K12)\" has taxid:83333, and \"Escherichia coli (strain K12 / MC4100 / BW2952)\" has taxid:595496
\n(2) The description mentioned the filename selected-taxon-list.tsv, while the actual provided file was named testsuperset-taxon-list.tsv. Thank you for pointing this out, we will fix the description to match the name of the provided file.
(3) I double checked your examples, and A0A1D6E0S8 GO:0009603 P and A0A1D6E0S8 GO:0005515 F are included in the GOA file on UniProt FTP, and the annotation A0A1D6E0S8 GO:0045087 P has an evidence code of \"IMP\" (in addition to \"IBA\") in the same GOA file. Please refer to the GOA files on UniProt FTP site as the standard data source since this is where we obtained the released data and will obtain the evaluation data from.
(4) The annotation X1WGX5 GO:0008354 is not included in the goa_uniprot_all.gaf.226.gz file, so it was not included in the train_terms.tsv. It is just an inconsistency between different data sources.
Thank you so much for the quick response! It really helps!
", "votes": 1, "replies": [ { "id": 3310604, "author_name": "Mateusz Pieniak", "author_url": "", "post_date": "2025-11-03T09:01:09.970000", "content": "Thanks for raising it up. It seems like QuickGO API is not reliable source of truth.
", "votes": 1, "replies": [] } ] }, { "id": 3310693, "author_name": "", "author_url": "", "post_date": "2025-11-03T13:46:08.387000", "content": "", "votes": 0, "replies": [] }, { "id": 3310737, "author_name": "JIAQI LUO", "author_url": "", "post_date": "2025-11-03T14:52:24.170000", "content": "I have another question about the species in the training and test superset. The provided training set contains roughly 1381 species (unique taxon ids), while the test superset contains about 8453 species (unique taxon ids). Would the final time-based evaluation only select proteins from the species in the training set or from the species in the test superset? Thanks!
", "votes": 0, "replies": [ { "id": 3310741, "author_name": "An Phan", "author_url": "", "post_date": "2025-11-03T15:03:19.393000", "content": "Both training set and test superset were selected from the same list of taxon ids (training set has fewer species because a large number of species does not have any experimentally annotated proteins). The final evaluation set will be a subset of the test superset, so the proteins will all come from the provided list of taxon ids in testsuperset-taxon-list.tsv.
Thank you so much for the clarification!
", "votes": 0, "replies": [] } ] }, { "id": 3311164, "author_name": "common-people", "author_url": "", "post_date": "2025-11-04T12:06:02.690000", "content": "Hello, sir, did you update the training data and test ways these three day for the reason this author stated? I found we could not get the score with pre script, so werid
", "votes": 0, "replies": [ { "id": 3311229, "author_name": "An Phan", "author_url": "", "post_date": "2025-11-04T14:34:57.537000", "content": "No data files were updated recently, only the description in the Data tab has been updated to match the correct description of species (eukaryotes + 13 bacteria + 1 archaea). Also, the file name selected-taxon-list.tsv in the description did not match the actual file provided testsuperset-taxon-list.tsv, so that has been fixed.