Hi @iddux1
\nCould you please explain the rationale behind the choice of GO termset for the training?
\nI see a lot of cases when QuickGO annotation has much more comprehensive and de-facto experimentally proven GO terms for the given protein then the list you provided in train_terms.tsv for the same protein.
Just one example: for X2JGI5 train_terms.tsv contains only these terms:
\n name IA\n: BPO organelle organization .\n: BPO cytoskeleton organization .\n: BPO biological_process \n: BPO anatomical structure morphogenesis .\n: BPO cellular process .\n: BPO cellular component assembly involved in morphogenesis .\n: BPO cellular component organization .\n: BPO cellular component assembly .\n: BPO actin filament-based process .\n: BPO actin cytoskeleton organization .\n: BPO cell differentiation .\n: BPO myofibril assembly \n: BPO actomyosin structure organization .\n: BPO developmental process .\n: BPO cellular component morphogenesis .\n: BPO muscle cell differentiation .\n: BPO cellular component biogenesis .\n: BPO cell development .\n: BPO anatomical structure formation involved in morphogenesis .````\n: BPO anatomical structure development .\n: BPO cellular developmental process .\n: BPO striated muscle cell differentiation .\n: BPO muscle cell development .\n: BPO striated muscle cell development .\n: BPO muscle structure development .\n: BPO organelle assembly .\n: BPO cellular component organization or biogenesis .\n: BPO supramolecular fiber organization .\n: BPO non-membrane-bounded organelle assembly .\nIn parallel, QuickGO list for the same protein (https://www.ebi.ac.uk/QuickGO/annotations?geneProductId=X2JGI5) is
\n stress fiber cellular_component\n actin binding molecular_function\n adherens junction cellular_component\n Z disc cellular_component\n actin cytoskeleton organization biological_process\n myofibril biological_process\n filamentous actin cellular_component\n muscle alpha-actinin binding molecular_function\n muscle development biological_process \nMost of terms are from GeneOntology Central, with non-experimental evidence code ECO:0000318 (ancestor evidence that is used in a manual assertion).
\nIt has to be stressed that recently all of them have been experimentally proven (see doi:10.1371/journal.pgen.1003342, doi:10.1534/genetics.119.302217). One of those references is mentioned by QuickGO (doi:10.1371/journal.pgen.1003342 = PMID:23505387), but not all experimental evidences from there are propagated to ECO codes.
Inter alia, in this specific case QuickGO terms belong to all 3 subontologies, and not only to BP as in your list.
\nWhen terms from train_terms.tsv for the given protein are missing in QuickGO, then they usually are the parents in ontology hierarchy - like GO:0030239 myofibril assembly in QuickGO for X2JGI5 immediately implies GO:0097435 supramolecular fiber organization, GO:0051146 striated muscle cell differentiation, GO:0042692 muscle cell differentiation, GO:0006996 organelle organization, GO:0032502 developmental process, GO:0048869 cellular developmental process, GO:0008150 biological_process, GO:0048856 anatomical structure development ).
\nSo, what was the rationale to compile train_terms.tsv as the basis for training?
Moreover, if the evaluation of prediction is based on annotations made in the same fashion as train_terms.tsv (effectively, ignoring experimental evidences from 2019 on), then the effective goal of this competition seems to be to mimic a particular annotation workflow, and not to understand the function of proteins.
\nAm I missing anything?
\nThank you in advance.
Hi @iddux1
\nCould you please explain the rationale behind the choice of GO termset for the training?
\nI see a lot of cases when QuickGO annotation has much more comprehensive and de-facto experimentally proven GO terms for the given protein then the list you provided in train_terms.tsv for the same protein.
Just one example: for X2JGI5 train_terms.tsv contains only these terms:
\n name IA\n: BPO organelle organization .\n: BPO cytoskeleton organization .\n: BPO biological_process \n: BPO anatomical structure morphogenesis .\n: BPO cellular process .\n: BPO cellular component assembly involved in morphogenesis .\n: BPO cellular component organization .\n: BPO cellular component assembly .\n: BPO actin filament-based process .\n: BPO actin cytoskeleton organization .\n: BPO cell differentiation .\n: BPO myofibril assembly \n: BPO actomyosin structure organization .\n: BPO developmental process .\n: BPO cellular component morphogenesis .\n: BPO muscle cell differentiation .\n: BPO cellular component biogenesis .\n: BPO cell development .\n: BPO anatomical structure formation involved in morphogenesis .````\n: BPO anatomical structure development .\n: BPO cellular developmental process .\n: BPO striated muscle cell differentiation .\n: BPO muscle cell development .\n: BPO striated muscle cell development .\n: BPO muscle structure development .\n: BPO organelle assembly .\n: BPO cellular component organization or biogenesis .\n: BPO supramolecular fiber organization .\n: BPO non-membrane-bounded organelle assembly .\nIn parallel, QuickGO list for the same protein (https://www.ebi.ac.uk/QuickGO/annotations?geneProductId=X2JGI5) is
\n stress fiber cellular_component\n actin binding molecular_function\n adherens junction cellular_component\n Z disc cellular_component\n actin cytoskeleton organization biological_process\n myofibril biological_process\n filamentous actin cellular_component\n muscle alpha-actinin binding molecular_function\n muscle development biological_process \nMost of terms are from GeneOntology Central, with non-experimental evidence code ECO:0000318 (ancestor evidence that is used in a manual assertion).
\nIt has to be stressed that recently all of them have been experimentally proven (see doi:10.1371/journal.pgen.1003342, doi:10.1534/genetics.119.302217). One of those references is mentioned by QuickGO (doi:10.1371/journal.pgen.1003342 = PMID:23505387), but not all experimental evidences from there are propagated to ECO codes.
Inter alia, in this specific case QuickGO terms belong to all 3 subontologies, and not only to BP as in your list.
\nWhen terms from train_terms.tsv for the given protein are missing in QuickGO, then they usually are the parents in ontology hierarchy - like GO:0030239 myofibril assembly in QuickGO for X2JGI5 immediately implies GO:0097435 supramolecular fiber organization, GO:0051146 striated muscle cell differentiation, GO:0042692 muscle cell differentiation, GO:0006996 organelle organization, GO:0032502 developmental process, GO:0048869 cellular developmental process, GO:0008150 biological_process, GO:0048856 anatomical structure development ).
\nSo, what was the rationale to compile train_terms.tsv as the basis for training?
Moreover, if the evaluation of prediction is based on annotations made in the same fashion as train_terms.tsv (effectively, ignoring experimental evidences from 2019 on), then the effective goal of this competition seems to be to mimic a particular annotation workflow, and not to understand the function of proteins.
\nAm I missing anything?
\nThank you in advance.
As you noted, the IBA evidence code is not one that we are using in the evaluation, and therefore we also omitted any annotations using that evidence code from the set of training terms. We do not consider IBA to be an experimental evidence code for the purposes of CAFA.
\n\n\nFor the training set, we include all proteins with annotated terms that have been validated by experimental or high-throughput evidence, traceable author statement (evidence code TAS), or inferred by curator (IC).
\n
You are welcome (even encouraged!) to include proteins that have IBA evidence codes (or any other ones) in your own training of course, as we state:
\n\n\nThe participants are not required to use these data and are also welcome to use any other data available to them.
\n
As for recent (or not so recent) experimental evidence in literature: by definition the literature precedes the biocurated data, sometimes by years. You are welcome to use any and all evidence you can gather for your own training. Inclusion of text mining has historically been shown to be quite helpful in CAFA competitions.
\nFor convenience, we also include all propagated ancestor terms. While QuickGO interface does not include those explicitly, as you noted, any term includes all of its ancestors implicitly following the True Path Rule.
", "rawMarkdown": "As you noted, the IBA evidence code is not one that we are using in the evaluation, and therefore we also omitted any annotations using that evidence code from the set of training terms. We do not consider IBA to be an experimental evidence code for the purposes of CAFA.\n\n>For the training set, we include all proteins with annotated terms that have been validated by experimental or high-throughput evidence, traceable author statement (evidence code TAS), or inferred by curator (IC). \n\nYou are welcome (even encouraged!) to include proteins that have IBA evidence codes (or any other ones) in your own training of course, as we state:\n\n>The participants are not required to use these data and are also welcome to use any other data available to them.\n\nAs for recent (or not so recent) experimental evidence in literature: by definition the literature precedes the biocurated data, sometimes by years. You are welcome to use any and all evidence you can gather for your own training. Inclusion of text mining has historically been shown to be quite helpful in CAFA competitions. \n\nFor convenience, we also include all propagated ancestor terms. While QuickGO interface does not include those explicitly, as you noted, any term includes all of its ancestors implicitly following the [True Path Rule](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC311077/ ).", "votes": null }, { "id": "2236879", "postDate": "04/27/2023 06:57:31", "content": "Clarification: You said \"I see a lot of cases when QuickGO annotation has much more comprehensive and de-facto experimentally proven GO terms for the given protein then the list you provided in train_terms.tsv for the same protein.\"
\nBut in the example you gave, QuickGO has less (experimental) annotations. Is that intentional? Or did you mean something else in the initial sentence?
As you noted, the IBA evidence code is not one that we are using in the evaluation, and therefore we also omitted any annotations using that evidence code from the set of training terms. We do not consider IBA to be an experimental evidence code for the purposes of CAFA.
\n\n\nFor the training set, we include all proteins with annotated terms that have been validated by experimental or high-throughput evidence, traceable author statement (evidence code TAS), or inferred by curator (IC).
\n
You are welcome (even encouraged!) to include proteins that have IBA evidence codes (or any other ones) in your own training of course, as we state:
\n\n\nThe participants are not required to use these data and are also welcome to use any other data available to them.
\n
As for recent (or not so recent) experimental evidence in literature: by definition the literature precedes the biocurated data, sometimes by years. You are welcome to use any and all evidence you can gather for your own training. Inclusion of text mining has historically been shown to be quite helpful in CAFA competitions.
\nFor convenience, we also include all propagated ancestor terms. While QuickGO interface does not include those explicitly, as you noted, any term includes all of its ancestors implicitly following the True Path Rule.
", "votes": null, "replies": [] }, { "id": 2236879, "author_name": "danofer", "author_url": "", "post_date": "04/27/2023 06:57:31", "content": "Clarification: You said \"I see a lot of cases when QuickGO annotation has much more comprehensive and de-facto experimentally proven GO terms for the given protein then the list you provided in train_terms.tsv for the same protein.\"
\nBut in the example you gave, QuickGO has less (experimental) annotations. Is that intentional? Or did you mean something else in the initial sentence?