Hi all!
\nI've been exploring ways to integrate protein family predictions into my models and during this process, I came across the CATH database. This resource (see an example at http://www.cathdb.info/version/latest/superfamily/1.20.190.20) not only provides information on protein families but also stores predicted GO term annotations for each family, which is something I found quite interesting.
I would like to ask if anyone here has experience working with the CATH database. Specifically, I'm interested in understanding the relationship between these predicted GO term annotations and the GO annotations provided by resources such as QuickGO/UniProt. Do these resources copy-transfer each other data?
\nMoreover, I would appreciate any advice on how to build a table consisting of: UniprotID, CATHfamilyID, GoTermsID from this resource. There is a lot of data listed in here, http://www.cathdb.info/wiki?id=data:index but I'm not sure where to start…
", "messages": [ { "id": 2313430, "postDate": "2023-06-22T16:58:20.233Z", "content": "Hi all!
\nI've been exploring ways to integrate protein family predictions into my models and during this process, I came across the CATH database. This resource (see an example at http://www.cathdb.info/version/latest/superfamily/1.20.190.20) not only provides information on protein families but also stores predicted GO term annotations for each family, which is something I found quite interesting.
I would like to ask if anyone here has experience working with the CATH database. Specifically, I'm interested in understanding the relationship between these predicted GO term annotations and the GO annotations provided by resources such as QuickGO/UniProt. Do these resources copy-transfer each other data?
\nMoreover, I would appreciate any advice on how to build a table consisting of: UniprotID, CATHfamilyID, GoTermsID from this resource. There is a lot of data listed in here, http://www.cathdb.info/wiki?id=data:index but I'm not sure where to start…
", "rawMarkdown": "Hi all!\nI've been exploring ways to integrate protein family predictions into my models and during this process, I came across the CATH database. This resource (see an example at http://www.cathdb.info/version/latest/superfamily/1.20.190.20) not only provides information on protein families but also stores predicted GO term annotations for each family, which is something I found quite interesting.\n\nI would like to ask if anyone here has experience working with the CATH database. Specifically, I'm interested in understanding the relationship between these predicted GO term annotations and the GO annotations provided by resources such as QuickGO/UniProt. Do these resources copy-transfer each other data? \n\nMoreover, I would appreciate any advice on how to build a table consisting of: UniprotID, CATHfamilyID, GoTermsID from this resource. There is a lot of data listed in here, http://www.cathdb.info/wiki?id=data:index but I'm not sure where to start...", "votes": 7 }, { "id": 2314610, "postDate": "2023-06-23T13:49:07.863Z", "content": "This is certainly a route I'd considered going down, given that I know Christine's work well and have generally used CATH as my preferred classification of domains (which for the benefit of anyone reading this who's unfamiliar, are structural and evolutionary units comprising proteins).
\nSome things to say that may or may not be obvious to people. Many proteins will have more than one kind of domain, so will have more than one domain with a distinct entry within CATH. Also, function is something that is normally assigned at a protein level (though sometimes a larger scale entity like a protein complex may be more appropriate), and not at a domain level. In many cases, there's no easy association of functions with specific domains - so a domain may correctly carry an annotation that either relates primarily to a different part of the protein, or else that is only possible through different domains together. Thus, transferring functional annotations within a family of domains will inevitably generate false positives.
\nThe obvious way of using CATH in a competition like this is through the FunFams (functional families). Indeed, quoting directly from the CATH website:
\nProteins are typically composed of one or more building blocks, called domains. Protein domains are generally considered as independently-folding units of structure.
\nWithin a superfamily, CATH-Gene3D creates Functional Families (FunFams) that aim to group together domains that share the same function. Therefore, if a region of protein sequence provides a highly significant match to a particular CATH-Gene3D FunFam, then there is a good chance they shares a similar function.
\nTo test this assertion, our function prediction pipeline (based on these FunFams) was submitted to the Critical Assessment of protein Function Annotation (CAFA). Our method was ranked highly for accuracy of function prediction according to a number of different scoring methods.
\nTo use CATH externally, I guess you could look at:
\n(1) Using their API;
\n(2) Scraping the data on the public web pages;
\n(3) Downloading a lot of data and fitting into your own workflow.
You will also need to think carefully about an appropriate heuristic or algorithm for inferring confidence scores (or probabilities) as well as the base predictions from the data available for each FunFam.
\nHowever, the scalability is a massive issue when we're dealing with ~140k proteins. For 140 proteins I'd be confident that CATH was the way for me to go; for a thousand times as many I might want something easier to scale within my very limited programming skills. I got as far as walking through possible processes without scalable code for a couple of FunFams, before deciding that the level of coding needed was beyond my expertise. It's just possible that someone else in my team might yet choose to pick this up, but I've personally moved on to other methodologies unrelated to CATH.
\nI expect that the lists you want would come from downloading CATH data and using pandas.DataFrame.join or similar. However, I've not got close to trying this in practice for CATH data in CAFA 5.
", "rawMarkdown": "This is certainly a route I'd considered going down, given that I know Christine's work well and have generally used CATH as my preferred classification of domains (which for the benefit of anyone reading this who's unfamiliar, are structural and evolutionary units comprising proteins).\n\nSome things to say that may or may not be obvious to people. Many proteins will have more than one kind of domain, so will have more than one domain with a distinct entry within CATH. Also, function is something that is normally assigned at a protein level (though sometimes a larger scale entity like a protein complex may be more appropriate), and not at a domain level. In many cases, there's no easy association of functions with specific domains - so a domain may correctly carry an annotation that either relates primarily to a different part of the protein, or else that is only possible through different domains together. Thus, transferring functional annotations within a family of domains will inevitably generate false positives.\n\nThe obvious way of using CATH in a competition like this is through the FunFams (functional families). Indeed, quoting directly from the [CATH website](http://www.cathdb.info/):\n\n*Proteins are typically composed of one or more building blocks, called domains. Protein domains are generally considered as independently-folding units of structure.*\n\n*Within a superfamily, CATH-Gene3D creates Functional Families (FunFams) that aim to group together domains that share the same function. Therefore, if a region of protein sequence provides a highly significant match to a particular CATH-Gene3D FunFam, then there is a good chance they shares a similar function.*\n\n*To test this assertion, our function prediction pipeline (based on these FunFams) was submitted to the Critical Assessment of protein Function Annotation (CAFA). Our method was ranked highly for accuracy of function prediction according to a number of different scoring methods.*\n\nTo use CATH externally, I guess you could look at:\n\n(1) Using their API;\n(2) Scraping the data on the public web pages;\n(3) Downloading a lot of data and fitting into your own workflow.\n\nYou will also need to think carefully about an appropriate heuristic or algorithm for inferring confidence scores (or probabilities) as well as the base predictions from the data available for each FunFam.\n\nHowever, the scalability is a massive issue when we're dealing with ~140k proteins. For 140 proteins I'd be confident that CATH was the way for me to go; for a thousand times as many I might want something easier to scale within my very limited programming skills. I got as far as walking through possible processes without scalable code for a couple of FunFams, before deciding that the level of coding needed was beyond my expertise. It's just possible that someone else in my team might yet choose to pick this up, but I've personally moved on to other methodologies unrelated to CATH. \n\nI expect that the lists you want would come from downloading CATH data and using [pandas.DataFrame.join](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.join.html) or similar. However, I've not got close to trying this in practice for CATH data in CAFA 5.", "votes": 4, "replies": [ { "id": 2315089, "postDate": "2023-06-23T19:55:35.960Z", "content": "This article https://elifesciences.org/reviewed-preprints/88229#c53 propose to use the FunFam hidden Markov model (HMM) with hmmsearch from HMMER3 (9495) and an E-value threshold of E <10-3. For machine learning, E-values were -log10-transformed to convert to a linear scale and transformed to a [n proteins x m FunFams] matrix, with 99.8% sparsity. So rather than just the presence/absence of a domain, they used probabilities. On the other hand, one could create a similar matrix of zeros and ones from the presence/absence of an Interpro family retrieved from the uniprot annotations…
", "rawMarkdown": "This article https://elifesciences.org/reviewed-preprints/88229#c53 propose to use the FunFam hidden Markov model (HMM) with hmmsearch from HMMER3 (9495) and an E-value threshold of E <10-3. For machine learning, E-values were -log10-transformed to convert to a linear scale and transformed to a [n proteins x m FunFams] matrix, with 99.8% sparsity. So rather than just the presence/absence of a domain, they used probabilities. On the other hand, one could create a similar matrix of zeros and ones from the presence/absence of an Interpro family retrieved from the uniprot annotations...", "votes": 2 } ] }, { "id": 2315612, "postDate": "2023-06-24T09:15:35.827Z", "content": "Some other protein families I put here:
\nhttps://www.kaggle.com/datasets/alexandervc/panther-protein-coding-gene-families/code
\nHere is uniprot reviewed with some additional information (and families, as far as I understand)
\nhttps://www.kaggle.com/datasets/yevgeniyantipin/uniprot-2023-02-reviewed-only
@nicolabordin - noted with thanks.
", "rawMarkdown": "@nicolabordin - noted with thanks. ", "votes": 1 }, { "id": 2318731, "postDate": "2023-06-26T14:41:36.163Z", "content": "Hi Nicola, thanks for your message!
\nI have got a couple of questions, please.
\n1) I run hmmsearch like this:
\nhmmsearch --tblout $output_file --noali -E 0.001 funfam-hmm3.lib $fasta_file > /dev/null
\nfor all sequences. But now I regret it. I removed the aligment info with > /dev/null to save space, but I need start/end of the hits to run cath-resolve hits. Which hmmsearch output file format do you generally use to assemble the input for cath-resolve-hits?
2) What about the go terms, how frequently is the assignment of go terms updated to each CATH family?
\nThanks!
\nMichele
for anyone interested, this is a notebook example to run hmmsearch and cath-resolve-hits on 100 random proteins.
\nhttps://www.kaggle.com/code/mtinti/predictfunfamiliesexample/
\nI'm trying to scale it up now…
This is certainly a route I'd considered going down, given that I know Christine's work well and have generally used CATH as my preferred classification of domains (which for the benefit of anyone reading this who's unfamiliar, are structural and evolutionary units comprising proteins).
\nSome things to say that may or may not be obvious to people. Many proteins will have more than one kind of domain, so will have more than one domain with a distinct entry within CATH. Also, function is something that is normally assigned at a protein level (though sometimes a larger scale entity like a protein complex may be more appropriate), and not at a domain level. In many cases, there's no easy association of functions with specific domains - so a domain may correctly carry an annotation that either relates primarily to a different part of the protein, or else that is only possible through different domains together. Thus, transferring functional annotations within a family of domains will inevitably generate false positives.
\nThe obvious way of using CATH in a competition like this is through the FunFams (functional families). Indeed, quoting directly from the CATH website:
\nProteins are typically composed of one or more building blocks, called domains. Protein domains are generally considered as independently-folding units of structure.
\nWithin a superfamily, CATH-Gene3D creates Functional Families (FunFams) that aim to group together domains that share the same function. Therefore, if a region of protein sequence provides a highly significant match to a particular CATH-Gene3D FunFam, then there is a good chance they shares a similar function.
\nTo test this assertion, our function prediction pipeline (based on these FunFams) was submitted to the Critical Assessment of protein Function Annotation (CAFA). Our method was ranked highly for accuracy of function prediction according to a number of different scoring methods.
\nTo use CATH externally, I guess you could look at:
\n(1) Using their API;
\n(2) Scraping the data on the public web pages;
\n(3) Downloading a lot of data and fitting into your own workflow.
You will also need to think carefully about an appropriate heuristic or algorithm for inferring confidence scores (or probabilities) as well as the base predictions from the data available for each FunFam.
\nHowever, the scalability is a massive issue when we're dealing with ~140k proteins. For 140 proteins I'd be confident that CATH was the way for me to go; for a thousand times as many I might want something easier to scale within my very limited programming skills. I got as far as walking through possible processes without scalable code for a couple of FunFams, before deciding that the level of coding needed was beyond my expertise. It's just possible that someone else in my team might yet choose to pick this up, but I've personally moved on to other methodologies unrelated to CATH.
\nI expect that the lists you want would come from downloading CATH data and using pandas.DataFrame.join or similar. However, I've not got close to trying this in practice for CATH data in CAFA 5.
", "votes": 4, "replies": [ { "id": 2315089, "author_name": "MT", "author_url": "", "post_date": "2023-06-23T19:55:35.960000", "content": "This article https://elifesciences.org/reviewed-preprints/88229#c53 propose to use the FunFam hidden Markov model (HMM) with hmmsearch from HMMER3 (9495) and an E-value threshold of E <10-3. For machine learning, E-values were -log10-transformed to convert to a linear scale and transformed to a [n proteins x m FunFams] matrix, with 99.8% sparsity. So rather than just the presence/absence of a domain, they used probabilities. On the other hand, one could create a similar matrix of zeros and ones from the presence/absence of an Interpro family retrieved from the uniprot annotations…
", "votes": 2, "replies": [] } ] }, { "id": 2315612, "author_name": "Alexander Chervov", "author_url": "", "post_date": "2023-06-24T09:15:35.827000", "content": "Some other protein families I put here:
\nhttps://www.kaggle.com/datasets/alexandervc/panther-protein-coding-gene-families/code
\nHere is uniprot reviewed with some additional information (and families, as far as I understand)
\nhttps://www.kaggle.com/datasets/yevgeniyantipin/uniprot-2023-02-reviewed-only
@nicolabordin - noted with thanks.
", "votes": 1, "replies": [] }, { "id": 2318731, "author_name": "MT", "author_url": "", "post_date": "2023-06-26T14:41:36.163000", "content": "Hi Nicola, thanks for your message!
\nI have got a couple of questions, please.
\n1) I run hmmsearch like this:
\nhmmsearch --tblout $output_file --noali -E 0.001 funfam-hmm3.lib $fasta_file > /dev/null
\nfor all sequences. But now I regret it. I removed the aligment info with > /dev/null to save space, but I need start/end of the hits to run cath-resolve hits. Which hmmsearch output file format do you generally use to assemble the input for cath-resolve-hits?
2) What about the go terms, how frequently is the assignment of go terms updated to each CATH family?
\nThanks!
\nMichele
for anyone interested, this is a notebook example to run hmmsearch and cath-resolve-hits on 100 random proteins.
\nhttps://www.kaggle.com/code/mtinti/predictfunfamiliesexample/
\nI'm trying to scale it up now…