![](\"https://i.ibb.co/PGGhF39/Sequence-Lenght.png\")
Recent advancements in Bio-Tech Industry have led to the enormous increase in the amount of Genetic/Protein Sequences, which is expected to increase further too. These studies help in further advancements such as
evolutionary relationships between organismsevolutionary processes that produced the diversity of life on Earthhow living organisms interact with their environment/how they respond to stimuliThough , Annotations of these Protein Sequences are not at the same pace, which is done with vitro/vivo experiments. These prediction process face many complex challenges such as
Sequence-Structure-Function SSF adds complex relations among protein sequence/structure/functionProteins do not only have a single function but may be Pleiotropic and have multiple different functionsGO contains strong/formally defined relations between functions that need to be taken into account during function prediction to ensure that theseHigher-level Physiological functions that go beyond simple molecular interactions, such as apoptosis or regulation of heart rate, will require other proteins and cannot usually be predicted by considering a single protein in isolation. There are 2 kinds of major Representations chooses.
Traditional Protein Sequences GO OntologyProtein Protein Interaction PPIFirstly these 2 forms of representations are trained on a Neural Network
features from the Protein Sequencesinteraction among different Proteins based on their locationFurther these features are used to in a Novel Deep Neuro-Symbolic Model that will predict the classes of the Proteins
| BPO | \nMFO | \nCCO | \nTotal | \n
|---|---|---|---|
28,647 | \n10,161 | \n3,907 | \n44,683 Classes/1,968 Obsolete | \n
Protein Sequences are taken from Uni-Prot containing 5,53,232 Proteins/5,25,931 Annotations.
| \n | Amount | \nTotal | \n
|---|---|---|
| BPO | \n19,181 | \n\n |
| MFO | \n6,221 | \n\n |
| CCO | \n2,358 | \n\n |
| \n | \n | 60,710 Proteins | \n
| \n | \n | 27,760 Classes | \n
The Embeddings are taken from A Neural Probabilistic Language Model
\nAlong this Protein-Protein Interaction - Neural Networks PPI-NNs String V10 combined with EggNOG DataBase are also used.
For the Uni-Prot, only Proteins with experimental codes with (EXP/IDA/IPI/IGI/IEP/TAS/IC) are considered. Protein with bigger than length 1002 and codes (B/O/J/U/X/Z) are ignored
The Sequences are first padded made to a length of 1,002 and then projected to a vector of 1,000 Dimensions
For the String V10, the data is first filtered by a confidence score of 300, further concerted with Orthology Relations from EggNOG DataBase. In total there are 84,78,935 Proteins/1,90,649 Edge Types/ 11,58,66,95,610 Edges. With this data, a Knowledge Graph is created with embeddings of size 256. Further 69,60,395 Proteins are mapped from Uni-Prot
A Classical 1-D CNN is used here. with the Convolution Function being
$$h(y) = \\sum\\limits_{x = 1}^{k}f(x) * g(y * d - x + c)$$
\nwhere
This is the same formula for the output layer of a Convolution
\nThe First Layer is intended to learn embeddings where each index is mapped to a dense vector by referring to a lookup embedding table with a size of 128, representing a Protein Sequence of length of 1002 as a matrix of 1000 x 128.
| Layer | \nFilters | \nFilter Size | \nStride | \n
|---|---|---|---|
| Conv1D - 1 | \n32 | \n128 | \n\n |
| Max-Pooling - 1 | \n\n | 64 | \n32 | \n
In the end, a vector of 832 Dimensions
The PPI Sequences is connected with a Layer of 1,024 Neurons
![](\"https://i.ibb.co/HzchbRR/Deep-Go-Model.png\")
Sigmoid as the Default Activation Function is used, along with Dropout Layers to avoid Overfitting for both the models
Multi-Output-Binary-Cross-Entropy , a loss function that is used for multi-label classification problems is used
$$loss = -\\sum(ylog(p) + (1 - y)log(1 - p))$$
\nRoot-Mean-Squared-Propagation Optimizer, an Adaptive Learning Rate Optimizer, which adjusts the learning rate of the algorithm based on the history of gradients is used with Batch size - 128/Lr - 0.01
$$w = β w_{-1} + (1 - β)g^2$$
\n$$α = α{-1}\\frac{1}{w{-1}}$$
First the dataset is separated with 4:1 Ratio
80%20%Training is done on NVIDIA Pascals X GPU, whcih took a little less than 3 Hours (BPO Model)
The Training loop can be seen on deepgo
\nThe test set for Comparison is from the 5 June 2017, which contains 1367 Proteins 3619 Annotations
![](\"https://i.ibb.co/XsDB7fB/D-Evalution-3.png\")
![](\"https://i.ibb.co/G3T4c6n/D-Evalution-2.png\")
The Evaluation is made on the metrics of F-Max Score
$$pr_i(t) = \\frac{\\sum_fI(fŒP_i(t)⋀fŒT_i)}{\\sum_fI(fŒP_i(t))}$$
\n$$pr_i(t) = \\frac{\\sum_fI(fŒP_i(t)⋀fŒT_i)}{\\sum_fI(fŒt_i)}$$
\n$$AvgPr(t) = \\frac{1}{m(t)}\\sum\\limits_{i = 1}^{m(t)}pr_i(t)$$
\n$$AvgRc(t) = \\frac{1}{n}\\sum\\limits_{i = 1}^{n}rc_i(t)$$
\n$$F_{max} = max(\\frac{2AvgPr(t)AvgRc(t)}{AvgPr(t) + AvgRc(t)})$$
where
\nThe Second Metric is the AUC-ROC Curve of Sensitivity
\nAdditionally Global AucRoc Curve is also calculated
\n![](\"https://i.ibb.co/3Mbx1LZ/D-Evalution-1.png\")
![](\"https://i.ibb.co/bH5T3B7/D-Evalution-5.png\")
| Paper Name | \nAuthors | \nPlace | \nYear | \n
|---|---|---|---|
| Tensorflow: A system for large-scale machine learning | \nAbadi,M. et al. | \nBerkeley, CA, USA | \n2016 | \n
| Molecular Biology of the Cell, 4th edn | \nAlberts,B. et al. | \nNew York, NY, USA | \n2002 | \n
| Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning | \nAlipanahi,B. et al. | \n- | \n2015 | \n
| Neuro-symbolic representation learning on biological knowledge graphs | \nAlshahrani,M. et al. | \n- | \n2017 | \n
| Gapped BLAST and PSI-BLAST: a new generation of protein database search programs | \nAltschul,S.F. et al. | \n- | \n1997 | \n
| Gene ontology: tool for the unification of biology | \nAshburner,M. et al. | \n- | \n2000 | \n
| Systematic functional annotation and visualization of biological networks | \nBaryshnikova,A. | \n- | \n2016 | \n
| A neural probabilistic language model | \nBengio,Y. et al. | \n- | \n2003 | \n
| UniProtKB/Swiss-Prot, the Manually Annotated Section of the UniProt KnowledgeBase: How to Use the Entry View | \nBoutet,E. et al. | \nNew York, NY | \n- | \n
| Information-theoretic evaluation of predicted ontological annotations | \nClark,W.T. and Radivojac,P. | \n- | \n2013 | \n
| Natural language processing (almost) from scratch | \nCollobert,R. et al. | \n- | \n2011 | \n
| A global genetic interaction network maps a wiring diagram of cellular function | \nCostanzo,M. et al. | \n- | \n2016 | \n
| Ffpred 3: feature-based function prediction for all gene ontology domains | \nCozzetto,D. et al. | \n- | \n2016 | \n
| Functional classification of cath superfamilies: a domain-based approach for protein function annotation | \nDas,S. et al. | \n- | \n2015 | \n
| Genome-scale phylogenetic function annotation of large and diverse protein families | \nEngelhardt,B.E. et al. | \n- | \n2011 | \n
| Interpro in 2017beyond protein family and domain annotations | \nFinn,R.D. et al. | \n- | \n2017 | \n
| Phylogenetic-based propagation of functional annotations within the gene ontology consortium | \nGaudet,P. et al. | \n- | \n2011 | \n
| The anatomy of phenotype ontologies: principles, properties and applications | \nGkoutos,G.V. et al. | \n- | \n2017 | \n
| Understanding the difficulty of training dee feedforward neural networks | \nGlorot,X. and Bengio,Y. | \n- | \n2010 | \n
| Gofdr: a sequence alignment based method for predicting protein functions | \nGong,Q. et al. | \n- | \n2016 | \n
| Learning distributed representations of concepts | \nHinton,G.E. | \nAmherst, MA | \n1986 | \n
| Protein function prediction from functional connectivity | \nHou,J. | \nSalt Lake City, UT, USA | \n2017 | \n
| eggnog 4.5: a hierarchical orthology framework with improved functional annotations for eukaryotic, prokaryotic and viral sequences | \nHuerta-Cepas,J. et al. | \n- | \n2016 | \n
| Predicting protein function by multi-label correlated semi-supervised learning | \nJiang,J.Q. and McQuay,L.J. | \n- | \n2012 | \n
| Protein Function Prediction Based on Patterns in Biological Networks | \nKirac,M. and Ozsoyoglu,G. | \n- | \n2008 | \n
| Convolutional networks for images, speech, and time series | \nLeCun,Y. and Bengio,Y. | \nCambridge, MA, USA | \n1998 | \n
| Critical assessment of methods of protein structure prediction (casp) round x | \nMoult,J. et al. | \n- | \n2014 | \n
| Protein annotation from protein interaction networks and gene ontology | \nNguyen,C.D. et al. | \n- | \n2011 | \n
| Annotating the human genome with disease ontology | \nOsborne,J. et al. | \n- | \n2009 | \n
| A large-scale evaluation of computational protein function prediction | \nRadivojac,P. et al. | \n- | \n2013 | \n
| Semantic similarity in a taxonomy: an Information-Based measure and its application to problems of ambiguity in natural language | \nResnik,P. | \n- | \n1999 | \n
| Network-based prediction of protein function | \nSharan,R. et al. | \n- | \n2007 | \n
| Hierarchical classification of gene ontology terms using the gostruct method | \nSokolov,A. and Ben-Hur,A. | \n- | \n2010 | \n
| String v10: proteinprotein interaction networks, integrated over the tree of life | \nSzklarczyk,D. et al. | \n- | \n2015 | \n
| Lecture 6.5—RmsProp: Divide the gradient by a running average of its recent magnitude | \nTieleman,T. and Hinton,G. | \n- | \n2012 | \n
| Roles for Text Mining in Protein Function Prediction | \nVerspoor,K.M. | \n- | \n2014 | \n
| Accurate de novo prediction of protein contact map by ultra-deep learning model | \nWang,S. et al. | \n- | \n2017 | \n
| Combfunc: predicting protein function using heterogeneous data sources | \nWass,M.N. et al. | \n- | \n2012 | \n
| Predicting effects of noncoding vari | \nZhou,J. and Troyanskaya,O.G. | \n- | \n2015 | \n
Recent advancements in Bio-Tech Industry have led to the enormous increase in the amount of Genetic/Protein Sequences, which is expected to increase further too. These studies help in further advancements such as
evolutionary relationships between organismsevolutionary processes that produced the diversity of life on Earthhow living organisms interact with their environment/how they respond to stimuliThough , Annotations of these Protein Sequences are not at the same pace, which is done with vitro/vivo experiments. These prediction process face many complex challenges such as
Sequence-Structure-Function SSF adds complex relations among protein sequence/structure/functionProteins do not only have a single function but may be Pleiotropic and have multiple different functionsGO contains strong/formally defined relations between functions that need to be taken into account during function prediction to ensure that theseHigher-level Physiological functions that go beyond simple molecular interactions, such as apoptosis or regulation of heart rate, will require other proteins and cannot usually be predicted by considering a single protein in isolation. There are 2 kinds of major Representations chooses.
Traditional Protein Sequences GO OntologyProtein Protein Interaction PPIFirstly these 2 forms of representations are trained on a Neural Network
features from the Protein Sequencesinteraction among different Proteins based on their locationFurther these features are used to in a Novel Deep Neuro-Symbolic Model that will predict the classes of the Proteins
| BPO | \nMFO | \nCCO | \nTotal | \n
|---|---|---|---|
28,647 | \n10,161 | \n3,907 | \n44,683 Classes/1,968 Obsolete | \n
Protein Sequences are taken from Uni-Prot containing 5,53,232 Proteins/5,25,931 Annotations.
| \n | Amount | \nTotal | \n
|---|---|---|
| BPO | \n19,181 | \n\n |
| MFO | \n6,221 | \n\n |
| CCO | \n2,358 | \n\n |
| \n | \n | 60,710 Proteins | \n
| \n | \n | 27,760 Classes | \n
The Embeddings are taken from A Neural Probabilistic Language Model
\nAlong this Protein-Protein Interaction - Neural Networks PPI-NNs String V10 combined with EggNOG DataBase are also used.
For the Uni-Prot, only Proteins with experimental codes with (EXP/IDA/IPI/IGI/IEP/TAS/IC) are considered. Protein with bigger than length 1002 and codes (B/O/J/U/X/Z) are ignored
The Sequences are first padded made to a length of 1,002 and then projected to a vector of 1,000 Dimensions
For the String V10, the data is first filtered by a confidence score of 300, further concerted with Orthology Relations from EggNOG DataBase. In total there are 84,78,935 Proteins/1,90,649 Edge Types/ 11,58,66,95,610 Edges. With this data, a Knowledge Graph is created with embeddings of size 256. Further 69,60,395 Proteins are mapped from Uni-Prot
A Classical 1-D CNN is used here. with the Convolution Function being
$$h(y) = \\sum\\limits_{x = 1}^{k}f(x) * g(y * d - x + c)$$
\nwhere
This is the same formula for the output layer of a Convolution
\nThe First Layer is intended to learn embeddings where each index is mapped to a dense vector by referring to a lookup embedding table with a size of 128, representing a Protein Sequence of length of 1002 as a matrix of 1000 x 128.
| Layer | \nFilters | \nFilter Size | \nStride | \n
|---|---|---|---|
| Conv1D - 1 | \n32 | \n128 | \n\n |
| Max-Pooling - 1 | \n\n | 64 | \n32 | \n
In the end, a vector of 832 Dimensions
The PPI Sequences is connected with a Layer of 1,024 Neurons
Sigmoid as the Default Activation Function is used, along with Dropout Layers to avoid Overfitting for both the models
Multi-Output-Binary-Cross-Entropy , a loss function that is used for multi-label classification problems is used
$$loss = -\\sum(ylog(p) + (1 - y)log(1 - p))$$
\nRoot-Mean-Squared-Propagation Optimizer, an Adaptive Learning Rate Optimizer, which adjusts the learning rate of the algorithm based on the history of gradients is used with Batch size - 128/Lr - 0.01
$$w = β w_{-1} + (1 - β)g^2$$
\n$$α = α{-1}\\frac{1}{w{-1}}$$
First the dataset is separated with 4:1 Ratio
80%20%Training is done on NVIDIA Pascals X GPU, whcih took a little less than 3 Hours (BPO Model)
The Training loop can be seen on deepgo
\nThe test set for Comparison is from the 5 June 2017, which contains 1367 Proteins 3619 Annotations
The Evaluation is made on the metrics of F-Max Score
$$pr_i(t) = \\frac{\\sum_fI(fŒP_i(t)⋀fŒT_i)}{\\sum_fI(fŒP_i(t))}$$
\n$$pr_i(t) = \\frac{\\sum_fI(fŒP_i(t)⋀fŒT_i)}{\\sum_fI(fŒt_i)}$$
\n$$AvgPr(t) = \\frac{1}{m(t)}\\sum\\limits_{i = 1}^{m(t)}pr_i(t)$$
\n$$AvgRc(t) = \\frac{1}{n}\\sum\\limits_{i = 1}^{n}rc_i(t)$$
\n$$F_{max} = max(\\frac{2AvgPr(t)AvgRc(t)}{AvgPr(t) + AvgRc(t)})$$
where
\nThe Second Metric is the AUC-ROC Curve of Sensitivity
\nAdditionally Global AucRoc Curve is also calculated
\n
| Paper Name | \nAuthors | \nPlace | \nYear | \n
|---|---|---|---|
| Tensorflow: A system for large-scale machine learning | \nAbadi,M. et al. | \nBerkeley, CA, USA | \n2016 | \n
| Molecular Biology of the Cell, 4th edn | \nAlberts,B. et al. | \nNew York, NY, USA | \n2002 | \n
| Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning | \nAlipanahi,B. et al. | \n- | \n2015 | \n
| Neuro-symbolic representation learning on biological knowledge graphs | \nAlshahrani,M. et al. | \n- | \n2017 | \n
| Gapped BLAST and PSI-BLAST: a new generation of protein database search programs | \nAltschul,S.F. et al. | \n- | \n1997 | \n
| Gene ontology: tool for the unification of biology | \nAshburner,M. et al. | \n- | \n2000 | \n
| Systematic functional annotation and visualization of biological networks | \nBaryshnikova,A. | \n- | \n2016 | \n
| A neural probabilistic language model | \nBengio,Y. et al. | \n- | \n2003 | \n
| UniProtKB/Swiss-Prot, the Manually Annotated Section of the UniProt KnowledgeBase: How to Use the Entry View | \nBoutet,E. et al. | \nNew York, NY | \n- | \n
| Information-theoretic evaluation of predicted ontological annotations | \nClark,W.T. and Radivojac,P. | \n- | \n2013 | \n
| Natural language processing (almost) from scratch | \nCollobert,R. et al. | \n- | \n2011 | \n
| A global genetic interaction network maps a wiring diagram of cellular function | \nCostanzo,M. et al. | \n- | \n2016 | \n
| Ffpred 3: feature-based function prediction for all gene ontology domains | \nCozzetto,D. et al. | \n- | \n2016 | \n
| Functional classification of cath superfamilies: a domain-based approach for protein function annotation | \nDas,S. et al. | \n- | \n2015 | \n
| Genome-scale phylogenetic function annotation of large and diverse protein families | \nEngelhardt,B.E. et al. | \n- | \n2011 | \n
| Interpro in 2017beyond protein family and domain annotations | \nFinn,R.D. et al. | \n- | \n2017 | \n
| Phylogenetic-based propagation of functional annotations within the gene ontology consortium | \nGaudet,P. et al. | \n- | \n2011 | \n
| The anatomy of phenotype ontologies: principles, properties and applications | \nGkoutos,G.V. et al. | \n- | \n2017 | \n
| Understanding the difficulty of training dee feedforward neural networks | \nGlorot,X. and Bengio,Y. | \n- | \n2010 | \n
| Gofdr: a sequence alignment based method for predicting protein functions | \nGong,Q. et al. | \n- | \n2016 | \n
| Learning distributed representations of concepts | \nHinton,G.E. | \nAmherst, MA | \n1986 | \n
| Protein function prediction from functional connectivity | \nHou,J. | \nSalt Lake City, UT, USA | \n2017 | \n
| eggnog 4.5: a hierarchical orthology framework with improved functional annotations for eukaryotic, prokaryotic and viral sequences | \nHuerta-Cepas,J. et al. | \n- | \n2016 | \n
| Predicting protein function by multi-label correlated semi-supervised learning | \nJiang,J.Q. and McQuay,L.J. | \n- | \n2012 | \n
| Protein Function Prediction Based on Patterns in Biological Networks | \nKirac,M. and Ozsoyoglu,G. | \n- | \n2008 | \n
| Convolutional networks for images, speech, and time series | \nLeCun,Y. and Bengio,Y. | \nCambridge, MA, USA | \n1998 | \n
| Critical assessment of methods of protein structure prediction (casp) round x | \nMoult,J. et al. | \n- | \n2014 | \n
| Protein annotation from protein interaction networks and gene ontology | \nNguyen,C.D. et al. | \n- | \n2011 | \n
| Annotating the human genome with disease ontology | \nOsborne,J. et al. | \n- | \n2009 | \n
| A large-scale evaluation of computational protein function prediction | \nRadivojac,P. et al. | \n- | \n2013 | \n
| Semantic similarity in a taxonomy: an Information-Based measure and its application to problems of ambiguity in natural language | \nResnik,P. | \n- | \n1999 | \n
| Network-based prediction of protein function | \nSharan,R. et al. | \n- | \n2007 | \n
| Hierarchical classification of gene ontology terms using the gostruct method | \nSokolov,A. and Ben-Hur,A. | \n- | \n2010 | \n
| String v10: proteinprotein interaction networks, integrated over the tree of life | \nSzklarczyk,D. et al. | \n- | \n2015 | \n
| Lecture 6.5—RmsProp: Divide the gradient by a running average of its recent magnitude | \nTieleman,T. and Hinton,G. | \n- | \n2012 | \n
| Roles for Text Mining in Protein Function Prediction | \nVerspoor,K.M. | \n- | \n2014 | \n
| Accurate de novo prediction of protein contact map by ultra-deep learning model | \nWang,S. et al. | \n- | \n2017 | \n
| Combfunc: predicting protein function using heterogeneous data sources | \nWass,M.N. et al. | \n- | \n2012 | \n
| Predicting effects of noncoding vari | \nZhou,J. and Troyanskaya,O.G. | \n- | \n2015 | \n
\n\nThe Embeddings are taken from [A Neural Probabilistic Language Model](https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf)\n\nAlong this `Protein-Protein Interaction - Neural Networks` `PPI-NNs` **[String V10](https://pubmed.ncbi.nlm.nih.gov/25352553/)** combined with **[EggNOG DataBase](http://eggnog5.embl.de/#/app/home)** are also used. \n\n# Pre-Processing 🔨\n****\nFor the **[Uni-Prot](https://www.uniprot.org/uniprotkb/?query=*)**, only `Proteins with experimental codes` with (`EXP`/`IDA`/`IPI`/`IGI`/`IEP`/`TAS`/`IC`) are `considered`. Protein with bigger than length `1002` and codes (`B`/`O`/`J`/`U`/`X`/`Z`) are ignored\n\nThe Sequences are first padded made to a length of `1,002` and then projected to a vector of `1,000` Dimensions\n****\nFor the **[String V10](https://pubmed.ncbi.nlm.nih.gov/25352553/)**, the data is first `filtered` by a `confidence score` of `300`, further concerted with `Orthology Relations` from **[EggNOG DataBase](http://eggnog5.embl.de/#/app/home)**. In total there are `84,78,935` `Proteins`/`1,90,649` `Edge Types`/ `11,58,66,95,610` `Edges`. With this data, a Knowledge Graph is created with embeddings of size `256`. Further `69,60,395` `Proteins` are mapped from `Uni-Prot`\n\n# Model 💻\n****\nA `Classical 1-D CNN` is used here. with the `Convolution Function` being \n\n$$h(y) = \\sum\\limits_{x = 1}^{k}f(x) * g(y * d - x + c)$$\nwhere \n* c = k - d + 1\n\nThis is the same formula for the output layer of a Convolution\n\nThe `First Layer` is intended to `learn embeddings` where each `index` is `mapped to a dense vector` by referring to a `lookup embedding table` with a `size` of `128`, representing a `Protein Sequence` of length of `1002` as a matrix of `1000 x 128`.\n\n|Layer|Filters|Filter Size|Stride|\n|---|---|---|---\n|Conv1D - 1|32|128||\n|Max-Pooling - 1||64|32\n\nIn the end, a vector of `832` `Dimensions`\n****\nThe PPI Sequences is connected with a Layer of `1,024` Neurons \n\n\n\n`Sigmoid` as the Default Activation Function is used, along with `Dropout Layers` to avoid Overfitting for both the models \n\n# Loss 💔\n`Multi-Output-Binary-Cross-Entropy` , a loss function that is used for `multi-label classification` problems is used \n\n$$loss = -\\sum(ylog(p) + (1 - y)log(1 - p))$$\n# Optimizer ⚙️\n`Root-Mean-Squared-Propagation` `Optimizer`, an `Adaptive Learning Rate` `Optimizer`, which `adjusts` the `learning rate` of the algorithm based on the `history of gradients` is used with `Batch size - 128`/`Lr - 0.01`\n\n$$w = β w_{-1} + (1 - β)g^2$$\n$$α = α_{-1}\\frac{1}{w_{-1}}$$\n\n# Training 🏋️♂️\nFirst the dataset is separated with `4:1` Ratio \n* **Training** - `80%`\n* **Validation** - `20%`\n\nTraining is done on `NVIDIA Pascals X GPU`, whcih took a little less than `3 Hours` (`BPO Model`)\n\nThe Training loop can be seen on [deepgo](https://github.com/bio-ontology-research-group/deepgo)\n\n# Comparison ⚖️\n* [CAFA 3 - FFPred3](http://bioinfadmin.cs.ucl.ac.uk/downloads/ffpred/cafa3/)\n* GoFDR\n\nThe test set for Comparison is from the `5 June 2017`, which contains `1367` `Proteins` `3619` `Annotations`\n\n\n\n\n\n# Evaluation 📄 \n\nThe Evaluation is made on the metrics of `F-Max Score`\n\n$$pr_i(t) = \\frac{\\sum_fI(fŒP_i(t)⋀fŒT_i)}{\\sum_fI(fŒP_i(t))}$$\n$$pr_i(t) = \\frac{\\sum_fI(fŒP_i(t)⋀fŒT_i)}{\\sum_fI(fŒt_i)}$$\n$$AvgPr(t) = \\frac{1}{m(t)}\\sum\\limits_{i = 1}^{m(t)}pr_i(t)$$\n$$AvgRc(t) = \\frac{1}{n}\\sum\\limits_{i = 1}^{n}rc_i(t)$$\n$$F_{max} = max(\\frac{2AvgPr(t)AvgRc(t)}{AvgPr(t) + AvgRc(t)})$$\n\nwhere \n* f - GO class\n* Pi(t) - predicted classes for\n* i - Protein\n* t - Threshold\n* Ti - annotated classes\n\nThe Second Metric is the AUC-ROC Curve of Sensitivity\n\nAdditionally Global AucRoc Curve is also calculated\n\n\n\n\n\n# Refrences 📚\n\n| Paper Name | Authors | Place | Year |\n| --- | --- | --- | --- |\n| Tensorflow: A system for large-scale machine learning | Abadi,M. et al. | Berkeley, CA, USA | 2016 |\n| Molecular Biology of the Cell, 4th edn | Alberts,B. et al. | New York, NY, USA | 2002 |\n| Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning | Alipanahi,B. et al. | - | 2015 |\n| Neuro-symbolic representation learning on biological knowledge graphs | Alshahrani,M. et al. | - | 2017 |\n| Gapped BLAST and PSI-BLAST: a new generation of protein database search programs | Altschul,S.F. et al. | - | 1997 |\n| Gene ontology: tool for the unification of biology | Ashburner,M. et al. | - | 2000 |\n| Systematic functional annotation and visualization of biological networks | Baryshnikova,A. | - | 2016 |\n| A neural probabilistic language model | Bengio,Y. et al. | - | 2003 |\n| UniProtKB/Swiss-Prot, the Manually Annotated Section of the UniProt KnowledgeBase: How to Use the Entry View | Boutet,E. et al. | New York, NY | - |\n| Information-theoretic evaluation of predicted ontological annotations | Clark,W.T. and Radivojac,P. | - | 2013 |\n| Natural language processing (almost) from scratch | Collobert,R. et al. | - | 2011 |\n| A global genetic interaction network maps a wiring diagram of cellular function | Costanzo,M. et al. | - | 2016 |\n| Ffpred 3: feature-based function prediction for all gene ontology domains | Cozzetto,D. et al. | - | 2016 |\n| Functional classification of cath superfamilies: a domain-based approach for protein function annotation | Das,S. et al. | - | 2015 |\n| Genome-scale phylogenetic function annotation of large and diverse protein families | Engelhardt,B.E. et al. | - | 2011 |\n| Interpro in 2017beyond protein family and domain annotations | Finn,R.D. et al. | - | 2017 |\n| Phylogenetic-based propagation of functional annotations within the gene ontology consortium | Gaudet,P. et al. | - | 2011 |\n| The anatomy of phenotype ontologies: principles, properties and applications | Gkoutos,G.V. et al. | - | 2017 |\n| Understanding the difficulty of training dee feedforward neural networks | Glorot,X. and Bengio,Y. | - | 2010 |\n| Gofdr: a sequence alignment based method for predicting protein functions | Gong,Q. et al. | - | 2016 |\n| Learning distributed representations of concepts | Hinton,G.E. | Amherst, MA | 1986 |\n| Protein function prediction from functional connectivity | Hou,J. | Salt Lake City, UT, USA | 2017 |\n| eggnog 4.5: a hierarchical orthology framework with improved functional annotations for eukaryotic, prokaryotic and viral sequences | Huerta-Cepas,J. et al. | - | 2016 |\n| Predicting protein function by multi-label correlated semi-supervised learning | Jiang,J.Q. and McQuay,L.J. | - | 2012 |\n| Protein Function Prediction Based on Patterns in Biological Networks | Kirac,M. and Ozsoyoglu,G. | - | 2008 |\n| Convolutional networks for images, speech, and time series | LeCun,Y. and Bengio,Y. | Cambridge, MA, USA | 1998 |\n| Critical assessment of methods of protein structure prediction (casp) round x | Moult,J. et al. | - | 2014 |\n| Protein annotation from protein interaction networks and gene ontology | Nguyen,C.D. et al. | - | 2011 |\n| Annotating the human genome with disease ontology | Osborne,J. et al. | - | 2009 |\n| A large-scale evaluation of computational protein function prediction | Radivojac,P. et al. | - | 2013 |\n| Semantic similarity in a taxonomy: an Information-Based measure and its application to problems of ambiguity in natural language | Resnik,P. | - | 1999 |\n| Network-based prediction of protein function | Sharan,R. et al. | - | 2007 |\n| Hierarchical classification of gene ontology terms using the gostruct method | Sokolov,A. and Ben-Hur,A. | - | 2010 |\n| String v10: proteinprotein interaction networks, integrated over the tree of life | Szklarczyk,D. et al. | - | 2015 |\n| Lecture 6.5—RmsProp: Divide the gradient by a running average of its recent magnitude | Tieleman,T. and Hinton,G. | - | 2012 |\n| Roles for Text Mining in Protein Function Prediction | Verspoor,K.M. | - | 2014 |\n| Accurate de novo prediction of protein contact map by ultra-deep learning model | Wang,S. et al. | - | 2017 |\n| Combfunc: predicting protein function using heterogeneous data sources | Wass,M.N. et al. | - | 2012 |\n| Predicting effects of noncoding vari | Zhou,J. and Troyanskaya,O.G. | - | 2015 |",
"votes": null
},
{
"id": "2416413",
"postDate": "08/31/2023 03:14:18",
"content": "Thank you for writing this summary.
\nHere's the citation and link to the paper as the link mentioned is broken.
\n\"DeepGO: predicting protein functions from sequence and interactions using a deep ontology-aware classifier\", Bioinformatics 2018
\nLink: [Abstract]
Thank you Kaushalya for the correct link :). I have updated the summary
", "rawMarkdown": "Thank you Kaushalya for the correct link :). I have updated the summary", "votes": null } ], "comments": [ { "id": 2416413, "author_name": "kaumad", "author_url": "", "post_date": "08/31/2023 03:14:18", "content": "Thank you for writing this summary.
\nHere's the citation and link to the paper as the link mentioned is broken.
\n\"DeepGO: predicting protein functions from sequence and interactions using a deep ontology-aware classifier\", Bioinformatics 2018
\nLink: [Abstract]
Thank you Kaushalya for the correct link :). I have updated the summary
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2336274": "**[Original Paper](https://academic.oup.com/bioinformatics/article/34/4/660/4265461?login=true)**\n\n# 1 | Pre-Requistics 📝\n\n* Sturcture of Amino Acids and their Sequences\n* GO Onotlogy\n* Protein-Protein Interaction\n* Convolution Neural Networks\n* Sigmoid Activation Function\n* Drop-Out Layers\n* AUC ROC Curve\n* Multi-Label Cross Entropy\n* Root Mean Squared Propagation\n\n# 2 | Starting 🌱\n\nRecent `advancements` in `Bio-Tech Industry` have led to the enormous `increase` in the amount of `Genetic`/`Protein` `Sequences`, which is expected to increase further too. These studies help in `further advancements` such as\n* **Phylogenetics** - Study of the `evolutionary relationships` between `organisms`\n* **Evolutionary Biology** - Subfield of biology that studies the `evolutionary processes` that produced the diversity of life on Earth\n* **Behavior of Biological Systems** - Study of `how living organisms interact with their environment`/`how they respond to stimuli`\n\nThough , `Annotations` of these `Protein Sequences` are `not` at the `same pace`, which is done with `vitro`/`vivo` experiments. These `prediction process` face many `complex challenges` such as \n* **Complex Relation** - `Sequence-Structure-Function` `SSF` adds `complex relations` among `protein sequence`/`structure`/`function`\n* **Multi Label Classification** - Several `Proteins` do `not` only have a `single function` but may be `Pleiotropic` and have `multiple different functions`\n* **Huge amount of Classes** - `GO` contains `strong`/`formally` defined `relations` between `functions` that need to be taken `into account` during `function prediction` to ensure that these\npredictions are consistent\n* **Non-Isolation Protein Function** - `Higher-level Physiological functions` that go beyond simple molecular interactions, such as apoptosis or regulation of heart rate, will `require other proteins` and `cannot usually be predicted` by `considering a single protein` in `isolation`. \n\n# Approach 👣\n\nThere are `2` kinds of major Representations chooses. \n* `Traditional Protein Sequences` `GO Ontology`\n* `Protein Protein Interaction` `PPI`\n\nFirstly these `2` forms of `representations` are `trained` on a `Neural Network` \n* **GO Ontology** - Learns the `features` from the `Protein Sequences`\n* **PPI** - Learns about the `interaction` among `different Proteins` based on their `location`\n \nFurther `these features` are used to in a `Novel Deep Neuro-Symbolic Model` that will `predict the classes` of the Proteins\n\n# Data 📊\n\n|BPO|MFO|CCO|Total\n|---|---|---|---|\n|`28,647`|`10,161`|`3,907`|`44,683` `Classes`/`1,968` `Obsolete`\n\n`Protein Sequences` are taken from **[Uni-Prot](https://www.uniprot.org/uniprotkb/?query=*)** containing `5,53,232` `Proteins`/`5,25,931` `Annotations`. \n\n||Amount|Total\n|---|---|---\n|BPO|19,181|\n|MFO|6,221|\n|CCO|2,358|\n|||60,710 Proteins\n|||27,760 Classes\n\n\n\nThe Embeddings are taken from [A Neural Probabilistic Language Model](https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf)\n\nAlong this `Protein-Protein Interaction - Neural Networks` `PPI-NNs` **[String V10](https://pubmed.ncbi.nlm.nih.gov/25352553/)** combined with **[EggNOG DataBase](http://eggnog5.embl.de/#/app/home)** are also used. \n\n# Pre-Processing 🔨\n****\nFor the **[Uni-Prot](https://www.uniprot.org/uniprotkb/?query=*)**, only `Proteins with experimental codes` with (`EXP`/`IDA`/`IPI`/`IGI`/`IEP`/`TAS`/`IC`) are `considered`. Protein with bigger than length `1002` and codes (`B`/`O`/`J`/`U`/`X`/`Z`) are ignored\n\nThe Sequences are first padded made to a length of `1,002` and then projected to a vector of `1,000` Dimensions\n****\nFor the **[String V10](https://pubmed.ncbi.nlm.nih.gov/25352553/)**, the data is first `filtered` by a `confidence score` of `300`, further concerted with `Orthology Relations` from **[EggNOG DataBase](http://eggnog5.embl.de/#/app/home)**. In total there are `84,78,935` `Proteins`/`1,90,649` `Edge Types`/ `11,58,66,95,610` `Edges`. With this data, a Knowledge Graph is created with embeddings of size `256`. Further `69,60,395` `Proteins` are mapped from `Uni-Prot`\n\n# Model 💻\n****\nA `Classical 1-D CNN` is used here. with the `Convolution Function` being \n\n$$h(y) = \\sum\\limits_{x = 1}^{k}f(x) * g(y * d - x + c)$$\nwhere \n* c = k - d + 1\n\nThis is the same formula for the output layer of a Convolution\n\nThe `First Layer` is intended to `learn embeddings` where each `index` is `mapped to a dense vector` by referring to a `lookup embedding table` with a `size` of `128`, representing a `Protein Sequence` of length of `1002` as a matrix of `1000 x 128`.\n\n|Layer|Filters|Filter Size|Stride|\n|---|---|---|---\n|Conv1D - 1|32|128||\n|Max-Pooling - 1||64|32\n\nIn the end, a vector of `832` `Dimensions`\n****\nThe PPI Sequences is connected with a Layer of `1,024` Neurons \n\n\n\n`Sigmoid` as the Default Activation Function is used, along with `Dropout Layers` to avoid Overfitting for both the models \n\n# Loss 💔\n`Multi-Output-Binary-Cross-Entropy` , a loss function that is used for `multi-label classification` problems is used \n\n$$loss = -\\sum(ylog(p) + (1 - y)log(1 - p))$$\n# Optimizer ⚙️\n`Root-Mean-Squared-Propagation` `Optimizer`, an `Adaptive Learning Rate` `Optimizer`, which `adjusts` the `learning rate` of the algorithm based on the `history of gradients` is used with `Batch size - 128`/`Lr - 0.01`\n\n$$w = β w_{-1} + (1 - β)g^2$$\n$$α = α_{-1}\\frac{1}{w_{-1}}$$\n\n# Training 🏋️♂️\nFirst the dataset is separated with `4:1` Ratio \n* **Training** - `80%`\n* **Validation** - `20%`\n\nTraining is done on `NVIDIA Pascals X GPU`, whcih took a little less than `3 Hours` (`BPO Model`)\n\nThe Training loop can be seen on [deepgo](https://github.com/bio-ontology-research-group/deepgo)\n\n# Comparison ⚖️\n* [CAFA 3 - FFPred3](http://bioinfadmin.cs.ucl.ac.uk/downloads/ffpred/cafa3/)\n* GoFDR\n\nThe test set for Comparison is from the `5 June 2017`, which contains `1367` `Proteins` `3619` `Annotations`\n\n\n\n\n\n# Evaluation 📄 \n\nThe Evaluation is made on the metrics of `F-Max Score`\n\n$$pr_i(t) = \\frac{\\sum_fI(fŒP_i(t)⋀fŒT_i)}{\\sum_fI(fŒP_i(t))}$$\n$$pr_i(t) = \\frac{\\sum_fI(fŒP_i(t)⋀fŒT_i)}{\\sum_fI(fŒt_i)}$$\n$$AvgPr(t) = \\frac{1}{m(t)}\\sum\\limits_{i = 1}^{m(t)}pr_i(t)$$\n$$AvgRc(t) = \\frac{1}{n}\\sum\\limits_{i = 1}^{n}rc_i(t)$$\n$$F_{max} = max(\\frac{2AvgPr(t)AvgRc(t)}{AvgPr(t) + AvgRc(t)})$$\n\nwhere \n* f - GO class\n* Pi(t) - predicted classes for\n* i - Protein\n* t - Threshold\n* Ti - annotated classes\n\nThe Second Metric is the AUC-ROC Curve of Sensitivity\n\nAdditionally Global AucRoc Curve is also calculated\n\n\n\n\n\n# Refrences 📚\n\n| Paper Name | Authors | Place | Year |\n| --- | --- | --- | --- |\n| Tensorflow: A system for large-scale machine learning | Abadi,M. et al. | Berkeley, CA, USA | 2016 |\n| Molecular Biology of the Cell, 4th edn | Alberts,B. et al. | New York, NY, USA | 2002 |\n| Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning | Alipanahi,B. et al. | - | 2015 |\n| Neuro-symbolic representation learning on biological knowledge graphs | Alshahrani,M. et al. | - | 2017 |\n| Gapped BLAST and PSI-BLAST: a new generation of protein database search programs | Altschul,S.F. et al. | - | 1997 |\n| Gene ontology: tool for the unification of biology | Ashburner,M. et al. | - | 2000 |\n| Systematic functional annotation and visualization of biological networks | Baryshnikova,A. | - | 2016 |\n| A neural probabilistic language model | Bengio,Y. et al. | - | 2003 |\n| UniProtKB/Swiss-Prot, the Manually Annotated Section of the UniProt KnowledgeBase: How to Use the Entry View | Boutet,E. et al. | New York, NY | - |\n| Information-theoretic evaluation of predicted ontological annotations | Clark,W.T. and Radivojac,P. | - | 2013 |\n| Natural language processing (almost) from scratch | Collobert,R. et al. | - | 2011 |\n| A global genetic interaction network maps a wiring diagram of cellular function | Costanzo,M. et al. | - | 2016 |\n| Ffpred 3: feature-based function prediction for all gene ontology domains | Cozzetto,D. et al. | - | 2016 |\n| Functional classification of cath superfamilies: a domain-based approach for protein function annotation | Das,S. et al. | - | 2015 |\n| Genome-scale phylogenetic function annotation of large and diverse protein families | Engelhardt,B.E. et al. | - | 2011 |\n| Interpro in 2017beyond protein family and domain annotations | Finn,R.D. et al. | - | 2017 |\n| Phylogenetic-based propagation of functional annotations within the gene ontology consortium | Gaudet,P. et al. | - | 2011 |\n| The anatomy of phenotype ontologies: principles, properties and applications | Gkoutos,G.V. et al. | - | 2017 |\n| Understanding the difficulty of training dee feedforward neural networks | Glorot,X. and Bengio,Y. | - | 2010 |\n| Gofdr: a sequence alignment based method for predicting protein functions | Gong,Q. et al. | - | 2016 |\n| Learning distributed representations of concepts | Hinton,G.E. | Amherst, MA | 1986 |\n| Protein function prediction from functional connectivity | Hou,J. | Salt Lake City, UT, USA | 2017 |\n| eggnog 4.5: a hierarchical orthology framework with improved functional annotations for eukaryotic, prokaryotic and viral sequences | Huerta-Cepas,J. et al. | - | 2016 |\n| Predicting protein function by multi-label correlated semi-supervised learning | Jiang,J.Q. and McQuay,L.J. | - | 2012 |\n| Protein Function Prediction Based on Patterns in Biological Networks | Kirac,M. and Ozsoyoglu,G. | - | 2008 |\n| Convolutional networks for images, speech, and time series | LeCun,Y. and Bengio,Y. | Cambridge, MA, USA | 1998 |\n| Critical assessment of methods of protein structure prediction (casp) round x | Moult,J. et al. | - | 2014 |\n| Protein annotation from protein interaction networks and gene ontology | Nguyen,C.D. et al. | - | 2011 |\n| Annotating the human genome with disease ontology | Osborne,J. et al. | - | 2009 |\n| A large-scale evaluation of computational protein function prediction | Radivojac,P. et al. | - | 2013 |\n| Semantic similarity in a taxonomy: an Information-Based measure and its application to problems of ambiguity in natural language | Resnik,P. | - | 1999 |\n| Network-based prediction of protein function | Sharan,R. et al. | - | 2007 |\n| Hierarchical classification of gene ontology terms using the gostruct method | Sokolov,A. and Ben-Hur,A. | - | 2010 |\n| String v10: proteinprotein interaction networks, integrated over the tree of life | Szklarczyk,D. et al. | - | 2015 |\n| Lecture 6.5—RmsProp: Divide the gradient by a running average of its recent magnitude | Tieleman,T. and Hinton,G. | - | 2012 |\n| Roles for Text Mining in Protein Function Prediction | Verspoor,K.M. | - | 2014 |\n| Accurate de novo prediction of protein contact map by ultra-deep learning model | Wang,S. et al. | - | 2017 |\n| Combfunc: predicting protein function using heterogeneous data sources | Wass,M.N. et al. | - | 2012 |\n| Predicting effects of noncoding vari | Zhou,J. and Troyanskaya,O.G. | - | 2015 |",
"2416413": "Thank you for writing this summary.\nHere's the citation and link to the paper as the link mentioned is broken.\n\"DeepGO: predicting protein functions from sequence and interactions using a deep ontology-aware classifier\", Bioinformatics 2018\nLink: [[Abstract](https://academic.oup.com/bioinformatics/article/34/4/660/4265461?login=true)]",
"2416414": "Thank you Kaushalya for the correct link :). I have updated the summary"
},
"source": "meta"
}