Hello to all!
\nI wanted to know if one of you had been interested in data augmentation techniques on embeddings (ProtBERT, T5 etc). As some classes of GO terms are very poorly represented, I thought of doing some kind of linear combinations of embeddings, in the same idea as MixUp: https://towardsdatascience.com/enhancing-neural-networks-with-mixup-in- pytorch-5129d261bc4a
Don’t know if it has sense to do it with embeddings. Wanted to have opinions.
\nGood luck to all!
Hello to all!
\nI wanted to know if one of you had been interested in data augmentation techniques on embeddings (ProtBERT, T5 etc). As some classes of GO terms are very poorly represented, I thought of doing some kind of linear combinations of embeddings, in the same idea as MixUp: https://towardsdatascience.com/enhancing-neural-networks-with-mixup-in- pytorch-5129d261bc4a
Don’t know if it has sense to do it with embeddings. Wanted to have opinions.
\nGood luck to all!
Surprisingly enough we discussed data augmentation today in my lab meeting, and how it can be applied to contrastive learning in bioinformatics and specifically for sequence and structure prediction. We are not experts, so everything following should be taken with a fistful of salt.
\nThe problem we identified is that, unlike image augmentation, any augmentation of, say, a sequence embedding representation may be non-functional. At the same time, there is some interesting literature, e.g.:
\nhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC9843584/
\nhttps://www.biorxiv.org/content/10.1101/2023.02.22.529597v2
Thank you very much for your feedback! indeed I have the same intuition concerning the raised aspects. the framework of the images seems to me very conducive to this kind of protocol. That said, I'm not sure that taking the barycenter of two embeddings can allow us to assert that the resulting embedding represents the GO terms of the two underlying embeddings. I'm going to dig the literature you sent me, if you have ideas for implementing certain things I'm in!
", "rawMarkdown": "Thank you very much for your feedback! indeed I have the same intuition concerning the raised aspects. the framework of the images seems to me very conducive to this kind of protocol. That said, I'm not sure that taking the barycenter of two embeddings can allow us to assert that the resulting embedding represents the GO terms of the two underlying embeddings. I'm going to dig the literature you sent me, if you have ideas for implementing certain things I'm in!", "votes": null }, { "id": "2259315", "postDate": "05/14/2023 21:08:56", "content": "I dug a bit into the idea of data augmentation for our project. In particular, I came across the BLAST utility (Basic Local Alignment Search Tool) which allows, from what I understood, to align protein sequences. As a result, I thought that this could be used as part of our data augmentation:
\nIf we consider that very similar additional sequences obtained from BLAST present same labels than reference query sequence (i.e same GO terms), we would basically be able to dispose of more data for training, and more precisely increase representation of some GO terms classes in order to increase their respective precision/accuracy when predicting.
\nWhat are your opinions about this hypothetical procedure ?
", "rawMarkdown": "I dug a bit into the idea of data augmentation for our project. In particular, I came across the BLAST utility (Basic Local Alignment Search Tool) which allows, from what I understood, to align protein sequences. As a result, I thought that this could be used as part of our data augmentation:\n- identify protein sequences that present quite rare GO terms (very sparse classes) : this would be our query sequences for BLAST => group (1)\n- proceed BLAST queries that will allow for each protein from group (1) to have very similar sequences of existing proteins => collect their respective sequences (can be done easily, possibility to directly download result sequences as fasta files)\n- use our favorite embedding pre-trained model (ProtBERT, T5, ESM2) to transform these new sequences into embedding vectors (must be same embedding space than one used for original training sequences)\n\nIf we consider that very similar additional sequences obtained from BLAST present same labels than reference query sequence (i.e same GO terms), we would basically be able to dispose of more data for training, and more precisely increase representation of some GO terms classes in order to increase their respective precision/accuracy when predicting. \n\nWhat are your opinions about this hypothetical procedure ?", "votes": null }, { "id": "2259320", "postDate": "05/14/2023 21:25:32", "content": "As organizer, I won't comment directly on the feasibility / usability of the idea, due to obvious conflict. I can give a couple of technical tips:
\nBLAST searches sequence databases using a query sequence, and finds similar sequences very fast (look to DIAMOND as well for something faster, though slightly less accurate). So the sequences you find will be influenced by which database you search. That is something to consider. DIAMOND: https://uni-tuebingen.de/en/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/algorithms-in-bioinformatics/software/diamond/
\nAt the risk of stating the obvious, you can install BLAST / DIAMOND and the databases to search in as standalone, instead of using the web version. Would be much faster.
\nTo parse BLAST results, Biopython is your friend.
\nGood luck.
", "rawMarkdown": "As organizer, I won't comment directly on the feasibility / usability of the idea, due to obvious conflict. I can give a couple of technical tips:\n\nBLAST searches sequence databases using a query sequence, and finds similar sequences very fast (look to DIAMOND as well for something faster, though slightly less accurate). So the sequences you find will be influenced by which database you search. That is something to consider. DIAMOND: https://uni-tuebingen.de/en/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/algorithms-in-bioinformatics/software/diamond/\n\nAt the risk of stating the obvious, you can install BLAST / DIAMOND and the databases to search in as standalone, instead of using the web version. Would be much faster.\n\nTo parse BLAST results, Biopython is your friend.\n\n\nGood luck.", "votes": null }, { "id": "2259333", "postDate": "05/14/2023 21:46:32", "content": "Thank you very much @iddux1 for your answer ! Indeed I have just seen that my best friend Biopython allows to launch requests locally. I will try to implement my procedure! I'll come back to you if I have any technical questions. 🫶🏻
", "rawMarkdown": "Thank you very much @iddux1 for your answer ! Indeed I have just seen that my best friend Biopython allows to launch requests locally. I will try to implement my procedure! I'll come back to you if I have any technical questions. 🫶🏻", "votes": null } ], "comments": [ { "id": 2256809, "author_name": "iddux1", "author_url": "", "post_date": "05/12/2023 18:26:13", "content": "Surprisingly enough we discussed data augmentation today in my lab meeting, and how it can be applied to contrastive learning in bioinformatics and specifically for sequence and structure prediction. We are not experts, so everything following should be taken with a fistful of salt.
\nThe problem we identified is that, unlike image augmentation, any augmentation of, say, a sequence embedding representation may be non-functional. At the same time, there is some interesting literature, e.g.:
\nhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC9843584/
\nhttps://www.biorxiv.org/content/10.1101/2023.02.22.529597v2
Thank you very much for your feedback! indeed I have the same intuition concerning the raised aspects. the framework of the images seems to me very conducive to this kind of protocol. That said, I'm not sure that taking the barycenter of two embeddings can allow us to assert that the resulting embedding represents the GO terms of the two underlying embeddings. I'm going to dig the literature you sent me, if you have ideas for implementing certain things I'm in!
", "votes": null, "replies": [] }, { "id": 2259315, "author_name": "henriupton", "author_url": "", "post_date": "05/14/2023 21:08:56", "content": "I dug a bit into the idea of data augmentation for our project. In particular, I came across the BLAST utility (Basic Local Alignment Search Tool) which allows, from what I understood, to align protein sequences. As a result, I thought that this could be used as part of our data augmentation:
\nIf we consider that very similar additional sequences obtained from BLAST present same labels than reference query sequence (i.e same GO terms), we would basically be able to dispose of more data for training, and more precisely increase representation of some GO terms classes in order to increase their respective precision/accuracy when predicting.
\nWhat are your opinions about this hypothetical procedure ?
", "votes": null, "replies": [ { "id": 2259320, "author_name": "iddux1", "author_url": "", "post_date": "05/14/2023 21:25:32", "content": "As organizer, I won't comment directly on the feasibility / usability of the idea, due to obvious conflict. I can give a couple of technical tips:
\nBLAST searches sequence databases using a query sequence, and finds similar sequences very fast (look to DIAMOND as well for something faster, though slightly less accurate). So the sequences you find will be influenced by which database you search. That is something to consider. DIAMOND: https://uni-tuebingen.de/en/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/algorithms-in-bioinformatics/software/diamond/
\nAt the risk of stating the obvious, you can install BLAST / DIAMOND and the databases to search in as standalone, instead of using the web version. Would be much faster.
\nTo parse BLAST results, Biopython is your friend.
\nGood luck.
", "votes": null, "replies": [ { "id": 2259333, "author_name": "henriupton", "author_url": "", "post_date": "05/14/2023 21:46:32", "content": "Thank you very much @iddux1 for your answer ! Indeed I have just seen that my best friend Biopython allows to launch requests locally. I will try to implement my procedure! I'll come back to you if I have any technical questions. 🫶🏻
", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "2256648": "Hello to all! \nI wanted to know if one of you had been interested in data augmentation techniques on embeddings (ProtBERT, T5 etc). As some classes of GO terms are very poorly represented, I thought of doing some kind of linear combinations of embeddings, in the same idea as MixUp: https://towardsdatascience.com/enhancing-neural-networks-with-mixup-in- pytorch-5129d261bc4a\n\nDon’t know if it has sense to do it with embeddings. Wanted to have opinions.\nGood luck to all!", "2256809": "Surprisingly enough we discussed data augmentation today in my lab meeting, and how it can be applied to contrastive learning in bioinformatics and specifically for sequence and structure prediction. We are not experts, so everything following should be taken with a fistful of salt. \n\nThe problem we identified is that, unlike image augmentation, any augmentation of, say, a sequence embedding representation may be non-functional. At the same time, there is some interesting literature, e.g.: \nhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC9843584/\nhttps://www.biorxiv.org/content/10.1101/2023.02.22.529597v2", "2257036": "Thank you very much for your feedback! indeed I have the same intuition concerning the raised aspects. the framework of the images seems to me very conducive to this kind of protocol. That said, I'm not sure that taking the barycenter of two embeddings can allow us to assert that the resulting embedding represents the GO terms of the two underlying embeddings. I'm going to dig the literature you sent me, if you have ideas for implementing certain things I'm in!", "2259315": "I dug a bit into the idea of data augmentation for our project. In particular, I came across the BLAST utility (Basic Local Alignment Search Tool) which allows, from what I understood, to align protein sequences. As a result, I thought that this could be used as part of our data augmentation:\n- identify protein sequences that present quite rare GO terms (very sparse classes) : this would be our query sequences for BLAST => group (1)\n- proceed BLAST queries that will allow for each protein from group (1) to have very similar sequences of existing proteins => collect their respective sequences (can be done easily, possibility to directly download result sequences as fasta files)\n- use our favorite embedding pre-trained model (ProtBERT, T5, ESM2) to transform these new sequences into embedding vectors (must be same embedding space than one used for original training sequences)\n\nIf we consider that very similar additional sequences obtained from BLAST present same labels than reference query sequence (i.e same GO terms), we would basically be able to dispose of more data for training, and more precisely increase representation of some GO terms classes in order to increase their respective precision/accuracy when predicting. \n\nWhat are your opinions about this hypothetical procedure ?", "2259320": "As organizer, I won't comment directly on the feasibility / usability of the idea, due to obvious conflict. I can give a couple of technical tips:\n\nBLAST searches sequence databases using a query sequence, and finds similar sequences very fast (look to DIAMOND as well for something faster, though slightly less accurate). So the sequences you find will be influenced by which database you search. That is something to consider. DIAMOND: https://uni-tuebingen.de/en/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/algorithms-in-bioinformatics/software/diamond/\n\nAt the risk of stating the obvious, you can install BLAST / DIAMOND and the databases to search in as standalone, instead of using the web version. Would be much faster.\n\nTo parse BLAST results, Biopython is your friend.\n\n\nGood luck.", "2259333": "Thank you very much @iddux1 for your answer ! Indeed I have just seen that my best friend Biopython allows to launch requests locally. I will try to implement my procedure! I'll come back to you if I have any technical questions. 🫶🏻" }, "source": "meta" }