\nso probably only one category is more promising:
\nhttps://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1835-8/figures/3
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2262596%2Fba31c988fa32bdb2670b9686da8a420e%2Fphoto_2023-04-19_21-15-02.jpg?generation=1681931721260219&alt=media\")
Hi everyone! I'm looking for a teamup for this year's CAFA. I've been im CAFA 3-4 (and other Kaggles) together with my old lab (we did well :)). I'm a senior DS, (ongoing) PhD in computational biology, with an MsC in protein function feature engineering and we also won the 2020 WIDS.
\n(I've also been pushing for CAFA to be on Kaggle for years! :))
", "messages": [ { "id": 2226719, "postDate": "2023-04-19T06:56:43.633Z", "content": "Hi everyone! I'm looking for a teamup for this year's CAFA. I've been im CAFA 3-4 (and other Kaggles) together with my old lab (we did well :)). I'm a senior DS, (ongoing) PhD in computational biology, with an MsC in protein function feature engineering and we also won the 2020 WIDS.
\n(I've also been pushing for CAFA to be on Kaggle for years! :))
", "rawMarkdown": "Hi everyone! I'm looking for a teamup for this year's CAFA. I've been im CAFA 3-4 (and other Kaggles) together with my old lab (we did well :)). I'm a senior DS, (ongoing) PhD in computational biology, with an MsC in protein function feature engineering and we also won the 2020 WIDS. \n\n(I've also been pushing for CAFA to be on Kaggle for years! :))", "votes": 10 }, { "id": 2227550, "postDate": "2023-04-19T20:34:24.480Z", "content": "I'm interested in collaborating, I'm a Phd in computer science and my main research is related with distributed computing (https://scholar.google.es/citations?hl=es&user=vg1HqBMAAAAJ&view_op=list_works&sortby=pubdate), althoug it's been a while that I'm not publishing anything because I'm not in university anymore, I started kaggle two years ago and I have participated in many life science related competitions, because it is my passion, and among other things I have adapted alphafold for using by kagglers. So, If you're interested or other person, please contact me.
", "rawMarkdown": "I'm interested in collaborating, I'm a Phd in computer science and my main research is related with distributed computing (https://scholar.google.es/citations?hl=es&user=vg1HqBMAAAAJ&view_op=list_works&sortby=pubdate), althoug it's been a while that I'm not publishing anything because I'm not in university anymore, I started kaggle two years ago and I have participated in many life science related competitions, because it is my passion, and among other things I have adapted alphafold for using by kagglers. So, If you're interested or other person, please contact me.", "votes": 1 }, { "id": 2227462, "postDate": "2023-04-19T19:09:10.763Z", "content": "Hi there, glad to hear this! I'm also a first-year PhD student in computational biology, but more on dynamic modeling than ML. For master I was a DS student, for bachelor I majored in pharmacy so I've a very multi-disciplinary background. Thinking about integrating ML into my research, I found this competition tho it's the first time for me. We can team up if you're interested, it should be fun! : )
", "rawMarkdown": "Hi there, glad to hear this! I'm also a first-year PhD student in computational biology, but more on dynamic modeling than ML. For master I was a DS student, for bachelor I majored in pharmacy so I've a very multi-disciplinary background. Thinking about integrating ML into my research, I found this competition tho it's the first time for me. We can team up if you're interested, it should be fun! : )", "votes": 1 }, { "id": 2227376, "postDate": "2023-04-19T18:01:33.933Z", "content": "Hi ! Sounds great !
\nMay I ask a question - I cannot find something like leaderbords for previous CAFAs, are there anywhere ?
\nWhat was the shake-up ?
It seems neither the site: https://biofunctionprediction.org/cafa/ , neither the review: https://www.biorxiv.org/content/10.1101/653105v1
\nneither the site: https://www.synapse.org/#!Synapse:syn21035948/wiki/597156
\ncontains the info (((
In the preprint you linked has the results of CAFA3 (now published in Genome Biology) Figures 3-5 show the top 10 performing methods in the three different ontologies using different metrics We never had a leaderboard that combined scores for all three ontologies, since we wanted to evaluate methods separately based on the different aspects of protein function (the Molecular Function (MFO), Biological Process (BPO), and Cellular component (CCO),. For information on these ontologies see: http://geneontology.org/docs/ontology-documentation/
\nTo be clear: the ontology terms are the labels that you need to predict for the protein sequences you are provided with.
\nSince Kaggle require a single leaderboard, we decided this time to use the arithmetic mean of the MFO BPO and CCO scores to provide the final score for the leaderboard.
\nCAFA2 has similar figures showing the top 10 methods using different metrics in different categories:
\nhttps://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1037-6
CAFA1: https://www.nature.com/articles/nmeth.2340
", "rawMarkdown": "In the preprint you linked has the results of CAFA3 (now published in [Genome Biology](https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1835-8)) Figures 3-5 show the top 10 performing methods in the three different ontologies using different metrics We never had a leaderboard that combined scores for all three ontologies, since we wanted to evaluate methods separately based on the different aspects of protein function (the Molecular Function (MFO), Biological Process (BPO), and Cellular component (CCO),. For information on these ontologies see: [http://geneontology.org/docs/ontology-documentation/](http://geneontology.org/docs/ontology-documentation/)\n\nTo be clear: the ontology terms are the labels that you need to predict for the protein sequences you are provided with.\n\nSince Kaggle require a single leaderboard, we decided this time to use the arithmetic mean of the MFO BPO and CCO scores to provide the final score for the leaderboard.\n\nCAFA2 has similar figures showing the top 10 methods using different metrics in different categories: \nhttps://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1037-6\n\nCAFA1: https://www.nature.com/articles/nmeth.2340", "votes": 2, "replies": [ { "id": 2227430, "postDate": "2023-04-19T18:47:52.107Z", "content": "@iddux1 thanks for the links!
", "rawMarkdown": "@iddux1 thanks for the links!", "votes": 2 }, { "id": 2227469, "postDate": "2023-04-19T19:17:49.497Z", "content": "Many many thanks !
\nWhat about shake-up ? (Difference in rankings between preliminary leaderboard and the final one.) Was it big ?
\nare the preliminary and final results tables - can we see them ? )
PS
\nLet me put a screenshot here:
\nso probably only one category is more promising:
\nhttps://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1835-8/figures/3
\n
In previous CAFAs we never had a leaderboard during the competition. The results were first shown about 6 months after the submission deadline. Kaggle requires a leaderboard during the competition, so this will be a first. Should be interesting to see the discrepancies, if any, between the in-competition leaderboard and the final results.
", "rawMarkdown": "In previous CAFAs we never had a leaderboard **during** the competition. The results were first shown about 6 months after the submission deadline. Kaggle requires a leaderboard during the competition, so this will be a first. Should be interesting to see the discrepancies, if any, between the in-competition leaderboard and the final results. ", "votes": 2 }, { "id": 2227497, "postDate": "2023-04-19T19:46:04.350Z", "content": "Thank you !
\nPS
\nIf you would be so kind to suggest whether the cross-validation schemes have been discussed in the literature for that task ? (Probably we should be careful placing homologs in the same folds, but may be for that task is not that much important ? )
", "rawMarkdown": "Thank you ! \n\nPS \n\nIf you would be so kind to suggest whether the cross-validation schemes have been discussed in the literature for that task ? (Probably we should be careful placing homologs in the same folds, but may be for that task is not that much important ? ) \n\n", "votes": 1 }, { "id": 2227574, "postDate": "2023-04-19T20:55:19.293Z", "content": "Leakage of homologs is a problem. One common technique is to use a clustering algorithm such as CD-HIT to ensure that sequence identity remains under a certain percentage threshold. I've heard 40%-70%, YMMV.
", "rawMarkdown": "Leakage of homologs is a problem. One common technique is to use a clustering algorithm such as CD-HIT to ensure that sequence identity remains under a certain percentage threshold. I've heard 40%-70%, YMMV.", "votes": 2 } ] } ] }, { "id": 2227115, "postDate": "2023-04-19T14:11:38.500Z", "content": "Yes, I'm interested in collaborating. I have plenty of experience of (mostly structural) bioinformatics, including several years working in and later with the Thornton group. This includes previous experience of protein function prediction. However, I'm aware that scaling up to the size of this dataset is possibly an issue. If you're interested, my publications are here.
", "rawMarkdown": "Yes, I'm interested in collaborating. I have plenty of experience of (mostly structural) bioinformatics, including several years working in and later with the Thornton group. This includes previous experience of protein function prediction. However, I'm aware that scaling up to the size of this dataset is possibly an issue. If you're interested, my publications are [here](http://chemistry.st-andrews.ac.uk/staff/jbom/group/publications-john.html).", "votes": 2 }, { "id": 2226987, "postDate": "2023-04-19T12:49:00.773Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2227550, "author_name": "Victor Fernandez Albor", "author_url": "", "post_date": "2023-04-19T20:34:24.480000", "content": "I'm interested in collaborating, I'm a Phd in computer science and my main research is related with distributed computing (https://scholar.google.es/citations?hl=es&user=vg1HqBMAAAAJ&view_op=list_works&sortby=pubdate), althoug it's been a while that I'm not publishing anything because I'm not in university anymore, I started kaggle two years ago and I have participated in many life science related competitions, because it is my passion, and among other things I have adapted alphafold for using by kagglers. So, If you're interested or other person, please contact me.
", "votes": 1, "replies": [] }, { "id": 2227462, "author_name": "Soleil_lll", "author_url": "", "post_date": "2023-04-19T19:09:10.763000", "content": "Hi there, glad to hear this! I'm also a first-year PhD student in computational biology, but more on dynamic modeling than ML. For master I was a DS student, for bachelor I majored in pharmacy so I've a very multi-disciplinary background. Thinking about integrating ML into my research, I found this competition tho it's the first time for me. We can team up if you're interested, it should be fun! : )
", "votes": 1, "replies": [] }, { "id": 2227376, "author_name": "Alexander Chervov", "author_url": "", "post_date": "2023-04-19T18:01:33.933000", "content": "Hi ! Sounds great !
\nMay I ask a question - I cannot find something like leaderbords for previous CAFAs, are there anywhere ?
\nWhat was the shake-up ?
It seems neither the site: https://biofunctionprediction.org/cafa/ , neither the review: https://www.biorxiv.org/content/10.1101/653105v1
\nneither the site: https://www.synapse.org/#!Synapse:syn21035948/wiki/597156
\ncontains the info (((
In the preprint you linked has the results of CAFA3 (now published in Genome Biology) Figures 3-5 show the top 10 performing methods in the three different ontologies using different metrics We never had a leaderboard that combined scores for all three ontologies, since we wanted to evaluate methods separately based on the different aspects of protein function (the Molecular Function (MFO), Biological Process (BPO), and Cellular component (CCO),. For information on these ontologies see: http://geneontology.org/docs/ontology-documentation/
\nTo be clear: the ontology terms are the labels that you need to predict for the protein sequences you are provided with.
\nSince Kaggle require a single leaderboard, we decided this time to use the arithmetic mean of the MFO BPO and CCO scores to provide the final score for the leaderboard.
\nCAFA2 has similar figures showing the top 10 methods using different metrics in different categories:
\nhttps://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1037-6
CAFA1: https://www.nature.com/articles/nmeth.2340
", "votes": 2, "replies": [ { "id": 2227430, "author_name": "serangu", "author_url": "", "post_date": "2023-04-19T18:47:52.107000", "content": "@iddux1 thanks for the links!
", "votes": 2, "replies": [] }, { "id": 2227469, "author_name": "Alexander Chervov", "author_url": "", "post_date": "2023-04-19T19:17:49.497000", "content": "Many many thanks !
\nWhat about shake-up ? (Difference in rankings between preliminary leaderboard and the final one.) Was it big ?
\nare the preliminary and final results tables - can we see them ? )
PS
\nLet me put a screenshot here:
\nso probably only one category is more promising:
\nhttps://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1835-8/figures/3
\n
In previous CAFAs we never had a leaderboard during the competition. The results were first shown about 6 months after the submission deadline. Kaggle requires a leaderboard during the competition, so this will be a first. Should be interesting to see the discrepancies, if any, between the in-competition leaderboard and the final results.
", "votes": 2, "replies": [] }, { "id": 2227497, "author_name": "Alexander Chervov", "author_url": "", "post_date": "2023-04-19T19:46:04.350000", "content": "Thank you !
\nPS
\nIf you would be so kind to suggest whether the cross-validation schemes have been discussed in the literature for that task ? (Probably we should be careful placing homologs in the same folds, but may be for that task is not that much important ? )
", "votes": 1, "replies": [] }, { "id": 2227574, "author_name": "Iddo Friedberg", "author_url": "", "post_date": "2023-04-19T20:55:19.293000", "content": "Leakage of homologs is a problem. One common technique is to use a clustering algorithm such as CD-HIT to ensure that sequence identity remains under a certain percentage threshold. I've heard 40%-70%, YMMV.
", "votes": 2, "replies": [] } ] } ] }, { "id": 2227115, "author_name": "John Mitchell", "author_url": "", "post_date": "2023-04-19T14:11:38.500000", "content": "Yes, I'm interested in collaborating. I have plenty of experience of (mostly structural) bioinformatics, including several years working in and later with the Thornton group. This includes previous experience of protein function prediction. However, I'm aware that scaling up to the size of this dataset is possibly an issue. If you're interested, my publications are here.
", "votes": 2, "replies": [] }, { "id": 2226987, "author_name": "", "author_url": "", "post_date": "2023-04-19T12:49:00.773000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2226719": "Hi everyone! I'm looking for a teamup for this year's CAFA. I've been im CAFA 3-4 (and other Kaggles) together with my old lab (we did well :)). I'm a senior DS, (ongoing) PhD in computational biology, with an MsC in protein function feature engineering and we also won the 2020 WIDS. \n\n(I've also been pushing for CAFA to be on Kaggle for years! :))", "2227550": "I'm interested in collaborating, I'm a Phd in computer science and my main research is related with distributed computing (https://scholar.google.es/citations?hl=es&user=vg1HqBMAAAAJ&view_op=list_works&sortby=pubdate), althoug it's been a while that I'm not publishing anything because I'm not in university anymore, I started kaggle two years ago and I have participated in many life science related competitions, because it is my passion, and among other things I have adapted alphafold for using by kagglers. So, If you're interested or other person, please contact me.", "2227462": "Hi there, glad to hear this! I'm also a first-year PhD student in computational biology, but more on dynamic modeling than ML. For master I was a DS student, for bachelor I majored in pharmacy so I've a very multi-disciplinary background. Thinking about integrating ML into my research, I found this competition tho it's the first time for me. We can team up if you're interested, it should be fun! : )", "2227376": "Hi ! Sounds great ! \nMay I ask a question - I cannot find something like leaderbords for previous CAFAs, are there anywhere ?\nWhat was the shake-up ? \n\nIt seems neither the site: https://biofunctionprediction.org/cafa/ , neither the review: https://www.biorxiv.org/content/10.1101/653105v1\nneither the site: https://www.synapse.org/#!Synapse:syn21035948/wiki/597156\ncontains the info ((( ", "2227115": "Yes, I'm interested in collaborating. I have plenty of experience of (mostly structural) bioinformatics, including several years working in and later with the Thornton group. This includes previous experience of protein function prediction. However, I'm aware that scaling up to the size of this dataset is possibly an issue. If you're interested, my publications are [here](http://chemistry.st-andrews.ac.uk/staff/jbom/group/publications-john.html).", "2226987": "" } }