I am using Deep learning NN to train CAFA5 model and non of the GPUs are working to predict the test data. Here is my notebook link https://www.kaggle.com/code/averma111/cafa5lightning/.
\nHow to effectively use GpUs?
", "messages": [ { "id": "2277835", "postDate": "05/28/2023 05:19:33", "content": "I am using Deep learning NN to train CAFA5 model and non of the GPUs are working to predict the test data. Here is my notebook link https://www.kaggle.com/code/averma111/cafa5lightning/.
\nHow to effectively use GpUs?
", "rawMarkdown": "I am using Deep learning NN to train CAFA5 model and non of the GPUs are working to predict the test data. Here is my notebook link https://www.kaggle.com/code/averma111/cafa5lightning/.\n\nHow to effectively use GpUs?", "votes": null }, { "id": "2278816", "postDate": "05/29/2023 01:46:41", "content": "You could train on GPU and download your models, then use a new notebook with CPU for inference.
", "rawMarkdown": "You could train on GPU and download your models, then use a new notebook with CPU for inference.", "votes": null }, { "id": "2279707", "postDate": "05/29/2023 14:45:15", "content": "Hi ! Just in case, I had some issues when testing with the size of the submission file. Indeed if you predict for a lot of protein ids and a lot of GO term classes, you can produce at the end a submission dataframe with a huge number of raws and saturate the memory. So it’s perhaps doesn’t come from GPU usage
", "rawMarkdown": "Hi ! Just in case, I had some issues when testing with the size of the submission file. Indeed if you predict for a lot of protein ids and a lot of GO term classes, you can produce at the end a submission dataframe with a huge number of raws and saturate the memory. So it’s perhaps doesn’t come from GPU usage", "votes": null }, { "id": "2280156", "postDate": "05/29/2023 23:09:21", "content": "For each protein, it makes sense to predict a maximum 1500 of GO terms(according to rules: https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/overview/evaluation). Is it still too big number of terms to fit into the RAM submission file?
", "rawMarkdown": "For each protein, it makes sense to predict a maximum 1500 of GO terms(according to rules: https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/overview/evaluation). Is it still too big number of terms to fit into the RAM submission file?", "votes": null }, { "id": "2280177", "postDate": "05/30/2023 00:01:39", "content": "Exactly this
", "rawMarkdown": "Exactly this", "votes": null }, { "id": "2280500", "postDate": "05/30/2023 06:20:54", "content": "Exactly no need to consider more than 1500. I tried this number and it’s too high by my side. 1000 works so perhaps I will try an in between like 1250. What about you?
", "rawMarkdown": "Exactly no need to consider more than 1500. I tried this number and it’s too high by my side. 1000 works so perhaps I will try an in between like 1250. What about you?", "votes": null }, { "id": "2282086", "postDate": "05/31/2023 11:04:50", "content": "Hi @henriupton! So far the maximum number of GO terms I predicted was N = 600 and I didn't face the out-of-memory errors. Now I predict the most common N terms, and, surprisingly for me, the simple model with top N = 500 received a higher leaderboard score than the same model with N = 600. So I didn't increase the number of terms to predict.
\nI understand that this is very basic approach (predict most common terms only, ignore the rest) and some kind of sparsity should be utilized to predict the most suitable GO term candidates for each individual protein.
\nComing back to memory issues I was not able to use ESM-2 embeddings (https://huggingface.co/docs/transformers/model_doc/esm) because of RAM, but I didn't figure out the issue and so far use T5 embeddings. I guess that HuggingFace provides some tricks for more memory-efficient computation, but I haven't figured it out yet.
\nBest,
\nAnton.
I am still wondering if they really consider 1500 terms as maximum. In the github that contains the code that computes the competition score for each submission, the script has an argument « max_terms » fixed at 500. They describe this argument as « the maximum number of GO terms considered for each target ». So it doesn’t match with the 1500 mentioned. If so, just consider 500 terms is enough and I think in this case it is the 500 most frequent GO terms.
\nI think considering more GO terms (600 in your case) has pros and cons: obviously it ignores less GO terms and really tries to predict a probability which is not necessarily 0 (as the ignoring case) and it tries to identify terms that are difficult to identify (that’s one of the main goal of the competition). But it’s a real trade off because it makes the classification work harder for the model.
\nFor EMS2 you can check my notebook if you want to see how to use them.
\nCheers
", "rawMarkdown": "I am still wondering if they really consider 1500 terms as maximum. In the github that contains the code that computes the competition score for each submission, the script has an argument « max_terms » fixed at 500. They describe this argument as « the maximum number of GO terms considered for each target ». So it doesn’t match with the 1500 mentioned. If so, just consider 500 terms is enough and I think in this case it is the 500 most frequent GO terms. \n\nI think considering more GO terms (600 in your case) has pros and cons: obviously it ignores less GO terms and really tries to predict a probability which is not necessarily 0 (as the ignoring case) and it tries to identify terms that are difficult to identify (that’s one of the main goal of the competition). But it’s a real trade off because it makes the classification work harder for the model. \n\nFor EMS2 you can check my notebook if you want to see how to use them.\n\nCheers", "votes": null }, { "id": "2282621", "postDate": "05/31/2023 17:39:02", "content": "\n\nI am still wondering if they really consider 1500 terms as maximum. In the github that contains the code that computes the competition score for each submission, the script has an argument « max_terms » fixed at 500. They describe this argument as « the maximum number of GO terms considered for each target ». So it doesn’t match with the 1500 mentioned. If so, just consider 500 terms is enough and I think in this case it is the 500 most frequent GO terms.
\n
From the rules:
\n\n\none target cannot be associated with more than 1500 terms for MF, BP, and CC subontologies combined.
\n
500 x 3 = 1,500
", "rawMarkdown": "> I am still wondering if they really consider 1500 terms as maximum. In the github that contains the code that computes the competition score for each submission, the script has an argument « max_terms » fixed at 500. They describe this argument as « the maximum number of GO terms considered for each target ». So it doesn’t match with the 1500 mentioned. If so, just consider 500 terms is enough and I think in this case it is the 500 most frequent GO terms. \n\nFrom the rules:\n\n> one target cannot be associated with more than 1500 terms for MF, BP, and CC subontologies combined.\n\n500 x 3 = 1,500", "votes": null }, { "id": "2283542", "postDate": "06/01/2023 10:35:13", "content": "Hi @iddux1, thank you for the comment! Now it became clear that ontology type (MF, BP, CC) should not be ignored.
", "rawMarkdown": "Hi @iddux1, thank you for the comment! Now it became clear that ontology type (MF, BP, CC) should not be ignored.", "votes": null }, { "id": "2283606", "postDate": "06/01/2023 11:39:08", "content": "Thank you ! Indeed it would be interesting to predict by grouping by ontology type, and finally pack all predictions
", "rawMarkdown": "Thank you ! Indeed it would be interesting to predict by grouping by ontology type, and finally pack all predictions", "votes": null }, { "id": "2283765", "postDate": "06/01/2023 13:39:05", "content": "True, but in test data there is no info about ontology type …
", "rawMarkdown": "True, but in test data there is no info about ontology type ...", "votes": null }, { "id": "2283768", "postDate": "06/01/2023 13:43:51", "content": "\n\nTrue, but in test data there is no info about ontology type …
\n
You can find that information in the obo file.
", "rawMarkdown": ">True, but in test data there is no info about ontology type …\n\nYou can find that information in the obo file.", "votes": null }, { "id": "2284027", "postDate": "06/01/2023 17:18:48", "content": "it's working fine on CPU also. Remember, whenever start clear the RAM.
", "rawMarkdown": "it's working fine on CPU also. Remember, whenever start clear the RAM.", "votes": null }, { "id": "2284219", "postDate": "06/01/2023 20:07:08", "content": "obo file is in train so it contains ontology for train not for test. Maybe I dont get it but how is it possible to use ontology in training when there is no ontology in test for predicting?
", "rawMarkdown": "obo file is in train so it contains ontology for train not for test. Maybe I dont get it but how is it possible to use ontology in training when there is no ontology in test for predicting?", "votes": null } ], "comments": [ { "id": 2278816, "author_name": "takanashihumbert", "author_url": "", "post_date": "05/29/2023 01:46:41", "content": "You could train on GPU and download your models, then use a new notebook with CPU for inference.
", "votes": null, "replies": [ { "id": 2280177, "author_name": "zmcxjt", "author_url": "", "post_date": "05/30/2023 00:01:39", "content": "Exactly this
", "votes": null, "replies": [] } ] }, { "id": 2279707, "author_name": "henriupton", "author_url": "", "post_date": "05/29/2023 14:45:15", "content": "Hi ! Just in case, I had some issues when testing with the size of the submission file. Indeed if you predict for a lot of protein ids and a lot of GO term classes, you can produce at the end a submission dataframe with a huge number of raws and saturate the memory. So it’s perhaps doesn’t come from GPU usage
", "votes": null, "replies": [ { "id": 2280156, "author_name": "calmscout", "author_url": "", "post_date": "05/29/2023 23:09:21", "content": "For each protein, it makes sense to predict a maximum 1500 of GO terms(according to rules: https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/overview/evaluation). Is it still too big number of terms to fit into the RAM submission file?
", "votes": null, "replies": [ { "id": 2280500, "author_name": "henriupton", "author_url": "", "post_date": "05/30/2023 06:20:54", "content": "Exactly no need to consider more than 1500. I tried this number and it’s too high by my side. 1000 works so perhaps I will try an in between like 1250. What about you?
", "votes": null, "replies": [ { "id": 2282086, "author_name": "calmscout", "author_url": "", "post_date": "05/31/2023 11:04:50", "content": "Hi @henriupton! So far the maximum number of GO terms I predicted was N = 600 and I didn't face the out-of-memory errors. Now I predict the most common N terms, and, surprisingly for me, the simple model with top N = 500 received a higher leaderboard score than the same model with N = 600. So I didn't increase the number of terms to predict.
\nI understand that this is very basic approach (predict most common terms only, ignore the rest) and some kind of sparsity should be utilized to predict the most suitable GO term candidates for each individual protein.
\nComing back to memory issues I was not able to use ESM-2 embeddings (https://huggingface.co/docs/transformers/model_doc/esm) because of RAM, but I didn't figure out the issue and so far use T5 embeddings. I guess that HuggingFace provides some tricks for more memory-efficient computation, but I haven't figured it out yet.
\nBest,
\nAnton.
I am still wondering if they really consider 1500 terms as maximum. In the github that contains the code that computes the competition score for each submission, the script has an argument « max_terms » fixed at 500. They describe this argument as « the maximum number of GO terms considered for each target ». So it doesn’t match with the 1500 mentioned. If so, just consider 500 terms is enough and I think in this case it is the 500 most frequent GO terms.
\nI think considering more GO terms (600 in your case) has pros and cons: obviously it ignores less GO terms and really tries to predict a probability which is not necessarily 0 (as the ignoring case) and it tries to identify terms that are difficult to identify (that’s one of the main goal of the competition). But it’s a real trade off because it makes the classification work harder for the model.
\nFor EMS2 you can check my notebook if you want to see how to use them.
\nCheers
", "votes": null, "replies": [ { "id": 2282621, "author_name": "iddux1", "author_url": "", "post_date": "05/31/2023 17:39:02", "content": "\n\nI am still wondering if they really consider 1500 terms as maximum. In the github that contains the code that computes the competition score for each submission, the script has an argument « max_terms » fixed at 500. They describe this argument as « the maximum number of GO terms considered for each target ». So it doesn’t match with the 1500 mentioned. If so, just consider 500 terms is enough and I think in this case it is the 500 most frequent GO terms.
\n
From the rules:
\n\n\none target cannot be associated with more than 1500 terms for MF, BP, and CC subontologies combined.
\n
500 x 3 = 1,500
", "votes": null, "replies": [ { "id": 2283542, "author_name": "calmscout", "author_url": "", "post_date": "06/01/2023 10:35:13", "content": "Hi @iddux1, thank you for the comment! Now it became clear that ontology type (MF, BP, CC) should not be ignored.
", "votes": null, "replies": [] }, { "id": 2283606, "author_name": "henriupton", "author_url": "", "post_date": "06/01/2023 11:39:08", "content": "Thank you ! Indeed it would be interesting to predict by grouping by ontology type, and finally pack all predictions
", "votes": null, "replies": [ { "id": 2283765, "author_name": "michalinahulak", "author_url": "", "post_date": "06/01/2023 13:39:05", "content": "True, but in test data there is no info about ontology type …
", "votes": null, "replies": [ { "id": 2283768, "author_name": "iddux1", "author_url": "", "post_date": "06/01/2023 13:43:51", "content": "\n\nTrue, but in test data there is no info about ontology type …
\n
You can find that information in the obo file.
", "votes": null, "replies": [ { "id": 2284219, "author_name": "michalinahulak", "author_url": "", "post_date": "06/01/2023 20:07:08", "content": "obo file is in train so it contains ontology for train not for test. Maybe I dont get it but how is it possible to use ontology in training when there is no ontology in test for predicting?
", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] } ] } ] } ] }, { "id": 2284027, "author_name": "dplg007", "author_url": "", "post_date": "06/01/2023 17:18:48", "content": "it's working fine on CPU also. Remember, whenever start clear the RAM.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2277835": "I am using Deep learning NN to train CAFA5 model and non of the GPUs are working to predict the test data. Here is my notebook link https://www.kaggle.com/code/averma111/cafa5lightning/.\n\nHow to effectively use GpUs?", "2278816": "You could train on GPU and download your models, then use a new notebook with CPU for inference.", "2279707": "Hi ! Just in case, I had some issues when testing with the size of the submission file. Indeed if you predict for a lot of protein ids and a lot of GO term classes, you can produce at the end a submission dataframe with a huge number of raws and saturate the memory. So it’s perhaps doesn’t come from GPU usage", "2280156": "For each protein, it makes sense to predict a maximum 1500 of GO terms(according to rules: https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/overview/evaluation). Is it still too big number of terms to fit into the RAM submission file?", "2280177": "Exactly this", "2280500": "Exactly no need to consider more than 1500. I tried this number and it’s too high by my side. 1000 works so perhaps I will try an in between like 1250. What about you?", "2282086": "Hi @henriupton! So far the maximum number of GO terms I predicted was N = `600` and I didn't face the out-of-memory errors. Now I predict the most common N terms, and, surprisingly for me, the simple model with top N = `500` received a higher leaderboard score than the same model with N = 600. So I didn't increase the number of terms to predict.\nI understand that this is very basic approach (predict most common terms only, ignore the rest) and some kind of sparsity should be utilized to predict the most suitable GO term candidates for each individual protein.\nComing back to memory issues I was not able to use ESM-2 embeddings (https://huggingface.co/docs/transformers/model_doc/esm) because of RAM, but I didn't figure out the issue and so far use T5 embeddings. I guess that HuggingFace provides some tricks for more memory-efficient computation, but I haven't figured it out yet.\nBest,\nAnton.", "2282605": "I am still wondering if they really consider 1500 terms as maximum. In the github that contains the code that computes the competition score for each submission, the script has an argument « max_terms » fixed at 500. They describe this argument as « the maximum number of GO terms considered for each target ». So it doesn’t match with the 1500 mentioned. If so, just consider 500 terms is enough and I think in this case it is the 500 most frequent GO terms. \n\nI think considering more GO terms (600 in your case) has pros and cons: obviously it ignores less GO terms and really tries to predict a probability which is not necessarily 0 (as the ignoring case) and it tries to identify terms that are difficult to identify (that’s one of the main goal of the competition). But it’s a real trade off because it makes the classification work harder for the model. \n\nFor EMS2 you can check my notebook if you want to see how to use them.\n\nCheers", "2282621": "> I am still wondering if they really consider 1500 terms as maximum. In the github that contains the code that computes the competition score for each submission, the script has an argument « max_terms » fixed at 500. They describe this argument as « the maximum number of GO terms considered for each target ». So it doesn’t match with the 1500 mentioned. If so, just consider 500 terms is enough and I think in this case it is the 500 most frequent GO terms. \n\nFrom the rules:\n\n> one target cannot be associated with more than 1500 terms for MF, BP, and CC subontologies combined.\n\n500 x 3 = 1,500", "2283542": "Hi @iddux1, thank you for the comment! Now it became clear that ontology type (MF, BP, CC) should not be ignored.", "2283606": "Thank you ! Indeed it would be interesting to predict by grouping by ontology type, and finally pack all predictions", "2283765": "True, but in test data there is no info about ontology type ...", "2283768": ">True, but in test data there is no info about ontology type …\n\nYou can find that information in the obo file.", "2284027": "it's working fine on CPU also. Remember, whenever start clear the RAM.", "2284219": "obo file is in train so it contains ontology for train not for test. Maybe I dont get it but how is it possible to use ontology in training when there is no ontology in test for predicting?" }, "source": "meta" }