{ "id": 329848, "title": "What if you don't have high-end GPU?", "url": "/competitions/AI4Code/discussion/329848", "author_name": "", "post_date": "2022-06-09T04:32:54.678466100Z", "votes": 19, "comment_count": 19, "views": 0, "content": "

I'm using colab pro for $10/month, I though it enough for this competition but I was wrong.
\nUsing gpu P100 cost 14hours training distilBert on 20% of data for one epochs from @suicaokhoailang 's idea.
\nI'm thinking of hiring online GPU, do you know any online renting gpu flatform which prices is reasonable?
\nMoney is a problem for students like me but knowledge has a price.

", "messages": [ { "id": "1815473", "postDate": "06/09/2022 04:32:54", "content": "

I'm using colab pro for $10/month, I though it enough for this competition but I was wrong.
\nUsing gpu P100 cost 14hours training distilBert on 20% of data for one epochs from @suicaokhoailang 's idea.
\nI'm thinking of hiring online GPU, do you know any online renting gpu flatform which prices is reasonable?
\nMoney is a problem for students like me but knowledge has a price.

", "rawMarkdown": "I'm using colab pro for $10/month, I though it enough for this competition but I was wrong.\nUsing gpu P100 cost 14hours training distilBert on 20% of data for one epochs from @suicaokhoailang 's idea.\nI'm thinking of hiring online GPU, do you know any online renting gpu flatform which prices is reasonable?\nMoney is a problem for students like me but knowledge has a price.", "votes": null }, { "id": "1815497", "postDate": "06/09/2022 05:13:51", "content": "

The cheapest platform I have seen is Q-block, where you can rent 2x RTX3090 with 1.2$/hour (compared with Jarvislab, 1.7$ for an A6000, ). However, I am using Colab pro+, and I think it is the cheapest way. Sometimes I got some annoying issues, but it is reasonable with a price 50$/month. I think if you have a limited budget, you can subscribe for several months for the competitions and then unsubscribe.

\n

Update: I just checked and we can rent an A6000 on Lambda for 1$/hour, making it the cheapest platform I've seen instead of Q-block (in terms of the GPUs RAM)

", "rawMarkdown": "The cheapest platform I have seen is Q-block, where you can rent 2x RTX3090 with 1.2$/hour (compared with Jarvislab, 1.7$ for an A6000, ~~or Lambda, I think 1.4$ or something~~). However, I am using Colab pro+, and I think it is the cheapest way. Sometimes I got some annoying issues, but it is reasonable with a price 50$/month. I think if you have a limited budget, you can subscribe for several months for the competitions and then unsubscribe.\n\nUpdate: I just checked and we can rent an A6000 on Lambda for 1$/hour, making it the cheapest platform I've seen instead of Q-block (in terms of the GPUs RAM)", "votes": null }, { "id": "1815846", "postDate": "06/09/2022 14:36:15", "content": "

Interesting, the only time I got a V100 in CollabPro+ was on the first day. After that it has all been P100 (same as Kaggle).

", "rawMarkdown": "Interesting, the only time I got a V100 in CollabPro+ was on the first day. After that it has all been P100 (same as Kaggle).", "votes": null }, { "id": "1815851", "postDate": "06/09/2022 14:43:29", "content": "

Weird… I usually got V100s and sometimes A100s. At the moment, I am training with 2 A100s…

", "rawMarkdown": "Weird... I usually got V100s and sometimes A100s. At the moment, I am training with 2 A100s...", "votes": null }, { "id": "1815857", "postDate": "06/09/2022 14:51:48", "content": "

whoa there!! A100s - I can only dream about it. V100, I got it as a Welcome drink the first time I shelled out 50 bucks. It has been a month since then and all I get is a P100.

", "rawMarkdown": "whoa there!! A100s - I can only dream about it. V100, I got it as a Welcome drink the first time I shelled out 50 bucks. It has been a month since then and all I get is a P100.", "votes": null }, { "id": "1815863", "postDate": "06/09/2022 14:59:53", "content": "

You may want to check the notebook settings, don't use the high-ram mode if you don't really need it. Sometimes I got even T4s with the high-ram setting.

", "rawMarkdown": "You may want to check the notebook settings, don't use the high-ram mode if you don't really need it. Sometimes I got even T4s with the high-ram setting.", "votes": null }, { "id": "1815881", "postDate": "06/09/2022 15:33:17", "content": "

Thanks for the tip! I didnt pay much attention to that. Will try and see

", "rawMarkdown": "Thanks for the tip! I didnt pay much attention to that. Will try and see", "votes": null }, { "id": "1816628", "postDate": "06/10/2022 11:46:22", "content": "

Look at using a TPU for training: https://www.kaggle.com/code/nickuzmenkov/ai4code-tensorflow-distilbert-baseline - works on Colab and Kaggle.

", "rawMarkdown": "Look at using a TPU for training: https://www.kaggle.com/code/nickuzmenkov/ai4code-tensorflow-distilbert-baseline - works on Colab and Kaggle.", "votes": null }, { "id": "1817217", "postDate": "06/11/2022 04:29:54", "content": "

l use chinese platform , where you can rent RTX3090 just 0.3$/hour. https://www.autodl.com/

", "rawMarkdown": "l use chinese platform , where you can rent RTX3090 just 0.3$/hour. [https://www.autodl.com/](url)", "votes": null }, { "id": "1817924", "postDate": "06/12/2022 03:29:27", "content": "

Split the datasets to 10 folds, after training each fold, save the checkpoint.

\n

Pay attention to resumtion, you should load everything back while resume from the last checkpoint, a good frame work should have done this for you, but if you write your code from scratch, considering:

\n", "rawMarkdown": "Split the datasets to 10 folds, after training each fold, save the checkpoint.\n\nPay attention to resumtion, you should load everything back while resume from the last checkpoint, a good frame work should have done this for you, but if you write your code from scratch, considering:\n\n* model (of course...)\n* optimizer\n* scheduler\n* scaler ( if amp is enabled )\n* other things that have state", "votes": null }, { "id": "1820889", "postDate": "06/15/2022 04:48:59", "content": "

From my position of lack of resources, it seems the most reasonable way. Thank you for your advice.👍👍

", "rawMarkdown": "From my position of lack of resources, it seems the most reasonable way. Thank you for your advice.👍👍", "votes": null }, { "id": "1824538", "postDate": "06/18/2022 12:30:17", "content": "

You can try a spot instance with RTX 5000 for $0.19 in Jarvislabs.ai which offers similar performance to V100.

", "rawMarkdown": "You can try a spot instance with RTX 5000 for $0.19 in Jarvislabs.ai which offers similar performance to V100.", "votes": null }, { "id": "1850694", "postDate": "07/10/2022 16:15:17", "content": "

I saw this tweet by Mark Tenenholtz:

\n
\n

The most unique \"trick\" that I can contribute is the use of @Tim_Dettmers' 8-bit Adam optimizer.

\n

The only GPU I have easy access to is an RTX 2070, which ain't much for this stuff.

\n

This optimizer saves a ton of memory at no accuracy hit (none that I could see, at least).

\n
\n

I haven't tried it yet, but it seems like a promising way to train big models with relatively small amounts of VRAM.

", "rawMarkdown": "I saw this tweet by [Mark Tenenholtz](https://twitter.com/marktenenholtz/status/1539579014804688896):\n\n> The most unique \"trick\" that I can contribute is the use of @Tim_Dettmers' 8-bit Adam optimizer.\n>\n> The only GPU I have easy access to is an RTX 2070, which ain't much for this stuff.\n>\n> This optimizer saves a ton of memory at no accuracy hit (none that I could see, at least).\n\nI haven't tried it yet, but it seems like a promising way to train big models with relatively small amounts of VRAM.", "votes": null }, { "id": "1852834", "postDate": "07/12/2022 12:08:23", "content": "

I personally think that some of the top players might not use deep learning methods as their submission times are very short (~1h).

", "rawMarkdown": "I personally think that some of the top players might not use deep learning methods as their submission times are very short (~1h).", "votes": null }, { "id": "1853057", "postDate": "07/12/2022 15:00:39", "content": "

Yeah, still don't know how to solve this challenge without deep learning. There is one team name 'simple gbdt' btw.

", "rawMarkdown": "Yeah, still don't know how to solve this challenge without deep learning. There is one team name 'simple gbdt' btw.", "votes": null }, { "id": "1853739", "postDate": "07/13/2022 05:28:41", "content": "

did you use deep learning method?

", "rawMarkdown": "did you use deep learning method?", "votes": null }, { "id": "1853831", "postDate": "07/13/2022 07:25:17", "content": "

For me yes.

", "rawMarkdown": "For me yes.", "votes": null }, { "id": "1853861", "postDate": "07/13/2022 08:10:54", "content": "

I have seen this also by other people on other competitions but never asked: How you even figure out this '1 hour'? Do you scrape the leaderboard or maybe track it 'manually' :-) ???🤔

", "rawMarkdown": "I have seen this also by other people on other competitions but never asked: How you even figure out this '1 hour'? Do you scrape the leaderboard or maybe track it 'manually' :-) ???🤔", "votes": null }, { "id": "1853873", "postDate": "07/13/2022 08:35:00", "content": "

For me yes manually 👀
\nWe have many chances to fortunately see it as people submit many times. Of course it should be not hard to build the scraping tool to track it, but manual is enough for me.👀

\n

NOTE: we should mind that the submission time could be distorted, like people could just do LB probing that usually doesn't take much time, or even embed codes like \"time.sleep(3*60*60)\". As well, of course different methods have different running times.
\nGazing the submission time is usually useless if we are already confident enough that we are in the right direction to tackle the problem (now I'm a bit worrying about I'm not there.😨)

", "rawMarkdown": "For me yes manually 👀\nWe have many chances to fortunately see it as people submit many times. Of course it should be not hard to build the scraping tool to track it, but manual is enough for me.👀\n\nNOTE: we should mind that the submission time could be distorted, like people could just do LB probing that usually doesn't take much time, or even embed codes like \"time.sleep(3\\*60\\*60)\". As well, of course different methods have different running times.\nGazing the submission time is usually useless if we are already confident enough that we are in the right direction to tackle the problem (now I'm a bit worrying about I'm not there.😨)", "votes": null }, { "id": "1853898", "postDate": "07/13/2022 09:24:43", "content": "

👍__👍

", "rawMarkdown": "👍______👍", "votes": null } ], "comments": [ { "id": 1815497, "author_name": "shinomoriaoshi", "author_url": "", "post_date": "06/09/2022 05:13:51", "content": "

The cheapest platform I have seen is Q-block, where you can rent 2x RTX3090 with 1.2$/hour (compared with Jarvislab, 1.7$ for an A6000, ). However, I am using Colab pro+, and I think it is the cheapest way. Sometimes I got some annoying issues, but it is reasonable with a price 50$/month. I think if you have a limited budget, you can subscribe for several months for the competitions and then unsubscribe.

\n

Update: I just checked and we can rent an A6000 on Lambda for 1$/hour, making it the cheapest platform I've seen instead of Q-block (in terms of the GPUs RAM)

", "votes": null, "replies": [ { "id": 1815846, "author_name": "allohvk", "author_url": "", "post_date": "06/09/2022 14:36:15", "content": "

Interesting, the only time I got a V100 in CollabPro+ was on the first day. After that it has all been P100 (same as Kaggle).

", "votes": null, "replies": [] }, { "id": 1815851, "author_name": "shinomoriaoshi", "author_url": "", "post_date": "06/09/2022 14:43:29", "content": "

Weird… I usually got V100s and sometimes A100s. At the moment, I am training with 2 A100s…

", "votes": null, "replies": [] }, { "id": 1815857, "author_name": "allohvk", "author_url": "", "post_date": "06/09/2022 14:51:48", "content": "

whoa there!! A100s - I can only dream about it. V100, I got it as a Welcome drink the first time I shelled out 50 bucks. It has been a month since then and all I get is a P100.

", "votes": null, "replies": [] }, { "id": 1815863, "author_name": "shinomoriaoshi", "author_url": "", "post_date": "06/09/2022 14:59:53", "content": "

You may want to check the notebook settings, don't use the high-ram mode if you don't really need it. Sometimes I got even T4s with the high-ram setting.

", "votes": null, "replies": [] }, { "id": 1815881, "author_name": "allohvk", "author_url": "", "post_date": "06/09/2022 15:33:17", "content": "

Thanks for the tip! I didnt pay much attention to that. Will try and see

", "votes": null, "replies": [] } ] }, { "id": 1816628, "author_name": "happycube", "author_url": "", "post_date": "06/10/2022 11:46:22", "content": "

Look at using a TPU for training: https://www.kaggle.com/code/nickuzmenkov/ai4code-tensorflow-distilbert-baseline - works on Colab and Kaggle.

", "votes": null, "replies": [] }, { "id": 1817217, "author_name": "richardzhangy26", "author_url": "", "post_date": "06/11/2022 04:29:54", "content": "

l use chinese platform , where you can rent RTX3090 just 0.3$/hour. https://www.autodl.com/

", "votes": null, "replies": [] }, { "id": 1817924, "author_name": "snaker", "author_url": "", "post_date": "06/12/2022 03:29:27", "content": "

Split the datasets to 10 folds, after training each fold, save the checkpoint.

\n

Pay attention to resumtion, you should load everything back while resume from the last checkpoint, a good frame work should have done this for you, but if you write your code from scratch, considering:

\n", "votes": null, "replies": [ { "id": 1820889, "author_name": "heuingttiang", "author_url": "", "post_date": "06/15/2022 04:48:59", "content": "

From my position of lack of resources, it seems the most reasonable way. Thank you for your advice.👍👍

", "votes": null, "replies": [] } ] }, { "id": 1824538, "author_name": "vishnus", "author_url": "", "post_date": "06/18/2022 12:30:17", "content": "

You can try a spot instance with RTX 5000 for $0.19 in Jarvislabs.ai which offers similar performance to V100.

", "votes": null, "replies": [] }, { "id": 1850694, "author_name": "ceshine", "author_url": "", "post_date": "07/10/2022 16:15:17", "content": "

I saw this tweet by Mark Tenenholtz:

\n
\n

The most unique \"trick\" that I can contribute is the use of @Tim_Dettmers' 8-bit Adam optimizer.

\n

The only GPU I have easy access to is an RTX 2070, which ain't much for this stuff.

\n

This optimizer saves a ton of memory at no accuracy hit (none that I could see, at least).

\n
\n

I haven't tried it yet, but it seems like a promising way to train big models with relatively small amounts of VRAM.

", "votes": null, "replies": [] }, { "id": 1852834, "author_name": "yufuin", "author_url": "", "post_date": "07/12/2022 12:08:23", "content": "

I personally think that some of the top players might not use deep learning methods as their submission times are very short (~1h).

", "votes": null, "replies": [ { "id": 1853057, "author_name": "locbaop", "author_url": "", "post_date": "07/12/2022 15:00:39", "content": "

Yeah, still don't know how to solve this challenge without deep learning. There is one team name 'simple gbdt' btw.

", "votes": null, "replies": [] }, { "id": 1853739, "author_name": "yuanzhezhou", "author_url": "", "post_date": "07/13/2022 05:28:41", "content": "

did you use deep learning method?

", "votes": null, "replies": [] }, { "id": 1853831, "author_name": "yufuin", "author_url": "", "post_date": "07/13/2022 07:25:17", "content": "

For me yes.

", "votes": null, "replies": [] }, { "id": 1853861, "author_name": "abdulkadirguner", "author_url": "", "post_date": "07/13/2022 08:10:54", "content": "

I have seen this also by other people on other competitions but never asked: How you even figure out this '1 hour'? Do you scrape the leaderboard or maybe track it 'manually' :-) ???🤔

", "votes": null, "replies": [] }, { "id": 1853873, "author_name": "yufuin", "author_url": "", "post_date": "07/13/2022 08:35:00", "content": "

For me yes manually 👀
\nWe have many chances to fortunately see it as people submit many times. Of course it should be not hard to build the scraping tool to track it, but manual is enough for me.👀

\n

NOTE: we should mind that the submission time could be distorted, like people could just do LB probing that usually doesn't take much time, or even embed codes like \"time.sleep(3*60*60)\". As well, of course different methods have different running times.
\nGazing the submission time is usually useless if we are already confident enough that we are in the right direction to tackle the problem (now I'm a bit worrying about I'm not there.😨)

", "votes": null, "replies": [] }, { "id": 1853898, "author_name": "abdulkadirguner", "author_url": "", "post_date": "07/13/2022 09:24:43", "content": "

👍__👍

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1815473": "I'm using colab pro for $10/month, I though it enough for this competition but I was wrong.\nUsing gpu P100 cost 14hours training distilBert on 20% of data for one epochs from @suicaokhoailang 's idea.\nI'm thinking of hiring online GPU, do you know any online renting gpu flatform which prices is reasonable?\nMoney is a problem for students like me but knowledge has a price.", "1815497": "The cheapest platform I have seen is Q-block, where you can rent 2x RTX3090 with 1.2$/hour (compared with Jarvislab, 1.7$ for an A6000, ~~or Lambda, I think 1.4$ or something~~). However, I am using Colab pro+, and I think it is the cheapest way. Sometimes I got some annoying issues, but it is reasonable with a price 50$/month. I think if you have a limited budget, you can subscribe for several months for the competitions and then unsubscribe.\n\nUpdate: I just checked and we can rent an A6000 on Lambda for 1$/hour, making it the cheapest platform I've seen instead of Q-block (in terms of the GPUs RAM)", "1815846": "Interesting, the only time I got a V100 in CollabPro+ was on the first day. After that it has all been P100 (same as Kaggle).", "1815851": "Weird... I usually got V100s and sometimes A100s. At the moment, I am training with 2 A100s...", "1815857": "whoa there!! A100s - I can only dream about it. V100, I got it as a Welcome drink the first time I shelled out 50 bucks. It has been a month since then and all I get is a P100.", "1815863": "You may want to check the notebook settings, don't use the high-ram mode if you don't really need it. Sometimes I got even T4s with the high-ram setting.", "1815881": "Thanks for the tip! I didnt pay much attention to that. Will try and see", "1816628": "Look at using a TPU for training: https://www.kaggle.com/code/nickuzmenkov/ai4code-tensorflow-distilbert-baseline - works on Colab and Kaggle.", "1817217": "l use chinese platform , where you can rent RTX3090 just 0.3$/hour. [https://www.autodl.com/](url)", "1817924": "Split the datasets to 10 folds, after training each fold, save the checkpoint.\n\nPay attention to resumtion, you should load everything back while resume from the last checkpoint, a good frame work should have done this for you, but if you write your code from scratch, considering:\n\n* model (of course...)\n* optimizer\n* scheduler\n* scaler ( if amp is enabled )\n* other things that have state", "1820889": "From my position of lack of resources, it seems the most reasonable way. Thank you for your advice.👍👍", "1824538": "You can try a spot instance with RTX 5000 for $0.19 in Jarvislabs.ai which offers similar performance to V100.", "1850694": "I saw this tweet by [Mark Tenenholtz](https://twitter.com/marktenenholtz/status/1539579014804688896):\n\n> The most unique \"trick\" that I can contribute is the use of @Tim_Dettmers' 8-bit Adam optimizer.\n>\n> The only GPU I have easy access to is an RTX 2070, which ain't much for this stuff.\n>\n> This optimizer saves a ton of memory at no accuracy hit (none that I could see, at least).\n\nI haven't tried it yet, but it seems like a promising way to train big models with relatively small amounts of VRAM.", "1852834": "I personally think that some of the top players might not use deep learning methods as their submission times are very short (~1h).", "1853057": "Yeah, still don't know how to solve this challenge without deep learning. There is one team name 'simple gbdt' btw.", "1853739": "did you use deep learning method?", "1853831": "For me yes.", "1853861": "I have seen this also by other people on other competitions but never asked: How you even figure out this '1 hour'? Do you scrape the leaderboard or maybe track it 'manually' :-) ???🤔", "1853873": "For me yes manually 👀\nWe have many chances to fortunately see it as people submit many times. Of course it should be not hard to build the scraping tool to track it, but manual is enough for me.👀\n\nNOTE: we should mind that the submission time could be distorted, like people could just do LB probing that usually doesn't take much time, or even embed codes like \"time.sleep(3\\*60\\*60)\". As well, of course different methods have different running times.\nGazing the submission time is usually useless if we are already confident enough that we are in the right direction to tackle the problem (now I'm a bit worrying about I'm not there.😨)", "1853898": "👍______👍" }, "source": "meta" }