Hi, as I had already mentioned in a previous comment, I am currently on a laptop which cannot handle training large datasets. As such, I am looking for cheaper (read: free) alternatives in cloud services. I would appreciate if some people could share the resources they generally use to train their models and the steps on setting them up. Thank you.
", "messages": [ { "id": "555901", "postDate": "06/19/2019 15:29:30", "content": "Hi, as I had already mentioned in a previous comment, I am currently on a laptop which cannot handle training large datasets. As such, I am looking for cheaper (read: free) alternatives in cloud services. I would appreciate if some people could share the resources they generally use to train their models and the steps on setting them up. Thank you.
", "rawMarkdown": "Hi, as I had already mentioned in a previous comment, I am currently on a laptop which cannot handle training large datasets. As such, I am looking for cheaper (read: free) alternatives in cloud services. I would appreciate if some people could share the resources they generally use to train their models and the steps on setting them up. Thank you.", "votes": null }, { "id": "555971", "postDate": "06/19/2019 16:52:22", "content": "You can use https://colab.research.google.com/
\n\nSee https://www.kaggle.com/getting-started/47096\nfor pointers.
\n\nI've heard that the following sites offer free access to GPU enabled cloud but never tried it:
\n\n\n\nColmetML https://www.comet.ml/
", "rawMarkdown": "You can use https://colab.research.google.com/\n\nSee https://www.kaggle.com/getting-started/47096\nfor pointers.\n\nI've heard that the following sites offer free access to GPU enabled cloud but never tried it:\n\nhttps://www.floydhub.com/\n\nColmetML https://www.comet.ml/", "votes": null }, { "id": "555985", "postDate": "06/19/2019 17:12:37", "content": "Why not Kaggle? You even get a free gpu (tesla P100 currently)
", "rawMarkdown": "Why not Kaggle? You even get a free gpu (tesla P100 currently)", "votes": null }, { "id": "556002", "postDate": "06/19/2019 17:39:00", "content": "I think Kaggle kernel's specs are not sufficient to reach the top ranks. Feel free to correct me though
", "rawMarkdown": "I think Kaggle kernel's specs are not sufficient to reach the top ranks. Feel free to correct me though", "votes": null }, { "id": "556014", "postDate": "06/19/2019 18:15:30", "content": "Have not tried it on Kaggle since I spent lots of dollars for my PC but I think you can use Kaggle but you will need to do things in pieces - and keep disk memory under the limits with frequent trimming of features that are not of importance. It would be a real pain.
\n\nSince I am using lightgbm for my models the free GPU is of no value as I think the current Kaggle library does not include GPU support.
", "rawMarkdown": "Have not tried it on Kaggle since I spent lots of dollars for my PC but I think you can use Kaggle but you will need to do things in pieces - and keep disk memory under the limits with frequent trimming of features that are not of importance. It would be a real pain.\n\nSince I am using lightgbm for my models the free GPU is of no value as I think the current Kaggle library does not include GPU support.", "votes": null }, { "id": "556031", "postDate": "06/19/2019 18:42:12", "content": "I play this competition only on Kaggle. 2 kernels, the first one creates the features, and the second one trains. it's not so bad :)
", "rawMarkdown": "I play this competition only on Kaggle. 2 kernels, the first one creates the features, and the second one trains. it's not so bad :)", "votes": null }, { "id": "556278", "postDate": "06/20/2019 04:18:51", "content": "Ah, so like creating features -> exporting to dataset -> import dataset in another kernel.\nDefinitely will try this, thank you
", "rawMarkdown": "Ah, so like creating features -> exporting to dataset -> import dataset in another kernel.\nDefinitely will try this, thank you", "votes": null }, { "id": "556403", "postDate": "06/20/2019 07:50:07", "content": "You can easily recompile lightgbm for gpu and you can also run many kernels at the same time. However finding some good features seems much more efficient than hours of compute!
", "rawMarkdown": "You can easily recompile lightgbm for gpu and you can also run many kernels at the same time. However finding some good features seems much more efficient than hours of compute!", "votes": null }, { "id": "556825", "postDate": "06/20/2019 18:04:13", "content": "try floydhub
", "rawMarkdown": "try floydhub", "votes": null }, { "id": "556832", "postDate": "06/20/2019 18:18:29", "content": "Try to use a kernel as data source. Create the features in one kernel, save them into csv, and use other kernel for modeling. Check this link: https://www.kaggle.com/product-feedback/45472
", "rawMarkdown": "Try to use a kernel as data source. Create the features in one kernel, save them into csv, and use other kernel for modeling. Check this link: https://www.kaggle.com/product-feedback/45472", "votes": null }, { "id": "556834", "postDate": "06/20/2019 18:20:16", "content": "As you're save dataset into csv, you can combine python and R in the same completion. R has some very useful tools for feature engineering.
", "rawMarkdown": "As you're save dataset into csv, you can combine python and R in the same completion. R has some very useful tools for feature engineering.", "votes": null }, { "id": "556872", "postDate": "06/20/2019 19:21:49", "content": "Seconding google colab
", "rawMarkdown": "Seconding google colab", "votes": null }, { "id": "557201", "postDate": "06/21/2019 02:06:29", "content": "Just signed up for floydhub.com Looks like I have 2 hours of GPU time and 20 hours of CPU time before free runs out. CPU looks like it's 0.48 dollars per hour and GPU is 4.50. Does not look to be much more capable than Kaggle so going to pass on even trying to use it.
", "rawMarkdown": "Just signed up for floydhub.com Looks like I have 2 hours of GPU time and 20 hours of CPU time before free runs out. CPU looks like it's 0.48 dollars per hour and GPU is 4.50. Does not look to be much more capable than Kaggle so going to pass on even trying to use it.", "votes": null } ], "comments": [ { "id": 555971, "author_name": "alexpcli", "author_url": "", "post_date": "06/19/2019 16:52:22", "content": "You can use https://colab.research.google.com/
\n\nSee https://www.kaggle.com/getting-started/47096\nfor pointers.
\n\nI've heard that the following sites offer free access to GPU enabled cloud but never tried it:
\n\n\n\nColmetML https://www.comet.ml/
", "votes": null, "replies": [ { "id": 556872, "author_name": "rsrgrimm", "author_url": "", "post_date": "06/20/2019 19:21:49", "content": "Seconding google colab
", "votes": null, "replies": [] }, { "id": 557201, "author_name": "pcjimmmy", "author_url": "", "post_date": "06/21/2019 02:06:29", "content": "Just signed up for floydhub.com Looks like I have 2 hours of GPU time and 20 hours of CPU time before free runs out. CPU looks like it's 0.48 dollars per hour and GPU is 4.50. Does not look to be much more capable than Kaggle so going to pass on even trying to use it.
", "votes": null, "replies": [] } ] }, { "id": 555985, "author_name": "jmtest", "author_url": "", "post_date": "06/19/2019 17:12:37", "content": "Why not Kaggle? You even get a free gpu (tesla P100 currently)
", "votes": null, "replies": [ { "id": 556002, "author_name": "tanmaypandey", "author_url": "", "post_date": "06/19/2019 17:39:00", "content": "I think Kaggle kernel's specs are not sufficient to reach the top ranks. Feel free to correct me though
", "votes": null, "replies": [] }, { "id": 556014, "author_name": "pcjimmmy", "author_url": "", "post_date": "06/19/2019 18:15:30", "content": "Have not tried it on Kaggle since I spent lots of dollars for my PC but I think you can use Kaggle but you will need to do things in pieces - and keep disk memory under the limits with frequent trimming of features that are not of importance. It would be a real pain.
\n\nSince I am using lightgbm for my models the free GPU is of no value as I think the current Kaggle library does not include GPU support.
", "votes": null, "replies": [] }, { "id": 556031, "author_name": "jmtest", "author_url": "", "post_date": "06/19/2019 18:42:12", "content": "I play this competition only on Kaggle. 2 kernels, the first one creates the features, and the second one trains. it's not so bad :)
", "votes": null, "replies": [] }, { "id": 556278, "author_name": "tanmaypandey", "author_url": "", "post_date": "06/20/2019 04:18:51", "content": "Ah, so like creating features -> exporting to dataset -> import dataset in another kernel.\nDefinitely will try this, thank you
", "votes": null, "replies": [] }, { "id": 556403, "author_name": "jmtest", "author_url": "", "post_date": "06/20/2019 07:50:07", "content": "You can easily recompile lightgbm for gpu and you can also run many kernels at the same time. However finding some good features seems much more efficient than hours of compute!
", "votes": null, "replies": [] } ] }, { "id": 556825, "author_name": "venkat555", "author_url": "", "post_date": "06/20/2019 18:04:13", "content": "try floydhub
", "votes": null, "replies": [] }, { "id": 556832, "author_name": "adrianoavelar", "author_url": "", "post_date": "06/20/2019 18:18:29", "content": "Try to use a kernel as data source. Create the features in one kernel, save them into csv, and use other kernel for modeling. Check this link: https://www.kaggle.com/product-feedback/45472
", "votes": null, "replies": [ { "id": 556834, "author_name": "adrianoavelar", "author_url": "", "post_date": "06/20/2019 18:20:16", "content": "As you're save dataset into csv, you can combine python and R in the same completion. R has some very useful tools for feature engineering.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "555901": "Hi, as I had already mentioned in a previous comment, I am currently on a laptop which cannot handle training large datasets. As such, I am looking for cheaper (read: free) alternatives in cloud services. I would appreciate if some people could share the resources they generally use to train their models and the steps on setting them up. Thank you.", "555971": "You can use https://colab.research.google.com/\n\nSee https://www.kaggle.com/getting-started/47096\nfor pointers.\n\nI've heard that the following sites offer free access to GPU enabled cloud but never tried it:\n\nhttps://www.floydhub.com/\n\nColmetML https://www.comet.ml/", "555985": "Why not Kaggle? You even get a free gpu (tesla P100 currently)", "556002": "I think Kaggle kernel's specs are not sufficient to reach the top ranks. Feel free to correct me though", "556014": "Have not tried it on Kaggle since I spent lots of dollars for my PC but I think you can use Kaggle but you will need to do things in pieces - and keep disk memory under the limits with frequent trimming of features that are not of importance. It would be a real pain.\n\nSince I am using lightgbm for my models the free GPU is of no value as I think the current Kaggle library does not include GPU support.", "556031": "I play this competition only on Kaggle. 2 kernels, the first one creates the features, and the second one trains. it's not so bad :)", "556278": "Ah, so like creating features -> exporting to dataset -> import dataset in another kernel.\nDefinitely will try this, thank you", "556403": "You can easily recompile lightgbm for gpu and you can also run many kernels at the same time. However finding some good features seems much more efficient than hours of compute!", "556825": "try floydhub", "556832": "Try to use a kernel as data source. Create the features in one kernel, save them into csv, and use other kernel for modeling. Check this link: https://www.kaggle.com/product-feedback/45472", "556834": "As you're save dataset into csv, you can combine python and R in the same completion. R has some very useful tools for feature engineering.", "556872": "Seconding google colab", "557201": "Just signed up for floydhub.com Looks like I have 2 hours of GPU time and 20 hours of CPU time before free runs out. CPU looks like it's 0.48 dollars per hour and GPU is 4.50. Does not look to be much more capable than Kaggle so going to pass on even trying to use it." }, "source": "meta" }