{
  "id": 78437,
  "title": "Do you use GCP or AWS for competitions?",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/78437",
  "author_name": "",
  "post_date": "2019-01-23T18:53:37.778708300Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I was wondering that if the top people on the leaderboard use gcp/aws/azure etc. to run the models for competitions like these with such a big dataset</p>",
  "messages": [
    {
      "id": "460464",
      "postDate": "01/23/2019 18:53:37",
      "content": "<p>I was wondering that if the top people on the leaderboard use gcp/aws/azure etc. to run the models for competitions like these with such a big dataset</p>",
      "rawMarkdown": "I was wondering that if the top people on the leaderboard use gcp/aws/azure etc. to run the models for competitions like these with such a big dataset",
      "votes": null
    },
    {
      "id": "460564",
      "postDate": "01/24/2019 01:02:19",
      "content": "<p>For most (but not all) competitions I think the rules might prevent you from running on anything other than a Kaggle kernel. As for technical capabilities, Kaggle kernels are topped by almost all the other options, but their ability to already come pre-loaded with many packages and with the given is pretty valuable. For any project with a moderate or large amount of data, Kaggle's kernels just won't cut it in terms of memory or processing power, you'll need to upgrade. Google is the next step up and is thankfully free, the setup is infinitely easier than AWS (which is a waking nightmare to use and pay for) but still requires some bash commands and installation of packages and repeated manual loading of datasets. AWS (and the paid-options at Google) provide the highest capabilities, but can be prohibitively expensive and has a notable learning curve. </p>",
      "rawMarkdown": "For most (but not all) competitions I think the rules might prevent you from running on anything other than a Kaggle kernel. As for technical capabilities, Kaggle kernels are topped by almost all the other options, but their ability to already come pre-loaded with many packages and with the given is pretty valuable. For any project with a moderate or large amount of data, Kaggle's kernels just won't cut it in terms of memory or processing power, you'll need to upgrade. Google is the next step up and is thankfully free, the setup is infinitely easier than AWS (which is a waking nightmare to use and pay for) but still requires some bash commands and installation of packages and repeated manual loading of datasets. AWS (and the paid-options at Google) provide the highest capabilities, but can be prohibitively expensive and has a notable learning curve.",
      "votes": null
    },
    {
      "id": "460574",
      "postDate": "01/24/2019 01:58:56",
      "content": "<p>GCP Deep learning VMs are a solid option they come pre-loaded with most of the packages for ML and DL, you can also scale them up or down base on your model requirements.</p>",
      "rawMarkdown": "GCP Deep learning VMs are a solid option they come pre-loaded with most of the packages for ML and DL, you can also scale them up or down base on your model requirements.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 460564,
      "author_name": "alecthekulak",
      "author_url": "",
      "post_date": "01/24/2019 01:02:19",
      "content": "<p>For most (but not all) competitions I think the rules might prevent you from running on anything other than a Kaggle kernel. As for technical capabilities, Kaggle kernels are topped by almost all the other options, but their ability to already come pre-loaded with many packages and with the given is pretty valuable. For any project with a moderate or large amount of data, Kaggle's kernels just won't cut it in terms of memory or processing power, you'll need to upgrade. Google is the next step up and is thankfully free, the setup is infinitely easier than AWS (which is a waking nightmare to use and pay for) but still requires some bash commands and installation of packages and repeated manual loading of datasets. AWS (and the paid-options at Google) provide the highest capabilities, but can be prohibitively expensive and has a notable learning curve. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 460574,
      "author_name": "cv13j0",
      "author_url": "",
      "post_date": "01/24/2019 01:58:56",
      "content": "<p>GCP Deep learning VMs are a solid option they come pre-loaded with most of the packages for ML and DL, you can also scale them up or down base on your model requirements.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "460464": "I was wondering that if the top people on the leaderboard use gcp/aws/azure etc. to run the models for competitions like these with such a big dataset",
    "460564": "For most (but not all) competitions I think the rules might prevent you from running on anything other than a Kaggle kernel. As for technical capabilities, Kaggle kernels are topped by almost all the other options, but their ability to already come pre-loaded with many packages and with the given is pretty valuable. For any project with a moderate or large amount of data, Kaggle's kernels just won't cut it in terms of memory or processing power, you'll need to upgrade. Google is the next step up and is thankfully free, the setup is infinitely easier than AWS (which is a waking nightmare to use and pay for) but still requires some bash commands and installation of packages and repeated manual loading of datasets. AWS (and the paid-options at Google) provide the highest capabilities, but can be prohibitively expensive and has a notable learning curve.",
    "460574": "GCP Deep learning VMs are a solid option they come pre-loaded with most of the packages for ML and DL, you can also scale them up or down base on your model requirements."
  },
  "source": "meta"
}