{
  "id": 9481,
  "title": "Hardware",
  "url": "/competitions/acquire-valued-shoppers-challenge/discussion/9481",
  "author_name": "",
  "post_date": "2014-06-16T14:00:14.090Z",
  "votes": null,
  "comment_count": 16,
  "views": 4445,
  "content": "<p>Hi folks,</p>\n\n<p>I have two questions on hardware</p>\n<ol>\n<li><span style=\"line-height: 1.4\">Do any of you know if there are servers that can be rented fairly cheap($10-20/month) with at least 4 cores and 16GB RAM?</span></li>\n<li><span style=\"line-height: 1.4\">Any suggestions on what a good laptop config should look like for participating in these competitions? Any suggestions in&nbsp;the $1000 price range?</span></li>\n</ol>",
  "messages": [
    {
      "id": "49199",
      "postDate": "06/16/2014 14:00:14",
      "content": "<p>Hi folks,</p>\n\n<p>I have two questions on hardware</p>\n<ol>\n<li><span style=\"line-height: 1.4\">Do any of you know if there are servers that can be rented fairly cheap($10-20/month) with at least 4 cores and 16GB RAM?</span></li>\n<li><span style=\"line-height: 1.4\">Any suggestions on what a good laptop config should look like for participating in these competitions? Any suggestions in&nbsp;the $1000 price range?</span></li>\n</ol>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49366",
      "postDate": "06/21/2014 18:00:58",
      "content": "<p>1. AWS will provide what you need</p>\n<p>2. in my case, I bought used macbook pro 2011 i7 at 550 dollars and I spent extra 150 to purchase&nbsp;the 16 gb ram. If you are looking for a&nbsp;desktop, you could get better computing computer at &lt;1000 dollars.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49367",
      "postDate": "06/21/2014 19:01:21",
      "content": "<p>http://aws.amazon.com/ec2/pricing/<br>https://cloud.google.com/products/compute-engine/</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49372",
      "postDate": "06/22/2014 04:45:20",
      "content": "<p>Thanks guys. I think I'll test out AWS!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49373",
      "postDate": "06/22/2014 07:22:34",
      "content": "<p>Try AWS spot requests.&nbsp;Some of the classes&nbsp;can be had for very reasonable rates. These requests&nbsp;have&nbsp;fluctuating prices based on demand.&nbsp;For example: m2.4xlarge (an older generation of AWS instance type)&nbsp;with 8 virtual cores and 68GB memory, usually average about 7 cents an hour, m2.2xlarge with half of the resources for normally half of that price. But they are more popular, so they get&nbsp;price burst more often.&nbsp; If you give a peak price wide enough margin, you can regularly have an instance running for weeks without interruption.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49391",
      "postDate": "06/23/2014 05:11:18",
      "content": "<p>Thanks Wei.</p>\n\n<p>As I understand, spot instances can shut down if someone else bids higher than me and there are no free instances left. The best way to work with spot instances is for your code to be able to keep 'saving its state' to disk and resume when the instance is up again. Does anyone know ways to achieve this in either R or python?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49412",
      "postDate": "06/23/2014 14:36:54",
      "content": "<p>[quote=TayShin;49366]</p>\n<p>1. AWS will provide what you need</p>\n<p>[/quote]</p>\n\n<p>What do you guys think of Digital Ocean or Linode as an alternative? &nbsp;To the OP, if you're using R, I've been using StatAce.com - they're very useable, have a broad range of server tiers, and are fairly generous with their free hours.</p>\n\n<p>[quote=TayShin;49366]</p>\n<p>2. in my case, I bought used macbook pro 2011 i7 at 550 dollars and I spent extra 150 to purchase&nbsp;the 16 gb ram. If you are looking for a&nbsp;desktop, you could get better computing computer at &lt;1000 dollars.&nbsp;</p>\n<p>[/quote]</p>\n\n<p>Wow... on Ebay? &nbsp;Going rate seems much higher than that. &nbsp;If you're looking for something small, though, EUROCOM has some interesting-looking options - you can snag an 11&quot; notebook with a i7 quad-core for around $800.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49418",
      "postDate": "06/23/2014 15:30:47",
      "content": "<p>Nostradamus, I use a program grive to sync to google drive storage, after any long runs. If the request stops during the run, that's just bad luck and I will just restart it again later. Kaggle competition work load is not that time critical, that's why AWS spot requests are ideal for this.</p>\n\n<p>Also, when I open an spot request, I give the max price some room above the average price base on looking at the 1 month history of price history. (There is a screen to look up the price history of each instances.)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49419",
      "postDate": "06/23/2014 15:53:58",
      "content": "<p>[quote=Phil Culliton;49412]</p>\n<p>[quote=TayShin;49366]</p>\n<p>1. AWS will provide what you need</p>\n<p>[/quote]</p>\n<p>What do you guys think of Digital Ocean or Linode as an alternative? &nbsp;To the OP, if you're using R, I've been using StatAce.com - they're very useable, have a broad range of server tiers, and are fairly generous with their free hours.</p>\n<p>[quote=TayShin;49366]</p>\n<p>2. in my case, I bought used macbook pro 2011 i7 at 550 dollars and I spent extra 150 to purchase&nbsp;the 16 gb ram. If you are looking for a&nbsp;desktop, you could get better computing computer at &lt;1000 dollars.&nbsp;</p>\n<p>[/quote]</p>\n<p>Wow... on Ebay? &nbsp;Going rate seems much higher than that. &nbsp;If you're looking for something small, though, EUROCOM has some interesting-looking options - you can snag an 11&quot; notebook with a i7 quad-core for around $800.</p>\n<p>[/quote]</p>\n<p>HaHa... I searched on craigslist where some people want to sell their product ASAP at lower price :D</p>\n<p>But I recommend to buy a customized desktop from&nbsp;newegg.com and make your own server by R studio server. In this way, you could use your high performing computer at any computer and everywhere.&nbsp;</p>\n<p>If you are trying to use r-studio server, all you have to do is&nbsp;</p>\n<p>1. install r studio server&nbsp;</p>\n<p>However, you will&nbsp;not get an access to your r-server if you are not using same adapter for internet. So, in order to use your r-server at anytime, any computer, anywhere,</p>\n<p>2. change from dynamic ip to static ip</p>\n<p>3. port forwarding</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49420",
      "postDate": "06/23/2014 16:03:36",
      "content": "<p>[quote=TayShin;49419]</p>\n<p>HaHa... I searched on craigslist where some people want to sell their product ASAP at lower price :D</p>\n<p>[/quote]</p>\n\n<p>Good find!</p>\n\n<p>[quote=TayShin;49419]</p>\n<p>But I recommend to buy a customized desktop from&nbsp;newegg.com and make your own server by R studio server. In this way, you could use your high performing computer at any computer and everywhere.&nbsp;</p>\n<p>If you are trying to use r-studio server, all you have to do is&nbsp;</p>\n<p>1. install r studio server&nbsp;</p>\n<p>However, you will&nbsp;not get an access to your r-server if you are not using same adapter for internet. So, in order to use your r-server at anytime, any computer, anywhere,</p>\n<p>2. change from dynamic ip to static ip</p>\n<p>3. port forwarding</p>\n<p>[/quote]</p>\n\n<p>True, but then you need to secure it and maintain it yourself, and getting that right isn't easy for everybody. &nbsp;Sites like StatAce.com (or Wakari, for Python, etc.) take care of that for you, and in some cases allow you to scale up or down as your needs change. &nbsp;So that may factor into the decision about which way to go - it did for me!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49421",
      "postDate": "06/23/2014 16:13:13",
      "content": "<p>Wei,</p>\n\n<p>Some of my runs take a couple of days at a time before I can write out anything. I guess there are ways I could split the job but it is a lot of effort. I was wondering if anyone had any success with joblib/python or some other package that created restore points periodically and restarted jobs from previous state seamlessly.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49423",
      "postDate": "06/23/2014 16:29:36",
      "content": "<p>[quote=Nostradamus;49421]</p>\n<p>Wei,</p>\n<p>Some of my runs take a couple of days at a time before I can write out anything. I guess there are ways I could split the job but it is a lot of effort. I was wondering if anyone had any success with joblib/python or some other package that created restore points periodically and restarted jobs from previous state seamlessly.</p>\n<p>[/quote]</p>\n<p>Nostradamus, you're amazing to get such good results using such unstable hardware~~ :) in my opinion, 1000 can buy you good parts and build a really powerful desktop~~</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49424",
      "postDate": "06/23/2014 16:38:24",
      "content": "<p>Carson,</p>\n\n<p>A laptop with 8GB RAM isn't&nbsp;exactly unstable hardware :)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49425",
      "postDate": "06/23/2014 16:48:53",
      "content": "<p>Nostradamus,</p>\n<p>Can your job be parallelized? A job runs days on your 8GB laptop, can be finished in hours on a 68GB 8cores AWS machine and cost you a couple of dollars if you can&nbsp;split it up onto multiple cores.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49426",
      "postDate": "06/23/2014 16:53:44",
      "content": "<p>Wei,</p>\n<p>Some packages/modules do and some don't support parallelization. For the current job I am running, I think I found a package that does so I'll go hunt down an AWS spot instance!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49448",
      "postDate": "06/24/2014 03:45:13",
      "content": "<p>I use a old HP XW8600 Workstation (Dual Quad core and 32 GB RAM) that I bought in eBay.</p>\n<p>http://www.ebay.com/itm/161272858365?_trksid=p2055119.m1438.l2649&amp;ssPageName=STRK%3AMEBIDX%3AIT</p>\n<p>These servers&nbsp;go on sale for ~$400 once in a while. you can also find many dell servers with similar config.It has worked out well for me so far.</p>\n\n<p>I also use Amazon AWS if I need more power (for parallilizable&nbsp;models). I typically choose c3.8xlarge (32 core) spot instances since I think it has the high CPU/cost ratio.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49449",
      "postDate": "06/24/2014 03:45:59",
      "content": "<p>I use a old HP XW8600 Workstation (Dual Quad core and 32 GB RAM) that I bought in eBay.</p>\n<p>http://www.ebay.com/itm/161272858365?_trksid=p2055119.m1438.l2649&amp;ssPageName=STRK%3AMEBIDX%3AIT</p>\n<p>These servers&nbsp;go on sale for ~$400 once in a while. you can also find many dell servers with similar config.It has worked out well for me so far.</p>\n\n<p>I also use Amazon AWS if I need more power (for parallilizable&nbsp;models). I typically choose c3.8xlarge (32 core) spot instances since I think it has the high CPU/cost ratio.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 49366,
      "author_name": "tayshin",
      "author_url": "",
      "post_date": "06/21/2014 18:00:58",
      "content": "<p>1. AWS will provide what you need</p>\n<p>2. in my case, I bought used macbook pro 2011 i7 at 550 dollars and I spent extra 150 to purchase&nbsp;the 16 gb ram. If you are looking for a&nbsp;desktop, you could get better computing computer at &lt;1000 dollars.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49367,
      "author_name": "mikeskim",
      "author_url": "",
      "post_date": "06/21/2014 19:01:21",
      "content": "<p>http://aws.amazon.com/ec2/pricing/<br>https://cloud.google.com/products/compute-engine/</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49372,
      "author_name": "",
      "author_url": "",
      "post_date": "06/22/2014 04:45:20",
      "content": "<p>Thanks guys. I think I'll test out AWS!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49373,
      "author_name": "weiwunyc",
      "author_url": "",
      "post_date": "06/22/2014 07:22:34",
      "content": "<p>Try AWS spot requests.&nbsp;Some of the classes&nbsp;can be had for very reasonable rates. These requests&nbsp;have&nbsp;fluctuating prices based on demand.&nbsp;For example: m2.4xlarge (an older generation of AWS instance type)&nbsp;with 8 virtual cores and 68GB memory, usually average about 7 cents an hour, m2.2xlarge with half of the resources for normally half of that price. But they are more popular, so they get&nbsp;price burst more often.&nbsp; If you give a peak price wide enough margin, you can regularly have an instance running for weeks without interruption.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49391,
      "author_name": "",
      "author_url": "",
      "post_date": "06/23/2014 05:11:18",
      "content": "<p>Thanks Wei.</p>\n\n<p>As I understand, spot instances can shut down if someone else bids higher than me and there are no free instances left. The best way to work with spot instances is for your code to be able to keep 'saving its state' to disk and resume when the instance is up again. Does anyone know ways to achieve this in either R or python?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49412,
      "author_name": "philculliton",
      "author_url": "",
      "post_date": "06/23/2014 14:36:54",
      "content": "<p>[quote=TayShin;49366]</p>\n<p>1. AWS will provide what you need</p>\n<p>[/quote]</p>\n\n<p>What do you guys think of Digital Ocean or Linode as an alternative? &nbsp;To the OP, if you're using R, I've been using StatAce.com - they're very useable, have a broad range of server tiers, and are fairly generous with their free hours.</p>\n\n<p>[quote=TayShin;49366]</p>\n<p>2. in my case, I bought used macbook pro 2011 i7 at 550 dollars and I spent extra 150 to purchase&nbsp;the 16 gb ram. If you are looking for a&nbsp;desktop, you could get better computing computer at &lt;1000 dollars.&nbsp;</p>\n<p>[/quote]</p>\n\n<p>Wow... on Ebay? &nbsp;Going rate seems much higher than that. &nbsp;If you're looking for something small, though, EUROCOM has some interesting-looking options - you can snag an 11&quot; notebook with a i7 quad-core for around $800.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49418,
      "author_name": "weiwunyc",
      "author_url": "",
      "post_date": "06/23/2014 15:30:47",
      "content": "<p>Nostradamus, I use a program grive to sync to google drive storage, after any long runs. If the request stops during the run, that's just bad luck and I will just restart it again later. Kaggle competition work load is not that time critical, that's why AWS spot requests are ideal for this.</p>\n\n<p>Also, when I open an spot request, I give the max price some room above the average price base on looking at the 1 month history of price history. (There is a screen to look up the price history of each instances.)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49419,
      "author_name": "tayshin",
      "author_url": "",
      "post_date": "06/23/2014 15:53:58",
      "content": "<p>[quote=Phil Culliton;49412]</p>\n<p>[quote=TayShin;49366]</p>\n<p>1. AWS will provide what you need</p>\n<p>[/quote]</p>\n<p>What do you guys think of Digital Ocean or Linode as an alternative? &nbsp;To the OP, if you're using R, I've been using StatAce.com - they're very useable, have a broad range of server tiers, and are fairly generous with their free hours.</p>\n<p>[quote=TayShin;49366]</p>\n<p>2. in my case, I bought used macbook pro 2011 i7 at 550 dollars and I spent extra 150 to purchase&nbsp;the 16 gb ram. If you are looking for a&nbsp;desktop, you could get better computing computer at &lt;1000 dollars.&nbsp;</p>\n<p>[/quote]</p>\n<p>Wow... on Ebay? &nbsp;Going rate seems much higher than that. &nbsp;If you're looking for something small, though, EUROCOM has some interesting-looking options - you can snag an 11&quot; notebook with a i7 quad-core for around $800.</p>\n<p>[/quote]</p>\n<p>HaHa... I searched on craigslist where some people want to sell their product ASAP at lower price :D</p>\n<p>But I recommend to buy a customized desktop from&nbsp;newegg.com and make your own server by R studio server. In this way, you could use your high performing computer at any computer and everywhere.&nbsp;</p>\n<p>If you are trying to use r-studio server, all you have to do is&nbsp;</p>\n<p>1. install r studio server&nbsp;</p>\n<p>However, you will&nbsp;not get an access to your r-server if you are not using same adapter for internet. So, in order to use your r-server at anytime, any computer, anywhere,</p>\n<p>2. change from dynamic ip to static ip</p>\n<p>3. port forwarding</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49420,
      "author_name": "philculliton",
      "author_url": "",
      "post_date": "06/23/2014 16:03:36",
      "content": "<p>[quote=TayShin;49419]</p>\n<p>HaHa... I searched on craigslist where some people want to sell their product ASAP at lower price :D</p>\n<p>[/quote]</p>\n\n<p>Good find!</p>\n\n<p>[quote=TayShin;49419]</p>\n<p>But I recommend to buy a customized desktop from&nbsp;newegg.com and make your own server by R studio server. In this way, you could use your high performing computer at any computer and everywhere.&nbsp;</p>\n<p>If you are trying to use r-studio server, all you have to do is&nbsp;</p>\n<p>1. install r studio server&nbsp;</p>\n<p>However, you will&nbsp;not get an access to your r-server if you are not using same adapter for internet. So, in order to use your r-server at anytime, any computer, anywhere,</p>\n<p>2. change from dynamic ip to static ip</p>\n<p>3. port forwarding</p>\n<p>[/quote]</p>\n\n<p>True, but then you need to secure it and maintain it yourself, and getting that right isn't easy for everybody. &nbsp;Sites like StatAce.com (or Wakari, for Python, etc.) take care of that for you, and in some cases allow you to scale up or down as your needs change. &nbsp;So that may factor into the decision about which way to go - it did for me!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49421,
      "author_name": "",
      "author_url": "",
      "post_date": "06/23/2014 16:13:13",
      "content": "<p>Wei,</p>\n\n<p>Some of my runs take a couple of days at a time before I can write out anything. I guess there are ways I could split the job but it is a lot of effort. I was wondering if anyone had any success with joblib/python or some other package that created restore points periodically and restarted jobs from previous state seamlessly.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49423,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "06/23/2014 16:29:36",
      "content": "<p>[quote=Nostradamus;49421]</p>\n<p>Wei,</p>\n<p>Some of my runs take a couple of days at a time before I can write out anything. I guess there are ways I could split the job but it is a lot of effort. I was wondering if anyone had any success with joblib/python or some other package that created restore points periodically and restarted jobs from previous state seamlessly.</p>\n<p>[/quote]</p>\n<p>Nostradamus, you're amazing to get such good results using such unstable hardware~~ :) in my opinion, 1000 can buy you good parts and build a really powerful desktop~~</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49424,
      "author_name": "",
      "author_url": "",
      "post_date": "06/23/2014 16:38:24",
      "content": "<p>Carson,</p>\n\n<p>A laptop with 8GB RAM isn't&nbsp;exactly unstable hardware :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49425,
      "author_name": "weiwunyc",
      "author_url": "",
      "post_date": "06/23/2014 16:48:53",
      "content": "<p>Nostradamus,</p>\n<p>Can your job be parallelized? A job runs days on your 8GB laptop, can be finished in hours on a 68GB 8cores AWS machine and cost you a couple of dollars if you can&nbsp;split it up onto multiple cores.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49426,
      "author_name": "",
      "author_url": "",
      "post_date": "06/23/2014 16:53:44",
      "content": "<p>Wei,</p>\n<p>Some packages/modules do and some don't support parallelization. For the current job I am running, I think I found a package that does so I'll go hunt down an AWS spot instance!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49448,
      "author_name": "prempandian",
      "author_url": "",
      "post_date": "06/24/2014 03:45:13",
      "content": "<p>I use a old HP XW8600 Workstation (Dual Quad core and 32 GB RAM) that I bought in eBay.</p>\n<p>http://www.ebay.com/itm/161272858365?_trksid=p2055119.m1438.l2649&amp;ssPageName=STRK%3AMEBIDX%3AIT</p>\n<p>These servers&nbsp;go on sale for ~$400 once in a while. you can also find many dell servers with similar config.It has worked out well for me so far.</p>\n\n<p>I also use Amazon AWS if I need more power (for parallilizable&nbsp;models). I typically choose c3.8xlarge (32 core) spot instances since I think it has the high CPU/cost ratio.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49449,
      "author_name": "prempandian",
      "author_url": "",
      "post_date": "06/24/2014 03:45:59",
      "content": "<p>I use a old HP XW8600 Workstation (Dual Quad core and 32 GB RAM) that I bought in eBay.</p>\n<p>http://www.ebay.com/itm/161272858365?_trksid=p2055119.m1438.l2649&amp;ssPageName=STRK%3AMEBIDX%3AIT</p>\n<p>These servers&nbsp;go on sale for ~$400 once in a while. you can also find many dell servers with similar config.It has worked out well for me so far.</p>\n\n<p>I also use Amazon AWS if I need more power (for parallilizable&nbsp;models). I typically choose c3.8xlarge (32 core) spot instances since I think it has the high CPU/cost ratio.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "49199": "",
    "49366": "",
    "49367": "",
    "49372": "",
    "49373": "",
    "49391": "",
    "49412": "",
    "49418": "",
    "49419": "",
    "49420": "",
    "49421": "",
    "49423": "",
    "49424": "",
    "49425": "",
    "49426": "",
    "49448": "",
    "49449": ""
  },
  "source": "meta"
}