{
  "id": 11077,
  "title": "Speed - order of magnitude improvement?",
  "url": "/competitions/avazu-ctr-prediction/discussion/11077",
  "author_name": "",
  "post_date": "2014-11-28T10:17:57.843Z",
  "votes": null,
  "comment_count": 4,
  "views": 1651,
  "content": "<p>Hi everyone!</p>\n<p>I've been making lots of tests and submissions for this competitions (which is really fun by the way). But the further I go, the more time it takes to run an experiment. My last attempts took 6 hours on my PC. And it's too much for many reasons.</p>\n<p>How I deal with the speed:</p>\n<p>- Simple Python code (refactor what I can to make it simpler)</p>\n<p>- PyPy 2.7 (PyPy3 is 2 times worse)</p>\n<p>- SSD memory</p>\n<p>I see the following opportunities to increase the speed:</p>\n<p>- Process data in mini-batches to get use of multiple cores</p>\n<p>- Redo the code to work with Anaconda + MKL + Numba (<a href=\"https://www.kaggle.com/c/avazu-ctr-prediction/forums/t/10927/beat-the-benchmark-with-less-than-1mb-of-memory/59144#post59144\" target=\"_blank\">stole from here</a>)</p>\n<p>- Get use of Cython</p>\n<p>- Redo the code in C++ (sic!)</p>\n<p>But these options are not very viable to me, since the first two, I believe, will give but a marginal improvement, the latter two are too much heavy-lifting since I know no C/C++.</p>\n<p>What works for you? How do you get a good speed? What are the options to get an order of magnitude improvement, but still work with reasonably regular Python?</p>\n<p>P.S. Or C/C++ is not that hard to learn for simple mathematical computations?</p>",
  "messages": [
    {
      "id": "59150",
      "postDate": "11/28/2014 10:17:57",
      "content": "<p>Hi everyone!</p>\n<p>I've been making lots of tests and submissions for this competitions (which is really fun by the way). But the further I go, the more time it takes to run an experiment. My last attempts took 6 hours on my PC. And it's too much for many reasons.</p>\n<p>How I deal with the speed:</p>\n<p>- Simple Python code (refactor what I can to make it simpler)</p>\n<p>- PyPy 2.7 (PyPy3 is 2 times worse)</p>\n<p>- SSD memory</p>\n<p>I see the following opportunities to increase the speed:</p>\n<p>- Process data in mini-batches to get use of multiple cores</p>\n<p>- Redo the code to work with Anaconda + MKL + Numba (<a href=\"https://www.kaggle.com/c/avazu-ctr-prediction/forums/t/10927/beat-the-benchmark-with-less-than-1mb-of-memory/59144#post59144\" target=\"_blank\">stole from here</a>)</p>\n<p>- Get use of Cython</p>\n<p>- Redo the code in C++ (sic!)</p>\n<p>But these options are not very viable to me, since the first two, I believe, will give but a marginal improvement, the latter two are too much heavy-lifting since I know no C/C++.</p>\n<p>What works for you? How do you get a good speed? What are the options to get an order of magnitude improvement, but still work with reasonably regular Python?</p>\n<p>P.S. Or C/C++ is not that hard to learn for simple mathematical computations?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59154",
      "postDate": "11/28/2014 12:23:16",
      "content": "<p>One option is to sign up for free Azure trial ($200 credit) and get 16 cores Xeon E5-2670 with 112Gb for ~$1.3 per hour. Might require some work to create VM image with competition data, but it sounds easier rather than to code in C++ :)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59155",
      "postDate": "11/28/2014 12:30:20",
      "content": "<p>It seems that to get use of all that monster, I'll have to parallel the pipeline somehow, right? Or it's just going to be x10 faster?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59157",
      "postDate": "11/28/2014 12:45:35",
      "content": "<p>That depends. Is your code CPU- or IO/RAM- bound? In the former case definitely you need to think multithreaded, although I have no idea how to do that in Python. But huge RAM might help as well to store&nbsp;the whole dataset and additional features&nbsp;in memory for lightning fast IO. MySQL has in-memory engine and other DBs for sure. Actually I'm ok with 28Gb Azure instance, but they have more just in case.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59182",
      "postDate": "11/28/2014 18:52:17",
      "content": "<p>[quote=Ivan Lobov;59150]</p>\n<p>- Redo the code in C++ (sic!)</p>\n<p>[/quote]</p>\n<p>A naive C++ re-implementation will not get you an order of magnitude speed increase over PyPy. &nbsp;I got perhaps a 3x speedup from re-implementing tingrtu's code in C++ on Tradeshift. &nbsp;I didn't use many tricks, but I&nbsp;have some (15 years or so) experience in&nbsp;C++ optimization, and I don't expect I'd have improved the speed&nbsp;much without going deep. &nbsp;You would have easy access to OpenMP via C++, which would give you a speedup - I don't recall how fast my OpenMP-enabled code was on Tradeshift, but it was nice.</p>\n<p>If you decide to go this route, I'd be happy to provide some (non-rule-breaking) help in doing so.</p>\n<p>You might consider implementing the code in Theano (which gives access to CUDA, and some access to OpenCL!), Numba, or Cython first, though. &nbsp;It'll likely be an easier transition.</p>\n<p>I definitely do suggest that every programmer have a working knowledge of C++. &nbsp;You never know when it might help you.</p>\n<p>[quote=Ivan Lobov;59150]</p>\n<p>P.S. Or C/C++ is not that hard to learn for simple mathematical computations?</p>\n<p>[/quote]</p>\n<p>The math would be relatively easy, yes, as long as you understood some C++ fundamentals. &nbsp;I think you'd find it harder to write all of the ancillary (data processing, etc.) code, though. &nbsp;Python makes a number of common operations much easier to do, and easier to understand. &nbsp;Trying to use pieces of&nbsp;C++ in the same manner as their Python &quot;counterparts&quot; (in the instances those exist) would likely cost you a lot of your performance gain.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 59154,
      "author_name": "dmitrynozhnin",
      "author_url": "",
      "post_date": "11/28/2014 12:23:16",
      "content": "<p>One option is to sign up for free Azure trial ($200 credit) and get 16 cores Xeon E5-2670 with 112Gb for ~$1.3 per hour. Might require some work to create VM image with competition data, but it sounds easier rather than to code in C++ :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59155,
      "author_name": "ivanlobov",
      "author_url": "",
      "post_date": "11/28/2014 12:30:20",
      "content": "<p>It seems that to get use of all that monster, I'll have to parallel the pipeline somehow, right? Or it's just going to be x10 faster?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59157,
      "author_name": "dmitrynozhnin",
      "author_url": "",
      "post_date": "11/28/2014 12:45:35",
      "content": "<p>That depends. Is your code CPU- or IO/RAM- bound? In the former case definitely you need to think multithreaded, although I have no idea how to do that in Python. But huge RAM might help as well to store&nbsp;the whole dataset and additional features&nbsp;in memory for lightning fast IO. MySQL has in-memory engine and other DBs for sure. Actually I'm ok with 28Gb Azure instance, but they have more just in case.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 59182,
      "author_name": "philculliton",
      "author_url": "",
      "post_date": "11/28/2014 18:52:17",
      "content": "<p>[quote=Ivan Lobov;59150]</p>\n<p>- Redo the code in C++ (sic!)</p>\n<p>[/quote]</p>\n<p>A naive C++ re-implementation will not get you an order of magnitude speed increase over PyPy. &nbsp;I got perhaps a 3x speedup from re-implementing tingrtu's code in C++ on Tradeshift. &nbsp;I didn't use many tricks, but I&nbsp;have some (15 years or so) experience in&nbsp;C++ optimization, and I don't expect I'd have improved the speed&nbsp;much without going deep. &nbsp;You would have easy access to OpenMP via C++, which would give you a speedup - I don't recall how fast my OpenMP-enabled code was on Tradeshift, but it was nice.</p>\n<p>If you decide to go this route, I'd be happy to provide some (non-rule-breaking) help in doing so.</p>\n<p>You might consider implementing the code in Theano (which gives access to CUDA, and some access to OpenCL!), Numba, or Cython first, though. &nbsp;It'll likely be an easier transition.</p>\n<p>I definitely do suggest that every programmer have a working knowledge of C++. &nbsp;You never know when it might help you.</p>\n<p>[quote=Ivan Lobov;59150]</p>\n<p>P.S. Or C/C++ is not that hard to learn for simple mathematical computations?</p>\n<p>[/quote]</p>\n<p>The math would be relatively easy, yes, as long as you understood some C++ fundamentals. &nbsp;I think you'd find it harder to write all of the ancillary (data processing, etc.) code, though. &nbsp;Python makes a number of common operations much easier to do, and easier to understand. &nbsp;Trying to use pieces of&nbsp;C++ in the same manner as their Python &quot;counterparts&quot; (in the instances those exist) would likely cost you a lot of your performance gain.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "59150": "",
    "59154": "",
    "59155": "",
    "59157": "",
    "59182": ""
  },
  "source": "meta"
}