{
  "id": 10840,
  "title": "vowpal wabbit starter code",
  "url": "/competitions/avazu-ctr-prediction/discussion/10840",
  "author_name": "",
  "post_date": "2014-11-05T03:56:53.360Z",
  "votes": 13,
  "comment_count": 11,
  "views": 5121,
  "content": "<p>Hi kagglers,</p>\n<p>I steal the great vw benchmark from Triskelion,&nbsp;https://www.kaggle.com/c/criteo-display-ad-challenge/forums/t/9583/beat-the-benchmark-with-vowpal-wabbit</p>\n<p>and made a few changes so it works now.</p>\n<p>vw is available at http://hunch.net/~vw/</p>\n<p>and just run 'sh run.sh' after you get vw installed.</p>\n<p>In the previous CTR contest, many people succeeded with vw. Let's learn it together this time. Any comments and advice are appreciated.</p>\n<p>Cheers!</p>\n<p>update: LB 0.423</p>",
  "messages": [
    {
      "id": "57337",
      "postDate": "11/05/2014 03:56:53",
      "content": "<p>Hi kagglers,</p>\n<p>I steal the great vw benchmark from Triskelion,&nbsp;https://www.kaggle.com/c/criteo-display-ad-challenge/forums/t/9583/beat-the-benchmark-with-vowpal-wabbit</p>\n<p>and made a few changes so it works now.</p>\n<p>vw is available at http://hunch.net/~vw/</p>\n<p>and just run 'sh run.sh' after you get vw installed.</p>\n<p>In the previous CTR contest, many people succeeded with vw. Let's learn it together this time. Any comments and advice are appreciated.</p>\n<p>Cheers!</p>\n<p>update: LB 0.423</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57340",
      "postDate": "11/05/2014 07:45:51",
      "content": "<p>we have change something in csv_to_vw.py.</p>\n<p>lucky.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57363",
      "postDate": "11/05/2014 21:36:45",
      "content": "<p>Increase <em>--passes</em> for instant improvement.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57440",
      "postDate": "11/06/2014 21:29:06",
      "content": "<p>Can I use this code on my laptop with only 4GB memory ?&nbsp;</p>\n<p>I am curious because I am having trouble working with this large file and I want a workable solution.&nbsp;</p>\n<p>Any input can be very helpful.</p>\n\n<p>thanks in advance and thanks in particular to Chen for the starter code provided here.&nbsp;</p>\n<p>Best</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57442",
      "postDate": "11/06/2014 22:34:31",
      "content": "<p>[quote=TD;57440]</p>\n<p>Can I use this code on my laptop with only 4GB memory ?&nbsp;</p>\n<p>[/quote]</p>\n<p>Sure, vw is known for its memory efficiency, which can be as low as 200 MB. Using pypy instead of python will speed preprocessing but it is only to be done once. &nbsp;&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57454",
      "postDate": "11/07/2014 05:01:19",
      "content": "<p>Thanks Chen for your input. I am going to try it out now.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57456",
      "postDate": "11/07/2014 06:09:57",
      "content": "<p>It seems like you are formatting all your features as categorical features. Doesn't that lose some of the information in the variables which have integer values?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57479",
      "postDate": "11/07/2014 18:52:13",
      "content": "<p>Hey Xueer,</p>\n\n<p>Thanks for the benchmark code. I ran it but my LB score&nbsp;1.0002701. Is there anything I am doing incorrectly.</p>\n\n<p>Thanks</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57497",
      "postDate": "11/08/2014 01:56:09",
      "content": "<p>[quote=Xueer Chen;57337]</p>\n<p>Hi kagglers,</p>\n<p>I steal the great vw benchmark from Triskelion,&nbsp;https://www.kaggle.com/c/criteo-display-ad-challenge/forums/t/9583/beat-the-benchmark-with-vowpal-wabbit</p>\n<p>and made a few changes so it works now.</p>\n<p>vw is available at http://hunch.net/~vw/</p>\n<p>and just run 'sh run.sh' after you get vw installed.</p>\n<p>In the previous CTR contest, many people succeeded with vw. Let's learn it together this time. Any comments and advice are appreciated.</p>\n<p>Cheers!</p>\n<p>update: LB 0.423</p>\n<p>[/quote]</p>\n<p>I'm curious about VW's speed? On what machine (e.g., number of CPU cores, amount of RAM) did you run VW? And how much time did the training take (not including pre-processing the data)?&nbsp;</p>\n\n<p>Thanks.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57540",
      "postDate": "11/09/2014 04:55:51",
      "content": "<p>If I do my own feature engineering, can I just pass the csv files with my new features into this script and run in with vw? Thank you</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "57570",
      "postDate": "11/10/2014 01:18:32",
      "content": "<p>[quote=Charles;57497]</p>\n<p>I'm curious about VW's speed? On what machine (e.g., number of CPU cores, amount of RAM) did you run VW? And how much time did the training take (not including pre-processing the data)?&nbsp;</p>\n<p>Thanks.</p>\n<p>[/quote]</p>\n<p>I've run a VW code similar to that on a Macbook Pro with a 2.6 GHz i5 processor (4 cores) and 8GB of RAM and it took about 2 and a half hours.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "318845",
      "postDate": "04/24/2018 15:23:07",
      "content": "<p>Hey @Xueer,</p>\n\n<p>why you didn't use numerical_features to vw file.</p>",
      "rawMarkdown": "Hey @Xueer,\n\nwhy you didn't use numerical_features to vw file.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 57340,
      "author_name": "wangheyu",
      "author_url": "",
      "post_date": "11/05/2014 07:45:51",
      "content": "<p>we have change something in csv_to_vw.py.</p>\n<p>lucky.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57363,
      "author_name": "zygmunt",
      "author_url": "",
      "post_date": "11/05/2014 21:36:45",
      "content": "<p>Increase <em>--passes</em> for instant improvement.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57440,
      "author_name": "attud27",
      "author_url": "",
      "post_date": "11/06/2014 21:29:06",
      "content": "<p>Can I use this code on my laptop with only 4GB memory ?&nbsp;</p>\n<p>I am curious because I am having trouble working with this large file and I want a workable solution.&nbsp;</p>\n<p>Any input can be very helpful.</p>\n\n<p>thanks in advance and thanks in particular to Chen for the starter code provided here.&nbsp;</p>\n<p>Best</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57442,
      "author_name": "xueerchen",
      "author_url": "",
      "post_date": "11/06/2014 22:34:31",
      "content": "<p>[quote=TD;57440]</p>\n<p>Can I use this code on my laptop with only 4GB memory ?&nbsp;</p>\n<p>[/quote]</p>\n<p>Sure, vw is known for its memory efficiency, which can be as low as 200 MB. Using pypy instead of python will speed preprocessing but it is only to be done once. &nbsp;&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57454,
      "author_name": "attud27",
      "author_url": "",
      "post_date": "11/07/2014 05:01:19",
      "content": "<p>Thanks Chen for your input. I am going to try it out now.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57456,
      "author_name": "goffinj",
      "author_url": "",
      "post_date": "11/07/2014 06:09:57",
      "content": "<p>It seems like you are formatting all your features as categorical features. Doesn't that lose some of the information in the variables which have integer values?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57479,
      "author_name": "ankitj",
      "author_url": "",
      "post_date": "11/07/2014 18:52:13",
      "content": "<p>Hey Xueer,</p>\n\n<p>Thanks for the benchmark code. I ran it but my LB score&nbsp;1.0002701. Is there anything I am doing incorrectly.</p>\n\n<p>Thanks</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57497,
      "author_name": "kuo000",
      "author_url": "",
      "post_date": "11/08/2014 01:56:09",
      "content": "<p>[quote=Xueer Chen;57337]</p>\n<p>Hi kagglers,</p>\n<p>I steal the great vw benchmark from Triskelion,&nbsp;https://www.kaggle.com/c/criteo-display-ad-challenge/forums/t/9583/beat-the-benchmark-with-vowpal-wabbit</p>\n<p>and made a few changes so it works now.</p>\n<p>vw is available at http://hunch.net/~vw/</p>\n<p>and just run 'sh run.sh' after you get vw installed.</p>\n<p>In the previous CTR contest, many people succeeded with vw. Let's learn it together this time. Any comments and advice are appreciated.</p>\n<p>Cheers!</p>\n<p>update: LB 0.423</p>\n<p>[/quote]</p>\n<p>I'm curious about VW's speed? On what machine (e.g., number of CPU cores, amount of RAM) did you run VW? And how much time did the training take (not including pre-processing the data)?&nbsp;</p>\n\n<p>Thanks.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57540,
      "author_name": "florianm",
      "author_url": "",
      "post_date": "11/09/2014 04:55:51",
      "content": "<p>If I do my own feature engineering, can I just pass the csv files with my new features into this script and run in with vw? Thank you</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 57570,
      "author_name": "fabiogabriel",
      "author_url": "",
      "post_date": "11/10/2014 01:18:32",
      "content": "<p>[quote=Charles;57497]</p>\n<p>I'm curious about VW's speed? On what machine (e.g., number of CPU cores, amount of RAM) did you run VW? And how much time did the training take (not including pre-processing the data)?&nbsp;</p>\n<p>Thanks.</p>\n<p>[/quote]</p>\n<p>I've run a VW code similar to that on a Macbook Pro with a 2.6 GHz i5 processor (4 cores) and 8GB of RAM and it took about 2 and a half hours.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 318845,
      "author_name": "sandiemann",
      "author_url": "",
      "post_date": "04/24/2018 15:23:07",
      "content": "<p>Hey @Xueer,</p>\n\n<p>why you didn't use numerical_features to vw file.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "57337": "",
    "57340": "",
    "57363": "",
    "57440": "",
    "57442": "",
    "57454": "",
    "57456": "",
    "57479": "",
    "57497": "",
    "57540": "",
    "57570": "",
    "318845": "Hey @Xueer,\n\nwhy you didn't use numerical_features to vw file."
  },
  "source": "meta"
}