{
  "id": 6339,
  "title": " Ways to handle the unbalanced data?",
  "url": "/competitions/expedia-personalized-sort/discussion/6339",
  "author_name": "",
  "post_date": "2013-11-19T14:07:37.517Z",
  "votes": null,
  "comment_count": 1,
  "views": 2503,
  "content": "<p>Since the data is highly unbalanced, what are the best ways to deal with this issue. Right now I am using stratified sampling so that I have almost equal number of 1s and 0s for the booking_bool. This definitely doesn't consider the searches in blocks. I mean for each srch_id, I am sampling just two rows, one with booking_bool value 1 and the other 0. I don't think this is the best way to do it.</p>\n<p>Which sort of methods did you guys try out?&nbsp;</p>\n<p>&nbsp;</p>",
  "messages": [
    {
      "id": "34899",
      "postDate": "11/19/2013 14:07:37",
      "content": "<p>Since the data is highly unbalanced, what are the best ways to deal with this issue. Right now I am using stratified sampling so that I have almost equal number of 1s and 0s for the booking_bool. This definitely doesn't consider the searches in blocks. I mean for each srch_id, I am sampling just two rows, one with booking_bool value 1 and the other 0. I don't think this is the best way to do it.</p>\n<p>Which sort of methods did you guys try out?&nbsp;</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "35473",
      "postDate": "11/28/2013 19:34:06",
      "content": "<p>For some analyses I also sampled a 0 and 1 from each query for speed/memory.</p>\n<p>But apart from that, why would you want to do it? Maybe you think too much in terms of classification. To sort queries, a non-discrete value is better (more like a probability than a class label).</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 35473,
      "author_name": "gertjac",
      "author_url": "",
      "post_date": "11/28/2013 19:34:06",
      "content": "<p>For some analyses I also sampled a 0 and 1 from each query for speed/memory.</p>\n<p>But apart from that, why would you want to do it? Maybe you think too much in terms of classification. To sort queries, a non-discrete value is better (more like a probability than a class label).</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "34899": "",
    "35473": ""
  },
  "source": "meta"
}