{
  "id": 25406,
  "title": "Imbalanced Classes",
  "url": "/competitions/bosch-production-line-performance/discussion/25406",
  "author_name": "",
  "post_date": "2016-11-13T19:29:01.417Z",
  "votes": null,
  "comment_count": 2,
  "views": 311,
  "content": "<p>How did people handle the heavily imbalanced classes? Through downsampling?</p>",
  "messages": [
    {
      "id": "144329",
      "postDate": "11/13/2016 19:29:01",
      "content": "<p>How did people handle the heavily imbalanced classes? Through downsampling?</p>",
      "rawMarkdown": "How did people handle the heavily imbalanced classes? Through downsampling?",
      "votes": null
    },
    {
      "id": "144335",
      "postDate": "11/13/2016 19:47:19",
      "content": "<p>We had a little luck upweighting the 1s by a factor of 3 or so in xgboost.  You can do this with either weights or the scale_pos_weight parameter; both ways seem to be very similar if not identical.   I haven't tried to look into the code to verify--if anyone knows for sure if these are the same or different please chime in. </p>",
      "rawMarkdown": "We had a little luck upweighting the 1s by a factor of 3 or so in xgboost.  You can do this with either weights or the scale_pos_weight parameter; both ways seem to be very similar if not identical.   I haven't tried to look into the code to verify--if anyone knows for sure if these are the same or different please chime in.",
      "votes": null
    },
    {
      "id": "144346",
      "postDate": "11/13/2016 22:32:58",
      "content": "<p>We found balancing the positive and negative samples is crucial for training neural networks, especially when it's trained by SGD in mini-batches where you might end up having no positive samples in some batches. We oversampled the positive class by 1/0.005811 in the training set but kept the validation set in the original proportions. An ensemble of 50 NNs gave us ~0.45 MCC in the public LB. As for tree models our results did not seem to be sensitive to whether the data is balanced or not. We tweaked scale_pos_weight but it only made xgboost converges faster, we did not see any improvements.</p>\n\n<p>Perhaps that was part of the reason we were failing behind because we didn't pay enough attention to balancing things out.</p>",
      "rawMarkdown": "We found balancing the positive and negative samples is crucial for training neural networks, especially when it's trained by SGD in mini-batches where you might end up having no positive samples in some batches. We oversampled the positive class by 1/0.005811 in the training set but kept the validation set in the original proportions. An ensemble of 50 NNs gave us ~0.45 MCC in the public LB. As for tree models our results did not seem to be sensitive to whether the data is balanced or not. We tweaked scale_pos_weight but it only made xgboost converges faster, we did not see any improvements.\r\n\r\nPerhaps that was part of the reason we were failing behind because we didn't pay enough attention to balancing things out.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 144335,
      "author_name": "sasrdw",
      "author_url": "",
      "post_date": "11/13/2016 19:47:19",
      "content": "<p>We had a little luck upweighting the 1s by a factor of 3 or so in xgboost.  You can do this with either weights or the scale_pos_weight parameter; both ways seem to be very similar if not identical.   I haven't tried to look into the code to verify--if anyone knows for sure if these are the same or different please chime in. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 144346,
      "author_name": "yuxipan",
      "author_url": "",
      "post_date": "11/13/2016 22:32:58",
      "content": "<p>We found balancing the positive and negative samples is crucial for training neural networks, especially when it's trained by SGD in mini-batches where you might end up having no positive samples in some batches. We oversampled the positive class by 1/0.005811 in the training set but kept the validation set in the original proportions. An ensemble of 50 NNs gave us ~0.45 MCC in the public LB. As for tree models our results did not seem to be sensitive to whether the data is balanced or not. We tweaked scale_pos_weight but it only made xgboost converges faster, we did not see any improvements.</p>\n\n<p>Perhaps that was part of the reason we were failing behind because we didn't pay enough attention to balancing things out.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "144329": "How did people handle the heavily imbalanced classes? Through downsampling?",
    "144335": "We had a little luck upweighting the 1s by a factor of 3 or so in xgboost.  You can do this with either weights or the scale_pos_weight parameter; both ways seem to be very similar if not identical.   I haven't tried to look into the code to verify--if anyone knows for sure if these are the same or different please chime in.",
    "144346": "We found balancing the positive and negative samples is crucial for training neural networks, especially when it's trained by SGD in mini-batches where you might end up having no positive samples in some batches. We oversampled the positive class by 1/0.005811 in the training set but kept the validation set in the original proportions. An ensemble of 50 NNs gave us ~0.45 MCC in the public LB. As for tree models our results did not seem to be sensitive to whether the data is balanced or not. We tweaked scale_pos_weight but it only made xgboost converges faster, we did not see any improvements.\r\n\r\nPerhaps that was part of the reason we were failing behind because we didn't pay enough attention to balancing things out."
  },
  "source": "meta"
}