{
  "id": 68257,
  "title": "Decrease in variance of losses for Group K Fold Validation?",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/68257",
  "author_name": "",
  "post_date": "2018-10-10T19:50:40.017056400Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hi Kagglers,</p>\n\n<p>After paying attention to feature engineering, the variance of losses on validation sets decreased(Using Group 5 fold) Example: Numbers are arbitrary.</p>\n\n<p>My previous losses on 5 folds:\n[1.5780,1.5980,1.5834,1.5697,1.5923]\nMy current losses on 5 fold:\n[1.5757,1.5812,1.5854,1.5802,1.5786]</p>\n\n<p>As you notice, the variance of losses decreased. How do you interpret this decrease?</p>",
  "messages": [
    {
      "id": "401864",
      "postDate": "10/10/2018 19:50:40",
      "content": "<p>Hi Kagglers,</p>\n\n<p>After paying attention to feature engineering, the variance of losses on validation sets decreased(Using Group 5 fold) Example: Numbers are arbitrary.</p>\n\n<p>My previous losses on 5 folds:\n[1.5780,1.5980,1.5834,1.5697,1.5923]\nMy current losses on 5 fold:\n[1.5757,1.5812,1.5854,1.5802,1.5786]</p>\n\n<p>As you notice, the variance of losses decreased. How do you interpret this decrease?</p>",
      "rawMarkdown": "Hi Kagglers,\n\nAfter paying attention to feature engineering, the variance of losses on validation sets decreased(Using Group 5 fold) Example: Numbers are arbitrary.\n\nMy previous losses on 5 folds:\n[1.5780,1.5980,1.5834,1.5697,1.5923]\nMy current losses on 5 fold:\n[1.5757,1.5812,1.5854,1.5802,1.5786]\n\nAs you notice, the variance of losses decreased. How do you interpret this decrease?",
      "votes": null
    },
    {
      "id": "402622",
      "postDate": "10/12/2018 02:36:28",
      "content": "<p>If you are validation your model on different subsets of out-of-sample data... I would take it as random error, you can find more randomness from more different froups of subsets of out-of-sample data.</p>\n\n<p>If you have changed the parameters or model - it seems current is better than previous model and parameters.</p>",
      "rawMarkdown": "If you are validation your model on different subsets of out-of-sample data... I would take it as random error, you can find more randomness from more different froups of subsets of out-of-sample data.\n\nIf you have changed the parameters or model - it seems current is better than previous model and parameters.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 402622,
      "author_name": "hemantrupani",
      "author_url": "",
      "post_date": "10/12/2018 02:36:28",
      "content": "<p>If you are validation your model on different subsets of out-of-sample data... I would take it as random error, you can find more randomness from more different froups of subsets of out-of-sample data.</p>\n\n<p>If you have changed the parameters or model - it seems current is better than previous model and parameters.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "401864": "Hi Kagglers,\n\nAfter paying attention to feature engineering, the variance of losses on validation sets decreased(Using Group 5 fold) Example: Numbers are arbitrary.\n\nMy previous losses on 5 folds:\n[1.5780,1.5980,1.5834,1.5697,1.5923]\nMy current losses on 5 fold:\n[1.5757,1.5812,1.5854,1.5802,1.5786]\n\nAs you notice, the variance of losses decreased. How do you interpret this decrease?",
    "402622": "If you are validation your model on different subsets of out-of-sample data... I would take it as random error, you can find more randomness from more different froups of subsets of out-of-sample data.\n\nIf you have changed the parameters or model - it seems current is better than previous model and parameters."
  },
  "source": "meta"
}