{
  "id": 22820,
  "title": "xgboost not reach optimal value even nround=900",
  "url": "/competitions/grupo-bimbo-inventory-demand/discussion/22820",
  "author_name": "",
  "post_date": "2016-08-12T03:31:31.307Z",
  "votes": null,
  "comment_count": 4,
  "views": 665,
  "content": "<p>I am using cross validation in xgboost and want to find the optimal parameters. However, even given nround=900, the test logloss is still decreasing. If keep growing the nround value, is there another overfitting issue will be caused by? Shall I adjust the eta, max.depth also? </p>\n\n<p>I just attached my code as below. Could you do me a favor to solve this problem? Thank you so much in advance!!</p>\n\n<p>require(xgboost)</p>\n\n<p>dep &lt;- train$dep\ntst &lt;- train[,!names(train) %in% c(&quot;dep&quot;)]\ntst1 &lt;- xgb.DMatrix(as.matrix(sapply(tst, as.numeric)), label=dep) </p>\n\n<p>param &lt;- list(&quot;objective&quot; = &quot;binary:logistic&quot;,\n              &quot;eval_metric&quot; = &quot;logloss&quot;,\n              &quot;eta&quot; = 0.1, &quot;max.depth&quot; = 6)</p>\n\n<p>bst.cv = xgb.cv(param=param, data = tst1,label =dep, nfold = 10, nrounds = 900)</p>",
  "messages": [
    {
      "id": "130995",
      "postDate": "08/12/2016 03:31:31",
      "content": "<p>I am using cross validation in xgboost and want to find the optimal parameters. However, even given nround=900, the test logloss is still decreasing. If keep growing the nround value, is there another overfitting issue will be caused by? Shall I adjust the eta, max.depth also? </p>\n\n<p>I just attached my code as below. Could you do me a favor to solve this problem? Thank you so much in advance!!</p>\n\n<p>require(xgboost)</p>\n\n<p>dep &lt;- train$dep\ntst &lt;- train[,!names(train) %in% c(&quot;dep&quot;)]\ntst1 &lt;- xgb.DMatrix(as.matrix(sapply(tst, as.numeric)), label=dep) </p>\n\n<p>param &lt;- list(&quot;objective&quot; = &quot;binary:logistic&quot;,\n              &quot;eval_metric&quot; = &quot;logloss&quot;,\n              &quot;eta&quot; = 0.1, &quot;max.depth&quot; = 6)</p>\n\n<p>bst.cv = xgb.cv(param=param, data = tst1,label =dep, nfold = 10, nrounds = 900)</p>",
      "rawMarkdown": "I am using cross validation in xgboost and want to find the optimal parameters. However, even given nround=900, the test logloss is still decreasing. If keep growing the nround value, is there another overfitting issue will be caused by? Shall I adjust the eta, max.depth also? \r\n\r\nI just attached my code as below. Could you do me a favor to solve this problem? Thank you so much in advance!!\r\n\r\nrequire(xgboost)\r\n\r\ndep <- train$dep\r\ntst <- train[,!names(train) %in% c(\"dep\")]\r\ntst1 <- xgb.DMatrix(as.matrix(sapply(tst, as.numeric)), label=dep) \r\n\r\nparam <- list(\"objective\" = \"binary:logistic\",\r\n              \"eval_metric\" = \"logloss\",\r\n              \"eta\" = 0.1, \"max.depth\" = 6)\r\n\r\nbst.cv = xgb.cv(param=param, data = tst1,label =dep, nfold = 10, nrounds = 900)",
      "votes": null
    },
    {
      "id": "131075",
      "postDate": "08/12/2016 19:13:41",
      "content": "<p>The way i see it, there are two potential issues:\n1. nround is not that much :-)\n2. I am rather sceptical whether xgb.cv is a good idea here - given that time is involved, you might end up validating performance on week 8 using model fitted on information from week 9, which is a bit like peeking into the future. It's not guaranteed to overfit, but the risk is there.</p>",
      "rawMarkdown": "The way i see it, there are two potential issues:\r\n1. nround is not that much :-)\r\n2. I am rather sceptical whether xgb.cv is a good idea here - given that time is involved, you might end up validating performance on week 8 using model fitted on information from week 9, which is a bit like peeking into the future. It's not guaranteed to overfit, but the risk is there.",
      "votes": null
    },
    {
      "id": "131093",
      "postDate": "08/12/2016 21:46:06",
      "content": "<p>There are two possible reasons I can think of.</p>\n\n<p>(1) you are using some highly risky features, for example the categorical ids (numbers). \n(2) you use some information in the testing data to form features for training data. </p>",
      "rawMarkdown": "There are two possible reasons I can think of.\r\n\r\n(1) you are using some highly risky features, for example the categorical ids (numbers). \r\n(2) you use some information in the testing data to form features for training data.",
      "votes": null
    },
    {
      "id": "131108",
      "postDate": "08/13/2016 00:45:02",
      "content": "<p>@Gillian, I wonder why you used binary:logistic and logloss for this competition? Shouldn't it be better to use reg:linear objective and rmse metric? </p>",
      "rawMarkdown": "Gillian, I wonder why you used binary:logistic and logloss for this competition? Shouldn't it be better to use reg:linear objective and rmse metric?",
      "votes": null
    },
    {
      "id": "131125",
      "postDate": "08/13/2016 08:17:12",
      "content": "<p>Excellent point @aldente, I completely missed it:  this is a regression problem @Gillian, not a classification one =&gt; at that rate the error can go decreasing for a while until it lands in a unit interval :-)</p>",
      "rawMarkdown": "Excellent point @aldente, I completely missed it:  this is a regression problem @Gillian, not a classification one => at that rate the error can go decreasing for a while until it lands in a unit interval :-)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 131075,
      "author_name": "konradb",
      "author_url": "",
      "post_date": "08/12/2016 19:13:41",
      "content": "<p>The way i see it, there are two potential issues:\n1. nround is not that much :-)\n2. I am rather sceptical whether xgb.cv is a good idea here - given that time is involved, you might end up validating performance on week 8 using model fitted on information from week 9, which is a bit like peeking into the future. It's not guaranteed to overfit, but the risk is there.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 131093,
      "author_name": "amazingbob",
      "author_url": "",
      "post_date": "08/12/2016 21:46:06",
      "content": "<p>There are two possible reasons I can think of.</p>\n\n<p>(1) you are using some highly risky features, for example the categorical ids (numbers). \n(2) you use some information in the testing data to form features for training data. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 131108,
      "author_name": "aldente",
      "author_url": "",
      "post_date": "08/13/2016 00:45:02",
      "content": "<p>@Gillian, I wonder why you used binary:logistic and logloss for this competition? Shouldn't it be better to use reg:linear objective and rmse metric? </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 131125,
      "author_name": "konradb",
      "author_url": "",
      "post_date": "08/13/2016 08:17:12",
      "content": "<p>Excellent point @aldente, I completely missed it:  this is a regression problem @Gillian, not a classification one =&gt; at that rate the error can go decreasing for a while until it lands in a unit interval :-)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "130995": "I am using cross validation in xgboost and want to find the optimal parameters. However, even given nround=900, the test logloss is still decreasing. If keep growing the nround value, is there another overfitting issue will be caused by? Shall I adjust the eta, max.depth also? \r\n\r\nI just attached my code as below. Could you do me a favor to solve this problem? Thank you so much in advance!!\r\n\r\nrequire(xgboost)\r\n\r\ndep <- train$dep\r\ntst <- train[,!names(train) %in% c(\"dep\")]\r\ntst1 <- xgb.DMatrix(as.matrix(sapply(tst, as.numeric)), label=dep) \r\n\r\nparam <- list(\"objective\" = \"binary:logistic\",\r\n              \"eval_metric\" = \"logloss\",\r\n              \"eta\" = 0.1, \"max.depth\" = 6)\r\n\r\nbst.cv = xgb.cv(param=param, data = tst1,label =dep, nfold = 10, nrounds = 900)",
    "131075": "The way i see it, there are two potential issues:\r\n1. nround is not that much :-)\r\n2. I am rather sceptical whether xgb.cv is a good idea here - given that time is involved, you might end up validating performance on week 8 using model fitted on information from week 9, which is a bit like peeking into the future. It's not guaranteed to overfit, but the risk is there.",
    "131093": "There are two possible reasons I can think of.\r\n\r\n(1) you are using some highly risky features, for example the categorical ids (numbers). \r\n(2) you use some information in the testing data to form features for training data.",
    "131108": "Gillian, I wonder why you used binary:logistic and logloss for this competition? Shouldn't it be better to use reg:linear objective and rmse metric?",
    "131125": "Excellent point @aldente, I completely missed it:  this is a regression problem @Gillian, not a classification one => at that rate the error can go decreasing for a while until it lands in a unit interval :-)"
  },
  "source": "meta"
}