I'm trying to write the metric for xgb (for the Scikit-Learn API).
\nCurrently, I don't know how to retrieve the weights in my function.
\nThe signature of the xgb metric is: myMetric(preds, dtrain) (no weights).
So I've tried to create a class and pass the weights into the constructor but I don't know how to \"filter\" them with the indexes of the subset used in the evaluation.
\nAnother solution could be to extract them from dtrain (but I failed to read values in a DMatrix).
Do you have an idea on how to perform that?
\n\nMy code:
\n\nclass NWRMSLE:\n def __init__(self, weights):\n self.weights = weights\n\n def NWRMSLE(self, a, p, w):\n a = np.array(a)\n p = np.array(p)\n w = np.array(w)\n weighted_errors = np.dot(np.square(np.log1p(p) - np.log1p(a)), np.transpose(w))\n weights_sum = np.sum(w)\n return math.sqrt(weighted_errors/weights_sum)\n\n def NWRMSLE_xgb(self, preds, dtrain):\n labels = dtrain.get_label()\n ### HERE pick the right weights\n w = foo(self.weights)\n ###\n ### OR extract them from dtrain\n w = foo(dtrain)\n ###\n NWRMSLE_score = self.NWRMSLE(labels, preds, w)\n return [('NWRMSLE', NWRMSLE_score)]\n\n\nThanks in advance! :-)
", "messages": [ { "id": "236445", "postDate": "10/27/2017 11:31:23", "content": "I'm trying to write the metric for xgb (for the Scikit-Learn API).
\nCurrently, I don't know how to retrieve the weights in my function.
\nThe signature of the xgb metric is: myMetric(preds, dtrain) (no weights).
So I've tried to create a class and pass the weights into the constructor but I don't know how to \"filter\" them with the indexes of the subset used in the evaluation.
\nAnother solution could be to extract them from dtrain (but I failed to read values in a DMatrix).
Do you have an idea on how to perform that?
\n\nMy code:
\n\nclass NWRMSLE:\n def __init__(self, weights):\n self.weights = weights\n\n def NWRMSLE(self, a, p, w):\n a = np.array(a)\n p = np.array(p)\n w = np.array(w)\n weighted_errors = np.dot(np.square(np.log1p(p) - np.log1p(a)), np.transpose(w))\n weights_sum = np.sum(w)\n return math.sqrt(weighted_errors/weights_sum)\n\n def NWRMSLE_xgb(self, preds, dtrain):\n labels = dtrain.get_label()\n ### HERE pick the right weights\n w = foo(self.weights)\n ###\n ### OR extract them from dtrain\n w = foo(dtrain)\n ###\n NWRMSLE_score = self.NWRMSLE(labels, preds, w)\n return [('NWRMSLE', NWRMSLE_score)]\n\n\nThanks in advance! :-)
", "rawMarkdown": "I'm trying to write the metric for xgb (for the Scikit-Learn API). \nCurrently, I don't know how to retrieve the weights in my function. \nThe signature of the xgb metric is: `myMetric(preds, dtrain)` (no weights). \n\nSo I've tried to create a class and pass the weights into the constructor but I don't know how to \"filter\" them with the indexes of the subset used in the evaluation. \nAnother solution could be to extract them from dtrain (but I failed to read values in a DMatrix).\n\nDo you have an idea on how to perform that?\n\nMy code:\nclass NWRMSLE:\n def __init__(self, weights):\n self.weights = weights\n\n def NWRMSLE(self, a, p, w):\n a = np.array(a)\n p = np.array(p)\n w = np.array(w)\n weighted_errors = np.dot(np.square(np.log1p(p) - np.log1p(a)), np.transpose(w))\n weights_sum = np.sum(w)\n return math.sqrt(weighted_errors/weights_sum)\n \n def NWRMSLE_xgb(self, preds, dtrain):\n labels = dtrain.get_label()\n ### HERE pick the right weights\n w = foo(self.weights)\n ###\n ### OR extract them from dtrain\n w = foo(dtrain)\n ###\n NWRMSLE_score = self.NWRMSLE(labels, preds, w)\n return [('NWRMSLE', NWRMSLE_score)]\n\n\nThanks in advance! :-)",
"votes": null
},
{
"id": "236453",
"postDate": "10/27/2017 12:04:26",
"content": "Have you thought about doing two separate models: one for perishables and one without?
\n\nIn another thread someone suggested doing a log+1 transform to the dependent variable, build the model, and then do a exp-1 transform on the prediction. This would allow you to use RMSE if you ignore weights, which, I think, you can do if you build two separate models from the start.
\n\nMy take away from the weights is that the company doing this competition cares more about models that accurately predict sales for their perishable goods. Another alternative for them would have been to add a shelf life metric to their data set.
", "rawMarkdown": "Have you thought about doing two separate models: one for perishables and one without? \n\nIn another thread someone suggested doing a log+1 transform to the dependent variable, build the model, and then do a exp-1 transform on the prediction. This would allow you to use RMSE if you ignore weights, which, I think, you can do if you build two separate models from the start. \n\nMy take away from the weights is that the company doing this competition cares more about models that accurately predict sales for their perishable goods. Another alternative for them would have been to add a shelf life metric to their data set.", "votes": null }, { "id": "236520", "postDate": "10/27/2017 15:03:38", "content": "Thanks for your inputs.\nI've found a solution. I'm dumb (or too tired ^^). I just have to give to my constructor the right array of weights. :-/
", "rawMarkdown": "Thanks for your inputs.\nI've found a solution. I'm dumb (or too tired ^^). I just have to give to my constructor the right array of weights. :-/", "votes": null }, { "id": "236522", "postDate": "10/27/2017 15:07:46", "content": "OK, here is a solution:
\n\nclass NWRMSLE:\n def set_weights(self, weights):\n self.weights = weights\n\n def _NWRMSLE(self, a, p, w):\n a = np.array(a)\n a[a < 0.0]=0.0\n p = np.array(p)\n p[p < 0.0]=0.0\n w = np.array(w)\n weighted_errors = np.dot(np.square(np.log1p(p) - np.log1p(a)), np.transpose(w))\n weights_sum = np.sum(w)\n return math.sqrt(weighted_errors/weights_sum)\n\n def NWRMSLE_xgb(self, preds, dtrain):\n labels = dtrain.get_label()\n NWRMSLE_score = self._NWRMSLE(labels, preds, self.weights)\n return [('NWRMSLE', NWRMSLE_score)]\n\n\nYou can use it like that:
\n\n_X_train, _X_cv = X[train_index], X[cv_index]\n_y_train, _y_cv = y[train_index], y[cv_index]\n_w_cv = w[cv_index]\neval_metric = NWRMSLE()\neval_metric.set_weights(_w_cv)\nclf.fit(_X_train, _y_train, eval_set=[(_X_cv, _y_cv)], eval_metric=eval_metric.NWRMSLE_xgb, early_stopping_rounds=...\n", "rawMarkdown": "OK, here is a solution:\n
class NWRMSLE:\n def set_weights(self, weights):\n self.weights = weights\n \n def _NWRMSLE(self, a, p, w):\n a = np.array(a)\n a[a < 0.0]=0.0\n p = np.array(p)\n p[p < 0.0]=0.0\n w = np.array(w)\n weighted_errors = np.dot(np.square(np.log1p(p) - np.log1p(a)), np.transpose(w))\n weights_sum = np.sum(w)\n return math.sqrt(weighted_errors/weights_sum)\n \n def NWRMSLE_xgb(self, preds, dtrain):\n labels = dtrain.get_label()\n NWRMSLE_score = self._NWRMSLE(labels, preds, self.weights)\n return [('NWRMSLE', NWRMSLE_score)]\n\nYou can use it like that: \n_X_train, _X_cv = X[train_index], X[cv_index]\n_y_train, _y_cv = y[train_index], y[cv_index]\n_w_cv = w[cv_index]\neval_metric = NWRMSLE()\neval_metric.set_weights(_w_cv)\nclf.fit(_X_train, _y_train, eval_set=[(_X_cv, _y_cv)], eval_metric=eval_metric.NWRMSLE_xgb, early_stopping_rounds=...\n", "votes": null }, { "id": "239318", "postDate": "11/03/2017 06:54:24", "content": "
I pass the perishable columns. Is that correct ?
\n\neval_metric = NWRMSLE()\neval_metric.set_weights(X_test.perishable) #perishable being the weights\nmodel_sklearn_xgb.fit(X_train, y_train,\n eval_set=eval_set,\n verbose=True,\n eval_metric = NWRMSLE_xgb,\n early_stopping_rounds=20)\nI think you can set sample weights in xgboost.DMatrix.
", "rawMarkdown": "I think you can [set sample weights in xgboost.DMatrix](http://xgboost.readthedocs.io/en/latest/python/python_api.html#xgboost.DMatrix).", "votes": null }, { "id": "239359", "postDate": "11/03/2017 09:24:05", "content": "It's the right column but not the right DataFrame. You have to use the perishable column of you eval_set.
\nSomething like: eval_set[0][0].perishable
\nIn my code : X_train['perishable'].iloc[cv_index] where cv_index is the index used for my eval_set.
Yes but I'm not sure it is possible to calculate: np.square(np.log1p(p) - np.log1p(a)) for a DMatrix (and then set sample weights on this result)
If you log transform the target, you might be able to use built-in MSE metric instead of building a custom one.
", "rawMarkdown": "If you log transform the target, you might be able to use built-in MSE metric instead of building a custom one.", "votes": null }, { "id": "239380", "postDate": "11/03/2017 10:00:08", "content": "I'm not sure to understand how do you combine the built-in MSE and the set_weight.
\nAs Brian says above:
\n\n\nThis would allow you to use RMSE if you ignore weights
\n
Do you have a piece of code to illustrate that?
", "rawMarkdown": "I'm not sure to understand how do you combine the built-in MSE and the set_weight. \nAs Brian says above: \n> This would allow you to use RMSE if you ignore weights \n\nDo you have a piece of code to illustrate that?", "votes": null }, { "id": "239389", "postDate": "11/03/2017 10:23:54", "content": "thanks for your help
\n\nX_train, X_test, y_train, y_test = train_test_split(df_train[colsToUse], target_train, test_size=0.33, random_state=7)
\n\neval_set = [(X_test, y_test)]
\n\nso i guess X_test.perishable is the right one, isn't it ?
", "rawMarkdown": "thanks for your help\n\nX_train, X_test, y_train, y_test = train_test_split(df_train[colsToUse], target_train, test_size=0.33, random_state=7)\n\neval_set = [(X_test, y_test)]\n\nso i guess X_test.perishable is the right one, isn't it ?", "votes": null }, { "id": "239394", "postDate": "11/03/2017 10:38:21", "content": "Yes. Perfect!
\nI thought that X_test was loaded from test.csv. :-)
Sorry for not being clear. The setup I'm trying to describe looks like this:
\n\nThis should get you what you want. (I think.)
\n\nI haven't tried it in XGBoost. But I've done it in LightGBM. I checked the values given by LightGBM against those given by scikit-learn and the they appears to be weighted correctly.
\n\nNonetheless, setting weights in training didn't make much difference in LB results. So that's a bummer. (Maybe I just screwed up somewhere.)
\n\nPlease feel free to correct me if I misunderstood the objective...
", "rawMarkdown": "Sorry for not being clear. The setup I'm trying to describe looks like this:\n\n1. Log transform the target variable\n2. Set weights when you initialize DMatrix\n3. Set eval_metric = \"rmse\" when training the model\n\nThis should get you what you want. (I think.)\n\nI haven't tried it in XGBoost. But I've done it in LightGBM. I checked the values given by LightGBM against those given by scikit-learn and the they appears to be weighted correctly.\n\nNonetheless, setting weights in training didn't make much difference in LB results. So that's a bummer. (Maybe I just screwed up somewhere.)\n\nPlease feel free to correct me if I misunderstood the objective...", "votes": null }, { "id": "239399", "postDate": "11/03/2017 10:56:12", "content": "Ah, I see. You're trying to use XGBoost's scikit-learn API, so you won't be initializing a DMatrix... (Somehow I failed to read the first line of the post) I'd recommend you take a look at XGBoost.train. It is the function I had in mind.
", "rawMarkdown": "Ah, I see. You're trying to use XGBoost's scikit-learn API, so you won't be initializing a DMatrix... (**Somehow I failed to read the first line of the post**) I'd recommend you take a look at [XGBoost.train](https://xgboost.readthedocs.io/en/latest//python/python_api.html#xgboost.train). It is the function I had in mind.", "votes": null }, { "id": "251397", "postDate": "12/01/2017 05:38:28", "content": "@Ceshine Lee Yes, it's great idea to including the weight while define DMatrix, somehow, the feval function how to include the weights from DMatrix? any example code for it? Thanks in advance!
", "rawMarkdown": "Ceshine Lee Yes, it's great idea to including the weight while define DMatrix, somehow, the feval function how to include the weights from DMatrix? any example code for it? Thanks in advance!", "votes": null }, { "id": "252022", "postDate": "12/02/2017 04:43:52", "content": "@Robin Hoo Sorry I don't know how to include the weights from DMatrix. As I said in the previous message, I misread Eric's original post. What I had in mind was just feeding weighted DMatrix to xgb.train function with built-in MSE loss (you need to log transform the target variable first). It should have the same effect as using a custom evaluation function with scikit-learn API.
", "rawMarkdown": "Robin Hoo Sorry I don't know how to include the weights from DMatrix. As I said in the previous message, I misread Eric's original post. What I had in mind was just feeding weighted DMatrix to xgb.train function with built-in MSE loss (you need to log transform the target variable first). It should have the same effect as using a custom evaluation function with scikit-learn API.", "votes": null } ], "comments": [ { "id": 236453, "author_name": "bdub76", "author_url": "", "post_date": "10/27/2017 12:04:26", "content": "Have you thought about doing two separate models: one for perishables and one without?
\n\nIn another thread someone suggested doing a log+1 transform to the dependent variable, build the model, and then do a exp-1 transform on the prediction. This would allow you to use RMSE if you ignore weights, which, I think, you can do if you build two separate models from the start.
\n\nMy take away from the weights is that the company doing this competition cares more about models that accurately predict sales for their perishable goods. Another alternative for them would have been to add a shelf life metric to their data set.
", "votes": null, "replies": [ { "id": 236520, "author_name": "ericdfr", "author_url": "", "post_date": "10/27/2017 15:03:38", "content": "Thanks for your inputs.\nI've found a solution. I'm dumb (or too tired ^^). I just have to give to my constructor the right array of weights. :-/
", "votes": null, "replies": [] } ] }, { "id": 236522, "author_name": "ericdfr", "author_url": "", "post_date": "10/27/2017 15:07:46", "content": "OK, here is a solution:
\n\nclass NWRMSLE:\n def set_weights(self, weights):\n self.weights = weights\n\n def _NWRMSLE(self, a, p, w):\n a = np.array(a)\n a[a < 0.0]=0.0\n p = np.array(p)\n p[p < 0.0]=0.0\n w = np.array(w)\n weighted_errors = np.dot(np.square(np.log1p(p) - np.log1p(a)), np.transpose(w))\n weights_sum = np.sum(w)\n return math.sqrt(weighted_errors/weights_sum)\n\n def NWRMSLE_xgb(self, preds, dtrain):\n labels = dtrain.get_label()\n NWRMSLE_score = self._NWRMSLE(labels, preds, self.weights)\n return [('NWRMSLE', NWRMSLE_score)]\n\n\nYou can use it like that:
\n\n_X_train, _X_cv = X[train_index], X[cv_index]\n_y_train, _y_cv = y[train_index], y[cv_index]\n_w_cv = w[cv_index]\neval_metric = NWRMSLE()\neval_metric.set_weights(_w_cv)\nclf.fit(_X_train, _y_train, eval_set=[(_X_cv, _y_cv)], eval_metric=eval_metric.NWRMSLE_xgb, early_stopping_rounds=...\n", "votes": null, "replies": [ { "id": 239318, "author_name": "savioz", "author_url": "", "post_date": "11/03/2017 06:54:24", "content": "
I pass the perishable columns. Is that correct ?
\n\neval_metric = NWRMSLE()\neval_metric.set_weights(X_test.perishable) #perishable being the weights\nmodel_sklearn_xgb.fit(X_train, y_train,\n eval_set=eval_set,\n verbose=True,\n eval_metric = NWRMSLE_xgb,\n early_stopping_rounds=20)\nIt's the right column but not the right DataFrame. You have to use the perishable column of you eval_set.
\nSomething like: eval_set[0][0].perishable
\nIn my code : X_train['perishable'].iloc[cv_index] where cv_index is the index used for my eval_set.
thanks for your help
\n\nX_train, X_test, y_train, y_test = train_test_split(df_train[colsToUse], target_train, test_size=0.33, random_state=7)
\n\neval_set = [(X_test, y_test)]
\n\nso i guess X_test.perishable is the right one, isn't it ?
", "votes": null, "replies": [] }, { "id": 239394, "author_name": "ericdfr", "author_url": "", "post_date": "11/03/2017 10:38:21", "content": "Yes. Perfect!
\nI thought that X_test was loaded from test.csv. :-)
I think you can set sample weights in xgboost.DMatrix.
", "votes": null, "replies": [ { "id": 239362, "author_name": "ericdfr", "author_url": "", "post_date": "11/03/2017 09:26:30", "content": "Yes but I'm not sure it is possible to calculate: np.square(np.log1p(p) - np.log1p(a)) for a DMatrix (and then set sample weights on this result)
If you log transform the target, you might be able to use built-in MSE metric instead of building a custom one.
", "votes": null, "replies": [] }, { "id": 239380, "author_name": "ericdfr", "author_url": "", "post_date": "11/03/2017 10:00:08", "content": "I'm not sure to understand how do you combine the built-in MSE and the set_weight.
\nAs Brian says above:
\n\n\nThis would allow you to use RMSE if you ignore weights
\n
Do you have a piece of code to illustrate that?
", "votes": null, "replies": [] }, { "id": 239398, "author_name": "ceshine", "author_url": "", "post_date": "11/03/2017 10:50:51", "content": "Sorry for not being clear. The setup I'm trying to describe looks like this:
\n\nThis should get you what you want. (I think.)
\n\nI haven't tried it in XGBoost. But I've done it in LightGBM. I checked the values given by LightGBM against those given by scikit-learn and the they appears to be weighted correctly.
\n\nNonetheless, setting weights in training didn't make much difference in LB results. So that's a bummer. (Maybe I just screwed up somewhere.)
\n\nPlease feel free to correct me if I misunderstood the objective...
", "votes": null, "replies": [] }, { "id": 239399, "author_name": "ceshine", "author_url": "", "post_date": "11/03/2017 10:56:12", "content": "Ah, I see. You're trying to use XGBoost's scikit-learn API, so you won't be initializing a DMatrix... (Somehow I failed to read the first line of the post) I'd recommend you take a look at XGBoost.train. It is the function I had in mind.
", "votes": null, "replies": [] }, { "id": 251397, "author_name": "robinhoo1973", "author_url": "", "post_date": "12/01/2017 05:38:28", "content": "@Ceshine Lee Yes, it's great idea to including the weight while define DMatrix, somehow, the feval function how to include the weights from DMatrix? any example code for it? Thanks in advance!
", "votes": null, "replies": [] }, { "id": 252022, "author_name": "ceshine", "author_url": "", "post_date": "12/02/2017 04:43:52", "content": "@Robin Hoo Sorry I don't know how to include the weights from DMatrix. As I said in the previous message, I misread Eric's original post. What I had in mind was just feeding weighted DMatrix to xgb.train function with built-in MSE loss (you need to log transform the target variable first). It should have the same effect as using a custom evaluation function with scikit-learn API.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "236445": "I'm trying to write the metric for xgb (for the Scikit-Learn API). \nCurrently, I don't know how to retrieve the weights in my function. \nThe signature of the xgb metric is: `myMetric(preds, dtrain)` (no weights). \n\nSo I've tried to create a class and pass the weights into the constructor but I don't know how to \"filter\" them with the indexes of the subset used in the evaluation. \nAnother solution could be to extract them from dtrain (but I failed to read values in a DMatrix).\n\nDo you have an idea on how to perform that?\n\nMy code:\nclass NWRMSLE:\n def __init__(self, weights):\n self.weights = weights\n\n def NWRMSLE(self, a, p, w):\n a = np.array(a)\n p = np.array(p)\n w = np.array(w)\n weighted_errors = np.dot(np.square(np.log1p(p) - np.log1p(a)), np.transpose(w))\n weights_sum = np.sum(w)\n return math.sqrt(weighted_errors/weights_sum)\n \n def NWRMSLE_xgb(self, preds, dtrain):\n labels = dtrain.get_label()\n ### HERE pick the right weights\n w = foo(self.weights)\n ###\n ### OR extract them from dtrain\n w = foo(dtrain)\n ###\n NWRMSLE_score = self.NWRMSLE(labels, preds, w)\n return [('NWRMSLE', NWRMSLE_score)]\n\n\nThanks in advance! :-)",
"236453": "Have you thought about doing two separate models: one for perishables and one without? \n\nIn another thread someone suggested doing a log+1 transform to the dependent variable, build the model, and then do a exp-1 transform on the prediction. This would allow you to use RMSE if you ignore weights, which, I think, you can do if you build two separate models from the start. \n\nMy take away from the weights is that the company doing this competition cares more about models that accurately predict sales for their perishable goods. Another alternative for them would have been to add a shelf life metric to their data set.",
"236520": "Thanks for your inputs.\nI've found a solution. I'm dumb (or too tired ^^). I just have to give to my constructor the right array of weights. :-/",
"236522": "OK, here is a solution:\nclass NWRMSLE:\n def set_weights(self, weights):\n self.weights = weights\n \n def _NWRMSLE(self, a, p, w):\n a = np.array(a)\n a[a < 0.0]=0.0\n p = np.array(p)\n p[p < 0.0]=0.0\n w = np.array(w)\n weighted_errors = np.dot(np.square(np.log1p(p) - np.log1p(a)), np.transpose(w))\n weights_sum = np.sum(w)\n return math.sqrt(weighted_errors/weights_sum)\n \n def NWRMSLE_xgb(self, preds, dtrain):\n labels = dtrain.get_label()\n NWRMSLE_score = self._NWRMSLE(labels, preds, self.weights)\n return [('NWRMSLE', NWRMSLE_score)]\n\nYou can use it like that: \n_X_train, _X_cv = X[train_index], X[cv_index]\n_y_train, _y_cv = y[train_index], y[cv_index]\n_w_cv = w[cv_index]\neval_metric = NWRMSLE()\neval_metric.set_weights(_w_cv)\nclf.fit(_X_train, _y_train, eval_set=[(_X_cv, _y_cv)], eval_metric=eval_metric.NWRMSLE_xgb, early_stopping_rounds=...\n", "239318": "I pass the perishable columns. Is that correct ?\n\n \n eval_metric = NWRMSLE()\n eval_metric.set_weights(X_test.perishable) #perishable being the weights\n model_sklearn_xgb.fit(X_train, y_train,\n eval_set=eval_set,\n verbose=True,\n eval_metric = NWRMSLE_xgb,\n early_stopping_rounds=20)", "239350": "I think you can [set sample weights in xgboost.DMatrix](http://xgboost.readthedocs.io/en/latest/python/python_api.html#xgboost.DMatrix).", "239359": "It's the right column but not the right DataFrame. You have to use the perishable column of you eval_set. \nSomething like: `eval_set[0][0].perishable` \nIn my code : `X_train['perishable'].iloc[cv_index]` where cv_index is the index used for my eval_set.", "239362": "Yes but I'm not sure it is possible to calculate: `np.square(np.log1p(p) - np.log1p(a))` for a DMatrix (and then set sample weights on this result)", "239374": "If you log transform the target, you might be able to use built-in MSE metric instead of building a custom one.", "239380": "I'm not sure to understand how do you combine the built-in MSE and the set_weight. \nAs Brian says above: \n> This would allow you to use RMSE if you ignore weights \n\nDo you have a piece of code to illustrate that?", "239389": "thanks for your help\n\nX_train, X_test, y_train, y_test = train_test_split(df_train[colsToUse], target_train, test_size=0.33, random_state=7)\n\neval_set = [(X_test, y_test)]\n\nso i guess X_test.perishable is the right one, isn't it ?", "239394": "Yes. Perfect! \nI thought that X_test was loaded from test.csv. :-)", "239398": "Sorry for not being clear. The setup I'm trying to describe looks like this:\n\n1. Log transform the target variable\n2. Set weights when you initialize DMatrix\n3. Set eval_metric = \"rmse\" when training the model\n\nThis should get you what you want. (I think.)\n\nI haven't tried it in XGBoost. But I've done it in LightGBM. I checked the values given by LightGBM against those given by scikit-learn and the they appears to be weighted correctly.\n\nNonetheless, setting weights in training didn't make much difference in LB results. So that's a bummer. (Maybe I just screwed up somewhere.)\n\nPlease feel free to correct me if I misunderstood the objective...", "239399": "Ah, I see. You're trying to use XGBoost's scikit-learn API, so you won't be initializing a DMatrix... (**Somehow I failed to read the first line of the post**) I'd recommend you take a look at [XGBoost.train](https://xgboost.readthedocs.io/en/latest//python/python_api.html#xgboost.train). It is the function I had in mind.", "251397": "Ceshine Lee Yes, it's great idea to including the weight while define DMatrix, somehow, the feval function how to include the weights from DMatrix? any example code for it? Thanks in advance!", "252022": "Robin Hoo Sorry I don't know how to include the weights from DMatrix. As I said in the previous message, I misread Eric's original post. What I had in mind was just feeding weighted DMatrix to xgb.train function with built-in MSE loss (you need to log transform the target variable first). It should have the same effect as using a custom evaluation function with scikit-learn API." }, "source": "meta" }