{
  "id": 67826,
  "title": "Question about Target Encoding and feature in Different Models",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/67826",
  "author_name": "",
  "post_date": "2018-10-05T21:18:43.654851200Z",
  "votes": 4,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi,\nI have 2 questions:</p>\n\n<ul>\n<li><p>Given an LGBM model, let's say I produce a feature that makes the CV score worse,  is there a chance that this feature would be good for a model of a different type? NN for example?\nOr i can generally assume that if score worsens (not stays the same even) so this feature is garbage?</p></li>\n<li><p>I have an LGBM model, I took my most important categorical feature (label encoded), and added Target encoding (mean encoding, freq encoding and NZ mean encoding).\nI simply added these to the model, and it got worse CV.\nI can understand if the model doesn't get better, but it got worse.\nAny thoughts on that?</p></li>\n</ul>\n\n<p>Thanks!!</p>",
  "messages": [
    {
      "id": "399477",
      "postDate": "10/05/2018 21:18:43",
      "content": "<p>Hi,\nI have 2 questions:</p>\n\n<ul>\n<li><p>Given an LGBM model, let's say I produce a feature that makes the CV score worse,  is there a chance that this feature would be good for a model of a different type? NN for example?\nOr i can generally assume that if score worsens (not stays the same even) so this feature is garbage?</p></li>\n<li><p>I have an LGBM model, I took my most important categorical feature (label encoded), and added Target encoding (mean encoding, freq encoding and NZ mean encoding).\nI simply added these to the model, and it got worse CV.\nI can understand if the model doesn't get better, but it got worse.\nAny thoughts on that?</p></li>\n</ul>\n\n<p>Thanks!!</p>",
      "rawMarkdown": "Hi,\nI have 2 questions:\n\n - Given an LGBM model, let's say I produce a feature that makes the CV score worse,  is there a chance that this feature would be good for a model of a different type? NN for example?\nOr i can generally assume that if score worsens (not stays the same even) so this feature is garbage?\n\n - I have an LGBM model, I took my most important categorical feature (label encoded), and added Target encoding (mean encoding, freq encoding and NZ mean encoding).\nI simply added these to the model, and it got worse CV.\nI can understand if the model doesn't get better, but it got worse.\nAny thoughts on that?\n\n\nThanks!!",
      "votes": null
    },
    {
      "id": "399511",
      "postDate": "10/06/2018 00:24:10",
      "content": "<p>If your new feature is correlated to an existing feature or features, then I can see that affecting your score negatively.</p>",
      "rawMarkdown": "If your new feature is correlated to an existing feature or features, then I can see that affecting your score negatively.",
      "votes": null
    },
    {
      "id": "399543",
      "postDate": "10/06/2018 04:12:30",
      "content": "<p>For your first question: I have found there are features that seem to benefit my NN models and even my LGB time-wise models, but don't benefit my straight-up LGB models. I believe this is a good thing as it creates a good ensemble.</p>",
      "rawMarkdown": "For your first question: I have found there are features that seem to benefit my NN models and even my LGB time-wise models, but don't benefit my straight-up LGB models. I believe this is a good thing as it creates a good ensemble.",
      "votes": null
    },
    {
      "id": "399722",
      "postDate": "10/06/2018 15:23:45",
      "content": "<p>As for the second question, simple target encoding without regularisation is prong to data leakage (especially for low-frequency high-cardinality features, like what we have), thus CV score can go down. Alternatively, it can be that label-encoding simply works better (especially in the case, when categorical features do not play a big role this can happen due to flucs in numerical-feature treatment)</p>",
      "rawMarkdown": "As for the second question, simple target encoding without regularisation is prong to data leakage (especially for low-frequency high-cardinality features, like what we have), thus CV score can go down. Alternatively, it can be that label-encoding simply works better (especially in the case, when categorical features do not play a big role this can happen due to flucs in numerical-feature treatment)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 399511,
      "author_name": "robertkag",
      "author_url": "",
      "post_date": "10/06/2018 00:24:10",
      "content": "<p>If your new feature is correlated to an existing feature or features, then I can see that affecting your score negatively.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 399543,
      "author_name": "jpmiller",
      "author_url": "",
      "post_date": "10/06/2018 04:12:30",
      "content": "<p>For your first question: I have found there are features that seem to benefit my NN models and even my LGB time-wise models, but don't benefit my straight-up LGB models. I believe this is a good thing as it creates a good ensemble.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 399722,
      "author_name": "mlisovyi",
      "author_url": "",
      "post_date": "10/06/2018 15:23:45",
      "content": "<p>As for the second question, simple target encoding without regularisation is prong to data leakage (especially for low-frequency high-cardinality features, like what we have), thus CV score can go down. Alternatively, it can be that label-encoding simply works better (especially in the case, when categorical features do not play a big role this can happen due to flucs in numerical-feature treatment)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "399477": "Hi,\nI have 2 questions:\n\n - Given an LGBM model, let's say I produce a feature that makes the CV score worse,  is there a chance that this feature would be good for a model of a different type? NN for example?\nOr i can generally assume that if score worsens (not stays the same even) so this feature is garbage?\n\n - I have an LGBM model, I took my most important categorical feature (label encoded), and added Target encoding (mean encoding, freq encoding and NZ mean encoding).\nI simply added these to the model, and it got worse CV.\nI can understand if the model doesn't get better, but it got worse.\nAny thoughts on that?\n\n\nThanks!!",
    "399511": "If your new feature is correlated to an existing feature or features, then I can see that affecting your score negatively.",
    "399543": "For your first question: I have found there are features that seem to benefit my NN models and even my LGB time-wise models, but don't benefit my straight-up LGB models. I believe this is a good thing as it creates a good ensemble.",
    "399722": "As for the second question, simple target encoding without regularisation is prong to data leakage (especially for low-frequency high-cardinality features, like what we have), thus CV score can go down. Alternatively, it can be that label-encoding simply works better (especially in the case, when categorical features do not play a big role this can happen due to flucs in numerical-feature treatment)"
  },
  "source": "meta"
}