{
  "id": 24328,
  "title": "AUC 99.94",
  "url": "/competitions/bosch-production-line-performance/discussion/24328",
  "author_name": "",
  "post_date": "2016-10-12T14:16:00.607Z",
  "votes": null,
  "comment_count": 2,
  "views": 597,
  "content": "<p>Hi guys,</p>\n\n<p>Addition of just one feature in my model takes my training AUC to 99.94 (from 89.xx) and corresponding MCC to .87 (from ~.39xx). This is clearly an overfit and LB confirms that too (MCC 0.25). If I remove that variable I fall back to more 'acceptable' range :)</p>\n\n<p>Is it fair to assume that this feature has prediction power but is more suited towards the training data? What next steps/changes would you recommend to make it suitable for training data?</p>",
  "messages": [
    {
      "id": "139092",
      "postDate": "10/12/2016 14:16:00",
      "content": "<p>Hi guys,</p>\n\n<p>Addition of just one feature in my model takes my training AUC to 99.94 (from 89.xx) and corresponding MCC to .87 (from ~.39xx). This is clearly an overfit and LB confirms that too (MCC 0.25). If I remove that variable I fall back to more 'acceptable' range :)</p>\n\n<p>Is it fair to assume that this feature has prediction power but is more suited towards the training data? What next steps/changes would you recommend to make it suitable for training data?</p>",
      "rawMarkdown": "Hi guys,\r\n\r\nAddition of just one feature in my model takes my training AUC to 99.94 (from 89.xx) and corresponding MCC to .87 (from ~.39xx). This is clearly an overfit and LB confirms that too (MCC 0.25). If I remove that variable I fall back to more 'acceptable' range :)\r\n\r\nIs it fair to assume that this feature has prediction power but is more suited towards the training data? What next steps/changes would you recommend to make it suitable for training data?",
      "votes": null
    },
    {
      "id": "139093",
      "postDate": "10/12/2016 14:23:33",
      "content": "<p>Hi, \nit sounds more like your accidently leaking information about the response in your feature. This extreme difference should not occur otherwise! I would check if you use the response in some way and if yes you should check if you do the cross validation properly and use in each fold only the information available in this fold.</p>",
      "rawMarkdown": "Hi, \r\nit sounds more like your accidently leaking information about the response in your feature. This extreme difference should not occur otherwise! I would check if you use the response in some way and if yes you should check if you do the cross validation properly and use in each fold only the information available in this fold.",
      "votes": null
    },
    {
      "id": "139099",
      "postDate": "10/12/2016 14:33:26",
      "content": "<p>Hi Malte,</p>\n\n<p>You are very close to what I think I did. Thanks for the suggestion.</p>",
      "rawMarkdown": "Hi Malte,\r\n\r\nYou are very close to what I think I did. Thanks for the suggestion.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 139093,
      "author_name": "malten",
      "author_url": "",
      "post_date": "10/12/2016 14:23:33",
      "content": "<p>Hi, \nit sounds more like your accidently leaking information about the response in your feature. This extreme difference should not occur otherwise! I would check if you use the response in some way and if yes you should check if you do the cross validation properly and use in each fold only the information available in this fold.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 139099,
      "author_name": "manavg123",
      "author_url": "",
      "post_date": "10/12/2016 14:33:26",
      "content": "<p>Hi Malte,</p>\n\n<p>You are very close to what I think I did. Thanks for the suggestion.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "139092": "Hi guys,\r\n\r\nAddition of just one feature in my model takes my training AUC to 99.94 (from 89.xx) and corresponding MCC to .87 (from ~.39xx). This is clearly an overfit and LB confirms that too (MCC 0.25). If I remove that variable I fall back to more 'acceptable' range :)\r\n\r\nIs it fair to assume that this feature has prediction power but is more suited towards the training data? What next steps/changes would you recommend to make it suitable for training data?",
    "139093": "Hi, \r\nit sounds more like your accidently leaking information about the response in your feature. This extreme difference should not occur otherwise! I would check if you use the response in some way and if yes you should check if you do the cross validation properly and use in each fold only the information available in this fold.",
    "139099": "Hi Malte,\r\n\r\nYou are very close to what I think I did. Thanks for the suggestion."
  },
  "source": "meta"
}