{
  "id": 886,
  "title": "AUC 0.98",
  "url": "/competitions/SemiSupervisedFeatureLearning/discussion/886",
  "author_name": "",
  "post_date": "2011-09-27T16:19:40.970Z",
  "votes": null,
  "comment_count": 3,
  "views": 2578,
  "content": "<p>Assuming that AUC will be similar between leaderboard data and 'offline' data, how much better is the winner supposed to get given we have 0.98 as sample entry on the leaderboard?</p>\r\n<p>&nbsp;</p>\r\n<p>Dirk</p>",
  "messages": [
    {
      "id": "5665",
      "postDate": "09/27/2011 16:19:40",
      "content": "<p>Assuming that AUC will be similar between leaderboard data and 'offline' data, how much better is the winner supposed to get given we have 0.98 as sample entry on the leaderboard?</p>\r\n<p>&nbsp;</p>\r\n<p>Dirk</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "5668",
      "postDate": "09/27/2011 16:57:38",
      "content": "<p>What is the AUC? </p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "5671",
      "postDate": "09/27/2011 17:02:57",
      "content": "<p>[quote=Aniket;5668]</p>\r\n<p>What is the AUC?</p>\r\n<p>[/quote]</p>\r\n<p>See <a href=\"http://www.kaggle.com/c/GiveMeSomeCredit/forums/t/865/evaluation-function/5543#post5543\">\r\nhere</a></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "5672",
      "postDate": "09/27/2011 17:32:58",
      "content": "<p>I'd expect that good submission methods can significantly improve on the baseline AUC of 0.98, and will likely be in the area of 0.999-something.</p>\r\n<p>It's helpful to think in terms of loss reduction. &nbsp;A classifier with AUC of 0.98 has an &quot;AUC Loss&quot; of 0.02 -- that is, it gets comparisons between a randomly selected positive and a randomly selected negative wrong about 2% of the time. &nbsp;Knocking this down\r\n to 0.02%, for example, would be good.</p>\r\n<p>This is one of the things that we're interested in seeing in this competition. &nbsp;When a simple baseline already gives &quot;good&quot; performance, can semi-supervised feature learning methods give &quot;great&quot; performance?</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 5668,
      "author_name": "",
      "author_url": "",
      "post_date": "09/27/2011 16:57:38",
      "content": "<p>What is the AUC? </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 5671,
      "author_name": "jeffmoser",
      "author_url": "",
      "post_date": "09/27/2011 17:02:57",
      "content": "<p>[quote=Aniket;5668]</p>\r\n<p>What is the AUC?</p>\r\n<p>[/quote]</p>\r\n<p>See <a href=\"http://www.kaggle.com/c/GiveMeSomeCredit/forums/t/865/evaluation-function/5543#post5543\">\r\nhere</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 5672,
      "author_name": "argv",
      "author_url": "",
      "post_date": "09/27/2011 17:32:58",
      "content": "<p>I'd expect that good submission methods can significantly improve on the baseline AUC of 0.98, and will likely be in the area of 0.999-something.</p>\r\n<p>It's helpful to think in terms of loss reduction. &nbsp;A classifier with AUC of 0.98 has an &quot;AUC Loss&quot; of 0.02 -- that is, it gets comparisons between a randomly selected positive and a randomly selected negative wrong about 2% of the time. &nbsp;Knocking this down\r\n to 0.02%, for example, would be good.</p>\r\n<p>This is one of the things that we're interested in seeing in this competition. &nbsp;When a simple baseline already gives &quot;good&quot; performance, can semi-supervised feature learning methods give &quot;great&quot; performance?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "5665": "",
    "5668": "",
    "5671": "",
    "5672": ""
  },
  "source": "meta"
}