{
  "id": 4204,
  "title": "ROC curve in R",
  "url": "/competitions/cause-effect-pairs/discussion/4204",
  "author_name": "",
  "post_date": "2013-04-04T20:44:33.633Z",
  "votes": null,
  "comment_count": 4,
  "views": 2601,
  "content": "<p>Hi</p>\r\n<p>What packages are people using for the AUC stats? I've tried packages &quot;verification&quot; and &quot;ROCR&quot; but they are for binary classes.</p>\r\n<p>thanks</p>",
  "messages": [
    {
      "id": "22185",
      "postDate": "04/04/2013 20:44:33",
      "content": "<p>Hi</p>\r\n<p>What packages are people using for the AUC stats? I've tried packages &quot;verification&quot; and &quot;ROCR&quot; but they are for binary classes.</p>\r\n<p>thanks</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "22188",
      "postDate": "04/04/2013 21:53:58",
      "content": "<p>You don't need special code for multiclass, we average the AUC results for 2 binary classification problems:</p>\r\n<p>Let Yhat be your predicted score in [-Inf, &#43;Inf] and Y the target values in {-1, 0 1}.&nbsp;<br>\r\nWe define Y1=Y; Y1(Y==0)=-1; and Y2=Y; Y2(Y==0)=&#43;1;&nbsp;<br>\r\nScore = 0.5*(AUC(Yhat, Y1)&#43;AUC(Yhat, Y2));&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "22229",
      "postDate": "04/05/2013 10:35:37",
      "content": "<p>Thanks but the language is too formal for me to understand.</p>\r\n<p>Do you mean you remove all -1 predictions and just do auc{0,1) then remove &#43;1 and do AUC{0,-1) then take the average.\r\n</p>\r\n<p>&nbsp;</p>\r\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "22238",
      "postDate": "04/05/2013 14:16:54",
      "content": "<p>No, first tou keep all entries, but first you substitute all zeros for -1 in the target vector, then compute the AUC, second you substitute all zeros for &#43;1 in the target vector and compute the AUC again, finally, you average the 2 AUCs.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "22768",
      "postDate": "04/14/2013 05:20:15",
      "content": "<p>It is difficult to think of the definition of multi-class AUC，and I made a mistake from the beginning.<br>\r\n<br>\r\n</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 22188,
      "author_name": "iguyon",
      "author_url": "",
      "post_date": "04/04/2013 21:53:58",
      "content": "<p>You don't need special code for multiclass, we average the AUC results for 2 binary classification problems:</p>\r\n<p>Let Yhat be your predicted score in [-Inf, &#43;Inf] and Y the target values in {-1, 0 1}.&nbsp;<br>\r\nWe define Y1=Y; Y1(Y==0)=-1; and Y2=Y; Y2(Y==0)=&#43;1;&nbsp;<br>\r\nScore = 0.5*(AUC(Yhat, Y1)&#43;AUC(Yhat, Y2));&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 22229,
      "author_name": "domcastro",
      "author_url": "",
      "post_date": "04/05/2013 10:35:37",
      "content": "<p>Thanks but the language is too formal for me to understand.</p>\r\n<p>Do you mean you remove all -1 predictions and just do auc{0,1) then remove &#43;1 and do AUC{0,-1) then take the average.\r\n</p>\r\n<p>&nbsp;</p>\r\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 22238,
      "author_name": "iguyon",
      "author_url": "",
      "post_date": "04/05/2013 14:16:54",
      "content": "<p>No, first tou keep all entries, but first you substitute all zeros for -1 in the target vector, then compute the AUC, second you substitute all zeros for &#43;1 in the target vector and compute the AUC again, finally, you average the 2 AUCs.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 22768,
      "author_name": "brucewu",
      "author_url": "",
      "post_date": "04/14/2013 05:20:15",
      "content": "<p>It is difficult to think of the definition of multi-class AUC，and I made a mistake from the beginning.<br>\r\n<br>\r\n</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "22185": "",
    "22188": "",
    "22229": "",
    "22238": "",
    "22768": ""
  },
  "source": "meta"
}