{
  "id": 11490,
  "title": "accuracy vs. nll",
  "url": "/competitions/datasciencebowl/discussion/11490",
  "author_name": "",
  "post_date": "2015-01-11T18:35:22.703Z",
  "votes": 1,
  "comment_count": 2,
  "views": 1361,
  "content": "<p>I am using SVM for classification and then converting its output to probabilities using a variety of techniques before evaluating nll. For an accuracy (121 class) of 73% I am getting an nll of 1.09 but I have a feeling my probability conversion techniques is sub-optimal and ideally I should get much better nll than what I have now.</p>\n<p>Do you mind sharing your nll and corresponding accuracy for reference?</p>",
  "messages": [
    {
      "id": "62610",
      "postDate": "01/11/2015 18:35:22",
      "content": "<p>I am using SVM for classification and then converting its output to probabilities using a variety of techniques before evaluating nll. For an accuracy (121 class) of 73% I am getting an nll of 1.09 but I have a feeling my probability conversion techniques is sub-optimal and ideally I should get much better nll than what I have now.</p>\n<p>Do you mind sharing your nll and corresponding accuracy for reference?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62632",
      "postDate": "01/12/2015 05:42:38",
      "content": "<p>Try Platt's scaling?</p>\n<p>I am curious to ask why not directly optimizing the Softmax loss (nll) as it is also the evaluation metric?</p>\n<p>In Tang's paper using L2-SVM on top of CNN instead of Softmax, it did get lower test error (higher accuracy), but interestingly it leads to a HIGHER softmax loss (see Table 3). It argues that the again of accuracy is largely due to a better objective function (L2-SVM in this case).</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "62633",
      "postDate": "01/12/2015 06:00:18",
      "content": "<p>Thanks for your response. I did try logloss but got worse results. I am not using CNN at all. I am doing unsupervised feature learning + SVM (pegasos algorithm).</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 62632,
      "author_name": "chenglongchen",
      "author_url": "",
      "post_date": "01/12/2015 05:42:38",
      "content": "<p>Try Platt's scaling?</p>\n<p>I am curious to ask why not directly optimizing the Softmax loss (nll) as it is also the evaluation metric?</p>\n<p>In Tang's paper using L2-SVM on top of CNN instead of Softmax, it did get lower test error (higher accuracy), but interestingly it leads to a HIGHER softmax loss (see Table 3). It argues that the again of accuracy is largely due to a better objective function (L2-SVM in this case).</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 62633,
      "author_name": "blackeagle",
      "author_url": "",
      "post_date": "01/12/2015 06:00:18",
      "content": "<p>Thanks for your response. I did try logloss but got worse results. I am not using CNN at all. I am doing unsupervised feature learning + SVM (pegasos algorithm).</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "62610": "",
    "62632": "",
    "62633": ""
  },
  "source": "meta"
}