{
  "id": 6117,
  "title": "KNN, K=1 misclassification rate",
  "url": "/competitions/digit-recognizer/discussion/6117",
  "author_name": "Stefan Zobel",
  "post_date": "2013-10-25T17:03:38.783000",
  "votes": 0,
  "comment_count": 0,
  "views": null,
  "content": "<p>Using the simplest k=1 nearest neighbor algorithm on the train set (classifying each one of the 42000 train instances against the remaining 39999 train instances)&nbsp;results in&nbsp;a misclassification rate of&nbsp;3,13 %.</p>\n<p>&nbsp;</p>\n<p>Is this a plausible magnitude or are there further considerations&nbsp;I should take into account (given that the goal is to estimate the performance of the algorithm)?</p>\n<p>&nbsp;</p>\n<p>Further, first experiments indicate that the misclassification rate doesn't seem to improve for k &gt; 1. Do you have any other suggestions in the context of KNN, how performance could be improved?</p>\n<p>&nbsp;</p>\n<p>&nbsp;EDIT:</p>\n<p>I was wrong about the insignificance of inspecting more neighbors. For a very simple weighting scheme where each class is assigned the sum of the inverse distances of its members I see a small improvement in the error rate in the range k = 3..8. The lowest train set error rate is 2,74% for k= 4. Maybe it's possible to reduce this even further by choosing a more sophisticated weighting scheme (e.g. exponential)? Thoughts?</p>",
  "messages": [
    {
      "id": 32683,
      "postDate": "2013-10-25T17:03:38.783Z",
      "content": "<p>Using the simplest k=1 nearest neighbor algorithm on the train set (classifying each one of the 42000 train instances against the remaining 39999 train instances)&nbsp;results in&nbsp;a misclassification rate of&nbsp;3,13 %.</p>\n<p>&nbsp;</p>\n<p>Is this a plausible magnitude or are there further considerations&nbsp;I should take into account (given that the goal is to estimate the performance of the algorithm)?</p>\n<p>&nbsp;</p>\n<p>Further, first experiments indicate that the misclassification rate doesn't seem to improve for k &gt; 1. Do you have any other suggestions in the context of KNN, how performance could be improved?</p>\n<p>&nbsp;</p>\n<p>&nbsp;EDIT:</p>\n<p>I was wrong about the insignificance of inspecting more neighbors. For a very simple weighting scheme where each class is assigned the sum of the inverse distances of its members I see a small improvement in the error rate in the range k = 3..8. The lowest train set error rate is 2,74% for k= 4. Maybe it's possible to reduce this even further by choosing a more sophisticated weighting scheme (e.g. exponential)? Thoughts?</p>"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "32683": ""
  }
}