{
  "id": 1112,
  "title": "Multiple choice with multiple correct answers?",
  "url": "/competitions/WhatDoYouKnow/discussion/1112",
  "author_name": "",
  "post_date": "2011-11-30T22:30:42.843Z",
  "votes": 1,
  "comment_count": 2,
  "views": 2327,
  "content": "<p>The training set contains the following lines:</p>\r\n<p>1,1,121052,2526,0,0,3,12,34 50 152 162 212 243,2011-11-10 18:37:50,2011-11-10 18:38:15,2011-11-10 18:38:17,14094,7,2,NULL,391<br>\r\n1,1,7120,2526,0,0,3,12,34 50 152 162 212 243,2011-02-17 18:18:39,2011-02-17 18:19:09,2011-02-17 18:19:20,14094,7,1,NULL,391<br>\r\n1,1,155663,2526,0,0,3,12,34 50 152 162 212 243,2009-11-22 23:25:18,2009-11-22 23:25:26,2009-11-22 23:25:26,14095,7,1,NULL,391</p>\r\n<p>That is, question 2526 is marked as correct for answerid 14094 and 14095. &nbsp;The question type is 0, 'MultipleChoiceOneCorrect'. &nbsp;Why are two different answers both being marked as correct?</p>\r\n<p>Thank you.</p>",
  "messages": [
    {
      "id": "6884",
      "postDate": "11/30/2011 22:30:42",
      "content": "<p>The training set contains the following lines:</p>\r\n<p>1,1,121052,2526,0,0,3,12,34 50 152 162 212 243,2011-11-10 18:37:50,2011-11-10 18:38:15,2011-11-10 18:38:17,14094,7,2,NULL,391<br>\r\n1,1,7120,2526,0,0,3,12,34 50 152 162 212 243,2011-02-17 18:18:39,2011-02-17 18:19:09,2011-02-17 18:19:20,14094,7,1,NULL,391<br>\r\n1,1,155663,2526,0,0,3,12,34 50 152 162 212 243,2009-11-22 23:25:18,2009-11-22 23:25:26,2009-11-22 23:25:26,14095,7,1,NULL,391</p>\r\n<p>That is, question 2526 is marked as correct for answerid 14094 and 14095. &nbsp;The question type is 0, 'MultipleChoiceOneCorrect'. &nbsp;Why are two different answers both being marked as correct?</p>\r\n<p>Thank you.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "6887",
      "postDate": "12/01/2011 00:03:00",
      "content": "<p>14094 is the correct answer. &nbsp;The question previously&nbsp;had&nbsp;14095 marked as being correct, but was later updated (at the end of 2009, apparently) so that 14095 was (appropriately) marked as incorrect. &nbsp;This is probably true of a few other questions as well,\r\n and is an (unfortunate) case of dirty/incorrect data -- particularly since it means that there are some examples (like the third one you list) where an answer is marked as correct when it should be incorrect (and there may also be cases where it should be\r\n correct but was marked incorrect). &nbsp;In these cases, the most recently observed answer/outcome is the most appropriate one.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "8400",
      "postDate": "02/02/2012 10:09:33",
      "content": "<p>In the worst case, I found that for question_id=2160, the answers with IDs 15979, 15981 and 15982 have been marked as correct (outcome=1), where the former two have also been marked as incorrect (outcome=2) in other occasions:</p>\r\n<p>(row no. in training.csv / outcome / answered<em>_</em>at<em> / </em>answer_id)<br>\r\n2448516 1 2011-10-13 02:11:17 15979<br>\r\n2125096 1 2011-10-13 03:04:12 15981<br>\r\n4002048 2 2011-10-16 07:18:38 15979<br>\r\n196716 2 2011-10-19 02:31:24 15981<br>\r\n240589 1 2011-10-21 13:09:42 15982</p>\r\n<p>This makes me wonder how reliable the outcomes are ...</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 6887,
      "author_name": "thomaslotze",
      "author_url": "",
      "post_date": "12/01/2011 00:03:00",
      "content": "<p>14094 is the correct answer. &nbsp;The question previously&nbsp;had&nbsp;14095 marked as being correct, but was later updated (at the end of 2009, apparently) so that 14095 was (appropriately) marked as incorrect. &nbsp;This is probably true of a few other questions as well,\r\n and is an (unfortunate) case of dirty/incorrect data -- particularly since it means that there are some examples (like the third one you list) where an answer is marked as correct when it should be incorrect (and there may also be cases where it should be\r\n correct but was marked incorrect). &nbsp;In these cases, the most recently observed answer/outcome is the most appropriate one.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 8400,
      "author_name": "thesuffocated",
      "author_url": "",
      "post_date": "02/02/2012 10:09:33",
      "content": "<p>In the worst case, I found that for question_id=2160, the answers with IDs 15979, 15981 and 15982 have been marked as correct (outcome=1), where the former two have also been marked as incorrect (outcome=2) in other occasions:</p>\r\n<p>(row no. in training.csv / outcome / answered<em>_</em>at<em> / </em>answer_id)<br>\r\n2448516 1 2011-10-13 02:11:17 15979<br>\r\n2125096 1 2011-10-13 03:04:12 15981<br>\r\n4002048 2 2011-10-16 07:18:38 15979<br>\r\n196716 2 2011-10-19 02:31:24 15981<br>\r\n240589 1 2011-10-21 13:09:42 15982</p>\r\n<p>This makes me wonder how reliable the outcomes are ...</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "6884": "",
    "6887": "",
    "8400": ""
  },
  "source": "meta"
}