{
  "id": 5915,
  "title": "Help: Percentage of true vs false predictions in artificially generated questions.csv",
  "url": "/competitions/accelerometer-biometric-competition/discussion/5915",
  "author_name": "",
  "post_date": "2013-09-29T02:17:11.940Z",
  "votes": null,
  "comment_count": 4,
  "views": 1553,
  "content": "<p>Whats a good mix of true vs false answers in an artificially created questions.csv for offline evaluation ? (We tried 50% but the AUC change trend we get isn't consistent with kaggle's evaluation).</p>",
  "messages": [
    {
      "id": "31743",
      "postDate": "09/29/2013 02:17:11",
      "content": "<p>Whats a good mix of true vs false answers in an artificially created questions.csv for offline evaluation ? (We tried 50% but the AUC change trend we get isn't consistent with kaggle's evaluation).</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31760",
      "postDate": "09/29/2013 07:09:30",
      "content": "<p>I guess you could try submitting all positives to see what ratio kaggle has in the public set =)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31761",
      "postDate": "09/29/2013 08:08:19",
      "content": "<p>hmm. aw thanks</p>\n<p>i suppose that won't work very well using the roc metric? (whatever the ratio is, it will score 0.5?) You could try 50% negative and 50% positive, ie zero's and one's.. though i think that will give a confusing result... but if you use something like 80/20% it might be meaningful?</p>\n<p>Added: I'm not sure if you need to submit twice.. eg first submit 80% positive, then that same 80% negative... actually i think the best you could do is submit all but one positive, i'm pretty sure that will do the trick =D</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31787",
      "postDate": "09/29/2013 18:34:29",
      "content": "<p>Hi ! The ratio is 50/50, Geoff has answered this question&nbsp;<a href=\"https://www.kaggle.com/c/accelerometer-biometric-competition/forums/t/5343/how-were-the-questions-generated\">here</a>.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31795",
      "postDate": "09/29/2013 19:44:23",
      "content": "<p>Thanks demytt and n3n7i. I should have scanned the forum before posting the question.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 31760,
      "author_name": "n3n77i",
      "author_url": "",
      "post_date": "09/29/2013 07:09:30",
      "content": "<p>I guess you could try submitting all positives to see what ratio kaggle has in the public set =)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31761,
      "author_name": "n3n77i",
      "author_url": "",
      "post_date": "09/29/2013 08:08:19",
      "content": "<p>hmm. aw thanks</p>\n<p>i suppose that won't work very well using the roc metric? (whatever the ratio is, it will score 0.5?) You could try 50% negative and 50% positive, ie zero's and one's.. though i think that will give a confusing result... but if you use something like 80/20% it might be meaningful?</p>\n<p>Added: I'm not sure if you need to submit twice.. eg first submit 80% positive, then that same 80% negative... actually i think the best you could do is submit all but one positive, i'm pretty sure that will do the trick =D</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31787,
      "author_name": "ademyttenaere",
      "author_url": "",
      "post_date": "09/29/2013 18:34:29",
      "content": "<p>Hi ! The ratio is 50/50, Geoff has answered this question&nbsp;<a href=\"https://www.kaggle.com/c/accelerometer-biometric-competition/forums/t/5343/how-were-the-questions-generated\">here</a>.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31795,
      "author_name": "sharadbanka",
      "author_url": "",
      "post_date": "09/29/2013 19:44:23",
      "content": "<p>Thanks demytt and n3n7i. I should have scanned the forum before posting the question.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "31743": "",
    "31760": "",
    "31761": "",
    "31787": "",
    "31795": ""
  },
  "source": "meta"
}