{
  "id": 1156,
  "title": "What kind of accuracies are top competitors getting against the valid_test data set?",
  "url": "/competitions/WhatDoYouKnow/discussion/1156",
  "author_name": "",
  "post_date": "2011-12-14T03:36:51.397Z",
  "votes": null,
  "comment_count": 3,
  "views": 2639,
  "content": "<p>Just wondering, since the&nbsp;<span>&nbsp;</span><a href=\"http://www.kaggle.com/c/ChessRatings2/Details/Evaluation\">Capped Binomial Deviance</a>&nbsp;metric does not provide a clear understanding of how well the top competitors are predicting the test data set.</p>\r\n<p>Cheers!</p>",
  "messages": [
    {
      "id": "7115",
      "postDate": "12/14/2011 03:36:51",
      "content": "<p>Just wondering, since the&nbsp;<span>&nbsp;</span><a href=\"http://www.kaggle.com/c/ChessRatings2/Details/Evaluation\">Capped Binomial Deviance</a>&nbsp;metric does not provide a clear understanding of how well the top competitors are predicting the test data set.</p>\r\n<p>Cheers!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "7118",
      "postDate": "12/14/2011 04:37:24",
      "content": "<p>Hi Cerebro...</p>\r\n<p>Below are valid_test and leaderboard scores for my last 3 submissions:</p>\r\n<table border=\"1\" cellspacing=\"1\" cellpadding=\"1\" align=\"left\">\r\n<tbody>\r\n<tr>\r\n<td><strong>valid_test</strong></td>\r\n<td><strong>Leaderboard</strong></td>\r\n</tr>\r\n<tr>\r\n<td>&nbsp;0.250705</td>\r\n<td>&nbsp;0.25190</td>\r\n</tr>\r\n<tr>\r\n<td>&nbsp;0.248331</td>\r\n<td>&nbsp;0.24941</td>\r\n</tr>\r\n<tr>\r\n<td>&nbsp;0.247345</td>\r\n<td>&nbsp;0.24844</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p>&nbsp;</p>\r\n<p>&nbsp;</p>\r\n<p>&nbsp;</p>\r\n<p>&nbsp;</p>\r\n<p>All of those are stacked ensembles of 3-6 predictors each.</p>\r\n<p>My best single-method score: 0.251520 (test), 0.25275 (leaderboard)</p>\r\n<p>&nbsp;</p>\r\n<p>(Note: I tried to make those numbers appear in a nice table, but apparently kaggle's table editor doesn't work.)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "7133",
      "postDate": "12/14/2011 14:16:11",
      "content": "<p>Here are the scores for all my submissions:</p>\r\n<table border=\"0\" frame=\"VOID\" rules=\"NONE\" cellspacing=\"0\">\r\n<colgroup><col width=\"139\"><col width=\"139\"><col width=\"94\"></colgroup>\r\n<tbody>\r\n<tr>\r\n<td align=\"CENTER\" width=\"139\" height=\"18\"><strong><span style=\"color:#0047ff\">valid_test</span></strong></td>\r\n<td align=\"CENTER\" width=\"139\"><strong><span style=\"color:#0047ff\">leaderboard</span></strong></td>\r\n<td align=\"CENTER\" width=\"94\"><strong><span style=\"color:#0047ff\">diff</span></strong></td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.25027</td>\r\n<td align=\"CENTER\">0.25109</td>\r\n<td align=\"CENTER\">0.00082</td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.24996</td>\r\n<td align=\"CENTER\">0.25064</td>\r\n<td align=\"CENTER\">0.00068</td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.24925</td>\r\n<td align=\"CENTER\"><span style=\"color:#000000\">0.25020</span></td>\r\n<td align=\"CENTER\">0.00095</td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.24902</td>\r\n<td align=\"CENTER\"><span style=\"color:#000000\">0.25005</span></td>\r\n<td align=\"CENTER\">0.00103</td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.24770</td>\r\n<td align=\"CENTER\">0.24900</td>\r\n<td align=\"CENTER\">0.00130</td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.24710</td>\r\n<td align=\"CENTER\">0.24831</td>\r\n<td align=\"CENTER\">0.00121</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p>James</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "7165",
      "postDate": "12/15/2011 04:42:42",
      "content": "<p>James, YetiMan, thank you for sharing! It is very helpful!</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 7118,
      "author_name": "yetiman",
      "author_url": "",
      "post_date": "12/14/2011 04:37:24",
      "content": "<p>Hi Cerebro...</p>\r\n<p>Below are valid_test and leaderboard scores for my last 3 submissions:</p>\r\n<table border=\"1\" cellspacing=\"1\" cellpadding=\"1\" align=\"left\">\r\n<tbody>\r\n<tr>\r\n<td><strong>valid_test</strong></td>\r\n<td><strong>Leaderboard</strong></td>\r\n</tr>\r\n<tr>\r\n<td>&nbsp;0.250705</td>\r\n<td>&nbsp;0.25190</td>\r\n</tr>\r\n<tr>\r\n<td>&nbsp;0.248331</td>\r\n<td>&nbsp;0.24941</td>\r\n</tr>\r\n<tr>\r\n<td>&nbsp;0.247345</td>\r\n<td>&nbsp;0.24844</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p>&nbsp;</p>\r\n<p>&nbsp;</p>\r\n<p>&nbsp;</p>\r\n<p>&nbsp;</p>\r\n<p>All of those are stacked ensembles of 3-6 predictors each.</p>\r\n<p>My best single-method score: 0.251520 (test), 0.25275 (leaderboard)</p>\r\n<p>&nbsp;</p>\r\n<p>(Note: I tried to make those numbers appear in a nice table, but apparently kaggle's table editor doesn't work.)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 7133,
      "author_name": "jpetterson",
      "author_url": "",
      "post_date": "12/14/2011 14:16:11",
      "content": "<p>Here are the scores for all my submissions:</p>\r\n<table border=\"0\" frame=\"VOID\" rules=\"NONE\" cellspacing=\"0\">\r\n<colgroup><col width=\"139\"><col width=\"139\"><col width=\"94\"></colgroup>\r\n<tbody>\r\n<tr>\r\n<td align=\"CENTER\" width=\"139\" height=\"18\"><strong><span style=\"color:#0047ff\">valid_test</span></strong></td>\r\n<td align=\"CENTER\" width=\"139\"><strong><span style=\"color:#0047ff\">leaderboard</span></strong></td>\r\n<td align=\"CENTER\" width=\"94\"><strong><span style=\"color:#0047ff\">diff</span></strong></td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.25027</td>\r\n<td align=\"CENTER\">0.25109</td>\r\n<td align=\"CENTER\">0.00082</td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.24996</td>\r\n<td align=\"CENTER\">0.25064</td>\r\n<td align=\"CENTER\">0.00068</td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.24925</td>\r\n<td align=\"CENTER\"><span style=\"color:#000000\">0.25020</span></td>\r\n<td align=\"CENTER\">0.00095</td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.24902</td>\r\n<td align=\"CENTER\"><span style=\"color:#000000\">0.25005</span></td>\r\n<td align=\"CENTER\">0.00103</td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.24770</td>\r\n<td align=\"CENTER\">0.24900</td>\r\n<td align=\"CENTER\">0.00130</td>\r\n</tr>\r\n<tr>\r\n<td align=\"CENTER\" height=\"18\">0.24710</td>\r\n<td align=\"CENTER\">0.24831</td>\r\n<td align=\"CENTER\">0.00121</td>\r\n</tr>\r\n</tbody>\r\n</table>\r\n<p>James</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 7165,
      "author_name": "arnaudsj",
      "author_url": "",
      "post_date": "12/15/2011 04:42:42",
      "content": "<p>James, YetiMan, thank you for sharing! It is very helpful!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "7115": "",
    "7118": "",
    "7133": "",
    "7165": ""
  },
  "source": "meta"
}