{
  "id": 4725,
  "title": "Comparing progress",
  "url": "/competitions/cause-effect-pairs/discussion/4725",
  "author_name": "",
  "post_date": "2013-05-31T22:56:18.250Z",
  "votes": null,
  "comment_count": 4,
  "views": 1659,
  "content": "<p>Am I right in thinking that it is no longer possible to gauge progress against others in this contest?</p>\r\n<p>I.e., the top scores on the leaderboard, even new ones, may or may not be using spurious features that will not exist in the final dataset.</p>\r\n<p>If this is true, would there be any interest in starting a thread self-reporting scores on SUP2Data (or a subset of it)? Competition leads to innovation...</p>",
  "messages": [
    {
      "id": "25012",
      "postDate": "05/31/2013 22:56:18",
      "content": "<p>Am I right in thinking that it is no longer possible to gauge progress against others in this contest?</p>\r\n<p>I.e., the top scores on the leaderboard, even new ones, may or may not be using spurious features that will not exist in the final dataset.</p>\r\n<p>If this is true, would there be any interest in starting a thread self-reporting scores on SUP2Data (or a subset of it)? Competition leads to innovation...</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "25018",
      "postDate": "06/01/2013 02:17:31",
      "content": "<p>Greg V@</p>\r\n<p><span>If this is true, would there be any interest in starting a thread self-reporting scores on SUP2Data (or a subset of it)? Competition leads to innovation...</span></p>\r\n<p><span>________________________&nbsp;</span></p>\r\n<p><span>Yes. It would be very good!</span></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "25067",
      "postDate": "06/02/2013 20:56:58",
      "content": "<p>I'll start with a sad, disappointing result:</p>\r\n<p>Original training data : 85% training;&nbsp; Applied to leaderboard validation set: 88%</p>\r\n<p></p>\r\n<p>Sup2 only</p>\r\n<p>Training : 69%; Applied to leaderboard validation set, in original form, : 63%</p>\r\n<p>oops</p>\r\n<p></p>\r\n<p></p>\r\n<p></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "25069",
      "postDate": "06/02/2013 21:54:13",
      "content": "<p>I have also seen a large drop:</p>\r\n<p>Training Data: 0.87</p>\r\n<p>Leaderboard: 0.87</p>\r\n<p>Sup2: 0.73</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "25070",
      "postDate": "06/02/2013 22:08:55",
      "content": "<p>Interesting, but why just a thread? &nbsp;&lt;s&gt; I propose we create our own website, with an automated leaderboard, where we can post results. &lt;/s&gt; .&nbsp;</p>\r\n<p></p>\r\n<p>This is ridiculous. The organisers haven't posted any updates for a week now (even on things they did promise to do, like investigating if it's possible to change the leaderboard scores), and kaggle itself seems totally absent. The competition is in auto-pilot.\r\n &nbsp;Writing two lines every other day with a status update can't be that hard.&nbsp;</p>\r\n<p></p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 25018,
      "author_name": "alexxanderlarko",
      "author_url": "",
      "post_date": "06/01/2013 02:17:31",
      "content": "<p>Greg V@</p>\r\n<p><span>If this is true, would there be any interest in starting a thread self-reporting scores on SUP2Data (or a subset of it)? Competition leads to innovation...</span></p>\r\n<p><span>________________________&nbsp;</span></p>\r\n<p><span>Yes. It would be very good!</span></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 25067,
      "author_name": "domcastro",
      "author_url": "",
      "post_date": "06/02/2013 20:56:58",
      "content": "<p>I'll start with a sad, disappointing result:</p>\r\n<p>Original training data : 85% training;&nbsp; Applied to leaderboard validation set: 88%</p>\r\n<p></p>\r\n<p>Sup2 only</p>\r\n<p>Training : 69%; Applied to leaderboard validation set, in original form, : 63%</p>\r\n<p>oops</p>\r\n<p></p>\r\n<p></p>\r\n<p></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 25069,
      "author_name": "thomasatkins",
      "author_url": "",
      "post_date": "06/02/2013 21:54:13",
      "content": "<p>I have also seen a large drop:</p>\r\n<p>Training Data: 0.87</p>\r\n<p>Leaderboard: 0.87</p>\r\n<p>Sup2: 0.73</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 25070,
      "author_name": "firfirikos",
      "author_url": "",
      "post_date": "06/02/2013 22:08:55",
      "content": "<p>Interesting, but why just a thread? &nbsp;&lt;s&gt; I propose we create our own website, with an automated leaderboard, where we can post results. &lt;/s&gt; .&nbsp;</p>\r\n<p></p>\r\n<p>This is ridiculous. The organisers haven't posted any updates for a week now (even on things they did promise to do, like investigating if it's possible to change the leaderboard scores), and kaggle itself seems totally absent. The competition is in auto-pilot.\r\n &nbsp;Writing two lines every other day with a status update can't be that hard.&nbsp;</p>\r\n<p></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "25012": "",
    "25018": "",
    "25067": "",
    "25069": "",
    "25070": ""
  },
  "source": "meta"
}