{
  "id": 1030,
  "title": "Why has the test data bid51-100 and ask51-100 populated?",
  "url": "/competitions/AlgorithmicTradingChallenge/discussion/1030",
  "author_name": "",
  "post_date": "2011-11-10T14:13:55.343Z",
  "votes": 1,
  "comment_count": 7,
  "views": 6211,
  "content": "<p>Why has the test data bid51-100 and ask51-100 populated? Aren't we supposed to predict that?</p>",
  "messages": [
    {
      "id": "6455",
      "postDate": "11/10/2011 14:13:55",
      "content": "<p>Why has the test data bid51-100 and ask51-100 populated? Aren't we supposed to predict that?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "6456",
      "postDate": "11/10/2011 14:32:49",
      "content": "<p>Great I won, can I have my money please :-)<br>\r\nThat was easy</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "6457",
      "postDate": "11/10/2011 15:09:51",
      "content": "<p>Hey Drik,</p>\r\n<p>Looks like you cracked it&nbsp; ! ! ! <br>\r\nCongratulations. <br>\r\n<br>\r\nI am new to this. <br>\r\nThus, I am not sure about these RMSE calculations. <br>\r\nThough after looking at Leaderboard it seems like entry with lower RMSE gets higher rank.\r\n<br>\r\nMeans no one can beat you now. <br>\r\n<br>\r\nI would welcome anybody explaining RMSE in Leaderboard. <br>\r\n<br>\r\nRegards, <br>\r\nBhavin</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "6458",
      "postDate": "11/10/2011 15:16:20",
      "content": "<p>This is a joke!</p>\r\n<p>Thank Jeff Moser!!!!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "6471",
      "postDate": "11/11/2011 18:17:55",
      "content": "<p>[quote=Dirk Nachbar;6455]</p>\r\n<p>Why has the test data bid51-100 and ask51-100 populated? Aren't we supposed to predict that?</p>\r\n<p>[/quote]</p>\r\n<p>Thanks for reporting this. This was an accident during the data generation phase. Per the rules, this issue required that we update the test set. Since the competition host has a wealth of data, they generated a fresh test set (that doesn't have the answers).\r\n Because this is a new test set, the benchmarks were updated and all previous entries were rescored.</p>\r\n<p>The training set now includes the previous training contents and the previous test set. Thus, the net effect of this issue is that you get more data.</p>\r\n<p>Sorry about the inconvenience this caused to early participants.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "6483",
      "postDate": "11/11/2011 21:14:34",
      "content": "<p>[quote=Jeff Moser;6471]&nbsp;Since the competition host has a wealth of data, they generated a fresh test set ...&nbsp;The training set now includes the previous training contents and the previous test set. Thus, the net effect of this issue is that you get more data.&nbsp;[/quote]</p>\r\n<p>Thanks for your swift work to correct this. &nbsp;I've got some follow-up questions about the data 'merge' you did:&nbsp;</p>\r\n<p>On another thread, the organizer(s) said the rows in the training file appeared in chronological order -- but I don't think anything was said about the testing file. So:</p>\r\n<ul>\r\n<li>Was the old test set just concatenated to the end of the old training set to make the new training set? &nbsp;Or is it interspersed, with row IDs changed/renumbered?&nbsp; (it looks like it was just concatenated, but just want to confirm)\r\n</li><li>Was the new testing set sampled from 'fresh' data in the same way that the old testing set was sampled? (i.e. same methodology and biases, but perhaps using just different random numbers to sample, etc). The benchmark scores on the leaderboard seemed to\r\n have jumped downward by 0.1 - 0.2 RMSE, which might indicate the old &amp; new test sets are mildly different for some reason -- maybe it's random, but I'd like to rule out any sampling differences. &nbsp;\r\n</li><li>Are the current &nbsp;testing and training sets sampled from 2 disjoint time periods? Or are they both randomly sampled from one contiguous time period? &nbsp;(as mentioned on another thread, dates are not included, so it's hard to tell...)\r\n</li></ul>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "6505",
      "postDate": "11/13/2011 11:38:43",
      "content": "<p>To avoid predicting current price with future info testing data must not be in chronological order but should be in random order instead, is the current testing data organized this way? Thx</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "7241",
      "postDate": "12/16/2011 17:34:09",
      "content": "<p>[quote=Capital Markets CRC;6506]</p>\r\n<p>The current testing data is in random order</p>\r\n<p>[/quote]</p>\r\n<p>&nbsp;</p>\r\n<p>Couldn't one still string together the event windows in chronological order by looking at the timestamps? p_value or p_tcount may be used to filter on days. Of course, this model will have no practical value outside this competition, as you pointed out.</p>\r\n<p>There are other subtle ways one could end up using data from the future without even knowing it. For example, if you use the standard deviation of the spread of each security (not that I think this is a good predictor), you would have to be careful not to\r\n look past the point in time that you are predicting for.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 6456,
      "author_name": "dirknbr",
      "author_url": "",
      "post_date": "11/10/2011 14:32:49",
      "content": "<p>Great I won, can I have my money please :-)<br>\r\nThat was easy</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 6457,
      "author_name": "",
      "author_url": "",
      "post_date": "11/10/2011 15:09:51",
      "content": "<p>Hey Drik,</p>\r\n<p>Looks like you cracked it&nbsp; ! ! ! <br>\r\nCongratulations. <br>\r\n<br>\r\nI am new to this. <br>\r\nThus, I am not sure about these RMSE calculations. <br>\r\nThough after looking at Leaderboard it seems like entry with lower RMSE gets higher rank.\r\n<br>\r\nMeans no one can beat you now. <br>\r\n<br>\r\nI would welcome anybody explaining RMSE in Leaderboard. <br>\r\n<br>\r\nRegards, <br>\r\nBhavin</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 6458,
      "author_name": "alexxanderlarko",
      "author_url": "",
      "post_date": "11/10/2011 15:16:20",
      "content": "<p>This is a joke!</p>\r\n<p>Thank Jeff Moser!!!!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 6471,
      "author_name": "jeffmoser",
      "author_url": "",
      "post_date": "11/11/2011 18:17:55",
      "content": "<p>[quote=Dirk Nachbar;6455]</p>\r\n<p>Why has the test data bid51-100 and ask51-100 populated? Aren't we supposed to predict that?</p>\r\n<p>[/quote]</p>\r\n<p>Thanks for reporting this. This was an accident during the data generation phase. Per the rules, this issue required that we update the test set. Since the competition host has a wealth of data, they generated a fresh test set (that doesn't have the answers).\r\n Because this is a new test set, the benchmarks were updated and all previous entries were rescored.</p>\r\n<p>The training set now includes the previous training contents and the previous test set. Thus, the net effect of this issue is that you get more data.</p>\r\n<p>Sorry about the inconvenience this caused to early participants.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 6483,
      "author_name": "chefele",
      "author_url": "",
      "post_date": "11/11/2011 21:14:34",
      "content": "<p>[quote=Jeff Moser;6471]&nbsp;Since the competition host has a wealth of data, they generated a fresh test set ...&nbsp;The training set now includes the previous training contents and the previous test set. Thus, the net effect of this issue is that you get more data.&nbsp;[/quote]</p>\r\n<p>Thanks for your swift work to correct this. &nbsp;I've got some follow-up questions about the data 'merge' you did:&nbsp;</p>\r\n<p>On another thread, the organizer(s) said the rows in the training file appeared in chronological order -- but I don't think anything was said about the testing file. So:</p>\r\n<ul>\r\n<li>Was the old test set just concatenated to the end of the old training set to make the new training set? &nbsp;Or is it interspersed, with row IDs changed/renumbered?&nbsp; (it looks like it was just concatenated, but just want to confirm)\r\n</li><li>Was the new testing set sampled from 'fresh' data in the same way that the old testing set was sampled? (i.e. same methodology and biases, but perhaps using just different random numbers to sample, etc). The benchmark scores on the leaderboard seemed to\r\n have jumped downward by 0.1 - 0.2 RMSE, which might indicate the old &amp; new test sets are mildly different for some reason -- maybe it's random, but I'd like to rule out any sampling differences. &nbsp;\r\n</li><li>Are the current &nbsp;testing and training sets sampled from 2 disjoint time periods? Or are they both randomly sampled from one contiguous time period? &nbsp;(as mentioned on another thread, dates are not included, so it's hard to tell...)\r\n</li></ul>",
      "votes": null,
      "replies": []
    },
    {
      "id": 6505,
      "author_name": "xiaoshilu",
      "author_url": "",
      "post_date": "11/13/2011 11:38:43",
      "content": "<p>To avoid predicting current price with future info testing data must not be in chronological order but should be in random order instead, is the current testing data organized this way? Thx</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 7241,
      "author_name": "anlthms",
      "author_url": "",
      "post_date": "12/16/2011 17:34:09",
      "content": "<p>[quote=Capital Markets CRC;6506]</p>\r\n<p>The current testing data is in random order</p>\r\n<p>[/quote]</p>\r\n<p>&nbsp;</p>\r\n<p>Couldn't one still string together the event windows in chronological order by looking at the timestamps? p_value or p_tcount may be used to filter on days. Of course, this model will have no practical value outside this competition, as you pointed out.</p>\r\n<p>There are other subtle ways one could end up using data from the future without even knowing it. For example, if you use the standard deviation of the spread of each security (not that I think this is a good predictor), you would have to be careful not to\r\n look past the point in time that you are predicting for.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "6455": "",
    "6456": "",
    "6457": "",
    "6458": "",
    "6471": "",
    "6483": "",
    "6505": "",
    "7241": ""
  },
  "source": "meta"
}