{
  "id": 299460,
  "title": "Is validation score around 0.01?",
  "url": "/competitions/g-research-crypto-forecasting/discussion/299460",
  "author_name": "ifashion",
  "post_date": "2022-01-08T05:54:03.133000",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>If only using data before 6/13 to train, do you experience similar score on PL?</p>",
  "messages": [
    {
      "id": 1642167,
      "postDate": "2022-01-08T05:54:03.133Z",
      "content": "<p>If only using data before 6/13 to train, do you experience similar score on PL?</p>",
      "rawMarkdown": "If only using data before 6/13 to train, do you experience similar score on PL?\n\n",
      "votes": 3
    },
    {
      "id": 1648743,
      "postDate": "2022-01-13T16:49:39.647Z",
      "content": "<p>Yes. What are people getting in CV? I suspect an average score of 0.01 on eventual true out-of-sample will be very good. From a theoretical perspective, by the Fundamental Theorem of Active Management, the information ratio is equal to the correlation of preds to actuals times the sqrt of the portfolio breadth (the number of independent bets).</p>\n<p>$$IR = IC \\times \\sqrt{\\text{breadth}}$$</p>\n<p>Since the target is residualized to the \"market\", you can assume these are all independent. So with 14 assets, and a corr of 0.01, the expected 15-minute IR is 0.01 x 3.74 = 0.0374. To annualized this you multiply by the sqrt of the number of periods in the year. In this case since crypto markets never close, the number of 15 minute periods is 365 x 24 x (60/15) = 35k. Thus the expected annualized information ratio (before transaction costs) is 7.0. That would be nice to achieve :-)</p>",
      "rawMarkdown": "Yes. What are people getting in CV? I suspect an average score of 0.01 on eventual true out-of-sample will be very good. From a theoretical perspective, by the Fundamental Theorem of Active Management, the information ratio is equal to the correlation of preds to actuals times the sqrt of the portfolio breadth (the number of independent bets).\n\n$$IR = IC \\times \\sqrt{\\text{breadth}}$$\n\nSince the target is residualized to the \"market\", you can assume these are all independent. So with 14 assets, and a corr of 0.01, the expected 15-minute IR is 0.01 x 3.74 = 0.0374. To annualized this you multiply by the sqrt of the number of periods in the year. In this case since crypto markets never close, the number of 15 minute periods is 365 x 24 x (60/15) = 35k. Thus the expected annualized information ratio (before transaction costs) is 7.0. That would be nice to achieve :-)",
      "votes": 1
    },
    {
      "id": 1646293,
      "postDate": "2022-01-11T16:00:03.733Z",
      "content": "<p>yes, i think it is</p>",
      "rawMarkdown": "yes, i think it is"
    },
    {
      "id": 1642395,
      "postDate": "2022-01-08T10:28:11.653Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1648743,
      "author_name": "marketneutral",
      "author_url": "",
      "post_date": "2022-01-13T16:49:39.647000",
      "content": "<p>Yes. What are people getting in CV? I suspect an average score of 0.01 on eventual true out-of-sample will be very good. From a theoretical perspective, by the Fundamental Theorem of Active Management, the information ratio is equal to the correlation of preds to actuals times the sqrt of the portfolio breadth (the number of independent bets).</p>\n<p>$$IR = IC \\times \\sqrt{\\text{breadth}}$$</p>\n<p>Since the target is residualized to the \"market\", you can assume these are all independent. So with 14 assets, and a corr of 0.01, the expected 15-minute IR is 0.01 x 3.74 = 0.0374. To annualized this you multiply by the sqrt of the number of periods in the year. In this case since crypto markets never close, the number of 15 minute periods is 365 x 24 x (60/15) = 35k. Thus the expected annualized information ratio (before transaction costs) is 7.0. That would be nice to achieve :-)</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1646293,
      "author_name": "Tom Lu",
      "author_url": "",
      "post_date": "2022-01-11T16:00:03.733000",
      "content": "<p>yes, i think it is</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1642395,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-01-08T10:28:11.653000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1642167": "If only using data before 6/13 to train, do you experience similar score on PL?\n\n",
    "1648743": "Yes. What are people getting in CV? I suspect an average score of 0.01 on eventual true out-of-sample will be very good. From a theoretical perspective, by the Fundamental Theorem of Active Management, the information ratio is equal to the correlation of preds to actuals times the sqrt of the portfolio breadth (the number of independent bets).\n\n$$IR = IC \\times \\sqrt{\\text{breadth}}$$\n\nSince the target is residualized to the \"market\", you can assume these are all independent. So with 14 assets, and a corr of 0.01, the expected 15-minute IR is 0.01 x 3.74 = 0.0374. To annualized this you multiply by the sqrt of the number of periods in the year. In this case since crypto markets never close, the number of 15 minute periods is 365 x 24 x (60/15) = 35k. Thus the expected annualized information ratio (before transaction costs) is 7.0. That would be nice to achieve :-)",
    "1646293": "yes, i think it is",
    "1642395": ""
  }
}