{
  "id": 301359,
  "title": "right evaluation window for short term crypto prediction",
  "url": "/competitions/g-research-crypto-forecasting/discussion/301359",
  "author_name": "",
  "post_date": "2022-01-17T10:00:23.015898900Z",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>TL;DR volatile crypto market and short prediction window requires a more agressive evaluation window. </p>\n<p>Our prediction includes t+16, so 16 minutes ahead.<br>\nWe will be forecasting a 3 months period.<br>\nMy question is: how will the correlation evaluation be performed 'timewise'?<br>\nIt doesn't make sense to perform a single correlation for the whole 3 months to evaluate the models.<br>\nHost says, the evaluation will be per 2-weeks basis.<br>\nBut isn't even 'two weeks' a little bit to far stretched for a target based on a  '16 minutes' prediction window?<br>\nIn other words: if I have a model that makes prediction for the next quarter hour, I will probably make use of it in the next couple of hours (maybe) and rerun the model.<br>\nSo, why should I even care to push my evaluation to two weeks, if I can run the model every day or more frequently?<br>\nThink about a model that is very good for the first day but is miserable thereafter. Isn't this model better than a model that is better in average but worse in terms of the short term I care about? </p>",
  "messages": [
    {
      "id": "1653176",
      "postDate": "01/17/2022 10:00:23",
      "content": "<p>TL;DR volatile crypto market and short prediction window requires a more agressive evaluation window. </p>\n<p>Our prediction includes t+16, so 16 minutes ahead.<br>\nWe will be forecasting a 3 months period.<br>\nMy question is: how will the correlation evaluation be performed 'timewise'?<br>\nIt doesn't make sense to perform a single correlation for the whole 3 months to evaluate the models.<br>\nHost says, the evaluation will be per 2-weeks basis.<br>\nBut isn't even 'two weeks' a little bit to far stretched for a target based on a  '16 minutes' prediction window?<br>\nIn other words: if I have a model that makes prediction for the next quarter hour, I will probably make use of it in the next couple of hours (maybe) and rerun the model.<br>\nSo, why should I even care to push my evaluation to two weeks, if I can run the model every day or more frequently?<br>\nThink about a model that is very good for the first day but is miserable thereafter. Isn't this model better than a model that is better in average but worse in terms of the short term I care about? </p>",
      "rawMarkdown": "TL;DR volatile crypto market and short prediction window requires a more agressive evaluation window. \n\nOur prediction includes t+16, so 16 minutes ahead.\nWe will be forecasting a 3 months period.\nMy question is: how will the correlation evaluation be performed 'timewise'?\nIt doesn't make sense to perform a single correlation for the whole 3 months to evaluate the models.\nHost says, the evaluation will be per 2-weeks basis.\nBut isn't even 'two weeks' a little bit to far stretched for a target based on a  '16 minutes' prediction window?\nIn other words: if I have a model that makes prediction for the next quarter hour, I will probably make use of it in the next couple of hours (maybe) and rerun the model.\nSo, why should I even care to push my evaluation to two weeks, if I can run the model every day or more frequently?\nThink about a model that is very good for the first day but is miserable thereafter. Isn't this model better than a model that is better in average but worse in terms of the short term I care about?",
      "votes": null
    },
    {
      "id": "1657931",
      "postDate": "01/20/2022 15:37:08",
      "content": "<p>Yeah, I have the same question and the other question is how will the host use the model we submitted?</p>",
      "rawMarkdown": "Yeah, I have the same question and the other question is how will the host use the model we submitted?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1657931,
      "author_name": "kailonglu",
      "author_url": "",
      "post_date": "01/20/2022 15:37:08",
      "content": "<p>Yeah, I have the same question and the other question is how will the host use the model we submitted?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1653176": "TL;DR volatile crypto market and short prediction window requires a more agressive evaluation window. \n\nOur prediction includes t+16, so 16 minutes ahead.\nWe will be forecasting a 3 months period.\nMy question is: how will the correlation evaluation be performed 'timewise'?\nIt doesn't make sense to perform a single correlation for the whole 3 months to evaluate the models.\nHost says, the evaluation will be per 2-weeks basis.\nBut isn't even 'two weeks' a little bit to far stretched for a target based on a  '16 minutes' prediction window?\nIn other words: if I have a model that makes prediction for the next quarter hour, I will probably make use of it in the next couple of hours (maybe) and rerun the model.\nSo, why should I even care to push my evaluation to two weeks, if I can run the model every day or more frequently?\nThink about a model that is very good for the first day but is miserable thereafter. Isn't this model better than a model that is better in average but worse in terms of the short term I care about?",
    "1657931": "Yeah, I have the same question and the other question is how will the host use the model we submitted?"
  },
  "source": "meta"
}