{
  "id": 300745,
  "title": "Modeling the data - What time value ",
  "url": "/competitions/g-research-crypto-forecasting/discussion/300745",
  "author_name": "",
  "post_date": "2022-01-14T02:11:32.268779900Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hey guys! I was wondering what kind of time value for the data you use when doing the modeling. </p>\n<p>Do you keep the minute data or do you resample to Hourly or Daily data for example? And if you resample, what target do you pick? Since you have one target available for every minute data!</p>\n<p>Thanks in advance!</p>",
  "messages": [
    {
      "id": "1649143",
      "postDate": "01/14/2022 02:11:32",
      "content": "<p>Hey guys! I was wondering what kind of time value for the data you use when doing the modeling. </p>\n<p>Do you keep the minute data or do you resample to Hourly or Daily data for example? And if you resample, what target do you pick? Since you have one target available for every minute data!</p>\n<p>Thanks in advance!</p>",
      "rawMarkdown": "Hey guys! I was wondering what kind of time value for the data you use when doing the modeling. \n\nDo you keep the minute data or do you resample to Hourly or Daily data for example? And if you resample, what target do you pick? Since you have one target available for every minute data!\n\nThanks in advance!",
      "votes": null
    },
    {
      "id": "1649937",
      "postDate": "01/14/2022 17:35:50",
      "content": "<p>The target you should pick should be the one furthest in time to ensure no target leakage.  But, you shouldn't resample.  If you resample to hours or days, many of the predictions points will be using data that is very similar but the variety in the target will be the same.  You are going to need data down to the minutes to predict with a horizon of 15 minutes.  You could try sampling to 2-5 minutes, but it didn't help when I did it.  </p>",
      "rawMarkdown": "The target you should pick should be the one furthest in time to ensure no target leakage.  But, you shouldn't resample.  If you resample to hours or days, many of the predictions points will be using data that is very similar but the variety in the target will be the same.  You are going to need data down to the minutes to predict with a horizon of 15 minutes.  You could try sampling to 2-5 minutes, but it didn't help when I did it.",
      "votes": null
    },
    {
      "id": "1650125",
      "postDate": "01/14/2022 20:32:47",
      "content": "<p>Thank you <a href=\"https://www.kaggle.com/petersk20\" target=\"_blank\">@petersk20</a> ! Don't exactly know what is meant with target leakage, will look further into it.. Will keep the data for minute data.. Only resampled for feature engineering, since avg_volume_20d/ EMAs/ Volatility I need to have daily values, after I joined back to the minute data</p>\n<p><a href=\"https://www.kaggle.com/vweimer/crypto-forecasting-all-until-modeling\" target=\"_blank\">https://www.kaggle.com/vweimer/crypto-forecasting-all-until-modeling</a> If you have time, take a look at my notebook if you'd like! Thx!</p>",
      "rawMarkdown": "Thank you @petersk20 ! Don't exactly know what is meant with target leakage, will look further into it.. Will keep the data for minute data.. Only resampled for feature engineering, since avg_volume_20d/ EMAs/ Volatility I need to have daily values, after I joined back to the minute data\n\nhttps://www.kaggle.com/vweimer/crypto-forecasting-all-until-modeling If you have time, take a look at my notebook if you'd like! Thx!",
      "votes": null
    },
    {
      "id": "1650170",
      "postDate": "01/14/2022 21:28:34",
      "content": "<p>If you group minute 1,2, and 3 together and use the target from minute 2.  Minute 3 will contain information that is not really available when predicting the actual target.  So if this is done.  The target from minute 3 should be used.</p>",
      "rawMarkdown": "If you group minute 1,2, and 3 together and use the target from minute 2.  Minute 3 will contain information that is not really available when predicting the actual target.  So if this is done.  The target from minute 3 should be used.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1649937,
      "author_name": "petersk20",
      "author_url": "",
      "post_date": "01/14/2022 17:35:50",
      "content": "<p>The target you should pick should be the one furthest in time to ensure no target leakage.  But, you shouldn't resample.  If you resample to hours or days, many of the predictions points will be using data that is very similar but the variety in the target will be the same.  You are going to need data down to the minutes to predict with a horizon of 15 minutes.  You could try sampling to 2-5 minutes, but it didn't help when I did it.  </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1650125,
      "author_name": "vweimer",
      "author_url": "",
      "post_date": "01/14/2022 20:32:47",
      "content": "<p>Thank you <a href=\"https://www.kaggle.com/petersk20\" target=\"_blank\">@petersk20</a> ! Don't exactly know what is meant with target leakage, will look further into it.. Will keep the data for minute data.. Only resampled for feature engineering, since avg_volume_20d/ EMAs/ Volatility I need to have daily values, after I joined back to the minute data</p>\n<p><a href=\"https://www.kaggle.com/vweimer/crypto-forecasting-all-until-modeling\" target=\"_blank\">https://www.kaggle.com/vweimer/crypto-forecasting-all-until-modeling</a> If you have time, take a look at my notebook if you'd like! Thx!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1650170,
          "author_name": "petersk20",
          "author_url": "",
          "post_date": "01/14/2022 21:28:34",
          "content": "<p>If you group minute 1,2, and 3 together and use the target from minute 2.  Minute 3 will contain information that is not really available when predicting the actual target.  So if this is done.  The target from minute 3 should be used.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1649143": "Hey guys! I was wondering what kind of time value for the data you use when doing the modeling. \n\nDo you keep the minute data or do you resample to Hourly or Daily data for example? And if you resample, what target do you pick? Since you have one target available for every minute data!\n\nThanks in advance!",
    "1649937": "The target you should pick should be the one furthest in time to ensure no target leakage.  But, you shouldn't resample.  If you resample to hours or days, many of the predictions points will be using data that is very similar but the variety in the target will be the same.  You are going to need data down to the minutes to predict with a horizon of 15 minutes.  You could try sampling to 2-5 minutes, but it didn't help when I did it.",
    "1650125": "Thank you @petersk20 ! Don't exactly know what is meant with target leakage, will look further into it.. Will keep the data for minute data.. Only resampled for feature engineering, since avg_volume_20d/ EMAs/ Volatility I need to have daily values, after I joined back to the minute data\n\nhttps://www.kaggle.com/vweimer/crypto-forecasting-all-until-modeling If you have time, take a look at my notebook if you'd like! Thx!",
    "1650170": "If you group minute 1,2, and 3 together and use the target from minute 2.  Minute 3 will contain information that is not really available when predicting the actual target.  So if this is done.  The target from minute 3 should be used."
  },
  "source": "meta"
}