Hey,
\nthe target we are predicting is based on a 15min timeframe. In contrast the OHLC data we are using for creating features is based on 1min time intervals. So there is a basic offset between prediction and predictor timeframes.
\nI was wondering if it should not be helpful, if not necessary, to downsample the OHLC data in order to work with insync data - especially when creating features like technical indicators, which heavily rely on the timeframe of interest (MACD, RSI, EMA, etc.)
\nI did not see anyone applying downsampling in the most shared kernels, so I was wondering why so?
\nAm I missing something?
Thx everybody!
\n(Sorry if double post, but i did not find anything on that topic so far)
Hey,
\nthe target we are predicting is based on a 15min timeframe. In contrast the OHLC data we are using for creating features is based on 1min time intervals. So there is a basic offset between prediction and predictor timeframes.
\nI was wondering if it should not be helpful, if not necessary, to downsample the OHLC data in order to work with insync data - especially when creating features like technical indicators, which heavily rely on the timeframe of interest (MACD, RSI, EMA, etc.)
\nI did not see anyone applying downsampling in the most shared kernels, so I was wondering why so?
\nAm I missing something?
Thx everybody!
\n(Sorry if double post, but i did not find anything on that topic so far)
I've found that downsampling 1/10 doesn't hurt performance but help widly avoiding out of memory errors. So I do it after the feature engineering (see: https://www.kaggle.com/lucasmorin/on-line-feature-engineering).
\nI suspect further downsampling might be needed to avoid target autocorrelation (or at least overlapping targets - see one of the first chapters of Lopez de Prado Book).
", "rawMarkdown": "I've found that downsampling 1/10 doesn't hurt performance but help widly avoiding out of memory errors. So I do it after the feature engineering (see: https://www.kaggle.com/lucasmorin/on-line-feature-engineering).\n\nI suspect further downsampling might be needed to avoid target autocorrelation (or at least overlapping targets - see one of the first chapters of Lopez de Prado Book).", "votes": null }, { "id": "1631775", "postDate": "12/28/2021 19:03:09", "content": "Thx for your answer. I seemed to miss that step when looking at your NB, but Ill have another look later : )
\nI think there are different possibilities applying and using downsampling in our case, given the data and the definition of our target. I even thought about creating 15 * 15min timeseries per asset out of the 1min data (each of those 15 series would then be highly correlated between each other of course). This would aswell remove target autocorrelation you mentiond within each subsampled series.
\nI'll play around a bit and share
", "rawMarkdown": "Thx for your answer. I seemed to miss that step when looking at your NB, but Ill have another look later : ) \n\nI think there are different possibilities applying and using downsampling in our case, given the data and the definition of our target. I even thought about creating 15 * 15min timeseries per asset out of the 1min data (each of those 15 series would then be highly correlated between each other of course). This would aswell remove target autocorrelation you mentiond within each subsampled series.\n\nI'll play around a bit and share", "votes": null } ], "comments": [ { "id": 1631272, "author_name": "lucasmorin", "author_url": "", "post_date": "12/28/2021 08:08:56", "content": "I've found that downsampling 1/10 doesn't hurt performance but help widly avoiding out of memory errors. So I do it after the feature engineering (see: https://www.kaggle.com/lucasmorin/on-line-feature-engineering).
\nI suspect further downsampling might be needed to avoid target autocorrelation (or at least overlapping targets - see one of the first chapters of Lopez de Prado Book).
", "votes": null, "replies": [ { "id": 1631775, "author_name": "belcebub", "author_url": "", "post_date": "12/28/2021 19:03:09", "content": "Thx for your answer. I seemed to miss that step when looking at your NB, but Ill have another look later : )
\nI think there are different possibilities applying and using downsampling in our case, given the data and the definition of our target. I even thought about creating 15 * 15min timeseries per asset out of the 1min data (each of those 15 series would then be highly correlated between each other of course). This would aswell remove target autocorrelation you mentiond within each subsampled series.
\nI'll play around a bit and share
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1630969": "Hey,\n\nthe target we are predicting is based on a 15min timeframe. In contrast the OHLC data we are using for creating features is based on 1min time intervals. So there is a basic offset between prediction and predictor timeframes. \nI was wondering if it should not be helpful, if not necessary, to downsample the OHLC data in order to work with insync data - especially when creating features like technical indicators, which heavily rely on the timeframe of interest (MACD, RSI, EMA, etc.) \nI did not see anyone applying downsampling in the most shared kernels, so I was wondering why so? \nAm I missing something? \n\nThx everybody! \n(Sorry if double post, but i did not find anything on that topic so far)", "1631272": "I've found that downsampling 1/10 doesn't hurt performance but help widly avoiding out of memory errors. So I do it after the feature engineering (see: https://www.kaggle.com/lucasmorin/on-line-feature-engineering).\n\nI suspect further downsampling might be needed to avoid target autocorrelation (or at least overlapping targets - see one of the first chapters of Lopez de Prado Book).", "1631775": "Thx for your answer. I seemed to miss that step when looking at your NB, but Ill have another look later : ) \n\nI think there are different possibilities applying and using downsampling in our case, given the data and the definition of our target. I even thought about creating 15 * 15min timeseries per asset out of the 1min data (each of those 15 series would then be highly correlated between each other of course). This would aswell remove target autocorrelation you mentiond within each subsampled series.\n\nI'll play around a bit and share" }, "source": "meta" }