TL;DR: Adhoc data shifting for target recreation for each asset, makes online target calculation based on predicted log_returns almost impossible, (I think)
\nYes, I guess I am following from behind but here is the deal.
\nI think it is extremely hard, if not 'close to impossible' to recreate the target online.
\nBy online I mean a situation, where you want to predict the log_returns and build your target from there, instead of a direct prediction of the target.
\nWhy is it a problem: To calculate the 'constructed target' using a model predicting only future log_returns (say log_return of t+16), you will need to calculate beta, which needs a window of 3749 prior 'log_return targets'. Good thing is, you can update your predictions with incomingdata. So you can update your prediction at time t, when t+16 data arrives.
\nTheoretically it is so simple, but practically, there is a huge problem. For each asset, you can't update your target at time t with data at time t+16, because, based on 'host/kaggle definition', if a target data is missing (such as the target for that asset at time t+5), t+16 becomes irrelevant for target calculations. Yes that is because, target is not based on an exact 't+16' interval but, a simple shift for each asset of the existing data(i.e. a shift(-16)), which totally weirds up the calculations..
TL;DR: Adhoc data shifting for target recreation for each asset, makes online target calculation based on predicted log_returns almost impossible, (I think)
\nYes, I guess I am following from behind but here is the deal.
\nI think it is extremely hard, if not 'close to impossible' to recreate the target online.
\nBy online I mean a situation, where you want to predict the log_returns and build your target from there, instead of a direct prediction of the target.
\nWhy is it a problem: To calculate the 'constructed target' using a model predicting only future log_returns (say log_return of t+16), you will need to calculate beta, which needs a window of 3749 prior 'log_return targets'. Good thing is, you can update your predictions with incomingdata. So you can update your prediction at time t, when t+16 data arrives.
\nTheoretically it is so simple, but practically, there is a huge problem. For each asset, you can't update your target at time t with data at time t+16, because, based on 'host/kaggle definition', if a target data is missing (such as the target for that asset at time t+5), t+16 becomes irrelevant for target calculations. Yes that is because, target is not based on an exact 't+16' interval but, a simple shift for each asset of the existing data(i.e. a shift(-16)), which totally weirds up the calculations..
Beta is based off of 3750 values and 16 of which we don't know. This will cause problems as you stated, but there are a lot less NA values near the end of the dataframe and 16 out of 3750 is not much. There is also a high correlation between the actual target and log returns of a single coin. Factoring this all in, it will slightly affect the performance of your model, but there is nothing to do about it and the effect will not make a well trained model unusable. Just calculate beta using the previous 3734 values and ignore the futures one.
\nI think the only solution would be for Gresearch to change the target calculation so that beta doesn't become 0 when there is an NA, but that is not going to happen.
", "rawMarkdown": "Beta is based off of 3750 values and 16 of which we don't know. This will cause problems as you stated, but there are a lot less NA values near the end of the dataframe and 16 out of 3750 is not much. There is also a high correlation between the actual target and log returns of a single coin. Factoring this all in, it will slightly affect the performance of your model, but there is nothing to do about it and the effect will not make a well trained model unusable. Just calculate beta using the previous 3734 values and ignore the futures one.\n\nI think the only solution would be for Gresearch to change the target calculation so that beta doesn't become 0 when there is an NA, but that is not going to happen.", "votes": null }, { "id": "1648248", "postDate": "01/13/2022 09:25:23", "content": "Thanks. This helps.
\nI have to say it all started when I looked at the famous target-reconstuction notebooks and saw a nonexisting timestamp t in data for an asset a, which suddenly catches a target (probably calculated with beta=0 but still a target) because t+1 and t+16 for that asset is full, and calculations are based on data shifting features. So the question was for me: should I predict that asset in 'test phase', where there is no data for that asset in that minute. (dogecoin, timestamp=1555089780)
Beta is based off of 3750 values and 16 of which we don't know. This will cause problems as you stated, but there are a lot less NA values near the end of the dataframe and 16 out of 3750 is not much. There is also a high correlation between the actual target and log returns of a single coin. Factoring this all in, it will slightly affect the performance of your model, but there is nothing to do about it and the effect will not make a well trained model unusable. Just calculate beta using the previous 3734 values and ignore the futures one.
\nI think the only solution would be for Gresearch to change the target calculation so that beta doesn't become 0 when there is an NA, but that is not going to happen.
", "votes": null, "replies": [ { "id": 1648248, "author_name": "abdulkadirguner", "author_url": "", "post_date": "01/13/2022 09:25:23", "content": "Thanks. This helps.
\nI have to say it all started when I looked at the famous target-reconstuction notebooks and saw a nonexisting timestamp t in data for an asset a, which suddenly catches a target (probably calculated with beta=0 but still a target) because t+1 and t+16 for that asset is full, and calculations are based on data shifting features. So the question was for me: should I predict that asset in 'test phase', where there is no data for that asset in that minute. (dogecoin, timestamp=1555089780)