My first baseline guess was the last known 15 minute return, calculated price(0) / price(-15).
\nFor completeness, this is how I converted my predicted returns to the competition target, r - bm.
\nr: price(0) / price(-15)
\nb: 0 if nan return in previous 3734 days (lag 15, since the most recent 15 don't have knows 15 minute returns yet), otherwise b calculated over the last 3734 days.
\nm: calculated using r prediction for all assets.
To my surprise this scored -.029 on the public leaderboard, and -.03 on my cv (roughly last 5 known 12 week periods). It was so bad that it was good. By flipping my prediction to -price(0) / price(-15), the score was reversed to .029 lb, .03 cv.
\nBy feeding only this feature to lightgbm I could only get around .022 or something. Even adding x day returns for x in (1 -32), it was only up to .025 or so. It took quite a lot features before lgbm could beat my simple baseline. I'm not sure about neural networks, but to me its interesting to see that lgbm couldn't beat the baseline easy. It makes me optimistic that there lots of clever advancements to be made in machine learning algorithms.
", "messages": [ { "id": 1673678, "postDate": "2022-02-02T21:59:16.847Z", "content": "My first baseline guess was the last known 15 minute return, calculated price(0) / price(-15).
\nFor completeness, this is how I converted my predicted returns to the competition target, r - bm.
\nr: price(0) / price(-15)
\nb: 0 if nan return in previous 3734 days (lag 15, since the most recent 15 don't have knows 15 minute returns yet), otherwise b calculated over the last 3734 days.
\nm: calculated using r prediction for all assets.
To my surprise this scored -.029 on the public leaderboard, and -.03 on my cv (roughly last 5 known 12 week periods). It was so bad that it was good. By flipping my prediction to -price(0) / price(-15), the score was reversed to .029 lb, .03 cv.
\nBy feeding only this feature to lightgbm I could only get around .022 or something. Even adding x day returns for x in (1 -32), it was only up to .025 or so. It took quite a lot features before lgbm could beat my simple baseline. I'm not sure about neural networks, but to me its interesting to see that lgbm couldn't beat the baseline easy. It makes me optimistic that there lots of clever advancements to be made in machine learning algorithms.
", "rawMarkdown": "My first baseline guess was the last known 15 minute return, calculated price(0) / price(-15). \n\nFor completeness, this is how I converted my predicted returns to the competition target, r - bm. \nr: price(0) / price(-15)\nb: 0 if nan return in previous 3734 days (lag 15, since the most recent 15 don't have knows 15 minute returns yet), otherwise b calculated over the last 3734 days.\nm: calculated using r prediction for all assets.\n\nTo my surprise this scored -.029 on the public leaderboard, and -.03 on my cv (roughly last 5 known 12 week periods). It was so bad that it was good. By flipping my prediction to -price(0) / price(-15), the score was reversed to .029 lb, .03 cv.\n\nBy feeding only this feature to lightgbm I could only get around .022 or something. Even adding x day returns for x in (1 -32), it was only up to .025 or so. It took quite a lot features before lgbm could beat my simple baseline. I'm not sure about neural networks, but to me its interesting to see that lgbm couldn't beat the baseline easy. It makes me optimistic that there lots of clever advancements to be made in machine learning algorithms.", "votes": 10 }, { "id": 1677280, "postDate": "2022-02-05T16:13:41.380Z", "content": "I did this too! It's a very reasonable benchmark if you believe there is mean reversion (on a relative basis!).
", "rawMarkdown": "I did this too! It's a very reasonable benchmark if you believe there is mean reversion (on a relative basis!). ", "votes": 1 }, { "id": 1676729, "postDate": "2022-02-05T09:09:30.967Z", "content": "The return is calculated using minutes instead of days.
", "rawMarkdown": "The return is calculated using minutes instead of days.", "votes": 1, "replies": [ { "id": 1677417, "postDate": "2022-02-05T17:47:43.553Z", "content": "Yes you're right, thank you. I fixed the typo
", "rawMarkdown": "Yes you're right, thank you. I fixed the typo" } ] }, { "id": 1673680, "postDate": "2022-02-02T22:00:55.907Z", "content": "You should back test this with multiple testing periods to ensure this wasn't just due to chance.
", "rawMarkdown": "You should back test this with multiple testing periods to ensure this wasn't just due to chance.", "votes": 1, "replies": [ { "id": 1673684, "postDate": "2022-02-02T22:01:58.770Z", "content": "I did, it scored .03 average of the past 5 known periods. It scored between .023 and .05!
", "rawMarkdown": "I did, it scored .03 average of the past 5 known periods. It scored between .023 and .05!", "votes": 2 } ] }, { "id": 1673706, "postDate": "2022-02-02T22:40:38.490Z", "content": "I had this as a feature (among many others) and got up to 0.075 out of sample on some of the assets.
\nIts simply just a low volatility thing I believe, which is why it worked better for some than others. That corr will drop significantly if the market trends very hard in one direction which might happen next 3 months..
", "rawMarkdown": "I had this as a feature (among many others) and got up to 0.075 out of sample on some of the assets. \n\nIts simply just a low volatility thing I believe, which is why it worked better for some than others. That corr will drop significantly if the market trends very hard in one direction which might happen next 3 months..", "votes": 1 }, { "id": 1680495, "postDate": "2022-02-07T21:12:43.117Z", "content": "Assume you have r(t), beta and m(t) for your original setup (positive return). In the negative case, r(t) will change sign, m(t) will change sign, and beta will stay the same.
\nIn other words the target you will get from the return formulation of (- price(0) / price(-15)) is the negative of the target you will get from price(0) / price(-15).
\nAnd as long as the 'pearson correlation coefficient' is concerned, in this case, shouldn't you get the negative of the positive case value? What am I missing?
", "rawMarkdown": "Assume you have r(t), beta and m(t) for your original setup (positive return). In the negative case, r(t) will change sign, m(t) will change sign, and beta will stay the same. \n\nIn other words the target you will get from the return formulation of (**-** price(0) / price(-15)) is the negative of the target you will get from price(0) / price(-15). \n\nAnd as long as the 'pearson correlation coefficient' is concerned, in this case, shouldn't you get the negative of the positive case value? What am I missing?", "replies": [ { "id": 1681023, "postDate": "2022-02-08T07:19:37.197Z", "content": "I don't think you are missing anything. You are exactly right, the score was the negative of the positive case prediction I first made. If you are wondering why I posted this at all, its because I thought it was cool that a simple baseline had such a strong correlation, and its funny to me that my first guess of \"just predict the last instance\", as is normal in time series actually gave a worse score than 0, which you would get if you predicted randomly. I also thought it was interesting that this baseline scored better than a lot of baselines with algorithms, and that lgbm took a lot features before it could match the same score.
", "rawMarkdown": "I don't think you are missing anything. You are exactly right, the score was the negative of the positive case prediction I first made. If you are wondering why I posted this at all, its because I thought it was cool that a simple baseline had such a strong correlation, and its funny to me that my first guess of \"just predict the last instance\", as is normal in time series actually gave a worse score than 0, which you would get if you predicted randomly. I also thought it was interesting that this baseline scored better than a lot of baselines with algorithms, and that lgbm took a lot features before it could match the same score." } ] }, { "id": 1674576, "postDate": "2022-02-03T15:52:48.357Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1677280, "author_name": "Chase", "author_url": "", "post_date": "2022-02-05T16:13:41.380000", "content": "I did this too! It's a very reasonable benchmark if you believe there is mean reversion (on a relative basis!).
", "votes": 1, "replies": [] }, { "id": 1676729, "author_name": "agenlu", "author_url": "", "post_date": "2022-02-05T09:09:30.967000", "content": "The return is calculated using minutes instead of days.
", "votes": 1, "replies": [ { "id": 1677417, "author_name": "Chris Miles", "author_url": "", "post_date": "2022-02-05T17:47:43.553000", "content": "Yes you're right, thank you. I fixed the typo
", "votes": 0, "replies": [] } ] }, { "id": 1673680, "author_name": "Kyle Peters", "author_url": "", "post_date": "2022-02-02T22:00:55.907000", "content": "You should back test this with multiple testing periods to ensure this wasn't just due to chance.
", "votes": 1, "replies": [ { "id": 1673684, "author_name": "Chris Miles", "author_url": "", "post_date": "2022-02-02T22:01:58.770000", "content": "I did, it scored .03 average of the past 5 known periods. It scored between .023 and .05!
", "votes": 2, "replies": [] } ] }, { "id": 1673706, "author_name": "JM", "author_url": "", "post_date": "2022-02-02T22:40:38.490000", "content": "I had this as a feature (among many others) and got up to 0.075 out of sample on some of the assets.
\nIts simply just a low volatility thing I believe, which is why it worked better for some than others. That corr will drop significantly if the market trends very hard in one direction which might happen next 3 months..
", "votes": 1, "replies": [] }, { "id": 1680495, "author_name": "GUNER", "author_url": "", "post_date": "2022-02-07T21:12:43.117000", "content": "Assume you have r(t), beta and m(t) for your original setup (positive return). In the negative case, r(t) will change sign, m(t) will change sign, and beta will stay the same.
\nIn other words the target you will get from the return formulation of (- price(0) / price(-15)) is the negative of the target you will get from price(0) / price(-15).
\nAnd as long as the 'pearson correlation coefficient' is concerned, in this case, shouldn't you get the negative of the positive case value? What am I missing?
", "votes": 0, "replies": [ { "id": 1681023, "author_name": "Chris Miles", "author_url": "", "post_date": "2022-02-08T07:19:37.197000", "content": "I don't think you are missing anything. You are exactly right, the score was the negative of the positive case prediction I first made. If you are wondering why I posted this at all, its because I thought it was cool that a simple baseline had such a strong correlation, and its funny to me that my first guess of \"just predict the last instance\", as is normal in time series actually gave a worse score than 0, which you would get if you predicted randomly. I also thought it was interesting that this baseline scored better than a lot of baselines with algorithms, and that lgbm took a lot features before it could match the same score.
", "votes": 0, "replies": [] } ] }, { "id": 1674576, "author_name": "", "author_url": "", "post_date": "2022-02-03T15:52:48.357000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1673678": "My first baseline guess was the last known 15 minute return, calculated price(0) / price(-15). \n\nFor completeness, this is how I converted my predicted returns to the competition target, r - bm. \nr: price(0) / price(-15)\nb: 0 if nan return in previous 3734 days (lag 15, since the most recent 15 don't have knows 15 minute returns yet), otherwise b calculated over the last 3734 days.\nm: calculated using r prediction for all assets.\n\nTo my surprise this scored -.029 on the public leaderboard, and -.03 on my cv (roughly last 5 known 12 week periods). It was so bad that it was good. By flipping my prediction to -price(0) / price(-15), the score was reversed to .029 lb, .03 cv.\n\nBy feeding only this feature to lightgbm I could only get around .022 or something. Even adding x day returns for x in (1 -32), it was only up to .025 or so. It took quite a lot features before lgbm could beat my simple baseline. I'm not sure about neural networks, but to me its interesting to see that lgbm couldn't beat the baseline easy. It makes me optimistic that there lots of clever advancements to be made in machine learning algorithms.", "1677280": "I did this too! It's a very reasonable benchmark if you believe there is mean reversion (on a relative basis!). ", "1676729": "The return is calculated using minutes instead of days.", "1673680": "You should back test this with multiple testing periods to ensure this wasn't just due to chance.", "1673706": "I had this as a feature (among many others) and got up to 0.075 out of sample on some of the assets. \n\nIts simply just a low volatility thing I believe, which is why it worked better for some than others. That corr will drop significantly if the market trends very hard in one direction which might happen next 3 months..", "1680495": "Assume you have r(t), beta and m(t) for your original setup (positive return). In the negative case, r(t) will change sign, m(t) will change sign, and beta will stay the same. \n\nIn other words the target you will get from the return formulation of (**-** price(0) / price(-15)) is the negative of the target you will get from price(0) / price(-15). \n\nAnd as long as the 'pearson correlation coefficient' is concerned, in this case, shouldn't you get the negative of the positive case value? What am I missing?", "1674576": "" } }