{ "id": 323250, "title": "7th place solution", "url": "/competitions/g-research-crypto-forecasting/discussion/323250", "author_name": "Patrick Yam", "post_date": "2022-05-05T13:50:08.734000", "votes": 74, "comment_count": 29, "views": 0, "content": "

First of all, thank you G-Research and Kaggle for hosting the competition. I really like the data provided in this competition, actual market data give us more room for creative method development and is much more interesting compared with anonymized tabular data.

\n

My approach is 99% on modeling and the only feature I added is the time of day. Here is a very brief description of my model, the final submission is an ensemble of 4 models trained with different sequence lengths.
\n\"\"

", "messages": [ { "id": 3160139, "postDate": "2025-03-26T12:22:24.243Z", "content": "

Thanks for sharing this great solution @wimwim. I was wondering how many training samples in total did you have for the (90, 14, 9) tensors. Did you consider the whole span of years (2018-2021) or only the most recent period?

", "rawMarkdown": "Thanks for sharing this great solution @wimwim. I was wondering how many training samples in total did you have for the (90, 14, 9) tensors. Did you consider the whole span of years (2018-2021) or only the most recent period? ", "votes": 1 }, { "id": 1778606, "postDate": "2022-05-05T13:50:08.733Z", "content": "

First of all, thank you G-Research and Kaggle for hosting the competition. I really like the data provided in this competition, actual market data give us more room for creative method development and is much more interesting compared with anonymized tabular data.

\n

My approach is 99% on modeling and the only feature I added is the time of day. Here is a very brief description of my model, the final submission is an ensemble of 4 models trained with different sequence lengths.
\n\"\"

", "rawMarkdown": "First of all, thank you G-Research and Kaggle for hosting the competition. I really like the data provided in this competition, actual market data give us more room for creative method development and is much more interesting compared with anonymized tabular data.\n\nMy approach is 99% on modeling and the only feature I added is the time of day. Here is a very brief description of my model, the final submission is an ensemble of 4 models trained with different sequence lengths.\n![](https://i.imgur.com/ECta6Oe.png)", "votes": 74 }, { "id": 1780544, "postDate": "2022-05-07T14:39:56.783Z", "content": "

Congratulations! I am really amazed when I see solutions without heavy feature engineering that use well designed deep learning models. Thank you for sharing your solution.

", "rawMarkdown": "Congratulations! I am really amazed when I see solutions without heavy feature engineering that use well designed deep learning models. Thank you for sharing your solution.", "votes": 3 }, { "id": 1778830, "postDate": "2022-05-05T17:58:56.577Z", "content": "

Wow, that's so different from what I did, really cool.

", "rawMarkdown": "Wow, that's so different from what I did, really cool.", "votes": 4, "replies": [ { "id": 1779590, "postDate": "2022-05-06T15:45:25.600Z", "content": "

Congrats on 2nd place!

", "rawMarkdown": "Congrats on 2nd place!", "votes": 2 } ] }, { "id": 1827340, "postDate": "2022-06-21T02:05:45.323Z", "content": "

I came across your solution and found it wonderful. Congrats to you!

\n

I'd like to ask, how did your model normalize input?

", "rawMarkdown": "I came across your solution and found it wonderful. Congrats to you!\n\nI'd like to ask, how did your model normalize input? ", "votes": 1, "replies": [ { "id": 1827815, "postDate": "2022-06-21T10:04:26.443Z", "content": "

Thanks for the question! The scale for features is very different between different assets and different timestamps, I decided to normalize input using only the data from the input sequence with the same asset (90 data points for each feature) to compute the mean and std.

", "rawMarkdown": "Thanks for the question! The scale for features is very different between different assets and different timestamps, I decided to normalize input using only the data from the input sequence with the same asset (90 data points for each feature) to compute the mean and std.", "votes": 4 } ] }, { "id": 1779535, "postDate": "2022-05-06T14:55:37.267Z", "content": "

did the author delete some content?

", "rawMarkdown": "did the author delete some content?", "votes": 1, "replies": [ { "id": 1779592, "postDate": "2022-05-06T15:46:52.280Z", "content": "

No, I didn't delete any content. If you can't see the picture, it could be because Imgur is blocked by your internet provider.

", "rawMarkdown": "No, I didn't delete any content. If you can't see the picture, it could be because Imgur is blocked by your internet provider.", "votes": 1 }, { "id": 1779612, "postDate": "2022-05-06T16:22:42.073Z", "content": "

I see, thanks.

", "rawMarkdown": "I see, thanks." } ] }, { "id": 1779408, "postDate": "2022-05-06T11:57:39.997Z", "content": "

Wait, what? No added features and just solving the problem with modeling? If such things are really possible I get back to learning ;)

", "rawMarkdown": "Wait, what? No added features and just solving the problem with modeling? If such things are really possible I get back to learning ;)", "votes": 1 }, { "id": 1778930, "postDate": "2022-05-05T23:09:57.840Z", "content": "

Great..🙌 lot to learn for beginner like me..Thanks for sharing this.. please share code also when you can.

", "rawMarkdown": "Great..🙌 lot to learn for beginner like me..Thanks for sharing this.. please share code also when you can.", "votes": 1 }, { "id": 1778912, "postDate": "2022-05-05T21:29:04.603Z", "content": "

Great result for no extra features

", "rawMarkdown": "Great result for no extra features", "votes": 1 }, { "id": 1782408, "postDate": "2022-05-09T15:00:00.797Z", "content": "

congratulations and thanks for sharing the content. Is it available to access the code?

", "rawMarkdown": "congratulations and thanks for sharing the content. Is it available to access the code? ", "votes": 2 }, { "id": 1779236, "postDate": "2022-05-06T08:18:08.357Z", "content": "

1stals- Thank you for the great writeup! I'm glad you enjoyed the data - I think it's some of the most interesting we've had on Kaggle in a while.

\n

It sounds like you have a really strong understanding of the data and how to model it. Do you have any advice for other Kagglers who are just starting out?

\n

Thanks again!\"

\n

This is a great writeup, thanks for sharing! Your approach sounds very robust and I'm sure it was a lot of work. Do you have any advice for other Kagglers who are just starting out?

", "rawMarkdown": "1stals- Thank you for the great writeup! I'm glad you enjoyed the data - I think it's some of the most interesting we've had on Kaggle in a while.\n\nIt sounds like you have a really strong understanding of the data and how to model it. Do you have any advice for other Kagglers who are just starting out?\n\nThanks again!\"\n\nThis is a great writeup, thanks for sharing! Your approach sounds very robust and I'm sure it was a lot of work. Do you have any advice for other Kagglers who are just starting out?", "votes": 2, "replies": [ { "id": 1779587, "postDate": "2022-05-06T15:42:32.013Z", "content": "

Thanks for your question! I always start by checking the data shape (i.e. is the data tabular? 1d? or 2d?), then pick some suitable neural network layers (e.g. LSTM/ Conv/ MHA) for it. I would say Transformer is a very strong candidate for data >= 1D, definitely give it a try.
\nI will perform feature engineering only when the model cannot 'see' the information from the input. For example, Transformer can't handle input with a very long sequence length, so we will have to create features to capture long-term information.

", "rawMarkdown": "Thanks for your question! I always start by checking the data shape (i.e. is the data tabular? 1d? or 2d?), then pick some suitable neural network layers (e.g. LSTM/ Conv/ MHA) for it. I would say Transformer is a very strong candidate for data >= 1D, definitely give it a try.\nI will perform feature engineering only when the model cannot 'see' the information from the input. For example, Transformer can't handle input with a very long sequence length, so we will have to create features to capture long-term information.", "votes": 7 } ] }, { "id": 1779096, "postDate": "2022-05-06T05:37:16.500Z", "content": "

Congrats on solo gold and prize my friend @wimwim

", "rawMarkdown": "Congrats on solo gold and prize my friend @wimwim ", "votes": 2, "replies": [ { "id": 1779588, "postDate": "2022-05-06T15:43:38.230Z", "content": "

Thanks for your kind words, my friend!

", "rawMarkdown": "Thanks for your kind words, my friend!" } ] }, { "id": 2074702, "postDate": "2022-12-24T14:05:46.190Z", "content": "

Congrats Patrick!

", "rawMarkdown": "Congrats Patrick!" }, { "id": 1800222, "postDate": "2022-05-24T19:06:44.393Z", "content": "

Congrats for your robust solution @wimwim!

\n

Can you comment a little bit about the CV strategy you used?

", "rawMarkdown": "Congrats for your robust solution @wimwim! \n\nCan you comment a little bit about the CV strategy you used?", "replies": [ { "id": 1800441, "postDate": "2022-05-25T01:21:59.773Z", "content": "

I use 3-fold Grouped CV and the key is timestamps.

", "rawMarkdown": "I use 3-fold Grouped CV and the key is timestamps.", "votes": 2 } ] }, { "id": 1799918, "postDate": "2022-05-24T12:01:01.273Z", "content": "

Thank you for sharing your approach. Wish I had the same understanding of neural net models! What kind of preprocessing did you use on price features or was it as simple as MinMaxScaler on train?

", "rawMarkdown": "Thank you for sharing your approach. Wish I had the same understanding of neural net models! What kind of preprocessing did you use on price features or was it as simple as MinMaxScaler on train?", "replies": [ { "id": 1799954, "postDate": "2022-05-24T12:51:22.067Z", "content": "

Thanks for the question! I use standardization for input scaling. I find that the features scaling could be very different between different assets (e.g. ETH vs BTC) and different timestamps (2020 vs 2021), so I calculate the mean and std for each asset and only use the data from the input sequence (90 data points) to make the model more stable and robust.

", "rawMarkdown": "Thanks for the question! I use standardization for input scaling. I find that the features scaling could be very different between different assets (e.g. ETH vs BTC) and different timestamps (2020 vs 2021), so I calculate the mean and std for each asset and only use the data from the input sequence (90 data points) to make the model more stable and robust.", "votes": 3 } ] }, { "id": 1779366, "postDate": "2022-05-06T10:47:31.477Z", "content": "

Ingenious! Congratulations on your score and 7th place!

", "rawMarkdown": "Ingenious! Congratulations on your score and 7th place!" }, { "id": 1833873, "postDate": "2022-06-26T12:15:18.240Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1784948, "postDate": "2022-05-11T16:08:53.213Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1782674, "postDate": "2022-05-09T19:02:01.937Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1779416, "postDate": "2022-05-06T12:12:07.823Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3045596, "postDate": "2024-11-14T16:04:44.130Z", "content": "

Great work. thank you for sharing

", "rawMarkdown": "Great work. thank you for sharing" }, { "id": 1779836, "postDate": "2022-05-06T21:43:19.167Z", "content": "

Great solution, thanks for sharing

", "rawMarkdown": "Great solution, thanks for sharing" } ], "comments": [ { "id": 3160139, "author_name": "Vitor", "author_url": "", "post_date": "2025-03-26T12:22:24.243000", "content": "

Thanks for sharing this great solution @wimwim. I was wondering how many training samples in total did you have for the (90, 14, 9) tensors. Did you consider the whole span of years (2018-2021) or only the most recent period?

", "votes": 1, "replies": [] }, { "id": 1780544, "author_name": "MarkFromItaly", "author_url": "", "post_date": "2022-05-07T14:39:56.783000", "content": "

Congratulations! I am really amazed when I see solutions without heavy feature engineering that use well designed deep learning models. Thank you for sharing your solution.

", "votes": 3, "replies": [] }, { "id": 1778830, "author_name": "Nathaniel Maddux", "author_url": "", "post_date": "2022-05-05T17:58:56.577000", "content": "

Wow, that's so different from what I did, really cool.

", "votes": 4, "replies": [ { "id": 1779590, "author_name": "Patrick Yam", "author_url": "", "post_date": "2022-05-06T15:45:25.600000", "content": "

Congrats on 2nd place!

", "votes": 2, "replies": [] } ] }, { "id": 1827340, "author_name": "katsu1110", "author_url": "", "post_date": "2022-06-21T02:05:45.323000", "content": "

I came across your solution and found it wonderful. Congrats to you!

\n

I'd like to ask, how did your model normalize input?

", "votes": 1, "replies": [ { "id": 1827815, "author_name": "Patrick Yam", "author_url": "", "post_date": "2022-06-21T10:04:26.443000", "content": "

Thanks for the question! The scale for features is very different between different assets and different timestamps, I decided to normalize input using only the data from the input sequence with the same asset (90 data points for each feature) to compute the mean and std.

", "votes": 4, "replies": [] } ] }, { "id": 1779535, "author_name": "yuanzhe zhou", "author_url": "", "post_date": "2022-05-06T14:55:37.267000", "content": "

did the author delete some content?

", "votes": 1, "replies": [ { "id": 1779592, "author_name": "Patrick Yam", "author_url": "", "post_date": "2022-05-06T15:46:52.280000", "content": "

No, I didn't delete any content. If you can't see the picture, it could be because Imgur is blocked by your internet provider.

", "votes": 1, "replies": [] }, { "id": 1779612, "author_name": "yuanzhe zhou", "author_url": "", "post_date": "2022-05-06T16:22:42.073000", "content": "

I see, thanks.

", "votes": 0, "replies": [] } ] }, { "id": 1779408, "author_name": "Ugibugibu", "author_url": "", "post_date": "2022-05-06T11:57:39.997000", "content": "

Wait, what? No added features and just solving the problem with modeling? If such things are really possible I get back to learning ;)

", "votes": 1, "replies": [] }, { "id": 1778930, "author_name": "Chirag Desai", "author_url": "", "post_date": "2022-05-05T23:09:57.840000", "content": "

Great..🙌 lot to learn for beginner like me..Thanks for sharing this.. please share code also when you can.

", "votes": 1, "replies": [] }, { "id": 1778912, "author_name": "JM", "author_url": "", "post_date": "2022-05-05T21:29:04.603000", "content": "

Great result for no extra features

", "votes": 1, "replies": [] }, { "id": 1782408, "author_name": "Milad A Shani", "author_url": "", "post_date": "2022-05-09T15:00:00.797000", "content": "

congratulations and thanks for sharing the content. Is it available to access the code?

", "votes": 2, "replies": [] }, { "id": 1779236, "author_name": "", "author_url": "", "post_date": "2022-05-06T08:18:08.357000", "content": "

1stals- Thank you for the great writeup! I'm glad you enjoyed the data - I think it's some of the most interesting we've had on Kaggle in a while.

\n

It sounds like you have a really strong understanding of the data and how to model it. Do you have any advice for other Kagglers who are just starting out?

\n

Thanks again!\"

\n

This is a great writeup, thanks for sharing! Your approach sounds very robust and I'm sure it was a lot of work. Do you have any advice for other Kagglers who are just starting out?

", "votes": 2, "replies": [ { "id": 1779587, "author_name": "Patrick Yam", "author_url": "", "post_date": "2022-05-06T15:42:32.013000", "content": "

Thanks for your question! I always start by checking the data shape (i.e. is the data tabular? 1d? or 2d?), then pick some suitable neural network layers (e.g. LSTM/ Conv/ MHA) for it. I would say Transformer is a very strong candidate for data >= 1D, definitely give it a try.
\nI will perform feature engineering only when the model cannot 'see' the information from the input. For example, Transformer can't handle input with a very long sequence length, so we will have to create features to capture long-term information.

", "votes": 7, "replies": [] } ] }, { "id": 1779096, "author_name": "KhanhVD", "author_url": "", "post_date": "2022-05-06T05:37:16.500000", "content": "

Congrats on solo gold and prize my friend @wimwim

", "votes": 2, "replies": [ { "id": 1779588, "author_name": "Patrick Yam", "author_url": "", "post_date": "2022-05-06T15:43:38.230000", "content": "

Thanks for your kind words, my friend!

", "votes": 0, "replies": [] } ] }, { "id": 2074702, "author_name": "HKMLGroup", "author_url": "", "post_date": "2022-12-24T14:05:46.190000", "content": "

Congrats Patrick!

", "votes": 0, "replies": [] }, { "id": 1800222, "author_name": "mavillan", "author_url": "", "post_date": "2022-05-24T19:06:44.393000", "content": "

Congrats for your robust solution @wimwim!

\n

Can you comment a little bit about the CV strategy you used?

", "votes": 0, "replies": [ { "id": 1800441, "author_name": "Patrick Yam", "author_url": "", "post_date": "2022-05-25T01:21:59.773000", "content": "

I use 3-fold Grouped CV and the key is timestamps.

", "votes": 2, "replies": [] } ] }, { "id": 1799918, "author_name": "devcon14", "author_url": "", "post_date": "2022-05-24T12:01:01.273000", "content": "

Thank you for sharing your approach. Wish I had the same understanding of neural net models! What kind of preprocessing did you use on price features or was it as simple as MinMaxScaler on train?

", "votes": 0, "replies": [ { "id": 1799954, "author_name": "Patrick Yam", "author_url": "", "post_date": "2022-05-24T12:51:22.067000", "content": "

Thanks for the question! I use standardization for input scaling. I find that the features scaling could be very different between different assets (e.g. ETH vs BTC) and different timestamps (2020 vs 2021), so I calculate the mean and std for each asset and only use the data from the input sequence (90 data points) to make the model more stable and robust.

", "votes": 3, "replies": [] } ] }, { "id": 1779366, "author_name": "LG", "author_url": "", "post_date": "2022-05-06T10:47:31.477000", "content": "

Ingenious! Congratulations on your score and 7th place!

", "votes": 0, "replies": [] }, { "id": 1833873, "author_name": "", "author_url": "", "post_date": "2022-06-26T12:15:18.240000", "content": "", "votes": 0, "replies": [] }, { "id": 1784948, "author_name": "", "author_url": "", "post_date": "2022-05-11T16:08:53.213000", "content": "", "votes": 0, "replies": [] }, { "id": 1782674, "author_name": "", "author_url": "", "post_date": "2022-05-09T19:02:01.937000", "content": "", "votes": 0, "replies": [] }, { "id": 1779416, "author_name": "", "author_url": "", "post_date": "2022-05-06T12:12:07.823000", "content": "", "votes": 0, "replies": [] }, { "id": 3045596, "author_name": "Ilya", "author_url": "", "post_date": "2024-11-14T16:04:44.130000", "content": "

Great work. thank you for sharing

", "votes": 0, "replies": [] }, { "id": 1779836, "author_name": "bturan19", "author_url": "", "post_date": "2022-05-06T21:43:19.167000", "content": "

Great solution, thanks for sharing

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3160139": "Thanks for sharing this great solution @wimwim. I was wondering how many training samples in total did you have for the (90, 14, 9) tensors. Did you consider the whole span of years (2018-2021) or only the most recent period? ", "1778606": "First of all, thank you G-Research and Kaggle for hosting the competition. I really like the data provided in this competition, actual market data give us more room for creative method development and is much more interesting compared with anonymized tabular data.\n\nMy approach is 99% on modeling and the only feature I added is the time of day. Here is a very brief description of my model, the final submission is an ensemble of 4 models trained with different sequence lengths.\n![](https://i.imgur.com/ECta6Oe.png)", "1780544": "Congratulations! I am really amazed when I see solutions without heavy feature engineering that use well designed deep learning models. Thank you for sharing your solution.", "1778830": "Wow, that's so different from what I did, really cool.", "1827340": "I came across your solution and found it wonderful. Congrats to you!\n\nI'd like to ask, how did your model normalize input? ", "1779535": "did the author delete some content?", "1779408": "Wait, what? No added features and just solving the problem with modeling? If such things are really possible I get back to learning ;)", "1778930": "Great..🙌 lot to learn for beginner like me..Thanks for sharing this.. please share code also when you can.", "1778912": "Great result for no extra features", "1782408": "congratulations and thanks for sharing the content. Is it available to access the code? ", "1779236": "1stals- Thank you for the great writeup! I'm glad you enjoyed the data - I think it's some of the most interesting we've had on Kaggle in a while.\n\nIt sounds like you have a really strong understanding of the data and how to model it. Do you have any advice for other Kagglers who are just starting out?\n\nThanks again!\"\n\nThis is a great writeup, thanks for sharing! Your approach sounds very robust and I'm sure it was a lot of work. Do you have any advice for other Kagglers who are just starting out?", "1779096": "Congrats on solo gold and prize my friend @wimwim ", "2074702": "Congrats Patrick!", "1800222": "Congrats for your robust solution @wimwim! \n\nCan you comment a little bit about the CV strategy you used?", "1799918": "Thank you for sharing your approach. Wish I had the same understanding of neural net models! What kind of preprocessing did you use on price features or was it as simple as MinMaxScaler on train?", "1779366": "Ingenious! Congratulations on your score and 7th place!", "1833873": "", "1784948": "", "1782674": "", "1779416": "", "3045596": "Great work. thank you for sharing", "1779836": "Great solution, thanks for sharing" } }