Hi everyone,
\nThis competition was weird in multiple aspects. One of them was about notebook sharing. As crypto is a hot topic I expected this competition to get a bit more traction and sharing. Maybe this is due to non-anonymised feature with low signal to noise ratio (nothing much to explore / exploit), available LB (possible overfitting), or money incentives (putting your own model into production).
\nIt turns out this was also due to some user publishing a lot of poor notebooks (non training, non converging, non submitting), sucking air from everyone else. Those notebooks weren't entirely wrong but showed some serious shortcomings. Now that kaggle has taken some action against this user, this is more obvious: by removing 20+ of his notebooks, there is only 18 golden notebooks in this competition. This is very few: Ubiquant competition, not even halfway trough, already has 15 golden notebooks.
\nHonestly I feel like this isn't due to a lack of good notebooks. So I have made a list of notebooks that showed some serious work or some new or unusual idea that in my opinion actually deserve a lot more attention and votes:
\nGenerally good EDAs:
\nhttps://www.kaggle.com/mishki/tutorial-time-series-exploration-crypto-price#7.-Power-transforms
\nhttps://www.kaggle.com/tensorchoko/g-research-crypto-forecasting-eda
\nhttps://www.kaggle.com/neomatrix369/everything-you-can-do-with-a-time-series-cryptos
Good EDA + whole Technical Analysis:
\nhttps://www.kaggle.com/kalilurrahman/g-res-crypto-eda-and-forecast#-6-Basic-MACD--(for-Bit-Coin)
Absolutely MAD crypto dashboard:
\nhttps://www.kaggle.com/toomuchsauce/g-crypto-interactive-dashboard-indicators
GM actually sharing good hindsigths/baselines:
\nLive EDA: https://www.kaggle.com/robikscube/predict-crypto-prices-twitch-stream
\nCausality test: https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie
\nTime in the market v.s. timing the market: https://www.kaggle.com/carlmcbrideellis/irrational-exuberance-s-p-vs-bitcoin
\nNN baseline: https://www.kaggle.com/berserker408/crypto-tf-keras-nn-with-asset-embedding
Target design (nearly half of the target weren't properly calculated) - this help by showing what to do (and what not to do) in real life:
\nhttps://www.kaggle.com/gengdaiziwang/are-we-training-models-on-an-inaccurate-target
CPCV - a rare attempt at crosss validation:
\nhttps://www.kaggle.com/osamurai/implementation-cpcv
Local API emulator to test your submissions locally:
\nhttps://www.kaggle.com/jagofc/local-api-emulator
LSTM training - serious attempt at deep learning:
\nhttps://www.kaggle.com/vmuzhichenko/g-research-parallel-lstm-training
Crypto sentiment analysis not usefull for this competition but getting signal from alternative data is generally how you make money in real life:
\nhttps://www.kaggle.com/minhajulhoque/reddit-crytocurrency-sentiment-analyser
Last but not least, a complete feature engineering pipeline. This one is a serie of notebooks, completely implementing data engineering, technical factors, how to design and optimise them, how to secure the whole process… etc.He hasn't a golden ring and this is not fancy ML stuff. But it is pro-level code bridging the gap between technical analysis and its practical use for machine learning. It is absolutely unbelievable that some of these notebooks don't even get to bronze. If you have missed those, you should really check them. Congrats @axzhang for doing and sharing such serious work.
\nPipeline Building-FeatureEngineering-Stage1: https://www.kaggle.com/axzhang/pipeline-building-featureengineering-stage1
\nPipeline Building-FeatureEngineering-Stage2: https://www.kaggle.com/axzhang/pipeline-building-featureengineering-stage2
\nFast and accurate online feature generator: https://www.kaggle.com/axzhang/fast-and-accurate-online-feature-generator
\nPipeline: Purged Kfold: https://www.kaggle.com/axzhang/pipeline-building-purgedkfold
This situation is a bit unfortunate but we can revert it with our upvotes. I am not asking you to upvote those blindly. But I suggest everyone to take a bit of time to learn from and support others. Feel free to share notebooks that I might have missed.
", "messages": [ { "id": "1697195", "postDate": "02/19/2022 12:45:22", "content": "Hi everyone,
\nThis competition was weird in multiple aspects. One of them was about notebook sharing. As crypto is a hot topic I expected this competition to get a bit more traction and sharing. Maybe this is due to non-anonymised feature with low signal to noise ratio (nothing much to explore / exploit), available LB (possible overfitting), or money incentives (putting your own model into production).
\nIt turns out this was also due to some user publishing a lot of poor notebooks (non training, non converging, non submitting), sucking air from everyone else. Those notebooks weren't entirely wrong but showed some serious shortcomings. Now that kaggle has taken some action against this user, this is more obvious: by removing 20+ of his notebooks, there is only 18 golden notebooks in this competition. This is very few: Ubiquant competition, not even halfway trough, already has 15 golden notebooks.
\nHonestly I feel like this isn't due to a lack of good notebooks. So I have made a list of notebooks that showed some serious work or some new or unusual idea that in my opinion actually deserve a lot more attention and votes:
\nGenerally good EDAs:
\nhttps://www.kaggle.com/mishki/tutorial-time-series-exploration-crypto-price#7.-Power-transforms
\nhttps://www.kaggle.com/tensorchoko/g-research-crypto-forecasting-eda
\nhttps://www.kaggle.com/neomatrix369/everything-you-can-do-with-a-time-series-cryptos
Good EDA + whole Technical Analysis:
\nhttps://www.kaggle.com/kalilurrahman/g-res-crypto-eda-and-forecast#-6-Basic-MACD--(for-Bit-Coin)
Absolutely MAD crypto dashboard:
\nhttps://www.kaggle.com/toomuchsauce/g-crypto-interactive-dashboard-indicators
GM actually sharing good hindsigths/baselines:
\nLive EDA: https://www.kaggle.com/robikscube/predict-crypto-prices-twitch-stream
\nCausality test: https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie
\nTime in the market v.s. timing the market: https://www.kaggle.com/carlmcbrideellis/irrational-exuberance-s-p-vs-bitcoin
\nNN baseline: https://www.kaggle.com/berserker408/crypto-tf-keras-nn-with-asset-embedding
Target design (nearly half of the target weren't properly calculated) - this help by showing what to do (and what not to do) in real life:
\nhttps://www.kaggle.com/gengdaiziwang/are-we-training-models-on-an-inaccurate-target
CPCV - a rare attempt at crosss validation:
\nhttps://www.kaggle.com/osamurai/implementation-cpcv
Local API emulator to test your submissions locally:
\nhttps://www.kaggle.com/jagofc/local-api-emulator
LSTM training - serious attempt at deep learning:
\nhttps://www.kaggle.com/vmuzhichenko/g-research-parallel-lstm-training
Crypto sentiment analysis not usefull for this competition but getting signal from alternative data is generally how you make money in real life:
\nhttps://www.kaggle.com/minhajulhoque/reddit-crytocurrency-sentiment-analyser
Last but not least, a complete feature engineering pipeline. This one is a serie of notebooks, completely implementing data engineering, technical factors, how to design and optimise them, how to secure the whole process… etc.He hasn't a golden ring and this is not fancy ML stuff. But it is pro-level code bridging the gap between technical analysis and its practical use for machine learning. It is absolutely unbelievable that some of these notebooks don't even get to bronze. If you have missed those, you should really check them. Congrats @axzhang for doing and sharing such serious work.
\nPipeline Building-FeatureEngineering-Stage1: https://www.kaggle.com/axzhang/pipeline-building-featureengineering-stage1
\nPipeline Building-FeatureEngineering-Stage2: https://www.kaggle.com/axzhang/pipeline-building-featureengineering-stage2
\nFast and accurate online feature generator: https://www.kaggle.com/axzhang/fast-and-accurate-online-feature-generator
\nPipeline: Purged Kfold: https://www.kaggle.com/axzhang/pipeline-building-purgedkfold
This situation is a bit unfortunate but we can revert it with our upvotes. I am not asking you to upvote those blindly. But I suggest everyone to take a bit of time to learn from and support others. Feel free to share notebooks that I might have missed.
", "rawMarkdown": "Hi everyone,\n\nThis competition was weird in multiple aspects. One of them was about notebook sharing. As crypto is a hot topic I expected this competition to get a bit more traction and sharing. Maybe this is due to non-anonymised feature with low signal to noise ratio (nothing much to explore / exploit), available LB (possible overfitting), or money incentives (putting your own model into production). \n\nIt turns out this was also due to some user publishing a lot of poor notebooks (non training, non converging, non submitting), sucking air from everyone else. Those notebooks weren't entirely wrong but showed some serious shortcomings. Now that kaggle has taken some action against this user, this is more obvious: by removing 20+ of his notebooks, **there is only 18 golden notebooks in this competition. This is very few: Ubiquant competition, not even halfway trough, already has 15 golden notebooks.**\n\nHonestly I feel like this isn't due to a lack of good notebooks. So I have made a list of notebooks that showed some serious work or some new or unusual idea that in my opinion actually deserve a lot more attention and votes:\n\nGenerally good EDAs:\nhttps://www.kaggle.com/mishki/tutorial-time-series-exploration-crypto-price#7.-Power-transforms\nhttps://www.kaggle.com/tensorchoko/g-research-crypto-forecasting-eda\nhttps://www.kaggle.com/neomatrix369/everything-you-can-do-with-a-time-series-cryptos\n\nGood EDA + whole Technical Analysis:\nhttps://www.kaggle.com/kalilurrahman/g-res-crypto-eda-and-forecast#-6-Basic-MACD--(for-Bit-Coin)\n\nAbsolutely MAD crypto dashboard:\nhttps://www.kaggle.com/toomuchsauce/g-crypto-interactive-dashboard-indicators\n\nGM actually sharing good hindsigths/baselines:\nLive EDA: https://www.kaggle.com/robikscube/predict-crypto-prices-twitch-stream\nCausality test: https://www.kaggle.com/carlmcbrideellis/granger-causality-part-ii-the-movie\nTime in the market v.s. timing the market: https://www.kaggle.com/carlmcbrideellis/irrational-exuberance-s-p-vs-bitcoin\nNN baseline: https://www.kaggle.com/berserker408/crypto-tf-keras-nn-with-asset-embedding\n\nTarget design (nearly half of the target weren't properly calculated) - this help by showing what to do (and what not to do) in real life:\nhttps://www.kaggle.com/gengdaiziwang/are-we-training-models-on-an-inaccurate-target\n\nCPCV - a rare attempt at crosss validation:\nhttps://www.kaggle.com/osamurai/implementation-cpcv\n\nLocal API emulator to test your submissions locally:\nhttps://www.kaggle.com/jagofc/local-api-emulator\n\nLSTM training - serious attempt at deep learning:\nhttps://www.kaggle.com/vmuzhichenko/g-research-parallel-lstm-training\n\nCrypto sentiment analysis not usefull for this competition but getting signal from alternative data is generally how you make money in real life:\nhttps://www.kaggle.com/minhajulhoque/reddit-crytocurrency-sentiment-analyser\n\n**Last but not least, a complete feature engineering pipeline**. This one is a serie of notebooks, completely implementing data engineering, technical factors, how to design and optimise them, how to secure the whole process... etc.He hasn't a golden ring and this is not fancy ML stuff. But it is pro-level code bridging the gap between technical analysis and its practical use for machine learning. **It is absolutely unbelievable that some of these notebooks don't even get to bronze.** If you have missed those, you should really check them. Congrats @axzhang for doing and sharing such serious work.\nPipeline Building-FeatureEngineering-Stage1: https://www.kaggle.com/axzhang/pipeline-building-featureengineering-stage1\nPipeline Building-FeatureEngineering-Stage2: https://www.kaggle.com/axzhang/pipeline-building-featureengineering-stage2\nFast and accurate online feature generator: https://www.kaggle.com/axzhang/fast-and-accurate-online-feature-generator\nPipeline: Purged Kfold: https://www.kaggle.com/axzhang/pipeline-building-purgedkfold\n\n**This situation is a bit unfortunate but we can revert it with our upvotes. I am not asking you to upvote those blindly. But I suggest everyone to take a bit of time to learn from and support others. Feel free to share notebooks that I might have missed.**", "votes": null }, { "id": "1697313", "postDate": "02/19/2022 14:27:27", "content": "What happened to he-who-shall-not-be-named?
", "rawMarkdown": "What happened to he-who-shall-not-be-named?", "votes": null }, { "id": "1697428", "postDate": "02/19/2022 16:08:44", "content": "His profile cannot be found, probably banned.
", "rawMarkdown": "His profile cannot be found, probably banned.", "votes": null }, { "id": "1697485", "postDate": "02/19/2022 16:48:48", "content": "To be fair though, it is sort of nice when someone introduces a new idea but doesn't give other Kaggle participants everything. He gave a good template for people to follow and people had to put in a little bit of work to get it working correctly.
", "rawMarkdown": "To be fair though, it is sort of nice when someone introduces a new idea but doesn't give other Kaggle participants everything. He gave a good template for people to follow and people had to put in a little bit of work to get it working correctly.", "votes": null }, { "id": "1697507", "postDate": "02/19/2022 17:02:39", "content": "Honestly It looked more like some mass copy paste to speed run the notebook GM than someone distillating ideas in discussions.
", "rawMarkdown": "Honestly It looked more like some mass copy paste to speed run the notebook GM than someone distillating ideas in discussions.", "votes": null }, { "id": "1697543", "postDate": "02/19/2022 17:21:13", "content": "Who is this and what did I miss?
", "rawMarkdown": "Who is this and what did I miss?", "votes": null }, { "id": "1697562", "postDate": "02/19/2022 17:32:07", "content": "Check this.
", "rawMarkdown": "Check [this.](https://www.kaggle.com/general/293597#1660634)", "votes": null }, { "id": "1697571", "postDate": "02/19/2022 17:35:41", "content": "Could of been the case. I didn't really pay close attention to any of his work because it was in Tensorflow and I was using Pytorch.
", "rawMarkdown": "Could of been the case. I didn't really pay close attention to any of his work because it was in Tensorflow and I was using Pytorch.", "votes": null }, { "id": "1697580", "postDate": "02/19/2022 17:40:39", "content": "Ah thanks, someone should have told them competition GM is the only one that really matters anyway 😁
", "rawMarkdown": "Ah thanks, someone should have told them competition GM is the only one that really matters anyway 😁", "votes": null }, { "id": "1697586", "postDate": "02/19/2022 17:44:15", "content": "The only bad thing though is that I was going to use the external data he provided to test my submissions during the evaluation period. They haven't been updating since he was banned though. It's a shame because I feel like he did give some value but that was overshadowed by everything else.
", "rawMarkdown": "The only bad thing though is that I was going to use the external data he provided to test my submissions during the evaluation period. They haven't been updating since he was banned though. It's a shame because I feel like he did give some value but that was overshadowed by everything else.", "votes": null }, { "id": "1697894", "postDate": "02/20/2022 00:15:44", "content": "Wow, thanks for citing my notebooks. It's very nice of you :) thank you again
", "rawMarkdown": "Wow, thanks for citing my notebooks. It's very nice of you :) thank you again", "votes": null }, { "id": "1698267", "postDate": "02/20/2022 08:47:17", "content": "I don't think anyone got the exact data (Vwap missing mostly, but also base currency). Main difference with other people working on getting external data was he spent more energy advertising \"his\" stuff than showing the source / code / giving credit / identifying & adressing the shortcomings. Some of his design choices even seemed to be oriented toward SEO rather than quality / practicality of use.
", "rawMarkdown": "I don't think anyone got the exact data (Vwap missing mostly, but also base currency). Main difference with other people working on getting external data was he spent more energy advertising \"his\" stuff than showing the source / code / giving credit / identifying & adressing the shortcomings. Some of his design choices even seemed to be oriented toward SEO rather than quality / practicality of use.", "votes": null }, { "id": "1699411", "postDate": "02/21/2022 07:02:22", "content": "Yeah, Some of them are very underrated👍
", "rawMarkdown": "Yeah, Some of them are very underrated👍", "votes": null }, { "id": "1701174", "postDate": "02/22/2022 15:06:08", "content": "Yeah, Some of them are very underrated👍
", "rawMarkdown": "Yeah, Some of them are very underrated👍", "votes": null }, { "id": "1701214", "postDate": "02/22/2022 15:36:04", "content": "Who and why downvoted this comment? Some people are so weird.
", "rawMarkdown": "Who and why downvoted this comment? Some people are so weird.", "votes": null }, { "id": "1702807", "postDate": "02/24/2022 00:05:55", "content": "Thanks, @lucasmorin; I need to get back on track on this competition. This content will help me.
", "rawMarkdown": "Thanks, @lucasmorin; I need to get back on track on this competition. This content will help me.", "votes": null }, { "id": "1702891", "postDate": "02/24/2022 03:33:41", "content": "This has been very helpful. Thank you.
", "rawMarkdown": "This has been very helpful. Thank you.", "votes": null }, { "id": "1730299", "postDate": "03/21/2022 05:28:56", "content": "@lucasmorin
\nThank you for your raising the issue and introduction of my notebook.
\nHonestly, I would be happier if I could get a gold for this but I am happy because this gave me a first silver medal:)
What happened to he-who-shall-not-be-named?
", "votes": null, "replies": [ { "id": 1697428, "author_name": "vivovinco", "author_url": "", "post_date": "02/19/2022 16:08:44", "content": "His profile cannot be found, probably banned.
", "votes": null, "replies": [] }, { "id": 1697543, "author_name": "julianmukaj", "author_url": "", "post_date": "02/19/2022 17:21:13", "content": "Who is this and what did I miss?
", "votes": null, "replies": [] }, { "id": 1697562, "author_name": "vivovinco", "author_url": "", "post_date": "02/19/2022 17:32:07", "content": "Check this.
", "votes": null, "replies": [] }, { "id": 1697580, "author_name": "julianmukaj", "author_url": "", "post_date": "02/19/2022 17:40:39", "content": "Ah thanks, someone should have told them competition GM is the only one that really matters anyway 😁
", "votes": null, "replies": [] } ] }, { "id": 1697485, "author_name": "petersk20", "author_url": "", "post_date": "02/19/2022 16:48:48", "content": "To be fair though, it is sort of nice when someone introduces a new idea but doesn't give other Kaggle participants everything. He gave a good template for people to follow and people had to put in a little bit of work to get it working correctly.
", "votes": null, "replies": [ { "id": 1697507, "author_name": "lucasmorin", "author_url": "", "post_date": "02/19/2022 17:02:39", "content": "Honestly It looked more like some mass copy paste to speed run the notebook GM than someone distillating ideas in discussions.
", "votes": null, "replies": [] }, { "id": 1697571, "author_name": "petersk20", "author_url": "", "post_date": "02/19/2022 17:35:41", "content": "Could of been the case. I didn't really pay close attention to any of his work because it was in Tensorflow and I was using Pytorch.
", "votes": null, "replies": [] }, { "id": 1697586, "author_name": "petersk20", "author_url": "", "post_date": "02/19/2022 17:44:15", "content": "The only bad thing though is that I was going to use the external data he provided to test my submissions during the evaluation period. They haven't been updating since he was banned though. It's a shame because I feel like he did give some value but that was overshadowed by everything else.
", "votes": null, "replies": [] }, { "id": 1698267, "author_name": "lucasmorin", "author_url": "", "post_date": "02/20/2022 08:47:17", "content": "I don't think anyone got the exact data (Vwap missing mostly, but also base currency). Main difference with other people working on getting external data was he spent more energy advertising \"his\" stuff than showing the source / code / giving credit / identifying & adressing the shortcomings. Some of his design choices even seemed to be oriented toward SEO rather than quality / practicality of use.
", "votes": null, "replies": [] } ] }, { "id": 1697894, "author_name": "axzhang", "author_url": "", "post_date": "02/20/2022 00:15:44", "content": "Wow, thanks for citing my notebooks. It's very nice of you :) thank you again
", "votes": null, "replies": [] }, { "id": 1699411, "author_name": "imnoob", "author_url": "", "post_date": "02/21/2022 07:02:22", "content": "Yeah, Some of them are very underrated👍
", "votes": null, "replies": [] }, { "id": 1701174, "author_name": "marknicholasakyapogu", "author_url": "", "post_date": "02/22/2022 15:06:08", "content": "Yeah, Some of them are very underrated👍
", "votes": null, "replies": [ { "id": 1701214, "author_name": "vivovinco", "author_url": "", "post_date": "02/22/2022 15:36:04", "content": "Who and why downvoted this comment? Some people are so weird.
", "votes": null, "replies": [] } ] }, { "id": 1702807, "author_name": "cv13j0", "author_url": "", "post_date": "02/24/2022 00:05:55", "content": "Thanks, @lucasmorin; I need to get back on track on this competition. This content will help me.
", "votes": null, "replies": [] }, { "id": 1702891, "author_name": "kravella", "author_url": "", "post_date": "02/24/2022 03:33:41", "content": "This has been very helpful. Thank you.
", "votes": null, "replies": [] }, { "id": 1730299, "author_name": "osamurai", "author_url": "", "post_date": "03/21/2022 05:28:56", "content": "@lucasmorin
\nThank you for your raising the issue and introduction of my notebook.
\nHonestly, I would be happier if I could get a gold for this but I am happy because this gave me a first silver medal:)