![\"\"](\"https://i.postimg.cc/cHrx6Cm0/upd14-1-4.jpg\")
\n
![\"\"](\"https://i.postimg.cc/9MKW928T/upd13-2-4.jpg\")
\n
![\"\"](\"https://i.postimg.cc/yNf6Vc0L/upd16-2-3.jpg\")
\n
![\"\"](\"https://i.postimg.cc/4ycXb0q0/upd12-3-4.jpg\")
It's interesting to analyze the leaderboard, trying to find some pattern in the models’ performance, here is a try.
\nFrom update 1 to 2 it was a major movement between the scores for many, maybe due to small evaluation set, so that can maybe be excluded from deeper analysis but the rest maybe something can be plotted.
\nThe updates between 1-2 is in a relative flat short term market, first bear then bull to the same level, update 2-3 pure bearish, but the update 3-4 is in a 60 degrees bull period, maybe that can affect the models if trained/validated more bullish, flat, or bearish condition.
\nI did a try to train one model with less bear and the second submission with all data which has more of flat/bear market than data before June last year. The first model did better in update 3-4 and the second in update 2-3, so it might have some effect or just coincident, but the ultimate should have been that they worked well in both 😉
\nBelow are the LB top 30 and its score changes between the updates.
\n
\n
\n
\n
It's interesting to analyze the leaderboard, trying to find some pattern in the models’ performance, here is a try.
\nFrom update 1 to 2 it was a major movement between the scores for many, maybe due to small evaluation set, so that can maybe be excluded from deeper analysis but the rest maybe something can be plotted.
\nThe updates between 1-2 is in a relative flat short term market, first bear then bull to the same level, update 2-3 pure bearish, but the update 3-4 is in a 60 degrees bull period, maybe that can affect the models if trained/validated more bullish, flat, or bearish condition.
\nI did a try to train one model with less bear and the second submission with all data which has more of flat/bear market than data before June last year. The first model did better in update 3-4 and the second in update 2-3, so it might have some effect or just coincident, but the ultimate should have been that they worked well in both 😉
\nBelow are the LB top 30 and its score changes between the updates.
\n
\n
\n
\n
Some other interesting graphs
\nBest score change, excluding scores below 0
\n![\"\"](\"https://i.postimg.cc/FHPHyzZd/ch-in-score.jpg\")
Total LB distribution
\n![\"\"](\"https://i.postimg.cc/SKJND4TZ/upd11-distr.jpg\")
There are still large shake ups with every update. So I guess nothing is final until the last version. I have also made a notebook recently, showing some shake up plots if you are interested in that :)
\nhttps://www.kaggle.com/competitions/g-research-crypto-forecasting/discussion/313417
Thanks for the link, in DS we can't have enough EDAs, graphs and plots ;)
\nYes lots of shake ups still to come. Meme Lord Capital team is more or less stable, impressive! I think in general those trained with the whole data have advantage in bear market and vice versa in bull, just a guess of course by study my own models…
Meme Lord Capital seem to have a very robust model, hope they share a little bit of info on their approach!
", "rawMarkdown": "Meme Lord Capital seem to have a very robust model, hope they share a little bit of info on their approach!", "votes": null }, { "id": "1750299", "postDate": "04/09/2022 13:53:22", "content": "That is right! Your graphs also go a bit more into detail with individual performance. In the beginning I speculated whether a bear or bull market is good for my model but after my ranking improved with every update, I have no idea 😄.
", "rawMarkdown": "That is right! Your graphs also go a bit more into detail with individual performance. In the beginning I speculated whether a bear or bull market is good for my model but after my ranking improved with every update, I have no idea 😄.", "votes": null }, { "id": "1752018", "postDate": "04/11/2022 10:20:35", "content": "Looks like -60 degree bearish from last update, maybe something for a trend-following model or/and models that did well between upd. 2-3, let's see on wednesday. Then we have the robust models that handles them all it seems ;) playing in an own league.
", "rawMarkdown": "Looks like -60 degree bearish from last update, maybe something for a trend-following model or/and models that did well between upd. 2-3, let's see on wednesday. Then we have the robust models that handles them all it seems ;) playing in an own league.", "votes": null }, { "id": "1754003", "postDate": "04/13/2022 09:44:52", "content": "Maybe an indication of the models performance relative to each other and LB/model trend, did an experimental plot below. Only based on 3 updates and specific market conditions, but anyway:
\n![\"\"](\"https://i.postimg.cc/gJSnCZXf/trend1.jpg\")
Some other interesting graphs
\nBest score change, excluding scores below 0
\n
Total LB distribution
\n
There are still large shake ups with every update. So I guess nothing is final until the last version. I have also made a notebook recently, showing some shake up plots if you are interested in that :)
\nhttps://www.kaggle.com/competitions/g-research-crypto-forecasting/discussion/313417
Thanks for the link, in DS we can't have enough EDAs, graphs and plots ;)
\nYes lots of shake ups still to come. Meme Lord Capital team is more or less stable, impressive! I think in general those trained with the whole data have advantage in bear market and vice versa in bull, just a guess of course by study my own models…
That is right! Your graphs also go a bit more into detail with individual performance. In the beginning I speculated whether a bear or bull market is good for my model but after my ranking improved with every update, I have no idea 😄.
", "votes": null, "replies": [] } ] }, { "id": 1749193, "author_name": "julianmukaj", "author_url": "", "post_date": "04/08/2022 10:27:26", "content": "Meme Lord Capital seem to have a very robust model, hope they share a little bit of info on their approach!
", "votes": null, "replies": [] }, { "id": 1752018, "author_name": "kirderf", "author_url": "", "post_date": "04/11/2022 10:20:35", "content": "Looks like -60 degree bearish from last update, maybe something for a trend-following model or/and models that did well between upd. 2-3, let's see on wednesday. Then we have the robust models that handles them all it seems ;) playing in an own league.
", "votes": null, "replies": [] }, { "id": 1754003, "author_name": "kirderf", "author_url": "", "post_date": "04/13/2022 09:44:52", "content": "Maybe an indication of the models performance relative to each other and LB/model trend, did an experimental plot below. Only based on 3 updates and specific market conditions, but anyway:
\n