Take a look at this paper \"Do We Really Need Deep Learning Models for Time Series Forecasting?\"
\nMy first point is that, perhaps, different time-series(tabular data) need to be classified into different classes. and solved somewhat differently. Yet the same top-notch Kagglers can find effective solutions for very different time-series data. In particular, two TS (time-series) competitions - Enefit-2024 and Optiver-2024 - are now in the concluding and private-evaluation stages. Enefit-2024 has time-periodic and high SNR (signal-noise ratio) data, while Optiver-2024 has low-SNR, financial, and multiple-regimes data.
\nMy second point is that ensemble of gbdt and DL works really well, and probably for good reasons? So by the very least, we need DL for ensembles of time-series forecasting, even if DL is not the best single model by itself. This is my own recent experience in Enefit-2024 and see the ensemble from 1st public place.
\nIt would be great to hear the opinions on the above topic and competitions/data comparison from the current and future)) Grandmasters: @hydantess @yuanzhezhou @aerdem4 @nimashahbazi @mathurinache @jacobyjaeger @cody11null @vitalykudelya
", "messages": [ { "id": "2635327", "postDate": "02/04/2024 10:35:45", "content": "Take a look at this paper \"Do We Really Need Deep Learning Models for Time Series Forecasting?\"
\nMy first point is that, perhaps, different time-series(tabular data) need to be classified into different classes. and solved somewhat differently. Yet the same top-notch Kagglers can find effective solutions for very different time-series data. In particular, two TS (time-series) competitions - Enefit-2024 and Optiver-2024 - are now in the concluding and private-evaluation stages. Enefit-2024 has time-periodic and high SNR (signal-noise ratio) data, while Optiver-2024 has low-SNR, financial, and multiple-regimes data.
\nMy second point is that ensemble of gbdt and DL works really well, and probably for good reasons? So by the very least, we need DL for ensembles of time-series forecasting, even if DL is not the best single model by itself. This is my own recent experience in Enefit-2024 and see the ensemble from 1st public place.
\nIt would be great to hear the opinions on the above topic and competitions/data comparison from the current and future)) Grandmasters: @hydantess @yuanzhezhou @aerdem4 @nimashahbazi @mathurinache @jacobyjaeger @cody11null @vitalykudelya
", "rawMarkdown": "Take a look at this paper [\"Do We Really Need Deep Learning Models for Time Series Forecasting?\"](https://arxiv.org/abs/2101.02118)\n\nMy first point is that, perhaps, different time-series(tabular data) need to be [classified into different classes.](https://www.kaggle.com/competitions/bench-tab-v1/discussion/469406#2634094) and **solved somewhat differently.** Yet the same top-notch Kagglers can find effective solutions for very different time-series data. In particular, two TS (time-series) competitions - Enefit-2024 and Optiver-2024 - are now in the concluding and private-evaluation stages. [Enefit-2024](https://www.kaggle.com/competitions/predict-energy-behavior-of-prosumers) has time-periodic and high SNR (signal-noise ratio) data, while [Optiver-2024](https://www.kaggle.com/competitions/optiver-trading-at-the-close/discussion/462369#2567415) has low-SNR, financial, and multiple-regimes data.\n\nMy second point is that **ensemble of gbdt and DL works really well**, and probably for good reasons? So by the very least, **we need DL for ensembles of time-series forecasting,** even if DL is not the best single model by itself. This is my own recent experience in Enefit-2024 and see [the ensemble from 1st public place.](https://www.kaggle.com/competitions/predict-energy-behavior-of-prosumers/discussion/472793)\n\nIt would be great to hear the opinions on the above topic and competitions/data comparison from the current and future)) Grandmasters: @hydantess @yuanzhezhou @aerdem4 @nimashahbazi @mathurinache @jacobyjaeger @cody11null @vitalykudelya", "votes": null }, { "id": "2635663", "postDate": "02/04/2024 14:46:20", "content": "I feel like the state of ML for TS is in a quite similar position as tabular data. Lot of litterature about NN architectures then in any serious competition they'll lag behind the usual FE + gbdt (and sometimes vanilla arch.).
", "rawMarkdown": "I feel like the state of ML for TS is in a quite similar position as tabular data. Lot of litterature about NN architectures then in any serious competition they'll lag behind the usual FE + gbdt (and sometimes vanilla arch.).", "votes": null }, { "id": "2635721", "postDate": "02/04/2024 15:07:02", "content": "Another thought - perhaps, your competition should be a SINGLE model competition, shouldn't it? Perhaps, effectively this is so anyway with naïve runs in the range of 20 hours
\nPerhaps, SINGLE model out of the box even? If people start using model-ensembles even here than it will confuse the problem/question.
", "rawMarkdown": "Another thought - perhaps, your competition should be a SINGLE model competition, shouldn't it? Perhaps, effectively this is so anyway with naïve runs in the range of 20 hours\n\nPerhaps, SINGLE model out of the box even? If people start using model-ensembles even here than it will confuse the problem/question.", "votes": null }, { "id": "2635731", "postDate": "02/04/2024 15:10:44", "content": "interesting - Literature versus competitions seems to be researchers (driven by fashions and funding and focused on the literature) versus practitioners (driven by speed/time and convenience) - wdyt? Eventually, but maybe after long time, the common solution will emerge … or not if quite new approaches/solutions appear, and people will forget old questions
", "rawMarkdown": "interesting - Literature versus competitions seems to be researchers (driven by fashions and funding and focused on the literature) versus practitioners (driven by speed/time and convenience) - wdyt? Eventually, but maybe after long time, the common solution will emerge ... or not if quite new approaches/solutions appear, and people will forget old questions", "votes": null }, { "id": "2635747", "postDate": "02/04/2024 15:33:45", "content": "I can't really enforce anything… so it's difficult to define and apply custom rules.
", "rawMarkdown": "I can't really enforce anything... so it's difficult to define and apply custom rules.", "votes": null }, { "id": "2635775", "postDate": "02/04/2024 15:51:59", "content": "Private dataset usually mean no competition, while publication is driven by apparent novelty and marginal performance gain. It incentivize some weird/bad practices (creating new models instead of improving existing ones, not properly benchmarking the models against existing ones).
", "rawMarkdown": "Private dataset usually mean no competition, while publication is driven by apparent novelty and marginal performance gain. It incentivize some weird/bad practices (creating new models instead of improving existing ones, not properly benchmarking the models against existing ones).", "votes": null }, { "id": "2635830", "postDate": "02/04/2024 16:18:22", "content": "agree
\n\"private dataset mean no competition\" - not clear, pls expand
", "rawMarkdown": "agree\n\n\"private dataset mean no competition\" - not clear, pls expand", "votes": null }, { "id": "2653381", "postDate": "02/15/2024 12:02:53", "content": "people that publish a paper on a private dataset didn't have competition to improve their score as they do their research on their own
", "rawMarkdown": "people that publish a paper on a private dataset didn't have competition to improve their score as they do their research on their own", "votes": null } ], "comments": [ { "id": 2635663, "author_name": "lucasmorin", "author_url": "", "post_date": "02/04/2024 14:46:20", "content": "I feel like the state of ML for TS is in a quite similar position as tabular data. Lot of litterature about NN architectures then in any serious competition they'll lag behind the usual FE + gbdt (and sometimes vanilla arch.).
", "votes": null, "replies": [ { "id": 2635721, "author_name": "mihailturlakov", "author_url": "", "post_date": "02/04/2024 15:07:02", "content": "Another thought - perhaps, your competition should be a SINGLE model competition, shouldn't it? Perhaps, effectively this is so anyway with naïve runs in the range of 20 hours
\nPerhaps, SINGLE model out of the box even? If people start using model-ensembles even here than it will confuse the problem/question.
", "votes": null, "replies": [ { "id": 2635747, "author_name": "lucasmorin", "author_url": "", "post_date": "02/04/2024 15:33:45", "content": "I can't really enforce anything… so it's difficult to define and apply custom rules.
", "votes": null, "replies": [] } ] }, { "id": 2635731, "author_name": "mihailturlakov", "author_url": "", "post_date": "02/04/2024 15:10:44", "content": "interesting - Literature versus competitions seems to be researchers (driven by fashions and funding and focused on the literature) versus practitioners (driven by speed/time and convenience) - wdyt? Eventually, but maybe after long time, the common solution will emerge … or not if quite new approaches/solutions appear, and people will forget old questions
", "votes": null, "replies": [ { "id": 2635775, "author_name": "lucasmorin", "author_url": "", "post_date": "02/04/2024 15:51:59", "content": "Private dataset usually mean no competition, while publication is driven by apparent novelty and marginal performance gain. It incentivize some weird/bad practices (creating new models instead of improving existing ones, not properly benchmarking the models against existing ones).
", "votes": null, "replies": [ { "id": 2635830, "author_name": "mihailturlakov", "author_url": "", "post_date": "02/04/2024 16:18:22", "content": "agree
\n\"private dataset mean no competition\" - not clear, pls expand
", "votes": null, "replies": [ { "id": 2653381, "author_name": "lucasmorin", "author_url": "", "post_date": "02/15/2024 12:02:53", "content": "people that publish a paper on a private dataset didn't have competition to improve their score as they do their research on their own
", "votes": null, "replies": [] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2635327": "Take a look at this paper [\"Do We Really Need Deep Learning Models for Time Series Forecasting?\"](https://arxiv.org/abs/2101.02118)\n\nMy first point is that, perhaps, different time-series(tabular data) need to be [classified into different classes.](https://www.kaggle.com/competitions/bench-tab-v1/discussion/469406#2634094) and **solved somewhat differently.** Yet the same top-notch Kagglers can find effective solutions for very different time-series data. In particular, two TS (time-series) competitions - Enefit-2024 and Optiver-2024 - are now in the concluding and private-evaluation stages. [Enefit-2024](https://www.kaggle.com/competitions/predict-energy-behavior-of-prosumers) has time-periodic and high SNR (signal-noise ratio) data, while [Optiver-2024](https://www.kaggle.com/competitions/optiver-trading-at-the-close/discussion/462369#2567415) has low-SNR, financial, and multiple-regimes data.\n\nMy second point is that **ensemble of gbdt and DL works really well**, and probably for good reasons? So by the very least, **we need DL for ensembles of time-series forecasting,** even if DL is not the best single model by itself. This is my own recent experience in Enefit-2024 and see [the ensemble from 1st public place.](https://www.kaggle.com/competitions/predict-energy-behavior-of-prosumers/discussion/472793)\n\nIt would be great to hear the opinions on the above topic and competitions/data comparison from the current and future)) Grandmasters: @hydantess @yuanzhezhou @aerdem4 @nimashahbazi @mathurinache @jacobyjaeger @cody11null @vitalykudelya", "2635663": "I feel like the state of ML for TS is in a quite similar position as tabular data. Lot of litterature about NN architectures then in any serious competition they'll lag behind the usual FE + gbdt (and sometimes vanilla arch.).", "2635721": "Another thought - perhaps, your competition should be a SINGLE model competition, shouldn't it? Perhaps, effectively this is so anyway with naïve runs in the range of 20 hours\n\nPerhaps, SINGLE model out of the box even? If people start using model-ensembles even here than it will confuse the problem/question.", "2635731": "interesting - Literature versus competitions seems to be researchers (driven by fashions and funding and focused on the literature) versus practitioners (driven by speed/time and convenience) - wdyt? Eventually, but maybe after long time, the common solution will emerge ... or not if quite new approaches/solutions appear, and people will forget old questions", "2635747": "I can't really enforce anything... so it's difficult to define and apply custom rules.", "2635775": "Private dataset usually mean no competition, while publication is driven by apparent novelty and marginal performance gain. It incentivize some weird/bad practices (creating new models instead of improving existing ones, not properly benchmarking the models against existing ones).", "2635830": "agree\n\n\"private dataset mean no competition\" - not clear, pls expand", "2653381": "people that publish a paper on a private dataset didn't have competition to improve their score as they do their research on their own" }, "source": "meta" }