![\"\"](\"https://raw.githubusercontent.com/jorijnsmit/msc-thesis-kaggle-jane-street/main/figures/cv_purged.png\")
When speaking about this competition, there are two main options for validation that comes to ming:
\nGroupKFold or PurgedGroupTimeSeriesSplit
\nWhere the main Idea of each is:
\nGroupKFold: We split the dataset into groups defined by the \"date\" of the samples. so there will never be a date that is both in the training set and in the validation set.
PurgedGroupTimeSeriesSplit: We again split the dataset into groups defined by the \"date\" of the samples. so there will never be a date that is both in the training set and in the validation set. However this time we also make sure that the dates are ordered in a chronological way (as they are in real life). Now! this method is easy to overfit since consecutive dates often strongly correlate to each other. so to fix this we take some \"days off\" between the different sets. Those dates we call \"purged\" since we \"purge\" the time series.
It is still not 100% clear by this stage what should we use but I converted a great notebook from the Jane Street competition earlier this year by @jorijnsmit [Go upvote him a bit ok? He earned it with respect for his this work on Jane Street].
\nOn Jane Street many people had used PurgedGroupTimeSeriesSplit [I think even the winning solution but don't catch me by the word].
\nAnyhow, the notebook is right here: https://www.kaggle.com/yamqwe/let-s-talk-validation-grouptimeseriessplit.
\nEnjoy!
\nHappy Kaggling!
\nCheers ^^
When speaking about this competition, there are two main options for validation that comes to ming:
\nGroupKFold or PurgedGroupTimeSeriesSplit
\nWhere the main Idea of each is:
\nGroupKFold: We split the dataset into groups defined by the \"date\" of the samples. so there will never be a date that is both in the training set and in the validation set.
PurgedGroupTimeSeriesSplit: We again split the dataset into groups defined by the \"date\" of the samples. so there will never be a date that is both in the training set and in the validation set. However this time we also make sure that the dates are ordered in a chronological way (as they are in real life). Now! this method is easy to overfit since consecutive dates often strongly correlate to each other. so to fix this we take some \"days off\" between the different sets. Those dates we call \"purged\" since we \"purge\" the time series.
It is still not 100% clear by this stage what should we use but I converted a great notebook from the Jane Street competition earlier this year by @jorijnsmit [Go upvote him a bit ok? He earned it with respect for his this work on Jane Street].
\nOn Jane Street many people had used PurgedGroupTimeSeriesSplit [I think even the winning solution but don't catch me by the word].
\nAnyhow, the notebook is right here: https://www.kaggle.com/yamqwe/let-s-talk-validation-grouptimeseriessplit.
\nEnjoy!
\nHappy Kaggling!
\nCheers ^^
cheers for the mention @yamqwe!
\nthe winner of the jane street competition indeed ended up using it: https://www.kaggle.com/c/jane-street-market-prediction/discussion/224348
\noriginal notebook in which i introduce the method: https://www.kaggle.com/jorijnsmit/found-the-holy-grail-grouptimeseriessplit
\nthe method also ended up in my master thesis. here's a diagram i made, hopefully giving a bit better intuition into what it does exactly:
\n
(from p.4 of https://github.com/jorijnsmit/msc-thesis-kaggle-jane-street/blob/main/Msc_Thesis-Smit-Predicting%20Profitability_of_Trades_in_Noisy_and_Obfuscated_Grouped_Time_Series-FINAL.pdf, with better explanation and more figures)
", "rawMarkdown": "cheers for the mention @yamqwe!\n\nthe winner of the jane street competition indeed ended up using it: https://www.kaggle.com/c/jane-street-market-prediction/discussion/224348\n\noriginal notebook in which i introduce the method: https://www.kaggle.com/jorijnsmit/found-the-holy-grail-grouptimeseriessplit\n\nthe method also ended up in my master thesis. here's a diagram i made, hopefully giving a bit better intuition into what it does exactly:\n\n\n\n(from p.4 of https://github.com/jorijnsmit/msc-thesis-kaggle-jane-street/blob/main/Msc_Thesis-Smit-Predicting%20Profitability_of_Trades_in_Noisy_and_Obfuscated_Grouped_Time_Series-FINAL.pdf, with better explanation and more figures)", "votes": 4 }, { "id": 1575690, "postDate": "2021-11-08T15:22:29.370Z", "content": "(to be fair, i am not fully aware of current status of this method in scikit-learn. maybe do some digging based on my links to see if there are better source code solutions to it already)
", "rawMarkdown": "(to be fair, i am not fully aware of current status of this method in scikit-learn. maybe do some digging based on my links to see if there are better source code solutions to it already)", "votes": 1 }, { "id": 1571406, "postDate": "2021-11-04T21:05:31.500Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1575688, "author_name": "J.J.H. Smit", "author_url": "", "post_date": "2021-11-08T15:21:24.287000", "content": "cheers for the mention @yamqwe!
\nthe winner of the jane street competition indeed ended up using it: https://www.kaggle.com/c/jane-street-market-prediction/discussion/224348
\noriginal notebook in which i introduce the method: https://www.kaggle.com/jorijnsmit/found-the-holy-grail-grouptimeseriessplit
\nthe method also ended up in my master thesis. here's a diagram i made, hopefully giving a bit better intuition into what it does exactly:
\n
(from p.4 of https://github.com/jorijnsmit/msc-thesis-kaggle-jane-street/blob/main/Msc_Thesis-Smit-Predicting%20Profitability_of_Trades_in_Noisy_and_Obfuscated_Grouped_Time_Series-FINAL.pdf, with better explanation and more figures)
", "votes": 4, "replies": [] }, { "id": 1575690, "author_name": "J.J.H. Smit", "author_url": "", "post_date": "2021-11-08T15:22:29.370000", "content": "(to be fair, i am not fully aware of current status of this method in scikit-learn. maybe do some digging based on my links to see if there are better source code solutions to it already)
", "votes": 1, "replies": [] }, { "id": 1571406, "author_name": "", "author_url": "", "post_date": "2021-11-04T21:05:31.500000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1570695": "When speaking about this competition, there are two main options for validation that comes to ming: \n\n**GroupKFold** or **PurgedGroupTimeSeriesSplit**\n\nWhere the main Idea of each is:\n\n- GroupKFold: We split the dataset into groups defined by the \"date\" of the samples. so there will never be a date that is both in the training set and in the validation set.\n\n- PurgedGroupTimeSeriesSplit: We again split the dataset into groups defined by the \"date\" of the samples. so there will never be a date that is both in the training set and in the validation set. However this time we also make sure that the dates are ordered in a chronological way (as they are in real life). Now! this method is easy to overfit since consecutive dates often strongly correlate to each other. so to fix this we take some \"days off\" between the different sets. Those dates we call \"purged\" since we \"purge\" the time series.\n\nIt is still not 100% clear by this stage what should we use but I converted a great notebook from the Jane Street competition earlier this year by @jorijnsmit [Go upvote him a bit ok? He earned it with respect for his this work on Jane Street].\n\nOn Jane Street many people had used PurgedGroupTimeSeriesSplit [I think even the winning solution but don't catch me by the word].\n\nAnyhow, the notebook is right here: https://www.kaggle.com/yamqwe/let-s-talk-validation-grouptimeseriessplit.\n\nEnjoy! \n\nHappy Kaggling! \nCheers ^^\n", "1575688": "cheers for the mention @yamqwe!\n\nthe winner of the jane street competition indeed ended up using it: https://www.kaggle.com/c/jane-street-market-prediction/discussion/224348\n\noriginal notebook in which i introduce the method: https://www.kaggle.com/jorijnsmit/found-the-holy-grail-grouptimeseriessplit\n\nthe method also ended up in my master thesis. here's a diagram i made, hopefully giving a bit better intuition into what it does exactly:\n\n\n\n(from p.4 of https://github.com/jorijnsmit/msc-thesis-kaggle-jane-street/blob/main/Msc_Thesis-Smit-Predicting%20Profitability_of_Trades_in_Noisy_and_Obfuscated_Grouped_Time_Series-FINAL.pdf, with better explanation and more figures)", "1575690": "(to be fair, i am not fully aware of current status of this method in scikit-learn. maybe do some digging based on my links to see if there are better source code solutions to it already)", "1571406": "" } }