{ "id": 303515, "title": "Autoregressive Model Pearsonr 0.74, Please provide the historical Target!", "url": "/competitions/g-research-crypto-forecasting/discussion/303515", "author_name": "", "post_date": "2022-01-28T02:10:04.595054400Z", "votes": null, "comment_count": 6, "views": 0, "content": "

Hi, Organizers:
\nI have build an Autoregressive(e.g AR) Model which get Pearsonr 0.74 on test data (split(supplemental_train.csv) -> train eval and test), I have carefully mask off current time Target

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
datasetLGBMAR/non-AR
volatility0.440.55 / 0.1341
crypto0.046490.74 / 0.01443
\n\n

Can you provide the historical Target as env.predict() return values?

\n
    sample_prediction_df['Target'] = 0  # make your predictions here\n    targets = env.predict(sample_prediction_df)   # register your predictions\n
\n

Best,

", "messages": [ { "id": "1666698", "postDate": "01/28/2022 02:10:04", "content": "

Hi, Organizers:
\nI have build an Autoregressive(e.g AR) Model which get Pearsonr 0.74 on test data (split(supplemental_train.csv) -> train eval and test), I have carefully mask off current time Target

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
datasetLGBMAR/non-AR
volatility0.440.55 / 0.1341
crypto0.046490.74 / 0.01443
\n\n

Can you provide the historical Target as env.predict() return values?

\n
    sample_prediction_df['Target'] = 0  # make your predictions here\n    targets = env.predict(sample_prediction_df)   # register your predictions\n
\n

Best,

", "rawMarkdown": "Hi, Organizers:\nI have build an Autoregressive(e.g AR) Model which get Pearsonr 0.74 on test data (split(supplemental_train.csv) -> train eval and test), I have carefully mask off current time Target\n\n| dataset | LGBM | AR/non-AR |\n| ----- | ---- | ---- |\n| volatility | 0.44 | **0.55 / 0.1341** |\n| crypto | 0.04649 | **0.74 / 0.01443** |\n* non-AR = mask off historical Target\n\nCan you provide the historical Target as env.predict() return values? \n\n```\n sample_prediction_df['Target'] = 0 # make your predictions here\n targets = env.predict(sample_prediction_df) # register your predictions\n```\n\nBest,", "votes": null }, { "id": "1666704", "postDate": "01/28/2022 02:16:10", "content": "

@andrewscherer Is this possible?

", "rawMarkdown": "andrewscherer Is this possible?", "votes": null }, { "id": "1666722", "postDate": "01/28/2022 02:47:39", "content": "

I'm afraid it is not a good idea, especially when the competition is ending. It would be unfair to others if the host changes the rule.

", "rawMarkdown": "I'm afraid it is not a good idea, especially when the competition is ending. It would be unfair to others if the host changes the rule.", "votes": null }, { "id": "1666800", "postDate": "01/28/2022 04:26:41", "content": "

The reason you get a high correlation is because the target contains information about the next 15 periods. So, you are leaking target information into later training rows.

", "rawMarkdown": "The reason you get a high correlation is because the target contains information about the next 15 periods. So, you are leaking target information into later training rows.", "votes": null }, { "id": "1667093", "postDate": "01/28/2022 09:59:54", "content": "

The target is built using data from the future. So it is normal that they do not provide it during inference.

", "rawMarkdown": "The target is built using data from the future. So it is normal that they do not provide it during inference.", "votes": null }, { "id": "1667456", "postDate": "01/28/2022 16:08:57", "content": "

The same thing for me, I wish them to update data for each week or month at least in the period of Submission. Because we can't use the Forecasting models without updating Data.

", "rawMarkdown": "The same thing for me, I wish them to update data for each week or month at least in the period of Submission. Because we can't use the Forecasting models without updating Data.", "votes": null }, { "id": "1667746", "postDate": "01/28/2022 22:00:07", "content": "

0.74 is impossible, it has either overfitting or leakage

", "rawMarkdown": "0.74 is impossible, it has either overfitting or leakage", "votes": null } ], "comments": [ { "id": 1666704, "author_name": "feitengli", "author_url": "", "post_date": "01/28/2022 02:16:10", "content": "

@andrewscherer Is this possible?

", "votes": null, "replies": [] }, { "id": 1666722, "author_name": "hhhhhhhhhz", "author_url": "", "post_date": "01/28/2022 02:47:39", "content": "

I'm afraid it is not a good idea, especially when the competition is ending. It would be unfair to others if the host changes the rule.

", "votes": null, "replies": [] }, { "id": 1666800, "author_name": "petersk20", "author_url": "", "post_date": "01/28/2022 04:26:41", "content": "

The reason you get a high correlation is because the target contains information about the next 15 periods. So, you are leaking target information into later training rows.

", "votes": null, "replies": [] }, { "id": 1667093, "author_name": "joelqv", "author_url": "", "post_date": "01/28/2022 09:59:54", "content": "

The target is built using data from the future. So it is normal that they do not provide it during inference.

", "votes": null, "replies": [] }, { "id": 1667456, "author_name": "youneseloiarm", "author_url": "", "post_date": "01/28/2022 16:08:57", "content": "

The same thing for me, I wish them to update data for each week or month at least in the period of Submission. Because we can't use the Forecasting models without updating Data.

", "votes": null, "replies": [] }, { "id": 1667746, "author_name": "liyangchenrice", "author_url": "", "post_date": "01/28/2022 22:00:07", "content": "

0.74 is impossible, it has either overfitting or leakage

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1666698": "Hi, Organizers:\nI have build an Autoregressive(e.g AR) Model which get Pearsonr 0.74 on test data (split(supplemental_train.csv) -> train eval and test), I have carefully mask off current time Target\n\n| dataset | LGBM | AR/non-AR |\n| ----- | ---- | ---- |\n| volatility | 0.44 | **0.55 / 0.1341** |\n| crypto | 0.04649 | **0.74 / 0.01443** |\n* non-AR = mask off historical Target\n\nCan you provide the historical Target as env.predict() return values? \n\n```\n sample_prediction_df['Target'] = 0 # make your predictions here\n targets = env.predict(sample_prediction_df) # register your predictions\n```\n\nBest,", "1666704": "andrewscherer Is this possible?", "1666722": "I'm afraid it is not a good idea, especially when the competition is ending. It would be unfair to others if the host changes the rule.", "1666800": "The reason you get a high correlation is because the target contains information about the next 15 periods. So, you are leaking target information into later training rows.", "1667093": "The target is built using data from the future. So it is normal that they do not provide it during inference.", "1667456": "The same thing for me, I wish them to update data for each week or month at least in the period of Submission. Because we can't use the Forecasting models without updating Data.", "1667746": "0.74 is impossible, it has either overfitting or leakage" }, "source": "meta" }