{ "id": 284913, "title": "External \"Elon Musk's Tweets\" dataset allowed? πŸ˜‚", "url": "/competitions/g-research-crypto-forecasting/discussion/284913", "author_name": "", "post_date": "2021-11-02T20:23:26.739000", "votes": 85, "comment_count": 25, "views": 0, "content": "

Hi,

\n

Just a quick question about external data usage. it is allowed at all?
\nFor example, is it allowed to use some external dataset of.. opinions expressed by some individuals?
\nIf no internet is allowed, can we for example use the new kaggle feature of an auto-updating dataset to create an auto dataset of tweets and use it?

\n

Not that I have any specific individual whose tweets I might want to add into the model's predictions.. πŸ˜†

", "messages": [ { "id": 1568531, "postDate": "2021-11-02T20:23:26.740Z", "content": "

Hi,

\n

Just a quick question about external data usage. it is allowed at all?
\nFor example, is it allowed to use some external dataset of.. opinions expressed by some individuals?
\nIf no internet is allowed, can we for example use the new kaggle feature of an auto-updating dataset to create an auto dataset of tweets and use it?

\n

Not that I have any specific individual whose tweets I might want to add into the model's predictions.. πŸ˜†

", "rawMarkdown": "Hi, \n\nJust a quick question about external data usage. it is allowed at all? \nFor example, is it allowed to use some external dataset of.. opinions expressed by some individuals?\nIf no internet is allowed, can we for example use the new kaggle feature of an auto-updating dataset to create **an auto dataset of tweets and use it**? \n\nNot that I have any specific individual whose tweets I might want to add into the model's predictions.. πŸ˜†", "votes": 85 }, { "id": 1568566, "postDate": "2021-11-02T21:15:45.740Z", "content": "

Imagine Elon musk participating in this competition and winning by manipulating the outcomes in forecast period using his tweets.

\n

There should be strict rules against him participating in this competition.

", "rawMarkdown": "Imagine Elon musk participating in this competition and winning by manipulating the outcomes in forecast period using his tweets.\n\nThere should be strict rules against him participating in this competition.", "votes": 48, "replies": [ { "id": 1568587, "postDate": "2021-11-02T21:59:16.360Z", "content": "

Elon Musk will become discussion grandmaster in a day if he joins

", "rawMarkdown": "Elon Musk will become discussion grandmaster in a day if he joins ", "votes": 32 }, { "id": 1568763, "postDate": "2021-11-03T03:03:31.843Z", "content": "

to the moonπŸ˜‰

", "rawMarkdown": "to the moonπŸ˜‰", "votes": 2 }, { "id": 1569381, "postDate": "2021-11-03T11:52:46.607Z", "content": "

Like his first wife said, \"It's Elon's world and the rest of us live in it\". Maybe he is already in between us πŸ‘€.

", "rawMarkdown": "Like his first wife said, \"It's Elon's world and the rest of us live in it\". Maybe he is already in between us πŸ‘€.", "votes": 4 }, { "id": 1572291, "postDate": "2021-11-05T15:25:44.470Z", "content": "

Well said and amazing name buddy… @shahnawazakhtar

", "rawMarkdown": "Well said and amazing name buddy... @shahnawazakhtar ", "votes": 1 }, { "id": 1573723, "postDate": "2021-11-06T21:12:38.273Z", "content": "

among usπŸ˜ƒπŸ˜ƒπŸ˜ƒπŸ˜ƒ

", "rawMarkdown": "among usπŸ˜ƒπŸ˜ƒπŸ˜ƒπŸ˜ƒ", "votes": 1 } ] }, { "id": 1568627, "postDate": "2021-11-02T23:11:37.457Z", "content": "

Internet access disabled

\n

You would better implement an 'Elon Musk' model offline in your submitted code to predict what he will say in the near future. πŸ˜„

", "rawMarkdown": "`Internet access disabled`\n\nYou would better implement an 'Elon Musk' model offline in your submitted code to predict what he will say in the near future. πŸ˜„", "votes": 31, "replies": [ { "id": 1584220, "postDate": "2021-11-16T10:54:34.620Z", "content": "

Would be a novel idea to develop an AI based on a person who is infamously against it 😬

", "rawMarkdown": "Would be a novel idea to develop an AI based on a person who is infamously against it 😬" } ] }, { "id": 1568796, "postDate": "2021-11-03T03:30:15.140Z", "content": "

Firstly, nice title for the topic @yamqwe ;)

\n

It is mentioned in the rules page that
\nFreely & publicly available external data is allowed, including pre-trained models

\n

So, I think we can use the external free datasets that are static. Accessing dynamically changing datasets may run the risk of future leakage(?). Admins will be able to help us on this one.

\n

I like the overall idea of using tweets / news in addition to do the short term forecasting as they play a vital role in the crypto space.

", "rawMarkdown": "Firstly, nice title for the topic @yamqwe ;) \n\nIt is mentioned in the rules page that\n`Freely & publicly available external data is allowed, including pre-trained models`\n\nSo, I think we can use the external free datasets that are static. Accessing dynamically changing datasets may run the risk of future leakage(?). Admins will be able to help us on this one.\n\nI like the overall idea of using tweets / news in addition to do the short term forecasting as they play a vital role in the crypto space.\n", "votes": 4, "replies": [ { "id": 1576149, "postDate": "2021-11-09T00:07:53.280Z", "content": "

Submission notebooks can only access the version of the dataset that existed when they were submitted, so you can't introduce leakage by updating your own dataset.

", "rawMarkdown": "Submission notebooks can only access the version of the dataset that existed when they were submitted, so you can't introduce leakage by updating your own dataset.", "votes": 8 }, { "id": 1584929, "postDate": "2021-11-16T21:50:42.200Z", "content": "

In my humble opinion, from past observations, it can be a case of hit or miss. Simpler the model and less dependencies are what will keep the model results consistent and stable.

", "rawMarkdown": "In my humble opinion, from past observations, it can be a case of hit or miss. Simpler the model and less dependencies are what will keep the model results consistent and stable." } ] }, { "id": 1569386, "postDate": "2021-11-03T11:55:38.433Z", "content": "

Crypto volatility and Elon's twitter schedule are significantly correlated.

", "rawMarkdown": "Crypto volatility and Elon's twitter schedule are significantly correlated. ", "votes": 2 }, { "id": 1569415, "postDate": "2021-11-03T12:25:03.823Z", "content": "

πŸ˜‚πŸ˜‚
\nlez take doge to the moon along with bitcoin.

", "rawMarkdown": "πŸ˜‚πŸ˜‚\nlez take doge to the moon along with bitcoin.", "votes": 1 }, { "id": 1569334, "postDate": "2021-11-03T10:59:38.807Z", "content": "

All you need to win this competition is to persuade Elon to do some tweets ;)

", "rawMarkdown": "All you need to win this competition is to persuade Elon to do some tweets ;)", "votes": 2 }, { "id": 1590125, "postDate": "2021-11-20T23:13:28.940Z", "content": "

Time for an IKO - Initial Kaggle Offering

", "rawMarkdown": "Time for an IKO - Initial Kaggle Offering" }, { "id": 1589972, "postDate": "2021-11-20T19:16:55.653Z", "content": "

Hi @yamqwe - thanks for all the great notebooks & datasets for this competition. One question - will auto datasets continue to update after the submission date of the competition?
\ni.e. if you have a notebook with internet access turned off, will your crypto currency datasets continue to be updated with the daily prices (not to mention the critical auto update Elon Musk dataset πŸ˜‚)?

", "rawMarkdown": "Hi @yamqwe - thanks for all the great notebooks & datasets for this competition. One question - will auto datasets continue to update after the submission date of the competition? \ni.e. if you have a notebook with internet access turned off, will your crypto currency datasets continue to be updated with the daily prices (not to mention the critical auto update Elon Musk dataset πŸ˜‚)?" }, { "id": 1587769, "postDate": "2021-11-18T23:59:20.267Z", "content": "

@yamqwe glad you brought this up though, Elon and his itchy Twitter finger were the first things that came to mind when signing up. I mean, can one truly model the \"Musk Effect\" on asset prices? Must we achieve elon-stationarity before forecasting? 😏

", "rawMarkdown": "@yamqwe glad you brought this up though, Elon and his itchy Twitter finger were the first things that came to mind when signing up. I mean, can one truly model the \"Musk Effect\" on asset prices? Must we achieve elon-stationarity before forecasting? 😏" }, { "id": 1584928, "postDate": "2021-11-16T21:48:38.153Z", "content": "

That only predicts the past right? The intent of the competition is to come up with a model that can be used with unseen future data. How will these tweets help in that case? Will a model from the new tweets be created to reassess/ensemble with the existing model then? Would like to know your thoughts as well on this?

", "rawMarkdown": "That only predicts the past right? The intent of the competition is to come up with a model that can be used with unseen future data. How will these tweets help in that case? Will a model from the new tweets be created to reassess/ensemble with the existing model then? Would like to know your thoughts as well on this?" }, { "id": 1573308, "postDate": "2021-11-06T14:00:30.510Z", "content": "

Hi All,
\nBelow are my points:

\n", "rawMarkdown": "Hi All,\nBelow are my points:\n* We can use Kaggle features for analyzing a dataset to process for a possible outcome\n* A feature where one can access the dataset offline would be beneficial in terms of accessing it\n" }, { "id": 1572837, "postDate": "2021-11-06T02:42:40.020Z", "content": "

'Elon Musk' == train-test-contamination πŸ€•

", "rawMarkdown": "'Elon Musk' == train-test-contamination πŸ€•" }, { "id": 1571811, "postDate": "2021-11-05T07:13:02.940Z", "content": "

wow~ It's attractive to add a pretrained model for sentimental analysis

", "rawMarkdown": "wow~ It's attractive to add a pretrained model for sentimental analysis " }, { "id": 1571781, "postDate": "2021-11-05T06:46:39.510Z", "content": "

nice thought ! ! !πŸ‘

", "rawMarkdown": "nice thought ! ! !πŸ‘" }, { "id": 1568859, "postDate": "2021-11-03T04:55:11.083Z", "content": "

If the internet is disabled, then how the live data from the market enters into the model? I read through timeseries API instruction provided. Will it take care?

", "rawMarkdown": "If the internet is disabled, then how the live data from the market enters into the model? I read through timeseries API instruction provided. Will it take care? " }, { "id": 1568560, "postDate": "2021-11-02T21:08:39.053Z", "content": "

Of course the other option is to do some analysis on Elon's tweets, and create a predictive model to drive sentiment as a feature, I'm going to guess the cycles are easier to predict πŸ˜ƒ but how the different coins move in out out of fashion may be trickier!

", "rawMarkdown": "Of course the other option is to do some analysis on Elon's tweets, and create a predictive model to drive sentiment as a feature, I'm going to guess the cycles are easier to predict πŸ˜ƒ but how the different coins move in out out of fashion may be trickier!" }, { "id": 1568553, "postDate": "2021-11-02T20:58:10.430Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1568566, "author_name": "Shahnawaz Akhtar", "author_url": "", "post_date": "2021-11-02T21:15:45.740000", "content": "

Imagine Elon musk participating in this competition and winning by manipulating the outcomes in forecast period using his tweets.

\n

There should be strict rules against him participating in this competition.

", "votes": 48, "replies": [ { "id": 1568587, "author_name": "", "author_url": "", "post_date": "2021-11-02T21:59:16.360000", "content": "

Elon Musk will become discussion grandmaster in a day if he joins

", "votes": 32, "replies": [] }, { "id": 1568763, "author_name": "Adwitiya Trivedi", "author_url": "", "post_date": "2021-11-03T03:03:31.843000", "content": "

to the moonπŸ˜‰

", "votes": 2, "replies": [] }, { "id": 1569381, "author_name": "Shahnawaz Akhtar", "author_url": "", "post_date": "2021-11-03T11:52:46.607000", "content": "

Like his first wife said, \"It's Elon's world and the rest of us live in it\". Maybe he is already in between us πŸ‘€.

", "votes": 4, "replies": [] }, { "id": 1572291, "author_name": "Kartik Khandelwal", "author_url": "", "post_date": "2021-11-05T15:25:44.470000", "content": "

Well said and amazing name buddy… @shahnawazakhtar

", "votes": 1, "replies": [] }, { "id": 1573723, "author_name": "Manh Lab", "author_url": "", "post_date": "2021-11-06T21:12:38.273000", "content": "

among usπŸ˜ƒπŸ˜ƒπŸ˜ƒπŸ˜ƒ

", "votes": 1, "replies": [] } ] }, { "id": 1568627, "author_name": "Yirun Zhang", "author_url": "", "post_date": "2021-11-02T23:11:37.457000", "content": "

Internet access disabled

\n

You would better implement an 'Elon Musk' model offline in your submitted code to predict what he will say in the near future. πŸ˜„

", "votes": 31, "replies": [ { "id": 1584220, "author_name": "Jayesh Mandlekar", "author_url": "", "post_date": "2021-11-16T10:54:34.620000", "content": "

Would be a novel idea to develop an AI based on a person who is infamously against it 😬

", "votes": 0, "replies": [] } ] }, { "id": 1568796, "author_name": "SRK", "author_url": "", "post_date": "2021-11-03T03:30:15.140000", "content": "

Firstly, nice title for the topic @yamqwe ;)

\n

It is mentioned in the rules page that
\nFreely & publicly available external data is allowed, including pre-trained models

\n

So, I think we can use the external free datasets that are static. Accessing dynamically changing datasets may run the risk of future leakage(?). Admins will be able to help us on this one.

\n

I like the overall idea of using tweets / news in addition to do the short term forecasting as they play a vital role in the crypto space.

", "votes": 4, "replies": [ { "id": 1576149, "author_name": "Sohier Dane", "author_url": "", "post_date": "2021-11-09T00:07:53.280000", "content": "

Submission notebooks can only access the version of the dataset that existed when they were submitted, so you can't introduce leakage by updating your own dataset.

", "votes": 8, "replies": [] }, { "id": 1584929, "author_name": "Mani Sarkar", "author_url": "", "post_date": "2021-11-16T21:50:42.200000", "content": "

In my humble opinion, from past observations, it can be a case of hit or miss. Simpler the model and less dependencies are what will keep the model results consistent and stable.

", "votes": 0, "replies": [] } ] }, { "id": 1569386, "author_name": "Shahnawaz Akhtar", "author_url": "", "post_date": "2021-11-03T11:55:38.433000", "content": "

Crypto volatility and Elon's twitter schedule are significantly correlated.

", "votes": 2, "replies": [] }, { "id": 1569415, "author_name": "AniketGoyal2913", "author_url": "", "post_date": "2021-11-03T12:25:03.823000", "content": "

πŸ˜‚πŸ˜‚
\nlez take doge to the moon along with bitcoin.

", "votes": 1, "replies": [] }, { "id": 1569334, "author_name": "Md Zarif Ul Alam", "author_url": "", "post_date": "2021-11-03T10:59:38.807000", "content": "

All you need to win this competition is to persuade Elon to do some tweets ;)

", "votes": 2, "replies": [] }, { "id": 1590125, "author_name": "Evan Cofsky", "author_url": "", "post_date": "2021-11-20T23:13:28.940000", "content": "

Time for an IKO - Initial Kaggle Offering

", "votes": 0, "replies": [] }, { "id": 1589972, "author_name": "Steve Roberts", "author_url": "", "post_date": "2021-11-20T19:16:55.653000", "content": "

Hi @yamqwe - thanks for all the great notebooks & datasets for this competition. One question - will auto datasets continue to update after the submission date of the competition?
\ni.e. if you have a notebook with internet access turned off, will your crypto currency datasets continue to be updated with the daily prices (not to mention the critical auto update Elon Musk dataset πŸ˜‚)?

", "votes": 0, "replies": [] }, { "id": 1587769, "author_name": "Ryan", "author_url": "", "post_date": "2021-11-18T23:59:20.267000", "content": "

@yamqwe glad you brought this up though, Elon and his itchy Twitter finger were the first things that came to mind when signing up. I mean, can one truly model the \"Musk Effect\" on asset prices? Must we achieve elon-stationarity before forecasting? 😏

", "votes": 0, "replies": [] }, { "id": 1584928, "author_name": "Mani Sarkar", "author_url": "", "post_date": "2021-11-16T21:48:38.153000", "content": "

That only predicts the past right? The intent of the competition is to come up with a model that can be used with unseen future data. How will these tweets help in that case? Will a model from the new tweets be created to reassess/ensemble with the existing model then? Would like to know your thoughts as well on this?

", "votes": 0, "replies": [] }, { "id": 1573308, "author_name": "Kundan Kumar", "author_url": "", "post_date": "2021-11-06T14:00:30.510000", "content": "

Hi All,
\nBelow are my points:

\n", "votes": 0, "replies": [] }, { "id": 1572837, "author_name": "spin fit", "author_url": "", "post_date": "2021-11-06T02:42:40.020000", "content": "

'Elon Musk' == train-test-contamination πŸ€•

", "votes": 0, "replies": [] }, { "id": 1571811, "author_name": "Edmund Sun", "author_url": "", "post_date": "2021-11-05T07:13:02.940000", "content": "

wow~ It's attractive to add a pretrained model for sentimental analysis

", "votes": 0, "replies": [] }, { "id": 1571781, "author_name": "pbokieee", "author_url": "", "post_date": "2021-11-05T06:46:39.510000", "content": "

nice thought ! ! !πŸ‘

", "votes": 0, "replies": [] }, { "id": 1568859, "author_name": "Kamal", "author_url": "", "post_date": "2021-11-03T04:55:11.083000", "content": "

If the internet is disabled, then how the live data from the market enters into the model? I read through timeseries API instruction provided. Will it take care?

", "votes": 0, "replies": [] }, { "id": 1568560, "author_name": "benvens", "author_url": "", "post_date": "2021-11-02T21:08:39.053000", "content": "

Of course the other option is to do some analysis on Elon's tweets, and create a predictive model to drive sentiment as a feature, I'm going to guess the cycles are easier to predict πŸ˜ƒ but how the different coins move in out out of fashion may be trickier!

", "votes": 0, "replies": [] }, { "id": 1568553, "author_name": "", "author_url": "", "post_date": "2021-11-02T20:58:10.430000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1568531": "Hi, \n\nJust a quick question about external data usage. it is allowed at all? \nFor example, is it allowed to use some external dataset of.. opinions expressed by some individuals?\nIf no internet is allowed, can we for example use the new kaggle feature of an auto-updating dataset to create **an auto dataset of tweets and use it**? \n\nNot that I have any specific individual whose tweets I might want to add into the model's predictions.. πŸ˜†", "1568566": "Imagine Elon musk participating in this competition and winning by manipulating the outcomes in forecast period using his tweets.\n\nThere should be strict rules against him participating in this competition.", "1568627": "`Internet access disabled`\n\nYou would better implement an 'Elon Musk' model offline in your submitted code to predict what he will say in the near future. πŸ˜„", "1568796": "Firstly, nice title for the topic @yamqwe ;) \n\nIt is mentioned in the rules page that\n`Freely & publicly available external data is allowed, including pre-trained models`\n\nSo, I think we can use the external free datasets that are static. Accessing dynamically changing datasets may run the risk of future leakage(?). Admins will be able to help us on this one.\n\nI like the overall idea of using tweets / news in addition to do the short term forecasting as they play a vital role in the crypto space.\n", "1569386": "Crypto volatility and Elon's twitter schedule are significantly correlated. ", "1569415": "πŸ˜‚πŸ˜‚\nlez take doge to the moon along with bitcoin.", "1569334": "All you need to win this competition is to persuade Elon to do some tweets ;)", "1590125": "Time for an IKO - Initial Kaggle Offering", "1589972": "Hi @yamqwe - thanks for all the great notebooks & datasets for this competition. One question - will auto datasets continue to update after the submission date of the competition? \ni.e. if you have a notebook with internet access turned off, will your crypto currency datasets continue to be updated with the daily prices (not to mention the critical auto update Elon Musk dataset πŸ˜‚)?", "1587769": "@yamqwe glad you brought this up though, Elon and his itchy Twitter finger were the first things that came to mind when signing up. I mean, can one truly model the \"Musk Effect\" on asset prices? Must we achieve elon-stationarity before forecasting? 😏", "1584928": "That only predicts the past right? The intent of the competition is to come up with a model that can be used with unseen future data. How will these tweets help in that case? Will a model from the new tweets be created to reassess/ensemble with the existing model then? Would like to know your thoughts as well on this?", "1573308": "Hi All,\nBelow are my points:\n* We can use Kaggle features for analyzing a dataset to process for a possible outcome\n* A feature where one can access the dataset offline would be beneficial in terms of accessing it\n", "1572837": "'Elon Musk' == train-test-contamination πŸ€•", "1571811": "wow~ It's attractive to add a pretrained model for sentimental analysis ", "1571781": "nice thought ! ! !πŸ‘", "1568859": "If the internet is disabled, then how the live data from the market enters into the model? I read through timeseries API instruction provided. Will it take care? ", "1568560": "Of course the other option is to do some analysis on Elon's tweets, and create a predictive model to drive sentiment as a feature, I'm going to guess the cycles are easier to predict πŸ˜ƒ but how the different coins move in out out of fashion may be trickier!", "1568553": "" } }