{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"---\n# G-Research Crypto Forecasting - EDA\n\n---","metadata":{}},{"cell_type":"markdown","source":"**Problem Statement:**\n\n* Over $40 billion worth of cryptocurrencies are traded every day. \n * They are among the most popular assets for speculation and investment, yet have proven wildly volatile. \n * Fast-fluctuating prices have made millionaires of a lucky few, and delivered crushing losses to others. \n* Could some of these price movements have been predicted in advance?\n\n* Use machine learning expertise to forecast short term returns in 14 popular cryptocurrencies. \n* Dataset contains millions of rows of high-frequency market data dating back to 2018 which can be used to build ML model. \n* Once the submission deadline has passed, final score will be calculated over the following 3 months using live crypto data as it is collected. \n \n---\n \n**Main Dataset:**\n\n* **train.csv** file contains training set.\n* **example_test.csv** file includes example of the data that will be delivered by the time series API.\n* **example_sample_submission.csv** file includes example of the data that will be delivered by the time series API. The data is just copied from train.csv.\n* **asset_details.csv** file provides the real name and of the cryptoasset for each Asset_ID and the weight each cryptoasset receives in the metric.\n* **supplemental_train.csv**- After the submission period is over this file's data will be replaced with cryptoasset prices from the submission period.\n\n**gresearch_crypto** - An unoptimized version of the time series API files for offline work. You may need Python 3.7 and a Linux environment to run it without errors.\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n**Importing Libraries:**\n\n* To get started we will use Python for data pre-processing and data analysis.\n\n* Import python libraries as necessary to get started for data load and later import other libraries as needed\n\n---","metadata":{}},{"cell_type":"code","source":"# Custom G-Research python module needed (Not used in this notebook)\nimport gresearch_crypto\n\n# Pandas for data manipulation\nimport pandas as pd\nimport numpy as np\n\n# module provides a portable way of using operating system dependent functionality\nimport os \n\n # Importing pyplot interface using matplotlib\nimport matplotlib.pyplot as plt \n\n# Importing seaborn library for interactive visualization\nimport seaborn as sns \n\n# Importing WordCloud for text data visualization\nfrom wordcloud import WordCloud\n\n# Importing matplotlib for plots\nimport matplotlib as mpl\n\n#Importing datetime for using datetime\nfrom datetime import datetime\n\n#Importing plotly Express for visualization\nimport plotly.express as px\n\n# Importing statsmodel for statistical methods\nimport statsmodels.api as sm\nfrom statsmodels.tsa.stattools import adfuller\n\n#Importing for customizing properties in matplotlib\nfrom pylab import rcParams","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:26:48.522610Z","iopub.execute_input":"2022-01-18T06:26:48.524590Z","iopub.status.idle":"2022-01-18T06:26:48.531777Z","shell.execute_reply.started":"2022-01-18T06:26:48.524532Z","shell.execute_reply":"2022-01-18T06:26:48.530794Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Custom G-Research python module requires this step\n#env = gresearch_crypto.make_env()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:26:48.533982Z","iopub.execute_input":"2022-01-18T06:26:48.534871Z","iopub.status.idle":"2022-01-18T06:26:48.555830Z","shell.execute_reply.started":"2022-01-18T06:26:48.534825Z","shell.execute_reply":"2022-01-18T06:26:48.554564Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Display files available in this competition folder\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-01-18T06:26:48.570539Z","iopub.execute_input":"2022-01-18T06:26:48.571207Z","iopub.status.idle":"2022-01-18T06:26:48.583222Z","shell.execute_reply.started":"2022-01-18T06:26:48.571167Z","shell.execute_reply":"2022-01-18T06:26:48.582483Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n# Data Definition/Description\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n**Data Definition:**\n\n---\n\n* **train.csv:** This dataset contains information on historic trades for several cryptoassets.","metadata":{}},{"cell_type":"code","source":"# load training data\ntrain_df = pd.read_csv('../input/g-research-crypto-forecasting/train.csv', low_memory=False)\n# load asset details data\nasset_details_df = pd.read_csv('../input/g-research-crypto-forecasting/asset_details.csv', low_memory=False)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:26:48.584663Z","iopub.execute_input":"2022-01-18T06:26:48.585345Z","iopub.status.idle":"2022-01-18T06:27:57.065479Z","shell.execute_reply.started":"2022-01-18T06:26:48.585297Z","shell.execute_reply":"2022-01-18T06:27:57.061169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the structure of train dataset?**\n\n---","metadata":{}},{"cell_type":"markdown","source":"| No. | Feature Name | Description of the feature |\n| :-- | :--| :--| \n|01| **timestamp**   | A timestamp for the minute covered by the row |\n|02| **Asset_ID** | An ID code for the cryptoasset                 |\n|03| **Count**   | The number of trades that took place this minute                 |\n|04| **Open**   | The USD price at the beginning of the minute  |\n|05| **High**   | The highest USD price during the minute|\n|06| **Low**   | The lowest USD price during the minute  |\n|07| **Close**   | The USD price at the end of the minute|\n|08| **Volume**   | The number of cryptoasset units traded during the minute|\n|09| **VWAP**   | The volume weighted average price for the minute  |\n|10| **Target**   | 15 minute residualized returns|","metadata":{}},{"cell_type":"code","source":"# get shape of dataframe\nprint('Shape of train dataset is:', train_df.shape)\n\n# print summary of dataframe\ntrain_df.info(show_counts=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:27:57.075299Z","iopub.execute_input":"2022-01-18T06:27:57.077041Z","iopub.status.idle":"2022-01-18T06:27:57.839170Z","shell.execute_reply.started":"2022-01-18T06:27:57.076974Z","shell.execute_reply":"2022-01-18T06:27:57.838332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**train dataset information:**\n\n* There are 24236806 data points (rows) and 10 feature (column) in train dataset.\n* There are seven columns which are of numerical float type and three column of numerical int type.\n* There are missing values (non-null count is not same) for VWAP and Target columns.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What does data looks like for train dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print first 10 rows of dataframe\ntrain_df.head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:27:57.840464Z","iopub.execute_input":"2022-01-18T06:27:57.841359Z","iopub.status.idle":"2022-01-18T06:27:57.867611Z","shell.execute_reply.started":"2022-01-18T06:27:57.841312Z","shell.execute_reply":"2022-01-18T06:27:57.866909Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What does data looks like for asset_details dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print summary of asset details dataframe\nasset_details_df.sort_values(by=['Weight'], ascending=False)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:27:57.869647Z","iopub.execute_input":"2022-01-18T06:27:57.870809Z","iopub.status.idle":"2022-01-18T06:27:57.894522Z","shell.execute_reply.started":"2022-01-18T06:27:57.870761Z","shell.execute_reply":"2022-01-18T06:27:57.893502Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the statistics description for train dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# print descriptive statistics for train dataset\ntrain_df.describe(include='all').round(1)","metadata":{"execution":{"iopub.status.busy":"2022-01-18T06:27:57.896609Z","iopub.execute_input":"2022-01-18T06:27:57.897159Z","iopub.status.idle":"2022-01-18T06:28:07.845426Z","shell.execute_reply.started":"2022-01-18T06:27:57.897111Z","shell.execute_reply":"2022-01-18T06:28:07.844318Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**train dataset data description:**\n\n* There are no missing values for timestamp and appears to be having normal distribution of datapoints.\n* There are no missing values for Asset_ID and appears to be having normal distribution of datapoints.\n* There are no missing values for Count, Open, High, Low, Close and Volume but appears to be having skewed distribution of datapoints.\n* There are missing values for VWAP and Target but appears to be having normal distribution of datapoints.","metadata":{}},{"cell_type":"markdown","source":"Need to merge train_df and asset_details_df","metadata":{}},{"cell_type":"code","source":"# merge dataframe using Asset_ID as key\ngresearch_crypto_df = pd.merge(train_df,asset_details_df,on=['Asset_ID'])","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:07.847054Z","iopub.execute_input":"2022-01-18T06:28:07.847854Z","iopub.status.idle":"2022-01-18T06:28:15.514829Z","shell.execute_reply.started":"2022-01-18T06:28:07.847813Z","shell.execute_reply":"2022-01-18T06:28:15.513871Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print first 10 rows of merged dataframe\ngresearch_crypto_df.head(10)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:15.516974Z","iopub.execute_input":"2022-01-18T06:28:15.517618Z","iopub.status.idle":"2022-01-18T06:28:15.544784Z","shell.execute_reply.started":"2022-01-18T06:28:15.517576Z","shell.execute_reply":"2022-01-18T06:28:15.543980Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print last 10 rows of merged dataframe\ngresearch_crypto_df.tail(10)","metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2022-01-18T06:28:15.546409Z","iopub.execute_input":"2022-01-18T06:28:15.547003Z","iopub.status.idle":"2022-01-18T06:28:15.571620Z","shell.execute_reply.started":"2022-01-18T06:28:15.546953Z","shell.execute_reply":"2022-01-18T06:28:15.570958Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Handle missing value, will drop them","metadata":{}},{"cell_type":"code","source":"# drop missing value from dataframe\ngresearch_crypto_df.dropna(axis=0, inplace=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:15.573222Z","iopub.execute_input":"2022-01-18T06:28:15.573766Z","iopub.status.idle":"2022-01-18T06:28:24.705739Z","shell.execute_reply.started":"2022-01-18T06:28:15.573721Z","shell.execute_reply":"2022-01-18T06:28:24.704486Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print summary of dataframe\ngresearch_crypto_df.info(show_counts=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:24.710012Z","iopub.execute_input":"2022-01-18T06:28:24.710617Z","iopub.status.idle":"2022-01-18T06:28:27.991431Z","shell.execute_reply.started":"2022-01-18T06:28:24.710573Z","shell.execute_reply":"2022-01-18T06:28:27.990329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# create sample of dataset for EDA (For faster execution and to avoid out of memory issue for notebook)\ngresearch_crypto_df_sample = gresearch_crypto_df.sample(n=1000000, random_state=1)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:27.993282Z","iopub.execute_input":"2022-01-18T06:28:27.993614Z","iopub.status.idle":"2022-01-18T06:28:30.353794Z","shell.execute_reply.started":"2022-01-18T06:28:27.993575Z","shell.execute_reply":"2022-01-18T06:28:30.352587Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print summary of sample dataframe\ngresearch_crypto_df_sample.info()","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:30.355601Z","iopub.execute_input":"2022-01-18T06:28:30.355907Z","iopub.status.idle":"2022-01-18T06:28:30.507612Z","shell.execute_reply.started":"2022-01-18T06:28:30.355870Z","shell.execute_reply":"2022-01-18T06:28:30.506607Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert timestamp to Datetime for EDA purpose\ngresearch_crypto_df_sample['Datetime'] = pd.to_datetime(gresearch_crypto_df_sample['timestamp'], unit='s')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:30.508980Z","iopub.execute_input":"2022-01-18T06:28:30.509232Z","iopub.status.idle":"2022-01-18T06:28:30.554329Z","shell.execute_reply.started":"2022-01-18T06:28:30.509202Z","shell.execute_reply":"2022-01-18T06:28:30.553475Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# print first 10 rows of sample dataframe\ngresearch_crypto_df_sample.head(10)","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:30.555963Z","iopub.execute_input":"2022-01-18T06:28:30.556523Z","iopub.status.idle":"2022-01-18T06:28:30.588037Z","shell.execute_reply.started":"2022-01-18T06:28:30.556465Z","shell.execute_reply":"2022-01-18T06:28:30.587001Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n# Data Analysis/EDA\n\n---","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution percentage for Asset_Name in sample dataset?**\n\n---","metadata":{}},{"cell_type":"code","source":"# distribution percentage for Asset_Name in sample dataset\nfig = px.pie(gresearch_crypto_df_sample, names='Asset_Name')\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:30.589583Z","iopub.execute_input":"2022-01-18T06:28:30.589850Z","iopub.status.idle":"2022-01-18T06:28:35.579122Z","shell.execute_reply.started":"2022-01-18T06:28:30.589819Z","shell.execute_reply":"2022-01-18T06:28:35.577202Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Top 5 cryptoasset (Bitcoin, Ethereum, Cardano, Binance Coin and Dogecoin) seems to have similar distribution in sample dataset.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Asset_Name with respect to Weight?**\n\n---","metadata":{}},{"cell_type":"code","source":"# setting the dimensions of the plot\nfig, ax = plt.subplots(figsize=(17,8))\n# distribution for Asset_Name with respect to Weight\nsns.barplot(data=gresearch_crypto_df_sample, x='Asset_Name', y='Weight',ax=ax)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:35.581527Z","iopub.execute_input":"2022-01-18T06:28:35.582000Z","iopub.status.idle":"2022-01-18T06:28:43.606579Z","shell.execute_reply.started":"2022-01-18T06:28:35.581921Z","shell.execute_reply":"2022-01-18T06:28:43.605140Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Bitcoin, Ethereum,Cardano, Binance Coin and Dogecoin appears to be top five crypto assest as per Weight.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Asset_Name with respect to Count?**\n\n---","metadata":{}},{"cell_type":"code","source":"# setting the dimensions of the plot\nfig, ax = plt.subplots(figsize=(17,8))\n# distribution for Asset_Name with respect to Count\nsns.barplot(data=gresearch_crypto_df_sample, x='Asset_Name', y='Count',ax=ax)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:43.608574Z","iopub.execute_input":"2022-01-18T06:28:43.608981Z","iopub.status.idle":"2022-01-18T06:28:51.362207Z","shell.execute_reply.started":"2022-01-18T06:28:43.608909Z","shell.execute_reply":"2022-01-18T06:28:51.360893Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Count: The number of trades that took place this minute\n\nBitcoin, Ethereum,Cardano, Binance Coin and Dogecoin apperas to be having higher count.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the trend for Asset_Name with respect to Count?**\n\n---","metadata":{}},{"cell_type":"code","source":"# setting the dimensions of the plot\nfig, ax = plt.subplots(figsize=(17,8))\n# trend for Asset_Name with respect to Count\nsns.lineplot(data=gresearch_crypto_df_sample, x='Datetime', y='Count',hue='Asset_Name', ax=ax)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:28:51.363898Z","iopub.execute_input":"2022-01-18T06:28:51.364204Z","iopub.status.idle":"2022-01-18T06:29:29.769391Z","shell.execute_reply.started":"2022-01-18T06:28:51.364171Z","shell.execute_reply":"2022-01-18T06:29:29.768033Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Asset_Name with respect to Volume?**\n\n---","metadata":{}},{"cell_type":"code","source":"# setting the dimensions of the plot\nfig, ax = plt.subplots(figsize=(17,8))\n# distribution for Asset_Name with respect to Volume\nsns.barplot(data=gresearch_crypto_df_sample, x='Asset_Name', y='Volume',ax=ax)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:29:29.771658Z","iopub.execute_input":"2022-01-18T06:29:29.772142Z","iopub.status.idle":"2022-01-18T06:29:37.496607Z","shell.execute_reply.started":"2022-01-18T06:29:29.772090Z","shell.execute_reply":"2022-01-18T06:29:37.495875Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Volume: The number of cryptoasset units traded during the minute\n\nDogecoin appears to be the leader.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the trend for Asset_Name with respect to Volume?**\n\n---","metadata":{}},{"cell_type":"code","source":"# setting the dimensions of the plot\nfig, ax = plt.subplots(figsize=(17,8))\n# trend for Asset_Name with respect to Volume\nsns.lineplot(data=gresearch_crypto_df_sample, x='Datetime', y='Volume',hue='Asset_Name', ax=ax)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:29:37.497946Z","iopub.execute_input":"2022-01-18T06:29:37.498554Z","iopub.status.idle":"2022-01-18T06:30:15.033051Z","shell.execute_reply.started":"2022-01-18T06:29:37.498514Z","shell.execute_reply":"2022-01-18T06:30:15.031736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Asset_Name with respect to High?**\n\n---","metadata":{}},{"cell_type":"code","source":"# setting the dimensions of the plot\nfig, ax = plt.subplots(figsize=(17,8))\n# distribution for Asset_Name with respect to High\nsns.barplot(data=gresearch_crypto_df_sample, x='Asset_Name', y='High',ax=ax)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:30:15.034676Z","iopub.execute_input":"2022-01-18T06:30:15.034977Z","iopub.status.idle":"2022-01-18T06:30:22.796968Z","shell.execute_reply.started":"2022-01-18T06:30:15.034945Z","shell.execute_reply":"2022-01-18T06:30:22.795620Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"High: The highest USD price during the minute\n\nBitcoin appears to be the leader.","metadata":{}},{"cell_type":"markdown","source":"---\n**Q: What is the trend for Asset_Name with respect to High?**\n\n---","metadata":{}},{"cell_type":"code","source":"# setting the dimensions of the plot\nfig, ax = plt.subplots(figsize=(17,8))\n# trend for Asset_Name with respect to High\nsns.lineplot(data=gresearch_crypto_df_sample, x='Datetime', y='High',hue='Asset_Name', ax=ax)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:30:22.798435Z","iopub.execute_input":"2022-01-18T06:30:22.798688Z","iopub.status.idle":"2022-01-18T06:31:00.641515Z","shell.execute_reply.started":"2022-01-18T06:30:22.798660Z","shell.execute_reply":"2022-01-18T06:31:00.640387Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the distribution for Asset_Name with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# setting the dimensions of the plot\nfig, ax = plt.subplots(figsize=(17,8))\n# distribution for Asset_Name with respect to Target\nsns.boxplot(data=gresearch_crypto_df_sample, x='Asset_Name', y='Target',ax=ax)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:00.643465Z","iopub.execute_input":"2022-01-18T06:31:00.643757Z","iopub.status.idle":"2022-01-18T06:31:02.173288Z","shell.execute_reply.started":"2022-01-18T06:31:00.643720Z","shell.execute_reply":"2022-01-18T06:31:02.172024Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Target: 15 minute residualized returns\n\nBitcoin appears to be having the most fluctuations followed by Dogecoin and Ethereum Classic.","metadata":{}},{"cell_type":"markdown","source":"Let us further explore trend for Bitcoin, Ethereum,Cardano, Binance Coin and Dogecoin with respect to Target.","metadata":{}},{"cell_type":"code","source":"# convert timestamp to Datetime for further analysis\ngresearch_crypto_df['Datetime'] = pd.to_datetime(gresearch_crypto_df['timestamp'], unit='s')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:02.175464Z","iopub.execute_input":"2022-01-18T06:31:02.176888Z","iopub.status.idle":"2022-01-18T06:31:03.003461Z","shell.execute_reply.started":"2022-01-18T06:31:02.176831Z","shell.execute_reply":"2022-01-18T06:31:03.002335Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Lookup asset name and return asset series\ndef asset_lookup (asset_name):\n    # check for Asset_Name\n    crypto_asset =gresearch_crypto_df.loc[gresearch_crypto_df['Asset_Name'] == asset_name]\n    # drop all other columns as of now\n    crypto_asset_df= crypto_asset.drop(columns=['timestamp','Asset_ID','Count','Open','High','Low','Close','Volume','VWAP','Weight','Asset_Name'], axis=1)\n    # sort by Datetime and set index\n    crypto_asset_df = crypto_asset_df.sort_values('Datetime')\n    crypto_asset_df = crypto_asset_df.groupby('Datetime')['Target'].sum().reset_index()\n    crypto_asset_df = crypto_asset_df.set_index('Datetime')\n    # group the data by start of month\n    return crypto_asset_df['Target'].resample('MS').mean()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:03.005251Z","iopub.execute_input":"2022-01-18T06:31:03.005538Z","iopub.status.idle":"2022-01-18T06:31:03.013813Z","shell.execute_reply.started":"2022-01-18T06:31:03.005505Z","shell.execute_reply":"2022-01-18T06:31:03.012830Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# check for stationarity using ADF Test for a given asset series\ndef asset_stationarity (asset_series):\n    result = adfuller(asset_series)\n    print('ADF Statistic: %f' % result[0])\n    print('p-value: %f' % result[1])\n    print('Critical Values:')\n    for key, value in result[4].items():\n     print('\\t%s: %.3f' % (key, value))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:03.015375Z","iopub.execute_input":"2022-01-18T06:31:03.015636Z","iopub.status.idle":"2022-01-18T06:31:03.030725Z","shell.execute_reply.started":"2022-01-18T06:31:03.015603Z","shell.execute_reply":"2022-01-18T06:31:03.029486Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# visualization for trend,seasonality and residual for a given asset series\ndef asset_decompose (asset_series):\n    rcParams['figure.figsize'] = 18, 8\n    decomposition = sm.tsa.seasonal_decompose(asset_series, model='additive')\n    fig = decomposition.plot()\n    plt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:03.032738Z","iopub.execute_input":"2022-01-18T06:31:03.033052Z","iopub.status.idle":"2022-01-18T06:31:03.049787Z","shell.execute_reply.started":"2022-01-18T06:31:03.033015Z","shell.execute_reply":"2022-01-18T06:31:03.048379Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend for Bitcoin with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# visualization for Bitcoin trend\nbitcoin = asset_lookup('Bitcoin')\nbitcoin.plot(figsize=(15, 6))\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:03.055149Z","iopub.execute_input":"2022-01-18T06:31:03.055538Z","iopub.status.idle":"2022-01-18T06:31:08.036541Z","shell.execute_reply.started":"2022-01-18T06:31:03.055490Z","shell.execute_reply":"2022-01-18T06:31:08.035070Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend, seasonality and residuals for Bitcoin with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# visualization for trend,seasonality and residual\nasset_decompose(bitcoin)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:08.038351Z","iopub.execute_input":"2022-01-18T06:31:08.038635Z","iopub.status.idle":"2022-01-18T06:31:09.410492Z","shell.execute_reply.started":"2022-01-18T06:31:08.038602Z","shell.execute_reply":"2022-01-18T06:31:09.409246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# check for stationarity using ADF Test\nasset_stationarity(bitcoin)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:09.412825Z","iopub.execute_input":"2022-01-18T06:31:09.413568Z","iopub.status.idle":"2022-01-18T06:31:09.449181Z","shell.execute_reply.started":"2022-01-18T06:31:09.413525Z","shell.execute_reply":"2022-01-18T06:31:09.448193Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend for Ethereum with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# visualization for trend\nethereum = asset_lookup('Ethereum')\nethereum.plot(figsize=(15, 6))\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:09.450602Z","iopub.execute_input":"2022-01-18T06:31:09.450854Z","iopub.status.idle":"2022-01-18T06:31:14.351556Z","shell.execute_reply.started":"2022-01-18T06:31:09.450821Z","shell.execute_reply":"2022-01-18T06:31:14.350565Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend, seasonality and residuals for Ethereum with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# visualization for trend,seasonality and residual\nasset_decompose(ethereum)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:14.353145Z","iopub.execute_input":"2022-01-18T06:31:14.354365Z","iopub.status.idle":"2022-01-18T06:31:15.266661Z","shell.execute_reply.started":"2022-01-18T06:31:14.354325Z","shell.execute_reply":"2022-01-18T06:31:15.265734Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# check for stationarity using ADF Test\nasset_stationarity(ethereum)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:15.267947Z","iopub.execute_input":"2022-01-18T06:31:15.268208Z","iopub.status.idle":"2022-01-18T06:31:15.281688Z","shell.execute_reply.started":"2022-01-18T06:31:15.268176Z","shell.execute_reply":"2022-01-18T06:31:15.280878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend for Cardano with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# visualization for trend\ncardano = asset_lookup('Cardano')\ncardano.plot(figsize=(15, 6))\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:15.282831Z","iopub.execute_input":"2022-01-18T06:31:15.283686Z","iopub.status.idle":"2022-01-18T06:31:20.100942Z","shell.execute_reply.started":"2022-01-18T06:31:15.283631Z","shell.execute_reply":"2022-01-18T06:31:20.099716Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend, seasonality and residuals for Cardano with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# visualization for trend,seasonality and residual\nasset_decompose(cardano)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:20.102518Z","iopub.execute_input":"2022-01-18T06:31:20.102782Z","iopub.status.idle":"2022-01-18T06:31:21.016108Z","shell.execute_reply.started":"2022-01-18T06:31:20.102749Z","shell.execute_reply":"2022-01-18T06:31:21.014792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# check for stationarity using ADF Test\nasset_stationarity(cardano)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:21.017809Z","iopub.execute_input":"2022-01-18T06:31:21.018972Z","iopub.status.idle":"2022-01-18T06:31:21.032555Z","shell.execute_reply.started":"2022-01-18T06:31:21.018902Z","shell.execute_reply":"2022-01-18T06:31:21.031726Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend for Binance Coin with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# visualization for trend\nbinancecoin = asset_lookup('Binance Coin')\nbinancecoin.plot(figsize=(15, 6))\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:21.033968Z","iopub.execute_input":"2022-01-18T06:31:21.035252Z","iopub.status.idle":"2022-01-18T06:31:25.841083Z","shell.execute_reply.started":"2022-01-18T06:31:21.035198Z","shell.execute_reply":"2022-01-18T06:31:25.839736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend, seasonality and residuals for Binance Coin with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# visualization for trend,seasonality and residual\nasset_decompose(binancecoin)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:25.843386Z","iopub.execute_input":"2022-01-18T06:31:25.844600Z","iopub.status.idle":"2022-01-18T06:31:26.795605Z","shell.execute_reply.started":"2022-01-18T06:31:25.844537Z","shell.execute_reply":"2022-01-18T06:31:26.794375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# check for stationarity using ADF Test\nasset_stationarity(binancecoin)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:26.797127Z","iopub.execute_input":"2022-01-18T06:31:26.797405Z","iopub.status.idle":"2022-01-18T06:31:26.813275Z","shell.execute_reply.started":"2022-01-18T06:31:26.797372Z","shell.execute_reply":"2022-01-18T06:31:26.812162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend for Dogecoin with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# visualization for trend\ndogecoin = asset_lookup('Dogecoin')\ndogecoin.plot(figsize=(15, 6))\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:26.814828Z","iopub.execute_input":"2022-01-18T06:31:26.815536Z","iopub.status.idle":"2022-01-18T06:31:31.011664Z","shell.execute_reply.started":"2022-01-18T06:31:26.815490Z","shell.execute_reply":"2022-01-18T06:31:31.010250Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n**Q: What is the trend, seasonality and residuals for Dogecoin with respect to Target?**\n\n---","metadata":{}},{"cell_type":"code","source":"# visualization for trend,seasonality and residual\nasset_decompose(dogecoin)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:31.013644Z","iopub.execute_input":"2022-01-18T06:31:31.014261Z","iopub.status.idle":"2022-01-18T06:31:31.931940Z","shell.execute_reply.started":"2022-01-18T06:31:31.014208Z","shell.execute_reply":"2022-01-18T06:31:31.930777Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# check for stationarity using ADF Test\nasset_stationarity(dogecoin)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:31.933415Z","iopub.execute_input":"2022-01-18T06:31:31.933664Z","iopub.status.idle":"2022-01-18T06:31:31.948835Z","shell.execute_reply.started":"2022-01-18T06:31:31.933634Z","shell.execute_reply":"2022-01-18T06:31:31.947378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---\n# Summary\n\n---","metadata":{}},{"cell_type":"markdown","source":"* Based on EDA and ADF test for top 5 crypto asset, most of the data point appears to be stationary.\n\n* And most of the top 5 assets are on the rise relatively and appears to be having good run.\n\n* Bitcoin crypto asset is kind of outlier in overall scenario and scheme of things.\n\n* New crypto asset may arrive in 2022 based on rising popularity of Cryptocurrency.\n\n* Cryptocurrency sector will continue to transition to become main asset class in 2022.","metadata":{}},{"cell_type":"markdown","source":"---\n**Thank you and Happy Learning.**\n\n---","metadata":{}},{"cell_type":"code","source":"thank_you_str=\"Thanks,Happy Learning,Collaboration,Thankyou,Keep Learning\"\n# create WordCloud with converted string\nwordcloud = WordCloud(width = 1000, height = 500, random_state=1, background_color='white', collocations=True).generate(thank_you_str)\nplt.figure(figsize=(20, 20))\nplt.imshow(wordcloud) \nplt.axis(\"off\")\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-01-18T06:31:31.951305Z","iopub.execute_input":"2022-01-18T06:31:31.951561Z","iopub.status.idle":"2022-01-18T06:31:32.566910Z","shell.execute_reply.started":"2022-01-18T06:31:31.951533Z","shell.execute_reply":"2022-01-18T06:31:32.566198Z"},"trusted":true},"execution_count":null,"outputs":[]}]}