{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":30894,"databundleVersionId":3147400,"sourceType":"competition"}],"dockerImageVersionId":31090,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 📌 Crypto Forecasting EDA | BTC & ETH: Candlesticks, Log Returns, Correlation\n\n---\n\n## 🧠 Intro\n\nYo! If you're diving into the G-Research Crypto Forecasting dataset and not sure where to start, this notebook’s for you. We're gonna explore some big players — Bitcoin (BTC) and Ethereum (ETH) — and see how they behave minute by minute. We'll pull some sick charts, crunch log returns, and peek at asset correlations.\n\n---\n\n## 📥 Load the Data\n\nLet’s first load the massive crypto dataset and check out what we’re working with. We’ll be using the `train.csv` and `asset_details.csv` files.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np \nimport matplotlib.pyplot as plt\nimport plotly.graph_objects as go\nfrom datetime import datetime\nimport time\nimport os","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-11T14:09:50.552471Z","iopub.execute_input":"2025-07-11T14:09:50.552747Z","iopub.status.idle":"2025-07-11T14:09:50.556779Z","shell.execute_reply.started":"2025-07-11T14:09:50.552724Z","shell.execute_reply":"2025-07-11T14:09:50.556135Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def totimestamp(s):\n    return int(time.mktime(datetime.strptime(s, \"%d/%m/%Y\").timetuple()))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-11T14:09:50.557826Z","iopub.execute_input":"2025-07-11T14:09:50.558074Z","iopub.status.idle":"2025-07-11T14:09:50.576540Z","shell.execute_reply.started":"2025-07-11T14:09:50.558057Z","shell.execute_reply":"2025-07-11T14:09:50.575792Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"data_folder = \"/kaggle/input/g-research-crypto-forecasting/\"\ncrypto_df = pd.read_csv(data_folder + 'train.csv')\nasset_details = pd.read_csv(data_folder + 'asset_details.csv')\ncrypto_df['datetime'] = pd.to_datetime(crypto_df['timestamp'], unit='s')\n\nprint(\"Loaded training data shape:\", crypto_df.shape)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-11T14:09:50.577817Z","iopub.execute_input":"2025-07-11T14:09:50.578046Z","iopub.status.idle":"2025-07-11T14:10:20.290779Z","shell.execute_reply.started":"2025-07-11T14:09:50.578026Z","shell.execute_reply":"2025-07-11T14:10:20.290028Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n## 🔎 Focus: BTC and ETH\n\nLet’s zoom into the two most iconic cryptos: Bitcoin (Asset_ID = 1) and Ethereum (Asset_ID = 6). These are super liquid and super correlated — perfect to start our EDA.","metadata":{}},{"cell_type":"code","source":"btc = crypto_df[crypto_df['Asset_ID'] == 1].set_index('timestamp')\neth = crypto_df[crypto_df['Asset_ID'] == 6].set_index('timestamp')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-11T14:10:20.291563Z","iopub.execute_input":"2025-07-11T14:10:20.291804Z","iopub.status.idle":"2025-07-11T14:10:20.912516Z","shell.execute_reply.started":"2025-07-11T14:10:20.291786Z","shell.execute_reply":"2025-07-11T14:10:20.911749Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 🕯️ Candlestick Visualization (BTC)\n\nVisualizing candles helps us understand price movement over time. We’ll use Plotly to render interactive OHLC charts for BTC.","metadata":{}},{"cell_type":"code","source":"btc_candle = btc.iloc[-200:]\nfig = go.Figure(data=[go.Candlestick(\n    x=pd.to_datetime(btc_candle.index, unit='s'),\n    open=btc_candle['Open'], high=btc_candle['High'],\n    low=btc_candle['Low'], close=btc_candle['Close']\n)])\nfig.update_layout(title='BTC Candlestick (Recent 200 mins)', xaxis_title='Time', yaxis_title='Price (USD)')\nfig.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-11T14:10:20.913947Z","iopub.execute_input":"2025-07-11T14:10:20.914210Z","iopub.status.idle":"2025-07-11T14:10:20.926547Z","shell.execute_reply.started":"2025-07-11T14:10:20.914193Z","shell.execute_reply":"2025-07-11T14:10:20.925771Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 📈 BTC vs ETH: Closing Price Trends\n\nLet’s reindex the data so we fill any missing time gaps (this happens a lot in minute-level data). Then we’ll plot closing prices over time to compare.","metadata":{}},{"cell_type":"code","source":"btc = btc.reindex(range(btc.index[0], btc.index[-1]+60, 60), method='pad')\neth = eth.reindex(range(eth.index[0], eth.index[-1]+60, 60), method='pad')\n\nplt.figure(figsize=(14, 5))\nplt.plot(pd.to_datetime(btc.index, unit='s'), btc['Close'], label='BTC')\nplt.plot(pd.to_datetime(eth.index, unit='s'), eth['Close'], label='ETH')\nplt.title(\"Bitcoin vs Ethereum: Close Price History\")\nplt.xlabel(\"Time\")\nplt.ylabel(\"Price\")\nplt.legend()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-11T14:24:14.982487Z","iopub.execute_input":"2025-07-11T14:24:14.983076Z","iopub.status.idle":"2025-07-11T14:24:17.464385Z","shell.execute_reply.started":"2025-07-11T14:24:14.983052Z","shell.execute_reply":"2025-07-11T14:24:17.463555Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n## 📉 Log Returns: Volatility Check\n\nRegular price curves are cool, but they don’t show how wild the ride is. For that, we calculate **log returns** — which reflect relative price changes and look like noise (great for modeling).\n","metadata":{}},{"cell_type":"code","source":"def log_return(series, periods=1):\n    return np.log(series).diff(periods=periods)\n\nbtc['log_return'] = log_return(btc['Close'])\neth['log_return'] = log_return(eth['Close'])\n\nplt.figure(figsize=(14, 5))\nplt.plot(pd.to_datetime(btc.index, unit='s'), btc['log_return'], label='BTC Log Return', alpha=0.7)\nplt.plot(pd.to_datetime(eth.index, unit='s'), eth['log_return'], label='ETH Log Return', alpha=0.7)\nplt.title(\"Log Returns Over Time\")\nplt.xlabel(\"Time\")\nplt.ylabel(\"Log Return\")\nplt.legend()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-11T14:10:24.147559Z","iopub.execute_input":"2025-07-11T14:10:24.147836Z","iopub.status.idle":"2025-07-11T14:10:26.767705Z","shell.execute_reply.started":"2025-07-11T14:10:24.147811Z","shell.execute_reply":"2025-07-11T14:10:26.767078Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 🔁 BTC-ETH Rolling Correlation (1000-minute window)\n\nLet’s see how closely BTC and ETH move together — not just overall, but over time. We use a rolling correlation to capture this dynamic vibe.\n","metadata":{}},{"cell_type":"code","source":"btc_mini = btc.loc[totimestamp('01/06/2021'):totimestamp('01/07/2021')]\neth_mini = eth.loc[totimestamp('01/06/2021'):totimestamp('01/07/2021')]\n\nbtc_log = log_return(btc_mini['Close']).dropna()\neth_log = log_return(eth_mini['Close']).dropna()\nrolling_corr = btc_log.rolling(1000).corr(eth_log)\n\nplt.figure(figsize=(14, 4))\nplt.plot(pd.to_datetime(btc_log.index, unit='s'), rolling_corr)\nplt.title(\"BTC-ETH Rolling Correlation (June 2021)\")\nplt.xlabel(\"Time\")\nplt.ylabel(\"Correlation\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-11T14:10:26.768820Z","iopub.execute_input":"2025-07-11T14:10:26.769063Z","iopub.status.idle":"2025-07-11T14:10:26.993876Z","shell.execute_reply.started":"2025-07-11T14:10:26.769044Z","shell.execute_reply":"2025-07-11T14:10:26.993182Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n## 🌐 Full Asset Correlation Matrix\n\nWe’ve just looked at BTC and ETH. But how about the whole crypto gang? Let’s build a heatmap of log return correlations across all 14 assets.\n","metadata":{}},{"cell_type":"code","source":"all_returns = pd.DataFrame()\n\nfor asset_id in asset_details['Asset_ID']:\n    asset = crypto_df[crypto_df['Asset_ID'] == asset_id].set_index('timestamp')\n    asset = asset.reindex(range(asset.index[0], asset.index[-1] + 60, 60), method='pad')\n\n    lret = log_return(asset['Close']).dropna()\n    lret.name = asset_details.loc[asset_details['Asset_ID'] == asset_id, 'Asset_Name'].values[0]\n\n    all_returns = pd.concat([all_returns, lret], axis=1)\n\nall_returns.dropna(inplace=True)\n\ncorr_matrix = all_returns.corr()\n\nplt.figure(figsize=(12, 10))\nplt.imshow(corr_matrix, cmap='coolwarm', interpolation='none')\nplt.xticks(range(len(corr_matrix)), corr_matrix.columns, rotation=90)\nplt.yticks(range(len(corr_matrix)), corr_matrix.columns)\nplt.colorbar()\nplt.title(\"Log Return Correlation Matrix Across All Assets\")\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-11T14:12:28.451357Z","iopub.execute_input":"2025-07-11T14:12:28.452088Z","iopub.status.idle":"2025-07-11T14:12:41.591982Z","shell.execute_reply.started":"2025-07-11T14:12:28.452063Z","shell.execute_reply":"2025-07-11T14:12:41.591313Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## ✅ Final Thoughts\n\nBoom! We now have:\n- Candlestick visuals of BTC 🕯️\n- Price & log return trends for BTC and ETH 📈\n- A rolling correlation view 🔁\n- A matrix showing how every asset moves together 🌐\n\nNext step? Use these insights to feed our model. Let’s go build a Ridge or LightGBM forecaster. Stay tuned for Part 2!\n\n🙏 If this helped, give it an upvote and follow along for more crypto ML deep dives. Peace!\n","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}