{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.11.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":30894,"databundleVersionId":3147400,"sourceType":"competition"}],"dockerImageVersionId":31090,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 📈 Notebook 2: Crypto Return Prediction | Ridge + LightGBM Baseline\n\n---\n\n## 🧠 Intro\n\nOkay, now that we’ve explored the data in Notebook 1, it’s time to actually predict stuff.\nHere’s what we’re doing:\n- Use **BTC** and **ETH**\n- Engineer features from prices (VWAP, shadows, returns)\n- Train two baseline models: **Ridge Regression** and **LightGBM**\n- Evaluate using **correlation score**\n\nLet’s gooo 🚀\n\n---\n\n## 🧾 Imports & Setup","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom datetime import datetime\nimport time\nfrom sklearn.linear_model import Ridge\nfrom sklearn.metrics import mean_squared_error\nfrom sklearn.preprocessing import StandardScaler\nfrom sklearn.model_selection import train_test_split\nimport lightgbm as lgb","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:39:38.543226Z","iopub.execute_input":"2025-07-13T15:39:38.543931Z","iopub.status.idle":"2025-07-13T15:39:43.879709Z","shell.execute_reply.started":"2025-07-13T15:39:38.543876Z","shell.execute_reply":"2025-07-13T15:39:43.879024Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 📥 Load the Data (BTC & ETH Only)","metadata":{}},{"cell_type":"code","source":"data_folder = '../input/g-research-crypto-forecasting/'\ncrypto_df = pd.read_csv(data_folder + 'train.csv')\ncrypto_df['datetime'] = pd.to_datetime(crypto_df['timestamp'], unit='s')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:39:43.880961Z","iopub.execute_input":"2025-07-13T15:39:43.881949Z","iopub.status.idle":"2025-07-13T15:40:45.658811Z","shell.execute_reply.started":"2025-07-13T15:39:43.881896Z","shell.execute_reply":"2025-07-13T15:40:45.658243Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 🔧 Feature Engineering Helpers","metadata":{}},{"cell_type":"code","source":"def totimestamp(s):\n    return int(time.mktime(datetime.strptime(s, \"%d/%m/%Y\").timetuple()))\n\ndef log_return(series, periods=1):\n    return np.log(series).diff(periods=periods)\n\ndef upper_shadow(df):\n    return df['High'] - np.maximum(df['Close'], df['Open'])\n\ndef lower_shadow(df):\n    return np.minimum(df['Close'], df['Open']) - df['Low']","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:40:45.659793Z","iopub.execute_input":"2025-07-13T15:40:45.660022Z","iopub.status.idle":"2025-07-13T15:40:45.665042Z","shell.execute_reply.started":"2025-07-13T15:40:45.660004Z","shell.execute_reply":"2025-07-13T15:40:45.664301Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 🔍 Prepare BTC and ETH Features","metadata":{}},{"cell_type":"code","source":"def prepare_features(asset_id, name):\n    asset = crypto_df[crypto_df['Asset_ID'] == asset_id].set_index('timestamp')\n    asset = asset.reindex(range(asset.index[0], asset.index[-1]+60, 60), method='pad')\n\n    df = pd.DataFrame(index=asset.index)\n    df[f'{name}_logret_1'] = log_return(asset['VWAP'], periods=1)\n    df[f'{name}_logret_5'] = log_return(asset['VWAP'], periods=5)\n    df[f'{name}_abs_logret_1'] = df[f'{name}_logret_1'].abs()\n    df[f'{name}_upper_shadow'] = upper_shadow(asset)\n    df[f'{name}_lower_shadow'] = lower_shadow(asset)\n    df[f'{name}_target'] = asset['Target']\n\n    return df.dropna()\n\nbtc_df = prepare_features(1, 'BTC')\neth_df = prepare_features(6, 'ETH')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:41:20.572485Z","iopub.execute_input":"2025-07-13T15:41:20.573045Z","iopub.status.idle":"2025-07-13T15:41:22.314036Z","shell.execute_reply.started":"2025-07-13T15:41:20.573022Z","shell.execute_reply":"2025-07-13T15:41:22.313252Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 🪄 Merge and Align the Two Assets","metadata":{}},{"cell_type":"code","source":"combined_df = pd.concat([btc_df, eth_df], axis=1).dropna()\nX = combined_df.drop(columns=['BTC_target', 'ETH_target'])\ny_btc = combined_df['BTC_target']\ny_eth = combined_df['ETH_target']","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:41:44.110120Z","iopub.execute_input":"2025-07-13T15:41:44.110389Z","iopub.status.idle":"2025-07-13T15:41:45.209706Z","shell.execute_reply.started":"2025-07-13T15:41:44.110366Z","shell.execute_reply":"2025-07-13T15:41:45.209162Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## ✂️ Train/Test Split by Time (Not Random!)","metadata":{}},{"cell_type":"code","source":"split_time = totimestamp('01/06/2021')\nX_train = X.loc[X.index < split_time]\nX_test = X.loc[X.index >= split_time]\ny_btc_train = y_btc.loc[X.index < split_time]\ny_btc_test = y_btc.loc[X.index >= split_time]\ny_eth_train = y_eth.loc[X.index < split_time]\ny_eth_test = y_eth.loc[X.index >= split_time]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:42:05.764381Z","iopub.execute_input":"2025-07-13T15:42:05.765085Z","iopub.status.idle":"2025-07-13T15:42:05.936106Z","shell.execute_reply.started":"2025-07-13T15:42:05.765059Z","shell.execute_reply":"2025-07-13T15:42:05.935243Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 🧼 Standardize Features\n","metadata":{}},{"cell_type":"code","source":"scaler = StandardScaler()\nX_train_scaled = scaler.fit_transform(X_train)\nX_test_scaled = scaler.transform(X_test)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:42:34.634301Z","iopub.execute_input":"2025-07-13T15:42:34.634979Z","iopub.status.idle":"2025-07-13T15:42:35.040186Z","shell.execute_reply.started":"2025-07-13T15:42:34.634949Z","shell.execute_reply":"2025-07-13T15:42:35.039419Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 📉 Model 1: Ridge Regression (BTC)","metadata":{}},{"cell_type":"code","source":"ridge_btc = Ridge(alpha=1.0)\nridge_btc.fit(X_train_scaled, y_btc_train)\ny_pred_btc = ridge_btc.predict(X_test_scaled)\nprint(\"BTC Ridge Corr:\", np.corrcoef(y_pred_btc, y_btc_test)[0, 1])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:43:03.277776Z","iopub.execute_input":"2025-07-13T15:43:03.278064Z","iopub.status.idle":"2025-07-13T15:43:03.470671Z","shell.execute_reply.started":"2025-07-13T15:43:03.278044Z","shell.execute_reply":"2025-07-13T15:43:03.469958Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 📉 Model 2: Ridge Regression (ETH)","metadata":{}},{"cell_type":"code","source":"ridge_eth = Ridge(alpha=1.0)\nridge_eth.fit(X_train_scaled, y_eth_train)\ny_pred_eth = ridge_eth.predict(X_test_scaled)\nprint(\"ETH Ridge Corr:\", np.corrcoef(y_pred_eth, y_eth_test)[0, 1])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:43:32.011469Z","iopub.execute_input":"2025-07-13T15:43:32.011735Z","iopub.status.idle":"2025-07-13T15:43:32.161647Z","shell.execute_reply.started":"2025-07-13T15:43:32.011713Z","shell.execute_reply":"2025-07-13T15:43:32.160894Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## ⚡ Model 3: LightGBM for BTC","metadata":{}},{"cell_type":"code","source":"train_data = lgb.Dataset(X_train, label=y_btc_train)\ntest_data = lgb.Dataset(X_test, label=y_btc_test)\n\nparams = {\n    'objective': 'regression',\n    'metric': 'rmse',\n    'verbosity': -1\n}\n\nlgb_model = lgb.train(params, train_data, valid_sets=[test_data])\ny_pred_lgb = lgb_model.predict(X_test)\nprint(\"BTC LightGBM Corr:\", np.corrcoef(y_pred_lgb, y_btc_test)[0, 1])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:48:07.302340Z","iopub.execute_input":"2025-07-13T15:48:07.302611Z","iopub.status.idle":"2025-07-13T15:48:11.198482Z","shell.execute_reply.started":"2025-07-13T15:48:07.302591Z","shell.execute_reply":"2025-07-13T15:48:11.197732Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 📊 Visualizing Predictions vs True Returns","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(14, 4))\nplt.plot(y_btc_test.index[:500], y_btc_test[:500], label='True BTC Returns', alpha=0.7)\nplt.plot(y_btc_test.index[:500], y_pred_btc[:500], label='Ridge BTC Predictions', alpha=0.7)\nplt.plot(y_btc_test.index[:500], y_pred_lgb[:500], label='LGB BTC Predictions', alpha=0.7)\nplt.title(\"BTC: True vs Predicted Returns (First 500 samples)\")\nplt.legend()\nplt.xlabel(\"Timestamp\")\nplt.ylabel(\"Return\")\nplt.grid(True)\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-07-13T15:50:42.510279Z","iopub.execute_input":"2025-07-13T15:50:42.510555Z","iopub.status.idle":"2025-07-13T15:50:42.834010Z","shell.execute_reply.started":"2025-07-13T15:50:42.510534Z","shell.execute_reply":"2025-07-13T15:50:42.833218Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## ✅ Conclusion\n\nWe built 3 baseline models:\n- Ridge Regression for BTC and ETH\n- LightGBM for BTC\n\n### 📌 Results:\n- **BTC Ridge Corr**: `0.0278`\n- **ETH Ridge Corr**: `0.0070`\n- **BTC LightGBM Corr**: `0.0217`\n\nClearly, the correlation scores are low, but that’s expected — crypto markets are noisy, volatile, and extremely non-stationary.\n\n🔧 To improve:\n- Add lag/rolling window features\n- Try additional crypto assets\n- Feature selection based on correlation\n- Time-aware cross-validation (like walk-forward)\n\n📊 The plots show that even though the signals are weak, the predicted curves try to align with some spikes and dips.\n\nNext step? Build a submission notebook using the time-series API, or try deep learning models like LSTM or GRU.\n\n🙌 Let’s keep experimenting and climbing that leaderboard!","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}