{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![](https://storage.googleapis.com/kaggle-competitions/kaggle/30894/logos/header.png?t=2021-09-14-17-32-48)\n\n\n## About this competition\nIn this competition, you'll use your machine learning expertise to **forecast short term returns in 14 popular cryptocurrencies**. We have amassed a dataset of millions of rows of high-frequency market data dating back to 2018 which you can use to build your model. Once the submission deadline has passed, **your final score will be calculated over the following 3 months using live crypto data** as it is collected.\n\n## Evaluation Metrics\n[Pearson correlation coefficient](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)\n\n## Data\nThis dataset contains information on historic trades for several cryptoassets, such as Bitcoin and Ethereum. Your challenge is to **predict their future returns**.\n\nAs historic cryptocurrency prices are not confidential this will be a forecasting competition using **the time series API**. Furthermore the public leaderboard targets are publicly available and are provided as part of the competition dataset. Expect to see many people submitting perfect submissions for fun. Accordingly, **THE PUBLIC LEADERBOARD FOR THIS COMPETITION IS NOT MEANINGFUL** and is only provided as a convenience for anyone who wants to test their code. The final private leaderboard will be determined using real market data gathered after the submission period closes.\n\n## Code Requirements\nThis is a code competition! In order for the \"Submit\" button to be active after a commit, the following conditions must be met:\n\n- CPU Notebook <= 9 hours run-time\n- GPU Notebook <= 9 hours run-time\n- **Internet access disabled**\n- Freely & publicly available external data is allowed, including pre-trained models\n- Submission file must be named submission.csv\n\nSo let's get the ball rolling!","metadata":{}},{"cell_type":"markdown","source":"# Libraries","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport gc\nimport pathlib\nfrom tqdm.auto import tqdm\nimport joblib\nimport pathlib\nimport json\nimport glob\nimport time\nimport datetime\nfrom scipy import stats\nfrom multiprocessing import Pool, cpu_count\n\n# models\nfrom xgboost import XGBRegressor\nimport lightgbm as lgb\n\n# visualize\nimport matplotlib.pyplot as plt\nimport matplotlib.style as style\nfrom matplotlib_venn import venn2, venn3\nimport seaborn as sns\nfrom matplotlib import pyplot\nfrom matplotlib.ticker import ScalarFormatter\nsns.set_context(\"talk\")\nstyle.use('seaborn-colorblind')\n\nimport warnings\nwarnings.simplefilter('ignore')\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-11-04T00:06:37.033013Z","iopub.execute_input":"2021-11-04T00:06:37.033422Z","iopub.status.idle":"2021-11-04T00:06:39.181275Z","shell.execute_reply.started":"2021-11-04T00:06:37.033289Z","shell.execute_reply":"2021-11-04T00:06:39.180258Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Config","metadata":{}},{"cell_type":"code","source":"class CFG:\n    INPUT_DIR = '/kaggle/input/g-research-crypto-forecasting/'\n    OUTPUT_DIR = './'\n    SEED = 20211103","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:06:39.183187Z","iopub.execute_input":"2021-11-04T00:06:39.183531Z","iopub.status.idle":"2021-11-04T00:06:39.190031Z","shell.execute_reply.started":"2021-11-04T00:06:39.183473Z","shell.execute_reply":"2021-11-04T00:06:39.188675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Utils\nThe data is huge! We might want to reduce memory usage somehow.","metadata":{}},{"cell_type":"code","source":"def reduce_mem_usage(df):\n    \"\"\" iterate through all the columns of a dataframe and modify the data type\n        to reduce memory usage.        \n    \"\"\"\n    start_mem = df.memory_usage().sum() / 1024**2\n    print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))\n    \n    for col in df.columns:\n        col_type = df[col].dtype\n        \n        if col_type != object:\n            c_min = df[col].min()\n            c_max = df[col].max()\n            if str(col_type)[:3] == 'int':\n                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                    df[col] = df[col].astype(np.int8)\n                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                    df[col] = df[col].astype(np.int16)\n                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                    df[col] = df[col].astype(np.int32)\n                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                    df[col] = df[col].astype(np.int64)  \n            else:\n                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n                    df[col] = df[col].astype(np.float16)\n                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n                    df[col] = df[col].astype(np.float32)\n                else:\n                    df[col] = df[col].astype(np.float64)\n#         else:\n#             df[col] = df[col].astype('category')\n\n    end_mem = df.memory_usage().sum() / 1024**2\n    print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))\n    print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))\n    \n    return df","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:06:39.191391Z","iopub.execute_input":"2021-11-04T00:06:39.192233Z","iopub.status.idle":"2021-11-04T00:06:39.205167Z","shell.execute_reply.started":"2021-11-04T00:06:39.192185Z","shell.execute_reply":"2021-11-04T00:06:39.204412Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load data","metadata":{}},{"cell_type":"markdown","source":"## train.csv - The training set\n\n- timestamp - A timestamp for the minute covered by the row.\n- Asset_ID - An ID code for the cryptoasset.\n- Count - The number of trades that took place this minute.\n- Open - The USD price at the beginning of the minute.\n- High - The highest USD price during the minute.\n- Low - The lowest USD price during the minute.\n- Close - The USD price at the end of the minute.\n- Volume - The number of cryptoasset units traded during the minute.\n- VWAP - The volume weighted average price for the minute.\n- Target - 15 minute residualized returns. See the ['Prediction and Evaluation' section of this notebook](https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition) for details of how the target is calculated.","metadata":{}},{"cell_type":"code","source":"%%time\n\ntrain = pd.read_csv(os.path.join(CFG.INPUT_DIR, 'train.csv')).pipe(reduce_mem_usage)\nprint(train.shape)\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:06:39.207086Z","iopub.execute_input":"2021-11-04T00:06:39.207657Z","iopub.status.idle":"2021-11-04T00:07:38.942595Z","shell.execute_reply.started":"2021-11-04T00:06:39.207623Z","shell.execute_reply":"2021-11-04T00:07:38.941629Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## asset_details.csv \n\nProvides the real name and of the cryptoasset for each Asset_ID and **the weight each cryptoasset receives in the metric**.\n","metadata":{}},{"cell_type":"code","source":"asset_details = pd.read_csv(os.path.join(CFG.INPUT_DIR, 'asset_details.csv'))\nasset_details['Asset_ID'] = asset_details['Asset_ID'].astype(np.int8)\nprint(asset_details.shape)\nasset_details","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:38.944817Z","iopub.execute_input":"2021-11-04T00:07:38.945125Z","iopub.status.idle":"2021-11-04T00:07:38.971299Z","shell.execute_reply.started":"2021-11-04T00:07:38.945077Z","shell.execute_reply":"2021-11-04T00:07:38.970447Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## example_sample_submission.csv\n","metadata":{}},{"cell_type":"code","source":"example_sample_submission = pd.read_csv(os.path.join(CFG.INPUT_DIR, 'example_sample_submission.csv'))\nprint(example_sample_submission.shape)\nexample_sample_submission.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:38.972875Z","iopub.execute_input":"2021-11-04T00:07:38.973434Z","iopub.status.idle":"2021-11-04T00:07:38.992070Z","shell.execute_reply.started":"2021-11-04T00:07:38.973384Z","shell.execute_reply":"2021-11-04T00:07:38.991085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## example_test.csv\n\nAn example of the data that will be delivered by the time series API. The data is just copied from train.csv.","metadata":{}},{"cell_type":"code","source":"%%time\n\ntest_df = pd.read_csv(os.path.join(CFG.INPUT_DIR, 'example_test.csv')).pipe(reduce_mem_usage)\nprint(test_df.shape)\ntest_df.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:38.993424Z","iopub.execute_input":"2021-11-04T00:07:38.993907Z","iopub.status.idle":"2021-11-04T00:07:39.029897Z","shell.execute_reply.started":"2021-11-04T00:07:38.993868Z","shell.execute_reply":"2021-11-04T00:07:39.029072Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Other files...\n\n- gresearch_crypto - An unoptimized version of the time series API files for offline work. You may need Python 3.7 and a Linux environment to run it without errors.\n\n- supplemental_train.csv - After the submission period is over this file's data will be replaced with cryptoasset prices from the submission period. The current copy, which is just filled approximately the right amount of data from train.csv is provided as a placeholder.","metadata":{}},{"cell_type":"markdown","source":"# EDA\n\nSome exploratory data analysis are performed here. \n\nYou might want to check [this official tutorial: Tutorial to the G-Research Crypto Competition\n](https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition) as well.","metadata":{}},{"cell_type":"code","source":"# dataframe info\ntrain.info()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:39.031503Z","iopub.execute_input":"2021-11-04T00:07:39.031836Z","iopub.status.idle":"2021-11-04T00:07:39.057376Z","shell.execute_reply.started":"2021-11-04T00:07:39.031801Z","shell.execute_reply":"2021-11-04T00:07:39.056755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# missing values?\ntrain.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:39.058622Z","iopub.execute_input":"2021-11-04T00:07:39.059400Z","iopub.status.idle":"2021-11-04T00:07:39.649275Z","shell.execute_reply.started":"2021-11-04T00:07:39.059351Z","shell.execute_reply":"2021-11-04T00:07:39.648317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"example_sample_submission.info()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:39.652578Z","iopub.execute_input":"2021-11-04T00:07:39.652915Z","iopub.status.idle":"2021-11-04T00:07:39.664869Z","shell.execute_reply.started":"2021-11-04T00:07:39.652872Z","shell.execute_reply":"2021-11-04T00:07:39.663857Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(3, 5, figsize=(20, 12), sharex=True)\nax = ax.flatten()\nfor i, asset in enumerate(train['Asset_ID'].unique()):\n    train.query('Asset_ID == @asset')['Target'].hist(bins=30, color='k', alpha=0.7, ax=ax[i])\n    asset_name = asset_details.query('Asset_ID == @asset')['Asset_Name'].values[0]\n    weight = asset_details.query('Asset_ID == @asset')['Weight'].values[0]\n    ax[i].set_title(f'{asset_name}\\n(weight={weight})')\n    \nax[-1].axis('off')\nplt.tight_layout()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:39.665963Z","iopub.execute_input":"2021-11-04T00:07:39.666861Z","iopub.status.idle":"2021-11-04T00:07:49.862947Z","shell.execute_reply.started":"2021-11-04T00:07:39.666816Z","shell.execute_reply":"2021-11-04T00:07:49.861410Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Feature Engineering\n\nYeah finally machine learning part!\n\nHere we generate sets of stock price features. There are some caveats to be aware of:\n\n- No Leak: we cannot use a feature which uses the future information (this is a forecasting task!)\n- Stationaly features: Our features have to work whenever (scales must be stationaly over the periods of time)\n\nAlso, I already add 'train' or 'validation' flag in a time-series split manner.","metadata":{}},{"cell_type":"code","source":"# select train and validation period\n\n# auxiliary function, from datetime to timestamp\ntotimestamp = lambda s: np.int32(time.mktime(datetime.datetime.strptime(s, \"%d/%m/%Y\").timetuple()))\n\n# train_window = [totimestamp(\"01/05/2021\"), totimestamp(\"30/05/2021\")]\n# test_window = [totimestamp(\"01/06/2021\"), totimestamp(\"30/06/2021\")]\ntrain_window = [totimestamp(\"01/01/2018\"), totimestamp(\"21/09/2020\")]\nvalid_window = [totimestamp(\"22/09/2020\"), totimestamp(\"21/09/2021\")]\n\ntrain = train.set_index(\"timestamp\")\nbeg_ = train.index[0].astype('datetime64[s]')\nend_ = train.index[-1].astype('datetime64[s]')\nprint('>> data goes from ', beg_, 'to ', end_, 'shape=', train.shape)\n\n# drop rows without target\ntrain.dropna(subset=['Target'], inplace=True)\n\n# add train flag\ntrain['train_flg'] = 1\ntrain.loc[valid_window[0]:valid_window[1], 'train_flg'] = 0","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:49.865035Z","iopub.execute_input":"2021-11-04T00:07:49.865602Z","iopub.status.idle":"2021-11-04T00:07:52.737828Z","shell.execute_reply.started":"2021-11-04T00:07:49.865554Z","shell.execute_reply":"2021-11-04T00:07:52.736913Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def add_asset_details(train, asset_details):\n    \"\"\"Add asset details to train df\n    \"\"\"\n    return train.merge(\n        asset_details\n        , how='left'\n        , on='Asset_ID'\n    )\n\n# merge asset_details\ntrain = add_asset_details(train, asset_details)","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:52.738972Z","iopub.execute_input":"2021-11-04T00:07:52.739556Z","iopub.status.idle":"2021-11-04T00:07:57.880670Z","shell.execute_reply.started":"2021-11-04T00:07:52.739467Z","shell.execute_reply":"2021-11-04T00:07:57.879704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_row_feats(df):\n    \"\"\"Feature engineering by row\n    \"\"\"\n    df['upper_shadow'] = df['High'] / df[['Close', 'Open']].max(axis=1)\n    df['lower_shadow'] = df[['Close', 'Open']].min(axis=1) / df['Low']\n    df['open2close'] = df['Close'] / df['Open']\n    df['high2low'] = df['High'] / df['Low']\n    mean_price = df[['Open', 'High', 'Low', 'Close']].mean(axis=1)\n    median_price = df[['Open', 'High', 'Low', 'Close']].median(axis=1)\n    df['high2mean'] = df['High'] / mean_price\n    df['low2mean'] = df['Low'] / mean_price\n    df['high2median'] = df['High'] / median_price\n    df['low2median'] = df['Low'] / median_price\n    df['volume2count'] = df['Volume'] / (df['Count'] + 1)\n    return df   \n    ","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:57.882574Z","iopub.execute_input":"2021-11-04T00:07:57.882883Z","iopub.status.idle":"2021-11-04T00:07:57.891690Z","shell.execute_reply.started":"2021-11-04T00:07:57.882845Z","shell.execute_reply":"2021-11-04T00:07:57.890609Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n\n# feature engineering\nfeature_df = get_row_feats(train)\n\nprint(feature_df.shape)\nfeature_df.tail()","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:07:57.893263Z","iopub.execute_input":"2021-11-04T00:07:57.893589Z","iopub.status.idle":"2021-11-04T00:08:21.342024Z","shell.execute_reply.started":"2021-11-04T00:07:57.893555Z","shell.execute_reply":"2021-11-04T00:08:21.341242Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Modeling\n\nAs a simple starter, let's use LightGBM. \n\nWe use time-series split as the validation strategy for this forecasting task.\n\nThere are two ways to try out: full model (using all the crypto data at once) and individual model (model for each asset).","metadata":{}},{"cell_type":"code","source":"target = 'Target'\ndrops = ['timestamp', 'Asset_Name', 'Weight', 'train_flg', 'Open', 'High', 'Low', 'Close', 'Volume', 'VWAP']\nfeatures = [f for f in train.columns if f not in drops + [target]]\ncategoricals = ['Asset_ID']\n\nprint('{:,} features: {}'.format(len(features), features))","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:08:21.343172Z","iopub.execute_input":"2021-11-04T00:08:21.343399Z","iopub.status.idle":"2021-11-04T00:08:21.350186Z","shell.execute_reply.started":"2021-11-04T00:08:21.343371Z","shell.execute_reply":"2021-11-04T00:08:21.349387Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Full Model","metadata":{}},{"cell_type":"code","source":"# parameters\nparams = {\n        'n_estimators': 10000,\n        'objective': 'regression',\n        'metric': 'rmse',\n        'boosting_type': 'gbdt',\n        'max_depth': -1,\n        'learning_rate': 0.01,\n        'subsample': 0.72,\n        'subsample_freq': 4,\n        'feature_fraction': 0.4,\n        'lambda_l1': 1,\n        'lambda_l2': 1,\n        'seed': 46,\n        }\n\n# train (full model)\nmodel = lgb.LGBMRegressor(**params)\nmodel.fit(\n    feature_df.query('train_flg == 1')[features],\n    feature_df.query('train_flg == 1')[target].values, \n    eval_set=[(feature_df.query('train_flg == 0')[features]\n               , feature_df.query('train_flg == 0')[target].values)],\n    verbose=-1, \n    early_stopping_rounds=100,\n    categorical_feature=categoricals,\n)\n\n# save model\njoblib.dump(model, os.path.join(CFG.OUTPUT_DIR, 'lgb_model_val.pkl'))\nprint('lgb model saved!')\n\n# feature importance\nfi_df = pd.DataFrame()\nfi_df['features'] = features\nfi_df['importance'] = model.booster_.feature_importance(importance_type=\"gain\")","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:08:21.351547Z","iopub.execute_input":"2021-11-04T00:08:21.351771Z","iopub.status.idle":"2021-11-04T00:10:42.405446Z","shell.execute_reply.started":"2021-11-04T00:08:21.351743Z","shell.execute_reply":"2021-11-04T00:10:42.404547Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# plot feature importance\nfig, ax = plt.subplots(1, 1, figsize=(7, 15))\nsns.barplot(\n    x='importance'\n    , y='features'\n    , data=fi_df.sort_values(by=['importance'], ascending=False)\n    , ax=ax\n)","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:10:42.406854Z","iopub.execute_input":"2021-11-04T00:10:42.407169Z","iopub.status.idle":"2021-11-04T00:10:42.721152Z","shell.execute_reply.started":"2021-11-04T00:10:42.407128Z","shell.execute_reply":"2021-11-04T00:10:42.720529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Individual Model","metadata":{}},{"cell_type":"code","source":"# train (full model)\nfor asset in feature_df['Asset_ID'].unique():\n    model = lgb.LGBMRegressor(**params)\n    model.fit(\n        feature_df.query('train_flg == 1 and Asset_ID == @asset')[features],\n        feature_df.query('train_flg == 1 and Asset_ID == @asset')[target].values, \n        eval_set=[(feature_df.query('train_flg == 0 and Asset_ID == @asset')[features]\n                   , feature_df.query('train_flg == 0 and Asset_ID == @asset')[target].values)],\n        verbose=-1, \n        early_stopping_rounds=100,\n    )\n    \n    # save model\n    asset_name = feature_df.query('Asset_ID == @asset')['Asset_Name'].values[0]\n    joblib.dump(model, os.path.join(CFG.OUTPUT_DIR, 'lgb_model_{}_val.pkl'.format(asset_name)))\n    print(f'lgb model for {asset_name} saved!')","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:10:42.722227Z","iopub.execute_input":"2021-11-04T00:10:42.723053Z","iopub.status.idle":"2021-11-04T00:13:41.421086Z","shell.execute_reply.started":"2021-11-04T00:10:42.723012Z","shell.execute_reply":"2021-11-04T00:13:41.420192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Validation score\nLet's see how good our model is in terms of the competition metric: weighted correlation.\n","metadata":{}},{"cell_type":"code","source":"# https://stackoverflow.com/questions/38641691/weighted-correlation-coefficient-with-pandas\ndef m(x, w):\n    \"\"\"Weighted Mean\"\"\"\n    return np.sum(x * w) / np.sum(w)\n\ndef cov(x, y, w):\n    \"\"\"Weighted Covariance\"\"\"\n    return np.sum(w * (x - m(x, w)) * (y - m(y, w))) / np.sum(w)\n\ndef corr(x, y, w):\n    \"\"\"Weighted Correlation\"\"\"\n    return cov(x, y, w) / np.sqrt(cov(x, x, w) * cov(y, y, w))\n\n# Compute the correlation\nprint('FULL MODEL *******************************************')\nmodel = joblib.load(os.path.join(CFG.OUTPUT_DIR, 'lgb_model_val.pkl'))\nval_df = train.query('train_flg == 0').copy()\nval_df['Prediction'] = model.predict(val_df[features])\nfor asset in val_df['Asset_ID'].unique():\n    tmp = val_df.query('Asset_ID == @asset')\n    coin = tmp['Asset_Name'].values[0]\n    r = corr(tmp['Prediction'], tmp['Target'], tmp['Weight'])\n    print('')\n    print('- {}: Validation Score (weighted correlation) = {:.4f}'.format(coin, r))\n\nr = corr(val_df['Prediction'], val_df['Target'], val_df['Weight'])\nprint('=> Overall Validation Score (weighted correlation) = {:.4f}'.format(r))","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:13:41.422446Z","iopub.execute_input":"2021-11-04T00:13:41.422745Z","iopub.status.idle":"2021-11-04T00:13:54.920800Z","shell.execute_reply.started":"2021-11-04T00:13:41.422713Z","shell.execute_reply":"2021-11-04T00:13:54.919606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('INDIVIDUAL MODEL *******************************************')\nval_df['Prediction'] = 0\nfor asset in val_df['Asset_ID'].unique():\n    # load model\n    model = joblib.load(os.path.join(CFG.OUTPUT_DIR, 'lgb_model_{}_val.pkl'.format(asset_name)))\n    \n    # inference\n    val_df.loc[val_df['Asset_ID'] == asset, 'Prediction'] = model.predict(val_df.loc[val_df['Asset_ID'] == asset, features])\n    tmp = val_df.query('Asset_ID == @asset')\n    asset_name = tmp['Asset_Name'].values[0]\n    r = corr(tmp['Prediction'], tmp['Target'], tmp['Weight'])\n    print('')\n    print('- {}: Validation Score (weighted correlation) = {:.4f}'.format(asset_name, r))\n    \nr = corr(val_df['Prediction'], val_df['Target'], val_df['Weight'])\nprint('=> Overall Validation Score (weighted correlation) = {:.4f}'.format(r))","metadata":{"execution":{"iopub.status.busy":"2021-11-04T00:14:28.422567Z","iopub.execute_input":"2021-11-04T00:14:28.423238Z","iopub.status.idle":"2021-11-04T00:14:43.761377Z","shell.execute_reply.started":"2021-11-04T00:14:28.423200Z","shell.execute_reply":"2021-11-04T00:14:43.759985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Fit with all the data (no validation)\n\nLooks like individual model suffers. So let's just stick to the full model.","metadata":{}},{"cell_type":"code","source":"# train (full model)\nparams['n_estimators'] = 101\nmodel = lgb.LGBMRegressor(**params)\nmodel.fit(\n    feature_df[features],\n    feature_df[target].values, \n    verbose=-1, \n    categorical_feature=categoricals,\n)\n\n# save model\njoblib.dump(model, 'lgb_model.pkl')\nprint('lgb model saved!')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submission\nThis competition uses Time-Series API. For details see:\n\nhttps://www.kaggle.com/c/g-research-crypto-forecasting/overview/evaluation","metadata":{}},{"cell_type":"code","source":"import gresearch_crypto\nenv = gresearch_crypto.make_env()   # initialize the environment\niter_test = env.iter_test()    # an iterator which loops over the test set and sample submission\nfor (test_df, sample_prediction_df) in iter_test:\n    # feature engineering\n    test_df = get_row_feats(test_df)\n    \n    # inference\n    sample_prediction_df['Target'] = model.predict(test_df[features])  # make your predictions here\n    \n    # register your predictions\n    env.predict(sample_prediction_df)   ","metadata":{"execution":{"iopub.status.busy":"2021-11-03T13:41:23.383901Z","iopub.execute_input":"2021-11-03T13:41:23.384802Z","iopub.status.idle":"2021-11-03T13:41:23.470683Z","shell.execute_reply.started":"2021-11-03T13:41:23.384754Z","shell.execute_reply":"2021-11-03T13:41:23.468611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"ALL DONE!","metadata":{}}]}