{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"This notebook can be considered a tutorial on how to use XGBoost for this competition and use Weights and Biases to make the most out of your XGBoost model. \n\nThis tutorial is based on [[Tutorial] Time Series forecasting with XGBoost](https://www.kaggle.com/robikscube/tutorial-time-series-forecasting-with-xgboost) by [Rob Mulla](https://www.kaggle.com/robikscube). ","metadata":{}},{"cell_type":"markdown","source":"# Setup and Imports","metadata":{}},{"cell_type":"code","source":"import os\nimport json\nimport time\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\nfrom datetime import datetime\nimport matplotlib.pyplot as plt\n\nimport xgboost as xgb\nfrom xgboost import plot_importance, plot_tree\nfrom sklearn.metrics import mean_squared_error, mean_absolute_error","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-11-08T21:32:35.934866Z","iopub.execute_input":"2021-11-08T21:32:35.935775Z","iopub.status.idle":"2021-11-08T21:32:35.941935Z","shell.execute_reply.started":"2021-11-08T21:32:35.935724Z","shell.execute_reply":"2021-11-08T21:32:35.940577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Weights and Biases comes preinstalled with Kaggle environment but it's recommended to get the latest version of the same. ","metadata":{}},{"cell_type":"code","source":"!pip install -q --upgrade wandb","metadata":{"execution":{"iopub.status.busy":"2021-11-08T21:15:55.163266Z","iopub.execute_input":"2021-11-08T21:15:55.163813Z","iopub.status.idle":"2021-11-08T21:16:08.442752Z","shell.execute_reply.started":"2021-11-08T21:15:55.163764Z","shell.execute_reply":"2021-11-08T21:16:08.441722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import wandb\nfrom wandb.xgboost import wandb_callback\n\nwandb.login()","metadata":{"execution":{"iopub.status.busy":"2021-11-08T21:16:08.444723Z","iopub.execute_input":"2021-11-08T21:16:08.444989Z","iopub.status.idle":"2021-11-08T21:22:34.434251Z","shell.execute_reply.started":"2021-11-08T21:16:08.444960Z","shell.execute_reply":"2021-11-08T21:22:34.432704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load Dataset\n\nIf you haven't already check out the [Tutorial to the G-Research Crypto Competition](https://www.kaggle.com/cstein06/tutorial-to-the-g-research-crypto-competition).","metadata":{}},{"cell_type":"code","source":"crypto_df = pd.read_csv('../input/g-research-crypto-forecasting/train.csv')\ncrypto_df.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-08T21:22:37.251841Z","iopub.execute_input":"2021-11-08T21:22:37.252501Z","iopub.status.idle":"2021-11-08T21:23:32.657413Z","shell.execute_reply.started":"2021-11-08T21:22:37.252457Z","shell.execute_reply":"2021-11-08T21:23:32.656439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The following are the columns available in the `train.csv` file.\n\n* `timestamp`: All timestamps are returned as second Unix timestamps (the number of seconds elapsed since 1970-01-01 00:00:00.000 UTC). Timestamps in this dataset are multiple of 60, indicating minute-by-minute data.\n\n* `Asset_ID`: The asset ID corresponding to one of the crytocurrencies (e.g. Asset_ID = 1 for Bitcoin). The mapping from Asset_ID to crypto asset is contained in `asset_details.csv`.\n\n* `Count`: Total number of trades in the time interval (last minute).\n\n* `Open`: Opening price of the time interval (in USD).\n\n* `High`: Highest price reached during time interval (in USD).\n\n* `Low`: Lowest price reached during time interval (in USD).\n\n* `Close`: Closing price of the time interval (in USD).\n\n* `Volume`: Quantity of asset bought or sold, displayed in base currency USD.\n\n* `VWAP`: The average price of the asset over the time interval, weighted by volume. VWAP is an aggregated form of trade data.\n\n* `Target`: Residual log-returns for the asset over a 15 minute horizon.","metadata":{}},{"cell_type":"code","source":"assets = pd.read_csv('../input/g-research-crypto-forecasting/asset_details.csv').sort_values(\"Asset_ID\").reset_index(drop=True)\nassets","metadata":{"execution":{"iopub.status.busy":"2021-11-08T21:23:32.659336Z","iopub.execute_input":"2021-11-08T21:23:32.659661Z","iopub.status.idle":"2021-11-08T21:23:32.685706Z","shell.execute_reply.started":"2021-11-08T21:23:32.659621Z","shell.execute_reply":"2021-11-08T21:23:32.684675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here we will log the raw dataset as W&B Artifacts to build data lineage as we train models and validate on different split of the dataset. \n\nThis might be an additional step but can be really useful to have in your arsenal. \n\nNote: This is a one time step. Once you have logged your raw_data you just need to log different splits of the same or preprocessed data. ","metadata":{}},{"cell_type":"code","source":"# The config below is for demonstration purposes. \nwandb_config = {'competition': 'gresearch', '_wandb_kernel': 'ayut'}\n\nrun = wandb.init(project='gresearch', config=wandb_config, job_type='raw_data')\nraw_data_artifact = wandb.Artifact('raw-data', type='raw-dataset')\nraw_data_artifact.add_file('../input/g-research-crypto-forecasting/train.csv')\nraw_data_artifact.add_file('../input/g-research-crypto-forecasting/asset_details.csv')    \nrun.log_artifact(raw_data_artifact)\nrun.finish()","metadata":{"execution":{"iopub.status.busy":"2021-11-06T20:29:35.376762Z","iopub.execute_input":"2021-11-06T20:29:35.377982Z","iopub.status.idle":"2021-11-06T20:31:11.130596Z","shell.execute_reply.started":"2021-11-06T20:29:35.377929Z","shell.execute_reply":"2021-11-06T20:31:11.129381Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Utils","metadata":{}},{"cell_type":"code","source":"# if you encounter a \"year is out of range\" error the timestamp\n# may be in milliseconds, try `ts /= 1000` in that case\ndef timestamp_to_utc(timestamp: int):\n    return datetime.utcfromtimestamp(timestamp).strftime('%Y-%m-%d %H:%M:%S')\n\ndef utc_to_timestamp(date_str):\n    return np.int32(time.mktime(datetime.strptime(date_str, \"%d/%m/%Y\").timetuple()))","metadata":{"execution":{"iopub.status.busy":"2021-11-08T22:47:43.295915Z","iopub.execute_input":"2021-11-08T22:47:43.296825Z","iopub.status.idle":"2021-11-08T22:47:43.303045Z","shell.execute_reply.started":"2021-11-08T22:47:43.296781Z","shell.execute_reply":"2021-11-08T22:47:43.301697Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Prepare Train-Validation Split\n\nNote that I have used the data that's used for LB score computation as `valid_df`. I will be using this `valid_df` for evaluating all my models. ","metadata":{}},{"cell_type":"code","source":"crypto_df['datetime'] = pd.to_datetime(crypto_df['timestamp'], unit='s')\ntrain_df = crypto_df[crypto_df['datetime'] < '2021-06-13 00:00:00']\nvalid_df = crypto_df[crypto_df['datetime'] >= '2021-06-13 00:00:00']\n\nprint(\"Number of samples in train_df: \", len(train_df))\nprint(\"Number of samples in valid_df: \", len(valid_df))","metadata":{"execution":{"iopub.status.busy":"2021-11-08T21:23:32.695866Z","iopub.execute_input":"2021-11-08T21:23:32.696516Z","iopub.status.idle":"2021-11-08T21:23:35.963491Z","shell.execute_reply.started":"2021-11-08T21:23:32.696472Z","shell.execute_reply":"2021-11-08T21:23:35.962327Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We will again save the splits as W&B Artifact and use the reference to the previously logged raw data to build the data lineage. \n\nThe step below might take some time, since a large `csv` file is being written on the disk. Again this is a one time process. You might want to repeat this for different splits of the raw dataset. For a competition that runs for months a good data version control can make a huge difference.","metadata":{}},{"cell_type":"code","source":"train_df.to_csv('train_df.csv', index=False)\nvalid_df.to_csv('valid_df.csv', index=False)\n\nrun = wandb.init(project='gresearch', config=wandb_config, job_type='data_split')\n# Notice the use of raw_artifact. This will act as reference for this split.\nraw_artifact = run.use_artifact('ayush-thakur/gresearch/raw-data:latest', type='raw-dataset')\n\ntrain_artifact = wandb.Artifact('train-data', type='train-split')\nvalid_artifact = wandb.Artifact('valid-data', type='valid-split')\n\ntrain_artifact.add_file('train_df.csv')\nvalid_artifact.add_file('valid_df.csv')\n\nrun.log_artifact(train_artifact)\nrun.log_artifact(valid_artifact)\n\nrun.finish()","metadata":{"execution":{"iopub.status.busy":"2021-11-06T21:02:07.715789Z","iopub.execute_input":"2021-11-06T21:02:07.716187Z","iopub.status.idle":"2021-11-06T21:03:12.499069Z","shell.execute_reply.started":"2021-11-06T21:02:07.716149Z","shell.execute_reply":"2021-11-06T21:03:12.497898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Features","metadata":{}},{"cell_type":"code","source":"# Features\nfeatues_col = [\"Count\", \"Open\", \"High\", \"Low\", \"Close\", \"Volume\", \"VWAP\"]\n\ndef upper_shadow(df):\n    return df['High'] - np.maximum(df['Close'], df['Open'])\n\ndef lower_shadow(df):\n    return np.minimum(df['Close'], df['Open']) - df['Low']\n\ndef log_return(series, periods=1):\n    return np.log(series).diff(periods=periods)\n\ndef fill_nan_inf(df):\n    # Fill NaN values\n    df = df.fillna(0)\n    # Fill Inf values\n    df = df.replace([np.inf, -np.inf], 0)\n    \n    return df\n\ndef create_features(df, label=False):\n    \"\"\"\n    Create time series features\n    \"\"\"\n    # Build features\n    up_shadow = upper_shadow(df)\n    low_shadow = lower_shadow(df)    \n    five_min_log_return = log_return(df.VWAP, periods=5)\n    abs_one_min_log_return = log_return(df.VWAP,periods=1).abs()    \n    features = df[featues_col]\n\n    # Concat all the features into one dataframe\n    X = pd.concat([features, up_shadow, low_shadow, \n                   five_min_log_return, abs_one_min_log_return], \n                  axis=1)\n    \n    # Rename feature columns\n    X.columns = featues_col+[\"up_shadow\", \"low_shadow\", \"five_min_log_return\", \"abs_one_min_log_return\"]\n    \n    # Fill NaN and Inf\n    X = fill_nan_inf(X)\n    \n    if label:\n        y = df.Target\n        # Fill NaN and Inf\n        y = fill_nan_inf(y)\n        \n        return X, y\n    \n    return X","metadata":{"execution":{"iopub.status.busy":"2021-11-08T21:23:35.964884Z","iopub.execute_input":"2021-11-08T21:23:35.965125Z","iopub.status.idle":"2021-11-08T21:23:35.976507Z","shell.execute_reply.started":"2021-11-08T21:23:35.965094Z","shell.execute_reply":"2021-11-08T21:23:35.975632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train a naive XGBRegressor on one crypto data\n\nIn this section, we will train an XGBRegressor, which is an implementation of the scikit-learn API for XGBoost regression.\n\nWe will take the crypto data of Bitcoin, fill the missing gaps in the series, compute features for train and validation splits. We will then initalize a W&B run and train an XGBRegressor with default parameters. Later in this notebook we will try to find the best parameters. \n\nLet's first start by initializing a W&B run and use the split artifact reference that we logged previously.","metadata":{}},{"cell_type":"code","source":"# Initialize a W&B run\nrun = wandb.init(project='gresearch', config=wandb_config, job_type='subset') \n\n# # Notice the use of splits.\ntrain_artifact = run.use_artifact('ayush-thakur/gresearch/train-data:latest', type='train-split')\nvalid_artifact = run.use_artifact('ayush-thakur/gresearch/valid-data:latest', type='valid-split')","metadata":{"execution":{"iopub.status.busy":"2021-11-06T21:20:21.164129Z","iopub.execute_input":"2021-11-06T21:20:21.164516Z","iopub.status.idle":"2021-11-06T21:20:29.74459Z","shell.execute_reply.started":"2021-11-06T21:20:21.164478Z","shell.execute_reply":"2021-11-06T21:20:29.743466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's prepare the features for just Bitcoin trading data.","metadata":{}},{"cell_type":"code","source":"# Get single crypto trading data\nbtc_train = train_df[train_df.Asset_ID==1]\nbtc_valid = valid_df[valid_df.Asset_ID==1]\n\n# Fill missing value\nbtc_train = btc_train.reindex(range(btc_train.index[0],btc_train.index[-1]+60,60),method='pad')\nbtc_valid = btc_valid.reindex(range(btc_valid.index[0],btc_valid.index[-1]+60,60),method='pad')\n\n# Create features\nX_train, y_train = create_features(btc_train, label=True)\nX_valid, y_valid = create_features(btc_valid, label=True)","metadata":{"execution":{"iopub.status.busy":"2021-11-06T21:20:34.235589Z","iopub.execute_input":"2021-11-06T21:20:34.235977Z","iopub.status.idle":"2021-11-06T21:20:35.124942Z","shell.execute_reply.started":"2021-11-06T21:20:34.235938Z","shell.execute_reply":"2021-11-06T21:20:35.123931Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since we have used the training and validation split and are going to train the model on a subset of the data we should log the subset as W&B artifacts. Here we will log the features dataframe for better sanity check in the future.","metadata":{}},{"cell_type":"code","source":"btc_subset_train = pd.concat([X_train, y_train], axis=1).to_csv('btc_subset_train.csv', index=False)\nbtc_subset_valid = pd.concat([X_valid, y_valid], axis=1).to_csv('btc_subset_train.csv', index=False)\n\nbtc_subset = wandb.Artifact('btc-data', type='subset')\nbtc_subset.add_file('btc_subset_train.csv')\nbtc_subset.add_file('btc_subset_train.csv')\nrun.log_artifact(btc_subset)\n\nrun.finish()","metadata":{"execution":{"iopub.status.busy":"2021-11-06T21:20:38.026122Z","iopub.execute_input":"2021-11-06T21:20:38.02649Z","iopub.status.idle":"2021-11-06T21:20:53.1486Z","shell.execute_reply.started":"2021-11-06T21:20:38.026451Z","shell.execute_reply":"2021-11-06T21:20:53.147733Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now finally let's train a simple regression and use W&B's XGBoost Callback to log the metrics and configs. ","metadata":{}},{"cell_type":"code","source":"# Initialize a W&B run\nrun = wandb.init(project='gresearch', config=wandb_config, job_type='train') \n\n# Initialize an XGBRegressor with some parameters.\nreg = xgb.XGBRegressor(n_estimators=1000)\n\n# Train the regressor. Note the use of wandb_callback\nreg.fit(X_train, y_train,\n        eval_set=[(X_valid, y_valid)],\n        early_stopping_rounds=50,\n        verbose=False,\n        callbacks=[wandb_callback()])","metadata":{"execution":{"iopub.status.busy":"2021-11-06T21:35:23.622573Z","iopub.execute_input":"2021-11-06T21:35:23.622935Z","iopub.status.idle":"2021-11-06T21:36:09.545361Z","shell.execute_reply.started":"2021-11-06T21:35:23.622901Z","shell.execute_reply":"2021-11-06T21:36:09.544291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's save the model along with model configuration. We will use the data lineage created so far and start building model lineage on top of it.","metadata":{}},{"cell_type":"code","source":"# Get the booster\nbstr = reg.get_booster()\n\n# Save the booster to disk\nmodel_name = f'{run.id}_model.json'\nmodel_path = f'./{model_name}'\nbstr.save_model(str(model_path))\n\n# Get the booster's config\nconfig = json.loads(bstr.save_config())\n\nmodel_artifact = wandb.Artifact(name=model_name, type='model', metadata=dict(config))\n# Notice the use of earlier artifact as reference for model artifact\nfeatures_artifact = run.use_artifact('ayush-thakur/gresearch/btc-data:v1', type='subset')\n\nmodel_artifact.add_file(model_path)\nrun.log_artifact(model_artifact)\nrun.finish()","metadata":{"execution":{"iopub.status.busy":"2021-11-06T21:37:23.735857Z","iopub.execute_input":"2021-11-06T21:37:23.736295Z","iopub.status.idle":"2021-11-06T21:37:29.935417Z","shell.execute_reply.started":"2021-11-06T21:37:23.736251Z","shell.execute_reply":"2021-11-06T21:37:29.93437Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# So Far\n\nSo far we built a data and model lineage and used `wandb_callback` for XGBoost. Using `wandb_callback` is like using a single line of code to keep a tab of your experiments.\n\n## [Check out the W&B Dashboard](https://wandb.ai/ayush-thakur/gresearch?workspace=user-ayush-thakur)\n\nThe image below shows the data and model lineage created so far. Imagine you are training tons of models on different splits of the same dataset. Taking the extra effor to build an MLOps pipeline around the same can be really useful in the long run. \n\n![img](https://i.imgur.com/jkVCZRi.png)\n\nThe image shown below is the logged metrics.\n\n![img](https://i.imgur.com/FXQTmts.png)","metadata":{}},{"cell_type":"markdown","source":"# XGBRegressor as Multi Output Regressor\n\nIn this section we will take in the trading data for one quarter (3 months) and try to forecast the returns for the 4th month. \n\nLet's see how things go. \n\nWe will use Sklearn's `MultiOutputRegressor`. Note however that this strategy doesn't use any dependence between different targets. \n\nFirst let's build our train and valid dataset for one quarter. We will use the data from ","metadata":{}},{"cell_type":"code","source":"# select training and test periods\n# 86400 corresponds to one day (24 hrs) in seconds. \n\ntrain_window = [utc_to_timestamp(\"01/01/2021\"), utc_to_timestamp(\"31/01/2021\")]\nvalid_window = [utc_to_timestamp(\"01/02/2021\")-86340, utc_to_timestamp(\"28/02/2021\")]\n\ntrain_window, valid_window","metadata":{"execution":{"iopub.status.busy":"2021-11-08T22:51:14.602086Z","iopub.execute_input":"2021-11-08T22:51:14.603161Z","iopub.status.idle":"2021-11-08T22:51:14.611903Z","shell.execute_reply.started":"2021-11-08T22:51:14.603081Z","shell.execute_reply":"2021-11-08T22:51:14.610824Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get single crypto trading data\nbtc_df = crypto_df[crypto_df.Asset_ID==1].set_index('timestamp')\n\n# Get the windowed data\nbtc_train = btc_df.loc[train_window[0]:train_window[1]]\nbtc_valid = btc_df.loc[valid_window[0]:valid_window[1]]\n\n# Fill missing value\nbtc_train = btc_train.reindex(range(train_window[0], train_window[1]+60,60),method='pad')\nbtc_valid = btc_valid.reindex(range(valid_window[0], valid_window[1]+60,60),method='pad')\n\n# # Create features\nX_train, y_train = create_features(btc_train, label=True)\nX_valid, y_valid = create_features(btc_valid, label=True)","metadata":{"execution":{"iopub.status.busy":"2021-11-08T23:07:44.611054Z","iopub.execute_input":"2021-11-08T23:07:44.611455Z","iopub.status.idle":"2021-11-08T23:07:45.122704Z","shell.execute_reply.started":"2021-11-08T23:07:44.611414Z","shell.execute_reply":"2021-11-08T23:07:45.121772Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_trains, y_trains, X_valids, y_valids = [], [], [], []\n\nfor i in tqdm(range(len(assets))):\n    row = assets.loc[i]\n    # Get single crypto trading data\n    df = crypto_df[crypto_df.Asset_ID==i].set_index('timestamp')\n    \n    # Get the windowed data\n    train = df.loc[train_window[0]:train_window[1]]\n    valid = df.loc[test_window[0]:test_window[1]]\n\n    # Fill missing value\n    train = train.reindex(range(train_window[0], train_window[1]+60,60),method='pad')\n    valid = valid.reindex(range(valid_window[0], valid_window[1]+60,60),method='pad')\n\n    # Create features\n    X_train, y_train = create_features(train, label=True)\n    X_valid, y_valid = create_features(valid, label=True)\n    \n    X_trains.append(X_train); y_trains.append(y_train)\n    X_valids.append(X_valid); y_valids.append(y_valid)","metadata":{"execution":{"iopub.status.busy":"2021-11-08T23:35:32.389365Z","iopub.execute_input":"2021-11-08T23:35:32.390163Z","iopub.status.idle":"2021-11-08T23:35:37.731885Z","shell.execute_reply.started":"2021-11-08T23:35:32.390122Z","shell.execute_reply":"2021-11-08T23:35:37.731020Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_all_train = np.concatenate(X_trains, axis=1)\nX_all_valid = np.concatenate(X_valids, axis=1)\ny_all_train = np.column_stack(y_trains)\ny_all_valid = np.column_stack(y_valids)\n\nX_all_train.shape, y_all_train.shape, X_all_valid.shape, y_all_valid.shape","metadata":{"execution":{"iopub.status.busy":"2021-11-08T23:16:06.660512Z","iopub.execute_input":"2021-11-08T23:16:06.660859Z","iopub.status.idle":"2021-11-08T23:16:06.712933Z","shell.execute_reply.started":"2021-11-08T23:16:06.660829Z","shell.execute_reply":"2021-11-08T23:16:06.711872Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# define the direct multioutput model and fit it\nfrom sklearn.multioutput import MultiOutputRegressor\nmreg = MultiOutputRegressor(xgb.XGBRegressor(n_estimators=1000))\n\nmreg.fit(X_all_train[:10], y_all_train[:10])","metadata":{"execution":{"iopub.status.busy":"2021-11-08T23:33:27.791154Z","iopub.execute_input":"2021-11-08T23:33:27.791919Z","iopub.status.idle":"2021-11-08T23:34:24.446612Z","shell.execute_reply.started":"2021-11-08T23:33:27.791883Z","shell.execute_reply":"2021-11-08T23:34:24.445711Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred_lr_all = mreg.predict(X_all_valid)","metadata":{"execution":{"iopub.status.busy":"2021-11-08T23:35:07.288013Z","iopub.execute_input":"2021-11-08T23:35:07.288360Z","iopub.status.idle":"2021-11-08T23:35:09.131054Z","shell.execute_reply.started":"2021-11-08T23:35:07.288323Z","shell.execute_reply":"2021-11-08T23:35:09.130348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_pred_lr_all","metadata":{"execution":{"iopub.status.busy":"2021-11-08T23:35:27.110115Z","iopub.execute_input":"2021-11-08T23:35:27.110998Z","iopub.status.idle":"2021-11-08T23:35:27.118246Z","shell.execute_reply.started":"2021-11-08T23:35:27.110942Z","shell.execute_reply":"2021-11-08T23:35:27.117334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# WORK IN PROGRESS\n\nI hope you will find it useful. If you have any questions feel free to comment or reach out. Plus if you think it can be improved further please let me know. \n\nUpcoming:\n\n* Extend the regression for the entire dataset.\n* Use a quarter worth of data and forecast for one month ahead. (Not sure how exactly)\n* Show how to use W&B Sweeps for Hyperparameter Optimization. ","metadata":{}}]}