{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Import","metadata":{}},{"cell_type":"markdown","source":"I think many people are having problems with the target variable this is where i will explain it nicely with code","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-11-30T15:26:40.048412Z","iopub.execute_input":"2021-11-30T15:26:40.048711Z","iopub.status.idle":"2021-11-30T15:26:40.053472Z","shell.execute_reply.started":"2021-11-30T15:26:40.048674Z","shell.execute_reply":"2021-11-30T15:26:40.052540Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load Data ","metadata":{}},{"cell_type":"code","source":"df=pd.read_csv('../input/g-research-crypto-forecasting/train.csv')\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-30T15:26:40.055301Z","iopub.execute_input":"2021-11-30T15:26:40.055631Z","iopub.status.idle":"2021-11-30T15:27:23.353657Z","shell.execute_reply.started":"2021-11-30T15:26:40.055599Z","shell.execute_reply":"2021-11-30T15:27:23.352851Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"crop the asset id","metadata":{}},{"cell_type":"code","source":"df=df[df['Asset_ID']==0]\ndf['date']=pd.to_datetime(df['timestamp'],unit='s')","metadata":{"execution":{"iopub.status.busy":"2021-11-30T15:27:23.355176Z","iopub.execute_input":"2021-11-30T15:27:23.355649Z","iopub.status.idle":"2021-11-30T15:27:23.657130Z","shell.execute_reply.started":"2021-11-30T15:27:23.355604Z","shell.execute_reply":"2021-11-30T15:27:23.656233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-30T15:27:23.658376Z","iopub.execute_input":"2021-11-30T15:27:23.658619Z","iopub.status.idle":"2021-11-30T15:27:23.676782Z","shell.execute_reply.started":"2021-11-30T15:27:23.658586Z","shell.execute_reply":"2021-11-30T15:27:23.675997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dfnumpy=np.array(df[['Close','Target']])\ndfnumpy[[0,1]]","metadata":{"execution":{"iopub.status.busy":"2021-11-30T15:27:23.679281Z","iopub.execute_input":"2021-11-30T15:27:23.679542Z","iopub.status.idle":"2021-11-30T15:27:23.707507Z","shell.execute_reply.started":"2021-11-30T15:27:23.679515Z","shell.execute_reply":"2021-11-30T15:27:23.706510Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"asset=pd.read_csv('../input/g-research-crypto-forecasting/asset_details.csv')\nasset","metadata":{"execution":{"iopub.status.busy":"2021-11-30T15:27:23.708812Z","iopub.execute_input":"2021-11-30T15:27:23.709061Z","iopub.status.idle":"2021-11-30T15:27:23.727179Z","shell.execute_reply.started":"2021-11-30T15:27:23.709032Z","shell.execute_reply":"2021-11-30T15:27:23.726417Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Lets peel the onion now","metadata":{}},{"cell_type":"markdown","source":"First computation\n$$R^a(t) = log (P^a(t+16)\\ /\\ P^a(t+1))$$\n\nFirst lets understand $P^a$ it means it means that P is the Close price or the price at the end of the minute for the asset id $a$ t is our current minute for which we want to compute the targets now $P^a$ are prices for asset id $a$ so t+16 is the price after 16 minutes from the current timestamp while t+1 is just the next timestamp , so $P^a(t+16)$ is the timestamp after the end of 15 minutes and $P^a(t+1)$ is at the start of the minute taking these two future price values we calculates $R^a$ with given formula\n\nCode is given below","metadata":{}},{"cell_type":"code","source":"t=0\nnum=dfnumpy[t+16,0]\ndenom=dfnumpy[t+1,0]\nr=np.log(num/denom)\nr","metadata":{"execution":{"iopub.status.busy":"2021-11-30T15:27:23.728569Z","iopub.execute_input":"2021-11-30T15:27:23.728770Z","iopub.status.idle":"2021-11-30T15:27:23.735689Z","shell.execute_reply.started":"2021-11-30T15:27:23.728745Z","shell.execute_reply":"2021-11-30T15:27:23.734640Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Second compuatition [can be ignored also],\n\n$$M(t) = \\frac{\\sum_a w^a R^a(t)}{\\sum_a w^a}$$\n\nin this formula we basically take a weighted average of the target values but as we can cancel out $w^a$ we can skip this we should have used this if we had many samples with different weights but here this can be ignored  as here $\\sum_a w^a$ is just $w^a$ with no change for only one sample so M=R","metadata":{}},{"cell_type":"code","source":"M=r","metadata":{"execution":{"iopub.status.busy":"2021-11-30T15:27:23.737400Z","iopub.execute_input":"2021-11-30T15:27:23.738169Z","iopub.status.idle":"2021-11-30T15:27:23.747352Z","shell.execute_reply.started":"2021-11-30T15:27:23.738124Z","shell.execute_reply":"2021-11-30T15:27:23.746612Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Beta calculation \n\n$\\beta$ is just the rolling average of $M^2$ with 3750 minutes so lets say i compute R value not just for one sample but for many samples if we define that by $M$ then the code is simple in pandas\n```df['beta']=(df['M']**2.rolling.mean(3750).mean()```  https://pandas.pydata.org/docs/reference/api/pandas.core.window.rolling.Rolling.mean.html \n\nbut as where we just have R for one sample or one minute we cannot get 3750 minutes so we were not able to compute this here and this will result into close but not exact measurements","metadata":{}},{"cell_type":"code","source":"beta=M**2","metadata":{"execution":{"iopub.status.busy":"2021-11-30T15:27:23.748769Z","iopub.execute_input":"2021-11-30T15:27:23.749577Z","iopub.status.idle":"2021-11-30T15:27:23.760406Z","shell.execute_reply.started":"2021-11-30T15:27:23.749542Z","shell.execute_reply":"2021-11-30T15:27:23.759732Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Target compuation come on this is simple\n$$\\text{Target}^a(t) = R^a(t) - \\beta^a M(t)$$\n\njust plug in the values and compute it","metadata":{}},{"cell_type":"code","source":"Target=r-beta*M\nTarget","metadata":{"execution":{"iopub.status.busy":"2021-11-30T15:28:53.225072Z","iopub.execute_input":"2021-11-30T15:28:53.226363Z","iopub.status.idle":"2021-11-30T15:28:53.233649Z","shell.execute_reply.started":"2021-11-30T15:28:53.226296Z","shell.execute_reply":"2021-11-30T15:28:53.233090Z"},"trusted":true},"execution_count":null,"outputs":[]}]}