{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**Quick intro**\n\nI wanted to see a notebook / tutorial that would take me through the **basics of working with time series**.  \n\nMost notebooks I saw were either not very rigorous or they took me straight into price prediction using some methods, which is not what I needed right away.  \n\nSo I wrote my own introductory notebook. I acquired most of the information I used here through reading <a href='https://machinelearningmastery.com/introduction-to-time-series-forecasting-with-python/'>Introduction to Time Series Forecasting With Python</a> by Jason Brownlee (took a few days to read an implement).   \n\nWhat this notebook is:  \n- a good starting point for understanding time series data and how it differs from problems with other type of tabular data \n- a cookbook we can use for exploration when starting to work with a new dataset\n\nWhat this notebook is not:\n- it is not about prediction. It stops at exploration and understanding the data.\n- it's not meant for advanced practitioners of asset price prediction - unless you want to revisit some concepts.  \n\n### Contents\n\n[1. Quick overview](#1.-Quick-overview)  \n[2. Dataset description](#2.-Dataset-description)  \n[3. Basic trading data visualization](#3.-Basic-trading-data-visualization)  \n[4. Preprocessing](#4.-Preprocessing)  \n[5. Feature engineering](#5.-Feature-engineering)  \n[6. Typical time series visualizations](#6.-Typical-time-series-visualizations)  \n[7. Power transforms](#7.-Power-transforms)  \n[8. Temporal structure of time series data](#8.-Temporal-structure-of-time-series-data)  \n[9. Model evaluation](#9.-Model-evaluation)","metadata":{}},{"cell_type":"markdown","source":"# 1. Quick overview\n\nThis dataset was made available on Kaggle as part of the <a href='https://www.kaggle.com/c/g-research-crypto-forecasting/overview'>G-Research Crypto forecasting competition</a>. The challenge proposed by G-Research was to predict price returns across a bundle of major cryptocurrencies for which we have approximately 3 years of historical price data.  ","metadata":{}},{"cell_type":"markdown","source":"# 2. Dataset description","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport os \nimport time\n\nimport matplotlib.pyplot as plt\n\nfrom datetime import datetime","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:19:22.718505Z","iopub.execute_input":"2022-01-24T21:19:22.719279Z","iopub.status.idle":"2022-01-24T21:19:22.72934Z","shell.execute_reply.started":"2022-01-24T21:19:22.719185Z","shell.execute_reply":"2022-01-24T21:19:22.728722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_folder = \"../input/g-research-crypto-forecasting/\"\n!ls $data_folder","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:19:22.730466Z","iopub.execute_input":"2022-01-24T21:19:22.730847Z","iopub.status.idle":"2022-01-24T21:19:23.489287Z","shell.execute_reply.started":"2022-01-24T21:19:22.730816Z","shell.execute_reply":"2022-01-24T21:19:23.488061Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Available files (provided by the organizers of the competition):\n- **train.csv**\n- **asset_details.csv**\n- example_sample_submission.csv\n- example_test.csv\n- supplemental_train.csv\n\nOnly the first two seem interesting for now.","metadata":{}},{"cell_type":"code","source":"start = time.time()\n\ncrypto_df = pd.read_csv(os.path.join(data_folder, 'train.csv'))\n\nend = time.time()\nprint(end - start) # just ouf curiosity, see how long it takes to read such a large csv file","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:19:23.490922Z","iopub.execute_input":"2022-01-24T21:19:23.491185Z","iopub.status.idle":"2022-01-24T21:20:28.261278Z","shell.execute_reply.started":"2022-01-24T21:19:23.491155Z","shell.execute_reply":"2022-01-24T21:20:28.26002Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"crypto_df.shape","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:28.264451Z","iopub.execute_input":"2022-01-24T21:20:28.264759Z","iopub.status.idle":"2022-01-24T21:20:28.275348Z","shell.execute_reply.started":"2022-01-24T21:20:28.264722Z","shell.execute_reply":"2022-01-24T21:20:28.27473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are approximately 24 million observations in our time series.","metadata":{}},{"cell_type":"code","source":"crypto_df.head(2)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:28.276286Z","iopub.execute_input":"2022-01-24T21:20:28.277075Z","iopub.status.idle":"2022-01-24T21:20:28.315405Z","shell.execute_reply.started":"2022-01-24T21:20:28.277037Z","shell.execute_reply":"2022-01-24T21:20:28.314634Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Data features**  \n\n- **timestamp** - Unix timestamps (the number of seconds elapsed since 1970-01-01 00:00:00.000 UTC). Timestamps in this dataset are multiple of 60, indicating minute-by-minute data.\n- **Asset_ID** - uniquely identifies the traded coin\n- **Count** - number of trades executed within the respective minute\n- **Open, High, Low, Close** - the usual price details for a given unit of time. \n- **Volume** - amount of units of this coin traded in the particular minute\n- **VWAP** - The average price of the asset over the time interval, weighted by volume. VWAP is an aggregated form of trade data.\n- **Target** - Residual log-returns for the asset over a 15 minute horizon <- we know this from the competition's official description.","metadata":{}},{"cell_type":"code","source":"start = crypto_df.iloc[0].timestamp.astype('datetime64[s]')\nend = crypto_df.iloc[-1].timestamp.astype('datetime64[s]')\n\nprint(f'Data from {start} until {end}')","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:28.31654Z","iopub.execute_input":"2022-01-24T21:20:28.316861Z","iopub.status.idle":"2022-01-24T21:20:28.328836Z","shell.execute_reply.started":"2022-01-24T21:20:28.316827Z","shell.execute_reply":"2022-01-24T21:20:28.327802Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We have approximately 3 years worth of data. This informs the type of time windows we can look at.  \n\nFor example, if we zoom out at an yearly resolution, we can only compare 2018, 2019 and 2020 (2021 is incomplete).","metadata":{}},{"cell_type":"markdown","source":"### Assets \n\nIn the list of transactions, assets are referred to by Asset_ID. Let's look into **asset_details.csv** to see what these 'assets' are.","metadata":{}},{"cell_type":"code","source":"asset_details_df = pd.read_csv(os.path.join(data_folder, 'asset_details.csv'))","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:28.330021Z","iopub.execute_input":"2022-01-24T21:20:28.330385Z","iopub.status.idle":"2022-01-24T21:20:28.349201Z","shell.execute_reply.started":"2022-01-24T21:20:28.330352Z","shell.execute_reply":"2022-01-24T21:20:28.348167Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"asset_details_df","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:28.350444Z","iopub.execute_input":"2022-01-24T21:20:28.351045Z","iopub.status.idle":"2022-01-24T21:20:28.362652Z","shell.execute_reply.started":"2022-01-24T21:20:28.350996Z","shell.execute_reply":"2022-01-24T21:20:28.361649Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"An interesting selection of 14 crypto coins. I say interesting because, as a crypto investor myself, I wonder what IOTA and Dogecoin are doing in there.","metadata":{}},{"cell_type":"markdown","source":"# 3. Basic trading data visualization","metadata":{}},{"cell_type":"markdown","source":"Get a subset of our data (the last 60 entries for example) and restrict the analysis to one asset (which makes it a univariate analysis). ","metadata":{}},{"cell_type":"code","source":"btc_mini_df = crypto_df[crypto_df.Asset_ID == 1].iloc[-60:]","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:28.364032Z","iopub.execute_input":"2022-01-24T21:20:28.364713Z","iopub.status.idle":"2022-01-24T21:20:28.651046Z","shell.execute_reply.started":"2022-01-24T21:20:28.364638Z","shell.execute_reply":"2022-01-24T21:20:28.650194Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"btc_mini_df.head(4)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:28.652445Z","iopub.execute_input":"2022-01-24T21:20:28.652782Z","iopub.status.idle":"2022-01-24T21:20:28.669164Z","shell.execute_reply.started":"2022-01-24T21:20:28.65274Z","shell.execute_reply":"2022-01-24T21:20:28.668243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's transform the timestamp into the index of our dataframe. This will facilitate some operations down the road.","metadata":{}},{"cell_type":"code","source":"btc_mini_df = btc_mini_df.set_index(\"timestamp\")","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:28.670386Z","iopub.execute_input":"2022-01-24T21:20:28.670617Z","iopub.status.idle":"2022-01-24T21:20:28.682309Z","shell.execute_reply.started":"2022-01-24T21:20:28.670591Z","shell.execute_reply":"2022-01-24T21:20:28.681516Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can use the **candlestick** function from the **graph_objects package** of plotly to visualize trading data as the common candlestick plot.","metadata":{}},{"cell_type":"code","source":"import plotly.graph_objects as go\n\nfig = go.Figure(data=[go.Candlestick(x=btc_mini_df.index, \n                                     open=btc_mini_df['Open'], \n                                     high=btc_mini_df['High'], \n                                     low=btc_mini_df['Low'], \n                                     close=btc_mini_df['Close'])])\nfig.show()","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2022-01-24T21:20:28.683937Z","iopub.execute_input":"2022-01-24T21:20:28.684471Z","iopub.status.idle":"2022-01-24T21:20:28.833115Z","shell.execute_reply.started":"2022-01-24T21:20:28.684427Z","shell.execute_reply":"2022-01-24T21:20:28.832232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Candlestick plots look nice and fancy, but later we'll go beyond this into exploring trends, cycles and seasonality of the dataset.  \n\nFor now, it's just a way to get a quick look at our data, so we have a mental model of what we're dealing with.","metadata":{}},{"cell_type":"markdown","source":"# 4. Preprocessing  \n\nDetecting features for which we have missing values is easy to do. We just look for null values per column.   ","metadata":{}},{"cell_type":"code","source":"btc_df = crypto_df[crypto_df.Asset_ID == 1].set_index('timestamp')\n\nbtc_df.info(show_counts =True)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:28.836931Z","iopub.execute_input":"2022-01-24T21:20:28.837191Z","iopub.status.idle":"2022-01-24T21:20:29.171056Z","shell.execute_reply.started":"2022-01-24T21:20:28.837162Z","shell.execute_reply":"2022-01-24T21:20:29.170096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Or, more clearly:","metadata":{}},{"cell_type":"code","source":"btc_df.isna().sum()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:29.173638Z","iopub.execute_input":"2022-01-24T21:20:29.173952Z","iopub.status.idle":"2022-01-24T21:20:29.218719Z","shell.execute_reply.started":"2022-01-24T21:20:29.173915Z","shell.execute_reply":"2022-01-24T21:20:29.217747Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Not a lot of data is missing. We can drop those rows, impute the values etc. I'm not going to use Target, so I won't go into this right now. \n\nThe more covert missing data is when we don't have any information at all for a particular minute. Which means that **whole rows may be missing** from our dataset.  \n\nWe should have one row per minute per asset. Since we extracted the data for a single asset, we expect consecutive rows to have a difference of 60 seconds between their index values.  \n\nFirst step is to look at the **time lag between consecutive entries** in our dataset.","metadata":{}},{"cell_type":"code","source":"# we start with 1 instead of 0 since there's nothing to compare the first entry to\n(btc_df.index[1:]-btc_df.index[:-1]).value_counts().head()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:29.220116Z","iopub.execute_input":"2022-01-24T21:20:29.220325Z","iopub.status.idle":"2022-01-24T21:20:29.24427Z","shell.execute_reply.started":"2022-01-24T21:20:29.220299Z","shell.execute_reply":"2022-01-24T21:20:29.243366Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see that we have 78 instances where two consecutive entries are 120 seconds apart, instead of 60 sec, and so on. \n\nBecause the gaps in data are so small, we can use a simple imputation method: fill in the missing data with the value from the most recent available minute.  \n\nThis is what the *method = 'pad'* parameter of the *reinde* function below does.    ","metadata":{}},{"cell_type":"code","source":"btc_df = btc_df.reindex(range(btc_df.index[0],btc_df.index[-1]+60,60), method='pad')","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:29.245769Z","iopub.execute_input":"2022-01-24T21:20:29.246314Z","iopub.status.idle":"2022-01-24T21:20:29.751861Z","shell.execute_reply.started":"2022-01-24T21:20:29.246266Z","shell.execute_reply":"2022-01-24T21:20:29.751011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Check the output","metadata":{}},{"cell_type":"code","source":"(btc_df.index[1:]-btc_df.index[:-1]).value_counts().head()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:29.753584Z","iopub.execute_input":"2022-01-24T21:20:29.753963Z","iopub.status.idle":"2022-01-24T21:20:29.789417Z","shell.execute_reply.started":"2022-01-24T21:20:29.753922Z","shell.execute_reply":"2022-01-24T21:20:29.788814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Great, no more time gaps in our dataset !  \n\nWe don't need the index as a timestamp anymore. For future analysis it will be easier to have it as a date.","metadata":{}},{"cell_type":"code","source":"btc_df['datetime'] = btc_df.apply(lambda r: np.float64(r.name).astype('datetime64[s]'), axis=1)\n\nbtc_df.set_index('datetime', inplace=True);\n\nbtc_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:29.790528Z","iopub.execute_input":"2022-01-24T21:20:29.791377Z","iopub.status.idle":"2022-01-24T21:20:52.679098Z","shell.execute_reply.started":"2022-01-24T21:20:29.791336Z","shell.execute_reply":"2022-01-24T21:20:52.678171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 5. Feature engineering","metadata":{}},{"cell_type":"markdown","source":"According to the usual taxonomy of Machine Learning, we have supervised, unsupervised and reinforcement learning problems. I'm looking to apply supervised learning methods for this data.  \n\nThe data does not look like the typical supervised learning dataset. In supervised learning data, for each row we expect to see several features and a label / value that we're trying to predict based on those features. \n\nLet's have a look again at our dataset below. Definitely not what we need, yet.","metadata":{}},{"cell_type":"code","source":"btc_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:52.680171Z","iopub.execute_input":"2022-01-24T21:20:52.68038Z","iopub.status.idle":"2022-01-24T21:20:52.696182Z","shell.execute_reply.started":"2022-01-24T21:20:52.680354Z","shell.execute_reply":"2022-01-24T21:20:52.695133Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Rephrasing the dataset into a supervised learning dataset. \n\nIn supervised learning datasets we have entries with a set of features (our **x**) and a label / value for our output variable which we want to predict (our **y**).  \n\nIn the case of time series, our output **y** is the price at time t. But what are the inputs for a possible prediction model ?   \n\nTo phrase it differently, for each entry in our dataset we need to have a set of feaures and a label (used for training and then outputted as a prediction of our model).  \n\nThis means we need to convert our original data:\n\n| time | value |\n| --- | --- |\n| 1514764860 | 13850.176 | \n| 1514764920 | 13828.102 |\n| 1514764980 | 13801.314 |\n| ... | ... |\n\ninto something of this form:\n\n| x | y |\n| --- | --- |\n| x1 | y1 |\n| x2 | y2 |\n| x3 | y3 |","metadata":{}},{"cell_type":"markdown","source":"#### Types of features (x) we can create from time series data:  \n- **date time** features \n- **lag** features  \n- **window** features   ","metadata":{}},{"cell_type":"markdown","source":"### 5.1 Date time features\n\nAgain, our data looks like this:  \n\n| timestamp | value |\n| --- | --- |\n| 1514764860 | 13850.176 | \n| 1514764920 | 13828.102 |\n| 1514764980 | 13801.314 |\n| ... | ... |\n\nWe can transform the timestamp into day, hour, min, this way creating three new features.  \n\nWhat types of questions we can answer with this approach ? We can imagine a problem where we're trying to predict the price for a specific time on a specific day.  \n\nThis makes more sense if our data was temperatures throughout the day, where we know there are daily cycles, yearly cycles etc, rather than stocks, but we're just using this for illustration purposes for now.  ","metadata":{}},{"cell_type":"code","source":"btc_mini_df = btc_df[-7201:].copy(deep=True)  # the 7201 comes from me cheating and looking ahead into the results, trying to get full days of data","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:52.697697Z","iopub.execute_input":"2022-01-24T21:20:52.697943Z","iopub.status.idle":"2022-01-24T21:20:52.709232Z","shell.execute_reply.started":"2022-01-24T21:20:52.697914Z","shell.execute_reply":"2022-01-24T21:20:52.708497Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"btc_mini_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:52.71063Z","iopub.execute_input":"2022-01-24T21:20:52.711153Z","iopub.status.idle":"2022-01-24T21:20:52.737979Z","shell.execute_reply.started":"2022-01-24T21:20:52.711118Z","shell.execute_reply":"2022-01-24T21:20:52.7372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"btc_mini_df['time'] = btc_mini_df.apply(lambda r:r.name, axis=1) # I need to move timestamp back into a column\n\n# and parse it into year, month, day and hour\nbtc_mini_df['year'] = [btc_mini_df.iloc[i].time.year for i in range(len(btc_mini_df))]  \nbtc_mini_df['month'] = [btc_mini_df.iloc[i].time.month for i in range(len(btc_mini_df))]\nbtc_mini_df['day'] = [btc_mini_df.iloc[i].time.day for i in range(len(btc_mini_df))]\nbtc_mini_df['hour'] = [btc_mini_df.iloc[i].time.hour for i in range(len(btc_mini_df))]","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:52.739448Z","iopub.execute_input":"2022-01-24T21:20:52.73998Z","iopub.status.idle":"2022-01-24T21:20:57.895077Z","shell.execute_reply.started":"2022-01-24T21:20:52.739944Z","shell.execute_reply":"2022-01-24T21:20:57.894066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"btc_mini_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:57.896649Z","iopub.execute_input":"2022-01-24T21:20:57.897037Z","iopub.status.idle":"2022-01-24T21:20:57.919602Z","shell.execute_reply.started":"2022-01-24T21:20:57.897004Z","shell.execute_reply":"2022-01-24T21:20:57.91787Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# average data per hour \ntmp = btc_mini_df.groupby(['year', 'month', 'day', 'hour']).mean()\n\n# restore the multilevel index created by groupby into the year, month, day, hour columns that we created earlier\ntmp.reset_index(inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:57.921001Z","iopub.execute_input":"2022-01-24T21:20:57.921362Z","iopub.status.idle":"2022-01-24T21:20:57.93878Z","shell.execute_reply.started":"2022-01-24T21:20:57.921315Z","shell.execute_reply":"2022-01-24T21:20:57.937756Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cols = ['year', 'month', 'day', 'hour', 'Close']\n\ntmp[cols].head(5)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:57.940028Z","iopub.execute_input":"2022-01-24T21:20:57.940608Z","iopub.status.idle":"2022-01-24T21:20:57.953414Z","shell.execute_reply.started":"2022-01-24T21:20:57.940553Z","shell.execute_reply":"2022-01-24T21:20:57.952745Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"And that would be our training data for a supervised learning model.\n\nTrying to predict coin prices based on day and hour will likely be a poor model.  \n\nBut we can enrich these features with additional ones. Here are a few candidates:\n- Weekend or not.\n- Season of the year.\n- Business quarter of the year.  \netc  \n\nI'm not convinced this would help much, so I'll explore further ways to build features.","metadata":{}},{"cell_type":"markdown","source":"### 5.2 Lag features  \n\nThis is the typical way in which time series data is tranformed into a supervised learning problem.  \n\nThe idea is to predict the value at the current timestamp based on the value from the previous timestamp(s).   \n\n**Nomenclature:**  \n- because we are predicting a single variable, this is called **univariate**  \n- the number of previous time steps we use in our prediction is called the **width of the time window** or **the lag**\n- if we predict only one future time step we are doing **one-step forecast**. We can also perform **multi-step forecast** and predict multiple next steps at once \n- this method is also called **sliding window**, with a window width of 1 in our case below.\n\nInitial dataset:  \n\n| time | value |  \n| --- | ---|   \n| t1 | v1  \n| t2 | v2  \n| t3 | v3\n\nbecomes:    \n\n| x | y | \n| --- | --- |     \n| ? | v1      \n| v1 | v2  \n| v2 | v3  \n\nThe actual value for timestamp is gone from our data. We don't care about it, as long as we preserve the order. We only care about the actual value at the previous point in time.\n\nWe use pandas' dataframe.shift() function, which shifts values vertically or horizontally, fills in with NaN values and leaves the index as it is.","metadata":{}},{"cell_type":"code","source":"tmp = btc_mini_df['Close'] # extract only the Close price\n\nlag_df = pd.concat([tmp.shift(1, axis = 0), tmp], axis=1) # downward shift by 1 step \n\n# the original price series becomes the time t value, \n# while the downward shifted series is time t+1\nlag_df.columns = ['Close(t)', 'Close(t+1)'] \n\nlag_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:57.954991Z","iopub.execute_input":"2022-01-24T21:20:57.955509Z","iopub.status.idle":"2022-01-24T21:20:57.969268Z","shell.execute_reply.started":"2022-01-24T21:20:57.955474Z","shell.execute_reply":"2022-01-24T21:20:57.968558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The same transformation as above, but this time we include the 3 previous values, for each future time t+1:","metadata":{}},{"cell_type":"code","source":"lag_df = pd.concat([tmp.shift(3), tmp.shift(2), tmp.shift(1), tmp], axis=1)\n\nlag_df.columns = ['Close(t-2)', 'Close(t-1)', 'Close(t)', 'Close(t+1)'] # rename columns for easier read\n\nlag_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:57.970346Z","iopub.execute_input":"2022-01-24T21:20:57.971097Z","iopub.status.idle":"2022-01-24T21:20:57.992584Z","shell.execute_reply.started":"2022-01-24T21:20:57.971058Z","shell.execute_reply":"2022-01-24T21:20:57.991696Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Finally, the lag does not have to be linear: we don't necessarily have to use the previous k values. We can include in the lag window values from the same day last week or the same hour the day before etc. ","metadata":{}},{"cell_type":"markdown","source":"### 5.3 Window features  \n\nWe saw how we can add a window of a certain width as features to be used by our model to forecast the value at future time t+1.  \n\nBesides using the raw values from the time window (i.e. Close(t-1), Close(t), etc), we can also compute *summary statistics* of these values and use them as features for prediction.  \n\nThe most common aggregate value is the mean of the lag window (also called **moving average** or **rolling mean**).\n\n| time | value |  \n| --- | --- |  \n| t1 | v1  \n| t2 | v2  \n| t3 | v3  \n| t4 | v4\n\n**shifted**  \n\n| time | value | \n| --- | --- |  \n| t1 | ?  \n| t2 | v1  \n| t3 | v2  \n| t4 | v3\n\nshifted, applied **rolling mean** (aka moving average ) with a window size of 2  \n\n| time | value | \n| --- | --- |  \n| t1 | ? |  \n| t2 | ? | \n| t3 | (v1+v2) / 2 | \n| t4 | (v2+v3) / 2 |\n\nFinal dataset\n\n| time | mean | Close(t+1) |  \n| --- | --- | --- |   \nt1&emsp;$\\;$|  ?  |  v1  |  \nt2&emsp;$\\;$|  ?  |  v2  |  \nt3&emsp;$\\;$|  (v1+v2) / 2  |  v3  |  \nt4&emsp;$\\;$|  (v2+v3) / 2  |  v4  |  \n\nThe moving average can be used as a naive prediction model: predict for next day the average of the last *w* days (where *w* is the width of the moving average window). But before doing this, the theory says we are supposed to have *stationary* data (having no obvious upward or downward trend and no seasonality), so we'll need some further preprocessing down the road. ","metadata":{}},{"cell_type":"code","source":"tmp = btc_mini_df['Close'] # extract only the Close price\n\nlag_df = tmp.shift(1) # downward shift by 1\n\nwindow = lag_df.rolling(window=2) # rolling window size of 2\nmeans = window.mean() # compute the means for the rolling windows\n\nnew_df = pd.concat([means, tmp], axis=1) # concatenate the two series vertically\n\nnew_df.columns = ['mean(t-1,t)', 't+1'] # rename columns for easier reading\n\nnew_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:57.993956Z","iopub.execute_input":"2022-01-24T21:20:57.994187Z","iopub.status.idle":"2022-01-24T21:20:58.012363Z","shell.execute_reply.started":"2022-01-24T21:20:57.994159Z","shell.execute_reply":"2022-01-24T21:20:58.011578Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are of course many summary statistics we can use besides the average for the previous time window.    \n\nAlso, the window does not have to have a fixed width. It can also be a forever expanding window.  \n\n| time | value |\n| --- | --- | \n| t1 | 1 | \n| t2 | 2 | \n| t3 | 3 | \n| t4 | 4 | \n| t4 | 5 |\n\n**expanding** rolling window  \n\n| # |values |\n| --- | --- |\n| 0 | 1 2 |\n| 1 | 1 2 3 |\n| 2 | 1 2 3 4 |\n| 4 | 1 2 3 4 5 |\n\nfinal engineered dataset having summay statistics (min, mean and max window values) computed for the **expanding** rolling window\n\n| # | min | mean | max | t+1 |\n| --- | --- | --- | --- | --- |\n| 0 | 1 | 1 | 1 | 2 |\n| 1 | 1 | 1.5 | 2 | 3 |\n| 2 | 1 | 2 | 3 | 4 |\n| 3 | 1 | 2.5 | 4 | 5 |","metadata":{}},{"cell_type":"code","source":"window = tmp.expanding()\n\ndataframe = pd.concat([window.min(), window.mean(), window.max(), tmp.shift(-1)], axis=1)\n\ndataframe.columns = ['min', 'mean', 'max', 't+1']\n\nprint(dataframe.head(5))","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:58.013382Z","iopub.execute_input":"2022-01-24T21:20:58.014319Z","iopub.status.idle":"2022-01-24T21:20:58.029337Z","shell.execute_reply.started":"2022-01-24T21:20:58.014269Z","shell.execute_reply":"2022-01-24T21:20:58.028483Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# have a look at the initial dataset again, to verify that tha dataframe above is correct\ntmp.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:58.030644Z","iopub.execute_input":"2022-01-24T21:20:58.031299Z","iopub.status.idle":"2022-01-24T21:20:58.038938Z","shell.execute_reply.started":"2022-01-24T21:20:58.031253Z","shell.execute_reply":"2022-01-24T21:20:58.038159Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='log_returns'></a>\n#### Log returns feature engineering","metadata":{}},{"cell_type":"markdown","source":"A typical feature for assets price time series is the ratio between the current price and the price at the previous time point. This is usually computed as a log of the ratio in time series modelling, because that facilitates some apperations (additions, etc). In this case, they are called **log returns**.\n\nTo compute the log return, we can simply take the difference between the logarithm of two consecutive prices. ","metadata":{}},{"cell_type":"code","source":"# helper function to compute the log returns\ndef log_returns(series, periods = 1):\n    return np.log(series).diff(periods = periods)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:58.040334Z","iopub.execute_input":"2022-01-24T21:20:58.04079Z","iopub.status.idle":"2022-01-24T21:20:58.051452Z","shell.execute_reply.started":"2022-01-24T21:20:58.040757Z","shell.execute_reply":"2022-01-24T21:20:58.050519Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f = plt.figure(figsize = (15,4))\n\nlret_btc = log_returns(btc_mini_df.Close,1)[1:]\n\nplt.plot(lret_btc)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:58.053141Z","iopub.execute_input":"2022-01-24T21:20:58.053815Z","iopub.status.idle":"2022-01-24T21:20:58.324037Z","shell.execute_reply.started":"2022-01-24T21:20:58.053765Z","shell.execute_reply":"2022-01-24T21:20:58.323137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To sum up, the three methods above are a few basic and common ways (by the book) in which time series are rephrased into a dataset on which we can apply supervised machine learning methods.  \n\nThat's it for feature engineering for now. I'll move on to visualizing the data.","metadata":{}},{"cell_type":"markdown","source":"# 6. Typical time series visualizations  ","metadata":{}},{"cell_type":"markdown","source":"There are 3 types of information to explore in a time series through visualization:\n\n<strong>temporal structure:</strong>  \n&emsp;- line plots  \n&emsp;- lag plots  \n&emsp;- autocorrelation plots  \n<strong>the distribution of observations:</strong>  \n&emsp;- histograms  \n&emsp;- density plots  \n<strong>the change in distribution over time intervals:</strong>  \n&emsp;- box and whisker plots  \n&emsp;- heatmap plots","metadata":{}},{"cell_type":"markdown","source":"### 6.1 Line plots","metadata":{}},{"cell_type":"code","source":"btc_df.head(3)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:58.325502Z","iopub.execute_input":"2022-01-24T21:20:58.325775Z","iopub.status.idle":"2022-01-24T21:20:58.342008Z","shell.execute_reply.started":"2022-01-24T21:20:58.325741Z","shell.execute_reply":"2022-01-24T21:20:58.341101Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#btc_df.plot(x='datetime', y='Close', figsize=(8,5))\nbtc_df.Close.plot(figsize=(20,5))\nplt.title('Evolution of BTC price')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:20:58.343579Z","iopub.execute_input":"2022-01-24T21:20:58.344101Z","iopub.status.idle":"2022-01-24T21:21:09.895619Z","shell.execute_reply.started":"2022-01-24T21:20:58.344054Z","shell.execute_reply":"2022-01-24T21:21:09.894451Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The plot is a bit dense, since it contains all the data we had (almost 3 years worth of data, on a 1-minuted resolution).  \n\nBut it's apparent that we don't see a noticeable pattern (no apperent pattern that repeats itself year after year, for example).  \n\nFor time series, it can be better to look at and compare plots from the same interval, like day-to-day, year-to-year etc.\n\nLet's assume we're naive about crypto prices and we want to investigate whether there is an daily seasonality, so we want to plot it in a more informative way. ","metadata":{}},{"cell_type":"code","source":"print(btc_mini_df.iloc[0].time)\nprint(btc_mini_df.iloc[-2].time)\nprint(btc_mini_df.iloc[-1].time)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:21:09.897158Z","iopub.execute_input":"2022-01-24T21:21:09.898148Z","iopub.status.idle":"2022-01-24T21:21:09.905128Z","shell.execute_reply.started":"2022-01-24T21:21:09.898097Z","shell.execute_reply":"2022-01-24T21:21:09.904423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"groups = btc_mini_df.groupby('day')\n\ndays = pd.DataFrame()\n\nfor name, group in groups:\n    if name == 21: # skip the last day, which seems to be incomplete \n        continue\n    days[name] = group.Close.values\n\ndays.plot(subplots=True, legend=False, figsize=(10,8), title='BTC price evolution throughout the day\\n2021-09-16 to 2021-09-20');","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:21:09.906098Z","iopub.execute_input":"2022-01-24T21:21:09.906837Z","iopub.status.idle":"2022-01-24T21:21:10.685979Z","shell.execute_reply.started":"2022-01-24T21:21:09.906799Z","shell.execute_reply":"2022-01-24T21:21:10.685067Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If we have some domain knowledge, we can say that we're not surprised we don't see any correlation between consecutive days.","metadata":{}},{"cell_type":"markdown","source":"### 6.2 Histograms and density plots  \n\nSome linear time series forecasting methods assume a well-behaved distribution of observations (like a normal distribution).  \n\nBefore doing statistical tests to formally assess the assumption of normality, we can easily visualize our data as a histogram.","metadata":{}},{"cell_type":"code","source":"btc_df.Close.hist(figsize=(8,5))\nplt.title('Histogram of closing prices for BTC')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:21:10.687415Z","iopub.execute_input":"2022-01-24T21:21:10.687937Z","iopub.status.idle":"2022-01-24T21:21:10.953301Z","shell.execute_reply.started":"2022-01-24T21:21:10.6879Z","shell.execute_reply":"2022-01-24T21:21:10.952323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's see it as a density plot too.\n","metadata":{}},{"cell_type":"code","source":"btc_df.Close.plot(kind='kde')\nplt.title('Density plot of closing prices for BTC')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:21:10.954638Z","iopub.execute_input":"2022-01-24T21:21:10.954915Z","iopub.status.idle":"2022-01-24T21:22:06.894688Z","shell.execute_reply.started":"2022-01-24T21:21:10.954882Z","shell.execute_reply":"2022-01-24T21:22:06.893525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see the same information from the line plot and no further insight.  \n","metadata":{}},{"cell_type":"markdown","source":"### 6.3 Box and whisker plots","metadata":{}},{"cell_type":"markdown","source":"We looked at the distribution of values across the whole timeframe (3 years worth of data).  \n\nBut it may be useful to examine the distribution in smaller time windows.  \n\nLet's group our data by year and visualize it as box and whisker plots.","metadata":{}},{"cell_type":"code","source":"print(btc_df.iloc[0].name)\nprint(btc_df.iloc[-2].name)\nprint(btc_df.iloc[-1].name)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:06.896362Z","iopub.execute_input":"2022-01-24T21:22:06.896713Z","iopub.status.idle":"2022-01-24T21:22:06.905051Z","shell.execute_reply.started":"2022-01-24T21:22:06.896648Z","shell.execute_reply":"2022-01-24T21:22:06.904025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"btc_df.groupby(pd.Grouper(freq='A')).size()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:06.911456Z","iopub.execute_input":"2022-01-24T21:22:06.912511Z","iopub.status.idle":"2022-01-24T21:22:06.960469Z","shell.execute_reply.started":"2022-01-24T21:22:06.91247Z","shell.execute_reply":"2022-01-24T21:22:06.959807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We don't have the same number of observations for each year. 2020 has one extra day, for example. 2018 is missing a single data point. And 2021 is quite incomplete. \nJust to proceed fast through this step, I'll use a quick dirty trick and join the data for each year while filling in missing values with NaN. It's not the best thing to do, but for the scope of this analysis it's good enough. ","metadata":{}},{"cell_type":"code","source":"groups = btc_df.groupby(pd.Grouper(freq='A')) # group by year\n\nyears = pd.DataFrame([])\n\nfor name, group in groups: # iterate through the years\n    tmp = group.groupby(pd.Grouper(freq='D')).Close.mean() # compute the daily mean\n    tmp.index = tmp.index.strftime('%m-%d') # transform the index into 'mm-dd' only\n    \n    years = years.join(tmp, rsuffix=name.year, how = \"outer\") # join together yearly series (on the 'mm-dd' index) \n    \nyears.boxplot(figsize=(8,6))\n\nplt.title('Box and whiskers plots for BTC close prices\\n years 2018 to 2020');","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:06.96212Z","iopub.execute_input":"2022-01-24T21:22:06.96245Z","iopub.status.idle":"2022-01-24T21:22:07.285341Z","shell.execute_reply.started":"2022-01-24T21:22:06.962407Z","shell.execute_reply":"2022-01-24T21:22:07.284337Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"[Short recap]   \nA box plot is interpretted like this:\n\n- The middle 50% of the data is contained in the block itself. The upper edge (hinge) of the box indicates the 75th percentile of the data set, and the lower hinge indicates the 25th percentile. \n- The horizontal line inside the box indicates the median value of the data.\n- If the median line within the box is not equidistant from the hinges, then the data is skewed.\n- The small horizontal ends of the vertical lines (the \"whiskers\") indicate the minimum and maximum data values, unless outliers are present in which case the whiskers extend to a maximum of 1.5 times the inter-quartile range (the hight of the box).\n- The points outside the ends of the whiskers are (suspected) outliers.  \n\nInsights from the plot above:  \n- median value for BTC changes slightly in the first 3 years\n- 2018 and 2020 are quite rich in outliers compared to the other 2\n- in 2021 BTC price spiked ","metadata":{}},{"cell_type":"markdown","source":"### 6.4 Heat map plots  \n\nAnother way to look at this data is to plot it as a 2D plot and color-encode price values. ","metadata":{}},{"cell_type":"code","source":"yrs = ['Close', 'Close2019', 'Close2020']\n\nplt.matshow(years[yrs].dropna().T, interpolation=None, aspect='auto')\n\nplt.title('BTC closing daily price\\nyears 2018->2020')\nplt.ylabel('years')\nplt.xlabel('days')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:07.287162Z","iopub.execute_input":"2022-01-24T21:22:07.287492Z","iopub.status.idle":"2022-01-24T21:22:07.638026Z","shell.execute_reply.started":"2022-01-24T21:22:07.287449Z","shell.execute_reply":"2022-01-24T21:22:07.637036Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The problem we see in the plot above is that matshow seems to scale the colors relative to the whole 3 years worth of data.  \nWe've seen in the box plots above that that 2021 has a much higher mean value than all previous years.  \n\nSo this upward going trend from one year to the next obscures the evolution of price throughout the year and makes it less visible.  \n\nTherefore I'll first normalize (from min 0 to max 1) the closing price for each year and replot.","metadata":{}},{"cell_type":"code","source":"norm_years=(years-years.min())/(years.max()-years.min())\n\nplt.matshow(norm_years[yrs].dropna().T, interpolation=None, aspect='auto')\n\nplt.title('BTC closing daily price\\nyears 2018->2020\\normalized per year')\nplt.ylabel('years')\nplt.xlabel('days')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:07.639242Z","iopub.execute_input":"2022-01-24T21:22:07.639458Z","iopub.status.idle":"2022-01-24T21:22:08.0054Z","shell.execute_reply.started":"2022-01-24T21:22:07.639431Z","shell.execute_reply":"2022-01-24T21:22:08.00449Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now it's easier to spot maximum and minimum per year, but we still don't see an pattern across years.","metadata":{}},{"cell_type":"markdown","source":"How about a daily pattern ?  \n\nLet's see a heatmap of price during the day for 10 consecutive days.","metadata":{}},{"cell_type":"code","source":"groups = btc_df.groupby(pd.Grouper(freq='D'))\n\ndays = pd.DataFrame([])\n\nfor i, (name, group) in enumerate(groups):\n    tmp = group.Close\n    tmp.index = tmp.index.strftime('%H:%M')\n    \n    days = days.join(tmp, rsuffix=name.year, how = \"outer\")\n    \n    if i == 9:\n        print('break')\n        break\n    \nplt.matshow(days.T, interpolation=None, aspect='auto')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:08.006883Z","iopub.execute_input":"2022-01-24T21:22:08.007146Z","iopub.status.idle":"2022-01-24T21:22:08.500055Z","shell.execute_reply.started":"2022-01-24T21:22:08.007113Z","shell.execute_reply":"2022-01-24T21:22:08.499063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Still not very informative. This doesn't seem to be the right way to look at assets prices and now we know.","metadata":{}},{"cell_type":"markdown","source":"### 6.5 Lag plots  \n\nTime series data implies a relationship between the value at a time t+1 and values and previous points in time.  \n\nThe step size we take to go back in time is called **lag** (lag of 1, lag 2 etc).  \n\nPandas provides the lag plot method. Let's examine the plot first.","metadata":{}},{"cell_type":"code","source":"from pandas.plotting import lag_plot\n\nlag_plot(btc_df.groupby(pd.Grouper(freq='D')).Close.mean())\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:08.501572Z","iopub.execute_input":"2022-01-24T21:22:08.501868Z","iopub.status.idle":"2022-01-24T21:22:08.729062Z","shell.execute_reply.started":"2022-01-24T21:22:08.501833Z","shell.execute_reply":"2022-01-24T21:22:08.728397Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see a strong positive relation between Close price at t and Close price at t+1.  \n\nLet's experiment with the lag value.","metadata":{}},{"cell_type":"code","source":"daily_df = btc_df.groupby(pd.Grouper(freq='D')).Close.mean()\ndaily_values = pd.DataFrame(daily_df.values)\n\nlags = 8\ncolumns = [daily_values]\n\nfor i in range(1,(lags + 1)):\n    columns.append(daily_values.shift(i)) # downward shift by i positions\n\ndataframe = pd.concat(columns, axis=1)\n\ncol_names = ['t'] + ['t-'+str(l) for l in range(1,(lags + 1))]\n\ndataframe.columns = col_names\n\nplt.figure(figsize=(16,6))\n\nfor i in range(1,(lags + 1)):\n    ax = plt.subplot(240 + i)\n    ax.set_title('t vs t-' + str(i))\n    plt.scatter(x=dataframe['t'].values, y=dataframe['t-'+str(i)].values)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:08.730132Z","iopub.execute_input":"2022-01-24T21:22:08.730548Z","iopub.status.idle":"2022-01-24T21:22:09.663283Z","shell.execute_reply.started":"2022-01-24T21:22:08.730515Z","shell.execute_reply":"2022-01-24T21:22:09.662432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Observations: the price at t correlates quite strongly with the price at previous time points (from t-1 to t-8), for lower price values (approximately half of maximum price). Beyond this value, the correlation becomes weaker the more we go back in time.  \n\nWe know from the first line plot we looked at that Bitcoin price surged in 2021. Also, common knowledge of crypto assets market says that prices get very volatile when there is a lot of hype and more people get into the market. So, when prices are high, we expect a lot of volatility (more abrupt and erratic price changes across days). This is what the plots above tell us too.  \n\nSo, what we conclude: there's a strong positive relationship with prices on previous days, but not so much when the market becomes hyped and volatile.","metadata":{}},{"cell_type":"markdown","source":"### 6.6 Autocorrelation plots\n\nLag plots showed a relationship between current price and price at previous time points.  \n\nWe can also quantify the relationship between the price and the actual lag value.  \n\nFor a lag=1, we can compute the correlation between current time step value and previous time step value. If we have n time steps in our data, we'll have n-1 correlation values. These values can be anywhere inthe interval [-1,1].  \n\n-1 (strongest negative correlation)  \n0 (no relationshop at all)  \n1 (strongest positive correlation)  \n\nThis computation can be done for lag=1 to any lag value we want.\n\nPandas provides the **autocorrelation_plot()** function for this.  \n\nValues outside the dotted lines are statistically significant.  \n\nFor many types of time series data this will look like a sinusoid, with a decreasing amplitude. It's not the case for asset prices (at least not for those with large liquidity  - traded by many people).  ","metadata":{}},{"cell_type":"code","source":"from pandas.plotting import autocorrelation_plot\n\nbtc_days_df = btc_df.groupby(pd.Grouper(freq='D')).Close.mean()\n\nautocorrelation_plot(btc_days_df)\n\nplt.title('Autocorrelation plot\\ndaily resolution')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:09.664743Z","iopub.execute_input":"2022-01-24T21:22:09.664963Z","iopub.status.idle":"2022-01-24T21:22:09.944093Z","shell.execute_reply.started":"2022-01-24T21:22:09.664935Z","shell.execute_reply":"2022-01-24T21:22:09.943016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see a statistically significant correlation with up tp 200-ish previous days average prices. The more we go back in time, the lower the correlation, until it starts to become slightly negative (statistically significant).  \n\nWe know markets have a global evolution (bull / bear market) and local trends (short term excitement for the price of an asset and then slight corrections).\n\nSo, depending on the time resolution our data has, I expect the autocorrelation plot to capture the underlying long or short term trend. ","metadata":{}},{"cell_type":"markdown","source":"# 7. Power transforms ","metadata":{}},{"cell_type":"markdown","source":"Let's see the linear plot again.","metadata":{}},{"cell_type":"code","source":"#btc_df.plot(x='datetime', y='Close', figsize=(8,5))\n\nmonth_sz = 60 * 24 * 30\nyear_sz = 12 * month_sz\n\nbtc_df.Close.plot(figsize=(20,5))\n\n# tail-rolling average transform\nrolling_m = btc_df.Close.rolling(window=month_sz)\nrolling_m_mean = rolling_m.mean()\nrolling_m_mean.plot(color='green')\n\nrolling_y = btc_df.Close.rolling(window=year_sz)\nrolling_y_mean = rolling_y.mean()\nrolling_y_mean.plot(color='red')\n\nplt.legend(['BTC price/min', '30-day rolling avg', '1-year rolling avg'])\nplt.title('Evolution of BTC price')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:09.945454Z","iopub.execute_input":"2022-01-24T21:22:09.945685Z","iopub.status.idle":"2022-01-24T21:22:47.1146Z","shell.execute_reply.started":"2022-01-24T21:22:09.945637Z","shell.execute_reply":"2022-01-24T21:22:47.113646Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='non_stationary'></a>\nWe see that the data is non-stationary: there is an **increasing trend**, yet non-linear (the 1-year rolling average in the plot above, going up at least in the last two years) and a **seasonality** component (local ups and downs).  \n\nFurthermore, data **variance** (difference between peaks and throughs in the local oscillaions) also changes (higher in the last year). \n\nThese make it more difficult to model the data with classical statistical methods.\n\nA common wasy to remove these and to improve the signal to noise ratio are power transforms.  \n\nTwo of the commonly used power transforms are: **square root** and **log transform**. ","metadata":{}},{"cell_type":"markdown","source":"### Square root\n\nSquare root helps bring the data into a linear trend and well-behaved distribution (i.e. Gaussian, uniform) *when* the data is quadratic.  \n\nHere's an example on made up data.","metadata":{}},{"cell_type":"code","source":"quad_data = [i**2 for i in range(1,200)]\nplt.figure(figsize=(15,6))\n\n## quadratic data plots\n# line plot\nplt.subplot(221)\nplt.plot(quad_data)\nplt.title('Made up data (qudratic)')\n\n# histogram\nplt.subplot(223)\nplt.hist(quad_data)\n\n## square root data plots\n# linear plots\nsq_data = np.sqrt(quad_data)\nplt.subplot(222)\nplt.plot(sq_data)\nplt.title('Data after transformation (squared root)')\n\n# histogram\nplt.subplot(224)\nplt.hist(sq_data)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:47.116104Z","iopub.execute_input":"2022-01-24T21:22:47.117002Z","iopub.status.idle":"2022-01-24T21:22:47.743612Z","shell.execute_reply.started":"2022-01-24T21:22:47.116943Z","shell.execute_reply":"2022-01-24T21:22:47.742632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here's the result on our real world data:","metadata":{}},{"cell_type":"code","source":"data = btc_df.Close\nplt.figure(figsize=(15,6))\n\n## quadratic data plots\n# line plot\nplt.subplot(221)\nplt.plot(data)\nplt.title('Real data (BTC closing price)')\n\n# histogram\nplt.subplot(223)\nplt.hist(data)\n\n## square root data plots\n# linear plots\nsq_data = np.sqrt(data)\nplt.subplot(222)\nplt.plot(sq_data)\nplt.title('Data after transformation (squared root)')\n\n# histogram\nplt.subplot(224)\nplt.hist(sq_data)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:47.744883Z","iopub.execute_input":"2022-01-24T21:22:47.745232Z","iopub.status.idle":"2022-01-24T21:22:49.080274Z","shell.execute_reply.started":"2022-01-24T21:22:47.745197Z","shell.execute_reply":"2022-01-24T21:22:49.079377Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The squared root doesn't seem to help because our data is not quadratic.","metadata":{}},{"cell_type":"markdown","source":"### Log transform\n\nLog transforms help when the data has an exponential trend, which is often likened to a hokey stick in asset price popular terminology.  \n\nWe could say we see something like a hockey stick starting in Jan 2021. Let's explore this.","metadata":{}},{"cell_type":"code","source":"data = btc_df.Close\nplt.figure(figsize=(15,6))\n\n## initial data plots\n# line plot\nplt.subplot(221)\nplt.plot(data)\nplt.title('Real data (BTC closing price)')\n\n# histogram\nplt.subplot(223)\nplt.hist(data)\n\n## square root data plots\n# linear plots\nsq_data = np.log(data)\nplt.subplot(222)\nplt.plot(sq_data)\nplt.title('Data after transformation (log transform)')\n\n# histogram\nplt.subplot(224)\nplt.hist(sq_data)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:49.081904Z","iopub.execute_input":"2022-01-24T21:22:49.082895Z","iopub.status.idle":"2022-01-24T21:22:50.521556Z","shell.execute_reply.started":"2022-01-24T21:22:49.082849Z","shell.execute_reply":"2022-01-24T21:22:50.520553Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It doesn't look like the right power transformation for our data.","metadata":{}},{"cell_type":"markdown","source":"### Automatic search for the right power transform\n\nscipy package offers a method to automatically search for the most suitable power transformation for a dataset: <a href='https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.boxcox.html'>boxcox function</a>.\n\n[Short overview]  \nBoxCox procedure identifies the appropriate exponent (lambda = l) to use to transform data into a well-behaved distribution.    \n\nThe returned lambda value indicates the power to which all data should be raised.  \n\nIn order to do this, the Box-Cox power transformation searches from lambda = -5 to lambda = +5 until the best value is found.  \n\nBest lambda means: the one for which the transformed data's standard deviation is the smallest. In this situation, the data has the highest likelihood – but not a guarantee – to be normally distributed.\n\nTable 1: Common Box-Cox Transformations\n\n| lambda | transformed Y |   \n| --- | --- |   \n| -2 | \tY^(-2) = 1/Y^2\n| -1 | \tY^(-1) = 1/Y^1\n| -0.5 | \tY^(-0.5) = 1/(Sqrt(Y))\n| | \tlog(Y)\n| 0.5 | \tY^(0.5) = Sqrt(Y)\n| 1 | \tY^1 = Y\n| 2 | \tY^2","metadata":{}},{"cell_type":"code","source":"# automatically box-cox transform a time series\nfrom scipy.stats import boxcox\n\nplt.figure(figsize=(15,6))\n\n## initial data plots\n# line plot\nplt.subplot(221)\nplt.plot(data)\nplt.title('Real data (BTC closing price)')\n\n# histogram\nplt.subplot(223)\nplt.hist(data)\n\n## transformed data plots\ntemp_df = pd.DataFrame(btc_df.Close)\ntemp_df.columns = ['Close']\n\ntemp_df['Close'], lambda_val = boxcox(temp_df['Close'])\nprint(f'best lambda: {lambda_val}')\n\n# line plot\nplt.subplot(222)\nplt.plot(temp_df['Close'])\n\n# histogram\nplt.subplot(224)\nplt.hist(temp_df['Close'])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:50.523043Z","iopub.execute_input":"2022-01-24T21:22:50.523328Z","iopub.status.idle":"2022-01-24T21:22:53.159482Z","shell.execute_reply.started":"2022-01-24T21:22:50.523294Z","shell.execute_reply":"2022-01-24T21:22:53.158415Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The best power transformation for our data seems to be the reciprocal square root (lambda = -0.5), according to the boxcox method.  \n\nThe distribution we obtain is the closest to a normal distribution, so far. But the trend is not linear. We see what happens with data from previous bull markets (2018 highs become more pronounced after this transformation). Too bad we don't have data from December 2017, when the actual peak of the previous bull market happened for BTC prices. My guess is it would approach the level of July 2021 in a transformed plot.  \n\nI won't explore this further for the moment, as I'm not convinced this is the right transformation we need for our data.","metadata":{}},{"cell_type":"markdown","source":"# 8. Temporal structure of time series data\n\n## 8.1 White noise\n\nIn a [previous section](#log_returns) I computed the log returns for the BTC close price. This plot reminded me of white noise.  \n\nWe can formally investigate if it actually is white noise .  \n\nA time series is white noise if the variables are independent and identically distributed with a mean of zero.  \n\nWhite noise is important in time series forecasting for two reasons:  \n1. **Prediction**: If a time series is white noise, then it's by definition random and cannot be predicted.\n2. **Diagnosis**: The errors of a time series model should be white noise. What does this mean ? That the erros contain no information, as all the information from the time series was harnessed by the model itself. And the opposite ? If the erors are not white noise, the model can be improved further.\n\nHowever, it's generally expected that any real like time series will contains a certain amount of white noise.\n\nA series is *not* white noise if:\n- the mean is non zero\n- the variance changes over time\n- the is a significant autocorrelation (with lagged values)","metadata":{}},{"cell_type":"code","source":"lret_btc = log_returns(btc_df.Close,1)[1:]","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:53.160783Z","iopub.execute_input":"2022-01-24T21:22:53.161077Z","iopub.status.idle":"2022-01-24T21:22:53.184454Z","shell.execute_reply.started":"2022-01-24T21:22:53.161043Z","shell.execute_reply":"2022-01-24T21:22:53.183532Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f = plt.figure(figsize = (15,4))\n\nplt.plot(lret_btc)\nplt.plot(lret_btc.index, [lret_btc.mean()] * len(lret_btc), color='red')\nplt.legend(['BTC log returns', 'mean'])\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:53.185815Z","iopub.execute_input":"2022-01-24T21:22:53.186551Z","iopub.status.idle":"2022-01-24T21:22:57.142102Z","shell.execute_reply.started":"2022-01-24T21:22:53.186505Z","shell.execute_reply":"2022-01-24T21:22:57.141094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(lret_btc.describe())","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:57.145741Z","iopub.execute_input":"2022-01-24T21:22:57.146048Z","iopub.status.idle":"2022-01-24T21:22:57.22776Z","shell.execute_reply.started":"2022-01-24T21:22:57.146011Z","shell.execute_reply":"2022-01-24T21:22:57.226626Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Mean and std are almost 0. Since variance = std^2, it's also close to 0. \n\nSince we have a lot of data, we can split it into shorter time intervals and see if the summary stats change. ","metadata":{}},{"cell_type":"code","source":"groups = lret_btc.groupby(pd.Grouper(freq='A')) # group by year\n\nyears = pd.DataFrame([])\n\nfor name, group in groups: # iterate through the years\n    tmp = group.groupby(pd.Grouper(freq='D')).mean() # compute the daily mean\n    tmp.index = tmp.index.strftime('%m-%d') # transform the index into 'mm-dd' only\n    \n    years = years.join(tmp, rsuffix=name.year, how = \"outer\") # join together yearly series (on the 'mm-dd' index) \n    \nyears.boxplot(figsize=(8,6))\nplt.ylim([-0.0002, 0.0002])\nplt.title('Box and whiskers plots for log returns of BTC close prices\\n years 2018 to 2021');","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:57.229086Z","iopub.execute_input":"2022-01-24T21:22:57.229455Z","iopub.status.idle":"2022-01-24T21:22:57.593699Z","shell.execute_reply.started":"2022-01-24T21:22:57.229419Z","shell.execute_reply":"2022-01-24T21:22:57.592649Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's see the autocorrelation for the log returns now.","metadata":{}},{"cell_type":"code","source":"lret_btc = log_returns(btc_df,1)[1:]\n\ntemp_df = lret_btc.groupby(pd.Grouper(freq='D')).Close.mean()\n\nautocorrelation_plot(temp_df)\n\nplt.title('Autocorrelation plot\\nlog returns\\ndaily resolution')\nplt.ylim([-0.25, 0.25])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:57.594857Z","iopub.execute_input":"2022-01-24T21:22:57.595135Z","iopub.status.idle":"2022-01-24T21:22:58.176707Z","shell.execute_reply.started":"2022-01-24T21:22:57.595105Z","shell.execute_reply":"2022-01-24T21:22:58.17532Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The spikes past the 95% (solid grey line) and 99% (dotted grey line) confidence levels look like a statistical fluke, in this context.  \n\nI conclude that log returns are white noise and cannot be predicted.","metadata":{}},{"cell_type":"markdown","source":"## 8.2 Random walk  \n\nFirst, let's see a perfect random walk dataset and then we'll look at our own data again.  \n\nA time series is constructed through a *random walk* process as follows:  \ny(t) = X(t-1) + rnd_step,  \nwhere rnd_step is randomly selected from {-1, 1} and os is x(0).  \n\nWe construct this dataset below.","metadata":{}},{"cell_type":"code","source":"from random import seed\nfrom random import random\n\nseed(101)\n\nvalues = [-1 if random() < 0.5 else 1] # x(0)\n\nfor i in range(1, 1000):\n    rnd_step = -1 if random() < 0.5 else 1\n    y_t = values[i-1] + rnd_step\n    values.append(y_t)\n\nplt.figure(figsize=(8,7))\n\n# linear plot\nplt.subplot(211)    \nplt.plot(values)\nplt.title('Line plot of a generated random walk time series')\n\n# correlogram\nplt.subplot(212)    \nautocorrelation_plot(values)\nplt.title('Autocorrelation plot for a generated random walk time series')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:58.178804Z","iopub.execute_input":"2022-01-24T21:22:58.180026Z","iopub.status.idle":"2022-01-24T21:22:58.569162Z","shell.execute_reply.started":"2022-01-24T21:22:58.179961Z","shell.execute_reply":"2022-01-24T21:22:58.568391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In section [section 6.6](#6.6-Autocorrelation-plots) I plotted the autocorrelation for BTC close price. Let's see it again.","metadata":{}},{"cell_type":"code","source":"from pandas.plotting import autocorrelation_plot\n\nbtc_days_df = btc_df.groupby(pd.Grouper(freq='D')).Close.mean()\n\nplt.figure(figsize=(8,4))\nautocorrelation_plot(btc_days_df)\n\nplt.title('Autocorrelation plot BTC price\\ndaily resolution')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:58.570253Z","iopub.execute_input":"2022-01-24T21:22:58.571094Z","iopub.status.idle":"2022-01-24T21:22:58.849357Z","shell.execute_reply.started":"2022-01-24T21:22:58.571049Z","shell.execute_reply":"2022-01-24T21:22:58.848374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The autocorrelation plot of the closing price looks exactly like that of a random walk. Random walks are tricky to predict. Still, most assets price data looks like a random walk, from what I read.  \n\nIn [section 7](#non_stationary) I mentioned our closing price time series looking **non-stationary**. It's time to apply formal methods to test this assumption.  \n\nThe *statsmodel* library provides the *adfuller()* method which implements the *Augmented Dickey-Fuller test*.  ","metadata":{}},{"cell_type":"code","source":"# calculate the stationarity of our closing price data\nfrom statsmodels.tsa.stattools import adfuller\n\n# statistical test\nresult = adfuller(btc_days_df)\n\nprint(f'ADF result: {result[0]} p={result[1]:.3f}') \n\nprint('Critical Values:')\nfor key, value in result[4].items():\n    print(f'\\t{key}: {value:.3f}')","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:58.850581Z","iopub.execute_input":"2022-01-24T21:22:58.850821Z","iopub.status.idle":"2022-01-24T21:22:59.194628Z","shell.execute_reply.started":"2022-01-24T21:22:58.850792Z","shell.execute_reply":"2022-01-24T21:22:59.191018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Null hypothesis (H0) of the ADF test: time series is non-stationary.  \nBased on the results, H0 cannot be rejected and the chances of this being a fluke are small.  \n\nThen the next interesting question is **how we can make it stationary**.  \n\nAn easy method is to **subtract the previous value** for each time step t. That would be an obvious thing to do since we know our dataset looks like a random walk and we know that we obtained a random walk by using a linear function of Close(t-1) to predict Close(t).","metadata":{}},{"cell_type":"code","source":"diff_df = btc_days_df.diff()[1:]\n\nplt.figure(figsize=(8,7))\n\n# linear plot\nplt.subplot(211)    \nplt.plot(diff_df)\nplt.title('Line plot of the diff time series')\n\n# correlogram\nplt.subplot(212)    \nautocorrelation_plot(diff_df)\nplt.title('Autocorrelation plot for the diff time series')\nplt.ylim([-0.25, 0.25])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:59.197077Z","iopub.execute_input":"2022-01-24T21:22:59.197508Z","iopub.status.idle":"2022-01-24T21:22:59.698729Z","shell.execute_reply.started":"2022-01-24T21:22:59.197458Z","shell.execute_reply":"2022-01-24T21:22:59.697687Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id='persistence_model'></a>\nSadly, it looks like we get white noise when we remove non-stationarity, which we know contains no structure that we can model and use for prediction.  \n\nA naive model for white noise is:  \n*Close(t) = Close(t-1)*  \nThis is also called the **persistence model**  \nJust because we know the next value is a function of the current value but we have no way to build a model of this relationship.  \n\nThe persistence model will be the baseline for any future model.","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import mean_squared_error\nfrom math import sqrt\n\n# prepare dataset\ntrain_size = int(len(btc_days_df) * 0.5)\ntrain, test = btc_days_df[0:train_size], btc_days_df[train_size:]\n\n# persistence\npreds = []\nprev = train[-1]\n\nfor i in range(len(test)):\n    preds.append(prev)\n    prev = test[i]\n    \nrmse = sqrt(mean_squared_error(test, preds))\nprint(f'Persistence model RMSE: {rmse:.3f}')","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:59.700063Z","iopub.execute_input":"2022-01-24T21:22:59.700321Z","iopub.status.idle":"2022-01-24T21:22:59.895161Z","shell.execute_reply.started":"2022-01-24T21:22:59.700288Z","shell.execute_reply":"2022-01-24T21:22:59.894109Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In conclusion, I found out that this data is likely a random walk time series. I think that discovering this through analysis is more valuable than having read this commonly known fact directly from some article dealing with forecasting security prices over time. ","metadata":{}},{"cell_type":"markdown","source":"## 8.3 Time series decomposition  \n\nA time series is conceptualized as having these types of components:\n1. **systematic components**\n    - **level** = overall average value \n    - **trend** = temporary upward or downward movement \n    - **seasonality** = a short-term cycle that repeats itself   \n2. **non-systematic components**\n    - **random noise**\n    \nThe 4 components are thought to combine in two possible ways into a time series:  \n- additive  \n    *Close(t) = level + trend + seasonality + noise*  \n        \n- multiplicative  \n    *Close(t) = level * trend * seasonality * noise*  \n    \nIn reality, time series data can have:\n- both additive and multiplicative components  \n- both upward and downward trends (especially security price data)  \n- non-repeating and repeating cycles","metadata":{}},{"cell_type":"code","source":"from statsmodels.tsa.seasonal import seasonal_decompose\n\n##whole data\ndata = btc_days_df\ndecomp = seasonal_decompose(data, model='additive')\n\nplt.figure(figsize=(20,12))\n\nplt.subplot(421)\ndata.plot()\nplt.ylabel('initial data')\nplt.title('Additive decomposition\\nWhole dataset (3.5 years)')\n\nplt.subplot(423)\ndecomp.trend.plot()\nplt.ylabel('trend')\n\nplt.subplot(425)\ndecomp.seasonal.plot()\nplt.ylabel('seasonal')\n\nplt.subplot(427)\ndecomp.resid.plot()\nplt.ylabel('residual')\n\n##small window\ndata = btc_days_df\ndecomp = seasonal_decompose(data, model='multiplicative')\n\nplt.subplot(422)\ndata.plot()\nplt.ylabel('initial data')\nplt.title('Multiplicative decomposition\\nWhole dataset (3.5 years)')\n\nplt.subplot(424)\ndecomp.trend.plot()\nplt.ylabel('trend')\n\nplt.subplot(426)\ndecomp.seasonal.plot()\nplt.ylabel('seasonal')\n\nplt.subplot(428)\ndecomp.resid.plot()\nplt.ylabel('residual')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:22:59.896885Z","iopub.execute_input":"2022-01-24T21:22:59.897415Z","iopub.status.idle":"2022-01-24T21:23:02.139862Z","shell.execute_reply.started":"2022-01-24T21:22:59.897368Z","shell.execute_reply":"2022-01-24T21:23:02.138809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from statsmodels.tsa.seasonal import seasonal_decompose\n\n##whole data\ndata = btc_days_df[:160]\ndecomp = seasonal_decompose(data, model='additive')\n\nplt.figure(figsize=(20,12))\n\nplt.subplot(421)\ndata.plot()\nplt.ylabel('initial data')\nplt.title('Additive decomposition\\nSmaller time window (0.5 years)')\n\nplt.subplot(423)\ndecomp.trend.plot()\nplt.ylabel('trend')\n\nplt.subplot(425)\ndecomp.seasonal.plot()\nplt.ylabel('seasonal')\n\nplt.subplot(427)\ndecomp.resid.plot()\nplt.ylabel('resdual')\n\n##small window\ndata = btc_days_df[:160]\ndecomp = seasonal_decompose(data, model='multiplicative')\n\nplt.subplot(422)\ndata.plot()\nplt.ylabel('initial data')\nplt.title('Multiplicative decomposition\\nSmaller time window (0.5 years)')\n\nplt.subplot(424)\ndecomp.trend.plot()\nplt.ylabel('trend')\n\nplt.subplot(426)\ndecomp.seasonal.plot()\nplt.ylabel('seasonal')\n\nplt.subplot(428)\ndecomp.resid.plot()\nplt.ylabel('resdual')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:02.14126Z","iopub.execute_input":"2022-01-24T21:23:02.142406Z","iopub.status.idle":"2022-01-24T21:23:03.807336Z","shell.execute_reply.started":"2022-01-24T21:23:02.142352Z","shell.execute_reply":"2022-01-24T21:23:03.806305Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"What I would like to find out is how to get more insight from these plots.  \n\nFor example:  \n- I see that additive and multiplicative decomposition produced similar plot. How to judge which one is better ?  \n- The seasonal components look similar, but the multiplicate is centered around 1 while the additive seems to have an average somewhere between 0 and 50  \n- The average residuals have a mean of 0 while the multiplicative have a mean of 1\n- Also, the amplitude of residuals is quite different, orders of magnitude different (4000 versus 1) My naive interpretation is that the additive model does not capture the information in the data well enough and it's even more wrong in periods of hype (start of 2018 and 2021).\n\nUnfortunately, all articles, blog posts and book chapters I found so far stop at showing these plots and briefly mentioning what trend, season and remainders mean. As I find out more about what conclusions we can draw, I will update this notebook.","metadata":{}},{"cell_type":"markdown","source":"## 8.4 Removing trends  \n\nTo clarify, a trend is a *systematic change* that does not seem to be periodic. In our case, a bull market or a bear market have upward / downward trends.  \n\nWhy we'd want to look into the trend of the dataset:  \n- can inform **which modelling algorithm** we can use\n- we could **remove** the identified trend and simplify prediction (remove information)\n- trend information can become an **extra feature** for training (add information)\n\nWe can group trends into two large groups:\n- deterministic - they are consistent(e.g. temperatures across years)\n- stochastic - they tend to swtich between up and down  \n\nWe can also call them:  \n- global trends\n- local trends (I guess the bear and the bull market would fit into a local trend)\n\nTime-series that contain a trend are labelled **non-stationary**. We can *detrend* the data and make it stationary.  \n\nThe interesting thing is that detrending \"is common when using statistical methods for time series forecasting, but does not always improve results when using machine learning models.\" (<a href='https://machinelearningmastery.com/introduction-to-time-series-forecasting-with-python/'>Introduction to Time Series Forecasting With Python</a> by Jason Brownlee).  \nSomething to keep in mind.\n\nThere are two main ways to detrend:  \n- differencing\n- model the trend \n\nIn [section 8.2](#detrend_differencing) we applied differencing to remove the trend when we assumed our data is a random walk. We were left with something that looked like white noise, which cannot be modelled.  \n\nLet's try to model the trend using a >1 degree **polynomial function**, since we know from just looking at it that our trend is not linear.  \n\nI will assume it's a cubic (3rd degree) polynomial:  \ny = a * x + b * x^2 + c * x^3 + d\n\nQuick overview of what polynomials look like:\n![image.png](attachment:35a0966e-05df-46a6-8bee-138c1df63616.png)\n\nSource: <a href='https://en.wikiversity.org/wiki/Algebra_II/Polynomial_Functions'>Wikiversity</a>\n\nIf you have to revisit curve fitting procedure (like I had to do), <a href='https://machinelearningmastery.com/curve-fitting-with-python/'>this tutorial is great</a>.  \n\nA summary from the tutorial mentioned above:   \n\"*Curve fitting involves first defining the functional form of the mapping function (also called the basis function or objective function), then searching for the parameters to the function that result in the minimum error.\" (<a href='https://machinelearningmastery.com/curve-fitting-with-python/'>Curve Fitting With Python</a>, by Jason Brownlee)*","metadata":{},"attachments":{"35a0966e-05df-46a6-8bee-138c1df63616.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAZAAAAE3CAYAAACEpheaAAAgAElEQVR4XuxdBXhVx9Zdxd3d3d3d3QoU11KkhVIoFAqluLu0SEtb3KU4tDjF3d01uCQESJD+aw0NL+WHErlJbpKZ9+WR5t5zzpw9e/baNnt/9DcH7LAUsBSwFLAUsBTwJQU+sgDiS4rZr1sKWApYClgKGApYALGMYClgKWApYCngJwpYAPET2exFlgKWApYClgIWQCwPWApYClgKWAr4iQIWQPxENnuRpYClgKWApYAFEMsDlgKWAkFGgRcvXuCx+2N4PvNAhAgREC16dIQLF85h8/Hw8IAnfyJFjozw4cP7/r5MUnV/8gSvXr1CtGjR8NFHH/noHnqvp0+fmneJFCmSj6/z0c3/+ZLmpGcokTZKlCgIEyaMby53yHcdDiAvX76E66NHePLkKSJEjIBYsWL5beEc8np+v4ne4/Hjx4YBIpP5AmJxtPDu7u74+xUZIGoUhA0b1u8TDgFXakO4ubkZWr+9WSUItFm0USRoAmuID7RGmlPUqFEDRBCID55QSEnoRKcADQheCyx6+eY57o/dcejQQezdswf37t1DjOgxkC9/fuTJlxcxY8b0za3e+V3x05HDR3D02BGUKF4CKVOl8tE9td43btxAwgQJuC+jYsP69eSBx6jxcU0fgduzZ89w6OAhHD1yBFmzZTXv5Cie1f64fes24sePh7CUTdu3bYOHxzOULVfe7I3AHg4FEL2cmGHfvn24f+8+BW8kQ7yixYoZhvApevuECLdu3sQjAlXyFCmMgPftEJO4kEmk8SRKlOhfl2sjH9h/APv37kXGzJlRqHAhhy2Oq6srNPfYceIgVsxYWL9+HZ6S4UqVKoXYsWP79jVC1PcF2L8vXkzwiM7NWuPNZpUgOHvmDHZs34HiJUogXfp0gfLeAq1jR49iD3k6Xbp05tnSJh0xxH9Xr1xBfAopAdOuXbsotK6jWrXqiBEjhiMe4dT38PT0xB+rV2PmjJncC7GRPn16XL50GVevXkH9hg2MsJYSIXD1khteR9a8/lvgrvu8reG/okL24sVzc+2qlSuwZPHv+PKrDmYf69DbK14nnvKydLTO+t3rv/fv24/hQ4eibbt2vKYwVq1aQVnjiqbNmhllWPfVczUiRoz4/+h85vRpjBoxEg8fPsQX7dqicJEi/+Kbt99J7/P8+ev56v7e3/e553O8+vuVuV5z3rljJyb//BOaNWuOgnyftX/8QfnxFB/Xqo1o5CMNzU338G5xeT3zXc/xD6M4DEBekAC///47Zs+chVSpUyNturS4QobYzY3RslUrNGzS2BBbRPD08ORihUW4f0xK72cZ9YIa/0JsEvYZF1kauhdRpk2Zgm1bt6H79z2QNm1aQzDdW+gvYntpcV6EM88lYTUHfVeC4aeJk5Anb1582vIzhPFmmt6kgO/Xuw/u37+PZi2ao2TJUoaZvcab+eqaf85hei3a25qG3keA5MXke3bvxpRff0PZ8uVQs2Yt/LFmNdyfPkGF8hUQJ25c8wgxtBcN3t48YijwsY7SaPzDPI68VjS9e/cuunb+htpVfAwaOuTNO2rtTp04ic2bN6NMubLIlCmT2Wz6u2irje/detPfJCTEX961+Xfyh17inzXUWun7XoLk6tWrGDtyNM5fOI/P231h+MC7lvfvM7gfka9g+M+7MPKikXj+5auXRtnRPE6cOIGhAwejbv26qFCpErb9tZXC8yo+qVvHKFte/Oqd570LUPGIPnOku8eR6/mhe70WsiOoNMXB1507I1HiRPRcuGLs6NE4c/oMevXrg7Bci9u3byNHzpxm/8maeMq9kjNXLri4uBjr5eH9h0iYKAF5IrORO7JSj1DzP8t7RI8RHefPnTPXtfjsM8SNF9dYuK5urhS20YxFcvHCBVy8eNGsa7p06ZGAgL5m1SrMnD4dVatXR7v27XHl8iXKH0/kyp3LWIqSHefOngPoMcrM52bNlu2NEivvy9IlSzFvzmx+Pw8aNGqIp7wmFt8zfYb0kAIpyyQVnx2Rckp8EIky6fq1a0bG5cqdGxkzZIAHZdXxY8dwhoqTeDkjeT5xkiRY+vsS3n8JKlWuZED2b4KLeEFz0Nx0zQXyq3gjU8bMyJQls+HJo5xzeO4TF5ebdMm5I2eOHMicOQvCR/CDW8/b4joMQLQQ3b/9FlmzZsNXX3c02rQshPE//EgXzUu079jRvOD+/fvgct0FMWJGR44cuQxRb1y/jtNkqPDhwuPy5ct0fYVHwUKFkZoMIUbRgl29ctUIlDx585jpT5s6hWbiYTSgttKgYUPcofDZt28v7t29hwQ07zJnyWqsk8OHDuMZmerJsyfmuXpeBi7Qhg0bMGfWbKPNfvnVVyRmZnNfacF//rEGP0/62QiqZp82N0wXL148s4gSTjt37EDSpMnIABFx8uQJw4wXzl9AuAjhUKhgIYJnOsPIJ44f54IeNwus56ZMmRLr163DwoULkTtXbrT54nPDNM+ePuN8MxuheOTwYbOBxDTpeI0YyjDd0SPmORcvXDQAUqBgQfMeIcXdoXe/xzX8tktXAyADBg96AyD6TKAuK0Q0fEDNTlachOy1q9eQLEUyFCTdxXMXzp83guXRg0dIkSol8ubNx80by2zQ/bSMXXhdHGq8EjgZMmY0/HX61CkjoK5eu4qiRYshBfnGnXywmhrydCoqyVOmwqctPjX8IQVGAk0WxOFDh4z1qnnI5SFX5BVq0OKLwoWLmPtIsZAAE5/IFy/eSM91+3PNH5g1cyatmuL45JM6xoUpq11CypX8prnq3aLFiMZ9khMZOVf998WLF8xzrl2/RhdGWMNvEpzBDUjWUHH69efJaEzFsnadum9EkvbHuDFj0atPH+zbuwc7d+5E/wEDzDv249+0jr369sX0qVPNXkmbNh2ukxaJEiVGl27fmj3347gfEJ3rKYErAaw1q9egntHeDx04iBTkoeIlS5COr7CR7ikpbh4EiMfcZ7Xr1TGW7sZ165GDazFy1Cj8+MMP5LdbGD5qJP7ashnTpkwjaCU0glluuC+/ao8SJUuad7hOWfbr5MlYuWIF0hOQylWsgO1UdHPyXp9/8YXhtX59+pr3jkuZMojvlixZcgNuR8knKQgsvfv2MTLvt19+Na6q55Q59ynXKhA09PcdvF/eAvlRtWoV0uAoHj56iK6UvX/99RcWL1xk9oGnp4exalq2bm3iP4P69+c+iI3EiRPj2JGjnH8i9B80kM9O5i/PkMMAZPXqVRg3egy6ffedIaYYWhv/0cNHJkgmRNSi79m1xwhTaRbSymXiXSD4jB05CtmIihEZNzlGFC1ZujRatGiBRSTI5k0bKSwz4tatW0hDayNjpoxYsXw5Be1plC1bzmgJs2bOwMGDB4neGQ1DRYoUGW2/bIexY8bgBpkud568ZtM9fPAQ9Rs1ICNeN8ROljQpOnT6+g0DyOqYRbN6zqxZSMLPtNCLJfB5feu2FPiM7bT9/AtUpNYYJ14cjBo+AoUIdrJsDpOh8+XLh67fdTOLNPmnn41mEz58OM79tnHnuZFJ165di3R8j7ac9+ZNm8gcdwm6X+PY8aOYO2OWYSyps2LGJs2bIVzYMBgyaAiFYV4j6A7xPbNmz0ZG6+sQX/GHtMXA+Py/AESuCgH+5Ek/URDUx0laI5s2bkSRokVx0+WG2dxffNkWWag0TPltCu7euWM2kTTHylWr8acK5s6ebSxW8Z746NWLl+je83scpEIzcvhIZM+R3Wyu5i0+M98RH0gLnT9vHv+eBFWqVsVxbt5odC917/GdUXoG9Otv3FrP6S5ZSutblrBcort37TYaYtsvv8RhrtUkWrrxE8Q3G/XiuQuoXK2K4T+BSPacOVD94+q4duUage8QvunalT73dRRUW4zQvHvnrtFuxaMnTxzHz+SplASmOHSB7t69ByVLlcRXVM7i/mO9BsZa+fcZWusldFXOmT0HrT9vY/aS15ALfGD/AaRxD+6nQ5QXuwyfy1oYSCEoun/Xsyc2b9yEmLFiGEVOoLN75y5047qs496SAjZk2DAqqTHwC0FqF0GoYeOG2Lr5L8qA62b9slPW/LV5i1EgJE8uXbqIKb/8xj3ZDgkTJsKwIUPQidZwPgrqEbyXeEzejqGDBiMehfr3vXoZBXnFsuVcp1SMQZR78w4HDxzApAkTUaZ8WSMbdI2e14rvKpk1iHxTn/OJFzce+vftZxRgue2WU6bNmDIVPQkg8+fMNa7NXvxdSogsj8RJEiM2Y8rjKNNatmljvCfjCZZSzGvX+cR4NqTQdPymE564PzFelCQEUbm6Jvw4nnSuSP5uQRqt416aZKx8Kaj+ib06DEAWzJ9PbW0q+g7oj7wUol6asVBQL3ji+AkMJCM0/+xTmu0NjPYweNAganxFkTpNakwaP8FsuPIVKxoz9tKlSwaxV61ciZO8tnrNj9+Y7KXLlDEbdvOmzejH50mDWEMACxc+ApKSYNp8mzdspCDvjrlz5hgNcQQ1CfmcxYTlK1QwG1/PKUANrgVdWN7HJZq0+l4JuiwqUhD0/v57aquZ0KpNK1oWz9BB86xQkc+NTdAci+979zJ+Ti3gju3b0adfXwNwskoGkRFlJm7kfKJFi2qYYhrpVLlKZQqOjw2jurq6oT4F45Rff2XSQWx07NzJBFN7f9/TWB+ly5YmQ05Cl65d+HtZA3DyH/8wcYLRckPC+BCAbCLQ/jr5F9Slhnjs6DHjQhhADUqa47DBg5Ejl9wc0bGKml+VatWotefADAKAgrO9uR7nzp4xboGUyVOamIbcAOLVG7Q6RtNN1a79l6hXv77R1rzGObo/pBRlzpLFbNAx5CGtnzRdl+s3COqDTXxPALKM/Pj1N9+gFBWfgRQQ8uVLi5YS8YRul77kJykZK8kXCswKbPr26o1GTRujTJlytKinGtdGtepV6f6Ya9a59eefG2Vh5LARqF23ttGkf6Jg+rxtW1TlO44YNty4QAbSWpNWHZyGhL5cyHXq1TUCVFq0LPHtW7eSFtPMHtrHGKRc4L379TMunwH8V5be9wQQuYJv3HAxlvqp06cILDfQqcs3FOjL6PoKayxYKbGLFy0izVcwRlDTeA4UX+pEl1lkKnYCesmhl9Twb9+5bQClQ6eOdGMlJM2H8/dOBIBCGDF8GBWVm8az0p9CuQIFcYuWLQ25ZYV4ZUF50f/A/v2YSHlWumwZE0NRPEVWpMDyFC0QWQMNGjci6MczCmhHKgfFihc3zx8zciTa87+nUxGSEi0vhWSpvBiSpVJI5NVp2aY18lMhlbUlb4eeNZsWrWSW5Jm+O4TyVYqQlIy5s+ca2VuxYiVDhxFDhqEn3YRSeJ0CQDau32AI1aHz1yjPjIDwdDfJ3SOinOXmlZCfSpOsH18qP1H9Gq0CIbNMt3zMupg3Zx46U0DmJiJOmjjRWCFaaBFC5uCli5cMIeXqqfXJJ8ZPuZUmm8xZuTz+/PMP46IKS239wvmL3FjH0LV7dyxZtNi4nyQszp09i6EU2AUKFDAWx2gKhAIFCnIxWtEr9L/0PPlNZVoWY+ZGpcqV0bdPL7oQBCCtzWJ91e5LVOBCCECktYwaO8ZsYAWAly9dxud2w/y5c+nbjMSN0M8AiBejSTv57ZdfCEyVUaNmTcOocpFV5n9P+e03CqSiaNy0qbE0xtN01jvWIHjOmTkbg6kJSVuSkJw5YwbG8HO5+ULC8A4g8kMPHDL4jVtGFsgbAGGM4OTJUwxQPjC0fUR3lgRpkmRJjTCRmyInXUxa84fUEHXfxk2b4NTJk6/djLSE5araQ+HRd+AAY8HMoEti9A/jjEvL+zhLfhFoyFdc65Pa+HHsOESlEtC1WzdjQWiDauM/f+5prJsu33Y17i1dI1dDZ1oTo+nnFwDJTSqL+xnTVeVeuEpXbT/ybsMmTcx+kXUujbs4+XLVipWMy7UgT1QxAlPPEVBFjRIZ8+cvQPt/AsLScvfs3mXAyacZRs7CK3I1jqawlD9Wgllu5rV//mksCVng31HbX0/30h+r1/D9+pn91YN01z6q16ABlYYh5t9yjCXKOpU86PYdr6EFIo/GkGFDqQxEMiC1d/deNGrSyCQqvI65dDLWQ7/evclj4c2+lqAdM2o0WrT6jNlXCTGU92/PNStavBgt1OHGAtG6S7FMSrdPj149cYdeBSmKcm2XZWzuXQCidRsycJBZn686djDvN4ausFYEBlkgCrZ7AYi8EWPJO9/wOfPoXo9NmSl3lhsVTCk8KemSjUkAHEulRvOUm1QAIgW9JgFS1rcUStHzKT0lcvlpHxQqWhjT+Fnzz1qgAhVfKbkjhg4nMPcxyr5TAIjcLT25gEJ4uaXk69WGHUxBHJ1/q1m7lnFTNeVL1KemJ7/wMGoJBYnQskAWzBWAdDUBskkTJhjNqhX9dxKuHtx08tlJO1Ggvis1wMu0UETwPlxQ+au7d/0WH1MgV65W1Qhd3U+CfBHdTwkIMNJivANISWY9jST65yV4SaPzniHmHUCq8n7a6DFjxMQ3FBASHF2padapW5eLEwdTf52KEWNHm4WTq2sFtR1ZJMuXLjVW1BCCqtxRMnXl8hLTyFIpT99obQKh5iArQ9rvVAKI/PUdqflEpzbdh/eRgCxN5pw1bSYBZIgRcitWLMdsurrGUOiFNADp+k0XA57S8iMoyYK0U+xLmvgvskAIINLilOEi14aARJtBG1Tuwo0UJnJz5cmTB3vpQ3/i/hTpM2YgUA8z8Q0FqXUvuYKk5V5n5tMc0lIAkiZNmvcCSB1eN5a+eSlFPSk8FLMbNmSoAffntGy0KeV+kqtCiomCmT1ouf5EZUgp7fI3i8fmUbGQOysTFRJt8Hr16xk3m6zS/1kg8/6xQNoYt9Yovl8tWiDSqBctWGjAqCA1Y+0TaeLBEUBEx810Q86moJRlpZ89e3YbTV8KVGdaE6eoKIyk9p+a65IiRUpsUryCCudnTMr5gWuhWJLiHCbWePyY8WAkZixk8s8/G4tFiuXOnTtMjoSC2Tu5RlJktU6KYY2m8JYLS2v28MEDrFu31ngnatAzIGU4bjzG4rhuP4wda+Y1YvQo4xFZxiB59uzZ8YCgc/v2LXzO5xanIuE1ZDn9OG6cWcNalHsTfny9TvmpuMqFvYXJIFJu48SNg+G0njtTdpVgTEa8K3kwmK4leUHkvldwXPLtMt2xbSinEiVMaGIoejfx3i4qTJIRUr6VbizLTnSRoquYmZQnyWQBbqvPW1MhrkJlZysGDxzId5MyX8A5AESanl5g5vQZhtHlT1Za4vPnL4zppqDlr79MNn7kTFmzckFcTDCwZetWxv8oC6QbXU65ufHHk/jHGHz+vO0XzHPeboBDi+z22I2BxntklPa4ecuFTDTOmGDKlviBfkGlRCZLntwQX5q+tMbz1HQSkugilgJqMh8VgJYVM4JMIstGwCBN8o3rgppn3159UIKmn3yTyixbSdM4T36iNc3jNXQfmayO+HGNj/UHbmQFd+Uvl69y9LixJgNNFkWC+AlMtoYLAbYhFzND+gxGwCg21IGawgZaboqByMctc3oRTW6vwNZ5umnqUMBEoAUzftyPGDFmtAnsSxuRr3Q80/neFnrOomH6dh5eWVjt27YzWTHaoB+F+cjQW1q9tLyJpHMDgoOUiwf3HxgrRZtYrlC5hEqXKYsF8+YaLV9ZPXJzaYMUKFQQE2j2C4gkjKTsSJhIqEgxkUtwwk+TzD28D/HLiKHDjAXRhjw8m3GU5RQeUnIEbFvpKpUr4xnz8Lfx92702+uz4QSrY3RHDaX1oViYMgYVHJXQVIaMApvZuAd69+xlsmjkxrlIq/l1DKSLEQL79uxjEkU65vzfMlq6QOPUqRN8v/kma0kuUwmpXXRHDCIfK14S3IYHrYkjlAe7GaNQTDQOrQOBrActtPoNGCMgWGzYsN64vyUwpVzJ9aS4ggSyrLzwdFsno/V5h3Ev0aA4vQY7dmw35zAiMZkhfsIExh2cNk1a4+aSVSKeCENLVGu0g/STUFfCzROCijL3FJORkiGFs2nz5mZfujOmUI6KnJJsBABSYgR6imvKTeU9K1IJH0oWEuhlz5bduNik7MqSSJWGfECZKM0/Mi1SgWa+fPmN3LpCq3QvwUfKrRIxpAjLfS/Pi+SiZJRcWfq73PECl7BhwxnrTZbSYyrb27dvMzFCWRWKb8gFp4OQkqNy66ZJm8Z4f3bt2olixYobOe2f4xUOi4GIebVBRPiDzHRQoFIIK+LILaWMEQXAtm7bahZSGn0+CmQRQdbE2XNnTcaMGEXZLfeYkSLrQETZxsMyysKSCyB3ntwmeCSzTRkRro/cCCDVSLQTRoP7iMROnjwZg4/3TJBNMQVpplpkaa0K0glQMjHrSiCjLC0tpp7lNeSPlYsjSZKk/F4mo6XoWXdv3zUBM72nYiJixpPUfuR2EJMqS0jms/yQYSj8lElzhPd/QddKNga9pTUKXPft3Wfmq7x0Bcikoeq95EbbTZeENCrlsuvZZt7Ujg4zs6gIF1wHM/WM06dOmuc44sCVswgerekWCuIbtPJEYw2BiMA5KwXuEWaiKbNF7gZtJNFGLg3RWVpWNvLSWbofJVykaCQlH5QsUdJonYYv+fOSwfOUKVO8PrjGcziZSeNz5D2toc7leB/iF/mzFZDXZrzGNNstf23BPQa2k1M40FtK6yaT8aHL1SQ+kstA1+j+CvKH+SiM2ayySOSK03sUoVtDQkH+fYGcgvZaxzu376Bw0SJGudi+gzx/+YqxxiSkpEDJopV1rLkoWKqkjVvkTdEhhgMO3gUVH0h58DqLIRpJc07wj+A3coVubB2ae1vQKbbpFWvV77qPlztGv+vnQ1mK3lOjvVLDdc2HhKrXd33q/tH8pHR4Py7gE3r79jlecljz9+ncfDKP933HoQDi/SHa2F5nLrz/3S8E0fVeOfr/RZS3D+ho0RxJRO8M61Oi6xrvh5Z8cp02kU+Y2Cf3Cq3feRcNHcEfRhBwfEgwvU13nwq0t/eKT4RgaF1j+95BT4EAA5CgfzU7A0sBSwFLAUuBgKSABZCApK69t6WApYClQAimgAWQELy49tUsBSwFLAUCkgIWQAKSuvbegUqBK8xMucRSL6YuGQOhcZnEoawrBdg/FBQN1InyYUqzVNkdpfS+qyBfYM8nJD5PSRlKOrjJMxyqUyY66wxHUlaYcCaaK86lE/I6nJqIJVKUkeXbGFtQrV+wABCl++nw2K1bN01Ot9Jaleqqw0WOqo7qiAV49fIVbjEvXFV+NTdl5NgReBRYMH+eOcUdhqm/Agx3pn0nYFmK5qxjpYwoZxAaEhYPmFWnCrFKI/2a5x2UZWaH4ymgdO2fWf5GpUyUEaqihkqhVcpro8aNnSL1WRmaK5Yvw9LFS0zW2AseSq3FM0cfs1BicCiYGiwARFVadWpVJdaV7ijtTVleOgOi+jI6NBTUiK00ROWsT+epZjGC5tmUday8iqw5fnvYO3qngMp0/8IDZCpb0fzTZjw7ooqtN015h2csd/0dD/XpLImARaeQdfgqAdO5vSsgr62Cu6aeVWTWUnv5kpV+meuvDEBl8ymlXJtaKbOPmOr9mIX0VJtIpUm8htKPlcIehSVRYvIz73yp++iw2s8Tf2LtqyvmrMqYH380RTHtcDwFVB5eBwJFd9XFi0ALRGdOlvFMl2roSbHQPlXdKKVdx4oV06y9l7XqtZaqlqu/68Co1lolV0z1b57h0bkQHQsQ7+hMkj5Xqr3XUNae7i3lUiD2dlMrpeT3YWkWnR9SPS2v0k1DRgzncYTkTmc5v71KwQJAtCF7skhjlChRTXVcLdD+A/sxc9oMU830C9btl5tCi6jv6nPv/UfEQDpopEXXZ8orV7FFT6L961r8nubwmfL9xTQqxieG0Xe955nf4WEnfa4Di961AwGG3CcDWRgtNQ+jqX7WMh72c2N5ahVGC06F7hy/jQPnjgIQVXdVMT1VKtBhMa2L6qWNYFkLleyvVKky1rHkze+LfufaPzfrq0OuOqSlw4k62a0Tx+ID1bxKniK5qfKrEjU62CXgUd0m9V9YOG8BLYn7pgqsFAUd7FNFatXrUpmOlxQcKrPRsFFjcwBOQ6VWdJZDvLtty1a6V86bagI6B2KH4ykgAFEZEtWgU/FD7Wmt4TiWHrp0gQeIeWhT/z2Dh591YFNAobYT5cqXN2eNJo4fbw4xShkQSKgqrkoiTf7pJ9N6QKfZdVasRo0arJIw2RzQi8kCjlV4Lq1WzdqmbLwqfusgtPhNZ72a8WCi3Kpe48A/pY2af/qpOd+zbNlSTP91Ckby9LsqMOsclDOPYAQgPVgdNwnad+hgDhvKAlHZZ1O9k+UAVNJCp8J1+CsMT3ipNkxtlsmWVqCCdjpcFjdOXISPFMEcYlRJCx1Y1ClTgUJqnlTNmTsnBcxa09hGtfNrflLL1NyXFqFChyrzLDDShm/LelgqwSKA0fU66WnKErCscjEeFBMjzGc5lZ4sRyIBZUfAUsA7gKhURUGePteQUO/SqbMpNpeNp9tVP6oaq9/mooWykHWlpCx0YG2kGdOmGSGiyrY6SawiihlZAqUMQWBQ/4HGkmzIAnjSJnWdTseXKl3KVE1Vt7oWPF2+lG4pHWBs0KgRXZk3Tdlv1Vkrx1pXXgrH666LrqwePctUNBjD+loWQAKGN7wARMrA9yw/o5JKGlNZGeCPNWtM6aO9PDCciKexJfRVWmXvnr3o9E1nUxpdgl+VvnVIWLJGB3p/YRuJ3rQYzpw6zZIqLVkJIi1WsrzJK7asUMMp1brSwWcpFSrQuJ7yRIU4Y1DmTGU9Kh2CbtKs6ZsDwJJjKtekA6M6ja/6gAIy1fhTxWVvJfoChkj+vGuwAhCVM9YGF4BoLF64gHVvfjHVa7V5k/HksWpUCdXlY27f4SvTpOd3FlT8isCjMswqribfqD7bQqkE6CAAACAASURBVO1UlXtVWiJL1iymTIROrddj+RJ1I1zH4m6qPyPr4o9Va4z1I+tDJSRU1FBz0fcVtP2TncHkQhGY6XS9SheownA79goow5o4dgQsBd4HIKqo+y3Lcqtpk7rULSYvKB4Sm1aBytzoJLhqKM1j1WZVLVXJEFUeGMn6U69evTANrH5j7TJ9pwIrRc9lo6BZtHxVyUD1t1T0Uxu/GQXGMhbS/JuuimwsDS+lQqV9VEpeQkhCzGsouKtKwSoSagEk4PjiXQCiHiAqMbRpwyZTmHU/qwbIlZmBSqGsz927d7KEUjtsYD+QJJQ3XVjYUG5OlUPfRlCZxHJMqgqs+6g80nnW+xtAYS/Zooq7ki2qJCF+k8tzB8Gk0D8dCVWDSuVWVDrJexVtHXpVnEbWq4p1qoij6lk58hB0QFE5WAPI3LlzMHPqdCPwZ06bbgoNaoHUFWw3LY7qNaqb8hD37t/DMNYl0oLMYhXbdaw11IalsgUeslBUaFHlVFTITOW8VYdJmqTu0ZjVUhXslLCR9iDAUMte9Wj4keXU5ToTgKiS6M80bQewSJnKTqh8u1wiqnelEt92BCwFvABE6yDXhMp7yGX0J/9b2mNb9gu5fvW6KaeuIKUKbMqtwHaE7AmT3PRYUBmRDuzLoqZFqsYqK1Sl9KVYdOrSxZQqUd+Z3xcuNu0A0mdMb8rQqKKrsmcmMp6RiC4NWT8qBCkXR1qWXlETNO++bwsgAcsLXnf3AhDt2e96fW88D2raNYZtHO7dvWP6x6h0jpp4qU6UQF8N3tRl8Cf2y1DjMdU3U/KFKuDupTwQgKhSt/a9innKizF4wECjNKrvhuIkcl/K5aUK5SpvpAKvMdkd0Y0xM1X6VY8RrxJEAg+5XadSSclfsIApqqoCkSp55GyZg+9atWAFIAo6SkNUuW9tTvXSuEA/sorRKduiFPuEZKUlYWrrcCHln9Ti3GAjqeEsYCh3k8pmqxiaKvCqU5h6Ocg/qsJsKucs/2QRChIJH/VxyMi2kHPZXOom3RuV2MPjddDN3ZjDCnrJNSHGk+mq/hCdunQ2Gu4yVuNVNc1efXqb8uJ2BCwFDIDQD61y+nIZqIf52TNn2aRnGeKzqmoPNo+6xhpbI+gTV2nvIiyFLRfSGRbFa0llYg5dSnJ3tWEBT2X7TRo/0dRwK1WmFBayAm4nVgkWEPy15S/znGKMvVWuUgWrGLRXCqYsX5Xw92B85DNatAqwi9eqsbd7efrUvcfMLIAELC+8ARAWJ1QxTMkKuZSk6MkK0F5txFiHWs6OJ+gr1qCqyAcYm1Jx089YIFPualmQLemmUmBcFohqnk2e8psBEMXPVAVZ3gn19IgWPRq+oExR59DNGzebOKhqeskToWC9Gjup0KyyM1Wg1SvQrhhLT4KU+hhVrVHN1O4TYKVjj3i5zpx9BBsA6dXje0PLGrU+fl10kL5LWRBNWPpZ/u3B7GIml0IjWgxnzp42VXEbNW5C98JNkxnVmH5HaZ2/MpahLAo1EBKAvGCmjTqcPWR1V/VoFrPIVaUGRGtWraZ109D0V1639k/T4Cc520/K/SD/5BfM7JB2Y/K4qbXKlFWOeTWWg15IrVX9y3v27W2CsnYEPAVmz5rJWNUUA/DS+BW8zsXWwRIWcgkoU05xKTUjU8KFNnGL1i3Z0Kmsqa6qILwKJuqsgDTDKJGjoFyl8gaUFPPKwe6BKq0tUFq0YJEJtCpVu3mL5gy8VjB8ogCr6WNNLVRWSmv2fVDhQ+/apJ6tys1ykQ6g0qEKqXY4ngIS4HI3q0dIJO5TCWZ1Ga1Ct6LAX1aEFExV21bqvT5X9etP6tQxLRbkUlIDJ62xCnKq6OYkKg+yViXcv2VJdqM8sijmr+zxo+KXUZnoI4ujMRuFPXZ7TCVimpFTSthRu28lbchD4cUPy5kR9gPjYHpeRCqj+rsqRKuMvDwqQZ1d+qFVCRYAokCnhPsWNqeSaadmVUL0ilUqoVq16mYxd1JjkGYo/3UElnhWUEzgogCVek+rC5diJ9EIELdpvkrLUNXUF6zOqniIglgKhCuYev78OUTkPaQ9KutGwuA3As9fbImp0t3SbtUHOQsrq3r5KWW6yo+pIK2sFcVIWv/TdvJDi2A/dwwFlOCgH+8VVrU+AhPvHTKlkaribczYTNtkn3lds3HjBuP6LM7qvQqmK/FCvVbUJlWBTvGUkjM0JDSUvukV/BT/iS/FJwIpAYueq+q+yt56lytCz9R9JLScXUg4ZnUC/y5aDy866+laB/2IH7Q++l2Kgs6HPGCr68hs2CVQ0eeyVHQI0ev8kHqMqC/5uAnjDX/pWi+rUveQVSkQkNtTvce9XJaSC0rsUQaXvBdSOL2vt3hAlpEXz2qeqtgbkXGX4MAXwQJAzAEbEtpUQpWvgpltXuWKvQS41+bVIS0FvcQIuk6uKWVgqTOYNuvCBQvoqrhoOn1JG1EwTLn+Gl73kHBRF0EvZtJ9xCRiEDW3isl8cT3j7QXW9dIu9aOGULpHcGCCwN/aQf9E75V5taar2RJZPdDVH8GNmqM6W8rqUC+R9w3v9/D+He8AFvRvamfgEwq8vZZSSCew42dUxi50huOulE66s6qw+df7gtvvW/eQzA/BAkB8wgDv+o6EvvowKIiqxj1qPqMDZLUZRJXP2gp3v1I25F1nDo0xfVdZWap7ocZd6kkRHAKZIW81gv6NpAxKGVW6rvsTd+N1UNOp4JAZFZjUC9EAIkIK/R+5PoK6+8mCkFtCgSwLHoHJZsHnWSFZWww+q+A8M7X88N9rEeIBxHlY0c7EUsBSwFIgZFHAAkjIWk/7NpYClgKWAoFGAQsggUZq+yBLAUsBS4GQRQELICFrPe3bWApYClgKBBoFLIAEGqntgywFLAUsBUIWBSyAhKz1tG9jKWApYCkQaBSwABJopLYPshSwFLAUCFkUsAASstbTvo2lgKWApUCgUcACSKCR2j7IUsBSwFIgZFHAAkjIWk/7NpYClgKWAoFGgVAPIC9Zzv0li6WpmqotbxJofBfsHqSqrq9YFkclt+2wFPAqvKrGcqF5hHoAuc8+IHdYLl4d5by6hIVmhrDv/m4KXGJzIpVwz8YS/gE1iE+q42hHMKCASvkfP3EC+dmlMjQXWAz1AHKaTYAOHDyAfHnzIX36dG9YV0XUzp09i3tsbat2lRcvXjS9RFSIUdaK11D/B/UVUQcxr17twYD/Q90UVZlZ3d+esWNgNrYsVpVmNXpS8yjvFXcfPnqI0ydOIWPmTG+6xknb/PPPtdBnDRs0cDjtbrp64OL9pyiYMqbpBWGH3yjw7DnLrrt7IlmsSH67gS+uunnzFmbPmYvGjRoYPvIaUjJOs4JvpMiRTGuAq+xYqPa2ifi796HGZXfu3GXTqAyISrnyofHsxSvccvNAytjOZfGEegDZtn07lrLjXE12ESxWrOibdbxMjVMdyeLFi4tKlStjCbvYlSlb1vRF9w4gajY0hy1vxTD1GzQM1drIhzZBUH4uwFDHwZx5cqNwkcJcs9mm4Vhalun27rq8e/eu6YGeIlUK06xMQw2B5rCToZoGtWZPCEdZqk8p8P48dQ8bzj9Aw1wJUCRVrKAkUbB/toB4zan7qJU9HmJFft3jJyCGlMsTJ05iErtPNmhQH8XYwlpDioa6D/6+eLFpaRs5UmScOnkSlatWNVXAvY8L589jHnmqZq2ayJwlywfbBlx79AxzD95G11IpAuKV/HzPUA0g0kqXLV+OWbPmsP1tQ9SpXfuNwJgxbbrpa96c7W0jsVPZwf0HTM9j+cHPnD5tgOQ8+7HLN+7h6YElixab3upJkibx82LYCwOGAi4u7HE+YQL7XG/H1990wt3bd03Xyc5duuAme4BcZ+vTXASWUydPmY188+YN7N+7H99+191YIefPX8BvU6aa5mPNmzc1vUL8M17RVXX5wROM334d6889RPn0sTGwUhpECve646EdfqOAhOzov64iT5LoaJL3f1aB3+72/qukNK5jv/SFCxejTOlS+JQyQuChtsjj2YTq9u3bpvnUuTPnjLL5KXuiq+X19evX2ak0Ey5fumQa4y1bthRZsmRF7TqfmI6o7xsvXv6NNafvGX75s3VOR7+Ov+4XqgFE2ubcufOxfsMGlC9XDo0JIupffvnSZQwZPAjXrlxD+45fYe+evYjEboYt27TG9WvXMWLYMOQvkB9HDh1mz/QGSEY3yMB+/dGKPdNLkaFsEyJ/8aTDL1ZXyr69eiMcO0S2b9+eisEy43Js0qwp9u/bh58mTkShIkVwilplc252T7olp0yZgg4dv0ZuAsuevXsxfvxENiQLh0aNGqJsmTJ+mqOUj+uPqGwcvYN5R+5g7w035E4YFQuaZEWqOM7lmvDTCwbxRbJAuq05jztuzzGPNI0R6X+uZkdO7RFbH//66xTsplzIli0runfrajwPmzdtwmi23k6dNh2tjipss73duEvVFnvbX1tpccxF/ATx8ZBxVzW1O3zoEOTp6NK1q+mD/r5x57Envlp2hkqHB3a2z+vIV/H3vUI1gJykeTll6nRqmOeRkb7IZnRpZKbvW53IBg0YgDhx4qBuvXr4acIkZMickcKjEV7Qaune9VucZACtQsWKaNf+S9Mzu2eP71GqTGk0bdbMAoi/2dKxN5Dm9123bihWvDgBvjSGDhqMajWqoVr1Grhx4wa6du5s+ph/UqcOPv3sM2Nhqotl3Qb1ULFiJSxfuRIzZ8ymq+sjfFyzBhoxDuKbwKmAY99VV/x+7A7WnX2Icw+esbmZB+LEjIifa2VAnRwJHPvCofRut908DYCspBtrbLV0aJwnYYBQQvGP3n36GjmRPHkytGv7helYqNbZP0/6CRUrVWBsLblxhQpI5AK/ypjHEPLdvj170LhpEzRt3hxbtmzGsiVL0X/gQKRgt8N3DfHOjkuPUPrXw8ifOBp2WAAJkDX1003lvli2fIVxXWTLng1VuNCJEyfCTRcXDOZiFyxcEGXKlEX/vv1QiL2x6xBMZF306dkT26ldDKO2UYT+T/nIe37Xg9pqHrT+vI0FED+tRsBddO7cOfTnhv/0sxZInSYNvu/WHU0+bYby5SsY4OhGDdCFQDJ67FiTMCHFYvSIkahcrSpq1KjBJIuDdDcsx4vnL1CnTm3kyJEDEWmRfmjQS4EjtDJG0a2y/bIrrhA0Xjx7wb9+hI/Ch0G97PExtV4mRA4f9kO3sp/7gAJ3qan3XnsRk7ZfQ/Vs8TGtfmbEieLYWIjiHwKO5ctXYuu2bShZsrixSJMmTYpNGzdh4fz5aNm6lYmXzZk921gfJUuVMsk4A/v1wyF6LQYOGoSixYth3dq1mDVjBgFkEFKmSvnON3z49AW6rT6HyTuuo1DqWNYC8QEfBOpXtu/YwcDXVpQtW4ZuqXza2iZD50f6Mj+uycB68RIGQNKmTYsGdHFtZZBsyi+/4datW2jXoT3q1auPO/R59urZiwGxj1G9xscWQAJ1BT/8MMU+fpo0Cd8w5pEkSRJaI91RuUpllKcFuWTx75g1cwZbHwOdv+1CJaKKyaqbOGEimjVvhhIlS5rsK/m7lXHXkiAUKVIkH6+xm8dLHHNxwwZaHmN2XsN9asl4BcSOEQEz62VG1cxxP/wC9hs+ooAskO//vIBfKWwjRQ2PXgw49yibykfX+uZLinfIa/Hj+En4rEVzZKfyKWBRzHTjho34ltbuvXt3MX3qNFSiBVKZiumCeXMxnxaJ4nFdun6D6h/XxIrly7B+7XrGTnuabMC3x3PGPlafvIf680/Aw/05cjNLb1f7PIgQ1nliZaHahaUF27dvPzZt2kxttBxy5XodoNq0cSMWLVzIAPqnKFCwIEYOH8HDhi+ZvVMEU3/9lZZJYcivTqxBX2oV96jFDh4wEO07fEUQKuAbXrTfDQQKLF64AOvXb8B3339vUipljSQmkMja+IWZNGXKlcXunbvM37oxcL6fADJ75iwG2b8xQU9pkwsWLjKuyjbULn174PQeU0vbLTmLRSfuIhx5xpPZVyWpTa76LAeiRrTWh6NYQEH0tovPYOXxu+aW6RNFxagqaVE9azxHPeLNfeSSGj1mnOEHub2Vvjt3zhycP3sO3Xv2wAPGOX6bPBk5cuZAqtRpMeW338hvWXHsyFHDg1993RG/M/FGlu+XlBtvHwGQ9XqZqd0fTz+GI9fdGHT/GxnpwlrdIjvSxI3i8Pfx6w0tgBBANtL0rFCh/BsAkcB4+vSpyYyQq2L9unVYx3MALVq2ROw4sc15EE9PTwMq0aNHxzaaskuXLEGvPn2QIIH1Z/uVGQPqusc89OXB9VL6rVyQS5f8jn3MsurYsSN48MKsocBBn0XgSfPVq1biKDf6t927m8/cyQ8LBSBPCSBtfA4gEgIPnzxHp5VnGTS/i9SMeShddySzaUYy66p1oSTSQexwEAXO3XuC2jOO4+g1Ctx/CFs5U1yMq5ke6eM5Vui+BpCx/wBIZpOF9djtsZEJsWLHMr9PmzrVKBv1meqrE6JKxPFk1qZGOAbdpZhmy57VxOKivHUWRJbr93+cx3haU3/TEhGAJIkbGdPqZEL5DHEcRDH/38YCyDsA5G2yPqTP84dx41C6dBlm6xRmNs7//Kru7u745eefjQla+5NPfBVc9f/y2Tv4lgJyNShwrnTLxk2aIAtPlnu3KFxuuBiXVs6cuVChUkVze78AiMDjAd0OBjyO8sBYnEiYUjcjokcMh+bzT2ElNcn40WxZFN+u3399//jNxyjy0yG40uKTYhApQlhECvsR2uRPjH4VmCbNuJOjxtsA8q77bmXm1b69e1CF50BkyXof8mAsplJSjwkZmWjBeM/cfME87zU8H/TpotPw4AFC96evQScW+WVohdT4vHBSR72Gv+9jAcQHACKhc5153BHp+1ZmlvcMHJ0D0anSuHHjIUbMGP5eEHuDgKeAtEStWQxaJDrn4X3zyvpUfEuWpNcJYb8AyH0KsU4rz70Bj2kMlufk+QRPCoQtFx6iYsY49tS5g5d61+WHqDz1GLInjIKt5x8yvhQPgyqn5in/Z8jMv2WM/+ET3z6dkk8ARLx0/+49RKdcePvwqQLxslISJEzw/xIynnq+xG97XJAwegScuuWOgUzCKJo8Om4/fo5qjJkNpVvOWYYFEB8AiBZLIKLxrjMeMl996xd3FgYIrfN433rq7/rxvp6+BRBlA30j8Dh2FxnjRsLUuq/BIxy1YnHRS2qY+t0Ox1FAAecdlx5i/ZkHqJw5DhrPO4mE1Nin18+EdP/EDMI6kOY+AZD/khv/JU/EI7I8ntCN1ZUZWL+fuIexVdMhRayI2ERg7F/x36faHUdF39/JAogPAcT3pLVXhBQK+AZA3oAHDwtmot9dbisv8Agp9HDG9xAouzJFWm6qu3Qddl5xDjt59mZ8jfSoyZReRw+fAoh/nqsDhGUnH8I9vte6ljmRLl5kuBNUYkfRAUnnUEAsgFgA8Q+Ph4prfQogd5hGKo1xNk+ZZyF4/FYnI3InjQ5Har6hguB+fElp7hKrjylkf9x2DT3XX0R/pvF+Xy6VH+/4/ssCGkDk8Dh+yw25ftiPnKxWsPPLvIjghKVuLIBYAHH45gppN/QJgNxy9UR3noKedeQ2sieIgsm1MxA8YljwCCJmWMFU3hpTDqMRa2LNaJDF4esQ0AAiF9aMfS5ov/I8WvMdfqyZwSlL/VsAsQASRFs8+Dz2QwDiwvpWPf64gFmHb1NbjIJJBI+8yWLYIHkQLvH+a66oP+cE4vEk+hTGoLJQi3fkCGgAURpvu99PYfnpB/j54/Sonzuhkzit/k1FCyAWQBy5r0Lkvf4LQG7w8FrPPy9ixqHbyM2DaxN45iB/8hg+PqkeIgnmBC+lQ4XdV1/A2nMP8GN1CmCev3HkCGgA0an6AuP34zl9Wdu+yI3UTlps0wKIBRBH7qsQea/3AYiEVG+Cx/SDt5CPWVY/1EiHAil0WDFEkiFYvZRcQJN4CK/TirPoWSYVBlRO49D5BySAvCRo7L78CCV/Ocz03RhY3yaX02btWQCxAOLQjRUSb/YuALn68Bn6rCN4HLiF/AyUj62eDoVYq8gO56HAHzyMV2XqUdTOFg+/0Y0V04Hl3QMSQAR+Y3j2o9+my+haNBn6Mm3XWTtVBjqAOFvfZ9XCeruUiTNsAWejkzPQJKjm8AZAWO7kC/aEucKYRz+CxzSCR8Fk0TGqWloUTmm7CQbV+rzvuYdZxLI1T3N78ozITyyb70iAD0gAURZZnZnHsINpyIsaZTWlS5zVqg0UAFFN+xssZX31oQfSsqRDgugfLoUdWMzorACiE8sHWEQtZuRwSEP/Z0QnTOELrDUK6ud4AYinxzOU+6QxBm26imn7b6IQA+Ujq6ZBYduKNqiX6J3P1zmKvv8A/Q+Mg7Qs8O++5P6ZdEABiFKRFVfLMW4forHM//6v8iKeE5e8CVAAceeR/LN3n+Coi7sp3/Dk+UsMoS/SmRrD76cFok5i5VhMMWdO52kXqZOqA9ZdwsEbj1E0VUzkoy80c/zIBnydVRvxz4Z05mtVkmLR4kU4ef0Rbqcrb8CjCNdjWJU0to+5Ey+crPjJu2/giwUn0aVMSoxgkylHjYACEB2IVAn3unNPoHK62JjfNBvLtztvUM2hACLf3V3WAHJhBsH1R544TA16HbMgtrGjVjjWsO9SPBkaMR3tdVGQoB+qaXWEVVf3sDVlYZZoz5o1E6tpslmDEwyxzLl7T/E1T9ReufMEaZiGqN7ZRRmkTUkrLjHr5CTgjyP9uk7w2k45hScsmDlj5QYM2uICl9hZUDRZNAyplBpFWJLdDuemwLoz91GH6bwlU8Yw6dVJY0ZyyIQDCkAkQ3syJXw8gW9I+dT4qlgyh59hcQgB/rmJvwFEKH/l4VPsv/oYBwkYZ1nDXoLvNH8e83CVSV5mDZowrIxZSKdynQhMVdfqoasr7rGjYLzYcRAzerQ3Na8cSWS/3kuBsxO04O6Ijqy3pUZEH5GOaVnSID1BJC3LO2clsBRIEQNZEkR1aLVRv845JF53/PoDdFt2DKvOPkGJDPGNFV3Euq2CxVKfvO2ODsvO4jLd5xN5GK8clTBHjIACEHltSv90kPv+KTa3zmnOEzmzx8EBAPK3cVNtPv8Ih1we4wr7PSvIeIlZKm7eAYQ+/FxsiOJMZR20MK6siCkQic3KrNFZk99ZrCMxubBWgPyAvlyYngDE4khhkSpWJKRgb4nkLK6WgSUzSqaJRUaLjigEFzscS4GLpL8C5tN3X0M+Wh6jamREiXS2i6BjqRxwd3NjHalBGy9j+JYr+JF1sb5kVpMjRkAAiJTxU3fcUXDCASSld2E3+5/HcGDmmCPe++17+BtAvN9QdexdGCy/TBBRCWVZJNuvuGIvswkEHF2KJ8cnOVjYzEmkdBi6sI4dPY69jIMUKlSAncUy4pWTuLBE10uk4bdsKnPx9hOkih8FxRgLUcpoBv4uN1Yygoj6S9gRMBTwAo+Zh+8gmus1dC8YA90aVUaYsJbmAUPxgLnr9L038SnbwranO2gk4yCOSEgJCACR/Jy88wa+4Z7/lK7+sQQ8R8w1YKj6+q4OBZC3J/qE5tgRNnnZe9kVu1haIAx16sE0/5NTg3aW4bxBdGAUtSYlH8hyK8gzBnmpASeyQfRAYZ1LtDx0zmMWT5gXShoV+Z4eRraYr9Cylc87EgbKRO1DPkiBLSyB3mLRKVruETGelQKyJIz2wWs+9IWAABBlXtZiC9sNFx9iYcMsqMLeH87ksXkXTQIUQLweqDReubbO0K+XhbWCkjkokPWhRfbJ586axvv8n8ZD8aKGN3V8nLESp0/oGxy/I/DotfYi5hA81Lu8Z4lEuLZvLZ64+66lbXB895A456tMi+226jxWnb6PX2plRD0HlDVxNIDIKSO+KzLxIBOOPsLeL/MgYQxmXDr5ggQKgDgzDZwVQJTG+67mVc5My5Awt8sPnrIw4kXMY2HEEnQZDq+aFtnjhcO8Bb7viR4S6BES3kF7adSWq+i6/Cx6sSVsH/74V7N3NIBIyZ6x7xbas/RK7Sxx8dMnGRGF50CcfVgAcdKT6M7OOCFxflcIHt+yAN/CY3dQnC7DkQQPnb/5UDXekEiLkPZOq3i2Qm6sQkw2GcOaZWn/6VLo1/d0NIAo/tFoznEs4zyn18nEWHEChHemlNX3EMoCiAUQv+6hEHOd3AdXCR5dV5/HIrahLc6zNqOqpzUplBoWQIL/Up+/9wSdlp/Dnmtu+PWTTKhGLd8/w9EAotPnRScdhJvnK54+z4MUsSM7vftK9LMAYgHEP/so2F8r8LjGlPPO7GH+O5sQCTykoaqToNewABLslxlyEfVnUkS/NRcxjNlN35ZO4a+XciSAiAcXHrqFVkvOoBxT8qc3yBxssistgFgA8ddGCs4XK+/+uuszdFx6FssYYC2anCXZ2bxHPcy9DwsgwXmV/zf3RewW2XLxaVRmccIRdE/6JxvUoQBCPmy16KTpKTOJ4NaUHQidPX3Xi6oWQCyAhAzp4Mu3MOBBt8FXy85gBbu+CTyU4pk98b/Bw7qwfElYJ/768VuP0WHpOVygu/I3xhnKsNaUX4cjAeSe+3MUm3TAHMA+2CEv0seL6tSnz73TzAKIBRC/7qFge52X5fElXQarzzwwhREn1kqPrInefT7AWiDBdqn/NfHnrObw3epzGMWT6RPrZEbbokn9/GKOBJB5dF+1pRVcnCWJZjTMjFiRw/t5XoF9oQUQCyCBzXNB+jwv8Gj3+xmsYaHPIszKUZG9zAmivVfrswASpEvm0IfP2OeCtkznrZOVNc0qpUESVnPwy3AkgNSbdQyLGX+bytTd+rkSBhv3lehmAcQCiF/2T7C8RuChNrRtCR5/nn/ttvqpdkZTGua/Or5ZAAmWy/3OSR9nZYz2IrVnCAAAIABJREFU1PbPsNjrzPqZ/ezGchSAqNhjNXZNfMqDw9vbOm/v8/dxgAUQCyAhRzr8x5u8AY/FBI8LAo8Y+JmWR/oPgIeNgYQ89ujOdO1h6y9hBAPWHVgfyy9VHhwFIKM2X0FvutQ+zZ0AgyqlpfsqeNVZswBiASTkSYi33kjgoR7m7RjzkOVRjODx0ycED1Yy9kmvaWuBhCwWWcazPl8uP4NsbIEw7uN0yBg/6jtf8BVbKLx8+ZKdFF6ZqhDhwoVDmDBhzHcdASDq/VH5t8PYzHp3a1vmRGkG9f17Qj6wV8oCiAWQwOa5QH2ecuxVh+3L308TPB6aQKXKRKRjTxWfgIe1QAJ1uQLlYbdYMfwLKhPLT93DkibZUCNrvP/33OfPn+PC+fPYsX07rl27hvjx46N02bJImzatARJHAMhGxuCaLTiFRNHCY2nzbE5VI9CnC2EBxAKIT3kl2H1P4KHaVu2XnMUf3KwlCB4KmKenxsnuAj4e1gLxMamCzReH0m3UgwUzu7LFRHceKowd5X+ZT7I6Nqxfjx/H/YDHjx8jQYIEuHnjBmLEioX+Awcgd548DgGQLivPYtyOGxhQLhXas09JtIjOX/vq7QW2AGIBJNhset9M1FQ3ZfmKDsvOmWyrUqxtNaF2ejbg8n2OvQUQ31A+eHx3+6WHaEvFQoUWpzOYnsdb5YErV66gR9dv4fnyBXr26oXUadJgy5bNGNR/IGrU/BjdunfH9evXMXrMWLRp3Yp9hDL7+qXvsElc5SlHcPiWO3a2zcVmZTF9fQ9nuMACiAUQZ+BDh8/B1D4ieKw6ex9lWB5iPE+YZ6DP2xeGx5s5WQBx+PIE+Q3Ve6MNT6VP33MDMxtlRaM8Cd+4NFcsX46hgwajbfsvUaduXUSMGBHn6c7q+GV7ZMyYEUNHDMet27f9BSAz999EJ5aYL8Fe7RNrZUAilm4PjsMCiAWQ4Mi3/znnc2yx3HkFwePMfZRLGwvjmG2TieDh12EBxK+Uc+7rftt9A53XXED1jLExtEraNzGIyZN+wqwZM9B/0CAUK1HcBM6PHD6CLp07o0yZMviu5/cmLuJXC+Tp85doNPsEK+/exYx6mVE3Z4JgdfbD+6paALEA4ty73JezO8Oe0l1XXcDK0/dQPm1sFkZMaw4J+mdYAPEP9Zz3WlmpLReext4bj7G0SVaUZ40sjTmzZ+GHMePwRdu2qFGrJp57PsesmTOx9PfF6P59D1StVt1fMRBlgbVjCZ3E7Hs+v3FWlpaP7LxE+sDMLIBYAAm2zPv2xE/xUJZy/Fecuo9K6WNjRLW0DmlfagEkxLDI/3sRU6GXAfXPCyRG33KpkYBC/djRo+jfty+ePn2KUqVL4w7dVbt27kLRYsXQtdu3iBU7tp8BRK6zluxLMvvgLYynZfxp/sSIEiH4Bc+9CBkgAKK8aa98aWdnPWftSPhfdBN9NYILjd/3Lo7kE53o/Y7NoJSaWYWa5LAqad5b28q3PPkGQJ6xpS2DpsGF7i9evDApp8FleJ23CMxOnCdYYLEFrZCTdHuuaJodJenyVBbWX1u2YPGiRbhx/QaiRYuK/AUKoEGjRogXL545E+LXNN61rPr8xdIziBQuDBY3zUrr2O+uVd+uq95LvOtI+joMQJ48eQJXVzf+uOLhw4fImTOHCT45+xCAbNq0GRUqlDdzdvahrJEHDx7g9JkziBc3HmLEiIHoMaIhSuQozj51PH/+Au7ujw2PPCCPxIoZE0mTJvW3kFN5ip5/XsRyVtWtnjEuBldOhSwO3Jji7YWLF+PZk6do06a1QzdgQC2ahMWxY8c41zCIFSum4ZNo0aL5m9aOnK94WVq+m5sbZcYjeHh4IEXK5OSLWI58zAfv1XftJQzcfBntCiZGH1ohcaP+L6VXabxRovDA6T8HCL1u9hpAxuFz8kOmTBk/+Ax9QV0HvyJ4/LznJkYw5vI5nxeYqbsuLi5wcblpADEmaRw9ejREjszGVQREvw5/AYgW/ubNm7hz5y6uXb+Gixcv48KFC4YpGjduaBjWmUfYMGFx9vxZnDh2EtlzZEPq1KnNqVNnHn+TCa9eu4ply1ciDk3p1KlSIWXqlEieNBkSJ06ERIkSIVKkSE7zChJk4o9bt24ZXrnMFMlLly7hypWryJUrJ/LmzYMIESL4br7K0SXPh6FwvPdRdEw+8Rwrjt5C6ohu+DZ/DORiP4+XOn7ugPERH/SMgm3X7t3GF162TGl85JtDJA6Yg19uobmu+eNP3OD5heTJkyFVSvJJyhRIkiQx+SQJ4saN4y/B4Zc5eV0joexCXrh9+45Jh7108RLOXziPsGHDoTTpmyxJEjaACpx9qMOkR1weY8DBl3APGw1ji0VG+qjP8bfkAHlM6/83/+d96G/37t3FylVrULZsGSSjEvT2d/51AS9Xd9rTTyNj4N6nuMr9MK5EdORMEJF86h9K+vxazfnU6dP466+tBjTSpJbcSGXmLpmRlDT3i8LvLwARou3ZuweHDx+jgLhptIhnNPPDhQuPNGlSO5W28y5SC3kfP3bDo0ePqKXFQtSoUZkX7vNFCZJvcoLSiCWINWLEiI7YBBIxQV4ecJJAjknN3lmGp6cnTpw4gZ279lDBuMiNdw+PaIFog+p0b9y4cX3tEjL6Ev/vvCuwxj0JHkRPg9xxXqBUmPOI8ew2N6XfNaq36aY7vXz1Enfv3iNvvELChAlfP9zJh+h7neAhJU8ALZ6IR1qnSZsGhQsVQoYM6X1Nd8e88t+UFXewe89uyo2jVCpcjNx4+vQZNf3Iho/1b2DuQwn3LWGz4C/3eMjgfgpVYt5F1LCv3oKN/729Vt/D04Pg7GL4IQoF8n+JDX3/BddjuXsyHEdKlI/9APn+vohwL5/xusDhJT3l0SNX3KQi95LnW17LjThIyEOSefLmRoH8+Y2V6tvhLwDRw+RnvXPnjhFoV2jW3aSJ5P7YHXXr1nEaC0QWWhhaGwIMmc2yMvRv2LBhcejQIezatZsBsqLIni0riRs4mo9vF8rr+5q7tLYNGzciTpw4SJ4sGVKkSI6k/FcuIWcdbgRq+ZPFJ1evXjM8kzlzJuTOldtXFojXdpPWOGzHHWy65okq6aKjc8E4yBovIli1yM8kEH94+YgFFl7W6BO6rlavWQNPD0/Ur18viASv715LpTjWb9gAVwqNBIkSIEWy5OSTFBTQCRE+fND3m9D87lKZuHJZcuOK4Q3RXvswGa3pwPQEyKA8fscDbVdfw60nLzC7RjLkThzlvRaa+ER7cMrUaWhAfkifPt1/Al54ItSOy7SON1DJfg5MrpwEhZJFCTTrw+hbsoBOn8FByjt5KJInJz/wR9apgMOvbix/A8jbbP2E7qu7NNFkKjtLAO85tWDlbSt2EJX+PzFo1H/ca8ExBvLsmQetEHejvQfXIeEh94EsP98GpY/z9G6vPy5iKfPoa2aOi74VUiHHOzoJ+oY2Uijc6VoRn7i7u5tgaTJuMCkZsvgWLF4EjycMogeTGIgE8H3yezRa1c7k0nzfmryOoboaYaaYQ1CMb1aotMh1dCySDL1YXuS/KuN6xUBen0TP9J/Tfez5Er1ZNmXs1mvoVjI5vi3179IpgfWubm6PjTWt2Ifc944YDgcQR0zKUfeQUFAK3qqVK7F923a4urkicqTIKFykMD6uVYu+4MQIjllYcgsp1iSfpayQK5cvGwtEv4fkITfBMRc39CR4rGC2Vc0s8RwCHorTnGVSwtIlS3CcgWdZGrFJy8pVK6NCxUrGWl2wcBGe0c3Spk3wyMISgCjudJ9AnZJxshvUmMPR8kjDshx2vJsCR8hbTeadwkVWbp7BlrdVs8RFhLCvq+++PXyThTWPKbudVp5DAgbnJ9fJiIIpgsZToPDC+XPnjZwIHyG84YnMWbL4yyIN0QAi/++cWbMwb+5c5M2Xn8TKjCOHDuPAvn1o9XkbNG/RIlgCiN5rOs1n+Y8zZMiIy5cvoWXr1rT6koRY2SCf+DFmW33/Bw8JvgGP1LQ8/JeoIXBQLG/0iJE4eOAAKlauhBh0BW7euAluro/Qi+cBcjG2NG/e/GAFIHIt79i+w/B+6bKlcWDvfgJiFZQoWTLE8ogjXmzidloJ5LHM7BMzk+1l31c7zacAcpLWcodlZ7Hp0iNMqJ7OnPuIyBTeoBiK9apApCwQuerF+1+0/eKNN8YvcwqxACIN7OSJk+jBwmfJGSPo1aePCXgdPHgQPbt/h0xE3v4D+jMz4Qw2bNho0niVFRQchtIdly1bhrEjRyFVqpSoU68ei7zVdBqXoaNp6AUePVh2YhVPmNcylkdqZPcneGiesub+/GMNax8NQeOmTdCqTRvjz161YjlGkb6ftvgUTZs3x7z5C4IVgIj/z509h65dvoFeKFfu3Ojw9dfB2u3paL561/0ePX2Br3hKfPbB22hXOLFp8hQj0v8/S+MTAHGn62oYDykO3HQZTXMnxKDKaYK0ZLss7Z9ZpkWWdtSoUdDuyy9RiqVZ/BMTC7EAojTGdevWYsiAgfj8y3Zo0rSp4ZfTp06hd8+ejIMkR79BA0xgKbgBiBhhz+496NSxI7Ou8mLIsGHUmn2fQREYG9K/z3hF4Xf8pk6YX8AaFkasxd4N/cqnQrZE/rM8vOaleMc4VlXdyIDz6LFjkYNngeQn3kSlYvCAQTw81gDNCCLz5gUvANH7Pbh/n/WbvqHb4hxGjBqJ/AUL+nc5QsX1cmU1nnsSFx56YHLNDKxVFf//ubJ8AiBLj91BR1ofUcKHxUS2EVDDqKAeq1etQt9evVGhUkVzql7nQfwzQiyAeFJLX0lijRo2HO07dkBDniKVVrZ50yYMGTgIFeiq6NK1Kw7QbbFhw6ZgZYHooObE8ePxx+o1yJgpE4YOH4648YJvQP19DCzwOEEXQJeV57H27APUyhYPAxgwz5LQMeCh5+pMwrDBQ5iJtwvjfvgBWZiJp7/NmjETC+bPR5duXVG2XDnMnRv8XFhSoMbzsJvHc0/jiitp3Vc+kpWKtc3cd9PUq0rGKrmq5FyKwj+ct/M/HwKQXZcfoRMLeu5jna1hFdOgXZGkiBQ+aFxXXi+tFPqxo0dj/br1KFq0KHr06unvuGmIBRBp6bspFLqzrn/2HNnRmq6Je/fum0JpOsDUvcd3prZNcAui65DmYmYELV+yDPny58MGMkMXahLlK1Tw0eYILl8SeMh/3JnBx3XnHtLyiIuBFdIgc0LHln5QYFGVV3+aMImuqmYoV748jhw9glnTpiMZ01779u9nah/NX7Aw2LiwpCgdPXIEoxjXSccU02NHjhpg7N6jR7DIyHIGHn3s8RJqOjVsyxUeTOUBwxrpUIg9ZcL+c2r7vwDkOMujdGdBT1WDVo2tXmVTIUnMoK3KoQSQ3379hXG+g0jMWOkeysYBgwczNpzXZBr6dYRYABFB7tOEnz51Kn5nBk1kpgZqY8WMHQtNmjVFpUpVeCIzUrADEAVHz549iyd0vSSngDvH31OkTIlkzMIKKcMLPL5efg4b2S+6JsFjMLW4jA4sT+JFK/GEej2MGz2GvLAXcXm46imt1/Tp06M1Ey3y8YCVSeMNRllYr7MP75BPziBtunS4xVPf0qqzZcsWYuNkAcH7an07ZPMVjN92DfmSx8D3ZVKiCjOzBCLvA5AD113Rj6VRVjBWV5vu1mEsWZI2btCkJXunic7dnDh+gmARhhmGsc3p//QZMpjDvH49A6L7h2gAkXCQH/jkyZO4zZTGSASMVKnTMPCc6k2ueXCzQAJiozjTPQ14sDBih6XnsPmiwCMehlZKbdrQBtSQtXqNh2DPMJX34aOHpsaYytoIoKWd2Wq8AUV5576vQPf6o2cYTEvk5503kCZeFDRnMLxNoaTweHgTY/7pSJiJHQndabGsYnbgqL+uYO/1xyzoGZsB+DTIybI6IXmEaADxvnDvq/RpAcR52FvgoZLsXy49i7+Y9qhsq6HMXEnHjRsYw6tKwdsmvQWQwKC+cz5DGXk3aIksPHzblH1/ynLseejSShfZE7eP7UDVcqXxd/T4WM8Ej13X3PDg2Qu0zJsI35RIbvp86LBsSB6hBkDet4gWQIKOvbU5t7E3db5kMUxu/Knbj9GOfaq3MgBZi66C4TT/0ziB+W8BJOh4xBmeLEtEMZFdl10xaec1E9tQZemwrzwQgyf9Pf4OgydM2c2VOCra5E+C2tnjm74ioWFYALENpYKMzx88eY4Gc47j508ymg34xeIz2H7VDbUJHiOqpkWqOM7Rqc0CSJCxiFM9+BUrYd90Y1mkRx5Yc+w6Zq/fjYw8yJsxWXwUShGDcZLoiBc1AqIF4wZRviW4BRALIL7lGYd9f+b+m+jILKu+DE7OOXSbrUUJHox5jCR4pIjNPgUOe5L/bmQBxH/0C4lXn77IfiA/TkTTJk2QLXNGdhUM896yJyHx/b3eyQKIBZAg4W8P+pKrTznCFN0HiM0aQToBrFTdUdXSIUUsNbkJkmm986EWQJxnLZxlJh86B+Is8wzoeVgAsQAS0Dz2zvuvox+58fyTuMPTvuD5qvjMk/+6cFJ8TAskA+sQhX9PEbugmKwFkKCgunM/0wLI6/WxAGIBJNB36kv6kj9beAozWaX0b1oiGmEYRFcr0aoZ4+DrosmQg+mPzmKFWAAJdBZx+gdaALEAYihgs7ACf6+uOnEPbZacxo0HzxCG5R1isVhd3Wzx0TRPQmRljSv1ifZeNiLwZ/jvJ1oACeoVcL7nWwCxAGIBJAj25R03D9SffQJbmKqbnHWGWrG8dYNcCZCEv0ckmHiVigiCqb33kRZAnGk1nGMuFkAsgFgACeS9qIOCA9ddws97XVCOxem6l9Jhq9fxDmdxV72LJBZAAplRgsHjLIBYALEAEsgbdc+VR/iBDXuqZIyH6sy4ih7x//dZCOQp+ehxFkB8RKZQ9SULIBZALIAE4pZX4PwvFkZMGzcSksWKFKxKPFgACURGCSaPsgBiAcQCSCBuVtWZkp/KiY53+PjtLYD4mFSh5osWQCyAWAAJNdvdfy9qAcR/9AuJV1sAsQBiASQk7uwAeCcLIAFA1GB+SwsgFkAsgATzTRxY07cAEliUDj7PsQBiAcQCSPDZr0E6UwsgQUp+p3y4BRALIBZAnHJrOt+kLIA435oE9YwsgFgAsQAS1LswmDzfAkgwWahAnKYFEAsgFkACccMF50dZAAnOqxcwc7cAYgHEAkjA7K0Qd1cLICFuSf39QhZALIBYAPH3NgodN7AAEjrW2TdvaQHEAogFEN/smFD8XQsgoXjx3/PqFkAsgFgAsXLBRxSwAOIjMoWqL1kAsQBiASRUbXm/v6wFEL/TLqReaQHEAogFkJC6ux38XhZAHEzQEHA7CyAWQCyAhICNHBivYAEkMKgcvJ5hAcQCiAWQ4LVng2y2FkCCjPRO+2ALIBZALIA47fZ0rolZAHGu9XCG2VgAsQBiAcQZdmIwmIMFkGCwSIE8RQsgFkAsgATypguuj7MAElxXLuDmbQHEAogFkIDbXyHqzhZAQtRyOuRlLIBYADEU2L59B/76ayvKli2DAgXyO4S57E1CFgUePnqIhYsWw+OZB1q1/AwRI0Zke/fg2N09ZK1LUL3Nq1evcP78efw4fiI+a/EpsmfPhrBhwwbVdIL0uR/9zRGkMwjChx8+fARTpk3H3Tt3kDp1ajRq2ABZsmQOwhnZRzsbBR65umL1mtVYveoPSHBUr14VtWrWNCBiR+ijgMTlxUuXMXLkSNx/8BBp06RB40YNQ63cCNUAcu7cOUybPhPHjx+n9VEATZs0QrJkyULfrrBv/F4KvHz5Cn+uXYtff/0NkSNHRv16dVCjRg1LsVBMgTtUOAcOHoIb128gffp06NrlG8SOHTtUUiRUA8jDhw8xf/5CaphrUPuTWmhQr57VLEPlNvjvlz548CAmTJyEqNGio2njRsiXL6+lUiimgKurG2bOnIntO3YaxfPrjl+FWmqEagDRqq9YuRLz5i1A8+bNUKF8uVDLCPbF30+By1euYOrUaQgXLjx93s2RJEkSS65QTAFPT09s3vIXFixYiJo1a6Ba1aqhlhqhHkB27d6DzZs2o0qVysiRI3uoZQT74u+ngJubG4XFIrx4+cIE0UNrwNTyyP8ocPbsOfz082S0btUSmTJlDLWkCfUAcv78BZNRkTNnDiRMmDDUMoJ98f+mwA66K1wZUK9UqaIllaUAFAdZuWo1GtSvZ2JjoXWEegB59OgRnj17hjhx4iB8+PChlQ/se3+AAvfvP4CHhwcSJ05kaWUpYGSGi4uLyd4MzSPUA4gWX6l5Nq8/NG8Dn7275ROf0Sm0fMvyA2ABJLRwu31PSwFLAUsBB1PAAoiDCWpvZylgKWApEFooYAEktKy0fU9LAUsBSwEHU8ACiIMJam9nKWApYCkQWihgASS0rLR9T0sBSwFLAQdTwAKIgwlqb2cpYClgKRBaKGABJLSstH1PSwFLAUsBB1PAAoiDCWpvZylgKWApEFooYAEktKy0fU9LAUsBSwEHU8ACiIMJam9nKWApYCkQWihgASS0rLR9T0sBSwFLAQdTIMgA5OnTpzhz+jRu375tunmly5AesWLG+s/Xu8PvPnjwAEnZNTBq1Kj/+q6aQ12/dg0pU6Zk459oPqpt5f7YHTdcbiAKq2kmSpw4QMp0v3jxAjdZdM2DPQRSpEhhCzY6mIHt7UIHBZ48eYJzZ88aeREvXjykz5Dh/8mAtynh7s79ff06YsSIgYSJfFYE8/nz57h44SKiUYYkTJTQRzJBNbEeuT7CLZebiM2irAkSJAiQRVExz2tXr5qmd8mSJw+QZ/j2poEOIOorfZUNeubOmYuTJ04gUuRI8HjmgZSpUqEBe5JnzpLlve+wevUq7GJZ7WbNmhvA8RpawAMHDmDRggX4rFUrpE2bFmHChPlPWjxgddUlvy/Gnj17UblyJVSoVMlhZZlV4Xfzxo28XxQUKFyQ/bRX4d6du2j26aeIGTOmb9coxH9f/TZOnzqFu3fvmpL6GTNlQpQoUQLkvbUJz7GXg6ovS0B8iE80CfGX5iYFJmHCRIgbL26AzE0CTwIiOgWebVr1PxJfunQJc2bNgnpwRIwQgfLiGVKxF3mTZk2Rhv++rxDqxYsXjUzImj0b+/28v+nTwQMH2db6GEqXLm1oP2nCRKRLnxZVqlbzkUy4RsV15rTpLPF+Gx/XqoWSpUo5hD9eUvk8ceIk9u7Zg+ofv26jPJt0EN82bNjIIc/w700CHUBkKfw6+Rfs3rkTdRvUIwOkw8lTJ7Bg7jzky58fHTt1MujPXYtwLK+uzSutIAIZZ8a0afhzzR/o1uM7g8BCYi9BI81EQihHzpxGSOs6WSsCLGksbzPZsaNHMXb0GMSKHQuft22L5LyfGgWFCxfOfFfP1JCAefnypfm7q5srXrFHdqxYsd4Inr9f8TkPH0CWhp6j71+5fBmjRo4y3+vUpTNcrrvg6dMnyJotm5mz7ieQCcPnSDsKw+dqnrqHnuPm9pi/PzfXh+TmRVqjM2fOmM13hYIzMpWJp0+eIm26tAZsHVUq+yrvvWrFShQvURzxqR1OHD8B+QrkQ8WKlXxkEYqvZnCOHp4eqN+gIfIXyO/ffWeu1/tfpnBct3YdihQtSl6ITsVqDpWozKhe42OHPCO43+Tx48cYN2Ysjh45gtp1P0HKFKnoNbiOeVRAM1HR6MR+5NGiRuO+fInwlBEa2l+irQD5LPlL+zJ16jRmT4cNF5b7yw1/c78JLLS/Zs+ehdXLV6JDp6+RM1cuHD582HhFBE7aj7J+JLf0t7d7f+hZ27ZupSwZjZIlS6EO22JLCQoTNoy5tz7XT1jeR3P0kifa/9GjR0ekSJHeLJG8Mg/4nFiUX3qOSsavXrUSM2fMxKBBg5E6bRqjdMv7kilzZnOd5uZOGkXjvbzmpvfUcySr9Jyo0aL6CAj9wiuBCiASkkfICP1690E5to+V4DaCmY16tm/fRoJ5IDuF7KaNmxAtRjR88kkd07hlPpklV+7cuHT5ElYuW44kdGG9pIB9/vwFKlephIqVq+D8ufNYvnQpGjVpbMBmzqzZOHnypLYpsmfPgcZNmxg3lcaN6zfw86RJWLF8GZKnSIlWbVrhKRfr3p17qPVJbcQho0yfPh2Ro0RGhvQZsX7dWoJKGFy/fs0sSJkyZckodQ2Dzp45C8eOHTUAkDkTNz41hXVr1xpBIAukTp06SJ4yOW7dvI1G7Kd9h5bIQmpFJ6jxsBgycufJjbpkOrXJnD9vHoHjJW7fuoV79+9R4JVAfVplH3Lt+WXhneEabcrhQ4cZi7RB44ZIkjgJrly5zLWbg2zUGqVMCHC1GdSrRTR+7vlaCIhvNB49eshN9NSArTaQ+Q43kL4v4fPq71c4dfwkhg4ZgpatW5lNfur0KeNmkEtR9xbtZWFIGdF9vA+B+ooVywlyM1C1WjUK9hqvgZ1zkIB4QR58+YrCi88zgosKhT67x/tpPtrYXsqL5nWX/ByJf5cw0vd37thhFJlmbKlconQpXGSDM32W6p8+ExIqro9cESVqFCNwNPRMvaeec//ePUQgjfSZT6wpZ1h338zhwP4D6N2rJ2rVqo2G3D9aI7NX5s/HLO7Rnr17mz4tu3fuwuft2hqar137J65dvopSZUpjy+bNyJI1Gy26xFhG+SChLEvSlSBSrUZ1xIweA7Oo1R8haOTNmw9du3cz+zd16lQoQV7R9WtWrzaKjWhcl/u+GPelaC2QOk0ZM2L4cOzduw+FCxdGseLFcI9rkotAVKpMGWM9qONpyVIljbIkXn9M1/kVyrIkSZOgRctWSE2g2rh+vVFyxLN6x49r1eS/UTGOwCR+Fd91+fZbLOR7J6A7rnLlykYurV652gBIvPjxUKNmLaPcrFqxgsB5Fs9Il0uXLhpeb96iBbITc9ZXAAAgAElEQVRRtjqaRwIVQLTw6/jSP1Cj6NGz579MPX2mTXGLwnMstfc41Bq6fNvVEHxg/wEoW66s0dBnExj0e7FixbFjx3bs37cP35Cwrg8fYfSoUejTty/27d2LPWxV26BJQ7MgSxcvoTlalVptc8O7z54+4zz+xNTfphJcsqMpN+/vixfjMi2HLl27GqDp2b07QSwm8hfMjx/HjkO2rNlRqWplbNywgUxzCv0HDcT2rdsMg0n4aW6L5i8kcxU3YDdxwgTEIHM2adYMW7dsgczcrykQly9fjsOHDhkGeU6NdtmSpShdtiwyUescPmQoLaEU5rNtvLe+N2DwoABZeN9s4oD67s7tO9C3T2+0YFvQ2hQQEoTa4HNmz8Zi9puWdilLUZujSdNmZn3mz51rXBIC8RWk5eqVq+Dp8YyWZBzUrF3LxMCWL11GEPGkwnGZtMuO48eO/R97VwFV1bZFlwp2KxaoCHZ3d3d3d3d3d3dgd3f3M57dgY0FmGBh159z6/Uj7wI39aJnj+H47+u9556z9zor55oLDsq/SG26yKixY2XdmrWSKUsmKVSwkEp90tl4/uw5Il47pDJLSKXKlZUiopdKWaKCuIozL4gUR8ZMGZWxYZoiR86cshcv/vFjx6RW7dpy2f2ynITcUcHf97wvdmHs8OI2lJxQLFRQS+BJ+sIxoDNSCMolS9asMmXSJDmGaDwdnqlAocJQhu+QPkkmpSCvBw/8I2tXr4FD8RheZ0Q4SiVhxMrJv4cPyxE8zycYJE9PL7hIX7E/9aDwCiiD+yetdWtWi9us2TJg0EDJniOHMtrKEcV+duvSVe07leLGDetlyvTpSsnPnT1Heep0JhfDyOTHWTFaGQzdkCplaikBp3MHpgn6InPQuEkTpUP27t4rterWVnvYt1dvFYlkyZpFZkyfIblxfpQ5Zj+YhRg0ZMiP6aXPkeVYs3q1+lO9Rg2JB0O1YukydVZ16taTrYggli1eDONXR+mkU/hTB/omdOhQsnD+AqmJ+0+FaGLqlCmSMUNGyGVm2YOI9OHDh9KqTWvolwPKgHXGu8B7Gjp4sNIRRYoVkelTpkrq1GmUHO7ZvVt8n/mq59m7e4/shZ6qjd+MFCWSivBLw/mhjOicEEvJyG8xIFTIffv1V8qWSxWh4Nkzt0mFPwOCwBx1py5dVE54+NBhULKFleexa8cO9fd8Ka/C+veAwq9cpYpKTczAhrZDGLocB5geI2rbd+yohI0ppQiw6o6Ojj/2jYpp6uTJEMqcKpfKF/nunbtKacWDhR8AA0fvkWm1WTNmQhjqwMJXhDexXybDY+zUrauKjFiL6di5sxJsKrgI4SPgv0PL2DFjlCfZElEWBfrRwwdSqWo1JdBJkyaVdh3aY8b2ZxkFz/gNnpne0uJFSxBRlYZQ1VK1nnFQXL3gffFZ/8RU1pIli+FFLpZRo0dJOqQedeE9vbb+2P8K2O+LOCeeW78BA5RSGDVypOTLl08KFy2KfZ0tsWPFlly5c8vGjRvVS5kHHuD8OXNVXa1Ktap4uXLBgFyATM1UhoEGolf3HlKseHEVBY/G9WLHdoBzUFrOQJFw37v26C6Zs2RRskIPn4p/7949yhggqQlnY41STsyrL0P6Y+umzdKzTx85CkPAXH3NWrWUEZg9c5ZKOzRt1kxFGUzN0uul7O3ds1eat2whz3x9ZcHceVIBxi8FItjFixZKTigEGqsJ48ZLtOjRpFy58vBw4RTBUFE+6ZHORRq4IvLt6dKnhyM0VxwcHKQv9ojvzZ+0uNdzYUD6Dx6k3kWdAWFKqxv0QO06ddW5b0JmYtLUKWqP58+bJ1dRO+B7xAxBvgL5JXmKFDIezkP5ChWUIl0BR2QzIhI6n3Q0NqxdJ12gSxI5J5LOHTpJeuwrIzw6iSNGjUKNNjGix6dIZT1T9Vpd6om6iw7iLBia5oiAmJKeNGGCMiC1atcR1m2ZbmN99/ix4yo66d6jh1LknTp8S5mFDx9OZV0GwTgkx3x1fsbXx1dFKAegb+bPmy+jce+sF49EKisWU3IuSWQ3DFrPvn2UDNCAzIOeqYgMyh3Ufljn64bfYb2ke9euiLYTS0vcn6Xl45caECrzM6dPy6B+A9QL07R5s+9piBfwLFcoFFXpsmWUNY8GRFYXvCw83BHDhuFlL/bDgDCUozfC6KRD+/bIZZdQyKwZyG23ad9O1UryIEJp066tShMQicE6A+scukUBnKIMSA4lUNPgAbDo1hlCScRGP3gh0ZCqyJYju8ybMwf32hyRT1Hl/Y1F2qV1x/ayDGmNTPBS2nXooJQfU2Moa6jazVgopugxYyFN11IpiEcPH0m5iuWVMqLQ0LvgvY2FkXgI9EYhRFVrV66SKtWrqvw3lSiFpXuvnsqA6FI2f5JyWLZsqSzBHo4cMxopq3Q/GZABffshnVgFKc/z4gQD0rtfP5WSHIuXmQaDNRJ6oR63PeSh90N4bbuU91cG8rNy2QrJiNQgz4wvzDlEcoOhXFu2aq1AGlQ8heGQUMlTSXeDwcgLo0TnhbU0hvxUBMq5gcxu2rgBgIv1yiFhupXyUAsGpBRSp8uXL5PtiIK6I2KlAdm1Y6f0x2+x3kWP18vbS3mmdJrad+yg0hqMsuhhMrK6ee26jBk1GuCPJiqVwWgna9ZsSpEthOJo0LiR+h0azxHDh0sdKEyfZz6ybvVapSBy5Myh5OTGjesyAt+1FgLod8kdo7sBcDYbNGoAg1lZXiDtyfN8hP1bBGesZ+8+GC3rrSL5iVMmK8XMGuvNmzeUIf9hQIDaYu2rKuooTPXQMG3AmdL5u3PHQ9av3aC8fKYOO3f8ZkCYGruA32LU6ujkqFKJRFQmgjHR1V6p0w4gRTVr5kyloFl/ZT2kGHQS9QrT5HQ0ayOFfgxpti9IcbaFjiLyswsyEowgmGZldDJsxHBxAQDoFaIcb9RN42F88sF/9iNSWYh3ZIxKY46EDMRAtE1dxtpLv4EDJBnkmHqJjndpOEKe9z2RpnuiajqU/57Ql/GRHqYzGzOWZR2MX2pAKIS0rlMnT5GzSB1QiWbKnBmbdwzWc67kzJ1LvWw8DB4Wlaeu2F0fAvTh/QdVzKxTt66qDRw6dBAv5mT1YtIzmYqXtBu+swUeISOa/gh7qZxnz5qlPHyGu/8xINlzKM+SUcJ+IKc6YNMZOfSHAmPYmytvbnVvNHY0IDy0cXjhu6OQv2P7dnng/UAGwDvi/U6fOlVy5cktJZGfHIvP8J5at2mjvJ2HEPLG8ESZLnkJyB9ffgroyOEjlADkyJVT5sxyk6o1qikDcgIGZNQfbkAOHTyklGwr7BEFn/WBs2fPwJHwhsJeJ20RpW0Hgi1evPjSHy8KZWEMDAhrQ4wcGYHEiBlDFaAZGb6CcmdkQsVQtEQxqV69hkrpUNYGDxykjHnqNGmkO1IfTCHRq2Q6lS8h0wCEdTP1FBcGhF4eFwufG5AeoYKiAWEBdg48YqZHyyKdxIhyJ+RAF4H8C/lgJMAoc/jQocobpLM0bdIUyHMvVcinAWFUHCtWLPz7HUSro1Uaz9XFVRkQpraosJYiIm3eogWeqYh43LqlUrkEnhAcsmPbduXJ0vCOg4K7cvmyDIfTEgcF3D9p0WBzH2/Bo26Cd5D1Jxp9bziblIFuPbsjnXdYZs6YhSi1v4pWhw0ZioK6/X8MyAyklRmVlq9Q8ScD4okzX4K9pmKnfDA1lgERccKETirSa9e5ozLqdEw94bT2hczy7JR8+DcgkC/WJ6jkeR1GnoxIdyFSaNOhHepdR1WNjL/DaKJrp87KgLi4JkG06ibNWrWAI5NfIa0I3OgJJ/bc+bPihmfrj99kHWc4nGnHBI6SGY6rGyLcuvXrSvHiJdV3aGwaAYXKtCv1bAfIK3WZMiD4zh9hQLjh9PRpCKiMVTEQijZP3jwoKDUWJ6eEyuOjMv2MzWah0wtprHpQ8iwostAUHkXyt1C+zD8WQKjPVMCVy+4qdKQyp2dPb8Mb3h8iTBTm00g7bKauMMmDpzKahpQXw2IWV0+hCDZ54kRVxE2aNJlKnVFRZc6WRUUQhAcXRq2Cln4Cai19AQSwRyF3GowGI6Ev+KFkSGcxvUYUBg+UOc7CUFRML1Bh9EY66ubNm8qDvYN0GcPfFAitqdj4/TEwJlWgIKiYmLIYg5oI0yl/agrrOQrgQ6DY79+9p6IFpgWoDAmH5F43adpM7e9LfK4fPncOxoWpxqrVqsGjiqfQKaxrZciYSZ0d6xilUKfatGGTFCtZHACGasqAMNqkQ1CtZg1V6GQKi84AvfeBKMIWRO2BBVoWX/fs3C2dkZ5kbSygAWGkyXMaN3oMvlMQqdOq8PqmKQNF5X0EBfEjkA8Wdl2/GxCirAgWGQFnIDt+rwEip0MHD8AgbVTOCgvsjLArA2zBVARlOAcUI40IUy7Mwzds3FilMhYtXIQ6Wke5B4VHpUQn5CcDAuP6p0UgPAMaYTp4xxCNvIMOoHPI9DIL1XynWESm0biH99AJSl++iMSIFUPJCetpfI/ppM0EcKYyUjx00NbBQSHoph0yGAQ10JF5/fKVDERt0w0OLM+/UuUqyvljgZuKxB6y1KRZEwXv1WUEqL8OIoVFp6IJwDg8t3lIoa4FUCYuaqkE9LA+2rhpY1XoZwTSGpkRyjpTS6lSpVb3SYeJNZYvnz8pBFcD6ELqAfa+9OreU0Ujg4cOUVFo4sTOKmJdvGiR7ER95DPugQV3PltJpMDnuLkp1BjlNQYMbm84LgSoNIOeDNEpLJ1nxE2nJ0ePkwXJ6LCS8WG5KRRMBRHx8ACh4rt3b1W4xlwkQ0YqXNZJuGFEuRD54hAX6QYUq+nN85oKYouXknA4IrhQX1TKhuko/wgE1lP88HlC/xj2EiHDsM8XOelo0aIqxRMWucmw9mEV2oq5VR46f4deEUNVGj4eFL/HMJS1Ex3El+kQ5rfDhv0GNf4MweA90Lh9+85jpTzoMfJa3BNGMUR+8fPqefA7hBr6h/r9Sd4lz5NNWxT448ePKweBe809zILItCNSCuegnFk/YFGbUQE/U6ZcGaVkJ8FoXL96DakgB7Utz1BELFuuHBBu7kgJFlYKgH0D9MYGIg3CesqESRNlAlIMhWA0KletoowG02hvIWsserPORaPkP0WxBWkIGiWifJj2YqTJCIBgCzoRzMEzxUYFR1RVr969VSqC9Zr7cBwGILd9FICPBQBt+L15DexdKAXqoONCWRiBGt+tWzdV9Mo8e6ZMmRWwY/36dSodR6QZFVbJkqVUSot59f2oodBZYaqMRof1wMHw1FkL/NMWvfaXr17gnfdRqb+ofPcARDh67KgUR6qIDhvPmO8732W+q3zXeYaUJ77LCpWH9521UN3fU5b4+TD4rA/eVYJr4kCfEBrLz/M6jEp5bdY+CAemrAV8H2nU/PA7hNfy36gfHngznR1K6TZen//GZmIu/ib/jZEk5Yd6j7/J56POiIk+IzqdvBaND+tw1E/xIW8vcG3qHaZYaTgf45l5j9FRK+P7oX4fKTDumXo2fJa/w7psFOjJEI3CCijYVCD8w80M2KfBv+cKrElI/Tu/F8TbEtw19H3Vlr/zpykGXQpAGVsI+WPUiaKhL4f4ddaAipUorowJ6xJv8YLxBeELQYXAP/REafDD4mWngVGGHZ8nmiocDD89S8qPzmGhg8EXk4qDjgGN9QdGsrgOHRK+cPyNgE2M/DwNOv+eioW/Q6i1Pa4RFc4GX0oqiI9QEB/wolPxfOvneaUgvnRwqAB4v3SYIuOz/B1+hw4FP/fm9Rt1PzRGdnb23xQOfpMNr+xyVgYUaRPeA++HDhA/o4Mrf4KDwt+xtIKwNZnjfqlnxOvPVGDYcGF/AEx0usQa90wZMmZvTdEjuvfhV/yOpfbol9dALHXj2nX+rB3gC6d7SVXzKBQko0Md3j4wZ8L/i2rqS6t7cfU5MoHtsqm/xWc05nf4+6b+1p8lIdrT2OIOaAbEFk9FuydtB7Qd0HYgBOyAZkBCwCFpt6jtgLYD2g7Y4g6EKANCcjEWolggYvGD6Q2iHGytyU4V+5G/5uL9GZPTtEUhsfV74n6zJsCCI3PjlA3WBvgnsBra734mda9Yf2J/z+/eW32/z9QhZYQ6hCvUd3ocHfedrdyzjjtLdz/UHbYsIyHKgBALT8gc0UqEukVEETFZsuTAXKdWeGdbUNTM3d8GdJPoIaLFyMOVHPBe0nRoyzo7wAI7qUqIvqLBIGqPiJU0wOInArWJraHYCBpgEyS50jJk/NaBry3r7gARWoRP3wNH1jcgRljVUU5+KDYO24KSppEj5JyQ/08AXxC95eiYEKwImW3WEQpRBmQPOF5GAQdN2FucuA6APAK6BzQMmwRbtW2rYLQ61A0tOT1Q/4ueKj0/CpCuOMvP69Ab/A7/v45Sg3/Pzwb0YhWr5/dr+L8+0UMUANJjfAAX07d7+SrtgcfODzI1bVlnBwh9ZIPhgQMHFRU8u8fv378HNFQUad22jeKt0nFE0QNlkBJQYegYXPn3OnnQFa9514pR9bss+P+s/yfS/b0+mdF9jpEpyfrYS8KOevIqselNW9bdgSugNmGDH3u2kqVILq/ghJL5IiWYCdiXwb4PnU7QsWLrfe+h1Mlq4V9GdDqHBoDyw//VAUICOge8tnJycI2Ai5Bj9ihRhxDaTxAJG09bQ7fZgoHTd0Ihy4CAZGwSegJatGkpJYCJ5yGRRI1cVWzGY4cpoZJkonwF3HgiUD+TM4YQSEIl2ZTz8BEoJEAxQu+PEEzCRr3Q+k+COx+fp2DnTSRRAMO8ik5QUgqwsZEUE4RMEqvNpiY2LBGqyeYk4u51QkJPmF3LpDZgNzIjpREgQ2QXK5vGtGWdHaABYaMe6dZJc0OaBzZsjgbPGM+1D5rNnEB1Q0JLcgSxJ4d9GjxrRUEDzD6HmxESTm+UZIj8N7/Xfoo6gz0+7E2icSLmn6yqZAV2TuKsPkfjRKqaG9evyWvIAAkd2bQakPqb8kpFxr4N0t6kS59O9W5oBsQ6cuH/qoxO6WSQuJA8dIR4Hzp4UDUF58yVW3X8M6tB+vfHjx+puS+uyZJKNBCq0uiz+ZnvPt97wql5luwJuu3hoZzENxjXYAejQJ1A+SApLD/HwVfMjjB95gXiS/b7hIZMumBkAbvm/Tu57B3p1rmrlClfVvUpRQYNO51lW14h0oA0hwFhUxU3n30AXcEpwwYfCgE73K9A+UfDxtNo1EGrfwGwrpJXn8R9MXGotP6cEcFmLtIvs/uZzKbsOSANiQea286dPaua0NhgRir4wkWKghlzP5g1l6iuz+foBUgBT6Ztu3aqoUw3Q4RpNjYAxoEAkqmX3mYV8O9UAP+OtqyzAz8MCF50du7rOM9WgKeKXEg9evZSDV1kSObZfIPRiqJQ4QS5CWPGqsZVNnWSLZd02x1AX0Flz/EBCRMlVMoiT568irTuNvi3woRGGgR0FA1Bk83fmwt2AQ+MFCAPGntLGoEVlZGxLvKhx8o0ChkW6MGyqY0pzsFaBGIdoQhwVf8GhBQ5umZdkpneBp0MswRkRf4H7zgdyKdo6iO9TvWaNRX3FilvwsNRIAWJD4gOGSEMHzUSPF39VCNxTFDqZAQjQpx4cUCls+HbbBs4G/nyF1D6g3qBDBR+mPXDOi6NC51eMglQH1E+zp09hwikuyIEZS9QLMgmB92RcsdWa3kh3oC8QdjXB6367ATNkzevUvBtwTvDdn/SiZC7pmWbtoo3ibQh3Xr1kPM4qMEDBirG07xg6uzfp68KYduC1uA4ultXrliJLuXKkjlzFpkJlk02jpEojfQojmDIrAGSNg94HpPBvdW6XRs1q8B/iMlIZQs4+fldZxx+O7Bucu6HtqyzA4EZEDI3cxhRa3APncNcCQ4KaoOUVkQMICLTcVwwFKRIkRJ02VtlJP4/DQF52Hbie10wSuDCuQuKi6tL966K5nvp4qWKUoTd4GxIJBUOOavolZ45dUYRLCaHx0m6FVKsD4Ny4ue4KBPsLHe/eFkZl+UgkmTnMyMQ1my0Zd0d0GdAqLRJgXMSLAgkueTkULI0Uy9sBUEmCTr7gl9rFfQBB7wNxXkykiDdOx3OYSNHSC/QyfAc+8CQsLFxyqTJkipNKqlWrbq6HumaGjRphJroOUQwHmouB3OodFrzQ/eQY4/EsbwXshiQboWcaHR49oOSnZxavcG4a6uRSIg0IC3atEIK69s0OXbqdgWjJqe5sQv5IPLgfNmJfmI6i2kl5hE5rKkKhsFwFCRDUQ4yYhojX4F8Ki1GGuQqoL4gQ++enbskLXiJYiLVxXD2I/KWxUDON2vqDHEE1w7TFuTGYVhbqEghqVGjpvI8md9kGMocO1NejGLWrlmj6MJ7wsiRTVNblt8B/ymsbvDgdPOiV61cAbLDxYqVeSMIFhkBcGIlc8s8JwIb6FWSMXfC5EmKip80IYwS6KVeunBJkdqNAN080xOkTjkBSm6S+NHDZCqTKS/m0skOS1oReqZkbGXqrFff3ioy1lG2UE5ZEE2N2S9bwOnGmgnpSDg/RlvW3YEfBgR7T/4rUpowUhwFPUC+MmcU1DljJRUAOQ54X2nwSbJIRmgyaHNQEwec0UhMAxksU9xDwZ47CKlqZ6Qse+D93gPOrIkgeiQPGq9HmhqyHJDnj2MCKHM6+WNEQvAPWaXJT0UZYYrrydMnkA+M9cb/XwJneOP69WAZngJd5WLdDTLx6iHOgJAXqVGThiqlxIlbZLgky2YtcP8zT80hQ2S+dHJ0UqkGThpMkjgJhq9MkfQYBtQGKScWqRi65gPzJcNFEvZxciB5kKg86IFWBmtnGjBlkiabBoRz1keB7DA9CNw4DYyCRCEieRqHDDGX/hLCQu5+KjTObaBBmYn5ypxkOBiEeeTS0Zbld0BnQF6DZ4qpCEYWD1BjIOszz6lTl84qlUVaf0YPzGtfuXpFpZfIprwPsz6GgbgyLniQFoJxdQdI7Tp36yLuly6r1MMwgCKYZqCcncP87LoN60u8OPHUNcjLdAZEnKRbrwEZdMHo1GuY1wGBUHJCehQqh/v37iO6maGGQ6mcOq4b2i40UmCNVUSiLevugDvOZwyMBcclt2jZUqURL10AoSrefb6/rHVyjEQ5zGvJlj07xuZ6KxnKDrZucp/R2WAtjfqE7MDvEXUMRQQyZNAgSQbyVaZOmbImbT9ZxTku2RPvPbnNmP5cj4F1zGQwe0HnkkPOaGTIwq3j+TuCAWtE55UtV1Y5HorpGeMBxoO/jXU1W1whyoDs2rlThg8ZpgrjtMiscdzAPAUeEL3MZ2BjnYgCJWsThOidPnFSKZPaoH8n8yaHw1Dhs9BJODBrILlw2FPgWVZDrpMDhzidjLlK1jCSJnOF53BMFUSrg8mVvP6cM0D6cHIa0atp2bq18ippQFhg3bxls0qj5cqdR/0dBSIvUmsNobg0KK91XgHWwfhSn+ZYUSCbuM/0Kkkq16BBA6QlSsi+/XvV+XH+SySksOhtMuXJ+eNMTZKQj7UO5qFZbGcN5OL5C2pU8qix34aD7UZkunrVSlUYpTPAGQ6UBYc4DkpBOANsQQrwfw//q5yP+mCQ1s0V4ZMrGhP8LyNjTs9k6mMokEHkc9KWdXfgMujuhw0arPaeSvs9RkPchHNIpB7T0w4OcVRKOmw4eziJGeTyxUuKv4zjGKgTVmFWD2eEsAbC86VjwAiEbM7JoX96gECTTsLcObMV+SIZtBm9EpDBGog7plVyVgyjTaa7T0I3cfIoU2aEmdPJUKMiUI9joZ8ADbI7p0+fQSEJAwIyrLtbhl89RBkQjhXleEcS2THHzD6QJM5J1HCg+AkSqJQAD5e5QxZLOfiFE8g4bIqU2zQaTC1EBGkd6yAcLsS0Fj/Pg+WgJwoNh9js37tPhZP0EngNprsYgm7GrBGiuYicITSXaAn/fQZMqW3bvlVOHz+pGtoyglmVhXmyfGrLOjtA+ONeFLc5Q0UH046FSYXZMQyMkwV17KucIX386HHF8pwa6SaeK5U6ZYreJuWJ58/5EpxOx+jlKWSA9N/0Gln4PoSX/DDmmLD2lhpjTumExIG3eBCInkNIn5IUkikMKgfWVPT1eLC4um/fXvkEJFdZzLq2tUZY65zS770q5wJt375NsTdTWdPJSACdwToEpxVSqZ8/d15NPGWkSrBNGUQCLHKz3sm/J+ElU5QE4JDgc+yE8SrlyZQXDQFl6RrQfKyb3YEDQyeDbMEcGcHiOcd5c1bHpw+fJEv2rEov+KdX5xTNPXt3Q74OI4X2Ws0jqoCpk7xPW10hyoAYuom6TnCdYqfXsQF8+/cwdyI38pGEch7Yv0+qVq+u5jDoWxQGHR48oBJgCoL1l6AawDiIiCNrWYvRlu3sAJFP7AXh+fGMmdpgxJguQzpVuNy7cw/SFG8wg3uQqm/oW5QLyljAPqMfMkPZIMxLWyFyBxTL73cGCcrKdgwMO4r0Uu58eTBO4gPm1K8CKCYLIOPd9D6frt+MyKmA/RsswgfXXa6To5CgO/5IAxLwVKnwGYbuQ6TBA/wAz4/d4WUwO4KpC239nTugg9ZygiHrVIxgSfNetHgxlQqzVejk33lav+epVVSBqGUrUJWv0Rf0DqmvWBgLy/G6SdEn8revv8KA6A6ZaYdHKGIy/RQHOU8t9/y3i/+356fhYLHTz++VmtPBfiBtaTvgfwdIUcRhVswq0OkMODPmb92tv8qA/K2HrD23tgPaDmg7YI0d0AyINXZVu6a2A9oOaDvwF+yAZkD+gkPWHlHbAW0HtB2wxg5oBsQau6pdU9sBbQe0HfgLdkAzIH/BIWuPqO2AtgPaDlhjBzQDYo1d1a6p7YC2A9oO/AU7oBmQv+CQtUfUdkDbAW0HrLEDmgGxxq5q19R2QNsBbQf+gh3QDMhfcMjaI6cFv/cAACAASURBVGo7oO2AtgPW2AHNgFhjV7Vrajug7YC2A3/BDmgG5PshkxdJ4z76CyTexEfU5MPEjftLvva3yodmQL4LOGdKkCVT40H6S954Ax+TiuHFyxdg8P2iSPS0pe1AwB3g9EIOi4oXCHvzn7xjmgHB6ZKh9zDmiESPEV0yBxgvqqPuJi0z/5tGJuD8Bh19M/8+MIr3z1++ynsoIbswoSRsmNB/skxZ9dm419hKCRP619Cl+/n5yQVMsAwXLrxkweAw/4skjGRr5bnzv8OExvnrOVvOGNEnN4Ft1Bc+4xc8I2Tl1zylVY/sj7445fEq2Hpv3roh5cqW+698UC6gO/g5/qGsBMx0GCof+LrSIZ/wAkQOF8Ym9lUzIDgGjjVdvmKlmlVdsmSJHwfDuQAH9v+DKWbPMfu8iBo0lChRYjVkxv8sCE5G3IohMhkzZJSUqVLqPVivF+/l5lMMMooTURyiaDNCTJX+p34f5PHrD9jHSBjYZX316g3ZWL9+A4ZDOUp5DH/SLb70x44ew3Che5IBc9avXb+uphAmxzRM/wqCzsnGjRvV1DrKjSHDox6/+iAPXr0Xl5gRJEp4O1O3SvveL9gBDjPbinkhXp7e0qlj+x+/SGfz3NmzMC5XJFeu3PIATL6cwZ4d43IjY5qhbtEB4RjuBPETSAaM1g1qBsibj5/l0gM/5TxlcYr6C54u+J/QDAj26CymE87DLPMsWbNIDUwo5PhIegtHMVJyGuZqcwgV5ySvxczk+o0aSVJMnKPHyZnaVCQ0NJs2bJTHTx6rmdz6hODw7eey65qv1M0SV5I7QPlpy6QduPTQTw55PJeKaRwkfrRwJl3DmC+5u1+RyVOmwkikl+bNmv4wAIxKpk2eLIkSO2N8ciKMM/VUY4+TYPyxbiARZeQT5GPHju2YmX5F2nfs+NOI28Du47z3Kzno8UKKp4ghKTRZMea4DPosPXguOwtEsY8fP5HZGIH9DtME27VtLXHifJs8euP6DZnBWerhw6lR2QcxLz1V2jRqVjrlgk5GGGQzPkA+9u3dg3lFp9X4XIfv39f3IE/gPC0+/VBiRwor9bPqH3Zm0AZY8EOaAcFmbses4vnzF0nGjOmlXt06aq61l6eXTMDc6jMYhdsR87GpIDgHvU3btuKJ4UP/7NsvRYoVlbOnz0iMmNElNuaLzJ45S3piNjK90IBr4ckHMgd/xpZ1lRyJolnwCP+uSx2790KmHfGU2hnjSamUsaz68HQiDmBM7UQ4EVkzZ5ZGjRqIo6Oj+Dz1kVkzZsjOnTulVq1acu/ePUni6iI1YEA4OnUnxp9y3jrHm4bDZDsXOBwTxo2XDjAgGeFlBjXJkg+0A47GrKNe0jaPkxRJFsOqz/i3Xdzv/Sc5fu+VSgPlSBhFoke0N2sLbt3ykBEjR6n5IHXq1JYc2bPJc2QslixcLKtWrJBqNaqryZbnz52DfNTEEKpkcETXSLToUTEeN7pch4xw7PKMadOlAZzTXLlz/WeKoe4Gb/u+lU6bbsKpiCijyriadd+W+vJfb0AYgq5cuVo2b9mCMDK+NG7SSLJAWdzFTOOhgwfLR4SiTZs1U2FmwoSJpF6D+uKN2ei9undHOiuR+Pr4SBf8NwVoyMDBUrt+HSldusxP5+P7+qMM2ntbNrr7yoyKyayu+CwlHLZ4nT03nknTtVelRbYE0qtIYqveIguj65F+2rB+ozr72rVrSM4cOeTxo0cyHs6Fx61bUqlKZfn30GE1W7sCptQ9wr/17dVLIkWOLL4wNG3atxMXV1fp07OXlCpbWqpWrRaogtA9zJzj3tJjh4eMh5JokDW+VZ/xb7v4AjhxXbZ7yDO8k3OrpJBG2U3fX6apjh47LpMmTZZIkSJJmdIlpTrGZD979kxmTp+hMhhNmjVRzidnqDdu2kScnZ1l0oQJmI1+St69fSt58uXDdMOKMnzYcMmZM6fUghFiBkTfOv/glRR0Oy/5E0WVjY3S28TR/fUGhFPGVq9ZK8cgCJyh3qhhA8mLlNXz589lUP8ByFunlRKlSsnwIUMlZ66cUrVaNRRxv8KA9EBN5Ih07tZZqteoKU8gIP379pO8+fNJQ3gS/tdZr1fSfO01ufXsnUwqm1TqZbGN8NMmJNDIm1h/8YlUnnteauPFn1MtlUSwtx4ggS/9ipWr5CRe9kiRIkrVKlWkaNEimFzoByUwEalKe8mdJ48sWbxYypQtI6XgODBqoeOxHmNym7dormSBc9gpL6yPtWzd+j+z1P1vwfO3H2X4vrsyBn+GlHKVvkWdjdwh7eOB7cDDl++ly9absuzkQ/WROtniyfBSLpIoun6FHdxOEn21e/duWb9hkwLYFClcWEWpTGHOmzMXY5K9lAysWbNGOA21ecuWaprhfmQvBg8YoP57yLChkjBxYunbu484OMSWtu3aKecj4KLO2XHVV8rMOitZXaPLunppJWH08MHdotX//a83IKxhMM+9gYVOpJ7Kly2rvAkPDw8ZNWKEUgw5cuaS/n36Sj54mZWhRHwQdQwZOBApiusyasxoya680scyoB8MCDwKRim6xYNfcvqRNFh9BamLUDKiuIt0L5TI6gf7J/7Ax89fZfYxL2mz4opkSRpDpiCay5XYeulAKoVLly4jxblDEjgmkMoVK0mUKJHlwYMHyotMhnRERqD23Ga5SfmK5aUUHA1f32cyZuRIlfoaMHigFClSVKH8evXoqQxI8xYtgjQglx/5SYeNN2UvDGXTAgllasXkEs7OekbyT5STwJ5pOxRw6w3X5f3nL/IOaKYYACgsqpFS8jibNsKYBXAvL2/ZAgCNj4+vtGndCmmpqCoCYYrT3s5eGqNuNmf2bOFI3BYwILEdHOBcrJVJ4yagtpoXDmhX1TrQt1dvVf9o1aa10j8B14u3n2T8gXsyeNdtcUIKa0q5pFIxrcNvP76/3oDwBDw9PWXZ8hWSIX16FLmKq0M5dvSozHZzUx5kNuQ1B/UfqFJW5SqUl2VLl8oFFN4fIl1RGgamFbxKL6S1hg4ajDxmQykET0S3bjx9o/KWWy89EeJPG+dxlCkVkkvEsLYBw/vtEmjEDdCDHLD7jrgdvi92EexlEFJYvYtY10P3RC1s7bp14pLEWcqV+wbTvHzpksyEgihUuJDkyJFLRsFg5MufV4oVLyErli1H6uJfoLPuI61VQDp06oRC+ifp0a0biqmVpFSZMkGmsFaffyy1V12RT28+Sa6k0WVyhWSS1UYQN0YclU1+dNZRb2m5/pp0zucElNsHWX72kaytn04qpzNdEb965SebNm9C6vKxdO7UUT33bTiflA+CbRo3baqiEcpDE/z306dPxA21UtZT6aD0HzhAXPE5Rqj5CxWQSkiDMhMScFGPVF50SS55+UkoRN2tcsSXaZVS/PZ91gyIzoAsgwEB0kZnQI4fO6aMSLkKFcTFxUUVuXj4BQsVlt0onrKA7n7ZXd68ea0Eg8W0aVOmSI/evSRFim8HS4952dmH0mj1VfmK/xb8yY2i6JTySSWzphSMFv7Tni+lzvIrcg1ILEGfRKEUMVUUkibef0N+oy8eyBfoXKxFOsq/AXG/fFn27tmjos1UaVLLqOEjxdHJERDNHLIZkWyWbFkVNPzhwwfSqHFjeYWU17DBQ6Rdxw6SCRFLYEX0BzCQA+Fhuv3rJYJoNVwEO5lQ2lVa5Xa01OP8tddhH1b/nR4yfPdtmV8rjTzx+yjdkc4ai5Ry27xOJkd5gRmQ7YD2pk2bVjkR27ZtlcMHDknlqlXlxo3rAGE8lZSpUytkVrHixcXJyUmGDR0q9Rs0UClR9pz5Xx8QMW289FSqL7r4DbqOZ8nqHE0W10wlKQln/41LMyCBGJCAZ3Lm9GlZumiJNG/VAqmuFD9h/VkMW4ew9NaNWyok1YWg1568ltbrrss+oGoETWE0IDHQAzKxXDKbgeH9Rtkz6qdZW1h9/onUWHzp+/e+il04OxlRIol0LWC9lKA+AxLwxtcjQiFMtzYQfCyS+l+EbG7dslmOHz2uZCNmzJiBUubsv/lcaiy/LE9QK6MBkQ9fpFGeBDIO8hLDTLSQUZv9B37Y88U76bLlluwDBHxJ9VTy+sNnaYF0VrmUMWUo0soJTISE6zMgAbfvDgA5i+YvkGw5skvxEiX+0wu0Cw7p/r37FOCCKL+AjYbsC2q17pqsQ3Sq5ALvQuSIYWUQ6mOdkeb8nUszIAYaEBbGmJ7IkDGDpEOqy7+X4IcwlkokRcqUqIdkV+fJjlF6DROYbsGZH77zQmJHDScRkc+umd5BRqBAypqItgzbAWLgh+69I5MPe6J4jfQfPLHw2Ni8QKSMBlopTVzrRCH3EYEsX7NRnF2SSPXypfXeLFFZ27dtlxwAWaSEDPhfrLGtQf+Qo6MTvMvcgaavniPHPfnQfRmFP0xv+iKFFRXdxq4xwsvwki7oCdFoVAyTFP2fOgH4d6v1NxBphJJZlZOjDvJVmqy5JtGw1zOBxkod1zRP3hADwlrJ9m3bVE0kT768P9U4+G8boDuiAtKbD/8WLkD6ijXUU/dfSt2VVyQ6ajYn772UqOgDSR07AhpNw6sa2e90LjQDYqABoVgSSUHFRZidfy+BXiaROZGBntB1Gr94R7z5S4mJNMTVx2+kHjznFvkTSvFkMeW2z1uplSmuyV6POS9SSP3ulUevZdCe26CBCSXuj98qA10nQxzxxT4Xco1hFWj0R6TJ9l+6J2M3HJHYsR2kRcks6B2IKuH1IL94/nQq2FzqXzaoIMhUwKg0YGrC/1ncefZWFgIdFBkK7QJSdNsBV26UOa5SdNnxm7UhL9oyfQc2X34qDdZcldLJY8gERHQsojdceRXIyLeyqnZqyW5ib5YhBoR3TQeUeoL1Df9sBJQPyg51in92C92TUs7ZhHwLOsMlZkQpNuuMFEkRC/WPZPIPItY8SaJJWiumcIPbcc2AGGFAgttMff/OOsiMI17SYdMNGYp0Sx+EnU8QkpLnKKaWljB4Sz2fv5MLD15L4hjhVCrCC/UCepKpEXm8B8UD6WFCW5jahPBrAiAOXEcKMtRXSe0YVcaVSSolkfaw9Hr1/rO8ePdR7NGd3Be5+s1XfWQGwBYFAdl8DiOZANGrhsYyfdfZwV0fXjwL6KNwhp9QV6gLNN96GJZdTTKY3LBpqAEx9c4ZgfhBNqIi+mAkkm38CSkFFoZtTTMIayOfoF9+JyBHMyBWNiAv8fKP/eeejELqhQXR1lpB1NR3SX3PA924XaDULz95g6bM5Ca/+MHdBPPOQ5Aym3r4nqQIh7SB/Rc5/zmuFHWOKuMBgrAWxYgXcvW90US499ZzmQYDUiFN7OBuVft3A3Zg2r+e0hYGZDCij37FnNU3WqI3axZShmsbZ5DKSCubsqxtQPzf00kYkOwwIGXSxZEtjbVGQlPOyyrfUTDeACgsS/0Qoaf9gKzZfMVHJkN4q2f8xpWjLdN24D4ikR7bbslRpAepYEunsg6dyQEUW8ssvCixQ72VmpHvSCKHaHI0QkZZetpbJuEc2wG5Y42le74j35+vjJWezxr3bqvXJABj+N670hdyMwnQ1/aIQrgG7rwNhog7QEUmk+Y5E0hYE/ptfqUBOY46Ts4Jp6Q8jJ3WiW5D0mZNA3IT+O32G2/IbSBrZkJ4CyAloS3Td+ABWI37IMWzAzWCaXjxK5mB4Q/sLhg1TkXE2A9Kpyac1TyfLoIk0VneJs4ttZdflDKoYzEKSRzDtA7moJ6etZCuSNGd9faT6YiwSmjFc9OF5fs3eZ4DAd+dcsxb5uId1BERkm+MtCZtYTx6FExkUjH6VxqQI3dQ85h8Wiqh9reuQTqz98USF9BSWNhFaxqQc2BWrYdcaxQgauZWSyGp4lgHLWQJYQgJ12D9aACUwSog3KbCgNS0QkRH+HX1JZdRwBbpnTWCvLx0QJxhQDLmKyHt1l2VHbeeycIqKa0STXr4vJHOm2/K1SdvlQEprJEpmi2WdDp6bL8lu26iQxwGRJcWJEqyLWqT+ZGWJDmhUzTjqUF+pQE5hGJ6/imnpSoAFatBZWILSzMgVjYgRFCURiokH1Aei2ukkpiRzGP/tAWh+Z33QBK8ofvuyOxTD0HnkAxkg5blFSNL6yawBlRZelmqpo8rw3NHlm2bN/5oJJx48L502nxDuhVMLAOQS49kYUYBdhx3RMR65/l7mV4puRRw0SJWc+XtOuplnWAobj/7tqcEJnAR2tt07XWJBuduDpw7U+pav9KAHEBdrOD001I9czxZWSeNudtike9rBsTKBoTU3KVAgFYnW3xZUiu1RQ7tb74I0xGj9t+TcSiKTkEXcTOkHyy5WDzvCW91DdA5o0uCb8j5q6xcvfaHAdkLL5Z0GImihgfNSFKLd8Ffffxa2sGAPEKn9AxANU3labLknoT0a5HBoNX66wrhRvhrxgTfBjrdQ7qQ2QEa6/X10pjEDvErDcg+yF6RGWelNmYKLa2tGRCbkUtrpbA4gnLpmYdSD95sJxAojofHrC3zduANOojHHbgvA/fflUlAtZGGwpLrChR4oVnnJW5ke9nYEJMn3/jImrX/58JiOqQDIpCt15/JMkSUlkZJkUyR7AV+6EKnATG1P8GSexLSr0XPnXRCaTANdDKob5Jg0iPXB/RYVEN/FiHT/7TIJPlNiPZ+lQHhCKw9gJMXdzsn9RB1L6ppG86oFoFYMQIhXQLhg7123QELbxKNhdcCmogv/aRDntJ9xy0ZCwPSxYI0JhxUtxfF+eKzz4HpNLasR6FSH5niENRg+gNqOxL9BF1QfLXEZDvd1rBm1gwd0rzmrMopJH0CrWZmrthscX8qNRFpVEodSyajbua/c7sJWLLnHfGWDegFqYAzN3b9MgMC2dx5zUdKYZQBZ8QsgPNiC0szIFY0IKTfGAyY4LJzj2UClI2tjKG0BcEz9R7YWDUNZIPtwWM0DLMcLMnGy4atGUDm9IaB6A3WgEElXBTAIiCZ4jpQrbfE7xeCxzoGxddEFpzLQKx/g1VXJS5qZbNAsZEc1N3aMm8HyLpbG1mAdoDvsgudM8V1qx8cgWGIZgnIaJwjAehxjKPO/1UGhHK/7YqvlJt/QZqAiXdO1Z8pc8zbIdO/rRkQKxqQu8ixdkTT24WHr1XTm8ZnZLqg+v/mbMAxm690l/7giKKSt9R6hPpHu43X5dCdl2pyJOct6DMgnMveBrxKj19/kHkovuZKbLlCN6GatcA4nAqGY3bVFDYxNMhS+/u7rqPkBQakH4z9YMiM/0WWiG6AhbdDLa07olljeaV+lQH5Bu54CnDHJWmd0xG1nP+Ozf4d+6sZECsaEHfwN5E+gbUQKpoM34t3v+Og/6TfXAQEVgMg23ogAhmJNJalFvmGCgDwEA0sv1sapVO5cn0GhKnJtohAFoAeY13dtBbtRfkH+XoqCaL2aEAcIoe11OP9tdeZgJpZ5w3XZCSijB6Ffh6DTChvm03XpSA4pUaXTmo0P92vMiCkLVmFTEYD1HI6YqYQGZptYWkGxIoGhNw1peZflHRxI8pyILDigs9IW+bvwIpzj6TWvAvSHrxik6AULLFo5P+F958PKBcSXu4A1xCptQKjcx+MNNcApD/GlU+uOpstVQfZjUJpWQwOqogO9DlIYUUBB5K2TN8BkhGOwnjgAew4h9INCLpgd3fj1dckdkQ7mVs9pSSNZVzK8FcZkHdg9lwCh6Ul4Mg9IG/DwOZtC0szIFY0IKTDKDj9jEqFrGuQNtA5ELYgCCHpHliDqLLggjTP7aTqBJZYRHfNPe4tnWEUuuRx+hHZBGZAiK5ri/RkVRRehwAgEc9CzsE2UN6UwbPVBxMv89z2YYzLyVtiL/6kaxD2zTEAkzCNcAbrHNnj//R4TDPXWOouj9BftAnvaDojmW1/lQFR8nnigXRG53x/ADd0fF6/+6w0A2JFA7LuwndFB4VkKUX3uwXGFn5/i7uPVFh8UepljCszwcgb3t788cA+UCAdAc/dDnguKVJqfO9wD8yAHL/L+RLXFX+SW5Xkkj7+t94Cc9cGNDFWAtKmGeDJbuh215Z5O0AgC7noVsHpILux7lx1V6VnX37BRdmHht8jrTKCOj+aUT/4qwwIAR7Tj3hKf0RTgwsnBqLz51ScUTdtwQ9rBsRKBuQtBHMOPNqOGJvZH4c9AF6qtiyzA7uQ5qmMomiZ5DEVpXt0zEc3d5HEsBAw9uBtl+2ofyT7jn4KzIA8hWJqBjbXXaA12VgvnRS1EOUIZ6JXn3Ne2hZNLFOg8LRl3g5wEmHPbR7yDwwEgSzlUv8XqlsDsrQKdbXdrTMbfY6/yoAwkiILwmhwtA0vluQHIaR5u2P+tzUDYiUD8uzNRxmJjukZJ71lTElXaWHhjmnzjz7kXoGF5prL3SWnUxQVgZibPmL947QXqLLBM5QPs6b3o6lMNy0yqJG2nZGPngCPcD5oJRqCacASa9GpB9IAPFzd4XCMsiBAwBL3FhKvces7t9g1cosBuVQ4aYz/PEZHnOMkKOZF6K2oiajWnuOnDVy/yoBwYuWof+7K9OMPZAzmCjXP5WjgHVr3Y5oBsZIB8UbHcldEHwcBCZ1eIZmU1+Y6WEySCXWth16JpEBJMQJx/t5ZbOoPMI2xCAXKNkhhtc6eQCbhvHQrKAPCJtHOoD3pmMtJeoFpILqZA8KI9XcD5LQVJucNwJzugVAU2jJvB9jZ33bDDQzrQgoI0Oycif+bohoNR68f6iSDkBpqA4RTFKDwDF2/yoDQIR20544sBoBkAtBittJTphkQKxkQ0riTZ4eT5hYA3ZEVY0m1ZZkdILqtMbq1OS54BrzKVCbOs9bdDb27XjAEK5Annwivv4G/aCIoA7ILPGet0TeSGumuiaibuMQyj96diCEapS6YWzES/S09oNC0Zd4OnMFUyeZINXKa40xQw6TTU6tajPRVJ+x5TczZGAhkX2wjoNPBGRCeKaeSMqphvczw2Obn52aNjmMM1oGjjU2P1UHpbgtLMyBWMiAch1oYlBjpoNzWgXrZGKG0BcGw5Xu48OCVtARfFKlHyBeVCaNmzVkstFZEX8kdRI0bcFbZ/Bn7oAwIiQ+bwJD5wDtcViuVZDbzPlgonYA892B0Ro+DIWtvYZ4vc/YopH73KMAO7Ox3jh5OZiBaddUD0yV0utm6a5IxXiSZCofEGFr3wAwIuat8IFfbwLN10vOV5IBMlU4Zy2Q2bsooMxq7MQd9Jmpj5U2gXbHGGWoGxEoGhJxKRWeckSqZ4sgaFFm1ZbkdIOFhO6QlnsArowHJ7WxeJzgL6GknngR9SFg50SbzT6mooAwIFX79Fe6yCRxFuzAWVV9+3ZinVnluGI/Jx7xkIqhvmoFaQ1vm7cB+MNjWYr0MCjywehmZBWotc1eQ6VV1UxvVCxKYASHpJscOuCG6+YIIJLx9aJV6apgtnkmTDx9hsmk71GqO3nul+oNsZdCYZkCsYECYU1+IYmgr0HJ3y++EYmhS894C7ds/7QALo1023xJ3zHmYaebQJdYdyD+Va/pZKZwkuuxpkfGn3wrKgPCDbUDtPh1Rw3IQL7IAa84isovcaUvQcTwZVPV1s1h21ok59xZSv7sDEUBFgBKqpImFInoKiaanMZORQhlAeW+AieAgzj+NEb0g+gzIZ4TGpEjphLRoejAAp0CKczeAH7GRcp1XLSWob4yDCnPvWVNtgVTcRThPC6unspk5MX+sAaFiePbmk7z59FnCYQ4AOW4CQ1dYms7dl0OPoAjmnnkkY8G9Y+mZFSH1ZbbUfTNi6Imc9eG7ACjAgJgzN5w56mU4p5aoZbRAk9nkANDZ4AzIGBRge6LPYEiRxIBWJpTIGE5k6qLX2hfX2obaCntRKiMnry3zdkD11aCzvwnOlhGIHXSBvkUGZqayTrTP+lMKM7hf12dAjiFt1gY9QndffkC6KZlUwTmS+mY6WKSnIXpoBi4rY5BevAfCkRusuIo06ztZVjOV5ADVjS2sP9KAMAI4jzz5UrBwXnn0RhyR/6ycxkFyJ46qtxZhaQNCBUdK7mvwaOZCYMxNbdiCoNjSPTCcp6LdgK7tWVD45ihapqE4L3vWScIjXaRl7p/hkcEZkFXo22iPEbSlkseQ4fh+/Gim09Xcg9x033pLjiMimobnKg06E22ZtwMrEc3VBO1NuyKJ/uMc+L8yU5Espm8FrXvJlDElNHlsDFgBDcgnpKvGHYBTseWmdCiYUIZAJojq4n102HID0POoMqF80h8zSQz4CfURykZN9Ks8hVO8qk7qH0OxDP2+tT73xxkQko6xA7wHKCk8oWicQDHhiyacj/A0O0A5dIaXGJCTytIGhMXVYmgGi4Oc+nrQI1iS7ttaghCSrktII5X+PEQOpKcwJ9Xz/O1HqbHEXc4gD76mdmop4Ppzn0BwBoRcSk3XXJeo4UKr9ESKOJFM3koPOBwdkPa84ftWNb0V0tOzYPLF/9IvLoRj0BARSHDEm70Q0Y4+dF91q9dD6jACahaGrIAGhOjLLnAoTnr7qWvpZozQqWwCIsR/779C42laoxsW74BypfLCS/IOBmodpiemNEPODHkuQz/zRxkQ5h6P3HkhtQCfffH+kzTG7OBy8OLcH7+RWce95LrPO2mDhr4+6Ax3iPJ/llNLGxAqlZxTz0gxdCeTlM9Qb8bQQ/vbP8eoYTgKlGPQ/EUD0tSMYjNH2GaafEqRIR5HJ3K8ABFEcAaE0RCLtOdB2b+jcTqkP0xPLVx78lparbsBL/OjuCHdoq9n4W8/e2Offybmu7RadUUGgHxwYBBsEJNhPHrvviOd0QfCIWX6aiX6fjugAdmJ9CObXPMkiqrG5yaO8X9odx/URIaj8ZSsv63xO5HCGp7u9IBTUXruBZX62gymBGd/1zV2Tyz5+T/GgBA29wTKoNayy3Lw3kvpBgjkYDRj2WHDicM+BGLDzggheZaEAQAAIABJREFUrz19K/2Aryets26wjCUNCH9r3UWEzUB1NMfgl1kan5El5VVdi1MJVfPXntsyGQyr7UyEu7ID3f2xn6SfeEqyYvLf0TZZfnSg6246OAPCzzG1sBLkijubZzJr5gvRQE3ADPsVN0Yqd43+3zzRoUM5BU5GJ6CXhsOA9EKdKrC15sJjkGPeQCoyJhS8q8E0+joD8vjRE+nYqYOM/+eedMV1+oBuZCjGDfhfrMfwN3KjfjGurKtRs15uAjhSxO28xEAhfkfj9GazL5i3s///9h9jQDhwhcin1oB3ko5iJfKEsZBC0i0KEzmU6mE+hwMK6jMwLrSg6zf4pyUNyIsfnDXA8xdxtujIVUsdeki/DhU/+yW6oPA9GgakG7rATVk09nyp66JPoE4GB5lTLRWixZ+vZIgB6QF8/uh/7sts1LvqE6ZpIoMue4fYfEolMRfzY5I7mJ4OM2U//rTvECBBOemNBjyOP+4cxPhjshvUXXlVXDBdckHNlAb3gugMyNMnKNY3bC3t11+VEzhHNvtVTvczCMIdXfEN8BuvUaNdiXRpuviGjyu+DsRhHowaSIx67h5kNcxlPbDUWf8xBoTT5Mpg3ONV5JFXgNOmFFJXYQIUwoizp0cyAJQAtaAwOEuCDX6WNCC8j/bwMg4hlUaIqUZhYilR/fk6MwGTbIVUwSAUJPvD2zNlkfByNPouSFBHhlMqmIC1U0MMCNMk3VBzawWkT09EtjFNpDQhnLg6ohkXULOQvSChBUflmrI/If07HPw1FhHBEPyZhGmEbcCKHdi6gxRROUB538Ho7EQh3VBWAZ0B8YUByVChkZSdf15yo+fEDRFkwNrnW9wPiRu3Is21v3lGyY+RyIYu1lWzTD0taeFU/NMyI2o0hqe/DP0NUz73RxgQQnaJcmgMnHQBRB9rUGSKHAifDQ+iNbqYT8JLGAGIbVsIlSUNCItdZeddlM+4p/X109pMscsU4bDl7yxAcbQRiqO9Sruo9IQpizQzjVa5y16PF7IU2Ho6HQGxN4YYEOa9WwK2mSFeRJkMp8FU0AS94PJ4phyOUWQhnCCNvcCUU/3/d17AYRwJAsIJcDamoq8mqFrZu4+fpeCsc3Iaxe/zHbJI6riGRQffDMhmue/1QKLkrSVt11yRHkiVBTYpsx3gvFNRb1lcO43UzBTX4EFknG6aDs2uOYDi+rd1JpuZLfRHGJCPQF5VW3xZtt/wlUVIQxB3HdiEOKay1mOMZUMQ1qWKHUFmokaRINQLWbxkuWTIkF5KlChultQyj51x8mnJhpz6PjQl2YqnYNZD2eCXlwGiXWfxJemE4TrjTZxKSAQWGXifw5CcaJtZb2HSEANyHcXv2phjzuL+uvppDFY+AbeVA8hKIIouBSTYYkywNKenxAaP7JffEvux2JhJtB4NSHAEhBWw95vANXUAYIp8aCo1BMlLA7Jly2b59/IdeZKmEmqtvqjLJZWq6fVzVU1FtNsDEPT2uYybwU69km7sCcmHtPvBVpl/+V4G9oMh3oCweH4dUUUBzHIIj9zzERw+sfhBobjZ8TsCqYvxOMz2eRNKs5QoTG1YI+nSm2dAaJz2gMKkJCC81TM6oA6T1mYO+k+7kTWAalcHYKI50kZMRZIsz5jFOsptpC1Sw6tzRarodIesEl7PNQwxIPReKyD9sQfdxkfgHZra5EXZKT7nnFSD8lkGA6IDeRjzXNpn/78DRNj1B9x7LRxG1iQCDpMKuFeEUE8Fjcw81EcZHRgiUzQg27dulpVHrsvx+KUkWQx7mY/0Y2AM0STgbIFoNQ061KeBdytxjPAGHdk571eSadwJKZoiluxG+stWVog3ICTUm4DGnb7wNJoAtjsa6IaIBuQHyejaCqksL7+P0jUjDM7VfyR12rRmRSDMuZLCoA9qLL3RbzJAo+O2mpxvdn8q1VADqYlxwaRfNxR2qbshGnvylZVDFFMuRUxZgZke+qJWQwwIr9kIUNEFGCC2EZxYHFpkiPcacHO2g3aj9KyzUh9Q84U1Ultt7/6WC5P+gwX03TjnaZARjpYOanFgU1+8u4TydkU9LKoB8+j9GIFs2yrT9l6VwzELSeMscQCACHySJGstHKHr9eq9bAeaytARuuxuz4VouRzGQmxqlN5mjjDEG5C38P5KzD4vR8F4ubtJehV6GuK5EYEzC95Ghy23pEj8UJLV75QUzJpWihc3PYXFyKY1cpz/3H4hs5ALrxQAhWEzp/4H3AjrDnVWuksRpHumwLuM46+vx5DHY8PpzCPeSCdgBjpmqw+CsdcnN4YakEGkM0GxdhygmyRBjGgExl93v0yfVIABaV4goQb/NuQQg/kMu7e7obP/BJzF6WjqY40rqEVEXku8v0VQ3NYBbIK7Db9Xr2T99t0ybI+H3I6bU4YUThjkuFnWayuhIXDT5Seyv2XmH0jQoH6H0fIBj2dSaOZZqQYa91V1bSezEeINyHmEdsXQYBM3kr1CNhhTeLyCwlTHTRj6dNtX8r8/J91LppUixYoFJzOB/vvd528l97SzqkFoT9P0kshGmn1MfiAb/iKnEjaE15/hOwW3sYglIrC6oC9oMcAX0wEFrpMlrt6GT0MNCIv6ndHNXBez1AcUTQIIufFjdlegrlNr0UXpACQX54toy7wd8EDvRCe83+z9Imda4WDGDjNNVBm1VJIergc5pqMBtDQ0ICt2HpQeex9KtCRpQYLpKmX1jM31/yRdMLhsPHix5lZNKXUyB58q07UglAYlSwN0yS8AwMJWltUMyIsXLyVcuLD4g/SQKfG8gTs0CxDKroBQNkL6ajhQVcYUHmnZ3ZB2aLvFQ2L7XkVBPbVUKJLbwF/++WP0LI4CupsPSI6i8Ip3AqttxceWT58+ia/vM4kVK6aECWMbkD5DN+75i+cSIXwEJRumLjIONAXqjlQ17PhNFjuiUZdiurEsXsgz6CDfjVQCZ4DoOy9PTy9Zu26duCRxlnLlygX6G/sxp6HJWh0wI4XREFwqiYXgYmoCo9gb/UPDAjShGfNwX758kcePH0ucOHHQGGlcbciY37HWZ9+8eS1fsB8RI0Y06/7ZO0Haf29kBjhMKk8wtP+E+edDr8VdEBaeQ02McOrg1utXL2Xh7hPSdu9LyZM2iSyuEXj9Q3ctDg0jAWd7kCp2AwgkOgxWUIvZki1I2VYBBJg1v5mo0Zi7ngB2HDVqNKWjzVkWMyDv378XTy8vuX37tty5fVd8fH2kOLz5JHjxrGVAvspXabPprqy88lxmlUkkFVLHEPuAnWBB7A5TFtfhnfTa7SW7znvJqDJO0jqfiyC7YfR6++mrLDj9RPod8JY2WWODndVJDTyy1mLxbvmKFWJvby+JEiUU58TO4uzsLDFj/nfms7XuwdDrvnv3Try9HyjZ4J+Hjx5LubKlJbFzYoAdDCOt8/9brFWc9EL0uP2eREDhe1zJhJI+XgSjzo0jTrO7uQuOTY43SyWxItr9506ofGlAtm7bpva4TOnSynAHXDQ8Hr7vpd5aD9Uktrp6UkkeO5zB588deA8lMR/y02OPp/TJF1/6FEwgH/B3piy+i3PnzZcIESJIooSQDciFM/Y6ZsyYplzO6t/5/PmzMngelI87d8TrvpdkzJhBMmfOJGHDmqbgONb88uN3kJG78ubjV5lYKqFkc4oUjIx8lXJL0MN110/2N0oJwsKIQUonz53v4ez9l2XQ6VBShc2olaDvgvgW72vzVbBi7LoveZwiy/BiThIvij3YBwLfZhqQ9e7PpOXWu9I8c2wZA3k3RUfpfoF6c82adfLs2TNJkCC+0h3U0/HjxzdaV5ttQF68eCHHjh2Xs+fOiY+Pj7x8+VJt6sePHyVhQieJFCmSSUrCEKn9HNpO3N5nkqcf7KSp/SmJF/ajceoIh/kFJ3HkbWzZ/zGppIv5UYqFuSWRP77AFhun2N6JvWz/mkquvA0vpUJdk0wRXwguYp2FW/v48ZN62T5DoXGPo0aJKjFiRpcUKVNI/rx5xckp8KYp69zUf6/KSOP4sROQjfOQjaeQjVfyCn8+fPwAhZzou2wYfzehQ30V78+RZPP7pPIpdBgpH/aWJAz9AgrbsDPjx56FiiozXqeVSJ8xMzvyRbHTd1j43Pv3H5RcM1qKHSuWohn5rwX5Rq+y8H16eRQmpjQNd1bih3qJzxp2P7zeRwkjxz45yj8fnKSg3V0pGO4+eolMix4+f/ksN296wIv/rPY4CmQjZozokjxlcshGPvVe2sJ69+69XLp0SY4cPaocjG+645W8fftWHBwclMELw65+E96jUJCRB1+iyPoPySTU1y9SIewNSRDmdZBnwu+sfO0q177Gl2qRbkryUE8kNL4b6MLxvvscWv59H0+OfE4sOULfkVKRvHFugZ87f+Puh4iy9mMqiWIPo2PvLjHlTZD65pOElvOf48q29y6S3c5LSoa7bZRsBbx/buf9+/fFz89PORlRokSRaNGiKbnInz+fpE4FRgYDI1ezDciHDx/k6dOncvfefXiXHnLrlod4eNzGi/dOSpQsIYmcEhp8M8YK9fnH76XfudCSELnKEZm/SNRg0s6MhHTREBUB/1A+bwDHP/HSF7nwNLR0zBBaKiYltM7wl5+ffPruq9Tb/1aihA8v0/OFk1h2H419HMM/jx/k4dOLoPdG78E1qYu4OCcRR0cnpC4clOL43UvJBpTvvXv3IBPfZIMRCB2M8hXKSkLHhEZ7PHwmBpl3X3+VyZegJN58ka7pQksWh9AGe2Vs8jzkjZktF0NL4QT20j0dZlbjogFlg/+fkfSpU2fEIXZsyZo1M/Zbv0IhAnjgWZE99z7J8GxfJWdcO4NJNJUn++GrLL/5WVbfCyVtUoWWKs6hBDbJpMUIZMnSZeq7SjZcXfC/LuLkmECltWxBNnhvlN3ncEDv3/eEM3RbPGD0bt66pfRJjhzZJQNg9WGZYjHBgNDTd3/+RYac/SzRwZTcPX1ocY4cKsiokOew5FYomX/jk9RL/FmqJ7VHhBu4HqAc3sPMj+HHXspju1jSNb2d5In9WT5Te1DXfNchX2GEdH4Hr/biUxjpdvyj3Hn5UWbmtRfXqKGDvC8y8G65+0VmXPsqdV1DS5MUQX8+OKGh3tuyZas8ePgQ+sJRXFxcxNUlCQxIQokXL64yJoZmjcw2ILqbZd71zZs36g8VxIMHD5Tg0pMw9GaCe3D//86DmAYUTW/A7pqgEDWwmDN49wOvBbx5/VouXLgg169dUzWDjJkySarUqVUK6A6U2pBN52TeTTspnzkxJggmUR3kzEsbslj/OAGa5vxuFyQfWDh3Nk1ncIepIdfX9xmmhc6fv4D9jY1cZlSVL+YfW6yHfJONt/jzWl7B8HlCYaREpBQLHr0pssHUI6cSdtt6W86hwWo6IJqc4fDZwJQPUwKTSF8C1FRPEDF2yB1Prl+5LO6XLyNF9VnSpE2j5IPpk3vw1Nav3wAFnFgqlK+glJ6+xXvqvd0DtCgYR4v6RQNwYkUCJbghEkRF9BRw8hG4n/lnHgPJlUQag4jT0OcJeD9Ms508eUrixo0rUaJGkUg2LBu8dyo0Gr3XeEf558GDh4imY6j0G99PUxbP4ygGjpHfivPQ3aokQ00jIhR14CdCA7IM+98eyK0qqWPJKJwj6xOBfYO/ceb+Cyk6+4y4OkSTpTW/8Zf5+PjK2bNnlV6JHCWyZMueQxInRrr2e5GNuqvk3IuAF/vKkTaZJCd0RmCqhp/1Q71u5lFvGQAC0f5A6JEU0lDdpG/vuN/UHeEjhJfo0aJLpMgRISORTNprixmQgDfKFAvDT0NDIVOEhENglmKgz8qaadSQ+bB0O/SsR7C085ET3r1rp9jZ2SO0/yKRI0eSRo0bSfmKlcTL01MWrN0mU2/Yy+OorjIXBbcGWZkPNOyuiOhxAyS4+85vHaZjMM/a2otCQEVh6gtm7fsL6vpMb9LQmSMbnN7XA/TYbN6bhYYs9l4Yukiy1xrNXCsuPpFJxeLLm7PbZcvGDfh6KBU18L6q16wh9Rs0EG8oM0OK6PztOQBkdIMRaQykTF+85JyCaegih1r/3R6y0d0HSJ5kaETV38ls6PW4xyFRNvh8ykhDvkNDRkxxMHR79C+oYaqj5yIDRtTOBTeVIcO+OIqhCkbgukSPoCiRgoOH7wFBa7Hpp6UsRuZubpZZOahzZs+RkydOACgSTpimc0JqqHXbtpIzV64fx8fZIPOA3FuH2SDs7QiMOYNfeIXRFBMPesqow/dlWFFn6YAeM3MX9bOdnXn7y3uwmgEx9wGD+z43NS/mWF+FJ3oBYygDG+TDiGj1qlUyf+5cKQsUTeWqVeXWzZsydfJkhGvxZcCggYw3ZeWKlXL8bSzZFyqFZI4THjDKpAbzWHHAUTUI3Vl4wwsBzQsOxhfcsxny70wPnYOX4+l5X7JkzSYXUIOKhyJYtuzZDfl6iP+MD2gq+gB9txJGwA0GhPh4QxcNfmH0W5zxfilDkvnItnmTJX+B/FKnbl0UFp/L1EkT5R1qHyNHj5IISAWuWbM2WBQWf5vKpBGQYZkALZ4FZt74QIgZuryA/OmxzUORcLJrmkrF1EUH6dLFi+LujrkUqIddOH9e5bnzFyhg6iVD5PdIDVMJjaL5MYOc7Mb+2bkDeyDS2+SadkY8YdAvdcz60zyPgN8hks/tmLd0XgV4bcaoMqVaOpk2bbrs3bMXslRHChctKv8ePiRuM2dJkaLFpE3bNoisvgEZhqBDfigizpEYOdEcTmdQs0HI8E3Sz6nHH8ho9Cu1yPXz1ExTDufqlStyBvojL+SDkbc9HOuChQsZncEIsQbkhPIU3AHjDCsb6qfDlEH9aA0Wi/r37qNC16Ejhqs83xMgPiaMG69esCHDhkpspNmWLlkq8ZKmkh0fksrSU14yDbjxJmgIC47OgKHnRYzPzQ5jljxmeDkMnppowcDyTDnwgN/5+OGj7N+/X2ZMmybpkSt++vSJtO/YEUX0wLtgLfG7tnKNl3ipOJVwGl6qOTAgnCJn6HoOg58MtBAvkU6r6bNBXjyCdzdmjNo7FnLnwoPcvHEjZGOYJEnqKqtXrzHIgHAaXQU0ibHGwi7jJAbAQHX3fBcknB033pSLoOWZVSmFFAmmZyGoZ6UBOYUU1gjIduYsWVBbuCXtOnZQ//03LTINlEFfDZkGOC2So2UNWeUA796G9NJusPIWQFNhYI3JDzFMrCuM/rLDV6VjujDSMLuTDOg/UFKmTiXdevRQIAA6q4MGDFR1p159+kiixN9GD3D0BPuGasPxGQQ26aAYnGnUBmPY1QL0CU0Eq3B9ZEfMXRcvXJTBAweCvimd3Lh6XVq2bS25cuc2OisQYg3IIhxAR+Qqa6DbewTYWPVhqZnmuQxPrGf3npIpSybpN2CAyms/evQIBmQccpR3ZOjw4RIRh7tk6VLJDOign2NWYMevSnjkrxfBa8kVDHac89cngl1zACaNtcsZX0aXSfafmRLmHra+7/PZbqHg2K1zF3mJQmSb9m2lYqXKRguANe7tV1yTaagheKmGg22VCrcZ6D8MWUyBs2EsP7jTHO0/iOvxqeLsGFdGwoDQSyeqcI6bm+zauQtGZbTEiRtPVq8xzIAQiZUf3cJker7YMRtIFQ0HMtD4kFrHG57v7CrJJXcwchfUs1I2iPOnbNy4cUOaNmsqderVM6vvxpC9tbXPkK2gNAgSawJeOw8kq8E5g7r774fIlhT/I1FXbYkx2IERonKCZLkFl+Xu3dvSK/lryZfKSUaNHCVVkOVo3rKlSr/duH5dBsGoxANctmv3bsh6fHN09t98Jg2RxkoHGeEcmaDSa75weFhfW4tekGloejU3vcnfp5z36tFTTp88KXXr15PGTZuaBK4IsQakLzZ0DBpyxpZMoiIFffxXfJGuIMro0aWbJEFBfzA8MiqJE8ePy6jhI5TH2atvHxTu3shSoFYyZkwvxUBl0hWdohP+9ZLGWePBO3BGR2rghGf0ZouDPPEWaBPWgjyxANgyDSydmPW+8dluwrvp07OnQgaNGTdWXFxNozU360Z+05f5/MP33pW+qINMhgExdCohI9HVqJs1gbIuGh/QqZ3jJUrE8DJ2/DiJBaTVTSjcIQMHSTig6YaPGqn2ds1aw1JY3IqqSJmsRVptL1gROLDM0HHGHDPQCArlPXoW5sFxyQhKd1MX94ZOUt9eveXO3TsyfsIERRRqTj3B1Hv5nd/besVHys49D0BDfKO6tzmdsDlqZGUwnXAKMhH6nFPKkTJQGN0Q/72ndE3iK5nTp5Uhg4dIyZIlpW37dmIPZ3UTamvTp06Hc1cJNdfGKFh/cypuwWEoz2gVKYztoGAKKlplupbTVHcionKrmELVe81dz58/lwH9+snZ02dk9Ngxki1HDqPTV7yHEGtAquFFXQPump0gr2O4H1iYyRdpJFIRZ8+cVYXR6DFiyLatW+W5r6+079RJipco8Z95IKTnbrH2uhy4/Vzmw3OpiYKmPu9Foa8wPrcAuLjSgV3zYMtMJnEgmSIMz3D/c2bPll07dqqoqjUikDJlyppyqRD7nQkH7ktndH+PRmcuO3oNWZxcOQKGZySixm45YsnHI8tl/+5dymtMhGa7vbt3y/XrN6Rlq5aQl5qqkdDQIjp/vwfSEmR5no4aWt0s8eG9GtbLcRH1s1oYgxwZNDiLQFWR3MG4znr/z07E4aKFCwEM2CRv3r2Vpi2aS82atf6a6FS3FxvBLVZx9jlpCqTdbNQmDV18/wvPviCRcXaH8E476OFZY91zCAhcpwE8Uyysp5SI4asMx2A4Hy+gnKtUq6p64TYhFRo1SjTp2qObQvbpFiPo/Oh6P4P09wVEq6mA+gxsPUEnfRtwdB0GqmwBnqM4UnLmLNZPVyxbJuvXrhMfQKbrNWooDQAYCWsCM0SINCDvkTbKO+OMKlpfxuYHVkDnJhOpdOniJZk5fTpgvOcxpTC0uCZLJrXq1JZChQsr5atvoNTi0w+lB0LZaMibLsGIyywY5BJwsRg7BHO5x4OBtxdo4fsiWjGEyNGcw+d3KZgH/jkgy5F2K42O7sMHDokjkB4dUAOh1/O3LDIft4bSHVwhqfQzcCohIbwNQcK4Ft7pStBupwz3QtxmzJCTx46p2gVx8dVqVpeyZcupJitDubB0e87phN3BANsqewLpBU6r4GgqdN875fkS6ZBLksohgiwHlXtcIwrw/s9bV/9gbaxchXJy/Ohx1UvRs1cveL+GDUn6U+RnHSLBKkhVtsG0yamokxm6OF8oB5hvLyAqvET9AmhuQEQmZ5SXQvRBOH2TWJ4S/cNTaY+oY/++fTIPNTT2PREFlwFGo36D+qr+FBBiX3beedkK1N3RdlkkJwr9gS0i9JoiOj0H7r4lGHzGLIc5i/WPSYhKi5csDvDNBQWh7o1MjK7Ab8y1Q6QBuYKDZbEyLHDY2xD+BTcBji8VN4m1gs/4bxa3eLi6A9VnQJjP7ohU1kwUaVuAf6Y/4HP+UTXMpTPtUBQhMpXS9kbpJLOjfj4lYw7E0M8S6kjjaAeoI5+J6YmQCts09JkDfm7uCW9pCih3bxiPYQZOJaTnlxsom8tIIZzDi5sMQ8U+wSNjBzQhl+QWCxs2nIQBxJHLWAOy65oPhpVdU1MF3ZDb1ue96nteTiNkJFsCPGqrAR81ZxDZD9mws/vRt0LZ+NtSWGro2MKL0gWQ6rGoHRi6+G43BifZEqSyliIDQVZte38tAoxid4B6nwa/oFNYaRr7rkoZdu7UUbUIvAXyk1yAdvZ2qimPe68Pst4c0fOckw9lFeajVwDqzp5dzXoWi/V1MLrg1rN3apa6qfNmdJfmPeqg9PxvLlPlI0QaENJecyhLfueoMhU5SgfMNTd0MT8c8EUKbKQtDVXj1dcUHfREsGzWBWEjh9mzxvEKKKAOmH3OPpRmyLGOx8SzsEYONTL0nrXP6d+BpZg0Vw/RRHuMJR5d2tWg/SflfuoJJxGJilzpkuOnCCEw2Vi7br1BKCzeJVkNSs+/pBTOLjAyOwVRP/P/VPtQVC0y/YxUR1PsShui6w6pskcjMB8ORpOVIKdEZsBQB4PPy8bBKUhx9gZIow2AMWRX9k/Pz/RVH0SZS84/kZ5IgyZ5fg5d3d8MiG7pk6WAezkYhIrDMINkdPEkCgQS2AgAzjWpuuSSPEKz6Xo4F+kTmF4fs/R5hkgDMh4DpPohj90NiqMzOjMNGfwS1MYFNRN9CVJZA9Dtfgc4/aFFkmBSWRyJgjz1ruvPpPG6a6rLdQcgm4kB2fwVxXNLC0BIvt561MDqrboKJF5sGYfmTRr3oBYVw7G7z6U0PEf2amxsmD5I9gJey9gI5B062fOA0p/pD/dO2QxmCVazQDCNsBlk2s2IfH1IPj9r3vun7/N+2qJ2MAgKur+BKU7dPTEirABq9yR4v7fh/Y4d6f9OqofPWynkdlbCIcJYUjGx3Dy5Tx4GMCCGPBsbT7sADNQcYJ0+YGAOLN3pCd1DKvd3YP7chkxHUiOZpw25F1M/EyINSEs0a7nBu1hWM7VUxvzzsIGEfoZuSlAGhNegERmGRp6rj99I/iTRxDVWBFkJ5RUeaY6xoDtgZOI/xDX0d7XPmbcDu9C41xjpopwJo6jGz+C8ffbssLbVFqnJ+hhZOgaY+uCmVxprQPhE5QEd3YwaC0EVeSAvwSGxmAJdeuahQmH1KJhQRpa2PpOBeTtv+98mvJ606d1QxxyB5rseqEcZs/j9Qmg2PentJ7tgQHT9IDpq9cqICCqB/WBR5SSycfMmpLAe/xSBGPJb2yAjTaDLqFM4MTGwWUZ3kbrKg4J7NFA1ceZRcN3xhvy2pT4T8gwIYtNiyBXvQaPPceSwsycKvPhk6CYFZ0B4HRa7Jv17H41eb+AJfJFYaBZsiUJpG+LETZg+Z+i9aZ8LfAfoJbYGHzrPAAAX6UlEQVQAHDc+UDKT8QKSvyyoRdRcfyiUcSi+jy6BDmCkDYLrDTDFgHBg0GR0KLthCl4tpKT0zVr3f5/saOZcG1LhDIEnSq4jbZm3A37vP8skpKEGoj+LTp6x9B+UlUkY+tQbzar10Ow3EfJFZ4MF7XYbr8sOQGrHw9DXShNFNmwyzYCwH6kSopx4SMGvq5sm0F4QRjzpJp6UpMhyHGuT2ab0TYgzIMwHsjnoMXLZ/7TIZBbcUSeihhgQfpbdzwdBj/AYuciUcSJgCBEKZIHwb5kn/tq3DdkBTqNsi4FBnKUxvWIyyYqhUEEtYu6rA/69BZHLZkycK5w0RrCkl6YYkP8PDEogPYEACq4Dmp3GY/65D6PjhVkPLtLSAlQVhuzfn/yZFxgONRZUIWOPeMoE1MeM3VOmO9mrkR8D4lhPOdgyI1i/w8uKc49U1JDdKQrADmklepiPstFEA/IYxqggUGIcZHUUhiFxDP39ZhyMlWLMcckO+abTbEsrxBkQKnB2cLpis+dj+ldwaQtDNttQA2LItbTP/LodYPc2R5begIfGKW1s3AtqEYGVdfIpue77VtUnXGIFPTCI1zLFgGwHQqfBmqvgYIoqbpVTSsxgxttSkfSHp7sOdZDJ5ZKi7yjur9vEP/SXfNF8NxxpZzcwVkxCfawRgC7GLqarWq27KguB5moEqpwCYAfog8K3H9NjSJnWwDmReXyTiSksRjk5p56W015+4t7lG1xY36KjlBHAj6LJYspupLBsaYU4AzL/xAMUnm5JHTT3DQI6IriX05DN1gyIIbtke58hI2+3bTcVI+8cKOqyoOAOat0D31QOFLgjYJDPqXZZg+Qf0l3HFANCqvmiSLOSYZV1kOBIFVkkpSE8BQqUmUAVlkgZ9HPY3knY3h09gVEeCPALyTZplGuj5mXsYuRB5d0YDutZNPEJZoNEBlCjBaj6Wafi+ZpjQHg/unrZ4VaZQJv033oZjcwhOM2MVKqkdQBDcFpjH8Oqnw9xBqQPjMdIQN8mAzbbCP0ZwRVBDdk9zYAYsku29xmylPaCPMw9/UixIJMxIKjFudL1oQwKuURTDAOGoPdMMSBsRGOvyZkHforRNVXcoBv4mOMmvfcjeM3zQWNiLs7f9k7q198RnYu+uzxkO9CSU9EDQrCNKYtpT4I15oJ6nQ3MBZJEl6aY1aJD/JlrQFqvByAIvSBLQPZYGf0mAVsBGAVtu/JUKmMeeiOAdebgc7a0QpwBqb3ssiwHkmZHs4xo6edAIvO3UzMg5u/h77gCX+5+KDyPAKUEXyxyogW2mNMeBej3EEDAe2KeQlegnQxp1jPFgPAeGnBWzYUnsgpd5ZxVElStjA2p1aAgwmIOybLaqQJNZfyOPQ6pv3kfUV1PkK3+C6qhaeWTS5lgotOgnpOy8xZAB6ZAdX1gus+ba0DI5zYEhKAET7QCICcgrTt/c+U5cLfB0LQH6GOcEQ2Rv+LsQpQBYQdoCaQG9ns8k5MoJumjFzFl0zQDYsqu2cZ3RgJl0wtNpRNBSxIU0oZKoBaU9FrUGdbWSSOlkCYyBABhqgFhAZcMzd3Aw0SerqDmPVwAHxIn1LliHMAaoHFMpTGxjROxjbu4g3Ql04KXQP8xA2nBoiBGtMYy14CQ1p2s4vVRTxkMuHG08D9Tzr/5+Fnmgg2jGxoXe8HxGYCeFltaIcqAcOhOpUWXxevVe3D1pwddtmW4fTQDYksiady9TAXWvx3SP0NQKCUXWWCLuP4c006LOxAtl5FWSkZ+IwN+ylQDQqqLumhyzItxpQtAjhgUJ9YpMB3kAs6/IPoBtjRKHyy02IDb/us/Qq6qtutvyD2ksjixMh/melhjmWtA9iDFVhe0KbnQyzQXadiANV3CkScgZT8KkOShoFPqmN/8aYSW3IcQZUD4otXHSxk9fBhZgtSAixEDe4LaNM2AWFKkfu21FiA33RicQl1AZkkPLrC0FClGCiF6jQws/yEULA2lvzHVgLAwzsl2bCI81TZLkJxYLJLmBxqnIvoN1gNerC3zd4CzOlqCUdsXdTLOV8kOyL01lrkG5AoipJLoMo8d0Q4d7xkkbgDmX9b5SNg699Qj9J24qrqvLa0QZUA2owjaCrj/vODAGg+PM0E0w0eGagbElsTOcvfC2Q2NgcvnYLGRaBgLbGzpeqBxmiKPXBK9H9MwP8RQllxTDQjTrXkQ8Zx+8FrOts8iaREt66vXsY6z8dJTqYLJec3R/8FRuNoyfwcuP/IDIeI1NYl0DuahZ7ASf5S5BoT9KlnB/OuDXiAOIXMMoNPYI9R1yy3ZgK51N/Q6VU5v+Ohm83cx+CuEKAPC+cOkJiCMjlTZMYLhPgr+8b99QotADN0p2/sch/q0gGHICvbbcSC8TBwjgt6bVNMLD96TgZAbDp8KjLgu4JdNNSCEgDZDZLQYPQREiPHFD6un6VQ3V5sU8P1RKzGUlt72TsK27oh1pborrmK+Smg1zjY4lgJT795cA0I5yaF6QV7p7QXhMKnm4Nw7AhjxItT5ilmplmPq84coAzIITTyD999DZ6kLoHSBs1cauxmaATF2x2zn88fvYZwvct0sUk+Bh5Y+vv66WDVwF62Dp7+1YTpVUCWG35BlsgHBxXWMru1yJZB+QNnoM1pseBv6v/auNSqLag2/aZApBlpkIiFkGIICqStNQSs1tcIUlDTFS4glJyXz0qnV5dTytBQ9qNEd8RqKRmiJaN41K0Q0RcVETS3U8KgpeQEyO887RZFH45tv9sB8fe/2j2sxs2fPM/PNs/d+3/d5kEXGftdTIUn/hMW2KGzByIrHbD9aQo+lFZBX/RsQgwrQ5U+v536MEghfKxKS8x8jZrY2NhSxGvc/eQqxmVR/JH/sO1UKuZNAJdJNeu6vqmMdikCexIzufegYfTQkmHpDgbUqkbqqbr7i70IgtiJlveO4Gj3h4/34gV2k1KiAq5rtXEAKZgdIpe/FsWwQ5K9DzdReAmGk2Pc6Cj/+e7A6WozsqqvVnRxFkHc0FGPzINr3Hqrpexh0m7PeE6qZEbEFQyRkawLwrNnhUdV295V3o4JAnoEtxDsokE7BBIir2yvrs7EXCCvx/lD6M62MbWW5FG+HIpDec8DUUMHdGN+aOinMqhACqZkfuYqrsjfDs1kHKB0xjoX4UPRGte6VbTecKx+ZvYvcb6ytWSDfpsPtzwiBcDV0KKRTOM6xG9IpV1NbPQjii/5gD/S8LsOJsAW1amwdrwcVz6em+sg5cpYiYDrXxssNjqItrql0a3R8Kghk2m/2FBOwtTr2ipRvnmC0Tc4jNzij5uC7d3MVsjhG70fv+Q5DIKfPl1MUlCu3f3+e1sGoR1UNCAMmBKL3tbHW8c9BziQRMY5ZA4KumqXChVjxmOX1wuw+CbIWemJnRgiEq4jve2875ULraAsyv0KxErly1bwH5BYGwb67YBGQjRReFdI81no6NTMaVmruiQlnGLKv0kDMVXnF2DtKFQSSiUSQ4ahlehS1Saz6W7kW5BB02+5M3EKt4F+zPaGtsl0Xe+/3yvMchkAKkFURg6AYZ7ekDwz8SxN6veAIgehFzFrHc9HeeGxjTYbA3YSr+D6whPvkzUU0CaZCTyIeYWsAvWJyoceRsDIyvPIYDenvVDgnvtvLnwZexTcmFzGcdm9s0/SvVg4PsRawDjyazYfOUDds/XTDTkUabGCrUkS291ZVEMjW73i1tAeTiDq0FGncFRMc9q/ZVoT3482vqBuyBz+Ns9774TAEsg76+yOQbdMCe5rJffzJ9xrZNva8CEIg9qBmnXO4FiQOcYRRkHp4BSRxZTUvBymXoAJ9NX6A98NzvLaNAXSjBMIppCk5v4p/susc26re6PKH7zUTTBbSM3vPydfUYmdFt7AOqA4+Elbt7pK6kx5BwsRCEEgd1P+Y0VQQCKsxt0EmFr8vO0a3/b1GiVewrOzMOlgx0HmbbcH3w2EIZAFmcQnLD1AUzOdfw0dCpSuXEIgZP63q6zMLZl9PgUDCIJ+eCJdBH48/fBVKkGfPSqYcQN+BeoxrSWZfa7RGtrA4RZMtdHvAQrc1tiCWYYvKDa5yFY2D+6kInj4LgnkBMhWvwgtEmhoE1kOhuUvKDorE9yL98SC63iTfHhUEwrsqrWdshUrCRdo/7p7fM8bYuI4nRwmQOhnXsYkuX3c1KFbdi8MQSNLG7+h5eCa81NmHRod726SkWvXt/3qEEIitSFnzOE7ljddSeWvRW9A9alUplXdbUQmECgvIEzVDmYNb/l+hVlV3ZIRAuG8O8gcm5dJP+EjsG9vuT0HQM/gbWyWzGutUkEccVlDS1CCwFjsWXeFbHo2spkXQPjOrqSAQ3qp6FCZ5K5C1tx7bmB0g686rZK4RmoL3gx00X4dMzyioLVitOQyBTECmzRTsdfMyPwbbAbbm8dsCuBCILShZ9xj23+BaEDaWmoOiscq6R+xnr61coYg7CVtIeoPURgmE1VR7zNxJX6JQbD2SP1iqvSKQXlxSTk9/UkhfQDE2BdXxD7UQHxBVb9lqSLA/+O5X9Di+FWlYgZjVVBAIr1RH4z3giUQKdLvYlsC1di3NAXUsvnssAPoeJkb9IHVjteYwBDIY8tjzUYmeDRvbnop/aEIgVnst9Y3nLOQeRiGIvgiFghnY72b59Io2HuQx7fOjNB06QlykpyeAXrE6tTeIzufzPvaLKw/SDLy7r8Hedkwnn99VgJn4HpyZr+X9LxkcpHt7TR9KznU0KxT0eGc7xWBVN69/oGk3r4pAWDDxZRSUcirveCSCcKyMV6+DFhbQVvjKsIJ0OLxIrNYcgkBYkZKluFdBxn1tbAiFKQZSCMRqr6X+8fwz+yBNhlLB21ihsq8CN35vHkIg9XNscX2BHHp7jJqMrkA4MLrhwBnqhaK24EZ1aRV8bNxQNa/5bLPTHGbJ3RHoXYHVyXUqzG30Q/e3PCMbweeHIWY5DNvdZiYnKCEQPIEl8I4ZBsmSSEyOk7Ha4FjZSVShd4EA6CnE8TaMCKE7dRTAVtdDdQgC4VzoIel7IeNervkl3I18epVNCEQlmjXTFwcb/7FsP7JV4KvwW5LFmv2nKTZjHzX1uEFTb64cXLd1lEYJhK/DiqrhkGs/fKZUsyFgZVje337niyJ6Abarz2FveyKEIKWpQ2AZtn16ob4mrrM3vQ+FArOaCgLhseUhVtdj1i4K8qxLnwxrpWUSFqMKPXD6VnJHEeEO1IDY4qBp1n1eq19LE0g5qnNd4NKWizzpOCiuNrjRBS9Dc+VLfSGQ6n7t1F+P6ylGIg7CClcpUF/lScaLK1D/AR+FV7s0pac72pd4oYJA+D2eiELHRHiXjOvgTa9Cdp5F8oZnfK0VGaZAwiTCgGOeejQdv8dMzOijZu+keCTdcGKFWU0VgXAqbwhUCzhHLx+qBQ2R9KEVEU5BEWEjaxYRMqaWJhDOgWbP81PYCxyDYBLLuE/sfgd51nNFXnctZYF0IRCzfl7V1y/LYo9EndAifDiWItuKVUsfTs3X3Cs3Im5mr6GQCgLhbSzOBrsvJZ+aIcV401Oh8D8vp/Zvf0VN4P+wBpk3euRVqg9Vx7oS19VwarQWU0I8rD8k8hMgDTIdRZxsKMapvCqTbxgdVQTCsbLQ6blUeLqUDiKV18u9Dm0+dJa6Ygv2Yf8GtHQob3Fa73lYmkBYvn3BjmIs4WrTBmjbREPnKNzXg75Bts2we267pnS3XpiFQPQiZs3jWbL95VXf0L+whXUHpEFeQEzEDx/suQbUWFUQiPahKbtEvaDHlYNsrIlQ5uWPwTiskKKDPbU6BWnGEeBV3Yc7T2heLxyAjoc8yNMISscieaLwxEVqjzohnwZ/1AgZv6I6AuFakD6QXlmFuNjqJ1iqqT7x949LF8YjpvcaMggtyB/WXoFsQDFQAjSM8o/9yIslqgsxvHIwdSJ8gePaN/lTUZaRl0EIxAh61jl3DXL/+X35DpbHtfG+lOCjPQPGY+zi9lee5H91B6oIhD8QK1B1Pgj2uz8ia4wD5o1vcqWZrMALGRNpxhG4hG/DEoitPoFgdGM3Vzpw8iJcS0EY+PKy4RgrAahuqlYgvHqagIzBt1BYOgMZg5yyyzI4y/b9QLMjm1vOSKoCR0uvQNgDPT6zkD6B2Bgqawi1/nRvMw/lBjFCIKp/VjXTH28VvbLyEE1FcLocWxYR+DBPgz6WnwHrY1UEwojw9spkLhzM+16bTfZreQvG5285gbyaeXpqrloAi9i+yHjbCydIDWR8M1piNv8GcL4felKqmyoC4Xc3DWobIzEBGgCyewWFgw9AQeHHssuwYA6lO2+uq3roSvqzNIHwHTILJ6MKHb8yqoW9TS4U40IbFyYURU0jkIXpFBIcTN27P6ioV+mmJhA4jMAj/xCPIwVyGIrIOJhuxDemqAj+M5mZdIefL0VERBi+pXNYFX2GvW2eLXdExbHewkbDA/ibd8AGXZM2HKEpa49o3wxuw1ELkvxocy1uqrr9SiDLqLi4mJ4d84zd3SOrmwqgzByOzDFv2A3MRLJQR/yfs7JyR7XRCgut2CxPIG9CRfU57GtfQJZCdJvGmtZRU8X7mIcPH6H0RYspOLglPdSzpxWfk4yphhDgd2PJ0qXk69uU+vTuXUOjkMvqQWA5tNEi5ubTL1iFeuMDnPTblpCePmw99szZMyCQLDpR/F8aN3YM6nt+sbueh1eooTPy6NAPpTQaqtFvIAYy9O5b6d3IAF0CoLaOXcVxlieQTBgFcQZWEYxVMlCN2QfLO5XtbEkJ5eRsodWr15C3dxOK7teXvLxEk0glxo7a17lz52jz5s9p46ZN5OnpSX2jIsnHx8dRb8dpxs3bWEMX76WtCEhHwIt+NrzEb0bmpupWXl5Ou3bvppUrPqUSfEdiYgZRUFALEIh9qwXOxBqJ+M2svOMa+dV1c6G5fQMoqtWtlszAYjwtTyCsaRO/dD/d7u5Kb0MnJuDWekrfg6NHj9H8D9IoNzeX6tWrRyPihlN4eJjSa0hnjonA8ePHac6cebQ1L48aNmxIj0X3o27dujrmzTjRqNkG9iVk4M1EQPr5+33odROC5wznufPnaVlWFmV8+BFdf70LPfDAffTkiDi7keY4yCoEzfsu2EPnkUXWFanomUNamuZlYvdAK51oeQJha8qnkI7XDym8YzrdrlvLqCqQeBaxOCODPsrIpMDAQBoyOIaaN/ev6jT5uxMgcOHCBcrKzqYFaQsxswyiwZhh3nWXeUVpTgBptdwiS9gkY+s7CckUyRH+WszUjHb58mXatm07TZmahAmGB1aoUdS1axdDl7rw08/0b6gTHMeW/cC7b0Pg38NQDM/QYGw42fIEsguBpUnrvqVBrRtRT5PSHdeuW0+pqbMoLKwjDRjQnxp4WE+0zIZnKYeYgMCXX+ZQ0rTp1KHDvRQ3PJbc3NxMuIp0qRoB9g/6D1QI5vcPoMBG5j2zI0e+pcmJiVS//k3YvYilZs3UpwqrxkZlf5YnkAMwAtoCuev2Td2pGYrDzGh79hTQvHnzqd297bRAqRULdsy4b+mzagQKCwtpJiYXIaEhNHDAgKpPkCMsgcB6eGukowiZhQldkb1pVjt16hTNxbejrKyMxjyTQHXqqC1UNGvcqvq1PIGchuTDRXgq3IIgGEsUmNFOnjxJy5dna7MHXoVIEwQqEDhx4gStXbeOvBp7UefOnQQYB0GA1So4mP5IJWl/M4ZeWlpKmzZ9RkePHaNhQ4eYcQlL92l5AuHA0nX8z8RlwaVLl+jrffu0ILqfr6+lH5gMrnoR4A8Eb1O4urqQn59f9V5crmY3AmzkxfpX7pA1MbNx2i4nWxwDgbRt29bMS1myb8sTSHWhxh8Kbs62BK0ufB35OmVl5cjvvyzvhiM/RBPHzhNQ3sLiCaizNSEQZ3vicr+CgCAgCChCQAhEEZDSjSAgCAgCzoaAEIizPXG5X0FAEBAEFCEgBKIISOlGEBAEBAFnQ0AIxNmeuNyvICAICAKKEBACUQSkdCMICAKCgLMhIATibE9c7lcQEAQEAUUICIEoAlK6EQQEAUHA2RAQAnG2Jy73KwgIAoKAIgSEQBQBKd0IAoKAIOBsCAiBONsTl/sVBAQBQUARAkIgioCUbgQBQUAQcDYEhECc7YnL/QoCgoAgoAiB/wEI0sGwoqtruAAAAABJRU5ErkJggg=="}}},{"cell_type":"code","source":"from sklearn.preprocessing import PolynomialFeatures\nfrom sklearn.linear_model import LinearRegression\nfrom scipy.optimize import curve_fit\n\ndata = btc_days_df\n\n# create our independent variable; our X values are the timesteps, \n# but we can make it just as well an array from 0 to ..., since the \n# actual value does not matter\nX = [i for i in range(0, len(data))]\nX = np.reshape(X, (len(X), -1))\n\ny = data.values # dependent variable is the closing price\n\npf = PolynomialFeatures(degree=3) # a cubic polynomial model\nXp = pf.fit_transform(X)          # transform X into a quadratic form (each row i will be: x[i]^0 x[i]^1 x[i]^2 x[i]^3) \n\n# fit the quadratic model through ordinary least squares Linear Regression\nmd2 = LinearRegression() \nmd2.fit(Xp, y)\n\ntrendp = md2.predict(Xp)\n\nplt.plot(X, y)\nplt.plot(X, trendp)\nplt.legend(['data', 'polynomial trend'])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:03.809194Z","iopub.execute_input":"2022-01-24T21:23:03.809508Z","iopub.status.idle":"2022-01-24T21:23:04.140812Z","shell.execute_reply.started":"2022-01-24T21:23:03.809469Z","shell.execute_reply":"2022-01-24T21:23:04.139632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Looks like a close fit.  \n\nLet's see the detrended data.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(8,7))\n\ndetrpoly = [y[i] - trendp[i] for i in range(0, len(y))]\n\nplt.subplot(211)    \nplt.plot(X, detrpoly)\nplt.title('polynomially detrended data')\n\n# correlogram\nplt.subplot(212)    \nautocorrelation_plot(detrpoly)\nplt.title('Autocorrelation plot for a generated random walk time series')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:04.142404Z","iopub.execute_input":"2022-01-24T21:23:04.142765Z","iopub.status.idle":"2022-01-24T21:23:04.524484Z","shell.execute_reply.started":"2022-01-24T21:23:04.14272Z","shell.execute_reply":"2022-01-24T21:23:04.523571Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Detrended data doesn't seem to have a trend we can spot with the naked-eye. Let's see the prediction error.","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import mean_squared_error, r2_score\n\nr2 = r2_score(y, trendp)\nrmse = np.sqrt(mean_squared_error(y, trendp))\nprint('r2:', r2)\nprint('rmse', rmse)","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:04.525781Z","iopub.execute_input":"2022-01-24T21:23:04.526024Z","iopub.status.idle":"2022-01-24T21:23:04.535286Z","shell.execute_reply.started":"2022-01-24T21:23:04.525997Z","shell.execute_reply":"2022-01-24T21:23:04.534105Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Remember the [persistence model](#persistence_model) from earlier ? It had better RMSE (1045). But of course we don't expect the trend to be a good prediction model for this problem. We just model it so we can remove it from the data and then find a good model on the detrended data. ","metadata":{}},{"cell_type":"markdown","source":"## 8.5 Removing seasonality   \n\n*Seasonality* is a short-term pattern that repeats itself at a fixed frequency. A one-time cycle is just that, a cycle, not seasonality. \n\nIts effect:  \n- may obscure the pattern in our data (and we remove it)\n- can be picked up by our modelling algorithm (we can use it as extra feature)  \nBoth are valid approaches.  \n\nThere are many method to deseasonalize a dataset. Not all can be used in forecasting problems. Some can only be used in analysis of past data. Something to keep in mind for other type of problems.  \n\nJust as with detrending, we can do:  \n- differencing  \n- modelling  \n\n**Differencing**\n\nIn [section 8.3](#8.3-Time-series-decomposition) we saw something that looked like weekly seasonality (easy to spot in the shorter time window plot there).  \n\nSo I'll subtract from each day the closing price of the day 1 week before.","metadata":{}},{"cell_type":"code","source":"tmp = btc_days_df\n\nlag_df = pd.concat([tmp.shift(7, axis = 0), tmp], axis=1)[7:] # remove the first 7 non differenced rows\n\nlag_df.columns = ['t', 't+7']\n\ndiff_data = lag_df['t+7'] - lag_df['t']\n\ndiff_data.plot()\nplt.title('Line plot of deseasonalized daily close price using differencing')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:04.537172Z","iopub.execute_input":"2022-01-24T21:23:04.537489Z","iopub.status.idle":"2022-01-24T21:23:04.880388Z","shell.execute_reply.started":"2022-01-24T21:23:04.537445Z","shell.execute_reply":"2022-01-24T21:23:04.879227Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Modelling seasonality**\n\nLike with detrending, we can model the seasonality and subtract it from the data.  \n\nSeasonality has the shape of the sine, which can be modelled as a 4th degree polynomial:  \ny = a * x + b * x^2 + c * x^3 + d * x^4 + e","metadata":{}},{"cell_type":"code","source":"groups = btc_df.groupby(pd.Grouper(freq='A'))\n\ncount = len(groups)\n\nplt.figure(figsize=(12,8))\n\nfor i, (name, group) in enumerate(groups):\n    plt.subplot(count*100 + 10 + i+1)\n    plt.plot(group.index.values, group.Close.values)\nplt.show","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:04.882076Z","iopub.execute_input":"2022-01-24T21:23:04.882505Z","iopub.status.idle":"2022-01-24T21:23:06.142921Z","shell.execute_reply.started":"2022-01-24T21:23:04.882456Z","shell.execute_reply":"2022-01-24T21:23:06.141738Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It's hard to see if there's a seasonal componentt, constistent across all years.\n\nUsing the daily closing price, I'm going to do the following:\n- break the dataset into four years\n- try to fit seasonality models (4th degree polynomials) of 3 different frequencies: weekly, monthly, trimestrial","metadata":{}},{"cell_type":"code","source":"groups = btc_days_df.groupby(pd.Grouper(freq='A'))\ndays_per_year = []\nfor name, group in groups:\n    days_per_year.append(len(group.values))\n\nplt.figure(figsize=(12,20))\n\nstart = 0\nyears = [2018, 2019, 2020, 2021]\ncount = len(days_per_year)\n\nlegend = ['Detrended series', 'Fitted weekly seasonality', 'Fitted monthly seasonality', 'Fitted trimester seasonality']\n\nfor i, days in enumerate(days_per_year):\n    series = detrpoly[start:start+days]\n    \n    plt.subplot(count*100 + 10 + i+1)\n    plt.title(f'Year {years[i]}')\n    \n    plt.plot(series)\n    \n    intervals = [7, 30, 90]\n    cols = ['red', 'green', 'magenta']\n\n    for cnt, inter in enumerate(intervals):\n        X = [i%inter for i in range(0, len(series))] # let's try to model a weekly seasonality as a sinusoid\n        y = series\n\n        degree = 4\n\n        coef = np.polyfit(X, y, degree)\n\n        # create curve\n        curve = list()\n        for i in range(len(X)):\n            value = coef[-1]\n            for d in range(degree):\n                value += X[i]**(degree-d) * coef[d]\n            curve.append(value)\n        \n        # plot curve over original data\n        plt.plot(curve, color=cols[cnt], linewidth=1)\n        plt.legend(legend)\n    start = start+days\nplt.show","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:06.144568Z","iopub.execute_input":"2022-01-24T21:23:06.145506Z","iopub.status.idle":"2022-01-24T21:23:07.039958Z","shell.execute_reply.started":"2022-01-24T21:23:06.145452Z","shell.execute_reply":"2022-01-24T21:23:07.039214Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This doesn't look like anything to me. The seasonality amplitude is so small, I don't think it matters if we remove it or not or if we add it as an egineered feature.","metadata":{}},{"cell_type":"markdown","source":"## 9. Model evaluation\n\nIn ML uspervised learning problems we usually use a train-test split to separate our dataset into data to use for training and data to use for testing.  \n\nHow do we do this with time series data, where we can't just do a random split, since we have to respect the order of the data ?\n\nApparently, there are three common method to split data for backtesting:\n1. Train-test split respecting the temporal order  \n2. Multiple train-test split respecting the temporal order  \n3. Walk-forward validation, where we update the model every time we reveice a new piece of data\n","metadata":{}},{"cell_type":"markdown","source":"1. Train-test split respecting the temporal order","metadata":{}},{"cell_type":"code","source":"X = btc_days_df.values\n\ntrain_size = int(len(X) * 0.7)\ntrain, test = X[0:train_size], X[train_size:len(X)]\n\nplt.plot(train)\nplt.plot([None for i in train] + [x for x in test])\n\nplt.title('70-30% train-test splits')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:07.041115Z","iopub.execute_input":"2022-01-24T21:23:07.041796Z","iopub.status.idle":"2022-01-24T21:23:07.24731Z","shell.execute_reply.started":"2022-01-24T21:23:07.041751Z","shell.execute_reply":"2022-01-24T21:23:07.246351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I don't think training on 2018-2020 data and predicting on 2021 data is a good idea ^^","metadata":{}},{"cell_type":"markdown","source":"2. Multiple train-test split respecting the temporal order\n\nsklearn provides the TimeSeriesSplit method.  \n\nWe decide on the number of splits first. Let's say we want 3 splits. TimeSeriesSplit will divide the data into 4 batches of equal size, respecting the temporal order. And the 4 batches are used for training different models in this way:\n\n![image.png](attachment:1c1773a4-d37a-4c56-b4f8-ec3911ddb153.png)\n\nThe size of the test set is contastant, while the size of the training set keep increasing.\n\nStill doesn't solve the problem I noticed with the previous train-test split method.  \n\nFurthermore, in practice, time series models are retrained as new data comes in. ","metadata":{},"attachments":{"1c1773a4-d37a-4c56-b4f8-ec3911ddb153.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAeYAAADDCAYAAABXhbL+AAAgAElEQVR4Xu2dC3wU1b3Hf7ub94tHIgQIco0CkUhULiKNiEJFNFKLYEGqPASkD4PoR7xFoRYtFG7lCtQUBAED+AC0AalAQdBWMZWHqAgEAeMDCAECkhd5J3fO0F03IcnubGY2ZzK/8eOHZPec/5zz/Q9895w5Z9ZWUFBQAx4kQAIkQAIkQAJSELBRzFLkgY0gARIgARIgAZUAxcwLgQRIgARIgAQkIkAxS5QMNoUESIAESIAEKGZeAyRAAiRAAiQgEQGKWaJksCkkQAIkQAIkQDHzGiABEiABEiABiQhQzBIlg00hARIgARIgAYqZ1wAJkAAJkAAJSESAYpYoGWwKCZAACZAACVDMvAZIgARIgARIQCICFLNEyWBTSIAESIAESIBi5jVAAiRAAiRAAhIRoJglSgabQgIkQAIkQAIUM68BEiABEiABEpCIAMUsUTLYFBIgARIgARKgmHkNkAAJkAAJkIBEBChmiZLBppAACZAACZCAKcWcmZmJ2NhYxMfHa8qgqLdv3z5ERkbinnvuQUxMjKb6LEwCJEACJEACjRHIzs5Gbm4ukpOTfQZlSjGPGzcOkyZN0tTxuXPnYvv27Rg6dChOnTqFXbt2Yc2aNZSzz5cOK5IACZAACdQlkJaWpr6UmprqMxzTiVl0Wvw/fPhwdOjQASkpKR5HzuITjCi3efNmV9np06erI+dp06b5DI8VSYAESIAESMBJYOvWrZgxYwYSExPRu3dv9OrVS9MA0hmnRYhZTBuIKer6DvGpJSMjAxs3bkR6erqriAD4wgsvqKNoHiRAAiRAAiTQVALCNYsWLUJcXJy1xCzAJSQkYMWKFa5PIs57xw2JWYywxfT17NmzXUVEnfHjx+Pw4cNNzQXrkwAJkAAJkIBKQNxqFaNlS01l1ydmT9dDfXP+FLMnanyfBEiABEhAKwGK+T+r3jhi1nrpsDwJkAAJkIARBChmL8XMe8xGXH6MSQIkQAItm0B+fr66QLhnz5548MEH0apVK48dppi93CfW0KpsQdj9vrNYIJaTk+MRPAuQAAmQAAlYg8CCBQtcXhBiHjJkiCrpfv361QvAsmIWW52ysrIwcOBAr5eji/vMGzZscO1j3rZtG9auXVtrq5WQcklJiTWuNvaSBEiABEjAIwHxMKro6Gh06dJFfTBVUlKSOoJu6BAztHPmzMHYsWPRsWNHDBs2zOM56hYw3XYp0YG8vDxs2rQJhYWF6N+/vwrKm4NP/vKGEsuQAAmQAAk4CXz55ZeNirg+UkLOYqDXtWtXDB48WDNMU4pZcy9ZgQRIgARIgARMQoBiNkmi2EwSIAESIAFrEKCYrZFn9pIESIAESMAkBChmkySKzSQBEiABErAGAYrZGnlmL0mABEiABExCgGI2SaLYTBIgARIgAWsQoJitkWf2kgRIgARIwCQEKGaTJIrNJAESIAESsAYBitkaeWYvSYAESIAETEKAYjZJothMEiABEiABaxCgmK2RZ/aSBEiABEjAJAQoZpMkis0kARIgARKwBgGK2Rp5Zi9JgARIgARMQoBiNkmi2EwSIAESIAFrEKCYrZFn9pIESIAESMAkBChmkySKzSQBEiABErAGAYrZGnlmL0mABEiABExCgGI2SaLYTBIgARIgAWsQoJitkWf2kgRIgARIwCQEKGaTJIrNJAESIAESsAYBitkaeWYvSYAESIAETEKAYjZJothMEiABEiABaxCgmK2RZ/aSBEiABEjAJAQoZpMkis0kARIgARKwBgGK2Rp5Zi9JgARIgARMQoBidktUUYVJsmaBZgY5Llqgl+boYllxlTkaylaSgB8JREZGGnY2itkN7aliYOzWagTaDePNwF4QqKwG+sbtQ3y7lxFgD/KiBosYRaC6pgqfvwpkf3EGRv5DZFT7W1LckydPIioqinmQIKlZWVnYs2ePYS2hmOuI+ZH3qvFDmWG8GdhLArd12YdrYl9CTY1iaR7NSuDQylCMTJmIPn36NGs7rH7yOXPmYPfu3Vi/fr3VUTR7/8eMGYNVq1YZ1g6KmWI27OJqSmCKuSn09K1LMevL09doFLOv5PSvRzHrz7TBiGIqmyNmPwJv5FQUsxx5EK2gmOXIBcUsRx5EKyhmP+aCYvYjbA+nopjlyQXFLEcuKGY58kAx+zkPFLOfgXPELA/wRlpCMcuRJopZjjxQzH7OA8XsZ+AUszzAKWbpc0Exy5OiFjWVvX//fnTs2BExMTHyEHZriQxijouyIzHG0SCf97+tQIUPC5XtNuDFn4bh+Y9LcL6kRkr+7o2SbSq7S+QNCLaHN8jtSP7H0jP1tYEcMftKTt96FLO+PJsSTTox5+Xl4ciRI2qfunXrpkmy48aNw6RJk5CcnAwRZ+rUqXj22WcRHx+vxhPiTkpK8sgrOzsbRUVFXpX1GEwyMffpEICfd7u0dzcmzIafdArA34/++OST53eWoKxKu1jF3uwPR0fhoY3F+PoH+R8YIZuYk2MfRGTgpQ+UsWHd0SqoPb668KHr6tl6fKGWS+2ysrfEjsbHuaubFMOoyhSzUWS1xaWYtfEysrRUYp47dy7S09PRt29fFBQU4NChQ1i4cCEGDx7sFYO6Yn7uuefwxBNPqGIWUn7xxRfV+I0dW7duxYwZM5CYmOixrFeNkkzM7m1OVqT88t3hSFqWr7Urpi8vm5jdgd7Z+TF0a3UL0g6M1I3z40kbsGD/UN3i6RmIYtaTpu+xKGbf2eldUxoxZ2ZmYsqUKdiyZYtrlCxGverITpmaFj/n5OSoo1ghWTGirTuidhezqCdiOkfP8+fPh3iaihhFR0RE1DsaXrlyJdLS0tQPBoWFhZYV87KUcOw8Uak+oexXN4bgizOVmLCpGF1a2fHH/mFIvMKhvrfpWAVm/OsinAPs/RNbYdjfinBMGTEPuSZQHY0fzKtC6n+HICTAhkwl5hPbi32aKtf7wjermHtGD0b/Dg/DbnPgyIWdcB9Jd2vdDwM6TkJoQBSKKs5j95m3sP/cFvyqxyr1Nechm6ApZr2vbt/iUcy+cTOiljRizsjIwGuvvQbxZ32HkOzSpUtdb4kR9fHjx7F8+XKXZOuKOSEhAYcPH1ZjLlq0CPn5+epIWLw+bdq0y04jxHzPPfdgzZo12Lt3r6XF3DHCjm/yq/H6wTKcU+4Zf3WuShXzje0DcOR8lfIoS2D5PRH4v10lWHOoXGVZV8x/uDUMb2eV4R1lqjw80IYFg8KwO6cST+5o/udUm1HMd8Q9iviom7Dpuz9DPMpyUOfJKK9S+B97CtEhXfBQtwXY8M3zuFh5AVHKVHigLRiHL/xLzQ1HzEb889myYlLM8uRTGjGL+7ojR45Eamoqxo4dexkhIebx48djxYoV6ihYHEKkq1evxvbt29XfGxKzeE+MhL2VrZayWlIpw+Iv9/Y2NJUtRsxXt3Hgp28UKAJouIfPJIciNtyGx967JNq6Yv59v1DcnF7gCvDIDcG4PyEIg9cUasFmSFmzidlhC0Rqz7VYnvWIMho+pzIRz/n+beIa/OXLYYiLuA7D4/+IJQfHoLTqcr6exFxSWInQyABDWHsKyhGzJ0L+eZ9i9g9nb84ijZjVf9iVKeqZM2eqI+E777wTEyZMcC3ccopZjIDdDzH6XbdunTpqppi9SfmPZRoTc25xtTJNXVIroBhFT7kpBDe0d6CD8rOQ9kfHKzF5m/JIs3rEPC5JEXFGkSvGfd2D8Jsbg3FnM4u5/Nz3uLHVLvS9ZbOUz8qu7x5zdMiVGN3tL/Um2Dk1PSguFQltblNGzPnY+O1snC35xlXeo5gLFDFHUcza/ga1rNIUszz5lErMTixC0GvXrsW2bdvUxV9ihOycyq67eEuI2TmKppi1XViNifmAcm94we5SV8DYcDu2jYrEnz8pVVZxlyO/rAbpQ8JRqMxiNyTmBxODMeod+cQsOmW2EXOb4E4Y2/2vWHzwQWXV/KUPQvUddlsAerQZgFs7jMXpkmxkZD+rFqOYtf3dsGJpilmerEspZicesUpbfBXZSy+9pIpZbH1yTls7y1DMvl9MWsQ8UZmGfqx3SK0V3O//Mkpd3EUx+56D+mrWN2K22ez4TeLr2HFica1tVA2d2aEI+jeJbygru0e4xLzowCiUV9eeBXHWL+GIWd8kmjAaxSxP0qQWs/u9XudU9s6dO12rtsXIesSIEXC+xhGztgtLi5hbh9jw4UNR2PJ1hbpPeZQyGhbHgbMUszbqnks3tF0qse1PcYcyXX2q+DC+LdyHDmHd0DG8hzqKvjqqD5Ki78bJ4kPqCW6MGYLCijy8cfRJ9ffJPd/GudLvcCz/E5woPoCc4qxaDaGYPeelpZegmOXJsDRiFiunxXaoXr16qXSOHj2qLtiaNWuWuo/ZuZ2qc+fO6pYnsZ1pyZIl6ne4OldYNyZmZ30RT3whu3MBWX2psMrir/jWdjx0XTDEQ0Xcj9HKazlF1dihPAXM/RBPDBvSNQjKzid8rmyh+jS3CrdfGeBalf2ssthr8b4ynL1YjaR2DoiHmSz74scvnxb3pm+7MhAL9/w4Rd5cfxVknsru0WYg2oVejX/mvHIZnqigdujZdjBCHBHKIq8iZfT8EfJKv0WQPRQ3XvEzRAREq3VOlxzDgfPvueqL9/u2fwAOeyD2nX0H+eWna8WuKK1GYIiy1L4ZDi7+agbo9ZySYpYjD6IV0ojZeV9ZTF2Lo1OnTurWJadAnfeYxZO93nzzTVXMAwYMqLWCW0x9p6Sk1No+5X5P2ilcIWYxPd7QIT4kiKeP1belqimpk21VdlP6Yva6MovZ7Gy1tp9i1krMmPIUszFcfYkqjZg9Nb6hxV+e6sn0PsUsTzYoZnlyQTHLkQuKWY48SDVi9oSEYvZEiO9rIUAxa6FlbFmK2Vi+3kanmL0lZXw504yYxVT35s2bdZ9eNh7xj2fgiNmftBs/F8UsTy4oZjlyQTHLkQdTjZjlQeZ7Syhm39npXZNi1puo7/EoZt/Z6VmTYtaTZtNimWbE3LRuylGbYpYjD6IVFLM8uaCY5cgFxSxHHjhi9nMeKGY/A2/kdBSzPLmgmOXIBcUsRx4oZj/ngWL2M3CKWR7gjbSEYpYjTRSzHHmgmP2cByHmMf+oluL7iP3cdalOJ74wq1/nfbgmNg025T8ezUegBjU4kB6MoLK26tex8mg+Ajt27FCfDzF06NDmawTPrBLYsGED1q9fbxgNm/K9yY18caBh55UycJHyIK0PTxCHDMmJDjuHtuG1H0spQ7us2IacL0txPvfHLzuxIgNZ+hwYGIiKitpP/JOlbVZrx3333WdYlylmw9AyMAmQAAmQAAloJ0Axa2fGGiRAAiRAAiRgGAGK2TC0DEwCJEACJEAC2glQzNqZsQYJkAAJkAAJGEaAYjYMLQOTAAmQAAmQgHYCFLN2ZqxBAiRAAiRAAoYRoJgNQ8vAJEACJEACJKCdAMWsnRlrkAAJkAAJkIBhBChmw9AyMAmQAAmQAAloJ0Axa2fGGiRAAiRAAiRgGAGK2TC0DEwCJEACJEAC2glQzNqZsQYJkAAJkAAJGEaAYjYMLQOTAAmQAAmQgHYCFLN2ZqxBAiRAAiRAAoYRoJgNQ8vAJEACJEACJKCdAMWsnRlrkAAJkAAJkIBhBChmw9AyMAmQAAmQAAloJ0Axa2fGGiRAAiRAAiRgGAGK2TC0DEwCJEACJEAC2glQzNqZsQYJkAAJkAAJGEaAYnZDW1QB/M9HNSirrDEMOAOTgNkI3JuwDQjINFuzW1x7HbZAZK2z47ujp1pc38zYobS0NMOaTTG7oT1VDDzyXjV+KDOMNwOTgOkIPNL7ZVTZ/226dre0BgfbI/Du9LN4ZfGrLa1rpuvPmDFjsH79esPaTTFTzIZdXAzcMghQzHLkMcQRhb8/cxprXntbjgZZuBVCzKtWrTKMAMVMMRt2cTFwyyBAMcuRR4pZjjyIVlDMfswFp7L9CJunMg0BilmOVFHMcuSBYvZzHihmPwPn6UxBgGKWI00Usxx5oJj9nAeK2c/AeTpTEKCY5UgTxSxHHihmP+eBYtYP+MNJwQ0Gy75QhX99X+nTyR6/KQRVym62l/aW+lSflbQToJi1MzOiBsVsBFXfYraoe8zjxo3DpEmTkJycrNIQ+8AeeOABxMTE+EZH51oUs35AV/8swhWsW7Qdp4pqUFh2aX/4RycqsPQz3/ak/e+AMBSW12DWxyX6NZaRGiUgm5gfT9rQYHuPF32Jv2X/vkVmlGKWJ61SiXnr1q1YsmQJDh06pBLq0aMHpk6d6hKtJ2zuYs7OzsbIkSOxfPlyJCUlQfz+/PPPIz09vcEwQuQbNmzAiRMnEBkZibFjxyI1NdXTab1+n2L2GpWmgptHRqoizTzh2yhZ08lYWHcCsonZvYMTrl2GL/I2Y+/ZDF367bAFILXnOizcP0yXeHoGoZj1pNm0WNKIef/+/RgxYgQWLlyIwYMHIy8vD59++im6du2K+Ph4r3pZd8TsXmnlypX44IMPGhSzOP/atWsxYcIE9XxOsT/99NMYNkyfv0QUs1dp1FyoPjFfd4UD834ahgW7S/FU31CEBQIPv1uMo+erMH9QGLq3dcBuA8QYe/nnZXjrcDmqlV8W3xWujL6r8fzOSyPmHb+MxLxdpfhtrxCEB9kQoNR57WCZzyNyzZ2zQAWzirljeAK6RNyo3PqoxLH8f+N82QlXttqHXoP/iuoF8TSt4orzOPTDB0q5CtwR9yh6tBmI3WfeUstm5r4uTYYpZmlSIc92qYyMDGzcuLFBcWZmZmLp0qXo3bs3hGQLCwvRt29fzJs3zzVVXVfMCQkJOHz4sFp+zpw5LuqiXmMjZ2fBuXPnIiIiQrdRM8VszIXfkJhXKdPdJwqrcd/bheqJxb1jIeNesQH4LLdSlfJ93YLwRJ8Q3L++CLmKkOuKOXNMFHKU14XUiytqkNTOgRX3RGDM34tw4GyVMR2yWFQzinlw5yno2uoWHDi/DaEBUcrP/fB29nTkFGchOqQLftn1//Dlua2orCnH1VE348NTK/BtwT5c22YA7uw8Ga8dmaJmOa/0O2myTTFLkwp5xOwcMa9bt06deq57CDFPmTLFNb0sRtTz58/HyZMnXZJtSMwilpim3rt3r1dCdp578uTJ6gcBMaWtx0Ex60Hx8hiNiXnq+xfx/rfKQ8obOISkl94djvszCvFtfv1iFgvB3jxUrkaIUkbNbw+PQPr+Mrxx8NJrPJpGwGxittnseKzn3/DywdEoqypSO9+3/QNIaN0f6V/9Frd2GIsrQq5CxjczLwMTF56I+6+ejQX7hzYNmgG1KWYDoPoYUpqpbNF+MWp+5plnMGjQIAwYMKDWFLIQ8/jx49URsPMQcu7Xrx82b96sTj/rKWbxQUFMa2/ZskW3xWMUs49XqYdqjYn5llUFKHH70pDoUBvGXx+Mn3QKQNtQO2qUYXOkIttfrC/ENxfqF/PIDUU4XlCttiI80IZ190XgdWU6m2LWJ59mE3PPtndiQKdf4aNT6S4ArYM74ProFFW44QFtMLb7X1FQcRa7Tq/BUWWa23lQzPpcMy09ilRiFrCFbDdt2oTVq1cjKipKnaoW0hViFj8LebsfYrp6xYoV6gIxvcTslPKsWbPU+916HRSzXiRrx2lMzL1W5NcqnKGMdi+U1mDaByXKNGMNEmMcmH9HeKNiTlbk7jwoZv1zaDYx3xI7Gr2uuBcbvnn+Mhhi1bY4AmxB6BJ5IwbG/Vr5ORhrjk1VvrwmBxSz/tdPS4wonZjdIYupZCHn2bNn+03MTinruejL2SeK2Zi/Qt6KOS7Kjk0jIvHYtmLXPuc7rgrE3NvDKGZjUuNVVLOJOTrkSjzYdQFeOnC/MuNyaSalsaNPu18gKfouLMuaQDF7gsX3VQJSi9n9vnB9U9li5XRKSopuU9lGSlnAppiN+VvnrZjF2fc8HIW3ssrxqnKPOEBZCTbrtlBc3y6AYjYmNV5FNZuYRacm9ViJC+WnsDMnHSVVhYgMjEFIQCSOXNiJK0KvUlZjByj3ny/CBjtubj8CIY4IrP/mOcQoC8Me6rYQa4/9Tnm/WJnuPoPKajnWKvAes1eXq18KSSNmIV5xOB8OIiQ5c+ZMDBw4UF0V7RTz8OHD1RG01sVfYgp80aJFWLNmTb33jI2WMsVs3PWsRcwJ0Q4sULZLRSj3lQuUB5KkKQu7pihP+/r1P4obvMfMqWzjcicim1HMdptDvc/cQ1llLbZEFSr3k/95chm+LtilrNBOxu2dHlHvNYsR9VcXPsT7J5egvPrSFrxbYh/CTe2Go6K6DKuPPIaC8jPGAvYyOsXsJSg/FJNGzOLhIi+88IL6cA9xxMXFYejQoa6tSvVtlxKLxP7whz943C7l5CjuQX/yySfqw0P27NlTC68YnYv/6zvcF5w1JSccMTeFHuu2VAIyi7mlMq+vXxSzPNmWRsyekDjF7M3+Y0+xmut9irm5yPO8MhOgmOXIDsUsRx5EKyhmP+aCYvYjbJ7KNAQoZjlSRTHLkQeK2c95oJj9DJynMwUBilmONFHMcuTBVGKWB5nvLaGYfWfHmi2XAMUsR24pZjnyQDH7OQ8Us5+B83SmIEAxy5EmilmOPFDMfs4Dxexn4DydKQhQzHKkiWKWIw8Us5/zQDH7GThPZwoCFLMcaaKY5cgDxeznPAgxj99WrTxQwM8n5ulIQGICFLMcyXGK+ZXFr8rRIAu34tFHH8WqVasMI2ArKCgQX3vLQyFQpHz74PSPa5RH8REJLwgScBK4u+s22AN3E0gzE3AgEAfXVeO7o6eauSU8vSDQ0AOv9KBDMetBkTFIgARIgARIQCcCFLNOIBmGBEiABEiABPQgQDHrQZExSIAESIAESEAnAhSzTiAZhgRIgARIgAT0IEAx60GRMUiABEiABEhAJwIUs04gGYYESIAESIAE9CBAMetBkTFIgARIgARIQCcCFLNOIBmGBEiABEiABPQgQDHrQZExSIAESIAESEAnAhSzTiAZhgRIgARIgAT0IEAx60GRMUiABEiABEhAJwIUs04gGYYESIAESIAE9CBAMetBkTFIgARIgARIQCcCFLNOIBmGBEiABEiABPQgQDHrQZExSIAESIAESEAnAhSzTiAZhgRIgARIgAT0IEAx60GRMUiABEiABEhAJwIUs04gGYYESIAESIAE9CBAMetBkTFIgARIgARIQCcCFLNOIBmGBEiABEiABPQgQDG7USyqANI+r0FppR5oGYMESIAE9CUQFbEBgQE5PgWNDbfhihCfqrJSHQIB9mD0i5lgGBeK2Q3tqWLgkfeq8UOZYbwZmARIgAR8JjA86XdoFZLrc31W1IdAsD0CD3VZrE+weqJQzBSzYRcXA5MACehLgGLWl6ev0cIcbTDqyr/4Wt1jPYqZYvZ4kbAACZCAHAQoZjnyQDH7MQ+cyvYjbJ6KBEhAMwGKWTMyQypQzIZgrT8oxexH2DwVCZCAZgIUs2ZkhlSgmA3BSjH7EStPpZFAeKANldVAWVWNxpos3tIJUMxyZJhi9mMeOGL2I2yTnypMkWdSOwcC7Zd3pFrx6dHz1ThzUbGrD8fMW0ORlVeFtVnlPtRmlZZMgGKWI7stSsxz585FSkoKkpKSVLqZmZlITk6Wg7TSCopZmlRI35C4SDue6x+KyCAbbDagVbANxYpHKxQrl1UBSz8rxUfHfdsQL8R8UBHzWxSz9NeBvxsom5gfT9rQIIKdp1Zi79n1/kbkl/NJJebs7GysW7cOhw8fVjufkJCAESNGID4+3isY48aNw6RJk1QZi1hC0itWrFB/z8vLw7JlyzBt2rQGY2VkZODTTz/FyZMnERkZiSFDhmDw4MFendubQhSzN5RYpi4BIeXX743A7MwS/PvkjzJWfK1KW4yg7eIX5RA/Ow/na/W9LorV/KesKCd+FrGch3scZsQ6BGQTc5AjzAV/0rWvYmnWw67fK6vLlOtd+ZTahGNM9zSs+iq1CRGMqSqNmIVIR44cieHDh6N///4oLCzEZ599pv7s7ajXXcwC1/79+12j561bt+LNN99Eenp6vSTF6Hrp0qW49957ERsbi9zcXMyZMwezZs3STc4UszEXcUuP2pCYb+4YgF9cG4RVX5bhl4nBaK0IfMGeUhwvqMbg+EAkxwWgbYgNF8pqsOKLMhw4e+kfsXkDw7DvdCXeOHhpKnvhoDB88F0lendwIDbcjpyiamw6VoFdOb6NyFt6Plpy/2QTszvr1OvWIu3AyHrxRwW1Q5A9FFU1FcgvP11L2JGBMXAKXshcvB/siEDKlU+iTXBHbPz2T2rMvNLvpEmtNGIWo9XXXnsN4s/6DiHZzZs3q6IWghXi7t27N1JTf/y0U1fM4nchYiHlGTNmqGETExPVkXhjI2fn+dPS0tQf3c/RlMxRzE2hZ926DYn5592CMO0nIfgstwqfKBLNVYT6+ZkqlFTUqK9/fKJS+YcKGKqUa60I+rf/KMYPpTV49xeR+Of3FZi3q1SFuvfhVjhyvgrrvypHviLxvp0CcL1yf3t4RpF1oVu052YU812dn8DVrW5GTvFh5cNpBzhsAViWdelxljfG/Ay3dhiL40UH4LAHoH1oV6w59hSErJNjH4QQ+v5z/1DLZua+Lk3WpRGzGLFOmTIFW7ZsQUxMzGWAxPvPPvssrr32WowaNUoV85IlS9CnTx+XZOuKWQhYTIuL0fjy5cuRlZWFqVOnIiIiwjWSbigTYur7gQcewFNPPcURszSXqzUb0piYZySH4KVPy/D6gTJVws4jJMCmPJP90gv3dQ/CY71D8MT2Ynx+uuoyMe9RxPzirhJ1MZiYwhYj8Vm3hWLQm4XWBG7hXptNzFeEXoVR18zDooOjlJ0G5crtGDvGJyzF1/m78M+cVzC620s4eP497MvbeFlWB3b6NeIiruNUtqfrffr06di1a1oq2A0AAAYoSURBVBdGjx6NW2+9tda9ZSHm8ePHY+fOnS5xO+8jO19rSMzivGL0u3fv3gansp1tEzHEcfDgQTz99NMYNmyYp2Z7/T5HzF6jYkE3Ao2J+YmbQvC7Dy7WmnZuo4yOO0TYcW/XIMRG2NAmxI4ro+x4csdF7MutvEzMHz4Uhed2lmDHt8q3rChHQrQDLyjT3T97i2K22oVoNjELubYLjXdNR4t8DYpLRfuwa7D00DjcEvsQbmp3P/aceRt7zmagvOqiK6UUs4arW0w7v/vuu3jvvfcgJDlx4kRVxM4R8/bt22tFE6Ni5wIvPcQsziOOo0ePqjLXU84Us4YLgUVdBBoTsxgJP7nj0khYHGIVt3jtZmU6Wqy6FveXe8Q4cJdyz5li5kXliYDZxDzymrkID2iD7wo/r9W18uqL+EhZtS2O1sp9ZDGdfVXkf+PUxSN46+tn1NcpZk9XQz3vi9GwmHYeOHCgeo/XuTir7uItvcXs3hRxv3vRokWo+2HAh+6oVShmX8lZu54WMd/Q3oG5t4dhpjIC/uQ/K7hHJQZh0g1C4BwxW/tK8tx7s4lZjIavbXO7V9PRoQFRGNt9EY5c2In3T75MMXu+HOov4T797JzKdm6lEjXEfeB+/fqpW6zE3mU9RszuLXHe996zZ4+vXahVj2LWBaPlgmgR89VtHJhzeyg+U0bQYq9zlLJSe+atYZzKttxV41uHzSbmiMBoTLx2OTZ992ccy//3fzptQ0RgWxRW5EF5CkAtEBN7rFDvPwsxi1H0dW0H4eWDo9UyNcp/shzSLP4SI+S6+5UnT56MqKgozJ49Wx0xi3vMf/rTn1z3fVeuXInVq1e7RrSNiVmUfeeddxpc9e3p/HokjGLWg6L1Yojp6fl3hOGvyiKvz5RtTs5jQJdATLwhGM8ro+Ovzv24nzPl6kCMvz4YZUrRwvIabPumAkOuCcQLn5QiSym3LCVcvSf9yueXvhh87dAIzFe2WTlH2P/Vyo7pt4Tikc3KF4jzsBQBs4lZJKdTeCLu6vy4chvnCjVXJZWF2PL9PHxf9AV+ftXv0Tm8JwLsQepWqpziLPwt+1lXTsclLEbroA7KwrGyBrdiNccFII2YhTjFvuG+ffuqHMTiK7G1ad68ea57zOJnIWpxFBQU4Pjx4+pqa+eTvhoTsxhd33333ejcubMao+6UuDi/GKGLc4rjxIkTiIuLc51fj+RQzHpQtF4Mh/Kh/6rWDpxStkMVK1uhnIcQtljk9b2yb9m5Atv53tVt7MpeTWUPs7I9Kre4Gl0U2Z4qqlHLXaOMqoWwTyuvi6O7stjrZGE1ipTXxBHsADpHOXDsh6Y9vMF6mTJ/j2UWs/npet8DacQsmixGreLBHuIQD/lwH0G732MWe5qLiooue/CIeL1jx46uVdt1H8kp5HzkyJEGt0s536/v/N4jbbgkxawHRcYgARIwigDFbBRZbXGlEnNjTW9o8Ze27jZvaYq5efnz7CRAAo0ToJjluEIoZj/mgWL2I2yeigRIQDMBilkzMkMqmEbMYpo5JyfH4xO7DKGkU1CKWSeQDEMCJGAIAYrZEKyag5pGzJp7JmEFilnCpLBJJEACLgIUsxwXA8XsxzxQzH6EzVORAAloJkAxa0ZmSAWK2RCs9QelmP0Im6ciARLQTIBi1ozMkAoUsyFYKWY/YuWpSIAEdCJAMesEsolhKOYmAtRSvUj58p6Ht1bDXvspcVpCsCwJkAAJGEagZ9xitAr5xqf4oQFAq2CfqrJSHQKBtlAM7fRHw7jYlCd0yfMAUsO6ycAkQAIkQAIkYA4CFLM58sRWkgAJkAAJWIQAxWyRRLObJEACJEAC5iBAMZsjT2wlCZAACZCARQhQzBZJNLtJAiRAAiRgDgIUsznyxFaSAAmQAAlYhADFbJFEs5skQAIkQALmIEAxmyNPbCUJkAAJkIBFCFDMFkk0u0kCJEACJGAOAhSzOfLEVpIACZAACViEAMVskUSzmyRAAiRAAuYgQDGbI09sJQmQAAmQgEUIUMwWSTS7SQIkQAIkYA4CFLM58sRWkgAJkAAJWIQAxWyRRLObJEACJEAC5iBAMZsjT2wlCZAACZCARQj8P8nrlS9KbJNqAAAAAElFTkSuQmCC"}}},{"cell_type":"markdown","source":"3. Walk-forward validation\n\nAgain, temporal order is respected. This is always the case with time series data.  \n\n- We start on small window of our dataset (we can chose a minimum window width to our liking). We train on this window and predict the value for the next time stamp.\n- We expand the window with the new observation, retrain, predict for the next point in time  \n- and so on...  \n\nWe can opt for an expanding window or for a sliding window (which slides 'to the left' instead of increasing in size).  \nWith this method, for 2 mil data points (our initial data size for BTC, at 1-min resolution), we'll end up training about 2 mil models. I don't know if it's feasible or we'd have to do some window jumps instead of sliding. Also, in this case, an expanding window might be computationally expensive (training of 1 mil data points), I don't know yet.  \n\nIn any case, this seems to be the way to do training in time series modeling.","metadata":{}},{"cell_type":"markdown","source":"**Evaluating the persistence model**\n\nThe persistence model was the naive one that predicts for the next time step the same value from the current time step. This seems like a good baselin model for security price, since the linear plot for asset price looked like a random walk anyway. And in random walk remember we  computed the next value as a linear transformation of the current value.  \n\nI'll go through formally evaluating the persistene model more like a practice for the serious models I will eventually build.  \n\n1. Transform the Closing price dataset into a supervised learning problem\n2. Establish the train and test sets\n3. Define the persistence model\n4. Make predictions and establish a baseline performance","metadata":{}},{"cell_type":"code","source":"data = btc_days_df\n\ndf = pd.concat([data.shift(1), data], axis=1)\ndf.columns = ['t', 't+1']\n\nprint(dataframe.head(5))","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:07.248426Z","iopub.execute_input":"2022-01-24T21:23:07.248688Z","iopub.status.idle":"2022-01-24T21:23:07.261842Z","shell.execute_reply.started":"2022-01-24T21:23:07.248631Z","shell.execute_reply":"2022-01-24T21:23:07.260736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'll do one 70-30% split","metadata":{}},{"cell_type":"code","source":"X = df.values\n\ntrain_size = int(len(X) * 0.7)\n\ntrain, test = X[1:train_size], X[train_size:]\n\nX_train, y_train = train[:,0], train[:,1]\nX_test, y_test = test[:,0], test[:,1]","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:07.263173Z","iopub.execute_input":"2022-01-24T21:23:07.263435Z","iopub.status.idle":"2022-01-24T21:23:07.273707Z","shell.execute_reply.started":"2022-01-24T21:23:07.263383Z","shell.execute_reply":"2022-01-24T21:23:07.272837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The naive persistence prediction model: receives the current value as input and predicts the same as output.","metadata":{}},{"cell_type":"code","source":"def pers_predict(x):\n    return x","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:23:07.275099Z","iopub.execute_input":"2022-01-24T21:23:07.276053Z","iopub.status.idle":"2022-01-24T21:23:07.288182Z","shell.execute_reply.started":"2022-01-24T21:23:07.275985Z","shell.execute_reply":"2022-01-24T21:23:07.287156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Evaluate using root mean squared error:  \n- because squaring errors punishes more the large errors  \n- taking the square root brings it back into the same units of measurement","metadata":{}},{"cell_type":"code","source":"preds = []\n\nfor x in X_test:\n    yhat = pers_predict(x)\n    preds.append(yhat)\n\nrmse = sqrt(mean_squared_error(y_test, preds))\n\nprint(f'RMSE: {rmse:.3f}')","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:33:28.081944Z","iopub.execute_input":"2022-01-24T21:33:28.083154Z","iopub.status.idle":"2022-01-24T21:33:28.092228Z","shell.execute_reply.started":"2022-01-24T21:33:28.083104Z","shell.execute_reply":"2022-01-24T21:33:28.091583Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We've kind of seen this before, in section 8.2, but now I have the whole procedure more formalized.  \n\nThere's no point to plot the predictions, since it will overlap on our data (shifting by one time step is not visible when we have 1300 datapoints in a small plot).\n\n**Visualize residuals**\n\nBeyond predicting and evaluating the error, we should look at the errors of each prediction. Here's why:  \n- the errors should have a mean at 0. If not, it means our model is biased (tends to err more on the positive or more on the negative side). If we have a biased model, we should know. The most naive way to correct for this is to subtract the mean of residuals from all future predictions.  \n- errors should be random (no pattern, no structure). Otherwise, it means the errors contain information. Information that should have been captured by our model. So we need to improve the model.  ","metadata":{}},{"cell_type":"code","source":"# calculate residuals\nresiduals = [y_test[i]-preds[i] for i in range(len(preds))]\nresiduals = pd.DataFrame(residuals)\n\nresiduals.plot()\nplt.title('Residuals plot for the persistence model')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:33:33.995986Z","iopub.execute_input":"2022-01-24T21:33:33.996773Z","iopub.status.idle":"2022-01-24T21:33:34.241174Z","shell.execute_reply.started":"2022-01-24T21:33:33.996726Z","shell.execute_reply":"2022-01-24T21:33:34.240468Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Examine the summary statistics of the residuals. The means is the most important.","metadata":{}},{"cell_type":"code","source":"# summary statistics\nprint(residuals.describe())","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:33:35.151542Z","iopub.execute_input":"2022-01-24T21:33:35.151872Z","iopub.status.idle":"2022-01-24T21:33:35.16414Z","shell.execute_reply.started":"2022-01-24T21:33:35.151834Z","shell.execute_reply":"2022-01-24T21:33:35.163187Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The mean is not 0. The persistence model tends to predict higher values than the real ones. This means we should correct for this bias. Solutions range from subtracting 76 (the mean of residuals) for predictions to complex bias correction methods.\n\n**Residuals distribution**\n\nThe distribution should be Gaussian, otherwise it means there's something worng with our model.  ","metadata":{}},{"cell_type":"code","source":"import seaborn as sns\n\nsns.histplot(data = residuals, kde=True)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:33:35.938794Z","iopub.execute_input":"2022-01-24T21:33:35.939336Z","iopub.status.idle":"2022-01-24T21:33:36.474727Z","shell.execute_reply.started":"2022-01-24T21:33:35.939291Z","shell.execute_reply":"2022-01-24T21:33:36.473351Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Difficult to tell with the naked eye. There's something called Q-Q plot that can help.  \n- it orders residuals ascendingly\n- plots them in a scatterplot against those of an ideal Gaussian distribution","metadata":{}},{"cell_type":"code","source":"from statsmodels.graphics.gofplots import qqplot\n\nres_arr = np.array(residuals)\nqqplot(res_arr, line='r')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:33:36.878379Z","iopub.execute_input":"2022-01-24T21:33:36.878732Z","iopub.status.idle":"2022-01-24T21:33:37.087178Z","shell.execute_reply.started":"2022-01-24T21:33:36.87869Z","shell.execute_reply":"2022-01-24T21:33:37.086098Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a href='https://stats.stackexchange.com/questions/101274/how-to-interpret-a-qq-plot'>This</a> is a usefull discussion on how to interpret the Q-Q plot. It seems like the residuals distribution is a heavy-tail, not a normal one. For an actual prediction model, this would be something to look into and correct.","metadata":{}},{"cell_type":"markdown","source":"**Autocorrelation plot of residuals**\n\nWe should find no autocorrelation. If there is some, we should improve our model and capture it.  \n\nAnd the plot below shows no significant autocorrelation.","metadata":{}},{"cell_type":"code","source":"autocorrelation_plot(residuals)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-01-24T21:33:38.32935Z","iopub.execute_input":"2022-01-24T21:33:38.329652Z","iopub.status.idle":"2022-01-24T21:33:38.56323Z","shell.execute_reply.started":"2022-01-24T21:33:38.329617Z","shell.execute_reply":"2022-01-24T21:33:38.562134Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"You've reached the temporary end. This is work in progress. There is no conclusion yet. I continue to update this notebook as I read more.","metadata":{}}]}