{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Time Series Data","metadata":{}},{"cell_type":"markdown","source":"## Introduction","metadata":{}},{"cell_type":"markdown","source":"Fortune-tellers are people who claim to be able to predict the future. A lot of people go to fortune-tellers because, they are curious about whet the futura holds. However, fortune-telling might not seem scientific enough for the Kaggle community. I would love to help you become better scientific fortune-tellers yourselves. So, I will compare time series forecasting methods for you to select from.","metadata":{}},{"cell_type":"markdown","source":"## Background Knowledge","metadata":{}},{"cell_type":"markdown","source":"Time series data is a sequence of data points that are collected over time intervals. Data points may be collected at regular time intervals like financial data or at irregular intervals like natural disaster data. \n\nTime series forecasting is making predictions about future based on historical time series data. In many cases, time series data is used to predict a target other than time of the future event, making time values a feature. For example, one may try to predict how much Tesla or gold will be worth tomorrow or a month from now. However there are also cases where time series data is used to predict the time of the next target event. For example, time series data about tectonics might be used to predict the next Istanbul earthquake. \n\n","metadata":{}},{"cell_type":"markdown","source":"## Important Notes","metadata":{}},{"cell_type":"markdown","source":"I have selected daily gold prices dataset to investigate the models. This dataset consists data about daily gold trades in terms of price and volume in terms of USD. Dataset contains following columns:\n1) \"Open\" (gold value when stock market opens), \n\n2) \"High\" (gold value when stock market opens), \n\n3) \"Low\" (gold value when stock market opens), \n\n4) \"Close\" (gold value when stock market closes),\n\n5) \"Volume\" (volume of gold traded). \n\nI have selected the \"Open\" column to be the target column. Each model is used to predict \"Open\" values for the next 30 stock market days. I have also trained and tested the models with different test parts of the data. In each training, the models didn't have access to future values. \n\nI have used mean absolute error to calculate errors. Mean absolute error is the average of absolute differences between predicted and true values.\n\nThere is no end to what might be done to improve prediction models. So I have tried to set some limits, and create a fair competition between models. I would like to share some of the choices that I made, for you to better understand the circumstances of the comparisons:\n\n    1) I created the models to take input from either 1 observation, or 1 type of information. For example, LSTM model only takes input from the \"Open\" column, while Dense model only takes input from one observation (row). \n\n    2) I haven't added other data that may be beneficial to predictions.\n    \n    3) I haven't covered improved versions of traditional models such as ARIMA, as ARIMA is basically improved version of moving average method.\n    \n    4) I haven't investigated any ensembling methods.\n\n","metadata":{}},{"cell_type":"markdown","source":"## Traditional Methods","metadata":{}},{"cell_type":"markdown","source":"Traditional methods are methods that have been used for decades, if not for more than a century. I have encountered these methods in my undergraduate production planning and control course. These methods rely on simple mathemathical operations to forecast future values.\n\nI will explain these methods using a simple case: Let's suppose I am a 12 year old child in summer holiday who is in dire need of a new bike; however, my parents told me that I would need to earn my own money if I want a bike. So, I opened a lemonade stand in front of our house. I have sold **3** cups of lemonade on **Monday**, and **5** cups of lemonade on **Tuesday**. Like my parents didn't give me a bike, life didn't give me lemons to make a lemonade. Thus, I need to forecast my future sales at the end of Tuesday for the week to decide how much lemons to buy.\n\nLet's see what my options are:\n\n1) **Naive**: Naive method assumes that the value will stay the same as the last observation. In this case, as I sold 5 cups of lemonade on Tuesday, I would predict that I would sell 5 cups of lemonade on Wednesday, Thursday, and Friday.\n\n2) **Simple Moving Average (SMA)**: SMA is a method that calculates the average of the specified amount of past values, and keeps calculating them by using the calculated values. In this case, using SMA with a frame of **2 days**, I would predict that I would sell (3 + 5) / 2 = 4 cups of lemonade on Wednesday, (5 + 4) / 2 = 4.5 on Thursday, and (4 + 4.5) / 2 = 4.75 on Friday.\n\n3) **Weighted Moving Average (WMA)**: WMA is a method that calculates the weighted average of the specified amount of past values with a specific weight given to more recent values. Generally the weight given to recent values is higher than 1. In this case, using WMA with a frame of **2 days** with a **weight of 4**, I would predict that I would sell (3 + 5 * 4)/(1 + 4) = 4.6 cups of lemonade on Wednesday, and (5 + 4.6 * 4) / (1 + 4) = 4.68 on Thursday.\n\n4) **Linear Regression (LR)**: Linear regression is a method where a line is drawn that minimizes the mean squared error. LR is great at capturing trends, so LR line is also called trend line by some programs like google sheets. In this case, seeing the trend of 2 cups of lemonade increase in sales per day, I would predict that I would sell 7 cups of lemonade on Wednesday, 9 on Thursday, and 11 on Friday.\n\nAs I don't know my sales in the future, I couldn't calculate the errors in this case. However, you may see the summary of predictions in Figure 1 below.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nlemonade_data = {\"Naive\":[3,5,5,5,5],\n               \"SMA2\": [3,5,4,4.5,4.25],\n               \"WMA2\": [3,5,4.6, 4.68, 4.663],\n               \"LR\": [3,5,7,9,11]}\n\nprint(pd.DataFrame.from_dict(data=lemonade_data, orient=\"index\", columns=[\"Monday\", \"Tuesday\", \"Wednesday\", \"Thursday\", \"Friday\"], ))\nprint(f\"{'Table 1 (Lemonade)' : ^60}\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:03.655012Z","iopub.execute_input":"2023-07-05T16:02:03.655498Z","iopub.status.idle":"2023-07-05T16:02:03.670941Z","shell.execute_reply.started":"2023-07-05T16:02:03.655465Z","shell.execute_reply":"2023-07-05T16:02:03.669673Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Naive model has perfomed the best by the mean of errors (see Table 2). I think this is a mainly a result of daily gold prices containing a lot of seemingly random fluctuations (noise). You may see a case where Naive outperformed all other models in Figure 1.\n\nEven though SMA and WMA performed better than using LR by the means of errors, LR based on recent data has the minimum value of error in a case considering all cases. You may see Figure 2 for the case where LR outperformed all predictions.\n\nNaive, SMA30, and LR based on recent data each outperformed all models in 3 cases. So, they are all valid methods of forecasting, and may outperform each other in some cases. WMA outperformed other methods in only 1 case; however, WMA might also be further optimized by changing the weights.","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport pandas as pd\n\ndf = pd.read_csv(\"/kaggle/input/time-series-forecasting-notebook-outputs/gold_test_y_105.csv\")\n\ndf.reset_index(inplace=True, drop=True)\ndf = df.reindex(index=range(29,-1,-1))\ndf.reset_index(inplace=True, drop=True)\n\nplt.plot(df.index, df[\"Open\"], label=\"True\")\nplt.plot(df.index, df[\"Naive Open\"],linestyle=\"dotted\", label=\"Naive\")\nplt.plot(df.index, df[\"SMA30 Open\"],linestyle=\"-.\", label=\"SMA30\")\nplt.plot(df.index, df[\"WMA30 Open\"],linestyle=\"-.\", label=\"WMA30\")\nplt.plot(df.index, df[\"LR All Data Open\"], linestyle=\"--\", label=\"LR All Data\")\nplt.plot(df.index, df[\"LR Recent Data Open\"], linestyle=\"--\", label=\"LR Recent Data\")\nplt.legend()\nplt.figtext(0.5, 0.01, \"Figure 1\", wrap=True, horizontalalignment='center', fontsize=12)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:03.673412Z","iopub.execute_input":"2023-07-05T16:02:03.673867Z","iopub.status.idle":"2023-07-05T16:02:04.011238Z","shell.execute_reply.started":"2023-07-05T16:02:03.673827Z","shell.execute_reply":"2023-07-05T16:02:04.009702Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport pandas as pd\n\ndf = pd.read_csv(\"/kaggle/input/time-series-forecasting-notebook-outputs/gold_test_y_60.csv\")\n\ndf.reset_index(inplace=True, drop=True)\ndf = df.reindex(index=range(29,-1,-1))\ndf.reset_index(inplace=True, drop=True)\n\nplt.plot(df.index, df[\"Open\"], label=\"True\")\nplt.plot(df.index, df[\"Naive Open\"],linestyle=\"dotted\", label=\"Naive\")\nplt.plot(df.index, df[\"SMA30 Open\"],linestyle=\"-.\", label=\"SMA30\")\nplt.plot(df.index, df[\"WMA30 Open\"],linestyle=\"-.\", label=\"WMA30\")\nplt.plot(df.index, df[\"LR All Data Open\"], linestyle=\"--\", label=\"LR All Data\")\nplt.plot(df.index, df[\"LR Recent Data Open\"], linestyle=\"--\", label=\"LR Recent Data\")\nplt.legend()\nplt.figtext(0.5, 0.01, \"Figure 2\", wrap=True, horizontalalignment='center', fontsize=12)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:04.013411Z","iopub.execute_input":"2023-07-05T16:02:04.013868Z","iopub.status.idle":"2023-07-05T16:02:04.368626Z","shell.execute_reply.started":"2023-07-05T16:02:04.013824Z","shell.execute_reply":"2023-07-05T16:02:04.367466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nfrom sklearn.metrics import mean_absolute_error\n\ncolumns_to_compare = ['Naive Open', 'SMA30 Open', 'WMA30 Open', 'LR All Data Open', 'LR Recent Data Open']\n\ncomparison_df = pd.DataFrame(columns=columns_to_compare)\n\nfor start_index in range(0, 136, 15):\n    temp = pd.read_csv(\"/kaggle/input/time-series-forecasting-notebook-outputs/gold_test_y_{i}.csv\".format(i=start_index))\n    list_of_comparisons = []\n    \n    for column in comparison_df:\n        list_of_comparisons.append(mean_absolute_error(\n        y_true=temp[\"Open\"],\n        y_pred=temp[column]))\n        \n    \n    comparison_df.loc[len(comparison_df)] = list_of_comparisons\n    \nprint(comparison_df.describe())\nprint(f\"{'Table 2' : ^60}\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:04.371895Z","iopub.execute_input":"2023-07-05T16:02:04.372362Z","iopub.status.idle":"2023-07-05T16:02:04.478039Z","shell.execute_reply.started":"2023-07-05T16:02:04.372322Z","shell.execute_reply":"2023-07-05T16:02:04.476918Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Deep Learning Methods","metadata":{}},{"cell_type":"markdown","source":"Deep learning methods are methods that have more than 1 layer, which enables them to recognize patterns that non-layered models might not recognize.\n\nDense, LGBM, and LSTM has been included as deep learning methods. Dense model has outperformed every other methor based on the mean of errors (see Table 3). Dense model has also outperformed every other method in the minimum error. You may see Figure 3 for the case where Dense outperformed every other method by the minimum error. However, there are cases where LSTM outperformed every other method (see Figure 4)","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport pandas as pd\n\ndf = pd.read_csv(\"/kaggle/input/time-series-forecasting-notebook-outputs/gold_test_y_0.csv\")\n\ndf.reset_index(inplace=True, drop=True)\ndf = df.reindex(index=range(29,-1,-1))\ndf.reset_index(inplace=True, drop=True)\n\nplt.plot(df.index, df[\"Open\"], label=\"True\")\nplt.plot(df.index, df[\"Dense\"],linestyle=\"dotted\", label=\"Dense\")\nplt.plot(df.index, df[\"LGBM\"],linestyle=\"-.\", label=\"LGBM\")\nplt.plot(df.index, df[\"LSTM\"],linestyle=\"-.\", label=\"LSTM\")\nplt.legend()\nplt.figtext(0.5, 0.01, \"Figure 3\", wrap=True, horizontalalignment='center', fontsize=12)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:04.609951Z","iopub.execute_input":"2023-07-05T16:02:04.610311Z","iopub.status.idle":"2023-07-05T16:02:48.264941Z","shell.execute_reply.started":"2023-07-05T16:02:04.610282Z","shell.execute_reply":"2023-07-05T16:02:48.264096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport pandas as pd\n\ndf = pd.read_csv(\"/kaggle/input/time-series-forecasting-notebook-outputs/gold_test_y_135.csv\")\n\ndf.reset_index(inplace=True, drop=True)\ndf = df.reindex(index=range(29,-1,-1))\ndf.reset_index(inplace=True, drop=True)\n\nplt.plot(df.index, df[\"Open\"], label=\"True\")\nplt.plot(df.index, df[\"Dense\"],linestyle=\"dotted\", label=\"Dense\")\nplt.plot(df.index, df[\"LGBM\"],linestyle=\"-.\", label=\"LGBM\")\nplt.plot(df.index, df[\"LSTM\"],linestyle=\"-.\", label=\"LSTM\")\nplt.legend()\nplt.figtext(0.5, 0.01, \"Figure 4\", wrap=True, horizontalalignment='center', fontsize=12)\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:48.266455Z","iopub.execute_input":"2023-07-05T16:02:48.267050Z","iopub.status.idle":"2023-07-05T16:02:48.549033Z","shell.execute_reply.started":"2023-07-05T16:02:48.267018Z","shell.execute_reply":"2023-07-05T16:02:48.547784Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nfrom sklearn.metrics import mean_absolute_error\n\ncolumns_to_compare = ['Naive Open', 'SMA30 Open', 'WMA30 Open', 'LR All Data Open', 'LR Recent Data Open', 'Dense', 'LGBM', 'LSTM']\n\ncomparison_df = pd.DataFrame(columns=columns_to_compare)\n\nfor start_index in range(0, 136, 15):\n    temp = pd.read_csv(\"/kaggle/input/time-series-forecasting-notebook-outputs/gold_test_y_{i}.csv\".format(i=start_index))\n    list_of_comparisons = []\n    \n    for column in comparison_df:\n        list_of_comparisons.append(mean_absolute_error(\n        y_true=temp[\"Open\"],\n        y_pred=temp[column]))\n        \n    \n    comparison_df.loc[len(comparison_df)] = list_of_comparisons\n    \nprint(comparison_df.describe())\nprint(f\"{'Table 3' : ^60}\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:04.479857Z","iopub.execute_input":"2023-07-05T16:02:04.480333Z","iopub.status.idle":"2023-07-05T16:02:04.608139Z","shell.execute_reply.started":"2023-07-05T16:02:04.480292Z","shell.execute_reply":"2023-07-05T16:02:04.606551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Recently Developed Methods","metadata":{}},{"cell_type":"markdown","source":"Recently, a deep learning method has been developed with collaboration of people from Google Research, Google Cloud, and UC San Diego (2). This method is named Time-series Dense Encoder (TiDE). They have developed encoder-decoder model based on Multi-Layer Perception (MLP). They claim that TiDE outperforms existing State of The Art (SoTA) neural network based models. As I haven't seen this model being used by Kaggle competition winners, I didn't add this model to comparison; however, I may also investigate this model if Kaggle community would prefer me to.","metadata":{}},{"cell_type":"markdown","source":"## Conclusion","metadata":{}},{"cell_type":"markdown","source":"There have been an immense interest in time series forecasting by human nature. In this essay, I have shown that deep learning methods outperforms traditional methods in many cases. However, this doesn't mean that traditional methods have become stale. Traditional methods can still outperform deep learning methods. In GoDaddy Microbusiness Density Forecasting, first place solution was based on linear regression (1).","metadata":{}},{"cell_type":"markdown","source":"## Supplementary Materials","metadata":{}},{"cell_type":"markdown","source":"### Traditional Methods","metadata":{}},{"cell_type":"code","source":"class cfg():\n    gold_df_path = \"/kaggle/input/daily-gold-price-2018-2023/Gold.csv\"\n    \n    df_start_index = 135","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:48.550647Z","iopub.execute_input":"2023-07-05T16:02:48.551625Z","iopub.status.idle":"2023-07-05T16:02:48.558147Z","shell.execute_reply.started":"2023-07-05T16:02:48.551582Z","shell.execute_reply":"2023-07-05T16:02:48.556604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nfrom sklearn.metrics import mean_absolute_error\nimport datetime\n\n\ngold = pd.read_csv(cfg.gold_df_path)\n\ngold['Date'] = gold['Date'].astype('datetime64[ns]')\n\ngold = gold[cfg.df_start_index:]\ngold.reset_index(drop=True,inplace=True)\n\ngold_test = gold[0:30]\ngold_test_x = gold_test.loc[:,[\"Date\"]]\ngold_test_y = gold_test.loc[:,[\"Open\"]]\ngold_train = gold[30:]\ngold_train_x = gold_train.loc[:,[\"Date\"]]\ngold_train_y = gold_train.loc[:,[\"Open\"]]","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:48.562341Z","iopub.execute_input":"2023-07-05T16:02:48.562690Z","iopub.status.idle":"2023-07-05T16:02:48.594472Z","shell.execute_reply.started":"2023-07-05T16:02:48.562662Z","shell.execute_reply":"2023-07-05T16:02:48.593263Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Naive","metadata":{}},{"cell_type":"code","source":"gold_test_y[\"Naive Open\"] = float(gold_train_y.iloc[0])\n\nprint(mean_absolute_error(\n    y_true=gold_test_y[\"Open\"],\n    y_pred=gold_test_y[\"Naive Open\"]\n))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:48.596230Z","iopub.execute_input":"2023-07-05T16:02:48.596553Z","iopub.status.idle":"2023-07-05T16:02:48.605310Z","shell.execute_reply.started":"2023-07-05T16:02:48.596525Z","shell.execute_reply":"2023-07-05T16:02:48.603492Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Simple Moving Average","metadata":{}},{"cell_type":"code","source":"df = gold.copy()\n\nfor i in range(len(df)):\n    if df.iloc[i][\"Date\"] not in list(gold_test_x[\"Date\"]):\n        df.at[i, \"SMA30\"] = float(df.iloc[i][\"Open\"])\n        \n\nfor i in range(len(gold_test_x),-1,-1):\n    df.at[i, \"SMA30\"] = float((df.iloc[i+1][\"SMA30\"] + \n                               df.iloc[i+2][\"SMA30\"] + \n                               df.iloc[i+3][\"SMA30\"] + \n                               df.iloc[i+4][\"SMA30\"] + \n                               df.iloc[i+5][\"SMA30\"] +\n                               df.iloc[i+6][\"SMA30\"] + \n                               df.iloc[i+7][\"SMA30\"] + \n                               df.iloc[i+8][\"SMA30\"] + \n                               df.iloc[i+9][\"SMA30\"] + \n                               df.iloc[i+10][\"SMA30\"] +\n                               df.iloc[i+11][\"SMA30\"] + \n                               df.iloc[i+12][\"SMA30\"] + \n                               df.iloc[i+13][\"SMA30\"] + \n                               df.iloc[i+14][\"SMA30\"] + \n                               df.iloc[i+15][\"SMA30\"] +\n                               df.iloc[i+16][\"SMA30\"] + \n                               df.iloc[i+17][\"SMA30\"] + \n                               df.iloc[i+18][\"SMA30\"] + \n                               df.iloc[i+19][\"SMA30\"] + \n                               df.iloc[i+20][\"SMA30\"] +\n                               df.iloc[i+21][\"SMA30\"] + \n                               df.iloc[i+22][\"SMA30\"] + \n                               df.iloc[i+23][\"SMA30\"] + \n                               df.iloc[i+24][\"SMA30\"] + \n                               df.iloc[i+25][\"SMA30\"] +\n                               df.iloc[i+26][\"SMA30\"] + \n                               df.iloc[i+27][\"SMA30\"] + \n                               df.iloc[i+28][\"SMA30\"] + \n                               df.iloc[i+29][\"SMA30\"] + \n                               df.iloc[i+30][\"SMA30\"]) / 30)\n    \ngold_test_y[\"SMA30 Open\"] = df.iloc[0:30][\"SMA30\"]\n\nprint(mean_absolute_error(\n    y_true=gold_test_y[\"Open\"],\n    y_pred=gold_test_y[\"SMA30 Open\"]\n))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:48.606939Z","iopub.execute_input":"2023-07-05T16:02:48.607663Z","iopub.status.idle":"2023-07-05T16:02:49.206380Z","shell.execute_reply.started":"2023-07-05T16:02:48.607630Z","shell.execute_reply":"2023-07-05T16:02:49.205365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Weighted Moving Average","metadata":{}},{"cell_type":"code","source":"df = gold.copy()\n\nfor i in range(len(df)):\n    if df.iloc[i][\"Date\"] not in list(gold_test_x[\"Date\"]):\n        df.at[i, \"WMA30\"] = float(df.iloc[i][\"Open\"])\n        \n\nfor i in range(len(gold_test_x),-1,-1):\n    df.at[i, \"WMA30\"] = float((df.iloc[i+1][\"WMA30\"] * 3.5 + \n                               df.iloc[i+2][\"WMA30\"] * 3.4 + \n                               df.iloc[i+3][\"WMA30\"] * 3.3 + \n                               df.iloc[i+4][\"WMA30\"] * 3.2  + \n                               df.iloc[i+5][\"WMA30\"] * 3.1 +\n                               df.iloc[i+6][\"WMA30\"] * 3.0 + \n                               df.iloc[i+7][\"WMA30\"] * 2.9 + \n                               df.iloc[i+8][\"WMA30\"] * 2.8 + \n                               df.iloc[i+9][\"WMA30\"] * 2.7 + \n                               df.iloc[i+10][\"WMA30\"] * 2.6 +\n                               df.iloc[i+11][\"WMA30\"] * 2.5 + \n                               df.iloc[i+12][\"WMA30\"] * 2.4 + \n                               df.iloc[i+13][\"WMA30\"] * 2.3 + \n                               df.iloc[i+14][\"WMA30\"] * 2.2 + \n                               df.iloc[i+15][\"WMA30\"] * 2.1 +\n                               df.iloc[i+16][\"WMA30\"] * 2.0 + \n                               df.iloc[i+17][\"WMA30\"] * 1.9 + \n                               df.iloc[i+18][\"WMA30\"] * 1.8 + \n                               df.iloc[i+19][\"WMA30\"] * 1.7 + \n                               df.iloc[i+20][\"WMA30\"] * 1.6 +\n                               df.iloc[i+21][\"WMA30\"] * 1.5 + \n                               df.iloc[i+22][\"WMA30\"] * 1.4 + \n                               df.iloc[i+23][\"WMA30\"] * 1.3 + \n                               df.iloc[i+24][\"WMA30\"] * 1.2 + \n                               df.iloc[i+25][\"WMA30\"] * 1.1 +\n                               df.iloc[i+26][\"WMA30\"] * 1.0 + \n                               df.iloc[i+27][\"WMA30\"] * 0.9 + \n                               df.iloc[i+28][\"WMA30\"] * 0.8 + \n                               df.iloc[i+29][\"WMA30\"] * 0.7 + \n                               df.iloc[i+30][\"WMA30\"] * 0.6) / 61.5)\n    \ngold_test_y[\"WMA30 Open\"] = df.iloc[0:30][\"WMA30\"]\n\nprint(mean_absolute_error(\n    y_true=gold_test_y[\"Open\"],\n    y_pred=gold_test_y[\"WMA30 Open\"]\n))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:49.207906Z","iopub.execute_input":"2023-07-05T16:02:49.208907Z","iopub.status.idle":"2023-07-05T16:02:49.781491Z","shell.execute_reply.started":"2023-07-05T16:02:49.208870Z","shell.execute_reply":"2023-07-05T16:02:49.780430Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Linear Regression","metadata":{}},{"cell_type":"code","source":"from sklearn.linear_model import LinearRegression\n\n\nmodel = LinearRegression()\nlr_x = np.array(gold_train_x[\"Date\"].map(datetime.datetime.toordinal))\nlr_y = np.array(gold_train_y)\n\nlr_x = lr_x.reshape(-1, 1)\n\nmodel.fit(lr_x, lr_y)\n\nlr_test_x = np.array(gold_test_x[\"Date\"].map(datetime.datetime.toordinal))\nlr_test_x = lr_test_x.reshape(-1, 1)\n\nlr_predict_all_data = model.predict(lr_test_x)\n\ngold_test_y[\"LR All Data Open\"] = lr_predict_all_data","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:49.782782Z","iopub.execute_input":"2023-07-05T16:02:49.783128Z","iopub.status.idle":"2023-07-05T16:02:49.797790Z","shell.execute_reply.started":"2023-07-05T16:02:49.783100Z","shell.execute_reply":"2023-07-05T16:02:49.796599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"n_recent_data = 30\n\nlr_recent_data_train_x = lr_x[:n_recent_data]\nlr_recent_data_train_y = lr_y[:n_recent_data]\n\nmodel.fit(lr_recent_data_train_x, lr_recent_data_train_y)\n\nlr_predict_recent_data = model.predict(lr_test_x)\n\ngold_test_y[\"LR Recent Data Open\"] = lr_predict_recent_data\n\nprint(\"LR All Data Open: \", mean_absolute_error(\n    y_true=gold_test_y[\"Open\"],\n    y_pred=gold_test_y[\"LR All Data Open\"]\n))\n\nprint(\"LR Recent Data Open: \", mean_absolute_error(\n    y_true=gold_test_y[\"Open\"],\n    y_pred=gold_test_y[\"LR Recent Data Open\"]\n))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:49.799298Z","iopub.execute_input":"2023-07-05T16:02:49.799597Z","iopub.status.idle":"2023-07-05T16:02:49.813303Z","shell.execute_reply.started":"2023-07-05T16:02:49.799571Z","shell.execute_reply":"2023-07-05T16:02:49.812333Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Deep Learning Methods","metadata":{}},{"cell_type":"markdown","source":"#### Dense","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\n\n\ndf = gold.copy()\n\n        \nfor i in range(len(df)):\n    if df.iloc[i][\"Date\"] not in list(gold_test_x[\"Date\"]):\n        df.at[i, \"Dense\"] = float(df.iloc[i][\"Open\"])\n \n        \ndf[\"Dense\"] = df[\"Dense\"].shift(30)\n\nx_train = df.iloc[60:, 1:-1]\nx_test = df.iloc[:30, 1:-1]\ny_train = df.iloc[60:, -1]\n\ncheckpoint_filepath = '/kaggle/working/'\nmodel_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(\n    filepath=checkpoint_filepath,\n    monitor='val_loss',\n    mode='min',\n    save_best_only=True)\n\nmodel = keras.Sequential()\nmodel.add(keras.Input(shape=(6,)))\nmodel.add(layers.Dense(8, activation=\"relu\", kernel_initializer = tf.keras.initializers.GlorotUniform(seed=42)))\nmodel.add(layers.Dense(4, activation=\"relu\", kernel_initializer = tf.keras.initializers.GlorotUniform(seed=42)))\nmodel.add(layers.Dense(1, activation=\"relu\", kernel_initializer = tf.keras.initializers.GlorotUniform(seed=42)))\nmodel.compile(optimizer=\"Adam\", loss=\"mae\")\nmodel.fit(x_train, y_train, epochs=96, batch_size=32, validation_split=0.3, callbacks = [model_checkpoint_callback])\n\ny_pred = model.predict(x_test)\n\ngold_test_y[\"Dense Open\"] = y_pred","metadata":{"scrolled":true,"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-07-05T16:02:49.814725Z","iopub.execute_input":"2023-07-05T16:02:49.815021Z","iopub.status.idle":"2023-07-05T16:03:14.262793Z","shell.execute_reply.started":"2023-07-05T16:02:49.814995Z","shell.execute_reply":"2023-07-05T16:03:14.261749Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Dense: \", mean_absolute_error(\n    y_true=gold_test_y[\"Open\"],\n    y_pred=gold_test_y[\"Dense Open\"]\n))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:03:14.264404Z","iopub.execute_input":"2023-07-05T16:03:14.264727Z","iopub.status.idle":"2023-07-05T16:03:14.272360Z","shell.execute_reply.started":"2023-07-05T16:03:14.264700Z","shell.execute_reply":"2023-07-05T16:03:14.270991Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### LGBM\n","metadata":{}},{"cell_type":"code","source":"import lightgbm as ltb\n    \nmodel = ltb.LGBMRegressor()\nmodel.fit(x_train, y_train)\n\ny_predi = model.predict(x_test)\n\ngold_test_y[\"LGBM Open\"] = y_predi\n\nprint(\"LGBM: \", mean_absolute_error(\n    y_true=gold_test_y[\"Open\"],\n    y_pred=gold_test_y[\"LGBM Open\"]\n))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:03:14.273898Z","iopub.execute_input":"2023-07-05T16:03:14.274223Z","iopub.status.idle":"2023-07-05T16:03:14.934209Z","shell.execute_reply.started":"2023-07-05T16:03:14.274194Z","shell.execute_reply":"2023-07-05T16:03:14.933353Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### LSTM","metadata":{}},{"cell_type":"code","source":"import datetime as dt\n\ndf = gold.copy()\n\nx_train= df.iloc[30:, 1]\n\ndef splitSequence(seq, n_steps, pred_length):\n    \n    #Declare X and y as empty list\n    X = []\n    y = []\n    \n    for i in range(len(seq)):\n        #get the last index\n        last_index = i + n_steps + pred_length\n        last_train_index = i + n_steps\n        \n        #if lastIndex is greater than length of sequence then break\n        if last_index > len(seq) - 1:\n            break\n            \n        #Create input and output sequence\n        seq_X, seq_y = seq[i:last_train_index], seq[last_train_index:last_index]\n        \n        #append seq_X, seq_y in X and y list\n        X.append(seq_X)\n        y.append(seq_y)\n        pass\n    #Convert X and y into numpy array\n    X = np.array(X)\n    y = np.array(y)\n    \n    return X,y \n    \n    pass\n\nn_steps = 30\npred_length=30\nX, y = splitSequence(np.array(x_train), n_steps = n_steps, pred_length=pred_length)\n\nn_features = 1\nX = X.reshape((X.shape[0], X.shape[1], n_features))\nX.shape\n\nmodel = tf.keras.Sequential()\nmodel.add(layers.LSTM(60, activation='relu', input_shape=(n_steps, n_features)))\nmodel.add(layers.Dense(30, activation='relu'))\nmodel.add(layers.Dense(30))\nmodel.compile(loss='mae', optimizer='Adam')\n\ncheckpoint_filepath = '/kaggle/working/'\nmodel_checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(\n    filepath=checkpoint_filepath,\n    monitor='val_loss',\n    mode='min',\n    save_best_only=True)\nmodel.fit(X,y, epochs=96, batch_size=32, validation_split=0.3, callbacks= [model_checkpoint_callback])\n\ndf = gold.copy()\n\n        \nfor i in range(len(df)):\n    if df.iloc[i][\"Date\"] not in list(gold_test_x[\"Date\"]):\n        df.at[i, \"LSTM\"] = float(df.iloc[i][\"Open\"])\n        \n\nx_pred = df[30:60][\"LSTM\"] \nx_pred = np.array(x_pred).reshape((1, 30, n_features))\n    \ngold_test_y[\"LSTM Open\"] = model.predict(x_pred)[0]","metadata":{"scrolled":true,"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-07-05T16:03:14.935631Z","iopub.execute_input":"2023-07-05T16:03:14.936245Z","iopub.status.idle":"2023-07-05T16:04:25.212274Z","shell.execute_reply.started":"2023-07-05T16:03:14.936214Z","shell.execute_reply":"2023-07-05T16:04:25.211338Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"LSTM: \", mean_absolute_error(\n    y_true=gold_test_y[\"Open\"],\n    y_pred=gold_test_y[\"LSTM Open\"]\n))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:04:25.214281Z","iopub.execute_input":"2023-07-05T16:04:25.214701Z","iopub.status.idle":"2023-07-05T16:04:25.224610Z","shell.execute_reply.started":"2023-07-05T16:04:25.214663Z","shell.execute_reply":"2023-07-05T16:04:25.223569Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Recently Developed Methods","metadata":{}},{"cell_type":"markdown","source":"#### TiDE","metadata":{}},{"cell_type":"markdown","source":"https://arxiv.org/abs/2304.08424","metadata":{}},{"cell_type":"markdown","source":"### Saving","metadata":{}},{"cell_type":"code","source":"gold_test_y.to_csv(\"./gold_test_y_{df_start_index}.csv\".format(df_start_index = cfg.df_start_index))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-05T16:04:25.226099Z","iopub.execute_input":"2023-07-05T16:04:25.226433Z","iopub.status.idle":"2023-07-05T16:04:25.239872Z","shell.execute_reply.started":"2023-07-05T16:04:25.226405Z","shell.execute_reply":"2023-07-05T16:04:25.238854Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## References","metadata":{}},{"cell_type":"markdown","source":"1) https://www.kaggle.com/competitions/godaddy-microbusiness-density-forecasting/discussion/395131\n\n2) https://arxiv.org/abs/2304.08424","metadata":{}},{"cell_type":"markdown","source":"## Submission","metadata":{}},{"cell_type":"code","source":"# submission - to be edited with peer feedback\nimport pandas as pd\ndf = pd.read_csv('/kaggle/input/2023-kaggle-ai-report/sample_submission.csv')\ndf.iloc[0,1] = 'Tabular and/or time series data'\ndf.iloc[1,1] = 'https://www.kaggle.com/code/umuttoygargoz/time-series-forecasting'\ndf.to_csv('submission.csv', index=False)\ndf.head()\n","metadata":{"execution":{"iopub.status.busy":"2023-07-05T16:04:25.539895Z","iopub.execute_input":"2023-07-05T16:04:25.540214Z","iopub.status.idle":"2023-07-05T16:04:25.557892Z","shell.execute_reply.started":"2023-07-05T16:04:25.540187Z","shell.execute_reply":"2023-07-05T16:04:25.556764Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}