{"metadata": {"kernelspec": {"name": "python3", "language": "python", "display_name": "Python 3"}, "language_info": {"codemirror_mode": {"version": 3, "name": "ipython"}, "file_extension": ".py", "pygments_lexer": "ipython3", "version": "3.6.4", "name": "python", "mimetype": "text/x-python", "nbconvert_exporter": "python"}}, "nbformat": 4, "cells": [{"cell_type": "markdown", "metadata": {"_cell_guid": "604d1554-d6e2-4622-8251-75c5c26881c0", "_uuid": "174e28dcc01536124fb84c52578c2bb685f4b76b"}, "source": ["# Sumary\n", "**Corporacion Favorita** has the highest incomes in Ecuador for selling grocery, clothes, toys, \u2026 by retail. Its large stores are located in the highland region specially in Pichincha state because Corporacion Rosado, the fourth company in incomes, controls the market in the coast region. For this reason, the selected data set has been delimited for Pichincha state in order to forecaste the time serie using regression trees."]}, {"cell_type": "markdown", "metadata": {"_cell_guid": "2abba5b2-7f15-4f29-bc14-34ae7cffce2d", "_uuid": "9425fc78865773ae3b028a5f17efe3e8340ce278"}, "source": ["# Load libraries "]}, {"cell_type": "code", "metadata": {"_cell_guid": "074f62a6-cd67-434f-a304-6251f922ed24", "_kg_hide-input": false, "_kg_hide-output": false, "_uuid": "674394230e725c585fba5507d09e967ad75ca58d"}, "execution_count": null, "source": ["import numpy\n", "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "import math\n", "from keras.models import Sequential\n", "from keras.layers import Dense,Dropout\n", "from keras.layers import LSTM\n", "from sklearn.preprocessing import MinMaxScaler\n", "from sklearn.metrics import mean_squared_error\n", "\n"], "outputs": []}, {"cell_type": "markdown", "metadata": {"_cell_guid": "0986e27a-6102-47a9-a483-1647b8f7012d", "_kg_hide-output": true, "_uuid": "1a022ca20afdbab2a37403ad9637d2be6e6730fc"}, "source": ["# Load data"]}, {"cell_type": "code", "metadata": {"_cell_guid": "af0cbf28-3e38-47f6-8ad9-d4bb0f42f316", "scrolled": true, "_uuid": "e23e1f4bf0b260a755bd80329e9c204caf2259fb"}, "execution_count": null, "source": ["from subprocess import check_output\n", "print(check_output([\"ls\", \"../input\"]).decode(\"utf8\"))"], "outputs": []}, {"cell_type": "code", "metadata": {"_cell_guid": "191490ab-f2d9-46cc-92cb-c9e57a3dcb10", "scrolled": true, "_uuid": "959bdacf13753ecf92db05e39c0f6d139c63dc15"}, "execution_count": null, "source": ["train = pd.read_csv(\"../input/train.csv\", parse_dates=['date'])\n", "#test=pd.read_csv(\"../input/test.csv\",parse_dates=['date'])\n", "stores = pd.read_csv(\"../input/stores.csv\") "], "outputs": []}, {"cell_type": "markdown", "metadata": {"_cell_guid": "44159970-f3e7-4205-94cf-4bc7d466ac5b", "_uuid": "d852c220a6e41f735d806ab242b095c2e0b6aa25"}, "source": ["# Data processing\n", " Subseting, join and aggregating  data"]}, {"cell_type": "code", "metadata": {"_cell_guid": "9cdd4553-77b9-4e35-8eeb-e623c8bc160a", "_uuid": "a18edae5fbd6555cf64efb7c5be86c09ef36be9e"}, "execution_count": null, "source": ["t=train.groupby([ 'store_nbr','date'], as_index=False).agg({\"unit_sales\": \"sum\"})\n", "train = pd.merge(t, stores, how='left', on=['store_nbr'])\n", "mask=train['state']=='Pichincha'\n", "train=train.loc[mask]\n", "train=train.groupby(['date'], as_index=False).agg({\"unit_sales\": \"sum\"})\n"], "outputs": []}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": ["train.head(10)"], "outputs": []}, {"cell_type": "code", "metadata": {"collapsed": true}, "execution_count": null, "source": [], "outputs": []}, {"cell_type": "code", "metadata": {"collapsed": true}, "execution_count": null, "source": ["#train1.groupby('month', as_index=False).agg({\"unit_sales\": \"sum\"})\n", "\n", "#MUPI_COM[\"valor\"].plot()"], "outputs": []}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": [], "outputs": []}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": [], "outputs": []}, {"cell_type": "markdown", "metadata": {"_cell_guid": "686a8ac2-1de0-4ad3-bc9d-12f9cc48e630", "_uuid": "538e37170e9bc170b1ee547f0cc6fe36fe482560"}, "source": ["## Sales for Pichincha state (train data)"]}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": ["train.tail(12)"], "outputs": []}, {"cell_type": "code", "metadata": {"_cell_guid": "fc3c546f-5f40-4108-81c8-aadea1b5a9e8", "scrolled": true, "_uuid": "8dde8829905e6b16db72705b75a5dcf22e2f8a07"}, "execution_count": null, "source": ["\n", "plt.plot(train['unit_sales'])\n", "plt.show()"], "outputs": []}, {"cell_type": "markdown", "metadata": {"_cell_guid": "c9740709-c4fa-45b9-a00f-0dc4831dcaa6", "_uuid": "45fa263568710ec4062503a6ad4e6f4f88c9e587"}, "source": ["There is a trend over time especially for the 2015 year, by 2015 and 2016 the slope gets lower and loses trend. It\u2019s easy to recognize how the sales increasing over the last quarters."]}, {"cell_type": "markdown", "metadata": {}, "source": ["# Sample split for train and test data"]}, {"cell_type": "code", "metadata": {"_cell_guid": "eeadc6bd-d69f-4848-87e3-63d8e01f6d38", "_uuid": "fd7230245de56b2f2cd69368f49a23af90cd1301"}, "execution_count": null, "source": ["numpy.random.seed(123)"], "outputs": []}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": ["train_size = int(len(train) * 0.75)\n", "test_size = len(train) - train_size\n", "\n", "print(train_size,test_size, len(train))"], "outputs": []}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": ["train1= train[0:train_size]\n", "test =  train[train_size:len(train)]\n", "print(len(train1), len(test))"], "outputs": []}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": ["train1=train1.set_index(\"date\")\n", "test=test.set_index(\"date\")\n", "train=train.set_index(\"date\")\n", "train1=train1.values\n", "test=test.values\n", "train=train.values"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["Determine the number of previous time steps to use as input variables to predict the next time period. \n", "    In this case (look_back) determinated to 1"]}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": ["def create_dataset(dataset, look_back=1):\n", "    dataX, dataY = [], []\n", "    for i in range(len(dataset)-look_back-1):\n", "        a = dataset[i:(i+look_back), 0]\n", "        dataX.append(a)\n", "        dataY.append(dataset[i + look_back, 0])\n", "    return numpy.array(dataX), numpy.array(dataY)\n"], "outputs": []}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": ["\n", "look_back = 1\n", "trainX, trainY = create_dataset(train1, look_back)\n", "testX, testY = create_dataset(test, look_back)"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["# Multilayer Perceptron model\n", "\n", "A simple network with 1 input (look_back)  , 1 hidden layer with 8 neurons and one (1) output layer."]}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": ["\n", "model = Sequential()\n", "model.add(Dense(8, input_dim=look_back, activation='relu'))\n", "model.add(Dense(1))\n", "model.compile(loss='mean_squared_error', optimizer='adam')\n", "model.fit(trainX, trainY, epochs=200, batch_size=2, verbose=2)"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": [" # Performance of the model"]}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": ["trainScore = model.evaluate(trainX, trainY, verbose=0)\n", "print('Train Score: %.2f MSE (%.2f RMSE)' % (trainScore, math.sqrt(trainScore)))\n", "testScore = model.evaluate(testX, testY, verbose=0)\n", "print('Test Score: %.2f MSE (%.2f RMSE)' % (testScore, math.sqrt(testScore)))"], "outputs": []}, {"cell_type": "code", "metadata": {}, "execution_count": null, "source": ["trainPredict = model.predict(trainX)\n", "testPredict = model.predict(testX)\n", " \n", "trainPredictPlot = numpy.empty_like(train)\n", "trainPredictPlot[:, :] = numpy.nan\n", "trainPredictPlot[look_back:len(trainPredict)+look_back, :] = trainPredict\n", " \n", "testPredictPlot = numpy.empty_like(train)\n", "testPredictPlot[:, :] = numpy.nan\n", "testPredictPlot[len(trainPredict)+(look_back*2)+1:len(train)-1, :] = testPredict\n", " \n", "plt.plot(train)\n", "plt.show()\n", "plt.plot(trainPredictPlot)\n", "plt.plot(testPredictPlot)\n", "plt.show()\n", "plt.plot(train)\n", "plt.plot(trainPredictPlot)\n", "plt.plot(testPredictPlot)\n", "plt.show()\n", "\n"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["The average error in the training data is 86.486 units and the averrage in the test data is 114.344 units sold per day"]}], "nbformat_minor": 1}