{"metadata": {"language_info": {"file_extension": ".py", "version": "3.6.3", "pygments_lexer": "ipython3", "name": "python", "mimetype": "text/x-python", "nbconvert_exporter": "python", "codemirror_mode": {"name": "ipython", "version": 3}}, "kernelspec": {"display_name": "Python 3", "name": "python3", "language": "python"}}, "nbformat_minor": 1, "cells": [{"metadata": {}, "cell_type": "markdown", "source": ["**Prophet** is a tool (made by Facebook) for predicting time-series. \n", "I was exploring it's capabilities for this competition, and will share this small starter script - maybe somebody will find something useful here."]}, {"metadata": {"_uuid": "27403e4d87aa96a6164c61e930ced6c788afd913", "_kg_hide-input": true, "_cell_guid": "c2ab6d8c-f0f3-415f-af83-b7a23af101f8", "collapsed": true, "_kg_hide-output": false}, "execution_count": null, "cell_type": "code", "source": ["%matplotlib inline\n", "import numpy as np\n", "import pandas as pd\n", "from fbprophet import Prophet\n", "from sklearn.metrics import mean_squared_log_error"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["STORE = 1\n", "ITEM = 105575\n", "print('Reading data for store {}, item {}...'.format(STORE, ITEM))\n", "it = pd.read_csv('../input/train.csv', iterator=True, chunksize=10000)\n", "df = pd.concat([c[(c['store_nbr'] == STORE) & (c['item_nbr'] == ITEM)] for c in it])\n", "print('Time-series data shape: {}'.format(df.shape))"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["TRAIN_SIZE = 365\n", "CV_SIZE = 16 #if you make it bigger, fill missing dates in cv with 0 if any\n", "X = df[-(TRAIN_SIZE+CV_SIZE):-CV_SIZE]\n", "y = df[-CV_SIZE:]\n", "print('Train on: {}, CV: {}'.format(X.shape, y.shape))\n", "\n", "X = X[['date','unit_sales']]\n", "X.columns = ['ds', 'y'] #Prophet names\n", "print(X.tail())"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["m = Prophet(yearly_seasonality=True)\n", "m.fit(X)\n", "print('Prophet fitted')"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["future = m.make_future_dataframe(periods=CV_SIZE)\n", "pred = m.predict(future)\n", "print(pred[['ds','yhat','yhat_lower','yhat_upper']].tail(5))\n", "m.plot(pred)"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["Looks like Prophet noticed some weekly seasonality in data and also managed to include it in predictions. Python API allows to extract different components of predicions to check it..."]}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["m.plot_components(pred)"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["So we can see that overall trend, yearly and weekly seasonality will influence predictions for this item. This looks like not typical item/store, because typically saturdays and sundays are top days, but here sundays are with lower popularity. But still we can see some spike around Christmas time in yearly graph. Let's try to score predictions to see, how good this tool can be..."]}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["pred['ds'] = pred['ds'].astype(str)\n", "data = pred[['ds','yhat']].merge(y, left_on='ds', right_on='date')\n", "\n", "items = pd.read_csv('../input/items.csv') #we need items for weights\n", "items['weight'] = 1 + items['perishable'] * 0.25\n", "data = data.merge(items[['item_nbr','weight']], how='left', on='item_nbr')\n", "\n", "score = np.sqrt(mean_squared_log_error(data['unit_sales'].clip(0, 999999), data['yhat'].fillna(0).clip(0, 999999), data['weight']))\n", "print('Score:{}'.format(score))"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["Score 0.5676 - not bad, if we are using only time-series data here (no *onpromotion*, etc important features). \n", "\n", "Just to note - scores differs quite a lot for different items/stores, so not all of them are so good.\n", "\n", "Enjoy."]}], "nbformat": 4}