{"nbformat": 4, "cells": [{"source": ["# Introduction\n", "\n", "This competition is hosted by a large grocery company in Ecuador called \"Corporacion Favorita\" where the aim of the game is to accurately predict and forecast the unit sales for items sold at various Favorita supermarket chains across Ecuador. Apart from the usual training and test data files provided, there are also quite a handful of other supplementary data files (5 extra files to be exact) provided to us. \n", "\n", "This notebook aims to take a deep-dive analysis into each of the files provided in this competition and to investigate what types of insights or observations can be derived from each. The structure of this analysis is as follows:\n", "\n", "**1. Data loading and inspection** - Loading the data as Python dataframes and conducting data quality checks\n", "\n", "**2. Supplementary Data exploration** - Exploration of all 5 supplementary files with a mix of D3.js visualizations and stacked barplots\n", "\n", "**3. Training data exploration **\n", "\n", "**4. Feature ranking with learning models** - Training a "], "metadata": {"_uuid": "b36e6423a34c79b224df04d497a36850c24ee3a2", "_cell_guid": "759c2537-ec1f-4f5a-96d5-7292a3ca168c"}, "cell_type": "markdown"}, {"source": ["# Importing the relevant libraries\n", "import IPython.display\n", "import json\n", "import pandas as pd\n", "import seaborn as sns\n", "import squarify\n", "%matplotlib inline\n", "import missingno as msno\n", "import plotly.offline as py\n", "py.init_notebook_mode(connected=True)\n", "import plotly.graph_objs as go\n", "import plotly.tools as tls\n", "import numpy as np\n", "from scipy.fftpack import fft\n", "from matplotlib import pyplot as plt\n", "# D3 modules\n", "from IPython.core.display import display, HTML, Javascript\n", "from string import Template\n"], "execution_count": null, "metadata": {"_uuid": "a5c67cb421d1deace8e5528e19deff588d3eff7b", "_cell_guid": "5c52ebc6-3d76-4daa-8f30-22a5e5dae4fa", "_kg_hide-input": true}, "cell_type": "code", "outputs": []}, {"source": ["# 1. Data loading and inspection checks\n", "\n", "To start off with, let us load in the various supplementary comma-separated value files with the Pandas package via the the read_csv function as follows. Borrowing from Inversion's very helpful kernel in creating a dataframe with all [Date-Store_Item combinations](https://www.kaggle.com/inversion/dataframe-with-all-date-store-item-combinations) (please do check it out ), I will load in the training data using some of his methods as well. "], "metadata": {"_uuid": "19f6a670b6b44aefdb6882a550a45c439d23d41e", "_cell_guid": "8c8e2fae-bfb5-4447-b6f0-5e70e5c92c9a"}, "cell_type": "markdown"}, {"source": ["items = pd.read_csv(\"../input/items.csv\")\n", "holiday_events = pd.read_csv(\"../input/holidays_events.csv\")\n", "stores = pd.read_csv(\"../input/stores.csv\")\n", "oil = pd.read_csv(\"../input/oil.csv\")\n", "transactions = pd.read_csv(\"../input/transactions.csv\",parse_dates=['date'])\n", "# I read in the full training data just to get prior information and here is the output:\n", "# Output: \"125,497,040 rows | 6 columns\"\n", "train = pd.read_csv(\"../input/train.csv\", nrows=6000000, parse_dates=['date'])"], "execution_count": null, "metadata": {"_uuid": "8f5e396190d05ba9a2cd021c480b2b63d89185a8", "_cell_guid": "8d831841-c0a1-4b1d-9fa5-3b309d852957", "collapsed": true, "_kg_hide-input": true}, "cell_type": "code", "outputs": []}, {"source": ["With regards to the training data, it contains a whooping 125,497,040 rows (and 6 columns). Therefore I will only load in 6 million rows of the training data (approx 5% of the data) just to get a rough idea of what is in store for us. "], "metadata": {"_uuid": "87d82898e50b545f48f0e807686cc6d317428857", "_cell_guid": "136c24ba-a573-4c17-ad4b-8122ad98d66c"}, "cell_type": "markdown"}, {"source": ["train.head()"], "execution_count": null, "metadata": {"_uuid": "24f0ede2d7060bc6f19adc993efee1a7b0cafbd7", "_cell_guid": "15a0cb65-fcf4-4381-ad5f-4a42a28a9507"}, "cell_type": "code", "outputs": []}, {"source": ["Taking a peek, we note that the training data only consists of 6 rather measly columns and coupled with the fact that we have approx 125 million rows, there does seem to be a discrepancy in the number of features that we are going to provide our learning model to train on. However that's where the other supplementary files comes in to play as we will most definitely have to join the fields \"store_nbr\" (store number) and \"item_nbr\" as well as dates ( bring in daily oil prices). So there is actually quite a lot of potential and avenues for feature enhancement and engineering."], "metadata": {"_uuid": "7ac3181b2087464ee187f39c4d71b8c614cfaeb1", "_cell_guid": "ed029374-989b-4069-b772-5f19d5790074"}, "cell_type": "markdown"}, {"source": ["### NULL or missing values check\n", "\n", "One standard check I like to carry out is to simply inspect all our data for any Null or missing values. If there are any, then we might have to think of strategies to handle them (eg. Imputation, removal of nulls etc). A good library to conveniently visualise missing values is via the \"missingno\" package as an aside. "], "metadata": {"_uuid": "78641a33978cbfee6b4835b6b44f1944d31e5576", "_cell_guid": "a3f35557-47e6-45a1-9502-b7a3fb3979de"}, "cell_type": "markdown"}, {"source": ["print(\"Nulls in Oil columns: {0} => {1}\".format(oil.columns.values,oil.isnull().any().values))\n", "print(\"=\"*70)\n", "print(\"Nulls in holiday_events columns: {0} => {1}\".format(holiday_events.columns.values,holiday_events.isnull().any().values))\n", "print(\"=\"*70)\n", "print(\"Nulls in stores columns: {0} => {1}\".format(stores.columns.values,stores.isnull().any().values))\n", "print(\"=\"*70)\n", "print(\"Nulls in transactions columns: {0} => {1}\".format(transactions.columns.values,transactions.isnull().any().values))"], "execution_count": null, "metadata": {"_uuid": "d1aea94631f9fe7393bec92a100b82954ea951af", "_cell_guid": "c9dfc8fa-55e6-4b2e-8420-ef6fe25b8c6b"}, "cell_type": "code", "outputs": []}, {"source": ["As we can see,  the only missing data occurs in the oil data file, which provides the historical daily price for oil. "], "metadata": {"_uuid": "45893f31594adcdfb68b4858cf3ebee7613f2988", "_cell_guid": "0d1b39c1-f02d-40ce-941f-9a3395a47cf5"}, "cell_type": "markdown"}, {"source": ["# 2. Supplementary Data Exploration"], "metadata": {"_uuid": "99e917d88a435cefffb96c342d5ab1c0683bb9a8", "_cell_guid": "eedc6081-ce78-45a3-a959-d22771cdf50f"}, "cell_type": "markdown"}, {"source": ["## 2a. Oil data\n", "\n", "First up we can take a look at the \"oil.csv\" provided to us. As alluded to in section 1, this file contains daily oil prices within a time range that covers both the train and test data timeframe so this is something to note should one's learning model take into account the trend in these oil prices. This supplementary oil data seems to be a very simple two column table with one column being the date and the other the daily oil price \"dcoilwtico\" which seems to be the abbreviation for [Crude oil prices: West Texas Intermediate - Cushing, Oklahoma](https://fred.stlouisfed.org/series/DCOILWTICO). \n", "\n"], "metadata": {"_uuid": "42233b22af155c7ee92e385ef3793f76a56ac0c0", "_cell_guid": "9a41b9ff-cec6-405c-a642-5321136fa198"}, "cell_type": "markdown"}, {"source": ["oil.head(3)"], "execution_count": null, "metadata": {"_uuid": "4c9c78274b22324a8b7bf6fd6d206f8cea2c8f3f", "_cell_guid": "35975467-e5f9-4129-abbe-08fa8fc87cd6"}, "cell_type": "code", "outputs": []}, {"source": ["**Interactive Visualisations with Plotly**\n", "\n", "Let us take a look at the underlying data by plotting the daily oil prices in a time series plot via the interactive Python visualisation library Plot.ly as follows. Here we invoke the Plot.ly scatter plot function by calling \"Scatter\" and it is a simple matter of providing the date range in the x-axis and the corresponding daily oil prices in the y-axis ( as an aside I have also simultaneously dropped nulls by calling dropna( ) in the oil dataframe). I've hidden the Plot.ly code as it can get quite long so unhide it if you want to see the syntax."], "metadata": {"_uuid": "bea836a2ba4002d3d5d07a9dc0c6155c7c72ff7f", "_cell_guid": "b90972a5-40eb-40bd-bad6-43f5f07f44af"}, "cell_type": "markdown"}, {"source": ["trace = go.Scatter(\n", "name='Oil prices',\n", "x=oil['date'],\n", "y=oil['dcoilwtico'].dropna(),\n", "    mode='lines',\n", "    line=dict(color='rgb(20, 15, 200, 0.8)'),\n", "#     #fillcolor='rgba(68, 68, 68, 0.3)',\n", "    fillcolor='rgba(0, 0, 216, 0.3)',\n", "    fill='tonexty' )\n", "data = [trace]\n", "\n", "layout = go.Layout(\n", "    yaxis=dict(title='Daily Oil price'),\n", "    title='Daily oil prices from Jan 2013 till July 2017',\n", "    showlegend = False)\n", "fig = go.Figure(data=data, layout=layout)\n", "py.iplot(fig, filename='pandas-time-series-error-bars')"], "execution_count": null, "metadata": {"_uuid": "b08346770a50e556c799c392ef603a0b3108aa6c", "_cell_guid": "9c1de4fa-3ee4-4d8d-92cd-32468b84eb54", "_kg_hide-input": true}, "cell_type": "code", "outputs": []}, {"source": ["#### *[THE ABOVE PLOT IS INTERACTIVE SO YOU CAN DRAG AND ZOOM ON IT. DOUBLE-CLICK TO GET BACK TO THE ORIGINAL SIZE]*"], "metadata": {"_uuid": "fba0982a687749b7558101d3a10aad0c44d3bed6", "_cell_guid": "84f6d13c-bbb2-41df-ab19-0d16840fdb24"}, "cell_type": "markdown"}, {"source": ["**Takeaway from the plots**\n", "\n", "This plot shows that the daily oil price is on a general downward trend from Jan 2013 till July 2017. Where the price of oil started out 2013 by increasing and even busting the 100 dollar mark for a good few months in 2013 and 2014, it reached the middle of 2014 where there was a drastic drop in the price of oil. Via some quick open-source research (i.e Googling), this trend checks out as it seems oil prices were kept fairly stable from 2010 till mid-2014 after which it drastically fell ( due to a confluence of reasons such as weak demand due to poor economic growth and surging alternative sources of crude oil from shale/tar sands).\n"], "metadata": {"_uuid": "91de04350832765db069a1b7db60d9ddf90f8ef5", "_cell_guid": "082fb224-e62a-4561-b6b8-f2b6ffcd4a05"}, "cell_type": "markdown"}, {"source": ["## 2b. Stores data\n", "\n", "With regards to the \"stores.csv\" file, the data dictionary on the Kaggle competition simply states that it contains metadata on the city, state, the store type and a column termed \"cluster\". Now this cluster column is a grouping to stores that are similar to each other and as we can see from the latter analysis, there are a total of 17 distinct clusters. With regards to the number of stores, there are a total of 54 stores (based off a unique list of store_nbr) and therefore I presume that all the unit sales and transactions are generated off the data collected from these 54 stores."], "metadata": {"_uuid": "4089fa8e74806befd03ee90257c9a76e32d23c7f", "_cell_guid": "d32bc964-fd62-4a4e-b78c-cc977b696c49"}, "cell_type": "markdown"}, {"source": ["stores.head(3)"], "execution_count": null, "metadata": {"_uuid": "0e8a36efec8344a8dfe445dc43ac0f7a42ab5a57", "_cell_guid": "d5695ece-8ddc-4aba-8742-de5680816579"}, "cell_type": "code", "outputs": []}, {"source": ["**Treemap plots of store attributes**\n", "\n", "Treemap plots inspired from I, Coder's kernel: [Novice to Grandmaster](https://www.kaggle.com/ash316/novice-to-grandmaster)\n"], "metadata": {"_uuid": "cc3864bddce56eebe22eae8c285aaac30e74fea3", "_cell_guid": "9022a564-6d9f-4cb3-b515-5ab2020ec091"}, "cell_type": "markdown"}, {"source": ["fig = plt.figure(figsize=(16, 11))\n", "marrimeko=stores.city.value_counts().to_frame()\n", "ax = fig.add_subplot(111, aspect=\"equal\")\n", "ax = squarify.plot(sizes=marrimeko['city'].values,label=marrimeko.index,\n", "              color=sns.color_palette('cubehelix_r', 28), alpha=1)\n", "ax.set_xticks([])\n", "ax.set_yticks([])\n", "fig=plt.gcf()\n", "fig.set_size_inches(40,25)\n", "plt.title(\"Treemap of store counts across different cities\", fontsize=18)\n", "plt.show();"], "execution_count": null, "metadata": {"_uuid": "4f651ee11cd7e0ced2323633be8382592733be29", "_cell_guid": "cf2f305c-5bdd-48bf-bfce-1e35b0caba9d"}, "cell_type": "code", "outputs": []}, {"source": ["fig = plt.figure(figsize=(14, 11))\n", "marrimeko=stores.state.value_counts().to_frame()\n", "ax = fig.add_subplot(111, aspect=\"equal\")\n", "ax = squarify.plot(sizes=marrimeko['state'].values,label=marrimeko.index,\n", "              color=sns.color_palette('viridis_r', 28), alpha=1)\n", "ax.set_xticks([])\n", "ax.set_yticks([])\n", "fig=plt.gcf()\n", "fig.set_size_inches(40,25)\n", "plt.title(\"Treemap of store counts across different States\", fontsize=18)\n", "plt.show()"], "execution_count": null, "metadata": {"_uuid": "537c04b48594c7b63ebee2f7b7c9b09c4a3beb7b", "_cell_guid": "3fae1e8e-70f2-4955-bd68-622b865e2135"}, "cell_type": "code", "outputs": []}, {"source": ["stores.state.unique()"], "execution_count": null, "metadata": {"_uuid": "6c365ba741ae13100211a53d18237c818c6f671f", "_cell_guid": "790e3663-60a6-4538-a0b9-fa7cab420be3"}, "cell_type": "code", "outputs": []}, {"source": ["**Inspecting the allocation of clusters to store numbers**\n", "\n", "We can now generate will be that of our store numbers ordered against their respective store clusters so that we can observe if there are any apparent trends or relationships in the data. To do so, I will take our stores Python dataframe and group it based on the columns \"store_nbr\" and \"cluster\" via the **groupby** and pivot statement. After which, I will unstack the grouping which means that I will pivot on the level of store_nbr index labels, returning a DataFrame having a new level of columns which are the store clusters whose inner-most level relate to the pivoted store_nbr index labels. This technique is commonly used for producing stacked barplots in Python but since we only have unique store_nbr numbers, therefore we will simply get barplots of store numbers ordered by their relevant clusters."], "metadata": {"_uuid": "4c70d8f57bda71fefc4e8e3af18ff1741421f5d4", "_cell_guid": "8e36f3ae-92ed-4733-b765-8e72ed235501"}, "cell_type": "markdown"}, {"source": ["# Unhide to see the sorted zip order\n", "neworder = [23, 24, 26, 36, 41, 15, 29, 31, 32, 34, 39, \n", "            53, 4, 37, 40, 43, 8, 10, 19, 20, 33, 38, 13, \n", "            21, 2, 6, 7, 3, 22, 25, 27, 28, 30, 35, 42, 44, \n", "            48, 51, 16, 0, 1, 5, 52, 45, 46, 47, 49, 9, 11, 12, 14, 18, 17, 50]"], "execution_count": null, "metadata": {"_uuid": "779d70abd511f4213aa473225a8a55a3e2b5cd6e", "_cell_guid": "f9cc20ad-060d-4a97-a95b-97fed748ddf5", "collapsed": true, "_kg_hide-input": true}, "cell_type": "code", "outputs": []}, {"source": ["# Finally plot the seaborn heatmap\n", "plt.style.use('dark_background')\n", "plt.figure(figsize=(15,12))\n", "store_pivot = stores.dropna().pivot(\"store_nbr\",\"cluster\", \"store_nbr\")\n", "ax = sns.heatmap(store_pivot, cmap='jet', annot=True, linewidths=0, linecolor='white')\n", "plt.title('Store numbers and the clusters they are assigned to')"], "execution_count": null, "metadata": {"_uuid": "b9ce5b599887b70ab7424ce0b5dc3d56c977e4a8", "_cell_guid": "5810d80f-34ad-4aa6-a657-7942f62025ac"}, "cell_type": "code", "outputs": []}, {"source": ["# plt.style.use('dark_background')\n", "# nbr_cluster = stores.groupby(['store_nbr','cluster']).size()\n", "# nbr_cluster.unstack().iloc[neworder].plot(kind='bar',stacked=True, colormap= 'tab20', figsize=(13,11),  grid=False)\n", "# plt.title('Store numbers and the clusters they are assigned to', fontsize=14)\n", "# plt.ylabel('')\n", "# plt.xlabel('Store number')\n", "# plt.show()"], "execution_count": null, "metadata": {"_uuid": "45b69903bc0dd485f02853b170aa8eb5b82c507a", "_cell_guid": "f36f8868-811e-4a27-aee2-08616c98ea99", "collapsed": true}, "cell_type": "code", "outputs": []}, {"source": ["**Takeaways from thse plot**\n", "\n", "From visualising the store numbers side-by-side based on the clustering, we can identify certain patterns. For example clusters 3, 6, 10 and 15 are the most common store clusters based off the fact that there are more store_nbrs attributed to them then the others while on the other end of the spectrum, we have clusters 5 and 17 which are only related to the stores 44 and 51 respectively."], "metadata": {"_uuid": "aadede69e3161187f8f3d685d6da39be953a1852", "_cell_guid": "241d6287-e818-473c-8f65-a7b378829949"}, "cell_type": "markdown"}, {"source": ["**Stacked Barplots of Types against clusters**\n", "\n", "Here it might be informative to look at the distribution of clusters based on the store type to see if we can identify any apparent relationship between types and the way the company has decided to cluster the particular store. Again we apply the groupby operation but this time on type and on cluster. This time when we pivot based off this grouped operation, we are able to get counts of each distinct cluster distributed and stacked on top of other clusters per store type as follows:"], "metadata": {"_uuid": "bc1a8e10d8d8c1ac6d64e4c9ca03261172e58eca", "_cell_guid": "2d6cd8fd-f690-421f-9799-00980770732c"}, "cell_type": "markdown"}, {"source": ["plt.style.use('seaborn-white')\n", "#plt.style.use('dark_background')\n", "type_cluster = stores.groupby(['type','cluster']).size()\n", "type_cluster.unstack().plot(kind='bar',stacked=True, colormap= 'PuBu', figsize=(13,11),  grid=False)\n", "plt.title('Stacked Barplot of Store types and their cluster distribution', fontsize=18)\n", "plt.ylabel('Count of clusters in a particular store type', fontsize=16)\n", "plt.xlabel('Store type', fontsize=16)\n", "plt.show()"], "execution_count": null, "metadata": {"_uuid": "d577a96db57a4932f76c7cff2843fdf57fc6e781", "_cell_guid": "6ca71516-7269-4f31-9366-3c21a41416ca"}, "cell_type": "code", "outputs": []}, {"source": ["**Takeaway from the plots**\n", "\n", "Most of the store types seem to contain a mix of the clusters, especially with regards to store type \"D\". Only type \"E\" stores seem to fall within the single cluster of cluster 10. However with regards to our initial plan of trying to suss out relationships between store types and clusters, it seems that there is nothing apparent that stands out. If we think of store types, one would normally think of categories such as convenience store types, huge general store types or bulk buy types."], "metadata": {"_uuid": "9da937aa8e118f9b8f1d144906e85003e0be82ed", "_cell_guid": "398b76df-92c5-4df0-b601-49506d51b4b2"}, "cell_type": "markdown"}, {"source": ["**Stacked barplot of types of stores across the different cities**\n", "\n", "Another interesting distribution to observe would be the types of stores that Corporacion Favorita has decided to open for each city in Ecuador as well as the absolute number of stores for that city. All these three things can be achieved by turning to our usual groupby operation and pivoting again by unstacking and plotting as follows:"], "metadata": {"_uuid": "ee5413119e88d368ea8b9638b66932f9ab353afe", "_cell_guid": "5aeaa9cb-6937-44f6-9424-2a936eec90fd"}, "cell_type": "markdown"}, {"source": ["# plt.style.use('dark_background')\n", "plt.style.use('seaborn-white')\n", "city_cluster = stores.groupby(['city','type']).store_nbr.size()\n", "city_cluster.unstack().plot(kind='bar',stacked=True, colormap= 'viridis', figsize=(13,11),  grid=False)\n", "plt.title('Stacked Barplot of Store types opened for each city')\n", "plt.ylabel('Count of stores for a particular city')\n", "plt.show()"], "execution_count": null, "metadata": {"_uuid": "dec0629f469dde60d4e01437d0d284c8d6fd0dbe", "_cell_guid": "85191c9e-44cb-43f0-a841-157aa3f00f06"}, "cell_type": "code", "outputs": []}, {"source": ["**Takeaways from the plot**: \n", "\n", "As observed from the stacked barplots, there are two cities that standout in terms of the variety of store types on offer - Guayaquil and Quito. These should come as no surprise as [Quito](https://en.wikipedia.org/wiki/Quito) is the capital city of Ecuador while [Guayaquil](https://en.wikipedia.org/wiki/Guayaquil) is the largest and most populous city in Ecuador. Therefore one would think it logical to expect Corporacion Favorita to target these major cities with the most diverse store types probably to capture different ends of the market (if we think store types as being high-end/premium/wholesale/discount etc) as well as opening up the highest number of stores evinced from the largest counts of store_nbrs attributed to those two cities."], "metadata": {"_uuid": "5a4421f1162f5e3228aef2dc36206396f55e1830", "_cell_guid": "10406822-b05c-4859-9dc3-55e47b3b3ff4"}, "cell_type": "markdown"}, {"source": ["## 2c. Holiday Events data\n", "\n", "Trudging on, we can inspect the \"holiday_events.csv\" file which contains data on the national, regional and local level of Ecuador. According to the data dictionary, we should pay special attention to the \"transferred\" column. \n", "\n", "*A holiday that is transferred officially falls on that calendar day, but was moved to another date by the government. A transferred day is more like a normal day than a holiday. To find the day that it was actually celebrated, look for the corresponding row where type is Transfer. For example, the holiday Independencia de Guayaquil was transferred from 2012-10-09 to 2012-10-12, which means it was celebrated on 2012-10-12. Days that are type Bridge are extra days that are added to a holiday (e.g., to extend the break across a long weekend). These are frequently made up by the type Work Day which is a day not normally scheduled for work (e.g., Saturday) that is meant to payback the Bridge.*"], "metadata": {"_uuid": "de3edff9d78f6ab0288447420facff18cb4c3b0b", "_cell_guid": "89fe7bab-068b-400a-9bc9-38ce324bf111"}, "cell_type": "markdown"}, {"source": ["holiday_events.head(3)"], "execution_count": null, "metadata": {"_uuid": "f3e8d2711c40e33fcb17e00b33fef1cfad73c5e4", "_cell_guid": "fbcb9370-f682-43fc-a0ac-624b6de630b1"}, "cell_type": "code", "outputs": []}, {"source": ["plt.style.use('seaborn-white')\n", "# plt.style.use('dark_background')\n", "holiday_local_type = holiday_events.groupby(['locale_name', 'type']).size()\n", "holiday_local_type.unstack().plot(kind='bar',stacked=True, colormap= 'magma_r', figsize=(12,10),  grid=False)\n", "plt.title('Stacked Barplot of locale name against event type')\n", "plt.ylabel('Count of entries')\n", "plt.show()"], "execution_count": null, "metadata": {"_uuid": "58b91bcdf53c45bbbb0e61b6fad3cce7459aa3c9", "_cell_guid": "e03a2932-cdd2-4eea-a1b0-0d46e93cccbb"}, "cell_type": "code", "outputs": []}, {"source": ["## D3.js Circular treemapping of Store Type to description\n", "\n", "Next, I shall implement a circular treemap via the D3.js visualization library which allows us to intuitively visualize data and their parent clusters that they belong to. The implementation of this treemap requires quite a lot of pre-processing and kernel feature hacking and therefore I have hidden this pre-work from you in the three hidden cells below. The first hidden cell involves preparation of the json file which shall be used as the input source for our D3.js plot while the second hidden cell creates the HTML structure for our embedded visual. The third hidden cell contains embedded javascript syntax for which the the D3 circular treemap is invoked."], "metadata": {"_uuid": "818debcb5c4ab55dd0c01cf5338e4aaa50a1d1e6", "_cell_guid": "6c5d6b30-fa44-46f9-b354-0574c1be9622"}, "cell_type": "markdown"}, {"source": ["# with pd.option_context('display.max_rows', None, 'display.max_columns', 3):\n", "#     print(holiday_events[['type','description']].apply(pd.Series.value_counts))"], "execution_count": null, "metadata": {"_uuid": "060589721e2786b39a3d4fe1d4dfa9a56003e1a7", "_cell_guid": "9c12769e-ce9b-4ced-876d-f5fd4fa6cde8", "collapsed": true}, "cell_type": "code", "outputs": []}, {"source": ["# Prepping the json file\n", "holiday_json = {\n", "\"name\": \"flare\",\n", "\"children\": [\n", "{\n", "\"name\": \"Additional\",\n", "\"children\":[\n", "{\"name\": \"Batalla de Pichincha\",       \"size\": 5.0},\n", "{\"name\": \"Cantonizacion de Cayambe\",   \"size\": 6.0},\n", "{\"name\": \"Cantonizacion de El Carmen\", \"size\": 6.0},\n", "{\"name\": \"Cantonizacion de Guaranda\",  \"size\": 6.0},\n", "{\"name\": \"Cantonizacion de Latacunga\", \"size\": 6.0},\n", "{\"name\": \"Cantonizacion de Libertad\",  \"size\": 6.0},\n", "{\"name\": \"Cantonizacion de Quevedo\",   \"size\": 6.0},\n", "{\"name\": \"Cantonizacion de Riobamba\",  \"size\": 6.0},\n", "{\"name\": \"Cantonizacion de Salinas\",   \"size\": 6.0},\n", "{\"name\": \"Cantonizacion del Puyo\",     \"size\": 6.0},\n", "{\"name\": \"Carnaval\",                   \"size\": 0.0},\n", "{\"name\": \"Dia de Difuntos\",            \"size\": 6.0},\n", "{\"name\": \"Dia de la Madre\",            \"size\": 5.0},\n", "{\"name\": \"Dia de la Madre-1\",          \"size\": 5.0},\n", "{\"name\": \"Dia del Trabajo\",             \"size\": 5.0},\n", "{\"name\": \"Fundacion de Guayaquil\",    \"size\": 5.0},\n", "{\"name\": \"Fundacion de Guayaquil-1\",  \"size\": 5.0},\n", "{\"name\": \"Fundacion de Quito\",        \"size\": 6.0},\n", "{\"name\": \"Fundacion de Quito-1\",      \"size\": 6.0},\n", "{\"name\": \"Navidad+1                                      \", \"size\": 6.0},\n", "{\"name\": \"Navidad-1                                      \", \"size\": 6.0},\n", "{\"name\": \"Navidad-2                                      \", \"size\": 6.0},\n", "{\"name\": \"Navidad-3                                      \", \"size\": 6.0},\n", "{\"name\": \"Navidad-4                                      \", \"size\": 6.0},\n", "]\n", "},\n", "{\n", "\"name\":  \"Holiday\",\n", "\"children\":[\n", "{\"name\": \"Fundacion de Ambato\",       \"size\": 6.0},\n", "{\"name\": \"Fundacion de Cuenca\",       \"size\": 7.0},\n", "{\"name\": \"Fundacion de Esmeraldas\",   \"size\": 6.0},\n", "{\"name\": \"Fundacion de Ibarra\",       \"size\": 7.0},\n", "{\"name\": \"Fundacion de Loja\",         \"size\": 6.0},\n", "{\"name\": \"Fundacion de Machala\",      \"size\": 6.0},\n", "{\"name\": \"Fundacion de Manta\",        \"size\": 6.0},\n", "{\"name\": \"Fundacion de Riobamba\",     \"size\": 6.0},\n", "{\"name\": \"Fundacion de Santo Domingo\", \"size\": 6.0}\n", "]\n", "},\n", "{\n", "\"name\": \"Event\",\n", "\"children\": [\n", "{\"name\": \"Inauguracion Mundial de futbol Brasil          \", \"size\": 1.0},\n", "{\"name\": \"Independencia de Ambato                        \", \"size\": 6.0},\n", "{\"name\": \"Independencia de Cuenca                        \", \"size\": 6.0},\n", "{\"name\": \"Independencia de Guaranda                      \", \"size\": 6.0},\n", "{\"name\": \"Independencia de Guayaquil                     \", \"size\": 6.0},\n", "{\"name\": \"Independencia de Latacunga                     \", \"size\": 6.0},\n", "{\"name\": \"Mundial de futbol Brasil: Cuartos de Final     \", \"size\": 2.0},\n", "{\"name\": \"Mundial de futbol Brasil: Ecuador-Francia      \", \"size\": 1.0},\n", "{\"name\": \"Mundial de futbol Brasil: Ecuador-Honduras     \", \"size\": 1.0},\n", "{\"name\": \"Mundial de futbol Brasil: Ecuador-Suiza        \", \"size\": 1.0},\n", "{\"name\": \"Mundial de futbol Brasil: Final                \", \"size\": 1.0},\n", "{\"name\": \"Mundial de futbol Brasil: Octavos de Final     \", \"size\": 4.0},\n", "{\"name\": \"Mundial de futbol Brasil: Semifinales          \", \"size\": 2.0},\n", "{\"name\": \"Mundial de futbol Brasil: Tercer y cuarto lugar\", \"size\": 1.0},\n", "{\"name\": \"Navidad                                        \", \"size\": 6.0},\n", "{\"name\": \"Primer Grito de Independencia                  \", \"size\": 6.0},\n", "{\"name\": \"Primer dia del ano                             \", \"size\": 5.0},\n", "{\"name\": \"Primer dia del ano-1                           \", \"size\": 5.0},\n", "{\"name\": \"Black Friday\",               \"size\": 3.0},\n", "{\"name\": \"Cyber Monday\",               \"size\": 3.0},\n", "{\"name\": \"Provincializacion Santa Elena                  \", \"size\": 6.0},\n", "{\"name\": \"Provincializacion de Cotopaxi                  \", \"size\": 6.0},\n", "{\"name\": \"Provincializacion de Imbabura                  \", \"size\": 6.0},\n", "{\"name\": \"Provincializacion de Santo Domingo             \", \"size\": 6.0},\n", "{\"name\": \"Terremoto Manabi                               \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+1                             \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+10                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+11                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+12                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+13                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+14                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+15                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+16                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+17                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+18                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+19                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+2                             \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+20                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+21                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+22                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+23                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+24                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+25                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+26                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+27                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+28                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+29                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+3                             \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+30                            \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+4                             \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+5                             \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+6                             \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+7                             \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+8                             \", \"size\": 1.0},\n", "{\"name\": \"Terremoto Manabi+9                             \", \"size\": 1.0}\n", "]\n", "},\n", "{\n", "\"name\": \"Transfer\",\n", "\"children\":[\n", "{\"name\": \"Traslado Batalla de Pichincha         \", \"size\": 2.0},\n", "{\"name\": \"Traslado Fundacion de Guayaquil       \", \"size\": 1.0},\n", "{\"name\": \"Traslado Fundacion de Quito           \", \"size\": 1.0},\n", "{\"name\": \"Traslado Independencia de Guayaquil   \", \"size\": 3.0},\n", "{\"name\": \"Traslado Primer Grito de Independencia\", \"size\": 2.0},\n", "{\"name\": \"Traslado Primer dia del ano           \", \"size\": 1.0},\n", "{\"name\": \"Viernes Santo                         \", \"size\": 5.0}\n", "]\n", "},\n", "    {\n", "\"name\": \"Bridge\",\n", "\"children\":[\n", "{\"name\": \"Puente Dia de Difuntos                         \", \"size\": 1.0},\n", "{\"name\": \"Puente Navidad                                 \", \"size\": 2.0},\n", "{\"name\": \"Puente Primer dia del ano                      \", \"size\": 2.0},\n", "]\n", "},\n", "{\n", "\"name\": \"Work Day\",\n", "\"children\":[\n", "    {\"name\": \"Recupero puente Navidad\", \"size\": 2.0},\n", "    {\"name\": \"ecupero puente primer dia del ano\", \"size\": 2.0},\n", "    {\"name\": \"Recupero Puente Navidad\", \"size\": 2.0},\n", "    {\"name\": \"Recupero Puente Primer dia del ano\", \"size\": 2.0},\n", "    {\"name\": \"Recupero Puente Dia de Difuntos\", \"size\": 2.0}\n", "]\n", "}\n", "] \n", "}                          "], "execution_count": null, "metadata": {"_uuid": "99b4b7cff46488020e073ef08fc97df5a4572c63", "_cell_guid": "2bc9fbf9-1ba5-461a-807c-19bf38f22608", "collapsed": true, "_kg_hide-input": true}, "cell_type": "code", "outputs": []}, {"source": ["# dumping the holiday_events data into a json file\n", "with open('output.json', 'w') as outfile:  \n", "    json.dump(holiday_json, outfile)\n", "pd.read_json('output.json').head()\n", "\n", "#Embedding the html string\n", "html_string = \"\"\"\n", "<!DOCTYPE html>\n", "<meta charset=\"utf-8\">\n", "<style>\n", "\n", ".node {\n", "  cursor: pointer;\n", "}\n", "\n", ".node:hover {\n", "  stroke: #000;\n", "  stroke-width: 1.5px;\n", "}\n", "\n", ".node--leaf {\n", "  fill: white;\n", "}\n", "\n", ".label {\n", "  font: 11px \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n", "  text-anchor: middle;\n", "  text-shadow: 0 1px 0 #fff, 1px 0 0 #fff, -1px 0 0 #fff, 0 -1px 0 #fff;\n", "}\n", "\n", ".label,\n", ".node--root,\n", ".node--leaf {\n", "  pointer-events: none;\n", "}\n", "\n", "</style>\n", "<svg width=\"760\" height=\"760\"></svg>\n", "\"\"\""], "execution_count": null, "metadata": {"_uuid": "864255ccad3efd30319518036670c893b5cc3a1b", "_cell_guid": "656e997a-b294-4146-babe-9e733242149c", "collapsed": true, "_kg_hide-input": true}, "cell_type": "code", "outputs": []}, {"source": ["# Finally embed the D3.js to produce the circular treemap\n", "js_string=\"\"\"\n", " require.config({\n", "    paths: {\n", "        d3: \"https://d3js.org/d3.v4.min\"\n", "     }\n", " });\n", "\n", "  require([\"d3\"], function(d3) {\n", "\n", "   console.log(d3);\n", "\n", "var svg = d3.select(\"svg\"),\n", "    margin = 20,\n", "    diameter = +svg.attr(\"width\"),\n", "    g = svg.append(\"g\").attr(\"transform\", \"translate(\" + diameter / 2 + \",\" + diameter / 2 + \")\");\n", "\n", "var color = d3.scaleSequential(d3.interpolateViridis)\n", "    .domain([-4, 4]);\n", "\n", "var pack = d3.pack()\n", "    .size([diameter - margin, diameter - margin])\n", "    .padding(2);\n", "\n", "d3.json(\"output.json\", function(error, root) {\n", "  if (error) throw error;\n", "\n", "  root = d3.hierarchy(root)\n", "      .sum(function(d) { return d.size; })\n", "      .sort(function(a, b) { return b.value - a.value; });\n", "\n", "  var focus = root,\n", "      nodes = pack(root).descendants(),\n", "      view;\n", "\n", "  var circle = g.selectAll(\"circle\")\n", "    .data(nodes)\n", "    .enter().append(\"circle\")\n", "      .attr(\"class\", function(d) { return d.parent ? d.children ? \"node\" : \"node node--leaf\" : \"node node--root\"; })\n", "      .style(\"fill\", function(d) { return d.children ? color(d.depth) : null; })\n", "      .on(\"click\", function(d) { if (focus !== d) zoom(d), d3.event.stopPropagation(); });\n", "\n", "  var text = g.selectAll(\"text\")\n", "    .data(nodes)\n", "    .enter().append(\"text\")\n", "      .attr(\"class\", \"label\")\n", "      .style(\"fill-opacity\", function(d) { return d.parent === root ? 1 : 0; })\n", "      .style(\"display\", function(d) { return d.parent === root ? \"inline\" : \"none\"; })\n", "      .text(function(d) { return d.data.name; });\n", "\n", "  var node = g.selectAll(\"circle,text\");\n", "\n", "  svg\n", "      .style(\"background\", color(-1))\n", "      .on(\"click\", function() { zoom(root); });\n", "\n", "  zoomTo([root.x, root.y, root.r * 2 + margin]);\n", "\n", "  function zoom(d) {\n", "    var focus0 = focus; focus = d;\n", "\n", "    var transition = d3.transition()\n", "        .duration(d3.event.altKey ? 7500 : 750)\n", "        .tween(\"zoom\", function(d) {\n", "          var i = d3.interpolateZoom(view, [focus.x, focus.y, focus.r * 2 + margin]);\n", "          return function(t) { zoomTo(i(t)); };\n", "        });\n", "\n", "    transition.selectAll(\"text\")\n", "      .filter(function(d) { return d.parent === focus || this.style.display === \"inline\"; })\n", "        .style(\"fill-opacity\", function(d) { return d.parent === focus ? 1 : 0; })\n", "        .on(\"start\", function(d) { if (d.parent === focus) this.style.display = \"inline\"; })\n", "        .on(\"end\", function(d) { if (d.parent !== focus) this.style.display = \"none\"; });\n", "  }\n", "\n", "  function zoomTo(v) {\n", "    var k = diameter / v[2]; view = v;\n", "    node.attr(\"transform\", function(d) { return \"translate(\" + (d.x - v[0]) * k + \",\" + (d.y - v[1]) * k + \")\"; });\n", "    circle.attr(\"r\", function(d) { return d.r * k; });\n", "  }\n", "});\n", "  });\n", " \"\"\""], "execution_count": null, "metadata": {"_uuid": "292ee7e31491002e45460819114413de2b3d2157", "_cell_guid": "5da85402-7e09-4ff1-a382-1986ea1122e0", "collapsed": true, "_kg_hide-input": true}, "cell_type": "code", "outputs": []}, {"source": ["**Displaying the HTML and D3 visuals on IPython notebook**\n", "\n", "Finally we invoke the following code to display the D3 visuals where I will be plotting the various descriptions/names under their parent holiday/event cluster types. The following plot is interactive and zoomable so please click onto the circles to zoom in and click anywhere outside to zoom back to the default view."], "metadata": {"_uuid": "5da75e8dcf414aabd18e93413905cedb63fb5187", "_cell_guid": "311edd5a-0c79-487e-8454-56e79f13124f"}, "cell_type": "markdown"}, {"source": ["h = display(HTML(html_string))\n", "j = IPython.display.Javascript(js_string)\n", "IPython.display.display_javascript(j)"], "execution_count": null, "metadata": {"_uuid": "4aae2fd83d72f761f34c6f2f56e79245a37e48b9", "_cell_guid": "66b0a945-572e-41ac-96a4-614a0d87fefc"}, "cell_type": "code", "outputs": []}, {"source": ["holiday_events.type.unique()"], "execution_count": null, "metadata": {"_uuid": "604d94f9914eec70b8ffcca9f1bdc2bed942b3ea", "_cell_guid": "742ab965-b7a4-4e6e-80da-93529e2f560e"}, "cell_type": "code", "outputs": []}, {"source": ["## 2d. Transactions data\n"], "metadata": {"_uuid": "1ac9c30bb9c2824871f3f58125e3c9f32f92c04f", "_cell_guid": "3874bf7e-23f5-4d9b-9041-c2057ea1fc1c"}, "cell_type": "markdown"}, {"source": ["\n", "**PERIODICITY IN TRANSACTION PATTERN**\n", "\n", "Inspecting the transactions file, we must be aware that the transactional data is only included for the training timeframe according to the competition's data dictionary. Furthermore the columns in this file relate only to the count of sales transactions for "], "metadata": {"_uuid": "0d4857a7715e4b5ab9d3f113b6d4ebfe90823382", "_cell_guid": "8e60cf1f-1d2b-477c-aa13-2ee0cee17897"}, "cell_type": "markdown"}, {"source": ["print(transactions.head(3))\n", "print(\"=\"*60)\n", "print(\"There are {0} rows and {1} columns in the transactions data\".\n", "      format(transactions.shape[0], transactions.shape[1]))"], "execution_count": null, "metadata": {"_uuid": "55407ca5bfee024a9fc66a82fb4e6ec789e4ea7a", "_cell_guid": "d81126d5-53a9-4e06-9009-3876d11562f2"}, "cell_type": "code", "outputs": []}, {"source": ["plt.style.use('seaborn-white')\n", "plt.figure(figsize=(13,11))\n", "plt.plot(transactions.date.values, transactions.transactions.values, color='darkblue')\n", "plt.axvline(x='2015-12-23',color='red',alpha=0.3)\n", "plt.axvline(x='2016-12-23',color='red',alpha=0.3)\n", "plt.axvline(x='2014-12-23',color='red',alpha=0.3)\n", "plt.axvline(x='2013-12-23',color='red',alpha=0.3)\n", "plt.axvline(x='2013-05-12',color='green',alpha=0.2, linestyle= '--')\n", "plt.axvline(x='2015-05-10',color='green',alpha=0.2, linestyle= '--')\n", "plt.axvline(x='2016-05-08',color='green',alpha=0.2, linestyle= '--')\n", "plt.axvline(x='2014-05-11',color='green',alpha=0.2, linestyle= '--')\n", "plt.axvline(x='2017-05-14',color='green',alpha=0.2, linestyle= '--')\n", "plt.ylim(-50, 10000)\n", "plt.title(\"Distribution of transactions per day from 2013 till 2017\")\n", "plt.ylabel('transactions per day', fontsize= 16)\n", "plt.xlabel('Date', fontsize= 16)\n", "plt.show()"], "execution_count": null, "metadata": {"_uuid": "0140db838832b5c6ac5c43cec5e9ae195ad8483e", "_cell_guid": "357aea86-5f53-4055-a4fc-daca14e509be"}, "cell_type": "code", "outputs": []}, {"source": ["**Takeaway from the plots**\n", "\n", "Interestingly when plotting transactions on a year-to-year basis, we can already pick out two different periodic spikes in transactions from the data (one in the solid Red line and the other the dotted Green line). The bigger yearly periodic spike in transactions seem to occur at the end of the year in December, specifically on the 23 December every year-end. Perhaps this is due to some sort of Christmas sale/discount that Corporacion Favorita holds every December thereby explaining the bump in transactions on this date. The weaker periodic spike given by the green dotted lines seems to occur around the middle of the months of May , specifically on the sunday in the second week of May. Perhaps another similar scheme is being implemented by Corporacion Favorita on those sundays."], "metadata": {"_uuid": "b11b8a4a186d39cc89fed3c5e565637190313a76", "_cell_guid": "fa0a11ce-eea0-4eb1-acf6-5e34a6cc12ab"}, "cell_type": "markdown"}, {"source": ["## 2e. Items data\n", "\n", "Moving onto the \"items.csv\" file, we can see that there are not many columns in the data. There is a unique item number identifier \"item_nbr\" which I presume relates to mappings to grocery items such as \"Housebrand tomatoes pack of 6\" or maybe \"4 blueberry muffins\" etc. The family column should be pretty self-explanatory in the sense that this relates to parent category that that particular item relates to."], "metadata": {"_uuid": "228d48b321425cde9803c66f28016bc9a895b928", "_cell_guid": "9bdb1afd-c4b8-4aa9-bd38-72f3c2371ca2"}, "cell_type": "markdown"}, {"source": ["items.head()"], "execution_count": null, "metadata": {"_uuid": "42ce19d3640226fdb4a358968064062da1a8c458", "_cell_guid": "b1a59a4d-5182-4faf-98f9-c795667ec1e3"}, "cell_type": "code", "outputs": []}, {"source": ["x, y = (list(x) for x in zip(*sorted(zip(items.family.value_counts().index, \n", "                                         items.family.value_counts().values), \n", "                                        reverse = False)))\n", "trace2 = go.Bar(\n", "    y=items.family.value_counts().values,\n", "    x=items.family.value_counts().index,\n", "    marker=dict(\n", "        color=items.family.value_counts().values,\n", "        colorscale = 'Portland',\n", "        reversescale = False\n", "    ),\n", "    orientation='v',\n", ")\n", "\n", "layout = dict(\n", "    title='Counts of items per family category',\n", "     width = 800, height = 800,\n", "    yaxis=dict(\n", "        showgrid=False,\n", "        showline=False,\n", "        showticklabels=True,\n", "#         domain=[0, 0.85],\n", "    ))\n", "\n", "fig1 = go.Figure(data=[trace2])\n", "fig1['layout'].update(layout)\n", "py.iplot(fig1, filename='plots')"], "execution_count": null, "metadata": {"_uuid": "bf78473155f05a66ca863f4a62f679f192f36b15", "_cell_guid": "8f3828e8-aa94-4fc9-9390-9cb723a08861", "_kg_hide-input": true, "scrolled": true}, "cell_type": "code", "outputs": []}, {"source": ["**Takeaways from the plot**\n", "\n", "As we can see from this barplot, the y-axis shows the counts of items while the x-axis displays the different family categories that the various retail items fall under, sorted from largest number of item counts to smallest number of items. As we can see from the plot, the top 3 family categories are the GROCERY I, BEVERAGES and CLEANING categories. "], "metadata": {"_uuid": "db93f32c724003b2218a671d502952cfa6b2d888", "_cell_guid": "8deef222-fc5e-4017-b2b8-7a7e79361e75"}, "cell_type": "markdown"}, {"source": ["x, y = (list(x) for x in zip(*sorted(zip(items['class'].value_counts().index, \n", "                                         items['class'].value_counts().values), \n", "                                        reverse = False)))\n", "trace2 = go.Bar(\n", "    x=items['class'].value_counts().index,\n", "    y=items['class'].value_counts().values,\n", "    marker=dict(\n", "        color=items['class'].value_counts().values,\n", "        colorscale = 'Jet',\n", "        reversescale = True\n", "    ),\n", "    orientation='v',\n", ")\n", "\n", "layout = dict(\n", "    title='Number of items attributed to a particular item class',\n", "     width = 800, height = 1400,\n", "    yaxis=dict(\n", "        showgrid=False,\n", "        showline=False,\n", "        showticklabels=True,\n", "#         domain=[0, 0.85],\n", "    ))\n", "\n", "fig1 = go.Figure(data=[trace2])\n", "fig1['layout'].update(layout)\n", "py.iplot(fig1, filename='plots')"], "execution_count": null, "metadata": {"_uuid": "38eb020f75e51481d7a55599655660dfa34dd626", "_cell_guid": "337167c7-57be-4364-a630-7866ad538013", "_kg_hide-input": true}, "cell_type": "code", "outputs": []}, {"source": ["**Takeaways from the plot**\n", "\n", "Plotting the count of "], "metadata": {"_uuid": "758354ad390fbcd9e387adf0af5800bb57419aa7", "_cell_guid": "d4ce0615-c97d-410f-a149-a3341c1f9f54"}, "cell_type": "markdown"}, {"source": ["items.head()"], "execution_count": null, "metadata": {"_uuid": "c5bc61fd6bb1535b31c5dd803e37d676c480dc35", "_cell_guid": "328189ca-6c73-4997-98df-b4f764849329"}, "cell_type": "code", "outputs": []}, {"source": ["plt.style.use('seaborn-white')\n", "fam_perishable = items.groupby(['family', 'perishable']).size()\n", "fam_perishable.unstack().plot(kind='bar',stacked=True, colormap= 'coolwarm', figsize=(12,10),  grid=False)\n", "plt.title('Stacked Barplot of locale name against event type')\n", "plt.ylabel('Count of entries')\n", "plt.show()"], "execution_count": null, "metadata": {"_uuid": "7221853e0e0a7ea50b390dd0bc1e152c6bfcc0cc", "_cell_guid": "aee3bdc2-d498-4f75-9d71-4a66d8f0fb8d"}, "cell_type": "code", "outputs": []}, {"source": ["**Takeaways from the plot**\n", "\n"], "metadata": {"_uuid": "024f93f1cb2d2f6f1fa1b0527aa1408be44f6abb", "_cell_guid": "f4cea522-3c2f-4d21-8660-be65cbb6d844"}, "cell_type": "markdown"}, {"source": ["# 3. Training Data exploration\n", "\n", "Finally we can now conduct an analysis of the training data. Since the entire training set is so large, as observed beforei"], "metadata": {"_uuid": "aa4196277ec16070ad7be3cfe5bf3ab8c12debc0", "_cell_guid": "88efa69d-9120-4a9c-8e84-5b01575033ad", "collapsed": true}, "cell_type": "markdown"}, {"source": ["train.head()"], "execution_count": null, "metadata": {"_uuid": "49c5909c93c11550ea6d5f6fd14b2be5ac1c6be3", "_cell_guid": "fd088fa4-7247-4031-811b-738f0a094b49"}, "cell_type": "code", "outputs": []}, {"source": ["plt.style.use('seaborn-deep')\n", "plt.figure(figsize=(13,11))\n", "plt.plot(train.date.values, train.unit_sales)\n", "plt.ylim(-50, 10000)\n", "plt.ylabel('transactions per day')\n", "plt.xlabel('Date')\n", "plt.show()"], "execution_count": null, "metadata": {"_uuid": "03af8fc136a4a0dc4c218c72da2854416df0d04c", "_cell_guid": "d42ee67e-d0a3-42a8-b57d-44ffd6acceee"}, "cell_type": "code", "outputs": []}, {"source": ["*TO BE CONTINUED*"], "metadata": {"_uuid": "c3e206e37456d3eef90c7010b143c4ff423e649a", "_cell_guid": "f23c42bd-5552-43c5-ae4a-18c97b88e96b", "collapsed": true}, "cell_type": "markdown"}], "metadata": {"language_info": {"pygments_lexer": "ipython3", "codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "name": "python", "mimetype": "text/x-python", "nbconvert_exporter": "python", "version": "3.6.3"}, "kernelspec": {"language": "python", "name": "python3", "display_name": "Python 3"}}, "nbformat_minor": 1}