{"metadata": {"kernelspec": {"language": "python", "name": "python3", "display_name": "Python 3"}, "language_info": {"nbconvert_exporter": "python", "codemirror_mode": {"name": "ipython", "version": 3}, "name": "python", "version": "3.6.3", "file_extension": ".py", "pygments_lexer": "ipython3", "mimetype": "text/x-python"}}, "nbformat_minor": 1, "nbformat": 4, "cells": [{"metadata": {"_uuid": "0040269d00cdb7f910368343b615af0916972195", "_cell_guid": "481c640b-90d4-4f2f-a698-bd003b38fb7e"}, "cell_type": "markdown", "source": ["# Objective\n", "[Heads or Tails](https://www.kaggle.com/headsortails/shopping-for-insights-favorita-eda) made a wonderful EDA in R two months ago and analyzed almost every aspects of the data  in Favorita competetion. No wonder he got more than 250 upvote with the kernel. For a beginner in Kaggle and a late paticipator in Favorita challenge, I felt astonished when I saw his beautiful plots and detailed analysis. I definitly want to go to the daRk side to enjoy all the fantastic features from ggplot2 library. However, since I already chose Python as my primary language in ML research, I just want to absorb all his ideas and skills in data analysis and visualization. What I can do is to reproduce his work in Python and practice my coding skills through this process. I found there are still many minor features I don't know how to express in the plots. Hope you can share your comments and methods after reading my kernel. Thanks."]}, {"metadata": {"_uuid": "5b654e5713b67bc8114258f44f7c0f63d018670b", "_cell_guid": "a6ab4e5d-ef36-4bee-ab37-b3a2aabe0890"}, "cell_type": "markdown", "source": ["## 1. Loading data"]}, {"metadata": {"_uuid": "c67563d59ca88f6990b23a21cc7027daea4b13a9", "collapsed": true, "_cell_guid": "b91e7856-6f64-4c30-9fbe-4cc0d1bf3342"}, "execution_count": null, "cell_type": "code", "source": ["import gc\n", "import time\n", "import numpy as np\n", "import pandas as pd\n", "\n", "from scipy.sparse import csr_matrix, hstack\n", "\n", "from sklearn.linear_model import Ridge, Lasso\n", "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n", "from sklearn.preprocessing import LabelBinarizer\n", "from sklearn.model_selection import train_test_split, cross_val_score, KFold, GridSearchCV\n", "import lightgbm as lgb\n", "from xgboost import XGBRegressor\n", "import xgboost\n", "from time import time\n", "from sklearn.metrics import make_scorer\n", "\n", "import matplotlib.pyplot as plt\n", "import seaborn as sns\n", "%matplotlib inline \n", "from scipy.stats import norm\n", "import random\n", "import time\n"], "outputs": []}, {"metadata": {"_uuid": "36b1c57f029ce908aeb02756287b22d9e66ab9e9", "_cell_guid": "dbcf4fec-891c-461a-9f02-c21c386ef5d6"}, "execution_count": null, "cell_type": "code", "source": ["from subprocess import check_output\n", "print (check_output(['ls', '../input']).decode('utf8'))"], "outputs": []}, {"metadata": {"_uuid": "a8fe31900089455ae9ed1dda378722b9d0ccd1bd", "_cell_guid": "02ad20d2-0cf7-4500-9add-b52c66da0cf6"}, "cell_type": "markdown", "source": ["The `train` dataset include more than 125 million lines. It takes forever to analyze them and plot. So I will only analyze 5% of them. "]}, {"metadata": {"_uuid": "2491d6f0fbe9b750c3728f01ddbf2ada47baa9e6", "_cell_guid": "93d71d35-a377-40b6-81dd-c62a00512d6e"}, "execution_count": null, "cell_type": "code", "source": ["train = pd.read_csv(\"../input/train.csv\")"], "outputs": []}, {"metadata": {"_uuid": "4af716e83c9a4ae96b8338eddb177ce56d45dcd9", "_cell_guid": "70f78449-f6b7-4838-b676-78ad76139269"}, "execution_count": null, "cell_type": "code", "source": ["train_sample = train.sample(frac = 0.05)\n", "train_sample.head()"], "outputs": []}, {"metadata": {"_uuid": "ec9e47cf615bfbaf5a16c222fa25119bd7a3ccf0", "collapsed": true, "_cell_guid": "6abf47a5-d9ad-44e3-8096-f646f2d7aae8"}, "execution_count": null, "cell_type": "code", "source": ["holiday_events = pd.read_csv('../input/holidays_events.csv')\n", "items = pd.read_csv('../input/items.csv')\n", "oil = pd.read_csv('../input/oil.csv')\n", "stores = pd.read_csv('../input/stores.csv')\n", "test = pd.read_csv('../input/test.csv')\n", "\n", "transactions = pd.read_csv('../input/transactions.csv')"], "outputs": []}, {"metadata": {"_uuid": "c544d6d911e4f83576b3f4606e226a05f122fd49", "_cell_guid": "a2ed282b-23d6-405e-b9ed-96ac52e0df3d"}, "execution_count": null, "cell_type": "code", "source": ["print (holiday_events.head())\n", "print (holiday_events.shape)\n", "print (holiday_events.describe())"], "outputs": []}, {"metadata": {"_uuid": "2440dc14d3884c250fab8e62d53569f33293b186", "_cell_guid": "85c20f95-2d3b-4b13-98d4-f99d53831803"}, "execution_count": null, "cell_type": "code", "source": ["print (items.head())\n", "print (items.shape)\n", "print (items.describe())"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["print (len(items['family'].value_counts().index))\n", "print (len(items['class'].value_counts().index))\n", "print (items['perishable'].value_counts())"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["From `item` dataset, we can find:\n", "    1. There are 4100 items.\n", "    2. These items can be classfied to 33 families (e.g. GROCERY I)\n", "    3. There are 337 classes\n", "    4. 3114 of them are not perishable, 986 of them are perishable"]}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["print (oil.head())\n", "print (oil.shape)\n", "print (oil.describe())"], "outputs": []}, {"metadata": {"scrolled": true}, "execution_count": null, "cell_type": "code", "source": ["import datetime\n", "train_sample['date'] = train_sample['date'].apply(datetime.datetime.strptime, args = (\"%Y-%m-%d\",))\n", "train_sample.sort_values(by = 'date', inplace = True)\n", "print (train_sample.head())\n", "print (train_sample.shape)\n", "print (train_sample.describe())\n", "print (train_sample.isnull().sum())\n"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["train_sample['onpromotion'].fillna(value = 'missing', inplace = True)\n", "#train_sample['onpromotion'].isnull().sum()\n", "print (train_sample['onpromotion'].value_counts())"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["print (test.head())\n", "print (test.shape)\n", "print (test.describe())\n", "print (test.isnull().sum())"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["print (test['onpromotion'].value_counts())"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["# 2. Individual feature visualisations\n", "\n", "In this step we will get an visual overview of the data by plotting the individual feature distribution for each data set sepeartely."]}, {"metadata": {}, "cell_type": "markdown", "source": ["## 3. Training data"]}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["daily_sale = pd.DataFrame()\n", "daily_sale['count'] = train_sample['date'].value_counts()\n", "daily_sale['date'] = train_sample['date'].value_counts().index\n", "daily_sale = daily_sale.sort_values(by = 'date')\n", "print (daily_sale.head(3))\n", "unit_sale = pd.DataFrame()\n", "unit_sale['count'] = train_sample[train['unit_sales']>0]['unit_sales'].value_counts()\n", "unit_sale['positive_unit_sales'] = train_sample[train['unit_sales']>0]['unit_sales'].value_counts().index\n", "unit_sale = unit_sale.sort_values(by = 'positive_unit_sales')"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["promotion_count = pd.DataFrame()\n", "promotion_count ['count'] = train_sample['onpromotion'].value_counts()"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["fig, (ax1,ax2,ax3) = plt.subplots(1,3,figsize = (18,5))\n", "sns.set_style('darkgrid')\n", "ax1.plot(daily_sale['date'], daily_sale['count'])\n", "ax1.set_xlabel('date', fontsize=15)\n", "ax1.set_ylabel('count', fontsize=15)\n", "ax1.tick_params(labelsize=15)\n", "sns.barplot(x = promotion_count.index, y = promotion_count['count'],ax = ax2)\n", "ax2.set_ylabel('count', fontsize = 15)\n", "ax2.set_xlabel ('onpromotion',fontsize =15)\n", "ax2.tick_params(labelsize = 15)\n", "ax2.ticklabel_format(style = 'sci',scilimits = (0,0), axis = 'y')\n", "plt.subplot(1,3,3)\n", "plt.loglog(unit_sale['positive_unit_sales'], unit_sale['count'])\n", "plt.ylabel('count', fontsize = 15)\n", "plt.xlabel('positive_unit_sales', fontsize = 15)\n", "plt.xticks(fontsize = 15)\n", "plt.yticks(fontsize = 15)\n", "plt.show()"], "outputs": []}, {"metadata": {"collapsed": true}, "execution_count": null, "cell_type": "code", "source": ["store_count = pd.DataFrame()\n", "store_count['count'] = train_sample['store_nbr'].value_counts().sort_index()"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["fig, ax = plt.subplots(figsize = (18, 3))\n", "sns.barplot(x = store_count.index, y = store_count['count'], ax = ax)\n", "ax.set_ylabel('count', fontsize = 15)\n", "ax.set_xlabel('store_nbr',fontsize = 15)\n", "ax.tick_params(labelsize=15)"], "outputs": []}, {"metadata": {"collapsed": true}, "execution_count": null, "cell_type": "code", "source": ["item_count = pd.DataFrame()\n", "item_count['count'] = train_sample['item_nbr'].value_counts().sort_index()\n", "plt.plot(item_count.index)"], "outputs": []}, {"metadata": {"scrolled": true}, "execution_count": null, "cell_type": "code", "source": ["fig, ax = plt.subplots(figsize = (18, 3))\n", "sns.barplot(x = item_count.index, y = item_count['count'], ax = ax)\n", "ax.set_ylabel('count', fontsize = 15)\n", "ax.set_xlabel('item_nbr',fontsize = 15)\n", "ax.tick_params(axis = 'x',which = 'both',top = 'off', bottom = 'off', labelbottom = 'off')"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["neg_unit_sale = pd.DataFrame()\n", "neg_unit_sale['unit_sales'] = (-train_sample[train_sample['unit_sales'] < 0]['unit_sales'])\n", "neg_unit_sale.head()"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["fig, ax = plt.subplots(figsize = (18, 5))\n", "#ax.set_xscale('log')\n", "np.log(neg_unit_sale['unit_sales']).plot.hist(ax = ax, log = True,edgecolor = 'white', bins = 50)\n", "ax.set_xlabel('neg_unit_sales (log10 scale)', fontsize=15)\n", "ax.set_ylabel('count', fontsize=15)\n", "ax.tick_params(labelsize=15)"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["## 2.2 Stores"]}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["city_count = pd.DataFrame()\n", "city_count['count'] = stores['city'].value_counts().sort_index()\n", "fig, (ax1,ax2) = plt.subplots(1,2,figsize = (18, 4))\n", "g = sns.barplot(x = city_count.index, y = city_count['count'], ax = ax1)\n", "ax1.set_ylabel('count', fontsize = 15)\n", "ax1.set_xlabel('city',fontsize = 15)\n", "ax1.tick_params(labelsize=15)\n", "g.set_xticklabels(city_count.index, rotation = 45)\n", "state_count =pd.DataFrame()\n", "state_count['count'] = stores['state'].value_counts().sort_index()\n", "g2 = sns.barplot(x = state_count.index, y = state_count['count'], ax = ax2)\n", "ax2.set_ylabel('count', fontsize = 15)\n", "ax2.set_xlabel('state',fontsize = 15)\n", "ax2.tick_params(labelsize=15)\n", "g2.set_xticklabels(state_count.index, rotation = 45)"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["type_count = pd.DataFrame()\n", "type_count['count'] = stores['type'].value_counts().sort_index()\n", "fig, (ax1,ax2) = plt.subplots(1,2,figsize = (18, 4))\n", "g = sns.barplot(x = type_count.index, y = type_count['count'], ax = ax1)\n", "ax1.set_ylabel('count', fontsize = 15)\n", "ax1.set_xlabel('type',fontsize = 15)\n", "ax.tick_params(labelsize=15)\n", "cluster_count = pd.DataFrame()\n", "cluster_count['count'] = stores['cluster'].value_counts().sort_index()\n", "g = sns.barplot(x = cluster_count.index, y = cluster_count['count'], ax = ax2)\n", "ax2.set_ylabel('count', fontsize = 15)\n", "ax2.set_xlabel('cluster',fontsize = 15)\n", "ax2.tick_params(labelsize=15)"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["import squarify\n"], "outputs": []}, {"metadata": {"scrolled": true}, "execution_count": null, "cell_type": "code", "source": ["fig, ax = plt.subplots(figsize = (16,8))\n", "grouped_city = np.log1p(stores.groupby(['city']).count())\n", "grouped_city['state'] = grouped_city.index\n", "current_palette = sns.color_palette()\n", "squarify.plot(sizes = grouped_city['store_nbr'], label = grouped_city.index, alpha = 0.8,color = current_palette)\n", "plt.rc('font', size = 25)\n", "plt.axis('off')"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["### For the treemap, I used squarify to show the `count` as areas of different `city` and `state`. I don't know how to plot sub treemap in a state square to illustrate  `city` count. If you know how to do it, please make a comment"]}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["fig, ax = plt.subplots(figsize = (16,8))\n", "grouped_state = np.log1p(stores.groupby(['state']).count())\n", "grouped_state['state'] = grouped_state.index\n", "current_palette = sns.color_palette()\n", "squarify.plot(sizes = grouped_state['store_nbr'], label = grouped_state.index, alpha = 0.8,color = current_palette)\n", "plt.rc('font', size = 20)\n", "plt.axis('off')"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["print (items.head())\n", "class_count = pd.DataFrame()\n", "class_count['count'] = items['class'].value_counts().sort_index()\n", "most_freq_class = items['class'].value_counts().head()\n", "perish_count = items['perishable'].value_counts()\n", "family_count = items['family'].value_counts().sort_index()"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["## 2.3 Items"]}, {"metadata": {"scrolled": true}, "execution_count": null, "cell_type": "code", "source": ["fig, (ax1, ax2, ax3) = plt.subplots(1,3,figsize = (18,3))\n", "sns.barplot(x = class_count.index, y = class_count['count'], ax = ax1)\n", "ax1.set_ylabel('count', fontsize = 15)\n", "ax1.set_xlabel('class',fontsize = 15)\n", "ax1.tick_params(labelsize = 6, axis = 'x',which = 'both',top = 'off', bottom = 'off', labelbottom = 'off')\n", "sns.barplot(x = most_freq_class.index, y = most_freq_class.values, ax = ax2)\n", "ax2.set_ylabel('n', fontsize = 15)\n", "ax2.set_xlabel('class-most frequent',fontsize = 15)\n", "ax2.tick_params(labelsize = 12)\n", "sns.barplot(x = perish_count.index, y = perish_count.values, ax = ax3)\n", "ax3.set_ylabel('count', fontsize = 15)\n", "ax3.set_xlabel('perish',fontsize = 15)\n", "ax3.tick_params(labelsize = 12)"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["fig, ax = plt.subplots(figsize = (18, 3))\n", "g = sns.barplot(x = family_count.index, y = family_count.values, ax = ax)\n", "ax.set_ylabel('count', fontsize = 15)\n", "ax.set_xlabel('family',fontsize = 15)\n", "ax.tick_params(labelsize = 12)\n", "g.set_xticklabels(family_count.index, rotation = 60)"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["fig, ax = plt.subplots(figsize = (16,8))\n", "grouped_family = np.log1p(items.groupby(['family']).count().head(20))\n", "current_palette = sns.color_palette()\n", "squarify.plot(sizes = grouped_family['class'], label = grouped_family.index, alpha = 0.8,color = current_palette)\n", "plt.rc('font', size = 12)\n", "plt.axis('off')"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["med_trans = transactions.groupby(['date']).median()\n", "med_trans['date'] = med_trans.index\n", "med_trans['date'] = med_trans['date'].apply(datetime.datetime.strptime, args = (\"%Y-%m-%d\",))\n", "med_trans.head()"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["## 2.4 Transactions\n", "\n", "I can reproduce Head and Tails' med_trans vs. date plot but the feature I missed is the trend curve. It seems very convinient and easy to fulfill in `R` with `geom_smooth` but I don't know how to do it in Python. Please let me know in the comment if you have a elegant solution. "]}, {"metadata": {"scrolled": true}, "execution_count": null, "cell_type": "code", "source": ["fig, ax = plt.subplots(figsize = (12,6))\n", "sns.set_style('darkgrid')\n", "plt.plot(med_trans['date'], med_trans['transactions'])\n", "#ax = sns.regplot(x='date', y='transactions', data = med_trans)\n", "ax.set_xlabel('date', fontsize=15)\n", "ax.set_ylabel('med_trans', fontsize=15)\n", "ax.tick_params(labelsize=15)"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["Since I don't know how to make smooth trend curve of time series plot, I don't have acess to the smoothed transations time series for each individual store. "]}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["transactions['date'] = transactions['date'].apply(datetime.datetime.strptime, args = (\"%Y-%m-%d\",))\n", "grouped_trans = transactions.groupby(['date','store_nbr']).sum().unstack()\n", "grouped_trans.head()\n", "fig, ax = plt.subplots(figsize = (12,6))\n", "grouped_trans.plot(ax = ax)"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": []}, {"metadata": {}, "cell_type": "markdown", "source": ["## 2.5 Oil\n", "\n", "Below is the plot of price over time together with its weekly changes (price - price 7 days later):"]}, {"metadata": {"collapsed": true}, "execution_count": null, "cell_type": "code", "source": ["oil['date'] = oil['date'].apply(datetime.datetime.strptime, args = (\"%Y-%m-%d\",))"], "outputs": []}, {"metadata": {"collapsed": true}, "execution_count": null, "cell_type": "code", "source": ["import statsmodels.formula.api as sm\n", "regression = (sm.ols(formula=\"dcoilwtico ~ date \", data=oil).fit())"], "outputs": []}, {"metadata": {"collapsed": true}, "execution_count": null, "cell_type": "code", "source": ["oil['trend'] = regression.predict(oil['date'])"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["fig, ax = plt.subplots(figsize = (18, 5))\n", "ax.plot(oil['date'],oil['dcoilwtico'], label = 'oilprice')\n", "ax.set\n", "ax.set_xlabel('date', fontsize=15)\n", "ax.set_ylabel('oilprice', fontsize=15)\n", "ax.tick_params(labelsize=15)"], "outputs": []}, {"metadata": {"collapsed": true}, "execution_count": null, "cell_type": "code", "source": ["lag7 = oil['dcoilwtico'] - oil['dcoilwtico'].shift(7)"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["fig, ax = plt.subplots(figsize = (18, 5))\n", "ax.plot(oil['date'],lag7, label = 'weekly variations in oil price')\n", "ax.set\n", "ax.set_xlabel('date', fontsize=15)\n", "ax.set_ylabel('weekly variations', fontsize=15)\n", "ax.tick_params(labelsize=15)"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["## 2.6 Holidays"]}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["holiday_events.head()"], "outputs": []}, {"metadata": {"collapsed": true}, "execution_count": null, "cell_type": "code", "source": ["holiday_events['date'] = holiday_events['date'].apply(datetime.datetime.strptime, args = (\"%Y-%m-%d\",))"], "outputs": []}, {"metadata": {"collapsed": true}, "execution_count": null, "cell_type": "code", "source": ["type_count = holiday_events['type'].value_counts().sort_index()\n", "locale_count = holiday_events['locale'].value_counts().sort_index()"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["fig, (ax1, ax2) = plt.subplots(1,2,figsize = (18, 4))\n", "sns.barplot(x = type_count.index, y = type_count.values, ax = ax1)\n", "ax1.set_ylabel('count', fontsize = 15)\n", "ax1.set_xlabel('type',fontsize = 15)\n", "sns.barplot(x = locale_count.index, y = locale_count.values, ax = ax2)\n", "ax2.set_ylabel('count', fontsize = 15)\n", "ax2.set_xlabel('locale',fontsize = 15)"], "outputs": []}, {"metadata": {"collapsed": true}, "execution_count": null, "cell_type": "code", "source": ["most_freq_descr = holiday_events['description'].value_counts().head(12).sort_index()\n", "#print (most_freq_descr)\n", "transferred_count = holiday_events['transferred'].value_counts()"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["fig, (ax1, ax2) = plt.subplots(1,2,figsize = (18, 4))\n", "sns.barplot(y = most_freq_descr.index, x = most_freq_descr.values, ax = ax1)\n", "ax1.set_ylabel('Description-most frequent', fontsize = 15)\n", "ax1.set_xlabel('Frequency',fontsize = 15)\n", "sns.barplot(x = transferred_count.index, y = transferred_count.values, ax = ax2)\n", "ax2.set_ylabel('count', fontsize = 15)\n", "ax2.set_xlabel('transferred',fontsize = 15)"], "outputs": []}, {"metadata": {}, "execution_count": null, "cell_type": "code", "source": ["locale_name_count = holiday_events['locale_name'].value_counts().sort_index()\n", "locale_name_count.head()\n", "fig, ax = plt.subplots(figsize = (18, 3))\n", "g = sns.barplot(x = locale_name_count.index, y = locale_name_count.values, ax = ax)\n", "ax.set_ylabel('count', fontsize = 15)\n", "ax.set_xlabel('locale_name',fontsize = 15)\n", "ax.tick_params(labelsize = 12)\n", "g.set_xticklabels(locale_name_count.index, rotation = 45)"], "outputs": []}, {"metadata": {}, "cell_type": "markdown", "source": ["## Summary\n", "\n", "I have completed the basic visualisation of datasets in the project. The following will be the analysis and plots combining different datasets together. The advantage of `ggplot` library in `R` is very obvious. It's easy to add legend, superposition of different plot, get smoothed trend curve and make plots pretty. "]}, {"metadata": {"collapsed": true}, "execution_count": null, "cell_type": "code", "source": [], "outputs": []}]}