{"cells":[{"metadata":{"_uuid":"143752ffd14edc16ecef0eb9f8b4907aaf7212d1"},"cell_type":"markdown","source":"# Introduction\n\n### Competition Objective\n\n*In this competition, you’re challenged to analyze a Google Merchandise Store (also known as GStore, where Google swag is sold) customer dataset to predict revenue per customer. Hopefully, the outcome will be more actionable operational changes and a better use of marketing budgets for those companies who choose to use data analysis on top of GA data.*\n\nThis is exploritory data analysis for the Google Analytics Customer Revenue Prediction Competion on Kaggle. The competition ended on November 30th 2018 and the results are evaluated on future data compiled between December 1st 2018 to January 31st 2019."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import warnings\nwarnings.simplefilter(action='ignore', category=FutureWarning)\n\nimport numpy as np\nimport pandas as pd\nfrom pandas.io.json import json_normalize\nfrom datetime import datetime\n\nimport json\nimport os\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nfrom plotly import tools\nfrom plotly.offline import init_notebook_mode, iplot, plot\nimport plotly.offline as py\nimport plotly.graph_objs as go","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"# About in the Data\n\n### Data Fields\n\n**fullVisitorId** - A unique identifier for each user of the Google Merchandise Store.\n\n**channelGrouping** - The channel via which the user came to the Store.\n\n**date** - The date on which the user visited the Store.\n\n**device** - The specifications for the device used to access the Store. (JSON format)\n\n**geoNetwork** - This section contains information about the geography of the user. (JSON format)\n\n**socialEngagementType** - Engagement type, either \"Socially Engaged\" or \"Not Socially Engaged\".\n\n**totals** - This section contains aggregate values across the session. (JSON format)\n\n**trafficSource** - This section contains information about the Traffic Source from which the session originated. (JSON format)\n\n**visitId** - An identifier for this session. This is part of the value usually stored as the _utmb cookie. This is only unique to the user. For a completely unique ID, you should use a combination of fullVisitorId and visitId.\n\n**visitNumber** - The session number for this user. If this is the first session, then this is set to 1.\n\n**visitStartTime** - The timestamp (expressed as POSIX time).\n\n**hits** - This row and nested fields are populated for any and all types of hits. Provides a record of all page visits.\n\n**customDimensions** - This section contains any user-level or session-level custom dimensions that are set for a session. This is a repeated field and has an entry for each dimension that is set.\n\n**totals** - This set of columns mostly includes high-level aggregate data.\n\n\n### Timeframe\n\nThe training data we are using is from August 2016 to May 2018.\n\n### Size and Format\n\nThe version 2 of the training data is fairly large at ~1.7 million rows. While in a csv file, many of the columns are in JSON format. \n\n# Read in the Data\n\nThe 1.7 million rows will need to be read in chunks of 100K to not overload memory. **customDimensions** and **hits** columns are extremely large and don't help much in an EDA so they are dropped to save space.  The JSON format columns need to also be processed into additional columns. This block takes around 8 minutes to complete."},{"metadata":{"trusted":true,"_uuid":"a64f07b884ddb8f7b395fac4c75bf764de83a528"},"cell_type":"code","source":"%%time\n\ndef load_df(csv_path='../input/train_v2.csv', nrows=None):\n    JSON_COLUMNS = ['device', 'geoNetwork', 'totals', 'trafficSource']\n    df = pd.DataFrame([])\n    for chunk in pd.read_csv(csv_path, \n                     converters={column: json.loads for column in JSON_COLUMNS}, \n                     dtype={'fullVisitorId': 'str'},\n                     chunksize=nrows):\n\n        chunk = chunk.drop(['customDimensions','hits'], axis=1)\n        df = df.append(chunk, ignore_index=True)\n        print(df.shape[0])\n    print(\"Dataframe assembled; Working on JSON columns...\")\n    for column in JSON_COLUMNS:\n        column_as_df = json_normalize(df[column])\n        column_as_df.columns = [f\"{column}.{subcolumn}\" for subcolumn in column_as_df.columns]\n        df = df.drop(column, axis=1).merge(column_as_df, right_index=True, left_index=True)\n    print(\"Done!\")\n    return df\n\ntrain = load_df('../input/train_v2.csv', 100000)\ntrain[\"totals.transactionRevenue\"] = train[\"totals.transactionRevenue\"].astype('float')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c0f08b3b747c4a6a8e5464a02d74d54c19f63f69"},"cell_type":"markdown","source":"# First Impressions"},{"metadata":{"trusted":true,"_uuid":"34f8ea1e7079acbc7357631bb87c7fe633f2b669"},"cell_type":"code","source":"print(\"Colunms in dataset: \" + str(train.shape[1]))\nprint(\"Rows in dataset: \" + str(train.shape[0]))\ntrain[:10]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"889f1d9e8accf880c05cfa2e1408fef53e50f4e3"},"cell_type":"markdown","source":"## First Impressions: What We Learned\n\n- Separating out the JSON columns balloons the columns up to 58. A lot of these will be useful, but many will not.\n\n- Many columns seem full of NaN values and many others have almost NaN equivalent strings like \"not available in demo dataset\" and \"(not set)\"\n"},{"metadata":{"_uuid":"93eb68f43c849a122695fb4952d289338b1e51b2"},"cell_type":"markdown","source":"# Looking at Na/Unique Values"},{"metadata":{"trusted":true,"_uuid":"f5a8293da6c19db86c9796821d1bb44f17cf84fd"},"cell_type":"code","source":"na_counts = pd.DataFrame(train.isnull().sum())\nna_counts.columns = ['na_counts']\nna_counts['%na'] = round(na_counts['na_counts'] / train.shape[0], 4) * 100\nuniq_counts = pd.DataFrame(train.nunique(dropna=False))\nuniq_counts.columns = ['uniq_counts']\nna_uniq_counts = na_counts.join(uniq_counts, how='outer').sort_values(by='na_counts', ascending=False)\n\nprint(\"There are \"+str(len(na_uniq_counts.loc[na_uniq_counts[\"na_counts\"] > 0]))+\" columns with na values\")\nconst_cols = [c for c in train.columns if train[c].nunique(dropna=False)==1 ]\nprint(\"There are \"+str(len(const_cols))+\" colunms with only one unique value\")\n\nna_uniq_counts\n# train = train.drop(const_cols, axis=1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"129a3a2f62a383ed8ce16f0272c431393065690d"},"cell_type":"markdown","source":"## Looking at Na/Unique Values: What We Learned\n\n- There are 19 columns with only one unique value. These are worthless for data prediction and can be dropped immediatly. Some of these are due to the demo version of google analytics so might have value with the full version.\n\n- 18 columns have na values. Of these 10 are more than 95% na values."},{"metadata":{"_uuid":"b1c583340996e2f8ae0b3054495264cb886a01dc"},"cell_type":"markdown","source":"# Transaction Revenue: Exploration"},{"metadata":{"trusted":true,"_uuid":"bb1f341da5b846125c250b49f1664c74c6507d86","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[40,20])\n\nfontdict = {'fontsize': 30, 'fontweight' : 'bold'}\n\nplt.subplot(231)\nplt.scatter(range(train.shape[0]), np.sort(train['totals.transactionRevenue'].values))\nplt.title(\"Revenue Range\", fontdict=fontdict)\nplt.tick_params(axis='both', which='major', labelsize=10)\nplt.xlabel('')\nplt.ylabel('')\n\nplt.subplot(232)\nsns.distplot( np.log(train[train['totals.transactionRevenue'] > 0][\"totals.transactionRevenue\"]), bins=60, kde=True)\nplt.title(\"Distribution of Revenue\", fontdict=fontdict)\nplt.tick_params(axis='both', which='major', labelsize=10)\nplt.xlabel('')\nplt.ylabel('')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"70e266f2ba5805ec1e5c19fe52e27e8d8abe1f3c"},"cell_type":"markdown","source":"## Transaction Revenue: What We Learned\n\n- The vast majority of visits had no revenue. In other words, no items were purchased during those visits.\n\n- The distribution of revenue looks normally distributed.\n"},{"metadata":{"_uuid":"d63e24491002b1494f98d786e1936d58c6962265"},"cell_type":"markdown","source":"# Visitors: Exploration"},{"metadata":{"trusted":true,"_uuid":"2b554b764cc3ba76f969f6254cbcd29c6c646238","_kg_hide-input":true},"cell_type":"code","source":"print(\"In the dataset of \"+str(train.shape[0])+\" visits there are...\")\nprint(\"Number of unique visitors: \"+str(train[\"fullVisitorId\"].nunique()))\nprint(\"Presentage of visits that bought something: %\"+str(train.loc[train[\"totals.transactionRevenue\"] > 0].shape[0]/train.shape[0]*100))\n\ntrain_first_visit = train.loc[train[\"visitNumber\"] == 1]\nprint(\"Percentage of visits that were for the first time: %\"+str(train_first_visit.shape[0]/train.shape[0]*100))\nprint(\"Presentage of first time visitors that bought something: %\"+str(train_first_visit.loc[train_first_visit[\"totals.transactionRevenue\"] > 0].shape[0]/train.shape[0]*100))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"65e956689f392191a153f0433cd9346bb71c1f3d"},"cell_type":"markdown","source":"## Visitors: What We Learned\n\n- Only ~1.1% of visits result in a sale.\n\n- Most of the visits to the site are from people visiting for the first time.\n\n- First time visitors are much less likely to purchase anything than returning visitors.\n"},{"metadata":{"_uuid":"d1d28e05062e7e40c6cc1fdea5b156ca1f6639a8"},"cell_type":"markdown","source":"#  Device Category: Exploration"},{"metadata":{"trusted":true,"_uuid":"532a4f4f5f0bdd0d24ab0b87c8659f5b22ab8aaa","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[20,16])\n\ndef create_countplot(column, title, plt_n, df=train, mean_ch = False):\n    labels = list(df[column].value_counts().index.unique())\n\n    fontdict = {'fontsize': 10, 'fontweight' : 'bold'}\n    plt.subplot(plt_n)\n    if mean_ch:\n        sns.barplot(x = column, y=\"totals.transactionRevenue\", data=df, order = labels[:10])\n    else:\n        sns.countplot(x = column, data=df, order = labels[:10])\n    plt.xticks(np.arange(0, len(labels[:10]), 1), labels, rotation=-80)\n    plt.title(title, fontdict=fontdict)\n    plt.tick_params(axis='both', which='major', labelsize=10)\n    plt.xlabel('')\n    plt.ylabel('')\n    \ncreate_countplot( 'device.deviceCategory', 'Device Used in Visits', 231)\ncreate_countplot( 'device.deviceCategory', 'Device Used in Visits with Purchase', 232, train.loc[train[\"totals.transactionRevenue\"] > 0])\ncreate_countplot( 'device.deviceCategory', 'Device Used in Visits with Purchase (Mean)', 233, train.loc[train[\"totals.transactionRevenue\"] > 0], True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2f90a3483ab04f1b77ee1383e2b02e8b791473f1"},"cell_type":"markdown","source":"## Device Category: What We Learned\n\n- Most visits are from a desktop computer, next mobile phones and very few from tablets.\n\n- Desktop visits are also more likely to make a purchase and spend more when they do. This intuitively makes sense, as most people use their phones on the go and are more likely to be casually browsing perhaps even saving purchases for when they are at a computer.\n\n- While mobile visitor are more common, mobile and tablet visitors seem to have similar behavior. \n"},{"metadata":{"_uuid":"6a0dd592c83ac932b16b46542066f917742a63a9"},"cell_type":"markdown","source":"# Device Browser: Exploration"},{"metadata":{"trusted":true,"_uuid":"9b34322982f23b16836483dee39ce8492c778dd1","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[20,16])\n\ncreate_countplot( 'device.browser', 'Browser Used in Visits', 231)\ncreate_countplot( 'device.browser', 'Browser Used in Visits with Purchase', 232, train.loc[train[\"totals.transactionRevenue\"] > 0])\ncreate_countplot( 'device.browser', 'Browser Used in Visits with Purchase (Mean)', 233, train.loc[train[\"totals.transactionRevenue\"] > 0], True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ba0d78d2c3a8719e12f42f1932d2775932992bb1"},"cell_type":"markdown","source":"## Device Browser: What We Learned\n\n- Most visits and visits with purchase are from Chrome browser. Makes sense as Chrome is a Google product and users are probably more likely to be Google fans.\n\n- Firefox seems to have larger purchases. However, the confidence is very low due to the small sample of Firefox purchases so this could very well be a fluke. Other than that, Chrome is confidently higher than the others besides Firefox."},{"metadata":{"_uuid":"b42c3618be2b261bded62e8e2580f507a3b1a9cc"},"cell_type":"markdown","source":"# Operating System: Exploration"},{"metadata":{"trusted":true,"_uuid":"3bcfb6b2f47c30ba4a72484867119cdb14780a08","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[20,16])\n\ncreate_countplot( 'device.operatingSystem', 'Operating System Used in Visits', 231)\ncreate_countplot( 'device.operatingSystem', 'Operating System Used in Visits with Purchase', 232, train.loc[train[\"totals.transactionRevenue\"] > 0])\ncreate_countplot( 'device.operatingSystem', 'Operating System Used in Visits with Purchase (Mean)', 233, train.loc[train[\"totals.transactionRevenue\"] > 0], True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"290656408c6c6ebdc9d85eccd19bf1ac65ef1fa1"},"cell_type":"markdown","source":"## Operating System: What We Learned\n\n- Windows is the most common operating system used to visit. In fact, the operating system used seems to just follow the most common models.\n\n- Despite this, Macintosh users were much much more likely than windows users in visits with purchase. Perhaps because Macintosh users tend to be considered more \"tech savvy\" and may have more interest in tech based swag.\n\n- Chrome OS, despite it rarity, is more likely to purchase something which makes sense since it is made by google.\n\n- Mobile operating systems tend to purchase less often and spend less when they do. This is consistent with the hypothesis that visitor tend to commit to purchase more at a desktop computer than on their phones.\n"},{"metadata":{"_uuid":"ab4ff407e7063d3249570cca163aceee1b895197"},"cell_type":"markdown","source":"# Network Domains: Exploration"},{"metadata":{"trusted":true,"_uuid":"83d74fdb58d7efe6bd01508d6a33de567acb3e3b","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[20,16])\n\ncreate_countplot( 'geoNetwork.networkDomain', 'Network Domain in Visits', 231)\ncreate_countplot( 'geoNetwork.networkDomain', 'Network Domain Used in Visits with Purchase', 232, train.loc[train[\"totals.transactionRevenue\"] > 0])\ncreate_countplot( 'geoNetwork.networkDomain', 'Network Domain in Visits with Purchase (Mean)', 233, train.loc[train[\"totals.transactionRevenue\"] > 0], True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"92d7067e02d74bdfa86bd846ccff8f405e2e619a"},"cell_type":"markdown","source":"## Network Domains: What We Learned\n\n- Most network domains are (not set) or unknown.unknown. Unfortunately, this means that we cannot get much useful information from this from a simple EDA. A careful machine learning model might still make use of this column.\n\n- The one small bit of information we can see is that comcastbusiness.net has much higher mean purchases than the other network domains. The subsample size is small but, even with the low confidence, the difference is still large enough to say that this is probably true.\n"},{"metadata":{"_uuid":"d41b539a4374d9fb792e3ddbfda4ae279e244e6a"},"cell_type":"markdown","source":"# Continent: Exploration"},{"metadata":{"trusted":true,"_uuid":"22ad9279a652c155487156a2c5d562e1b16c587f","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[20,16])\n\ncreate_countplot( 'geoNetwork.continent', 'Continent in Visits', 231)\ncreate_countplot( 'geoNetwork.continent', 'Continent in Visits with Purchase', 232, train.loc[train[\"totals.transactionRevenue\"] > 0])\ncreate_countplot( 'geoNetwork.continent', 'Continent in Visits with Purchase (Mean)', 233, train.loc[train[\"totals.transactionRevenue\"] > 0], True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"81988cb830af4fdb172645ec9900b5be6849ad2f"},"cell_type":"markdown","source":"## Continent: What We Learned\n\n- Americas are the most likely visitors and overwhelmingly the most likely to make a purchase. In fact, almost every purchase was made in the Americas.\n\n- Despite the abnormal spike in africa mean purchase, that comparison is unusable due to the small subsample sizes of non-Americas purchases.\n"},{"metadata":{"_uuid":"8fc838d1c6320d09da21a89fcad8d1d7b216241c"},"cell_type":"markdown","source":"# Sub Continent: Exploration"},{"metadata":{"trusted":true,"_uuid":"19a5306accc9b66edb456a2e2f7942eee02eebb1","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[20,16])\n\ncreate_countplot( 'geoNetwork.subContinent', 'Sub Continent in Visits', 231)\ncreate_countplot( 'geoNetwork.subContinent', 'Sub Continent in Visits with Purchase', 232, train.loc[train[\"totals.transactionRevenue\"] > 0])\ncreate_countplot( 'geoNetwork.subContinent', 'Sub Continent in Visits with Purchase (Mean)', 233, train.loc[train[\"totals.transactionRevenue\"] > 0], True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7d701eedcbaa7644effe78d097db31cc0a424a7b"},"cell_type":"markdown","source":"## Sub Continent: What We Learned\n\n- These charts parrots the results of the last ones, but narrows down the area to just Northern America. Unsurprisingly, purchases in South America are about as insignificant as all other non-North America sub continents."},{"metadata":{"_uuid":"1e412003aed8af292936614ab240757b94427dee"},"cell_type":"markdown","source":"# Traffic Source: Exploration"},{"metadata":{"trusted":true,"_uuid":"6e4249a0c935d6b902794ca1277a7a7d8bf7a774","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[20,16])\n\ncreate_countplot( 'trafficSource.source', 'Traffic Source in Visits', 231)\ncreate_countplot( 'trafficSource.source', 'Traffic Source in Visits with Purchase', 232, train.loc[train[\"totals.transactionRevenue\"] > 0])\ncreate_countplot( 'trafficSource.source', 'Traffic Source in Visits with Purchase (Mean)', 233, train.loc[train[\"totals.transactionRevenue\"] > 0], True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"125113013a958906b78bf3953787602e3a995bf6"},"cell_type":"markdown","source":"## Traffic Source: What We Learned\n\n- Most visitors come from google, youtube.com, or a direct link.\n\n- visitors that followed a direct link were way more likely to make a purchase. In contrast, those that came from google were likely just browsing.\n\n- Surprisingly, despite a large portion of overall visitors coming by way of youtube.com, almost none of them made a purchase.\n"},{"metadata":{"_uuid":"b3de8aaa0d27151d2b9b17a3febdbcb488f581d9"},"cell_type":"markdown","source":"# Traffic Medium: Exploration"},{"metadata":{"trusted":true,"_uuid":"09b44ec31514d4de37370acb8a5804878adb25f2","_kg_hide-input":true},"cell_type":"code","source":"plt.figure(figsize=[20,16])\n\ncreate_countplot( 'trafficSource.medium', 'Traffic Medium in Visits', 231)\ncreate_countplot( 'trafficSource.medium', 'Traffic Medium in Visits with Purchase', 232, train.loc[train[\"totals.transactionRevenue\"] > 0])\ncreate_countplot( 'trafficSource.medium', 'Traffic Medium in Visits with Purchase (Mean)', 233, train.loc[train[\"totals.transactionRevenue\"] > 0], True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"811d1925c0879afb7e7cfdb5a40655ab29b0f023"},"cell_type":"markdown","source":"## Traffic Medium: What We Learned\n\n- Unsurprisingly, organic traffic is very high. Since this traffic typically comes from search engines and Google owns the most popular one. \n\n- The high number of referral visits suggests social media success. Agian not that surpising considering Googles strong brand."},{"metadata":{"_uuid":"7fdb89567b8b08c4fc2e1684ea328aec3ad065b0"},"cell_type":"markdown","source":"# Date and Time: Explorations"},{"metadata":{"trusted":true,"_uuid":"ad479624dde8e1ca5eb3693a71e9a374052fe67b","_kg_hide-output":false,"_kg_hide-input":true},"cell_type":"code","source":"def time_stamp_features(df):\n    df[\"date\"] = pd.to_datetime(df[\"date\"], format=\"%Y%m%d\") # seting the column as pandas datetime\n    df[\"weekday\"] = df['date'].dt.weekday #extracting week day\n    df[\"day\"] = df['date'].dt.day # extracting day\n    df[\"month\"] = df['date'].dt.month # extracting day\n    df[\"year\"] = df['date'].dt.year # extracting day\n    df[\"hour\"] = (df['visitStartTime'].apply(lambda x: str(datetime.fromtimestamp(x).hour))).astype(int)\n    return df\ntime_train = time_stamp_features(train)\np_time_train = time_train.loc[time_train[\"totals.transactionRevenue\"] > 0]\n\ndef create_timeplot(var, title, xlabel, ylabel, labels = []):\n    plt.figure(figsize=[40,16])\n    \n    df_count = time_train[var].value_counts().to_frame().reset_index().sort_values('index') \n    df_sum = df_count[var].sum()\n\n    sns.lineplot(x = \"index\", y = var, data = df_count, estimator=lambda x: x / df_sum * 100, label=\"%Visits\")\n\n    df_count = p_time_train[var].value_counts().to_frame().reset_index().sort_values('index')\n    df_sum = df_count[var].sum()\n\n    sns.lineplot(x = \"index\", y = var, data = df_count, color=\"Orange\", estimator=lambda x: x / df_sum * 100, label=\"%Visits with Purchase\")\n\n    fontdict = {'fontsize': 40, 'fontweight' : 'bold'}\n\n    plt.title(title, fontdict=fontdict)\n    plt.tick_params(axis='both', which='major', labelsize=30)\n    if labels != []:\n        plt.xticks(np.arange(0, len(labels), 1), labels)\n    plt.xlabel(xlabel, fontdict=fontdict)\n    plt.ylabel(ylabel, fontdict=fontdict)\n    plt.legend(fontsize=30)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"19ffd76c5c77d3202c3dd38d8aa8f2338b1454aa"},"cell_type":"markdown","source":"# Hour of the Day: Exploration"},{"metadata":{"trusted":true,"_uuid":"f135027627c8f95ecdd251bab69cf9954e953a8b","_kg_hide-input":true},"cell_type":"code","source":"create_timeplot(\"hour\",\"Visits by Hour of the Day\",\"Hour of the Day\", \"% of Visits\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b87f35d3ded60eb11ea1c7e72d47cc3266dac4f5"},"cell_type":"markdown","source":"## Hour of the Day: What We Learned\n\n- Even though people visit the site from all over the world, there is a clear difference in visit at different times during the day. This is likely because the vast majority of visits and purchases are from america and the more active hours for the site are during the afternoon-night in america.\n\n- While both lines follow the same trend, the visits with purchase line is much more pronounced. Noting again that these lines follow american time zones, we can also infer that while people might still visit the site different hours of the day, few people seem less inclined to make purchases during late night/early morning hours and are more likely to actually pull out their credit cards and make a purchase in the evening.\n"},{"metadata":{"_uuid":"6bd2a6b1ddc0f962040777d992a32be27569fca6"},"cell_type":"markdown","source":"# Date: Exploration"},{"metadata":{"trusted":true,"_uuid":"6a7a01ee5f26b7dec93018f9df04be9205ef6b78","_kg_hide-input":true},"cell_type":"code","source":"create_timeplot(\"date\",\"Visits by Date\",\"Date\", \"% of Visits\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"33ba5c6ce79a4901785d7a4445b06de103a153c0"},"cell_type":"markdown","source":"# Date: What We Learned\n\n- The data has very clear regular waves for both Visits and Visits with Purchase. This is likely due to regular weekly tendencies in visitor behavior.\n\n- There are 2 anomalous spikes in the data: a blue one at the end of 2017 and an orange one around 2018-04. This is probably just noise since they are short blips unless there was some event, like a one day sale, that might have triggered them.\n"},{"metadata":{"_uuid":"55b6578bce2edc131e636bf421181e939021c5c8"},"cell_type":"markdown","source":"# Weekday: Exploration"},{"metadata":{"trusted":true,"_uuid":"395b63af5629afc050f295a8ae76c7f7b0bec66a","_kg_hide-input":true},"cell_type":"code","source":"create_timeplot(\"weekday\",\"Visits by Weekday\",\"Weekday\", \"% of Visits\", [\"Monday\",\"Tuesday\",\"Wendsday\", \"Thursday\", \"Friday\", \"Saturday\", \"Sunday\"])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"606af76efa86dfd3a8dc8c6ab775822e7f4473b2"},"cell_type":"markdown","source":"## Weekday: What We Learned\n\n- There are more visits and purchases on the site during weekdays and less on the weekends. This explains the wavey data in the visits by date chart."},{"metadata":{"_uuid":"589161cde54d947ceafb500ad15f42874d04b0ca"},"cell_type":"markdown","source":"# Page Views and Hits: Exploration"},{"metadata":{"trusted":true,"_uuid":"334e861873942320190a9ebdcb109eff2a81cf7d","_kg_hide-input":true},"cell_type":"code","source":"train['totals.pageviews'] = train['totals.pageviews'].astype(\"float\")\ntrain['totals.hits'] = train['totals.hits'].astype(\"float\")\n\nplt.figure(figsize=[20,16])\ndef create_viewplot(column, title, plt_n, df=train, mean_ch = False):\n    df = df.loc[df[column] <= 60].sort_values(column) \n    labels = np.arange(0, 60, step=5)\n    fontdict = {'fontsize': 10, 'fontweight' : 'bold'}\n    plt.subplot(plt_n)\n    if mean_ch:\n        sns.barplot(x = column, y=\"totals.transactionRevenue\", data=df)\n    else:\n        sns.countplot(x = column, data=df)\n    plt.xticks(np.arange(-1, 60, step=5),labels, rotation=-80)\n    plt.title(title, fontdict=fontdict)\n    plt.tick_params(axis='both', which='major', labelsize=10)\n    plt.xlabel('')\n    plt.ylabel('')\n\ncreate_viewplot( 'totals.pageviews', '# of Page Views in Visit', 231)\ncreate_viewplot( 'totals.pageviews', '# of Page Views in Visit with Purchase', 232, train.loc[train[\"totals.transactionRevenue\"] > 0])\ncreate_viewplot( 'totals.pageviews', '# of Page Views in Visit with Purchase (Mean)', 233, train.loc[train[\"totals.transactionRevenue\"] > 0], True)\n\ncreate_viewplot( 'totals.hits', '# of hits in Visit', 234)\ncreate_viewplot( 'totals.hits', '# of hits in Visit with Purchase', 235, train.loc[train[\"totals.transactionRevenue\"] > 0])\ncreate_viewplot( 'totals.hits', '# of hits in Visit with Purchase (Mean)', 236, train.loc[train[\"totals.transactionRevenue\"] > 0], True)\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a3e31802e2ba92582a1a9a0e4b1ea1565d526af4"},"cell_type":"markdown","source":"## Weekday: What We Learned\n\n- The Page Views and Hits charts are all very similar and there are no real notable differences. This is likely because they are so strongly correlated with each other and only have slightly different meanings.\n\n- Purchase visits max around 15-20 page view/hits and follow a right-skewed distribution.\n\n- While the means revenue graphs have some spikes throughout the charts, the low confidence rates and scattered occurrences suggest that this is likely just from small subsample sizes. \n\n- That being said, there might still be some evidence that single page/hit visitors make larger purchases, on the rare chance that they do, since both means graphs show rather large spikes at 1 visit hits. This *might* be true because they might have been following direct links to the site and already had their purchase in mind with no interest in browsing.\n"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}