{"cells":[{"metadata":{"_uuid":"5878e851921e40394ea5875edec3b0602e893a7d"},"cell_type":"markdown","source":"This kernel is a beginners approach to exploring the data set, in order to prepare for the development of a predicting model. This is my first Kernel and competition, and would love to hear everyone's feedback! \n\nIn this kernel, I will be going through each feature of the data and discussing its relevance when trying to predict revenue per customer."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport json\nimport matplotlib.pyplot as plt\n\n\nfrom pandas.io.json import json_normalize\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5f2a7508113f8f49de2bfee43d225739a654e03c"},"cell_type":"markdown","source":"To flatten the JSON fields in the data set, I used Julián Peller's \"1 - Quick start: read csv and flatten json fields\", thanks for this!\n\n"},{"metadata":{"trusted":true,"_uuid":"ccabb06c250c73f32b33b8c1662e2a10da4ccc16"},"cell_type":"code","source":"def load_df(csv_path='../input/train.csv', nrows=None):\n    JSON_COLUMNS = ['device', 'geoNetwork', 'totals', 'trafficSource']\n    \n    df = pd.read_csv(csv_path, \n                     converters={column: json.loads for column in JSON_COLUMNS}, \n                     dtype={'fullVisitorId': 'str'}, # Important!!\n                     nrows=nrows)\n    \n    for column in JSON_COLUMNS:\n        column_as_df = json_normalize(df[column])\n        column_as_df.columns = [f\"{column}.{subcolumn}\" for subcolumn in column_as_df.columns]\n        df = df.drop(column, axis=1).merge(column_as_df, right_index=True, left_index=True)\n    print(f\"Loaded {os.path.basename(csv_path)}. Shape: {df.shape}\")\n    return df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"627f881c902ecc433e647a24db752abf1e24d198"},"cell_type":"markdown","source":"Import train and test data frames. Specify a number of rows for quicker execution. "},{"metadata":{"trusted":true,"_uuid":"e81e9f36799336b40976dab941c4f2b976680a69","scrolled":true},"cell_type":"code","source":"train_df = load_df()#nrows=20000)\ntest_df = load_df(\"../input/test.csv\")#,nrows=20000)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ae838b288e8b5fe680dd274c8f594be36c950712"},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b70f555e673998681659a777b54f3fb286053fc4"},"cell_type":"code","source":"features_not_available = []\nfor a,b in zip(train_df.loc[0], train_df.columns):\n    if a == 'not available in demo dataset':\n        features_not_available.append(b)\ntrain_df = train_df.drop(features_not_available, axis=1)\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ab4daa6921a45851f10d3d2f871b3b04eb8c9ff0"},"cell_type":"markdown","source":"# Data fields\n## 1. channelGrouping\n\nThe channel via which the user came into the store. Here we want to explore:\n- Channel density i.e. how many visits per channel\n- Channel revenue i.e. how much revenue each channel generates"},{"metadata":{"trusted":true,"_uuid":"09317c04bfb90ea8ad60ea8d4001b81c1200dacd"},"cell_type":"code","source":"channel_density = train_df.groupby(['channelGrouping']).channelGrouping.count().sort_values()\nplt.xticks(rotation=60)\nplt.xlabel('Channels')\nplt.ylabel('Visits')\nplt.title('Channel Density')\nx_den = channel_density.index\ny_den = channel_density\nfor a,b in zip(x_den, y_den):\n    plt.text(a, b+60, str(round(100*b/train_df.shape[0]))+'%')\nplt.bar(x_den, y_den)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0f6a5fd22b638a519c6a8082d5ac8d17e38b610d"},"cell_type":"code","source":"train_df[\"totals.transactionRevenue\"] = train_df[\"totals.transactionRevenue\"].astype('float')\nchannel_revenue = train_df.groupby('channelGrouping')['totals.transactionRevenue'].sum().sort_values().reset_index()\nplt.xticks(rotation=60)\nplt.xlabel('Channels')\nplt.ylabel('Revenue')\nplt.title('Revenue per channel')\nplt.bar(channel_revenue['channelGrouping'], channel_revenue['totals.transactionRevenue'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3e57db984b257224e1da18fb3da25b3378d2994b"},"cell_type":"markdown","source":"Channel breakdown:\n- Referral: Traffic that occurs when a user finds you through a site other than a major search engine\n- Social: Traffic from a social network, such as Facebook, LinkedIn, Twitter, or Instagram\n- Organic search: Traffic from search engine results that is earned, not paid\n- Paid search: Traffic from search engine results that is the result of paid advertising via Google AdWords or another paid search platform\n- Email: Traffic from email marketing that has been properly tagged with an email parameter\n- Other: If traffic does not fit into another source or has been tagged as “Other” via a URL parameter, it will be bucketed into “Other” traffic.\n- Direct: Any traffic where the referrer or source is unknown.\n\n## 2. Device\nThe specifications for the device used to access the Store.\n### 2.1 device.browser\nWhat internet browser was used to access the store."},{"metadata":{"trusted":true,"_uuid":"8285075d551b25e44202b95681e873fc00b74ca0"},"cell_type":"code","source":"device_browser = train_df.groupby('device.browser')['totals.transactionRevenue'].agg(['count', 'size', 'mean']).sort_values('count', ascending=False)[0:8]\ndevice_browser.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e4d1e6ec7d9c8c553e4ed57d8285f41759000c8a"},"cell_type":"code","source":"plt.xticks(rotation=60)\nplt.xlabel('Browser')\nplt.ylabel('Count')\nplt.title('Device browser count')\nplt.bar(device_browser.index, device_browser['count'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9642bec5464b856bc130b47ce3b4e4be69484ab8"},"cell_type":"code","source":"plt.xticks(rotation=60)\nplt.xlabel('Browser')\nplt.ylabel('Revenue')\nplt.title('Device browser total revenue')\nplt.bar(device_browser.index, device_browser['size'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d25cf32be65c6c29ff4fb114e659da22b8f0270e"},"cell_type":"code","source":"plt.xticks(rotation=60)\nplt.xlabel('Browser')\nplt.ylabel('Mean revenue')\nplt.title('Device browser mean revenue')\nplt.bar(device_browser.index, device_browser['mean'])","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}