{"cells": [{"cell_type": "markdown", "metadata": {}, "source": "# GStore Revenue \u2014 flattening the JSON columns + the 80/20 reality\n\nThe first hurdle in this competition is that four columns (`device`, `geoNetwork`, `totals`, `trafficSource`) are **JSON strings**. This notebook shows how to flatten them, then looks at the heavily zero-inflated revenue that defines the problem. Reads a sample so it runs fast."}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": "import glob, os, json\nimport numpy as np, pandas as pd\nimport matplotlib.pyplot as plt\n\nBASE = '/kaggle/input/ga-customer-revenue-prediction'\nif not os.path.exists(f'{BASE}/train.csv'):\n    BASE = os.path.dirname(glob.glob('/kaggle/input/**/train*.csv', recursive=True)[0])\n\nJSON_COLS = ['device', 'geoNetwork', 'totals', 'trafficSource']\n\ndef load_flat(path, nrows=200_000):\n    df = pd.read_csv(path, nrows=nrows, dtype={'fullVisitorId': str},\n                     converters={c: json.loads for c in JSON_COLS})\n    for c in JSON_COLS:\n        flat = pd.json_normalize(df[c]).add_prefix(f'{c}.')\n        df = df.drop(columns=[c]).join(flat)\n    return df\n\ntrain = load_flat(f'{BASE}/train.csv')\nprint('flattened shape:', train.shape)\nprint('columns:', list(train.columns)[:25], '...')\ntrain.head(3)", "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": "## The target: revenue is overwhelmingly zero\n\n`totals.transactionRevenue` is missing (= no purchase) for the vast majority of sessions. The competition predicts the **log of per-customer total revenue**, so the modelling challenge is really *which rare sessions convert, and how much*."}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": "rev = pd.to_numeric(train.get('totals.transactionRevenue'), errors='coerce').fillna(0)\nnonzero = (rev > 0).mean()\nprint(f'sessions with revenue > 0: {nonzero:.2%}')\nlogrev = np.log1p(rev[rev > 0])\nplt.figure(figsize=(9, 4))\nplt.hist(logrev, bins=50, color='seagreen')\nplt.xlabel('log1p(transactionRevenue) | revenue>0'); plt.ylabel('sessions')\nplt.title(f'Revenue is zero for {1-nonzero:.1%} of sessions; log-normal among buyers')\nplt.show()", "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": "## Where does revenue come from? (channel & device)"}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": "train['rev'] = rev\nfig, ax = plt.subplots(1, 2, figsize=(13, 4))\nby_ch = train.groupby('channelGrouping')['rev'].mean().sort_values()\nby_ch.plot.barh(ax=ax[0], color='teal'); ax[0].set_title('mean revenue by channelGrouping')\nif 'device.deviceCategory' in train.columns:\n    by_dev = train.groupby('device.deviceCategory')['rev'].mean().sort_values()\n    by_dev.plot.barh(ax=ax[1], color='slateblue'); ax[1].set_title('mean revenue by deviceCategory')\nplt.tight_layout(); plt.show()", "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": "## Takeaways\n\n- Flatten `device / geoNetwork / totals / trafficSource` with `json.loads` + `pd.json_normalize` before anything else.\n- Revenue is ~99% zero; predict log-revenue and expect most of the signal in *whether* a session/customer converts (channel, device, geography, visit behaviour).\n- Aggregate to the **fullVisitorId** level for the final target, and drop constant columns (several flattened fields have a single value).\n\nFork and build on it. \ud83d\uded2"}], "metadata": {"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}, "language_info": {"name": "python"}}, "nbformat": 4, "nbformat_minor": 5}