{"cells":[{"metadata":{},"cell_type":"markdown","source":"**Google Analytics Customer Revenue Prediction\n**"},{"metadata":{},"cell_type":"markdown","source":"**Problem Statement**\n\nThe 80/20 rule has proven true for many businesses–only a small percentage of customers produce most of the revenue. As such, marketing teams are challenged to make appropriate investments in promotional strategies.\n\n**GStore**\n\nRStudio, the developer of free and open tools for R and enterprise-ready products for teams to scale and share work, has partnered with Google Cloud and Kaggle to demonstrate the business impact that thorough data analysis can have.\n\nIn this kaggle competition,we are challenged to analyze a Google Merchandise Store (also known as GStore, where Google swag is sold) customer dataset to predict revenue per customer.\n\n**File Descriptions**\n\ntrain.csv - the training set - contains the same data as the BigQuery rstudio_train_set.\ntest.csv - the test set - contains the same data as the BigQuery rstudio_test_set.\nsampleSubmission.csv - a sample submission file in the correct format. Contains all fullVisitorIds in test.csv.\n\n**Data Fields**\n\nfullVisitorId- A unique identifier for each user of the Google Merchandise Store.\n\nchannelGrouping - The channel via which the user came to the Store.\n\ndate - The date on which the user visited the Store.\n\ndevice - The specifications for the device used to access the Store.\n\ngeoNetwork - This section contains information about the geography of the user.\n\nsessionId - A unique identifier for this visit to the store.\n\nsocialEngagementType - Engagement type, either \"Socially Engaged\" or \"Not Socially Engaged\".\n\ntotals - This section contains aggregate values across the session.\n\ntrafficSource - This section contains information about the Traffic Source from which the session originated.\n\nvisitId - An identifier for this session. This is part of the value usually stored as the _utmb cookie. \nThis is only unique to the user. For a completely unique ID, we should use a combination of fullVisitorId and visitId.\n\nvisitNumber - The session number for this user. If this is the first session, then this is set to 1.\n\nvisitStartTime - The timestamp (expressed as POSIX time)."},{"metadata":{},"cell_type":"markdown","source":"**Objectives:**\n1- I will explore if we have some difference between the browser and if browser is significant to predict sells. \n\n2- Which countrys and continents have more acesses and sales ? How it's distributed?!\n\n3- Which type of device are most normal in our dataset?\n\n4- What's the mobile % of accesses? \n\n5- Which is the most frequent Operational System? \n\n6- What's the most frequent channelGrouping ?\n\n7- Whats the most frequent Weekdays, months, days, year with highest accesses and revenue?\n\nAnd another bunch of ideas that I will have when start exploring."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport json\nimport gc\nimport sys\nimport math\n\nfrom pandas.io.json import json_normalize\nfrom datetime import datetime\n\nfrom sklearn.metrics import mean_squared_error\nimport time\nfrom pandas.core.common import SettingWithCopyWarning\nimport warnings\nimport lightgbm as lgb\nfrom sklearn.model_selection import GroupKFold\n\n# I don't like SettingWithCopyWarnings ...\nwarnings.simplefilter('error', SettingWithCopyWarning)\ngc.enable()\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gc.enable()\n\nfeatures = ['channelGrouping', 'date', 'fullVisitorId', 'visitId',\\\n       'visitNumber', 'visitStartTime', 'device.browser',\\\n       'device.deviceCategory', 'device.isMobile', 'device.operatingSystem',\\\n       'geoNetwork.city', 'geoNetwork.continent', 'geoNetwork.country',\\\n       'geoNetwork.metro', 'geoNetwork.networkDomain', 'geoNetwork.region',\\\n       'geoNetwork.subContinent', 'totals.bounces', 'totals.hits',\\\n       'totals.newVisits', 'totals.pageviews', 'totals.transactionRevenue',\\\n       'trafficSource.adContent', 'trafficSource.campaign',\\\n       'trafficSource.isTrueDirect', 'trafficSource.keyword',\\\n       'trafficSource.medium', 'trafficSource.referralPath',\\\n       'trafficSource.source', 'customDimensions']\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_df(csv_path):\n    JSON_COLUMNS = ['device', 'geoNetwork', 'totals', 'trafficSource']\n    ans = pd.DataFrame()\n    dfs = pd.read_csv(csv_path, sep=',',\n            converters={column: json.loads for column in JSON_COLUMNS}, \n            dtype={'fullVisitorId': 'str'}, # Important!!\n            chunksize=100000)\n    for df in dfs:\n        df.reset_index(drop=True, inplace=True)\n        for column in JSON_COLUMNS:\n            column_as_df = json_normalize(df[column])\n            column_as_df.columns = [f\"{column}.{subcolumn}\" for subcolumn in column_as_df.columns]\n            df = df.drop(column, axis=1).merge(column_as_df, right_index=True, left_index=True)\n\n        #print(f\"Loaded {os.path.basename(csv_path)}. Shape: {df.shape}\")\n        use_df = df[features]\n        del df\n        gc.collect()\n        ans = pd.concat([ans, use_df], axis=0).reset_index(drop=True)\n        #print(ans.shape)\n    return ans","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ntrain = load_df('/kaggle/input/ga-customer-revenue-prediction/train_v2.csv')\ntest = load_df('/kaggle/input/ga-customer-revenue-prediction/test_v2.csv')\n\nprint('train date:', min(train['date']), 'to', max(train['date']))\nprint('test date:', min(test['date']), 'to', max(test['date']))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"list(train.columns.values)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.shape\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"list(test.columns.values)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Knowing the missing values¶**"},{"metadata":{"trusted":true},"cell_type":"code","source":"# code chunk that I saw in Gabriel Preda kernel\ndef missing_values(data):\n    total = data.isnull().sum().sort_values(ascending = False) # getting the sum of null values and ordering\n    percent = (data.isnull().sum() / data.isnull().count() * 100 ).sort_values(ascending = False) #getting the percent and order of null\n    df = pd.concat([total, percent], axis=1, keys=['Total', 'Percent']) # Concatenating the total and percent\n    print(\"Total columns at least one Values: \")\n    print (df[~(df['Total'] == 0)]) # Returning values of nulls different of 0\n    \n    print(\"\\n Total of Sales % of Total: \", round((train[train['totals.transactionRevenue'] != np.nan]['totals.transactionRevenue'].count() / len(train['totals.transactionRevenue']) * 100),4))\n    \n    return ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# calling the missing values function\nmissing_values(train) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# only train feature\nfor c in train.columns.values:\n    if c not in test.columns.values: print(c)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train['totals.transactionRevenue'].fillna(0, inplace=True)\ntrain['totals.transactionRevenue'] = np.log1p(train['totals.transactionRevenue'].astype(float))\nprint(train['totals.transactionRevenue'].describe())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test['totals.transactionRevenue'] = np.nan","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"all_data = train.append(test, sort=False).reset_index(drop=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(all_data.info())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"null_cnt = train.isnull().sum().sort_values()\nprint(null_cnt[null_cnt > 0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for col in ['trafficSource.keyword',\n            'trafficSource.referralPath',\n            'trafficSource.adContent']:\n    all_data[col].fillna('unknown', inplace=True)\n\n# fillna numeric feature\nall_data['totals.pageviews'].fillna(1, inplace=True)\nall_data['totals.newVisits'].fillna(0, inplace=True)\nall_data['totals.bounces'].fillna(0, inplace=True)\nall_data['totals.pageviews'] = all_data['totals.pageviews'].astype(int)\nall_data['totals.newVisits'] = all_data['totals.newVisits'].astype(int)\nall_data['totals.bounces'] = all_data['totals.bounces'].astype(int)\n\n# fillna boolean feature\nall_data['trafficSource.isTrueDirect'].fillna(False, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# drop constant column\nconstant_column = [col for col in all_data.columns if all_data[col].nunique() == 1]\n#for c in constant_column:\n#    print(c + ':', train[c].unique())\n\nprint('drop columns:', constant_column)\nall_data.drop(constant_column, axis=1, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# pickup any visitor\nall_data[all_data['fullVisitorId'] == '7813149961404844386'].sort_values(by='visitNumber')[\n    ['date','visitId','visitNumber','totals.hits','totals.pageviews']].head(20)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_rev = train[train['totals.transactionRevenue'] > 0].copy()\nprint(len(train_rev))\ntrain_rev.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def plotCategoryRateBar(a, b, colName, topN=np.nan):\n    if topN == topN: # isNotNan\n        vals = b[colName].value_counts()[:topN]\n        subA = a.loc[a[colName].isin(vals.index.values), colName]\n        df = pd.DataFrame({'All':subA.value_counts() / len(a), 'Revenue':vals / len(b)})\n    else:\n        df = pd.DataFrame({'All':a[colName].value_counts() / len(a), 'Revenue':b[colName].value_counts() / len(b)})\n    df.sort_values('Revenue').plot.barh(colormap='jet')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Custom Dimensions**"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique customDimensions count:', train['customDimensions'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'customDimensions')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Date**"},{"metadata":{"trusted":true},"cell_type":"code","source":"format_str = '%Y%m%d'\nall_data['formated_date'] = all_data['date'].apply(lambda x: datetime.strptime(str(x), format_str))\nall_data['_year'] = all_data['formated_date'].apply(lambda x:x.year)\nall_data['_month'] = all_data['formated_date'].apply(lambda x:x.month)\nall_data['_quarterMonth'] = all_data['formated_date'].apply(lambda x:x.day//8)\nall_data['_day'] = all_data['formated_date'].apply(lambda x:x.day)\nall_data['_weekday'] = all_data['formated_date'].apply(lambda x:x.weekday())\n\nall_data.drop(['date','formated_date'], axis=1, inplace=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**ChannelGrouping\n\nThe channel via which the user came to the Store.**"},{"metadata":{"trusted":true},"cell_type":"code","source":"plotCategoryRateBar(all_data, train_rev, 'channelGrouping')\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**fullVisitorId**\n\nA unique identifier for each user of the Google Merchandise Store.\n\n**visitId**\n\nAn identifier for this session. This is part of the value usually stored as the _utmb cookie. This is only unique to the user.\n\nFor a completely unique ID, you should use a combination of fullVisitorId and visitId.\n\n\n**newVisits**"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('train all:', len(train))\nprint('train unique fullVisitorId:', train['fullVisitorId'].nunique())\nprint('train unique visitId:', train['visitId'].nunique())\nprint('-' * 30)\nprint('test all:', len(test))\nprint('test unique fullVisitorId:', test['fullVisitorId'].nunique())\nprint('test unique visitId:', test['visitId'].nunique())\n\n#print('common fullVisitorId:', len(pd.merge(train, test, how='inner', on='fullVisitorId'))) # 183434","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(all_data['visitNumber'].value_counts()[:5])\nprint('-' * 30)\nprint(all_data['totals.newVisits'].value_counts())\nprint('-' * 30)\nprint(all_data['totals.bounces'].value_counts())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"all_data['_visitStartHour'] = all_data['visitStartTime'].apply(\n    lambda x: str(datetime.fromtimestamp(x).hour))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"***Device***"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique browser count:', train['device.browser'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'device.browser', 10)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pd.crosstab(all_data['device.deviceCategory'], all_data['device.isMobile'], margins=False)\n\nall_data['isMobile'] = True\nall_data.loc[all_data['device.deviceCategory'] == 'desktop', 'isMobile'] = False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique operatingSystem count:', train['device.operatingSystem'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'device.operatingSystem', 10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**geoNetwork**"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique geoNetwork.city count:', train['geoNetwork.city'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'geoNetwork.city', 10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique geoNetwork.region count:', train['geoNetwork.region'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'geoNetwork.region', 10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique geoNetwork.subContinent count:', train['geoNetwork.subContinent'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'geoNetwork.subContinent', 10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique geoNetwork.continent count:', train['geoNetwork.continent'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'geoNetwork.continent')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique geoNetwork.metro count:', train['geoNetwork.metro'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'geoNetwork.metro', 10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique geoNetwork.networkDomain count:', train['geoNetwork.networkDomain'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'geoNetwork.networkDomain', 10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**totals**"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(all_data['totals.hits'].value_counts()[:10])\n\nall_data['totals.hits'] = all_data['totals.hits'].astype(int)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(all_data['totals.pageviews'].value_counts()[:10])\n\nall_data['totals.pageviews'] = all_data['totals.pageviews'].astype(int)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**trafficSource**"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique trafficSource.adContent count:', train['trafficSource.adContent'].nunique())\n\nplotCategoryRateBar(all_data, train_rev, 'trafficSource.adContent', 10)\n\nall_data['_adContentGMC'] = (all_data['trafficSource.adContent'] == 'Google Merchandise Collection').astype(np.uint8)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique trafficSource.campaign count:', train['trafficSource.campaign'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'trafficSource.campaign', 10)\n\nall_data['_withCampaign'] = (all_data['trafficSource.campaign'] != '(not set)').astype(np.uint8)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(all_data['trafficSource.isTrueDirect'].value_counts())\nplotCategoryRateBar(all_data, train_rev, 'trafficSource.isTrueDirect')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique trafficSource.keyword count:', train['trafficSource.keyword'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'trafficSource.keyword', 10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique trafficSource.medium count:', train['trafficSource.medium'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'trafficSource.medium')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique trafficSource.referralPath count:', train['trafficSource.referralPath'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'trafficSource.referralPath', 10)\n\nall_data['_referralRoot'] = (all_data['trafficSource.referralPath'] == '/').astype(np.uint8)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('unique trafficSource.source count:', train['trafficSource.source'].nunique())\nplotCategoryRateBar(all_data, train_rev, 'trafficSource.source', 10)\n\nall_data['_sourceGpmall'] = (all_data['trafficSource.source'] == 'mall.googleplex.com').astype(np.uint8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Aggregate**"},{"metadata":{"trusted":true},"cell_type":"code","source":"_='''\n'''\nall_data['_meanHitsPerDay'] = all_data.groupby(['_day'])['totals.hits'].transform('mean')\nall_data['_meanHitsPerWeekday'] = all_data.groupby(['_weekday'])['totals.hits'].transform('mean')\nall_data['_meanHitsPerMonth'] = all_data.groupby(['_month'])['totals.hits'].transform('mean')\nall_data['_sumHitsPerDay'] = all_data.groupby(['_day'])['totals.hits'].transform('sum')\nall_data['_sumHitsPerWeekday'] = all_data.groupby(['_weekday'])['totals.hits'].transform('sum')\nall_data['_sumHitsPerMonth'] = all_data.groupby(['_month'])['totals.hits'].transform('sum')\n\nfor feature in ['totals.hits', 'totals.pageviews']:\n    info = all_data.groupby('fullVisitorId')[feature].mean()\n    all_data['_usermean_' + feature] = all_data.fullVisitorId.map(info)\n    \nfor feature in ['visitNumber']:\n    info = all_data.groupby('fullVisitorId')[feature].max()\n    all_data['_usermax_' + feature] = all_data.fullVisitorId.map(info)\n\ndel info","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"all_data['_source.country'] = all_data['trafficSource.source'] + '_' + all_data['geoNetwork.country']\nall_data['_campaign.medium'] = all_data['trafficSource.campaign'] + '_' + all_data['trafficSource.medium']\nall_data['_browser.category'] = all_data['device.browser'] + '_' + all_data['device.deviceCategory']\nall_data['_browser.os'] = all_data['device.browser'] + '_' + all_data['device.operatingSystem']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Select feature **"},{"metadata":{"trusted":true},"cell_type":"code","source":"null_cnt = all_data.isnull().sum().sort_values()\nprint(null_cnt[null_cnt > 0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"all_data.drop(['visitId','visitStartTime'],axis=1,inplace=True)\n\nfor i, t in all_data.loc[:, all_data.columns != 'fullVisitorId'].dtypes.iteritems():\n    if t == object:\n        all_data[i].fillna('unknown', inplace=True)\n        all_data[i] = pd.factorize(all_data[i])[0]\n        #all_data[i] = all_data[i].astype('category')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Prediction**"},{"metadata":{"trusted":true},"cell_type":"code","source":"all_data.info()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = all_data[all_data['totals.transactionRevenue'].notnull()]\ntest = all_data[all_data['totals.transactionRevenue'].isnull()].drop(['totals.transactionRevenue'], axis=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_id = train['fullVisitorId']\ntest_id = test['fullVisitorId']\n\nY_train_reg = train.pop('totals.transactionRevenue')\n#Y_train_cls = (Y_train_reg.fillna(0) > 0).astype(np.uint8)\nX_train = train.drop(['fullVisitorId'], axis=1)\nX_test  = test.drop(['fullVisitorId'], axis=1)\n\nprint(X_train.shape, X_test.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del all_data, train, test, train_rev\ngc.collect()\n\nprint(pd.DataFrame([[val for val in dir()], [sys.getsizeof(eval(val)) for val in dir()]],\n                   index=['name','size']).T.sort_values('size', ascending=False).reset_index(drop=True)[:10])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import StratifiedKFold, GroupKFold\nfrom sklearn.metrics import roc_auc_score\nimport lightgbm as lgb\nparams={'learning_rate': 0.01,\n        'objective':'regression',\n        'metric':'rmse',\n        'num_leaves': 31,\n        'verbose': 1,\n        'random_state':42,\n        'bagging_fraction': 0.6,\n        'feature_fraction': 0.6\n       }\n\nfolds = GroupKFold(n_splits=5)\n\noof_preds = np.zeros(X_train.shape[0])\nsub_preds = np.zeros(X_test.shape[0])\nfor fold_, (trn_, val_) in enumerate(folds.split(X_train, Y_train_reg, groups=train_id)):\n    trn_x, trn_y = X_train.iloc[trn_], Y_train_reg.iloc[trn_]\n    val_x, val_y = X_train.iloc[val_], Y_train_reg.iloc[val_]\n    \n    reg = lgb.LGBMRegressor(**params, n_estimators=3000)\n    reg.fit(trn_x, trn_y, eval_set=[(val_x, val_y)], early_stopping_rounds=50, verbose=500)\n    \n    oof_preds[val_] = reg.predict(val_x, num_iteration=reg.best_iteration_)\n    sub_preds += reg.predict(X_test, num_iteration=reg.best_iteration_) / folds.n_splits\n\npred = sub_preds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Plot feature importance\nfeature_importance = reg.feature_importances_\nfeature_importance = 100.0 * (feature_importance / feature_importance.max())\nsorted_idx = np.argsort(feature_importance)\nsorted_idx = sorted_idx[len(feature_importance) - 30:]\npos = np.arange(sorted_idx.shape[0]) + .5\n\nplt.figure(figsize=(12,8))\nplt.barh(pos, feature_importance[sorted_idx], align='center')\nplt.yticks(pos, X_train.columns[sorted_idx])\nplt.xlabel('Relative Importance')\nplt.title('Variable Importance')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission = pd.DataFrame({'fullVisitorId':test_id, 'PredictedLogRevenue':pred})\n\nsubmission[\"PredictedLogRevenue\"] = np.expm1(submission[\"PredictedLogRevenue\"])\nsubmission[\"PredictedLogRevenue\"] = submission[\"PredictedLogRevenue\"].apply(lambda x : 0.0 if x < 0 else x)\nsubmission[\"PredictedLogRevenue\"] = submission[\"PredictedLogRevenue\"].fillna(0.0)\n\nsubmission_sum = submission[['fullVisitorId', 'PredictedLogRevenue']].groupby('fullVisitorId').sum().reset_index()\nsubmission_sum[\"PredictedLogRevenue\"] = np.log1p(submission_sum[\"PredictedLogRevenue\"])\nsubmission_sum.to_csv(\"submission.csv\", index=False)\nsubmission_sum.head(20)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission_sum['PredictedLogRevenue'].describe()\n","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}