{"cells":[{"metadata":{"_uuid":"6eb129ae618ca11173fdbee6ee8a11c176901d2a"},"cell_type":"markdown","source":"Well we don't have time intervals in our training data set to compare. But **Demo Account** has some information.\nIf we compare in users report `2017-09-01 to 2017-10-15` (train) with `2017-12-01 to 2018-01-30` (test) the same intervals but last year. \n\nIt is just a simple report I've downloaded from Google Analytics. Test period contains only 3185 users with revenue, so it is a one file with 5000 records after after sorting by \nrevenue.\n\n**Update:**   fix error in metric, now real revenue is multiplied to million as in dataset. Added proportion of time periods in calculation.\n\n#### Sample data:\n\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\ndf = pd.read_csv('../input/gmsga2017/Analytics 3 Raw Data View User Explorer 20171201-20180131 20170501-20171015.csv',skiprows=5)\ndf.head(3).T","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b32e2cc4a4d873a8bde38304fb4a3b6036da60a8"},"cell_type":"code","source":"df['Revenue'] = pd.to_numeric(df.Revenue.str.replace(',','').str[3:])\ndf['Range'] = df['Date Range'].replace({'01-Dec-2017 - 31-Jan-2018':'Test', '01-May-2017 - 15-Oct-2017':'Train'})\ndf=df[['Sessions','Revenue','Range','Client ID']].set_index(['Range','Client ID']).unstack('Range')\ndf.columns = ['_'.join(t) for t in df.columns]\ndf.sample(3).T","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c76d2131b530538150642d38cd8f81cca56858a8"},"cell_type":"code","source":"train_user_revenue_on_test = df[(df.Revenue_Test>0)&(df.Sessions_Train>0)].Revenue_Test\ntrain_user_revenue_on_test.agg(['sum','count','mean','min','max'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a07a71933bc3fb7d23cafef8ee0974c6f8e62029"},"cell_type":"code","source":"total_abs_error = np.log1p(train_user_revenue_on_test*10**6).sum()\nprint ('total abs error',total_abs_error)\ntotal_square_error = (np.log1p(train_user_revenue_on_test*10**6)**2).sum()\nprint ('total square error',total_square_error)\ndates = pd.to_datetime([\n    '2 August 2017',#old test start\n    '30 April 2018',#old test end\n    '1 May 2018',#new test start\n    '15 October 2018']) #new test end\nprop = (dates[3] - dates[2])/(dates[1] - dates[0])\nprint('new test length as proportion of old one',prop) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d0cea64c659a435cd973930ab7f6ad122f8e7ec3"},"cell_type":"code","source":"df_sub = pd.read_csv('../input/ga-customer-revenue-prediction/sample_submission.csv')\nprint('total users', df_sub.shape[0], 'RMSE on all zero:',np.sqrt(total_square_error/(df_sub.shape[0]*prop)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0cd3d91385632ce31c71130ae775f58751a61265"},"cell_type":"markdown","source":"I'm not sure about how many users will be on new submissions, but you can see that only 0.58%  of users still active after 3 months.  "},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true,"_uuid":"aa33453bacfafb734dc117fc6e6623fe9039a28c"},"cell_type":"code","source":"pd.DataFrame([[4.85,1.60,0.58]], index = ['User Retention'], columns =['1 Month','2 Month','3 Month'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"186da5c0d9e2a4609b2612f7cac3240ab8b2f60a"},"cell_type":"markdown","source":"So all zero submissions looks like a strong baseline."},{"metadata":{"_uuid":"0d61f4bb82d94609bde668ceb98d65f96b08b06d"},"cell_type":"markdown","source":""}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}