{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"@author: André Daniël VOLSCHENK\n\nKaggle project {Google Analytics Customer Revenue Prediction}\nkaggle.com/andredanielvolschenk\n\n# Preface\nThis is the final notebook in my series of notebooks on the Google Store (GStore) revenue prediction competition.  \nParts 1 to 3:  \nPart 1 : https://www.kaggle.com/andredanielvolschenk/gstore-part-1-data-cleansing  \nPart 2 : https://www.kaggle.com/andredanielvolschenk/gstore-part-2-visuals-eda-feature-engineering  \nPart 3 : https://www.kaggle.com/andredanielvolschenk/gstore-part-3-feature-reduction  \n\nThe aim of this notebook is as a follow up from Part 3. Here the intention is to generate a final estimator using `train_v2` and make predictions using `test_v2`.\n\n# Contents\n* [Setup](#Setup)\n* [Train LGBM for regression](#Train-LGBM-for-regression)\n\n# Setup\n## Import Libraries"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport statistics as ss\n\nimport json\nimport pandas.io.json as pdjson\nimport ast # Abstract Syntax Trees : The ast module helps Python applications to process trees of the Python abstract syntax grammar.\nimport datetime as dt\n\nimport gc   # Garbage Collector : gc exposes the underlying memory management mechanism of Python\ngc.enable()\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nimport lightgbm as lgb\nimport sklearn.metrics as sklm\n\nimport os\nprint(os.listdir(\"../input\"))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5dea01edd65bbc8d15562e78c32c450cd3f4d927"},"cell_type":"markdown","source":"## Select features to use\nPart 3 in this series provided us with a chart ranking feature importances as shown below:  \n\n![](https://i.imgur.com/GYiNb8g.png)  \n\nLets select the feature we wish to use in this notebook. We still have a large number of rows (train_v2.csv has 1'708'337 observations and test_v2.csv has 401'589 observations) so we do not want to store many column since we wish to use all our observations this time.  \n\nLets use `hits_page.pageTitle_hitsum` and all features ranked higher.  \n\nOur feature list is then (from least to most relevant :  \n\n'hits_page.pageTitle_hitssum' , 'hits_hitsNumber' , 'hits_referer_hitssum' , 'geoNetwork_country_hitssum',  'device_operatingSystem' , 'geoNetwork_networkDomain_hitssum' , 'hits_page.pagePathLevel1_hitssum' ,  'hits_page.pagePathLevel3_count' , 'hits_page.pageTitle_count' , 'hits_referer_count' , 'hits_hour' , 'geoNetwork_country_count' , 'week' , 'trafficSource_referralPath' , 'hits_appInfo.landingScreenName_hitssum' , 'hits_appInfo.exitScreenName_hitssum' , 'hits_page.pagePathLevel4_count' , 'hits_appInfo.landingScreenName_count' ,  'geoNetwork_city_hitssum' , 'hour' , 'hits_page.pagePathLevel1_count' , 'hits_appInfo.exitScreenName_count' , 'geoNetwork_metro_count' , 'day' , 'geoNetwork_networkDomain_hitssum' , 'totals_pageviews' , 'totals_hits' , 'geoNetwork_city_count' , 'count' , 'geoNetwork_networkDomain_count' , 'totals_timeOnSite' , 'visitNumber' , 'totals_transactionRevenue' , 'fullVisitorId'  \n\nThats 34 features.  \n\nThe original columns that we need to make these are:  \nhits , geoNetwork , device , date , trafficSource , visitStartTime , totals , visitNumber , fullVisitorId  \n\nThe JSON column in these are :  \nhits , geoNetwork, device , trafficSource, totals  \n\n## Load and prepare data as per Part 3\nHere we load and flatten `train_v2` and `test_v2`. Feature extraction is performed, cardinality is reduced, and only the relevant features are stored.  \nHigh Cardinality features are aggregated as per Part 3, relevent features are converted to numeric or categorical as per Part 3. Categorical features are represented as ordinals as in Part 3."},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"97cad8a57ffe9377d3038bfb6976fcd5b1f4dc3f"},"cell_type":"code","source":"\ndef extraction(df): # here we declare a function to do all feature extraction as per Part 2.\n    \n    df['date'] = df['date'].apply(lambda x: dt.date(int(str(x)[:4]), int(str(x)[4:6]), int(str(x)[6:])))\n    #% feature representation\n    df.date = pd.to_datetime(df.date, errors='coerce')\n    #% feature extraction - time and date features\n    # Get the month value from date\n    df['month'] = df['date'].dt.month\n    # Get the week value from date\n    df['week'] = df['date'].dt.week\n    # Get the weekday value from date\n    #df['weekday'] = df['date'].dt.weekday\n    # Get the year\n    df['year'] = df['date'].dt.year\n    # Get the day of the month\n    df['day'] = df['date'].dt.day\n    # period counter\n    df['count'] = ((df['year'] - 2016) * 24) + (df['month']-1)*2 + round( df['day']/30 ) - 13\n    # drop date\n    df = df.drop(labels=['date'], axis=1)\n    \n    df['visitStartTime'] = pd.to_datetime(df['visitStartTime'], unit='s')\n    df['hour'] = df['visitStartTime'].dt.hour\n    df = df.drop(labels=['visitStartTime'], axis=1)\n    \n    return df\n\ndef make_countsum(df, dfstr):\n    df[dfstr] = df[dfstr].astype('str')\n    \n    df['totals_hits']=df['totals_hits'].fillna(0).astype('int')\n    df['totals_pageviews']=df['totals_pageviews'].fillna(0).astype('int')\n    \n    df[str(dfstr+'_count')] = df[dfstr]\n    df[str(dfstr+'_count')]=df.groupby(dfstr).transform('count')\n    \n    df[str(dfstr+'_hitssum')] = df.groupby(dfstr)['totals_hits'].transform('sum')\n    df[str(dfstr+'_viewssum')] = df.groupby(dfstr)['totals_pageviews'].transform('sum')\n    del(df[dfstr])\n    return df\n\n\ndef cardinality_redux(df): # this function covnerts high cardinality categorical features to numeric aggregates\n    lst = ['geoNetwork_city', 'geoNetwork_metro', 'geoNetwork_region', 'geoNetwork_country', \n           'geoNetwork_networkDomain', 'hits_appInfo.exitScreenName', \n           'hits_appInfo.landingScreenName', 'hits_appInfo.screenName', #'hits_eventInfo.eventLabel', \n           'hits_page.pagePath', 'hits_page.pagePathLevel1', 'hits_page.pagePathLevel2', \n           'hits_page.pagePathLevel3', 'hits_page.pagePathLevel4', 'hits_page.pageTitle', \n           'hits_referer', 'trafficSource_adContent', 'trafficSource_adwordsClickInfo.gclId']\n    for dfstr in lst:\n        df = make_countsum(df, dfstr)\n    return df\n\n\ndef aggregate(df, col, leave): # fn to aggregate all categories in df[col] except for cols in leave\n    df[col] = df[col].astype('str')\n    include = df[col].unique()  # array of all unique categories\n    include = list(include)\n    include = set(include).difference(set(leave))  # set: take out 'leave' from include\n    include = list(include)\n    df.loc[df[col].isin(include), col] = \"grouped\"  # rename all cols in 'include' to 'grouped'\n    return df\n\ndef all_agg(data):\n    data = aggregate(data, 'device_operatingSystem', leave=['Windows', 'Macintosh', 'Android', 'iOS', 'Linux', 'Chrome OS'])\n    data = aggregate(data, 'trafficSource_referralPath', leave=['/'])\n    return data\n\ndef encode_num(data):\n    data['hits_hitNumber'] = pd.to_numeric(data['hits_hitNumber'], errors='coerce', downcast='unsigned')\n    data['hits_hour'] = pd.to_numeric(data['hits_hour'], errors='coerce', downcast='unsigned')\n    data['totals_timeOnSite'] = pd.to_numeric(data['totals_timeOnSite'], errors='coerce', downcast='unsigned')\n    return data\n\ndef clean(data):\n    # Convert to object:\n    data['hits_isEntrance'] = data['hits_isEntrance'].astype('str')\n    data['hits_isExit'] = data['hits_isExit'].astype('str')\n    data['hits_promotionActionInfo.promoIsView'] = data['hits_promotionActionInfo.promoIsView'].astype('str')\n    data['trafficSource_adwordsClickInfo.isVideoAd'] = data['trafficSource_adwordsClickInfo.isVideoAd'].astype('str')\n    data['trafficSource_isTrueDirect'] = data['trafficSource_isTrueDirect'].astype('str')\n    \n    # Encode nans as 0:\n    data['totals_sessionQualityDim'].fillna(0, inplace=True)\n    data['totals_timeOnSite'].fillna(0, inplace=True)\n    \n    # replace nans with mode:\n    data['hits_hitNumber'].fillna(ss.mode(data['hits_hitNumber']), inplace=True)\n    data['hits_hour'].fillna(ss.mode(data['hits_hour']), inplace=True)\n    return data\n\n\ndef onehot(data):                    # TODO this is not OHE yet\n    for col in data.columns:\n        x=data[col]\n        if (x.apply(np.isreal).all(axis=0)) & ((str(x.dtypes) != 'category')): # if numeric, but not category\n            #print(col, 'is numeric')\n            1+1\n        else:\n            data[col] = data[col].astype('category',copy=False)\n    return data\n\n\ndef load_df(csv_path):\n    \n    json_vars = ['device', 'geoNetwork', 'totals', 'trafficSource']  # 'hits' will be handled seperately\n    \n    # final_vars taken directly from the end of Part 3\n    final_vars = ['hits_page.pageTitle_hitssum' , 'hits_hitNumber' , 'hits_referer_hitssum' , \n              'geoNetwork_country_hitssum',  'device_operatingSystem' , \n              'geoNetwork_networkDomain_viewssum' , 'hits_page.pagePathLevel1_hitssum' ,  \n              'hits_page.pagePathLevel3_count' , 'hits_page.pageTitle_count' , 'hits_referer_count' ,\n              'hits_hour' , 'geoNetwork_country_count' , 'week' , 'trafficSource_referralPath' ,\n              'hits_appInfo.landingScreenName_hitssum' , 'hits_appInfo.exitScreenName_hitssum' ,\n              'hits_page.pagePathLevel4_count' , 'hits_appInfo.landingScreenName_count' , \n              'geoNetwork_city_hitssum' , 'hour' , 'hits_page.pagePathLevel1_count' ,\n              'hits_appInfo.exitScreenName_count' , 'geoNetwork_metro_count' , 'day' ,\n              'geoNetwork_networkDomain_hitssum' , 'totals_pageviews' , 'totals_hits' ,\n              'geoNetwork_city_count' , 'count' , 'geoNetwork_networkDomain_count' ,\n              'totals_timeOnSite' , 'visitNumber' , 'totals_transactionRevenue' , 'fullVisitorId' ]\n    \n    # the column from the original data we need to import to make the above\n    usecols = ['hits' , 'geoNetwork' , 'device' , 'date' , 'trafficSource' , 'visitStartTime' ,\n           'totals' , 'visitNumber' , 'fullVisitorId']\n    \n    print('created json_var, final_var, and usecols')\n    \n    # lets append json_vars with final_vars, because we still need to import the json vars before expanding them\n    all_vars  = json_vars + final_vars + usecols # the master list of columns to import\n    \n    \n    ans = pd.DataFrame()\n    \n    dfs = pd.read_csv(csv_path, sep=',',\n                      converters={column: json.loads for column in json_vars},\n                      dtype={'fullVisitorId': 'str'}, # Important!!\n                      usecols = usecols,   # import only the ones we really need\n                      chunksize = 50000, # 100 000\n                      nrows=1000000  # TODO: remove this !\n                     )\n                        # if memory runs out, try decrease chunksize\n    \n    for df in dfs:\n        df.reset_index(drop = True,inplace = True)\n        \n        device_list=df['device'].tolist()\n        #deleting unwanted columns before normalizing\n        for device in device_list:\n            del device['browserVersion'],device['browserSize'],device['flashVersion'],device['mobileInputSelector'],device['operatingSystemVersion'],device['screenResolution'],device['screenColors']\n        df['device']=pd.Series(device_list)\n        \n        geoNetwork_list=df['geoNetwork'].tolist()\n        for network in geoNetwork_list:\n            del network['latitude'],network['longitude'],network['networkLocation'],network['cityId']\n        df['geoNetwork']=pd.Series(geoNetwork_list)\n        \n        df['hits']=df['hits'].apply(ast.literal_eval)\n        df['hits']=df['hits'].str[0]\n        df['hits']=df['hits'].apply(lambda x: {'index':np.NaN,'value':np.NaN} if pd.isnull(x) else x)\n        \n        # not hits is normal JSON, so we can add it\n        json_vars = ['device', 'geoNetwork', 'totals', 'trafficSource', 'hits']  # 'hits' will be handled seperately\n    \n        \n        for column in json_vars:\n            column_as_df = pdjson.json_normalize(df[column])\n            column_as_df.columns = [f\"{column}_{subcolumn}\" for subcolumn in column_as_df.columns]\n            df = df.drop(column, axis=1).merge(column_as_df, right_index=True, left_index=True)\n        \n        print(f\"Loaded {os.path.basename(csv_path)}. Shape: {df.shape}\")\n        \n        \n        df = extraction(df) # FEATURE EXTRACTION\n        \n        df = cardinality_redux(df) # DEAL WITH HIGH CARDINALITY\n        \n        df = all_agg(df)\n        \n        df = encode_num(df)\n        \n        #df = clean(df)\n        \n        \n        # we wont see each and every column in each chunk that we load, so we need to find where our master list intersects with the actual data\n        final_vars = set(final_vars).intersection(df.columns)\n        final_vars = list(final_vars)\n        df = df[final_vars]\n        gc.collect()\n        \n        df = onehot(df)\n        \n        ans = pd.concat([ans, df], axis = 0).reset_index(drop = True)\n        del(df)\n        print('Stored shape:', ans.shape)\n        \n    return ans\n\nprint(\" The 'load_df' function has been created\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d998031689cc7fd56a1eab1dd052cd181e2544ed"},"cell_type":"markdown","source":"Now lets load in `train_v2` as `data1`."},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"5edaff2f912d1870e44870e628ec08fb01dfbc6a","scrolled":false},"cell_type":"code","source":"data1 = load_df('../input/train_v2.csv')\nprint('data1 shape: ', data1.shape)\nprint(\"data1 loaded\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4a881210878fc7def1fa93bc134c745185c38ab9"},"cell_type":"markdown","source":"Lets check to make sure we have all the column that we wanted:"},{"metadata":{"_kg_hide-input":true,"trusted":true,"scrolled":false,"_uuid":"432e533970b8b98f58af3b7b06e93d2a779d0344"},"cell_type":"code","source":"final_vars = ['hits_page.pageTitle_hitssum' , 'hits_hitNumber' , 'hits_referer_hitssum' , \n              'geoNetwork_country_hitssum',  'device_operatingSystem' , \n              'geoNetwork_networkDomain_viewssum' , 'hits_page.pagePathLevel1_hitssum' ,  \n              'hits_page.pagePathLevel3_count' , 'hits_page.pageTitle_count' , 'hits_referer_count' ,\n              'hits_hour' , 'geoNetwork_country_count' , 'week' , 'trafficSource_referralPath' ,\n              'hits_appInfo.landingScreenName_hitssum' , 'hits_appInfo.exitScreenName_hitssum' ,\n              'hits_page.pagePathLevel4_count' , 'hits_appInfo.landingScreenName_count' , \n              'geoNetwork_city_hitssum' , 'hour' , 'hits_page.pagePathLevel1_count' ,\n              'hits_appInfo.exitScreenName_count' , 'geoNetwork_metro_count' , 'day' ,\n              'geoNetwork_networkDomain_hitssum' , 'totals_pageviews' , 'totals_hits' ,\n              'geoNetwork_city_count' , 'count' , 'geoNetwork_networkDomain_count' ,\n              'totals_timeOnSite' , 'visitNumber' , 'totals_transactionRevenue' , 'fullVisitorId' ]\n\nprint('List of Columns not successully loaded: ',list( set(final_vars) - set(data1.columns) ) )\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b37eba786831cc5c17a65cc59a90b3cc863b393e"},"cell_type":"markdown","source":"Great! All the columns we wanted are loaded!  \n\n# Train LGBM for regression\nIn Part 3 we used the LGBM because of its speed. We needed this due to the large number of column in our data. We have now reduced the number of columns significantly, however we now use all our observations too, so we will make use of the LGBM again to handle this. Fortunately, Gradient Boosting methods are insensitive to feature distributions, so no further transforms are neccessary!  \n\nLets first declare the `makeSet` function as we did in Part 3:"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"7949495eac9899b1d49ecc790c842731f8233adf"},"cell_type":"code","source":"def makeSet(count, data, verbose=1):\n    \n    # PART 1 ----- Get targets --------\n    \n    targets = data[ (data['count']>=count+14) & (data['count']<=count+17)][['fullVisitorId', 'totals_transactionRevenue']]\n    targets['revenue'] = targets['totals_transactionRevenue']\n    targets = targets.drop(labels=['totals_transactionRevenue'], axis=1)\n    \n    targets['fullVisitorId'] = targets.fullVisitorId.astype('str')\n    targets = targets.groupby('fullVisitorId').sum()\n    \n    targets['fullVisitorId'] = targets.index\n    targets.reset_index(drop=True, inplace=True)\n    \n    # PART 2 ----- Fill in train set --------\n    \n    train = data[ (data['count']>=count) & (data['count']<=count+10)]\n    train=train.copy()\n    train['revenue'] = 0 # set all to 0 for now\n    \n    loyals = targets[targets.revenue>0]['fullVisitorId']\n    \n    if verbose:\n        print(loyals.shape[0], 'buyers in blue-box')\n        print( len( list(set(train.fullVisitorId.unique()) & set(targets.fullVisitorId.unique())) ) ,'customers in BOTH green and blue boxes')\n        print( len( set(train.fullVisitorId.unique()) & set( loyals ) ) ,'customers return to make a purchase')\n    \n    loyal_purchaser = set(train.fullVisitorId.unique()) & set( loyals )\n    for loyal in loyal_purchaser:\n        train.loc[train.fullVisitorId==loyal,'revenue'] = targets[targets.fullVisitorId==loyal]['revenue'].values[0]\n    \n    revenue=train['revenue']\n    train.drop(labels=['revenue'], axis=1, inplace=True)\n    return train, revenue\n\nprint(\"The 'makeSet' function has been created\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d8ac6271af28f6e17ae6e7e252f9e2fc8fcc55e0"},"cell_type":"markdown","source":"Next we import `test_v2` as `data2`.  \nRecall that `data2` will have periods 43 through 53 (11 periods). \nIf we append `data1` with `data2`, we can do the following:  \n\ntrain with sets 1 through 36   (uses periods 1 through 53  ... i.e. all the periods in `train_v2` plus `test_v2`.  \nuse the model to predict periods 57 to 60 (1 dec 2018 till 31 jan 2019) from set 43.  \n\nIn summary... train with sets 1 through 36 ... and then predict set 43.  \n\nFirst and foremost we create `data2` and append it to `data1` to form `data`, remembering to log-transform `totals_transactionRevenue`:"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"c54fbbe4d7691c78efc125a396ec7fb1ecd6e37b"},"cell_type":"code","source":"data2 = load_df('../input/test_v2.csv')\nprint('data2 shape: ', data2.shape)\nprint(\"data2 loaded\")\n\ndata=data1\ndel(data1)\ndata = data.append(data2)\ndel(data2)\n\ndata['totals_transactionRevenue'].fillna(0, inplace=True)\ndata['totals_transactionRevenue'] = np.log1p(data['totals_transactionRevenue'].astype(float))\n\ndata['fullVisitorId'] = data['fullVisitorId'].astype('category',copy=False)\n\nprint('data shape: ', data.shape)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e7801659cbcd6fa0af28d8c820ccd4e28b1d649a"},"cell_type":"markdown","source":"Now lets train our model with all the train sets 1 through 36 and predict set 43."},{"metadata":{"_kg_hide-input":true,"trusted":true,"scrolled":false,"_uuid":"add1d43a40a68537f4c1c2fcbb480dd2a11d2f13"},"cell_type":"code","source":"def rmse(y_true, y_pred):\n    return np.sqrt(sklm.mean_squared_error(y_true, y_pred))\n\n\ndef main():\n    importances = pd.DataFrame()\n    feature_name = data.columns\n    params = { 'metric': 'rmse' }\n    est_lgbm = lgb.LGBMRegressor(boosting_type='gbdt', num_leaves=32, max_depth=5,\n                                  learning_rate=0.01, n_estimators=10000, subsample=0.8, \n                                  subsample_freq=1, colsample_bytree=0.8,\n                                  reg_alpha=0.05, reg_lambda=0.05, random_state=1, \n                                  n_jobs = -1, **params)\n    \n    \n    trn_x, trn_y = makeSet(1, data, 0)\n    for count in range(2,19):          # (2,37) will range from 2 to 36\n        x, y = makeSet(count, data, 0)\n        trn_x=trn_x.append(x)\n        trn_y=trn_y.append(y)\n        del(x,y)\n    #\n    \n    val_x, val_y = makeSet(43, data, 0)  # 43\n    # val_y will be all zeros\n    \n    # Train estimator.\n    est_lgbm.fit(trn_x, trn_y,\n                 eval_set=[(val_x, val_y)],\n                 early_stopping_rounds=50, \n                 verbose=False)\n    # Prediction and evaluation on validation data set.\n    val_pred = est_lgbm.predict(val_x)\n    rmse_valid = rmse(val_y, np.maximum(0, val_pred))\n    \n    gc.collect()\n    \n    fullVisitorId = val_x['fullVisitorId']\n    \n    return fullVisitorId, val_pred, rmse_valid\n    \nif __name__ == '__main__':\n    fullVisitorId, val_pred, rmse_valid = main()\n#\nprint('done')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"932a7ccfe364178acfed4d387620986ace56009e"},"cell_type":"markdown","source":"Lets prepare the submission dataframe `submission`:"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"852283dc80d01a98c591e356cbe5bf04a2733163"},"cell_type":"code","source":"fullVisitorId = pd.DataFrame(fullVisitorId)\nfullVisitorId = fullVisitorId.reset_index(drop=True)\nval_pred = pd.DataFrame(val_pred)\n\nsubmission = fullVisitorId.copy()\nsubmission['PredictedLogRevenue'] = val_pred\n\ndel(fullVisitorId, val_pred)\n\nprint('Will there be any buyers in Dec 2018 or Jan 2019?', (submission.PredictedLogRevenue>0).any())\n\n#print('done')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7a1f07496e1dab22625d51cd840f4d77f063b5c9"},"cell_type":"markdown","source":"We should have exaclty 296530 rows, just like `sample_submission_v2`, because that is how many unique `fullVisitorId`s there are in the test set."},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"e0df5891d7f8b8c7fde24cf68849a6c518ecc4e7"},"cell_type":"code","source":"print( submission.shape)\nsubmission.drop_duplicates(subset='fullVisitorId', inplace=True)\nprint( submission.shape)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a5e60ab0e87e9798f89c181807b6b203ca23499e"},"cell_type":"markdown","source":"Looks correct now\n\nFinally we can output the results. The results will be saved under Output\nScroll to the top of this notebook to see the headings: Notebook, Code, Data, Output, Comments, Log, Versions, Fork"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"dd185881713f94b83898aa81d51ae53a5697eb86"},"cell_type":"code","source":"#submit file\nsubmission.to_csv(\"../working/submission.csv\", index=False)\n\nprint(\"Submitted to 'Output'\")","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}