{"cells":[{"metadata":{"_uuid":"061f7a55c69d38f4415cade1d3692df378d91b09"},"cell_type":"markdown","source":"## Summary of the Notebook\nI'm ***KH Huang***, an enthusiastic data scientist from *Taiwan*. \n\nHere, I try to **replicate SRK's baseline model**, which was originally writen in Python, **in R**, using mainly tidyverse and XGBoost.  \nMy objective is to make sure I understand every part of his code and improve my R skills.\n\nOnly preprocessing and model building are included.  \nFor the EDA part, please find **SRK's awesome post**:\n\n#### Reference: SRK's Baseline Model \nhttps://www.kaggle.com/sudalairajkumar/simple-exploration-baseline-ga-customer-revenue\n"},{"metadata":{"_uuid":"09f87f305bf7fcb3363d5da46d85f669fea51054"},"cell_type":"markdown","source":"## Import Packages"},{"metadata":{"_uuid":"6531e5fec8f7b67e1cdfd8871eca5bb03481e572","_execution_state":"idle","trusted":true},"cell_type":"code","source":"library(tidyverse)\nlibrary(jsonlite)\nlibrary(recipes)\nlibrary(lubridate)\nlibrary(xgboost)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"bc31c69b90de6d6bfe78f20c6656439134dbe39d"},"cell_type":"markdown","source":"## Load Data\nthe IDs might be all numbers, so we should convert them to characters manually"},{"metadata":{"trusted":true,"_uuid":"9f1b4a19038f3fab92a3b81c73f611f6d06ae8b2"},"cell_type":"code","source":"data_dir <- \"../input/\"\n\n# load data\ntrain_data <- read.csv(paste0(data_dir, \"train.csv\"), \n                           stringsAsFactors = FALSE, \n                           colClasses=c(\"fullVisitorId\" = \"character\",\n                                        \"sessionId\" = \"character\",\n                                        \"visitId\" = \"character\"))\ntest_data <- read.csv(paste0(data_dir, \"test.csv\"),\n                          stringsAsFactors = FALSE, \n                          colClasses=c(\"fullVisitorId\" = \"character\",\n                                       \"sessionId\" = \"character\",\n                                       \"visitId\" = \"character\"))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8e3578c2d06cfb1e1fbbb93a70eafabf4c9b3a00"},"cell_type":"markdown","source":"## Process the JSON Columns\nconvert json columns to tidy format\n#### Reference: ML's JSON Processing \nhttps://www.kaggle.com/mrlong/r-flatten-json-columns-to-make-single-data-frame"},{"metadata":{"trusted":true,"_uuid":"77be3c29ca8076db35e81c5eeb6b00ed77e0af69"},"cell_type":"code","source":"# wrap up the JSON processing\njsonClmn2df <- function(json_clmn_name, data) {\n    json_data <- paste(\"[\", paste(data[[json_clmn_name]], collapse = \",\"), \"]\") %>% fromJSON(flatten = T)\n    colnames(json_data) <- paste(json_clmn_name, colnames(json_data), sep = \"_\")\n    return(json_data)\n}","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0e96fc836cf4c9ffef7770123af3747c80088568"},"cell_type":"markdown","source":"JSON columns in the data include \"device\", \"geoNetwork\", \"totals\", \"trafficSource\""},{"metadata":{"trusted":true,"_uuid":"4ecd4f9635ad62b5c5aecb92d29042fc2f230f67"},"cell_type":"code","source":"train_data <- train_data %>% \n    bind_cols(\n        jsonClmn2df(\"device\", train_data),\n        jsonClmn2df(\"geoNetwork\", train_data),\n        jsonClmn2df(\"totals\", train_data),\n        jsonClmn2df(\"trafficSource\", train_data)\n    ) %>%\n    select(-c(\"device\", \"geoNetwork\", \"totals\", \"trafficSource\"))\ntest_data <- test_data %>% \n    bind_cols(\n        jsonClmn2df(\"device\", test_data),\n        jsonClmn2df(\"geoNetwork\", test_data),\n        jsonClmn2df(\"totals\", test_data),\n        jsonClmn2df(\"trafficSource\", test_data)\n    ) %>%\n    select(-c(\"device\", \"geoNetwork\", \"totals\", \"trafficSource\"))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"68be699a3de56afa9dd50c74ac1925feeec61e1f"},"cell_type":"markdown","source":"## Find Columns having only Constant Values\nThe constant-value columns in ***training set*** should be removed because they're not helpful for model training.  \nThe constant-value columns in ***testing set*** should be removed because they won't contribute to the prediction process."},{"metadata":{"trusted":true,"_uuid":"d10533a62d6b0c66be1a8b5bc1e0cb6abc4c4f02"},"cell_type":"code","source":"get_cnst_clmns <- function(df) {\n    cnst_clmns <- c()\n    for (clmn in colnames(df)) {\n        n_uniq <- df %>% select(clmn) %>% n_distinct()\n        if (n_uniq == 1) {\n            cnst_clmns <- c(cnst_clmns, clmn)\n        }\n    }\n    return(cnst_clmns)\n}\ncnst_clmns <- union(get_cnst_clmns(train_data), get_cnst_clmns(test_data))\nprint(\"Columns having only constant values: \")\nprint(cnst_clmns)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8378cee1a19d7ebf230e8795a1897822ed9f337b"},"cell_type":"markdown","source":"## Find Columns Existing only in Training Data\nThese columns (except for the target variable, of course) should be removed because they will not contribute to prediction  \n(these columns may be contribute to model training, though)"},{"metadata":{"trusted":true,"_uuid":"58a7d3b562211b54ade9aa42985096b1bbb10040"},"cell_type":"code","source":"print(\"Columns existing in training data but not in testing data: \")\nprint(setdiff(colnames(train_data), colnames(test_data)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0d596434ce4e7da08433ad35651b363748a82f5d"},"cell_type":"markdown","source":"## Drop unuseful columns\nDrop two kinds of unuseful columns:\n- Columns having only constant values\n- Columns existing in training data but not in testing data  \n\nBesides, our baseline model will not use the information of shopping order, so \"sessionId\" would also be removed for now."},{"metadata":{"trusted":true,"_uuid":"6d4a43d56eccd16e357ad69aacbb3caaeb9cbd57"},"cell_type":"code","source":"cols_to_drop <- c(cnst_clmns, 'sessionId')\ntrain_data <- train_data %>% \n    select(-cols_to_drop) %>% \n    select(-trafficSource_campaignCode)\ntest_data <- test_data %>% \n    select(-cols_to_drop)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c3a00f75a14587f10a989cf66d653569743b4674"},"cell_type":"markdown","source":"## Clean Up Data: Numerical\nParse the data type."},{"metadata":{"trusted":true,"_uuid":"4c4095da3ea5015067e798e9cb327730795c7dfd"},"cell_type":"code","source":"num_cols <- c(\"totals_hits\", \"totals_pageviews\", \"visitNumber\", \n              \"visitStartTime\", 'totals_bounces',  'totals_newVisits') \ntrain_data <- train_data %>%\n    mutate_at(num_cols, as.numeric)  %>% \n    mutate(date = ymd(date),\n           totals_transactionRevenue = as.numeric(totals_transactionRevenue))\ntest_data <- test_data %>%\n    mutate_at(num_cols, as.numeric)  %>% \n    mutate(date = ymd(date))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d6eab11394eb5348197f34802db1069ab1ad8998"},"cell_type":"markdown","source":"## Clean Up Data: Categorical\nParse the data type and impute NAs.  \nI replace NA with the character \"NA\" here.  \nThat is, we consider all the missing values as one \"all the others\" type."},{"metadata":{"trusted":true,"_uuid":"923a8442f614ac90bd35a8bc72bd40846b42c803"},"cell_type":"code","source":"cat_cols <- c(\"channelGrouping\", \"device_browser\", \n              \"device_deviceCategory\", \"device_operatingSystem\", \n              \"geoNetwork_city\", \"geoNetwork_continent\", \n              \"geoNetwork_country\", \"geoNetwork_metro\",\n              \"geoNetwork_networkDomain\", \"geoNetwork_region\", \n              \"geoNetwork_subContinent\", \"trafficSource_adContent\", \n              \"trafficSource_adwordsClickInfo.adNetworkType\", \n              \"trafficSource_adwordsClickInfo.gclId\", \n              \"trafficSource_adwordsClickInfo.page\", \n              \"trafficSource_adwordsClickInfo.slot\", \"trafficSource_campaign\",\n              \"trafficSource_keyword\", \"trafficSource_medium\", \n              \"trafficSource_referralPath\", \"trafficSource_source\",\n              'trafficSource_adwordsClickInfo.isVideoAd', 'trafficSource_isTrueDirect')\ntrain_data <- train_data %>%\n    mutate_if(is.logical, as.character) %>% \n    mutate_if(is.character, function(x) {replace_na(x, \"NA\")}) \ntest_data <- test_data %>% \n    mutate_if(is.logical, as.character) %>% \n    mutate_if(is.character, function(x) {replace_na(x, \"NA\")})  ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"32cb08edbb18c63014dd262c14ab79b2f71b6019"},"cell_type":"markdown","source":"## Factorize the Categorical Columns\nTransform the character columns to R factors (which are basically integers to the predictive models)   \nSince we use tree-based models (e.g., XGBoost),  \n*factorizing*, instead of *one-hot encoding*, is acceptable for now\n\n**recipes** package is very useful for these processing tasks  \nAs far as I know, this package provides APIs most comparable to the handy **sklearn.preprocessing.LabelEncoder**\n"},{"metadata":{"trusted":true,"_uuid":"1370df0180a7129123d9aa241ebb7d07f2df7782"},"cell_type":"code","source":"rec_obj <- recipe(\n    totals_transactionRevenue ~ .,\n    data = train_data\n) %>% \n    step_string2factor(cat_cols) %>%\n    prep(training = train_data,\n         stringsAsFactors = FALSE)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c9f56685935ed0d9876ce8832a05e8636cc1ee91"},"cell_type":"markdown","source":"## Keep the customer IDs\nwe would evaluate the performance by grouping the user IDs later"},{"metadata":{"trusted":true,"_uuid":"e3d89b91973f1eab860e2c43c9fef6a3173c03ac"},"cell_type":"code","source":"train_id <- train_data %>% pull(fullVisitorId)\ntest_id <- test_data %>% pull(fullVisitorId)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4a11c0bf55a07137d71e049fc2accce0a36d62cb"},"cell_type":"markdown","source":"## Wrap Up Data Preprocessing\n- Select the variables which would be provided as inputs for the predictive model\n- Replace all NAs in both numerical and categorical (factor) columns with 0\n- Impute 0 for missing target values\n- Transform the targets to log values"},{"metadata":{"trusted":true,"_uuid":"3569a8517bb54a6b72d3c610bdcdc57ee03d0c02"},"cell_type":"code","source":"transform_df2xy <- function(df, has_target = TRUE) {\n    x <- cbind(\n        df %>% \n            select(num_cols) %>% \n            replace(is.na(.), 0), \n        bake(rec_obj, \n             newdata = (df %>% select(cat_cols))\n        ) %>% \n            mutate_if(is.factor, as.numeric) %>%\n            replace(is.na(.), 0)\n    )\n    if (has_target) {\n        y <- df %>% \n            pull(totals_transactionRevenue) %>% \n            as.numeric() %>% \n            replace_na(0) %>%\n            log1p()\n        return(list(x = x, y = y))\n    } else {\n        return(list(x = x, y = NULL))\n    }\n}","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"393f8866f44dffae2187f54460b4b5f6253356c5"},"cell_type":"markdown","source":"## Split Up Training/Validation Set\nfollowing SRK's idea, we keep data after 2017-05-31 as validation set"},{"metadata":{"trusted":true,"_uuid":"ef5c874e4ad47b3ee836439278ebef0dac4c298e"},"cell_type":"code","source":"train_index <- which(train_data$date <= ymd(20170531))\nvalid_index <- which(train_data$date > ymd(20170531))\ntrain_set <- transform_df2xy(df = train_data[train_index, ], has_target = TRUE)\nvalid_set <- transform_df2xy(df = train_data[valid_index, ], has_target = TRUE)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a391e62a960ff97e044b2fb2bef768a2e501fdcf"},"cell_type":"markdown","source":"## Predictive Model: XGBoost\nMost of the XGBoost parameters are set as default  \nWe only tune the \"nrounds\" using validation set"},{"metadata":{"trusted":true,"_uuid":"c86d71124fcf14eccf0d00cd18ee129ae4b2d85c"},"cell_type":"code","source":"dtrain <- xgb.DMatrix(data = train_set$x %>% as.matrix(), \n                      label = train_set$y)\ndvalid <- xgb.DMatrix(data = valid_set$x %>% as.matrix(), \n                      label = valid_set$y)\n\nparams <- list(\n    objective = \"reg:linear\",\n    eval_metric = \"rmse\",\n    booster = \"gbtree\"\n)\n\nmy_xgb <- xgb.train(data = dtrain, \n                    params = params, \n                    nrounds = 50,\n                    watchlist = list(dvalid = dvalid),\n                    early_stopping_rounds = 5, \n                    print_every_n = 3)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a435601062df3d62306054c3b519ed91580e734a"},"cell_type":"markdown","source":"## Check the Model Performance on Validation Set\n- Find the right \"nrounds\" we need\n- Have a look at feature importances to some hints for further feature engineering\n\nNote that the RMSE indicated here is **not** exactly the same evaluation metric as the one this competition observes"},{"metadata":{"trusted":true,"_uuid":"2e2e3e2e8e5f96cc973dcadbf126baf303c52320"},"cell_type":"code","source":"print(my_xgb, verbose = TRUE)\nxgb.importance(train_set$x %>% colnames(), \n               model = my_xgb) %>%\n    xgb.plot.importance(top_n = 15)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"64f3e1b176ebb645d4aca3f2c1122457f4878e14"},"cell_type":"markdown","source":"## Evaluate according to the Competition's Evaluation Metric\nPlease find the [thread](https://www.kaggle.com/c/ga-customer-revenue-prediction/discussion/66737)  \nIn short, sum up the original revenue values for each user, instead of summing up the log-transformed values\n"},{"metadata":{"trusted":true,"_uuid":"22737e3bfcfa7e3098529a697a3258e2f9ba7f81"},"cell_type":"code","source":"eval_valid_rmse <- function(log_pred, valid_y, valid_id) {\n    tmp_df <- data.frame(fullVisitorId = valid_id, totals_transactionRevenue = valid_y) %>%\n        replace_na(list(totals_transactionRevenue = 0)) %>%\n        mutate(pred = expm1(log_pred)) %>%\n        group_by(fullVisitorId) %>%\n        summarize(user_revenue_sum_target = sum(totals_transactionRevenue),\n                  user_revenue_sum_pred = sum(pred))\n    return(\n        sqrt(\n            mean(\n                (log1p(tmp_df$user_revenue_sum_target) - log1p(tmp_df$user_revenue_sum_pred))^2\n            )\n        )\n    )\n}","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4313270e330731d194af0615734175c31a284ac7"},"cell_type":"markdown","source":"## Check the Model Performance on Validation Set: Official Evaluation Metric\nWhen evaluating, note that\n- the model outputs the **log-value** of revenue prediction  \n- the model would somehow extrapolate **negative revenue prediction**"},{"metadata":{"trusted":true,"_uuid":"be3afac798431ab47bf42bba31ffbd568cbd8e9e"},"cell_type":"code","source":"my_valid_log_pred <- predict(my_xgb, newdata = dvalid)\nmy_valid_log_pred[my_valid_log_pred < 0] <- 0\nprint(\n    eval_valid_rmse(log_pred = my_valid_log_pred, \n                    valid_y = valid_set$y, \n                    valid_id = train_data[valid_index, ] %>% pull(fullVisitorId))\n)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"040bad8f23550893d6fb79c2e9bc8aad81caf122"},"cell_type":"markdown","source":"## Prediction: Model Training\nmove toward the testing stage, using all training data (need not to keep validation data aside)  "},{"metadata":{"trusted":true,"_uuid":"d6ad72f43ad17ecccd9754de935aea26359c3376"},"cell_type":"code","source":"train_set <- transform_df2xy(df = train_data, has_target = TRUE)\ntest_set <- transform_df2xy(df = test_data, has_target = FALSE)\ndtrain <- xgb.DMatrix(data = train_set$x %>% as.matrix(), \n                      label = train_set$y)\ndtest <- xgb.DMatrix(data = test_set$x %>% as.matrix())\n\nmy_xgb <- xgb.train(data = dtrain, \n                    params = params, \n                    nrounds = 15,\n                    watchlist = list(dtrain = dtrain),\n                    print_every_n = 2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"92705283360cfff4baa7b041dfc083a6ce2217dd"},"cell_type":"markdown","source":"## Prediction: Deal With the Model Outputs\nAgain, sum up the original values of revenue, instead of summing up the log-transformed values"},{"metadata":{"trusted":true,"_uuid":"3139feba54f46e4951950ec335a64b133488298f"},"cell_type":"code","source":"my_test_log_pred <- predict(my_xgb, newdata = dtest)\nmy_test_log_pred[my_test_log_pred < 0] <- 0\n\npred_df <- data.frame(fullVisitorId = test_id) %>%\n    mutate(my_test_pred = expm1(my_test_log_pred)) %>%\n    group_by(fullVisitorId) %>%\n    summarize(PredictedLogRevenue = log1p(sum(my_test_pred)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7698b067e643e8692623c8491348705df48b869a"},"cell_type":"markdown","source":"## Prediction: Get the Results"},{"metadata":{"trusted":true,"_uuid":"aeea1689eedb7bd7fe6cb688338299d312bce3e0"},"cell_type":"code","source":"subm <- read.csv(paste0(data_dir, \"sample_submission.csv\"), \n                 colClasses=c(\"fullVisitorId\" = \"character\"))\nsubm <- subm %>% \n    select(-PredictedLogRevenue) %>%\n    left_join(., pred_df, by = \"fullVisitorId\")\nwrite.csv(subm, file = \"my_submission.csv\", row.names = FALSE)\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c5f894a25d360b91f60dc2dbcf2b3f1f020734e2"},"cell_type":"markdown","source":"## Conclusion\nUsing SRK's variable selection and R XGBoost default parameters with nrounds = 15,  \nthe public leaderboard score of this baseline model is **1.46**\n\nMy immediate reflection on this script is the naming convention  \nI should learn more about concise naming strategies\n\nPlease leave a comment if you find any way I could improve this notebook  \nThanks for your reading and enjoy the competition!"}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}