{"cells":[{"metadata":{"trusted":true,"_uuid":"a4226c785e956286af0dc76605cac028989deb7d"},"cell_type":"markdown","source":"The challenge with the hits column -beyond the logistics of reading it into R- is that the usual methods for converting it into an R friendly format wont work, this is because standard json uses double quotes. \n\n-Initial step is to convert the single quotes to double. \n\n-there are some general cases that need to be addressed  e.g. when the value of a key is a logical, or when apostrophes are present within quoted strings\n\nThe code below deals with those, BUT there might be a few more  inconsistencies related to quotation use in the original data. This means that the regex below might need to be adapted slightly in order to work for all rows in both train and test sets.\n\n"},{"metadata":{"_uuid":"8e3912bd6fe4bf8572952f0ef9175bebfa0d1fde","_execution_state":"idle","trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"library(jsonlite)\nlibrary(tidyverse)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a00b2f6b93218cfd137f8c16804f11c684d5605d","_kg_hide-input":false,"_kg_hide-output":true},"cell_type":"code","source":"train_sample  <- read_csv(\"../input/train_v2.csv\", n_max=1000)  ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"85d34c49b34b3bd219fb18b53da0ea3db6e806e1"},"cell_type":"code","source":"fix_json_hits <- function(jsonstring) {\n  jsonstring %>%\n    str_replace_all(c(\n      '\"' = \"'\", # there are few double quotes in the original, converted to single as a first step, for uniformity\n      \"'(?![a-z])|(?<=\\\\{|\\\\s)'\" = '\"', # converts single quotes to double, except apostrophes\n      \"True\" = '\"True\"', # add double quotes to logical values\n      '\" : \"' = '\":\"', # helper\n      '(?<!: )\"(?= |é)' = \"\" # remove last remaining double quotes\n    ))\n} \n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7bc06727ab77c5ae23d6046fa66f3ebabb5bfcdb"},"cell_type":"markdown","source":"\n  "},{"metadata":{"trusted":true,"_uuid":"1bbf77a5735d0303eb3d22b3e420d36101cf5912"},"cell_type":"code","source":"# fixes the inconsistencies in the hits column and converts it to an R nested list\n\nhit_type_df  <- train_sample %>%\n  select(fullVisitorId, visitStartTime, hits) %>%\n  rename(csv_hits = hits) %>%\n  mutate(fixed_json_hits = fix_json_hits(csv_hits)) %>%\n  mutate(r_object_hits = map(fixed_json_hits, fromJSON)) \n   ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1d64423bddee537b498849c038e1255c35050ded"},"cell_type":"code","source":"# first captures the page element and as a second step its subelements \n\npagePath_df <- hit_type_df %>%\n  mutate(page_hits = map(r_object_hits, \"page\")) %>%\n  mutate(pages_pagePaths = map(page_hits, \"pagePath\"))\n\n# likewise for the eCom actions\n\naction_type_df <- hit_type_df %>%\n  mutate(ecom_action=map(r_object_hits, \"eCommerceAction\")) %>% \n  mutate(action_type=map(ecom_action, \"action_type\"))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"790dbd5cf5acf98657be9474f44b215c56f207fa"},"cell_type":"code","source":"pagePath_df  %>% \n  select(fullVisitorId, visitStartTime, pages_pagePaths) %>%\n  sample_n(20)  \n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bc9c4dcd6b60cbd5f75c657c4402a2b0626f6eb7"},"cell_type":"code","source":"action_type_df  %>% \n  select(fullVisitorId, visitStartTime, ecom_action) %>% \n  sample_n(20) \n  ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b82e8ad632cd6fa73d692b5f9791fb6916c45f59"},"cell_type":"markdown","source":"The idea is to decide first which hit types or subtypes are relevant e.g. pagepath, product price,action types etc  and then capture those ones in a systematic way.\n\nIn any case, while there is a wealth of information within the hit data that worth exploring, when it comes to user engagement with respect to purchase intent, a good proxy for this is the session Quality score  - provided by Google ML, very likely based on this type of hit level data\n\n*Analytics calculates the Session Quality dimension and the Average Session Quality metric to estimate a user’s proximity to conversion. User engagement is evaluated for each session, and the resulting proximity to conversion is expressed as a score of 1-100 for each session during the date range, with 1 being the farthest from and 100 being the closest to a transaction. A value of 0 indicates that the metric is not calculated for the selected time range.*\nhttps://support.google.com/analytics/answer/7303153?hl=en"}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}