{"cells":[{"metadata":{"_uuid":"91fffe23bc2f8cc70baeab6df2c4854a2e84ee35"},"cell_type":"markdown","source":"** >>> \"They'll be back...or may be not...\" **"},{"metadata":{"_uuid":"cd908ac92c31dc6775a7c4f311071c3c2cdb4a1b"},"cell_type":"markdown","source":"Here we will have some talk about customers/buyers returning to purchase from some history period"},{"metadata":{"trusted":true,"_uuid":"e7b08a43be6e429529d4b56b36ac97d44e359cb7","_kg_hide-output":true,"_kg_hide-input":false},"cell_type":"code","source":"library(tidyverse) # metapackage with lots of helpful functions\nlibrary(data.table)\nlibrary(stringi)\nlibrary(caret)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"27d9d14a2741cbe57e55688d0c26583a482b8372"},"cell_type":"markdown","source":"Load the data from prepared file. \nHere we:\n* Use the test & train sets in one bulk\n* Use only columns:  date,target,fullVisitorId\n* Column target was divided by 1e6 to be more close to real world"},{"metadata":{"trusted":true,"_uuid":"a17f3b49559b5ed6dd71509fb78ca9b0d0dd1697","scrolled":true},"cell_type":"code","source":"dat <- readRDS(\"../input/tot-short/tot_short.rds\")\ncat( \">>> class of the total dataset: \", class(dat) , \"\\n\") \ncat( \">>> total dataset dimensions: \", dim(dat) , \"\\n\")\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d42f7abd3b1c158935c682a747a570c298a102ea"},"cell_type":"code","source":" #-- select only datas before 2018-10-01. Oct.2018 has only 15 days, so let's skip it...\n  z <- dat[ date < '2018-10-01', .(date, m = data.table::month(date), year = data.table::year(date), fullVisitorId, target)]\n\n  #-- create factor column from year+month\n  z$myear <- as.factor( z$year*100 + z$m )\n  \n  #-- We need the BUYERS only (with at least one purchase). And do aggregation the sum of purchases by visitorId and date  \n  z <- z[target > 0, .(fullVisitorId, myear, target)][ order(myear) ]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"eae9972ca2aca69cdfabb3e721eed338c71622d4"},"cell_type":"markdown","source":"Let's have a look at the trend for total number of unique buyers per month"},{"metadata":{"trusted":true,"_uuid":"b667263a6c3de4c0e52afbc61623bebc486874c6"},"cell_type":"code","source":"stats_m <- z[, .( value = 1 ), by = .(fullVisitorId, myear) ]\nstats_m <- stats_m[, .(value = .N, month = myear), by = .(myear) ][ order(myear) ]\n\nt( stats_m[, -1 ] )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"4c0094475847c73b01430f2599bf7e6e81c5fb55"},"cell_type":"code","source":" \n  ggplot( data = stats_m, aes( x = as.Date( paste0( stats_m$month, '01'), '%Y%m%d' ) , y = value, label = round(value,1) ) ) + \n    geom_line(color = \"gray\") +\n    labs(x = NULL, y = \"Number of unique buyers\", title = \"Total number of the unique buyers per month\" ) + \n    geom_smooth( formula = y ~ x, method = 'lm', se = F ) +\n    scale_x_date(date_labels = \"%b %Y\", date_breaks = \"1 month\") +\n    scale_y_continuous( labels = function(x ){ return(paste( x, \"\")) } ) +\n    theme( axis.text.x = element_text(angle = 90, size = 8, vjust = 0.5) ) + \n    geom_label( nudge_x = 0,  nudge_y = 0.25  )  ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"08bfdd5b00d0f20b2a8e9e9df82a58db8c353f77"},"cell_type":"markdown","source":"Oops, the buyers number is trended to be negative over time. It is not very good (("},{"metadata":{"trusted":true,"_uuid":"6be576bf6b329abe5a7b19d16de46d542a9ed676","scrolled":true},"cell_type":"code","source":" \n  #-- continue to form the datas...   \n\n  z <- z[, .( .N, tsum = sum( target ) ), by = .(fullVisitorId, myear ) ]\n  \n  #-- Expand horisontally the data.frame with date columns\n  z <- cbind( z, class2ind(  z$myear, drop2nd = T) )\n  \n  #-- Collapse rows with aggregation by date columns for every Visitor. \n  z <- z[, lapply(.SD, sum), by = .(fullVisitorId), .SDcols = names(z)[ -(1:4) ]]\n  \n  #-- Add the summarize column (?)\n  z <- cbind( z, rowSums(z[, -c(1 ), with = F]) )\n  \n \n  head(z)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f68e573f8ee03d5b057723c70e6b4156187678d5"},"cell_type":"markdown","source":"Let's have a look at the visitors, having the purchases in the reporting month **AND** having at least one purchase within historical period.\nFirstly, define the function *calc_historical_period*"},{"metadata":{"trusted":true,"_uuid":"528177cece6c2759fd3a891543e974c719a3e41b","_kg_hide-input":true},"cell_type":"code","source":"##-- define the function\ncalc_historical_period <- function(z, skip_periods = 1, history_periods = 6 ) {\n    \n    ret  <- data.table();\n\n    #--  from (current_month - skip_periods) to (current_month - history_periods)\n\n    #print( colnames(z) )\n\n    #-- get dates in descending order\n    calc_dates <- colnames(z)[ 27:(2 + history_periods) ]\n\n    #print( calc_dates )\n    \n    #-- For every month as current\n    for (p in calc_dates ) {\n      \n      #-- save only those, who made the purchase in the current month\n      z1 <- z[  which( z[[ p ]] > 0 ) ]\n\n      #\n      #-- now we select only those who made purchases within \n      #   the previous 6 months \n\n      #-- get the previous periods as column names\n      prev_periods  <- colnames(z)[ \n          (which(colnames(z) == p) - skip_periods ):(which(colnames(z) == p) - history_periods )  \n      ]\n      \n      z2 <- z[, c('fullVisitorId', prev_periods), with = F ]\n      \n      #-- calculate the number of transactions for buyers in current month and \n      # filter only positive ones\n      z2$transQty <- rowSums( z2[, -1] )\n      z2 <- z2[ transQty > 0 & fullVisitorId %in% unique( z1$fullVisitorId ) ]\n      \n      #-- calculate the filnal ratio.  \n      ratio <-  nrow(z2) * 100 / nrow(z1) \n      \n      ret <- rbind( ret, data.frame( value = round(ratio, 3), month = p ) ) \n    }\n    \n    ret\n}","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8236871d51b5a95c25dfce7b3232112fd9501b55"},"cell_type":"markdown","source":"*Now we have a look at the percentage of buyers returns to purchases. Historical period: 6 months*"},{"metadata":{"trusted":true,"_uuid":"dd780155ae77bb8eab5ca2d66aa384be54808cb7"},"cell_type":"code","source":"ret_past_6 <- calc_historical_period( z, 1, 6 )\n\nt(ret_past_6)\n\n##\n##-- and plot it\n##\nggplot( data = ret_past_6, aes( x = as.Date( paste0( ret_past_6$month, '01'), '%Y%m%d' ) , y = value, label = round(value,1) ) ) + \n      geom_line( color = \"gray\") +\n      geom_point(color = \"red\") +\n      labs(x = NULL, y = \"Return percentage\", title = 'Return percentage of the buyers from past period ( 6 months back )' ) + \n      geom_smooth( formula = y ~ x, method = 'lm', se = F ) +\n      scale_x_date(date_labels = \"%b %Y\", date_breaks = \"1 month\") +\n      scale_y_continuous( labels = function(x ){ return(paste( x, \"%\")) } ) +\n      theme( axis.text.x = element_text(angle = 90, size = 8, vjust = 0.5) ) + \n      geom_label( nudge_x = 0,  nudge_y = 0.25  )","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d6442049e7febcb95ced110f7506ac91b5a2a802","_execution_state":"idle","trusted":false},"cell_type":"markdown","source":"*And Historical period: 3 months*"},{"metadata":{"trusted":true,"_uuid":"03a034ff6a648e2fc292f7ffca166949281483f4"},"cell_type":"code","source":"ret_past_3 <- calc_historical_period(z, 1, 3 )\n\nt(ret_past_3)\n\nggplot( data = ret_past_3, aes( x = as.Date( paste0( ret_past_3$month, '01'), '%Y%m%d' ) , y = value, label = round(value,1) ) ) + \n      geom_line(color=\"gray\") +\n      geom_point(color = \"red\") +\n      labs(x = NULL, y = 'Return percentage', title = 'Return percentage of the buyers from past period ( 3 months back )' ) + \n      geom_smooth( formula = y ~ x, method = 'lm', se = F ) +\n      scale_x_date(date_labels = \"%b %Y\", date_breaks = \"1 month\") +\n      scale_y_continuous( labels = function(x ){ return(paste( x, \"%\")) } ) +\n      theme( axis.text.x = element_text(angle = 90, size = 8, vjust = 0.5) ) + \n      geom_label( nudge_x = 0,  nudge_y = 0.25  )\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2859d8502e78d37b1a9fa23fa023e734bb43e562"},"cell_type":"markdown","source":"*We can see the significant downward trend in return buyers from histrical periods. It is also not good...*\n"},{"metadata":{"_uuid":"940e4e064e3a975f65eaa984ef10077e5d8ce100"},"cell_type":"markdown","source":"And from other hand we can draw the percentage of the buyers, having the purchase in the report month and made at least one purchase within next period, f.e. 3  months. \nDefine the function *calc_future_period*"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"7eb6198f799ecfdf3884ced38128520a82b435a8"},"cell_type":"code","source":"#\n#-- define function\n#\ncalc_future_period <- function(z, skip_periods = 1, future_periods = 6 ) {\n  \n    ret  <- data.table();\n\n    #--  from (current_month + skip_periods) to (current_month + future_periods)\n\n    calc_dates <- colnames(z)[ 2:((ncol(z) - 1) - future_periods ) ]\n\n    #print( calc_dates )\n    \n    #-- For every month as current\n    for (p in calc_dates ) {\n      \n      #-- save only those, who made the purchase in the current month\n      z1 <- z[  which( z[[ p ]] > 0 ) ]\n\n      #\n      #-- now we select only those who had at least one visit and had NO any purchases within \n      #   the previous 6 months from the current\n      #\n\n      #-- get the previous periods as column names\n      x <- which(colnames(z) == p)\n      future_m  <- colnames(z)[ \n        (x + skip_periods ):(x + future_periods )  \n      ]\n      \n      z2 <- z[, c('fullVisitorId', future_m), with = F ]\n      \n      #-- calculate the number of transactions for buyers in current month and \n      #   filter only positive ones\n      z2$transQty <- rowSums( z2[, -1] )\n      z2 <- z2[ transQty > 0 & fullVisitorId %in% unique( z1$fullVisitorId ) ]\n      \n      #-- calculate the filnal ratio.  Visitors who made at least one purchase \n      #   within previous 6 months relatively buyers in current month\n      ratio <-  nrow(z2) * 100 / nrow(z1) \n      \n      ret <- rbind( ret, data.frame( z0 = nrow(z1), z2 = nrow(z2), value = round(ratio, 3), month = p ) )  \n    }\n    \n    ret\n    \n}","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"92d0b7cfa50de5d7e43f4674b3f5712b557bc8dd"},"cell_type":"markdown","source":"* And as expected, there is also the defined negative trend over time. * "},{"metadata":{"trusted":true,"_uuid":"fa31667252bbfcfff3529f2adddcd6a88744ed92"},"cell_type":"code","source":"ret_future_3 <- calc_future_period( z, 1 , 3 )\n\nt( ret_future_3 )\n\nggplot( data = ret_future_3, aes( x = as.Date( paste0( ret_future_3$month, '01'), '%Y%m%d' ) , y = value, label = round(value,1) ) ) + \n      geom_line(color=\"gray\") +\n      geom_point(color = \"red\") +\n      labs(x = NULL, y = NULL, title = \"Percentage of 'return' buyers within next 3 months\" ) + \n      geom_smooth( formula = y ~ x, method = 'lm', se = F ) +\n      scale_x_date(date_labels = \"%b %Y\", date_breaks = \"1 month\") +\n      scale_y_continuous( labels = function(x ){ return(paste( x, \"%\")) } ) +\n      theme( axis.text.x = element_text(angle = 90, size = 8, vjust = 0.5) ) + \n      geom_label( nudge_x = 0,  nudge_y = 0.25  )","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"096f6c17fb442194aa8d79047264f541bb0fbd73"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"2c23d2b10df9f1424d0f05275e5605543c45497a"},"cell_type":"markdown","source":""}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}