{"cells":[{"metadata":{"_uuid":"b1f468e43c639606cd544ee842e256fbf298f146"},"cell_type":"markdown","source":"# This Kernel aims to explore the data (EDA) of the Google Analytics Customer Revenue Prediction Dataset"},{"metadata":{"_uuid":"3d9bcc033f14a988813bbef9b366d32b57c51ae7"},"cell_type":"markdown","source":"### Table of Contents\n\n1. [Prepare the DataSet and Load Libraries](#section1)\n2. [Word-Cloud of Most Frequent Traffic Source Keywords](#section2)
\n3. [Average and Sum of Revenue by Channel Grouping](#section3)\n4. [Average and Sum of Revenue by Device Browser](#section4)\n5. [Average and Sum of Revenue by Device OS](#section5)\n6. [Average and Sum of Revenue by Device Browser and OS](#section6)\n7. [Average and Sum of Revenue by Device Category](#section7)\n8. [Average and Sum of Revenue by Device Browser and Device Category](#section8)\n9. [Distribution of Total and Average Revenue by Continent](#section9)\n10. [Correlation Matrix of all Session-Related Metrics and other binary variables with Transaction Revenue](#section10)\n11. [Continuous Variables versus \"is Transaction Revenue present\"](#section11)\n12. [Scatterplot and Bar Plot of all Continuous Variables and Time Variables to the Log of Revenue](#section12)\n13. [Average and Sum of Revenue by Traffic Source Campaign](#section13)\n14. [Average and Sum of Revenue by Traffic Sources](#section14)\n15. [Average and Sum of Revenue by Traffic Source Medium](#section15)\n16. [Average and Sum of Revenue by Traffic Source Medium and Traffic Source and Traffic Source Campaign (TreeMap)](#section16)\n17. [Analysis of Revenue versus Traffic Source Keyword search terms](#section17)\n18. [Analysis by Time](#section18)\n19. [Seasonality Tests](#section19)\n20. [Seasonality Tests Continued - (ARIMA tests with ACF and PACF by Day and Hour)](#section20)\n21. [Number Of Sessions Per User And Sessions with the most (Log) Revenue](#section21)"},{"metadata":{"_uuid":"8419b9508baccb3f9516173b58618191eccbec50"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"63fce02ae930e73e40e12d453582f267bd197160"},"cell_type":"markdown","source":"**1. Prepare the DataSet and Load Libraries:** \n\n> a. Filling NAs with 0 \n> b. Calculating log of the transaction revenue \n> c. Feature Engineering\n> d. Drop columns with a lot of missing values "},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"de9e8d7e9458e99b99315e1a37a1bfb968e45407"},"cell_type":"code","source":"# Install libraries\n#install.packages('ggplot2')\nlibrary(ggplot2)\nlibrary (magrittr)\nlibrary(dplyr)\nlibrary (maps)\nlibrary(rgdal)\nlibrary(ggmap)\nlibrary(osmar)\nlibrary(geosphere)\nlibrary(reshape)\nlibrary(tm)\nlibrary(ngram)\nlibrary(wordcloud)\nlibrary(corrplot)\nlibrary(scales)\nlibrary(ggpmisc)\nlibrary(lubridate)\nlibrary(forecast)\nlibrary(aTSA)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"0640f6eb00bbc572b346d6abd1bf31b383a982ef"},"cell_type":"code","source":"# Read dataset and drop columns with lots of missing values\ndfgoog = read.csv('../input/flat-train-data/flat_train.csv',colClasses = c(fullVisitorId=\"character\"))\ndfgoog_subset = subset(dfgoog,select=-c(device_browserVersion,totals_visits,device_browserSize,trafficSource_adwordsClickInfo.criteriaParameters,trafficSource_adwordsClickInfo.page,trafficSource_adwordsClickInfo.slot,trafficSource_adwordsClickInfo.gclId,trafficSource_adwordsClickInfo.adNetworkType,trafficSource_adwordsClickInfo.isVideoAd,trafficSource_adContent,trafficSource_campaignCode,device_operatingSystemVersion,device_mobileDeviceBranding,device_mobileDeviceModel,device_mobileInputSelector,device_mobileDeviceInfo,device_mobileDeviceMarketingName,device_flashVersion,device_language,device_screenColors,device_screenResolution,geoNetwork_cityId,geoNetwork_latitude,geoNetwork_longitude,geoNetwork_networkLocation))\ndfgoog_subset = subset(dfgoog_subset,select=-c(socialEngagementType))\n\n# Convert NA to 0\ndfgoog_subset$totals_newVisits[is.na(dfgoog_subset$totals_newVisits)] = 0\ndfgoog_subset$totals_bounces[is.na(dfgoog_subset$totals_bounces)] = 0\ndfgoog_subset$totals_pageviews[is.na(dfgoog_subset$totals_pageviews)] = 0\ndfgoog_subset$totals_hits[is.na(dfgoog_subset$totals_hits)] = 0\n\n# Feature Engineering\ndfgoog_subset$logrevenue = log(dfgoog_subset$totals_transactionRevenue)\ndfgoog_subset$totals_transactionRevenue[is.na(dfgoog_subset$totals_transactionRevenue)] = 0\ndfgoog_subset$logrevenue[is.na(dfgoog_subset$logrevenue)] = 0\ndfgoog_subset$year = substr(dfgoog_subset$date,1,4)\ndfgoog_subset$month = substr(dfgoog_subset$date,5,6)\ndfgoog_subset$day = substr(dfgoog_subset$date,7,8)\n\n# Domain one-hot\ndfgoog_subset$isnet = grepl('.net',dfgoog_subset$geoNetwork_networkDomain,fixed=TRUE)\ndfgoog_subset$iscom = grepl('.com',dfgoog_subset$geoNetwork_networkDomain,fixed=TRUE)\ndfgoog_subset$isorg = grepl('.org',dfgoog_subset$geoNetwork_networkDomain,fixed=TRUE)\ndfgoog_subset$device_isMobile = as.integer(as.logical(dfgoog_subset$device_isMobile))\ndfgoog_subset$iscom = as.integer(as.logical(dfgoog_subset$iscom))\ndfgoog_subset$isnet = as.integer(as.logical(dfgoog_subset$isnet))\ndfgoog_subset$isorg = as.integer(as.logical(dfgoog_subset$isorg))\ndfgoog_subset$isnotnetorgcom = as.integer(as.logical((dfgoog_subset$isnet == 0 & dfgoog_subset$isorg == 0 & dfgoog_subset$iscom == 0)))\ndfgoog_subset$trafficSource_isTrueDirect[dfgoog_subset$trafficSource_isTrueDirect == \"\"] = \"false\"\ndfgoog_subset$trafficSource_isTrueDirect = factor(dfgoog_subset$trafficSource_isTrueDirect)\ndfgoog_subset$trafficSource_isTrueDirect = as.integer(as.logical(dfgoog_subset$trafficSource_isTrueDirect))\ndfgoog_subset$trafficSource_isTrueDirect[is.na(dfgoog_subset$trafficSource_isTrueDirect)] = 0\n\ndfgoog_subset$is_revenue = as.integer(as.logical(dfgoog_subset$logrevenue > 0))\n\n\n# Text processing for Traffic Source Keyword\n# Get the count of keywords in a search term\nstrSplit = function(strtext){\n return (strsplit(strtext, ' '))\n}\n\ndfgoog_subset$keywords = dfgoog_subset$trafficSource_keyword\ndfgoog_subset$keywords = lapply(as.character(dfgoog_subset$keywords),removePunctuation)\ndfgoog_subset$keywords = lapply(as.character(dfgoog_subset$keywords),stripWhitespace)\ndfgoog_subset$keywords = lapply(as.character(dfgoog_subset$keywords),tolower)\ndfgoog_subset$clean_keywords = dfgoog_subset$keywords\n\ndfgoog_subset$split_text = lapply(as.character(dfgoog_subset$clean_keywords),strSplit)\ndfgoog_subset$keyword_count = lapply(as.character(dfgoog_subset$split_text),wordcount)\ndfgoog_subset$keywordlength = lapply(dfgoog_subset$clean_keywords,nchar)\n\ndfgoog_subset$date_2 = paste(substr(dfgoog_subset$date,1,4),'-',substr(dfgoog_subset$date,5,6),'-',substr(dfgoog_subset$date,7,8),sep='')\ndfgoog_subset$date_2= ymd(dfgoog_subset$date_2) \ndfgoog_subset$weekdays = weekdays(as.Date(dfgoog_subset$date_2))\ndfgoog_subset$posix_date = (as.POSIXct(dfgoog_subset$visitStartTime, origin=\"1970-01-01\"))\n\ndfgoog_subset$hour_visit = hour(dfgoog_subset$posix_date)\ndfgoog_subset$min_visit = minute(dfgoog_subset$posix_date)\ndfgoog_subset$sec_visit = second(dfgoog_subset$posix_date)\n\n# Two binning types for minutes and seconds\ndfgoog_subset$minbin = cut(dfgoog_subset$min_visit,c(-Inf,10,20,30,40,50,60,Inf))\ndfgoog_subset$minbin_2 = cut(dfgoog_subset$min_visit,c(-Inf,30,60,Inf))\n\ndfgoog_subset$secbin = cut(dfgoog_subset$sec_visit,c(-Inf,10,20,30,40,50,60,Inf))\ndfgoog_subset$secbin_2 = cut(dfgoog_subset$sec_visit,c(-Inf,30,60,Inf))\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"29bb17a28674819fb539787049593b32b6d6476e"},"cell_type":"markdown","source":" "},{"metadata":{"_uuid":"e564fa5440bb21ab1481afb07dad5142a510973c"},"cell_type":"markdown","source":"**2. Word-Cloud of Most Frequent Traffic Source Keywords**\n\n> The top Traffic Source Keywords provided are:
\n> 1. 6qehscssdk0z36ri
\n> 2. google
\n> 3. merchandise
\n> 4. store
\n> 5. targeting
"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"0d093247f2745fa83a21826b0b67ea6534747d72"},"cell_type":"code","source":"keywords = dfgoog_subset$trafficSource_keyword\nkeywords = as.character(keywords)\nkeywords = removePunctuation(keywords)\nconcat_keywords = paste(keywords,collapse = ' ')\nconcat_keywords = gsub('not provided','',concat_keywords)\nconcat_keywords = stripWhitespace(concat_keywords)\nconcat_keywords = tolower(concat_keywords)\n#head(concat_keywords)\n\nsplit_keywords = strsplit(concat_keywords,' ')\ndocs = data.frame(table(split_keywords))\n\nwordcloud(docs$split_keywords,docs$Freq,min.freq=3)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f1ca12f23747cd2d8a64f8e64562a26780a3e0b1"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"4eb044077ff387c1a00d6212d7e9d8136ad62ad0"},"cell_type":"markdown","source":"**3. Average and Sum of Revenue by Channel Grouping**\n\n> The Channels with the most Total Revenue:\n> 1. Referrals\n> 2. Direct Channel\n> 3. Organic Search\n> \n\n> The Channels with the most Average Revenue:\n> 1. Display \n> 2. Referrals\n> 3. Affiliates"},{"metadata":{"trusted":true,"_uuid":"cfc22e859266fc58df2c88f63cde99f95e3c6a05","_kg_hide-input":true},"cell_type":"code","source":"ggplot(dfgoog_subset,aes(x=channelGrouping,y=totals_transactionRevenue,fill=channelGrouping)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + theme(axis.text.x=element_blank()) + ggtitle(\"Sum of Revenue by Channel Grouping\")\nggplot(dfgoog_subset,aes(x=channelGrouping,y=totals_transactionRevenue,fill=channelGrouping)) + geom_bar(stat=\"summary\",fun.y=\"mean\") + theme(axis.text.x=element_blank()) + ggtitle(\"Average Revenue by Channel Grouping\") ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"13b1faa0a7d486f7e59f72cbe973a46822c5614f"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"cea13737f5c5c4d8e22ebe246c59d616c4cb6df2"},"cell_type":"markdown","source":"**4. Average and Sum of Revenue by Device Browser** \n\n> Browsers with highest Total Revenue:\n> 1. Chrome\n> 2. Firefox\n> 3. Safari\n\n> Browsers with highest Average Revenue\n> 1. Firefox\n> 2. Chrome\n> 3. Edge"},{"metadata":{"trusted":true,"_uuid":"17dc9b9f1190c1f8244f81a6e176238a1a82a0b7","_kg_hide-input":true,"_kg_hide-output":false,"scrolled":false},"cell_type":"code","source":"dfpie_sum = dfgoog_subset %>% \n group_by(device_browser) %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000000) %>%\n arrange(desc(share))\n\nbp_sum = ggplot(dfpie_sum[c(1:5),],aes(x=\"\",y=share,fill=device_browser)) + geom_bar(width=1,size=1,stat=\"identity\")\npie_sum = bp_sum + coord_polar(\"y\") + geom_text(aes(label=paste0(\"$\",round(share))),position=position_stack(vjust=0.5),check_overlap=TRUE) + theme(axis.line=element_blank(),axis.text=element_blank(),axis.ticks=element_blank()) + ggtitle(\"Top Five Sum of Revenue by Device Browser in billions \")\n\ndfpie_avg = dfgoog_subset %>% \n group_by(device_browser) %>%\n summarise(volume=mean(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000) %>%\n arrange(desc(share))\n\nbp_avg = ggplot(dfpie_avg[c(1:5),],aes(x=\"\",y=share,fill=device_browser)) + geom_bar(width=1,size=1,stat=\"identity\")\npie_avg = bp_avg + coord_polar(\"y\") + geom_text(aes(label=paste0(\"$\",round(share,2))),position=position_stack(vjust=0.5),check_overlap=TRUE) + theme(axis.line=element_blank(),axis.text=element_blank(),axis.ticks=element_blank()) + ggtitle(\"Top Five Average of Revenue by Device Browser in millions\")\n\npie_sum\npie_avg\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0c6eccb6ee976e8e259a7819e6a06279d8d0657b"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"498c8f47920c689dccd7ebec21c8a13ba7023d0e"},"cell_type":"markdown","source":"**5. Average and Sum of Revenue by Device OS** \n\n> OS with highest Total Revenue:\n> 1. Macintosh\n> 2. Windows\n> 3. Chrome OS\n\n> OS with highest Average Revenue\n> 1. Chrome OS\n> 2. Macintosh\n> 3. Linux"},{"metadata":{"trusted":true,"_uuid":"5af1596cfe9940989d0b59adccee95d7e180c6e7","_kg_hide-input":true},"cell_type":"code","source":"\n\n\ndfpie_sum = dfgoog_subset %>% \n group_by(device_operatingSystem) %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000000) %>%\n arrange(desc(share))\n\nbp_sum = ggplot(dfpie_sum[c(1:5),],aes(x=\"\",y=share,fill=device_operatingSystem)) + geom_bar(width=1,size=1,stat=\"identity\")\npie_sum = bp_sum + coord_polar(\"y\") + geom_text(aes(label=paste0(\"$\",round(share))),position=position_stack(vjust=0.5),check_overlap=TRUE) + theme(axis.line=element_blank(),axis.text=element_blank(),axis.ticks=element_blank()) + ggtitle(\"Top Five Sum of Revenue by Device OS in billions \")\n\ndfpie_avg = dfgoog_subset %>% \n group_by(device_operatingSystem) %>%\n summarise(volume=mean(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000) %>%\n arrange(desc(share))\n\nbp_avg = ggplot(dfpie_avg[c(1:5),],aes(x=\"\",y=share,fill=device_operatingSystem)) + geom_bar(width=1,size=1,stat=\"identity\")\npie_avg = bp_avg + coord_polar(\"y\") + geom_text(aes(label=paste0(\"$\",round(share,5))),position=position_stack(vjust=0.5),check_overlap=TRUE) + theme(axis.line=element_blank(),axis.text=element_blank(),axis.ticks=element_blank()) + ggtitle(\"Top Five Average of Revenue by Device OS in millions\")\n\npie_sum\npie_avg\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9e5e0401909583953ffec85e7d456d64dc8fdfd9"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"78ec1df6348e09265529f9cb3180e10412fbb07c"},"cell_type":"markdown","source":"**6. Average and Sum of Revenue by Device Browser and OS** \n\n> OS + Browser with highest Total Revenue:\n> 1. Macintosh + Chrome\n> 2. Windows + Chrome\n> 3. Chrome OS + Chrome\n\n> OS + Browser with highest Average Revenue\n> 1. Chrome OS + Chrome\n> 2. Macintosh + Chrome\n> 3. Windows + Firefox"},{"metadata":{"trusted":true,"_uuid":"0ad5a1b0f563247912648d23241b8447bbb75092","_kg_hide-input":true},"cell_type":"code","source":"# Filter by top browsers and OS only\ndfpie_sum = dfgoog_subset %>% \n group_by(device_operatingSystem,device_browser) %>%\n filter(device_browser == \"Chrome\" | device_browser == \"Safari\" | device_browser == \"Edge\" | device_browser == \"Firefox\" | device_browser == \"Internet Explorer\") %>%\n filter(device_operatingSystem == \"Chrome OS\" | device_operatingSystem == \"Android\" | device_operatingSystem == \"Linux\" | device_operatingSystem == \"Macintosh\" | device_operatingSystem == \"Windows\") %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000000) %>%\n arrange(desc(share))\n\ndfpie_avg = dfgoog_subset %>% \n group_by(device_operatingSystem,device_browser) %>%\n filter(device_browser == \"Chrome\" | device_browser == \"Safari\" | device_browser == \"Edge\" | device_browser == \"Firefox\" | device_browser == \"Internet Explorer\") %>%\n filter(device_operatingSystem == \"Chrome OS\" | device_operatingSystem == \"Android\" | device_operatingSystem == \"Linux\" | device_operatingSystem == \"Macintosh\" | device_operatingSystem == \"Windows\") %>%\n summarise(volume=mean(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000) %>%\n arrange(desc(share))\n\nbp_sum = ggplot(dfpie_sum,aes(x=device_browser,y=share,fill=device_browser)) + geom_bar(width=1,size=1,stat=\"identity\") + geom_text(aes(label=paste0(\"$\",round(share))),position=position_stack(vjust=1),check_overlap=TRUE) + facet_wrap(~device_operatingSystem) + theme(axis.line.x=element_blank(),axis.text.x=element_blank(),axis.ticks.x=element_blank()) + ggtitle(\"Total Revenue by Device OS and Browser in billions\")\nbp_avg = ggplot(dfpie_avg,aes(x=device_browser,y=share,fill=device_browser)) + geom_bar(width=1,size=1,stat=\"identity\") + geom_text(aes(label=paste0(\"$\",round(share,2))),position=position_stack(vjust=1),check_overlap=TRUE) + facet_wrap(~device_operatingSystem) + theme(axis.line.x=element_blank(),axis.text.x=element_blank(),axis.ticks.x=element_blank()) + ggtitle(\"Average Revenue by Device OS and Browser in millions\")\n\nbp_sum\nbp_avg","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"17915e419a0a59145cddfb109c651171041af9e2"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"152bede900ff98976b8bc4831429dd19be485794"},"cell_type":"markdown","source":"**7. Average and Sum of Revenue by Device Category**\n\n> Desktops have the highest Total and Average Revenue \n"},{"metadata":{"trusted":true,"_uuid":"83af6894b23fc5e1ff1c4a9358ade37232e40ebf","_kg_hide-input":true},"cell_type":"code","source":"ggplot(dfgoog_subset,aes(x=device_deviceCategory,y=totals_transactionRevenue,fill=device_deviceCategory)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + theme(axis.text.x=element_blank()) + ggtitle(\"Total Revenue by Device Category\") \nggplot(dfgoog_subset,aes(x=device_deviceCategory,y=totals_transactionRevenue,fill=device_deviceCategory)) + geom_bar(stat=\"summary\",fun.y=\"mean\") + theme(axis.text.x=element_blank()) + ggtitle(\"Average Revenue by Device Category\") ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a132242b9c4a53f088209dc9c2979f1c37330ceb"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"c4c8369e29b7d2205f37cd85461defadb075c665"},"cell_type":"markdown","source":"**8. Average and Sum of Revenue by Device Browser and Device Category**\n\n> Category + Browser with highest Total Revenue:\n> 1. Desktop + Chrome\n> 2. Desktop + Firefox\n> 3. Mobile + Chrome\n\n> OS + Browser with highest Average Revenue\n> 1. Desktop + Chrome\n> 2. Desktop + Firefox\n> 3. Desktop + Edge"},{"metadata":{"trusted":true,"_uuid":"77e8ecbc1a7c4938d2ec5c21b40603271c33c1da","_kg_hide-input":true},"cell_type":"code","source":"# Filter by top browsers and OS only\ndfpie_sum = dfgoog_subset %>% \n group_by(device_deviceCategory,device_browser) %>%\n filter(device_browser == \"Chrome\" | device_browser == \"Safari\" | device_browser == \"Edge\" | device_browser == \"Firefox\" | device_browser == \"Internet Explorer\") %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000000) %>%\n arrange(desc(share))\n\ndfpie_avg = dfgoog_subset %>% \n group_by(device_deviceCategory,device_browser) %>%\n filter(device_browser == \"Chrome\" | device_browser == \"Safari\" | device_browser == \"Edge\" | device_browser == \"Firefox\" | device_browser == \"Internet Explorer\") %>%\n summarise(volume=mean(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000) %>%\n arrange(desc(share))\n\nbp_sum = ggplot(dfpie_sum,aes(x=device_browser,y=share,fill=device_browser)) + geom_bar(width=1,size=1,stat=\"identity\") + geom_text(aes(label=paste0(\"$\",round(share))),position=position_stack(vjust=1),check_overlap=TRUE) + facet_wrap(~device_deviceCategory) + theme(axis.line.x=element_blank(),axis.text.x=element_blank(),axis.ticks.x=element_blank()) + ggtitle(\"Total Revenue by Device Category and Browser in billions\")\nbp_avg = ggplot(dfpie_avg,aes(x=device_browser,y=share,fill=device_browser)) + geom_bar(width=1,size=1,stat=\"identity\") + geom_text(aes(label=paste0(\"$\",round(share,2))),position=position_stack(vjust=1),check_overlap=TRUE) + facet_wrap(~device_deviceCategory) + theme(axis.line.x=element_blank(),axis.text.x=element_blank(),axis.ticks.x=element_blank()) + ggtitle(\"Average Revenue by Device Category and Browser in millions\")\n\nbp_sum\nbp_avg","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a8b987aacbec124389a8d302c1b762e58cf7e69b"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"5209eeccd683306a3641570258953a995e86adcf"},"cell_type":"markdown","source":"**9. Distribution of Total and Average Revenue by Continent** \n\n> It is clear that North American Revenue outstrips revenue from other parts of the world."},{"metadata":{"trusted":true,"_uuid":"6de16a2350f898361c0574fa3a1d872cda4b5cb4","_kg_hide-input":true},"cell_type":"code","source":"cont_shp = readOGR(dsn = \"../input/worldcontinents\",layer = \"world-continents\")\ncontinents_map <- fortify(cont_shp, region=\"CONTINENT\")\n\n# Rename Oceania to Australia and Americas to North America\ndfgoog_subset$geoNetwork_continent = as.character(dfgoog_subset$geoNetwork_continent)\ndfgoog_subset$geoNetwork_continent[dfgoog_subset$geoNetwork_continent == 'Oceania'] <- 'Australia'\ndfgoog_subset$geoNetwork_continent[dfgoog_subset$geoNetwork_continent == 'Americas'] <- 'North America'\ndfgoog_subset$geoNetwork_continent = as.factor(dfgoog_subset$geoNetwork_continent)\n\n# \"#FFFFFF\", \"#9999FF\", \"#66FFFF\", \"#66FFCC\", \"#99FF33\", \"#FFFF33\",\ncolours = c( \"#FFCC33\", \"#FF9900\", \"#FF6600\", \"#FF0000\", \"#CC0000\")\n\n# Get the average and total revenues\ndfcont_sum = dfgoog_subset %>% \n group_by(geoNetwork_continent) %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(total_revenue=volume/1000000000)\n\n# Get the average and total revenues\ndfcont_avg = dfgoog_subset %>% \n group_by(geoNetwork_continent) %>%\n summarise(volume=mean(totals_transactionRevenue)) %>%\n mutate(avg_revenue=volume/1000000)\n\n# Plot the total\ngg <- ggplot()\ngg <- gg + geom_map(data=continents_map,\n map=continents_map,\n aes(x=long, y=lat, map_id=id),\n color=\"black\")\ngg <- gg + geom_map(data=dfcont_sum,\n map=continents_map,\n aes(map_id=geoNetwork_continent, fill=total_revenue),\n color=\"black\")\ngg <- gg + scale_fill_distiller(\"Total Revenue in billions\") # needs latest ggplot2\ngg <- gg + scale_fill_gradientn(colours = colours ) # needs latest ggplot2\ngg <- gg + coord_equal()\ngg <- gg + theme_bw()\ngg <- gg + labs(x=NULL, y=NULL)\ngg <- gg + theme(panel.border=element_blank())\ngg <- gg + theme(panel.grid=element_blank()) + theme(axis.line.x=element_blank(),axis.text.x=element_blank(),axis.ticks.x=element_blank())\ngg <- gg + ggtitle('Total Revenue by Continent (in Billions)')\ngg\n\ngg_bar = ggplot(dfcont_sum,aes(x=geoNetwork_continent,y=total_revenue,fill=geoNetwork_continent)) + geom_bar(stat=\"identity\") + geom_text(data=dfcont_sum, aes(x=geoNetwork_continent,y=total_revenue,label=round(total_revenue,0))) + theme(axis.text.x=element_blank()) + ggtitle(\"Total Revenue by Continent (billions)\") \ngg_bar\n \n# Plot the average\ngg <- ggplot()\ngg <- gg + geom_map(data=continents_map,\n map=continents_map,\n aes(x=long, y=lat, map_id=id),\n color=\"black\")\ngg <- gg + geom_map(data=dfcont_avg,\n map=continents_map,\n aes(map_id=geoNetwork_continent, fill=avg_revenue),\n color=\"black\")\ngg <- gg + scale_fill_distiller(\"Avg. Revenue in millions\") # needs latest ggplot2\ngg <- gg + scale_fill_gradientn(colours = colours ) # needs latest ggplot2\ngg <- gg + coord_equal()\ngg <- gg + theme_bw()\ngg <- gg + labs(x=NULL, y=NULL)\ngg <- gg + theme(panel.border=element_blank())\ngg <- gg + theme(panel.grid=element_blank()) + theme(axis.line.x=element_blank(),axis.text.x=element_blank(),axis.ticks.x=element_blank())\ngg <- gg + ggtitle(\"Average Revenue by Continent (in millions)\")\ngg\n\ngg_bar = ggplot(dfcont_avg,aes(x=geoNetwork_continent,y=avg_revenue,fill=geoNetwork_continent)) + geom_bar(stat=\"identity\") + geom_text(data=dfcont_avg, aes(x=geoNetwork_continent,y=avg_revenue,label=round(avg_revenue,2))) + theme(axis.text.x=element_blank()) + ggtitle(\"Average Revenue by Continent (millions)\") \ngg_bar\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"59262c824dc2fb948ac70068584a9bc2e591c72f"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"ad57954466b4c33efaa0792f9b0aadf076f0e3f7"},"cell_type":"markdown","source":"**10. Correlation Matrix of all Session-Related Metrics and other binary variables with Transaction Revenue** \n\n> Some interesting points stand out:
\n> a. Direct Traffic Sources have a high negative correlation with number of New Visits: (cor: -0.63737026 considering positive revenue records and -0.61 considering all records )
\n> b. Log Revenue is positively correlated with Direct Traffic Source visits and Number Of Visits in a Session
\n> c. Log Revenue is negatively correlated with number of New Visits and visits from a Mobile Device
\n> \n> This suggests that New Visits are less likely to come from Direct Traffic, and they do not result in revenue during the session. Sessions from Mobile Devices are less likely to result in revenue."},{"metadata":{"trusted":true,"_uuid":"65e8cb8834490aea6c80548f859345bc2e6c509d","_kg_hide-input":true},"cell_type":"code","source":"# First, with all the data\n#print ('Correlation Matrix considering all Data')\nsession_vars = subset(dfgoog_subset,select=c(logrevenue,totals_hits,totals_pageviews,totals_bounces,totals_newVisits,totals_transactionRevenue,isnet,iscom,isorg,isnotnetorgcom,visitNumber,device_isMobile,trafficSource_isTrueDirect))\ncor_sess = cor(session_vars)\n#cor_sess\ncorsess_melt = melt(cor_sess)\nggplot(data=corsess_melt,aes(x=X1,y=X2,fill=value)) + geom_tile(color=\"white\") + scale_fill_gradient2(low=\"blue\",high=\"red\",mid=\"white\",midpoint=0,limit=c(-1,1)) + theme(axis.text.x=element_text(angle=45,vjust=1,hjust=1)) + ggtitle('Correlation Matrix of Session Statistics (all records)')\n\n# Next, with data with positive revenue only\n#print ('Correlation Matrix considering only Data with positive revenue')\nsession_vars = session_vars[session_vars$totals_transactionRevenue > 0,]\ncor_sess = cor(session_vars)\n#cor_sess\ncorsess_melt = melt(cor_sess)\nggplot(data=corsess_melt,aes(x=X1,y=X2,fill=value)) + geom_tile(color=\"white\") + scale_fill_gradient2(low=\"blue\",high=\"red\",mid=\"white\",midpoint=0,limit=c(-1,1)) + theme(axis.text.x=element_text(angle=45,vjust=1,hjust=1)) + ggtitle('Correlation Matrix of Session Statistics(+ve Revenue)')\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2faac9bc9a008fd0effb73764754354d65ed6a1b"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"92d89162be4d8fc6c1cb7665c1158f32c06f10cc"},"cell_type":"markdown","source":"**11. Continuous Variables versus \"is Transaction Revenue present\"** \n\n> All Continuous variables are skewed towards the '0' flag of the binary variable 'Is Revenue Present'\n"},{"metadata":{"trusted":true,"_uuid":"0bbd575218189fe3d09e8437bf63c90ed2577376","_kg_hide-input":true},"cell_type":"code","source":"session_vars = subset(dfgoog_subset,select=c(is_revenue,logrevenue,totals_hits,totals_pageviews,totals_bounces,totals_newVisits,visitNumber,month,day,hour_visit,min_visit,sec_visit))\nggplot(session_vars, aes(x = as.factor(is_revenue), y = totals_hits,fill = as.factor(is_revenue))) + geom_bar(stat=\"summary\",fun.y=\"sum\")\nggplot(session_vars, aes(x = as.factor(is_revenue), y = totals_pageviews,fill = as.factor(is_revenue))) + geom_bar(stat=\"summary\",fun.y=\"sum\")\nggplot(session_vars, aes(x = as.factor(is_revenue), y = totals_bounces,fill = as.factor(is_revenue))) + geom_bar(stat=\"summary\",fun.y=\"sum\")\nggplot(session_vars, aes(x = as.factor(is_revenue), y = totals_newVisits,fill = as.factor(is_revenue))) + geom_bar(stat=\"summary\",fun.y=\"sum\")\nggplot(session_vars, aes(x = as.factor(is_revenue), y = visitNumber,fill = as.factor(is_revenue))) + geom_bar(stat=\"summary\",fun.y=\"sum\")\n\nggplot(session_vars, aes(x = as.factor(is_revenue), y = hour_visit,fill = as.factor(is_revenue))) + geom_bar(stat=\"summary\",fun.y=\"sum\")\nggplot(session_vars, aes(x = as.factor(is_revenue), y = min_visit,fill = as.factor(is_revenue))) + geom_bar(stat=\"summary\",fun.y=\"sum\")\nggplot(session_vars, aes(x = as.factor(is_revenue), y = sec_visit,fill = as.factor(is_revenue))) + geom_bar(stat=\"summary\",fun.y=\"sum\")\n\n\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"24b08993fd9a8946e995599e46a192be274acf90"},"cell_type":"markdown","source":""},{"metadata":{"_kg_hide-input":true,"_uuid":"88ef5d8b85c48ea2c4627e0e361866efdfa960dc"},"cell_type":"markdown","source":"**12 Scatterplot and Bar Plot of all Continuous Variables and Time Variables to the Log of Revenue**\n\n> The scatterplots show that binning with 100 units in each bin may be a useful discriminator. Upto 50 units in each bin for the visitNumber, may be useful. "},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"869c6069274762324aa5c60d861cf623884b8fc3"},"cell_type":"code","source":"ggplot(session_vars,aes(x=totals_hits,y=logrevenue)) + geom_point() + ggtitle('Total Hits versus Log Revenue')\nggplot(session_vars,aes(x=totals_pageviews,y=logrevenue)) + geom_point() + ggtitle('Total PageViews versus Log Revenue')\nggplot(session_vars,aes(x=totals_bounces,y=logrevenue)) + geom_point() + ggtitle('Total Bounces versus Log Revenue')\nggplot(session_vars,aes(x=totals_newVisits,y=logrevenue)) + geom_point() + ggtitle('Total New Visits versus Log Revenue')\nggplot(session_vars,aes(x=visitNumber,y=logrevenue)) + geom_point() + ggtitle('Visit Number versus Log Revenue')\n\nggplot(session_vars,aes(x=hour_visit,y=logrevenue,fill = hour_visit)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + ggtitle('Hour of Visit versus Log Revenue')\nggplot(session_vars,aes(x=min_visit,y=logrevenue,fill = min_visit)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + ggtitle('Minute of Visit versus Log Revenue')\nggplot(session_vars,aes(x=sec_visit,y=logrevenue, fill = sec_visit)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + ggtitle('Second of Visit versus Log Revenue')\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f4e683e21336c03e6c26b849b58cfb28e6fe3c22"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"5ea733d51c635a69272b71d5b235c081cc4ff1f8"},"cell_type":"markdown","source":"**13. Average and Sum of Revenue by Traffic Source Campaign** \n\n> a. Revenue is mostly generated without aid of campaigns
\n> b. Dynamic Search Ads and AW - Accessories campaigns generate most of the remaining revenue
\n> c. AW - Accessories and Dynamic Search Ads campaigns generate the highest average revenue
\n"},{"metadata":{"trusted":true,"_uuid":"ebc40faaa4cdc8dd36be3633834ebff4122cc844","_kg_hide-input":true},"cell_type":"code","source":"ggplot(dfgoog_subset,aes(x=trafficSource_campaign,y=totals_transactionRevenue,fill=trafficSource_campaign)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + theme(axis.text.x=element_blank()) + ggtitle(\"Sum of Revenue by Traffic Source Campaign\")\nggplot(dfgoog_subset[dfgoog_subset$trafficSource_campaign != '(not set)',],aes(x=trafficSource_campaign,y=totals_transactionRevenue,fill=trafficSource_campaign)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + theme(axis.text.x=element_blank()) + ggtitle(\"Sum of Revenue by Traffic Source Campaign - V2\")\nggplot(dfgoog_subset,aes(x=trafficSource_campaign,y=totals_transactionRevenue,fill=trafficSource_campaign)) + geom_bar(stat=\"summary\",fun.y=\"mean\") + theme(axis.text.x=element_blank()) + ggtitle(\"Average of Revenue by Traffic Source Campaign\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f6cd34c280e77f8d3222e0e78ac438e6545ee4a"},"cell_type":"markdown","source":" "},{"metadata":{"trusted":true,"_uuid":"af8de3e2d3030157589c5b53ebb47c2066b06a37"},"cell_type":"markdown","source":"**14. Average and Sum of Revenue by Traffic Sources**
\n\n> The top traffic sources by sum of revenue are:
\n> 1. mall.googleplex.com
\n> 2. direct sources
\n> 3. google
\n\n> The top traffic sources by average of revenue are:
\n> 1. basecamp.com
\n> 2. mall.aol.com
\n> 3. mall.google.com
\n"},{"metadata":{"trusted":true,"_uuid":"0b0403c016447ac0fa10dce3e543905cf16bbdb2","_kg_hide-input":true},"cell_type":"code","source":"\ndfpie_sum = dfgoog_subset %>% \n group_by(trafficSource_source) %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000000) %>%\n arrange(desc(share))\n\nbp_sum = ggplot(dfpie_sum[c(1:5),],aes(x=\"\",y=share,fill=trafficSource_source)) + geom_bar(width=1,size=1,stat=\"identity\")\npie_sum = bp_sum + coord_polar(\"y\") + geom_text(aes(label=paste0(\"$\",round(share))),position=position_stack(vjust=0.5),check_overlap=TRUE) + theme(axis.line=element_blank(),axis.text=element_blank(),axis.ticks=element_blank()) + ggtitle(\"Top Five Sum of Revenue by Traffic Source in billions \")\npie_sum\n\n\n\ndfpie_sum = dfgoog_subset %>% \n group_by(trafficSource_source) %>%\n summarise(volume=mean(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000) %>%\n arrange(desc(share))\n\nbp_sum = ggplot(dfpie_sum[c(1:5),],aes(x=\"\",y=share,fill=trafficSource_source)) + geom_bar(width=1,size=1,stat=\"identity\")\npie_sum = bp_sum + coord_polar(\"y\") + geom_text(aes(label=paste0(\"$\",round(share))),position=position_stack(vjust=0.5),check_overlap=TRUE) + theme(axis.line=element_blank(),axis.text=element_blank(),axis.ticks=element_blank()) + ggtitle(\"Top Five Average of Revenue by Traffic Source in millions\")\npie_sum\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a447ff65c5f3c12e2991a0272be24b184711a3b7"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"66a1ef702f777636344c38800ddaa65bdc737121"},"cell_type":"markdown","source":"**15. Average and Sum of Revenue by Traffic Source Medium**\n\n> Top traffic source mediums by total revenue:
\n> 1. Referrals
\n> 2. No Medium
\n> 3. Organic Medium
\n\n> Top traffic source mediums by average revenue:
\n> 1. CPM
\n> 2. No Medium
\n> 3. Referrals
\n"},{"metadata":{"trusted":true,"_uuid":"d49b567d19f565857492842b11f75780d7db7b2f","_kg_hide-input":true},"cell_type":"code","source":"ggplot(dfgoog_subset,aes(x=trafficSource_medium,y=totals_transactionRevenue,fill=trafficSource_medium)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + theme(axis.text.x=element_blank()) + ggtitle(\"Sum of Revenue by Traffic Source Medium\")\nggplot(dfgoog_subset,aes(x=trafficSource_medium,y=totals_transactionRevenue,fill=trafficSource_medium)) + geom_bar(stat=\"summary\",fun.y=\"mean\") + theme(axis.text.x=element_blank()) + ggtitle(\"Average of Revenue by Traffic Source Medium\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3bafc039e31983f37ef38ff7832e35103b263b74"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"d0036f1afd3aac111e3f5b6543752f4f27d7c65c"},"cell_type":"markdown","source":"**16. Average and Sum of Revenue by Traffic Source Medium and Traffic Source and Traffic Source Campaign (TreeMap)** \n\n> In this multi-dimensional analysis of total revenue, using filters by top Medium, Campaign, and Source markers :
\n> Total Revenue is largest from
\n> > 1. mall.googleplex.com with no Traffic Campaign set and from the referral Traffic Source\n\n\n"},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"f193135502bafe618736283ef998205f44c1f461"},"cell_type":"code","source":"library (treemap)\ndfpie_sum = dfgoog_subset %>% \n group_by(trafficSource_medium,trafficSource_campaign,trafficSource_source) %>%\n filter(trafficSource_medium == \"(none)\" | trafficSource_medium == \"organic\" | trafficSource_medium == \"referral\" | trafficSource_medium == \"cpm\" | trafficSource_medium == \"cpc\") %>%\n filter(trafficSource_campaign == \"(not set)\" | trafficSource_campaign == \"AW - Dynamic Search Ads Whole Site\" | trafficSource_campaign == \"AW - Accessories\") %>%\n filter(trafficSource_source == \"(direct)\" | trafficSource_source == \"google\" | trafficSource_source == \"mall.googleplex.com\") %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000000) %>%\n arrange(desc(share))\n\n#dfpie_sum\ntreemap(dfpie_sum,index=c('trafficSource_medium','trafficSource_campaign','trafficSource_source'),vSize='share',aspRatio=1,overlap.labels = 1,align.labels=list(c('left','top'),c('right','bottom'),c('center','center')),title=\"Treemap: Total Revenue by Traffic Medium (top left), Traffic Campaign (bottom right), and Traffic Source (center)\",fontsize.title=9)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"406620b78fdd8dca4029638869948163e3f10fea"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"708ac8cffbe7bf28c849acac671cea26361a3b01"},"cell_type":"markdown","source":"**17. Analysis of Revenue versus Traffic Source Keyword search terms**"},{"metadata":{"trusted":true,"_uuid":"d95b0197b857f1c0e9e0b4468e4d8e808d0ac670","_kg_hide-input":true,"scrolled":false},"cell_type":"code","source":"# Filter out records where no keyword is provided\ndfgoog_keyword = dfgoog_subset[dfgoog_subset$clean_keywords != 'not provided' & dfgoog_subset$clean_keywords != '',]\ndfgoog_keyword$keywordlength = as.numeric(dfgoog_keyword$keywordlength)\ndfgoog_keyword$logrevenue = as.numeric(dfgoog_keyword$logrevenue)\ndfgoog_keyword$keyword_count = as.numeric(dfgoog_keyword$keyword_count)\n\n# Most frequent number of search terms\ncnts = as.data.frame(table(dfgoog_keyword$keyword_count))\ncolnames(cnts)[1] = 'n_search_terms'\nggplot(cnts,aes(x=n_search_terms,y=Freq,fill=n_search_terms)) + geom_bar(stat=\"identity\") + geom_text(aes(label=Freq)) + ggtitle('Count of traffic source keyword searches by number of search terms')\n\ndfgoog_keyword_1 = dfgoog_keyword %>%\n filter(keyword_count == 1) %>%\n group_by(keyword_count,as.character(keywords)) %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000000) %>%\n arrange(desc(share))\n\ndfgoog_keyword_2 = dfgoog_keyword %>%\n filter(keyword_count == 2) %>%\n group_by(keyword_count,as.character(keywords)) %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000000) %>%\n arrange(desc(share))\n\ndfgoog_keyword_3 = dfgoog_keyword %>%\n filter(keyword_count == 3) %>%\n group_by(keyword_count,as.character(keywords)) %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000000) %>%\n arrange(desc(share))\n\ndfgoog_keyword_4 = dfgoog_keyword %>%\n filter(keyword_count == 4) %>%\n group_by(keyword_count,as.character(keywords)) %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(share=volume/1000000000) %>%\n arrange(desc(share))\n\n\n#dfgoog_keyword_5 = dfgoog_keyword %>%\n# filter(keyword_count == 5) %>%\n# group_by(keyword_count,as.character(keywords)) %>%\n# summarise(volume=sum(totals_transactionRevenue)) %>%\n# mutate(share=volume/1000000000) %>%\n# arrange(desc(share))\n\ndfgoog_keyword_all = rbind(dfgoog_keyword_1[c(1:5),],dfgoog_keyword_2[c(1:5),],dfgoog_keyword_3[c(1:5),]) #dfgoog_keyword_4[c(1:5),],dfgoog_keyword_5[c(1:5),])\ncolnames(dfgoog_keyword_all)[2] = 'keywords'\ncolnames(dfgoog_keyword_all)[4] = 'revenue'\nbp_sum = ggplot(dfgoog_keyword_all,aes(x=keywords,y=revenue,fill=keywords)) + geom_bar(width=1,size=1,stat=\"identity\") + geom_text(aes(label=paste0(\"$\",round(revenue,2))),position=position_stack(vjust=1),check_overlap=TRUE) + facet_wrap(~keyword_count) + theme(axis.line.x=element_blank(),axis.text.x=element_blank(),axis.ticks.x=element_blank()) + ggtitle(\"Total Revenue by Keyword Search Term in billions (facet by # Keywords)\")\nbp_sum","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5927a141a56f3828cf8e9f99898395d8c6e9863f"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"3457302d0495ce4f069a4b8a78b3e0c4d818ac4b"},"cell_type":"markdown","source":"**18. Analysis by Time**\n\n"},{"metadata":{"trusted":true,"_uuid":"fcd75975383db6c7899c7988755b326481a813ef","_kg_hide-input":true},"cell_type":"code","source":"\n# By Day\nggplot(dfgoog_subset,aes(x=day,y=logrevenue,fill=day)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + ggtitle('Daily Total Revenue by Day of Month') + theme(axis.text.x=element_blank())\n\n# By Day of Week\nggplot(dfgoog_subset,aes(x=weekdays,y=logrevenue,fill=weekdays)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + ggtitle('Daily Total Revenue by Day of Month') + theme(axis.text.x=element_blank())\n\n# By Day of Week - Faceted\nggplot(dfgoog_subset,aes(x=weekdays,y=logrevenue,fill=weekdays)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + ggtitle('Daily Total Revenue by Day of Week and Month') + facet_wrap(~month) + theme(axis.text.x=element_blank())\n\n# By Day and Month\nggplot(dfgoog_subset,aes(x=day,y=logrevenue,fill=day)) + geom_bar(stat=\"summary\",fun.y=\"sum\") + ggtitle('Daily Total Revenue by Month and Day') + facet_wrap(~ month,ncol = 2,scale=\"free_x\") + theme(axis.text.x=element_blank())\n\n# 2016 time plot\ndfgoog_time = dfgoog_subset %>%\n group_by(year,date_2) %>%\n filter(year == '2016') %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(revenue = volume / 1000000000) %>%\n arrange(date_2)\n\ndfgoog_time$date_2 = ymd(dfgoog_time$date_2)\nggplot(dfgoog_time,aes(x=date_2,y=revenue)) + geom_line() + ggtitle('Time Series Plot of Daily Total Revenue (2016)') + scale_x_date(labels = date_format(\"%b-%Y\")) \n\n# 2017 time plot\ndfgoog_time = dfgoog_subset %>%\n group_by(year,date_2) %>%\n filter(year == '2017') %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(revenue = volume / 1000000000) %>%\n arrange(date_2)\n\nggplot(dfgoog_time,aes(x=date_2,y=revenue)) + geom_line() + ggtitle('Time Series Plot of Daily Total Revenue (2017)') + scale_x_date(labels = date_format(\"%b-%Y\")) \n\ndfgoog_time_hour = dfgoog_subset %>%\n group_by(date_2,hour_visit) %>%\n summarise(volume=sum(totals_transactionRevenue)) %>%\n mutate(revenue = volume / 1000000000) %>%\n arrange(date_2,hour_visit)\n\nts.plot(ts(dfgoog_time_hour$revenue,frequency=1),xlab = \"Time Series (hour of the year [2016 - 2017])\",ylab = \"log revenue\")\n\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3b6d890e45bc7238d8304f46670dfe829ca30772"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"d250179f01307f2ad04663e5741a16a9fbedb934"},"cell_type":"markdown","source":"**19. Seasonality Tests ** \n\n> Adjusted R-squared is very high when log Revenue is regressed to hour, binned minute, and binned seconds, for records with positive revenue only."},{"metadata":{"trusted":true,"_uuid":"eed0a30b6cd50abc320b273b609148c39c565baa","_kg_hide-input":true},"cell_type":"code","source":"\ndfgoog_subset_2 = dfgoog_subset[dfgoog_subset$logrevenue > 0,]\n\nprint ('Regression of Hour of visit to LogRevenue')\nsummary(lm(logrevenue ~ as.factor(hour_visit) - 1,data=dfgoog_subset_2))\n\nprint ('Regression of Minute of visit to LogRevenue - binned')\nsummary(lm(logrevenue ~ as.factor(minbin) - 1,data=dfgoog_subset_2))\nsummary(lm(logrevenue ~ as.factor(minbin_2) - 1,data=dfgoog_subset_2))\n\nprint ('Regression of Second of visit to LogRevenue - binned')\nsummary(lm(logrevenue ~ as.factor(secbin) - 1,data=dfgoog_subset_2))\nsummary(lm(logrevenue ~ as.factor(secbin_2) - 1,data=dfgoog_subset_2))\n\nprint ('Regression of day of week to LogRevenue - binned')\nsummary(lm(logrevenue ~ as.factor(weekdays) - 1,data=dfgoog_subset_2))\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9651391327d1394fe7e6626478ac3956de2b2574"},"cell_type":"markdown","source":""},{"metadata":{"_uuid":"36bca9372e34c6a920ff2433f75636702f866c40"},"cell_type":"markdown","source":"**20. Seasonality Tests Continued - (ARIMA tests with ACF and PACF by Day and Hour) ** \n\n> ARIMA models fitted to daily and hourly time series data, and ACF and PACF plots generated.
\n> This offers some interesting suggestions on ARIMA models to apply as an ensemble to a regression model.\n"},{"metadata":{"trusted":true,"_uuid":"e94bd6700297c1885758e8b41b3484f5a2a9e6c4","_kg_hide-input":true},"cell_type":"code","source":"\ndfgoog_subset_2 = dfgoog_subset[dfgoog_subset$logrevenue > 0,]\ndfgoog_subset_2 = dfgoog_subset$logrevenue\n\n# Time series objects for hour and day\nts_hour = ts(dfgoog_time_hour$revenue,frequency=1)\nts_date = ts(dfgoog_time$revenue,frequency = 7)\n\nprint ('Test of Stationarity, and ACF / PACF plots - for Day Time Series Object')\nadf.test(ts_date)\nacf(ts_date)\npacf(ts_date)\nprint ('Preliminary ARIMA test for Daily Time Series Object')\nauto.arima(ts_date)\n\n\nprint ('Test of Stationarity, and ACF / PACF plots - for Hour Time Series Object')\nadf.test(ts_hour)\nacf(ts_hour)\npacf(ts_hour)\nprint ('Preliminary ARIMA test for Hourly Time Series Object')\nauto.arima(ts_hour)\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5b2e56cde8a91b3149655dc080a60e554bccab2d"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"1e9ee2a98960fd4288f55fdff45a109c298db7ad"},"cell_type":"markdown","source":"**21. Number Of Sessions Per User And Sessions with the most (Log) Revenue**"},{"metadata":{"trusted":true,"_uuid":"863e08f9490e4231789c6922abf98460defc2023","_kg_hide-input":true},"cell_type":"code","source":"dfgoog_subset$visitId = as.factor(dfgoog_subset$visitId)\ndfgoog_subset$fullVisitorId = as.factor(dfgoog_subset$fullVisitorId)\n\nnumber_sessions = as.data.frame(table(dfgoog_subset$fullVisitorId))\nsessions_revenue = dfgoog_subset %>%\n group_by(as.factor(fullVisitorId)) %>%\n summarise(volume= sum(logrevenue)) %>%\n arrange(desc(volume))\n\ncolnames(number_sessions)[1] = 'fullVisitorId'\ncolnames(sessions_revenue)[1] = 'fullVisitorId'\nmerged_dataset = merge(number_sessions,sessions_revenue)\nggplot(merged_dataset,aes(x=Freq,y=volume)) + geom_point() + ggtitle('Number of Sessions versus Total Log Revenue')\nprint ('Correlation between session count and total transaction revenue per user')\ncor(merged_dataset$volume,merged_dataset$Freq)\n\n\n\n","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}