The first thing I did when I saw the Google Analytics Demo Account was posted online was to try to familiarize myself with the environment. I was searching around trying to understand it, and then I clicked the User Explorer tab. I found out that the Client ID for each user actually had the visitID of the first session of the visitor. And I found that if I click on this user, I can map each visit to how much he spent based on the number of pageviews he made in that session. So, I simply exported all of this information, and was changing my submission based on what this user export said! This is how I was able to skyrocket to #1 on LB before the reset, and it spawned the \"Insight or Dodgy?\" discussion that led to this competition's reset.
\n\nFirst we must realize that the test set was going to be graded on December 1 2018 to January 31 2019. Therefore I reserved December 1 2017 to January 31 2018 as my validation dataset, instead of using the standard May 1 2018 to October 15 2018 that was issued as the \"test set\". I did this because I thought this period was holiday/Christmas period, and therefore I thought the behavior of the customers during this period would be significantly different from the behavior of the customers during other periods.
\n\nFirst, I convert all columns with 2 unique values into binary. I then impute some NAs based on what they should logically be. I updated some values, that for anyone who \"bounced\" from the website, their sessionQuality was set to 1 (the lowest possible) and their timeOnSite was set to 1 (the lowest possible).
\n\nFor everyone who didn't bounce, and there was no logical imputation for NA, I imputed with the median. I created new binary columns representing if the column was originally NA or not.
\n\nI did a lot of feature engineering in this competition:
\n\nRecall this is at a visit-based level, but we need to make user-based predictions. Therefore I had to aggregate all these features by taking maxes, mins, sums, means, standard deviations, heads, tails and modes. The aggregations I chose were based on what I thought would be important. I also made some aggregations based on time, e.g. I took the mean of the customer's lifetime and the mean from the last 6 months, because I saw that features like these can improve the model from the previous Home Credit competition.
\n\nI took logs of some aggregated features if I found their distribution was better. This was determined using kernel density plots.
\n\nThere were two parts to my solution:\n1. First, make a model to predict if the buyer will make a purchase in the future\n2. A regression that predicts how much they will spend in the future
\n\nFirst, I tuned my parameters using the validation set. I predicted 0 or 1 did you make a purchase or not. Performance was looking good.
\n\nOnce I was satisfied with my validation, I retrained the model on the entire dataset using 4 StratifiedKFold. This is one of my errors--I should have used TimeSplitFold instead.
\n\nI generated binary OOFs to feed to the regression, and I achieved an AUC of 0.95255 and Log Loss of 0.0020877.
\n\nAdd these predictions to a LightGBM fitted to minimize MSE. This got my mean absolute error to 0.01147 and my root mean square error to 0.320785.
\n\nRun this procedure 4 times using 4 different seeds and average the results. Then you obtain my submission!
\n\nThings I tried that didn't work:
\n\nThings that I know to do for next time:
\n\nThe first thing I did when I saw the Google Analytics Demo Account was posted online was to try to familiarize myself with the environment. I was searching around trying to understand it, and then I clicked the User Explorer tab. I found out that the Client ID for each user actually had the visitID of the first session of the visitor. And I found that if I click on this user, I can map each visit to how much he spent based on the number of pageviews he made in that session. So, I simply exported all of this information, and was changing my submission based on what this user export said! This is how I was able to skyrocket to #1 on LB before the reset, and it spawned the \"Insight or Dodgy?\" discussion that led to this competition's reset.
\n\nFirst we must realize that the test set was going to be graded on December 1 2018 to January 31 2019. Therefore I reserved December 1 2017 to January 31 2018 as my validation dataset, instead of using the standard May 1 2018 to October 15 2018 that was issued as the \"test set\". I did this because I thought this period was holiday/Christmas period, and therefore I thought the behavior of the customers during this period would be significantly different from the behavior of the customers during other periods.
\n\nFirst, I convert all columns with 2 unique values into binary. I then impute some NAs based on what they should logically be. I updated some values, that for anyone who \"bounced\" from the website, their sessionQuality was set to 1 (the lowest possible) and their timeOnSite was set to 1 (the lowest possible).
\n\nFor everyone who didn't bounce, and there was no logical imputation for NA, I imputed with the median. I created new binary columns representing if the column was originally NA or not.
\n\nI did a lot of feature engineering in this competition:
\n\nRecall this is at a visit-based level, but we need to make user-based predictions. Therefore I had to aggregate all these features by taking maxes, mins, sums, means, standard deviations, heads, tails and modes. The aggregations I chose were based on what I thought would be important. I also made some aggregations based on time, e.g. I took the mean of the customer's lifetime and the mean from the last 6 months, because I saw that features like these can improve the model from the previous Home Credit competition.
\n\nI took logs of some aggregated features if I found their distribution was better. This was determined using kernel density plots.
\n\nThere were two parts to my solution:\n1. First, make a model to predict if the buyer will make a purchase in the future\n2. A regression that predicts how much they will spend in the future
\n\nFirst, I tuned my parameters using the validation set. I predicted 0 or 1 did you make a purchase or not. Performance was looking good.
\n\nOnce I was satisfied with my validation, I retrained the model on the entire dataset using 4 StratifiedKFold. This is one of my errors--I should have used TimeSplitFold instead.
\n\nI generated binary OOFs to feed to the regression, and I achieved an AUC of 0.95255 and Log Loss of 0.0020877.
\n\nAdd these predictions to a LightGBM fitted to minimize MSE. This got my mean absolute error to 0.01147 and my root mean square error to 0.320785.
\n\nRun this procedure 4 times using 4 different seeds and average the results. Then you obtain my submission!
\n\nThings I tried that didn't work:
\n\nThings that I know to do for next time:
\n\n