I'm curious if anyone has thought about trying a pseudo-uplift modeling approach for this problem.
The benchmark for this competition looks at customers who have purchased in the same category/brand/company before. To me, it seems that those features (and some of the others mentioned in forum) are capturing customers who are likely to be repeat customers because they are ALREADY repeat customers (i.e. if you buy a specific kind of water 10 times before you get a coupon for it, you're probably going to keep buying it after the coupon).
To BEAT the benchmark, it seems that it would be helpful to find features that predict customers who are likely to become repeat customers as a result of the offer, which is a causal question.
Uplift modeling is used to predict second-order effects, i.e. if I send you an offer, you will buy my product because you got the offer. Usually, you would have test and control groups to implement this modeling approach. But I was wondering if you might be able to steal some of these ideas for this contest using an approach that looks something like this:
1) Group A: Identify customers who have not made repeat purchases of the product that they are being offered. (This is your "test" group.)
2) Group B: Identify other customers who have not made repeat purchases of the product that Group A was offered, but who are not offered that product themselves. (This is your "control" group.)
3) Create an uplift model (see http://stochasticsolutions.com/papers.html) using Group A as your test group and Group B as your control. (You are trying to find features that make Group A much more likely to be repeat purchases than Group B, since they might be roughly homogenous other than the fact Group A got an offer for the product).
4) Apply uplift model scores to test set.
Theoretically, this should give higher weight to features that are likely to be causal, and therefore it might be an answer to the problems stemming from the fact that the test set has different customers in different time periods than the training set.
Any thoughts? Suggestions?
", "messages": [ { "id": "49925", "postDate": "07/01/2014 20:14:31", "content": "I'm curious if anyone has thought about trying a pseudo-uplift modeling approach for this problem.
The benchmark for this competition looks at customers who have purchased in the same category/brand/company before. To me, it seems that those features (and some of the others mentioned in forum) are capturing customers who are likely to be repeat customers because they are ALREADY repeat customers (i.e. if you buy a specific kind of water 10 times before you get a coupon for it, you're probably going to keep buying it after the coupon).
To BEAT the benchmark, it seems that it would be helpful to find features that predict customers who are likely to become repeat customers as a result of the offer, which is a causal question.
Uplift modeling is used to predict second-order effects, i.e. if I send you an offer, you will buy my product because you got the offer. Usually, you would have test and control groups to implement this modeling approach. But I was wondering if you might be able to steal some of these ideas for this contest using an approach that looks something like this:
1) Group A: Identify customers who have not made repeat purchases of the product that they are being offered. (This is your "test" group.)
2) Group B: Identify other customers who have not made repeat purchases of the product that Group A was offered, but who are not offered that product themselves. (This is your "control" group.)
3) Create an uplift model (see http://stochasticsolutions.com/papers.html) using Group A as your test group and Group B as your control. (You are trying to find features that make Group A much more likely to be repeat purchases than Group B, since they might be roughly homogenous other than the fact Group A got an offer for the product).
4) Apply uplift model scores to test set.
Theoretically, this should give higher weight to features that are likely to be causal, and therefore it might be an answer to the problems stemming from the fact that the test set has different customers in different time periods than the training set.
Any thoughts? Suggestions?
", "rawMarkdown": "", "votes": null }, { "id": "50337", "postDate": "07/11/2014 08:50:34", "content": "create features that you think are causal and try classification, I don't see any difference between the two approaches.
", "rawMarkdown": "", "votes": null }, { "id": "50357", "postDate": "07/11/2014 15:48:51", "content": "It's a very different approach. This paper (especially sections 2.5 and 3) explains the difference: http://stochasticsolutions.com/pdf/sig-based-up-trees.pdf. The short answer is you are optimizing a different quantity through uplift modeling than you would with traditional classification / propensity models. Instead of modeling how likely someone is to make a repeat purchase, you model how much more likely they are to purchase if they have received an offer as opposed to not receiving an offer. It requires a different modeling design and optimization process. Implementations are tough to come by though, in that this is still a new/evolving discipline. There is a package in R called "uplift" that R users can use. Not sure about Python or some others.
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 50337, "author_name": "anshul", "author_url": "", "post_date": "07/11/2014 08:50:34", "content": "create features that you think are causal and try classification, I don't see any difference between the two approaches.
", "votes": null, "replies": [] }, { "id": 50357, "author_name": "nsusanj", "author_url": "", "post_date": "07/11/2014 15:48:51", "content": "It's a very different approach. This paper (especially sections 2.5 and 3) explains the difference: http://stochasticsolutions.com/pdf/sig-based-up-trees.pdf. The short answer is you are optimizing a different quantity through uplift modeling than you would with traditional classification / propensity models. Instead of modeling how likely someone is to make a repeat purchase, you model how much more likely they are to purchase if they have received an offer as opposed to not receiving an offer. It requires a different modeling design and optimization process. Implementations are tough to come by though, in that this is still a new/evolving discipline. There is a package in R called "uplift" that R users can use. Not sure about Python or some others.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "49925": "", "50337": "", "50357": "" }, "source": "meta" }