This competition presents an interesting in that it is a regression problem - predicting a certain revenue - but very few of the training observations are non-zero.
\n\nI noticed that this right-skews the prediction distribution of my regression model, presumably as so many values are NaN/zero.
\n\nI was therefore wondering if it would be useful to run a first round classification estimator to identify non-zero customers, then run a regression model trained only on non-zero customers on that subset. It seems to me that this could have a couple of advantages:
\n\nscale_pos_weight parameter that could help highlight the few, non-zero revenue customers.I have not tried this yet, but has anyone tried anything like this before?
", "messages": [ { "id": 389310, "postDate": "2018-09-18T14:22:19.667Z", "content": "This competition presents an interesting in that it is a regression problem - predicting a certain revenue - but very few of the training observations are non-zero.
\n\nI noticed that this right-skews the prediction distribution of my regression model, presumably as so many values are NaN/zero.
\n\nI was therefore wondering if it would be useful to run a first round classification estimator to identify non-zero customers, then run a regression model trained only on non-zero customers on that subset. It seems to me that this could have a couple of advantages:
\n\nscale_pos_weight parameter that could help highlight the few, non-zero revenue customers.I have not tried this yet, but has anyone tried anything like this before?
", "rawMarkdown": "This competition presents an interesting in that it is a regression problem - predicting a certain revenue - but very few of the training observations are non-zero.\n\nI noticed that this right-skews the prediction distribution of my regression model, presumably as so many values are NaN/zero. \n\nI was therefore wondering if it would be useful to run a first round classification estimator to identify non-zero customers, then run a regression model trained only on non-zero customers on that subset. It seems to me that this could have a couple of advantages:\n\n1. Classifiers usually have a `scale_pos_weight` parameter that could help highlight the few, non-zero revenue customers.\n2. The regression model would only be trained on non-zero customers, meaning that **if** the classifier is decent the prediction distribution will more closely match the reality\n\nI have not tried this yet, but has anyone tried anything like this before? ", "votes": 11 }, { "id": 389482, "postDate": "2018-09-18T19:38:00.967Z", "content": "Why zero/non-zero only? Create several models: greater than 10$, greater than 100$, etc.
", "rawMarkdown": "Why zero/non-zero only? Create several models: greater than 10$, greater than 100$, etc.", "votes": 7 }, { "id": 389359, "postDate": "2018-09-18T15:35:08.957Z", "content": "I have, with decent results - it's something i definitely intend to try this time around.
", "rawMarkdown": "I have, with decent results - it's something i definitely intend to try this time around.", "votes": 3 }, { "id": 399174, "postDate": "2018-10-05T10:59:33.030Z", "content": "I'm working with this 2-step approach as well. When I run a simple regression using out-of-the-box features on the subset of training data with purchases I already get a low RMSE (<1.0), so the trick is getting a high quality pre-classification prediction. I'm working on that right now but struggling to get a good enough classification; the overall RMSE is hovering around 2.0. Would be curious as to how much success others are seeing with this approach and how they're improving the classification.
", "rawMarkdown": "I'm working with this 2-step approach as well. When I run a simple regression using out-of-the-box features on the subset of training data with purchases I already get a low RMSE (<1.0), so the trick is getting a high quality pre-classification prediction. I'm working on that right now but struggling to get a good enough classification; the overall RMSE is hovering around 2.0. Would be curious as to how much success others are seeing with this approach and how they're improving the classification." }, { "id": 391410, "postDate": "2018-09-21T18:44:20.540Z", "content": "That’s exactly what I did as the first thing I tried to see how it would perform - better result than I expected. I used very few data transforms and intend to build this approach out with encoded vars and further data cleaning.
", "rawMarkdown": "That’s exactly what I did as the first thing I tried to see how it would perform - better result than I expected. I used very few data transforms and intend to build this approach out with encoded vars and further data cleaning." }, { "id": 399208, "postDate": "2018-10-05T12:11:58.147Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 389482, "author_name": "Alexander Firsov", "author_url": "", "post_date": "2018-09-18T19:38:00.967000", "content": "Why zero/non-zero only? Create several models: greater than 10$, greater than 100$, etc.
", "votes": 7, "replies": [] }, { "id": 389359, "author_name": "Konrad Banachewicz", "author_url": "", "post_date": "2018-09-18T15:35:08.957000", "content": "I have, with decent results - it's something i definitely intend to try this time around.
", "votes": 3, "replies": [] }, { "id": 399174, "author_name": "Dan Cripe", "author_url": "", "post_date": "2018-10-05T10:59:33.030000", "content": "I'm working with this 2-step approach as well. When I run a simple regression using out-of-the-box features on the subset of training data with purchases I already get a low RMSE (<1.0), so the trick is getting a high quality pre-classification prediction. I'm working on that right now but struggling to get a good enough classification; the overall RMSE is hovering around 2.0. Would be curious as to how much success others are seeing with this approach and how they're improving the classification.
", "votes": 0, "replies": [] }, { "id": 391410, "author_name": "DuraQ", "author_url": "", "post_date": "2018-09-21T18:44:20.540000", "content": "That’s exactly what I did as the first thing I tried to see how it would perform - better result than I expected. I used very few data transforms and intend to build this approach out with encoded vars and further data cleaning.
", "votes": 0, "replies": [] }, { "id": 399208, "author_name": "", "author_url": "", "post_date": "2018-10-05T12:11:58.147000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "389310": "This competition presents an interesting in that it is a regression problem - predicting a certain revenue - but very few of the training observations are non-zero.\n\nI noticed that this right-skews the prediction distribution of my regression model, presumably as so many values are NaN/zero. \n\nI was therefore wondering if it would be useful to run a first round classification estimator to identify non-zero customers, then run a regression model trained only on non-zero customers on that subset. It seems to me that this could have a couple of advantages:\n\n1. Classifiers usually have a `scale_pos_weight` parameter that could help highlight the few, non-zero revenue customers.\n2. The regression model would only be trained on non-zero customers, meaning that **if** the classifier is decent the prediction distribution will more closely match the reality\n\nI have not tried this yet, but has anyone tried anything like this before? ", "389482": "Why zero/non-zero only? Create several models: greater than 10$, greater than 100$, etc.", "389359": "I have, with decent results - it's something i definitely intend to try this time around.", "399174": "I'm working with this 2-step approach as well. When I run a simple regression using out-of-the-box features on the subset of training data with purchases I already get a low RMSE (<1.0), so the trick is getting a high quality pre-classification prediction. I'm working on that right now but struggling to get a good enough classification; the overall RMSE is hovering around 2.0. Would be curious as to how much success others are seeing with this approach and how they're improving the classification.", "391410": "That’s exactly what I did as the first thing I tried to see how it would perform - better result than I expected. I used very few data transforms and intend to build this approach out with encoded vars and further data cleaning.", "399208": "" } }