We have some time before new train and test become available. One can sit and wait refreshing forum in browser or try to use what we have now to align with new competition objectives.
\n\nIt looks like all visit based models cannot be used in updated competition, while user based ones should fit well. \nBelow are some ideas off the top of my head that can be checked right now.
\n\nValidation strategy. New test time range will be 01 Dec 2018 - 31 Jan 2019. One can check what is the best validation strategy using old train data. There are two options I see: a) train on Aug - 15 Oct and validate on Dec-Jan, or b) train on previous months, skip 1.5 month, predict on next 2 months.
We are not interested in one-time visitors/one-time buyers in test set anymore, as they will not impact metrics (only users from train will be evaluated). So it sounds like a good idea to create two models predicting probability that an user become returning visitor and returning buyer.
Based on #2 - remove all one-time visitors from train and see how this impact the metric
Find out who are returning buyers as it looks like they are in the focus now. Are they corporate guys accessing from specific network domains or individuals loving google products.
We have some time before new train and test become available. One can sit and wait refreshing forum in browser or try to use what we have now to align with new competition objectives.
\n\nIt looks like all visit based models cannot be used in updated competition, while user based ones should fit well. \nBelow are some ideas off the top of my head that can be checked right now.
\n\nValidation strategy. New test time range will be 01 Dec 2018 - 31 Jan 2019. One can check what is the best validation strategy using old train data. There are two options I see: a) train on Aug - 15 Oct and validate on Dec-Jan, or b) train on previous months, skip 1.5 month, predict on next 2 months.
We are not interested in one-time visitors/one-time buyers in test set anymore, as they will not impact metrics (only users from train will be evaluated). So it sounds like a good idea to create two models predicting probability that an user become returning visitor and returning buyer.
Based on #2 - remove all one-time visitors from train and see how this impact the metric
Find out who are returning buyers as it looks like they are in the focus now. Are they corporate guys accessing from specific network domains or individuals loving google products.
Regarding point 1, use of the leak would still provide a slight advantage. The Google Demo Account would provide information in that 1.5 month gap. Kaggle may need or want to provide an updated dataset in the last day or two of the competition to eliminate this advantage.
\n\nMakes sense to consider other strategies and these are all points worth investigating further.
", "rawMarkdown": "Regarding point 1, use of the leak would still provide a slight advantage. The Google Demo Account would provide information in that 1.5 month gap. Kaggle may need or want to provide an updated dataset in the last day or two of the competition to eliminate this advantage.\n\nMakes sense to consider other strategies and these are all points worth investigating further.", "votes": null }, { "id": "405307", "postDate": "10/17/2018 08:01:57", "content": "Good point, Michael! We need to know what Kaggle team thinks about this. @philculliton, @juliaelliott could you please clarify?
", "rawMarkdown": "Good point, Michael! We need to know what Kaggle team thinks about this. @philculliton, @juliaelliott could you please clarify?", "votes": null }, { "id": "405324", "postDate": "10/17/2018 08:40:42", "content": "Wouldn't people using GA demo data have more features to add to dataset of entire training data? As many ones were removed from the competition data, people with access to demo data can bring more features to the their model.
", "rawMarkdown": "Wouldn't people using GA demo data have more features to add to dataset of entire training data? As many ones were removed from the competition data, people with access to demo data can bring more features to the their model.", "votes": null } ], "comments": [ { "id": 405300, "author_name": "datahobbit", "author_url": "", "post_date": "10/17/2018 07:49:34", "content": "Regarding point 1, use of the leak would still provide a slight advantage. The Google Demo Account would provide information in that 1.5 month gap. Kaggle may need or want to provide an updated dataset in the last day or two of the competition to eliminate this advantage.
\n\nMakes sense to consider other strategies and these are all points worth investigating further.
", "votes": null, "replies": [ { "id": 405307, "author_name": "alexfir", "author_url": "", "post_date": "10/17/2018 08:01:57", "content": "Good point, Michael! We need to know what Kaggle team thinks about this. @philculliton, @juliaelliott could you please clarify?
", "votes": null, "replies": [] }, { "id": 405324, "author_name": "tpthegreat", "author_url": "", "post_date": "10/17/2018 08:40:42", "content": "Wouldn't people using GA demo data have more features to add to dataset of entire training data? As many ones were removed from the competition data, people with access to demo data can bring more features to the their model.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "405280": "We have some time before new train and test become available. One can sit and wait refreshing forum in browser or try to use what we have now to align with new competition objectives.\n\nIt looks like all visit based models cannot be used in updated competition, while user based ones should fit well. \nBelow are some ideas off the top of my head that can be checked right now.\n\n1. Validation strategy. New test time range will be 01 Dec 2018 - 31 Jan 2019. One can check what is the best validation strategy using old train data. There are two options I see: a) train on Aug - 15 Oct and validate on Dec-Jan, or b) train on previous months, skip 1.5 month, predict on next 2 months. \n\n2. We are not interested in one-time visitors/one-time buyers in test set anymore, as they will not impact metrics (only users from train will be evaluated). So it sounds like a good idea to create two models predicting probability that an user become returning visitor and returning buyer. \n\n3. Based on #2 - remove all one-time visitors from train and see how this impact the metric\n\n4. Find out who are returning buyers as it looks like they are in the focus now. Are they corporate guys accessing from specific network domains or individuals loving google products.", "405300": "Regarding point 1, use of the leak would still provide a slight advantage. The Google Demo Account would provide information in that 1.5 month gap. Kaggle may need or want to provide an updated dataset in the last day or two of the competition to eliminate this advantage.\n\nMakes sense to consider other strategies and these are all points worth investigating further.", "405307": "Good point, Michael! We need to know what Kaggle team thinks about this. @philculliton, @juliaelliott could you please clarify?", "405324": "Wouldn't people using GA demo data have more features to add to dataset of entire training data? As many ones were removed from the competition data, people with access to demo data can bring more features to the their model." }, "source": "meta" }