My final solution was an ensemble of XGBoost, random forests, k-nearest neighbors, and kernel density estimation. My XGBoost, RF, and KNN methods ended up being highly similar to what has already been discussed in the forums so in this post I will share my KDE method, which scored ~0.604 on the public LB.
\n\nThe idea of this problem is very simple, given a new check-in, can we predict the business of the check-in out of 100000+ businesses? Using, KDE, the goal is to generate a kernel density for every business in the training set using relevant properties. Then, for each observation in the test set, compute the probability of the check-in being at each of the 100000+ businesses and sort the results from highest to lowest.
\n\nProblems faced:
\n\nSolution: break the data set into smaller grids as in the KNN and XGBoost methods
\n\nSolution: estimate a kernel for each property (e.g., x, y, log(accuracy), day of week, hour of day), and then multiply the probabilities. This assumes that the properties are independent (which I believe they are) so you're basically computing Pr(check-in = business | x, y, log(accuracy), day of week, hour of day) = Pr(check-in = business | x)*Pr(check-in = business | y)*Pr(check-in = business | log(accuracy))*Pr(check-in = business | day of week)*Pr(check-in = business | hour of day).
\n\nSolution: I used the kernel density code from sklearn. They offer a number of different kernels. The best way is to start with the Gaussian kernel, and to start with Scott's rule for bandwidth selection (described in the Scipy kde documentation). Then, tweak the values to optimize the kernel and bandwidth with your validation set. I found that the Exponential kernel was slightly better than the Gaussian kernel.
\n\nSolving the three problems above produces a score of ~0.59.
\n\nNext, we know that check-ins do not happen consistently across time. Some businesses have check-ins early in the time period and others later and some happen at seemingly randomly periods (while still following day of week and hour of day trends). One theory to test is that businesses that have check-ins later in the training set are more likely to appear in the test set. This can be modeled in the KDE framework by simply multiplying the final probability by the average date of check-in for the business, so that businesses with later check-ins will have higher probabilities than businesses with earlier check-ins. Doing this improves the score to ~0.595.
\n\nAnother way to do the above is to estimate a KDE for time (e.g., week number), and set all of the test set's week number to be that of the final week in the training set. Doing this instead of the above method scores ~0.594, but an ensemble of both methods (while keeping the other parts the same) improves the score to 0.602 on the public LB.
\n\nFinally, the grid method cuts off boarders at specific locations which limits kernels to be estimated using only the data within the grid. By shifting the grid slightly in both directions (x and y), you end up using a different data set to estimate the kernels. Ensembling the results from the two different grids improves score to 0.604.
\n\nAttached is a not-so-clean Python script that I used. Please let me know if something is unclear or wrong in the script. Hope this was useful/interesting for people!
", "messages": [ { "id": "126440", "postDate": "07/08/2016 13:04:51", "content": "My final solution was an ensemble of XGBoost, random forests, k-nearest neighbors, and kernel density estimation. My XGBoost, RF, and KNN methods ended up being highly similar to what has already been discussed in the forums so in this post I will share my KDE method, which scored ~0.604 on the public LB.
\n\nThe idea of this problem is very simple, given a new check-in, can we predict the business of the check-in out of 100000+ businesses? Using, KDE, the goal is to generate a kernel density for every business in the training set using relevant properties. Then, for each observation in the test set, compute the probability of the check-in being at each of the 100000+ businesses and sort the results from highest to lowest.
\n\nProblems faced:
\n\nSolution: break the data set into smaller grids as in the KNN and XGBoost methods
\n\nSolution: estimate a kernel for each property (e.g., x, y, log(accuracy), day of week, hour of day), and then multiply the probabilities. This assumes that the properties are independent (which I believe they are) so you're basically computing Pr(check-in = business | x, y, log(accuracy), day of week, hour of day) = Pr(check-in = business | x)*Pr(check-in = business | y)*Pr(check-in = business | log(accuracy))*Pr(check-in = business | day of week)*Pr(check-in = business | hour of day).
\n\nSolution: I used the kernel density code from sklearn. They offer a number of different kernels. The best way is to start with the Gaussian kernel, and to start with Scott's rule for bandwidth selection (described in the Scipy kde documentation). Then, tweak the values to optimize the kernel and bandwidth with your validation set. I found that the Exponential kernel was slightly better than the Gaussian kernel.
\n\nSolving the three problems above produces a score of ~0.59.
\n\nNext, we know that check-ins do not happen consistently across time. Some businesses have check-ins early in the time period and others later and some happen at seemingly randomly periods (while still following day of week and hour of day trends). One theory to test is that businesses that have check-ins later in the training set are more likely to appear in the test set. This can be modeled in the KDE framework by simply multiplying the final probability by the average date of check-in for the business, so that businesses with later check-ins will have higher probabilities than businesses with earlier check-ins. Doing this improves the score to ~0.595.
\n\nAnother way to do the above is to estimate a KDE for time (e.g., week number), and set all of the test set's week number to be that of the final week in the training set. Doing this instead of the above method scores ~0.594, but an ensemble of both methods (while keeping the other parts the same) improves the score to 0.602 on the public LB.
\n\nFinally, the grid method cuts off boarders at specific locations which limits kernels to be estimated using only the data within the grid. By shifting the grid slightly in both directions (x and y), you end up using a different data set to estimate the kernels. Ensembling the results from the two different grids improves score to 0.604.
\n\nAttached is a not-so-clean Python script that I used. Please let me know if something is unclear or wrong in the script. Hope this was useful/interesting for people!
", "rawMarkdown": "My final solution was an ensemble of XGBoost, random forests, k-nearest neighbors, and kernel density estimation. My XGBoost, RF, and KNN methods ended up being highly similar to what has already been discussed in the forums so in this post I will share my KDE method, which scored ~0.604 on the public LB.\r\n\r\nThe idea of this problem is very simple, given a new check-in, can we predict the business of the check-in out of 100000+ businesses? Using, KDE, the goal is to generate a kernel density for every business in the training set using relevant properties. Then, for each observation in the test set, compute the probability of the check-in being at each of the 100000+ businesses and sort the results from highest to lowest. \r\n\r\n**Problems faced:**\r\n\r\n1. 100000+ business is too many. \r\n\r\nSolution: break the data set into smaller grids as in the KNN and XGBoost methods\r\n\r\n2. How to incorporate the various properties of the data?\r\n\r\nSolution: estimate a kernel for each property (e.g., x, y, log(accuracy), day of week, hour of day), and then multiply the probabilities. This assumes that the properties are independent (which I believe they are) so you're basically computing Pr(check-in = business | x, y, log(accuracy), day of week, hour of day) = Pr(check-in = business | x)*Pr(check-in = business | y)*Pr(check-in = business | log(accuracy))*Pr(check-in = business | day of week)*Pr(check-in = business | hour of day).\r\n\r\n3. How to select the bandwidth/kernel type?\r\n\r\nSolution: I used the kernel density code from sklearn. They offer a number of different kernels. The best way is to start with the Gaussian kernel, and to start with Scott's rule for bandwidth selection (described in the Scipy kde documentation). Then, tweak the values to optimize the kernel and bandwidth with your validation set. I found that the Exponential kernel was slightly better than the Gaussian kernel.\r\n\r\n**Solving the three problems above produces a score of ~0.59.**\r\n\r\nNext, we know that check-ins do not happen consistently across time. Some businesses have check-ins early in the time period and others later and some happen at seemingly randomly periods (while still following day of week and hour of day trends). One theory to test is that businesses that have check-ins later in the training set are more likely to appear in the test set. This can be modeled in the KDE framework by simply multiplying the final probability by the average date of check-in for the business, so that businesses with later check-ins will have higher probabilities than businesses with earlier check-ins. Doing this improves the score to ~0.595. \r\n\r\nAnother way to do the above is to estimate a KDE for time (e.g., week number), and set all of the test set's week number to be that of the final week in the training set. Doing this instead of the above method scores ~0.594, but an ensemble of both methods (while keeping the other parts the same) improves the score to 0.602 on the public LB.\r\n\r\nFinally, the grid method cuts off boarders at specific locations which limits kernels to be estimated using only the data within the grid. By shifting the grid slightly in both directions (x and y), you end up using a different data set to estimate the kernels. Ensembling the results from the two different grids improves score to 0.604.\r\n\r\nAttached is a not-so-clean Python script that I used. Please let me know if something is unclear or wrong in the script. Hope this was useful/interesting for people!", "votes": null }, { "id": "126444", "postDate": "07/08/2016 13:37:21", "content": "@Qingchen, thank you very much for your idea of KDE. It is always good experience for me to read your post.
", "rawMarkdown": "Qingchen, thank you very much for your idea of KDE. It is always good experience for me to read your post.", "votes": null }, { "id": "126462", "postDate": "07/08/2016 15:54:53", "content": "How long did this method take to train?
", "rawMarkdown": "How long did this method take to train?", "votes": null }, { "id": "126474", "postDate": "07/08/2016 17:57:15", "content": "@Qinchen, thanks very much for this write up! By how much x,y did you shift the grid. Does "slightly" mean something like 0.04 (x), 0.01 (y) or are you doing it by multiples of the grid size?
", "rawMarkdown": "Qinchen, thanks very much for this write up! By how much x,y did you shift the grid. Does \"slightly\" mean something like 0.04 (x), 0.01 (y) or are you doing it by multiples of the grid size?", "votes": null }, { "id": "126483", "postDate": "07/08/2016 19:56:46", "content": "@Jared: the exponential kernel took significantly longer to train (~14 hours) than the Gaussian kernel (~4 hours), all on my i7 Macbook Pro.
\n\n@Larry: my grids were 50 by 50 so 0.2 x and y per grid. I shifted by 0.1x and 0.1y together.
", "rawMarkdown": "Jared: the exponential kernel took significantly longer to train (~14 hours) than the Gaussian kernel (~4 hours), all on my i7 Macbook Pro.\r\n\r\n@Larry: my grids were 50 by 50 so 0.2 x and y per grid. I shifted by 0.1x and 0.1y together.", "votes": null }, { "id": "126488", "postDate": "07/08/2016 20:27:08", "content": "Thanks for the idea sharing. I also tried some KDE estimation. In my case I create one KDE on x, y log_accuracy and other time features. The result turns to be pretty bad. In your case, you have independent KDE for all the features. I am guess that is why your KDE model is much better?
\n\nAnother maybe dummy question please, how do you ensemble the KDE with other models? Would the probability given from KDE be in the different scale with models like XGBoost and RF?
\n\nThank you again.
", "rawMarkdown": "Thanks for the idea sharing. I also tried some KDE estimation. In my case I create one KDE on x, y log_accuracy and other time features. The result turns to be pretty bad. In your case, you have independent KDE for all the features. I am guess that is why your KDE model is much better?\r\n\r\nAnother maybe dummy question please, how do you ensemble the KDE with other models? Would the probability given from KDE be in the different scale with models like XGBoost and RF?\r\n\r\nThank you again.", "votes": null }, { "id": "126496", "postDate": "07/08/2016 21:14:02", "content": "Yes, using independent KDE is indeed better than a joint kernel. You can kind of infer this from the KNN parameters because manhattan distance ended up being better than Euclidean distance. Of course, these things are quickly testable so always do that to make sure you're doing it right.
\n\nRegarding ensembling, I normalize the likelihood scores to sum to one so that all of the methods' results are on the same scale.
", "rawMarkdown": "Yes, using independent KDE is indeed better than a joint kernel. You can kind of infer this from the KNN parameters because manhattan distance ended up being better than Euclidean distance. Of course, these things are quickly testable so always do that to make sure you're doing it right. \r\n\r\nRegarding ensembling, I normalize the likelihood scores to sum to one so that all of the methods' results are on the same scale.", "votes": null }, { "id": "126508", "postDate": "07/09/2016 00:44:52", "content": "@Qingchen, congratulations, thank you so much for sharing, first competition I've met with KDE and KNN solutions :)
", "rawMarkdown": "Qingchen, congratulations, thank you so much for sharing, first competition I've met with KDE and KNN solutions :)", "votes": null }, { "id": "126577", "postDate": "07/10/2016 03:41:54", "content": "@Qingchen is it possible for you to share the script? I was trying to implement your method about KDE, but I got stuck and I got nowhere to go. It would be very helpful for someone like me who want to learn from you about KDE.
\n\nThanks :) \nCongrats!
", "rawMarkdown": "Qingchen is it possible for you to share the script? I was trying to implement your method about KDE, but I got stuck and I got nowhere to go. It would be very helpful for someone like me who want to learn from you about KDE.\r\n\r\nThanks :) \r\nCongrats!", "votes": null }, { "id": "126589", "postDate": "07/10/2016 09:48:27", "content": "@ArthurLu: I uploaded a script to my first post. Please let me know if something is wrong or if you don't understand something from it. Hope it's helpful.
", "rawMarkdown": "ArthurLu: I uploaded a script to my first post. Please let me know if something is wrong or if you don't understand something from it. Hope it's helpful.", "votes": null }, { "id": "126597", "postDate": "07/10/2016 13:06:45", "content": "@Qingchen Thanks for your sharing! :) Really appreciate it
", "rawMarkdown": "Qingchen Thanks for your sharing! :) Really appreciate it", "votes": null }, { "id": "126600", "postDate": "07/10/2016 15:20:26", "content": "scores[:, i] = (scores_x[:, i]*scores_y[:, i]*scores_acc[:, i]*scores_dow[:, i]*scores_hod[:, i])*np.power(place_num[places[i]], 0.7)*np.power(place_dates[places[i]], 1.2)\n@Quingchen, the above code is at line 190, I assume you are multiplying all marginal probabilities, but could you explain why multiplying np.power(place_num[places[i]], 0.7) and np.power(place_dates[places[i]], 1.2)?\nAnd where 0.7 and 1.2 comes from?
Thanks!
", "rawMarkdown": "scores[:, i] = (scores_x[:, i]*scores_y[:, i]*scores_acc[:, i]*scores_dow[:, i]*scores_hod[:, i])*np.power(place_num[places[i]], 0.7)*np.power(place_dates[places[i]], 1.2)\r\n\r\n@Quingchen, the above code is at line 190, I assume you are multiplying all marginal probabilities, but could you explain why multiplying `np.power(place_num[places[i]], 0.7)` and `np.power(place_dates[places[i]], 1.2)`?\r\nAnd where 0.7 and 1.2 comes from?\r\n\r\nThanks!", "votes": null }, { "id": "126610", "postDate": "07/10/2016 18:49:19", "content": "@AuthurLu: That part is simply something to tune which can be tested with a validation set. Taking the value to a certain exponent puts greater or less weight on the distribution of that feature. Let's say that all else equal, one place with average check-in date of 200 will be weighted half as much as the place with check-in date of 400. However, if I take the exponent of 0.5 for this feature, then the place with check-in date of 400 will be weighted 0.7 times as much, so this feature becomes less discriminative.
", "rawMarkdown": "AuthurLu: That part is simply something to tune which can be tested with a validation set. Taking the value to a certain exponent puts greater or less weight on the distribution of that feature. Let's say that all else equal, one place with average check-in date of 200 will be weighted half as much as the place with check-in date of 400. However, if I take the exponent of 0.5 for this feature, then the place with check-in date of 400 will be weighted 0.7 times as much, so this feature becomes less discriminative.", "votes": null }, { "id": "126735", "postDate": "07/12/2016 02:27:14", "content": "@Quinchen Thanks for explaining details of the pro tip. :)
", "rawMarkdown": "Quinchen Thanks for explaining details of the pro tip. :)", "votes": null } ], "comments": [ { "id": 126444, "author_name": "aikinogard", "author_url": "", "post_date": "07/08/2016 13:37:21", "content": "@Qingchen, thank you very much for your idea of KDE. It is always good experience for me to read your post.
", "votes": null, "replies": [] }, { "id": 126462, "author_name": "jturkewitz", "author_url": "", "post_date": "07/08/2016 15:54:53", "content": "How long did this method take to train?
", "votes": null, "replies": [] }, { "id": 126474, "author_name": "larryfreeman", "author_url": "", "post_date": "07/08/2016 17:57:15", "content": "@Qinchen, thanks very much for this write up! By how much x,y did you shift the grid. Does "slightly" mean something like 0.04 (x), 0.01 (y) or are you doing it by multiples of the grid size?
", "votes": null, "replies": [] }, { "id": 126483, "author_name": "qwang88", "author_url": "", "post_date": "07/08/2016 19:56:46", "content": "@Jared: the exponential kernel took significantly longer to train (~14 hours) than the Gaussian kernel (~4 hours), all on my i7 Macbook Pro.
\n\n@Larry: my grids were 50 by 50 so 0.2 x and y per grid. I shifted by 0.1x and 0.1y together.
", "votes": null, "replies": [] }, { "id": 126488, "author_name": "chengcen", "author_url": "", "post_date": "07/08/2016 20:27:08", "content": "Thanks for the idea sharing. I also tried some KDE estimation. In my case I create one KDE on x, y log_accuracy and other time features. The result turns to be pretty bad. In your case, you have independent KDE for all the features. I am guess that is why your KDE model is much better?
\n\nAnother maybe dummy question please, how do you ensemble the KDE with other models? Would the probability given from KDE be in the different scale with models like XGBoost and RF?
\n\nThank you again.
", "votes": null, "replies": [] }, { "id": 126496, "author_name": "qwang88", "author_url": "", "post_date": "07/08/2016 21:14:02", "content": "Yes, using independent KDE is indeed better than a joint kernel. You can kind of infer this from the KNN parameters because manhattan distance ended up being better than Euclidean distance. Of course, these things are quickly testable so always do that to make sure you're doing it right.
\n\nRegarding ensembling, I normalize the likelihood scores to sum to one so that all of the methods' results are on the same scale.
", "votes": null, "replies": [] }, { "id": 126508, "author_name": "aldente", "author_url": "", "post_date": "07/09/2016 00:44:52", "content": "@Qingchen, congratulations, thank you so much for sharing, first competition I've met with KDE and KNN solutions :)
", "votes": null, "replies": [] }, { "id": 126577, "author_name": "arthurlu", "author_url": "", "post_date": "07/10/2016 03:41:54", "content": "@Qingchen is it possible for you to share the script? I was trying to implement your method about KDE, but I got stuck and I got nowhere to go. It would be very helpful for someone like me who want to learn from you about KDE.
\n\nThanks :) \nCongrats!
", "votes": null, "replies": [] }, { "id": 126589, "author_name": "qwang88", "author_url": "", "post_date": "07/10/2016 09:48:27", "content": "@ArthurLu: I uploaded a script to my first post. Please let me know if something is wrong or if you don't understand something from it. Hope it's helpful.
", "votes": null, "replies": [] }, { "id": 126597, "author_name": "arthurlu", "author_url": "", "post_date": "07/10/2016 13:06:45", "content": "@Qingchen Thanks for your sharing! :) Really appreciate it
", "votes": null, "replies": [] }, { "id": 126600, "author_name": "arthurlu", "author_url": "", "post_date": "07/10/2016 15:20:26", "content": "scores[:, i] = (scores_x[:, i]*scores_y[:, i]*scores_acc[:, i]*scores_dow[:, i]*scores_hod[:, i])*np.power(place_num[places[i]], 0.7)*np.power(place_dates[places[i]], 1.2)\n@Quingchen, the above code is at line 190, I assume you are multiplying all marginal probabilities, but could you explain why multiplying np.power(place_num[places[i]], 0.7) and np.power(place_dates[places[i]], 1.2)?\nAnd where 0.7 and 1.2 comes from?
Thanks!
", "votes": null, "replies": [] }, { "id": 126610, "author_name": "qwang88", "author_url": "", "post_date": "07/10/2016 18:49:19", "content": "@AuthurLu: That part is simply something to tune which can be tested with a validation set. Taking the value to a certain exponent puts greater or less weight on the distribution of that feature. Let's say that all else equal, one place with average check-in date of 200 will be weighted half as much as the place with check-in date of 400. However, if I take the exponent of 0.5 for this feature, then the place with check-in date of 400 will be weighted 0.7 times as much, so this feature becomes less discriminative.
", "votes": null, "replies": [] }, { "id": 126735, "author_name": "arthurlu", "author_url": "", "post_date": "07/12/2016 02:27:14", "content": "@Quinchen Thanks for explaining details of the pro tip. :)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "126440": "My final solution was an ensemble of XGBoost, random forests, k-nearest neighbors, and kernel density estimation. My XGBoost, RF, and KNN methods ended up being highly similar to what has already been discussed in the forums so in this post I will share my KDE method, which scored ~0.604 on the public LB.\r\n\r\nThe idea of this problem is very simple, given a new check-in, can we predict the business of the check-in out of 100000+ businesses? Using, KDE, the goal is to generate a kernel density for every business in the training set using relevant properties. Then, for each observation in the test set, compute the probability of the check-in being at each of the 100000+ businesses and sort the results from highest to lowest. \r\n\r\n**Problems faced:**\r\n\r\n1. 100000+ business is too many. \r\n\r\nSolution: break the data set into smaller grids as in the KNN and XGBoost methods\r\n\r\n2. How to incorporate the various properties of the data?\r\n\r\nSolution: estimate a kernel for each property (e.g., x, y, log(accuracy), day of week, hour of day), and then multiply the probabilities. This assumes that the properties are independent (which I believe they are) so you're basically computing Pr(check-in = business | x, y, log(accuracy), day of week, hour of day) = Pr(check-in = business | x)*Pr(check-in = business | y)*Pr(check-in = business | log(accuracy))*Pr(check-in = business | day of week)*Pr(check-in = business | hour of day).\r\n\r\n3. How to select the bandwidth/kernel type?\r\n\r\nSolution: I used the kernel density code from sklearn. They offer a number of different kernels. The best way is to start with the Gaussian kernel, and to start with Scott's rule for bandwidth selection (described in the Scipy kde documentation). Then, tweak the values to optimize the kernel and bandwidth with your validation set. I found that the Exponential kernel was slightly better than the Gaussian kernel.\r\n\r\n**Solving the three problems above produces a score of ~0.59.**\r\n\r\nNext, we know that check-ins do not happen consistently across time. Some businesses have check-ins early in the time period and others later and some happen at seemingly randomly periods (while still following day of week and hour of day trends). One theory to test is that businesses that have check-ins later in the training set are more likely to appear in the test set. This can be modeled in the KDE framework by simply multiplying the final probability by the average date of check-in for the business, so that businesses with later check-ins will have higher probabilities than businesses with earlier check-ins. Doing this improves the score to ~0.595. \r\n\r\nAnother way to do the above is to estimate a KDE for time (e.g., week number), and set all of the test set's week number to be that of the final week in the training set. Doing this instead of the above method scores ~0.594, but an ensemble of both methods (while keeping the other parts the same) improves the score to 0.602 on the public LB.\r\n\r\nFinally, the grid method cuts off boarders at specific locations which limits kernels to be estimated using only the data within the grid. By shifting the grid slightly in both directions (x and y), you end up using a different data set to estimate the kernels. Ensembling the results from the two different grids improves score to 0.604.\r\n\r\nAttached is a not-so-clean Python script that I used. Please let me know if something is unclear or wrong in the script. Hope this was useful/interesting for people!", "126444": "Qingchen, thank you very much for your idea of KDE. It is always good experience for me to read your post.", "126462": "How long did this method take to train?", "126474": "Qinchen, thanks very much for this write up! By how much x,y did you shift the grid. Does \"slightly\" mean something like 0.04 (x), 0.01 (y) or are you doing it by multiples of the grid size?", "126483": "Jared: the exponential kernel took significantly longer to train (~14 hours) than the Gaussian kernel (~4 hours), all on my i7 Macbook Pro.\r\n\r\n@Larry: my grids were 50 by 50 so 0.2 x and y per grid. I shifted by 0.1x and 0.1y together.", "126488": "Thanks for the idea sharing. I also tried some KDE estimation. In my case I create one KDE on x, y log_accuracy and other time features. The result turns to be pretty bad. In your case, you have independent KDE for all the features. I am guess that is why your KDE model is much better?\r\n\r\nAnother maybe dummy question please, how do you ensemble the KDE with other models? Would the probability given from KDE be in the different scale with models like XGBoost and RF?\r\n\r\nThank you again.", "126496": "Yes, using independent KDE is indeed better than a joint kernel. You can kind of infer this from the KNN parameters because manhattan distance ended up being better than Euclidean distance. Of course, these things are quickly testable so always do that to make sure you're doing it right. \r\n\r\nRegarding ensembling, I normalize the likelihood scores to sum to one so that all of the methods' results are on the same scale.", "126508": "Qingchen, congratulations, thank you so much for sharing, first competition I've met with KDE and KNN solutions :)", "126577": "Qingchen is it possible for you to share the script? I was trying to implement your method about KDE, but I got stuck and I got nowhere to go. It would be very helpful for someone like me who want to learn from you about KDE.\r\n\r\nThanks :) \r\nCongrats!", "126589": "ArthurLu: I uploaded a script to my first post. Please let me know if something is wrong or if you don't understand something from it. Hope it's helpful.", "126597": "Qingchen Thanks for your sharing! :) Really appreciate it", "126600": "scores[:, i] = (scores_x[:, i]*scores_y[:, i]*scores_acc[:, i]*scores_dow[:, i]*scores_hod[:, i])*np.power(place_num[places[i]], 0.7)*np.power(place_dates[places[i]], 1.2)\r\n\r\n@Quingchen, the above code is at line 190, I assume you are multiplying all marginal probabilities, but could you explain why multiplying `np.power(place_num[places[i]], 0.7)` and `np.power(place_dates[places[i]], 1.2)`?\r\nAnd where 0.7 and 1.2 comes from?\r\n\r\nThanks!", "126610": "AuthurLu: That part is simply something to tune which can be tested with a validation set. Taking the value to a certain exponent puts greater or less weight on the distribution of that feature. Let's say that all else equal, one place with average check-in date of 200 will be weighted half as much as the place with check-in date of 400. However, if I take the exponent of 0.5 for this feature, then the place with check-in date of 400 will be weighted 0.7 times as much, so this feature becomes less discriminative.", "126735": "Quinchen Thanks for explaining details of the pro tip. :)" }, "source": "meta" }