Hi Guys
\n\nConsidering this as a classification problem, I'm just wondering what type of classifier would be suitable.
\n\nTreating each place_id as a class label, it is a hugely multiclass problem with over 100,000 classes.
\n\nThere are over 200 classes which occur only once and around 28,000 classes with less than 100 occurrences.
\n\nYet ignoring these items with occur less frequency would not reduce the significantly reduce the number of classes which is several orders of magnitude larger.
", "messages": [ { "id": "120406", "postDate": "05/18/2016 01:17:23", "content": "Hi Guys
\n\nConsidering this as a classification problem, I'm just wondering what type of classifier would be suitable.
\n\nTreating each place_id as a class label, it is a hugely multiclass problem with over 100,000 classes.
\n\nThere are over 200 classes which occur only once and around 28,000 classes with less than 100 occurrences.
\n\nYet ignoring these items with occur less frequency would not reduce the significantly reduce the number of classes which is several orders of magnitude larger.
", "rawMarkdown": "Hi Guys\r\n\r\nConsidering this as a classification problem, I'm just wondering what type of classifier would be suitable.\r\n\r\nTreating each place_id as a class label, it is a hugely multiclass problem with over 100,000 classes.\r\n\r\n# total # classes: 108390\r\n# classes with freq = 1: 246\r\n# classes with freq < 10: 1201\r\n# classes with freq < 100: 28223\r\n\r\nThere are over 200 classes which occur only once and around 28,000 classes with less than 100 occurrences.\r\n\r\nYet ignoring these items with occur less frequency would not reduce the significantly reduce the number of classes which is several orders of magnitude larger.", "votes": null }, { "id": "120407", "postDate": "05/18/2016 01:37:19", "content": "Given the large amount of data,\ntuning any model with gridsearch over parameters with cross validation would prove to be very computation intensive.
\n\nHowever, with such a large amount of data I wonder if crossvalidation is even needed?
", "rawMarkdown": "Given the large amount of data,\r\ntuning any model with gridsearch over parameters with cross validation would prove to be very computation intensive.\r\n\r\nHowever, with such a large amount of data I wonder if crossvalidation is even needed?", "votes": null }, { "id": "120427", "postDate": "05/18/2016 05:52:12", "content": "Choosing a model is is up to your imagination, for example you could use simple logistic regression that model the probability of pair of place and location, here are the steps:
\n\nA regressor would predict real numbers (e.g 100.391) where as place_ids are categorical (integers such as 100) right? So how do you convert regressed real value output to ids - do you round it up?
\n\nMore importantly how do you get 3 estimates from a regression model?
\n\nThis is why I was thinking more along the lines of using a classification model which can handle a large number of classes such as a random forest.\nMulticlass methods which are based on one-v-rest etc would train impractically large number of models so I would avoid them.
", "rawMarkdown": "A regressor would predict real numbers (e.g 100.391) where as place_ids are categorical (integers such as 100) right? So how do you convert regressed real value output to ids - do you round it up?\r\n\r\nMore importantly how do you get 3 estimates from a regression model?\r\n\r\nThis is why I was thinking more along the lines of using a classification model which can handle a large number of classes such as a random forest.\r\nMulticlass methods which are based on one-v-rest etc would train impractically large number of models so I would avoid them.", "votes": null }, { "id": "120472", "postDate": "05/18/2016 14:40:11", "content": "@dingo, logistic regression does classification so don't have to worry about converting.
\n\n[quote=dingo;120433]
\n\nA regressor would predict real numbers (e.g 100.391) where as place_ids are categorical (integers such as 100) right? So how do you convert regressed real value output to ids - do you round it up?
\n\nMore importantly how do you get 3 estimates from a regression model?
\n\nThis is why I was thinking more along the lines of using a classification model which can handle a large number of classes such as a random forest.\nMulticlass methods which are based on one-v-rest etc would train impractically large number of models so I would avoid them.
\n\n[/quote]
", "rawMarkdown": "dingo, logistic regression does classification so don't have to worry about converting.\r\n\r\n[quote=dingo;120433]\r\n\r\nA regressor would predict real numbers (e.g 100.391) where as place_ids are categorical (integers such as 100) right? So how do you convert regressed real value output to ids - do you round it up?\r\n\r\nMore importantly how do you get 3 estimates from a regression model?\r\n\r\nThis is why I was thinking more along the lines of using a classification model which can handle a large number of classes such as a random forest.\r\nMulticlass methods which are based on one-v-rest etc would train impractically large number of models so I would avoid them.\r\n\r\n[/quote]", "votes": null }, { "id": "120723", "postDate": "05/20/2016 04:50:36", "content": "That's right. Sorry I was thinking of linear regression not logistic.
\n\nMulticlass with logistic regression would be one vs rest type of strategy right? So we are looking at training 108390 odd logistic regression models right?
", "rawMarkdown": "That's right. Sorry I was thinking of linear regression not logistic.\r\n\r\nMulticlass with logistic regression would be one vs rest type of strategy right? So we are looking at training 108390 odd logistic regression models right?", "votes": null }, { "id": "120748", "postDate": "05/20/2016 09:29:23", "content": "@dingo, this may be the way, but in this particular problem we don't actually have correct answers in terms of multiple classes - it's only one, though we have been given an opportunity to answer up to three classes. Thereby, if you are aiming for a logistic regression way, a multinomial logistic regression is what you are looking for.
", "rawMarkdown": "dingo, this may be the way, but in this particular problem we don't actually have correct answers in terms of multiple classes - it's only one, though we have been given an opportunity to answer up to three classes. Thereby, if you are aiming for a logistic regression way, a [multinomial logistic regression][1] is what you are looking for.\r\n\r\n\r\n [1]: https://en.wikipedia.org/wiki/Multinomial_logistic_regression", "votes": null }, { "id": "120822", "postDate": "05/20/2016 19:01:29", "content": "For classification problems, normally it boils down to regression based model (logistic regression, SVM) and tree based model (decision tree) - and essentially these models/classifiers aim toward "drawing" the decision boundary.
\n\nI think it'll be a combination/ensemble of both approaches for this problem - tree based models assume that decision boundary are parallel to axis (think 2D) and regression based models assume that decision boundary is a smooth line.
\n\nHence, for features like x,y coordinates, they probably would be good candidates for tree based models - as we are dealing with a 2D spacial map here, while for feature like time it might make slight more sense to use regression based models.
\n\nmy 2 cents.
", "rawMarkdown": "For classification problems, normally it boils down to regression based model (logistic regression, SVM) and tree based model (decision tree) - and essentially these models/classifiers aim toward \"drawing\" the decision boundary.\r\n\r\nI think it'll be a combination/ensemble of both approaches for this problem - tree based models assume that decision boundary are parallel to axis (think 2D) and regression based models assume that decision boundary is a smooth line.\r\n\r\nHence, for features like x,y coordinates, they probably would be good candidates for tree based models - as we are dealing with a 2D spacial map here, while for feature like time it might make slight more sense to use regression based models.\r\n\r\nmy 2 cents.", "votes": null }, { "id": "120825", "postDate": "05/20/2016 19:18:34", "content": "@jasoos Good points.
\n\nOne challenge here, as pointed out, is that it seems ... time consuming ... to run a model on the full data set. E.g., a tree. I'm somewhat new to those algorithms, so perhaps I'm unaware of software sporting advanced efficiency (I'm using caret and h2o packages in R).
\n\nAs a result, I've been trying to think "outisde of the box", which is easy for me b/c I'm barely inside it to begin with ;)
\n\nAn approach gaining traction in the forums and scripts is to run a model on separate pieces of the full grid. Idea being that many small models are more manageable than a single large model. I've still found this to be computationally intensive, to say the least.
", "rawMarkdown": "jasoos Good points.\r\n\r\nOne challenge here, as pointed out, is that it seems ... time consuming ... to run a model on the full data set. E.g., a tree. I'm somewhat new to those algorithms, so perhaps I'm unaware of software sporting advanced efficiency (I'm using caret and h2o packages in R).\r\n\r\nAs a result, I've been trying to think \"outisde of the box\", which is easy for me b/c I'm barely inside it to begin with ;)\r\n\r\nAn approach gaining traction in the forums and scripts is to run a model on separate pieces of the full grid. Idea being that many small models are more manageable than a single large model. I've still found this to be computationally intensive, to say the least.", "votes": null }, { "id": "120828", "postDate": "05/20/2016 20:02:37", "content": "Thanks @RyanBatt, a lot of guys are using the "xgboost" package, seems that package runs pretty fast - am also learning how to apply this package to this problem, doesn't seem easy...
", "rawMarkdown": "Thanks @RyanBatt, a lot of guys are using the \"xgboost\" package, seems that package runs pretty fast - am also learning how to apply this package to this problem, doesn't seem easy...", "votes": null }, { "id": "120833", "postDate": "05/20/2016 20:15:17", "content": "@jasoos If you're working in R, checkout the caret package ... it's a wrapper for lots of other packages (xgboost included!). I think it has something like 200+ packages/ algorithms, that they've wrangled into the some in/out formats. Makes it easy to tweak the right hyperparameters as you experiment with different implementations.
That said, xgboost wasn't 'fast enough'.
", "rawMarkdown": "jasoos If you're working in R, checkout the `caret` package ... it's a wrapper for lots of other packages (xgboost included!). I think it has something like 200+ packages/ algorithms, that they've wrangled into the some in/out formats. Makes it easy to tweak the right hyperparameters as you experiment with different implementations.\r\n\r\nThat said, xgboost wasn't 'fast enough'.", "votes": null } ], "comments": [ { "id": 120407, "author_name": "srimalj", "author_url": "", "post_date": "05/18/2016 01:37:19", "content": "Given the large amount of data,\ntuning any model with gridsearch over parameters with cross validation would prove to be very computation intensive.
\n\nHowever, with such a large amount of data I wonder if crossvalidation is even needed?
", "votes": null, "replies": [] }, { "id": 120427, "author_name": "halshater", "author_url": "", "post_date": "05/18/2016 05:52:12", "content": "Choosing a model is is up to your imagination, for example you could use simple logistic regression that model the probability of pair of place and location, here are the steps:
\n\nA regressor would predict real numbers (e.g 100.391) where as place_ids are categorical (integers such as 100) right? So how do you convert regressed real value output to ids - do you round it up?
\n\nMore importantly how do you get 3 estimates from a regression model?
\n\nThis is why I was thinking more along the lines of using a classification model which can handle a large number of classes such as a random forest.\nMulticlass methods which are based on one-v-rest etc would train impractically large number of models so I would avoid them.
", "votes": null, "replies": [] }, { "id": 120472, "author_name": "georgeliu", "author_url": "", "post_date": "05/18/2016 14:40:11", "content": "@dingo, logistic regression does classification so don't have to worry about converting.
\n\n[quote=dingo;120433]
\n\nA regressor would predict real numbers (e.g 100.391) where as place_ids are categorical (integers such as 100) right? So how do you convert regressed real value output to ids - do you round it up?
\n\nMore importantly how do you get 3 estimates from a regression model?
\n\nThis is why I was thinking more along the lines of using a classification model which can handle a large number of classes such as a random forest.\nMulticlass methods which are based on one-v-rest etc would train impractically large number of models so I would avoid them.
\n\n[/quote]
", "votes": null, "replies": [] }, { "id": 120723, "author_name": "srimalj", "author_url": "", "post_date": "05/20/2016 04:50:36", "content": "That's right. Sorry I was thinking of linear regression not logistic.
\n\nMulticlass with logistic regression would be one vs rest type of strategy right? So we are looking at training 108390 odd logistic regression models right?
", "votes": null, "replies": [] }, { "id": 120748, "author_name": "pershinmr", "author_url": "", "post_date": "05/20/2016 09:29:23", "content": "@dingo, this may be the way, but in this particular problem we don't actually have correct answers in terms of multiple classes - it's only one, though we have been given an opportunity to answer up to three classes. Thereby, if you are aiming for a logistic regression way, a multinomial logistic regression is what you are looking for.
", "votes": null, "replies": [] }, { "id": 120822, "author_name": "howardxie", "author_url": "", "post_date": "05/20/2016 19:01:29", "content": "For classification problems, normally it boils down to regression based model (logistic regression, SVM) and tree based model (decision tree) - and essentially these models/classifiers aim toward "drawing" the decision boundary.
\n\nI think it'll be a combination/ensemble of both approaches for this problem - tree based models assume that decision boundary are parallel to axis (think 2D) and regression based models assume that decision boundary is a smooth line.
\n\nHence, for features like x,y coordinates, they probably would be good candidates for tree based models - as we are dealing with a 2D spacial map here, while for feature like time it might make slight more sense to use regression based models.
\n\nmy 2 cents.
", "votes": null, "replies": [] }, { "id": 120825, "author_name": "zeraes", "author_url": "", "post_date": "05/20/2016 19:18:34", "content": "@jasoos Good points.
\n\nOne challenge here, as pointed out, is that it seems ... time consuming ... to run a model on the full data set. E.g., a tree. I'm somewhat new to those algorithms, so perhaps I'm unaware of software sporting advanced efficiency (I'm using caret and h2o packages in R).
\n\nAs a result, I've been trying to think "outisde of the box", which is easy for me b/c I'm barely inside it to begin with ;)
\n\nAn approach gaining traction in the forums and scripts is to run a model on separate pieces of the full grid. Idea being that many small models are more manageable than a single large model. I've still found this to be computationally intensive, to say the least.
", "votes": null, "replies": [] }, { "id": 120828, "author_name": "howardxie", "author_url": "", "post_date": "05/20/2016 20:02:37", "content": "Thanks @RyanBatt, a lot of guys are using the "xgboost" package, seems that package runs pretty fast - am also learning how to apply this package to this problem, doesn't seem easy...
", "votes": null, "replies": [] }, { "id": 120833, "author_name": "zeraes", "author_url": "", "post_date": "05/20/2016 20:15:17", "content": "@jasoos If you're working in R, checkout the caret package ... it's a wrapper for lots of other packages (xgboost included!). I think it has something like 200+ packages/ algorithms, that they've wrangled into the some in/out formats. Makes it easy to tweak the right hyperparameters as you experiment with different implementations.
That said, xgboost wasn't 'fast enough'.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "120406": "Hi Guys\r\n\r\nConsidering this as a classification problem, I'm just wondering what type of classifier would be suitable.\r\n\r\nTreating each place_id as a class label, it is a hugely multiclass problem with over 100,000 classes.\r\n\r\n# total # classes: 108390\r\n# classes with freq = 1: 246\r\n# classes with freq < 10: 1201\r\n# classes with freq < 100: 28223\r\n\r\nThere are over 200 classes which occur only once and around 28,000 classes with less than 100 occurrences.\r\n\r\nYet ignoring these items with occur less frequency would not reduce the significantly reduce the number of classes which is several orders of magnitude larger.", "120407": "Given the large amount of data,\r\ntuning any model with gridsearch over parameters with cross validation would prove to be very computation intensive.\r\n\r\nHowever, with such a large amount of data I wonder if crossvalidation is even needed?", "120427": "Choosing a model is is up to your imagination, for example you could use simple logistic regression that model the probability of pair of place and location, here are the steps: \r\n\r\n- extract features about the places (temporal features like activity in days of week, spacial features, etc.) \r\n- extract features about the location (this is a bit harder but you could start with x, y and accuracy)\r\n- train a model on positive examples (location and true place) with negative examples sampled form other places (preferably harder places to guess)\r\n- in prediction phase use a heuristic to generate candidate places the could be assigned to a location (e.g. near places) and use the model to find out which of them has the highest probability.", "120433": "A regressor would predict real numbers (e.g 100.391) where as place_ids are categorical (integers such as 100) right? So how do you convert regressed real value output to ids - do you round it up?\r\n\r\nMore importantly how do you get 3 estimates from a regression model?\r\n\r\nThis is why I was thinking more along the lines of using a classification model which can handle a large number of classes such as a random forest.\r\nMulticlass methods which are based on one-v-rest etc would train impractically large number of models so I would avoid them.", "120472": "dingo, logistic regression does classification so don't have to worry about converting.\r\n\r\n[quote=dingo;120433]\r\n\r\nA regressor would predict real numbers (e.g 100.391) where as place_ids are categorical (integers such as 100) right? So how do you convert regressed real value output to ids - do you round it up?\r\n\r\nMore importantly how do you get 3 estimates from a regression model?\r\n\r\nThis is why I was thinking more along the lines of using a classification model which can handle a large number of classes such as a random forest.\r\nMulticlass methods which are based on one-v-rest etc would train impractically large number of models so I would avoid them.\r\n\r\n[/quote]", "120723": "That's right. Sorry I was thinking of linear regression not logistic.\r\n\r\nMulticlass with logistic regression would be one vs rest type of strategy right? So we are looking at training 108390 odd logistic regression models right?", "120748": "dingo, this may be the way, but in this particular problem we don't actually have correct answers in terms of multiple classes - it's only one, though we have been given an opportunity to answer up to three classes. Thereby, if you are aiming for a logistic regression way, a [multinomial logistic regression][1] is what you are looking for.\r\n\r\n\r\n [1]: https://en.wikipedia.org/wiki/Multinomial_logistic_regression", "120822": "For classification problems, normally it boils down to regression based model (logistic regression, SVM) and tree based model (decision tree) - and essentially these models/classifiers aim toward \"drawing\" the decision boundary.\r\n\r\nI think it'll be a combination/ensemble of both approaches for this problem - tree based models assume that decision boundary are parallel to axis (think 2D) and regression based models assume that decision boundary is a smooth line.\r\n\r\nHence, for features like x,y coordinates, they probably would be good candidates for tree based models - as we are dealing with a 2D spacial map here, while for feature like time it might make slight more sense to use regression based models.\r\n\r\nmy 2 cents.", "120825": "jasoos Good points.\r\n\r\nOne challenge here, as pointed out, is that it seems ... time consuming ... to run a model on the full data set. E.g., a tree. I'm somewhat new to those algorithms, so perhaps I'm unaware of software sporting advanced efficiency (I'm using caret and h2o packages in R).\r\n\r\nAs a result, I've been trying to think \"outisde of the box\", which is easy for me b/c I'm barely inside it to begin with ;)\r\n\r\nAn approach gaining traction in the forums and scripts is to run a model on separate pieces of the full grid. Idea being that many small models are more manageable than a single large model. I've still found this to be computationally intensive, to say the least.", "120828": "Thanks @RyanBatt, a lot of guys are using the \"xgboost\" package, seems that package runs pretty fast - am also learning how to apply this package to this problem, doesn't seem easy...", "120833": "jasoos If you're working in R, checkout the `caret` package ... it's a wrapper for lots of other packages (xgboost included!). I think it has something like 200+ packages/ algorithms, that they've wrangled into the some in/out formats. Makes it easy to tweak the right hyperparameters as you experiment with different implementations.\r\n\r\nThat said, xgboost wasn't 'fast enough'." }, "source": "meta" }