Hi,
\nHow are you guys taking advantage of the decision tree structure of the responses? Has reducing the number of responses by utilizing the dependencies (i.e. Class 10.1 + Class 10.2 + Class 10.3 = Class 4.1) helped your RMSE?
\n\n
Thanks!
", "messages": [ { "id": "38768", "postDate": "02/14/2014 03:38:49", "content": "Hi,
\nHow are you guys taking advantage of the decision tree structure of the responses? Has reducing the number of responses by utilizing the dependencies (i.e. Class 10.1 + Class 10.2 + Class 10.3 = Class 4.1) helped your RMSE?
\n\n
Thanks!
", "rawMarkdown": "", "votes": null }, { "id": "39711", "postDate": "03/02/2014 18:03:15", "content": "Hi,
\nI have not utilized those dependencies yet. I think that one should use those rules to post edit estimates so that one replaces the worst RMSE component using deterministic rule. Say if in your example rule Class 10.3 would have highest RMSE (in training set) then I would l use:
\nClass 10.3 = Class4.1 - Class10.1 - Class 10.2
\nHowever, there is one but. Namely, let's assume that 4 classes are estimated independently (which is perhaps not the optimal case - as they are negatively correlated due to sum rule) but for simplicity. Then prediction variance would be:
VAR[Class 10.3] = VAR[Class 4.1 estimate] + VAR[Class 10.1 estimate] + VAR[Class 10.2 estimate]
Now the question comes that what was the initial MSE of Class 10.3 estimate vs MSE of Class 10.3 "based on rule". Depending which of these has smaller MSE then I would use it. Also estimation of MSE is likely biased which makes things a bit more complicated.
", "rawMarkdown": "", "votes": null }, { "id": "39999", "postDate": "03/07/2014 19:35:46", "content": "Generally speaking, I think that modeling P(labels) in any problem should bring you an additional performance boost, and specifically in this type of problem where the labels are very correlated with each other.
More to the point, if you are using, say, 37 independent regressors, then interpreting the results with a multivariate gaussian distribution or a gaussian mixture model trained on the training labels should improve things, I think.
or, even better, just average the k nearest neighbors of your prediction vector and the 60K available training label vectors...
For me, right now I use these label dependencies by simply predicting the first 5 PCA coefficients of the labels instead of predicting the full 37 labels vector, but this is just an intermediate solution for me, it will probably change a lot as I try new things...
I use a gradient-boost classifier, which supports multi-label classification. The constraints are used as multipliers: for example, class10.1 + class10.2 + class10.1 = class4.1, so I first predict the probability of the galaxy being in class4.1, then predict the probability for each of the subclasses of class 10, and then multiply the result by the class4.1 probability.
\nThe same goes for all other dependencies.
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 39711, "author_name": "sfin2015", "author_url": "", "post_date": "03/02/2014 18:03:15", "content": "Hi,
\nI have not utilized those dependencies yet. I think that one should use those rules to post edit estimates so that one replaces the worst RMSE component using deterministic rule. Say if in your example rule Class 10.3 would have highest RMSE (in training set) then I would l use:
\nClass 10.3 = Class4.1 - Class10.1 - Class 10.2
\nHowever, there is one but. Namely, let's assume that 4 classes are estimated independently (which is perhaps not the optimal case - as they are negatively correlated due to sum rule) but for simplicity. Then prediction variance would be:
VAR[Class 10.3] = VAR[Class 4.1 estimate] + VAR[Class 10.1 estimate] + VAR[Class 10.2 estimate]
Now the question comes that what was the initial MSE of Class 10.3 estimate vs MSE of Class 10.3 "based on rule". Depending which of these has smaller MSE then I would use it. Also estimation of MSE is likely biased which makes things a bit more complicated.
", "votes": null, "replies": [] }, { "id": 39999, "author_name": "selfishgene", "author_url": "", "post_date": "03/07/2014 19:35:46", "content": "Generally speaking, I think that modeling P(labels) in any problem should bring you an additional performance boost, and specifically in this type of problem where the labels are very correlated with each other.
More to the point, if you are using, say, 37 independent regressors, then interpreting the results with a multivariate gaussian distribution or a gaussian mixture model trained on the training labels should improve things, I think.
or, even better, just average the k nearest neighbors of your prediction vector and the 60K available training label vectors...
For me, right now I use these label dependencies by simply predicting the first 5 PCA coefficients of the labels instead of predicting the full 37 labels vector, but this is just an intermediate solution for me, it will probably change a lot as I try new things...
I use a gradient-boost classifier, which supports multi-label classification. The constraints are used as multipliers: for example, class10.1 + class10.2 + class10.1 = class4.1, so I first predict the probability of the galaxy being in class4.1, then predict the probability for each of the subclasses of class 10, and then multiply the result by the class4.1 probability.
\nThe same goes for all other dependencies.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "38768": "", "39711": "", "39999": "", "40096": "" }, "source": "meta" }