Hi. I use caret to find optimal parameters, but sometimes I don't need final model - especially if some other parameter is intentionally small to make the search faster. I guess I may not be alone. I finally got angry at all that wasted time and money, sometimes\r\n the final fitting took longer than all of cross-validation.
\r\nI made a quick fix by modyfing the file train.default.R by adding a very simple and crude if. The original version is 5.15-044
\r\nNow you can disable the creation of a final model by setting trainBest to false in train, example:
\r\nmodel <- train(train, activities, method = "rf", trControl = fitControl, metric = 'Rsquared', tuneGrid = RFGrid, trainBest = FALSE, trace = 0, ntree = 75)
\r\nfinalModel and preProcess in result = NA with this option.
\r\nInstall by "install.packages("caret_modified_by_pawzaw.tar.gz")".
\r\nWithout trainBest set to FALSE caret behaves as original.
", "messages": [ { "id": "15150", "postDate": "10/03/2012 21:24:58", "content": "Hi. I use caret to find optimal parameters, but sometimes I don't need final model - especially if some other parameter is intentionally small to make the search faster. I guess I may not be alone. I finally got angry at all that wasted time and money, sometimes\r\n the final fitting took longer than all of cross-validation.
\r\nI made a quick fix by modyfing the file train.default.R by adding a very simple and crude if. The original version is 5.15-044
\r\nNow you can disable the creation of a final model by setting trainBest to false in train, example:
\r\nmodel <- train(train, activities, method = "rf", trControl = fitControl, metric = 'Rsquared', tuneGrid = RFGrid, trainBest = FALSE, trace = 0, ntree = 75)
\r\nfinalModel and preProcess in result = NA with this option.
\r\nInstall by "install.packages("caret_modified_by_pawzaw.tar.gz")".
\r\nWithout trainBest set to FALSE caret behaves as original.
", "rawMarkdown": "", "votes": null }, { "id": "15251", "postDate": "10/07/2012 17:37:23", "content": "I think Max has done a great job with caret. It's helpful to have a consistent interface on top of a great many models. He's very accepting of patches (even informal ones) if you sent this his way.
\r\nHowever, I think it's more aimed at real-life work instead of competition silliness. We used it for our first few contests and it was a solid asset. When you really start ensembling though, it's not quite there. Max had started working on a followup package,\r\n but I believe his day job got in the way:
\r\nhttp://r-forge.r-project.org/projects/fusebox/
", "rawMarkdown": "", "votes": null }, { "id": "15252", "postDate": "10/07/2012 18:06:15", "content": "Yes, caret is a great package.
", "rawMarkdown": "", "votes": null }, { "id": "15635", "postDate": "10/15/2012 18:35:17", "content": "[quote=pawzaw;15150]
\r\nHi. I use caret to find optimal parameters, but sometimes I don't need final model - especially if some other parameter is intentionally small to make the search faster. I guess I may not be alone. I finally got angry at all that wasted time and money, sometimes\r\n the final fitting took longer than all of cross-validation.
\r\nI made a quick fix by modyfing the file train.default.R by adding a very simple and crude if. The original version is 5.15-044
\r\nNow you can disable the creation of a final model by setting trainBest to false in train, example:
\r\nmodel
\r\nfinalModel and preProcess in result = NA with this option.
\r\nInstall by "install.packages("caret_modified_by_pawzaw.tar.gz")".
\r\nWithout trainBest set to FALSE caret behaves as original.
\r\n[/quote]
\r\n\r\n
It's funny, I've sometimes had the OPPOSITE problem! I want to fit the final model in caret with NO cross validation. It provides a nice standardized interface and "predict" functions.
", "rawMarkdown": "", "votes": null }, { "id": "20025", "postDate": "02/04/2013 18:31:54", "content": "You can also use the "custom" method to do this. You have to manually pass a list to train control with the following parameters:
\r\nparameters = grid of tuning parameters
\r\nmodel = function to fit the model
\r\nprediciton = prediciton function (value for regression, classes for classificaiton)
\r\nprobability = probability prediction function for classification models
\r\nsort = function to sort tuning parameters from least to most complex, for breaking ties
\r\nThe "model" function must have an argument called "last." When that argument is TRUE, caret is fitting the final model, and you can use an "if" statement to return NULL.
\r\nThis requires a little work on your part to manage the model fitting and prediction functions, but allows you to do this without patching caret. You can read more here (towards the end).
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 15251, "author_name": "biasvariance", "author_url": "", "post_date": "10/07/2012 17:37:23", "content": "I think Max has done a great job with caret. It's helpful to have a consistent interface on top of a great many models. He's very accepting of patches (even informal ones) if you sent this his way.
\r\nHowever, I think it's more aimed at real-life work instead of competition silliness. We used it for our first few contests and it was a solid asset. When you really start ensembling though, it's not quite there. Max had started working on a followup package,\r\n but I believe his day job got in the way:
\r\nhttp://r-forge.r-project.org/projects/fusebox/
", "votes": null, "replies": [] }, { "id": 15252, "author_name": "rkirana", "author_url": "", "post_date": "10/07/2012 18:06:15", "content": "Yes, caret is a great package.
", "votes": null, "replies": [] }, { "id": 15635, "author_name": "zachmayer", "author_url": "", "post_date": "10/15/2012 18:35:17", "content": "[quote=pawzaw;15150]
\r\nHi. I use caret to find optimal parameters, but sometimes I don't need final model - especially if some other parameter is intentionally small to make the search faster. I guess I may not be alone. I finally got angry at all that wasted time and money, sometimes\r\n the final fitting took longer than all of cross-validation.
\r\nI made a quick fix by modyfing the file train.default.R by adding a very simple and crude if. The original version is 5.15-044
\r\nNow you can disable the creation of a final model by setting trainBest to false in train, example:
\r\nmodel
\r\nfinalModel and preProcess in result = NA with this option.
\r\nInstall by "install.packages("caret_modified_by_pawzaw.tar.gz")".
\r\nWithout trainBest set to FALSE caret behaves as original.
\r\n[/quote]
\r\n\r\n
It's funny, I've sometimes had the OPPOSITE problem! I want to fit the final model in caret with NO cross validation. It provides a nice standardized interface and "predict" functions.
", "votes": null, "replies": [] }, { "id": 20025, "author_name": "zachmayer", "author_url": "", "post_date": "02/04/2013 18:31:54", "content": "You can also use the "custom" method to do this. You have to manually pass a list to train control with the following parameters:
\r\nparameters = grid of tuning parameters
\r\nmodel = function to fit the model
\r\nprediciton = prediciton function (value for regression, classes for classificaiton)
\r\nprobability = probability prediction function for classification models
\r\nsort = function to sort tuning parameters from least to most complex, for breaking ties
\r\nThe "model" function must have an argument called "last." When that argument is TRUE, caret is fitting the final model, and you can use an "if" statement to return NULL.
\r\nThis requires a little work on your part to manage the model fitting and prediction functions, but allows you to do this without patching caret. You can read more here (towards the end).
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "15150": "", "15251": "", "15252": "", "15635": "", "20025": "" }, "source": "meta" }