r - Selecting tuning parameters with caret using standard deviation of custom metric -

i'm using caret custom fitting metric, need maximize not metric lower bound of it's confidence interval. i'd maximize mean(metric) - k * stddev(metric). know how manually, there way tell caret automatically select best parameters using function?

yes, can define own selection metric through "summaryfunction" parameter of "traincontrol" object , "metric" parameter of call train(). details on pretty documented in "alternate performance metrics" section on caret's model tuning page: http://caret.r-forge.r-project.org/training.html

i don't think gave enough information write you're looking for, here example using code twoclasssummary function:

> library(caret) > data(titanic) >  > #an example custom function > roc <- function (data, lev = null, model = null) { +   require(proc) +   if (!all(levels(data[, "pred"]) == levels(data[, "obs"])))  +     stop("levels of observed , predicted data not match") +   rocobject <- try(proc:::roc(data$obs, data[, lev[1]]), silent = true) +   rocauc <- if (class(rocobject)[1] == "try-error")  +     na +   else rocobject$auc +   out <- c(rocauc, sensitivity(data[, "pred"], data[, "obs"], lev[1]), specificity(data[, "pred"], data[, "obs"], lev[2])) +   names(out) <- c("roc", "sens", "spec") +   out + } >  > #your train control specs > tc <- traincontrol(method="cv",classprob=true,summaryfunction=roc) > #yoru model selection metric specificed > train(survived~.,data=data.frame(titanic),method="rf",trcontrol=tc,metric="roc") 32 samples  4 predictors  2 classes: 'no', 'yes'   no pre-processing resampling: cross-validation (10 fold)   summary of sample sizes: 28, 29, 30, 30, 28, 28, ...   resampling results across tuning parameters:    mtry  roc    sens  spec  roc sd  sens sd  spec sd   2     0.9    0.2   0.25  0.175   0.35     0.425     4     0.85   0.4   0.6   0.211   0.459    0.459     6     0.875  0.35  0.6   0.212   0.412    0.459    roc used select optimal model using  largest value. final value used model mtry = 2.