Summary
Objectives:
We illustrate a recently proposed two-step bootstrap model averaging (bootstrap MA)
approach to cope with model selection uncertainty. The predictive performance is investigated
in an example and in a simulation study. Results are compared to those derived from
other model selection methods.
Methods:
In the framework of the linear regression model we use the two-step bootstrap MA,
which consists of a screening step to eliminate covariates thought to have no influence
on the response, and a model-averaging step. We also apply the full model, variable
selection using backward elimination based on Akaike’s Information Criterion (AIC),
the Bayes Information Criterion (BIC) and the bagging approach. The predictive performance
is measured by the mean squared error (MSE) and the coverage of confidence intervals
for the true response.
Results:
We obtained similar results for all approaches in the example. In the simulation
the MSE was reduced by all approaches in comparison to the full model. The smallest
values are obtained for bootstrap MA. Only the bootstrap MA and the full model correctly
estimated the nominal coverage. The backward elimination procedures led to substantial
underestimation and bagging to an overestimation of the true coverage. The screening
step of bootstrap MA eliminates most of the unimportant factors.
Conclusion:
The new bootstrap MA approach shows promising results for predictive performance.
It increases practical usefulness by eliminating unimportant factors in the screening
step.
Keywords
Model uncertainty - bootstrap model averaging - variable selection - prediction
