Finding Best Model in Multiple Regression Applying Reversible Jump MCMC

The purpose of this study was to demonstrate reversible jump MCMC (RJ-MCMC) in multiple regression. The RJ-MCMC determined the best number of predictors given the data. Specifically, in the empirical analysis, the results showed that 12 predictors were selected as the appropriate predictors to explain science achievement among 20 predictors. It seems that the RJ-MCMC prefer to a simpler model compared to the other model selection methods (i.e., AIC, forward selection, backward selection, and stepwise selection). However, BIC suggested the same number of the variables suggested by the RJ-MCMC. To compare the model selection based on BIC and the RJ-MCMC, the simulation study was performed. The general trend for both model selection results is that the accuracy and the precision of model selection improves when the sample size is large or the number of the predictors is small. However, the model selection via the RJ-MCMC shows the better performance than the performance based on BIC when the number of predictors increases. Also, the results might imply that the RJ-MCMC allows to select the variable even the magnitude of the variables is small with parsimoniousness.