Applications of Adaptive Shrinkage in Multiple Statistical Problems

Bayesian shrinkage method provides attractive approaches in large-scale data anal- ysis with advantages in prediction and interpretation. Stephens (2016) presented a flexible Empirical Bayes method, "Adaptive Shrinkage" or ash, to the normal means problem. We explore the idea of adaptive shrinkage in multiple statistics problems, such as sparse factor analysis, sparse logistic factor analysis, estimation of precision matrix, multivariate regression and multiple testing with correlated observation.,In this dissertation, we propose a novel Empirical Bayes Factor Analysis (EBFA) and provide a framework of variational inference algorithm. We can apply any Em- pirical Bayes method, which provides first and second order moments, for normal means problem, and we focus on "Adaptive Shrinkage" method. Both factors and loadings have flexible shrinkage priors in this framework. We extend this EBFA method to logistic factor analysis on binary and binomial distributed data. We show that we can apply the same variational framework in this model by taking a tight lower bound of the likelihood function or augment the model with a Polya-Gamma random variable. We also apply the EBFA method to estimate the precision matrix and conduct a comparison of the methods based on sparse assumption and methods based on low rank assumption. We also introduce a hybrid procedure to combine these two types of methods. We provide a Variational Bayes (VB) algorithm for Bayesian sparse regression with a flexible prior. We also show that this VB approach can solve the normal means problem with correlated observation. Numerical studies on both real data and simulated data illustrate the efficacy of our methods.