The use of data-dependent priors is strangely ubiquitous in Bayesian statistical modeling given the alarm it raises within some cadres. Indeed, the researcher who employs a data-dependent prior faces the inescapable truth that he or she has used the data twice: the first time in establishing prior belief, and the second in updating the prior with the likelihood to obtain the posterior distribution. In this dissertation, we introduce the Adjusted Data-Dependent Bayesian Paradigm as a principled approach to using data-dependent priors in weak accordance with Bayes' theorem.
Suppose that a researcher peeks at the data through some summary statistic prior to applying the Bayesian update. This novel method systematically chooses the posterior distribution that captures the information regarding model parameters that is contained in the data above and beyond the information contained in the observed statistic. In special situations the adjusted approach is formally equivalent to other Bayesian methods, but these cases are necessarily rare. The adjusted procedure imposes a null update when the observed statistic is sufficient, choosing the posterior distribution that is equal to the prior. Conversely, observing a non-sufficient statistic will invite a non-trivial update.
Of particular interest is how analyses under the adjusted and naive (unadjusted) procedures compare. Implementation strategies are described for imposing the adjustment in low and high dimensional settings. Posterior simulation is emphasized using Markov Chain Monte Carlo (MCMC) techniques modified to accommodate the adjusted paradigm. Black box strategies (e.g. preprocessing data) used to fit Dirichlet Process (DP) mixture models are cast as data-dependent and we demonstrate how to apply the adjustment in these settings. Additionally, we compare the predictive power of the naive and adjusted techniques in analyzing the classic galaxies data set popularized in Roeder (1990).