Bayesian
Data Analysis
Instructor: David Madigan, Department of
Statistics, madigan@stat.rutgers.edu
Time: Tuesdays and Thursdays, 11:10-12:50 (but may be
changeable)
First class will be Thursday September 5th
Course
Description
This course introduces graduate students to Bayesian
statistical analysis. There are no graduate-level prerequisites, although
students are expected to be familiar with essential features of probability and
statistical inference as usually covered in an intermediate undergraduate
course. A basic premise in Bayesian analysis is that probability measures a
degree of belief in any uncertain event, and thus is personal. The implication
for scientific applications is that all inference proceeds by manipulating
joint probability distributions. The first part of the course studies such
probability distributions and conditional independence concepts in more detail.
Monte Carlo methods arise naturally from these discussions, and we will
consider static and dynamic methods by taking different
graphical summaries of the dependence structure of a
joint distribution. We take a predictive approach to Bayesian analysis,
following de Finetti, and motivate the development of statistical modeling via
exchangeability and the de Finetti theorem. From here, we
study the key components of Bayesian analysis in
one-layer problems, including prior, posterior, and predictive distributions.
We then discuss more advanced modeling and inference problems. The theory and
methods are illustrated with examples from a wide range of current research
topics, and calculations are done in the Splus/R computer language and BUGS, a
free software package for Bayesian data analysis.
Scope of Bayesian Analysis:
accounting for
uncertainty; large parameter spaces; combining
information;
knowledge representation; probability as degree
of belief.
Structure of Joint Distributions:
directed
acyclic graphs; conditional
independence;
example structures, independence, Markov chain, hidden
Markov model;
direct simulation;
undirected
graphs; cliques; Markov random field; Gibbs distribution;
Gibbs sampler;
Hammersley Clifford Theorem.
Statistical Inference I:
de Finetti's
theorem: exchangeability, Polya urn.
one-layer
problems: prior, likelihood, posterior, prior predictive,
posterior
predictive. Bayes rule. Some exponential family examples
including
normal, dirichlet-multinomial.
Priors:
conjugate, non-informative, Jeffreys
Monte Carlo
Methods
Decision Theory (a sampling):
risk, loss,
Bayes risk, admissibility, Bayes estimate.
classification
Stein effect,
shrinkage.
hypothesis
testing: prior/posterior odds; Bayes factor;
connection to
p-values.
Statistical Inference II:
further
examples such as ridge regression, splines, state-space models;
hierarchical
modeling; model checking/selection/averaging; large-sample
theory;
Schwartz criterion; robustness; nonparametric Bayes, as time
permits.