BAYESIAN DATA ANALYSIS

• Time: Tuesdays and Thursday's 11:30am to 12:50pm
• Location: Hill 552
• Instructor: David Madigan
• Email: madigan@stat.rutgers.edu

12/10/02 Here my R code for Homework 8

12/03/02 Homework 9 posted. Due Friday, December 20

12/03/02 Link to Weakliem's critique of BIC.

11/26/02 I fixed a couple of typos in the marginal likelihood slides to do with the the harmonic mean estimator and BIC (thanks Rong and Aimin)

11/26/02 Here are the HMM Notes that I seem to have forgotten to post before.

11/12/02 Homework 8 posted. Due Thursday November 21

11/4/02 Homework 7 posted. Due Tuesday November 12

10/31/02 Here's my R code for homework 6 (it now includes the requisite Hastings adjustment for the independence sampler - thanks Pai-Hsi for pointing this out)

10/23/02 Homework 6 posted. Due Thursday 10/31.

10/23/02 R code for the Metropolis-Hastings Beta sampler. Play with this and see the effect of different proposal distributions on convergence.

10/22/02 The Gilks and Wild adaptive rejection sampling paper

10/17/02 The "air" example that crashed in class today works just fine with WinBUGS 1.4 beta

10/15/02: WinBUGS code for Chapter 3, Question 8

10/15/02: Homework 5 posted.

10/10/02: Here's the WinBUGS code for the hospital example.

• Course Syllabus

  Here's a brief description of what this course is about.

• Course Text

  We will be using Bayesian Data Analysis, by Gelman, Meng, Stern, and Rubin (Chapman and Hall, 1995), as the course text.  You can get the book at Amazon or other places.

  We will also make use of:

  • B.P. Carlin and T.A. Louis (2000) Bayes and Empirical Bayes Methods for Data Analysis, Chapman and Hall.
  • W.G. Gilks, S. Richardson, and D.J. Spiegelhalter (1995) Markov Chain Monte Carlo in Practice, CRC Press.
  • P. Congdon (2001). Bayesian Statistical Modelling, Wiley.
  • Michael Ross Chernick's list of Bayesian Books

  Here are some more relevant books:

  • R.O. Duda, P.E. Hart, and D.G. Stork (2000) Pattern Classification (2nd edition), Wiley.
  • C.M. Bishop (1996) Neural Networks for Pattern Recognition, Oxford University Press.

• Course Work

  We will have regular homework assignments and possibly some sort of project. Let me know if you have specific ideas.

Homework.

Homework 1
Any two questions from Chapter 2 of Gelman et al.
Due September 17th

Homework 2
Exercises 2 and 8 of Chapter 3 of Gelman et al.
Due September 26th

Homework 3
Q1. Do Question 3 from Chapter 3 of Gelman et al. This code should be useful.
Q2. Question 5 from Chapter 4
Due October 8

Homework 4
Q1. Bayesian learning for acyclic directed probabilistic graphical models. Consider two models for three binary variables A, B, and C. Model 1 is A->-B->-C; Model 2 is A->-B     C. The available data comprise three examples: (1,0,1), (1,1,0), and (1,0,0). Using uniform prior distributions, compute the Bayes factor for Model 1 against Model 2.
Q2. Read this paper by David Heckerman
Due October 15

Homework 5
Bayesian binary regression with a probit model using BUGS.
Q1. Finney (1947) describes a binary regression problem with two continuous valued predictors and a binary response. Here are the data in BUGS-ready format:

list(n=39,x1=c(3.7,3.5,1.25,0.75,0.8,0.7,0.6,1.1,0.9,0.9,0.8,0.55,0.6,1.4,0.75,2.3,3.2,
0.85,1.7,1.8,0.4,0.95,1.35,1.5,1.6,0.6,1.8,0.95,1.9,1.6,2.7,2.35,1.1,1.1,1.2,0.8,
0.95,0.75,1.3),x2=c(0.825,1.09,2.5,1.5,3.2,3.5,0.75,1.7,0.75,0.45,0.57,2.75,3.0,
2.33,3.75,1.64,1.6,1.415,1.06,1.8,2.0,1.36,1.35,1.36,1.78,1.5,1.5,1.9,0.95,0.4,
0.75,0.03,1.83,2.2,2.0,3.33,1.9,1.9,1.625),y=c(1,1,1,1,1,1,0,0,0,0,0,0,0,1,1,
1,1,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,0,1,1,1,0,0,1))

Q2 (optional). Suppose instead of the diffuse normal prior for b_i, i=0,1,2, you use a normal prior with mean zero and variance v_i, and assume the v_i's are independently exponentially distributed with some hyperparameter gamma (i.e., a hierarchical model). Fit this model using BUGS. How different are the posterior distributions from this model? How sensitive are they to the choice of gamma?

Due October 22nd

Homework 6
Do Question 2 from Chapter 11 of Gelman et al. (I expect most people will use R or S-plus for this, but Matlab would be OK too. Please do not use BUGS/WinBUGS for this homework assignment.)
Due October 30

Homework 7
Write a one or two page summary of Tipping's paper on Relevance Vector Machines. Specifically for the classification models of Section 3, are there alternative models that generalize or specialize Tippings' that you would like to explore? (recall, for example, Question 2 of Homework 5)
Due November 12

Homework 8
Implement a sequential Monte Carlo ("particle filter") algorithm for univariate Gaussian data with known variance (see page 44-45 of the Gelman et al. book). For example, try M=100 particles, n=100 and N=1000 with resample-move steps when the ESS drops below 50.
Due November 21

Homework 9
Read this paper about model uncertainty by David Draper. Suppose we want to make inference about the true mean of these 100 iid observations (these are the NB10 measurements that Draper discusses - see his normal quantile-quantile plot, etc.)

Using BUGS, consider two models for these data and examine the posterior variance of the mean under both. The first models the data as a Gaussian. The second models the data using a t-distribution but accounts for the uncertainty in the degrees of freedom (just like in Draper's paper).

You might find the Blocker example in the BUGS Examples Volume I useful (but feel free to deal with the degrees of freedom in other ways if you wish, e.g., a Poisson prior with a hyper-prior on the mean of the Poisson?).

Due December 20

Class Topics.
I will post links to materials we use in the class here.

DATE	TOPICS	LINKS
September 5	Introduction to the Bayesian approach	Mostly based on Sujit Ghosh's notes
September 10	Introduction to the Bayesian approach (cont.)	PPT
September 12	Introduction to the Bayesian approach (cont.)	PPT
September 17	First-cut Bayesian Computation Bayes Factors	PPT Mostly based on Sujit Ghosh's notes
September 19	Exercises from Chapter 2 of Gelman et al. Introduction to Probabilistic Graphical Models	(better) R Code for Q4 (Ming Ouyang) PPT
September 26	Exercises from Chapter 2 and 3 of Gelman et al. Introduction to Probabilistic Graphical Models	R Code for Q8 (version shown in class) PPT
October 1	Introduction to Probabilistic Graphical Models	PPT
October 8	Introduction to Probabilistic Graphical Models	PPT
October 10	Hierarchical Models	PPT
October 15	Monte Carlo Methods	Hard copy handout
October 17	Monte Carlo Methods	-
October 22	Monte Carlo Methods	-
October 24	MCMC diagnostics Adaptive Rejection Sampling and Slice Sampling	Mostly based on Kate Cowles Notes ARS from here and Slice Sampling from here.
October 29	Monte Carlo Methods	-
October 31	Monte Carlo Methods	-
November 5	Bayesian Computation - Analytic Approximation	Mostly based on Sujit Ghosh's notes
November 7	Bayesian Regression	Mostly based on Francesca Dominici's notes
November 12	Sequential Monte Carlo	PPT
November 14	Model checking	PPT
November 19	Professor Hirsh (Guest Lecture)	-
November 21	Model checking	PPT
November 26	Computing Model Probabilities	PPT
December 3	Bayesian Model Averaging	PPT
December 10	Bayesian Nonparametrics	PPT

Other Bayesian Courses

• Francesca Dominici's course at Johns Hopkins
• Joe Ibrahim's Bayesian Biostatistics course at Harvard
• Sujit Ghosh's course at NCSU
• Paula Sebastiani's course at UMass
• Lawrence Joseph's course at McGill
• Michael Newton's course at Wisconsin
• Gary Rosner's course at Rice
• Kate Cowles's course at Iowa

General Pointers to Bayesian and related Web pages

• Yang and Berger's catalog of noninformative priors
• The likelihood principle
• There are R Tutorials here
• The BUGS Project
• R
• Figueiredo's Bayesian Tutorial
• Wasserman's Decision Theory Notes
• Cole's Bayesian Notes
• Kevin Murphy's Bayes net software list.
• Gillian Raab's WinBUGS Course
• Andrew Moore's Statistical Data Mining tutorials.