BAYESIAN DATA ANALYSIS

Spring 2004

NEWS

5/10/04 Thanks for a fun class!

Homework.

Homework 1
Any two questions from Chapter 2 of Gelman et al.
Due February 6th

Homework 2
Any two questions from Chapter 3 of Gelman et al.
Due February 13th

Homework 3
Consider a univariate normal model with mean mu and variance tau. Suppose I use a Beta(2,2) prior for mu (somehow I know mu is between zero and one) and a log-normal(1,10) prior for tau (recall that if a random variable X is log-normal(m,v) then log X is N(m,v) - the textbook has an expression for the log-normal density). I assume a priori that mu and tau are independent. Use a grid-based approximation to generate a random sample from the joint posterior distribution of mu and tau. Provide a scatterplot of the random sample.

Here are the data: 

2.3656491  2.4952035  1.0837817  0.7586751  0.8780483  1.2765341
1.4598699  0.1801679 -1.0093589  1.4870201 -0.1193149  0.2578262

Note, this is a lot like the bioassay example we did in class.
Due February 20th

Homework 4
Same as Homework 3, but devise and implement a Metropolis-Hastings algorithm to estimate the posterior probability that mu is bigger than 0.5. Do not use BUGS/WinBUGS for this assignment, but feel free to use anything else.
Due March 5th

Homework 5
Any two questions from Chapter 5 of Gelman et al.
Due March 12th

Homework 6
Write a short (one or two pages) summary of this paper by David Heckerman.
Due March 26th

Homework 7
Same as Homework 3 but this time use WinBUGS. Use the "Correlation" tool on the inference menu to draw a scatterplot of the MCMC output for mu and tau.
Due April 2nd

Homework 8
Bayesian binary regression with a probit model using BUGS.
Q1. Finney (1947) describes a binary regression problem with two continuous valued predictors and a binary response. Here are the data in BUGS-ready format: 

list(n=39,x1=c(3.7,3.5,1.25,0.75,0.8,0.7,0.6,1.1,0.9,0.9,0.8,0.55,0.6,1.4,0.75,2.3,3.2,
0.85,1.7,1.8,0.4,0.95,1.35,1.5,1.6,0.6,1.8,0.95,1.9,1.6,2.7,2.35,1.1,1.1,1.2,0.8,
0.95,0.75,1.3),x2=c(0.825,1.09,2.5,1.5,3.2,3.5,0.75,1.7,0.75,0.45,0.57,2.75,3.0,
2.33,3.75,1.64,1.6,1.415,1.06,1.8,2.0,1.36,1.35,1.36,1.78,1.5,1.5,1.9,0.95,0.4,
0.75,0.03,1.83,2.2,2.0,3.33,1.9,1.9,1.625),y=c(1,1,1,1,1,1,0,0,0,0,0,0,0,1,1,
1,1,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,0,1,1,1,0,0,1))
The objective here is to build a predictive model that predicts y using x1 and x2. One approach is the so called probit model: Pr(y=1|x1,x2) = g(b0 + b1*x1 + b2*x2) where g is the standard normal cumulative distribution function. Use BUGS to compute posterior distributions for b0, b1, and b2 using diffuse normal priors for each. Please provide your BUGS code as well as the posterior distributions.

Q2 (optional). Suppose instead of the diffuse normal prior for bi, i=0,1,2, you use a normal prior with mean zero and variance vi, and assume the vi's are independently exponentially distributed with some hyperparameter gamma (i.e., a hierarchical model). Fit this model using BUGS. How different are the posterior distributions from this model? How sensitive are they to the choice of gamma?
Due April 9th

Homework 9
Implement a sequential Monte Carlo ("particle filter") algorithm for univariate Gaussian data with known variance (see Section 2.6 of the Gelman et al. book). For example, try M=100 particles, n=100 and N=1000 with resample-move steps when the ESS drops below 50.
Due April 16th

Homework 10
Consider n observations y1, y2, ..., yn from a univariate normal model with known variance. Assume a Gaussian prior for the mean.
(a) derive an algebraic expresion for the marginal likelihood of the data
(b) simulate some data and compute the marginal likelihood
(c) compute the marginal likelihood via Monte Carlo by sampling from the prior
(d) compute the marginal likelihood via Monte Carlo by sampling from the posterior (i.e., the harmonic mean estimator)
(e) for (c) and (d) do the simulations a few times and compare your results with the exact answer from (b)
Due April 23rd

Class Topics.
I will post links to materials we use in the class here.

DATE TOPICS LINKS
January 28th Introduction to the Bayesian approach Mostly based on Sujit Ghosh's notes
January 30th Introduction to the Bayesian approach (cont.) PPT
February 6 Multiparameter models
First-cut Bayesian Computation 		PPT (mostly based on Kate Cowles' notes and Francesca Dominici's notes)
PPT
February 13 Large-Sample Bayes
Hierarchical Models
Bayes Factors 		PPT
PPT
Mostly based on Sujit Ghosh's notes
February 20 Monte Carlo
MCMC 		DOC HTML
DOC HTML
February 27 I am away - we need to rearrange this class.
March 5 More Monte Carlo Some Gibbs Sampling examples.
March 12 Adaptive Rejection Sampling
Probabilistic Graphical Models 		PPT
PPT
March 19 Spring break
March 26 Probabilistic Graphical Models
WinBUGS		 PPT
Software Tutorial Material
April 2 WinBUGS continued
MCMC Diagnostics 		-
PPT
April 8 review class -
April 9 Sequential Monte Carlo PPT
April 16 Computing the Marginal Likelihood PPT
April 16 Bayesian Hidden Markov Models PPT bugs
April 30 Two project presentations plus BMA

Other Bayesian Courses

General Pointers to Bayesian and related Web pages

[]
[]

OLD NEWS

4/30/04 Here's a paper comparing different estimators of the marginal likelihood. The harmonic mean estimator doesn't work very well although its not as bad as the example in Homework 10.

4/17/04 I posted Homework 10.

4/9/04 I posted Homework 9. If you need extra time, that's fine.

4/3/04 Extra Class!. Thursday 4/8 at 10am we'll have an extra class. I'll just do a review session. I'll go over WinBUGS again and show an example of calling BUGS from another program. I'll also review any other material that you suggest.

4/3/04 I posted Homework 8 below - its another BUGS exercise. Again, if you need extra time that's fine.

3/27/04 I posted Homework 7 below - its a WinBUGS exercise. I did not spend as much time as I would have liked on WinBUGS in class on Friday. If you are finding the software confusing, you can hand in the homework a few days late - I will spend quite a bit of time on WinBUGS this coming Friday.

3/4/04 Jacek Rawicki very kindly typed up the Monte Carlo and MCMC notes.

2/20/04 No class on 2/27/04. We'll arrange a makeup class later.

2/20/04: Here's the simple Metropolis example for R that I went over today in class. Here's a two-dimensional example based on Francesca Dominici's code. I working on scanning the lecture notes.

2/16/04: Idea: some students in the class would like to do a project. Lets have an optional project in place of *4* homework assignments. I expect there will be about 12 homework assignments. So, you can either do 12 homework assignments, or, 8 homework assignments plus a project. If you intend to do a project please consult me in advance.

2/12/04: I posted homework 3 and some new lecture notes.

2/7/04: I have heard about some Bayesian internship opportunities this summer in California. Let me know if you are interested.

2/6/04: Here is the R code for today's bioassay example. This is a modified version of Francesca Dominici's program.

2/3/04: JAGS is an open source alternative to BUGS. If you are interested in using JAGS let me know - one of the students in the class has managed to get it compiled.

2/2/04: Here are the football data and the corresponding R code I used in class on Friday.

1/29/04: Here is the R code I used yesterday for showing the beta priors and posteriors.