Distributed Markov Chain Monte Carlo for Bayesian Hierarchical Models

69 Pages Posted: 9 May 2017 Last revised: 27 Jul 2017

See all articles by Federico Bumbaca

Federico Bumbaca

University of California, Irvine - Paul Merage School of Business

Sanjog Misra

University of Chicago - Booth School of Business

Peter E. Rossi

University of California, Los Angeles (UCLA) - Anderson School of Management

Date Written: May 7, 2017

Abstract

This article proposes a distributed Markov chain Monte Carlo (MCMC) algorithm for estimating Bayesian hierarchical models when the number of cross-sectional units is very large and the objects of interest are the unit-level parameters. The two-stage algorithm is asymptotically exact, retains the flexibility of a standard MCMC algorithm, and is easy to implement. The algorithm constructs an estimator of the posterior predictive distribution of the unit-level parameters in the first stage, and uses the estimator as the prior distribution in the second stage for the unit-level draws. Both stages are embarrassingly parallel. The algorithm is demonstrated with simulated data from a hierarchical logit model and is shown to be faster and more efficient (in effective sample size generated per unit of computing) than a single machine algorithm by at least an order of magnitude. For a relatively small number of observations per cross-sectional unit, the algorithm is both faster and has better mixing properties than the standard hybrid Gibbs sampler. We illustrate our approach with data on 1,100,000 donors to a charitable organization, and simulations with up to 100 million units.

Keywords: distributed Markov Chain Monte Carlo, Bayesian hierarchical model

JEL Classification: C11, C23

Suggested Citation

Bumbaca, Federico and Misra, Sanjog and Rossi, Peter E., Distributed Markov Chain Monte Carlo for Bayesian Hierarchical Models (May 7, 2017). Available at SSRN: https://ssrn.com/abstract=2964646 or http://dx.doi.org/10.2139/ssrn.2964646

Federico Bumbaca

University of California, Irvine - Paul Merage School of Business ( email )

Paul Merage School of Business
Irvine, CA California 92697-3125
United States

Sanjog Misra

University of Chicago - Booth School of Business ( email )

5807 South Woodlawn Avenue
Chicago, IL 60637
United States

Peter E. Rossi (Contact Author)

University of California, Los Angeles (UCLA) - Anderson School of Management ( email )

110 Westwood Plaza
Los Angeles, CA 90095-1481
United States
773-294-8616 (Phone)

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
338
Abstract Views
2,227
Rank
162,865
PlumX Metrics