Regression Density Estimation With Variational Methods and Stochastic Approximation

REFERENCES

• Biernacki , C. , Celeux , G. and Govaert , G. 2003 . “Choosing Starting Values for the EM Algorithm for Getting the Highest Likelihood in Multivariate Gaussian Mixture Models,” . Computational Statistics and Data Analysis , 41 : 561 – 575 .
   Web of Science ®Google Scholar
• Bishop , C. M. 2006 . Pattern Recognition and Machine Learning , New York : Springer .
   Google Scholar
• Bishop , C. M. and Svensén , M. 2003 . “Bayesian Hierarchical Mixtures of Experts,” . In Proceedings of the 19th Conference on Uncertainty in Artificial Intelligence , Edited by: Kjaerulff , U. and Meek , C. 57 – 64 . Waltham, MA : Morgan Kaufmann .
   Google Scholar
• Blei , D. M. and Jordan , M. I. 2006 . “Variational Inference for Dirichlet Process Mixtures,” . Bayesian Analysis , 1 : 121 – 144 .
   Web of Science ®Google Scholar
• Braun , M. and McAuliffe , J. 2010 . “Variational Inference for Large-Scale Models of Discrete Choice,” . Journal of the American Statistical Association , 105 : 324 – 335 .
   Web of Science ®Google Scholar
• Boughton , W. 2004 . “The Australian Water Balance Model,” . Environmental Modelling and Software , 19 : 943 – 956 .
   Web of Science ®Google Scholar
• Constantinopoulos , C. and Likas , A. 2007 . “Unsupervised Learning of Gaussian Mixtures Based on Variational Component Splitting,” . IEEE Transactions on Neural Networks , 18 : 745 – 755 .
   PubMed Web of Science ®Google Scholar
• Corduneanu , A. and Bishop , C. M. 2001 . “Variational Bayesian Model Selection for Mixture Distributions,” . In Artificial Intelligence and Statistics , Edited by: Jaakkola , T. and Richardson , T. 27 – 34 . Waltham, MA : Morgan Kaufmann .
   Google Scholar
• de Freitas , N. , Højen-Sørensen , P. , Jordan , M. I. and Russell , S. 2001 . “Variational MCMC,” . In Uncertainty in Artificial Intelligence (UAI): Proceedings of the 17th Conference , Edited by: Breese , J. and Koller , D. 120 – 127 . San Francisco , CA : Morgan Kaufmann .
   Google Scholar
• De Iorio , M. , Müller , P. , Rosner , G. L. and MacEAchern , S. N. 2004 . “An ANOVA Model for Dependent Random Measures,” . Journal of the American Statistical Association , 99 : 205 – 215 .
   Web of Science ®Google Scholar
• Dunson , D. B. , Pillai , N. and Park , J.-H. 2007 . “Bayesian Density Regression,” . Journal of the Royal Statistical Society, Series B , 69 : 163 – 183 .
   Google Scholar
• Frühwirth-Schnatter , S. 2004 . “Estimating Marginal Likelihoods for Mixture and Markov Switching Models Using Bridge Sampling Techniques,” . The Econometrics Journal , 7 : 143 – 167 .
   Google Scholar
• Geweke , J. and Amisano , G. 2010 . “Comparing and Evaluating Bayesian Predictive Distributions of Asset Returns,” . International Journal of Forecasting , 26 : 216 – 230 .
   Web of Science ®Google Scholar
• Geweke , J. and Keane , M. 2007 . “Smoothly Mixing Regressions,” . Journal of Econometrics , 138 : 252 – 291 .
   Web of Science ®Google Scholar
• Ghahramani , Z. and Beal , M. J. 2000 . “Variational Inference for Bayesian Mixtures of Factor Analysers,” . In Advances in Neural Information Processing Systems (Vol. 12) , Edited by: Solla , S. A. , Leen , T. K. and Müller , K-R . 831 – 864 . Cambridge : MIT Press .
   Google Scholar
• Griffin , J. E. and Steel , M. F. J. 2006 . “Order-Based Dependent Dirichlet Processes,” . Journal of the American Statistical Association , 101 : 179 – 194 .
   Web of Science ®Google Scholar
• Honkela , A. and Valpola , H. 2003 . “On-Line Variational Bayesian Learning,” . In Proceedings of the 4th International Symposium on Independent Component Analysis and Blind Signal Separation (ICA 2003) , 803 – 808 . Berlin : Springer .
   Google Scholar
• Jacobs , R. , Jordan , M. , Nowlan , S. and Hinton , G. 1991 . “Adaptive Mixtures of Local Experts,” . Neural Computation , 3 : 79 – 87 .
   PubMed Web of Science ®Google Scholar
• Ji , C. , Shen , H. and West , M. 2010 . “Bounded Approximations for Marginal Likelihoods,” . Technical Report, ISDS, Duke University. Available at http://ftp.stat.duke.edu/WorkingPapers/10-05.html
   Google Scholar
• Jiang , W. and Tanner , M. 1999 . “Hierarchical Mixtures-of-Experts for Exponential Family Regression Models: Approximation and Maximum Likelihood Estimation,” . The Annals of Statistics , 27 : 987 – 1011 .
   Web of Science ®Google Scholar
• Jordan , M. I. , Ghahramani , Z. , Jaakkola , T. S. and Saul , L. K. 1999 . “An Introduction to Variational Methods for Graphical Models,” . In Learning in Graphical Models , Edited by: Jordan , M. I. 105 – 158 . Cambridge , MA : MIT Press .
   Google Scholar
• Jordan , M. I. and Jacobs , R. A. 1994 . “Hierarchical Mixtures of Experts and the EM Algorithm,” . Neural Computation , 6 : 181 – 214 .
   Web of Science ®Google Scholar
• Li , F. , Villani , M. and Kohn , R. 2010a . “Modeling Conditional Densities Using Finite Smooth Mixtures,” . In Mixtures: Estimation and Applications , Edited by: Mengersen , K. L. , Robert , C. P. and Titterington , D. M. Chichester, UK : John Wiley & Sons, Ltd. .
   Google Scholar
• Li , F. , Villani , M. and Kohn , R. 2010b . “Flexible Modeling of Conditional Distributions Using Smooth Mixtures of Asymmetric Student t Densities,” . Journal of Statistical Planning and Inference , 140 : 3638 – 3654 .
   Web of Science ®Google Scholar
• MacEachern , S. N. 1999 . “Dependent Nonparametric Processes,” . In ASA Proceedings of the Section on Bayesian Statistical Science , Alexandria, VA : American Statistical Association, pp. 50–55 .
   Google Scholar
• McCullagh , P. and Nelder , J. A. 1989 . Generalized Linear Models , (2nd ed.) , London : Chapman and Hall .
   Google Scholar
• McGrory , C. A. and Titterington , D. M. 2007 . “Variational Approximations in Bayesian Model Selection for Finite Mixture Distributions,” . Computational Statistics and Data Analysis , 51 : 5352 – 5367 .
   Web of Science ®Google Scholar
• Norets , A. 2010 . “Approximation of Conditional Densities by Smooth Mixtures of Regressions,” . The Annals of Statistics , 38 : 1733 – 1766 .
   Web of Science ®Google Scholar
• O’Hagan , A. 2006 . “Bayesian Analysis of Computer Code Outputs: A Tutorial,” . Reliability Engineering and System Safety , 91 : 1290 – 1300 .
   Web of Science ®Google Scholar
• Ormerod , J. T. and Wand , M. P. 2010 . “Explaining Variational Approximations,” . The American Statistician , 64 ( 2 ) : 140 – 153 .
   Web of Science ®Google Scholar
• Peng , F. , Jacobs , R. A. and Tanner , M. A. 1996 . “Bayesian Inference in Mixtures-of-Experts and Hierarchical Mixtures-of-Experts Models with an Application to Speech Recognition,” . Journal of the American Statistician Association , 91 : 953 – 960 .
   Web of Science ®Google Scholar
• Pepelyshev , A. 2010 . “The Role of the Nugget Term in the Gaussian Process Method,” . In MODA 9—Advances in Model-Oriented Design and Analysis: Contributions to Statistics , 149 – 156 . New York : Springer .
   Google Scholar
• Pesaran , M. H. and Timmermann , A. 2002 . “Market Timing and Return Prediction Under Model Instability,” . Journal of Empirical Finance , 9 : 495 – 510 .
   Google Scholar
• Robbins , H. and Monro , S. 1951 . “A Stochastic Approximation Method,” . Annals of Mathematical Statistics , 22 : 400 – 407 .
   Google Scholar
• Spall , J. C. 2003 . Introduction to Stochastic Search and Optimization: Estimation, Simulation and Control , Hoboken , NJ : Wiley .
   Google Scholar
• Spiegelhalter , D. J. , Best , N. G. , Carlin , B. P. and Van der Linde , A. 2002 . “Bayesian Measures of Model Complexity and Fit” (with discussion), . Journal of the Royal Statistical Society, Series B , 64 : 583 – 616 .
   Google Scholar
• Ueda , N. and Ghahramani , Z. 2002 . “Bayesian Model Search for Mixture Models Based on Optimizing Variational Bounds,” . Neural Networks , 15 : 1223 – 11241 .
   PubMed Web of Science ®Google Scholar
• Vehtari , A. and Lampinen , J. 2002 . “Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities,” . Neural Computation , 14 : 2439 – 2468 .
   PubMed Web of Science ®Google Scholar
• Villani , M. , Kohn , R. and Giordani , P. 2009 . “Regression Density Estimation Using Smooth Adaptive Gaussian Mixtures,” . Journal of Econometrics , 153 : 155 – 173 .
   Web of Science ®Google Scholar
• Waterhouse , S. , MacKay , D. and Robinson , T. 1996 . “Bayesian Methods for Mixtures of Experts,” . In Advances in Neural Information Processing Systems, (Vol. 8) , Edited by: Touretzky , D. S. , Mozer , M. C. and Hasselmo , M. E. 351 – 357 . Cambridge , MA : MIT Press .
   Google Scholar
• Wand , M. P. 2002 . “Vector Differential Calculus in Statistics,” . The American Statistician , 56 : 55 – 62 .
   Web of Science ®Google Scholar
• Wang , B. and Titterington , D. M. 2005 . “Inadequacy of Interval Estimates Corresponding to Variational Bayesian Approximations,” . In Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics , Edited by: Cowell , R. G. and Ghahramani , Z. 373 – 380 . Society for Artificial Intelligence and Statistics .
   Google Scholar
• Wedel , M. 2002 . “Concomitant Variables in Finite Mixture Models,” . Statistica Neerlandica , 56 : 362 – 375 .
   Web of Science ®Google Scholar
• West , M. 1985 . “Generalized Linear Models: Outlier Accommodation, Scale Parameters and Prior Distributions,” . In Bayesian Statistics 2 , Edited by: Bernardo , J. M. , DeGroot , M. H. , Lindley , D. V. and Smith , A. F. M. 531 – 538 . North Holland : Amsterdam .
   Google Scholar
• Wood , S. A. , Jiang , W. and Tanner , M. A. 2002 . “Bayesian Mixture of Splines for Spatially Adaptive Nonparametric Regression,” . Biometrika , 89 : 513 – 528 .
   Web of Science ®Google Scholar
• Wood , S. A. , Kohn , R. , Cottet , R. , Jiang , W. and Tanner , M. 2008 . “Locally Adaptive Nonparametric Binary Regression,” . Journal of Computational and Graphical Statistics , 17 : 352 – 372 .
   Web of Science ®Google Scholar
• Wu , B. , McGrory , C. A. and Pettitt , A. N. 2012 . “A New Variational Bayesian Algorithm With Application to Human Mobility Pattern Modeling,” . Statistics and Computing , 22 ( 1 ) : 185 – 203 .
   Web of Science ®Google Scholar