Overview
Research investigated the performance of Markov chain Monte Carlo algorithms in the context of Bayesian inference, specifically focusing on the classical spiked Wigner inference problem. The study aimed to connect the recovery performance of Glauber dynamics on the annealed posterior to that of Approximate Message Passing (AMP), which is recognized for achieving Bayes-optimal performance.
Research Context
Markov chain Monte Carlo algorithms have been observed to achieve near-optimal performance in various Bayesian inference scenarios. However, establishing a rigorous theoretical foundation for these observations has presented challenges. The classical spiked Wigner inference problem involves recovering a planted Boolean spike from noisy matrix measurements.
Approach
The research employed an analysis centered on an auxiliary Markov chain referred to as restricted Gaussian dynamics (RGD). The methodology involved examining the relationship between RGD's evolution and the iterates of AMP within the spiked Wigner inference problem. The analysis considered the conditions under which RGD converges and how its performance relates to Bayes optimality.
Findings
- Restricted Gaussian dynamics (RGD) can be reduced to an effective one-dimensional recursion. This recursion mirrors the evolution of the Approximate Message Passing (AMP) iterates.
- From an initial warm start, RGD rapidly converges to a fixed point within correlation space. When executed on the posterior, this convergence recovers Bayes-optimal performance.
- The phase transition necessary for non-trivial inference is recovered. This recovery is conditional on mixing results for the Sherrington-Kirkpatrick (SK) model, which are widely believed to hold.
Why This Matters
This research contributes to the theoretical understanding of Markov chain Monte Carlo algorithms, particularly concerning their performance in Bayesian inference. By establishing a rigorous connection between Glauber dynamics, restricted Gaussian dynamics, and Approximate Message Passing in the spiked Wigner inference problem, the study provides a framework for analyzing near-optimal performance, addressing a long-standing challenge in the field.