Approximate Bayesian inference in high-dimensional systems is often enabled by approximate message passing methods. In linear models, Generalized Approximate Message Passing (GAMP) handles non-identically independently distributed (niid) priors and niid measurement channels applied to niid linear-mixture outputs. GAMP was originally derived through large-system-limit approximations of belief propagation, which can be viewed as an alternating-minimization procedure for the Lagrangian of the Bethe free energy.

To obtain algorithms with stronger convergence behavior, it is natural to move from approximate alternating minimization to exact alternating minimization on a large-system-limit Bethe free energy. I will discuss two such approaches: one inspired by ADMM, and another that alternately enforces the Karush-Kuhn-Tucker conditions. To avoid repeated matrix inversions, we introduce adaptive and accelerated gradient-descent variants, where the choice of adaptation is crucial for stability and speed.

Bilinear models arise in semi-blind multi-user MIMO channel estimation and provide a route to mitigating pilot contamination. Starting from second-order statistics of the unknown data, we derive a tight Bayesian Cramér-Rao bound, where a key expectation is simplified by large-system asymptotics. I will then review message-passing-type approximate MMSE algorithms based on expectation propagation, highlighting how different levels of prior information can be exploited.

The talk will also compare several complexity variants, including their impact on convergence speed, estimation performance, and distributed implementation. I will conclude by outlining open directions, including convergence theory, further asymptotic simplifications, more decentralized semi-blind architectures, and acceleration.

Approximate Message Passing (AMP) algorithms admit precise asymptotic characterizations through state evolution, providing powerful tools for analyzing high-dimensional inference problems. In this talk, we present a Gaussian coupling framework for a broad class of generalized first-order methods (GFOMs) that extends these ideas beyond the AMP setting. For every GFOM, we construct an explicit conditionally Gaussian comparison process and establish dimension-free, non-asymptotic bounds that couple the algorithm iterates to their Gaussian counterparts. The resulting guarantees hold at finite dimension and apply to a wide range of iterative algorithms generated by random matrix operations and nonlinear updates.

Diffusion models over discrete spaces have recently shown striking empirical success, yet their theoretical foundations remain incomplete. In this talk, we explore the sampling efficiency of score-based discrete diffusion models under a continuous-time Markov chain formulation, with a focus on τ-leaping-based samplers. We establish sharp convergence guarantees for attaining ε accuracy in Kullback-Leibler divergence for both uniform and masking noising processes. (1) For uniform discrete diffusion, we show that the τ-leaping algorithm achieves an iteration complexity of order O(d/ε), with d the ambient dimension of the target distribution, eliminating linear dependence on the vocabulary size S and improving existing bounds by a factor of d. (2) For masking discrete diffusion, we introduce a modified τ-leaping sampler whose convergence rate is governed by an intrinsic information-theoretic quantity, termed the effective total correlation, which is bounded by d log S but can be sublinear or even constant for structured data. As a consequence, the sampler provably adapts to low-dimensional structure without prior knowledge or algorithmic modification, yielding sublinear convergence rates for various practical examples, such as hidden Markov models, image data, and random graphs.

Neural scaling laws underlie many of the recent advances in deep learning, yet their theoretical understanding remains largely confined to linear models. In this work, we present a systematic analysis of scaling laws for shallow neural networks in the feature learning regime. Leveraging connections with matrix compressed sensing and LASSO, we derive a detailed phase diagram for the scaling exponents of the excess risk as a function of sample complexity and weight decay. This analysis uncovers crossovers between distinct scaling regimes and plateau behaviours, mirroring phenomena widely reported in the empirical neural scaling literature. Furthermore, we establish a precise link between these regimes and the spectral properties of the trained network weights, which we characterize in detail. Consequently, we provide a theoretical validation of recent empirical observations connecting the emergence of power-law tails in the weight spectrum with network generalization performance, yielding an interpretation from first principles.

We study a class of diffusion processes on \(\mathbb{R}^n\) interacting through a symmetric matrix \(X\). When eigenvectors of \(X\) are random and uniformly distributed on the orthogonal group, we derive a dynamical mean-field limit for the empirical law of sample paths, extending a classical characterization due to Sompolinsky and Zippelius when \(X\) is GOE. This limit takes the form of a generalized Langevin equation with correlated Gaussian noise and memory, whose correlation and response kernels relate to those of the original diffusion process through a system of convolution equations depending on \(X\) via the free cumulants of its eigenvalue distribution.

Specializing to the overdamped Langevin diffusion for a S-K type model, we analyze this mean-field limit under an assumption that the dynamics are rapidly mixing, and show that the generalized Langevin equation converges in time to a replica-symmetric prediction for the empirical law of coordinates under the Gibbs measure. Consequently, we prove that the free energy of the Gibbs measure converges to a replica-symmetric limit under an explicit high-temperature condition, which in the case of Ising spins is \(\|X\| < 1/2\). By recent dynamical universality results, the same conclusion then holds for models without random disorder, where \(X\) is deterministic and satisfies a set of delocalization conditions.

This is joint work with Theodor Misiakiewicz, Leda Wang, and Garrett G. Wen.