Approximate Bayesian inference in high-dimensional systems is often enabled by approximate message passing methods. In linear models, Generalized Approximate Message Passing (GAMP) handles non-identically independently distributed (niid) priors and niid measurement channels applied to niid linear-mixture outputs. GAMP was originally derived through large-system-limit approximations of belief propagation, which can be viewed as an alternating-minimization procedure for the Lagrangian of the Bethe free energy.

To obtain algorithms with stronger convergence behavior, it is natural to move from approximate alternating minimization to exact alternating minimization on a large-system-limit Bethe free energy. I will discuss two such approaches: one inspired by ADMM, and another that alternately enforces the Karush-Kuhn-Tucker conditions. To avoid repeated matrix inversions, we introduce adaptive and accelerated gradient-descent variants, where the choice of adaptation is crucial for stability and speed.

Bilinear models arise in semi-blind multi-user MIMO channel estimation and provide a route to mitigating pilot contamination. Starting from second-order statistics of the unknown data, we derive a tight Bayesian Cramér-Rao bound, where a key expectation is simplified by large-system asymptotics. I will then review message-passing-type approximate MMSE algorithms based on expectation propagation, highlighting how different levels of prior information can be exploited.

The talk will also compare several complexity variants, including their impact on convergence speed, estimation performance, and distributed implementation. I will conclude by outlining open directions, including convergence theory, further asymptotic simplifications, more decentralized semi-blind architectures, and acceleration.

Approximate Message Passing (AMP) algorithms admit precise asymptotic characterizations through state evolution, providing powerful tools for analyzing high-dimensional inference problems. In this talk, we present a Gaussian coupling framework for a broad class of generalized first-order methods (GFOMs) that extends these ideas beyond the AMP setting. For every GFOM, we construct an explicit conditionally Gaussian comparison process and establish dimension-free, non-asymptotic bounds that couple the algorithm iterates to their Gaussian counterparts. The resulting guarantees hold at finite dimension and apply to a wide range of iterative algorithms generated by random matrix operations and nonlinear updates.

Diffusion models over discrete spaces have recently shown striking empirical success, yet their theoretical foundations remain incomplete. In this talk, we explore the sampling efficiency of score-based discrete diffusion models under a continuous-time Markov chain formulation, with a focus on τ-leaping-based samplers. We establish sharp convergence guarantees for attaining ε accuracy in Kullback-Leibler divergence for both uniform and masking noising processes. (1) For uniform discrete diffusion, we show that the τ-leaping algorithm achieves an iteration complexity of order O(d/ε), with d the ambient dimension of the target distribution, eliminating linear dependence on the vocabulary size S and improving existing bounds by a factor of d. (2) For masking discrete diffusion, we introduce a modified τ-leaping sampler whose convergence rate is governed by an intrinsic information-theoretic quantity, termed the effective total correlation, which is bounded by d log S but can be sublinear or even constant for structured data. As a consequence, the sampler provably adapts to low-dimensional structure without prior knowledge or algorithmic modification, yielding sublinear convergence rates for various practical examples, such as hidden Markov models, image data, and random graphs.

Neural scaling laws underlie many of the recent advances in deep learning, yet their theoretical understanding remains largely confined to linear models. In this work, we present a systematic analysis of scaling laws for shallow neural networks in the feature learning regime. Leveraging connections with matrix compressed sensing and LASSO, we derive a detailed phase diagram for the scaling exponents of the excess risk as a function of sample complexity and weight decay. This analysis uncovers crossovers between distinct scaling regimes and plateau behaviours, mirroring phenomena widely reported in the empirical neural scaling literature. Furthermore, we establish a precise link between these regimes and the spectral properties of the trained network weights, which we characterize in detail. Consequently, we provide a theoretical validation of recent empirical observations connecting the emergence of power-law tails in the weight spectrum with network generalization performance, yielding an interpretation from first principles.

General first order methods (GFOMs), including various gradient descent and AMP algorithms, constitute a broad class of iterative algorithms in modern statistical learning problems. Some GFOMs also serve as constructive proof devices, iteratively characterizing the empirical distributions of statistical estimators in the large system limits for any fixed number of iterations.

In this talk, I will describe recent theoretical progress on the entrywise dynamics of a large class of GFOMs. Our theory provides a precise distributional characterization of GFOM iterates at the coordinate level, and establishes universality across a broad class of heterogeneous random matrix models.

I will illustrate the utility of these general results through two applications. The first develops an algorithmic method for proving refined universality results for regularized least-squares estimators in linear models, as well as for regularized maximum likelihood estimators in logistic regression. The second develops an algorithmic debiasing method that enables statistical inference along the entire gradient descent trajectory. This inference procedure remains valid at each iteration, without requiring algorithmic convergence, and is therefore particularly well suited for modern learning problems with complex algorithmic dynamics.