Mathematical Foundations of Infinite-Dimensional Statistical Models

This book provides a comprehensive account of the mathematical foundations of nonparametric models, for example, density estimation and regression, and develops a general framework for studying estimation and inference in infinite dimensional problems.

The first part of the book, namely Chapters 1–4, provides a rigorous and thorough account of the theory of empirical processes and Gaussian processes, and Chapters 5–8 study numerous statistical applications related to nonparametric estimation, hypothesis testing and confidence intervals.

Chapter 1 starts with an introduction of the main models to be studied in this book and ends with a formal proof of the asymptotic equivalence (in the Le Cam sense) of the Gaussian sequence model (or white noise model) and nonparametric Gaussian fixed-design regression. Chapter 2 provides a rigorous and detailed study of Gaussian processes, and develops some of the main theoretical results in this theory—concentration of the supremum of a (sub)-Gaussian process, Dudley’s metric entropy bound and Sudakov’s lower bound, isoperimetric and log-Sobolev inequalities, etc. The theory of empirical processes, focussing on finite sample bounds, is described in Chapter 3. This approach nicely complements the classical book on empirical process theory and weak convergence by van der Vaart and Wellner (1996). I especially like the treatment of the celebrated Talagrand’s inequalities in Sections 3.3 and 3.4; I have used this material very closely to teach a class on empirical processes at Columbia.

Chapter 4 provides a solid background in functional analysis and approximation theory; in particular, the theory of wavelets and Besov spaces that are used in the later chapters of the book.

Building upon the theory developed in Chapters 1–4, the book describes a broad spectrum of statistical applications such as: (i) minimax lower bounds (Chapter 6.3), (ii) nonparametric likelihood methods (Chapter 7.2), (iii) Bayesian nonparametrics (Chapter 7.3), and (ii) adaptive inference (Chapter 8). A prominent theme in these chapters is the emphasis on the minimax paradigm for nonparametric function estimation and inference. Such a rigorous and detailed treatment of this framework with examples is not available in accessible form in other books on nonparametric theory. It nicely complements the classical book on the minimax paradigm for nonparametric function estimation by Tsybakov (2009).

Let me provide a sampler of some of the interesting statistical applications described in this book that caught my attention. Theoretical properties of kernel and projection-type estimators are discussed in Chapter 5.1. Chapter 5.3.1 provides an introduction to estimation of functionals—integrals of smooth functions—of a density, whereas Chapter 5.3.2 studies the deconvolution problem. The minimax testing paradigm with examples is studied in Chapter 6.2. Chapter 7.3.4 discusses the scope of Bernstein-von Mises type results—the asymptotic normality of the posterior distribution—in infinite dimensional settings.

Let me now describe, in a bit more detail, the main content of Chapter 8 which investigates adaptive inference in nonparametric statistics. To understand this phenomenon, let us consider construction of minimax optimal nonparametric testing procedures (as described in Chapter 6.2). It is known that such constructions depend strongly on regularity properties of the nonparametric model under consideration (such as the Hölder constants of the underlying function class) that are usually not known in practice. The idea of an adaptive procedure is to construct a test that does not require knowledge of these parameters but can still perform optimally for any given value of these unknowns. The authors illustrate that in certain cases such adaptive tests exist. In particular, Chapters 8.1.1 and Chapters 8.1.2 quantify precisely the price to be paid for such adaptation, under different loss functions.

To summarize, in Chapters 8.1 and.82, the authors show that full adaptation is possible in many testing and estimation problems, and that mild losses occur for some adaptive testing problems. In contrast, the theory of adaptive confidence sets—and, more generally, the problem of adaptive uncertainty quantification—is more intricate, and the price for adaptation can be severe unless some additional structural assumptions on the parameter space are imposed. In particular, the authors introduce a class of “self-similar” functions for which a unified theory of estimation, testing and confidence sets can be demonstrated to exist. As far as I know, such a detailed and nuanced study of adaptive procedures was not available in a textbook before.

The book contains many exercises and problems that aim to help the reader gain a better understanding of the theory presented. Notes in every chapter give a bibliographic account of the historical developments of the ideas. The coverage of this book, in terms of the variety of topics studied, is commendable. Another nice feature of this book is that each chapter is reasonably self-contained and can be read independently. However, the book can sometimes feel a bit dense especially when used to teach a graduate course in statistics on modern nonparametric methods. I usually complement this book with the more recent book Wainwright (2019) which has more (modern) statistical applications and a gentler introduction to some of the topics.

In summary, this book will be invaluable to any researcher (including graduate students) interested in nonparametric problems in statistics and machine learning.

Bodhisattva Sen
Columbia University, New York, NY
[email protected]