This is a continuition from my previous blog post on Banach and Hilbert Spaces. I hope to cover some important results and why we should care in statistics.

Weak Convergence

The idea behind the large numbers of convergences is to have a notion of a sequence of values (maybe numbers or functions) approaching some value. This can differ with how we define the “distance” between two values. This is why convergence in measure exists for measure spaces, convergence in metric space exists for metric spaces, and what I am doing to focus on: convergence in norm (strong convergence) and convergence in inner product (weak convergence). Notice that this will apply to hilbert spaces.

A sequence of vectors \(v_n\) in an inner product space \(H\) converges strongly to a vector \(v\) if the norm between \(v_n\) and \(v\) is zero as \(n \to \infty\), i.e. \(\lim_{n \to \infty} \| v_n - v \| = 0\). Similarly, that sequence converges weakly if for all \(y \in H\), \(\lim_{n \to \infty} <x_n , y> = <x, y>\).

Strong convergence implies weak convergence. The proof requires using the triangle inequality a bunch and is probably a good exercise for me to do when I read this again.

Riesz Representation for Hilbert Spaces

Every continuous linear functional on a Hilbert space is an inner product with a unique element

Which then implies that a hilbert space is isomorphic to its dual space? Why exactly is that important?

Orthogonal Projections

Tanget Spaces

Need to channel my knowledge from differential geometry…

Examples in Statistics

I can think of two ways hilbert spaces play into topics I am interested in: functional data analysis (FDA) and semiparametric theory.

For FDA, each observation (which is a function) lives in a function space. For all the math to work out really nicely, we usually think of a subset of a general function space, such as only square-integrable functions equipped with an inner product so we can discuss angles and norms between different functions. This fits nicely with hilbert spaces. There is also something called the reproducing kernel hilbert space (RKHS).

Since I haven’t learned about nusiance parameters, I find it difficult to understand the connection of hilbert spaces to semiparametric theory that well. All I know is that the nonparametric part (which we make no assumptions about its distribution, which means it lives in an infinite dimensional space. Since we don’t care about it for estimation, this is labeled as “nusiance”) needs to be understood first before we talk about the parametric part (finite dimensional, we make some assumptions about their distribution?)

There are also machine learning applications, but I don’t really care about that right now. Maybe I will in the future?