Talk for BIRS Workshop on Kernel Approx. and GPs
Geometric Gaussian Processes
Alexander Terenin
Geometric Gaussian Processes
AISTATS 2021 Best Student Paper
Matérn Gaussian Processes
$\nu = 1/2$
$\nu = 3/2$
$\nu = 5/2$
$\nu = \infty$
$$ \htmlData{class=fragment fade-out,fragment-index=9}{ \footnotesize \mathclap{ k_\nu(x,x') = \sigma^2 \frac{2^{1-\nu}}{\Gamma(\nu)} \del{\sqrt{2\nu} \frac{\norm{x-x'}}{\kappa}}^\nu K_\nu \del{\sqrt{2\nu} \frac{\norm{x-x'}}{\kappa}} } } \htmlData{class=fragment d-print-none,fragment-index=9}{ \footnotesize \mathclap{ k_\infty(x,x') = \sigma^2 \exp\del{-\frac{\norm{x-x'}^2}{2\kappa^2}} } } $$
$\sigma^2$: variance
$\kappa$: length scale
$\nu$: smoothness
$\nu\to\infty$: recovers squared exponential kernel
Riemannian Geometry
How should Matérn kernels generalize to this setting?
Geodesics
$$ k_\infty^{(d_g)}(x,x') = \sigma^2\exp\del{-\frac{d_g(x,x')^2}{2\kappa^2}} $$
Result. Let $M$ be a complete Riemannian manifold without boundary. If $k_\infty^{(d_g)}$ is positive semi-definite for all $\kappa$, then $M$ is isometric to a Euclidean space.
Need a different candidate generalization
Feragen et al. (2015)
Stochastic Partial Differential Equations
$$ \htmlData{class=fragment,fragment-index=0}{ \htmlClass{anchor-1}{\del{\frac{2\nu}{\kappa^2} - \Delta}^{\frac{\nu}{2}+\frac{d}{4}} f = \c{W}} } \qquad \htmlData{class=fragment,fragment-index=2}{ \htmlClass{anchor-2}{\vphantom{\del{\frac{2\nu}{\kappa^2} - \Delta}^{\frac{\nu}{2}+\frac{d}{4}}} e^{-\frac{\kappa^2}{4}\Delta} f = \c{W}} } $$
Matérn
squared exponential
$\Delta$: Laplacian
$\c{W}$: (rescaled) white noise
$e^{-\frac{\kappa^2}{4}\Delta}$: (rescaled) heat semigroup
Generalizes well to the Riemannian setting
Whittle (1963)
Lindgren et al. (2011)
Riemannian Matérn Kernels: compact spaces
$$ k_\nu(x,x') = \frac{\sigma^2}{C_\nu} \sum_{n=0}^\infty \del{\frac{2\nu}{\kappa^2} - \lambda_n}^{\nu-\frac{d}{2}} f_n(x) f_n(x') $$
$\lambda_n, f_n$: Laplace–Beltrami eigenpairs
Analytic expressions for circle, sphere, ..
Riemannian Matérn Kernels
$k_{1/2}(\htmlStyle{color:rgb(0, 0, 0)!important}{\bullet},\.)$
Example: regression on the surface of a dragon
(a) Ground truth
(b) Posterior mean
(c) Std. deviation
(d) Posterior sample
Stationary Kernels on Compact Lie Groups
$$ \begin{aligned} \htmlData{fragment-index=0,class=fragment}{ k(g_1,g_2) } & \htmlData{fragment-index=0,class=fragment}{ = \sum_{n=1}^\infty a(\lambda_n) \v{f}_n(g_1)\v{f}_n(g_2) } \\ & \htmlData{fragment-index=1,class=fragment}{ = \sum_{\lambda\in\Lambda} a^{(\lambda)} \f{Re} \chi^{(\lambda)}(g_2^{-1} \. g_1) } \end{aligned} $$
$\Lambda$: set of irreducible unitary representations
$\chi^{(\lambda)}$: characters
$a^{(\lambda)}$: for Matérn, explicit function of Laplace–Beltrami eigenvalues
compute using signatures and truncate sum
Weyl character formula:ratio of polynomials
also compute using signatures
Yaglom (1961)
Stationary Kernels on Compact Homogeneous Spaces
$$ \begin{aligned} \htmlData{fragment-index=0,class=fragment}{ k(g_1,g_2) } & \htmlData{fragment-index=0,class=fragment}{ = \sum_{n=1}^\infty a(\lambda_n) \v{f}_n(g_1)\v{f}_n(g_2) } \\ & \htmlData{fragment-index=1,class=fragment}{ = \sum_{\lambda\in\Lambda} \sum_{j,k=1}^{r_\lambda} a^{(\lambda)}_{jk} \pi^{(\lambda)}_{jk}(g_2^{-1} \. g_1) } \end{aligned} $$
$\Lambda$: irreducible unitary representations
$\pi^{(\lambda)}_{jk}$: zonal spherical functions
$a^{(\lambda)}$: PSD matrix, for Matérn turns out to be diagonal
compute using generalized periodic summation
Yaglom (1961)
Example: real projective space
Kernel
Prior samples
Example: regression on a real projective space
(a) Ground truth
(b) Posterior mean
(c) Std. deviation
(d) Posterior sample
Stationary Kernels on Non-compact Symmetric Spacess
$$ \htmlData{fragment-index=0,class=fragment}{ k(x_1, x_2) = k(\htmlClass{anchor-1}{g_1\. H,g_2\. H}) } \htmlData{fragment-index=2,class=fragment}{ = \int_\Lambda \pi^{(\lambda)}(g_2^{-1} \. g_1) \d\mu_k(\lambda) } $$
$\Lambda$: set of all irreducible unitary representations
$\pi^{\smash{(\lambda)}}$: zonal spherical functions
$\mu_k$: finite measure
Approach: compute all quantities using Iwasawa decomposition
cosets of $x_1$ and $x_2$where $X = G/H$
including infinite-dimensional ones
one per $\lambda$ for symmetric spaces
Yaglom (1961)
Examples: hyperbolic space and symmetric positive-definite matrices
Hyperbolic space
Symmetric positive-definite matrices
spectral measure $\mu_k$: very closely related to Gaussian orthogonal ensemble from random matrix theory
Geometric Kernels in Python
Thank you!
Thank you!
V. Borovitskiy,* P. Mostowsky,* A. Terenin,* M. P. Deisenroth. Matérn Gaussian Processes on Riemannian Manifolds. NeurIPS, 2020.
V. Borovitskiy,* I. Azangulov,* P. Mostowsky,* A. Terenin,* M. P. Deisenroth, N. Durrande. Matérn Gaussian Processes on Graphs. AISTATS, 2021. Best Student Paper Award.
M. J. Hutchinson,* A. Terenin,* V. Borovitskiy,* S. Takao,* Y. W. Teh, M. P. Deisenroth. Vector-valued Gaussian Processes on Riemannian Manifolds via Gauge Independent Projected Kernels. NeurIPS, 2021.
N. Jaquier,* V. Borovitskiy,* A. Smolensky, A. Terenin, T. Asfour, L. Rozo. Geometry-aware Bayesian Optimization in Robotics using Riemannian Matérn Kernels. CoRL, 2021.
I. Azangulov, A. Smolensky, A. Terenin, V. Borovitskiy. Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces I: the Compact Case. JMLR, 2024.
I. Azangulov, A. Smolensky, A. Terenin, V. Borovitskiy. Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces II: Non-compact Symmetric Spaces. JMLR, 2024.
S. Holalkere, D. Bindel, S. Sellán, A. Terenin. Stochastic Poisson Surface Reconstruction with One Solve using Geometric Gaussian Processes. ICML, 2025.
P. Mostowsky, V. Dutordoir, I. Azangulov, N. Jaquier, M. J. Hutchinson, A. Ravuri, L. Rozo, A. Terenin, V. Borovitskiy. The GeometricKernels Package: Heat and Matérn Kernels for Geometric Learning on Manifolds, Meshes, and Graphs. JMLR, 2025.
*Equal contribution