*

A fundamental problem in machine learning is the following. A data of the form $\{(x_{j},y_{j})\}$ is given, assumed to be sampled from an unknown probability distribution. The goal is to approximate the function $f(x)=\mathbb{E}(y|x)$ from the data. Typically, the points $x_{j}$ belong to an ambient Euclidean space of a very high dimension, leading to the so called curse of dimensionality. One of the strategies to counter this “curse” is to assume the manifold hypothesis; i.e., assume that the points $x_{j}$ are located on an unknown low dimensional submanifold of the ambient space. Examples of some well known techniques in this direction, dimensionality reduction in particular, are Isomaps tenenbaum2000global , maximum variance unfolding (MVU) (also called semidefinite programming (SDP)) weinberger2005nonlinear , locally linear embedding (LLE) roweis2000nonlinear , local tangent space alignment method (LTSA) zhang2004principal , Laplacian eigenmaps (Leigs) belkin2003laplacian , Hessian locally linear embedding (HLLE) david2003hessian , diffusion maps (Dmaps) coifmanlafondiffusion , and randomized anisotropic transform chuiwang2010 . A recent survey of these methods is given by Chui and Wang in chuidimred2015 . An excellent introduction to the subject of diffusion geometry can be found in the special issue achaspissue of Applied and Computational Harmonic Analysis, 2006. The application areas are too numerous to mention exhaustively. They include, for example, document analysis coifmanmauro2006 , face recognition niyogiface ; ageface2011 ; chuiwang2010 , hyperspectral imaging chuihyper , semi-supervised learning niyogi1 ; niyogi2 , image processing donoho2005image ; arjuna1 , cataloguing of galaxies donoho2002multiscale , and social networking bertozzicommunity .

A good deal of research in the theory of manifold learning deals with the problem of understanding the geometry of the data defined manifold. For example, it is shown in jones2010universal ; jones2008parameter that an atlas on the unknown manifold can be defined in terms of the heat kernel corresponding to the Laplace-Beltrami operator on the manifold. Other constructions of the atlas are given in chui_deep ; shaham2018provable ; schmidt2019deep with applications to the study of deep networks. Function approximation on manifolds based on scattered data (i.e., data points $x_{j}$ whose locations are not prescribed analytically) has been studied in detail in many papers, starting with mauropap , e.g., frankbern ; modlpmz ; eignet ; compbio ; heatkernframe ; mhaskar2020kernel . A theory was applied successfully in mhas_sergei_maryke_diabetes2017 to construct deep networks for predicting blood sugar levels based on continuous glucose monitoring devices.

A fundamental role in this theory is played by the heat kernel on the manifold corresponding to an appropriate elliptic partial differential operator. In coifmanmauro2006 ; heatkernframe , a muti-resolution analysis is constructed using the heat kernel. Another important tool is the theory of localized kernels based on the eigen-decomposition of the heat kernel. These were introduced in mauropap based on certain assumptions on the spectral function and the property of finite speed of wave propagation. In the context of manifolds, this later property was proved in sikora2004riesz ; frankbern to be equivalent to the so called Gaussian upper bounds on the heat kernels. Although such bounds are studied in many contexts by many authors, e.g., grigoryan1995upper ; grigor1997gaussian ; davies1990heat ; kordyukov1991p , we could not locate a reference where such a bound was proved for a general smooth manifold. We have therefore supplied a proof in mhaskar2020kernel . In [tauberian, , Theorem 4.3], we have proved a very general recipe that yields localized kernels based on the Gaussian upper bound on the heat kernel in what we have termed a data defined space (or data space in some other papers).

The problem of transfer learning (or meta-learning) involves learning the parameters of an approximation process based on one data set, and using this information to quickly learn the corresponding parameters on another data set, e.g., valeriyasmartphone ; maskey2023transferability ; maurer2013sparse . In the context of manifold learning, a data set (point cloud) determines a manifold, so that different data sets would correspond to different manifolds. In the context of data spaces, we can therefore interpret transfer learning as “lifting” a function from one data space (the base data space) to another (the target data space). This viewpoint allows us to unify the topic of transfer learning with the study of some inverse problems in image/signal processing. For example, the problem of synthetic aperture radar (SAR) imaging can be described in terms of an inverse Radon transform nolan2002synthetic ; cheney2009fundamentals ; munson1983tomographic . The domain and range of the Radon transform are different, and hence, the problem amounts to approximating the actual image on one domain based on observations of its Radon transform, which are located on a different domain. Another application is in analyzing hyperspectral images changing with time coifmanhirn . A similar problem arises in analyzing the progress of Alzheimer’s disease from MRI images of the brain taken over time, where one is interested in the development of the cortical thickness as a function on the surface of the brain, a manifold which is changing over time kim2014multi .

Motivated by these applications and the paper coifmanhirn of Coifman and Hirn, we studied in tauberian the question of lifting a function from one data space to another, when certain landmarks from one data space were identified with those on the other data space. For example, it is known lerch2005focal that in spite of the changing brain, one can think of each brain to be parametrized by an inner sphere, and the cortical thickness at certain standard points based on this parametrization are important in the prognosis of the disease. In tauberian we investigated certain conditions on the two data spaces which allow the lifting of a function from one to the other, and analyzed the effect on the smoothness of the function as it is lifted.

In many applications, the data about the function is available only on a part of the base data space. The novel part of this paper is to investigate the following questions of interest: (1) determine on what subsets of the target data space the lifting is defined, and (2) how the local smoothness on the base data space translates into the local smoothness of the lifted function. In limited angle tomography, one observes the Radon transform on a limited part of a cylinder and needs to reconstruct the image as a function on a ball from this data. A rudimentary introduction to the subject is given in the book natterer2001mathematics of Natterer. We do not aim to solve the limited angle tomography problem itself, but we will study in detail an example motivated by the singular value decomposition of the Radon transform, which involves two different systems of orthogonal polynomials on the interval $[-1,1]$. The theory of transplantation theorems muckenhoupt1986transplantation deals with the following problem. We are given the coefficients in the expansion of a function $f$ on $[-1,1]$ in terms of Jacobi polynomials with certain parameters (the base space expansion in our language), and use them as the coefficients in an expansion in terms of Jacobi polynomials with respect to a different set of parameters (the target space in our language). Under what conditions on $f$ and the parameters of the two Jacobi polynomial systems will the expansion in the target space converge and in which $L^{p}$ spaces? While old fashioned, the topic appears to be of recent interest diaz2021discrete ; arenas2019weighted . We will illustrate our general theory by obtaining a localized transplantation theorem for uniform approximation.

In Section 2, we review certain important results in the context of a single data space (our abstraction of a manifold). In particular, we present a characterization of local approximation of functions on such spaces. In Section 3, we review the notion of joint spaces (introduced under a different name in tauberian ). The main new result of our paper is to study the lifting of a function from a subset (typically, a ball) on one data space to another. These results are discussed in Section 4. The proofs are given in Section 5. An essential ingredient in our constructions is the notion of localized kernels which, in turn, depend upon a Tauberian theorem. For the convenience of the reader, this theorem is presented in Appendix 5.1. Appendix 5.2 lists some important properties of Jacobi polynomials which are required in our examples.

As mentioned in the introduction, a good deal of research on manifold learning is devoted to the question of learning the geometry of the manifold. For the purpose of harmonic analysis and approximation theory on the manifold, we do not need the full strength of the differentiability structure on the manifold. Our own understanding of the correct hypotheses required to study these questions has evolved, resulting in a plethora of terminology such as data defined manifolds, admissible systems, data defined spaces, etc., culminating in our current understanding with the definition of a data space given in mhaskar2020kernel . For the sake of simplicity, we will restrict our attention in this paper to the case of compact spaces. We do not expect any serious problems in extending the theory to the general case, except for a great deal of technical details.

Thus, the set up is the following.

We consider a compact metric measure space ${\mathbb{X}}$ with metric $d$ and a probability measure $\mu^{*}$. We take $\{\lambda_{k}\}_{k=0}^{\infty}$ to be a non-decreasing sequence of real numbers with $\lambda_{0}=0$ and $\lambda_{k}\to\infty$ as $k\to\infty$, and $\{\phi_{k}\}_{k=0}^{\infty}$ to be an orthonormal set in $L^{2}(\mu^{*})$. We assume that each $\phi_{k}$ is continuous. The elements of the space

$$\Pi_{n}=\mathsf{span}\{\phi_{k}:\lambda_{k}<n\}$$

(1)

are called diffusion polynomials (of order $<n$). We write $\displaystyle\Pi_{\infty}=\bigcup_{n>0}\Pi_{n}$. We introduce the following notation.

$${\mathbb{B}}(x,r)=\{y\in{\mathbb{X}}:d(x,y)\leq r\},\qquad x\in{\mathbb{X}},\ r>0.$$

(2)

If $A\subseteq{\mathbb{X}}$ we define

$${\mathbb{B}}(A,r)=\bigcup_{x\in A}{\mathbb{B}}(x,r).$$

(3)

With this set up, the definition of a compact data space is the following.

The primary example of a data space is, of course, a Riemannian manifold.

{uda}

Let ${\mathbb{X}}$ be a smooth, compact, connected Riemannian manifold (without boundary), $d$ be the geodesic distance on ${\mathbb{X}}$, $\mu^{*}$ be the Riemannian volume measure normalized to be a probability measure, $\{\lambda_{k}\}$ be the sequence of eigenvalues of the (negative) Laplace-Beltrami operator on ${\mathbb{X}}$, and $\phi_{k}$ be the eigenfunction corresponding to the eigenvalue $\lambda_{k}$; in particular, $\phi_{0}\equiv 1$. We have proved in [mhaskar2020kernel, , Appendix A] that the Gaussian upper bound is satisfied. Therefore, if the condition in Equation (4) is satisfied, then $({\mathbb{X}},d,\mu^{*},\{\lambda_{k}\}_{k=0}^{\infty},\{\phi_{k}\}_{k=0}^{\infty})$ is a data space with exponent equal to the dimension of the manifold.

{uda}

In this example, we let $\mathbb{X}=[0,\pi]$ and for $\theta_{1},\theta_{2}\in\mathbb{X}$ we simply define the distance as

$$d(\theta_{1},\theta_{2})=\left|\theta_{1}-\theta_{2}\right|.$$

(6)

We will consider the so-called trigonometric functions nowak2011sharp

$$\phi_{n}^{(\alpha,\beta)}(\theta)=(1-\cos\theta)^{\alpha/2+1/4}(1+\cos\theta)^{\beta/2+1/4}p_{n}^{(\alpha,\beta)}(\cos\theta),$$

(7)

where $p_{n}^{(\alpha,\beta)}$ are orthonormalized Jacobi polynomials defined as in Appendix 5.2 and $\alpha,\beta\geq-1/2$. We define

$$d\mu^{*}(\theta)=\frac{1}{\pi}d\theta.$$

(8)

We see that a change of variables $x=\cos\theta$ in Equation (96) results in the following orthogonality condition

$$\int_{0}^{\pi}\phi_{n}^{(\alpha,\beta)}(\theta)\phi_{m}^{(\alpha,\beta)}(\theta)d\theta=\delta_{n,m}.$$

(9)

So our orthonormal set of functions with respect to $\mu^{*}$ will be $\{\sqrt{\pi}\phi_{n}^{(\alpha,\beta)}\}$. It was proven in nowak2011sharp that with

$$\lambda_{n}=n+\frac{\alpha+\beta+1}{2},$$

(10)

we have

$$\pi\sum_{n=0}^{\infty}\exp\left(-\lambda_{n}^{2}t\right)\phi_{n}^{(\alpha,\beta)}(\theta_{1})\phi_{n}^{(\alpha,\beta)}(\theta_{2})\lesssim t^{-1/2}\exp\left(-c\frac{d(\theta_{1},\theta_{2})^{2}}{t}\right),\quad\theta_{1},\theta_{2}\in\mathbb{X}.$$

(11)

In conclusion,

$$\Xi=(\mathbb{X},d,\mu^{*},\{\lambda_{n}\},\{\sqrt{\pi}\phi_{n}^{(\alpha,\beta)}\})$$

(12)

is a data space with exponent $1$.

The following example illustrates how a manifold with boundary can be transformed into a closed manifold as in Example 2. We will use the notation and facts from Appendix 5.2 without always referring to them explicitly. We adopt the notation

$$\SS^{q}=\{{\bf x}\in{\mathbb{R}}^{q+1}:|{\bf x}|=1\},\qquad\SS^{q}_{+}=\{{\bf x}\in\SS^{q}:x_{q+1}\geq 0\}.$$

(13)

{uda}

Let $\mu^{*}_{q}$ denote the volume measure of $\mathbb{S}^{q}$, normalized to be a probability measure. Let ${\mathbb{H}}_{n}^{q}$ be the space of the restrictions to $\SS^{q}$ of homogeneous harmonic polynomials of degree $n$ on $q+1$ variables, and $\{Y_{n,k}\}_{k}$ be an orthonormal (with respect to $\mu^{*}_{q}$) basis for ${\mathbb{H}}_{n}^{q}$. The polynomials $Y_{n,k}$ are eigenfunctions of the Laplace-Beltrami operator on the manifold $\SS^{q}$ with eigenvalues $n(n+q-1)$. The geodesic distance between $\xi,\eta\in\mathbb{S}^{q}$ is $\arccos(\xi\cdot\eta)$, so the Gaussian upper bound for manifolds takes the form

$$\sum_{n,k}\exp(-n(n+q-1)t)Y_{n,k}({\bf x})\overline{Y_{n,k}({\bf y})}\lesssim t^{-q/2}\exp\left(-c\frac{(\arccos({\bf x}\cdot{\bf y}))^{2}}{t}\right).$$

(14)

As a result, $(\mathbb{S}^{q},\arccos(\circ\cdot\circ),\mu^{*}_{q},\{\lambda_{n}\}_{n},\{Y_{n,k}\}_{n,k})$ is a data space with dimension $q$.

Now we consider

$${\mathbb{X}}={\mathbb{B}}^{q}=\{{\bf x}\in{\mathbb{R}}^{q}:|{\bf x}|\leq 1\}.$$

We can identify ${\mathbb{B}}^{q}$ with $\SS^{q}_{+}$ as follows. Any point ${\bf x}\in{\mathbb{B}}^{q}$ has the form ${\bf x}=\omega\sin\theta$ for some $\omega\in\SS^{q-1}$, $\theta\in[0,\pi/2]$. We write $\hat{{\bf x}}=(\omega\sin\theta,\cos\theta)\in\SS^{q}_{+}$. With this identification, $\SS^{q}_{+}$ is parameterized by ${\mathbb{B}}^{q}$ and we define

	$\displaystyle d\mu^{*}({\bf x})=d\mu^{*}_{q}(\hat{{\bf x}})=$	$\displaystyle\frac{\operatorname{Vol}(\mathbb{B}^{q})}{\operatorname{Vol}(\mathbb{S}^{q}_{+})}(1-|{\bf x}|^{2})^{-1/2}dm^{*}({\bf x})$		(15)
	$\displaystyle=$	$\displaystyle\frac{\Gamma((q+1)/2)}{\sqrt{\pi}\Gamma(q/2+1)}(1-|{\bf x}|^{2})^{-1/2}dm^{*}({\bf x}),$

where $\mu^{*}_{q}$ is the probability volume measure on $\mathbb{S}^{q}_{+}$, and $m^{*}$ is the probability volume measure on $\mathbb{B}^{q}$. It is also convenient to define the distance on ${\mathbb{B}}^{q}$ by

$$d({\bf x}_{1},{\bf x}_{2})=\arccos(\hat{{\bf x}}_{1}\cdot\hat{{\bf x}}_{2})=\arccos({\bf x}_{1}\cdot{\bf x}_{2}+\sqrt{1-|{\bf x}_{1}|^{2}}\sqrt{1-|{\bf x}_{2}|^{2}}).$$

(16)

All spherical harmonics of degree $2n$ are even functions on $\mathbb{S}^{q}$. So with the identification of measures as above, one can represent the even spherical harmonics as an orthonormal system of functions on $\mathbb{B}^{q}$. That is, by defining

$$P_{2n,k}({\bf x})=\sqrt{2}Y_{2n,k}(\hat{{\bf x}}),$$

(17)

we have

	$\displaystyle\int_{{\mathbb{B}}^{q}}P_{2n,k}({\bf x})\overline{P_{2n^{\prime},k^{\prime}}({\bf x})}d\mu^{*}({\bf x})=$	$\displaystyle 2\int_{\mathbb{S}_{+}^{q}}Y_{2n,k}(\hat{\bf x})\overline{Y_{2n^{\prime},k^{\prime}}(\hat{\bf x})}d\mu^{*}_{q}(\hat{\bf x})$		(18)
	$\displaystyle=$	$\displaystyle\int_{\mathbb{S}^{q}}Y_{2n,k}(\xi)\overline{Y_{2n^{\prime},k^{\prime}}(\xi)}d\mu^{*}_{q}(\xi)$
	$\displaystyle=$	$\displaystyle\delta_{(n,k),(n^{\prime},k^{\prime})}.$

To show the Gaussian upper bound for $Y_{2n,k}$ on $\mathbb{B}^{q}$, we first see that in view of the addition formula (101) and (98), we deduce

		$\displaystyle\sum_{k=1}^{\operatorname{dim}(\mathbb{H}_{2n}^{q})}P_{2n,k}({\bf x})\overline{P_{2n,k}({\bf y})}$		(19)
	$\displaystyle=$	$\displaystyle\sum_{k=1}^{\operatorname{dim}(\mathbb{H}_{2n}^{q})}Y_{2n,k}(\hat{\bf x})\overline{Y_{2n,k}(\hat{\bf y})}$
	$\displaystyle=$	$\displaystyle\frac{\omega_{q}}{\omega_{q-1}}p_{2n}^{(q/2-1,q/2-1)}(1)p_{2n}^{(q/2-1,q/2-1)}(\hat{\bf x}\cdot\hat{\bf y})$
	$\displaystyle=$	$\displaystyle\frac{\omega_{q}}{\omega_{q-1}}2^{(q-1)/2}p_{n}^{(q/2-1,-1/2)}(1)p_{n}^{(q/2-1,-1/2)}(\cos(2\arccos(\hat{\bf x}\cdot\hat{\bf y}))).$

In light of Equation (95) we define

$$\lambda_{n}=\sqrt{n(n+q/2-1/2)},$$

(20)

which is conveniently not dependent upon $k$. Using (LABEL:eq:specialjacobigauss), we see that for $t>0$

		$\displaystyle\sum_{n=0}^{\infty}\sum_{k=1}^{\operatorname{dim}(\mathbb{H}_{2n}^{q})}\exp\left(-\lambda_{n}^{2}t\right)P_{2n,k}(\mathbf{x})\overline{P_{2n,k}({\bf y})}$		(21)
	$\displaystyle\sim$	$\displaystyle\sum_{n=0}^{\infty}\exp(-n(n+q/2-1/2)t)p_{n}^{(q/2-1,-1/2)}(1)p_{n}^{(q/2-1,-1/2)}(\cos(2\arccos(\hat{\bf x}\cdot\hat{\bf y})))$
	$\displaystyle\lesssim$	$\displaystyle t^{-q/2}\left(-4c\frac{\arccos(\hat{{\bf x}}_{1}\cdot\hat{{\bf x}}_{2})^{2}}{t}\right).$

Therefore, $(\mathbb{B}^{q},d,\mu^{*},\{\lambda_{n}\}_{n},\{P_{2n,k}\}_{n,k})$ is a data space with exponent $q$.

In this section, we will assume $\Xi$ to be a fixed data space and omit its mention from the notations. We will mention it later in other parts of the paper in order to avoid confusion. Next, we define smoothness classes of functions on ${\mathbb{X}}$. In the absence of any differentiability structure, we do this in a manner that is customary in approximation theory. We define first the degree of approximation of a function $f\in L^{p}(\mu^{*})$ by

$$E_{n}(p,f)=\min_{P\in\Pi_{n}}\|f-P\|_{p,\mu^{*}},\qquad n>0,1\leq p\leq\infty,\ f\in L^{p}(\mu^{*}).$$

(22)

We find it convenient to denote by $X^{p}$ the space $\{f\in L^{p}(\mu^{*}):\displaystyle\lim_{n\to\infty}E_{n}(p,f)=0\}$; e.g., in the manifold case, $X^{p}=L^{p}(\mu^{*})$ if $1\leq p<\infty$ and $X^{\infty}=C({\mathbb{X}})$. In the case of Example 2, we need to restrict ourselves to even functions.

A central theme in approximation theory is to characterize the smoothness spaces $W_{\gamma,p}$ in terms of the degree of approximation from some spaces; in our case we consider $\Pi_{n}$’s.

For this purpose, we define some localized kernels and operators.

The kernels are defined by

$$\Phi_{n}(H;x,y)=\sum_{m=0}^{\infty}H\left(\frac{\lambda_{m}}{n}\right)\phi_{m}(x)\phi_{m}(y),$$

(25)

where $H:{\mathbb{R}}\to{\mathbb{R}}$ is a compactly supported function.

The operators corresponding to the kernels $\Phi_{n}$ are defined by

$$\sigma_{n}(H;f,x)=\int_{\mathbb{X}}\Phi_{n}(H;x,y)f(y)d\mu^{*}(y)=\sum_{k:\lambda_{k}<n}H\left(\frac{\lambda_{k}}{n}\right)\hat{f}(k)\phi_{k}(x),$$

(26)

where

$$\hat{f}(k)=\int_{\mathbb{X}}f(y)\phi_{k}(y)d\mu^{*}(y).$$

(27)

The following proposition recalls an important property of these kernels. Proposition 2.4 is proved in mauropap , and more recently in much greater generality in [tauberian, , Theorem 4.3].

In the remainder of this paper, we fix a filter $h$; i.e., an infinitely differentiable function $h:[0,\infty)\to[0,1]$, such that $h(t)=1$ for $0\leq t\leq 1/2$, $h(t)=0$ for $t\geq 1$. The domain of the filter $h$ can be extended to ${\mathbb{R}}$ by setting $h(-t)=h(t)$. Since $h$ is fixed, its mention will be omitted from the notation unless we feel that this would cause a confusion. The following theorem gives a crucial property of the operators, proved in several papers of ours in different contexts, see mhaskar2020kernel for a recent proof.

While Theorem 2.5 gives, in particular, a characterization of the global smoothness spaces $W_{\gamma,p}$, the characterization of local smoothness requires two more assumptions: the partition of unity and product assumption.

In the most important manifold case, the partition of unity assumption is always satisfied [docarmo_riemannian, , Chapter 0, Theorem 5.6]. It is shown in geller2011band ; modlpmz that the strong product assumption is satisfied if $\phi_{k}$’s are eigenfunctions of certain differential equations on a Riemannian manifold and the $\lambda_{k}$’s are the corresponding eigenvalues. We do not know of any example where this property does not hold, yet cannot prove that it holds in general. Hence, we have listed it as an assumption.

Our characterization of local smoothness (compbio ; heatkernframe ; mhaskar2020kernel ) is the following.

A direct corollary is the following.

In order to motivate our definitions in this section, we first consider a couple of examples.

{uda}

Let $\Xi_{j}=({\mathbb{X}}_{j},d_{j},\mu_{j}^{*},\{\lambda_{j,k}\}_{k=0}^{\infty},\{\phi_{j,k}\}_{k=0}^{\infty})$, $j=1,2$ be two data spaces with exponent $q$. We denote the heat kernel in each case by

$$K_{j,t}(x,y)=\sum_{k=0}^{\infty}\exp(-\lambda_{j,k}^{2}t)\phi_{j,k}(x)\phi_{j,k}(y),\qquad j=1,2,\ x,y\in{\mathbb{X}},\ t>0,$$

In the paper coifmanhirn , Coifman and Hirn assumed that ${\mathbb{X}}_{1}={\mathbb{X}}_{2}={\mathbb{X}}$, $\mu_{1}^{*}=\mu_{2}^{*}=\mu^{*}$, and proposed the diffusion distance between points $x_{1},x_{2}$ to be the square root of

$$K_{1,2t}(x_{1},x_{2})+K_{2,2t}(x_{2},x_{2})-2\int_{\mathbb{X}}K_{1,t}(x_{1},y)K_{2,t}(y,x_{2})d\mu^{*}(y).$$

Writing, in this example only,

$$A_{j,k}=\int_{\mathbb{X}}\phi_{1,j}(y)\phi_{2,k}(y)d\mu^{*}(y),$$

(35)

we get

$$\int_{\mathbb{X}}K_{1,t}(x_{1},y)K_{2,t}(y,x_{2})d\mu^{*}(y)=\sum_{j,k}\exp\left(-(\lambda_{1,j}^{2}+\lambda_{2,k}^{2})t\right)A_{j,k}\phi_{1,j}(x_{1})\phi_{2,k}(x_{2}).$$

(36)

Furthermore, the Gaussian upper bound conditions imply that

	$\displaystyle\int_{\mathbb{X}}K_{1,t}(x_{1},y)K_{2,t}(y,x_{2})d\mu^{*}(y)$	$\displaystyle\lesssim t^{-q}\int_{\mathbb{X}}\exp\left(-c\frac{d_{1}(x_{1},y)^{2}+d_{2}(y,x_{2})^{2}}{t}\right)d\mu^{*}(y)$		(37)
		$\displaystyle\lesssim t^{-q}\exp\left(-c\frac{\left(\min_{y\in{\mathbb{X}}}\left(d_{1}(x_{1},y)+d_{2}(y,x_{2})\right)\right)^{2}}{t}\right).$

Writing, in this example only,

$$d_{1,2}(x_{1},x_{2})=\min_{y\in{\mathbb{X}}}\left(d_{1}(x_{1},y)+d_{2}(y,x_{2})\right)=d_{2,1}(x_{2},x_{1}),$$

we observe that for any $x_{1},x_{1}^{\prime},x_{2},x_{2}^{\prime}\in{\mathbb{X}}$,

	$\displaystyle d_{1,2}(x_{1},x_{2})$	$\displaystyle\leq d_{1,2}(x_{1}^{\prime},x_{2})+d_{1}(x_{1},x_{1}^{\prime}),$
	$\displaystyle d_{1,2}(x_{1},x_{2})$	$\displaystyle\leq d_{1,2}(x_{1},x_{2}^{\prime})+d_{1}(x_{2},x_{2}^{\prime}).$

{uda}

In this example we let $\alpha_{i},\beta_{i}\geq-1/2$ for $i=1,2$ and assume that $a=\left|\alpha_{1}-\alpha_{2}\right|/2,b=\left|\beta_{1}-\beta_{2}\right|/2\in\mathbb{N}$. Then we select the following two data spaces as defined in Example 2

$$\Xi_{i}=([0,\pi],d_{i},\frac{1}{\pi}d\theta,\{\lambda_{i,n}\},\{\sqrt{\pi}\phi^{(\alpha_{i},\beta_{i})}_{n}\}).$$

(38)

Since both spaces already have the same distance, we will define a joint distance for the systems accordingly:

$$d_{1,2}(\theta_{1},\theta_{2})=d_{1}(\theta_{1},\theta_{2})=d_{2}(\theta_{1},\theta_{2})=|\theta_{1}-\theta_{2}|.$$

(39)

Similar to Example 3 above, we are considering two data spaces with the same underlying space and measure. However, we now proceed in a different manner. Let us denote

$$\varOmega(\theta)=(1-\cos\theta)^{a}(1+\cos\theta)^{b}.$$

(40)

Let $\overline{\alpha}=\max(\alpha_{1},\alpha_{2})$ and $\overline{\beta}=\max(\beta_{1},\beta_{2})$. Then we define

	$\displaystyle A_{m,n}=$	$\displaystyle\int_{0}^{\pi}\phi_{m}^{(\alpha_{1},\beta_{1})}(\theta)\phi_{n}^{(\alpha_{2},\beta_{2})}(\theta)\varOmega(\theta)d\theta$		(41)
	$\displaystyle=$	$\displaystyle\int_{0}^{\pi}p_{m}^{(\alpha_{1},\beta_{1})}(\cos\theta)p_{n}^{(\alpha_{2},\beta_{2})}(\cos\theta)(1-\cos\theta)^{\overline{\alpha}+1/2}(1+\cos\theta)^{\overline{\beta}+1/2}d\theta.$

The orthogonality of the Jacobi polynomials tells us that $A_{m,n}=0$ at least when $m>n+2a+2b$ or $n>m+2a+2b$. Furthermore, we have the following two sums

$$\sum_{n}A_{m,n}\phi_{n}^{(\alpha_{2},\beta_{2})}(\theta)=\varOmega(\theta)\phi_{m}^{(\alpha_{1},\beta_{1})}(\theta),\hskip 5.0pt\sum_{m}A_{m,n}\phi_{m}^{(\alpha_{1},\beta_{1})}(\theta)=\varOmega(\theta)\phi_{n}^{(\alpha_{2},\beta_{2})}(\theta).$$

(42)

We define $\ell_{m.n}=\sqrt{\lambda_{1,m}^{2}+\lambda_{2,n}^{2}}$, utilize the Gaussian upper bound property for $\Xi_{i}$ and Equation (42) to deduce as in Example 3 that

		$\displaystyle\left|\pi\sum_{m,n}\exp\left(-\ell_{m,n}^{2}t\right)A_{m,n}\phi_{m}^{(\alpha_{1},\beta_{1})}(\theta_{1})\phi_{n}^{(\alpha_{2},\beta_{2})}(\theta_{2})\right|$		(43)
	$\displaystyle=$	$\displaystyle\left|\int_{0}^{\pi}K_{1,t}(\theta_{1},\phi)K_{2,t}(\phi,\theta_{2})\varOmega(\phi)d\phi\right|$
	$\displaystyle\lesssim$	$\displaystyle t^{-1}\exp\left(-c\frac{d_{1,2}(\theta_{1},\theta_{2})^{2}}{t}\right).$

We note (cf. [mhaskar2020kernel, , Lemma 5.2]) that

		$\displaystyle\pi\sum_{m,n:\ell_{m,n}<N}\left|A_{m,n}\phi_{m}^{(\alpha_{1},\beta_{1})}(\theta_{1})\phi_{n}^{(\alpha_{2},\beta_{2})}(\theta_{2})\right|$		(44)
	$\displaystyle\lesssim$	$\displaystyle\left|\left|\varOmega\right|\right|_{[0,\pi]}\sum_{m:\lambda_{1,m}<N}\left|\phi_{m}^{(\alpha_{1},\beta_{1})}(\theta_{1})\phi_{m}^{(\alpha_{1},\beta_{1})}(\theta_{2})\right|$
	$\displaystyle\lesssim$	$\displaystyle N.$

Motivated by these examples, we now give a series of definitions, culminating in Definition 3.3. First, we define the notion of a joint distance.

For convenience of notation we denote $d_{2,1}(x_{2},x_{1})=d_{1,2}(x_{1},x_{2})$. Then for $r>0$, $x_{1}\in\mathbb{X}_{1}$, $x_{2}\in\mathbb{X}_{2}$, $A_{1}\subset{\mathbb{X}}_{1}$, $A_{2}\subset{\mathbb{X}}_{2}$, we define

	$\displaystyle\mathbb{B}_{1}(x_{1},r)$	$\displaystyle=\{z\in\mathbb{X}_{1}:d_{1}(x_{1},z)\leq r\},$	$\displaystyle\mathbb{B}_{2}(x_{2},r)$	$\displaystyle=\{z\in\mathbb{X}_{2}:d_{2}(x_{2},z)\leq r\},$
	$\displaystyle\mathbb{B}_{1,2}(x_{1},r)$	$\displaystyle=\{z\in\mathbb{X}_{2}:d_{1,2}(x_{1},z)\leq r\},$	$\displaystyle\mathbb{B}_{2,1}(x_{2},r)$	$\displaystyle=\{z\in\mathbb{X}_{1}:d_{2,1}(x_{2},z)\leq r\},$

	$\displaystyle d_{1,2}(A_{1},x_{2})=$	$\displaystyle\inf_{x\in A_{1}\subseteq\mathbb{X}_{1}}d_{1,2}(x,x_{2}),$		(46)
	$\displaystyle d_{1,2}(x_{1},A_{2})=d_{2,1}(A_{2},x_{1})=$	$\displaystyle\inf_{y\in A_{2}\subseteq\mathbb{X}_{2}}d_{2,1}(y,x_{1}).$

We recall here that an infimum over an empty set is defined to be $\infty$.