\SetKwComment

Comment/* */ \coltauthor\NameZihao Hu \Email[email protected]
\addrGeorgia Institute of Technology and \NameGuanghui Wang \Email[email protected]
\addrGeorgia Institute of Technology and \NameJacob Abernethy \Email[email protected]
\addrGeorgia Institute of Technology, Google Research

Minimizing Dynamic Regret on Geodesic Metric Spaces

Abstract

In this paper, we consider the sequential decision problem where the goal is to minimize the general dynamic regret on a complete Riemannian manifold. The task of offline optimization on such a domain, also known as a geodesic metric space, has recently received significant attention. The online setting has received significantly less attention, and it has remained an open question whether the body of results that hold in the Euclidean setting can be transplanted into the land of Riemannian manifolds where new challenges (e.g., curvature) come into play. In this paper, we show how to get optimistic regret bound on manifolds with non-positive curvature whenever improper learning is allowed and propose an array of adaptive no-regret algorithms. To the best of our knowledge, this is the first work that considers general dynamic regret and develops “optimistic” online learning algorithms which can be employed on geodesic metric spaces.

keywords:

Riemannian Manifolds, Optimistic Online Learning, Dynamic Regret

1 Introduction

Online convex optimization (OCO) in Euclidean space is a well-developed area with numerous applications. In each round, the learner takes an action from a decision set, while the adversary chooses a loss function. The long-term performance metric of the learner is (static) regret, which is defined as the difference between the learner’s cumulative loss and the loss of playing the best-fixed decision in hindsight. As the name suggests, OCO requires both the losses and the decision set to be convex. From the theoretical perspective, convex functions and sets are well-behaved objects with many desirable properties that are generally required to obtain tight regret bounds.

Typical algorithms in OCO, such as mirror descent, determine how one should adjust parameter estimates in response to arriving data, typically by shifting parameters against the gradient of the loss. But in many cases of interest, the underlying parameter space is not only non-convex but non-Euclidean. The hyperboloid, for example, arising from the solution set of a degree-two polynomial, is a Riemannian manifold that has garnered interest as a tool in tree-embedding tasks (Lou et al., 2020). On such manifolds, we do have a generalized notion of convexity, known as geodesic convexity (Udriste, 2013). There are many popular problems of interest (Hosseini and Sra, 2015; Vishnoi, 2018; Sra et al., 2018) where the underlying objective function is geodesically convex (gsc-convex) under a suitable Riemannian metric. But there has thus far been significantly limited research on how to do adaptive learning in such spaces and to understand when regret bounds are obtainable.

Table 1: Summary of bounds.

\delta

describes the discrepancy between the decision set and the comparator set. We define

\zeta

in Def. 1 and let

B_{T}\coloneqq\min\{V_{T},F_{T}\}

Algorithm	Type	Dynamic regret
Radar	Standard	$O(\sqrt{{\zeta}(1+P_{T})T})$
	Lower bound	$\Omega(\sqrt{(1+P_{T})T})$
Radar_v	Gradient-variation	$O(\sqrt{{\zeta}(\frac{1+P_{T}}{\delta^{2}}+V_{T})(P_{T}+1)})$
Radar_s	Small-loss	$O(\sqrt{{\zeta}((1+P_{T}){\zeta}+F_{T})(P_{T}+1)})$
Radar_b	Best-of-both-worlds	$O\left(\sqrt{\zeta(P_{T}(\zeta+\frac{1}{\delta^{2}})+B_{T}+1)(P_{T}+1)+B_{T}\ln T}\right)$

Let $\mathcal{N}$ be a gsc-convex subset of a geodesic metric space $\mathcal{M}$ . In this paper, we consider the problem of minimizing the general dynamic regret on $\mathcal{N}$ , defined as

\text{D-Regret}_{T}:=\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t}),

where $\mathbf{x}_{1},\ldots,\mathbf{x}_{T}\in\mathcal{N}$ is the sequence of actions taken by the learner, whose loss is evaluated relative to the sequence of “comparator” points $\mathbf{u}_{1},\dots,\mathbf{u}_{T}\in\mathcal{N}$ . There has been recent work establishing that sublinear regret is possible as long as $\mathcal{N}$ and the $f_{t}$ ’s are gsc-convex, for example using a Riemannian variant of Online Gradient Descent (Wang et al., 2021). But so far there are no such results that elicit better D-Regret using adaptive algorithms.

What do we mean by “adaptive” in this context? In the online learning literature there have emerged three key quantities of interest in the context of sequential decision making, the comparator path length, the gradient variation, and the comparator loss, defined respectively as:

$\displaystyle\textstyle P_{T}$	$\displaystyle:=$	$\displaystyle\textstyle\sum_{t=2}^{T}d(\mathbf{u}_{t},\mathbf{u}_{t-1}),$	(1)
$\displaystyle\textstyle V_{T}$	$\displaystyle:=$	$\displaystyle\textstyle\sum_{t=2}^{T}\sup_{\mathbf{x}\in\mathcal{N}}\\|\nabla f_{t-1}(\mathbf{x})-\nabla f_{t}(\mathbf{x})\\|^{2},$
$\displaystyle\textstyle F_{T}$	$\displaystyle:=$	$\displaystyle\textstyle\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t}).$

Let us start by considering regret minimization with respect to path length. While it has been observed that $O(\sqrt{(1+P_{T})T})$ is optimal in a minimax sense (Zhang et al., 2018), a great deal of research for the Euclidean setting (Srebro et al., 2010; Chiang et al., 2012; Rakhlin and Sridharan, 2013) has shown that significantly smaller regret is achievable when any one of the above quantities is small. These are not just cosmetic improvements either, many fundamental applications of online learning rely on these adaptive methods and bounds. We give a thorough overview in Section 3.

The goal of the present paper is to translate to the Riemannian setting an array of adaptive regret algorithms and prove corresponding bounds. We propose a family of algorithms which we call Radar, for Riemannian adaptive dynamic regret. The three important variants of Radar are Radar_v, Radar_s, and Radar_b, we prove regret bounds on each, summarized in Table 1. We allow improper learning for the gradient-variation bound, which means the player can choose $\mathbf{x}_{1},\dots,\mathbf{x}_{T}$ from a slightly larger set $\mathcal{N}_{\delta G}$ (formally defined in Definition 4).

As a general matter, convex constraints on a Riemannian manifold introduce new difficulties in optimization that are not typically present in the Euclidean case, and there has been limited work on addressing these. To the best of our knowledge, there are only three papers considering how to incorporate constraints on manifolds, and these all make further assumptions on either the curvature or the diameter of the feasible set. Martínez-Rubio (2022) only applies to hyperbolic and spherical spaces. Criscitiello and Boumal (2022) works for complete Riemannian manifolds with sectional curvature in $[-K,K]$ but the diameter of the decision set is at most $\textstyle O(\frac{1}{\sqrt{K}})$ . Martínez-Rubio and Pokutta (2022) mainly works for locally symmetric Hadamard manifolds. Our paper is the first to consider the projective distortion in the online setting that applies to all Hadamard manifolds as long as improper learning is allowed without imposing further constraints on the diameter or the curvature.

Obtaining adaptive regret guarantees in the Riemannian setting is by no means a trivial task, as the new geometry introduces various additional technical challenges. Here is but one example: whereas the cost of a (Bregman) projection into a feasible region can be controlled using a generalized “Pythagorean” theorem in the Euclidean setting, this same issue becomes more difficult on a manifold as we encounter geometric distortion due to curvature. To better appreciate this, for a Hadamard manifold $\mathcal{M}$ , assume the projection of $\mathbf{x}\in\mathcal{M}$ onto a convex subset $\mathcal{N}\subset\mathcal{M}$ is ${\mathbf{z}}$ . While it is true that for any $\mathbf{y}\in\mathcal{N}\setminus\{\mathbf{z}\}$ the angle between geodesics $\overline{\mathbf{z}\mathbf{x}}$ and $\overline{\mathbf{z}\mathbf{y}}$ is obtuse, this property is only relevant at the tangent space of $\mathbf{z}$ , yet we need to analyze gradients at the tangent space of ${\mathbf{x}}$ . The use of parallel transport between $\mathbf{x}$ and $\mathbf{z}$ unavoidably incurs extra distortion and could potentially lead to $O(T)$ regret.

The last challenge comes from averaging on manifolds. For example, many adaptive OCO algorithms rely on the meta-expert framework, described by Van Erven and Koolen (2016), that runs several learning algorithms in parallel and combines them through appropriately-weighted averaging. There is not, unfortunately, a single way to take convex combinations in a geodesic metric space, and all such averaging schemes need to account for the curvature of the manifold and incorporate the associated costs. We finally find the Fréchet mean to be a desirable choice, but the analysis must proceed with care.

The key contributions of our work can be summarized as follows:

•

We develop the optimistic mirror descent (OMD) algorithm on Hadamard manifolds¹¹1We focus on Hadamard manifolds in the main paper and extend the guarantee to CAT $(\kappa)$ spaces in Appendix B.2. in the online improper learning setting. Interestingly, we also show Optimistic Hedge, a variant of OMD, works for gsc-convex losses. We believe these tools may have significant applications to research in online learning and Riemannian optimization.
•

We combine our analysis on OMD with the meta-expert framework (Van Erven and Koolen, 2016) to get several adaptive regret bounds, as shown in Table 1.
•

We develop a novel dynamic regret lower bound, which renders our $O(\sqrt{\zeta(1+P_{T})T})$ bound to be tight up to the geometric constant $\zeta$ .

2 Preliminaries

In this section, we introduce background knowledge of OCO and Riemannian manifolds.

OCO in Euclidean space.

We first formally describe OCO in Euclidean space. For each round $t=1,\dots,T$ , the learner makes a decision $\mathbf{x}_{t}\in\mathcal{X}$ based on historical losses $f_{1},\dots,f_{t-1}$ where $\mathcal{X}$ is a convex decision set, and then the adversary reveals a convex loss function $f_{t}$ . The goal of the learner is to minimize the difference between the cumulative loss and that of the best-fixed decision in hindsight: $\text{Regret}_{T}=\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\min_{\mathbf{x}\in\mathcal{X}}\sum_{t=1}^{T}f_{t}(\mathbf{x}),$ which is usually referred to as the static regret, since the comparator is a fixed decision.

In the literature, there exist a large number of algorithms (Hazan et al., 2016) on minimizing the static regret. However, the underlying assumption of the static regret is the adversary’s behavior does not change drastically, which can be unrealistic in real applications. To resolve this issue, dynamic regret stands out, which is defined as (Zinkevich, 2003)

\text{Regret}_{T}(\mathbf{u}_{1},\dots,\mathbf{u}_{T})=\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t}),

where $\mathbf{u}_{1},\dots,\mathbf{u}_{T}\in\mathcal{X}$ is a comparator sequence. Dynamic regret receives considerable attention recently (Besbes et al., 2015; Jadbabaie et al., 2015; Mokhtari et al., 2016; Zhang et al., 2017, 2018; Zhao et al., 2020; Wan et al., 2021; Zhao and Zhang, 2021; Baby and Wang, 2021) due to its flexibility. However, dynamic regret can be as large as $O(T)$ in general. Thus, regularizations need to be imposed on the comparator sequence to ensure no-regret online learning. A common assumption (Zinkevich, 2003) is the path-length (see Equation (1)) of the comparator sequence to be bounded. We refer to the corresponding dynamic regret as general dynamic regret because any assignments to $\mathbf{u}_{1},\dots,\mathbf{u}_{T}$ subject to the path-length constraint are feasible.

Riemannian manifolds. Here, we give a brief overview of Riemannian geometry, but this is a complex subject, and we refer the reader to, e.g., Petersen (2006) for a full treatment. A Riemannian manifold $(\mathcal{M},g)$ is a smooth manifold $\mathcal{M}$ equipped with a Riemannian metric $g$ . The tangent space $T_{\mathbf{x}}\mathcal{M}\cong\mathbb{R}^{d}$ , generalizing the concept of the tangent plane, contains vectors tangent to any curve passing $\mathbf{x}$ . The Riemannian metric $g$ induces the inner product $\langle\mathbf{u},\mathbf{v}\rangle_{\mathbf{x}}$ and the Riemannian norm $\|\mathbf{u}\|_{\mathbf{x}}=\sqrt{\langle\mathbf{u},\mathbf{u}\rangle_{\mathbf{x}}}$ where $\mathbf{u},\mathbf{v}\in T_{\mathbf{x}}\mathcal{M}$ (we omit the reference point $\mathbf{x}$ when it is clear from the context). We use $d(\mathbf{x},\mathbf{y})$ to denote the Riemannian distance between $\mathbf{x},\mathbf{y}\in\mathcal{M}$ , which is the greatest lower bound of the length of all piecewise smooth curves joining $\mathbf{x}$ and $\mathbf{y}$ .

A curve connecting $\mathbf{x},\mathbf{y}\in\mathcal{M}$ is a geodesic if it is locally length-minimizing. For two points $\mathbf{x},\mathbf{y}\in\mathcal{M}$ , suppose there exists a geodesic $\gamma(t):[0,1]\rightarrow\mathcal{M}$ such that $\gamma(0)=\mathbf{x},\gamma(1)=\mathbf{y}$ and $\gamma^{\prime}(0)=\mathbf{v}\in T_{\mathbf{x}}\mathcal{M}$ . The exponential map ${\textnormal{Exp}}_{\mathbf{x}}(\cdot):T_{\mathbf{x}}\mathcal{M}\rightarrow\mathcal{M}$ maps $\mathbf{v}\in T_{\mathbf{x}}\mathcal{M}$ to $\mathbf{y}\in\mathcal{M}$ . Correspondingly, the inverse exponential map ${\textnormal{Exp}}_{\mathbf{x}}^{-1}(\cdot):\mathcal{M}\rightarrow T_{\mathbf{x}}\mathcal{M}$ maps $\mathbf{y}\in\mathcal{M}$ to $\mathbf{v}\in T_{\mathbf{x}}\mathcal{M}$ . Since traveling along a geodesic is of constant velocity, we indeed have $d(\mathbf{x},\mathbf{y})=\|{\textnormal{Exp}}_{\mathbf{x}}^{-1}\mathbf{y}\|_{\mathbf{x}}$ . Also, it is useful to compare tangent vectors in different tangent spaces. Parallel transport $\Gamma_{\mathbf{x}}^{\mathbf{y}}\mathbf{u}$ translates $\mathbf{u}$ from $T_{\mathbf{x}}\mathcal{M}$ to $T_{\mathbf{y}}\mathcal{M}$ and preserves the inner product, i.e., $\langle\mathbf{u},\mathbf{v}\rangle_{\mathbf{x}}=\langle\Gamma_{\mathbf{x}}^{\mathbf{y}}\mathbf{u},\Gamma_{\mathbf{x}}^{\mathbf{y}}\mathbf{v}\rangle_{\mathbf{y}}$ .

The curvature of Riemannian manifolds reflects the extent to which the manifold differs from a Euclidean surface. For optimization purposes, it usually suffices to consider the sectional curvature. Following Zhang and Sra (2016); Wang et al. (2021), in this paper we mainly consider Hadamard manifolds, which are complete and single-connected manifolds with non-positive sectional curvature. On such manifolds, every two points are connected by a unique and distance-minimizing geodesic by Hopf-Rinow Theorem (Petersen, 2006).

A subset $\mathcal{N}$ of $\mathcal{M}$ is gsc-convex if for any $\mathbf{x},\mathbf{y}\in\mathcal{N}$ , there exists a geodesic connecting $\mathbf{x},\mathbf{y}$ and fully lies in $\mathcal{N}$ . A function $f:\mathcal{N}\rightarrow\mathbb{R}$ is gsc-convex if $\mathcal{N}$ is gsc-convex and the composition $f(\gamma(t))$ satisfies $f(\gamma(t))\leq(1-t)f(\gamma(0))+tf(\gamma(1))$ for any geodesic $\gamma(t)\subseteq\mathcal{N}$ and $t\in[0,1]$ . An alternative definition of geodesic convexity is

\textstyle f(\mathbf{y})\geq f(\mathbf{x})+\langle\nabla f(\mathbf{x}),{\textnormal{Exp}}_{\mathbf{x}}^{-1}\mathbf{y}\rangle,\qquad\forall\;\mathbf{x},\mathbf{y}\in\mathcal{N}.

where the Riemannian gradient $\nabla f(\mathbf{x})\in T_{\mathbf{x}}\mathcal{M}$ is the unique vector determined by $Df(\mathbf{x})[\mathbf{v}]=\langle\mathbf{v},\nabla f(\mathbf{x})\rangle$ and $Df(\mathbf{x})[\mathbf{v}]$ is the differential of $f$ along $\mathbf{v}\in T_{\mathbf{x}}\mathcal{M}$ .

Similarly, a $L$ -geodesically-smooth (L-gsc-smooth) function $f$ satisfies $\|\Gamma_{\mathbf{y}}^{\mathbf{x}}\nabla f(\mathbf{y})-\nabla f(\mathbf{x})\|\leq L\cdot d(\mathbf{x},\mathbf{y})$ for all $\mathbf{x},\mathbf{y}\in\mathcal{N}$ , or

\textstyle f(\mathbf{y})\leq f(\mathbf{x})+\langle\nabla f(\mathbf{x}),{\textnormal{Exp}}_{\mathbf{x}}^{-1}\mathbf{y}\rangle+\frac{L}{2}d(\mathbf{x},\mathbf{y})^{2}.

3 Related Work

In this part, we briefly review past work on OCO in Euclidean space, online optimization and optimism on Riemannian manifolds.

3.1 OCO in Euclidean Space

Static regret.

We first consider work on static regret. In Euclidean space, it is well known that online gradient descent (OGD) guarantees $O(\sqrt{T})$ and $O(\log T)$ regret for convex and strongly convex losses (Hazan et al., 2016), which are also minimax optimal (Abernethy et al., 2008). However, the aforementioned bounds are not fully adaptive due to the dependence on $T$ . Therefore, there is a tendency to replace $T$ with problem-dependent quantities. Srebro et al. (2010) first notice that the smooth and non-negative losses satisfy the self-bounding property, thus establishing the small-loss bound $O(\sqrt{F_{T}^{\star}})$ where $F_{T}^{\star}=\sum_{t=1}^{T}f_{t}(\mathbf{x}^{\star})$ is the cumulative loss of the best action in hindsight. Chiang et al. (2012) propose extra-gradient to get $O(\sqrt{V_{T}})$ gradient-variation regret bound for convex and smooth losses where $V_{T}=\sum_{t=2}^{T}\sup_{\mathbf{x}\in\mathcal{X}}\|\nabla f_{t-1}(\mathbf{x})-\nabla f_{t}(\mathbf{x})\|_{2}^{2}$ . Rakhlin and Sridharan (2013) generalize the work of Chiang et al. (2012) and propose optimistic mirror descent, which has become a standard tool in online learning since then.

Dynamic regret.

Now we switch to the related work on dynamic regret. Zinkevich (2003) propose to use OGD to get a $O\left(\eta T+\frac{1+P_{T}}{\eta}\right)$ regret bound where $\eta$ is the step size, but the result turns out to be $O((1+P_{T})\sqrt{T})$ since the value of $P_{T}$ is unknown to the learner. The seminal work of Zhang et al. (2018) use Hedge to combine the advice of experts with different step sizes, and show a $O\left(\sqrt{(1+P_{T})T}\right)$ regret. A matching lower bound is also established therein. Zhao et al. (2020) utilize smoothness to get a gradient-variation bound, a small-loss bound, and a best-of-both-worlds bound in Euclidean space.

3.2 Online Learning and Optimism on Riemannian Manifolds

In the online setting, Bécigneul and Ganea (2019) consider adaptive stochastic optimization on Riemannian manifolds but their results only apply to the Cartesian product of one-manifolds. Maass et al. (2022) study the restricted dynamic regret on Hadamard manifolds under the gradient-free setting and provide $O(\sqrt{T}+P_{T}^{\star})$ bound for gsc-strongly convex and gsc-smooth functions, where $P_{T}^{\star}$ is the path-length of the comparator formed by $\mathbf{u}_{t}=\operatorname*{argmin}_{\mathbf{x}\in\mathcal{X}}f_{t}(\mathbf{x})$ . On Hadamard manifolds, Wang et al. (2021) apply Riemannian OGD (R-OGD) to get $O(\sqrt{T})$ upper bound and $\Omega(\sqrt{T})$ randomized lower bound. Comparatively, we focus on general and adaptive dynamic regret on Hadamard manifolds. Our minimax lower bound is also novel.

There also exist algorithms considering optimism on Riemannian manifolds. Zhang et al. (2022) propose Riemannian Corrected Extra Gradient (RCEG) for unconstrained minimax optimization on manifolds. Karimi et al. (2022) consider a Robbins-Monro framework on Hadamard manifolds which subsumes Riemannian stochastic extra-gradient. By imposing the weakly asymptotically coercivity and using a decaying step size, the trajectory is guaranteed to be finite (Karimi et al., 2022). However, our paper is the first to consider the constrained case and the online setting. For the improper learning setting, we show a constant step size achieves the same guarantee as in Euclidean space.

4 Path Length Dynamic Regret Bound on Manifolds

In this section, we present the results related to the minimax path-length bound on manifolds. Before diving into the details, following previous work (Zinkevich, 2003; Wang et al., 2021), we introduce some standard assumptions and definitions.

Assumption 1

$\mathcal{M}$ is a Hadamard manifold and its sectional curvature is lower bounded by $\kappa\leq 0$ .

Assumption 2

The decision set $\mathcal{N}$ is a gsc-convex compact subset of $\mathcal{M}$ with diameter upper bounded by $D$ , i.e., $\sup_{\mathbf{x},\mathbf{y}\in\mathcal{N}}d(\mathbf{x},\mathbf{y})\leq D$ . For optimistic online learning, we allow the player chooses decisions from $\mathcal{N}_{\delta M}$ , which is defined in Definition 4 and the diameter becomes $(D+2\delta M)$ .

Assumption 3

The norm of Riemannian gradients are bounded by $G$ , i.e., $\sup_{\mathbf{x}\in\mathcal{N}}\|\nabla f_{t}(\mathbf{x})\|\leq G.$ When improper learning is allowed, we assume $\sup_{\mathbf{x}\in\mathcal{N}_{\delta M}}\|\nabla f_{t}(\mathbf{x})\|\leq G.$

Definition 1

Under Assumptions 1, 2, we denote ${\zeta}\coloneqq\sqrt{-\kappa}D\operatorname{coth}(\sqrt{-\kappa}D)$ . When improper learning is allowed, ${\zeta}\coloneqq\sqrt{-\kappa}(D+2\delta M)\operatorname{coth}(\sqrt{-\kappa}(D+2\delta M))$ , where $M$ is in Definition 4. Note that, on manifolds of zero sectional curvature ( $\kappa=0$ ), we have $\zeta=\lim_{x\to 0}x\cdot\coth{x}=1$ .

The seminal work of Zinkevich (2003) shows that the classical OGD algorithm can minimize the general dynamic regret in Euclidean space. Motivated by this, we consider the Riemannian OGD (R-OGD) algorithm (Wang et al., 2021):

\mathbf{x}_{t+1}=\Pi_{\mathcal{N}}{\textnormal{Exp}}_{\mathbf{x}_{t}}(-\eta\nabla f_{t}(\mathbf{x}_{t})),

(2)

which is a natural extension of OGD to the manifold setting. We show R-OGD can also minimizes the general dynamic regret on manifolds. Due to page limitation, we postpone details to Appendix A.

Theorem 1

Suppose Assumptions 1, 2 and 3 hold. Then the general dynamical regret of R-OGD defined in Equation (2) satisfies

\text{D-Regret}_{T}\leq\frac{D^{2}+2DP_{T}}{2\eta}+\frac{\eta{\zeta}G^{2}T}{2}.

(3)

Theorem 1 implies that R-OGD yields $O(\frac{P_{T}+1}{\eta}+\eta T)$ general dynamic regret bound, which means the optimal step size is $\eta=O{\scriptstyle\left(\sqrt{\frac{1+P_{T}}{T}}\right)}$ . However, this configuration of $\eta$ is invalid, as $P_{T}$ is unknown to the learner. Although a sub-optimal choice for $\eta$ , i.e., $\textstyle{\eta=O\left(\frac{1}{\sqrt{T}}\right)}$ , is accessible, the resulting algorithm suffers $O((1+P_{T})\sqrt{T})$ regret.

The meta-expert framework (Van Erven and Koolen, 2016) consists of a meta algorithm and some expert algorithm instances. The constructions are modular such that we can use different meta algorithms and expert algorithms to achieve different regret guarantees. For optimizing dynamic regret, the seminal work of Zhang et al. (2018) propose Ader based on this framework. In every round $t$ , each expert runs OGD with a different step size, and the meta algorithm applies Hedge to learn the best weights. The step sizes used by the experts are carefully designed so that there always exists an expert which is almost optimal. The regret of Ader is $O(\sqrt{(1+P_{T})T})$ , which is minimax-optimal in Euclidean space (Zhang et al., 2018).

However, it is unclear how to extend Ader to manifolds at first glance since we need to figure out the ”correct” way to do averaging. In this paper, we successfully resolve this problem using the Fréchet mean and the geodesic mean. Our proposed algorithm, called Radar, consists of $N$ instances of the expert algorithm (Algorithm 4), each of which runs R-OGD with a different step size, and a meta algorithm (Algorithm 4), which enjoys a regret approximately the same as the best expert. We denote the set of all step sizes $\{\eta_{i}\}$ by $\mathcal{H}$ . In the $t$ -th round, the expert algorithms submit all $\mathbf{x}_{t,i}$ ’s ( $i=1,\dots,N$ ) to the meta algorithm. Then the meta algorithm either computes the Fréchet mean or the geodesic mean (see Algorithm E in Appendix E for details) as $\mathbf{x}_{t}$ . After receiving $f_{t}$ , the meta algorithm updates the weight of each expert $w_{t+1,i}$ via Hedge and sends $\nabla f_{t}(\mathbf{x}_{t,i})$ to the $i$ -th expert, which computes $\mathbf{x}_{t+1,i}$ by R-OGD. The regret of the meta algorithm of Radar can be bounded by Lemma 4.1.

{algorithm2e}

[H] Radar: Meta Algorithm \KwDataLearning rate $\beta$ , set of step sizes $\mathcal{H}$ , initial weights $w_{1,i}=\frac{N+1}{i(i+1)N}$ \For $t=1,\dots,T$ Receive $\mathbf{x}_{t,i}$ from experts with stepsize $\eta_{i}$

$\mathbf{x}_{t}=\operatorname*{argmin}_{\mathbf{x}\in\mathcal{N}}\sum_{i\in[N]}w_{t,i}d(\mathbf{x},\mathbf{x}_{t,i})^{2}$

Observe the loss function $f_{t}$

Update $w_{t+1,i}$ by Hedge with $f_{t}(\mathbf{x}_{t,i})$

Send gradient $\nabla f_{t}(\mathbf{x}_{t,i})$ to each expert

{algorithm2e}

[H] Radar: Expert Algorithm \KwData A step size $\eta_{i}$ Let $\mathbf{x}_{1,i}^{\eta}$ be any point in $\mathcal{N}$

\For

$t=1,\dots,T$ Submit $\mathbf{x}_{t,i}$ to the meta algorithm

Receive gradient $\nabla f_{t}(\mathbf{x}_{t,i})$ from the meta algorithm

Update:

$\mathbf{x}_{t+1,i}=\Pi_{\mathcal{N}}{\textnormal{Exp}}_{\mathbf{x}_{t,i}}(-\eta_{i}\nabla f_{t}(\mathbf{x}_{t,i}))$

Lemma 4.1.

Under Assumptions 1, 2, 3, and setting $\beta=\sqrt{\frac{8}{G^{2}D^{2}T}}$ , the regret of Algorithm 4 satisfies

\textstyle\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq\sqrt{\frac{G^{2}D^{2}T}{8}}\left(1+\ln\frac{1}{w_{1,i}}\right).

We show that, by configuring the step sizes in $\mathcal{H}$ carefully, Radar ensures a $O(\sqrt{(1+P_{T})T})$ bound on geodesic metric spaces.

Theorem 2.

Set $\textstyle\mathcal{H}=\left\{\eta_{i}=2^{i-1}\sqrt{\frac{D^{2}}{G^{2}{\zeta}T}}\big{|}i\in[N]\right\}$ where $N=\lceil\frac{1}{2}\log_{2}(1+2T)\rceil+1$ and $\beta=\sqrt{\frac{8}{G^{2}D^{2}T}}$ . Under Assumptions 1, 2, 3, for any comparator sequence $\mathbf{u}_{1},\dots,\mathbf{u}_{T}\in\mathcal{N}$ , the general dynamic regret of Radar satisfies

\text{D-Regret}_{T}=O(\sqrt{{\zeta}(1+P_{T})T}).

Remark 4.2.

Note that if $\mathcal{M}$ is Euclidean space, then ${\zeta}=1$ and we get $O(\sqrt{(1+P_{T})T)}$ regret, which is the same as in Zhang et al. (2018).

A disadvantage of Radar is $\Theta(\log T)$ gradient queries are required at each round. In Euclidean space, Zhang et al. (2018) use a linear surrogate loss to achieve the same bound by $O(1)$ gradient queries. But on manifolds, the existence of such functions implies the sectional curvature of the manifold is everywhere $0$ (Kristály et al., 2016). It is interesting to investigate if $\Omega(\log T)$ gradient queries are necessary to achieve dynamic regret on manifolds. We would also like to point out that $O(\log T)$ is reasonable small and the work of Zhang et al. (2018) still needs $O(\log T)$ computational complexity per round.

Using the Busemann function as a bridge, we show the following dynamic regret lower bound, with proof deferred to Appendix A.4.

Theorem 3.

There exists a comparator sequence which satisfies $\sum_{t=2}^{T}d(\mathbf{u}_{t},\mathbf{u}_{t-1})\leq P_{T}$ and encounters $\Omega(\sqrt{(1+P_{T})T})$ dynamic regret on Hadamard manifolds.

Although the regret guarantee in Theorem 2 is optimal up to constants in terms of $T$ and $P_{T}$ by considering the corresponding lower bound, it still depends on $T$ and thus cannot adapt to mild environments. In Euclidean space, the smoothness of losses induces adaptive regret bounds, including the gradient-variation bound (Chiang et al., 2012) and the small-loss bound (Srebro et al., 2010). It is then natural to ask if similar bounds can be established on manifolds by assuming gsc-smoothness. We provide an affirmative answer to this question and show how to get problem-dependent bounds under the Radar framework.

5 Gradient-variation Bound on Manifolds

In this section, we show how to obtain the gradient-variation bound on manifolds under the Radar framework with alternative expert and meta algorithms.

Expert Algorithm.

For minimax optimization on Riemannian manifolds, Zhang et al. (2022) propose Riemannian Corrected Extra Gradient (RCEG), which performs the following iterates:

\begin{split}\textstyle&\mathbf{x}_{t}={\textnormal{Exp}}_{\mathbf{y}_{t}}(-\eta\nabla f_{t-1}(\mathbf{y}_{t}))\\ &\mathbf{y}_{t+1}={\textnormal{Exp}}_{\mathbf{x}_{t}}\left(-\eta\nabla f_{t}(\mathbf{x}_{t})+{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}(\mathbf{y}_{t})\right).\end{split}

However, this algorithm does not work in the constrained case, which has been left as an open problem (Zhang et al., 2022). The online improper learning setting (Hazan et al., 2018; Baby and Wang, 2021) allows the decision set to be different from (usually larger than) the set of strategies we want to compete against. Under such a setting, we find the geometric distortion due to projection can be bounded in an elegant way, and generalize RCEG to incorporate an optimism term $M_{t}\in T_{\mathbf{y}_{t}}\mathcal{M}$ .

Definition 4.

We use $M_{t}$ to denote the optimism at round $t$ and assume there exists $M$ such that $\|M_{t}\|\leq M$ for all t. We define $\mathcal{N}_{c}=\{\mathbf{x}|d(x,\mathcal{N})\leq c\}$ where $d(\mathbf{x},\mathcal{N})\coloneqq\inf_{\mathbf{y}\in\mathcal{N}}d(\mathbf{x},\mathbf{y})$ . In the improper setting, we allow the player to choose decisions from $\mathcal{N}_{\delta M}$ .

Theorem 5.

Suppose all losses $f_{t}$ are $L$ -gsc-smooth on $\mathcal{M}$ . Under Assumptions 1, 2, 3, the iterates

\begin{split}\textstyle&\mathbf{x}_{t}^{\prime}={\textnormal{Exp}}_{\mathbf{y}_{t}}(-\eta M_{t})\\ &\mathbf{x}_{t}=\Pi_{\mathcal{N}_{\delta M}}\mathbf{x}_{t}^{\prime}\\ &\mathbf{y}_{t+1}=\Pi_{\mathcal{N}}{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}\left(-\eta\nabla f_{t}(\mathbf{x}_{t}^{\prime})+{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{y}_{t})\right).\end{split}

(4)

satisfies

\begin{split}\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\leq\eta{\zeta}\sum_{t=1}^{T}\|\nabla f_{t}(\mathbf{y}_{t})-M_{t}\|^{2}+\frac{D^{2}+2DP_{T}}{2\eta}.\end{split}

for any $\mathbf{u}_{1},\dots,\mathbf{u}_{T}\in\mathcal{N}$ and $\eta\leq\frac{\delta M}{G+(G^{2}+2\zeta\delta^{2}M^{2}L^{2})^{\frac{1}{2}}}$ . Specifically, we achieve $\eta{\zeta}V_{T}+\frac{D^{2}+2DP_{T}}{2\eta}$ regret by choosing $M_{t}=\nabla f_{t-1}(\mathbf{y}_{t})$ . In this case, $M=G$ and we need $\eta\leq\frac{\delta}{1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}}$ .

Proof sketch.

We use the special case $M_{t}=\nabla f_{t-1}(\mathbf{y}_{t})$ to illustrate the main idea of the proof. We first decompose $f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{u}_{t})$ into two terms,

\begin{split}&f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{u}_{t})={(f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t}^{\prime}))}+{(f_{t}(\mathbf{x}_{t}^{\prime})-f_{t}(\mathbf{u}_{t}))}\\ \leq&G\cdot d(\mathbf{x}_{t},\mathbf{x}_{t}^{\prime})+{(f_{t}(\mathbf{x}_{t}^{\prime})-f_{t}(\mathbf{u}_{t}))}\leq\underbrace{G\cdot d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})}_{\text{troublesome term }1}+\underbrace{(f_{t}(\mathbf{x}_{t}^{\prime})-f_{t}(\mathbf{u}_{t}))}_{\text{unconstrained RCEG}}\end{split}

where the first inequality is because the gradient Lipschitzness condition, and the second one follows from the non-expansiveness of the projection. For the unconstrained RCEG term, we have the following decomposition,

\begin{split}&f_{t}(\mathbf{x}_{t}^{\prime})-f_{t}(\mathbf{u}_{t})\leq\underbrace{\frac{1}{2\eta}(2\eta^{2}\zeta L^{2}-1)d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})^{2}}_{\text{troublesome term }2}\\ +&\underbrace{\eta\zeta\|\nabla f_{t}(\mathbf{y}_{t})-\nabla f_{t-1}(\mathbf{y}_{t})\|^{2}}_{\eta\zeta V_{T}}+\underbrace{\frac{1}{2\eta}\left(d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t+1},\mathbf{u}_{t})^{2}\right)}_{\frac{D^{2}+2DP_{T}}{2\eta}}\end{split}

where the second and the third term corresponds to the gradient variation term and the dynamic regret term, respectively.

In the improper learning setting, we can show $d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\geq\delta G$ . Combining both troublesome terms, it suffices to find $\eta$ which satisfies

2\eta G+2\eta^{2}\zeta L^{2}\lambda-\lambda\leq 0,\;\forall\lambda\coloneqq d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\geq\delta G.

Remark 5.1.

We generalize Theorem 5 from Hadamard manifolds to CAT $(\kappa)$ spaces in Appendix B.2. Note that although we allow the player to make improper decisions, $V_{T}$ is still defined on $\mathcal{N}$ instead of $\mathcal{N}_{\delta G}$ . For the static setting, $P_{T}=0$ and the resulting regret bound is $O(\sqrt{V_{T}}+\frac{1}{\delta})$ . Also, in this setting, we can use an adaptive step-size

\eta_{t}=\min\left\{\frac{1}{\sqrt{1+\sum_{s=2}^{t}\|\nabla f_{t}(\mathbf{y}_{t})-\nabla f_{t-1}(\mathbf{y}_{t})\|^{2}}},\frac{\delta}{1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}}\right\}

to eliminate the dependence on $V_{T}$ .

Meta algorithm.

Intuitively, we can run OMD with different step sizes and apply a meta algorithm to estimate the optimal step size. Previous studies in learning with multiple step sizes usually adopt Hedge to aggregate the experts’ advice. However, the regret of Hedge is $O(\sqrt{T\ln N})$ and thus is undesirable for our purpose. Inspired by optimistic online learning (Rakhlin and Sridharan, 2013; Syrgkanis et al., 2015), Zhao et al. (2020) adopt Optimistic Hedge as the meta algorithm to get $O(\sqrt{(V_{T}+P_{T})P_{T}})$ gradient-variation bound. After careful analysis, we show Optimistic Hedge works for gsc-convex losses regardless of the geometric distortion and get the desired gradient-variation bound.

{algorithm2e}

[H] Radar_v: Expert Algorithm \KwDataA step size $\eta_{i}$ Let $\mathbf{x}_{1,i}^{\eta}$ be any point in $\mathcal{N}$
\For $t=1,\dots,T$ Submit $\mathbf{x}_{t,i}$ to the meta algorithm
Receive gradient $\nabla f_{t}(\cdot)$ from the meta algorithm
Each expert runs Equation (4) with $M_{t}=\nabla f_{t-1}(\mathbf{y}_{t})$ , $M=G$ and step size $\eta_{i}$

{algorithm2e}

[H] Radar_v: Meta Algorithm \KwDataA learning rate $\beta$ , a set of step sizes $\mathcal{H}$ , initial weights $w_{1,i}=w_{0,i}=\frac{1}{N}$ \For $t=1,\dots,T$ Receive all $\mathbf{x}_{t,i}$ ’s from experts with step size $\eta_{i}$
$\bar{\mathbf{x}}_{t}=\operatorname*{argmin}_{\mathbf{x}\in\mathcal{N}_{\delta G}}\sum_{i\in[N]}w_{t-1,i}d(\mathbf{x},\mathbf{x}_{t,i})^{2}$
Update $w_{t,i}\propto\exp\left(-\beta\left(\sum_{s=1}^{t-1}\ell_{s,i}+m_{t,i}\right)\right)$ by Equation (5)
$\mathbf{x}_{t}=\operatorname*{argmin}_{\mathbf{x}\in\mathcal{N}_{\delta G}}\sum_{i\in[N]}w_{t,i}d(\mathbf{x},\mathbf{x}_{t,i})^{2}$
Observe $f_{t}(\cdot)$ and send $\nabla f_{t}(\cdot)$ to experts

We denote $\boldsymbol{\ell}_{t},\mathbf{m}_{t}\in\mathbb{R}^{N}$ as the surrogate loss and the optimism at round $t$ . The update rule of Optimistic Hedge is:

\textstyle w_{t,i}\propto\exp\left(-\beta\left(\sum_{s=1}^{t-1}\ell_{s,i}+m_{t,i}\right)\right),

which achieves adaptive regret due to the optimism. The following technical lemma (Syrgkanis et al., 2015) is critical for our analysis of Optimistic Hedge, and the proof is in Appendix B.3 for completeness.

Lemma 5.2.

For any $i\in[N]$ , Optimistic Hedge satisfies

\begin{split}\textstyle\sum_{t=1}^{T}\left\langle\mathbf{w}_{t},\boldsymbol{\ell}_{t}\right\rangle-\ell_{t,i}\leq\frac{2+\ln N}{\beta}+\beta\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}-\frac{1}{4\beta}\sum_{t=2}^{T}\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}^{2}\end{split}

Following the insightful work of Zhao et al. (2020), we also adopt the Optimistic Hedge algorithm as the meta algorithm, but there are some key differences in the design of the surrogate loss and optimism. To respect the Riemannian metric, we propose the following:

\begin{split}&\ell_{t,i}=\left\langle\nabla f_{t}(\mathbf{x}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle\\ &m_{t,i}=\left\langle\nabla f_{t-1}(\bar{\mathbf{x}}_{t}),{\textnormal{Exp}}_{\bar{\mathbf{x}}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle\\ \end{split}

(5)

where $\mathbf{x}_{t}$ and $\bar{\mathbf{x}}_{t}$ are Fréchet averages of $\mathbf{x}_{t,i}$ w.r.t. linear combination coefficients $\mathbf{w}_{t}$ and $\mathbf{w}_{t-1}$ respectively. Under the Fréchet mean, we can show

f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t,i})\leq\left\langle\mathbf{w}_{t},\boldsymbol{\ell}_{t}\right\rangle-\ell_{t,i},

which ensures Lemma 5.2 can be applied to bound the meta-regret and the geodesic mean does not meet this requirement. We also emphasize that the design of the surrogate loss and optimism is highly non-trivial. As we will see in the proof of Theorem 6, the combination of the surrogate loss and the gradient-vanishing property of the Fréchet mean ensures Lemma 5.2 can be invoked to upper bound the regret of the meta algorithm. However, $\mathbf{m}_{t}$ cannot rely on $\mathbf{x}_{t}$ thus, we need to design optimism based on the tangent space of ${\bar{\mathbf{x}}_{t}}$ , which incurs extra cost. Luckily, under Equation (5), we find a reasonable upper bound of this geometric distortion by showing

\begin{split}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}&\leq O(1)\cdot\sup_{\mathbf{x}\in\mathcal{N}_{\delta G}}\|\nabla f_{t}(\mathbf{x})-\nabla f_{t-1}(\mathbf{x})\|^{2}+O(1)\cdot d(\mathbf{x}_{t},\bar{\mathbf{x}}_{t})^{2}\\ &\leq O(1)\cdot\sup_{\mathbf{x}\in\mathcal{N}_{\delta G}}\|\nabla f_{t}(\mathbf{x})-\nabla f_{t-1}(\mathbf{x})\|^{2}+\tilde{O}(1)\cdot\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}^{2}.\end{split}

Thus we can apply the negative term in Lemma 5.2 to eliminate undesired terms in $\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}$ .

Algorithms 5 and 5 describe the expert algorithm and meta algorithm of Radar_v. We show the meta-regret and total regret of Radar_v in Theorems 6 and 7, respectively. Detailed proof in this section is deferred to Appendix B.

Theorem 6.

Assume all losses are $L$ -gsc-smooth on $\mathcal{M}$ . Then under Assumptions 1, 2, 3, the regret of Algorithm 5 satisfies:

\begin{split}\textstyle&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\\ \leq&\frac{2+\ln N}{\beta}+3D^{2}\beta(V_{T}+G^{2})+\sum_{t=2}^{T}\left(3\beta(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})-\frac{1}{4\beta}\right)\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}^{2}.\end{split}

Theorem 7.

Let $\textstyle\beta=\min\left\{\sqrt{\frac{2+\ln N}{3D^{2}V_{T}}},\frac{1}{\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}}\right\}$ , $\textstyle\mathcal{H}=\left\{\eta_{i}=2^{i-1}\sqrt{\frac{D^{2}}{8{\zeta}G^{2}T}}\big{|}i\in[N]\right\}$ , where $\textstyle N=\left\lceil\frac{1}{2}\log\frac{8\zeta\delta^{2}G^{2}T}{({1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}})^{2}}\right\rceil+1$ . Assume all losses are $L$ -gsc-smooth on $\mathcal{M}$ and allow improper learning. Under Assumptions 1, 2 and 3, the regret of Radar_v satisfies

\text{D-Regret}_{T}={O}\left(\sqrt{{\zeta}(V_{T}+(1+P_{T})/\delta^{2})(1+P_{T})}\right).

In Theorem 7, $\beta$ relies on $V_{T}$ , and this dependence can be eliminated by showing a variant of Lemma 5.2 with an adaptive learning rate $\beta_{t}$ .

6 Small-loss Bound on Manifolds

For dynamic regret, the small-loss bound replaces the dependence on $T$ by $F_{T}=\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})$ , which adapts to the function values of the comparator sequence. In Euclidean space, Srebro et al. (2010) show this adaptive regret by combining OGD with the self-bounding property of smooth and non-negative functions, which reads $\|\nabla f(\mathbf{x})\|_{2}^{2}\leq 4L\cdot f(\mathbf{x})$ where $L$ is the smoothness constant. We show a similar conclusion on manifolds and defer proof details in this part to Appendix C.

Lemma 6.1.

Suppose $f:\mathcal{M}\rightarrow\mathbb{R}$ is both $L$ -gsc-smooth and non-negative on its domain where $\mathcal{M}$ is a Hadamard manifold, then we have $\|\nabla f(\mathbf{x})\|^{2}\leq 2L\cdot f(\mathbf{x})$ .

To facilitate the discussion, we denote $\bar{F}_{T}=\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})$ and $\bar{F}_{T,i}=\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})$ . We use R-OGD as the expert algorithm (Algorithm 4) and Hedge with surrogate loss

\ell_{t,i}=\left\langle\nabla f_{t}(\mathbf{x}_{t}),\textnormal{Exp}^{-1}_{\mathbf{x}_{t}}\mathbf{x}_{t,i}\right\rangle

as the meta algorithm (Algorithm 4). The following Lemma considers the regret of a single expert and shows that R-OGD achieves a small-loss dynamic regret on geodesic metric spaces.

Lemma 6.2.

Suppose all losses are $L$ -gsc-smooth and non-negative on $\mathcal{M}$ . Under Assumptions 1, 2, by choosing any step size $\eta\leq\frac{1}{2{\zeta}L}$ , R-OGD achieves $O\left(\frac{P_{T}}{\eta}+\eta F_{T}\right)$ regret.

Again, we can not directly set $\textstyle\eta=O\left(\frac{1+P_{T}}{F_{T}}\right)$ because $P_{T}$ is unknown, which is precisely why we need the meta algorithm. The meta-regret of Hedge is as follows.

Lemma 6.3.

Suppose all losses are $L$ -gsc-smooth and non-negative on $\mathcal{M}$ . Under Assumptions 1, 2, by setting the learning rate of Hedge as $\beta=\sqrt{\frac{(2+\ln N)}{D^{2}\bar{F}_{T}}}$ , the regret of the meta algorithm satisfies

\begin{split}\textstyle\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq 8D^{2}L(2+\ln N)+\sqrt{8D^{2}L(2+\ln N)F_{T,i}}.\end{split}

Now as we have the guarantee for both the expert algorithm and the meta algorithm, a direct combination yields the following general dynamic regret guarantee.

Theorem 8.

Suppose all losses are $L$ -gsc-smooth and non-negative on $\mathcal{M}$ . Under Assumptions 1, 2. Setting $\mathcal{H}=\left\{\eta_{i}=2^{i-1}\sqrt{\frac{D}{2{\zeta}LGT}}\big{|}i\in[N]\right\}$ where $N=\lceil\frac{1}{2}\log\frac{GT}{2LD{\zeta}}\rceil+1$ and $\beta=\sqrt{\frac{(2+\ln N)}{D^{2}\bar{F}_{T}}}$ . Then for any comparator $\mathbf{u}_{1},\dots,\mathbf{u}_{T}\in\mathcal{N}$ , we have

\text{D-Regret}_{T}=O(\sqrt{\zeta({\zeta}(P_{T}+1)+F_{T})(P_{T}+1)}).

Remark 6.4.

If we take $\mathcal{M}$ as Euclidean space, the regret guarantee shown in Theorem 8 becomes $O(\sqrt{(P_{T}+F_{T}+1)(P_{T}+1)})$ , which matches the result of Zhao et al. (2020).

7 Best-of-both-worlds Bound on Manifolds

Now we already achieve the gradient-variation bound and the small-loss bound on manifolds. To highlight the differences between them, we provide an example in Appendix D.1 to show under certain scenarios, one bound can be much tighter than the other. The next natural question is, is that possible to get a best-of-both-worlds bound on manifolds?

We initialize $N\coloneqq N^{v}+N^{s}$ experts as shown in Theorems 7 and 8 where $N^{v}$ and $N^{s}$ are the numbers of experts running OMD and R-OGD, respectively. For each expert $i\in[N]$ , the surrogate loss and the optimism are

\begin{split}&\ell_{t,i}=\left\langle\nabla f_{t}(\mathbf{x}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle\\ &m_{t,i}=\gamma_{t}\left\langle\nabla f_{t-1}(\bar{\mathbf{x}}_{t}),{\textnormal{Exp}}_{\bar{\mathbf{x}}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle.\\ \end{split}

(6)

$\gamma_{t}$ controls the optimism used in the meta algorithm. When $\gamma_{t}=1$ , the optimism for the gradient-variation bound is recovered, and $\gamma_{t}=0$ corresponds to the optimism for the small-loss bound.

Following Zhao et al. (2020), we use Hedge for two experts to get a best-of-both-worlds bound. The analysis therein relies on the strong convexity of $\|\nabla f_{t}(\mathbf{x}_{t})-\mathbf{m}\|_{2}^{2}$ in $\mathbf{m}$ , which is generally not the case on manifolds. So an alternative scheme needs to be proposed. We denote

\begin{split}&m_{t,i}^{v}=\left\langle\nabla f_{t-1}(\bar{\mathbf{x}}_{t}),\textnormal{Exp}^{-1}_{\bar{\mathbf{x}}_{t}}\mathbf{x}_{t,i}\right\rangle\\ &m_{t,i}^{s}=0,\\ \end{split}

(7)

while $\mathbf{m}_{t}^{v}$ and $\mathbf{m}_{t}^{s}$ be the corresponding vectors respectively. Then $\mathbf{m}_{t}=\gamma_{t}\mathbf{m}_{t}^{v}+(1-\gamma_{t})\mathbf{m}_{t}^{s}$ , which is exactly the combination rule of Hedge. The function $\|\boldsymbol{\ell}_{t}-\mathbf{m}\|_{\infty}^{2}$ is convex with respect to $\mathbf{m}$ but not strongly convex so we instead use $d_{t}(\mathbf{m})\coloneqq\|\boldsymbol{\ell}_{t}-\mathbf{m}\|_{2}^{2}$ for Hedge, and the learning rate is updated as

{\gamma_{t}=\frac{\exp\left(-\tau\left(\sum_{r=1}^{t-1}d_{r}(\mathbf{m}_{r}^{v})\right)\right)}{\exp\left(-\tau\left(\sum_{r=1}^{t-1}d_{r}(\mathbf{m}_{r}^{v})\right)\right)+\exp\left(-\tau\left(\sum_{r=1}^{t-1}d_{r}(\mathbf{m}_{r}^{s})\right)\right)}}

(8)

Algorithm 7 summarizes the meta algorithm as well as the expert algorithm for Radar_b.

{algorithm2e}

Radar_b: Algorithm \KwData Learning rates $\beta$ for Optimistic Hedge and $\gamma_{t}$ for Hedging the two experts, $\mathcal{H}=\{\eta_{i}\}$ consists of $N=N^{v}+N^{s}$ step sizes, $\tau=\frac{1}{8NG^{2}D^{2}}$ \For $t=1,\dots,T$ Run Algorithms 5 and 4 on the first $N^{v}$ experts and the later $N^{s}$ experts, resp. $\bar{\mathbf{x}}_{t}=\operatorname*{argmin}_{\mathbf{x}\in\mathcal{N}_{\delta G}}\sum_{i\in[N]}w_{t-1,i}d(\mathbf{x},\mathbf{x}_{t,i})^{2}$
Update $\gamma_{t}$ as in Equation (8)
Update $w_{t,i}\propto\exp\left(-\beta\left(\sum_{s=1}^{t-1}\ell_{s,i}+m_{t,i}\right)\right)$ by Equation (6) $\mathbf{x}_{t}=\operatorname*{argmin}_{\mathbf{x}\in\mathcal{N}_{\delta G}}\sum_{i\in[N]}w_{t,i}d(\mathbf{x},\mathbf{x}_{t,i})^{2}$
Observe $f_{t}$ and send $\nabla f_{t}(\cdot)$ to each expert

In Theorem 9 we show the guarantee of the meta algorithm of Radar_b and postpone proof details of this section to Appendix D.

Theorem 9.

Setting learning rates $\tau=\frac{1}{8NG^{2}D^{2}}$ and

\begin{split}\beta=\min\left\{\sqrt{\frac{2+\ln N}{N(D^{2}\min\{3(V_{T}+G^{2}),\bar{F}_{T}\}+8G^{2}D^{2}\ln 2)}},\frac{1}{\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}}\right\}.\end{split}

Suppose all losses are $L$ -gsc-smooth and non-negative on $\mathcal{M}$ . Under Assumptions 1, 2, 3, the regret of the meta algorithm satisfies

\textstyle\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})=O\left(\sqrt{\ln T\min\{V_{T},\bar{F}_{T}\}}\right)

where $N=N^{v}+N^{s}$ and $\bar{F}_{T}=\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})$ .

Finally, we show the regret of Radar_b is bounded by the smaller of two problem-dependent bounds as follows.

Theorem 10.

Suppose all losses are $L$ -gsc-smooth and non-negative on $\mathcal{M}$ and allow improper learning. Under Assumptions 1, 2, 3, if we set the set of candidate step sizes as

\mathcal{H}=\mathcal{H}^{v}\cup\mathcal{H}^{s},

(9)

where $\mathcal{H}^{v}$ and $\mathcal{H}^{s}$ are sets of step sizes in Theorem 7 with $N=N^{v}$ and Theorem 8 with $N=N^{s}$ respectively. Then Algorithm 7 satisfies

\text{D-Regret}_{T}=O\left(\sqrt{\zeta(P_{T}(\zeta+1/\delta^{2})+B_{T}+1)(1+P_{T})+\ln T\cdot B_{T}}\right)

where $B_{T}\coloneqq\min\{V_{T},F_{T}\}$ .

Remark 7.1.

Comparing to the result in Zhao et al. (2020), we find the result of Theorem 10 has an additional $\sqrt{\ln T}$ factor, which comes from our construction of hedging two experts. It will be interesting to remove this dependence.

8 Conclusion

In this paper, we consider adaptive online learning on Riemannian manifolds. Equipped with the idea of learning with multiple step sizes and optimistic mirror descent, we derive a series of no-regret algorithms that adapt to quantities reflecting the intrinsic difficulty of the online learning problem in different aspects. In the future, it is interesting to investigate how to achieve optimistic online learning in the proper learning setting. Moving forward, one could further examine whether $\Omega(\log T)$ gradient queries in each round are truly necessary. A curvature-dependent lower bound like the one in Criscitiello and Boumal (2022) for Riemannian online optimization also remains open.

\acks

We would like to thank three anonymous referees for constructive comments and suggestions. We gratefully thank the AI4OPT Institute for funding, as part of NSF Award 2112533. We also acknowledge the NSF for their support through Award IIS-1910077. GW would like to acknowledge an ARC-ACO fellowship provided by Georgia Tech.

References

Abernethy et al. (2008) Jacob Abernethy, Peter L Bartlett, Alexander Rakhlin, and Ambuj Tewari. Optimal strategies and minimax lower bounds for online convex games. In Proceedings of the 21st Annual Conference on Learning Theory, pages 415–423, 2008.
Ahn and Sra (2020) Kwangjun Ahn and Suvrit Sra. From nesterov’s estimate sequence to riemannian acceleration. In Proceedings of the 33rd Annual Conference on Learning Theory, pages 84–118. PMLR, 2020.
Alimisis et al. (2020) Foivos Alimisis, Antonio Orvieto, Gary Bécigneul, and Aurelien Lucchi. A continuous-time perspective for modeling acceleration in riemannian optimization. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, pages 1297–1307. PMLR, 2020.
Baby and Wang (2021) Dheeraj Baby and Yu-Xiang Wang. Optimal dynamic regret in exp-concave online learning. In Proceedings of 34th Conference on Learning Theory, pages 359–409. PMLR, 2021.
Bacák (2014a) Miroslav Bacák. Computing medians and means in hadamard spaces. SIAM journal on optimization, 24(3):1542–1566, 2014a.
Bacák (2014b) Miroslav Bacák. Convex analysis and optimization in Hadamard spaces. de Gruyter, 2014b.
Ballmann (2012) Werner Ballmann. Lectures on spaces of nonpositive curvature, volume 25. Birkhäuser, 2012.
Bécigneul and Ganea (2019) Gary Bécigneul and Octavian-Eugen Ganea. Riemannian adaptive optimization methods. In 7th International Conference on Learning Representations, 2019.
Bento et al. (2021) GC Bento, JX Neto, and IDL Melo. Elements of convex geometry in hadamard manifolds with application to equilibrium problems. arXiv preprint arXiv:2107.02223, 2021.
Besbes et al. (2015) Omar Besbes, Yonatan Gur, and Assaf Zeevi. Non-stationary stochastic optimization. Operations research, 63(5):1227–1244, 2015.
Bhatia (2009) Rajendra Bhatia. Positive definite matrices. In Positive Definite Matrices. Princeton university press, 2009.
Bridson and Haefliger (2013) Martin R Bridson and André Haefliger. Metric spaces of non-positive curvature, volume 319. Springer Science & Business Media, 2013.
Cesa-Bianchi and Lugosi (2006) Nicolo Cesa-Bianchi and Gábor Lugosi. Prediction, learning, and games. Cambridge university press, 2006.
Chiang et al. (2012) Chao-Kai Chiang, Tianbao Yang, Chia-Jung Lee, Mehrdad Mahdavi, Chi-Jen Lu, Rong Jin, and Shenghuo Zhu. Online optimization with gradual variations. In Proceedings of the 25th Annual Conference on Learning Theory, pages 6.1–6.20, 2012.
Criscitiello and Boumal (2022) Christopher Criscitiello and Nicolas Boumal. Negative curvature obstructs acceleration for strongly geodesically convex optimization, even with exact first-order oracles. In Proceedings of the 35th Annual Conference on Learning Theory, pages 496–542. PMLR, 2022.
Duchi et al. (2012) John C Duchi, Alekh Agarwal, and Martin J Wainwright. Dual averaging for distributed optimization: Convergence analysis and network scaling. IEEE Transactions on Automatic Control, 57(3):592–606, 2012.
Hazan et al. (2018) Elad Hazan, Wei Hu, Yuanzhi Li, and Zhiyuan Li. Online improper learning with an approximation oracle. Advances in Neural Information Processing Systems, 31, 2018.
Hazan et al. (2016) Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends® in Optimization, 2(3-4):157–325, 2016.
Hosseini and Sra (2015) Reshad Hosseini and Suvrit Sra. Matrix manifold optimization for gaussian mixtures. In Advances in Neural Information Processing Systems 29, 28:910–918, 2015.
Jadbabaie et al. (2015) Ali Jadbabaie, Alexander Rakhlin, Shahin Shahrampour, and Karthik Sridharan. Online optimization: Competing with dynamic comparators. In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics, pages 398–406. PMLR, 2015.
Karimi et al. (2022) Mohammad Karimi, Ya-Ping Hsieh, Panayotis Mertikopoulos, and Andreas Krause. Riemannian stochastic approximation algorithms. arXiv preprint arXiv:2206.06795, 2022.
Kristály et al. (2016) Alexandru Kristály, Chong Li, Genaro López-Acedo, and Adriana Nicolae. What do ‘convexities’ imply on hadamard manifolds? Journal of Optimization Theory and Applications, 170:1068–1074, 2016.
Lou et al. (2020) Aaron Lou, Isay Katsman, Qingxuan Jiang, Serge Belongie, Ser-Nam Lim, and Christopher De Sa. Differentiating through the fréchet mean. In Proceeddings of the 37th International Conference on Machine Learning, pages 6393–6403. PMLR, 2020.
Luo and Schapire (2015) Haipeng Luo and Robert E Schapire. Achieving all with no parameters: Adanormalhedge. In Proceedings of the 28th Annual Conference on Learning Theory, pages 1286–1304. PMLR, 2015.
Maass et al. (2022) Alejandro I Maass, Chris Manzie, Dragan Nesic, Jonathan H Manton, and Iman Shames. Tracking and regret bounds for online zeroth-order euclidean and riemannian optimization. SIAM Journal on Optimization, 32(2):445–469, 2022.
Martínez-Rubio (2022) David Martínez-Rubio. Global riemannian acceleration in hyperbolic and spherical spaces. In Proceedings of the 33rd International Conference on Algorithmic Learning Theory, pages 768–826. PMLR, 2022.
Martínez-Rubio and Pokutta (2022) David Martínez-Rubio and Sebastian Pokutta. Accelerated riemannian optimization: Handling constraints with a prox to bound geometric penalties. arXiv preprint arXiv:2211.14645, 2022.
Mokhtari et al. (2016) Aryan Mokhtari, Shahin Shahrampour, Ali Jadbabaie, and Alejandro Ribeiro. Online optimization in dynamic environments: Improved regret rates for strongly convex problems. In 2016 IEEE 55th Conference on Decision and Control, pages 7195–7201. IEEE, 2016.
Petersen (2006) Peter Petersen. Riemannian geometry, volume 171. Springer, 2006.
Rakhlin and Sridharan (2013) Alexander Rakhlin and Karthik Sridharan. Online learning with predictable sequences. In Conference on Learning Theory, pages 993–1019. PMLR, 2013.
Sakai (1996) Takashi Sakai. Riemannian geometry, volume 149. American Mathematical Soc., 1996.
Sra et al. (2018) Suvrit Sra, Nisheeth K Vishnoi, and Ozan Yildiz. On geodesically convex formulations for the brascamp-lieb constant. In Proceedings of the 21st International Conference on Approximation Algorithms for Combinatorial Optimization Problems. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2018.
Srebro et al. (2010) Nathan Srebro, Karthik Sridharan, and Ambuj Tewari. Smoothness, low noise and fast rates. In Advances in neural information processing systems 23, 2010.
Sturm (2003) Karl-Theodor Sturm. Probability measures on metric spaces of nonpositive curvature. Heat Kernels and Analysis on Manifolds, Graphs, and Metric Spaces: Lecture Notes from a Quarter Program on Heat Kernels, Random Walks, and Analysis on Manifolds and Graphs, 338:357, 2003.
Sun et al. (2019) Yue Sun, Nicolas Flammarion, and Maryam Fazel. Escaping from saddle points on riemannian manifolds. In In Advances in Neural Information Processing Systems 32, pages 7276–7286, 2019.
Syrgkanis et al. (2015) Vasilis Syrgkanis, Alekh Agarwal, Haipeng Luo, and Robert E Schapire. Fast convergence of regularized learning in games. In Advances in Neural Information Processing Systems 28, pages 2989–2997, 2015.
Udriste (2013) Constantin Udriste. Convex functions and optimization methods on Riemannian manifolds, volume 297. Springer Science & Business Media, 2013.
Van Erven and Koolen (2016) Tim Van Erven and Wouter M Koolen. Metagrad: Multiple learning rates in online learning. In In Advances in Neural Information Processing Systems 29, pages 3666–3674, 2016.
Vishnoi (2018) Nisheeth K Vishnoi. Geodesic convex optimization: Differentiation on manifolds, geodesics, and convexity. arXiv preprint arXiv:1806.06373, 2018.
Wan et al. (2021) Yuanyu Wan, Bo Xue, and Lijun Zhang. Projection-free online learning in dynamic environments. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, pages 10067–10075, 2021.
Wang et al. (2021) Xi Wang, Zhipeng Tu, Yiguang Hong, Yingyi Wu, and Guodong Shi. No-regret online learning over riemannian manifolds. In Advances in Neural Information Processing Systems 34, pages 28323–28335, 2021.
Zhang and Sra (2016) Hongyi Zhang and Suvrit Sra. First-order methods for geodesically convex optimization. In The 29th Annual Conference on Learning Theory, pages 1617–1638. PMLR, 2016.
Zhang et al. (2017) Lijun Zhang, Tianbao Yang, Jinfeng Yi, Rong Jin, and Zhi-Hua Zhou. Improved dynamic regret for non-degenerate functions. In Advance in Neural Information Processing Systems 30, pages 732–741, 2017.
Zhang et al. (2018) Lijun Zhang, Shiyin Lu, and Zhi-Hua Zhou. Adaptive online learning in dynamic environments. In Advances in Neural Information Processing Systems, 31, 2018.
Zhang et al. (2022) Peiyuan Zhang, Jingzhao Zhang, and Suvrit Sra. Minimax in geodesic metric spaces: Sion’s theorem and algorithms. arXiv preprint arXiv:2202.06950, 2022.
Zhao and Zhang (2021) Peng Zhao and Lijun Zhang. Improved analysis for dynamic regret of strongly convex and smooth functions. In Proceedings of the 3rd Conference on Learning for Dynamics and Control, pages 48–59, 2021.
Zhao et al. (2020) Peng Zhao, Yu-Jie Zhang, Lijun Zhang, and Zhi-Hua Zhou. Dynamic regret of convex and smooth functions. In In Advances in Neural Information Processing Systems 33, pages 12510–12520, 2020.
Zhou and Huang (2019) Li-wen Zhou and Nan-jing Huang. A revision on geodesic pseudo-convex combination and knaster–kuratowski–mazurkiewicz theorem on hadamard manifolds. Journal of Optimization Theory and Applications, 182(3):1186–1198, 2019.
Zinkevich (2003) Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th International Conference on Machine Learning, pages 928–936, 2003.

Appendix A Omitted Proof for Section 4

A.1 Proof of Theorem 1

We denote $\mathbf{x}_{t+1}^{\prime}={\textnormal{Exp}}_{\mathbf{x}_{t}}(-\eta\nabla f_{t}(\mathbf{x}_{t}))$ and start from the geodesic convexity:

\begin{split}f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{u}_{t})\stackrel{{\scriptstyle\rm(1)}}{{\leq}}&\langle\nabla f_{t}(\mathbf{x}_{t}),-{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}(\mathbf{u}_{t})\rangle\\ =&\frac{1}{\eta}\langle{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t+1}^{\prime},{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{u}_{t}\rangle\\ \stackrel{{\scriptstyle\rm(2)}}{{\leq}}&\frac{1}{2\eta}\left(\|{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{u}_{t}\|^{2}-\|{\textnormal{Exp}}_{\mathbf{x}_{t+1}^{\prime}}^{-1}\mathbf{u}_{t}\|^{2}+{\zeta}\|{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t+1}^{\prime}\|^{2}\right)\\ \stackrel{{\scriptstyle\rm(3)}}{{\leq}}&\frac{1}{2\eta}\left(\|{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{u}_{t}\|^{2}-\|{\textnormal{Exp}}_{\mathbf{x}_{t+1}}^{-1}\mathbf{u}_{t}\|^{2}\right)+\frac{\eta{\zeta}G^{2}}{2}\\ =&\frac{1}{2\eta}\left(\|{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{u}_{t}\|^{2}-\|{\textnormal{Exp}}_{\mathbf{x}_{t+1}}^{-1}\mathbf{u}_{t+1}\|^{2}+\|{\textnormal{Exp}}_{\mathbf{x}_{t+1}}^{-1}\mathbf{u}_{t+1}\|^{2}-\|{\textnormal{Exp}}_{\mathbf{x}_{t+1}}^{-1}\mathbf{u}_{t}\|^{2}\right)+\frac{\eta{\zeta}G^{2}}{2}\\ \stackrel{{\scriptstyle\rm(4)}}{{\leq}}&\frac{1}{2\eta}\left(\|{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{u}_{t}\|^{2}-\|{\textnormal{Exp}}_{\mathbf{x}_{t+1}}^{-1}\mathbf{u}_{t+1}\|^{2}+2D\|{\textnormal{Exp}}_{\mathbf{u}_{t}}^{-1}\mathbf{u}_{t+1}\|\right)+\frac{\eta{\zeta}G^{2}}{2},\end{split}

(10)

where for the second inequality we use Lemma E.5, while the third is due to Lemma E.7 and Assumption 3. For the last inequality, we invoke triangle inequality and Assumption 2.

WLOG, we can assume $\mathbf{u}_{T+1}=\mathbf{u}_{T}$ and sum from $t=1$ to $T$ :

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{u}_{t})\leq\frac{D^{2}}{2\eta}+\frac{DP_{T}}{\eta}+\frac{\eta{\zeta}G^{2}T}{2}.

(11)

A.2 Proof of Lemma 4.1

This is a generalization of (Cesa-Bianchi and Lugosi, 2006, Theorem 2.2) to the Riemannian manifold. Let $L_{t,i}=\sum_{s=1}^{t}f_{s}(\mathbf{x}_{s,i})$ and $W_{t}=\sum_{i=1}^{N}w_{1,i}e^{-\beta L_{t,i}}$ . We have the following lower bound for $\ln W_{T}$ ,

\ln(W_{T})=\ln\left(\sum_{i\in[N]}w_{1,i}e^{-\beta L_{t,i}}\right)\geq-\beta\min_{i\in[N]}\left(L_{T,i}+\frac{1}{\beta}\ln\frac{1}{w_{1,i}}\right).

For the next step, we try to get an upper bound on $\ln W_{T}$ . When $t\geq 2$ , we have

\ln\left(\frac{W_{t}}{W_{t-1}}\right)=\ln\frac{\sum_{i\in[N]}w_{1,i}e^{-\beta L_{t-1,i}}e^{-\beta f_{t}(\mathbf{x}_{t,i})}}{\sum_{i\in[N]}w_{1,i}e^{-\beta L_{t-1,i}}}=\ln\left(\sum_{i\in[N]}w_{t,i}e^{-\beta f_{t}(\mathbf{x}_{t,i})}\right),

where the updating rule of Hedge

w_{t,i}=\frac{w_{1,i}e^{-\beta L_{t-1,i}}}{\sum_{j\in[N]}w_{1,j}e^{-\beta L_{t-1,j}}}

is applied. Therefore

\begin{split}&\ln W_{T}=\ln W_{1}+\sum_{t=2}^{T}\ln\left(\frac{W_{t}}{W_{t-1}}\right)=\sum_{t=1}^{T}\ln\left(\sum_{i\in[N]}w_{t,i}e^{-\beta f_{t}(\mathbf{x}_{t,i})}\right)\\ \leq&\sum_{t=1}^{T}\left(-\beta\sum_{i\in[N]}w_{t,i}f_{t}(\mathbf{x}_{t,i})+\frac{\beta^{2}G^{2}D^{2}}{8}\right)\leq\sum_{t=1}^{T}\left(-\beta f_{t}(\mathbf{x}_{t})+\frac{\beta^{2}G^{2}D^{2}}{8}\right),\end{split}

(12)

where the first inequality follows from Hoeffding’s inequality and $f_{t}(\mathbf{x}^{\star})\leq f_{t}(\mathbf{x})\leq f_{t}(\mathbf{x}^{\star})+G\cdot D$ holds for any $\mathbf{x}\in\mathcal{N}$ and $\mathbf{x}^{\star}=\operatorname*{argmin}_{\mathbf{x}\in\mathcal{N}}f_{t}(\mathbf{x})$ , and the second inequality is due to both the Fréchet mean and the geodesic mean satisfy Jensen’s inequality. For the Fréchet mean, we can apply Lemma E.8. While Lemmas E.1 and E.11 ensure the geodesic mean satisfies the requirement.

Combining the lower and upper bound for $\ln W_{T}$ , we see

-\beta\min_{i\in[N]}\left(L_{T,i}+\frac{1}{\beta}\ln\frac{1}{w_{1,i}}\right)\leq\sum_{t=1}^{T}\left(-\beta f_{t}(\mathbf{x}_{t})+\frac{\beta^{2}G^{2}D^{2}}{8}\right).

After simplifying, we get

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\min_{i\in[N]}\left(\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})+\frac{1}{\beta}\ln\frac{1}{w_{1,i}}\right)\leq\frac{\beta G^{2}D^{2}T}{8}.

Setting $\beta=\sqrt{\frac{8}{G^{2}D^{2}T}}$ , we have

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq\sqrt{\frac{G^{2}D^{2}T}{8}}\left(1+\ln\frac{1}{w_{1,i}}\right).

A.3 Proof of Theorem 2

Each expert performs R-OGD, so by Theorem 1 we have

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})-f_{t}(\mathbf{u}_{t})\leq\frac{D^{2}+2DP_{T}}{2\eta}+\frac{\eta{\zeta}G^{2}T}{2}.

(13)

holds for any $i\in[N]$ . Now it suffices to verify that there always exists $\eta_{k}\in\mathcal{H}$ which is close to the optimal stepsize

\eta^{\star}=\sqrt{\frac{D^{2}+2DP_{T}}{{\zeta}G^{2}T}}.

(14)

By Assumption 2,

0\leq P_{T}=\sum_{t=2}^{T}d(\mathbf{u}_{t-1},\mathbf{u}_{t})\leq TD.

Thus

\sqrt{\frac{D^{2}}{TG^{2}{\zeta}}}\leq\eta^{*}\leq\sqrt{\frac{D^{2}+2TD^{2}}{TG^{2}{\zeta}}}

It is obvious that

\min\mathcal{H}=\sqrt{\frac{D^{2}}{TG^{2}{\zeta}}},\text{ and }\max\mathcal{H}\geq 2\sqrt{\frac{D^{2}+2TD^{2}}{TG^{2}{\zeta}}}

Therefore, there exists $k\in[N-1]$ such that

\eta_{k}=2^{k-1}\sqrt{\frac{D^{2}}{TG^{2}{\zeta}}}\leq\eta^{*}\leq 2\eta_{k}

(15)

The dynamic regret of the $k$ -th expert is

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k})-\sum_{t=1}^{T}f_{t}\left(\mathbf{u}_{t}\right)\\ \stackrel{{\scriptstyle\rm(1)}}{{\leq}}&\frac{D^{2}}{2\eta_{k}}+\frac{DP_{T}}{\eta_{k}}+\left(\frac{\eta_{k}TG^{2}{\zeta}}{2}\right)\\ \stackrel{{\scriptstyle\rm(2)}}{{\leq}}&\frac{D^{2}}{\eta^{*}}+\frac{2DP_{T}}{\eta^{*}}+\left(\frac{\eta^{*}TG^{2}{\zeta}}{2}\right)\\ {=}&\frac{3}{2}\sqrt{TG^{2}{\zeta}\left(D^{2}+2DP_{T}\right)}.\end{split}

(16)

The second inequality follows from Equation (15) and we use Equation (14) to derive the last equality.

Since the initial weight of the $k$ -th expert satisfies

w_{1,k}=\frac{N+1}{k(k+1)N}\geq\frac{1}{(k+1)^{2}},

the regret of the meta algorithm with respect to the $k$ -th expert is bounded by

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k})\leq\sqrt{\frac{G^{2}D^{2}T}{8}}\left(1+2\ln(k+1)\right)

(17)

in view of Lemma 4.1. Combining Equations (LABEL:expert-rader) and (17), we have

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\\ \leq&\frac{3}{2}\sqrt{TG^{2}{\zeta}\left(D^{2}+2DP_{T}\right)}+\sqrt{\frac{G^{2}D^{2}T}{8}}\left(1+2\ln(k+1)\right)\\ \leq&\sqrt{2\left(\frac{9}{4}TG^{2}{\zeta}\left(D^{2}+2DP_{T}\right)+\frac{G^{2}D^{2}T}{8}\left(1+2\ln(k+1)\right)^{2}\right)}\\ =&O\left(\sqrt{{\zeta}(1+P_{T})T}\right),\end{split}

(18)

where $\sqrt{a}+\sqrt{b}\leq\sqrt{2(a+b)}$ is applied to derive the second inequality, and for the equality follows from $\ln k=O(\log\log P_{T})=o(\sqrt{P_{T}})$ .

A.4 Minimax Dynamic Regret Lower Bound on Hadamard Manifolds

In this part, we first establish a $\Omega(\sqrt{T})$ minimax static regret lower bound on Hadamard manifolds following the classical work of Abernethy et al. (2008), then follow the reduction in Zhang et al. (2018) to get its dynamic counterpart. We focus on the manifold of SPD matrices (Bhatia, 2009) $\mathcal{S}^{n}_{++}=\{p:p\in\mathbb{R}^{n\times n},p=p^{T}\text{and }p\succ 0^{n\times n}\}$ , which becomes Hadamard when equipped with the affine-invariant metric $\left\langle U,V\right\rangle_{p}=tr(p^{-1}Up^{-1}V)$ where $tr(\cdot)$ is the trace operator. The tangent space of $\mathcal{S}^{n}_{++}$ is $\mathcal{S}^{n}=\{U:U\in\mathbb{R}^{n\times n},U=U^{T}\}$ . Under the affine-invariant metric, we have

\begin{split}&{\textnormal{Exp}}_{p}U=p^{\frac{1}{2}}\exp\left(p^{-\frac{1}{2}}Up^{-\frac{1}{2}}\right)p^{\frac{1}{2}}\\ &\textnormal{Exp}^{-1}_{p}q=p^{\frac{1}{2}}\log\left(p^{-\frac{1}{2}}qp^{-\frac{1}{2}}\right)p^{\frac{1}{2}}\\ &d(p,q)=\sqrt{\sum_{i=1}^{n}(\log\lambda_{i}(q^{-\frac{1}{2}}pq^{-\frac{1}{2}}))^{2}}.\end{split}

For technical reason, we restrict the manifold to be the manifold of SPD matrices with diagonal entries $\mathcal{D}_{++}^{n}=\{p:p\in\mathbb{R}^{n\times n},p_{i,i}>0,p\text{ is diagonal}\}$ and its tangent space is $\mathcal{D}^{n}=\{U:U\in\mathbb{R}^{n\times n},U\text{ is diagonal}\}$ . A key component in the proof is the Busemann function (Ballmann, 2012) on $\mathcal{D}_{++}^{n}$ equipped with affine-invariant metric has a closed form, which we describe as follows.

Definition 11.

(Ballmann, 2012) Suppose $\mathcal{M}$ is a Hadamard manifold and $c:[0,\infty)$ is a geodesic ray on $\mathcal{M}$ with $\|\dot{c}(0)\|=1$ . Then the Busemann function associated with $c$ is defined as

b_{c}(p)=\lim_{t\rightarrow\infty}(d(p,c(t))-t).

Busemann functions enjoy the following useful properties.

Lemma A.1.

(Ballmann, 2012) A Busemann function $b_{c}$ satisfies

1)

$b_{c}$ is gsc-convex;
2)

$\nabla b_{c}(c(t))=\dot{c}(t)$ for any $t\in[0,\infty)$ ;
3)

$\|\nabla b_{c}(p)\|\leq 1$ for every $p\in\mathcal{M}$ .

Lemma A.2.

(Bridson and Haefliger, 2013, Chapter II.10) On $\mathcal{D}_{++}^{n}$ , suppose $c(t)={\textnormal{Exp}}_{I}(tX)$ , $p={\textnormal{Exp}}_{I}(Y)$ and $\|X\|=1$ , then the Busemann function is

b_{c}(p)=-\text{tr}(XY).

Proof A.3.

We first compute

\begin{split}d(p,c(t))^{2}=&d({\textnormal{Exp}}_{I}(Y),{\textnormal{Exp}}_{I}(tX))^{2}\\ =&d(e^{Y},e^{tX})^{2}\\ =&\sum_{i=1}^{n}(\log\lambda_{i}(e^{-tX+Y}))^{2}\\ =&d(I,e^{Y-tX})^{2}\\ =&d(I,{\textnormal{Exp}}_{I}(Y-tX))^{2}\\ =&\|Y-tX\|^{2}\\ =&tr((Y-tX)(Y-tX))\\ =&tr(Y^{2})-2t\cdot tr(XY)+t^{2}\cdot tr(X^{2}),\end{split}

(19)

where we use facts that ${\textnormal{Exp}}_{I}(X)=e^{X}$ and $d(p,q)=\sqrt{\sum_{i=1}^{n}(\log\lambda_{i}(q^{-\frac{1}{2}}pq^{-\frac{1}{2}}))^{2}}$ . Meanwhile,

\lim_{t\rightarrow\infty}\frac{d(p,c(t))}{t}=1

due to the triangle inequality on $\triangle(p,c(0),c(t))$ . Therefore,

\begin{split}b_{c}(p)=&\lim_{t\rightarrow\infty}(d(p,c(t))-t)\\ =&\lim_{t\rightarrow\infty}\frac{d(p,c(t))^{2}-t^{2}}{2t}\\ =&-tr(XY).\end{split}

Remark A.4.

We consider $\mathcal{D}_{++}^{n}$ to ensure $X$ and $Y$ commute thus also $e^{Y}$ and $e^{tX}$ commute. This is necessary to get a closed form of the Busemann function.

Now we describe the minimax game on $\mathcal{D}_{++}^{n}$ . Each round $t$ , the player chooses $p_{t}$ from $\mathcal{N}=\{p:d(p,I)\leq\frac{D}{2}\}=\{p:p={\textnormal{Exp}}_{I}(Y),\|Y\|\leq\frac{D}{2}\}$ , and the adversary is allowed to pick a geodesic $c^{t}$ , which determines a loss function in

\begin{split}\mathcal{F}_{t}=&\{\alpha_{t}G_{t}b_{c}(p):\|\dot{c}(0)\|=1,\alpha_{t}\in[0,1]\}\\ =&\{\alpha_{t}G_{t}\cdot-tr(X_{t}Y):\|X_{t}\|=1,\alpha_{t}\in[0,1]\}\\ =&\{-tr(X_{t}Y):\|X_{t}\|\leq G_{t}\}\\ \end{split}

(20)

The domain is gsc-convex by Lemmas E.13 and E.14. Each loss function is gsc-convex and has a gradient upper bound $G_{t}$ using the second item of Lemma A.1. WLOG, we assume $D=2$ , and the value of the game is

\mathcal{V}_{T}(\mathcal{N},\{\mathcal{F}_{t}\})\coloneqq\inf_{\|Y_{1}\|\leq 1}\sup_{\|X_{1}\|\leq G_{1}}\dots\inf_{\|Y_{T}\|\leq 1}\sup_{\|X_{T}\|\leq G_{T}}\left[\sum_{t=1}^{T}-tr(X_{t}Y_{t})-\inf_{\|Y\|\leq 1}\sum_{t=1}^{T}-tr(X_{t}Y)\right]

Lemma A.5.

The value of the minimax game $\mathcal{V}_{T}$ can be written as

\mathcal{V}_{T}(\mathcal{N},\{\mathcal{F}_{t}\})=\inf_{\|Y_{1}\|\leq 1}\sup_{\|X_{1}\|\leq G_{1}}\dots\inf_{\|Y_{T}\|\leq 1}\sup_{\|X_{T}\|\leq G_{T}}\left[\sum_{t=1}^{T}-tr(X_{t}Y_{t})+\left\|\sum_{t=1}^{T}X_{t}\right\|\right]

Proof A.6.

This is obvious due to Cauchy–Schwarz inequality:

\inf_{\|Y\|\leq 1}\sum_{t=1}^{T}-tr(X_{t}Y)=-\sup_{\|Y\|\leq 1}tr\left(\sum_{t=1}^{T}X_{t}Y\right)=-\left\|\sum_{t=1}^{T}X_{t}\right\|.

Lemma A.7.

For $n>2$ , the adversary guarantees at least $\sqrt{\sum_{t=1}^{T}G_{t}^{2}}$ regardless of the player’s strategy.

Proof A.8.

Each round, after the player chooses $Y_{t}$ , the adversary chooses $X_{t}$ such that $\|X_{t}\|=G_{t}$ , $\left\langle X_{t},Y_{t}\right\rangle=0$ and $\left\langle X_{t},\sum_{s=1}^{t-1}X_{s}\right\rangle=0$ . This is always possible when $n>2$ . Under this strategy, $\sum_{t=1}^{T}-tr(X_{t}Y_{t})=0$ and we can show $\left\|\sum_{t=1}^{T}X_{t}\right\|=\sqrt{\sum_{t=1}^{T}G_{t}^{2}}$ by induction. The case for $T=1$ is obvious. Assume $\left\|\sum_{s=1}^{t-1}X_{s}\right\|=\sqrt{\sum_{s=1}^{t-1}G_{s}^{2}}$ , then

\left\|\sum_{s=1}^{t}X_{s}\right\|=\left\|\sum_{s=1}^{t-1}X_{s}+X_{t}\right\|=\sqrt{\left\|\sum_{s=1}^{t-1}X_{s}\right\|^{2}+\|X_{t}\|^{2}}=\sqrt{\sum_{s=1}^{t}G_{t}^{2}}.

where the second equality is due to $\left\langle X_{t},\sum_{s=1}^{t-1}X_{s}\right\rangle=0$ .

Lemma A.9.

Let $X_{0}=0$ . If the player plays

Y_{t}=\frac{\sum_{s=1}^{t-1}X_{s}}{\sqrt{\left\|\sum_{s=1}^{t-1}X_{s}\right\|^{2}+\sum_{s=t}^{T}G_{s}^{2}}},

then

\sup_{\|X_{1}\|\leq G_{1}}\sup_{\|X_{2}\|\leq G_{2}}\dots\sup_{\|X_{T}\|\leq G_{T}}\left[\sum_{t=1}^{T}-tr(X_{t}Y_{t})+\left\|\sum_{t=1}^{T}X_{t}\right\|\right]\leq\sqrt{\sum_{t=1}^{T}G_{t}^{2}}.

Proof A.10.

Let $\Gamma_{t}^{2}=\sum_{s=t}^{T}G_{s}^{2}$ , $\Gamma_{T+1}=0$ and $\tilde{X}_{t}=\sum_{s=1}^{t}X_{s}$ . We define

\Phi_{t}(X_{1},\dots,X_{t-1})=\sum_{s=1}^{t-1}-tr(X_{s}Y_{s})+\sqrt{\left\|\sum_{s=1}^{t-1}X_{s}\right\|^{2}+\Gamma_{t}^{2}}

and $\Phi_{1}=\sqrt{\sum_{t=1}^{T}G_{t}^{2}}$ . We further let

\Psi_{t}(X_{1},\dots,X_{t-1})=\sup_{\|X_{t}\|\leq G_{t}}\dots\sup_{\|X_{T}\|\leq G_{T}}\left[\sum_{s=1}^{T}-tr(X_{s}Y_{s})+\left\|\sum_{s=1}^{T}X_{s}\right\|\right]

be the payoff of the adversary when he plays $X_{1},\dots,X_{t-1}$ and then plays optimally.

We do backward induction for this argument, which means for all $t\in\{1,\dots,T+1\}$ ,

\Psi_{t}(X_{1},\dots,X_{t-1})\leq\Phi_{t}(X_{1},\dots,X_{t-1}).

The case of $t=T+1$ is obvious because $\Psi_{T+1}=\Phi_{T+1}$ . Assume the argument holds for $t+1$ and we try to show the case for $t$ .

\begin{split}&\Psi_{t}(X_{1},\dots,X_{t-1})\\ =&\sup_{\|X_{t}\|\leq G_{t}}\Psi_{t+1}(X_{1},\dots,X_{t})\\ \leq&\sup_{\|X_{t}\|\leq G_{t}}\Phi_{t+1}(X_{1},\dots,X_{t})\\ =&\sum_{s=1}^{t-1}-tr(X_{s}Y_{s})+\sup_{\|X_{t}\|\leq G_{t}}\left[-tr(X_{t}Y_{t})+\sqrt{\left\|\sum_{s=1}^{t}X_{s}\right\|^{2}+\Gamma_{t+1}^{2}}\right].\end{split}

(21)

Now it suffices to show

\sup_{\|X_{t}\|\leq G_{t}}\left[-tr(X_{t}Y_{t})+\sqrt{\left\|\sum_{s=1}^{t}X_{s}\right\|^{2}+\Gamma_{t+1}^{2}}\right]\leq\sqrt{\left\|\sum_{s=1}^{t-1}X_{s}\right\|^{2}+\Gamma_{t}^{2}}

to establish our argument. Recall that

Y_{t}=\frac{\sum_{s=1}^{t-1}X_{s}}{\sqrt{\left\|\sum_{s=1}^{t-1}X_{s}\right\|^{2}+\sum_{s=t}^{T}G_{s}^{2}}},

and denote $\tilde{X}_{t-1}=\sum_{s=1}^{t-1}X_{s}$ . It turns out that what we need to show is

\sup_{\|X_{t}\|\leq G_{t}}-tr\left(\frac{\left\langle X_{t},\tilde{X}_{t-1}\right\rangle}{\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}}\right)+\sqrt{\|\tilde{X}_{t-1}+X_{t}\|^{2}+\Gamma_{t+1}^{2}}\leq\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}.

We use the Lagrange multiplier method to prove this argument. Let

g(X_{t})=-tr\left(\frac{\left\langle X_{t},\tilde{X}_{t-1}\right\rangle}{\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}}\right)+\sqrt{\|\tilde{X}_{t-1}+X_{t}\|^{2}+\Gamma_{t+1}^{2}}+\lambda(\|X_{t}\|^{2}-G_{t}^{2}),

then the stationary point of $g$ satisfies

\frac{\partial g(X_{t})}{\partial X_{t}}=-\frac{\tilde{X}_{t-1}}{\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}}+\frac{\tilde{X}_{t-1}+X_{t}}{\sqrt{\|\tilde{X}_{t-1}+X_{t}\|^{2}+\Gamma_{t+1}^{2}}}+2\lambda X_{t}=0

and

\lambda(\|X_{t}\|^{2}-G_{t}^{2})=0.

We first consider that $\tilde{X}_{t-1}$ is co-linear with $X_{t}$ . When $\lambda=0$ , we have $X_{t}=c\tilde{X}_{t-1}$ where

c=\frac{\Gamma_{t+1}}{\Gamma_{t}}-1.

If $\tilde{X}_{t-1}$ is co-linear with $X_{t}$ and $\lambda\neq 0$ , we know $\|X_{t}\|=G_{t}$ and again let $X_{t}=G_{t}\frac{\tilde{X}_{t-1}}{\|\tilde{X}_{t-1}\|}$ or $X_{t}=-G_{t}\frac{\tilde{X}_{t-1}}{\|\tilde{X}_{t-1}\|}$ . Then we need to ensure

g(c\tilde{X}_{t-1})\leq\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}

holds for $c=\frac{\Gamma_{t+1}}{\Gamma_{t}}-1,\frac{G_{t}}{\|\tilde{X}_{t-1}\|}$ and $-\frac{G_{t}}{\|\tilde{X}_{t-1}\|}$ .

By Lemma A.11, it suffices to verify

(c^{2}-1)\|\tilde{X}_{t-1}\|^{2}\Gamma_{t}^{2}+(\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2})\Gamma_{t+1}^{2}\leq\Gamma_{t}^{4}.

If $c=\frac{\Gamma_{t+1}}{\Gamma_{t}}-1$ , we have to ensure

\begin{split}&(c^{2}-1)\|\tilde{X}_{t-1}\|^{2}\Gamma_{t}^{2}+(\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2})\Gamma_{t+1}^{2}-\Gamma_{t}^{4}\\ =&\left(\frac{\Gamma_{t+1}^{2}}{\Gamma_{t}^{2}}-2\frac{\Gamma_{t+1}}{\Gamma_{t}}\right)\|\tilde{X}_{t-1}\|^{2}\Gamma_{t}^{2}+\|\tilde{X}_{t-1}\|^{2}\Gamma_{t+1}^{2}+\Gamma_{t}^{2}\Gamma_{t+1}^{2}-\Gamma_{t}^{4}\\ =&2(\Gamma_{t+1}-\Gamma_{t})\Gamma_{t+1}\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}(\Gamma_{t+1}^{2}-\Gamma_{t}^{2})\leq 0.\end{split}

(22)

For the case where $c^{2}=\frac{G_{t}^{2}}{\|\tilde{X}_{t-1}\|^{2}}$ , we have

\begin{split}&(c^{2}-1)\|\tilde{X}_{t-1}\|^{2}\Gamma_{t}^{2}+(\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2})\Gamma_{t+1}^{2}-\Gamma_{t}^{4}\\ =&\left(\frac{G_{t}^{2}}{\|\tilde{X}_{t-1}\|^{2}}-1\right)\|\tilde{X}_{t-1}\|^{2}\Gamma_{t}^{2}+(\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2})\Gamma_{t+1}^{2}-\Gamma_{t}^{4}\\ =&\Gamma_{t}^{2}(G_{t}^{2}+\Gamma_{t+1}^{2}-\Gamma_{t}^{2})-G_{t}^{2}\|\tilde{X}_{t-1}\|^{2}\\ =&-G_{t}^{2}\|\tilde{X}_{t-1}\|^{2}\leq 0.\end{split}

(23)

The only case left is when $X_{t}$ is not parallel to $\tilde{X}_{t-1}$ . $\lambda=0$ implies $X_{t}=0$ and thus

g(0)=\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t+1}^{2}}\leq\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}.

If $\lambda\neq 0$ then $\|X_{t}\|=G$ . We have

-\frac{\tilde{X}_{t-1}}{\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}}+\frac{\tilde{X}_{t-1}}{\sqrt{\|\tilde{X}_{t-1}+X_{t}\|^{2}+\Gamma_{t+1}^{2}}}=0

which in turn implies $\left\langle X_{t},\tilde{X}_{t-1}\right\rangle=0$ . This is the maximum point of $g$ as now

g(X_{t})=\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}.

Thus we finished the induction step and the lemma was established.

Lemma A.11.

-tr\left(\frac{\left\langle X_{t},\tilde{X}_{t-1}\right\rangle}{\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}}\right)+\sqrt{\|\tilde{X}_{t-1}+X_{t}\|^{2}+\Gamma_{t+1}^{2}}\leq\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}.

holds for $X_{t}=c\tilde{X}_{t-1}$ iff

(c^{2}-1)\|\tilde{X}_{t-1}\|^{2}\Gamma_{t}^{2}+(\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2})\Gamma_{t+1}^{2}\leq\Gamma_{t}^{4}.

Proof A.12.

The statement we want to show is

-\frac{c\|\tilde{X}_{t-1}\|^{2}}{\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}}+\sqrt{\|\tilde{X}_{t-1}\|^{2}(1+c)^{2}+\Gamma_{t+1}^{2}}\leq\sqrt{\|\tilde{X}_{t-1}\|^{2}+\Gamma_{t}^{2}}.

Let $\alpha=\|\tilde{X}_{t-1}\|^{2}$ , $\beta=\Gamma_{t}^{2}$ and $\gamma=\Gamma_{t+1}^{2}$ . Following a series of algebraic manipulations, we get

(c^{2}-1)\alpha\beta+(\alpha+\beta)\gamma\leq\beta^{2}.

And the argument is proved after plugging back $\alpha,\beta,\gamma$ .

Theorem 12.

There exists a game on $\mathcal{D}_{++}^{n}$ such that we can exactly compute the value of the minimax regret. Specifically, the decision set of the player is $\mathcal{N}=\{p:p={\textnormal{Exp}}_{I}(Y),\|Y\|\leq\frac{D}{2}\}$ , and the adversary is allowed to pick a loss function in

\mathcal{F}_{t}=\{\alpha_{t}G_{t}b_{c}(p):\|\dot{c}(0)\|=1,\alpha_{t}\in[0,1]\}=\{-tr(X_{t}Y):\|X_{t}\|\leq G_{t}\}\\ .

Then the minimax value of the game is

\mathcal{V}_{T}(\mathcal{N},\{\mathcal{F}_{t}\})=\frac{D}{2}\sqrt{\sum_{t=1}^{T}G_{t}^{2}}.

In addition, the optimal strategy of the player is

Y_{t}=\frac{\sum_{s=1}^{t-1}X_{s}}{\sqrt{\left\|\sum_{s=1}^{t-1}X_{s}\right\|^{2}+\sum_{s=t}^{T}G_{s}^{2}}}.

Proof A.13.

The proposition is a direct conclusion of Lemmas A.5, A.7, and A.9.

Theorem 13.

There exists a comparator sequence which satisfies $\sum_{t=2}^{T}d(\mathbf{u}_{t},\mathbf{u}_{t-1})\leq P_{T}$ and the dynamic minimax regret lower bound on Hadamard manifolds is $\Omega(G\sqrt{D^{2}+DP_{T}})$ .

Proof A.14.

We combine Theorem 12 with a reduction in Zhang et al. (2018) to finish the proof. By Theorem 12 we have

\mathcal{V}_{T}=\inf_{\|Y_{1}\|\leq 1}\sup_{\|X_{1}\|\leq G}\dots\inf_{\|Y_{T}\|\leq 1}\sup_{\|X_{T}\|\leq G}\left[\sum_{t=1}^{T}-tr(X_{t}Y_{t})-\inf_{\|Y\|\leq 1}\sum_{t=1}^{T}-tr(X_{t}Y)\right]=\frac{GD\sqrt{T}}{2}.

Note that the path-length is upper bounded by $TD$ . For any $\tau\in[0,TD]$ , we define the set of comparators with path-length bounded by $\tau$ as

C(\tau)=\left\{\mathbf{u}_{1},\dots,\mathbf{u}_{T}\in\mathcal{N}:\sum_{t=2}^{T}d(\mathbf{u}_{t},\mathbf{u}_{t-1})\leq\tau\right\}

where $\mathcal{N}=\{\mathbf{u}:d(I,\mathbf{u})\leq\frac{D}{2}\}$ is a gsc-convex subset and the minimax dynamic regret w.r.t. $C(\tau)$ is

\mathcal{V}_{T}(C(\tau))=\inf_{\small\|Y_{1}\|\leq 1}\sup_{\small\|X_{1}\|\leq G}\dots\inf_{\small\|Y_{T}\|\leq 1}\sup_{\small\|X_{T}\|\leq G}\left[\sum_{t=1}^{T}-tr(X_{t}Y_{t})-\inf_{\mathbf{u}_{1},\dots,\mathbf{u}_{T}\in C(\tau)}\sum_{t=1}^{T}-tr(X_{t}\textnormal{Exp}^{-1}_{I}\mathbf{u}_{t})\right].

We distinguish two cases. When $\tau\leq D$ , we invoke the minimax static regret directly to get

\mathcal{V}_{T}(C(\tau))\geq\mathcal{V}_{T}=\frac{GD\sqrt{T}}{2}.

(24)

For the case of $\tau\geq D$ , WLOG, we assume $\lceil\tau/D\rceil$ divides $T$ and let $L$ be the quotient. We construct a subset of $C(\tau)$ , named $C^{\prime}(\tau)$ , which contains comparators that are fixed for each consecutive $L$ rounds. Specifically,

C^{\prime}(\tau)=\left\{\mathbf{u}_{1},\dots,\mathbf{u}_{T}\in\mathcal{N}:\mathbf{u}_{(i-1)L+1}=\dots\mathbf{u}_{iL},\forall i\in[1,\lceil\tau/D\rceil]\right\}.

Note that the path-length of comparators in $C^{\prime}(\tau)$ is at most $(\lceil\tau/D\rceil-1)D\leq\tau$ , which implies $C^{\prime}(\tau)$ is a subset of $C(\tau)$ . Thus we have

\mathcal{V}_{T}(C(\tau))\geq\mathcal{V}_{T}(C^{\prime}(\tau)).

(25)

The objective of introducing $C^{\prime}(\tau)$ is we can set $\mathbf{u}_{(i-1)L+1}=\dots=\mathbf{u}_{(iL)}$ to be the offline minimizer of the $i$ -th segment and invoke the minimax lower bound for the static regret for each segment. Thus we have

\begin{split}&\mathcal{V}_{T}(C^{\prime}(\tau))\\ =&\inf_{\small\|Y_{1}\|\leq 1}\sup_{\small\|X_{1}\|\leq G}\dots\inf_{\small\|Y_{T}\|\leq 1}\sup_{\small\|X_{T}\|\leq G}\left[\sum_{t=1}^{T}-tr(X_{t}Y_{t})-\inf_{\mathbf{u}_{1},\dots,\mathbf{u}_{T}\in C^{\prime}(\tau)}\sum_{t=1}^{T}-tr(X_{t}\textnormal{Exp}^{-1}_{I}\mathbf{u}_{t})\right]\\ =&\inf_{\small\|Y_{1}\|\leq 1}\sup_{\small\|X_{1}\|\leq G}\dots\inf_{\small\|Y_{T}\|\leq 1}\sup_{\small\|X_{T}\|\leq G}\left[\sum_{t=1}^{T}-tr(X_{t}Y_{t})-\sum_{i=1}^{\lceil\tau/D\rceil}\inf_{\|Y\|\leq 1}\sum_{t=(i-1)L+1}^{iL}-tr(X_{t}Y)\right]\\ =&\lceil\tau/D\rceil\frac{GD\sqrt{L}}{2}=\frac{GD\sqrt{T\lceil\tau/D\rceil}}{2}\geq\frac{G\sqrt{TD\tau}}{2}.\end{split}

(26)

Combining Equations (24), (25) and (26) yields

\mathcal{V}_{T}(C(\tau))\geq\frac{G}{2}\max(D\sqrt{T},\sqrt{TD\tau})=\Omega(G\sqrt{T(D^{2}+D\tau)}).

Appendix B Omitted Proof for Section 5

B.1 Proof of Theorem 5

We first argue $\mathcal{N}_{c}$ is gsc-convex for any $c\geq 0$ to ensure the algorithm is well-defined. By Lemma E.13, $d(\mathbf{x},\mathcal{N})$ is gsc-convex on Hadamard manifolds. The sub-level set of a gsc-convex function is a gsc-convex set due to Lemma E.14, which implies $\mathcal{N}_{c}$ is gsc-convex. We notice that

f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{u}_{t})=(f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t}^{\prime}))+(f_{t}(\mathbf{x}_{t}^{\prime})-f_{t}(\mathbf{u}_{t}))

and derive upper bounds for two terms individually. If $\mathbf{x}_{t}^{\prime}\in\mathcal{N}_{\delta M}$ then $\mathbf{x}_{t}^{\prime}=\mathbf{x}_{t}$ and $f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t}^{\prime})=0$ . If this is not the case, by Lemma E.7, we have

d(\mathbf{x}_{t}^{\prime},\mathbf{x}_{t})\leq d(\mathbf{x}_{t}^{\prime},\mathbf{z})\leq d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})

where $\mathbf{z}$ is the intersection of $\mathcal{N}_{\delta}$ and the geodesic segment connecting $\mathbf{x}_{t}^{\prime}$ and $\mathbf{y}_{t}$ . Thus

f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t}^{\prime})\leq\left\langle\nabla f_{t}(\mathbf{x}_{t}),-\textnormal{Exp}^{-1}_{\mathbf{x}_{t}}{\mathbf{x}_{t}^{\prime}}\right\rangle\leq\|\nabla f_{t}(\mathbf{x}_{t})\|\cdot d(\mathbf{x}_{t}^{\prime},\mathbf{x}_{t})\leq G\cdot d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t}),

(27)

where we notice $\mathbf{x}_{t}\in\mathcal{N}_{\delta M}$ and use Assumption 3. Let $\mathbf{y}_{t+1}^{\prime}={\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}\left(-\eta\nabla f_{t}(\mathbf{x}_{t}^{\prime})+{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{y}_{t})\right)$ . The second term $f_{t}(\mathbf{x}_{t}^{\prime})-f_{t}(\mathbf{u}_{t})$ can be bounded by

\begin{split}&f_{t}(\mathbf{x}_{t}^{\prime})-f_{t}(\mathbf{u}_{t})\stackrel{{\scriptstyle\rm(1)}}{{\leq}}-\langle{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{u}_{t}),\nabla f_{t}(\mathbf{x}_{t}^{\prime})\rangle\\ =&\frac{1}{\eta}\langle{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{u}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{y}_{t+1}^{\prime})-{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{y}_{t})\rangle\\ \stackrel{{\scriptstyle\rm(2)}}{{\leq}}&\frac{1}{2\eta}\left({\zeta}d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t+1}^{\prime})^{2}+d(\mathbf{x}_{t}^{\prime},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t+1}^{\prime},\mathbf{u}_{t})^{2}\right)-\frac{1}{2\eta}\left(d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})^{2}+d(\mathbf{x}_{t}^{\prime},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}\right)\\ =&\frac{1}{2\eta}\left({\zeta}d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t+1}^{\prime})^{2}-d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})^{2}+d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t+1}^{\prime},\mathbf{u}_{t})^{2}\right)\\ =&\frac{1}{2\eta}\left({\zeta}\|-\eta\nabla f_{t}(\mathbf{x}_{t}^{\prime})+{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{y}_{t})\|^{2}-d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})^{2}+d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t+1}^{\prime},\mathbf{u}_{t})^{2}\right)\\ =&\frac{1}{2\eta}\left({\zeta}\|-\eta\nabla f_{t}(\mathbf{x}_{t}^{\prime})+\eta\Gamma_{\mathbf{y}_{t}}^{\mathbf{x}_{t}^{\prime}}M_{t}\|^{2}-d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})^{2}+d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t+1}^{\prime},\mathbf{u}_{t})^{2}\right)\\ \stackrel{{\scriptstyle\rm(3)}}{{\leq}}&\frac{1}{2\eta}\left({\zeta}\|-\eta\nabla f_{t}(\mathbf{x}_{t}^{\prime})+\eta\Gamma_{\mathbf{y}_{t}}^{\mathbf{x}_{t}^{\prime}}M_{t}\|^{2}-d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})^{2}+d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t+1},\mathbf{u}_{t})^{2}\right)\end{split}

(28)

where the second inequality follows from Lemmas E.5 and E.6 and the third one is due to the non-expansive property of projection onto Hadamard manifolds. We apply $\Gamma_{\mathbf{x}}^{\mathbf{y}}{\textnormal{Exp}}_{\mathbf{x}}^{-1}\mathbf{y}=-{\textnormal{Exp}}_{\mathbf{y}}^{-1}\mathbf{x}$ to derive the last equality.

Now we can get the desired squared term $\|\nabla f_{t}(\mathbf{y}_{t})-M_{t}\|^{2}$ by considering

\begin{split}&\|\nabla f_{t}(\mathbf{x}_{t}^{\prime})-\Gamma_{\mathbf{y}_{t}}^{\mathbf{x}_{t}^{\prime}}M_{t}\|^{2}\\ =&\|\nabla f_{t}(\mathbf{x}_{t}^{\prime})-\Gamma_{\mathbf{y}_{t}}^{\mathbf{x}_{t}^{\prime}}\nabla f_{t}(\mathbf{y}_{t})+\Gamma_{\mathbf{y}_{t}}^{\mathbf{x}_{t}^{\prime}}\nabla f_{t}(\mathbf{y}_{t})-\Gamma_{\mathbf{y}_{t}}^{\mathbf{x}_{t}^{\prime}}M_{t}\|^{2}\\ \leq&2\left(\|\nabla f_{t}(\mathbf{x}_{t}^{\prime})-\Gamma_{\mathbf{y}_{t}}^{\mathbf{x}_{t}^{\prime}}\nabla f_{t}(\mathbf{y}_{t})\|^{2}+\|\Gamma_{\mathbf{y}_{t}}^{\mathbf{x}_{t}^{\prime}}\nabla f_{t}(\mathbf{y}_{t})-\Gamma_{\mathbf{y}_{t}}^{\mathbf{x}_{t}^{\prime}}M_{t}\|^{2}\right)\\ \leq&2L^{2}d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})^{2}+2\|\nabla f_{t}(\mathbf{y}_{t})-M_{t}\|^{2},\end{split}

(29)

where in the first inequlity we use $\|\mathbf{x}+\mathbf{y}\|^{2}\leq 2\|\mathbf{x}\|^{2}+2\|\mathbf{y}\|^{2}$ holds for any SPD norm $\|\cdot\|$ , and the second inequality is due to the smoothness of $f$ and parallel transport is an isometry. Combining Equations (27), (28) and (29), we have

\begin{split}&f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{u}_{t})\\ \leq&Gd(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})+\frac{\eta{\zeta}}{2}\left(2\|\nabla f_{t}(\mathbf{y}_{t})-M_{t}\|^{2}+2L^{2}d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})^{2}\right)\\ &-\frac{1}{2\eta}d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})^{2}+\frac{1}{2\eta}(d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t+1},\mathbf{u}_{t})^{2})\\ \leq&\eta{\zeta}\|\nabla f_{t}(\mathbf{y}_{t})-M_{t}\|^{2}+\frac{1}{2\eta}\left(2\eta G+2\eta^{2}{\zeta}L^{2}d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})-d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\right)d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\\ &+\frac{1}{2\eta}(d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t+1},\mathbf{u}_{t})^{2}).\end{split}

(30)

Now we show

\frac{1}{2\eta}\left(2\eta G+2\eta^{2}{\zeta}L^{2}d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})-d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\right)d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\leq 0

(31)

holds for any $t\in[T]$ . First we consider the case that $d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\leq\delta M$ , which means $\mathbf{x}_{t}^{\prime}\in\mathcal{N}_{\delta M}$ and $f_{t}(\mathbf{x}_{t})=f_{t}(\mathbf{x}_{t}^{\prime})$ . Thus Equation (31) is implied by

2\eta^{2}{\zeta}L^{2}d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})-d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\leq 0,

which is obviously true by considering our assumption on $\eta$ :

\eta\leq\frac{\delta M}{G+(G^{2}+2\zeta\delta^{2}M^{2}L^{2})^{\frac{1}{2}}}\leq\frac{1}{\sqrt{2\zeta L^{2}}}.

When $d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\geq\delta M$ , to simplify the proof, we denote $\lambda=d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})$ and try to find $\eta$ such that

h(\eta;\lambda)\coloneqq 2\eta G+2\eta^{2}\zeta L^{2}\lambda-\lambda\leq 0

(32)

holds for any $\lambda\geq\delta M$ . We denote the only non-negative root of $h(\eta;\lambda)$ as $\eta(\lambda)$ , which can be solved explicitly as

\eta(\lambda)=\frac{-G+(G^{2}+2\zeta\lambda^{2}L^{2})^{\frac{1}{2}}}{2\zeta\lambda L^{2}}.

Applying Lemma E.9 with $a=G$ and $b=2\zeta L^{2}$ , we know $\eta(\lambda)$ increases on $[0,\infty)$ . Thus $\eta(\lambda)\geq\eta(\delta M)$ holds for any $\lambda\geq\delta M$ . Combining with the fact that $h(0;\lambda)=-\lambda<0$ , we know $h(\eta;\lambda)\leq 0$ holds for any $\eta\leq\eta(\lambda)$ , so we can simply set

\eta\leq\min_{\lambda}\eta(\lambda)=\eta\left(\delta M\right)=\frac{\delta M}{G+(G^{2}+2\zeta\delta^{2}M^{2}L^{2})^{\frac{1}{2}}}.

to ensure $h(\eta;\lambda)\leq 0$ always holds.

Now it suffices to bound

\begin{split}&\sum_{t=1}^{T}\frac{1}{2\eta}(d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t+1},\mathbf{u}_{t})^{2})\\ \leq&\frac{d(\mathbf{y}_{1},\mathbf{u}_{1})^{2}}{2\eta}+\sum_{t=2}^{T}\frac{\left(d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t},\mathbf{u}_{t-1})^{2}\right)}{2\eta}\\ \leq&\frac{D^{2}}{2\eta}+2D\frac{\sum_{t=2}^{T}d(\mathbf{u}_{t},\mathbf{u}_{t-1})}{2\eta}=\frac{D^{2}+2DP_{T}}{2\eta}.\end{split}

(33)

Finally, we apply the telescoping-sum on Equation (30),

\begin{split}\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\leq\eta{\zeta}\sum_{t=1}^{T}\|\nabla f_{t}(\mathbf{y}_{t})-M_{t}\|^{2}+\frac{D^{2}+2DP_{T}}{2\eta}.\end{split}

(34)

B.2 Extension of Theorem 5 to CAT $(\kappa)$ Spaces

In this part, we show how to get optimistic regret bound on CAT $(\kappa)$ spaces, the sectional curvature of which is upper bounded by $\kappa$ . Note that Hadamard manifolds are complete CAT $(0)$ spaces. To proceed, we make the following assumption.

Assumption 4

The sectional curvature of manifold $\mathcal{M}$ satisfies $-\kappa_{1}\leq\kappa\leq\kappa_{2}$ where $\kappa_{1}\geq 0$ . We define

\mathcal{D}(\kappa)\coloneqq\begin{cases}\infty,\quad&\kappa\leq 0\\ \frac{\pi}{\sqrt{\kappa}},\quad&\kappa>0.\end{cases}

The diameter of the gsc-convex set $\mathcal{N}\subset\mathcal{M}$ is $D$ and we assume $D+2\delta M\leq\mathcal{D}(\kappa_{2})$ . The gradient satisfies $\sup_{\mathbf{x}\in\mathcal{N}_{\delta M}}\|\nabla f_{t}(\mathbf{x})\|\leq G.$

Lemma B.1.

(Alimisis et al., 2020, Corollary 2.1) Let $\mathcal{M}$ be a Riemannian manifold with sectional curvature upper bounded by $\kappa_{2}$ and $\mathcal{N}\subset\mathcal{M}$ be a gsc-convex set with diameter upper bounded by $\mathcal{D}(\kappa_{2})$ . For a geodesic triangle fully lies within $\mathcal{N}$ with side lengths $a,b,c$ , we have

a^{2}\geq{\xi}(\kappa_{2},D)b^{2}+c^{2}-2bc\cos A

where ${\xi}(\kappa_{2},D)=\sqrt{-\kappa_{2}}D\operatorname{coth}(\sqrt{-\kappa_{2}}D)$ when $\kappa_{2}\leq 0$ and ${\xi}(\kappa_{2},D)=\sqrt{\kappa_{2}}D\operatorname{cot}(\sqrt{\kappa_{2}}D)$ when $\kappa_{2}>0$ .

Definition 14.

We define $\zeta=\zeta(-\kappa_{1},D+2\delta M)$ and $\xi=\xi(\kappa_{2},D+2\delta M)$ where $\xi(\cdot,\cdot)$ and $\zeta(\cdot,\cdot)$ are defined in Lemmas B.1 and E.5, respectively.

Lemma B.2.

(Bridson and Haefliger, 2013) For a CAT $(\kappa)$ space $\mathcal{M}$ , a ball of diameter smaller than $\mathcal{D}(\kappa)$ is convex. Let $C$ be a convex subset in $\mathcal{M}$ . If $d(\mathbf{x},C)\leq\frac{\mathcal{D}(\kappa)}{2}$ then $d(\mathbf{x},C)$ is convex and there exists a unique point $\Pi_{C}(\mathbf{x})\in\mathcal{C}$ such that $d(\mathbf{x},\Pi_{C}(\mathbf{x}))=d(\mathbf{x},C)=\inf_{\mathbf{y}\in C}d(\mathbf{x},\mathbf{y})$ .

Theorem 15.

Suppose all losses $f_{t}$ are $L$ -gsc-smooth on $\mathcal{M}$ . Under Assumptions 4, the iterates

\begin{split}\textstyle&\mathbf{x}_{t}^{\prime}={\textnormal{Exp}}_{\mathbf{y}_{t}}(-\eta M_{t})\\ &\mathbf{x}_{t}=\Pi_{\mathcal{N}_{\delta M}}\mathbf{x}_{t}^{\prime}\\ &\mathbf{y}_{t+1}=\Pi_{\mathcal{N}}{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}\left(-\eta\nabla f_{t}(\mathbf{x}_{t}^{\prime})+{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{y}_{t})\right).\end{split}

(35)

satisfies

\begin{split}\textstyle\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\leq\eta{\zeta}\sum_{t=1}^{T}\|\nabla f_{t}(\mathbf{y}_{t})-M_{t}\|^{2}+\frac{D^{2}+2DP_{T}}{2\eta}.\end{split}

for any $\mathbf{u}_{1},\dots,\mathbf{u}_{T}\in\mathcal{N}$ and $\eta\leq\min\left\{\frac{\xi\delta M}{G+(G^{2}+2\zeta\xi\delta^{2}M^{2}L^{2})^{\frac{1}{2}}},\frac{\mathcal{D}(\kappa_{2})}{2(G+2M)}\right\}$ .

Proof B.3.

We highlight key differences between the proof of Theorem 5. Again we let $\mathbf{y}_{t+1}^{\prime}={\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}\left(-\eta\nabla f_{t}(\mathbf{x}_{t}^{\prime})+{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{y}_{t})\right)$ . First, we need to argue the algorithm is well-defined. The diameter of $\mathcal{N}_{\delta M}$ is at most $D+2\delta M$ by triangle inequality, so $\mathcal{N}_{\delta M}$ is gsc-convex by Assumption 4 and Lemma B.2. We also need to ensure that $d(\mathbf{x}_{t}^{\prime},\mathcal{N}_{\delta M})\leq\frac{\mathcal{D}(\kappa_{2})}{2}$ and $d(\mathbf{y}_{t+1}^{\prime},\mathcal{N})\leq\frac{\mathcal{D}(\kappa_{2})}{2}$ and apply Lemma B.2 to show the projection is unique and non-expansive. For $d(\mathbf{x}_{t}^{\prime},\mathcal{N}_{\delta M})$ , we have

d(\mathbf{x}_{t}^{\prime},\mathcal{N}_{\delta M})\leq d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\leq\eta M\leq\frac{\mathcal{D}(\kappa_{2})}{2}.

by $\eta\leq\frac{\mathcal{D}(\kappa_{2})}{2(G+2M)}$ . Similarly, for $d(\mathbf{y}_{t+1}^{\prime},\mathcal{N})$

\begin{split}&d(\mathbf{y}_{t+1}^{\prime},\mathcal{N})\leq d(\mathbf{y}_{t+1}^{\prime},\mathbf{y}_{t})\leq d(\mathbf{y}_{t+1}^{\prime},\mathbf{x}_{t}^{\prime})+d(\mathbf{y}_{t},\mathbf{x}_{t}^{\prime})\\ \leq&\|-\eta\nabla f_{t}(\mathbf{x}_{t}^{\prime})+\eta\Gamma_{\mathbf{y}_{t}}^{\mathbf{x}_{t}^{\prime}}M_{t}\|+\eta M\\ \leq&\eta(G+2M)\leq\frac{\mathcal{D}(\kappa_{2})}{2}.\end{split}

We can bound $f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t}^{\prime})$ in the same way as Theorem 5, but we now use Lemmas B.1 and E.5 to bound $f_{t}(\mathbf{x}_{t}^{\prime})-f_{t}(\mathbf{u}_{t})$ .

\begin{split}&f_{t}(\mathbf{x}_{t}^{\prime})-f_{t}(\mathbf{u}_{t}){\leq}-\langle{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{u}_{t}),\nabla f_{t}(\mathbf{x}_{t}^{\prime})\rangle\\ =&\frac{1}{\eta}\langle{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{u}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{y}_{t+1}^{\prime})-{\textnormal{Exp}}_{\mathbf{x}_{t}^{\prime}}^{-1}(\mathbf{y}_{t})\rangle\\ {\leq}&\frac{1}{2\eta}\left({\zeta}d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t+1}^{\prime})^{2}+d(\mathbf{x}_{t}^{\prime},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t+1}^{\prime},\mathbf{u}_{t})^{2}\right)-\frac{1}{2\eta}\left(\xi d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})^{2}+d(\mathbf{x}_{t}^{\prime},\mathbf{u}_{t})^{2}-d(\mathbf{y}_{t},\mathbf{u}_{t})^{2}\right).\\ \end{split}

(36)

Finally, we need to show

2\eta G+2\eta^{2}{\zeta}L^{2}d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})-\xi d(\mathbf{x}_{t}^{\prime},\mathbf{y}_{t})\leq 0

(37)

Following the proof of Theorem 5, we find

\eta\leq\frac{\xi\delta M}{G+(G^{2}+2\zeta\xi\delta^{2}M^{2}L^{2})^{\frac{1}{2}}}

satisfies the required condition. The guarantee is thus established.

B.3 Proof of Lemma 5.2

We first show

\begin{split}\sum_{t=1}^{T}\left\langle\boldsymbol{\ell}_{t},\mathbf{w}_{t}-\mathbf{w}^{*}\right\rangle\leq\frac{\ln N+R(\mathbf{w}^{*})}{\beta}+\beta\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}-\frac{1}{2\beta}\sum_{t=1}^{T}\left(\|\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\|_{1}^{2}+\|\mathbf{w}_{t}-\mathbf{w}_{t-1}^{\prime}\|_{1}^{2}\right)\end{split}

(38)

holds for any $\mathbf{w}^{*}\in\Delta_{N}$ , where

\mathbf{w}_{t}=\operatorname*{argmin}_{\mathbf{w}\in\Delta_{N}}\;\beta\left\langle\sum_{s=1}^{t-1}\boldsymbol{\ell}_{s}+\mathbf{m}_{t},\mathbf{w}\right\rangle+R(\mathbf{w}),\quad t\geq 1

and

\mathbf{w}_{t}^{\prime}=\operatorname*{argmin}_{\mathbf{w}\in\Delta_{N}}\;\beta\left\langle\sum_{s=1}^{t}\boldsymbol{\ell}_{s},\mathbf{w}\right\rangle+R(\mathbf{w}),\quad t\geq 0.

Note that $R(\mathbf{w})=\sum_{i\in[N]}w_{i}\ln w_{i}$ is the negative entropy. According to the equivalence between Hedge and follow the regularized leader with the negative entropy regularizer, we have $w_{t,i}\propto e^{-\beta\left(\sum_{s=1}^{t-1}\ell_{s,i}+m_{t,i}\right)}$ and $w_{t,i}^{\prime}\propto e^{-\beta\left(\sum_{s=1}^{t}\ell_{s,i}\right)}.$ To prove Equation (38), we consider the following decomposition:

\left\langle\boldsymbol{\ell}_{t},\mathbf{w}_{t}-\mathbf{w}^{*}\right\rangle=\left\langle\boldsymbol{\ell}_{t}-\mathbf{m}_{t},\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\right\rangle+\left\langle\mathbf{m}_{t},\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\right\rangle+\left\langle\boldsymbol{\ell}_{t},\mathbf{w}_{t}^{\prime}-\mathbf{w}^{*}\right\rangle.

Since $R(\cdot)$ is $1$ -strongly convex w.r.t. the $\boldsymbol{\ell}_{1}$ norm, by Lemma E.16, we have $\|\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\|_{1}\leq\beta\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}$ and

\left\langle\boldsymbol{\ell}_{t}-\mathbf{m}_{t},\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\right\rangle\leq\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}\|\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\|_{1}\leq\beta\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}

by Hölder’s inequality. Hence it suffices to show

\sum_{t=1}^{T}\left\langle\mathbf{m}_{t},\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\right\rangle+\left\langle\boldsymbol{\ell}_{t},\mathbf{w}_{t}^{\prime}-\mathbf{w}^{*}\right\rangle\leq\frac{\ln N+R(\mathbf{w}^{*})}{\beta}-\frac{1}{2\beta}\sum_{t=1}^{T}\left(\|\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\|_{1}^{2}+\|\mathbf{w}_{t}-\mathbf{w}_{t-1}^{\prime}\|_{1}^{2}\right)

(39)

to prove Equation (38).

Equation (39) holds for $T=0$ because $R(\mathbf{w}^{*})\geq-\ln N$ holds for any $\mathbf{w}^{*}\in\Delta_{N}$ . To proceed, we need the following proposition:

g(\mathbf{w}^{*})+\frac{c}{2}\|\mathbf{w}-\mathbf{w}^{*}\|^{2}\leq g(\mathbf{w})

(40)

holds for any $c$ -strongly convex $g(\cdot):\mathcal{W}\rightarrow\mathbb{R}$ where $\mathbf{w}^{*}=\operatorname*{argmin}_{\mathbf{w}\in\mathcal{W}}g(\mathbf{w})$ . This fact can be easily seen by combining the strong convexity and the first-order optimality condition for convex functions.

Assume Equation (39) holds for round $T-1\;(T\geq 1)$ and we denote

C_{T}=\frac{1}{2\beta}\sum_{t=1}^{T}\left(\|\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\|_{1}^{2}+\|\mathbf{w}_{t}-\mathbf{w}_{t-1}^{\prime}\|_{1}^{2}\right).

Now for round $T$ ,

\begin{split}&\sum_{t=1}^{T}\left(\left\langle\mathbf{m}_{t},\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\right\rangle+\left\langle\boldsymbol{\ell}_{t},\mathbf{w}_{t}^{\prime}\right\rangle\right)\\ \stackrel{{\scriptstyle\rm(1)}}{{\leq}}&\sum_{t=1}^{T-1}\left\langle\boldsymbol{\ell}_{t},\mathbf{w}_{T-1}^{\prime}\right\rangle+\frac{\ln N+R(\mathbf{w}_{T-1}^{\prime})}{\beta}-C_{T-1}+\left\langle\mathbf{m}_{T},\mathbf{w}_{T}-\mathbf{w}_{T}^{\prime}\right\rangle+\left\langle\boldsymbol{\ell}_{T},\mathbf{w}_{T}^{\prime}\right\rangle\\ \stackrel{{\scriptstyle\rm(2)}}{{\leq}}&\sum_{t=1}^{T-1}\left\langle\boldsymbol{\ell}_{t},\mathbf{w}_{T}\right\rangle+\frac{\ln N+R(\mathbf{w}_{T})}{\beta}-C_{T-1}+\left\langle\mathbf{m}_{T},\mathbf{w}_{T}-\mathbf{w}_{T}^{\prime}\right\rangle+\left\langle\boldsymbol{\ell}_{T},\mathbf{w}_{T}^{\prime}\right\rangle-\frac{1}{2\beta}\|\mathbf{w}_{T}-\mathbf{w}_{T-1}^{\prime}\|_{1}^{2}\\ =&\left(\sum_{t=1}^{T-1}\left\langle\boldsymbol{\ell}_{t},\mathbf{w}_{T}\right\rangle+\left\langle\mathbf{m}_{T},\mathbf{w}_{T}\right\rangle+\frac{\ln N+R(\mathbf{w}_{T})}{\beta}\right)+\left\langle\boldsymbol{\ell}_{T}-\mathbf{m}_{T},\mathbf{w}_{T}^{\prime}\right\rangle-C_{T-1}-\frac{1}{2\beta}\|\mathbf{w}_{T}-\mathbf{w}_{T-1}^{\prime}\|_{1}^{2}\\ \stackrel{{\scriptstyle\rm(3)}}{{\leq}}&\left(\sum_{t=1}^{T-1}\left\langle\boldsymbol{\ell}_{t},\mathbf{w}_{T}^{\prime}\right\rangle+\left\langle\mathbf{m}_{T},\mathbf{w}_{T}^{\prime}\right\rangle+\frac{\ln N+R(\mathbf{w}_{T}^{\prime})}{\beta}\right)+\left\langle\boldsymbol{\ell}_{T}-\mathbf{m}_{T},\mathbf{w}_{T}^{\prime}\right\rangle\\ &-C_{T-1}-\frac{1}{2\beta}\|\mathbf{w}_{T}-\mathbf{w}_{T-1}^{\prime}\|_{1}^{2}-\frac{1}{2\beta}\|\mathbf{w}_{T}-\mathbf{w}_{T}^{\prime}\|_{1}^{2}\\ =&\sum_{t=1}^{T}\left\langle\boldsymbol{\ell}_{t},\mathbf{w}_{T}^{\prime}\right\rangle+\frac{\ln N+R(\mathbf{w}_{T}^{\prime})}{\beta}-C_{T}\\ \stackrel{{\scriptstyle\rm(4)}}{{\leq}}&\sum_{t=1}^{T}\left\langle\boldsymbol{\ell}_{t},\mathbf{w}^{*}\right\rangle+\frac{\ln N+R(\mathbf{w}^{*})}{\beta}-C_{T}.\end{split}

(41)

The first inequality is due to the induction hypothesis with $\mathbf{w}^{\star}=\mathbf{w}_{T-1}^{\prime}$ . The second and the third ones are applications of Equation (40). Specifically, $\mathbf{w}_{T-1}^{\prime}$ and $\mathbf{w}_{T}$ minimize $\sum_{t=1}^{T-1}\beta\left\langle\boldsymbol{\ell}_{t},\mathbf{w}\right\rangle+{R(\mathbf{w})}$ and $\sum_{t=1}^{T-1}\beta\left\langle\boldsymbol{\ell}_{t},\mathbf{w}\right\rangle+\beta\left\langle\mathbf{m}_{T},\mathbf{w}\right\rangle+{R(\mathbf{w})}$ respectively. The forth inequality follows from $\mathbf{w}_{T}^{\prime}$ minimizes $\beta\sum_{t=1}^{T}\left\langle\boldsymbol{\ell}_{t},\mathbf{w}\right\rangle+R(\mathbf{w})$ .

We now demonstrate how to remove the dependence on $\mathbf{w}_{t}^{\prime}$ :

\begin{split}&\frac{1}{2\beta}\sum_{t=1}^{T}\left(\|\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\|_{1}^{2}+\|\mathbf{w}_{t}-\mathbf{w}_{t-1}^{\prime}\|_{1}^{2}\right)\\ \geq&\frac{1}{2\beta}\sum_{t=1}^{T}\left(\|\mathbf{w}_{t}-\mathbf{w}_{t}^{\prime}\|_{1}^{2}+\|\mathbf{w}_{t+1}-\mathbf{w}_{t}^{\prime}\|_{1}^{2}\right)-\frac{1}{2\beta}\|\mathbf{w}_{T+1}-\mathbf{w}_{T}^{\prime}\|_{1}^{2}\\ \geq&\frac{1}{4\beta}\sum_{t=2}^{T}\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}^{2}-\frac{2}{\beta},\end{split}

(42)

where the last inequality follows from $\|\mathbf{x}+\mathbf{y}\|^{2}\leq 2(\|\mathbf{x}\|^{2}+\|\mathbf{y}\|^{2})$ holds for any norm. Now the proof is completed by combining Equations (38) and (42).

B.4 Proof of Theorem 6

We first show $f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t,i})\leq\left\langle\mathbf{w}_{t},\boldsymbol{\ell}_{t}\right\rangle-\ell_{t,i}$ so that Lemma 5.2 can be invoked to bound the regret of the meta algorithm. We start from gsc-convexity,

\begin{split}&f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t,i})\leq-\left\langle\nabla f_{t}(\mathbf{x}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle\\ =&\left\langle\nabla f_{t}(\mathbf{x}_{t}),\sum_{j=1}^{N}w_{t,j}{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,j}\right\rangle-\left\langle\nabla f_{t}(\mathbf{x}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle\\ =&\sum_{j=1}^{N}w_{t,j}\ell_{t,j}-\ell_{t,i}=\left\langle\mathbf{w}_{t},\boldsymbol{\ell}_{t}\right\rangle-\ell_{t,i},\end{split}

(43)

where the first equality is due to the gradient of $\sum_{i=1}^{N}w_{t,i}d(\mathbf{x},\mathbf{x}_{t,i})^{2}$ vanishes at $\mathbf{x}_{t}$ . Now we can bound the regret as

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t,i})\leq\sum_{t=1}^{T}\left\langle\mathbf{w}_{t},\boldsymbol{\ell}_{t}\right\rangle-\ell_{t,i}\\ \leq&\frac{2+\ln N}{\beta}+\beta\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}-\frac{1}{4\beta}\sum_{t=2}^{T}\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}^{2}\\ \end{split}

(44)

It now suffices to bound $\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}$ in terms of the gradient-variation $V_{T}$ and $\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}^{2}$ . We start with the definition of the infinity norm.

\begin{split}&\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}\\ =&\max_{i\in[N]}(\ell_{t,i}-m_{t,i})^{2}\\ =&\max_{i\in[N]}\left(\left\langle\nabla f_{t}(\mathbf{x}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle-\left\langle\nabla f_{t-1}(\bar{\mathbf{x}}_{t}),{\textnormal{Exp}}_{\bar{\mathbf{x}}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle\right)^{2}\\ =&\max_{i\in[N]}\Bigl{(}\Bigl{.}\left\langle\nabla f_{t}(\mathbf{x}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle-\left\langle\nabla f_{t-1}(\mathbf{x}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle+\left\langle\nabla f_{t-1}(\mathbf{x}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle\\ &-\left\langle\Gamma_{\bar{\mathbf{x}}_{t}}^{\mathbf{x}_{t}}\nabla f_{t-1}(\bar{\mathbf{x}}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle+\left\langle\Gamma_{\bar{\mathbf{x}}_{t}}^{\mathbf{x}_{t}}\nabla f_{t-1}(\bar{\mathbf{x}}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle-\left\langle\Gamma_{\bar{\mathbf{x}}_{t}}^{\mathbf{x}_{t}}\nabla f_{t-1}(\bar{\mathbf{x}}_{t}),\Gamma_{\bar{\mathbf{x}}_{t}}^{\mathbf{x}_{t}}{\textnormal{Exp}}_{\bar{\mathbf{x}}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle\Bigl{.}\Bigl{)}^{2}\\ \stackrel{{\scriptstyle\rm(1)}}{{\leq}}&3\max_{i\in[N]}\Bigl{(}\Bigl{.}\left\langle\nabla f_{t}(\mathbf{x}_{t})-\nabla f_{t-1}(\mathbf{x}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle^{2}+\left\langle\nabla f_{t-1}(\mathbf{x}_{t})-\Gamma_{\bar{\mathbf{x}}_{t}}^{\mathbf{x}_{t}}\nabla f_{t-1}(\bar{\mathbf{x}}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle^{2}\\ &+\left\langle\Gamma_{\bar{\mathbf{x}}_{t}}^{\mathbf{x}_{t}}\nabla f_{t-1}(\bar{\mathbf{x}}_{t}),{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}\mathbf{x}_{t,i}-\Gamma_{\bar{\mathbf{x}}_{t}}^{\mathbf{x}_{t}}{\textnormal{Exp}}_{\bar{\mathbf{x}}_{t}}^{-1}\mathbf{x}_{t,i}\right\rangle^{2}\Bigl{.}\Bigl{)}\\ \stackrel{{\scriptstyle\rm(2)}}{{\leq}}&3\left(D^{2}\sup_{\mathbf{x}\in\mathcal{N}}\|\nabla f_{t}(\mathbf{x})-\nabla f_{t-1}(\mathbf{x})\|^{2}+D^{2}L^{2}d(\mathbf{x}_{t},\bar{\mathbf{x}}_{t})^{2}+G^{2}\|{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}(\mathbf{x}_{t,i})-\Gamma_{\bar{\mathbf{x}}_{t}}^{\mathbf{x}_{t}}{\textnormal{Exp}}_{\bar{\mathbf{x}}_{t}}^{-1}(\mathbf{x}_{t,i})\|^{2}\right)\end{split}

(45)

where the first inequality relies on fact that $(a+b+c)^{2}\leq 3(a^{2}+b^{2}+c^{2})$ holds for any $a,b,c\in\mathbb{R}$ , and the second one follows from Assumptions 2, 3, $L$ -gsc-smoothness and Hölder’s inequality.

By Lemma E.12, for a Hadamard manifold with sectional curvature lower bounded by $\kappa$ , $h(\mathbf{x})\coloneqq\frac{1}{2}d(\mathbf{x},\mathbf{x}_{t,i})^{2}$ is $\frac{\sqrt{\kappa}D}{\tanh(\sqrt{\kappa}D)}$ -smooth (which is exactly ${\zeta}$ as in Definition 1) on Hadamard manifolds. Thus

\|{\textnormal{Exp}}_{\mathbf{x}_{t}}^{-1}(\mathbf{x}_{t,i})-\Gamma_{\bar{\mathbf{x}}_{t}}^{\mathbf{x}_{t}}{\textnormal{Exp}}_{\bar{\mathbf{x}}_{t}}^{-1}(\mathbf{x}_{t,i})\|=\|-\nabla h(\mathbf{x}_{t})+\Gamma_{\bar{\mathbf{x}}_{t}}^{\mathbf{x}_{t}}\nabla h(\bar{\mathbf{x}}_{t})\|\leq{\zeta}d(\mathbf{x}_{t},\bar{\mathbf{x}}_{t}).

(46)

We need to bound $d(\mathbf{x}_{t},\bar{\mathbf{x}}_{t})$ in terms of $\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}$ to make full use of the negative term in Lemma 5.2. By Lemma E.3

d(\mathbf{x}_{t},\bar{\mathbf{x}}_{t})\leq\sum_{i=1}^{N}w_{t,i}\cdot d(\mathbf{x}_{t,i},\mathbf{x}_{t,i})+D\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}=D\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}.

(47)

Combining Equations (44), (45), (46) and (47), we have

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t,i})\leq\frac{2+\ln N}{\beta}+3\beta D^{2}(V_{T}+G^{2})+\sum_{t=2}^{T}\left(3\beta(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})-\frac{1}{4\beta}\right)\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}^{2},

(48)

where the $3\beta D^{2}G^{2}$ term is due to the calculation of $V_{T}$ starts from $t=2$ , while $\mathbf{w}_{0}=\mathbf{w}_{1}$ ensures $\mathbf{x}_{1}=\bar{\mathbf{x}}_{1}$ and thus $d(\mathbf{x}_{1},\bar{\mathbf{x}}_{1})=0$ .

B.5 Proof of Theorem 7

The optimal step size, according to Theorem 5 is

\eta^{\star}=\min\left\{\frac{\delta}{1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}},\sqrt{\frac{D^{2}+2DP_{T}}{2{\zeta}V_{T}}}\right\}.

Based on Assumption 3, we know $V_{T}$ has an upper bound $V_{T}=\sum_{t=2}^{T}\sup_{\mathbf{x}\in\mathcal{N}}\|\nabla f_{t}(\mathbf{x})-\nabla f_{t-1}(\mathbf{x})\|^{2}\leq 4G^{2}T$ . Therefore, $\eta^{\star}$ can be bounded by

\sqrt{\frac{D^{2}}{8{\zeta}G^{2}T}}\leq\eta^{\star}\leq\frac{\delta}{1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}}.

According to the construction of $\mathcal{H}$ ,

\min\mathcal{H}=\sqrt{\frac{D^{2}}{8{\zeta}G^{2}T}},\qquad\max\mathcal{H}\geq 2\frac{\delta}{1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}}.

Therefore, there always exists $k\in[N]$ such that $\eta_{k}\leq\eta^{\star}\leq 2\eta_{k}$ . We can bound the regret of the $k$ -th expert as

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\leq\eta_{k}{\zeta}V_{T}+\frac{D^{2}+2DP_{T}}{2\eta_{k}}\leq\eta^{\star}{\zeta}V_{T}+\frac{D^{2}+2DP_{T}}{\eta^{\star}}\\ \leq&\zeta V_{T}\sqrt{\frac{D^{2}+2DP_{T}}{2\zeta V_{T}}}+{(D^{2}+2DP_{T})}\cdot\max\left\{\sqrt{\frac{2\zeta V_{T}}{D^{2}+2DP_{T}}},\frac{1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}}{\delta}\right\}\\ =&\frac{3}{2}\sqrt{2(D^{2}+2DP_{T}){\zeta}V_{T}}+(D^{2}+2DP_{T})\frac{1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}}{\delta}.\\ \end{split}

(49)

Since the dynamic regret can be decomposed as the sum of the meta-regret and the expert-regret,

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})=\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k})+\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t}).

Applying Theorem 6 with $\beta\leq\frac{1}{\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}}$ , we have

\left(3\beta(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})-\frac{1}{4\beta}\right)\leq 0

and

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq\frac{2+\ln N}{\beta}+3\beta D^{2}(V_{T}+G^{2}).

We need to consider two cases based on the value of $\beta$ .

If $\sqrt{\frac{2+\ln N}{3D^{2}(V_{T}+G^{2})}}\leq\frac{1}{\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}}$ , then

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq 2\sqrt{3D^{2}(V_{T}+G^{2})(2+\ln N)}.

Otherwise, we have

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq 2(2+\ln N)\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}.

In sum,

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\\ \leq&\max\left\{2\sqrt{3D^{2}(V_{T}+G^{2})(2+\ln N)},2(2+\ln N)\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}\right\}.\\ \end{split}

(50)

Combining Equations (50) and (49), we have

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\\ \leq&\max\left\{2\sqrt{3D^{2}(V_{T}+G^{2})(2+\ln N)},2(2+\ln N)\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}\right\}\\ &+\frac{3}{2}\sqrt{2(D^{2}+2DP_{T}){\zeta}V_{T}}+(D^{2}+2DP_{T})\frac{1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}}{\delta}\\ =&O\left(\sqrt{(V_{T}+{\zeta}^{2}\ln N)\ln N}\right)+O\left(\sqrt{{\zeta}(V_{T}+(1+P_{T})/\delta^{2})(1+P_{T})}\right)\\ =&{O}\left(\sqrt{{\zeta}(V_{T}+(1+P_{T})/\delta^{2})(1+P_{T})}\right),\end{split}

(51)

where we use ${O}(\cdot)$ to hide $O(\log\log T)$ following Luo and Schapire (2015) and Zhao et al. (2020) and $N=O(\log T)$ leads to $\ln N=O(\log\log T)$ .

Appendix C Omitted Proof for Section 6

C.1 Proof of Lemma 6.1

By $L$ -gsc-smoothness, we have

f(\mathbf{y})\leq f(\mathbf{x})+\left\langle\nabla f(\mathbf{x}),{\textnormal{Exp}}_{\mathbf{x}}^{-1}(\mathbf{y})\right\rangle+\frac{L\cdot d(\mathbf{x},\mathbf{y})^{2}}{2}.

Setting $\mathbf{y}={\textnormal{Exp}}_{\mathbf{x}}\left(-\frac{1}{L}\nabla f(\mathbf{x})\right)$ , we have

\begin{split}0\leq&f(\mathbf{y})\leq f(\mathbf{x})-\frac{1}{L}\|\nabla f(\mathbf{x})\|^{2}+\frac{L}{2}\cdot\frac{1}{L^{2}}\|\nabla f(\mathbf{x})\|^{2}\\ =&f(\mathbf{x})-\frac{1}{2L}\|\nabla f(\mathbf{x})\|^{2},\end{split}

(52)

where we use the non-negativity of $f$ . The above inequality is equivalent to

\|\nabla f(\mathbf{x})\|^{2}\leq 2L\cdot f(\mathbf{x}),

in which the constant is two times better than that of Srebro et al. (2010).

C.2 Proof of Lemma 6.2

The proof is similar to the proof of Theorem 1. Let $\mathbf{x}_{t+1}^{\prime}={\textnormal{Exp}}_{\mathbf{x}_{t}}\left(-\eta\nabla f_{t}(\mathbf{x}_{t})\right)$ , then analog to Equation (10), we have

\begin{split}f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{u}_{t})\leq&\frac{1}{2\eta}\left(\|\textnormal{Exp}^{-1}_{\mathbf{x}_{t}}\mathbf{u}_{t}\|^{2}-\|\textnormal{Exp}^{-1}_{\mathbf{x}_{t+1}}\mathbf{u}_{t+1}\|^{2}+2D\|\textnormal{Exp}^{-1}_{\mathbf{u}_{t}}\mathbf{u}_{t+1}\|\right)+\frac{\eta{\zeta}\|\nabla f_{t}(\mathbf{x}_{t})\|^{2}}{2}\\ \leq&\frac{1}{2\eta}\left(\|\textnormal{Exp}^{-1}_{\mathbf{x}_{t}}\mathbf{u}_{t}\|^{2}-\|\textnormal{Exp}^{-1}_{\mathbf{x}_{t+1}}\mathbf{u}_{t+1}\|^{2}+2D\|\textnormal{Exp}^{-1}_{\mathbf{u}_{t}}\mathbf{u}_{t+1}\|\right)+\eta{\zeta}Lf_{t}(\mathbf{x}_{t}),\end{split}

(53)

where for the second inequality we apply Lemma 6.1. WLOG, we can assume $\mathbf{u}_{T+1}=\mathbf{u}_{T}$ and sum from $t=1$ to $T$ :

\sum_{t=1}^{T}\left(f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{u}_{t})\right)\leq\frac{D^{2}+2DP_{T}}{2\eta}+\eta{\zeta}L\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t}).

After simplifying, we get

\begin{split}\sum_{t=1}^{T}\left(f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{u}_{t})\right)\leq&\frac{D^{2}+2DP_{T}}{2\eta(1-\eta{\zeta}L)}+\frac{\eta{\zeta}L\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})}{1-\eta{\zeta}L}\\ =&\frac{D^{2}+2DP_{T}}{2\eta(1-\eta{\zeta}L)}+\frac{\eta{\zeta}LF_{T}}{1-\eta{\zeta}L}\\ \leq&\frac{D^{2}+2DP_{T}}{\eta}+2\eta{\zeta}LF_{T}\\ =&O\left(\frac{1+P_{T}}{\eta}+\eta F_{T}\right).\\ \end{split}

(54)

where $\eta\leq\frac{1}{2\zeta L}$ is used to obtain the second inequality.

C.3 Proof of Lemma 6.3

We again apply Lemma 5.2, with the surrogate loss $\ell_{t,i}=\left\langle\nabla f_{t}(\mathbf{x}_{t}),\textnormal{Exp}^{-1}_{\mathbf{x}_{t}}\mathbf{x}_{t,i}\right\rangle$ and the optimism $m_{t,i}=0$ for any $i\in[N]$ . In this way,

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\\ \leq&-\sum_{t=1}^{T}\left\langle\nabla f_{t}(\mathbf{x}_{t}),\textnormal{Exp}^{-1}_{\mathbf{x}_{t}}\mathbf{x}_{t,i}\right\rangle=\sum_{t=1}^{T}\left\langle\mathbf{w}_{t},\boldsymbol{\ell}_{t}\right\rangle-w_{t,i}\\ \leq&\frac{2+\ln N}{\beta}+\beta\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}\|_{\infty}^{2}-\frac{1}{4\beta}\sum_{t=2}^{T}\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}^{2}\\ \leq&\frac{2+\ln N}{\beta}+\beta D^{2}\sum_{t=1}^{T}\|\nabla f_{t}(\mathbf{x}_{t})\|^{2}\\ \leq&\frac{2+\ln N}{\beta}+2\beta D^{2}L\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})=\frac{2+\ln N}{\beta}+2\beta D^{2}L\bar{F}_{T},\end{split}

(55)

where the second inequality follows from Lemma 5.2, the third one follows from Assumption 2 and Hölder’s inequality, while the last inequality is due to Lemma 6.1. By setting $\beta=\sqrt{\frac{2+\ln N}{2LD^{2}\bar{F}_{T}}}$ , the regret of the meta algorithm is upper bounded by

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq\sqrt{8D^{2}L(2+\ln N)\bar{F}_{T}}=\sqrt{8D^{2}L(2+\ln N)\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})}.

(56)

Although $\bar{F}_{T}$ is unknown similar to the case of Optimistic Hedge, we can use techniques like the doubling trick or a time-varying step size $\beta_{t}=O\left(\frac{1}{\sqrt{1+\bar{F}_{t}}}\right)$ to overcome this hurdle.

The RHS of Equation (56) depends on the cumulative loss of $\mathbf{x}_{t}$ , which remains elusive. Here we apply an algebraic fact that $x-y\leq\sqrt{ax}$ implies $x-y\leq a+\sqrt{ay}$ holds for any non-negative $x,y$ and $a$ . Then

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq 8D^{2}L(2+\ln N)+\sqrt{8D^{2}L(2+\ln N)\bar{F}_{T,i}}

(57)

where we remind $\bar{F}_{T,i}=\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})$ .

C.4 Proof of Theorem 8

Recall the regret of the meta algorithm as in Lemma 6.3:

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq 8D^{2}L(2+\ln N)+\sqrt{8D^{2}L(2+\ln N)\bar{F}_{T,i}}.

(58)

On the other hand, we know the regret can be decomposed as

\begin{split}\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})=&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})+\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\\ =&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})+\bar{F}_{T,i}-F_{T}.\end{split}

(59)

We can first show there exists an almost optimal step size and bound the regret of the corresponding expert. That regret immediately provides an upper bound of $\bar{F}_{T,i}$ in terms of $F_{T}$ . This argument eliminates the dependence on $\bar{F}_{T,i}$ and leads to a regret bound solely depending on $F_{T}$ and $P_{T}$ .

Now we bound the regret of the best expert. Note that due to Lemma 6.2, the optimal step size is $\eta=\min\left\{\frac{1}{2{\zeta}L},\sqrt{\frac{D^{2}+2DP_{T}}{2{\zeta}LF_{T}}}\right\}$ . According to Assumptions 2, 3, $F_{T}\leq GDT$ . Thus the optimal step size $\eta^{\star}$ is bounded by

\sqrt{\frac{D}{4LGT}}\leq\eta^{\star}\leq\frac{1}{2{\zeta}L}.

Due to our construction of $\mathcal{H}$ , there exists $k\in[N]$ such that $\eta_{k}\leq\eta^{\star}\leq 2\eta_{k}$ .

According to Lemma 6.2,

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\\ \leq&\frac{D^{2}+2DP_{T}}{2\eta_{k}(1-\eta_{k}{\zeta}L)}+\frac{\eta_{k}{\zeta}LF_{T}}{1-\eta_{k}{\zeta}L}\leq\frac{D^{2}+2DP_{T}}{\eta_{k}}+2\eta_{k}{\zeta}LF_{T}\\ \leq&\frac{2(D^{2}+2DP_{T})}{\eta^{\star}}+2\eta^{\star}{\zeta}LF_{T}\\ \leq&2(D^{2}+2DP_{T})\left(2{\zeta}L+\sqrt{\frac{2{\zeta}LF_{T}}{D^{2}+2DP_{T}}}\right)+2{\zeta}LF_{T}\cdot\sqrt{\frac{D^{2}+2DP_{T}}{2{\zeta}LF_{T}}}\\ =&4{\zeta}L(D^{2}+2DP_{T})+3\sqrt{2{\zeta}LF_{T}(D^{2}+2DP_{T})}\\ \leq&\sqrt{2\left(16{\zeta}^{2}L^{2}(D^{2}+2DP_{T})^{2}+18{\zeta}LF_{T}(D^{2}+2DP_{T})\right)}\end{split}

(60)

where we apply $\eta^{\star}\leq\sqrt{\frac{D^{2}+2DP_{T}}{2\zeta LF_{T}}}$ , $\frac{1}{\eta^{\star}}\leq 2{\zeta}L+\sqrt{\frac{2\zeta LF_{T}}{D^{2}+2DP_{T}}}$ and $\sqrt{a}+\sqrt{b}\leq\sqrt{2(a+b)}$ .

Now as we combine Equations (58), (59), (60),

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\\ \leq&8D^{2}L(2+\ln N)+\sqrt{8D^{2}L(2+\ln N)\bar{F}_{T,k}}+\sqrt{2\left(16{\zeta}^{2}L^{2}(D^{2}+2DP_{T})^{2}+18{\zeta}LF_{T}(D^{2}+2DP_{T})\right)}\\ \leq&8D^{2}L(2+\ln N)+\sqrt{8D^{2}L(2+\ln N)\left(F_{T}+\sqrt{2\left(16{\zeta}^{2}L^{2}(D^{2}+2DP_{T})^{2}+18{\zeta}LF_{T}(D^{2}+2DP_{T})\right)}\right)}\\ &+\sqrt{2\left(16{\zeta}^{2}L^{2}(D^{2}+2DP_{T})^{2}+18{\zeta}LF_{T}(D^{2}+2DP_{T})\right)}\\ =&O(\sqrt{\zeta({\zeta}(1+P_{T})+F_{T})(P_{T}+1)}).\end{split}

(61)

where we again use $O(\cdot)$ to hide the $\log\log T$ term.

Appendix D Omitted Proof for Section 7

D.1 Necessity of Best-of-both-worlds Bound

We highlight the necessity of achieving a best-of-both-worlds bound by computing the Fréchet mean in the online setting on the $d$ -dimensional unit Poincaré disk. The Poincaré disk looks like a unit ball in Euclidean space, but its Riemannian metric blows up near the boundary:

\left\langle\mathbf{u},\mathbf{v}\right\rangle_{\mathbf{x}}=\frac{4\left\langle\mathbf{u},\mathbf{v}\right\rangle_{2}}{(1-\|\mathbf{x}\|_{2}^{2})^{2}}

and has constant sectional curvature $-1$ . We use $\mathbf{0}$ to denote the origin of the Poincaré ball and $\mathbf{e}_{i}$ to be the $i$ -th unit vector in the standard basis. The Poincaré ball has the following property (Lou et al., 2020):

\begin{split}&d(\mathbf{x},\mathbf{y})=\operatorname{arcosh}\left(1+\frac{2\|\mathbf{x}-\mathbf{y}\|_{2}^{2}}{(1-\|\mathbf{x}\|_{2}^{2})(1-\|\mathbf{y}\|_{2}^{2})}\right)\\ &\textnormal{Exp}^{-1}_{\mathbf{0}}\mathbf{y}=\operatorname{arctanh}(\|\mathbf{y}\|_{2})\frac{\mathbf{y}}{\|\mathbf{y}\|_{2}}.\end{split}

Now consider the following loss function

f_{t}(\mathbf{x})=\sum_{i=1}^{2d}\frac{d(\mathbf{x},\mathbf{x}_{t,i})^{2}}{2d}

where $\mathbf{x}_{t,i}=\frac{t}{2T}\mathbf{e}_{i}$ for $1\leq i\leq d$ and $\mathbf{x}_{t,i}=-\frac{t}{2T}\mathbf{e}_{i-d}$ for $d+1\leq i\leq 2d$ . We choose $\mathcal{N}$ to be the convex hull of $\pm\frac{1}{2}\mathbf{e}_{i}$ , $i=1,\dots,d$ . And the comparator is $\mathbf{u}_{t}=\operatorname*{argmin}_{\mathbf{u}_{t}\in\mathcal{N}}f_{t}(\mathbf{u}_{t})$ which is indeed the origin $\mathbf{0}$ due to symmetry. Now we can bound $V_{T}$ by

\begin{split}V_{T}=&\sum_{t=2}^{T}\sup_{\mathbf{x}\in\mathcal{N}}\|\nabla f_{t}(\mathbf{x})-\nabla f_{t-1}(\mathbf{x})\|^{2}\\ =&\frac{1}{4d^{2}}\sum_{t=2}^{T}\sup_{\mathbf{x}\in\mathcal{N}}\left\|\sum_{i=1}^{2d}\left(\textnormal{Exp}^{-1}_{\mathbf{x}}\mathbf{x}_{t,i}-\textnormal{Exp}^{-1}_{\mathbf{x}}\mathbf{x}_{t-1,i}\right)\right\|^{2}\\ \leq&\frac{1}{4d^{2}}\sum_{t=2}^{T}c\cdot\left(\sum_{i=1}^{2d}d(\mathbf{x}_{t,i},\mathbf{x}_{t-1,i})\right)^{2}=\sum_{t=2}^{T}c\left(\operatorname{arcosh}\left(1+\frac{2(\frac{t}{2T}-\frac{t-1}{2T})^{2}}{(1-(\frac{t}{2T})^{2})(1-(\frac{t-1}{2T})^{2})}\right)\right)^{2}\\ \leq&\sum_{t=2}^{T}c\left(\operatorname{arcosh}\left(1+\frac{8}{9T^{2}}\right)\right)^{2}\\ \leq&\sum_{t=2}^{T}c\cdot\left(\frac{8}{9T^{2}}+\sqrt{\frac{64}{81T^{4}}+\frac{16}{9T^{2}}}\right)^{2}=\sum_{t=2}^{T}c\cdot O\left(\frac{1}{{T^{2}}}\right)=O\left(\frac{1}{T}\right),\\ \end{split}

(62)

where the first inequality is due to triangle inequality and Lemma E.15. We note that $c$ is a constant depending on the diamater of $\mathcal{N}$ and the sectional curvature of $\mathcal{M}$ . The second one is due to $t\leq T$ , while the third inequality follows from $\operatorname{arcosh}(1+x)\leq x+\sqrt{x^{2}+2x}$ .

Similarly, we can evaluate $F_{T}$

\begin{split}F_{T}=&\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})=\sum_{t=1}^{T}f_{t}(\mathbf{0})=\sum_{t=1}^{T}\sum_{i=1}^{2d}\frac{d(\mathbf{0},\mathbf{x}_{t,i})^{2}}{2d}\\ =&\sum_{t=1}^{T}\operatorname{arcosh}\left(\frac{1+\frac{t^{2}}{4T^{2}}}{1-\frac{t^{2}}{4T^{2}}}\right)=\int_{0}^{T}\operatorname{arcosh}\left(\frac{1+\frac{t^{2}}{4T^{2}}}{1-\frac{t^{2}}{4T^{2}}}\right)dt+O(1)\\ =&2T\int_{0}^{\frac{1}{2}}\operatorname{arcosh}\frac{1+a^{2}}{1-a^{2}}da+O(1)\\ =&2T\left(a\cdot\operatorname{arcosh}\frac{1+a^{2}}{1-a^{2}}+\ln(1-a^{2})\right)\big{|}_{0}^{\frac{1}{2}}+O(1)=\Theta(T).\end{split}

(63)

When the input losses change smoothly, $V_{T}\ll F_{T}$ and the gradient-variation bound is much tighter than the small-loss bound.

There also exist scenarios in which the small-loss bound is tighter. We still consider computing the Fréchet mean on the Poincaré disk

f_{t}(\mathbf{x})=\sum_{i=1}^{n}d(\mathbf{x},\mathbf{x}_{t,i})^{2}/n,

but assume $\mathbf{x}_{t,i}=\mathbf{y}_{i}$ when $t$ is odd and $x_{t,i}=-\mathbf{y}_{i}$ when $t$ is even. We restrict $\mathbf{y}_{1},\dots,\mathbf{y}_{n}\in\mathbb{B}(\frac{\mathbf{e}_{1}}{2},T^{-\alpha})$ where $\mathbb{B}(\mathbf{p},r)$ is the geodesic ball centered at $\mathbf{p}$ and with radius $r$ . $\mathcal{N}$ is the convex hull of $\mathbb{B}(\frac{\mathbf{e}_{1}}{2},T^{-\alpha})\cup\mathbb{B}(-\frac{\mathbf{e}_{1}}{2},T^{-\alpha})$ . Since the input sequence is alternating, $\sup_{\mathbf{x}\in\mathcal{N}}\|\nabla f_{t}(\mathbf{x})-\nabla f_{t-1}(\mathbf{x})\|^{2}$ is a constant over time, and we can lower bounded it by

\begin{split}&\sup_{\mathbf{x}\in\mathcal{N}}\|\nabla f_{t}(\mathbf{x})-\nabla f_{t-1}(\mathbf{x})\|^{2}\\ =&\sup_{\mathbf{x}\in\mathcal{N}}\left\|\frac{1}{n}\left(\sum_{i=1}^{n}\textnormal{Exp}^{-1}_{\mathbf{x}}\mathbf{x}_{t,i}-\sum_{i=1}^{n}\textnormal{Exp}^{-1}_{\mathbf{x}}\mathbf{x}_{t-1,i}\right)\right\|^{2}\\ =&\sup_{\mathbf{x}\in\mathcal{N}}\left\|\frac{2}{n}\sum_{i=1}^{n}\textnormal{Exp}^{-1}_{\mathbf{x}}\mathbf{y}_{i}\right\|^{2}\geq\frac{4}{n^{2}}\left\|\sum_{i=1}^{n}\textnormal{Exp}^{-1}_{\mathbf{0}}\mathbf{y}_{i}\right\|^{2}\\ =&\frac{4}{n^{2}}\left(\left\|\left(\sum_{i=1}^{n}\textnormal{Exp}^{-1}_{\mathbf{0}}\mathbf{y}_{i}\right)^{\parallel}\right\|^{2}+\left\|\left(\sum_{i=1}^{n}\textnormal{Exp}^{-1}_{\mathbf{0}}\mathbf{y}_{i}\right)^{\perp}\right\|^{2}\right)\\ \geq&\frac{4}{n^{2}}\left\|\left(\sum_{i=1}^{n}\textnormal{Exp}^{-1}_{\mathbf{0}}\mathbf{y}_{i}\right)^{\parallel}\right\|^{2}\geq\frac{4}{n^{2}}\left\|n\cdot\operatorname{arctanh}\left(\frac{1}{2}-T^{-\alpha}\right)\mathbf{e}_{1}\right\|_{\mathbf{0}}^{2}\\ =&16\operatorname{arctanh}^{2}\left(\frac{1}{2}-T^{-\alpha}\right)\\ \geq&16\left(\frac{\frac{1}{2}-T^{-\alpha}}{\frac{3}{2}-T^{-\alpha}}\right)^{2}=\Omega(1).\\ \end{split}

(64)

where we use $\mathbf{a}^{\parallel}$ and $\mathbf{a}^{\perp}$ to denote components parallel and orthogonal to the direction of $\mathbf{e}_{1}$ , respectively. The key observation is the lower bound attains when $\left(\sum_{i=1}^{n}\textnormal{Exp}^{-1}_{\mathbf{0}}\mathbf{y}_{i}\right)^{\perp}$ is zero, and each $\mathbf{y}_{i}$ has the smallest component along $\mathbf{e}_{1}$ , i.e., $\mathbf{y}_{i}=\left(\frac{1}{2}-T^{-\alpha}\right)\mathbf{e}_{1}$ . We also use $\operatorname{arctanh}(x)\geq\frac{x}{1+x}$ . Thus we have $V_{T}=\Omega(T)$ . Now we consider $F_{T}$ . By Lemma E.8, we know that $\mathbf{u}_{t}$ lies within the same geodesic ball as $\mathbf{x}_{t,i}$ , $i\in[n]$ . Thus

F_{T}=\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})=\sum_{t=1}^{T}\sum_{i=1}^{n}d(\mathbf{u}_{t},\mathbf{x}_{t,i})^{2}/n\leq T\cdot\left(\frac{2}{T^{\alpha}}\right)^{2}=O(T^{1-2\alpha}).

We can see whenever $\alpha>0$ , $F_{T}=o(T)$ but $V_{T}=\Omega(T)$ .

D.2 Proof of Theorem 9

By Lemma 5.2 we have

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-f_{t}(\mathbf{x}_{t,i})\leq\frac{2+\ln N}{\beta}+\beta\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}-\frac{1}{4\beta}\sum_{t=2}^{T}\|\mathbf{w}_{t}-\mathbf{w}_{t-1}\|_{1}^{2}

(65)

We bound $\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}$ in terms of $\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}^{v}\|_{\infty}^{2}$ and $\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}^{s}\|_{\infty}^{2}$ as follows. By Assumptions 2 and 3, $\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{2}^{2}\leq 4NG^{2}D^{2}$ and we can compute $d_{t}(\mathbf{m})=\|\boldsymbol{\ell}_{t}-\mathbf{m}\|_{2}^{2}$ is $\frac{1}{8NG^{2}D^{2}}$ -exp-concave. . We have

\begin{split}&\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{\infty}^{2}\leq\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}\|_{2}^{2}\\ \leq&\min\left\{\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}^{v}\|_{2}^{2},\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}^{s}\|_{2}^{2}\right\}+8NG^{2}D^{2}\ln 2\\ \leq&N\min\left\{\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}^{v}\|_{\infty}^{2},\sum_{t=1}^{T}\|\boldsymbol{\ell}_{t}-\mathbf{m}_{t}^{s}\|_{\infty}^{2}\right\}+8NG^{2}D^{2}\ln 2,\\ \end{split}

(66)

where for the second inequality we use Lemma E.17 and for the first and the third one the norm inequality $\|\mathbf{x}\|_{\infty}\leq\|\mathbf{x}\|_{2}\leq\sqrt{N}\|\mathbf{x}\|_{\infty}$ is used.

Combining Equations (50), (65), (66) and Lemma 6.3, we have

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq\frac{2+\ln T}{\beta}+\beta N\left(D^{2}\min\{3V_{T},\bar{F}_{T}\}+8G^{2}D^{2}\ln 2\right)

holds for any $\beta\leq\frac{1}{\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}}$ and $i\in[N]$ .

Suppose $\beta^{\star}=\sqrt{\frac{2+\ln N}{N(D^{2}\min\{3(V_{T}+G^{2}),\bar{F}_{T}\}+8G^{2}D^{2}\ln 2)}}\leq\frac{1}{\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}}$ , then

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq 2\sqrt{(2+\ln N)N(D^{2}\min\{3(V_{T}+G^{2}),\bar{F}_{T}\}+8G^{2}D^{2}\ln 2)}.

Otherwise

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\leq 2(2+\ln N)\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}.

In sum, we have

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})\\ \leq&\max\left\{2\sqrt{(2+\ln N)N(D^{2}\min\{3(V_{T}+G^{2}),\bar{F}_{T}\}+8G^{2}D^{2}\ln 2)},2(2+\ln N)\sqrt{12(D^{4}L^{2}+D^{2}G^{2}{\zeta}^{2})}\right\}\\ =&O(\log T\cdot\min\{V_{T},\bar{F}_{T}\}).\end{split}

(67)

D.3 Proof of Theorem 10

By Theorem 9, we know

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,i})=O\left(\sqrt{\ln T(\min\{V_{T},\bar{F}_{T}\})}\right)

holds for any $i\in[N^{v}+N^{s}]$ . WLOG, we assume $k$ and $k^{\prime}$ to be indexes of the best experts for the gradient-variation bound and the small-loss bound, respectively. Then by Theorem 7,

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k})-f_{t}(\mathbf{u}_{t})\leq\frac{3}{2}\sqrt{2(D^{2}+2DP_{T}){\zeta}V_{T}}+(D^{2}+2DP_{T})\frac{1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}}{\delta}

(68)

while by Theorem 8,

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k^{\prime}})-f_{t}(\mathbf{u}_{t})\leq\sqrt{2\left(16{\zeta}^{2}L^{2}(D^{2}+2DP_{T})^{2}+18{\zeta}LF_{T}(D^{2}+2DP_{T})\right)}.

(69)

Since the regret admits the following decompositions

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\\ =&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k})+\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\\ =&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k^{\prime}})+\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t,k^{\prime}})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\\ ,\end{split}

(70)

we indeed have

\begin{split}&\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})\\ \leq&O\left(\sqrt{\ln T(\min\{V_{T},\bar{F}_{T}\})}\right)+\min\left\{\frac{3}{2}\sqrt{2(D^{2}+2DP_{T}){\zeta}V_{T}}+(D^{2}+2DP_{T})\frac{1+(1+2\zeta\delta^{2}L^{2})^{\frac{1}{2}}}{\delta},\right.\\ &\left.\sqrt{2\left(16{\zeta}^{2}L^{2}(D^{2}+2DP_{T})^{2}+18{\zeta}LF_{T}(D^{2}+2DP_{T})\right)}\right\}\\ =&O\left(\sqrt{\ln T(\min\{V_{T},\bar{F}_{T}\})}\right)\\ &+O\left(\min\left\{\sqrt{{\zeta}(1+P_{T})((1+P_{T})/\delta^{2}+V_{T})},\sqrt{{\zeta}(1+P_{T})(\zeta(1+P_{T})+F_{T})}\right\}\right)\end{split}

(71)

Note that $\bar{F}_{T}$ can be processed similarly as in Lemma 6.3 and Theorem 8 to get $F_{T}$ . In sum, the regret is bounded by

\sum_{t=1}^{T}f_{t}(\mathbf{x}_{t})-\sum_{t=1}^{T}f_{t}(\mathbf{u}_{t})=O\left(\sqrt{\zeta(P_{T}(\zeta+1/\delta^{2})+\min\{V_{T},F_{T}\}+1)(1+P_{T})+\ln T\min\{V_{T},F_{T}\}}\right).

Appendix E Technical Lemmas

We need the following technical lemmas.

Lemma E.1.

(Bento et al., 2021, Theorem 2.1) Suppose $f:\mathcal{M}\rightarrow\mathbb{R}$ is geodesically convex and $\mathcal{M}$ is Hadamard. The geodesic mean $\bar{\mathbf{x}}_{k}$ w.r.t coefficients $a_{1},\dots,a_{N}$ ( $\sum_{i=1}^{N}a_{i}=1$ , $a_{i}\geq 0$ ) is defined as:

\begin{split}&\bar{\mathbf{x}}_{1}=\mathbf{x}_{1}\\ &\bar{\mathbf{x}}_{k}={\textnormal{Exp}}_{\bar{\mathbf{x}}_{k-1}}\left(\frac{a_{k}}{\sum_{i=1}^{k}a_{i}}{\textnormal{Exp}}_{\bar{\mathbf{x}}_{k-1}}^{-1}\mathbf{x}_{k}\right),\quad k>1.\end{split}

(72)

Then we have

f(\bar{\mathbf{x}}_{N})\leq\sum_{i=1}^{N}a_{i}f(\mathbf{x}_{i}).

(73)

Proof E.2.

We use induction to show a stronger statement

f(\bar{\mathbf{x}}_{k})\leq\sum_{i=1}^{k}\frac{a_{i}}{\sum_{j=1}^{k}a_{j}}f(\mathbf{x}_{i})

holds for $k=1,\dots,N$ .

For $k=1$ , this is obviously true because $\bar{\mathbf{x}}_{1}=\mathbf{x}_{1}$ . Suppose

f(\bar{\mathbf{x}}_{k})\leq\sum_{i=1}^{k}\frac{a_{i}}{\sum_{j=1}^{k}a_{j}}f(\mathbf{x}_{i})

holds for some $k$ , then by geodesic convexity,

\begin{split}f(\bar{\mathbf{x}}_{k+1})\leq&\left(1-\frac{a_{k+1}}{\sum_{j=1}^{k+1}a_{j}}\right)f(\bar{\mathbf{x}}_{k})+\frac{a_{k+1}}{\sum_{j=1}^{k+1}a_{j}}f(\mathbf{x}_{k+1})\\ \leq&\sum_{i=1}^{k}\frac{a_{i}}{\sum_{j=1}^{k+1}a_{j}}f(\mathbf{x}_{i})+\frac{a_{k+1}}{\sum_{j=1}^{k+1}a_{j}}f(\mathbf{x}_{k+1})\\ =&\sum_{i=1}^{k+1}\frac{a_{i}f(\mathbf{x}_{i})}{\sum_{j=1}^{k+1}a_{j}}.\end{split}

(74)

The first inequality is due to gsc-convexity: for the geodesic determined by $\gamma(0)=\bar{\mathbf{x}}_{k}$ and $\gamma(1)=\mathbf{x}_{k+1}$ we have $\bar{\mathbf{x}}_{k+1}=\gamma\left(\frac{a_{k+1}}{\sum_{i=1}^{k+1}a_{i}}\right)$ and thus $f(\gamma(t))\leq(1-t)f(\gamma(0))+tf(\gamma(1))$ . For the second inequality, we use the induction hypothesis. Given $\sum_{i=1}^{N}a_{i}=1$ , the lemma is proved.

The computation of the geodesic averaging is summarized in Algorithm E, which serves as a sub-routine of Radar.

{algorithm2e}

[H] Geodesic Averaging \KwData $N$ points $\mathbf{x}_{1},\dots,\mathbf{x}_{N}\in\mathcal{N}$ and $N$ real weights $w_{1},\dots,w_{N}$ . Let $\bar{\mathbf{x}}_{1}=\mathbf{x}_{1}$
\For $k=2,\dots,N$ $\bar{\mathbf{x}}_{k}={\textnormal{Exp}}_{\bar{\mathbf{x}}_{k-1}}\left(\frac{w_{k}}{\sum_{i=1}^{k}w_{i}}{\textnormal{Exp}}^{-1}_{\bar{\mathbf{x}}_{k-1}}\mathbf{x}_{k}\right)$
Return $\bar{\mathbf{x}}_{N}$ .

Lemma E.3.

Suppose $\mathbf{x}_{1},\dots,\mathbf{x}_{N},\mathbf{y}_{1},\dots,\mathbf{y}_{N}\in\mathcal{N}$ where $\mathcal{N}$ is a gsc-convex subset of a Hadamard manifold $\mathcal{M}$ and the diameter of $\mathcal{N}$ is upper bounded by $D$ . Let $\bar{\mathbf{x}}$ and $\bar{\mathbf{y}}$ be the weighted Fréchet mean with respect to coefficient vectors $\mathbf{a}$ and $\mathbf{b}$ $(\mathbf{a},\mathbf{b}\in\Delta_{N})$ , defined as $\bar{\mathbf{x}}=\operatorname*{argmin}_{\mathbf{x}\in\mathcal{N}}\sum_{i=1}^{N}a_{i}\cdot d(\mathbf{x},\mathbf{x}_{i})^{2}$ and $\bar{\mathbf{y}}=\operatorname*{argmin}_{\mathbf{y}\in\mathcal{N}}\sum_{i=1}^{N}b_{i}\cdot d(\mathbf{y},\mathbf{y}_{i})^{2}$ . Then we have

d(\bar{\mathbf{x}},\bar{\mathbf{y}})\leq\sum_{i=1}^{N}a_{i}\cdot d(\mathbf{x}_{i},\mathbf{y}_{i})+D\sum_{i=1}^{N}|a_{i}-b_{i}|.

(75)

Proof E.4.

Recall that on Hadamard manifolds, the following inequality (Sturm, 2003, Prop. 2.4)

d(\mathbf{x},\mathbf{y})^{2}+d(\mathbf{u},\mathbf{v})^{2}\leq d(\mathbf{x},\mathbf{v})^{2}+d(\mathbf{y},\mathbf{u})^{2}+2d(\mathbf{x},\mathbf{u})\cdot d(\mathbf{y},\mathbf{v})

holds for any $\mathbf{x},\mathbf{y},\mathbf{u},\mathbf{v}\in\mathcal{M}$ . A direct application of the above inequality yields

d(\mathbf{x}_{i},\bar{\mathbf{y}})^{2}+d(\mathbf{y}_{i},\bar{\mathbf{x}})^{2}\leq d(\mathbf{x}_{i},\bar{\mathbf{x}})^{2}+d(\mathbf{y}_{i},\bar{\mathbf{y}})^{2}+2d(\bar{\mathbf{x}},\bar{\mathbf{y}})\cdot d(\mathbf{x}_{i},\mathbf{y}_{i})\qquad\forall i\in[N].

(76)

By (Bacák, 2014a, Theorem 2.4):

\sum_{i=1}^{N}a_{i}\cdot d(\mathbf{x}_{i},\bar{\mathbf{x}})^{2}+\sum_{i=1}^{N}b_{i}\cdot d(\mathbf{y}_{i},\bar{\mathbf{y}})^{2}+2d(\bar{\mathbf{x}},\bar{\mathbf{y}})^{2}\leq\sum_{i=1}^{N}a_{i}\cdot d(\mathbf{x}_{i},\bar{\mathbf{y}})^{2}+\sum_{i=1}^{N}b_{i}\cdot d(\mathbf{y}_{i},\bar{\mathbf{x}})^{2}

(77)

Multiplying Equation (76) by $a_{i}$ , summing from $i=1$ to $n$ and adding Equation (77), we have

\begin{split}2d(\bar{\mathbf{x}},\bar{\mathbf{y}})^{2}\leq&2d(\bar{\mathbf{x}},\bar{\mathbf{y}})\sum_{i=1}^{N}a_{i}\cdot d(\mathbf{x}_{i},\mathbf{y}_{i})+\sum_{i=1}^{N}(a_{i}-b_{i})d(\mathbf{y}_{i},\bar{\mathbf{y}})^{2}+\sum_{i=1}^{N}(b_{i}-a_{i})d(\mathbf{y}_{i},\bar{\mathbf{x}})^{2}\\ =&2d(\bar{\mathbf{x}},\bar{\mathbf{y}})\sum_{i=1}^{N}a_{i}\cdot d(\mathbf{x}_{i},\mathbf{y}_{i})+\sum_{i=1}^{N}(a_{i}-b_{i})\cdot(d(\mathbf{y}_{i},\bar{\mathbf{y}})-d(\mathbf{y}_{i},\bar{\mathbf{x}}))\cdot(d(\mathbf{y}_{i},\bar{\mathbf{y}})+d(\mathbf{y}_{i},\bar{\mathbf{x}}))\\ \leq&2d(\bar{\mathbf{x}},\bar{\mathbf{y}})\sum_{i=1}^{N}a_{i}\cdot d(\mathbf{x}_{i},\mathbf{y}_{i})+2D\sum_{i=1}^{N}|a_{i}-b_{i}|d(\bar{\mathbf{x}},\bar{\mathbf{y}}),\end{split}

(78)

where for the last inequality we use the triangle inequality for geodesic metric spaces and Assumption 2. Now dividing both sides by $2d(\bar{\mathbf{x}},\bar{\mathbf{y}})$ and we complete the proof.

Lemma E.5.

(Zhang and Sra, 2016, Lemma 5). Let $\mathcal{M}$ be a Riemannian manifold with sectional curvature lower bounded by $\kappa\leq 0$ . Consider $\mathcal{N}$ , a gsc-convex subset of $\mathcal{M}$ with diameter $D$ . For a geodesic triangle fully lies within $\mathcal{N}$ with side lengths $a,b,c$ , we have

a^{2}\leq{\zeta}(\kappa,D)b^{2}+c^{2}-2bc\cos A

where ${\zeta}(\kappa,D):=\sqrt{-\kappa}D\operatorname{coth}(\sqrt{-\kappa}D)$ .

Lemma E.6.

(Sakai, 1996, Prop. 4.5) Let $\mathcal{M}$ be a Riemannian manifold with sectional curvature upper bounded by $\kappa\leq 0$ . Consider $\mathcal{N}$ , a gsc-convex subset of $\mathcal{M}$ with diameter $D$ . For a geodesic triangle fully lies within $\mathcal{N}$ with side lengths $a,b,c$ , we have

a^{2}\geq b^{2}+c^{2}-2bc\cos A.

Lemma E.7.

(Bacák, 2014b, Theorem 2.1.12) Let $(\mathcal{H},d)$ be a Hadamard space and $C\subset\mathcal{H}$ be a closed convex set. Then $\Pi_{C}{\mathbf{x}}$ is singleton and $d(\mathbf{x},\Pi_{C}\mathbf{x})\leq d(\mathbf{x},\mathbf{z})$ for any $\mathbf{z}\in C\setminus\{\Pi_{C}\mathbf{x}\}$ .

Lemma E.8.

(Sturm, 2003, Prop. 6.1 and Theorem 6.2) Suppose $\mathbf{x}_{1},\dots,\mathbf{x}_{N}\in\mathcal{N}$ where $\mathcal{N}$ is a bounded gsc-convex subset of a Hadamard space. $\bar{\mathbf{x}}$ is the weighted Fréchet mean of $\mathbf{x}_{1},\dots,\mathbf{x}_{N}\in\mathcal{N}$ w.r.t. non-negative $w_{1},\dots,w_{N}$ such that $\sum_{i=1}w_{i}=1$ and $f:\mathcal{N}\rightarrow R$ is a gsc-convex function. Then $\bar{\mathbf{x}}\in\mathcal{N}$ and

f(\bar{\mathbf{x}})\leq\sum_{i=1}^{N}w_{i}f(\mathbf{x}_{i}).

Lemma E.9.

Let

g(x)\coloneqq\frac{-a+({a^{2}+bx^{2}})^{\frac{1}{2}}}{x},

where $a,b\in\mathbb{R}^{+}$ , then $g(x)$ increases on $[0,\infty)$ .

Proof E.10.

Taking the derivative w.r.t. $x$ , we have

\begin{split}g^{\prime}(x)=&\frac{\frac{1}{2}\cdot 2bx(a^{2}+bx^{2})^{-\frac{1}{2}}\cdot x-(-a+(a^{2}+bx^{2})^{\frac{1}{2}})\cdot 1}{x^{2}}\\ =&\frac{bx^{2}-(-a\sqrt{a^{2}+bx^{2}}+a^{2}+bx^{2})}{x^{2}\sqrt{a^{2}+bx^{2}}}\\ =&\frac{a\sqrt{a^{2}+bx^{2}}-a^{2}}{x^{2}\sqrt{a^{2}+bx^{2}}}\geq 0\end{split}

holds for any $x>0$ . By L’Hôpital’s rule, $g(0)=0$ and $g^{\prime}(0)=\frac{b}{2a}$ . Thus we know $g(x)$ increases on $[0,\infty)$ .

Lemma E.11.

(Zhou and Huang, 2019, Theorem 3.1) On a Hadamard manifold $\mathcal{M}$ , a subset $C$ gsc-convex, iff it contains the geodesic convex combinations of any countable points in $C$ .

Lemma E.12.

(Ahn and Sra, 2020, Prop. H.1) Let $\mathcal{M}$ be a Riemannian manifold with sectional curvatures lower bounded by $-\kappa<0$ and the distance function $d(\mathbf{x})=\frac{1}{2}d(\mathbf{x},\mathbf{p})^{2}$ where $\mathbf{p}\in\mathcal{M}$ . For $D\geq 0$ , $d(\cdot)$ is gsc-smooth within the domain $\{\mathbf{u}\in\mathcal{M}:d(\mathbf{u},\mathbf{p})\leq D\}$ .

Lemma E.13.

(Ballmann, 2012, Corollary 5.6) Let $\mathcal{M}$ be a Hadamard space and $C\subset\mathcal{M}$ a convex subset. Then $d(\mathbf{z},C)$ is gsc-convex for $\mathbf{z}\in\mathcal{M}$ .

Lemma E.14.

(Bacák, 2014b, Section 2.1) Let $\mathcal{H}$ be a Hadamard manifold, $f:\mathcal{H}\rightarrow(-\infty,\infty)$ be a convex lower semicontinuous function. Then any $\beta$ -sublevel set of $f$ :

\{\mathbf{x}\in\mathcal{H}:f(\mathbf{x})\leq\beta\}

is a closed convex set.

Lemma E.15.

(Sun et al., 2019, Lemma 4) Let the sectional curvature of $\mathcal{M}$ is in $[-K,K]$ and $\mathbf{x},\mathbf{y},\mathbf{z}\in\mathcal{M}$ with pairwise distance upper bounded by $R$ . Then

\|\textnormal{Exp}^{-1}_{\mathbf{x}}\mathbf{y}-\textnormal{Exp}^{-1}_{\mathbf{x}}\mathbf{z}\|\leq(1+c(K)R^{2})d(\mathbf{y},\mathbf{z}).

Lemma E.16.

(Duchi et al., 2012, Lemma 2) Let

\Pi_{\mathcal{X}}^{\psi}(\mathbf{z},\alpha)=\operatorname*{argmin}_{\mathbf{x}\in\mathcal{X}}\left\langle\mathbf{z},\mathbf{x}\right\rangle+\frac{\psi(\mathbf{x})}{\alpha}

where $\psi(\cdot)$ is $1$ -strongly convex w.r.t. $\|\cdot\|$ , then

\|\Pi_{\mathcal{X}}^{\psi}(\mathbf{u},\alpha)-\Pi_{\mathcal{X}}^{\psi}(\mathbf{v},\alpha)\|\leq\alpha\|\mathbf{u}-\mathbf{v}\|_{\star}.

Lemma E.17.

(Cesa-Bianchi and Lugosi, 2006, Prop. 3.1 and Theorem 3.2) Suppose the loss function $\ell_{t}$ is exp-concave for $\eta>0$ , then the regret of Hedge is $\frac{\ln N}{\eta}$ , where $N$ is the number of experts.

Minimizing Dynamic Regret on Geodesic Metric Spaces

Abstract

keywords:

1 Introduction

2 Preliminaries

OCO in Euclidean space.

3 Related Work

3.1 OCO in Euclidean Space

Static regret.

Dynamic regret.

3.2 Online Learning and Optimism on Riemannian Manifolds

4 Path Length Dynamic Regret Bound on Manifolds

Assumption 1

Assumption 2

Assumption 3

Definition 1

Theorem 1

Lemma 4.1.

Theorem 2.

Remark 4.2.

Theorem 3.

5 Gradient-variation Bound on Manifolds

Expert Algorithm.

Definition 4.

Theorem 5.

Proof sketch.

Remark 5.1.

Meta algorithm.

Lemma 5.2.

Theorem 6.

Theorem 7.

6 Small-loss Bound on Manifolds

Lemma 6.1.

Lemma 6.2.

Lemma 6.3.

Theorem 8.

Remark 6.4.

7 Best-of-both-worlds Bound on Manifolds

Theorem 9.

Theorem 10.

Remark 7.1.

8 Conclusion

References

Appendix A Omitted Proof for Section 4

A.1 Proof of Theorem 1

A.2 Proof of Lemma 4.1

A.3 Proof of Theorem 2

A.4 Minimax Dynamic Regret Lower Bound on Hadamard Manifolds

Definition 11.

Lemma A.1.

Lemma A.2.

Proof A.3.

Remark A.4.

Lemma A.5.

Proof A.6.

Lemma A.7.

Proof A.8.

Lemma A.9.

Proof A.10.

Lemma A.11.

Proof A.12.

Theorem 12.

Proof A.13.

Theorem 13.

Proof A.14.

Appendix B Omitted Proof for Section 5

B.1 Proof of Theorem 5

B.2 Extension of Theorem 5 to CAT(κ)(\kappa) Spaces

Assumption 4

Lemma B.1.

Definition 14.

Lemma B.2.

Theorem 15.

Proof B.3.

B.3 Proof of Lemma 5.2

B.4 Proof of Theorem 6

B.5 Proof of Theorem 7

Appendix C Omitted Proof for Section 6

C.1 Proof of Lemma 6.1

C.2 Proof of Lemma 6.2

B.2 Extension of Theorem 5 to CAT $(\kappa)$ Spaces