Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning

In the past decade, VIO has been an active topic in the field of SLAM. Due to the complementary sensors, VIO exhibits enhanced robustness compared to VO across various scenarios and makes scale observable in monocular setups.

Classic VIO systems are mainly based on tightly coupled approaches, wherein visual and IMU constraints are fused through filtering or nonlinear optimization. Filter-based methods, such as MSCKF [22, 17], and ROVIO [2], use the extended Kalman filter (EKF) to propagate and update the current state. While nonlinear-optimization-based methods, like VINS [24, 25], ORB-SLAM3 [6] and DM-VIO [30], adopt local visual-inertial bundle adjustment for the state estimation, achieving more accurate pose tracking.

Classic VIO uses optical flow or hand-craft features for visual association, constructing motion constraints based on photometric or reprojection errors. As for IMU processing, modeling IMU bias as the random walk is a common practice [11]. Classic VIO methods have gained widespread application, but the manually defined data association and IMU bias modeling are often inaccurate in challenging scenarios, leading to suboptimal results or even failure.

In recent years, learning-based methods in VO, IO, and VIO have been extensively researched[45, 4, 1], yielding promising results.

End-to-end learning-based VO utilizes pose and depth estimation networks to replace classic modules of tracking and mapping, trained in either a supervised [40, 43] or self-supervised [45, 18] manner. Some end-to-end learning-based VIO approaches have also been proposed [9, 28, 14]. For instance, SelfVIO [1] leverages networks to encode and adaptively fuse visual and IMU information, estimating depth and pose by self-supervised learning as VO.

Despite promising results on some datasets, end-to-end learning-based VO and VIO exhibit lower accuracy than classic approaches, particularly struggling to get accurate pose estimates under complex motions. Therefore, some propose to combine learning techniques with traditional modules [32, 3, 7]. DROID-SLAM [33] and DPVO [35] combine iterative visual correspondence updates with differentiable bundle adjustment, demonstrating excellent performance across multiple datasets. iSLAM [12] combines learning-based VO with graph optimization, further enhancing performance through self-supervised learning. From the IMU perspective, learning-based data preprocessing and bias estimation techniques are also proposed [4, 21, 7, 44]. Zhang et al. [44] learn to denoise IMU measurements and use preintegration loss derived from ground truth poses to train the network. Buchanan et al. [4] explicitly propose models for the dynamics of bias by networks and incorporate them into factor graph optimization, replacing traditional random walk models.

These methods combining learning with classic computations provide us with significant inspiration. To the best of our knowledge, our approach is the first to simultaneously integrate learning-based visual association and IMU bias modeling into a VIO framework and refine them by self-supervised learning.

Generally speaking, classic VO/VIO algorithms are manually pre-defined, and learning-based ones are usually pre-trained on dedicated datasets and then inference on scenes with similar distribution. These methods may suffer from domain shift problems if directly applied to a different scenario, leading to inferior performance.

To address this issue, Li et al. [19, 20] propose an online-learning VO framework to generalize better on unseen environments and use meta-learning techniques to facilitate fast adaptation. Vödisch et al. [36, 38] introduce online continual learning for SLAM, incorporating a replay buffer to prevent catastrophic forgetting and using asynchronous network updates to optimize system efficiency.

While these online learning methods have shown adaptability in autonomous driving scenarios such as Cityscape[10] and KITTI [13], they often fail in complex environments and under complex motions. In our method, the states from optimization provide feedback signals to the networks, enabling self-supervised online continual learning, which has proved effective in EuRoC [5] and VI-TUM datasets [27].

The tracking pipeline of our VIO system is illustrated in Fig.2, featured by the framework that integrates learning and classic optimization. The orange shapes in the diagram represent computations performed by the neural network, while the blue shapes indicate traditional manual calculations.

(A) Feature encoder receives the latest RGB image frame and encodes it into feature maps by convolutional neural networks, providing stable feature encoding for subsequent visual correspondence.

(B) Feature map sampling module randomly selects several feature points and their neighborhood from the feature map, generating feature patches. These patches are considered as keypoints, facilitating subsequent matching and depth and pose estimation. The module can be differentiable [35], which made the gradient backpropagation possible.

(C) Visual correspondence predictor predicts visual matching relationship among the keyframes in the factor graph. It takes initial matches generated by reprojection as input and outputs the updates relative to the initial matching, which can also be viewed as reprojection residuals. The pose and depth estimation are generated from IMU state propagation or visual-inertial bundle adjustment, which will be detailed in the following content.

(D) IMU bias predictor takes IMU bias estimation from the previous time step, along with the accelerometer and gyroscope measurements between previous and current image frames, as inputs to predict the IMU bias $\hat{\mathbf{b}}_{a}$, $\hat{\mathbf{b}}_{g}$ for the current time step. The IMU timestamps and bias configuration follow the settings in [24, 6]. Specifically, we synchronize the IMU preintegration interval with the timestamps of the image frames, assuming that the bias remains fixed within the interval.

(E) Differentiable IMU preintegration integrate IMU data in body poses of the previous image frame, which is independent of the initial conditions and can be treated as a single observation between two adjacent image frames. Assuming the timestamps of $i^{th}$ and ${i+1}^{th}$ IMU frames are between the timestamps of $k^{th}$ and ${k+1}^{th}$ image frames. Let $\hat{\bm{\alpha}}_{i}^{b_{k}}$, $\hat{\bm{\beta}}_{i}^{b_{k}}$, $\hat{\bm{\gamma}}_{i}^{b_{k}}$ represent the preintegration of translation, velocity and rotation until the $i^{th}$ IMU frame, where $b_{k}$ denotes the preintegration is under the body coordinate at the $k^{th}$ image frame. The body coordinate aligns with the IMU coordinate. Then, the preintegration until ${i+1}^{th}$ IMU frame can be expressed in the following form[24]:

$$\hat{\bm{\alpha}}_{i+1}^{b_{k}}=\hat{\bm{\alpha}}_{i}^{b_{k}}+\hat{\bm{\beta}}_{i}^{b_{k}}{\delta}t+\frac{1}{2}\mathbf{R}(\hat{\bm{\gamma}}_{i}^{b_{k}})\hat{\mathbf{a}}_{i}{\delta}t^{2}$$

(1)

$$\hat{\bm{\beta}}_{i+1}^{b_{k}}=\hat{\bm{\beta}}_{i}^{b_{k}}+\mathbf{R}(\hat{\bm{\gamma}}_{i}^{b_{k}})\hat{\mathbf{a}}_{i}{\delta}t$$

(2)

$$\hat{\bm{\gamma}}_{i+1}^{b_{k}}=\hat{\bm{\gamma}}_{i}^{b_{k}}\otimes\begin{bmatrix}\frac{1}{2}\hat{\bm{\omega}}_{i}{\delta}t\\ 1\end{bmatrix}$$

(3)

where $\hat{\mathbf{a}}_{i}$, $\hat{\bm{\omega}}_{i}$ are acceleration and angular velocity measurements of $i^{th}$ IMU frame, after subtracting the network-predicted bias $\hat{\mathbf{b}}_{a}$, $\hat{\mathbf{b}}_{g}$, respectively. $\otimes$ represents the product in quaternion.

The propagation from $i^{th}$ to ${i+1}^{th}$ IMU frame is differentiable with respect to $\hat{\mathbf{a}}_{i}$ and $\hat{\bm{\omega}}_{i}$. Henceforth, the preintegration from the ${k^{th}}$ to ${k+1}^{th}$ image frame is also differentiable, which allows the gradient backpropagate from preintegration to the IMU bias predictor.

The propagation of covariance and approximate bias update techniques are also adopted in our system, which are detailed in [11]. The differentiation and backpropagation of Lie Group can refer to [29, 34, 39].

(F) Visual-inertial bundle adjustment (VIBA) are standard techniques for solving state variables in classic VIO systems, which we also adopt with some distinctions.

Our factor graph comprises states within a sliding window of $n$ image frames, incorporating visual constraints, IMU constraints, and their interrelations. The full state vector in the factor graph is:

$$\mathcal{X}=\{{\mathbf{p}^{w}_{b}},\mathbf{q}^{w}_{b},{\mathbf{v}^{w}_{b}},{\mathbf{b}_{a}},{\mathbf{b}_{g}},{\mathbf{d}_{0..m}}\}_{0..n}$$

(4)

where $\mathbf{p}^{w}_{b}$, $\mathbf{q}^{w}_{b}$ and $\mathbf{v}^{w}_{b}$ denote translation, quaternion and velocity of the body. Bolded $\mathbf{b}_{a}$ and $\mathbf{b}_{g}$ are biases of the accelerator and gyroscope. ${\mathbf{d}_{0..m}}$ represents the depth of $m$ features from the frame.

The visual constraints in the factor graph are reprojection residuals, which reflect the coordinate error between the matches predicted by the network and the matches projected under current states. We construct reprojection residuals for each feature, and the network provides the confidence of each residual.

Reprojection residuals:

$$\begin{split}\mathbf{r}_{v}=\sum_{i,j\in\mathcal{G}}\|\hat{p}_{j}^{t}-KT^{t}_{s}\mathbf{d}(p_{i}^{s})K^{-1}p_{i}^{s}\|\end{split}$$

(5)

where $p_{i}^{s}$, $\hat{p}_{j}^{t}$ are a pair of matching points predicted by the network. $T^{t}_{s}$ denotes the coordinate transformation between the frames of the matching points in the image. $K$ is intrinsic to the camera. All such matching points constitute visual constraints in the factor graph $\mathcal{G}$.

IMU constraints can be categorized into two types: preintegration residuals and bias residuals. For two consecutive image frames $I_{k}$ and $I_{k+1}$, the following expressions are given:

$$\mathbf{r}_{\mathcal{I}_{k+1}^{k}}=\{\mathbf{r}_{\Delta\mathrm{p}_{k+1}^{k}},\mathbf{r}_{\Delta\mathrm{v}_{k+1}^{k}},\mathbf{r}_{\Delta\mathrm{R}_{k+1}^{k}},\mathbf{r}_{\Delta\mathrm{(b_{a})}_{k+1}^{k}},\mathbf{r}_{\Delta\mathrm{(b_{g})}_{k+1}^{k}}\}$$

(6)

Preintegration residuals:

$$\begin{split}\mathbf{r}_{\Delta\mathrm{p}_{k+1}^{k}}=&\mathrm{R}_{w}^{b_{k}}\Big{(}{\mathbf{p}}_{b_{k+1}}^{w}-{\mathbf{p}}_{b_{k}}^{w}-{\mathbf{v}}_{b_{k}}^{w}\Delta t_{k}\\ &+\frac{1}{2}g^{w}\Delta t_{k}^{2}\Big{)}-\hat{\bm{\alpha}}^{b_{k}}_{b_{k+1}}(\mathbf{b}_{a},\mathbf{b}_{g})\end{split}$$

(7)

$$\begin{split}\mathbf{r}_{\Delta\mathrm{v}_{k+1}^{k}}=&\mathrm{R}_{w}^{b_{k}}\Big{(}{\mathbf{v}}_{b_{k+1}}^{w}-{\mathbf{v}}_{b_{k}}^{w}+g^{w}\Delta t_{k}\Big{)}\\ &-\hat{\bm{\beta}}^{b_{k}}_{b_{k+1}}(\mathbf{b}_{a},\mathbf{b}_{g})\end{split}$$

(8)

$$\begin{split}\mathbf{r}_{\Delta\mathrm{R}_{k+1}^{k}}=&\mathrm{Log}\left(\mathbf{R}(\hat{\bm{\gamma}}_{b_{k+1}}^{b_{k}}(\mathbf{b}_{g}))^{\mathrm{T}}\mathrm{R}_{w}^{b_{k}}\mathrm{R}_{b_{k+1}}^{w}\right)\end{split}$$

(9)

where $\hat{\bm{\alpha}}^{b_{k}}_{b_{k+1}}$, $\hat{\bm{\beta}}^{b_{k}}_{b_{k+1}}$, $\hat{\bm{\gamma}}_{b_{k+1}}^{b_{k}}$ denotes preintegration terms and $g^{w}$ is gravity under the world coordinate.

Bias residuals:

$$\begin{split}\mathbf{r}_{\Delta\mathrm{(b_{a})}_{k+1}^{k}}=(\mathbf{b}_{a})^{k}-(\hat{\mathbf{b}}_{a})^{k}\end{split}$$

(10)

$$\begin{split}\mathbf{r}_{\Delta\mathrm{(b_{g})}_{k+1}^{k}}=(\mathbf{b}_{g})^{k}-(\hat{\mathbf{b}}_{g})^{k}\end{split}$$

(11)

where $\hat{\mathbf{b}}_{a}$, $\hat{\mathbf{b}}_{g}$ are the bias predicted by the network.

We model accelerator and gyroscope bias as a Gaussian distribution with the network-predicted bias value as the mean, and the variance is manually set. Similar methods are also elaborated in [4]. In addition to the differences in bias residuals compared to traditional random walk methods, there are other distinctions in treating bias. Firstly, we only conduct preintegration by the network-predicted bias ($\hat{\mathbf{b}}_{a}$, $\hat{\mathbf{b}}_{g}$). We never reintegrate measurements by the updated bias (${\mathbf{b}}_{a}$, ${\mathbf{b}}_{g}$) after optimization. Second, the updated bias from the current frame serves as inputs to the predictor, along with measurements from the next frame, generating a new bias prediction instead of being directly passed to the next time step.

The final optimization objective is composed of IMU residuals and visual residuals, which can be written as:

$$\begin{split}\mathcal{X}^{*}=\underset{\mathcal{X}}{\arg\min}\Big{(}\|\mathbf{r}_{\mathrm{v}}\|_{\Sigma_{p}}^{2}+\sum_{k=0}^{n}\|\mathbf{r}_{\mathcal{I}_{k+1}^{k}}\|_{\Sigma_{\mathcal{I}_{k+1}^{k}}}^{2}\Big{)}\end{split}$$

(12)

Considering the Gauss-Newton method is naturally differentiable, we use it to solve the factor graph, iterating twice for each timestamp. This facilitates gradient backpropagation to the neural network during training and online adaptation.

In our approach, factor graph optimization can provide feedback signals for visual and IMU networks, enabling self-supervised learning updates.

The feature encoder and visual correspondence predictor adopt similar network structures to DPVO [35]. The difference is that our predictor does not compose a hidden state, making the inference of the correspondences in each iteration an independent process irrelevant to the previous network state. The modifications make our pipeline more akin to the classic SLAM, enhancing system interpretability.

The poses and depths optimized by VIBA can be used to construct photometric loss through reprojection:

$$\begin{split}\mathbf{L}_{visual}=\sum_{i\in\mathcal{G}}\|I^{t}({p}_{i}^{*})-I^{s}(p_{i})\|\end{split}$$

(13)

$$\begin{split}{p}_{i}^{*}=KT^{t}_{s}\mathbf{d}(p_{i}^{s})K^{-1}p_{i}^{s}\end{split}$$

(14)

where $I$ represents the intensity of the pixel.

The bias prediction network consists of normalization layers, fully connected layers, and a GRU [8] module. The bias updated from the previous timestamp is initially normalized and then encoded as hidden states for the GRU. Simultaneously, the IMU measurements are normalized and concatenated with the normalized bias. After encoding by a fully connected layer, they serve as inputs to the GRU. The GRU’s output is then transformed into the current bias estimation via another fully connected layer.

After visual-inertial bundle adjustment, the refined poses and velocities are feedback to the network, generating loss function as:

$$\mathbf{L}_{imu}=\mathbf{L}_{\Delta\mathrm{p}}+\mathbf{L}_{\Delta\mathrm{v}}+\mathbf{L}_{\Delta\mathrm{R}}$$

(15)

$$\begin{split}\mathbf{L}_{\Delta\mathrm{p}}=&\mathrm{R}_{w}^{b_{n-1}}\Big{(}{\mathbf{p}}_{b_{n}}^{w}-{\mathbf{p}}_{b_{n-1}}^{w}-{\mathbf{v}}_{b_{n-1}}^{w}\Delta t\\ &+\frac{1}{2}g^{w}\Delta t^{2}\Big{)}-\hat{\bm{\alpha}}^{b_{n-1}}_{b_{n}}(\mathbf{\hat{b}}_{a},\mathbf{\hat{b}}_{g})\end{split}$$

(16)

$$\begin{split}\mathbf{L}_{\Delta\mathrm{v}}=&\mathrm{R}_{w}^{b_{n-1}}\Big{(}{\mathbf{v}}_{b_{n}}^{w}-{\mathbf{v}}_{b_{n-1}}^{w}+g^{w}\Delta t\Big{)}\\ &-\hat{\bm{\beta}}^{b_{n-1}}_{b_{n}}(\mathbf{\hat{b}}_{a},\mathbf{\hat{b}}_{g})\end{split}$$

(17)

$$\begin{split}\mathbf{L}_{\Delta\mathrm{R}}=&\mathrm{Log}\left(\mathbf{R}(\hat{\bm{\gamma}}_{b_{n}}^{b_{n-1}}(\mathbf{\hat{b}}_{g}))^{\mathrm{T}}\mathrm{R}_{w}^{b_{n-1}}\mathrm{R}_{b_{n}}^{w}\right)\end{split}$$

(18)

The IMU loss is almost the same with preintegration residuals Eq.(7)-(9), while the constrained entities shifted from the system’s states to the parameters of the networks. Besides, there are some other slight differences to be noticed. 1) The loss function constrains the bias predicted by the network rather than the bias updated by VIBA. 2) In the feedback loop, we manually set the weights for each loss function (all set to 1 here), while in VIBA, the weights for each residual are determined by the covariance [11]. 3) Bias residuals Eq.(10)-(11) are not included in the loss function.

The feedback-based self-supervised update mechanism enhances the consistency between the learning and classic optimization modules, collectively improving the overall performance.

Performance degradation is often encountered when a learning algorithm is transferred to an unseen environment. In such cases, adaptive fine-tuning of the network is a common practice. However, in our scenario, network fine-tuning often requires customized data preprocessing and training strategy, posing an additional workload and delaying the deployment of the VIO system.

To reconcile the contradiction between VIO deployment and network adaptation, we propose a “learning within VIO” strategy known as online continual learning. In this mechanism, the VIO system can be considered an automatic dataloader, responsible for organizing and optimizing training data and feeding it to the neural network. In addition to being able to run continuously alongside VIO, our online continual learning mechanism is also highly flexible. 1) We can start or stop it at any time without interrupting the execution of VIO. 2) We can train the visual correspondence or the IMU bias predictor independently or simultaneously.

The VIO system can be summarized as follows:

Initialization: Our VIO system starts with an initialization process, which includes map initialization and IMU initialization. After IMU initialization, we align the body coordinate and recover the metric scale of the pose and the map. The initialization process provides the system with a good initial state and builds the platform for online learning.

Tracking: The pipeline of tracking are mainly shown in Fig.2. We compute the initial pose of the incoming frame by IMU state propagation and add it to the factor graph. The factor graph maintains a sliding window containing states of the latest 10 keyframes. For efficiency considerations, during online continual learning, the visual constraints for each keyframe are only derived from several keyframes preceding and following it. Otherwise, the visual constraints of each keyframe may encompass several keyframes with optical flow magnitudes below a threshold, forming a co-visibility relationship.

Feedback: After tracking, we utilize the feedback-based self-supervised updates described in Sec.3.2 to implement online continual learning for the networks. During online adaptation, we fix the weights of the feature encoder and fine-tune the predictors for both visual and IMU. All inputs and constraints for the networks come from the factor graph, including the images, IMU measurements, and state estimates, generating loss functions as Eq.(13-18)

Keyframing: After each tracking session, keyframe culling is performed. Our keyframing strategy is similar to DPVO [35]. However, considering the temporal constraints of IMU, the gap between two keyframes should not exceed 3 frames.

Our method is implemented by Python and PyTorch [23], with specific components like VIBA utilizing C++ and CUDA programming for acceleration. The visual network requires pre-training, while the IMU bias network does not. In the following content, we will present the methods of pre-training and online continual learning separately, primarily focusing on the latter.

Datasets: The TartanAir[41] dataset is adopted to pre-train our visual model, which is a large-scale simulated dataset widely used in visual learning tasks. We choose the EuRoC [5] and TUM-VI [27] datasets for online continual learning and validation. The EuRoC is captured by a UAV visual-inertial platform, while the TUM-VI is collected by a handheld visual-inertial device. Both datasets include environments with complex lighting conditions and intricate motion patterns, making them widely used in VIO. To give the quantified results, we align the estimated trajectories with the provided ground-truth and compute the Root Mean Square Error (RMSE) of the Absolute Trajectory Error (ATE) [31].

Pre-training settings: The visual part of our network (module (A)and(C) in Fig.2) requires pre-training. Following the strategy in DPVO [35], we train our visual model on the TartanAir dataset [41] for 240000 iterations with a batch size of 1. Note that due to minor modifications in our networks compared to DPVO [35](discussed in Sec.3.2), the performance after pre-training may not be identical.

Online continual learning settings: After pre-training the visual network, we perform online continual learning on the EuRoC and TUM-VI datasets. Our PC configuration includes an Intel i9-9900 CPU and an NVIDIA GTX 3090 GPU with 24GB of VRAM.

During online continual learning, our VIO performs tracking and carries out gradient backpropagation for each incoming frame. This process constitutes one iteration of training. For tracking stability, we may not update the networks at every iteration. Completing an entire sequence is considered as one epoch, and online continual learning requires training multiple epochs, involving the continuous replay of one sequence.

For the EuRoC dataset, we perform online continual learning in MH_01. Similarly, we conduct continual learning in room1 of the TUM-VI dataset. In both settings, we replay the sequences for 60 epochs. The visual predictor’s network weights update every 100 iterations, while the IMU bias predictor’s weights update with each iteration. The learning rate for the visual predictor is set to be $1\times 10^{-5}$. For the IMU bias predictor, since it is not pre-trained, we conduct visual BA for the first 30 epochs to ensure tracking stability, with the learning rate of $1\times 10^{-4}$. After that, we perform VIBA with the learning rate of $1\times 10^{-6}$.

To evaluate the effectiveness of online continual learning for visual and IMU bias networks, we take the pre-trained visual model and IMU bias random walk as our baseline model and then compare it with visual adaptation, IMU adaptation, and joint adaptation. The results are summarized in Tab.1. To eliminate the influence of the metric scale, all results are Sim(3) aligned and averaged over five trials.

Additionally, since the visual networks of our method are mainly borrowed from DPVO [35], it’s reasonable to present their results for reference, as shown in the first row of Tab.1, where the results are Sim(3) aligned and are the median of five trials.

Compared to the baseline, online continual learning for either visual or IMU networks improves trajectory accuracy, and the joint adaptation achieves the best performance, resulting in over 10% improvements across all sequences.

To further validate the continual adaptability of our method, we perform additional statistical tests during online continual learning. We conduct 10 validation experiments for the visual model after every 10 epochs to assess its trajectory accuracy and distribution. The overall statistical results are illustrated in Fig.3(a). We can observe that the ATE distribution of trajectories continually decreases during adaptation. For the IMU model, although the continual learning of IMU bias contributes less significantly to the improvement of trajectory accuracy compared to the visual model, we statistically compare its errors with those of the random walk model in 10 experiments after every 30 epochs. As shown in Fig.3(b), the IMU bias model presents lower trajectory drift and variance than bias random walk, thus leading to more robust performance.

Two examples of each dataset are selected for trajectory comparison of our model, with and without online continual learning. As shown in Fig. 4, the trajectories of our online adaptation model are closer to the ground-truth than that of our pre-trained model on four sequences, as clearly indicated by the red boxes.

To evaluate the overall performance of our VIO system, we compare our method with state-of-the-art VIO approaches on the EuRoC [5] and TUM-VI [27] datasets. Our system constructs a more extensive set of keyframe association constraints containing temporally adjacent keyframes and spatially neighboring keyframes, as described in Sec. 3.3. We perform online continual learning on MH_01 of the EuRoC dataset and room1 of the TUM-VI dataset, then generalize to other sequences in the same dataset. The results are computed as the median of three trials. All trajectories are in metric scale and SE(3)-aligned with the ground-truth.

Results on EuRoC: As presented in Tab.2, we choose the state-of-the-art classic VIO methods and learning-based approaches as comparison. Compared with classic methods, our method outperforms most approaches in terms of RMSE ATE and is comparative to the performance of the DM-VIO [30]. Furthermore, our method exhibits significantly superior performance compared to other learning-based VIO methods. We also observe the performance of our method exceeds the VIO system Kimera [26] and iSLAM [12], both with stereo-inertial settings.

Results on TUM-VI: To evaluate the generalization ability of our method, we also conduct experiments on another TUM-VI dataset [27], which is a highly challenging handheld dataset with large-scale scenes, compared with the EuRoC dataset [5].

We compare our method to the classic state-of-the-art VIO methods as presented in Lukas et al. [30]. The results are reported in Tab.3. Our method can achieve better accuracy on 7 sequences than alternative methods, even compared to the DM-VIO method, which shows the best result among all methods. However, in other sequences, DM-VIO reported better results than ours, which can be attributed mainly to its more robust initialization and long-term scale refinement.