Adaptive Attitude Estimation Using a Hybrid Model-Learning Approach

Micro-electro-mechanical systems (MEMS) inertial sensors have revolutionized the consumer market. This is due to the availability of advanced technology enabling accurate sensing and with respect to size, weight, and power usage, it is light and small. Inertial sensors are commonly integrated in many modern products such as smartphones, autonomous drones, virtual reality headsets, and low cost robotic toys.
An inertial measurement unit (IMU) contains three orthogonal gyroscopes, accelerometers, and, frequently, magnetometers. IMU is used to measure angular velocity, specific force, and magnetic field, mainly for navigation purposes. A common navigation task is orientation measurement, often referred to as an attitude and heading reference system (AHRS) [1]. A popular definition of attitude in the literature is the description of the relative angular displacement between two reference frames [2]. Attitude determination is a basic requirement in many disciplines and platforms such as motion tracking [3, 4, 5], pedestrian dead reckoning [6, 7, 8, 9], and aerial vehicles tracking [10, 11, 12, 13]. Attitude can also be estimated based on images from an on-board camera [14, 15]. Although very effective, image processing relies on the quality of the image, which is in many cases not sufficient or even worst, unavailable (if the camera is placed in a bag). Also image processing requires heavy calculation resources and is available as frequently as the cameras capture rate. IMU based AHRS is an accurate, reliable, and a computationally efficient task that can be carried out at a fast sensor rate.
The main challenge in AHRS is to deal with external accelerations and magnetic disturbances, such as determining smartphone attitude while a pedestrian is accelerating. The common smartphone integrates low-cost MEMS IMU readings, which are characterized by low resolution signals incorporating high amplitude noise and time varying bias. To achieve an accurate attitude solution, the nine dimensional IMU signals must be fused properly. The fusion algorithm must consider each sensor’s signal composition for it to be weighted correctly in the attitude solution. Most algorithms assume a constant signal composition (equal for all axes), but, as discussed later, the amplitude of disturbances incorporated in the sensor signal varies based on time, dynamics, and location. A few classical studies addressed this issue in the literature by using assumptions and heuristics as a basis for a weighting policy. Although these methods have achieved some success, a more appealing strategy is to use data-driven approaches. In the last decade, ground breaking results have been obtained with learning approaches in fields such as image processing [16], speech recognition[17], and even navigation [18]. Adaptation of learning approaches for attitude estimation is a new trend, making this field of study intriguing for many researchers.
This paper rests on the foundations of our prior studies [19, 20], addressing the integration of a model-based and an optimization-based data driven method for the purpose of attitude adaptive determination using low-cost inertial sensors. Based on this approach, a hybrid model-learning framework for attitude estimation is proposed. The learning algorithm aims to cope with the varying amplitudes of linear accelerations applied by the walking user. The output of a neural network (NN) is plugged into model equations to form an adaptive complementary filter structure. That is, instead of using constant or model-based adaptive weights, the accelerometer weights in each axis are determined by a unique NN. Instead of end-to-end deep learning solutions using a ”black box” strategy (input inertial data, output attitude), the proposed method relies on a model and uses learning to determine only the required weights within the model equations. The motivation of such s strategy is to make use of the established knowledge of the deterministic nature of this problem on the one hand, with stochastic behavior learned from data on the other.
To demonstrate the improvement in performance, the new approach is compared to [19] and also to popular AHRS such as [3, 10, 2]. Our analysis uses experimental data collected in [21] for four different smartphone locations: 1) Pocket, 2) Texting, 3) Body, and 4) Bag.
The rest of the paper is organized as follows: Section II gives a thorough review of the various available in the literature, which form the basis for this research. Section III describes the mathematical background needed for the derivation of the proposed approach. Section IV presents the proposed framework; namely, the structure of the adaptive filter, the proposed data-driven structure, and the training process of the model. A comparison of the proposed approach to other AHRS approaches is in Section V, Section VI gives the conclusions.

Attitude estimation has been studied widely in the literature for many years. The Triad method proposed by [22] is one of the earliest solutions to attitude determination. Two-directional measurements, namely magnetic field and a 3D solar cell, are considered to compute the attitude rotation matrix of a satellite. Wahba [23] formulated this problem to calculate the optimal orthogonal matrix given two or more directional measurements by optimizing a least squares cost function. The QUaternion ESTimator (QUEST) [24] method calculates an optimal attitude solution in a quaternion form. In [25], the authors proposed a complementary filter with an Euler angles representation of the attitude. Gyroscope and accelerometer data are processed to estimate the pitch and roll angles. A frequency domain analysis is proposed as a method for the tuning procedure. A thorough historical review of the early stages of attitude determination is given in [26].
Due to the nonlinear relations between the vector observations and attitude representation, the extended Kalman filter (EKF) is probably the most accepted approach in the literature. In this framework, the attitude kinematic equations, numerically propagated by gyro measurements, are treated as the system model. The accelerometer and magnetometer vector observations are processed with a linearization of the observation model to update the estimation and propagate the covariance matrix. An optimal solution is calculated by minimizing the error covariance based on assumptions on the distribution and magnitude of the process and measurement noise covariance. The EKF can be divided into two categories: a total state and an error state implementations. In the total state approach, employed in [9, 27, 28], a vector quaternion representation of the attitude is estimated. In the error state approach, a state vector consisting of misalignment angles is promoted to update attitude represented in the orthogonal matrix form. Examples of error state EKF can be found in [2] and [29]. The generalization of this problem as a Kalman Filtering (KF) estimation scenario enables the use of state augmentation. The gyro bias and linear accelerations are modeled and added to the estimation problem to improve accuracy. Furthermore, instead of assuming a zero mean and a normal distribution, in the unscented Kalman filter (UKF), deterministic sampling is used to approximate the mean and covariance of the state distribution. This framework was used for attitude determination in [6, 12].
The work described in [30] creatively formulates AHRS as a linear system. This can be done by constructing of the system process matrix using the gyro measurement; this step is common when using a quaternion form. Further, the observation matrix is constructed using the accelerometer and magnetometer readings. Another derivation of a KF approach to AHRS is proposed in [31]. A unique selection of the state vector leads to linear process and measurement models. The simplified approach in [13] and[32] converts the gravity measurement to an observation of the attitude quaternion.
Although KF based solutions are well established mathematically, they are known to be sensitive to the tuning of process and measurement covariance matrices. Another drawback is the computation complexity involved in calculating the filter gain. Complementary filters (CF) have the advantage of an easier and intuitive tuning process and demand less computing resources. Examples of CF are given in [25, 10, 5, 3]. In [25], body rates are converted to predictions of the pitch and roll angles derivatives. The gravity measurement and prediction are then subtracted to produce a gravity residual. The residual is filtered and used to produce updated estimations of pitch and roll derivatives. In [10, 5], the accelerometers and magnetometers are processed to produce an estimation of the attitude quaternion error. The attitude error quaternion is used to update a predicted estimation of the quaternion derivative. A very popular algorithm is the Madgwick [3] filter. The Jacobeans of gravity and magnetic measurement models are used for a gradient descent step. The step length is the tunable parameter and constitutes the filter gain.
Performance of all solutions mentioned above depends on parameter tuning that reflects the characteristics of the scenario (such as each sensor’s noise and amplitude of disturbances). It is well known that the tuning procedure of a filter to a specific environment is not a trivial task and that the achieved performance degrades if the environment is not maintained. Moreover, in a dynamic environment, static parameters does not lead to optimal performance. This brings a motivation for an algorithm that tunes the fusion weights online to reflect variations in the environment. A few studies in the literature address the AHRS problem in an adaptive Kalman filter (AKF) approach. Such an algorithm is proposed in [30]. Noise covariance of measurement and process are adjusted to achieve consistency with the measured residual. This is done by formulating an optimization problem to which an analytic solution is available. In [11], the measured residual is used to adjust a recursive least square model with exponential age weighting. The resulting model is used to tune the measurement noise covariance matrix. In [29], a minimum volume covering ellipsoid algorithm is used to adjust the measurement covariance, while in [33], the process noise covariance is adjusted based on the measured segment of the residual samples. An interesting algorithm is presented in [34], where the measurement noise is adjusted proportionally to the measured acceleration norm. Additionally, an adjustable scale factor is used to control the responsiveness of the adaptation process.
It is important to stress that the adaptation of noise and measurement covariance of a KF relies on the validity of the filter assumptions. In the derivation of an AHRS problem as in a KF framework, linear accelerations and magnetic disturbances are treated as Gaussian distributed uncorrelated signals. Yet, neither of the mentioned signals meet these definitions. The violation of these underlining assumptions lead to a non-optimal performance of the AKF compared to the non-adaptive filters. Nevertheless, the value in robustness and self-tunable adaptive algorithms is significant in some applications. This was shown in [19] where [11, 33] were compared to a few non-adaptive algorithms optimized for optimal performance on the tested data.
Several other papers propose integration of heuristics to deal with variations in the environment. Examples can be found in [7, 6, 8, 9]. The vector observation norms are compared to known values of gravity and magnetic fields. This operation is performed to detect the presence of disturbances in the sensor measurements, which would then lead to rejection of the disturbed measurements. Disturbance detection can be performed for each sample [7, 6] or based on a buffer of measurements like in [8, 9].
The task of choosing the best among the multitude of model based AHRS algorithms is not an easy one. Informative performance surveys are given in [8, 7, 19]. AHRS determination of a smartphone is characterized by low-cost sensors and a dynamic/magnetic environment of variate nature. The inherent deficiencies of the classic approaches and the availability of recorded data constitute a fertile substrate to a new era of data driven solutions. Recently a few studies featured neural networks (NN) for the purpose of attitude estimation. When approaching the task of developing a data driven attitude estimator, there is no agreed convention for structure, data handling, training process, network type, input, etc. In [35, 36, 37] a deep neural network (DNN) is used to enhance the performance of the state estimator. In [38] a convolutional neural network (CNN) is used to denoise the gyroscope measurements of an IMU by estimating calibration parameters and bias. This process substantially improved the accuracy of attitude estimation calculated by gyro integration. In [39] a long short term memory (LSTM) based architecture called OriNet was proposed to propagate the attitude estimation using the gyro measurements. The proposed structure consists of a genetic algorithm for gyro bias estimation. This network could also be thought of as a calibration process to use with a single IMU on which it was trained. A very distinguishable approach is presented in [40, 41], employing a pure end-to-end strategy in which the NN is fed by the raw sensor data and delivers prediction of the attitude quaternion, was employed. State-of-the-art DL tools are used to achieve the optimal choice of network configuration and hyper parameter tuning. Several publicly available datasets are used for training, achieving a robust solution.
In [42, 43, 21, 16] DNNs were used for pedestrian dead reckoning (PDR). Although not the topic of this work, the use of DNNs to process IMU data for navigation purposes is relevant. In [42] a bi-directional LSTM was used on segments of IMU samples to predict the change in heading and location. In [43] a ResNet [44] structure was used to regress a velocity vector in the horizontal plane. In [21] same task was carried out by support vector regression[45] and a support vector machine.
In [19, 20], a model-data-driven approach to AHRS was proposed by the authors. The main concept was to use the well established mathematical relations of attitude and vector observations but to replace tuning, heuristics models, and assumptions with data driven decisions. The data based empirical functions, designed to predict the optimal fusion weights, are nested in a complementary filter based structure. In [19], accelerometer weights were adjusted based on an estimation of the linear acceleration. Accelerometers and gyroscopes were used to estimate the attitude (roll and pitch angles). Estimated linear accelerations were mapped to fusion weights by the use of an empirical function. This function was obtained by an optimization process using ground truth (GT) dataset. In the following study [20], the previous fundamental concept was employed and elaborated to include magnetometer readings achieving accurate heading estimation in the presence of magnetic disturbances. Additionally the heading update is decoupled from the attitude update. As a result, the gravity estimation accuracy is preserved even if the heading estimation is erroneous due to a magnetic disturbed environment. This method of applying a data driven function combined with classical equations demonstrated best performance compared to a selection of common approaches from the literature.

The attitude (roll and pitch angles) of a smartphone relative to the ground is measured based on gyroscopes and accelerometers mounted on the smartphone. Rotation matrices and Euler angles (roll, pitch, and yaw) are the attitude representations used in this work.
Two reference frames are employed:

• •

  Body frame (b-frame): fixed to the IMU axes such that the x-axis points to the right side of the smartphone screen, y-axis points to the top of the screen, and z-axis points towards the outer side of the screen, completing the right-handed coordinate system, as pretested in Fig. 1.

• •

  Reference frame (r-frame): fixed to the ground so that the x-axis points north, z-axis points down, and y-axis completes the right-handed coordinate system pointing east.

The foundations for the proposed data-driven approach are based on our previous work [19, 20]. There, the update weights of the accelerometers and magnetometers were determined by an empiric function obtained by an optimization process. In this paper, this empiric function is replaced by a NN model aiming to predict the optimal fusion weights based on the momentary residual. In this manner, instead of an end-to-end deep learning approach, the proposed method employs a solid theoretical AHRS algorithm and incorporates a NN to predict the weights in a hybrid fashion. Although a single block in the pipeline is replaced, major advantages are achieved. The NN is a differentiable function and thus enables analytical calculations of the gradient in the back propagation stage. The analytical calculation replaces a forward difference gradient approximation used in our former proposed approaches. Moreover, the training process depicted in Section IV-E adopts batch optimization and therefore only a portion of the training set is processed in each training iteration. Another benefit is that the NN generalizes from a batch of shuffled short segments to full length experiments and thus is more robust. The framework, which is comprised of the structure paired with the proposed training process is not only superior in performance but is also much faster to optimize.
The proposed model focuses on gravity estimation (like in [19]) without the use of magnetometers. Nevertheless the same solution can also be used for heading estimation (like in [20]).
The proposed model consists of two supplementary components: a complementary model-based filter structure and a NN for gain determination that requires a training procedure. We highlight two properties that enable accurate attitude estimation in a dynamic environment:

1. 1.

   Deep Learning (DL) Based Adaptation Policy –The linear accelerations are estimated and inserted to a NN. The NN is trained, as described in Section IV-E, to determine the update weights of the filter for optimal performance on the train set.

2. 2.

   Component Based Update –The gravity is updated in each axis separately based on the estimated acceleration in the appropriate direction (Section IV-B).

A hybrid model-learning approach for adaptive attitude estimation was proposed. The initial data driven framework (APGE, AAE), introduced by the authors in [19, 20] was elaborated into a hybrid model-learning framework. Instead of using an empiric function obtained by an optimization process, the method integrates a NN nested in a complementary filter based structure. The training process enables stochastic batch optimization and, therefore, performs significantly quicker than the optimization process needed in the former configuration.
The NN is designed to predict the optimal fusion weights. Thus, it is carefully optimized to meet the requirements of predicting the accelerometer multiplicative weights. Using the soft threshold custom activation function. Additionally, an input augmentation layer designed to encourage nonlinear behavior, is integrated in the network structure.
The performance of our approach has been tested on experimental data compared to a selection of commonly used estimators including Madgwick, Mahony, ES-EKF, and APGE. The DAE approach performed better than Madgwick, Mahony, and ES-EKF by 37%. The improvement of the AGPE (both DAE and APGE share a similar model-based structure) averaged 10%.