A Data-Efficient Model-Based Learning Framework for the
Closed-Loop Control of Continuum Robots

Along the centre-line of the rod, we have defined the position $p(s)$, orientation $R(s)$, the linear velocity $v(s)$, angular velocity $u(s)$, internal force $n(s)$ and internal moment $m(s)$ at the length s of the rod. And $q$ is linear velocity and $\omega$ is angular velocity for the rod’s time $t$ at length $s$. And $p_{s}$ represents taking derivative of position respect to length of the rod s and so on. For the Cosserat rod model, most internal forces come from tendon tensions, which are compression forces. Under the condition of little external load and lightweight of the continuum robot, we have considered the internal forces compress the spine.

With this assumption, multiple trials of experiments have been conducted to determine the relationship between the length change of our compressible soft spine with compression force. A specimen of two sections of the compressible spine is used for testing. The change of total length of the specimen is recorded with respect to total forces applied on top of the specimen. Finally, we fit all the experiment data points using a regression method and simplify it into a linear equation, a decrease of compressible spine length with respect to force is represented as :

where $c_{spine}$ is the scalar coefficient obtained from experiments with a value of $0.2\frac{mm}{N}$ for each section of the compressible spine, and $\left\|n(s)\right\|$ is the magnitude of vector $n(s)$ at length $s$ of the rod. The length stops decreasing after compression force exceeding 20 Newtons.

Our continuum robot has 11 sections of compressible segments as shown in Figure 1(b). Meanwhile, internal forces $n(s)$ are different at these 11 sections due to weight and external load effects are calculated using linear constitutive law:

where $K_{se}$ is stiffness matrix for shear and extension and $v^{*}$ is the straight rod reference configuration with a value of [0;0;1]. Therefore, instead of directly reducing the total length of the continuum robots, we have iteratively reduced the length on each segment of the continuum robots with respect to internal force at the length s of the rod inside the shooting method of the Cosserat rod model. The detailed algorithm is presented in Algorithm 1.

We have used four 3D-printed Thermoplastic Polyurethane (TPU) spines in the continuum robot. Meanwhile, the Cosserat mathematical model is based on a single rod. We treat these four separate spines as one centre TPU spine with the same cross-section area to simplify our model. Meanwhile, the spine has triangular shapes along its length, resulting changes in its mechanical propriety. Therefore, we need to re-calibrate Young’s modulus ($E$). There is an existing relationship between Young’s modulus and shear modulus ($G$) for isotropic and homogeneous materials, which is $G=E/(2\times(1+v)$, where $v$ is Poisson’s ratio. For the convenience of calibration, we apply this existing relationship and calibrate Young’s modulus of the spine.

The Poisson’s ratio value is set to a fixed value of 0.3897 obtained from [27]. We have performed experiments by pulling tendons using different weights and recording tip position values using motion-tracking cameras. To calibrate, we solve an optimisation problem of minimising the error terms between experiment data and simulated mathematical data by finding suitable Young’s Modulus of the compressible spine. The final calibrated Young’s Modulus value is $2.33e^{9}$.

The dynamic modelling of soft continuum robots was presented previously in [19]. In their approach, a fully dynamic model with $n$ degree-of-freedom in configuration-space variable q is represented as:

where $M(q)$ is the inertial term, $C(q,\dot{q})$ is the damping term and $G(q)$ is the gravitational and stiffness effects. $\tau$ is generalized force acting on the system. This representation is later reduced to a first-order dynamic neural network in [20]. In the first-order dynamic representation, they considered the inertial term mainly takes effects during initial transient motion, and most soft continuum robots are lightweight. The first-order dynamic model is then represented as:

For the first-order recurrent neural network, $\dot{q}$ is provided and feed into the input. Regarding under-actuated systems actuated by $m$ control inputs $u$, a $B^{T}$ describing the $n\times{m}$ matrix of control input $u$ influences on $\tau$ is included in Equation (4) [28]. However, during experiments, changing positions are more easily recorded. In their approach, fixed time-step positions are recorded and fed into the neural network to represent discretized dynamics. The mapping in the RNN is now represented as:

where $u_{i}$ is the control input in the under-actuated system and $x_{i}$ is the position at time-step $i$.

We followed the approach of first-order dynamic recurrent neural network modelling. From the tendon-length control Cosserat dynamic model, we generated 10000 data points using the continuous random exploration of the actuators with a maximum travel distance of 0.003 m each step. The tendon length control is slower than pneumatic pressure control. By imposing the maximum travel distance so it can capture the dynamic effects under the assumption of 10 Hz control frequency. In addition, to not restricting the mathematical model from exploring the workspace, we have two tendons actuating each time. We are only minimizing the tendon length errors of two actuating tendons leaving the remaining two tendon lengths calculated using Euler’s integration method along their tendon paths.

The neural network consists of a single-layer recurrent neural network of 30 neurons with a linear output layer. We divided 10000 data points with a ratio of 70:30 for the training and validation dataset. To prevent our model from over-fitting, we have adapted the early stopping method. With the training of 780 epochs, the validation mean absolute error is around 0.0024 meters.

For modelling the error between our recurrent neural network and the real continuum robot with data efficiency, we employed a non-parametric method GPR. Now we consider an input data $X={x_{i}}$ and output data $Y={y_{i}}$ for i = 1,2,…,N. GPR could approximate any nonlinear function $y_{i}\sim f(x_{i})+\epsilon$, where $\epsilon$ is a white Gaussian noise with zero mean and variance $\sigma^{2}_{n}$ [29]. And $f(x_{i})\sim GP(m(x_{i}),k(x_{i},x^{\prime}))$. The mean function and $k(x_{i},x^{\prime})$ defined the GPR. For our application, we have chosen the zero-mean function and square exponential (SE) kernel. The SE kernel is represented as:

where $\sigma^{2}_{s}$ is the signal variance, $\lambda:=diag([l^{2}_{1},...,l^{2}_{D}])$ and depends on the characteristic length-scale $l_{i}$.

For making a prediction given input set $x_{*}$, the mean output value and variance could be represented as:

We used control input and simulated position as input to GPR, and the difference between our dynamic model and real experiment data as output, which can be described in the following equation:

Where $p_{sim}$ could either generated from our learned dynamic neural network or directly from simulated mathematical model.

Different from tendon force control, all the tendon length control inputs have to be previously calculated in the tendon-length control dynamic Cosserat rod model to ensure fitting the shape of the continuum robot. During this process, we also obtained the simulated position data at each point.

Furthermore, manually correcting the input to RNN at each discontinuity is cumbersome, making the usage of simulated position data obtained from the mathematical model a preferred choice. Since our learned dynamic model is trained from a mathematical model, this approach is equivalent to modelling error between our trained dynamic model and real robot. The output of GPR will be the value of the predicted mean calculated in Equation (7) when given a test input.

We have collected 1000 real points across workspace in GPR training. When a small number of points are needed, we sparsely select random points with larger intervals between them while maintaining the overall shape of the workspace. To prove the robustness of the GPR model, for the case of 100 real points, we first divide 1000 by 100 to get ten points for each cluster of points, then we randomly select 1 point from 10 points in each cluster. The cross-validation error for a GPR model fitting 1000 real data points is around 0.002 meters.

We have trained a similar inverse mapping of continuum robot using simulation data presented in [20]. The training process is similar to forward dynamic model training with a single RNN layer with 30 neurons and a linear output layer using 10000 simulated data :

where $x_{i}$ and $x_{i+1}$ are the current position and next target position, $u_{i}$ is the control input. After training of 698 epochs, the validation mean absolute error is around 0.0024 meters. All the data points are taken from the same dataset with modification of the data order to train this dynamic inverse mapping model. To predict given a target position $x_{target}$, we represent it as:

The inverse mapping of the continuum robot is trained with simulation data, while there is still a sim-to-real gap to fill. For the controller, we have added an error term $e_{gp}$ obtained from GPR to correct the behaviour of our trained inverse mapping:

Given a real target point $x_{real\>target}$ in the real workspace, now we need to estimate the target point $x_{s-target}$ in the simulated workspace. Term $e_{gp}$ is also dependent on the input value of $x_{s-target}$. It then becomes an optimisation problem of finding suitable $x_{s-target}$ value. We take an initial guess of $x_{s-target}=x_{real\>target}$. feeding it into the RNN and receive tendon length control outputs. Then feeding the control outputs and $x_{s-target}$ into the GPR model to generate $e_{gp}$. This error term will be added to $x_{s-target}$ as a correction of initial guess.

For open-loop control, the initial position does not change during the control time step. However, for closed-loop control, the initial position is constantly updated according to the time step. Meanwhile, we expect the current position in the simulated workspace instead of the real workspace as an input for our inverse mapping model. To simplify the optimisation step, we used the error predicted $e_{gp}$ from the GPR model at the simulated target point and treating it as an error term for each current position, adding that to the real position feedback input gives us a simulated current position. Thus, as the robot gets closer to the simulated target point, the error term becomes more accurate. The detailed algorithm is presented in Algorithm 2.

Given the GPR model trained from real experiment data, we can generate $e_{gp}$ using Equation (8) for each simulated input. Assuming each $e_{gp}$ is accurate, we will then obtain accurate real workspace positions by using the following equation to generate approximated real data:

With approximated real workspace points, we can retrain the RNN in control policy A using Equation (9). One benefit of using this method is to allow the RNN to learn the real dynamics of the continuum robot, similar to training the RNN with real experimental data. After training of 76 epochs, the validation means absolute error is around 0.0019 meters fitting 10000 generated data points.

This control policy takes advantage of both control policy A and B, and combines them into a hybrid model. Control policy A relies on an accurate simulation model, and the GPR model compensates for the error. Most of the time, the simulation model is a good approximation of the real robot, and it works well when the simulation model is accurate.

Due to simplifications and many assumptions in the simulation model, some physics could not be captured well. This problem becomes more critical for our application, where soft materials are used. We use control policy B to learn the physics missing or heavily distorted in the real robot from the same GPR model generated data. While the approximated data generated from the GPR model could be less accurate than real data, it will outperform control policy A when the simulation model is not accurate.

We have presented the control policy C to connect policy A and B using the final position tracking error as a criteria and checking $L_{A}$ and $L_{B}$ to avoid an infinite loop. If the absolute position error is greater than a constant value $e_{c}$, we will switch to the control output generated from policy B assuming the simulation model is not accurate at this location as presented in Figure 2. And $e_{c}$ is chosen based on the average goal-reaching error of policy A : 0.015m. Currently, policy C relies on feedback to check $e_{c}$, works in the closed-loop setup.

In the tendon-length control Cosserat dynamic model, we have generated 10000 data points. Using the same control inputs, we perform experiments with all these 10000 points on the real robot. We then present the difference between the simulated workspace and the real workspace in Figure 3.

After calibration, the error in z-axis is not apparent. However, due to the gravitational effect in the negative x direction and our continuum robot’s unique triangular shape spine design, the error in the direction of the x-axis is significant where gravitational force is in the direction of negative x-axis.

To test the capability of different controllers, we have experimented with our controllers by randomly picking 50 target points from another 960 unique points spaced equally in the real workspace and recording their position error. All the points selected for testing are not used for the training or validation process of the neural network or the GPR model.

While we have collected 1000 pre-defined real data points for GPR training, we sparsely random-picking a certain number of them by maintaining the relative equal distance between each point. We have performed experiments with 50, 100, 200, 400, 600, 800, 1000 of real data to train GPR models and test our controller’s performance to find out the impact of real training data on our GPR models and controllers. Meanwhile, controllers that trained from only real data with 500, 1000, 2000, 3000, 5000, 7500 and 10000 real data points have been tested in the open/closed-loop case. The results are shown in Figure 4.

For the controller trained directly with real data, more data points did increase the performance and decrease standard deviation error.

As for our model-based controller with GPR, fitting more real data in GPR does not necessarily improve the performance of our controller. The main reason could be the more uncertainties and errors added to the GPR model when fitting more data points. On the other hand, performing an inversion of matrix in GPR with a larger dataset costs $O(N^{3})$. We find that a train data size around 100 is a good train data selection criteria for modelling the errors between dynamic model and real robot while maintain low standard deviation error.

The previous section found that using 100 real points trained GPR model is data-efficient while having good performance. This section presents control policy A, B, and C performance using the GPR model trained from only 100 real data points. Here, We compare our three control policies in Figure 2 with controller $\pi_{real}$ directly trained from 10000 real data using Equation (9). The performance with standard deviation is presented in Table I. Control policy B has been retrained using the same GPR model with 10000 generated data points.

We can find that the closed-loop control improves the accuracy for control policy A, B, and controller $\pi_{real}$ compared to open-loop control. Both control policy A and B achieve comparable performances compared to controller $\pi_{real}$. Control policy A is slighter better than control policy B. Control policy C has outperformed controller $\pi_{real}$ with average goal-reaching an error of 1.1cm and a lower standard deviation error of 0.46cm. The uncertainties due to the soft spine deformation and long recovery time from stress aside from dynamic effects cause controller $\pi_{real}$ to be less accurate. With model-based learning with GPR framework, we reduced the uncertainties in data and improved the performance of our controller.

In addition, we have tested the same controllers performance when a tip-load of 20 grams is added to the soft continuum robot. In this case, the learned dynamic models from simulation and real data mismatched with real robot dynamics due to significant deformation of the continuum robot. Our proposed control policy A, B and C still outperformed the controller $\pi_{real}$.

We have connected 33 real data points in the workspace with linear interpolation to generate a target path as shown in Figure 5 (a) to perform closed-loop path tracking experiments. The resulted path tracking plots showed both control policy A and B are stable and closely match the target path with a completion time of around 135, 150 seconds, respectively. Furthermore, the overall path tracking error is similar for control policy A, B, C and $\pi_{real}$. The proposed control policy C is a hybrid model of policy A and B. It will switch between policy A or B based on a preset criteria, this approach results in a small oscillation of 3-5mm around the target path and longer completion time of 196 seconds. However, control policy C has much smaller starting point position error than other control policies, proving its advantage in goal reaching tests.