Soft Multicopter Control Using
Neural Dynamics Identification

Making a drone’s body soft opens up brand new horizons to advance its maneuverability, safety, and functionalities. The intrinsic property of soft materials to deform and absorb energy during collision allows safe human-machine interactions [1, 2, 3]. In circumscribed environments, soft drones can naturally deform their bodies to travel through gaps and holes, making them effective for emergency rescues. Moreover, the ability to perform controlled deformation enables soft drones to perform secondary functionalities apart from aerial locomotion, such as flapping wings, grasping objects, and even operating machines, any additional mechanical parts.

Despite the various advantages that this promises, to date a reliable and practical algorithm that controls soft drones to fly and deform has been lacking, due to multifaceted challenges. Unlike how it is for rigid drones, which are fully defined by 12-dimensional state vectors, describing the state of soft drones is far from trivial. Since a continuum body deforms in infinite DOFs, one needs to design discrete representations both compact —— so that the underactuated control problem is feasible, and measurable —— so that the controller can adjust to unmodelled errors. Even if such discretizations are obtained, the dynamic interplay of these state variables cannot be derived analytically in closed-form, as the dynamics in the full space is governed by complex PDEs. Finally, if one is to adopt machine-learning methods to model the dynamics, a problem is posed by the complexity of soft body simulation, which inevitably leads to scarce data, making brute-force learning an ill-fated avenue.

Bridging the deformation mechanics, deep learning, and optimal control, our method is so designed to successfully overcome the abovementioned challenges. First, we adopt the polar decomposition theorem to build a low-dimensional decomposed state space consisting of three geometric, latent variables each representing rotation, translation, and deformation, which can be synthesized from the readings of a set of onboard Inertial Measurement Units (IMUs). We define the interdependencies of these three variables and learn them with lightweight neural networks, thereby learning a neural simulator in a latent space on which the controller will be based. With automatic differentiation, we then extract the numeric gradients of the learned system to be controlled with a Linear Quadratic Regulator (LQR). Due to the fact that LQR requires the system to be linearized around fixed points that are inaccessible, we extend it with a novel online-relinearization scheme that iteratively converges to the desired target, bringing robustness and convenience for human piloting.

As shown in Figure. 1, our system takes soft drone geometries as input, and returns functionals that compute control matrices based on the drone’s current state. To the best of our knowledge, the proposed approach is the first to control soft drones that are meant to deform significantly in flight; we show that we can not only regulate such deformation for balanced locomotion, but also capitalize on the deforming ability to perform various feats in the air.

Multicopter     Over the last few years, quadcopters have virtually dominated the commercial UAV industry, thanks to their simple mechanical structures, optimized efficiency for hovering, and easy-to-control dynamics that has been extensively studied by [4, 5] and many more. Various methods have been successfully developed to control quadcopters, including PD/PID [6], LQR [7, 8], differential flatness [9], sliding mode [10], and MPC [11] methods. Recent research explores the control of non-conventional multicopter designs, including ones with extra rotors [12], asymmetric structures [7], articulated linkage [13], or with gliding wings [14]. The problem of controlling drones fabricated with soft materials is still understudied.

Aerial Deformation     Recent works have explored the potential of drones to actively deform in flight [15]. [13, 16] achieve impressive results with their multi-linked drones in passing through small openings or grasping objects, but the added mechanical components in their designs imply additional cost, fabrication complexity, energy consumption, and maladroitness. [17] proposes a lightweight, planar folding mechanism actuated by servo motors that is effective in controlling the drones to travel through confined spaces, but the simplified mechanism limits the ability to perform extra functionalities such as grasping. [18] propose the incorporation of a cable-actuated soft gripper with a rigid quadcopter to achieve load manipulation. While previous works add additional actuators to control deformation, we control deformation jointly with locomotion using rotors only.

Data-driven Soft Robot Control     The modeling and control of soft robots is a challenging problem due to the high DOFs and the non-linear dynamics [1], which together make closed-form solutions unfeasible to be derived [19], and therefore making data-driven approaches favorable. [20, 21, 22], and many more, use deep reinforcement learning to train neural network controllers for soft robots. [23, 24] marries machine learning with control theory, applying MPC or PD control methods on models learned from data. [19] propose the possibility of end-to-end supervised learning with a differentiable soft-body simulator, although their current design does not facilitate feedback mechanism in real-life due to the assumption of full state measurement.

actuation $\mathbf{u}$ will be passed into the three trained network modules as inputs. Then we extract the numeric gradients to form the Jacobian matrices, which will be used to assemble the matrices $\mathbf{A}$ and $\mathbf{B}$ representing the linearized dynamics. Then, $\mathbf{A}$ and $\mathbf{B}$ will be passed into the LQR algorithm to form the control matrix $\mathbf{K}$, given which we simply need to pass in our current state $\mathbf{x}$, our next waypoint state $\mathbf{x}_{wp}$, and the current actuation $\mathbf{u}$, to obtain the new actuation $\mathbf{u}_{next}$, which will be applied to the drone’s rotors to complete the control loop. The $\mathbf{x}_{wp}$ will be computed by a Proportional-Derivative (PD) controller given final goal state $\mathbf{x}_{goal}$. The control loop will operate at $100Hz$ while the recomputation of $\mathbf{K}$ takes place at $10Hz$.

Geometric State Decomposition     Let $\hat{\Omega}$, $\Omega$ denote the volume of the drone’s undeformed and deformed geometry, and let $\hat{\mathbf{z}}$, $\mathbf{z}$ be their respective finite discretizations as particles forming the solid body, with $\mathbf{z}=\Phi(\hat{\mathbf{z}})$. We can always write:

for some rotation matrix $\mathbf{R}$, some position $\mathbf{p}\in\mathbf{R}^{3}$, and some non-linear function $S$ which reduces to $\Phi$ when $\mathbf{R=I}$ and $\mathbf{p=0}$. In other words, we first define a local reference frame by the $SO3$ transformation resulting from $\mathbf{R}$ and $\mathbf{p}$, and express the deformation in such a frame with $S$, thereby decomposing the total transformation into a rigid component and a deformable component. In this way, the rigid, $SO3$ component can be simulated side by side with the deformable component $S$, an approach proposed by [25] and adopted by various works in the deformation simulation community [26, 27, 28], taking advantage of the fact that the local-frame deformation $S$ is much nicer to work with than the world-frame deformation $\Phi$. In this work, we adopt the same philosophy and learn evolution rules for $\mathbf{R}$, $\mathbf{p}$, and $S$ in juxtaposition.

Measurability      Since our goal is to build feedback controllers for $\mathbf{z}$, i.e, $(\mathbf{R},\mathbf{p},S)$, we would need to define $\mathbf{R}$, $\mathbf{p}$, and $S$ in such a way that they can be measured by sensors. In this work, we adopt the following strategy. As shown in Figure. 3 we plant Inertial Measuring Units (IMUs), which are compact, low-cost MEMS outputting rotational and translational status, at the drone’s geometric center and around its periphery, next to the propellers. For the local frame rotation $\mathbf{R}$, we use the Euler angles $\mathbf{e}=(e1,e2,e3)$ measured at the geometric center. For local frame origin $\mathbf{p}$, we average the positional measurements of all the IMUs to approximate the center of mass. For the local frame deformation $S$, we obtain an approximate representation $\mathbf{s}$ by computing a scalar angle difference $s_{i}$ between the $Y$-axis measured by each peripheral IMU and the $Y$-axis defined by $\mathbf{R}$, and concatenate these scalars together. In the shown case, each entry of $\mathbf{s}$ represents how its associated wing is folded, where the sign indicates inward or outward and the magnitude indicates the angle. Such a design of $\mathbf{s}$ facilitates interactive piloting since it is intuitive to describe one’s desired deformation in terms of the bent angles. Since we have parameterized $\mathbf{z}$ with $\mathbf{s}$, $\mathbf{e}$ and $\mathbf{p}$, controlling the dynamics of $\mathbf{z}$ now relegates to controlling the dynamics of $(\mathbf{s,e,p})$.

Dynamic Coupling     The drone’s body will be actuated by a set of $m$ propellers, each providing a scalar thrust along the normal of the surface it is planted on. Let $\mathbf{u}\in\mathbf{R}^{m}$ denote the thrusts. The second-order dynamics of $\mathbf{z}$ is given by the momentum conservation:

where $m$ represents the particle mass. We note here the translational and rotational symmetries of the experienced forces. Since $f_{actuation}$ depends on the rotors’ normal directions which are computed locally with neighboring particles, it is unchanged when the whole of $\mathbf{z}$ is rotated or translated. The translational and rotational invariance of $f_{stress}$ and $f_{damping}$ is based on the principle of material frame-indifference in continuum mechanics, which states that the behavior of a material is independent of the reference frame [29]. Combining the symmetry with the equality $\mathbf{z}=\mathbf{R}S(\hat{\mathbf{z}})+\mathbf{p}$, and ignoring the Coriolis forces resulting from the evolving $\mathbf{R}$, we get that:

Since $\hat{\mathbf{z}}$ is constant, $S(\hat{\mathbf{z}})$ is a function of $\mathbf{s}$, and we know $\mathbf{R}$ is a function of $\mathbf{e}$, then the dynamics coupling of $\mathbf{s}$, $\mathbf{e}$, and $\mathbf{p}$ are as follows. Since $\ddot{\mathbf{p}}$ is the average of $\ddot{\mathbf{z}}$, it depends on $\mathbf{u}$, $\mathbf{e}$, $\dot{\mathbf{e}}$, $\mathbf{s}$ and $\dot{\mathbf{s}}$. Since $\mathbf{R}$ is measured in the local geometric center of $\mathbf{z}$, $\ddot{\mathbf{e}}$ depends on $\mathbf{u}$, $\mathbf{e}$, $\dot{\mathbf{e}}$, $\mathbf{s}$ and $\dot{\mathbf{s}}$ as well. For $\ddot{\mathbf{s}}$, since it is measured by projecting $\mathbf{z}$ onto the local frame by left-multiplying $\mathbf{R^{-1}}$, the $\mathbf{R}$ component in $\ddot{\mathbf{z}}$ cancels out for $\ddot{\mathbf{s}}$ and $\ddot{\mathbf{s}}$ depends on $\mathbf{u}$, $\mathbf{s}$ and $\dot{\mathbf{s}}$ only.

These interdependencies will be modelled by learned neural networks, in particular, we train networks $\{\mathbf{d},\mathbf{g},\mathbf{h}\}$ such that $\mathbf{\dot{s}}_{next}\mathbf{=d(s,\dot{s},u)}$, $\mathbf{\dot{e}}_{next}\mathbf{=g(s,\dot{s},e,\dot{e},u)}$, and $\mathbf{\dot{p}}_{next}\mathbf{=h(s,\dot{s},e,\dot{e},\dot{p},u)}$.

Learning the Dynamics      Training the networks $\mathbf{d}$, $\mathbf{g}$ and $\mathbf{h}$ can be done in a relatively straightforward fashion. The three networks share the same lightweight architecture consisting of four residual blocks [30] featuring linear layers as previously explored by [31, 32]. The three networks will be trained separately using the same reservoir of data samples of the form {$\mathbf{s},\mathbf{e},\mathbf{p},\dot{\mathbf{s}},\dot{\mathbf{e}},\dot{\mathbf{p}},\mathbf{u},\dot{\mathbf{s}}_{next},\dot{\mathbf{e}}_{next},\dot{\mathbf{p}}_{next}$} generated from Finite Element Method (FEM) simulation. We use Adam optimizer and L1 loss for optimization, with hyperparameter details given in the supplement. There are two techniques that we adopt that are worth reporting. First, for data generation, we would apply a constant random thrust uniformly sampled in the range $[-T\_max,T\_max]$, where $T\_max$ denotes a rotor’s max output thrust, to the drone’s rotors for $1s$, before a new random thrust is applied. The soft drone would be dancing and twisting in the air, but despite that, this method leads to successful trainings, while generating data from a guiding controller, or switching random control signals at every instant, would fail. Secondly, we’ve found that trainings converge much better when we predict the next frame’s velocity using the current one, instead of directly predicting the acceleration. Although this approach can lead to local minima for settling at the input, current velocity, that does not happen in our experiments and the acceleration can be convincingly reconstructed from the two velocities, as we shall elaborate in the results section.

Controlling the Learned Dynamics     Once the networks $\mathbf{d}$, $\mathbf{g}$ and $\mathbf{h}$ are trained, the dynamics can be expressed as

To control such learned dynamics, we summon the Linear Quadratic Regulator (LQR), a well-proven method for controlling rigid drones. Since LQR requires a linear system, we will perform first-order Taylor expansion around an operating point $\mathbf{(x^{*},u^{*})}$ with Jacobian matrices from automatic differentiation which can be done with PyTorch. In particular, we write:

where $\mathbf{A}=\frac{\partial{\mathbf{f}}}{\partial{\mathbf{x}}}\big{|}_{\mathbf{x^{*},u^{*}}}$, and $\mathbf{B}=\frac{\partial{\mathbf{f}}}{\partial{\mathbf{u}}}\big{|}_{\mathbf{x^{*},u^{*}}}$, as shown in Equation 6:

If we make the assumption that for $\mathbf{(x^{*},u^{*})}$ and $\mathbf{(x,u)}$ close enough to each other, $\mathbf{f(x,u)-f(x^{*},u^{*})}\approx\mathbf{f(x-x^{*},u-u^{*})}$, then we have:

Once the linear system is obtained, LQR outputs the control matrix $\mathbf{K}$ and the control policy $\mathbf{u-u^{*}=-K(x-x^{*})}$ that drives $\mathbf{x}$ to $\mathbf{x^{*}}$ while keeping $\mathbf{u}$ close to $\mathbf{u^{*}}$ by minimizing the cost function $\int_{0}^{\infty}\mathbf{(x-x^{*})^{T}Q(x-x^{*})+(u-u^{*})^{T}R(u-u^{*})}dt$. The $\mathbf{Q}$ and $\mathbf{R}$ matrices are cost matrices used to manage the tradeoff between the two objectives. The optimization is done by solving the Continuous-time Algebraic Riccati Equation with SciPy.

To test the efficacy of our method, we design a number of soft drone models in 2D and 3D featuring asymmetrical structures and odd numbers of rotors, as depicted in Figure. 4. We conduct training on each model individually and use the generated controllers to direct the models to perform hovering, target reaching, velocity tracking, and active deformation.

Secondly, since the LQR controller relies on the Taylor-expanded version of the learned networks, it is necessary to verify how well the networks’ gradient matches that of the actual dynamics. Since the ground-truth gradient is difficult to obtain, we test this by reproducing temporal sequences using the linearized version as in Equation. 5 relinearized at $20Hz$. As shown in the right subfigure, even with error accumulation, the linearized evolution sequence (Jacobian Matrix Update) keeps up with the ground truth for over a minute, showing the linearized neural network can reliably approximate the real dynamics locally.

Aerial Deformation     Our method successfully controls the drones to deform into specified configurations while hovering or tracking a velocity. The first experiment instructs the flower drone to maintain its location and orientation while deforming into two different configurations: first with lateral petals raised and axial petals flat, then with lateral petals flat and axial petals raised, as shown in Figure. 6. In the middle subfigure, the pitch of all eight petals are controlled as expected in a smooth manner: petals $1,\ 3,\ 5,\ 7$ will rise first while petals $2\ ,4\ ,6\ ,8\$ stay still, before the roles are gradually switched. The right subfigure shows that this is done while maintaining precise control of balance and position. Ever since the initial drop in the $-Y$ direction immediately after release, other rotational and translational movements are contained within $\pm 3^{\circ}$ and $\pm 5cm$.

of Figure. 8, our controller increases the wing pitch by over $73^{\circ}$, reducing its width for over 30%. As seen in the second subfigure, it does so while maintaining a steady velocity in the $(+X,+Y)$ direction, and deviating for less than $5cm$ in the $Z$-axis throughout the entire sequence, thus leading to the successful object avoidance as depicted in the bottom of Figure. 7.

SOTA Comparison     We compare our method with three different controllers: traditional LQR, geometry-updating LQR, and a neural network controller trained end-to-end using the strategy proposed by [19]. Our metrics will base on locomotion, since existing controllers for soft drone deformation is unavailable. The first candidate computes an LQR control matrix using the relevant quantities —— fixed point, rotor position, rotor orientation, and rotational inertia, calculated from the drone’s rest shape. The second recomputes the control matrix at every timestep with the relevant quantities updated based on the deformation, with further details given in the supplement. The third takes our trained network dynamic system as a differentiable simulator to train another network controller. We instruct all controllers to direct the flower drone to travel from position $(0,0,0)$ to $(3,3,3)$. As depicted in Figure. 9, our method (left) successfully drives the $(x,y,z)$ coordinates of the drone to the target within 7% error, while the other controllers fail due to the fragility of the soft drone dynamics. As shown in the table below, we compare their performance numerically with three metrics: final error, thrust usage, and the survival time before illegal configurations are encountered, which shows that our method excels the other candidates by far.

Ablation Testing: State Decomposition     The state decomposition is helpful for defining a local frame in which deformation can be expressed compactly as scalars. In this experiment, we train without decomposing and feed the network with the concatenated vector of the position, center orientation, and peripheral orientations. As displayed in Figure. 10, the validation loss is unable to converge successfully, signifying that the network fails to learn the correct patterns. This can be due to the fact that such unprocessed states double the size of the decomposed version, and also that the evolution rules of the position and orientation are highly unalike, thus demanding the network to evolve into multi-modal behaviors.

Ablation Testing: Online Relinearization     We show the importance of the online relinearing mechanism by testing the controller performance where the input targets $\mathbf{(x}_{goal},\mathbf{u}_{goal})$ deviates from a fixed point $\mathbf{(x^{*},u^{*})}$ by different margins. We first obtain a fixed point by trial-and-error and then fine-tune it with gradient descent using our learned networks. Then, we keep the goal location untouched, and pollute the rest of the fixed point by adding uniform random noise proportional to each state variable by 10%, 20%, 30%, 40%, and 80%. We direct the rod drone to translate for 2 units along the $+X$ direction. The results are depicted in the left 2 subfigures of Figure. 11. The first subfigure shows the results without online relinearization, where the control quality deteriorates significantly when over 20% noise is added, and start to generate NANs after 40% noise is added. The second subfigure shows the results with online relinearization, in which the controller performs steadily even with 80% noise added. The significance is that during deployment the pilot only needs to input the target configuration without worrying about the hard-to-compute fixedness of such configuration, thereby making our control system feasible for human piloting.

Effect of Relinearization Frequency     The parameter $n$ controls the relinearization frequency $\frac{100}{n}Hz$. The more often the system is linearized the more accurate the linear approximation is. And also, since we set a new waypoint at each linearization, whose distance away is inversely proportional to $n$, frequent linearzation sets many adjacent waypoints while infrequent linearization sets long-term, sparse waypoints. In this experiment, we test the performance of 9 different values of $n$: $\{1,2,5,10,20,30,40,50,100\}$, and show that it is not the case that larger $n$ implies better performance. As shown in the right two subfigures of Figure. 11, $[5,\ 20]$ is clearly the sweet-spot of this parameter, where smaller $n$ leads to overshoots, and larger $n$ insufficient actuation.

We propose a computational system to generate controllers for soft multicopters that jointly controls the locomotion and active deformation, without relying on extra mechanical parts. Our method takes advantage of a physics-inspired decomposed state space, and train neural networks to represent the dynamics. We control the neural dynamics system using an LQR controller enhanced with a novel online relinearization scheme. We use our method to successfully generate controllers for a variety of soft multicopters to perform hovering, target reaching, velocity tracking, and active deformation.

Sim2Real Transfer      Our method is well-suited for real-world adaptations. First, we limit the interfacing between the simulator and the learning system strictly to sensor readings, so we make sure that no unrealistic benefit is gained from experimenting virtually. Secondly, we form a straightforward guideline for sensor deployment, featuring accessible gadgets with easy installation. Thirdly, the computation of LQR optimization and neural network evaluation can be realistically carried out in real-time by onboard computers, as previously explored by [33, 34]. Finally, being data-driven, our method betters the analytic approaches for Sim2Real adaptation, since it learns from data which contain the unmodelled nuances of the real world. The main challenge for Sim2Real transfer would be the experimental designs to generate meaningful data using the fabricated drones.

Limitations     With our approach there are several limitations. First, the sensor placement requires human design, and there lacks a mechanism to tell, before training, if a sensing scheme would work. Secondly, every drone requires a separate training with no knowledge transfer. Thirdly, we use dual rotors with counter-rotation to cancel the spinning torque effects, which complicates the manufacturing process. Finally, the current control scheme is ineffective for aggressive maneuvers.

In the future, we plan to build an automated system for optimizing sensor locations as well as a unified neural dynamics platform that facilitates knowledge transfer, and to evaluate our approach on real-world soft multicopters.

In this document, we present the supplementary materials to our published paper. In Section B, we describe the specifications of the 2D and 3D drone models used in our training and testing, including the soft material properties, the rotor designs, and the sensor placements. In Section C we introduce the details of how the state vector $\mathbf{s}$ and $\mathbf{e}$ are obtained from sensor readings. In Section D we propose a general guideline for deploying IMU sensors for arbitrary drone shapes. In Section E we describe the simulation environment, the simulation model used, and the noise treatment. In Section F, we specify the details of the learning module, including the network structure used, the data generation scheme, as well as the techniques and hyperparameters used along the training procedure. In Section G, we specify the parameters used in our control module and the mathematical derivation of the benchmark LQR controller. In Section H, we present a discussion about the system design choices, the assumptions we have made, and the potential challenges for the fabrication and control of real-world soft multicopters.

We present a general guideline for selecting where IMUs are deployed in the left part of Figure. 15. Given an arbitrary drone shape in 3D (pressed onto the X-Z plane), we build a tree with the root node being the geometric center of the drone, and the child nodes being the rotors. The IMUs will be inserted at the edges of the tree near the outer rotor. The effectiveness of this approach is contingent on the simple modality of the soft drone’s deformation. For instance, if you take a look at the right part of Figure. 15, for the above case, the deformed shape of the drone’s arm can be approximately reconstructed from the three measurements, whereas in the lower case, the three measurements are far from enough to describe the deformed shape, as the deformation is highly multi-modal, while these higher-order deformation modes are effectively beyond the controlling capacity of the drone’s rotors. As a result, it is the task in the design of these drones (mostly selecting the modulus and thickness) so that the drone is soft enough to perform significant deformation, while the deformation mode of the drone is simple. In practice, this IMU insertion guideline works for our various examples.