Soft Walks: Real-Time, Two-Ways Interaction
between a Character and Loose Grounds

Real-time animation of characters walking on loose grounds such as sand, snow, mud or soil, possibly covered with grass, is a key element in video games and virtual-reality applications involving natural environments. Modeling plausible walking motions on such terrains requires two-ways interactions. Indeed, not only should the character’s feet deform the loose material of the ground, but this deformation should, in return, dynamically impact the walking style, since both balance and motion IKs are to be adapted to the evolving terrain.

While recent work based on deep and/or reinforcement learning allows to animate character motion on terrains with a variety of rigid slopes and obstacles, real-time character animation on loose grounds and deformable vegetation remains a challenge. Indeed, capturing enough learning data would be difficult, while optimizing motion-controllers would require costly training sessions where all collisions and interactions between the character and the environment are fully simulated. These simulations remain notably out of reach when loose material and soft interactions are involved.

This work takes a different path, in proposing a very simple and efficient, yet versatile, method to modify a character’s walking style, through two-ways interactions between specific layers of its motion controller and the environment, the latter being consistently deformed through efficient, procedural functions. Since it achieves real-time performances, our method is well suited to game engines such as Unity (Unity Technologies), which we use for illustration.

Our main contribution is a new, multi-layer animation modifier for the character (Sec. 3) which applies at different levels of details while reusing and extending standard game-engine principles, namely combination of approximated physics-based dynamics, pre-set motions and inverse kinematics (IK). We first introduce, at a high level, a global orientation controller allowing the character to dynamically tilt and improve balance by targeting an equilibrium state that depends on the nature and slope of the terrain. Second, at a finer scale, we model two-ways interactions between the character’s feet and the environment by using simple distance-based rules used to both dynamically deform the environment and adapt feet placement and character walking style in real-time. Lastly, we show how such layered control can be used to enhance standard game engines in different scenarios, such as walking on sand, soil, mud, snow and grass with different slopes (Sec. 4). In each case, a specific deformation is applied to the environment, which consistently impacts the walking style. The method runs in real-time, and opens the way to extensions to other loose environments, as well as refinement in adding more layers to the character’s motion controller.

Modeling plausible character motion is a complex problem, which couples behavioral intents with physics-based equilibrium constraints. Fully kinematic models were explored in early work [SM01] leading to real-time adaptation of walking cycles to uneven terrains, at the price of lacking dynamic response to terrain changes. Coupling preset gaits with dynamic events was handled through the help of animation controllers [YLvdP07] that convert differences between the actual pose and the prescribed one as torques, which are then fed into a physically-based simulation computing dynamic state changes. Coupled with reinforcement-learning [PALvdP18] and/or adapted reward functions [KLP20], these approaches can model a wide range of plausible complex motions up to acrobatic effects [WGH20]. These motions automatically adapt to their simulated environment, at the price of their high computational cost and indirect control, and thus cannot be directly applied to loose deforming terrains in standard game engines. In reverse, gathering and processing large prerecorded motion data-bases allow for the use of deep-learning methods [HKS17] which, once learned, can achieve real-time performance. Still, the need of an extensive motion data-base would not scale well to the variety of loose environments targeted in this work.

Closer to our goals, simple dynamic models applying forces to the center of mass to control the general balance of a character, seen as an inverted pendulum [KH10], allow a good high-level trade-off between efficiency and plausibility. Mitake et al. [MAA^∗09] proposed a high level representation of an oscillating proxy volume, triggering pre-recorded detailed animations (walking, running, falling). This work achieves real-time animation but is restricted to pre-set animations on flat grounds. Other works attempt to exploit to a greater degree the environmental impact on the dynamic model. Bermudez et al. [BTZZ18] proposed a method for applying drag forces to a character walking in a fluid and adapting its motion to the medium. Similarly to ours, this approach rely on high level controller-based model. However, this previous work only take into account the influence of the environment, ie. the fluid, toward the character’s gait, but not the coupled two-ways interactions between them as in this work.

Our method takes as input an animated character model from a standard game engine, ie. a mesh geometry, a rigged skeleton with preset animations, an IK system applied to feet bones, on top of a simple proxy-geometry used for collision processing, and a global dynamic behavior computed in simulating a single rigid body model expressed by a center of mass and an inertia tensor. This model allows the character to walk on flat and sloppy terrains, but does not allow to dynamically adapt its gait to varying ground slopes, leading to a robotic-looking motion that lacks dynamic movement. Furthermore, it does not model any two-ways interaction with the environment, therefore restricting the animation to rigid grounds.

The aforementioned elements can be seen as a layered model for character motion-control, where each layer acts at a different level of detail. They span from global control for the rigid body that controls the position and orientation of the coarse geometric collider, to local control for the IK, which positions, computed using dynamic ray-casting, allow for accurate feet positioning on the ground. In this work, we enhance the two layers we just mentioned, by firstly adding an extra swing-control term to the global rigid body model, and secondly by dynamically adapting feet IKs, enabling them to trigger deformations of the surrounding ground and grass. Taken together, these contributions result in consistent interactions between a walking character and an arbitrary, natural-looking environment.

We implemented our method within the Unity Game Engine¹¹1https://unity.com, running in real-time on standard laptops, and used it to model interactions between a walking character and a variety of terrains, namely dry sand, soil, mud and snow. We considered $10\times 10$ meters terrains with a maximum height of $10$ meters and character sizes ranging from $1$ to $1.9$ meters. Height fields were typically sampled using $512^{2}$ points.

Footprints computed on terrains without vegetation are illustrated in Fig. 3. Controller parameters in Eq. (1) were set to $\alpha=30$, and $\beta=6$. The variety of prints was achieved by tuning the depth, compression, and smoothness parameters described in Section 3.2. Dry sand was set to show limited depth changes, while exhibiting non-compressible behavior and large smoothing. Soil was set to smaller depth but higher compression than sand. Mud and snow were set to allow deeper carving, prints on mud being set to be sharper that those on snow. The resulting instantaneous change of terrain height – controlled through the depth coefficient – has an immediate effect on the character’s gait, as shown on the joint video.

The grass was represented as billboards (see Figure 2 and the accompanying video), and its deformation was implemented in Unity using shader-graphs to allow efficient space deformations. We considered two typical grass heights : small ($0.5$ meters) and medium ($0.9$ meters). The size of the vegetation influences the maximal height of the platform set at one third of the vegetation’s height. We also forced the character to swing less in the case of higher vegetation, to model friction. To this end, we adapted the controller parameter $\beta$ to range linearly from 6 (no vegetation) to 4 (medium vegetation).

More examples combining different terrains, slopes and vegetation are shown in Fig. Soft Walks: Real-Time, Two-Ways Interaction between a Character and Loose Grounds and in the supplemental video. In all situations shown, the character automatically adapts its gait while dynamically interacting with the surrounding, loose environment.

This work shows that simple layers acting at different levels of motion controllers can enrich real-time character animation, and handle two-ways interactions with natural environments. Simple and efficient, our method achieves consistent changes of the environment and the character’s gait.

Among limitations, sand footprints do not account for slope, leading to symmetrical shapes even on steep dunes where sand-slides should be taking place. Secondly, our handling of vegetation is limited, billboards being unable to capture realistic deformations. We also fail to handle different natures and densities of vegetation, such as plastic deformations for broken rods. Lastly, our prototype implementation only models straight walks on small terrains. Extending balance control to orthogonal motion directions would allow the character to turn, while using adaptive, local subdivision of the terrain would allow to freely navigate the character on larger and visually richer terrains.

An interesting avenue for future work would be the addition of new control layers to other parts of the character to improve both its global and local adaptations to the environment. We are particularly interested in extending the approach to other part of the character such as the motion of his arms through high vegetation to clear bushes or protecting its head. Extension toward more accurate impact on the environment could be explored through learning local deformation that could be obtained from simulation or real data, while reinforcement learning could be used to improve the response on some layer of the character such as its feet placement. Lastly, our method was specifically designed for walking motions on quite stable terrains.Generalizing control and interaction layers could enable to handle more complex cases, such as a character or any other creature, running, jumping or climbing a scree with rolling stones.

This work has been funded by the Marie Sklodowska-Curie Innovative Training Network CLIPE, in the framework of Horizon 2020.