Clutter-Aware Spill-Free Liquid Transport via Learned Dynamics

A challenge in spill-free trajectory generation is that the volume of the robot configuration space that contains spill-free trajectories is small and therefore inefficient to explore with random sampling alone [20]. This dictates the need for an informed approach to explore dynamically-valid robot states. However, developing a model for combined robot and object dynamics, specifically spillage dynamics, is computationally expensive, task-specific, and impractical for real-world control. To address this, we need to mathematically formalize how container properties inform the construction of the spill-free space. For example, for a given container that contains a greater volume of liquid, it cannot be tilted as much as when it contains less liquid. Additionally, in scenarios where containers have the same liquid volume, taller containers can be tilted more than shorter ones.

To sample spill-free configurations from the spill-free space, we first calculate the maximum tilt angle that a container can withstand without spilling. Then, we sample states ensuring that their orientation remains within this range. Specifically, under quasi-static conditions, we consider the scenario that involves tilting a container to the point where it cannot retain all its content, i.e., to the angle $\theta_{max}$, such that any $\theta_{t}\leq\theta_{max}$ would be spill-free and any $\theta_{t}>\theta_{max}$ would cause spillage (Fig. 3). Quasi-static conditions are assumed because they simplify analytical modeling without requiring expensive computational fluid dynamics calculations.

To formalize the spillage scenario, we model the container as a frustum of a cone, which accurately represents most common container shapes and can provide an upper bound on the maximum tilt angle of containers with other shapes, such as round bottom flasks or curvy wine glasses (Fig. 3(a)). Specifically, we simplify curved containers to the largest possible frustums of cones that can fit inside the original shape, leading to predicting spillage at a lesser tilt angle than the real container’s capabilities. This guarantees a $\theta$ less than $\theta_{max}$, ensuring the generation of spill-free states. Lastly, we simplify the frustum of the cone representation from 3D to 2D, resulting in a trapezoid. This simplification also allows us to represent planar symmetric containers such as rectangular prisms and can be directly extended to 3D.

To solve for the maximum tilt angle, we assume that the information we have from the container is the bottom circle radius $r_{b}$, the top circle radius $r_{u}$, the cup height $h_{c}$, and the water height $h_{w}$ (Fig. 3(a)). With this information, we can derive other relevant variables for calculating the maximum tilt angle. As shown in Fig. 3, two spillage scenarios could occur: (i) one where the liquid forms a trapezoid and a triangle (Fig. 3(b)), and (ii) another where the liquid only forms a triangle (Fig. 3(c)). This happens because static liquid stays parallel to the horizontal ground and depending on the liquid level, these cases can occur.

To calculate the maximum tilt angle, for both spillage cases, we leverage the key insight that the total amount of liquid remains constant before and after tilting the container. By equating the liquid area in the initial and tilted configurations, we derive an algebraic formula to compute the angle. The resulting equations, Eq. 1 for case I and Eq. 2 for case II, provide the respective maximum tilt angles.

Maximum tilt angle for case I:

where $r_{w^{\prime}}=\frac{h_{c}r_{u}r_{b}-h_{c}r_{2}r_{b}+h_{w}r_{2}r_{w}+h_{w}r_{2}r_{b}}{h_{c}r_{2}+h_{c}r_{b}}$.

Maximum tilt angle for case II:

where $r_{w}=r_{u}-[(\frac{r_{u}-r_{b}}{h_{c}})(h_{c}-h_{w})]$.

The next section explains how these equations are incorporated in the sampling-based path planner to inform the construction of a spill-free path.

Having formalized the maximum tilt angle $\theta_{max}$, it is now used to guide the exploration of states that avoid spills, such as an upside-down container, or a container that is tilted beyond its capacity (i.e., $>$ $\theta_{max}$). To achieve this, we implemented an informed sampler to sample robot configurations where the container orientation is less than the spill threshold $\theta_{max}$. Orientation refers to the container’s Euler angles, and it is sampled such that the angle between the vertical axis and the axis about which the container is symmetric, is less than $\theta_{max}$. This informed sampler is then incorporated into a geometric RRT* path planner, to generate spill-free paths. RRT* is chosen due to its asymptotic optimality properties. By preferring transitions with minimal cost, RRT* generates smooth motion profiles that maintain orientation stability [21]. The next section describes how the path generated by RRT* is parameterized to ensure a spill-free trajectory.

While RRT* generates spill-free paths, to ensure the generation of a spill-free trajectory, we must carefully parameterize the path such that the velocity, acceleration, and jerk, do not cause spillage.

To do this, it is necessary to first model the risk of spillage with respect to the trajectory. To capture this relationship, we employed a transformer-based machine learning model trained to classify whether a container’s trajectory is spill-free or not. This model is termed the Spill-Free Classifier (SFC), and utilizes the transformer architecture, as depicted in Fig. 4.

The input to SFC is the container trajectory and container properties, where the trajectory is the position and orientation over time and the properties represent the container’s top and bottom radius, container height, and liquid fill level. Furthermore, as shown in Fig. 4, the trajectory is first passed through the transformer encoder layer. Then, the container properties are concatenated with it. The SFC model employs the transformer architecture because its input is sequence data (the trajectory) and transformers have become the leading architecture for handling sequential data due to their efficient parallel training capabilities and long context window [22].

Lastly, to train the SFC model, we utilized a motion capture system to collect container trajectories (comprised of position and orientation over time) performed by individuals across different container types and liquid volumes (Fig. 6). We collected a total of 700 trajectories, including 655 human-performed trajectories and 45 trajectories from the Panda robot. These trajectories encompassed a variety of paths from holding cups upside down and following zig-zag patterns to transporting cups in straight or typical paths between two points.

Having modeled the relationship between the container trajectory and spillage risk, we leverage SFC to generate spill-free trajectories. We introduce the Spill-Free Time Parameterization (SFTP), which guarantees the creation of spill-free trajectories by utilizing a time-parameterization algorithm alongside the SFC model. The key idea is to find the maximum jerk that does not cause spillage. To achieve this, SFTP works by iteratively (line 5, Algorithm 1) reducing the maximum jerk of the trajectory (line 12, Algorithm 1), querying SFC after each iteration (line 8, Algorithm 1) until SFC predicts the resulting trajectory is spill-free. Jerk constraints are especially critical in preventing sudden changes in acceleration that could cause spillage. SFTP only modifies the jerk parameter since it directly affects acceleration and velocity. Lastly, for time parameterization the Ruckig algorithm [23] is used since it can enforce constraints on higher-order derivatives including jerk, acceleration, and velocity.

In summary, to generate a spill-free collision-free trajectory (Fig. 2), first, the container shape and fill level are leveraged to inform the generation of a path using RRT*. The spill-free classifier (SFC) model is then used alongside the Ruckig time-parameterization algorithm to ensure the generation of a spill-free trajectory. We refer to this combined process of generating the path and the trajectory, the spill-free RRT* (SFRRT*) algorithm.

The performance of SFRRT* depends on the spill constraint enforced during path planning. This constraint restricts the container’s orientation by imposing maximum allowable tilt angles to prevent spillage. As described previously, these angles are calculated using Eq. 1 and Eq. 2. Table I, last column, presents the maximum tilt angles for the containers used in the experiments. To train the spill-free classifier (SFC) model, we deployed it on TensorFlow and optimized it using a binary cross-entropy loss function for 400 epochs with a batch size of 32. SFC achieved an accuracy of 94%. Detailed training parameters are available on the open-source code.

Finally, several key parameters are set in SFTP (Algorithm 1). The bounds on velocity, acceleration, jerk and are set according to the robotic arm’s kinematic constraints [24]. Moreover, the jerk decrease factor $rate$ is set to 2, meaning when SFTP fails to find a spill-free trajectory, $J_{max}$ will be halved and SFTP will reevaluate for spillage under the updated constraint. This $rate$ selection balances execution time and planning time: a smaller rate would extend the time to find a spill-free trajectory, while a larger rate could lead to unnecessarily slow trajectories due to large jerk constraints.

To measure the success rate of the experiments presented in Table I, each experiment was repeated five times and the result is presented in Table II. Success is defined as the absence of spillage and environmental collisions during the executed motion. Overall, we achieved a 100% success rate for all experiments except experiment 5. For experiment 5, spill-free trajectories were accomplished in four out of five trials. Experiment 5 faced challenges since it used a container with a maximum tilt angle of 35^∘, smaller than others, limiting the trajectory planning options and increasing spillage risk. Additionally, the SFC model, with its 94% accuracy, had difficulty adapting to this constrained scenario, contributing to the lower success rate.

A key capability of SFRRT* is its ability to navigate around obstacles in the environment while ensuring the container does not spill. To analyze this ability, we focus on the end effector orientation allowable ranges. In SFRRT*, the container orientation is constrained by the maximum tilt angle of the container, i.e. it can generate spill-free trajectories, so long as the container is not tilted past the maximum tilt angle. By enabling the robot to tilt the container to higher angles, it can more flexibly maneuver in cluttered spaces, such as a fume hood, which may require tilting chemistry vessels to fit them inside.

To illustrate this expanded ability, Fig. 7 shows the maximum possible and maximum utilized tilt angles of all 30 trials of Table I. In every experiment, SFRRT* utilizes a larger tilt range than the SpillNot [3]. Notably, SpillNot’s tilt angles are not contingent on the fill level and are always limited to a maximum of 15^∘ to maintain low linearization error ([3], Fig 4). We also chose [3] for comparison because, unlike other methods, it does not rely on sensor-based slosh modeling, and allows translational motion in all axes.

As seen in Fig. 7, using SFRRT* the robot achieves a maximum tilt angle of 45^∘ when transporting the basic container with a 30% fill level (Table I, experiment 6), surpassing the SpillNot method’s 15^∘. The range of orientation for this specific container is 3x larger than [3]. Crucially, the maximum tilt angle for the flute glass with a 70% fill level is 72^∘, a 4.8x increase in orientation range.

Furthermore, it is important to enable orientation changes along all axes to avoid obstacles. For example, maintaining a fixed pitch angle throughout the trajectory prevents avoidance of obstacles in certain cluttered environments. SFRRT* can generate trajectories where all three axes of orientation are utilized. Fig. 8 displays the container orientation over all the 30 trajectories generated for Table I. As shown, there are changes in Euler angles across all axes, the range for roll, pitch, and yaw angles are $[-49,41]$, $[-30,30]$, and $[-16,9]$ degrees, respectively. In contrast, the SpillNot method imposes a fixed yaw angle throughout the trajectory [3].

The Spill-Free Classifier (SFC) model is trained using 75% of the 700 trajectories and achieved an accuracy of 94% for the validation set. To evaluate the generalization capability of the SFC, its accuracy was tested on novel containers not present in the training dataset. Specifically, a round bottom flask, beaker, and graduated cylinder were used to gather multiple trajectories using a motion capture system (Table III). For each container type, 12 trajectories were gathered with varying fill levels.

As shown in Table III, SFC demonstrates good generalization capability. It achieves strong performance for the beaker and graduated cylinder, with 11 out of 12 data points accurately predicted. However, its accuracy is lower for the round bottom flask, with only 8 out of 12 data points predicted correctly. This is because the SFC model input for container properties only includes details that capture curvy-shaped containers by estimating them to a frustum of a cone. By representing these shapes as frustums of cones, the SFC model ensures a conservative yet safe portrayal of container properties. While this cautious approach may occasionally lead to conservative false negatives, it guarantees the generation of spill-free trajectories.

We also explore a different model architecture for SFC. The current architecture (Fig. 4) passes only the trajectory to the transformer layer before concatenating with container properties. We modified this by concatenating properties at each trajectory step before passing it to the transformer. We call this modified architecture SFC_m. SFC_m achieved only 88% accuracy, lower than the 94% of SFC, and demonstrated lower accuracy on novel containers than SFC (Table III).

Lastly, SFC is able to take advantage of the container shape to generate trajectories with different velocities. As seen in Fig. 10, considering the same containers, with different fill levels we see that it consistently generates higher velocities for containers with lower fill levels, which are less likely to spill. Moreover, when comparing different containers in the same scenario (as in Table I), those with higher tilt angles consistently achieve higher velocities. For example, in the comparison between the wine glass and the flute glass, the wine glass consistently achieves lower mean velocities. This is attributed to the geometry of the containers. As shown in Table I, the wine glass has a lower maximum tilt angle compared to the flute glass, indicating smaller room to maneuver and a higher susceptibility to spillage. Conversely, the flute glass, with its higher maximum tilt angle consistently achieves higher mean velocities.

In this section, two key elements of SFRRT* are modified to evaluate their impact on performance: the informed sampler and the spill-free classifier (SFC) model.

Informed Sampler Variation: SFRRT*_u. To avoid sampling states such as an upside-down container, or a container tilted beyond its capacity, the informed sampler samples states within the maximum allowable tilt angles for the container (Table I, last column). To measure its impact, we replace it with a uniform sampler, creating SFRRT*_u. Then, experiments 1 and 2 are executed 5 times on the robot, and the success rate is shown in Table IV. The success rate drops significantly since the path generation lacks state sampling constraints, as a result, the time parameterization algorithm struggles to create a spill-free trajectory using this path.

Spill-Free Classifier Variation: SFRRT*_r. In SFRRT*, the SFC model assists with time parameterization by selecting the jerk constraints to ensure a spill-free trajectory. To evaluate the impact of SFC, we introduce SFRRT*_r by replacing the SFC model with a random jerk selection. More specifically, a random jerk is selected out of all the jerks that were ever chosen by SFC. The path is then parameterized using that jerk. Table IV shows the success rate for experiments 1 and 2. The observed drop in success rate is due to the random selection of jerk constraints without considering the path itself, in contrast to using the SFC model which takes the path into account when choosing the jerk constraints.