UniDoorManip: Learning Universal Door Manipulation Policy Over Large-scale and Diverse Door Manipulation Environments

Building a door manipulation environment that simulates the real world and transferring the policy trained in simulation to the real world has been the main approach for door manipulation tasks in recent years. However, recent works covering door manipulation have mainly two drawbacks: 1)Unrealistic simulation. Besides using push or pull to open the door[21, 37, 33, 6, 5, 4], Robosuite [40] benchmarks opening doors with latching mechanism as standardized task but employs small doors with large handles. 2)Lack of diversity in the dataset. PartNet-Mobility [36] and AKB-48 [17] provide a diverse dataset for articulated objects including doors. Focusing on the cross-category diversity, they ignore the intra-category diversity of doors. To address these two problems, we build a door manipulation environment with diverse realistic manipulation mechanisms, and equip this environment with a large-scale door dataset covering diverse types, handles and geometries for universal manipulation policy learning.

Extensive investigations have been conducted in the field of articulated objects, encompassing various facets such as reconstructing the dynamic structure of articulated objects [1, 10, 14, 23, 19, 12, 3], estimating 6d pose [11, 18, 15, 32, 6], comprehending the intricacies of manipulating them [13, 37, 4, 5, 22, 9, 7, 2, 26], and point-level visual affordance [21, 33, 31]. Among these areas of study, particular attention has been dedicated to the manipulation of articulated objects, with a specific emphasis on generating optimal action sequences. Previous works [29, 5] have explored the training of a single universal policy using state-based  [29] or visual-based RL [5, 7], which suffer from vast exploration space and complicated manipulation mechanisms for the door manipulation task. Otherwise, we propose a novel framework that disentangles the whole manipulation process into three stages with respective universal policies and integrates them into the whole universal policy leveraging conditioned training.

In recent years, several works have proposed their datasets for door manipulation as illustrated in Table 1. DoorGym [29] claims a large-scale and scalable dataset specifically for door manipulation. Due to the hard-coded generation and the same templates, the doors they construct lack the diversity of geometry, which leads to bad performance on unseen doors. Limited by the environment, the door boards don’t have a corresponding urdf or object entity, which can not be transferred to other environments. PartNet-Mobility [36] and AKB-48 [17] provide a diverse dataset for articulated objects including doors. Focusing on the cross-category diversity, they ignore the intra-category diversity of doors, which are not suitable for our goal. Hence we introduce a large-scale diverse dataset specifically for door manipulation.

Compared with the current dataset providing the whole door without part composition over diverse configuration, we construct our dataset in the formulation of two door parts, named body and handle. Given the composition method we provide, users of our dataset can configure their personalized door objects with diverse settings according to their needs. In total, we create 328 bodies and 204 handles which can be composed of thousands of door objects, as shown in Table 1.

To satisfy the need for diversity, we make efforts at both the category and intra-category level. For category level, We select 6 representative categories that cover most of the door-opening scenes we encounter in the real world, including Interior, Window, Car, Safe, Storagefurniture and Refrigerator. For diversity in the intra-category, we collect hundreds of door bodies and handles with irregular geometries and diverse manipulation mechanisms as illustrated in Figure 1. Table 1 provides a detailed statistic comparison between previous datasets and ours. Notice that we do the calculation for all object assets based on whether their manipulation mechanisms are similar to doors.

To closely emulate the real world, we specifically configure our environment both from the manipulation and the observation. To simulate the latching mechanism, we apply a force $\mathcal{F}_{door}\left(\theta\right)$ in the reverse direction of the opening to the doors until the handle joint angle $\theta_{h}$ have reached the opening threshold $thre_{door}$. Furthermore, we also add a resilient force to both the handle $\mathcal{F}_{handle}\left(\theta\right)$ and door to ensure the robustness of the manipulation policy. The forces are formulated as:

$$\mathcal{F}_{door}\left(\theta_{d},\theta_{h}\right)=\left\{\begin{aligned} F_{f}&,&\quad\theta_{h}&\leq thre\\ k_{1}\theta_{d}&,&\quad\theta_{h}&>thre\\ \end{aligned}\right.$$

(1)

$$\mathcal{F}_{handle}\left(\theta_{h}\right)=k_{2}\theta_{h}$$

(2)

Where $\theta_{d}$ represents the door joint angle. Here we set $k_{1}=3,k_{2}=3,F_{f}=150$. The unit of force is $Newton$. Instead of a parallel or suction gripper, we utilize the whole robot arm with a mobile base and parallel gripper as the agent to manipulate the doors. In this condition, collision between doors and robot arm as well as the joint limitation of the robot arm itself must be taken into consideration. Unlike previous works using the pseudo perfect observation [7, 6, 33, 21] or partial observation [5, 37, 4] which don’t consider the occlusion caused by the door itself or the robot arm, we put a fixed camera in front of the door and robot arm and acquire the partially occluded point cloud generated by the depth image as the observation. Table 2 shows the detailed comparison between our environment and others from various aspects.

We disentangle the door manipulation task into three stages: handle grasping, handle manipulation and door opening. Following UMPNet [37], we define the policy $\pi$ for each stage to be generating robot actions $a_{t}$, given the partial point cloud observation $O_{t}$ and the robot state $S_{t}$ at time $t$. Here, the robot action $a_{t}=(p_{t},r_{t})$ indicates the next end-effector pose of robot arm, consisting of position $p_{t}\in\mathbb{R}^{3}$ and orientation $r_{t}\in\mathbb{SO}(3)$. The robot state $S_{t}$ consists of its joint angles and velocities.

In stage one, given the initial point cloud observation $O_{0}$ and the contact point $p_{0}\in O_{0}$, the model outputs a per-point affordance map $A_{O_{0}}$ where a higher score on $p_{0}$ indicates a greater chance of successful door manipulation after grasping $p_{0}$ with action $a_{0}$.

In stage two, given the observation $O_{1}$ after grasping, the goal is to manipulate the handle in the right direction to unlock the door. Given $O_{1}$ and robot state $S_{1}$, we train a manipulation policy to generate the action $a_{1}$ for diverse mechanisms.

In stage three, the goal is to open the door as wide as possible. Given the current visual observation $O_{t}$ and robot state $S_{t}$ at time $t$, we employ a manipulation policy to generate the opening action $a_{t}$ in a close-loop manner until the door joint angle $\theta_{d}$ exceeds a threshold.

As door manipulation includes multiple different stages, it is difficult to directly train a universal policy in an end-to-end manner covering all the manipulation policies in those diverse stages. Therefore, we disentangle the door manipulation into three distinct stages. Due to the similarity of the manipulation policies in each stage, it is easy to train a universal and generalizable policy in each stage and merge them into a whole universal manipulation policy.

While separated, these related stages have internal dependencies. For example, if the robot arm grasps the handle too far outward, it may result in disengagement while rotating the handle. Also, if the robot arm has a strange state in handle manipulation, it’s easy to collide with the doors while the door is opening. Hence, to integrate all policies for each stage, we employ a conditioned training formulation leveraging these internal relations. Because we can evaluate the results of handle grasping during handle manipulation and evaluate the results of handle manipulation during door opening, we train the policy for door opening first. Then we train the handle manipulation policy using data collected based on the door opening policy. Given the policies for the latter two stages, we can obtain the first policy for initial handle grasping.

Due to the diverse geometries of different handle types, it’s challenging to train a universal and generalizable manipulation policy for initial handle grasping. While existing heuristic methods leveraging the pose estimation [6] or keypoint representation [30] can not generalize to objects with irregular shapes, we employ point-level visual affordance [8] for manipulation, which predicts a point-level score map on the target object, indicating actionability for downstream tasks.

The policy for handle grasping consists of three models: affordance predictor $\mathcal{E}_{\text{grasp }}$, action sampler and action discriminator $\mathcal{D}_{grasp}$. During inference, taking the initial observed point cloud $O_{0}\in\mathbb{R}^{3\times 4096}$ and the contact point $p\in\mathbb{R}^{3}$ as input, the affordance predictor $\mathcal{E}_{\text{grasp}}$ predicts a point-level score map where we choose point with highest score as the grasp point. We employ a segmentation version PointNet++ [25] to extract a per-point feature from $O_{0}$, where $f_{p_{0}^{i}\mid O_{0}}\in\mathbb{R}^{128}$ represents the feature of point $p_{0}^{i}$, and use MLP to encode the contact point $p$ into a latent representation $f_{p_{0}}\in\mathbb{R}^{128}$. Based on the grasp point, the action sampler proposes alternative actions $\left\{a_{0}^{0},a_{0}^{1},\ldots,a_{0}^{k}\right\}$ by sampling random orientations from the normal plane of the handle axis and combining each of them with the grasp point. Taking the concatenation of the point cloud feature and the action feature extracted by another MLP, the action discriminator $\mathcal{D}_{grasp}$ outputs an action score. We select the action with the highest action score to grasp the handle.

In the reverse order of inference, we train the model of action discriminator $\mathcal{D}_{grasp}$ firstly which is supervised by the final door joint angle $\theta_{d}$ using $L_{1}$ loss. For training affordance predictor $\mathcal{E}_{\text{grasp }}$, we sample 50 random actions based on the contact point $p_{0}$ selected from $O_{0}$. Then we estimate their action scores using $\mathcal{D}_{grasp}$ and regress the prediction to the mean score of the top-10 rated action with $L_{1}$ loss.

Leveraging these diverse manipulation mechanisms provided by our proposed environment, we employ a generative network architecture to learn the high dimensional data distribution and output the handle manipulation action. The policy for handle manipulation is composed of two modules: action generator $\mathcal{G}_{handle}$ and action discriminator $\mathcal{D}_{handle}$. The action generator $\mathcal{G}_{handle}$ is implemented as a conditional variational autoencoder(cVAE) [28], composed of an action encoder that maps the input action $a_{t}$ into a Gaussian noise $z$ and an action decoder that reconstructs the action input from the noise vector. We set the current point cloud $O_{t}$ and robot state $S_{t}$ as the conditions of the encoder and decoder, which are implemented both by MLPs. Given the alternative actions $\left\{a_{1}^{0},a_{1}^{1},\ldots,a_{1}^{k}\right\}$ generated by $\mathcal{G}_{handle}$, plus $O_{t}$ and $S_{t}$, the action discriminator predict action scores which we select the action with the highest score as the manipulation action.

For training action generator $\mathcal{G}_{handle}$, we use the KL divergence loss for regularizing Gaussian bottleneck noises, the $L_{1}$ loss to regress the action position $p_{t}$ and the a 6D-rotation loss [39] for supervise the action orientation$r_{t}$. The total loss can be formulated as:

$$\mathcal{L}_{h}=\lambda_{KL}\mathcal{L}_{kl}+\lambda_{pos}\mathcal{L}_{pos}+\lambda_{rot}\mathcal{L}_{rot}$$

(3)

where we use a set of hyper-parameters to adjust the loss balance. For training action discriminator $\mathcal{D}_{handle}$, we use $L_{1}$ loss to supervise the action score with the residual handle joint angle between before and after the action execution.

To tackle the diverse geometries and long action sequences, Instead of open-loop manipulation [33], we employ a closed-loop formulation that iteratively generates the next action $a_{t}$ for door opening based on the current observation including the partially occluded point cloud $O_{t}$ and the robot state $S_{t}$ for door opening stage. We follow the implementation of the policy for handle manipulation and employ an action generator $G_{door}$ and an action discriminator $D_{door}$ to output the door opening action. Different from the previous discriminator, we supervise the action score with the residual door joint angle between before and after the action execution.

To train policies for the three stages, we collect data including the input observation and ground truth supervision leveraging the rule-based method. For different manipulation mechanisms and stages, we use the door states like the joint axis of the handle and body which can be acquired in the simulation environment to calculate the next action.

In the reverse order of inference, we train the door-opening policy first. By supervising the policy with the final door manipulation result, the door opening policy can generate action that facilitates to open the door as wide as it can. Then we replace the rule for the door opening with the trained policy and collect data for handle manipulation policy training. In this way, the handle manipulation policy is integrated with the door opening policy, which outputs action that facilitates door opening. Finally, we replace the latter two rules with trained two policies and collect data for affordance training. Due to the supervision of the final door joint angle for affordance score, the policy for the initial handle grasping can predict a foresightful grasp pose which helps to avoid the bad cases in the following stages, like the disengagement between the end-effector and handle, the collision between the robot arm and door.

We conduct our experiments on the representative door manipulation tasks: pull door. The initial closed door can only be unlocked and pulled to open. The robot arm needs to pull the door until the door joint angle $\theta_{d}$ is larger than a threshold $thre_{door}$. Here, we set $thre_{door}$ to be $45^{\circ}$.

For training and testing, to show the generalizability of our proposed framework across diverse categories, geometries and manipulation mechanisms, we train the universal manipulation policy on the door category including Interior, Window, Safe and Car. We carefully split the object assets of the aforementioned categories into the train and test part which ensures the universal policy is tested on unseen shapes. Furthermore, we also leave two more categories, Storagefurniture and Refrigerator, for extensive tests on unseen novel categories.

For evaluation metric, we use the task success rate as our main evaluation metric. To reduce the evaluation noise, we conduct each experiment 3 times using random seeds and report the mean performance as well as the variance.

We compare our method with the following baselines: 1) GAPartNet + GT: following the heuristic manipulation method of GAPartNet, we directly obtain the initial ground truth like poses of the handle and board from simulation and implement a heuristic motion planning method for door manipulation. 2)DoorGym: we introduce the state-based PPO [27] implemented in DoorGym as our baseline. Here we use a flying gripper as the agent and put the gripper close to the handle in the beginning. 3)PartManip: we introduce a visual-based PPO used in PartManip as our baseline. The setting is similar to DoorGym. 4)VAT-MART: a method that proposes visual action trajectory for downstream manipulation task in an open-loop formulation. For a fair comparison, We develop an implementation version in our environment.

Figure 3 shows the whole manipulation sequence of our universal manipulation in four categories, including Safe, Window, Interior and Car. The results show that our universal policy can generalize over diverse categories, geometries and manipulation mechanisms. Located in the upper left of the initial pictures, we use a heap map to visualize the affordance score where the redder point indicates the higher score. It’s worth mentioning that the affordance map represents where the fingers of the end-effector contact with the object, which just matches the result affordance of the round handle for the category Interior.

Table  3 shows that our framework outperforms all baselines for both the train and test categories in the pull door task. When encountering handles with irregular types and geometries, GAPartNet + GT can not calculate a proper initial grasp pose even provided with ground truth handle pose. Besides, the heuristic motion planning method can not generalize well in unseen objects. For the two RL-based methods DoorGym and PartManip, due to the large exploration space for multiple stages and complicated reward engineering for diverse categories, these methods fail to train an end-to-end policy for door manipulation. Open-loop method VAT-MART predicts the action trajectory with the initial observation, which leads to accumulation error and inability to real-time adjustments.

To further evaluate the importance of different components of our method, we conducted an ablation study by comparing our method with four ablations: 1) Ours w/o disentangle.: instead of training policies for the latter two stages separately, we use a single policy to generate the manipulation action after handle grasping. 2)Ours w/o condition.: ours without conditioned training. 3)Ours w/o state.: ours without the input observation of robot state. 4)Ours w/o mob.: ours that uses a fixed base robot arm instead of a mobile robot arm.

The experimental results presented in Table 3 and Figure 4 provide evidence supporting the efficacy of the four components we have devised in enhancing the performance of the universal manipulation policy. Figure 4 depicts the trends observed in the performance of Ours and Ours w/o condition. as the door joint angle $\theta_{d}$ increases. Both approaches exhibit a similar downward trend, indicating that conditioned training contributes to performance improvement. Conversely, Ours w/o disentangle. yields the lowest success rate across most categories and all $\theta_{d}$ values due to the inappropriate state(Figure 5(b)), underscoring the criticality of disentangling the door manipulation process for effective universal manipulation policy learning. Notably, Ours w/o mobile. experiences a rapid decline as $\theta_{d}$ slightly increases and achieves a zero success rate in the Car category, suggesting that the absence of a mobile base makes it prone to collisions with doors as shown in Figure 5 (a). Finally, Ours w/o state. exhibits a rapid decline in performance after $\theta_{d}$ reaches $30^{\circ}$, demonstrating that incorporating robot state information can mitigate degradation in point cloud data(Figure 5(c)) as $\theta_{d}$ increases.

We conduct experiments that involve interacting with various real-world doors including Interior, Window, Safe and Storagefurniture as shown in Figure 6. We employ a Franka Emika Panda Robot Arm to perform door manipulation tasks. For input of observation, we obtain real-time point clouds from an Azure Kinect DK depth camera and robot state from the robot arm. The manipulation process in Figure 6 demonstrates that our universal policy can effectively transfer to real-world scenarios. See supplementary for more details.