Object-Aware Impedance Control for Human-Robot Collaborative Task with Online Object Parameter Estimation

We investigate a human-robot collaborative task scenario involving an elongated object in which parameters such as the mass, center of mass, and moment of inertia are not provided in advance.

The dynamics of the object are as follows [9, 18]:

where

$\boldsymbol{M}_{o}=\begin{bmatrix}m_{o}\boldsymbol{I}_{3}&\boldsymbol{0}_{3}\\ \boldsymbol{0}_{3}&\boldsymbol{J}_{0}\end{bmatrix}$, $\boldsymbol{C}_{o}=\begin{bmatrix}-m_{o}\boldsymbol{g}\\ {\boldsymbol{\omega}}_{o}\times\boldsymbol{J}_{o}{\boldsymbol{\omega}}_{o}\end{bmatrix}$

$\boldsymbol{x}_{o}=[\boldsymbol{p}_{o}^{T},\boldsymbol{q}_{o}^{T}]^{T}\in SE(3)$ is the state of the object expressed in the world frame {w}, as illustrated in Fig. 1, and $\boldsymbol{p}_{o},\boldsymbol{q}_{o}$ represent translation and orientation, respectively. Let $\dot{\boldsymbol{x}}_{o}=[\boldsymbol{v}_{o}^{T},{\boldsymbol{\omega}}_{o}^{T}]^{T}$ and $\ddot{\boldsymbol{x}}_{o}=[\dot{\boldsymbol{v}}_{o}^{T},\dot{{\boldsymbol{\omega}}}_{o}^{T}]^{T}$ be twist and acceleration vectors, respectively. $\boldsymbol{F}_{o}=[\boldsymbol{f}_{o}^{T},\boldsymbol{t}_{o}^{T}]^{T}\in\mathbb{R}^{6}$ is the total object wrench, where $\boldsymbol{f}_{o}\in\mathbb{R}^{3}$ is the force and $\boldsymbol{t}_{o}\in\mathbb{R}^{3}$ is the torque acting on the object. $\boldsymbol{M}_{o}\in\mathbb{R}^{6\times 6}$ and $\boldsymbol{C}_{o}\in\mathbb{R}^{6}$ are the inertia matrix and nonlinear matrix containing Coriolis effects and gravity, respectively.

Because human and robot wrenches both act on the object, the relative kinematics can be represented as follows:

where $\boldsymbol{F}_{r}=[\boldsymbol{f}_{r}^{T},\boldsymbol{t}_{r}^{T}]^{T}\in\mathbb{R}^{6}$ and $\boldsymbol{F}_{h}=[\boldsymbol{f}_{h}^{T},\boldsymbol{t}_{h}^{T}]^{T}\in\mathbb{R}^{6}$ are the robot and human wrenches at each grasping point, respectively. $\boldsymbol{G}$ represents the kinematic relations in the robot and human frames with respect to the object frame and is defined as follows:

where ${}^{o}\boldsymbol{p}_{r}\in\mathbb{R}^{3}$ and ${}^{o}\boldsymbol{p}_{h}\in\mathbb{R}^{3}$ are vectors from the object to the robot/human grasp points, respectively. $|{}^{o}\boldsymbol{p}_{r}|_{x}\in\mathbb{R}^{3\times 3}$ denotes the skew matrix, where vector ${}^{o}\boldsymbol{p}_{r}$ represents the cross product. ${}^{o}\boldsymbol{G}_{r}\in\mathbb{R}^{6\times 6}$ and ${}^{o}\boldsymbol{G}_{h}\in\mathbb{R}^{6\times 6}$ are partial grasp matrices from the object to the robot/human grasp points, respectively.

The kinematic relations between the object frame and robot frame can be represented in terms of the velocity:

They can also be represented in terms of the acceleration as follows:

According to (4), $F_{o}$ can be represented by ${}^{r}\boldsymbol{F}_{o}$ as follows:

where

${}^{o}\boldsymbol{G}_{r}=\begin{bmatrix}\boldsymbol{I}_{3}&\boldsymbol{0}_{3}\\ |{}^{o}\boldsymbol{p}_{r}|_{\times}&\boldsymbol{I}_{3}\end{bmatrix}$

${}^{r}\boldsymbol{F}_{o}=\begin{bmatrix}{}^{r}\boldsymbol{f}_{o}\\ {}^{r}\boldsymbol{t}_{o}\end{bmatrix}=\begin{bmatrix}\boldsymbol{f}_{r}+\boldsymbol{f}_{h}\\ \boldsymbol{t}_{r}+{}^{r}\boldsymbol{p}_{h}\times\boldsymbol{f}_{h}+\boldsymbol{t}_{h}\end{bmatrix}$

The variable ${}^{r}\boldsymbol{F}_{o}$ denotes the wrench exerted on the object with respect to the robot frame and can be directly estimated based on the robot’s sensory feedback and measurement capabilities.

By substituting (10) and (12) into (2) and manipulating the result with the parallel axis theorem, the object dynamics (2) can be represented with respect to the robot frame $\{r\}$ as follows:

where ${}^{r}\boldsymbol{J}_{o}$ is the object inertia matrix expressed in the robot frame.

For human-robot collaboration tasks, obtaining human motion and wrench measurements requires that the human participant be equipped with additional sensors for motion tracking or force/torque feedback [9]. However, in most real-world scenarios, it is challenging to use additional sensors, leading to limited available human information. Therefore, we estimated the object parameters in a scenario without human motion information and wrench measurements. In this study, the objective of the parameter estimation process is not to precisely obtain the real parameters of the object. Instead, the aim is to accurately recognize the intended human motion by compensating for the effects of the object dynamics, thereby enabling safe and precise human-robot collaboration. Consequently, in the proposed approach, we focus on estimating the effective object parameters that the robot needs during the collaborative task.

According to (4) and (12), the wrench ${}^{r}F_{o}$ can be represented as follows:

where

${}^{r}\boldsymbol{G}_{h}=\begin{bmatrix}\boldsymbol{I}_{3}&\boldsymbol{0}_{3}\\ |{}^{r}\boldsymbol{p}_{h}|_{\times}&\boldsymbol{I}_{3}\end{bmatrix}$

As previously mentioned, since we do not account for the human wrench during the estimation process, $F_{h}$ is set to zero as follows:

By substituting (19) into (15), the object dynamics affecting the robot can be represented as follows:

where $\boldsymbol{A}(\boldsymbol{a},\dot{\boldsymbol{\omega}},\boldsymbol{\omega},\boldsymbol{g})$ is a data matrix containing the linear and angular accelerations and angular velocities [18]. The parameter vector $\boldsymbol{\theta}\in\mathbb{R}^{n_{\theta}},n_{\theta}=10$ consists of the object’s dynamic parameters and is defined as follows:

where $\boldsymbol{m}_{o}$ is the object mass, ${}^{r}\boldsymbol{p}_{o}$ is the center of mass between the robot and object frames, and ${}^{r}J_{o,vec}$ is the moment of inertia, with ${}^{r}J_{o,vec}=\begin{bmatrix}{}^{r}\boldsymbol{J}_{o,xx}&{}^{r}\boldsymbol{J}_{o,xy}&{}^{r}\boldsymbol{J}_{o,xz}&{}^{r}\boldsymbol{J}_{o,yy}&{}^{r}\boldsymbol{J}_{o,yz}&{}^{r}\boldsymbol{J}_{o,zz}\end{bmatrix}^{T}$.

The extended Kalman filter (EKF) is adopted as an estimator because $\boldsymbol{A}(\boldsymbol{a},\dot{\boldsymbol{\omega}},\boldsymbol{\omega},\boldsymbol{g})$ is time varying. With the filter state $\hat{\boldsymbol{x}}(k)=\hat{\boldsymbol{\phi}}_{eff}$, the prediction of the EKF is formulated as follows:

where $\boldsymbol{Q}\in\mathbb{R}^{n_{\boldsymbol{\theta}}\times n_{\theta}}$ is the process noise covariance matrix. Then, the correction process is formulated with the effective object dynamics as follows:

where $\boldsymbol{R}\in\mathbb{R}^{6\times 6}$ is the measurement noise covariance matrix, $\boldsymbol{P}$ is the estimated error covariance matrix, and $\boldsymbol{H}(k)=\frac{\partial}{\partial\boldsymbol{x}}\boldsymbol{A}(\boldsymbol{a},\dot{\boldsymbol{\omega}},\boldsymbol{\omega},\boldsymbol{g})\hat{\boldsymbol{\phi}}_{eff}|_{\hat{\boldsymbol{x}}(k|k-1)}$ is obtained by backward differentiation. $\bar{\boldsymbol{F}}_{r}(k)$ is directly measured based on the robot sensory feedback, and $\hat{\boldsymbol{\phi}}_{eff}$ is the value estimated according to this filter.

With the estimated effective object dynamics, the real object dynamics can be expressed according to (17) as follows:

Here, $\boldsymbol{F}_{h}$ denotes the intended human motion transmitted to the robot through ${}^{r}\boldsymbol{G}_{h}$. Therefore, the real object dynamics are expressed as a sum of the effective object dynamics and the intended human motion projected by the object geometry.

The dynamics of the nonholonomic mobile manipulator are briefly reviewed [23]. The mobile manipulator consists of a four-wheel differential-drive mobile base and manipulator, as depicted in Fig. 1. The generalized coordinates include 3-degree-of-freedom (DOF) rigid body motion ($\boldsymbol{\zeta}$) due to the presence of a floating base, wheel angular motion ($\boldsymbol{q}_{b}$) and arm joint motion ($\boldsymbol{q}_{m}$) and are written as follows:

where $\boldsymbol{\zeta}=\begin{bmatrix}{x}_{c}\quad{y}_{c}\quad{\phi}\end{bmatrix}^{T}\in\mathbb{R}^{3}$, $\boldsymbol{q}_{b}=\begin{bmatrix}\theta_{r}\quad\theta_{l}\end{bmatrix}^{T}\in\mathbb{R}^{2}$ and $\boldsymbol{q}_{m}\in\mathbb{R}^{n_{m}}$. Note that $n_{b}$ has only two components despite the presence of the four wheels because the left and right pairs of wheels are dependent on each other. As the manipulator used in our system has 7-DOF motion, the total DOF of the generalized coordinates is $n=n_{\boldsymbol{\zeta}}+n_{b}+n_{m}=3+2+7=12$.

The differential-drive mobile base is characterized by two nonholonomic constraints: no lateral slip and pure rolling motion. The constraint matrix $\boldsymbol{A}_{c}(\boldsymbol{q})$ can be represented as follows:

where $\boldsymbol{A}_{c}(\boldsymbol{q})\in\mathbb{R}^{n_{\boldsymbol{\zeta}}\times n}$. The transformation matrix $\boldsymbol{S}(\boldsymbol{q})$ is in the null space of the constraint matrix $\boldsymbol{A}_{c}(\boldsymbol{q})$ [24].

where $\boldsymbol{S}(\boldsymbol{q})\in\mathbb{R}^{n\times(n_{b}\times n_{m})}$ satisfies $\boldsymbol{A}_{c}(\boldsymbol{q})\boldsymbol{S}(\boldsymbol{q})=0$ and $\boldsymbol{\eta}=\begin{bmatrix}\dot{q}^{T}_{b}\quad\dot{q}^{T}_{m}\end{bmatrix}^{T}\in\mathbb{R}^{n_{b}+n_{m}}$ is the reduced velocity vector of the generalized coordinate of the robot. The Jacobian matrix between the Cartesian velocity space and the reduced joint velocity space can be derived as follows:

where $\boldsymbol{J}\in\mathbb{R}^{6\times(n_{b}+n_{m})}$. The nonholonomic constraint can be included in the equation of motion of the mobile manipulator by introducing a Lagrangian multiplier as follows:

where $\tilde{\boldsymbol{\tau}}_{input}\in\mathbb{R}^{n}$ and $\tilde{\boldsymbol{\tau}}_{ext}\in\mathbb{R}^{n}$ are the joint torque input vector and the external disturbance vector, respectively. $\tilde{\boldsymbol{M}}(q)\in\mathbb{R}^{n\times n}$, $\tilde{\boldsymbol{C}}(\boldsymbol{q},\dot{\boldsymbol{q}})\in\mathbb{R}^{n\times n}$, and $\tilde{\boldsymbol{g}}(\boldsymbol{q})\in\mathbb{R}^{n}$ are the inertia matrix, Coriolis and centrifugal matrix, and gravity vector, respectively. $\boldsymbol{\lambda}\in\mathbb{R}^{n_{\boldsymbol{\zeta}}}$ is the Lagrangian multiplier for the nonholonomic constraints.

By substituting (43) and its time derivative into (47) and multiplying by $\boldsymbol{S}^{T}(\boldsymbol{q})$ on both sides of the equation, the nonconstrained equation of motion can be reformulated as follows:

where

$\boldsymbol{M}(\boldsymbol{q})=\boldsymbol{S}(\boldsymbol{q})^{T}\tilde{\boldsymbol{M}}(\boldsymbol{q})\boldsymbol{S}(\boldsymbol{q})$

$\boldsymbol{C}(\boldsymbol{q},\dot{\boldsymbol{q}})=\boldsymbol{S}(\boldsymbol{q})^{T}\begin{bmatrix}\tilde{\boldsymbol{M}}(\boldsymbol{q})\dot{\boldsymbol{S}}(\boldsymbol{q})+\tilde{\boldsymbol{C}}(q,\dot{q})S(q)\end{bmatrix}$

$\boldsymbol{g}(\boldsymbol{q})=\boldsymbol{S}(\boldsymbol{q})^{T}\tilde{\boldsymbol{g}}(\boldsymbol{q})$

$\boldsymbol{\tau}_{input}=\boldsymbol{S}(\boldsymbol{q})^{T}\tilde{\boldsymbol{\tau}}$

$\boldsymbol{\tau}_{ext}=\boldsymbol{S}(\boldsymbol{q})^{T}\tilde{\boldsymbol{\tau}}_{ext}$

Consequently, the 3-DOF rigid body motion is eliminated from the state vector of the reformulated equation, enabling the use of the nonholonomic constraints.

Due to the redundancy of the whole-body robot, quadratic programming (QP) optimization can be introduced to develop a multipriority control framework. The redundancy is due to the difference between the number of joints and the number of Cartesian DOFs, and the Jacobian can be introduced to address this redundancy. The QP optimization can be formulated as follows:

where $\boldsymbol{w_{j}}$ and $\boldsymbol{w_{t}}$ are the weight vectors in the cost function. $\dot{\boldsymbol{\eta}}_{d1}$ and $\boldsymbol{\ddot{x}}_{d1}$ are the joint and Cartesian space desired accelerations introduced in the cost function, and $\dot{\boldsymbol{\eta}}_{d0}$ and $\boldsymbol{\ddot{x}}_{d0}$ are the desired accelerations introduced in the constraints of the QP formulation. $\underline{$\boldsymbol{\eta}$}_{l}$ and $\bar{\boldsymbol{\eta}}_{u}$ are the lower and upper limits of the joint acceleration, respectively.

If the solution of (54) exits, it should satisfy the equality and inequality constraints. On the other hand, the goal of the optimization problem is to find a solution that minimizes the resultant cost of the loss function under the given constraints. Consequently, it is not necessary for each term in the loss function to converge to zero. With this perspective, the tasks designated by the constraints must be satisfied, and they have the highest priority (priority = 0), while the tasks in the loss function have lower priority (priority = 1). In summary, the QP optimization process can be formulated by selecting tasks based on the candidates, such as priority 0 tasks $(\dot{\boldsymbol{\eta}}_{d0},\boldsymbol{\ddot{x}}_{d0}),(\boldsymbol{\eta}_{l},\boldsymbol{\eta}_{u})$ and/or priority 1 tasks ($\dot{\boldsymbol{\eta}}_{d1},\boldsymbol{\ddot{x}}_{d1}$).

In the joint space, the desired acceleration can be obtained through a PD controller as follows:

where $i=0,1$ denotes the priorities. $\boldsymbol{K}_{P\boldsymbol{\eta}}$ and $\boldsymbol{K}_{D\boldsymbol{\eta}}$ are the proportional and derivative gains in joint space, respectively, and $\begin{bmatrix}\boldsymbol{q}^{T}_{b}&\boldsymbol{q}^{T}_{m}\end{bmatrix}^{T}_{d}$ denotes the desired joint angle. Additionally, the desired acceleration in Cartesian space can be obtained as follows:

where $\boldsymbol{K}_{P\boldsymbol{x}}$ and $\boldsymbol{K}_{D\boldsymbol{x}}$ are the proportional and derivative gains in Cartesian space, respectively, and $\boldsymbol{x}_{d},\dot{\boldsymbol{x}}_{d}$ are the desired pose and velocity, respectively. In this paper, instead of (59), we adopt a Cartesian space object-aware impedance controller.

According to the solution shown in (54), $\dot{\boldsymbol{\eta}}^{*}$, the desired input torque for the real robot, can be obtained with the feedback linearization technique as follows:

For a human-robot collaborative task with an object, the external force is determined by the real object dynamics as follows:

By substituting (37) into (49), the equation of motion considering the object dynamics can be represented as follows:

The external forces acting on the robot include the human wrench, which is exerted through the object, and the effective object dynamics that the robot endures under the specific configuration. If there is no object, ${}^{r}\boldsymbol{G}_{h}$ becomes an identity matrix, reducing the external forces to $\boldsymbol{F}_{ext}=^{r}\boldsymbol{G}_{h}\boldsymbol{F}_{h}=\boldsymbol{F}_{h}$, signifying that the intended human motion is applied directly to the robot. On the other hand, in the absence of human-robot collaboration, in which only the robot performs the task, the robot bears all the object dynamics. Hence, (37) becomes $\boldsymbol{F}_{ext}=\boldsymbol{A}(\boldsymbol{a},\dot{\boldsymbol{\omega}},\boldsymbol{\omega},\boldsymbol{g})\hat{\boldsymbol{\phi}}_{eff}=\boldsymbol{A}(\boldsymbol{a},\dot{\boldsymbol{\omega}},\boldsymbol{\omega},\boldsymbol{g})\hat{\boldsymbol{\phi}}_{real}$, representing that the real object dynamics are manifested as external forces.

For human-robot collaborative tasks, the target impedance can be designed as follows:

The crucial point is that the external force in the desired impedance is $\boldsymbol{F}_{h}$ and not ${}^{r}\boldsymbol{G}_{h}\boldsymbol{F}_{h}$. When the human and the robot are cooperatively holding an object, the desired impedance represents the relationship between the robot motion and the intended human motion, excluding the influence of the object. Note that if the stiffness of the desired impedance is set to zero, the robot can actively follow the human operator based on the estimated human motion.

To implement the impedance controller using the QP formulation, the Cartesian space task $\ddot{\boldsymbol{x}}_{d0}$ with priority 0 can be expressed as follows:

The above equation is similar to (59); however, instead of a PD controller, an impedance controller including the desired inertia and external force is employed.

By assigning the impedance controller as the highest priority task and solving the QP problem, the object-aware impedance controller can be represented as follows:

where $\boldsymbol{J}^{+}_{0}$ is the pseudoinverse of $\boldsymbol{J}_{0}$. Without loss of generality, the existence of a feasible solution for the impedance controller can be guaranteed due to the its highest priority. Therefore, the desired input torque can be represented by substituting (74) into (66) as follows:

Alternatively, by gathering the terms related to $F_{h}$, (80) can be represented as follows:

where $\boldsymbol{M}_{F}=(\boldsymbol{M}_{x}\boldsymbol{M}^{-1}_{e}-{{}^{r}\boldsymbol{G}_{h}})$ and $\boldsymbol{M}_{x}=\boldsymbol{J}^{+T}_{0}\boldsymbol{M}\boldsymbol{J}^{+}_{0}$, which is the inertia in Cartesian space [25]. For a general impedance controller, external force feedback can be avoided when inertia shaping is not performed, as expressed by $\boldsymbol{M}_{e}=\boldsymbol{M}_{\boldsymbol{x}}$ [26]. However, in scenarios in which an object is present between the robot and human, avoiding the external forces feedback requires specific inertia shaping with considering object geometry, which is expressed as $\boldsymbol{M}_{e}={}^{r}\boldsymbol{G}^{-1}_{h}\boldsymbol{M}_{x}$, thereby resulting in $\boldsymbol{M}_{F}=0$. Accordingly, the desired impedance is formulated as follows:

where ${}^{r}\boldsymbol{G}^{-1}_{h}={}^{h}\boldsymbol{G}_{r}$. However, understanding the intended human motion, denoted as $\boldsymbol{F}_{h}$, necessitates the use of external force measurements. Consequently, ifurther shaping of the desired inertia is employed to enable efficient interactions between the human and the robot through the object as follows:

where $\boldsymbol{P}_{d}\in\mathbb{R}^{6\times 6}$ is the desired weighting matrix of the external wrenches, which is determined in the subsequent section.

For parameter estimation, the development of estimation methods such as least squares or adaptive algorithms is crucial. However, to enhance accuracy and precision, the design of a probing signal is also important [29]. The object should be perturbed to satisfy the persistent excitation condition, which is achieved by noncolinear angular motion. However, the input perturbation signal should not affect human motion during collaborative tasks to guarantee safety. Pivoted angular motion with respect to the human grasp point enables the object to be perturbed without significantly affecting the human motion.

Similar to (8), the kinematic relation between the robot and hand grasping frames can be represented as follows:

The pivoted perturbation motion can be realized with reduced kinematics considering only translational motion as follows:

where ${}^{h}\boldsymbol{G}^{T}_{r,t}=\begin{bmatrix}\boldsymbol{I}_{3}&|^{h}\boldsymbol{p}_{r}|_{\times}\end{bmatrix}$. The robot end-effector motion in Cartesian space had redundancy in terms of making only translational motion of human grasping frame, thereby allowing the creation of the desired robot motion as follows:

where $\boldsymbol{z}_{id}\in\mathbb{R}^{6}$ is an arbitrary vector projected in the null space of ${}^{h}\boldsymbol{G}^{T}_{r,t}$. By introducing angular perturbation motion to the vector $\boldsymbol{z}_{id}$, collaborative transportation (translational motion) and estimation with object perturbations can be performed simultaneously, with the human grasping point maintained as a pivot [9]. In this paper, the angular motions are chosen as $\boldsymbol{z}_{id}=\begin{bmatrix}\boldsymbol{0}_{1\times 3}&\boldsymbol{\omega}^{T}_{id}\end{bmatrix}^{T}$, where $\boldsymbol{\omega}_{id}=-0.2\cos(2\pi F_{f}t)\times\boldsymbol{I}_{3}$ [rad/s], with $F_{f}=0.4$ [Hz]. The object to be estimated is a 1.5 m long aluminum object with additional attached weights, and the true parameters are represented in Table I. The coordinates representing the center of mass are illustrated in Fig. 1; the y-axes of the human and robot frames are aligned with the longitudinal direction of the object, expressed as ${}^{r}\boldsymbol{p}_{h}=[0,-1.5,0]^{T}[m]$.

In this paper, the object length is assumed to be known before the object is held by the human and robot; well-known artificial intelligence (AI)-based algorithms such as the region-based convolutional neural network (R-CNN) can be used to estimate the object length.

The effective object parameters depend on the human-robot grasping configuration. In this experiment, the human and the robot grasp both ends of the object as rigidly as possible; additionally, the human participant attempts to hold the object as parallel to the robot as possible. Accordingly, the effective parameters of the object are presented in Table I. Additional sensors are not employed to estimate human information, such as the human wrench or motion.

The objective of this paper is to effectively eliminate the object dynamics during collaborative tasks and not to estimate the true object parameters; thus, the moment of inertia of the object is not estimated because this parameter is less important than the mass and center of mass if motion with large acceleration does not occur during the task. Moreover, perturbing the object with large accelerations is undesirable because such motion can potentially affect human safety. In this section, the human and the robot remain static to accurately verify the parameter estimation performance, and in the following section, the estimation is performed during the collaborative transportation task.

The initial values of the estimator are $\hat{\boldsymbol{\theta}}(0)=\boldsymbol{0}_{10}$ and $\boldsymbol{P}(0)=\boldsymbol{I}_{10}$. The weighting matrix of the estimator is set to $\boldsymbol{Q}=10^{-5}\times diag(1.0,0.01,1.0,0.001,0,0,0,0,0,0)$ and $\boldsymbol{R}=diag(200,1000,1000,1000,1000,50)$. The results are represented in Fig. 4 and Table I. Since angular perturbation motion $\boldsymbol{z}_{id}$ was realized in the null space of ${}^{h}\boldsymbol{G}^{T}_{r,t}$, the robot also showed transitional motion. On the other hand, the hand motion, estimated with (95), was well pivoted, as the linear motion remained static, including only some fluctuations, which may be introduced due to the difficulty of humans holding the object consistently during the duration of the estimation. The angular velocities of the human, robot and object are the same, as they are connected rigidly during the task.

For the estimation, most unmodeled bias and disturbance terms are subtracted from the calibration motion based on the measured external wrench, resulting in the calibrated external wrench depicted in Fig. 4. The result of $\hat{m}_{o}$ converges within $t=5[sec]$ to $\hat{m}_{o}=1.131[kg]$, with a relative error and an effective mass of $0.016[kg]$. The results of ${}^{r}\hat{\boldsymbol{p}}_{o}$ converge within $t=8[sec]$ to ${}^{r}\hat{\boldsymbol{p}}_{o}=[8,-7,44]^{T}[mm]$, with a relative error and an effective mass of $[-8,7,-5]^{T}[mm]$. Because the estimation results for the mass and the center of mass both converge within 10 seconds, the required estimation time for our collaborative task scenario is set to 10 seconds.

A scenario was selected to demonstrate the effectiveness of our proposed methods, including the online parameter estimation strategy and the Cartesian impedance controller considering object dynamics. The scenario involves human-robot collaborative transportation and assembly tasks with an unknown object. A jig, as depicted in Fig. 2, is utilized for the object assembly task. The specific steps of the scenario are as follows:

1. 1.

   Object lifting: The human and the robot lift an object together.

2. 2.

   Object parameter estimation during the collaborative transportation task: The human and robot transport the object and approach the jig. During the transportation task, the parameter estimation is performed while inducing an object perturbation signal in the null space of the desired robot motion.

3. 3.

   Continuation of the collaborative transportation task: Once the estimation process is completed, the perturbation signal is no longer applied, regardless of whether the transportation task was completed. In this scenario, the transportation task continues even after the estimation is completed.

4. 4.

   Object dynamics compensation: After the transportation phase, the object dynamics are compensated for to complete the design of the object-aware impedance controller, as represented in (86).

5. 5.

   Following the intended human motion: The human intention following control, also called as follow-me control [5], is implemented by setting the stiffness of the desired impedance, represented in (68), to zero. This enables active tracking of the intended human motion to locate the end of the object on the robot grasping side at the assembly point of the jig.

6. 6.

   Object assembly onto the jig: The human accomplishes the assembly task by securely attaching both ends of the object to the jig. Meanwhile, the robot returns to its initial position.

As reflected in the above scenario, we attempted to minimize the duration of the transportation task by conducting the parameter estimation during this task employing a nonthreatening perturbation signal. Therefore, by incorporating the object dynamics into the impedance controller in real time, the intended human motion can be transmitted to the robot without any additional action during the scenario. Note that the human worker performs the task with one hand holding the object, and their other hand is used to manipulate the joystick for the step transition. Therefore, the human can successfully perform the scenario without additional control PCs or coworkers. The whole scenario is available at https://www.youtube.com/watch?v=bGH6GAFlRgA, where the task was captured in a single continuous take.

With several practice of collaborative task, it was observed that transmitting intended human motion to the robot through the object becomes more challenging as the longer the object. The results showed that the use of rotational wrench at the robot side is more efficient to understand the translational human motion. To incorporate this observation into the weighting matrix of the external wrenches, $\boldsymbol{M}_{\boldsymbol{F}}$, presented in (86), we defined $\boldsymbol{P}_{d}$ as follows:

With the coordinate system illustrated in Fig. 1, the translational motion of the robot along the +y and +z directions can be adjusted based on the rotational motion of the human along the -z and -y axes. Moreover, the robot’s +x transitional motion can be adjusted based on the +x translation motion of the human due to the alignment of the x-axis of the robot’s end effector, the human grasping point and the object.

The overall results of the scenario are presented in Fig. 5. The object parameter estimation results during the nonstationary transportation task are $mass=1.154[kg]$ and ${}^{r}\boldsymbol{p}_{h}=[-0.026,-0.03,0.057]^{T}(m)$. Comparatively, the results exhibited larger errors than those obtained in the stationary task. The bias in the results may be influenced by the human’s inability to maintain the alignment between the robot’s grasping position and the object during the transportation phase; especially, the negative bias of x and y-axes center of gravity indicates that human participant tends to walk rapidly and more raise the object than the robot based on the coordinate system in Fig. 1. Nevertheless, the estimation errors are acceptable as the robot could maintain its posture without tilting downward during human intention following control.

During the transportation phase, an impedance controller with nonzero stiffness and damping, $\boldsymbol{K}_{e}=diag[100,100,100,400,400,600]^{T}$, $\boldsymbol{C}_{e}=0.5\times(2\sqrt{\boldsymbol{K}_{e}})$, was employed to move the robot’s end effector from the initial position, $\boldsymbol{p}_{initial}=[0,0,0]^{T}$, $\boldsymbol{q}_{initial}=[0,0,0,1]^{T}$, to the final position, $\boldsymbol{p}_{final}=[0,1.5,0]^{T}$, $\boldsymbol{q}_{final}=[0,0,0,1]^{T}$, by tracking the desired trajectory. In the whole-body controller defined in (54), high-priority tasks (priority 0) include the trajectory tracking control in the Cartesian space and limiting the input torques, while low-priority tasks (priority 1) include maintaining the initial robot arm configuration. Notably, in the tracking control task, the emphasis lies that the human is allowed to track the desired trajectory, whereas the end effector does not exactly follow the trajectory due to the presence of the identification input. After the transportation phase is completed, the input force and torques were modulated due to the feedforward incorporation of the object dynamics, as shown in Fig. 5.

To attach the object to the jig, precise alignment is necessary. To achieve this, the intended human motion following the control strategy was executed by configuring the highest priority task of the whole-body controller to maintain the initial Cartesian position using an impedance controller with zero translational stiffness. The priority 1 task attempts to maintain the initial configuration of the robot arm, as in the previous experiment. The results show that the external wrench $\boldsymbol{F}_{ext}$ does not significantly increase during the object alignment, indicating that the robot successfully follows the intended human motion. Moreover, the x-, y-, and z-axes of the object position are adjusted by the x-axis of the external force and the z- and y-axes of the external torques, as represented in (99). Note that some vibrations appear in $\boldsymbol{F}_{ext}$ during the transportation phase due to the motion of the mobile base, and these vibrations can be eliminated by tuning the velocity controller.

Consequently, the proposed human-robot collaborative method can realize transportation and assembly tasks in a single operation.