ABatRe-Sim: A Comprehensive Framework for Automated Battery Recycling Simulation

Despite the growing interest in robotic battery recycling, a suitable model integrated in robotic operating system (ROS) is still lacking. The CAD drawings of battery packs are considered proprietary information by the EV companies and are generally not available to the research community. The complexity and technical details make it challenging to create such drawings. Furthermore, importing such complex models into ROS-based simulation environment presents a challenge, as the complexity of these models can lead to unexpected behaviors in the physics engine of simulators such as Gazebo [8]. While some sorting tasks have been simulated in the Gazebo environment, they typically involve objects that are either lying flat on the ground [4] or imported as a single piece [5]. To our knowledge, a simulation platform involving stacked or assembled battery objects is largely unavailable.

To address the aforementioned challenges, a detailed battery model is created in SolidWorks with major components necessary for robotic disassembly. As shown in Fig. 2, the silver blocks represent battery modules, connected by high voltage cables in orange. The switch on the right side (orange) is Manual Service Disconnect (MSD), which, when unplugged, breaks down the pack voltage by half. The orange box on the left side is the contactor box, which contains high-voltage relays (contactors). The two orange lines on the left-hand sides are the positive bus bar and the negative bus bar, respectively. The red box is the BMS controller which is the “brain” of the battery pack (marked red only for demonstration purpose). The components mentioned above account for the major structural parts of the battery pack. Since adding more components of the same category only involves repetitive work, we designed a generic EV battery pack with four modules in Gazebo simulation, as shown in Fig. 1. This battery pack also includes the battery base, two cables, positive and negative bus bars, one contactor, one MSD, and multiple bolts.

In this framework, we will demonstrate the detection and disassembly of three main classes: bolts, cables, and modules. The shape is imported through model editors. Its physics properties are further hand-modified in XML scripts to reduce the processing workload. Each component is set to static in its XML script, but will only be replaced with a movable version prior to picking due to the fact that fewer interacting parts will reduce the physics engine processing load. For example, movable bolts will be spawned prior to the unbolting action while static bolts will be simultaneously deleted. The same approach is applied to cables and modules. In this way, a seamless transition is achieved without noticeable visual differences. We use the built-in physics engine in Gazebo to simulate the object interactions. Specifically, we exploit the grasp plugin libgazebo_grasp_fix.so [8] to establish a connection between two components when the gripper approaches a particular position, and the object can be dropped from the gripper when it is open.

We have also taken into account scenarios where the battery pack is received damaged or in an imperfect condition, which could be due to internal failure, transportation damage, or aging. To address these imperfections, we have created image filters that are available to simulate such conditions. Fig. 3 illustrates four simulated conditions that could occur when a pack is received at the disassembly station. Image A depicts a dent or deformation that could occur after the pack is pressed by other objects, resulting in convex shapes on the right edges of the two packs on the right. Image B simulates the case of an old improperly maintained pack where coolant leaked out, leaving behind corrosion and green coolant residuals. Image C simulates a long-sitting pack where a light layer of dust is present on top. Image D simulates the result of careless disassembly, where the pack is left with multiple marks on the metal and paint surfaces. These scenarios account for the possibility of a poorly maintained or transported battery pack. We plan to continue adding more scenarios to improve the capability of simulating real-world situations.

While a number of robot arms can be used for this application, we use Universal Robot 10, a 6-axis robotic arm, as the default in the simulation platform. Note that the user can choose other robotic arms in the simulation (see Section IV for more details). In ROS, robots are described using Unified Robot Description Format (URDF) files, which describe the robot’s joint structures and kinematic relations. The files can include additional details such as the visual and collision properties of links, joints, sensors, etc. XACRO files (XML Macros) are extensions of URDF that allow for smarter and more modular descriptions of robots. Mesh files provide 3D models for the visual representation of robots in simulations. These files are often in STL or Collada format. The package for the robot is referenced from the Universal Robot Github Page [9]. Moreover, as shown in Fig. 4, two types of grippers are incorporated with the robot arm to facilitate the battery disassembly. Specifically, a Robotiq gripper [10] is utilized for the first two operations, which involve unscrewing the bolts and removing them as well as lifting the cables. Following the removal of these parts, a vacuum gripper is employed to pick up the modules. The vacuum gripper is driven by the libgazebo_ros_vacuum_gripper.so, which essentially creates linkages between objects when a service is called. The animation resembles that of a vacuum gripper. The robot arm and gripper are connected by creating an additional link in-between. The robot and gripper files are open-source ROS packages on GitHub [9, 10]. Inspirations from other researchers’ integration with Universal Robots  [11, 12] also contribute to this project.

Furthermore, a sensor is needed to understand the scene, i.e., identifying and localizing the position of the objects. To that end, we adopt Kinetic RGB-D camera as the sensor modality, which is placed on top of the battery pack, as depicted by the green object in Fig.1. In Gazebo, the Kinect has a ready plugin that renders color and depth images through ROS topics. In our node, the color image is streamed into the object detection node for inferencing different battery parts (see Section III-A) and the depth image is then used to generate the target positions to guide the manipulations.

Detecting various parts of the battery pack is a key step to perform disassembly. Deep learning-based techniques have been widely used for object detection and have received significant attention in recent years. There are numerous machine learning frameworks such as PyTorch and TensorFlow, and there are also open-source implementations that use pre-trained models for common objects with webcams. However, it should be noted that although there exist many Gazebo simulation integrations for object detection, fewer of them use more recent frameworks like TensorFlow 2 and recent learning algorithms like YOLOv5.

For the object detection task, we utilize TensorFlow 2.11.0 to deploy the learning algorithm. The model architecture is constructed with SSD ResNet50 V1 FPN 640x640[13], which is selected from the open-source TensorFlow model zoo. The model structure is shown in Fig. 6. Different from SSD [14] that uses VGG16 as its backbone, the SSD ResNet FPN uses ResNet50 as the backbone and provides additional convolutional layers for classification [15]. ResNet50 is a deeper backbone than VGG16 as it has 50 layers and it is introduced to address the problem of vanishing gradients in deep neural network [16]. The architecture also utilizes Feature Pyramid Network (FPN) for better feature extraction. FPN is a type of multi-scale feature extraction method that enhances the detection of objects of varying scales and sizes [17].

The task of image labeling is known to be a laborious process, requiring significant investments in time and human resources. Our research has determined that a dataset consisting of 120 labeled images is sufficient for our model to detect all three classes in a simulated environment, which exhibits lower complexity than real-world images. For the effective deployment of this approach in real-world applications, a larger dataset, and a corresponding increase in the volume of labeled images, will likely be needed. To account for different conditions in the real world, data augmentation techniques [18] is applied to extend the dataset, enabling tests under a broader range of conditions. Specifically, a data augmentation script is included in the simulation framework to characterize more variations in the data and improve the robustness of the models trained on the dataset. The tool can enlarge the dataset with random brightness, contrast, crop portion, flip orientation, Gaussian noise level, and rotation position. A sample of the enlarged dataset, consisting of eight distinct pictures, is shown in Fig. 7.

The model training is performed on a separate computer with Ubuntu 20.04LTS operating system, an Intel Core i9-9820x CPU, and two NVIDIA GeForce RTX 2080Ti GPUs. In our training process, we specified the number of classes to be 3, the batch size to be 4, the warm-up steps to be 2000, and the training steps to be 25000. The entire training process took 1 hour 15 minutes and 9 seconds. Once the training is complete, the model is exported as a benchmark model for object detection purposes. The following are the final training metrics: classification loss of 0.03157, localization loss of 0.01083, regulation loss of 0.01415, and total loss of 0.05655. The exported model can then be employed for inferencing objects of interest. The inference node and Gazebo simulation were experimented on a Dell Precision PC with an Intel i7-11700 CPU and no dedicated GPU, and we only experience a small amount of latency.

The streaming of Kinetic camera images, object detection inference, and the communication of the process information (stage flags, coordinates) are all integrated into a single Python script called Vision Algorithm. The pseudo code for this module is shown in Algorithm 1, which processes the image and generates the bounding boxes with corresponding pixel coordinates. The pixel coordinates are transformed into the coordinates under the world frame, which are then passed to the planning and control module.

The planning and control algorithm of robotic motion is designed using the components from the open-source ROS package, MoveIt!, and can be further modified by users with the functions provided in the demo. MoveIt! is a comprehensive software framework for robotic motion planning and manipulation. Once a robot is set up and initialized in the MoveIt! configuration, users can conveniently finish the task of planning and control for the robot through the API provided. By employing the Planning Scene Interface, users can add the collision objects in the scene to achieve collision-free planning. The arm (Arm group) and the gripper (Gripper group) are powered by MoveGroup, which is a component of MoveIt! that provides interfaces for controlling motion planning and execution. In our demo, we exploit the Rapidly-exploring Random Tree (RRT) [19] algorithm for the planning of the robot as a C++ node in ROS, which makes it convenient for users to edit their own planning and control algorithms if necessary. With the Kinematics and Dynamics Library, we obtain a group of joint angles given a desired pose in the Cartesian workspace. Then we iteratively take samples in the joint configuration space and use the functions that MoveIt! provides to check the validity of the sampled states. As the iteration progresses, a collision-free joint trajectory is then generated. When every point in the joint configuration space has the same probability of being sampled, this algorithm can be guaranteed successful if there is a feasible path. Taking some biased sampling strategy can accelerate the progress. Having obtained a feasible joint trajectory, we then use the Iterative Spline Parameterization algorithm from MoveIt! to interpolate a timed trajectory that is executable for the robot.

The planning algorithm discussed above is outlined in Algorithm 3. To briefly explain, Lines 4-11 initializes the algorithm. Line 5 obtains one group of joint configuration $q_{t}$ which complies with the given target pose $p_{t}$, and the validity of $q_{t}$ is checked. Then a tree structure $\{V,\,E\}$ is initialized, where $V$ is the set of nodes, and $E$ is the set of edges. One node $v$ has two components, one being the joint configuration $q$, and the other being the distance to the root of the tree, i.e., $v_{0}$. In the while-loop, we first take the biased sampling, which has a probability of 0.2 for choosing $q_{t}$ as a result and 0.8 for uniform sampling. Then with the sampled $q_{s}$, the nearest node $q_{nearest}\in V$ is found. In Line 15, $Steer$ function is used to replace $q_{s}$ with $q_{new}$ if $q_{s}$ is too far away from or too close to $q_{nearest}$, in order to efficiently explore the joint configuration space and find a feasible trajectory. After verifying the validity of the edge $(v_{nearest},v_{new})$, $v_{new}$ is added to the tree $\{V,\,E\}$. In Line 22-23, the neighboring nodes of $v_{new}$ is checked to see whether those nodes are closer to $v_{0}$, if their ”father” node is changed to $v_{new}$. If $q_{t}$ is added to the tree, a valid trajectory is then obtained by back-propagation in the tree $\{V,\,E\}$ from $v_{t}$.

In our simulator, high-level control actions, such as unscrewing bolts, removing cables, and lifting modules, are executed based on different stage flags as shown in Algorithm 2, while the C++ node handles low-level trajectory generation and inverse kinematics. In order to initiate the simulation, it is crucial to launch all necessary functions, such as Gazebo world, frame transformation, vision, and control in ROS nodes, and establish communication through ROS topics. For the disassembling task, the first step is to unscrew the bolts. The detected bolts are appended to a list for preparation in the perception node. The first item in the list has its coordinates sent to the frame transformation node, which converts the $u,v$ coordinate frame to the camera frame and then to the world frame for the robot to pick up. Given the corresponding pose of targets, a collision-free trajectory for the robot is generated and executed by the C++ node. After picking the bolt, the simulation adds a twisting motion to simulate unscrewing. Once all bolts in the first class are removed, the simulation proceeds to the second stage – cable picking. With the bolts no longer locking down the cables, they are then ready to be picked. The whole workflow is demonstrated in Fig. 9. Specifically, at the beginning of the simulation, A1 displays the first object that is targeted for disassembly (dark blue circle indicates the desired location to place the end effector of the robot), while A2 shows the arm moving to the location to pick it up. B1 shows the real-time disassembling process, where the perception node selects the next bolt while B2 shows the robot executing the task. In C1, it is displayed that there is only one bolt left, and C2 shows a zoomed-in view of the gripper for picking. In D1, the transition between the first and second stages is shown as dark blue circle moving from the last bolt to the first cable whereas in D2 the gripper goes on to pick up the cables. After completing stage two, the modules are left to be removed. Since the modules require a vacuum gripper to pick them up, and the URDF does not support the dynamic changing of grippers, we switch the gripper by relaunching the corresponding URDF file and loading the vacuum gripper plugin. In E1, the perception node selects the first module to pick, while E2 shows the robot successfully picking this object. Finally, all the modules are removed. The objects MSD, positive bus bar, and negative bus are not implemented in this simulation, as the process for disassembling them follows the same concepts as for the bolts and cables. The outcomes of the benchmark algorithm are presented in Table I. Notably, all the assigned tasks were performed successfully. However, it is worth-noting that the analysis also reveals a prolonged execution time for bolt disassembly, as generating a viable path took several attempts in some cases, highlighting a needed area for further enhancements over our C++ planner. While the current object detection algorithm is able to accurately detect all objects and their locations, the detection score is relatively low compared to that of larger objects. There is still room for improvement in detecting smaller objects with higher accuracy, which is a known challenge in computer vision. Nonetheless, the framework offers a comprehensive solution to automated battery recycling, and meanwhile there is certainly potential for further improvements and extensions, which will be discussed next.