Binary Neural Network in Robotic Manipulation: Flexible Object Manipulation for Humanoid Robot Using Partially Binarized Auto-Encoder on FPGA

End-to-end learning of robotic manipulation is composed of three phases: (1) data collection, (2) model training, and (3) model deployment on a robot.

To reduce the model size, [5] has proposed binarization of synaptic parameters and activations, i.e., a real-valued weight or activation is approximated by either $+1$ or $-1$. The real-valued input tensor is binarized using the following activation function:

$\displaystyle x^{b}=Sign(x)=\begin{cases}+1&x\geq 0,\\ -1&otherwise.\end{cases}$

(1)

However, since the derivative of the sign function is almost zero everywhere, naive binarization leads to poor network performance. To alleviate this, [5] proposed a technique called “straight-through estimator,” where the derivative of the sign function is substituted by that of hard hyperbolic tangent, i.e.,

$\displaystyle\frac{\partial}{\partial x}Sign(x)$

$\displaystyle\approx\begin{cases}1&\left|x\right|<1,\\ 0&otherwise.\end{cases}$

(2)

By constraining the weights and activations to be binary, we can (1) dramatically reduce model size, e.g., considering float32 as baseline, binarization achieves 1/32 model size reduction, and (2) multiply-and-accumulate (MAC) operations can be realized by using only XNOR gates and 1’s counters.

Even with “straight-through estimator,” training of BNN is unstable and gradients tend to explode frequently. Hence, BNN usually comes with batch normalization (BN) layer which basically standardizes activations, i.e., it re-scales and re-centers input activations so that their means and variances become zero and one, respectively[10]. Let $x$ is the input to the BN layer, it computes:

$\displaystyle y$

$\displaystyle=\gamma\frac{x-\mu}{\sqrt{\sigma^{2}+\epsilon}},$

(3)

where $y$ is the output of the BN layer, $\gamma$ and $\beta$ are trainable parameters, $\mu$ and $\sigma^{2}$ are the empirical mean and variances of mini-batch, and $\epsilon$ is a small constant for numerical stabilization.

One may argue that adding BN requires additional hardware resource since Eq. (3) includes a division and multiplication. However, remembering that BNN requires only the sign of activations, the BN layer at inference time can be realized simply by shifting bias term:

$\displaystyle Sign\left\{BN(x)\right\}$

$\displaystyle=\begin{cases}+1&x\geq\lfloor\mu-\frac{\sigma}{\gamma}\beta\rfloor,\\ -1&otherwise,\end{cases}$

(4)

where $\lfloor\cdot\rfloor$ is the floor function.

Owing to the straight-through estimator and BN layers, BNNs can achieve comparable performance with the full-precision counterpart. However, we find that naive replacement of DCAE with BNNs results in corruption of extracted image features, which motivates us to develop “partiall” binarization technique.

Fig. 1 shows the overall system, which is composed of two neural networks: a PB-DCAE to convert raw images into low-dimensional feature vectors and a long-short term memory (LSTM) to generate joint angles of robot arms. Since the LSTM accounts for only 1.2% of whole model size, we leave the LSTM unquantized to maintain its performance. Zynq SoC platform is selected to cope with heterogeneous arithmetic precisions, i.e., the PB-DCAE requires binary operations whereas the LSTM requires floating point operations, and the PB-DCAE and the LSTM are implemented onto a programmable-logic (PL) and processing-system (PS) having an ARM processor, respectively. Our model is firstly trained by using GPU. Then, the parameters of PB-DCAE are converted to C++ header file and converted to “.bit” file via Vivado toolchain to configure the PL. The LSTM is re-implemented by using Numpy from scratch so that it can be efficiently executed on the resource limited ARM processor.

For the ease of stabilization of training, we employed a two-stage training, where the PB-DCAE is trained firstly to yield well self-organized image features, followed by training of LSTM.

PB-DCAE training: PB-DCAE is trained to reconstruct input images provided to the input of the encoder. Since PB-DCAE has a hourglass-shaped structure, i.e., middle layers have fewer neurons than input or output layers, the network is forced to “reconstruct” images by using only degenerated information. Hence, the encoder part tries to find better compressive expressions of input images for the decoder to successfully recover original images. After the training, the decoder part is removed and the parameters of the encoder are frozen.

LSTM training: The image features extracted by the trained PB-DCAE is concatenated with the joint angles and the gripper command to form a visuo-motor sequence, $\bm{X}=(\bm{x}_{1},\bm{x}_{2},\cdots,\bm{x}_{N})$. Here, $\bm{x}_{t}$ is a vector containing the extracted image feature and the joint angles and the gripper command at time step $t$. $N$ is the sequence length. At each time step, the LSTM takes the input $\bm{x}_{t}$ and the current state $\bm{h}_{t}$, and outputs $\bm{y}_{t+1}$ and the updated state $\bm{h}_{t+1}$. Let $f_{\mathrm{LSTM}}(\cdot,\cdot)$ be the mathematical function representing the LSTM’s behaviour, the relationship between inputs and outputs can be written as: $\left\{\bm{y}_{t+1},\bm{h}_{t+1}\right\}=f_{\mathrm{LSTM}}(\bm{x}_{t},\bm{h}_{t})$. By recursively feeding $\left\{\bm{y}_{t+1},\bm{h}_{t+1}\right\}$ to the LSTM, it can generate the sequence: ${\bm{Y}}=(\bm{y}_{1},\bm{y}_{2},\cdots,\bm{y}_{t})$. In our application, we want to predict visuo-motor sequence and hence the training objective is to minimize mean-square-loss between $\bm{X}$ and $\bm{Y}$. Unfortunately, however, this “multi-step” prediction task is difficult to train and loss tend to diverge frequently especially for the longer sequences. Hence, to stabilize the training, we propose to exploit single-step prediction loss in combinaiton with multi-step prediction loss. Our training setup is shown in Fig. 2.

1) Multi-step prediction loss: We firstly compute “multi-step” prediction loss by recursively feeding LSTM’s outputs to its inputs as shown in the lower part of Fig. 2. Let $\bm{Y}^{\mathrm{m}}=\left(\bm{y}_{1}^{\mathrm{m}},\bm{y}_{2}^{\mathrm{m}},\cdots,\bm{y}_{t}^{\mathrm{m}}\right)$ be the generated sequence. Then, the multi-step prediction loss $L_{m}$ is given by:

$\displaystyle L_{m}$

$\displaystyle=\frac{1}{2N}\sum_{t=1}^{N}(\bm{x}_{t}-\bm{y}_{t}^{m})^{2}.$

(5)

2) Single-step prediction loss: Unlike the multi-step prediction setup, the LSTM takes “teacher” visuo-motor sequence at each time step, i.e., given an element of visuo-motor sequence $\bm{x}_{t}$ and the internal state $\bm{h}_{t}^{s}$, it computes $\left\{\bm{y}_{t+1}^{s},\bm{h}_{t+1}^{s}\right\}=f(\bm{y}_{t}^{s},\bm{h}_{t}^{s})$. Then, by recursively feeding only the internal state $\bm{h}_{t+1}^{s}$, the next output $\bm{y}_{t+2}^{s}$ is computed as shown in the upper part of Fig. 2: $\left\{\bm{y}_{t+2}^{s},\bm{h}_{t+2}^{s}\right\}=f_{\mathrm{LSTM}}(\bm{x}_{t+1},\bm{h}_{t+1}^{s})$. Repeating this procedure, we have a sequence of single-step ahead prediction of visuo-motor sequence: $\bm{Y}^{s}=(\bm{y}_{1}^{s},\bm{y}_{2}^{s},\cdots,\bm{y}_{t}^{s})$. Then, the single-step prediction loss $L_{s}$ is given by:

$\displaystyle L_{s}$

$\displaystyle=\frac{1}{2N}\sum_{t=1}^{N}(\bm{x}_{t}-\bm{y}_{t}^{s})^{2}.$

(6)

3) Internal state loss: Since internal states have no teacher signal, the small error may accumulate and cause gradient explosion when making multi-step prediction. Hence, we further introduce an additional loss $L_{h}$ to make close the internal states for single-step prediction $\bm{h}_{t}^{s}$ and those for multi-step prediction $\bm{h}_{t}^{m}$:

$\displaystyle L_{h}$

$\displaystyle=\frac{1}{2N}\sum_{t=1}^{N}(\bm{h}_{t}^{s}-\bm{h}_{t}^{m})^{2}.$

(7)

The final loss function is given by linear combination of the three losses as follows:

$$L=\alpha L_{m}+\beta L_{s}+\gamma L_{h},$$

(8)

where $\alpha$, $\beta$, and $\gamma$ are hyper-parameters.

The proposed model is implemented on Zynq UltraScale+ MPSoC ZCU102 board. The on-chip RAM and LUT resources of this FPGA board are very limited, so it is necessary to implement a reduced hardware resource usage while maintaining the inference throughput. The overall design flow is shown in Fig. 3.

First, PB-DCAE is trained using PyTorch framework and the network parameters are extracted. Next, a C++ model of the proposed PB-DCAE is created and synthesized using two Xilinx Vivado software to yield an executable “.bit” file containing the network topology to properly configure the FPGA board. Finally, download all the generated files to a Xilinx ZCU102 FPGA board and test/measure the performance. PYNQ (PYthon Productivity of zynQ) environment is installed onto ZCU102 board so that we can write programs and view the results via Jupyter Notebook. Note again that The image feature extraction module is implemented on PL whereas the LSTM-based motion generation module is implemented on PS. Since the DCAE-based image feature extraction module accounts for 99.94% of total multiplications required for single step inference the performance of the entire system can be greatly improved even when the LSTM remains on the PS-side.

FPGA implementation: Fig. 4 shows the block diagram of PB-DCAE implemented on FPGA. A dedicated computation unit is assigned for each layer to minimize data movements, and each layer is connected via AXI-Stream interface. To process the data stream efficiently with a small hardware resources, shift registers are prepared to hold only the data necessary for the computation and perform convolution. Binarization allows floating-point multiply-and-accumulate operations to be replaced by XNOR and bit counting, which can be implemented in small circuits. In addition, although BN requires multipliers and adders, it can be converted to integer comparisons by transforming the expression together with the Sign function that follows, thus reducing the circuit size and implementation. The model size can be reduced by binarization. Since the model size is compressed by binarization, all the parameters can be expanded to the on-chip BRAM. Therefore, feature maps and binarization weights can be stored in on-chip memory for PB-DCAE computation. Since no external I/O is involved, latency due to communication can be minimized and the system can be executed with low power consumption.

The dataset for the experiments was collected by using Nextage Open developed by Kawada Robotics. It has two arms with grippers and a camera mounted on robot’s head. Each arm has six degrees of freedom (DoF). To collect the training data, a robot teleoperating environment is deveoped, where a human operator can control the gripper positions by using 3D mouse. Using this environment, the operator is requested to complete a cloth-folding task, during which camera images (142$\times$142$\times$3=60,492 dimensions, RGB), joint angles of both arms (6$\times$2=12 dimensions), and a gripper command (1 dimension) are recorded. Note here that since we only used right gripper to pick up the cloth, we only recorded the gripper command attached at the right arm. A total of 75 task sequences were collected, 50 of which were used for training and 25 for evaluation. As a result, approximately 22k and 11k steps of data are used to train and to validate the model, respectively. $\alpha$, $\beta$, and $\gamma$ in Eq. (8) are set to be 0.1, 1.0, and 0.1, respectively. The model architecture is summarized in Tab. I.

To validate the impact of partial binarization, we firstly compare the quality of reconstructed image using three different DCAEs, i.e., a full-precision DCAE, a fully-binarized DCAE, and a PB-DCAE, as shown in Fig. 5. The left most image is the raw input image which is captured when the robot moves its grippers to pick the towel placed at the center. The remaining three images are those reconstructed by three different DCAEs.

Observing Fig. 5, we see no noticeable difference between images reconstructed by the full-precision DCAE and the PB-DCAE. However, the fully-binarized DCAE outputs the corrupted image. To quantitatively compare the quality of reconstructed images, peak signal-to-noise ratios (PSNRs), one of the common method to measure the image quality of lossy compression codecs, are calculated. PSNR is expressed by

$\displaystyle PSNR=10\log_{10}{\frac{MAX_{I}^{2}}{MSE}}.$

(9)

Higher PSNR means higher image quality and, according to [13], PSNR of over 20 dB is sufficient quality for practical applications. As shown in Fig. 5, the average PSNR for full-precision DCAE was 21.85 dB while that for fully-binarized DCAE was 7.164 dB, which again demonstrates that naive binarization leads to corruption of reconstructed images. We also confirmed that PSNR for PB-DCAE was 20.78 dB which is comparable to that of the full-precision DCAE.

We then examine the impact of binarization on the prediction accuracy of the joint angles and the gripper command. For this purpose, we compare the multi-step prediction accuracy of the LSTM, i.e., given only image features, joint angles, a gripper command at time step $t=1$, the LSTM is requested to predict the whole remaining sequence for $t=2,3,\cdots,N$ by recursively feeding the predicted output into the LSTM. The reconstruction errors calculated by the mean square error between the original and the predicted sequences are 2.98^∘, 6.32^∘, and 3.43^∘ each corresponds to the image features extracted by full-precision DCAE, fully-binarized DCAE, and PB-DCAE, respectively. We again confirmed that the PB-DCAE achieved comparable performance with the full-precision DCAE although the model size was dramatically reduced from 51.3MB to 2.20MB, as shown Tab. I.

We finally compare the energy efficiency of the proposed method with the conventional method relying on a GPU-based platform. The power consumptions of Zynq ZCU102 board, CPU, and GPU are measured by using Maxim powertool, “s-tui” command, and “nvidia-smi” command, respectively. Tab. II summarizes the processing time, power consumption, and energy efficiency for CPU, GPU, and Zynq platforms. Studying Tab. II, we can confirm that the Zynq platform achieved the highest energy efficiency, which corresponds to 10$\times$ and 3.7$\times$ improvements compared to CPU and GPU platform, respectively, while satisfying the processing speed above 30 FPS which is the upper bound determined by the frame rate of a commercial RGB camera.