EV-Planner: Energy-Efficient Robot Navigation via Event-Based Physics-Guided Neuromorphic Planner

The network architecture used for filtering objects based on their motion consists of a single spike-based neural layer, whose parameters are fine-tuned to directly represent the speed of the moving object. The LIF neuron model [8] has a membrane potential ($U[t]$) and a leak factor ($\beta$). At first, the initial membrane potential ($U[t_{0}]$) and the threshold value ($U_{th}$) get initialized. Next, according to equation [1] for each time step ($t$) the weighted sum of the inputs ($WX[t]$) gets stored in $U[t]$. Finally, if at any instance of time, the stored $U[t]$ exceeds $U_{th}$ then the neuron produces an output spike and resets $U[t]$.

In our approach hyper-parameters like $\beta$ and $U_{th}$ are fine-tuned to select nearby input spikes during any given instance of time. Figure 2 demonstrates the difference in the output spikes generated from the neuron according to the frequency of the input events. As shown, the frequency of the generated events is directly proportional to the speed of the moving object. Exploiting this property of spiking neurons and event data, we consequently design an SNN capable of filtering out a particular object of interest, based on speed.

To detect an object, the initial step involves connecting the output pixels from the event camera to the spiking neurons. Then, we apply a weighted sum ($W$) from a neighborhood of pixels and pass that as input to the spiking neuron. Here, the neurons connected to the specific neighborhood will register more than one input pixel every consecutive time step. This enables the SNN with the ability to identify pixels as fast-moving. In the next step, we locate the minimum ($X_{min},Y_{min}$) and maximum ($X_{max},Y_{max}$) pixel values in two dimensions and subsequently draw the bounding box. Finally, we obtain the center coordinates of the moving object using Eqn. (2).

Figure 3 depicts our network architecture. We selected a reasonable weighted neighborhood size ($W$) (see Section IV) as input to every spiking neuron. The weights for the neighborhood were normalized to obtain a summation value close to one. Note that the SNN mentioned above detects the moving object. But we still need to isolate the input events which represent the object of interest. Therefore, after detection, we recover the input events around the neighborhood of the spiking neuron output. Hence, the SNN works asynchronously to find the output spikes and recover the inputs from the previous time step (one-time step delay). Generally, after separating the objects based on the movement speed, their corresponding events are spatially grouped together using some clustering technique for predicting the object class. However, in this work, we only consider a single moving object. Hence, our current SNN-based design does not require any spatial clustering technique, which reduces the latency and energy overheads of our object detection and tracking. Please note that the SNN is scene-independent and can be deployed in various scenarios with minimal fine-tuning. Figure 4 illustrates the output of the event-based object (gate) tracking using SNN, situated at different depths (see Section V-A for more details).

We model the actuation energy of the drone in flight by considering an electrical model of the LiPo battery-powered brushless DC (BLDC) motors of the rotor propellers [38]. Figure 5 represents the equivalent circuit of each BLDC motor by considering the resistive and inductive windings. The energy for a particular flight path requires overcoming the motor and load frictions (given by the coefficients $J_{m}$ and $J_{L}$). The instantaneous motor current is:

$K_{T}$ is the torque constant. $T_{f}$ and $T_{L}\;\omega(t)$ are the motor and load friction torques respectively. $D_{f}$ represents the viscous damping coefficient. For the BLDC motors of drone propellers, we can safely neglect $T_{f}$ and $D_{f}$.

$n_{B}$ is the number of propeller blades, $m_{B}$ is the mass of the blade and $r$ is the propeller radius. $\epsilon$ is the clearance between the blade and motor.

The motor voltage is as follows:

$R$ and $L$ denote the resistive and inductive impedances of the phase windings respectively. $K_{E}$ represents the motor voltage constant. We also neglect the electronic speed controller (ESC) voltage drops and any non-idealities present in the LiPo battery. For small drones (which we assume in this work), the propellers have direct connections with the shafts. At steady-state operation, the motor voltage becomes:

For flight from time $0$ to $t$, then the total actuation energy of the flight path for all practical purposes can be written as:

Using equations (3) and (6), the energy expression becomes:

where,

Using the expression given in equation (8), and plugging in the constant values given in equations (9), we are thus able to calculate the energy for a particular trajectory up to time $t$. Given a certain distance away from a target destination (depth), an energy-efficient planner should be able to approximately estimate the flight time that would minimize the energy. We utilize the knowledge obtained through the aforementioned equations by simulating several trajectories to reach targets situated at different depths.

We implemented the energy model using the values in Table II. We present key insights into our physics-guided simulation. Figure 7 presents the actuation energy for traversing different depths. Using our physics-guided simulation, we report an average instantaneous power of $\sim 124W$ for a $2700mAh$ battery. For each depth, we swipe the linear velocity as the control variable from $1$ m/s to $16$ m/s (the top speed of the drone in this study) in steps of $1$ m/s. We observe that the energy is the highest when the robot speed is the minimum. The time taken by the drone to reach the destination varies inversely with speed. As a consequence, if we use Eqn. (8) to estimate the energy, then the effect of time dominates at a lower speed for which the energy adds up. With increasing speed, the energy burned drops dramatically in the beginning due to a reduction in flight time. Beyond a certain speed, the energy again increases. This is because, for higher speeds, the motor currents drawn from the battery increases, resulting in an increase in instantaneous power and consequently energy. We observe there is a sweet spot for a near-optimal velocity which yields the least energy for a particular trajectory. This establishes the need for an interpolation a near-optimal velocity, given the target depth.

We collect several trajectories as part of our training dataset by flying the drone across different depths and by varying the linear velocity as shown in Figure 7. A vanilla ANN can be trained with several such depth-velocity pairs. However, the ANN will have no knowledge about the underlying source data distribution.

As shown in Equation (10a), we fit 5th-order polynomial curves to the energy-velocity profile obtained from the simulations, for each depth. To approximately evaluate the optimal velocity for near-minimum energy, we equate the derivative of the fitted polynomial to zero as shown in Eqn. (10b). As shown in Table III, for each training data point, the expression from Eqn. (10b) using the corresponding polynomial coefficients is added as an additional regularization term in the loss function. While a mid-range value of $v$ will cover most cases, addition of physical equations makes the model more interpretable. We use the standard mean-square error loss along with the additional physical constraints:

$n$ is the number of training samples. $v^{*}$ is the desired velocity obtained from the groundtruth, while $v^{pred}$ is the velocity the network learns. $\lambda$ is the regularization coefficient.

Note, the output of the PgNN is a velocity. Using Eqn. (12), we calculate an approximate trajectory time using which we estimate the position of the moving gate, in the future. Also, the training set is small because we are constraining the training via physical equations. This enhances sample-efficiency which accelerates training. While the training set consists of a similar range of depths as seen during inference, the inference can have a depth value not available in training data, making the proposed method superior than a regression-less lookup-based approach.

As shown in Figure 7, we assume the moving gate has a linear velocity along the Y axis as $v_{r}$. We use our SNN-based object tracking to estimate $v_{r}$. Consider the position of the gate along the Y axis to be $y_{1}$ at any given instant, obtained from the object tracker. For a sensing interval of $\delta t$, let the position of the same point in the gate after $\delta t$ be $y_{2}$. Then, to estimate the instantaneous velocity of the gate, we use the expression given:

We assume the gate is constrained within distances $+L$ and $-L$, beyond which the gate reverses the direction of motion. This is the only prior knowledge that we use during inference. Algorithm 1 presents the pseudo-code of our symbolic predictive planner. The planner accepts the variables $t_{traj}$ (trajectory time) and $L$ as the input arguments along with $y_{1}$ and $y_{2}$. $y_{1}$, $y_{2}$ are the outputs from the SNN and $t_{traj}$ is the output from the PgNN. The information from the two networks are now combined as discussed next. The algorithm outputs $y^{*}$ – the desired destination of the robot after $t_{traj}$, which will enable collision-free gate crossing. The distance $d_{1}$ traversed by the gate is computed during the trajectory time $t_{traj}$ (Line 3). Also, the distance $d_{2}$ (which is the distance of the gate from the nearest end ($-L$ or $+L$) ) is computed by considering the different sub-cases. If the gate moves right (line 5), then we consider the cases where the gate is in the positive Y axis (Line 6), or negative Y axis (Line 9). Accordingly, the value of $d_{2}$ is evaluated (Line 7 or Line 10). If $d_{1}$ $>$ $d_{2}$, then the gate will change the direction of motion (from right to left), and $y^{*}$ is computed (Line 15). If $d_{1}$ $<$ $d_{2}$, then the gate keeps moving right, and $y^{*}$ is computed accordingly (Line 18). Similarly, if the gate is detected to be moving left (Line 21), $y^{*}$ is evaluated symbolically by stepping through the hard-coded rules (Lines 22 - 36). Once $y^{*}$ is obtained, we use the minimum jerk trajectory [39] which is optimized to make sure the amount of current drawn from the battery is the lowest possible which results in almost zero rate of change of acceleration. Consequently, the planned trajectory burns near-minimum motor currents for the concerned navigation task. Note, that during PgNN training, the predicted velocity is along the depth axis (path AB = depth $d$ in Figure 8). However, during inference, there will be a component of the drone velocity along the Y axis ($v_{y}$ in Figure 8) in the same direction as $v_{r}$, as the target destination is C which depends on the speed of the moving ring ($v_{r}$ which cannot be known in advance). The predicted drone velocity $v^{pred}$ is only a proxy to calculate $t_{traj}$ (see Eqn. 12). However, a lower value of $t_{traj}$ will reduce the difference between paths AB and AC, making the actuation-energy near-minimum. Also, note, the flight path has a parabolic nature, but for the sake of explanation we simplified them as straight lines in Figure 8. While it is beyond the scope of this work to mathematically derive lower bounds, our method is reasonably energy-efficient (see Sections V-C, V-D and V-E).