Active Exploration for Real-Time Haptic Training

Autoencoders (AEs) are a neural network variant which find lower dimensional data representations in a manner analogous to non-linear principal component analysis. Autoencoders are typically composed of two elements: an encoder network and a decoder network. The encoder network takes full dimension input data and returns a lower dimensional latent representation. The decoder network takes an element of the latent space and returns a signal in the same space as the original input; training is based on requiring the decoder output to approximate the original input data. The encoder network can be used as a preprocesser for efficiently training other models to perform tasks such as classification or control.

To ensure better out-of-distribution performance, variational autoencoders (VAEs) modify the encoder to produce a multivariate latent distribution. This distribution is then sampled to provide a latent vector for the decoder network [1]. The latent distribution acts as a regularizer, and improves performance in regions of the latent space with sparse training data [2].

A final modification of the AE structure introduces a conditional vector associated with each data point. The conditional vector is provided as an input to both constituent networks as shown in figure 3. During inference the conditional input to the decoder can be varied to predict how sensor readings would change given different conditional parameters, so that the decoder can be used as a generative model. The resulting model is called a Conditional Variational Autoencoder (CVAE) [3].

Unsupervised training makes VAEs particularly promising for use in settings where novelty is expected (e.g., infrastructure monitoring [4]). VAEs and CVAEs have also been used for sensor modeling, including for compressing radar images and modeling soft sensors [5, 6, 7]. CVAEs are particularly promising for perceptual modeling, because the conditional vector provides a way to extract predicted sensor data across a variety of sensor states [8, 9], making them particularly appropriate for haptic perception.

Active learning (AL) is a machine learning strategy in which information about the model state or performance is used to guide data collection. This can result in lower losses or greater data efficiency relative to conventional training strategies. In deep learning AL can improve training times by biasing the data set, or reduce data labeling costs by prioritizing certain samples for manual review [10]. In a robotics context AL can be used to guide the behavior of a system to maximize model learning. For example, an active learning approach increases the rate at which Koopman operators can be re-trained on simulated quadroter dynamics in flight [11]. AL can additionally be powerful for training perceptual sensor models, since it enables a data collection system to focus on only the most important parts of a sensor target.

Training speed, data efficiency, and energy use are all key considerations when training learned sensor models. Active learning can be used to substantially improve training performance of CVAEs on camera-like sensors [12]. However, active learning has not been used for generating perceptual models of tactile sensors. Since tactile sensors tend to be delicate and slow to collect samples, they represent a particularly promising application area of AL, motivating the present work.

We demonstrate our methods using a BioTac sensor from SynTouch. The BioTac is a biomimetic finger-tip sensor designed to have sensing capabilities similar to those of a human finger [13]. The BioTac has three primary sensing modalities: an ultrasonic pressure sensor which provides spectral data, an array of 19 electrodes which provide spatial pressure data, and a temperature sensor which provides a measurement of heat-flux[13]. The BioTac is composed of a solid inner core, containing the sensing electronics, a soft conductive skin, and an electrolytic fluid layer[13]. The outer skin contains “fingerprint” ridges which help the ultrasonic sensor resolve textures, and it is held in place by a rigid “fingernail” which also serves to seal the fluid chamber [13].

Early research using the BioTac showed near-human microvibration and impact detection performance using the ultrasonic sensor[14]. In [15] it was demonstrated that a set of 3 features generated from the ultrasonic data can be used to accurately identify various textures using Bayesian exploration. Recently, a spiking network was applied for rapid texture identification using the BioTac’s electrode array across 20 different materials [16]. The BioTac is extensively used in haptic robotic manipulation, including for haptic teleoperation and manipulation of diverse objects [17, 18, 19, 20].

Although the BioTac sensor provides a rich and sensitive data stream, it experiences drift and is non-linear in force response beyond 2N [15]. Finite Element Analysis (FEA) simulations and data sets have improved modeling of the BioTac sensor [21, 22, 23]. However, while there are potentially adequate simulation capabilities for a BioTac in a manipulation context, there are no simulations suitable for predicting BioTac texture response.

Our learning process (Fig. 2) alternates between periods in which the model is trained using previously collected data, and periods in which the state of the model is used to guide the collection of new data.

Our work uses the CVAE architecture shown in Fig. 3. This model architecture was selected for its ability to efficiently compress structured data, and because it can be used to make predictions about sensor values for different system states. The model’s data input has 57 dimensions consisting of three concatenated 19 dimensional sensor readings, and its conditional input is the position of the sensor in Cartesian space. The model’s output distribution is a maximum likelihood Gaussian modeling the encoder input. The output distribution has a scalar diagonal variance matrix with a magnitude provided by the decoder. This variance output from the decoder provides a measurement of the network’s uncertainty for a particular pairing of a latent space element and conditional vector, which might reflect either underlying variance in the data or the training state of the network.

The encoder and decoder networks are composed of fully connected layers and taper symmetrically towards the center of the network. Each half of the network has an inter-layer ratio of $0.8$, with an initial layer width of $300$, and a depth of $4$ layers. The encoder outputs means and variances for a $6$ dimensional latent distribution with a diagonal variance matrix. During training a value is sampled from this distribution and passed to the decoder as the model’s latent vector. The model is trained with a batch size of $256$ and a dropout value of $p=0.2$. Leaky ReLU activation functions are used for all neurons except the output layer, which uses a signmoid function to enforce consistent scaling. The network is initialized with Kaiming Uniform Initialization for $a=sqrt(5)$[24].

Generating exploratory motions for tactile sensing involves balancing two competing priorities: The sensor must explore high-salience parts of a scene under many contact conditions in order to build a repeatable model, but it must also continue to explore other parts of the environment and other contact conditions in case they contain high salience features. The coverage of a sensor trajectory relative to the environmental domain and contact conditions can be specified by minimizing the ergodic metric [25] relative to a continuous target distribution provided by a learning model.

In order to compare a trajectory (a set of states parameterized by one continuous variable time $t$) to a 2D (or higher dimensional) distribution, the trajectory is first decomposed into a series of delta functions, Eq. (1).

These delta functions can be represented using a Fourier decomposition of (1) using basis functions of the form (2) where $n$ is the dimension of the state space, $k$ is an index over the coefficients of the multidimensional Fourier transform, $L_{i}$ is the length of the $i^{th}$ dimension and $h_{k}$ is a normalizing factor to ensure the basis remains orthogonal.

The Fourier coefficients corresponding to the trajectory are given by (4) and the coefficients approximating the target distribution are given by (5).

The ergodic metric can then be computed by taking the (Sobolev) distance between the time-averaged trajectory statistics and the target distribution (6).

For faster computation, we use a version of the sample-based ergodic controller described in [26]. This formulation approximates the ergodic measure based on Kullback-Leibler divergence and replaces the Fourier based ergodic metric above with (7). $P(s)$ is the target distribution, $q(s)=q(s|x(t))$, and s represents a sample.

The expectation can then be replaced with a sample based approximation as shown in (8). In our case, the distribution being sampled is provided by the model under training. This allows the target distribution to evolve as the model learns and new areas of high tactile complexity are discovered.

Haptic exploration benefits from this specification by always representing the entire domain and the likelihood that a sensor reading anywhere in the domain will improve the generative predictions for the haptic sensor. As a result, the exploration strategy can always take into account the entire domain and the state of the learning model over that whole domain, avoiding fixating on a small subset of the domain that happens to be feature rich.

The premise of our approach to generating exploratory motions is that by spending more time in areas of the conditional space with high model entropy, the sensor will collect a higher quality data set. Since the model output is structured as a multivariate Gaussian, (9)—which describes the entropy of a multivariate Gaussian—can be used to calculate the model entropy for a given decoder input. For (9): $x\sim N_{D}(\mu,\Sigma)$ is the network output distribution, $Y$ is the conditional vector, $Z$ is the latent vector, and $D=57$ is the dimension of a data point.

Since for our model the output variance ($\Sigma=\sigma I_{D}$) is a scalar matrix, (9) simplifies to (10).

This value can then be sampled for different conditional vectors to provide the target distribution required for ergodic exploration. A fixed length trajectory is then optimized relative to this target distribution using the ergodic metric described by (8). As an example of this process, figure 4 shows two target distributions (Network Entropy) and the exploratory trajectories that result (Training Trajectory). In practice, each time the distribution is sampled we use several latent vectors drawn from the preceding round of data collection and then average the result. This ensures a consistent distribution during the trajectory generation process.

To encourage exploration we add a constant value to all sampled entropies, ensuring that at least $5\%$ of the sensor’s time is spent in low-entropy areas of the conditional space. This helps prevent the neural network from fixating on high-entropy areas.

The initial data collection round—prior to any training—needs to be specified. To encourage exploration the initial target distribution is set to a uniform distribution across the conditional space. This ensures the sensor explores the entire scene before beginning the active learning process.

For comparison purposes, we use a passive sampling approach that does not use the state of the learning model, specifically relying on random walks, though raster scanning could also be used. Random walks can be vulnerable to over-exploring low information areas of the space, or to missing key areas—like a leaf, piece of leather, or another “object”—entirely. Rastering strategies ensure good coverage, but can take a long time to achieve good coverage and risk being dominated by irrelevant data. This is particularly an issue for tactile sensors, which must explore not only a range of positions/states, but also a range of contact parameters.