AcousTac: Tactile sensing with acoustic resonance for electronics-free soft skin

In Section 2, we describe the theoretical basis for AcousTac design, which includes models of taxel length and resonant frequency, relationships of taxel deformation with force, and proposed alterations of boundary conditions. The taxel implementation is presented in Section 3, along with a description of the experimental and signal processing methods used to characterize the sensor. Results from experimental characterizations of different single taxels in Section 4 reveal the effect of tube length and end cap modifications on the range and sensitivity of AcousTac. We demonstrate a system of AcousTac sensors in a 4-taxel array and a 3-taxel astrictive gripper in Section 5. Design guidelines, additional observations, and limitations and future work are described in Section 6. Section 7 summarizes the impact of this work in the broader context.

In one dimensional (1D) theory, a tube that is open on one end and closed on the other has a fundamental resonant wavelength, $\lambda$, four times the tube length, $L$ (Fig. 1A). Frequency, $f$, is related to wavelength, $\lambda$, by $\lambda=c/f$, where $c$ is the the speed of sound. Resonant frequency, $f_{oc}$, of an open-closed tube is therefore

For a tube open on both ends (Fig. 1B), the resonant wavelength is twice the tube length, such that the resulting frequency, $f_{oo}$, is

For both the open-open and open-closed boundary condition, frequency is inversely proportional to length. The frequency will be higher for a smaller sensors with shorter tubes. We propose to fit experimental data to a modification of Eqn. 1 in this case, with the constants $b_{i}$ such that

In the case of a tube with a soft end cap at one end, this boundary condition appears to lie between the open and closed idealized cases, as the soft material partially damps and reflects vibrations, still resulting in resonance. When external forces deform the soft end-cap (Fig. 1C), the length of the tube changes; the frequency also changes. Linear approximation of Eqn. 1 shows that the shorter the tube length $L$, the greater frequency change $\Delta f$ for a change in length $\delta$, where $L$ is the unloaded length. As such, shorter tubes are more sensitive to end cap deformations. For a linear approximation of the open-closed tube, change in frequency relates to length as

The above 1D theory assumes a thin tube, $L>>D$, where $D$ is the inner diameter of the tube. In practice, taxel tubes have a non-zero diameter and a particular inlet geometry which makes the measured resonant frequency deviate from theory. Regardless, the 1D model serves as a useful design tool to predict first-order trends. Additional factors known to contribute to resonant frequency, are kept constant for this study. For example, as flow rate increases, higher harmonics dominate the resonance; this work assumes the fundamental frequency mode across all designs.

Deformations of the end-cap alter tube length and result from externally applied normal compressive forces. Frequency is therefore a measure of force. Designers can tune force sensing range and resolution by varying end-cap stiffness. Both material selection and geometry alter the force-deformation response. Generally, the stiffer the cap, the greater the force range and the lower the resolution and sensitivity. The Hertzian contact theory model for small deformations of a compliant rubber hemisphere relates force $F$ to displacement $\delta$ as

In this work, the end-cap is a hemispherical shell with a wall thickness $t$ and inner diameter $D$ that experiences large deformations (Fig. 1D). We therefore propose an empirically fit approximation of Eqn. 5, for each cap design

Due to the physical properties of the soft compliant end-cap, the unloaded cap behaves somewhere in between an open and closed boundary condition, and we refer to this as damped. In this damped boundary condition, the cap is a flexible, oscillating surface, resulting in a frequency higher than in the closed condition and lower than in the open condition. When a rigid object contacts the cap, the boundary condition undergoes a transition from damped to closed, lowering the resonant frequency.

This will occur even without gross deformations or substantial changes in tube length $L$. However, this transition between different boundary conditions occurs across a range of light contact forces greater than zero, until there is no relative vibration between the object and cap surface. For forces higher than this boundary condition transition phase, gross deformations reduce length and increase frequency. We therefore expect a nonmonotonic sensor signal, with a minimum value under light contact forces.

In this work, we propose and investigate two design features to mitigate or remove this resonance property, and produce reliable measures of force. Specifically, we explore the introduction of a hole or mass at the tip of the end-cap.

A rigid tube and compliant end-cap compose each taxel. We individually fabricate these components and assemble them together into a single taxel, which is then rigidly mounted to an acrylic plate for testing. The rigid tube is a specified length $L$ (Fig. 2A), additive manufactured with PLA (Makergear M3-1D 3D Printer, nozzle diameter 0.35 mm). The length of the resonant tube varies between 41 and 65 mm, in increments of 6 mm. These 6 mm intervals ensure the resonant frequencies of the different taxels do not overlap. The inner diameter is 6 mm. The smallest cross section is 1.4 mm, located immediately before the edge-orifice, the angled cutout of the tube wall. The inlet and edge-orifice geometry are consistent across all taxels.

The end-cap is the soft element in the taxel. We fabricate and test a range of cap thicknesses $t$, hole diameters $h$, and added mass $m$ (Fig. 2B) to assess the effect of these parameters. We cast silicone (Smooth-On Dragon Skin 30) in a two-part mold to make the caps. All caps have the same 7 mm inner diameter hemisphere, regardless of thickness, and a 5 mm long cylindrical cavity, also 7 mm in diameter, that stretches over the open end of the rigid tube.

To assess taxel stiffness and force range, the wall thickness $t$ of the caps ranges from 1 to 5 mm (t1 through t5), at 1 mm increments. To test the effect of hole size, we compare caps with 3 mm wall thickness, across 0, 1, 3 and 5 mm hole sizes. A complete set of thicknesses (t1 through t5) are tested for both without (h0) and with a 3 mm hole (h3) to test coupling between these parameters. The caps with holes are cast with dowel pins of specified diameter in the negative mold. We add cylindrical magnets to the compliant caps to assess the addition of mass. Each magnet has a mass of 50 mg, a diameter of 2 mm, and a height of just less than 1mm. We test caps of 1, 3, and 5 mm wall thickness, and vary the number of magnets between 1 and 4 (m1 through m4). This results in an added mass of up to 200 mg. For the 1 mm thickness cap, we adhere 1 magnet to the outer surface with glue. For 3 mm and 5 mm thickness, the first magnet is embedded into the cap during the casting process. We increase mass by adding more magnets to the inner surface of the cap, which utilizes their magnetic attraction to attach them.

In this study, we characterize all the different end-cap designs using one resonant tube length $L=59$ mm. When comparing the different rigid tube lengths, we use a single end-cap design with 3 mm wall thickness, and no hole nor mass (t3h0).

We characterize each taxel individually using the experimental setup shown in Fig. 2C. The taxel is fixed to the tabletop. Wall compressed air flows through flexible 8.3 mm (3/8”) tubing and a flow meter (analog 1-10 LPM) before entering the resonant tube, which is kept between 4 to 5 L/min. A robot arm (UR-10, Universal Robots) and 6DOF wrist force/torque sensor (Axia80, ATI) cyclically palpate the taxel, moving at 0.5 mm/s until at least 5 N of normal force. We increase the tested force range with cap thickness. Robot Operating System (ROS) software records force and position at 2.0 kHz and 3.3 kHz, respectively. A smartphone (iPhone 11) located approximately 0.5 m away records sound at a sampling rate of 44.1 kHz. Ambient lab noise is approximately 65 dB, less than the approximately 90 dB sound level when a taxel is active.

Fig. 2D shows an example of the audio signal and processing outcome for four palpations of the t3h0 end-cap. We perform data processing using the pipeline detailed in Li, et al [36]. We extract the resonant frequency from the raw amplitude data at 25 Hz, using the built-in MATLAB functions spectrogram() and tfridge() and binning frequency in 2.5 Hz intervals. For caps with holes, we are also interested in the amplitude of the audio signal. We generate an envelope of the raw audio by finding the max absolute amplitude value for every 22.7 ms (1000 data points) then taking a smooth average for every 113 ms (5000 data points). The amplitude is then downsampled to 25 Hz to match the sampling rate of extracted frequency.

Characterization tests without a hole h0t1-h0t5 or with a mass m1-m4 all consist of at least 3 palpation cycles performed within 80 sec each. In the amplitude thresholding tests when a hole is present, h3t1-h3t5 and h1t3-h5t3, we only report points above the set threshold, which is approximately 12 sec of data over at least 2 palpation cycles. The specific amplitude thresholds in each hole case are individually selected as the minimum value to avoid a nonmonotonic relationship between force and frequency in that design’s data set.

In later demonstrations with systems of 3 and 4 taxels emitting sound simultaneously, we assume that the achievable frequency range of each taxel does not overlap with any other. This is achieved by assigning a unique tube length to each taxel. We process data within the expected frequency range for each taxel separately.

In Fig. 3A, we plot the measured frequencies for the four rigid tube lengths from two normal force loads: 5 N and 10 N. We also show the 1D theory for an open-closed cavity from Eqn. 1. The measured frequency is consistent with the theory, in which longer tubes exhibit lower resonant frequency. Table 1 presents fitting constants from Eqn. 3. The end-cap geometry is a negligible constant length offset and is not included in our calculations.

Fig. 3B shows the full characterization curves for frequency as a function of force, with loads ranging from zero to greater than 10 N. By design, the frequency ranges do not overlap each other to streamline later signal processing when multiple taxels emit sound at the same time. In Fig. 3C, we plot the same data but now as the net change in frequency; measured frequency $f$ is subtracted by unloaded frequency $f_{0}$ to inspect the calibration curve shape. All four taxels demonstrate a nonmonotonic frequency relationship, with both local maxima and minima observed at forces less than 3 N. As force increases from initial contact, the frequency increases, then decreases, then increases again. This behavior is likely due to the boundary condition changing from damped to closed as the force increases. After the boundary condition transitions, at forces greater than 3 N, the shorter length taxels with higher unloaded frequency have steeper slopes. This is consistent with Eqn. 4. We do not observe any hysteresis in the signal for these taxel lengths.

We plot cap deformation versus force in Fig. 4A for caps with no hole (t1h0-t5h0). In the tested displacements, thicker caps provide a more linear force displacement curve, which is steeper than thinner caps at lower displacements. The softer, thinner caps have a linear region at low forces, then start to saturate. Table 2 shows the fitting constants for these caps using Eqn. 6.

Fig. 4B shows how the amplitude of sound produced by the taxel changes with deformation. A negative value of $\delta$ denotes the distance between contact surface and top of the cap, e.g., a -2 mm deformation means they are separated by 2 mm. Prior to contact, thinner caps exhibit lower audio amplitudes and are closer to the open boundary condition than thicker caps. However all cap thicknesses demonstrate a downward deviation in audio amplitude centered between 0 and 2 mm of contact. In Fig. 4C, the resonant frequency $f$ is plotted as a function of force $F$. All the caps produce nonmonotonic boundary condition transitions, while the thinner caps show greater frequency fluctuations during the boundary condition change. The thinnest cap signal saturates at around 3 N of load, where the cap is compressed against the rigid resonant tube and can deform no more. The force-frequency slopes are consistent with cap stiffness trends.

Fig. 4D shows the audio amplitude with varying cap wall thickness in designs with a 3 mm diameter hole (t1h3-t5-h3). As compared with the cases without a hole, all taxels have a lower amplitude when there is no contact. Upon contact, varied deviations in the amplitude arise. However, all signals increase and remain above the unloaded amplitude with enough deflection. This amplitude cross-over threshold occurs for the smallest deflection for the thinnest cap, which also has the lowest amplitude prior to initial contact. In Fig. 4E, the force-frequency relationship is plotted while employing the amplitude crossover threshold to produce monotonic results. The shape of the force sensing curves is similar to those found in Fig. 4C, except for the omission of data at the lowest force levels. Table 3 reports the each force taxel’s resulting approximate linear ranges and sensitivities. The lower bound of force range is from the minimum detectable force after amplitude thresholding. Visual inspection dictates the upper bound as where the signal begins to saturate.

In Fig. 4F, amplitude is plotted across caps with the same thickness, 3 mm, but with varying hole size (t3h0-t3h5). Larger holed caps exhibit lower sound amplitude prior to contact, which is expected as they are more physically similar to an open tube. As a result, the larger hole allows for a lower amplitude threshold. As seen in the monotonic force-frequency curves in Fig. 4G and tabulated values in Table 4, caps with larger holes capture smaller forces in their sensing range when using amplitude thresholds. With the exception of the t3h5 calibration curve, for which the hole is a substantial size compared with the cap and lower forces are measured, these sensitivities are within 8% of each other.

Fig. 5 depicts how magnets attached to the tip of the cap alter force-frequency relationships in the transition region, where A, B, and C show the results for the 1, 3, and 5 mm cap thicknesses, respectively. The added mass varies from 50 g (m1) to 200 g (m4), with darker lines indicating higher mass. Overall, increasing mass mitigates the frequency fluctuations from the boundary condition transition. We observe nonmonotonic behavior in the tested force range with 50 mg added mass in all the three cap thicknesses. This boundary condition effect resolves with 200 mg for 1 mm thickness and 3 mm thicknesses, and 50 g for 5 mm thickness caps. As expected, the thinner caps require more mass to remove the boundary condition transition. For both the 1 and 3 mm thicknesses, intermediate levels of mass shifts the minima frequency locations to higher force levels. The mechanism that causes the t3m3 force-frequency curve to have this upward then downward sloping shape is unclear; the decrease is out of the typical force range for a boundary condition transition and perhaps due to the mechanics of the added mass on the cap. The magnets have nonzero volume and are rigid, which may account for the hysteretic effect observed within some transition regions, e.g., in t1m3.

Two multi-taxel systems are shown in Fig. 6: (A) a tactile array and (B) an astrictive gripper.

First, we combine four taxels of different lengths for an indicied, force and contact sensitive 2x2 array. The taxel lengths are consistent with those in Sec. 4.1: 41, 47, 59, and 65 mm. We position the four taxels in a grid pattern, spaced 15 mm apart center-to-center. The end-caps have a 3 mm wall thickness and 3 mm hole (t3h3). For the processing of the signal from the L59 taxel, we use the sensitivity value from Table 3 to translate frequency to force. For the three other taxel lengths, we scale the force by a ratio of $L^{2}$ according to Eqn. 4. All taxels receive air from the same source. As varying flow rate produces slight differences in each taxel’s frequency, and can change as neighboring taxel holes get covered, the measured frequency at light contact is set as the unloaded frequency as an initialization calibration procedure.

Next, we integrate three taxels into a force sensitive astrictive gripper. Inspired by prior work [24, 37], we implement passive suction cups to grip onto objects. The taxels are evenly spaced 15.9 mm apart, with rigid tube lengths of 41, 47, and 59 mm. The caps have 2.5 mm wall thickness, and we adhere a suction cup to each cap using an air-cured silicone glue (Smooth-On Sil-Poxy). The suction cups are similarly cast from silicone (Smooth-On Dragon Skin 30) and are 13 mm in diameter and 2.5 mm thick. The suction cups sit on posts 7 mm in diameter and 11 mm tall. We affix the caps to the rigid tube and wet the suction surface with tap water to increase suction force. Because the introduction of suction cup structures to the caps alters the frequency response of this system, we report frequency and not force.

Shown in Fig. 7A, we contact and palpate the caps individually with one finger and together with a piece of acrylic over four different sequences. First (i), we make light contact with each taxel individually, started from L65 and ending with L41. In the same order, we push down on each taxel individually with greater force (ii). In the third sequence (iii), an acrylic plate is pushed down on all the taxels simultaneously. In the last sequence (iv), the same acrylic piece applies a rolling forceful contact across the array in two orientations rotated by 90 degrees, side to side and front to back. Fig. 7B shows the audiospectrogram in the relevant frequency range. We extract the peak frequencies from the spectrogram (Fig. 7C) and translate them to force (Fig. 7D). Signals are as expected. With rolling contact, we observe two taxels loaded prior to the other two, showing the different between the two rolling orientations. We use a linear mapping enabled by the addition of a hole and therefore monotonic force-frequency relationship. This tactile array demonstrates the practicality of holes in the caps, exhibiting low amplitude of sound when not in contact, and resonance when loaded.

Shown in Fig. 8A, we manipulate a 500 g weight, imparting tension and compression via suction cup gripping. Fig. 8B shows the net frequency of the three taxels. In the demonstration, the suction cups adhere to the smooth, flat surface of the weight (i). The gripper lifts the weight and hefts repeatedly with weight still attached (ii), varying the tension loading. In part (iii), we compress the suction cups against the weight. The resonant frequency signals show expected trends, in which the sensor captures the oscillating tension forces at approximately 4 Hz in (ii) and the larger compression force at about 1 Hz in (iii).

Spatial resolution is not the focus of this current work and could be improved in future work. Taxels need to be spaced far enough apart such that a deformed cap does not interfere with the neighboring taxel caps. Therefore, as taxel diameters get smaller, they can be packed more closely. As taxels reduce in size, output frequency, air supply, fabrication precision, and microphone requirements may need to change. Another issue could arise where the surrounding taxels interfere with the airflow near the edge-orifice. If the edge-orifice area of the tube is obstructed, resonance is inhibited, though we do not observe any issues for taxels spaced 15 mm apart. For sensors at substantially different length scales, a resonance generating geometry other than the current edge-orifice may be more appropriate from a resonance, packing, and physical implementation standpoint. With the addition of more taxels, future work should also investigate the limit on how many simultaneous frequencies can be produced within unique frequency ranges.

Throughout this work, we assume superposition holds for sounds signals from systems of multiple taxels. This seems to be true for the current design, where each taxel operates the same independently of the other taxels. However, with closely overlapping frequency bands, we observe heterodyning and difficulty processing the signals as a result. We also observe a dependence on microphone location due to constructive and destructive signal interference. Each taxel is separated by rigid materials in this study. When resonating soft structures are physically coupled together, their resonant frequencies can interact in complex and nonlinear ways [38], a topic to further investigate in fully-soft resonating structures.

We characterized AcousTac under normal loading conditions, yet oblique loading occurs in the real robot manipulation. As a preliminary test, we sheared caps with and without a hole under 5N normal force, with up to 2N lateral force. We did not find the sliding and shear forces to affect the normal force measurements. By modifying the cap geometry, AcousTac structures could potentially be specialized for other particular types of loading and deformation measurements in future work.

AcousTac uses the ambient atmosphere to transmit information. The atmosphere must be gaseous, as this resonance generation utilizes the compressible nature of the ambient air. The atmosphere also must be dense enough such that the sound propagates, i.e. not the vacuum of space. Extreme changes in temperature, density, and humidity would require additional calibration steps as they alter resonance and sound propagation [39]. As an open fluid system, the robot also needs to be able to provide a continuous flow of air when activating the sensors. Pneumatic systems for locomotion in untethered robots [40] and robots in extreme environments [41] could potentially power AcousTac for sensing on mobile robots. One relatively untapped benefit of active acoustic tactile sensing is that sound can be measured across large distances, even without physical contact with the resonating system. Thus, if loud enough, information about robot contact state is detected by any microphone within audible range.