Distributed Node-Specific Block-Diagonal LCMV Beamforming in Wireless Acoustic Sensor Networks

We consider the WASNs with $J$ sensor nodes, where the set of nodes is denoted as $\mathcal{J}=\left\{1,\cdots,j,\cdots,J\right\}$. Each node $j$ is equipped with $M_{j}$ microphones and thus the total number of microphones is $M=\sum_{j=1}^{J}M_{j}$. The distributed speech enhancement problem is often formulated in the short-time Fourier transform (STFT) domain and the vector $\mathbf{y}\left(f,l\right)$ is given by

where $f$ denotes the frequency index, and $l$ denotes the time-frame index. $\mathbf{y}_{j}\left(f,l\right)$ is an $M_{j}\times 1$ vector consisting of locally received microphone signals at the $j$th node, and the superscript $T$ denotes the transpose operator. $\mathbf{y}\left(f,l\right)$ can be modeled as

where $\mathbf{s}\left(f,l\right)$ is a signal vector containing $S$ speech sources, $\mathbf{n}\left(f,l\right)$ is a noise vector, and

is a full-rank $M\times S$ ATF matrix. In particularly, $\mathbf{H}_{j}\left(f,l\right)$ is the ATF matrix between the $S$ speech sources and the $M_{j}$ microphones at the $j$th node. In the following, $\mathbf{H}\left(f,l\right)$ can be approximated as time-invariant in each frame and all derivations refer to a single frequency bin. The frame index $l$ and the frequency index $f$ will be omitted when no confusion arises.

For the centralized LCMV beamformer, node $j$ applies a $M$-dimensional estimator $\mathbf{w}_{j}$ to the $M$-dimensional microphone signals $\mathbf{y}$ to obtain the node-specific output $d_{j}=\mathbf{w}^{H}_{j}\mathbf{y}$, where the superscript $H$ denotes the conjugate transpose operator. $\mathbf{w}_{j}$ can be obtained from the following general optimization problem [32][33]

where $\mathbf{R}_{nn}=E\left\{\mathbf{n}\mathbf{n}^{H}\right\}$ is the noise covariance matrix and $E\{\cdot\}$ denotes the expected value operator. $\mathbf{g}_{j}$ is an $S\times 1$ desired response vector for the $S$ speech sources. Its entries usually consist of ones and zeros to preserve the target sources and eliminate other interfering sources simultaneously. The solution of (4) can be given by

The node-specific output can be expressed as

where $g_{j}\left(k\right)$ and $s\left(k\right)$ are the $k$th entries of $\mathbf{g}_{j}$ and $\mathbf{s}$, respectively. The superscript “$*$” marker denotes the conjugate operator.

Equations (5) and (6) require each node to have access to all microphone signals $\mathbf{y}$ to estimate $\mathbf{R}_{nn}$ and then obtain $d_{j}$. Therefore, all the locally received signals $\mathbf{y}_{j}$ at the $j$th node need to be transmitted, which results in a large communication bandwidth and a large transmission power in the WASNs. Besides, the computational power grows dramatically with the increase of $M$ when computing the inversion of $\mathbf{R}_{nn}$.

If the reverberation time of a room is moderate or large enough and/or the noise is far away from the microphones, i.e., the distance between the noise and the microphones is larger than the reverberation radius, its reverberant sound field is diffuse, homogenous, and isotropic [22] and [34]. In this case, the normalized correlation ${\Omega}_{m,m_{1}}\left(f\right)$ between two microphones $m$ and $m_{1}$ with distance ${\lambda}_{m,m_{1}}$ at a frequency $f$ can be given by

where $c=343~{}\mathrm{m/s}$ is the sound speed. The correlation can be roughly divided into two frequency regions: one highly correlated at low frequencies and the other much less correlated at high frequencies. The boundary between the two regions occurs at the first zero-crossing frequency $f_{c}=c/\left(2{\lambda}_{m,m_{1}}\right)$. When the distance ${\lambda}_{m,m_{1}}$ is large, the frequency $f_{c}$ is small. For example, $f_{c}$ equals 171.5 Hz for ${\lambda}_{m,m_{1}}=1$ m.

For the WASNs, the microphones within a node are often nearby, whereas the microphones from different nodes are further away. The noise can be assumed to be uncorrelated across the different nodes [22], and then $\mathbf{R}_{nn}\left(l\right)$ has the following block-diagonal form approximately

where $\mathbf{\Delta}_{nn,j}\left(l\right)$ is the noise covariance matrix at the $j$th node, and $\mathbf{0}_{M_{j_{1}}\times M_{j_{2}}}$ is an $M_{j_{1}}\times M_{j_{2}}$ null matrix.

Note that the noise covariance matrix used in the existing node-specific algorithms, such as [14, 24, 26, 27], and [28], is full-element. In the non-node-specific algorithm BD-LCMV [22], the block-diagonal $\mathbf{R}_{nn}\left(l\right)$ is adopted and the weight vector associated with the $j$th node is given by

where $\bm{\mu}\left(l\right)$ is a Lagrange multiplier shared by all nodes. Then the dual optimization problem is introduced to compute the optimal $\bm{\mu}\left(l\right)$ and is solved by PDMM. Finally, the signal $e_{j}(l)=\mathbf{w}^{H}_{j}(l)\mathbf{y}_{j}(l)$ is exchanged between nodes and the output $d_{j}(l)$ is obtained by summing all $e_{j}(l)$.

In this paper, the proposed beamformer has a completely new scheme. From (5) and (8), the weight vector corresponding to the $l$th-frame microphone signals can be expressed by

The $S$-dimensional compressed signals $\mathbf{z}_{j}\left(l\right)$ and the $S\times S$ matrix $\mathbf{D}_{j}\left(l\right)$ related to the $j$th node are defined as

From (6) and (LABEL:BDWeight), the node-specific output can be rewritten as

where $\tilde{\mathbf{z}}\left(l\right)$ is the product of $\mathbf{D}^{-1}\left(l\right)$ and $\mathbf{z}\left(l\right)$,

In particularly, $\mathbf{D}\left(l\right)$ and $\mathbf{z}\left(l\right)$ are obtained by summing all $\mathbf{z}_{j}\left(l\right)$ and all $\mathbf{D}_{j}\left(l\right)$ separately,

The above derivations assume that the noise covariance matrix $\mathbf{\Delta}_{nn,j}\left(l\right)$ at the $j$th node is perfectly known. However, for practical applications, $\mathbf{\Delta}_{nn,j}\left(l\right)$ is unknown and needs to be estimated from the noisy observations. This often requires a hard or soft VAD to determine whether the speakers are present or not. It is noted that the design of a VAD mechanism is a hot research topic on its own, and is out of the scope of this paper [18, 24, 35]. In the following, two methods including non-recursive (moving average) smoothing and first-order recursive smoothing are considered to estimate $\mathbf{\Delta}_{nn,j}\left(l\right)$.

In general, the ATF matrix $\mathbf{H}$ is unknown and needs to be estimated online. Some subspace estimation algorithms can be used to estimate the column space of $\mathbf{H}$ [19, 20]. Define $\mathbf{Q}=\left[\mathbf{q}_{1},\cdots,\mathbf{q}_{S}\right]$ to be any basis that spans the column space of $\mathbf{H}$,

where $\mathbf{\Theta}$ is an $S\times S$ matrix comprised of the projection coefficients of the original ATFs on the basis vectors.

When the speakers only change their positions slowly with respect to their initial positions, such as teleconferencing, we can estimate the column space $\mathbf{Q}$ at the initial stage (e.g., in a centralized way) [22]. This may cause some estimation error of $\mathbf{Q}$ if the speakers have some slight movements and therefore robust beamformer is preferred.

The goal for node $j$ is to estimate the target signal from the signal vector $\mathbf{s}$ as observed by one of node $j$’s microphones, referred to as the reference microphone. Without loss of generality, the first microphone of each node is chosen as the reference microphone. For node $j$, the index of its reference microphone is equal to $m=1+\sum_{q=1}^{\left(j-1\right)}M_{q}$. Accordingly, the weight vector in (5) can be rewritten as

where $Q\left(m,k\right)$ is the entry in the $m$th row and $k$th column of $\mathbf{Q}$. $\bar{g}_{j}\left(k\right)$ and $g_{j}\left(k\right)$ are the $k$th entry of $\bar{\mathbf{g}}_{j}$ and that of $\mathbf{g}_{j}$, respectively. The node-specific output in (6) is modified by the following equation [14]

where $H\left(m,k\right)$ is the entry in the $m$th row and $k$th column of $\mathbf{H}$.

It is obvious that different nodes have different desired response vectors due to different reference microphones, i.e., $\bar{\mathbf{g}}_{j}\not=\bar{\mathbf{g}}_{q}$ with $j\not=q$. Therefore, each node can extract a node-specific target signal based on node-specific reference microphone to preserve the spatial information of the target source, such as time difference cues, which are very important in target source localization.

When the noise covariance matrix $\mathbf{\Delta}_{nn,j}$ in (8) is replaced by the noisy covariance matrix $\mathbf{\Delta}_{yy,j}$, that is to say,

one can obtain the distributed node-specific block-diagonal linearly constrained minimum power (DNBD-LCMP) beamformer, which can be given by

where this beamformer does not require an estimate of the noise covariance matrix.

Particularly, when $\mathbf{\Delta}_{nn,j}$ is an identity matrix $\mathbf{I}_{j}$, the DNBD-LCMV becomes the distributed node-specific delay and sum (DNDS) beamformer

where this beamformer only needs to be calculated once at the beginning. However, it cannot control the sound sources that are not included in $\bar{\mathbf{g}}_{j}$.

Similar to [22], the proposed beamformers including DNBD-LCMV/DNBD-LCMP, and DNDS can be implemented in the WASNs with arbitrary topologies. This can be achieved by a slight modification to the data transmission process of each node, where each node aggregates its transmitted data including $\mathbf{z}_{j}\left(l\right)$ and $\mathbf{D}_{j}\left(l\right)$ with the transmitted data from its neighbors. This will allow for the transmitted data to disperse through the WASNs by means of an in-network summation, and is demonstrated for the tree topology, as shown in Fig. 3.

We denote $\mathcal{N}_{j}$ as the set of neighbors of node $j$ in the topology with node $j$ excluded. After the tree is formed with any tree formation algorithm [38, 39], one arbitrary node is assigned as the root node and the nodes only communicate with their neighbors (for example, $\mathcal{N}_{6}=\{5,7,8,9\}$ and node 1 is the root node in Fig. 3). The following data-driven signal flow is executed for each block of microphone signals (non-recursive smoothing method is considered below):

1) Any leaf node, i.e., a non-root node $q$ having only a single neighbor, can immediately fire and transmits $\mathbf{z}_{q}\left(l\right)$ and $\mathbf{D}_{q}\left(l\right)$ to its single neighbor (toward the root node). Any non-root node $j$ with more than a single neighbor waits until it has received the transmitted data from all its neighbors except for a single neighbor that has yet to fire, say node $j^{{}^{\prime}}$, and then computes the following sum

Next, $\check{\mathbf{z}}_{j}\left(l\right)$ and $\check{\mathbf{D}}_{j}\left(l\right)$ are transmitted to node $j^{{}^{\prime}}$ (toward the root node). This process is repeated at every non-root node in the tree until the root node is reached.

2) Once the data-driven signal flow has reached the root node, say node $j^{{}^{\prime\prime}}$, the vector $\tilde{\mathbf{z}}\left(l\right)$ is obtained by

The vector $\tilde{\mathbf{z}}\left(l\right)$ is now flooded through the WASNs (away from the root node) so that it reaches every node, where the nodes simple act as relays to pass $\tilde{\mathbf{z}}\left(l\right)$ further through the tree. Finally, the output $d_{j}\left(l\right)={\mathbf{g}}^{H}_{j}\tilde{\mathbf{z}}\left(l\right)$ is obtained.

Based on the data-driven signal flow above, we see that any leaf node will transmit only a single block of compressed signals $\mathbf{z}_{q}\left(l\right)$. Any non-leaf node will transmit a maximum of two blocks of signals including $\check{\mathbf{z}}_{j}\left(l\right)$ and $\tilde{\mathbf{z}}\left(l\right)$, first toward the root node and then away from the root node. It is worth noting that $\mathbf{D}_{q}\left(l\right)$ and $\check{\mathbf{D}}_{j}\left(l\right)$ only need to be transmitted once for each block of microphone signals and can be ignored.

When the first-order recursive smoothing method is applied, for any leaf node $q$, $\bar{\mathbf{c}}_{q}\left(l\right)$ and $c_{q}\left(l\right)$ are transmitted to reconstruct $\mathbf{D}_{q}\left(l\right)$ and $\mathbf{z}_{q}\left(l\right)$ with (20) and (21) at the non-root node $j$. The rest is the same as the non-recursive smoothing method. In particularly, $\check{\mathbf{D}}_{j}\left(l\right)$ needs to be transmitted for each noise-only frame.

The dimensions of the simulated room is $5~{}\mathrm{m}\times 5~{}\mathrm{m}\times 3~{}\mathrm{m}$ with reverberation times $T_{60}=0.3~{}\mathrm{s}$ and $T_{60}=0.5~{}\mathrm{s}$. The WASNs consist of $J=4$ nodes, each having $M_{j}=6$ microphones forming a uniform linear array with an inter-microphone distance of 3 cm. Four point sound sources including $S=2$ speech sources and two babble noise sources are presented. The configuration of the nodes and sound sources are depicted in Fig. 4. The two speakers that produce speech sentences taken from the NOIZEUS corpus [43] have the same power. The two babble directional noise sources are mutually uncorrelated with power that is $10\%$ of the power of any speaker, i.e., 10 dB SNR. Besides the two babble directional noise sources, all microphone signals have an uncorrelated white Gaussian noise component with 30 dB SNR with respect to the superimposed speech signals in the first microphone of node 1. The sampling frequency is 8 kHz and the sound speed is $c=343~{}\mathrm{m/s}$.

In order to approximate the real acoustic scenario, these positions of the speakers, where the column space $\mathbf{Q}$ was estimated, were uniformly distributed over a sphere centered around the true source positions depicted in Fig. 4. These positions were referred to as training positions. For Speaker 1 and Speaker 2, its erroneous training positions had radii $r_{1}$ and $r_{2}$ ranging from 0 to 10 cm, respectively. Therefore, the column space estimation error can be modeled as a function of positional error between the training positions and the true source positions [22]. For every value of the positional error, the average performance of 100 different setups were measured. Each setup used the same source signals at the true source positions. However, a different set of training positions mentioned previously and different realizations of the microphone-self noise were used in each setup.

In the experiment, the non-recursive smoothing method was adopted to estimate $\mathbf{\Delta}_{yy,j}$ and $\mathbf{\Delta}_{nn,j}$, and the estimation was performed on the entire length of signal [24, 44],

where $\mathcal{L}_{y}$ is the set of frames of the entire time horizon, and $\mathcal{L}^{{}^{\prime}}_{y}$ is the set of frames of the noisy signals used to estimate $\mathbf{\Delta}_{nn,j}$. The set $\mathcal{L}^{{}^{\prime}}_{y}$ is used to simulate the VAD error, where the noisy frames containing speech component are erroneously detected as noise-only frames. The error can be measured by the following scalar

When $\mathcal{L}^{{}^{\prime}}_{y}$ is an empty set, i.e., $\mathcal{L}^{{}^{\prime}}_{y}=\{\varnothing\}$ and $R=0$, an ideal VAD is considered.

The SNR is the ratio between the powers of the desired speech component and the noise, and can be defined as

where $\bar{d}_{j}(t)$ and $\bar{n}_{j}(t)$ denote the time domain beamformer output and the time domain noise component (also containing the residual competing speech component [18]) at the $j$th node, respectively.

The TDOA error $\Delta{T}_{j1,s}$ between the $j$th node and the 1st node for the $s$th speaker can expressed by

where $\mathbf{x}_{j1}$ is the location of the 1st microphone at the $j$th node, $\bar{\mathbf{x}}_{s}$ is the location of the $s$th speaker, and $\lVert\cdot\rVert$ denotes the two-norm of a vector. $T_{j1,s}$ is the theoretical TDOA and $\hat{T}_{j1,s}$ is the estimated TDOA based on the output signals of different nodes using the generalized cross-correlation phase transform (GCC-PHAT) [45]. More details about TDOA can be found in [46]. The ATE is defined by

For the following performance comparison, the first microphone of each node is chosen as the reference microphone and the experimental results for Speaker 1 are presented. In particularly, LC-DANSE [14] is time-recursive and converges to the solution of its centralized algorithm. Therefore, LC-DANSE is replaced by the centralized LCMV [22].

Fig. 5 shows the performance of different beamformers under different positional errors in terms of reverberation time $T_{60}=0.3$ s and VAD error $R=5\%$. The spectrograms of the desired signal received by the first microphone of node 1 and the output signals of node 1 of different beamformers were depicted in Fig. 6 for $T_{60}=0.3$ s, $R=5\%$, and $r_{1}=r_{2}=5$ cm.

From Fig. 5, first, one can get that the performance of the five beamformers decreases with increasing positional error. When there is no positional error, i.e., $r_{1}=r_{2}=0$ cm, the SNR and the STOI of DNBD-LCMP are higher than LCMP and DNDS, and are slightly lower than LCMV and DNBD-LCMV. When there is positional error, the SNR and the STOI of DNBD-LCMP are significantly reduced, and are much lower than DNDS, LCMV, and DNBD-LCMV. For different positional error values, the SNR and the STOI of LCMP are the lowest. LCMP and DNBD-LCMP had lower robustness to the positional error than DNDS, LCMV, and DNBD-LCMV. Second, for DNDS, LCMV, and DNBD-LCMV, the SNR and the STOI of DNDS are the lowest when the positional error is zero and then approach LCMV and DNBD-LCMV with increasing positional error (even higher than LCMV). This is related to the fact that DNDS cannot well suppress the directional noise sources that are not included in the desired response vector but has more robust performance to the positional error [47]. DNBD-LCMV has higher SNR and STOI than LCMV for non-zero positional error and is less sensitive to the positional error, where $\mathbf{\Delta}_{nn,j}$ has lower dimensions than the full-element noise sample covariance matrix used in the LCMV and is numerically more favorable. Third, the ATEs of different beamformers do not appear to be a significant difference.

From Fig. 6, the speech component in the output signal of LCMP is almost completely removed, where the column space estimation error results in removal of the actual speech component and preservation in the direction of the wrongly estimated column space. NBD-LCMP has the similar problem to LCMP. However, the performance degradation was not that great as with LCMP, this is because DNBD-LCMP has lower degrees of freedom to satisfy the wrong distortionless response and suppresses less speech component than LCMP [22]. Note that LCMP and DNBD-LCMP will not be further considered in the following experiments due to their poor performance in signal model mismatch cases. LCMV causes more speech distortion than DNBD-LCMV. DNDS preserved the speech component like DNBD-LCMV. However, it also preserves more noise component around 0.5 kHz.

In this subsection, we compare the performance of DNDS, LCMV, and DNBD-LCMV under different VAD errors with the reverberation time $T_{60}=0.3$ s and the positional error $r_{1}=r_{2}=5$ cm, as shown in Fig. 7. First, for an ideal VAD, i.e., $R=0$, the LCMV has the highest SNR and STOI. However, VAD error is often inevitable for practical applications. The SNR and the STOI of LCMV deteriorate significantly and become the lowest with increasing VAD error, which indicates that LCMV is the most sensitive to the VAD error. Second, when the VAD error becomes larger, the SNR and the STOI of DNBD-LCMV are reduced and gradually approach the DNDS, which is independent of the VAD error. Third, for different beamformers, their ATEs are almost identical and do not become larger with increasing VAD error.

In this subsection, we compare the performance of DNDS, LCMV, and DNBD-LCMV when the reverberation time increases to $T_{60}=0.5$ s, as shown in Fig. 8. First, one can observe that the SNR and the STOI of LCMV are the lowest for non-zero positional error and the VAD error. The SNR and the STOI of DNBD-LCMV are reduced, and gradually approach or even lower than DNDS with increasing positional error or the VAD error. Second, no significant difference can be observed in the ATEs of different beamformers, which is similar to Fig. 5 (c) and Fig. 7 (c).