Audio-Visual Event Recognition through the lens of Adversary

We first describe the audio encoding part of CSN. Following TALNet architecture proposed by [3], we employ stacked convolutions and pooling layers to first extract high-dimensional features from raw frames. In particular, the audio encoder consists of 10 convolution layers of 3x3 and 5 pooling layers are insert after every 2 convolution layers, which reduce the frame rate from 40 Hz to 10 Hz. The outputs of the convolution encoder are fed into 2 transformers blocks to further model the global interaction among frames. Each transformer block consists of 1 layer of multi-head scaled dot-product attention and 2 feed-forward layers which are connected by residual connections. Finally, we use attention pooling [3] to make final predictions. The audio encoder of CSN is depicted in the left branch of Fig 1.

For encoding and representing videos, our model employs the R(2+1)D block similar to [22] which decomposes the 3D (spatial-temporal) CNN into a spatial 2D convolution followed by a temporal 1D convolution. Specifically, let $h,w,k$ denotes the height and width of $k$ frames in a sliding window. For a video clip with $M$ frames, there are $N=\lceil{M/k}\rceil$ windows in it, and the size of each window is denoted as $[k,w,h]$. For each $k$ frame window, our 3D CNN employs a sequence of $[1,w,h]$ 2D convolutional kernel followed by a $[k,1,1]$ temporal convolutional kernel to encode the video clip into a size $H$ vector. For the windows in the video, we utilize a 1D temporal CNN and a linear layer to extend its temporal receptive field and map the visual context for multi-modal fusion with the audio model. Essentially, the 3D CNN encodes the video into $\mathbf{v}\in\mathcal{R}^{H\times N}$, where $H$ is the embedding size for multimodal fusion. For late-fusion, we use additional 2 layers of transformers and an attention pooling layer for prediction. The visual modules are shown on the right side of Fig 1.

In order to understand the trade-offs between early, middle and late fusion, we performed 4 different types of fusions in our study. As is shown in right side of Fig. 1, we analyze the effects of fusing video bottle-neck features with the audio pipeline at different stages. Specifically, we investigated late fusion, two levels of middle fusion, and early fusion. For late fusion (maximum model parameters), we aggregate the output predictions from the full audio and visual models. For early fusion (minimum model parameters), we concatenate the visual bottleneck feature with the spectrogram. In middle fusion, we concatenate the visual feature before and after the transformer blocks with the audio bottleneck features.

As is pointed by [17], in training multi-modal models, the different modalities tend to overfit and generalize at different rates. Thus, we normalize and balance the gradients of the two modalities before we transform and concatenate the latent dimensions in the fusion process.

Originally, single-modal AER models can be formulated as a minimization of $\mathbf{E}_{x,y\sim D}[L(f(x),y)]$ where $L$ is the loss function, $f$ is the classifier mapping from input $x$ to label $y$, and $D$ is the data distribution. We evaluate the quality of our classifier based on the loss, and a smaller loss usually indicates a better classifier. In this work, since we are using adversarial noises to deactivate the models, we form $\underset{\delta}{\max}[\mathbf{E}_{x,y\sim D}[L(f(x^{\prime}),y)]]$, where $x^{\prime}=x+\delta$ is our perturbed audio input. It is common to define a perturbation set $C(x)$ that constrains $x^{\prime}$, such that the maximization problem becomes $\underset{x^{\prime}\in C(x)}{\max}[\mathbf{E}_{x,y\sim D}[L(f(x^{\prime}),y)]]$. The set $C(x)$ is usually defined as a ball of small radius of the perturbation size $\epsilon$ (of either $\ell_{\infty},\ell_{2}\,$ or $\,\ell_{1}$) around $x$.

To solve such a constrained optimization problem, one of the most common methods utilized to circumvent the non-exact-solution issue is the Projected Gradient Descent (PGD) method:

where $\mathcal{P}_{\epsilon}$ is the projection onto the $\ell_{p}$ ball of radius $\epsilon$, and $\alpha$ is the gradient step size. To practically implement gradient descent methods, we use very small step size and iterate it by $\epsilon/\alpha$ steps.

In this work, we are particularly interested in measuring the attack potency through the benchmarks including $\epsilon$ in the $\ell_{p}$ norm ball, and step size $\alpha$. Since our main focus is to maximally exploit the model’s universal weakness, we only discuss untargeted attacks that drift the model’s prediction away from the true label $y$. We do not discuss targeted attacks where we have to minimize the loss towards $y_{target}$ in the mean time, since doing this will inevitably further constrain our optimization and make our attack less potent. Note that our $\delta$ is a universal perturbation trained on the entire training set, and it is tested against the entire evaluation set to measure its attack success rate.

In a multi-modal setting, the loss formulation is $L_{multi}=L(f(x_{m_{1}}\oplus x_{m_{2}}\oplus\cdots\oplus x_{m_{k}}),y)$, where $x_{m_{k}}$ denotes inputs from multiple modalities. In this work, we are dealing with 2 modalities: $x_{audio}$ and $x_{video}$ We study the adversarial perturbation computed against the audio input $\delta_{audio}$. Our optimization goal is: $\underset{\delta_{audio}}{\max}[\mathbf{E}_{x,y\sim D}[L(f((x_{audio}+\delta_{audio})\oplus x_{video}),y)]]$ , and thus, our PGD step is:

Since we focus more on the robustness of the audio modality in this work, we will study the effect of $\delta_{video}$ in a future work, and we expect the same formulation applies to the visual modality.

We downloaded the entire audio and visual track of Google Audio Set [1]³³3Unfortunately, some links are broken, we were able to download 98% of the 2 million clips of Audioset which contains 2 million 10-second YouTube video clips, summing up to 5,800 hours. AudioSet is annotated with 527 types of sound events occurring in the clips, but they are weakly labeled in the sense that no information about the time span (onset and offset) of the events is annotated in each 10-second excerpt. We train and test the models according to the train and test split described in [4]. The input for the audio branch are matrices of Mel filter bank features, which have 400 frames(time domain) and 64 Mel-filter (frequency domain) bins. For the visual branch, we employ a 3D CNN backbone to encode the spatial-temporal context in videos as is mentioned in Section 3.2.

3D-CNN: Our 3D CNN video encoding backbone is initialized with the network pre-trained on IG65M [23]. For training on the videos in Google Audio Set, we freeze the clip-level 3D-CNN backbone and fine-tune the video-level 1D CNN and the transformer layer.

As we can see from Table 1, late fusion prevails in terms of performance with and without the presence of the adversarial noise computed in the same way (full-frequency and time domain, the same size as input). However, late fusion is the most expensive model as it requires 2 full-sized branches of audio and visual models with a huge amount of parameters. Evidently, we can observe the trend that the deeper level the fusion is, the more robust the model would be against the adversary.

SpecAugment [24] introduced a simple but effective technique for data augmentation in speech recognition task, where they mask out certain frequency band or time steps in the original spectrogram input. Inspired by this, we modify the spectrograms to construct adversarial examples. In particular, we distinguish frequency bands that are robust features versus those that are non-robust. Our results also partially explains why [24]’s approach achieved significant performance improvement, that is by masking out non-robust frequency bands, the model is forced to learn more robust features that are going to generalize better in test setting.

To identify the robust/non-robust regions in the audio features, we compute constrained adversarial noises as is shown in Fig. 2 (a,b,c). Against the best CSN audio-only model (without fusion), we computed different permutation of adversarial perturbations that mask out different frequencies and temporal portions with different attack strength. The results are shown in Table. 2 and Table. 3. As we can observe, adversarial noises present on lower frequencies tend to have higher attack potency. This suggests that the lower frequency bands contribute more to the overall performance of the model while the features there are not robust. Attacks on higher frequencies tend to be less effective. Not surprisingly, higher $\epsilon$ leads to more effective attack. Interestingly, using the $\ell_{\infty}$ attack, attacks on the different frequency lead to similar accuracy drop. This is because $\ell_{\infty}$ is too potent as is shown in Fig. 2 (d).⁴⁴4Due to space constraint, we often discuss adversarial perturbation using $\ell_{2}$ norm instead of $\ell_{\infty}$ norm in this work. We have performed the equivalent experiments using $\ell_{\infty}$ norm, and observed the same trend.In the temporal domain, we can observe nearly uniform drop in performance despite of the different temporal masks, suggesting the temporal domain contributes equally to robustness.

We also compute the adversarial noise with the same $\ell_{2}$ norm and $\epsilon=0.3$ to attack different neural network models. We run 3 trials of every model with different seeds to reported the average performance. The results are shown in table 4⁵⁵5To avoid repeating, refer to Table 1 for early/middle fusion result. We observe that replacing the recurrent layer of CRNN with transformers greatly boost the audio classification performance, suggesting that self-attention modules could lead to best accuracy empirically. Under the influence of adversarial noise, our full late fusion model still has the highest performance, whereas ResNet seems to suffer the least in terms of performance drop. This is an interesting yet not fully understood phenomena that pure convolutional modules appear to be more robust than recurrent or self-attention layers. We conjecture that the residual links in ResNet could be obfuscating the gradients making them more robust against gradient based attacks.

As a further step towards understanding what every model is learning and their reaction under the impact of adversarial noises. We further compare the performances of CSN (audio) and CRNN on different classes. Some of the classes that have large discrepancy are shown here. Although CRNN’s overall performance is much lower than CSN, it outperform CSN on several classes such as Mains hum by a large margin. From Table. 5, we can see that CSN has better adversarial robustness overall compared to CRNN. It is interesting to notice that classes trained with more data (the AudioSet is unbalanced) such as speech and music are relatively robust against attacks as is shown in Table 6, and the more well-defined structured sound such as sine wave does not get affected much by adversarial noise. Plus, the higher frequency classes’ accuracy tend to drop significantly with adversarial noise.