CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency

Let $\boldsymbol{I}\in\mathbb{R}^{c\times h\times w}$ be an input image with ‘$c$’ channels. A deep visual classifier $\mathcal{K}(\boldsymbol{I})$ maps $\boldsymbol{I}$ to a prediction vector $\boldsymbol{y}^{\ell}\in\mathbb{R}^{L}$, where ‘$L$’ is the total number of classes. Here, ‘$\ell$’ indicates the predicted label of $\boldsymbol{I}$ under the premise that the $\ell^{\text{th}}$ coefficient of $\boldsymbol{y}$ has the largest value. It is well-known that a neural network is a hierarchical composition of representation layers. Rebuffi et al. [21] demonstrated that among these layers, those closer to the input learn class-insensitive features. Thus, the deeper layers hold more promise for computing image saliency for a model. Grad-CAM [25] takes a pragmatic approach to single out the last convolutional layer to estimate the saliency map.

Let us denote the $k^{\text{th}}$ activation map of the last convolutional layer of a network as $\boldsymbol{A}_{k}(\boldsymbol{I})\in\mathbb{R}^{m\times n}$. Focusing on Grad-CAM, the technique first computes an intermediate representation $\boldsymbol{S}(\boldsymbol{I})\in\mathbb{R}^{m\times n}$, such that $\boldsymbol{S}^{(i,j)}(\boldsymbol{I})=\sum_{k}w_{k}\boldsymbol{A}^{(i,j)}_{k}(\boldsymbol{I})$, where $w_{k}$ is given by Eq. (1). Henceforth, we ignore the argument $(\boldsymbol{I})$ for clarity, unless required.

In the above expressions, $\boldsymbol{X}^{(i,j)}$ is the $(i,j)$ coefficient of $\boldsymbol{X}$. The computed $\boldsymbol{S}$ is later extended to the final saliency map $\boldsymbol{\Psi}\in\mathbb{R}^{h\times w}$, as $f(\boldsymbol{S}):\boldsymbol{S}\rightarrow\boldsymbol{\Psi}$, where the function $f(.)$ must account for interpolating an $m\times n$ matrix for a $h\times w$ grid (along other complementary transformations).

Observing Grad-CAM (and similar methods) from the above perspective reveals two performance drain-holes in backpropagation image saliency computations.

Though centered around Grad-CAM, the above discussion points to a simple, yet powerful generic notion for effective backpropagation saliency computation. That is, to better leverage the backpropagated gradients, the differential information of the gradients (in $\boldsymbol{W}_{k}$) is still at our disposal to exploit. Fusing activation maps with this information promises more precise image saliency.

From the above discussion, it is clear that whereas useful techniques exist for backpropagation image saliency, the paradigm is yet to fully harness the backpropagated gradients and resolution enhancement of the activation maps for precise image saliency. Both limitations are rooted in the very nature of the underlying signals. Leveraging these signals from multiple layers can potentially help in partially overcoming the issues. However, this possibility is also restricted by the class-insensitivity of the earlier network layers and combinatorial nature of the problem. Moreover, there is evidence that multi-layer fusion can often adversely affect image saliency [21]. Hence, a technique specifically targeting the class-sensitive last layer, while allowing minimal loss of differential information across backpropagated gradients and improving activation map upsampling, holds significant promises for better saliency computation.

Building directly on the insights in § 3.3, we devise CAMERAS - an enhanced resolution and sanity preserving class activation mapping scheme. The approach is illustrated in Fig. (1-top) and explained below in a top-down manner, keeping the flow of the above discussion.

Our method eventually computes a saliency map as:

where $\boldsymbol{\overset{*}{W_{k}}}\in\mathbb{R}^{h\times w}$ encodes the differential information of the backpropagated gradients for the $k^{\text{th}}$ activation map in a network layer, $\boldsymbol{\overset{*}{A_{k}}}\in\mathbb{R}^{h\times w}$ is an enhanced resolution encoding for the activation map itself, and $f(.)$ performs an element-wise normalisation in the range $[0,1]$. The $\boldsymbol{\overset{*}{W_{k}}}$ and $\boldsymbol{\overset{*}{A_{k}}}$ are defined as follows

where $\varphi_{t}(\boldsymbol{X},\zeta_{t})$ is the $t^{\text{th}}$ up-sampling applied to resize $\boldsymbol{X}$ to the dimensions $\zeta_{t}$ - provided as a tuple. We fix $\zeta_{o}=(h,w)$ for $\boldsymbol{I}\in\mathbb{R}^{c\times h\times w}$. This will be explained shortly. We compute the $(i,j)$ coefficient of $\boldsymbol{W}_{k}$ as $\left(\frac{\partial\boldsymbol{y}^{\ell}}{\partial\boldsymbol{A}_{k}^{(i,j)}}\right)$. The overall process of generating an image saliency map with CAMERAS is summarized as Algorithm 1.

The algorithm computes the desired saliency map $\boldsymbol{\Psi}$ by an iterative multi-scale accumulation of activation maps and gradients for the $\kappa^{\text{th}}$ layer of the model. In the $t^{\text{th}}$ iteration, the input image $\boldsymbol{I}$ gets up-sampled to $\zeta_{t}$ based on the maximum desired size $\zeta_{m}$ and the number of steps $N$ allowed to reach that size (lines 4,5). Provided that the input up-scaling does not alter the model prediction, the activation maps and backpropagated gradients to the $\kappa^{\text{th}}$ layer are also up-sampled and stored. We show this on lines 6-10 of the algorithm. Notice that, we use calligraphic symbols to distinguish 3D tensors from matrices (e.g. $\mathcal{A}$ instead of $\boldsymbol{A}$ for activations) in the algorithm for clarity. The newly introduced symbol $\nabla\mathcal{J}(\kappa,\ell)$ on line 9 denotes the collective backpropagated gradients to the $\kappa^{\text{th}}$ layer $w.r.t.$ the predicted label $\ell$. Also notice, on lines 8, 9, up-sampling of the activation maps and gradients are performed to match the original image size $\zeta_{o}$. This is because, the same accumulated signals are eventually transformed into the saliency map of the original image. We iteratively accumulate the up-sampled activation maps and gradients, and finally compute their averages on lines 12 and 13. On line 14, we compute the saliency map by solving Eq. (2). Here, matrix notation is intentionally used to match the original equation.

In Algorithm 1, CAMERAS is shown to expect four input parameters, along with the classifier and the image. We discuss the choice of $\varphi(.)$ in § 3.4.1 where we eventually propose to keep this function fixed. The algorithm optionally allows $\kappa$ for computing saliency maps using layers other than the last convolutional layer of the model. For all the experiments presented in the main paper, we keep $\kappa$ fixed to the last convolutional layer. This is due to the well-known class-sensitivity of the deeper layers of CNNs [21]. Essentially, the only choice to be made is for the values of parameters $\zeta_{m}$ and $N$, which are related as $\zeta_{m}=cN\zeta_{o}$, where $c$ is the step size. Trading-off performance with efficiency, the choice of these parameters is mainly governed by the available computational resources. For larger $\zeta_{m}$ and $N$ values, performance of CAMERAS roughly improves monotonically - generally saturating at $\zeta_{m}\approx(1K,1K)$ for $\zeta_{o}=(224,224)$ for the popular ImageNet models. For this $\zeta_{m}$ range, the performance is largely insensitive for $N\in[5,10]$. We give further analysis of parameter values in the supplementary material. In the presented experiments, we empirically choose $\zeta_{m}=(1K,1K)$ and $N=7$.

In Fig. 2, we qualitatively compare the results of our technique with the existing saliency methods for ImageNet models of ResNet, DenseNet and Inception. Representative maps of randomly chosen images are provided. See the supplementary material for more visualisations. The high quality of CAMERAS maps is apparent in the figure. A quick inspection reveals that our technique maintains its performance across a variety of scenarios, including clear objects (Loudspeaker, Mountain tent), occluded objects (Bulbul), and multiples instances of objects (Accordion, Basket). Observing carefully, CAMERAS maps provide precise maps even for small and relatively complex geometric shapes, e.g. Volleyball, Spotlight, Loafer, Hognose snake. Interestingly, our method is able to attach appropriate importance even to the reflection of the Hognose snake. Adoption of saliency maps to complex geometric shapes is a direct consequence of enabling precise saliency mapping while sealing-off any external influence on the maps. CAMERAS is able to maintain its characteristic precision across different models and images. These are highly promising results for explainability of modern deep visual classifiers.

A quantum leap in performance with CAMERAS is also observed in our quantitative results. Saliency maps are typically evaluated by measuring their correlation with the semantic annotations of the image. Pointing game [34] is a popular metric for that purpose, which considers the computed saliency for every object class in the image. If the maximal point in the saliency map is contained within the object, it is considered a hit; otherwise, a miss. The performance is measured as the percentage of successful hits. We refer to [34] for more details on the metric. Table 1 benchmarks the performance of CAMERAS for pointing game on $4,952$ images of PASCAL VOC test set [7], and $\sim 50$K images of COCO 2014 validation set [16]. Our technique consistently shows superior performance, achieving up to $27.5\%$ error reduction. The gain is higher for ResNet as compared to VGG due the better performance of the original ResNet that permits better saliency.

The pointing game generally disregards precision of the saliency maps by focusing only on the maximal points. Arguably, crudeness of the saliency maps computed by the earlier methods influenced this evaluation metric. The possibility of precise saliency computation (by CAMERAS) calls for new metrics that account for the finer details of saliency maps. We propose ‘positive map density’ and ‘negative map density’ as two suitable metrics, respectively defined as: $\rho^{+}_{\text{map}}=P(\mathcal{K}(\boldsymbol{I}\odot\boldsymbol{\Psi}))/\sum_{i}\sum_{j}{\boldsymbol{\Psi}^{(i,j)}}\times(h\times w)$, and $\rho^{-}_{\text{map}}=P(\mathcal{K}(\boldsymbol{I}\odot\boldsymbol{1}-\boldsymbol{\Psi}))/\sum_{i}\sum_{j}{(\boldsymbol{1}-\boldsymbol{\Psi}^{(i,j)}})\times(h\times w)$. Here, $P(.)$ is the predicted probability of the actual label of the object. Other notations follow the conventions from above. For an estimated saliency map, the value of $\rho^{+}_{\text{map}}$ improves if higher importance is attached to a smaller number of pixels that retain higher confidence of the model on the original label. In the extreme case of all the pixels deemed maximally important (saliency value 1), the score depicts the model’s confidence on the object label. On the other hand, the value of $\rho^{-}_{\text{map}}$ decreases if lesser importance is attached by the saliency method to more pixels that do not influence the prediction confidence on the original label. Lower value of this metric is more desired.

Combined $\rho^{+}_{\text{map}}$ and $\rho^{-}_{\text{map}}$ provide a comprehensive quantification of the quality of the saliency map. We provide results of our technique, Grad-CAM [25] and the recent Norm-Grad [21] on our metrics in Table 2. Due to page limits, we provide further discussion on the proposed metrics in the supplementary material.

Gradient-based adversarial attacks algorithmically compute minimal perturbations to image pixels to maximally change the model prediction. This objective coincides with the objective of image saliency computation, thereby providing a natural sanity check for the saliency methods. That is, the effects of corruption with an adversarial perturbation to image pixels should correspond to the importance of the pixels identified by the saliency method. Leveraging this fact, we develop a sanity check for saliency methods that operates on pixel-by-pixel basis, which is more suited for precise image saliency. We provide details of the test in the supplementary material due to page limits.