Counterfactual Explanations via Latent Space Projection and Interpolation

For SharpShooter, crafting counterfactuals requires creating new samples from a learned data distribution. Generative models, which attempt to learn $p(X)$ are well suited for the task. Broadly speaking these approaches fall into two categories generative adversarial networks (GANs) Goodfellow et al. [2014] and normalizing flows Kobyzev et al. [2020], Rezende and Mohamed [2015]. Besides when used for improving the latent capacity of VAEs, both GANs and normalizing flows focus largely on generation (or $p(x))$, and less on the latent space of the hidden variables themselves. For instance, GANs learn to generate synthetic samples from noise while normalizing flows use a series of transforms and inverse transforms to learn relationships between hidden variables and the input.

To gain greater control over the latent space we use variational autoencoders (VAE) Kingma and Welling [2013]. Variational autoencoders have a similar encoder-decoder structure to autoencoders, however VAEs additionally learn a posterior distribution over latent codes through the use of variational inference Blei et al. [2017]. In practice VAEs are learned through maximizing a lower bound on $p(x)$ frequently known as the evidence lower bound (ELBO) Doersch [2021]. This formulation as a lower bound helps decompose the objective function into two parts: a reconstruction component related to the decoding network and a Kullback–Leibler divergence ($KL$) term that measures the asymmetric distance from the latent codes $p(Z|X)$ to the standard normal distribution of the prior $p(Z)$ in the standard VAE. The normalization of latent space in VAEs through the $KL$ divergence helps provide structure over the latent codes $p(Z|X)$. In SharpShooter, we use the $\beta$-VAE to encourage continuous latent code spaces Higgins et al. [2016].

Much of the work on local explanations for machine learned models has focused on local attributions, per sample feature importances and their aggregation Shrikumar et al. [2017], Ribeiro et al. [2016], Lundberg and Lee [2017], Datta et al. [2016], Ibrahim et al. [2019]. While useful for model developers, feature importances are less helpful in other contexts. The output and processes used to create these explanations are often difficult to understand for those not well versed with the background such as end-users.

A second branch of explainability is focused on counterfactual explanations; explanations that represent plausible changes for a user or observation to change their predicted classification. Counterfactual explanations are useful for multiple audiences, with most of the work centering on explanations for the people subject to a disadvantageous decision. Wachter et al Wachter et al. [2017] popularized the notion of using counterfactuals as way to provide recourse to data subjects in response to the EU General Data Protection Regulation (“GDPR”). They leverage the principle of "the closest possible world" in motivating their work. The notions of recourse Ustun et al. [2019] and fairness Joshi et al. [2019] are still major themes of research. This leaves a gap of providing counterfactual explanations for the audience that facilitates, mitigates and deliberates before a model gets deployed.

A good survey of available methods is available in Verma et al Verma et al. [2020]. Here we will frame much of that work through the lens of our use case, and highlight selected relevant work. We broadly separate the literature into two relevant branches, methods that involve input feature perturbations and those that use perturbations of hidden variables of generative or probabilistic models. We include comparisons to a method from each branch in our experiments in our experiments in Section 4.

One large branch of the literature creates counterfactuals by perturbing the input feature space. For instance, early work Lash et al. [2017] under the name of inverse classification maintained sparsity by partitioning the features into immutable and mutable features, and imposing budgetary constraints on allowable changes to the mutable features. Likewise Laugel et al Laugel et al. [2018] advocated a sampling approach with their growing spheres method to traverse the input space where Gomez et al Gomez et al. [2020] submitted a heuristic search method supported with visualisations to the FICO data challenge FICO [2018]. Their considerations of data distribution and feature importance, along with counterfactual explanations are in line with our current work.

Dhurandhar et al Dhurandhar et al. [2018] utilize gradient descent in input space to find contrastive explanations, which decompose into pertinent positives and pertinent negatives. They included an autoencoder loss to keep the explanations in sample. Van Looveren et al  Looveren and Klaise [2019] accelerate previous gradient descent methods with the introduction of prototypes, guiding the gradient descent towards the average value of the target class as determined by averaging the training sets representation in the latent space. Russell Russell [2019] improves on the stability of the optimization problem posed in Wachter et al. [2017] and extends beyond considering only categorical features Ustun et al. [2019] in posing a mixed integer programming formulation. Where Poyiadzi et al Poyiadzi et al. [2020] create counterfactuals by forming a network of feasible paths following high density regions of input space. GRACE Le et al. [2020] was designed for neural networks on tabular data that combines the concepts of contrastive explanations with interventions by performing constrained gradient descent adding an additional loss that is a measure of information gain to keep the resulting explanations sparse.

While intuitive in idea, input feature perturbation methods, without regularization, can generate counterfactuals unconvincing to the human eye that resemble traditional adversarial attacks Szegedy et al. [2013]. To address this shortcoming, latent space perturbations methods utilize the machinery of generative and probabilistic models to insure counterfactuals have high probability under the data distribution $p(X)$.

For instance ExplainGAN Samangouei et al. [2018], a method for finding counterfactual explanations for images, jointly trains multiple autoencoders using the signal from the classifier and discriminators to inform the learned representations. Their approach is focused on deep learning and computer vision tasks, while our approach is domain and model agnostic. Pawelczyk et al Pawelczyk et al. [2020] focus on tabular datasets by perform growing sphere perturbation in the latent space of a conditional variational autoencoder to generate counterfactuals. Where Balasubramanian et al Balasubramanian et al. [2020] and Joshi et al Joshi et al. [2019] directly perform gradient descent in the latent space of a variational autoencoder with various regularization terms. While other work focuses on topics of causality and counterfactuals within the context of generative models Mahajan et al. [2020].

Both latent space and input space methods differ substantially in the types of models and the datasets they can be used for, as well as the amount of model access they require (see Verma et al Verma et al. [2020] for a full list). By contrast SharpShooter is a model and data agnostic (although our primary focus is on tabular data).

The starting point of the SharpShooter algorithm is the target variational autoencoder (TVAE). We train the TVAE on the subset of the data that is labeled as the preferred target class. A sample from the base class ($x_{b}$) is passed through the TVAE to give a reconstruction $\hat{x}_{t}$. This process creates a projection of the base observation to the target class.

This raises an interesting question: how does an autoencoder make sense of data from a class it has never seen before? If the model has only ever seen the target class, how does it reconstruct a base class sample? Does it place these latent codes in an area of very low data distribution and the sample becomes unrecognizable as the target class? or does the model generate valid reconstructions?

We find that with the use of the proper prior, the TVAE will correctly produce an in-sample projection. While too strong of a prior will collapse the latent distribution, a prior with wider variance prevents the creation of viable counterfactuals. This weighting is closely determined by the $KL$ coefficient $\beta$. A higher $\beta$ biases the model away from purely optimizing reconstruction loss and ensures that codes still have a roughly normally distributed shape. This term ensures that even points with low mass in $p(x)$ have codes close to the prior distribution $p(z)$.

The limited capacity of amortized inference models (such as a VAE) to both reconstruct and perform inference is known as the amortization gap Cremer et al. [2018]. In prior contexts, the amortization gap has been seen as an obstacle to overcome, but the SharpShooter algorithm takes advantage of the lossy gap in inference when translating bases to target counterfactuals. If reconstruction and latent code inference becomes too direct then predictions on the unseen class stop classifying as the target class. We show this trade-off and investigate it in the appendix and in Figures 11-17.

We train the unified variational autoencoder (UVAE) on both classes. The encoder of the UVAE is used to embed both a base class sample, $z_{b}$, and its target class projection, $z_{t}$, in a common learned representation. We then use linear interpolation to sample along the line between a base sample and its TVAE-generated projection in UVAE space. We use the UVAE to estimate the joint data distribution and its latent space so that we can ensure counterfactuals both lie on the data manifold, and that they are close to their original sample in latent distance. While a powerful method that seems to succeed in our experiments on MNIST and the Lending Club datasets, there are additional difficulties building non-degenerate $\beta$-VAEs on the highly imbalanced UCI Credit dataset with its higher tabular dimensionality.

We examine the quality of counterfactuals by tabulating these metrics surveyed from literature: (i) time required to find a CF for a given sample (ii) validity (val), the percentage success in generating counterfactuals that cross the decision boundary (iii) proximity (prox), distance from input sample to counterfactual in latent space (iv) sparsity (spars), percentage of features changed (v) classifier shift - pass CF sample through UAE and check the change in classification score (vi) reconstruction loss - a measure of the CF being in sample. Further details about metrics and the classifiers used in the experiments can be found in the appendix.

For MNIST, we formulate the problem as a binary classification problem of predicting fours (our base class) and nines (our target class). While multiclass image counterfactuals are not the primary focus of SharpShooter, MNIST provides a naturally intuitive visualization of the impact of projecting from the base class to the target class. The counterfactual examples from SharpShooter shown in Figure 3 are exhibit a blend of features from the input (e.g. angle of strokes) and from the target class (e.g. rounder and more closed tops). Interestingly, all methods show struggles in interpreting the last out-of-sample image of a four.

The quality of counterfactual explanations as measured by the metrics discussed in the appendix are summarized in Table 1. These metrics show SharpShooter outperforming the other methods in validity, classifier shift, reconstruction score, and time, but lagging in proximity and sparsity.

Note that the thin layer of changed pixels in the counterfactuals for GDI in Figure 3 is reflected by the method’s poor performance in classifier shift and reconstruction score. The noisy transformations seen in those counterfactuals is not reproduced in UVAE reconstructions - leading to the lower performance in CS and RECON. This is why we introduce these two measures as metrics of realism of counterfactuals: the UVAE smooths out samples far from the underlying data distribution. Meanwhile, SharpShooter counterfactuals lie closer to the data distribution, and appear more like the target class in Figure 3.

Following new banking regulation in 1990 Wang [2021], Taiwan faced a credit card debt crisis. The UCI credit default dataset Yeh and hui Lien [2009], contains demographic, billing and payment information and labels of default from the period of April to September of 2005 while the crisis is deepening. The UCI credit dataset contains a high number of categorical variables. Complicating things further, the labels have a large class imbalance on the scale of 20 to 1 in favor of the non-defaulted class. A classifier is built to predict default with an AUC of 0.76 - one of the drawbacks of modeling an unsteady temporal process as steady, there is noise on the labels in the form of samples that have yet to default.

Due to these complications, training a standard VAE for either part of the SharpShooter process proves more difficult than training for the other datasets. The nature of $KL$ divergence attempts to form spherical latent distributions and is zero-avoidingDieng et al. [2019], while at the same time the reconstruction loss on the categorical variables encourages latent codes to be further apart. The tension caused by these competing forces can cause for either collapsed latent spaces or areas of low probability mass. This push and pull can be seen in the TSNE of UCI credit’s latent codes (see Figure 7) which show islands of mass, even within the same class type hinting at clusters in the latent space. The shape of the latent space for the TVAE and UVAE thus requires careful tuning parameters such as $\beta$ and categorical reconstruction weights.

The UCI credit dataset serves as a good counterexample necessitating the use of more expressive generational models with fundamentally different code distributions $p(z|x)$ for tabular datasets with higher categorical dimensionality.

Changes made by the three methods on the continuous variables of the dataset - presented in a scaled log value - can be seen in Figure 4. The SharpShooter changes in bill amount are in striking contrast to the gradient descent methods. While similar for April, a structural change is seen in May, and the later months suggest drastic reductions in bill amounts. The reductions suggested in payment amount correlate with lower bill amounts.

Likewise Figure 5 shows values for education, marital status, and pay timing. For categorical variables, SharpShooter shows a tendency to create counterfactuals where the payment is on time, and the marital status changes to married. On the other hand, GDI and GDL tend not to make many changes to categorical variables. This could be due to a relative inability of gradient descent methods to change categorical variables, a property not shared by the VAE-based SharpShooter.

Lending Club is a peer-to-peer lending company that would periodically release data on its loan portfolio. This tabular dataset included information on whether a borrower defaulted on their loan, the size of the loan, their annual income, debt to income ratio, FICO score, and the loan length. We train a classifier to predict default using five continuous and one categorical feature and obtains an AUC of 0.95. A summary of quality metrics for the methods is seen in Table 3.

For this experiment, the latent dimension of both autoencoders is two. In low dimensional space, the reconstruction loss performance of SharpShooter becomes very close to GDL. Otherwise Sharpshooter shows high validity and generates counterfactuals that are more sparse. Even though SharpShooter sees a larger classifier shift and reconstruction loss on this dataset, time is still much lower than the other two methods.

To illustrate these differences between methods the distributions of changes made for counterfactual explanations are shown in Figure 6. Given the latent search space was only 2, it is unsurprising that SharpShooter and GDL were not very large. The most notable difference between the two sets of counterfactuals was with respect to loan size, which was mostly unchanged in SharpShooter counterfactuals. Meanwhile, gradient descent in the input space makes the largest changes for variable like debt to income ratio (dti) and annual income.