Towards a Unified Framework For Interpretable Planning

Legibility of a behavior was originally formalized as the ability of the behavior to reveal its underlying objective. The usual scenario considered involves an observer who has access to a set of possible goals ($\mathbb{G}$) for the agent and is trying to identify the exact goal from the observed behavior. The generation of legible behavior is usually formalized as generating behavior that maximizes the probability of the original goal

$$\tau^{*}_{\mathcal{L}}=argmax_{\tau}P(g^{R}|\tau)$$

Where $g^{R}$ is the original goal of the agent. In general legibility can be generalized to implicit communication of any model parameter SZ:MZaamas20.

A behavior is assumed to be explicable if the behavior meets the user’s expectation for the agent in the given task. In the binary form this is usually taken to mean that the behavior is explicable if it is one of the plans the observer would have expected the agent to generate. In the continuous form this is usually translated as being proportional to the distance between the current trace and the closest expected behavior. Thus the problem of generating the most explicable behavior is usually formalized as

$$\tau^{*}_{\mathcal{E}}=\argmin_{\tau}\mathcal{D}(\tau,\tau^{E}_{\mathcal{M}^{R}_{h}})$$

Where $\mathcal{D}$ is some distance function and $\tau^{E}_{\mathcal{M}^{R}_{h}}$ is the expected behavior for the model $\mathcal{M}^{R}$. While there is no consensus on the distance function or expected behavior, a reasonable possibility for the expected set may be the set of optimal plans and the distance could be cost difference kulkarni2020designing.

This measure correspond to the user’s ability to complete a given behavior given the current behavior prefix. Now the goal of the agent is to choose a behavior prefix such that

$$\hat{\tau}^{*}_{\mathcal{P}}=argmax_{\hat{\tau}}P(\tau|\hat{\tau},\mathcal{M}^{R}_{h})$$

Where $P(\tau^{\prime}|\hat{\tau},\mathcal{M}^{R}_{h})$ is the probability assigned to a future behavior $\tau^{\prime}$ given the current observed prefix $\hat{\tau}$ under the model $\mathcal{M}^{R}_{h}$.

One might note that the first two measures were defined with respect to entire behavior traces while Predictability is defined with respect to prefix. This also points to the larger point that while one could meaningfully describe an offline notion of legibility and explicability (i.e. the score is assigned to the complete behavior trace), predictability only makes sense in an online settings (i.e. the observer is in the process of observing the agent in the middle of execution). Since our objective is to provide a unified discussion of all the metrics, we will limit ourselves to online versions of legibility and predictability. The descriptions provided above can easily be extended to such settings by considering prefixes (i.e. probability of goals/models given a prefix and the distance between current prefix and expected prefixes). As we will see in the next section internally this would require considering the most likely completions.

The legibility score of a behavior prefix for a specific model parameter can be defined to be directly proportional to the probability of the human’s belief in that parameter’s value in the true model, i.e.,

	$\displaystyle\mathcal{L}^{\theta}(\tau^{^{\prime}})\propto P(\Theta=\Theta(M^{R})|\hat{\tau}_{obs})$
	$\displaystyle\propto\Sigma_{\mathcal{M}\in\mathbb{M}\setminus\{\mathcal{M}^{0}\}~{}\textrm{Where}~{}\Theta(\mathcal{M}^{R})=\Theta(\mathcal{M})}P(\mathcal{M}|\hat{\tau}_{obs})$

While this may appear to be a direct generalization of the legibility descriptions previously discussed, there are some important points of departure. For one there is no assumption that the actual parameter being conveyed or the actual robot model is part of the hypothesis set being maintained by the user. Thus it is not always guaranteed that high legibility score can be guaranteed. Also note that the parameter is not tied to a single model in the set. Finally, the presence of $\mathcal{M}^{0}$ with non-zero prior distribution would affect what constitutes legible behavior. Since most of the earlier work had an explicit assumption that the observer is certain that the robot’s model is one of the few they are considering. This means in many cases existing legible behavior generation methods would create extremely circuitous route that is more likely in one model than others. For example in Figure 1, a legible planner might suggest the robot take a step towards the left to reveal its trying to deliver coffee, even though that correspond to an extremely suboptimal plan. In more realistic settings, where people aren’t so certain about possible robot models such behavior would leave them confused as opposed to help them understand the exact model. By explicitly incorporating $M^{0}$, we hope to capture such dynamics.

The explicability score of a behavior prefix can be defined to be directly proportional to the probability assigned to all model that are not $M^{0}$

$\displaystyle\mathcal{E}(\tau^{^{\prime}})\propto\Sigma_{\mathcal{M}\in\mathbb{M}\setminus\{\mathcal{M}^{0}\}}P(\mathcal{M}|\hat{\tau}_{obs})$

While at first this might look unconnected to the distance based formulation that appear in literature, we will go onto show that these formulations are in fact equivalent under reasonable assumptions. To start with let us consider a setting where the set of possible models at the user’s end consists of just $\mathcal{M}^{0}$ and another model $\mathcal{M}^{R}_{h}$. Moreover to keep ourselves in settings previously explored by previous explicability works, let us assume that we are dealing with offline settings. Such settings will be captured in our formulation by restricting the completions ($\tau^{^{\prime}}$). Now the above formulation would have the explicability score to be

	$\displaystyle\mathcal{E}(\tau^{^{\prime}})\propto P(\mathcal{M}^{R}_{h}|\hat{\tau}_{obs})$
	$\displaystyle\propto P(\hat{\tau}_{obs}|\mathcal{M}^{R}_{h})*P(\mathcal{M}^{R}_{h})$

Now given the fact that $\hat{\tau}_{obs}$ corresponds to the entire plan, $P(\hat{\tau}_{obs}|\mathcal{M}^{R}_{h})$ is the same as the likelyhood function described earlier, which gives us.

$\displaystyle\mathcal{E}(\tau^{^{\prime}})\propto P_{\mathcal{L}}(\mathcal{M}^{R}_{h},\hat{\tau}_{obs})*P(\mathcal{M}^{R}_{h})$

Now let us consider two plausible models for likelihood here. For the first one, let us assume the normative model, where the agent is expected to be optimal. Which would mean $P_{\mathcal{L}}(\mathcal{M}^{R}_{h},\hat{\tau}_{obs})$ is either $1$ thus leading to high explicability or $0$ thus not explicable. This brings us to the original binary explicability formulation used in works like balancing; chakraborti2019explicability etc. Now another possible likelihood function could be to use a noisy rational model. Such distributions have been previously considered for works like fisac2020generating. Here the likelihood function would be given as

$$P_{\mathcal{L}}(\mathcal{M}^{R}_{h},\hat{\tau}_{obs})\propto e^{C*-C(\hat{\tau}_{obs})}$$

Where $C*$ is the cost of the best possible behavior. This would now again map the formulation to the distance based formulation for explicability where the distance is defined on the cost kulkarni2020designing. Regardless of the specific formulation, in the end explicability is a measure that is meant to capture the user’s understanding of the robot model. Earlier formulations rely on using the space of expected plans as a stand-in for the model itself. This is further supported by the fact that works that have looked at updating user’s perceived explicability value of a plan do it by providing information about the model and not by directly modifying the users understanding of the expected set of plans. Now moving onto the more general settings, where we have now multiple possible models. Notice that if we have models with the same prior belief, then the formulation would make no difference between plans that work in both models equally well versus one that works in individual models, which would mirror our intuitions about behavior in such scenarios.

The final measure we are interested in capturing in this formulation is predictability. Which we could be captured as

$\displaystyle\mathcal{P}(\tau^{^{\prime}})\propto P(\tau^{\prime}=\tau|\hat{\tau}_{obs})$

Where $tau$ is the completion considered by the agent. This is more or less a direct translation of the predictability measure to this more general setting (for example in earlier works like fisac2020generating generally you consider a single possible model). One interesting point to note here is predictability only optimizes for the probability of the current completion, which allows for the system to choose unlikely prefixes for cases where the agent is only required to achieve required levels of predictability after a few steps.