This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

About a Proof Pearl: A Purported Solution to
a PoplMark Challenge Problem that Is Not One

Gopalan Nadathur
Abstract

The PoplMark Challenge comprises a set of problems intended to measure the strength of reasoning systems in the realm of mechanizing programming language meta-theory at the time the challenge was enunciated. Included in the collection is the exercise of demonstrating transitivity of subtyping for a specific algorithmic formulation of subtyping for an extension of System F. The challenge represented by this problem derives from the fact that, for the given formulation, subtyping must be proved simultaneously with another property called narrowing. In a paper published as a proof pearl, Brigitte Pientka claimed to have presented a solution to the problem in which “the full power of parametric and higher-order judgments” is exploited to “get the narrowing lemma for free.” We show this claim to be inaccurate. In particular, we show that the simplification is in substantial part the result of changing the formulation of the subtyping relation in a way that modifies the challenge rather than the outcome of the manner in which the argument is mechanized.

The PoplMark Challenge [1] identified a collection of benchmarks for measuring the status of reasoning frameworks at the time that it was presented from the perspective of mechanizing the meta-theory of programming languages. The problems in this collection are oriented around the typed λ\lambda-calculus with second-order polymorphism known as System F extended with subtyping and records. The formalization task that is of specific interest here is what is referred to in the mentioned paper as the transitivity of algorithmic subtyping. This problem, which was identified as Challenge 1A, was considered worthy of inclusion in the benchmarks because the proof of transitivity requires an intricate inductive argument in which an auxiliary lemma called the narrowing lemma must be established simultaneously. In [3], Brigitte Pientka claimed to have presented a solution in the Twelf system that “plays to the strengths of the logical framework LF” and that as a benefit gets the “tedious narrowing lemma, which must normally be proved separately, for free.” Unfortunately, what is really presented does not constitute a solution to Challenge 1A. Specifically, the “simplification” in the proof results not from the choice of encoding but from changing the formalization of subtyping to a form that comes close to including narrowing in a typing rule.

We provide substance to the above observations in the rest of this note. We first recall the original formalization task and then explain our comments about the “solution” presented in [3].

The formalization task

Challenge 1A relates solely to the type language that includes subtyping but does not encompass record types. The two syntactic entities that are relevant to describing the subtyping relation in this context are types and type environments. Expressions in these categories are given by the following grammar rules:

TypesT::=X|Top|TT|X<​:T.TType EnvironmentsΓ::=|Γ,X<​:​bT\begin{array}[]{rrcl}\mbox{\it Types}&\quad T&::=&X\ |\ \mbox{\tt Top}\ |\ T\rightarrow T\ |\ \forall X\,\mbox{\tt<\!:}\,T.T\\[3.0pt] \mbox{\it Type Environments}&\quad\Gamma&::=&\cdot\ |\ \Gamma,X\,\mbox{\tt<\!:\!}_{b}\,T\end{array}

The token XX represents variables in these rules. A type of the form X<​:T1.T2\forall X\,\mbox{\tt<\!:}\,T_{1}.T_{2} represents a binding of XX over the type T2T_{2}. Type environments bind variables that occur in types within their scope and they identify subtyping constraints for the variables so bound. Variables bound by an environment are assumed to be named uniquely and the subtyping rules implicitly permit the renaming of bound variables in types to ensure that this property holds. The presentation in [1] uses the same symbol for both <​: and <​:​b\mbox{\tt<\!:\!}_{b}. We avoid this overloading here to highlight issues that arise from the confusion of the two in [3].

SA-TopΓS<​:Top\Gamma\vdash S\,\mbox{\tt<\!:}\,\mbox{\tt Top} SA-Refl-TVarΓX<​:X\Gamma\vdash X\,\mbox{\tt<\!:}\,X
X<​:​bUΓΓU<​:TSA-Trans-TVarΓX<​:T\Gamma\vdash X\,\mbox{\tt<\!:}\,TX\,\mbox{\tt<\!:\!}_{b}\,U\in\Gamma\qquad\Gamma\vdash U\,\mbox{\tt<\!:}\,T ΓT1<​:S1ΓS2<​:T2SA-ArrowΓS1S2<​:T1T2\Gamma\vdash S_{1}\rightarrow S_{2}\,\mbox{\tt<\!:}\,T_{1}\rightarrow T_{2}\Gamma\vdash T_{1}\,\mbox{\tt<\!:}\,S_{1}\qquad\Gamma\vdash S_{2}\,\mbox{\tt<\!:}\,T_{2}
ΓT1<​:S1Γ,X<​:​bT1S2<​:T2SA-AllΓX<​:S1.S2<​:X<​:T1.T2\Gamma\vdash\forall X\,\mbox{\tt<\!:}\,S_{1}.S_{2}\,\mbox{\tt<\!:}\,\forall X\,\mbox{\tt<\!:}\,T_{1}.T_{2}\Gamma\vdash T_{1}\,\mbox{\tt<\!:}\,S_{1}\qquad\Gamma,X\,\mbox{\tt<\!:\!}_{b}\,T_{1}\vdash S_{2}\,\mbox{\tt<\!:}\,T_{2}
Figure 1: Rules Defining the Subtyping Relation

The subtyping relation that is of interest is written as ΓS<​:T\Gamma\vdash S\,\mbox{\tt<\!:}\,T, to be read as “SS is a subtype of TT relative to the type environment Γ\Gamma.” This relation is determined by the rules in Figure 1. The SA-Top rule has the requirement that SS must be well-scoped with respect to Γ\Gamma, i.e. all its free variables must be bound in Γ\Gamma. Similarly, the SA-Refl-TVar rule has the proviso that the variable XX must be bound in Γ\Gamma.111It can be shown that these assumptions suffice to ensure that ΓU<​:V\Gamma\vdash U\,\mbox{\tt<\!:}\,V is derivable only for types UU and VV that are well-scoped. It may be desirable to impose well-scoping requirements on Γ\Gamma as well, in which case the order in which bindings appear in it will be important. We discuss this observation further later in the note. This collection of rules has an algorithmic flavor in that it determines a process for checking a purported subtyping relation that is syntax-directed. The form of the SA-Trans-TVar rule is crucial to this property. Note that because of the form of this rule it is not automatically the case that ΓX<​:U\Gamma\vdash X\,\mbox{\tt<\!:}\,U is derivable if X<​:​bUX\,\mbox{\tt<\!:\!}_{b}\,U appears in Γ\Gamma. However, this turns out to be the case if UU is well-scoped with respect to Γ\Gamma because then ΓU<​:U\Gamma\vdash U\,\mbox{\tt<\!:}\,U is derivable.

Challenge 1A consists of providing a mechanized proof of the fact that the subtyping relation defined by the rules in Figure 1 is transitive. Specifically, a proof is to be provided for the following proposition within a chosen system for formal reasoning:

Prop 1.

If ΓS<​:Q\Gamma\vdash S\,\mbox{\tt<\!:}\,Q and ΓQ<​:T\Gamma\vdash Q\,\mbox{\tt<\!:}\,T are derivable then so is ΓS<​:T\Gamma\vdash S\,\mbox{\tt<\!:}\,T.

The difficulty in showing this property, and the reason why it is interesting as a challenge problem, is that it must be proved simultaneously with the following property that is called narrowing:

Prop 2.

If Γ1,X<​:​bQ,Γ2M<​:N\Gamma_{1},X\,\mbox{\tt<\!:\!}_{b}\,Q,\Gamma_{2}\vdash M\,\mbox{\tt<\!:}\,N and Γ1P<​:Q\Gamma_{1}\vdash P\,\mbox{\tt<\!:}\,Q are derivable then Γ1,X<​:​bP,Γ2M<​:N\Gamma_{1},X\,\mbox{\tt<\!:\!}_{b}\,P,\Gamma_{2}\vdash M\,\mbox{\tt<\!:}\,N is also derivable.

In the standard informal proof, the properties are established by an induction on the structure of QQ, with the first property being shown first and then used in showing the second. The proof of the first property requires an additional induction on the derivation of ΓS<​:Q\Gamma\vdash S\,\mbox{\tt<\!:}\,Q and that of the second property similarly requires an additional induction on the derivation of Γ1,X<​:​bQ,Γ2M<​:N\Gamma_{1},X\,\mbox{\tt<\!:\!}_{b}\,Q,\Gamma_{2}\vdash M\,\mbox{\tt<\!:}\,N.

While the description of the challenge allows for small changes to the subtyping rules, such as those needed to accommodate a higher-order abstract syntax style of formalization, it places a requirement that these such changes should result in an “obviously equivalent” system. As an example, mention is made in [1] of a declarative presentation of subtyping that explicitly include rules for reflexivity and transitivity. Such a presentation obviates a proof of transitivity of subtyping and thus constitutes another “nice example how the original problem specification influences the proofs about it” in the sense hinted at at the end of Section 2 in [3]. However, it also illustrates fairly starkly how changing the “problem specification” in such ways is not acceptable because it undermines the intended challenge.

The issues with the purported higher-order solution

The claimed solution to the challenge in [3] uses an encoding of the subtyping rules in LF that is based on a higher-order abstract syntax style. However, it is unnecessary to get into the details of this encoding to illuminate the difficulty with the claims therein. For simplicity of presentation, we will adhere to a presentation style that is close to that of the original description of the challenge. We note that this style is also used at the outset in [3].

The presentation of subtyping in [3] begins by confusing subtyping in bindings in type environments with subtyping judgements. In the description of the challenge in this note, we have highlighted the distinction between the two by deliberately using the symbol <​:​b\mbox{\tt<\!:\!}_{b} for one and <​: for the other. In keeping with the presentation in [1], the symbol <​: is used in [3] for both.222The symbol that is actually used in [3] is \leq but we will assume it to be <​: here to ensure consistency with the earlier discussion. However, unlike in [1] where a distinction is maintained despite the overloading of notation, the two notions are conflated in [3] by the inclusion of the following rule:

X<​:TΓSA-HypΓX<​:T\Gamma\vdash X\,\mbox{\tt<\!:}\,TX\,\mbox{\tt<\!:}\,T\in\Gamma

A further distinction between the subtyping rules in the original presentation and the one in [3] is that the latter builds in transitivity with respect to variable subtyping directly by replacing the SA-Trans-TVar rule with the following:

ΓX<​:UΓU<​:TSA-Tr-TVarΓX<​:T\Gamma\vdash X\,\mbox{\tt<\!:}\,T\Gamma\vdash X\,\mbox{\tt<\!:}\,U\qquad\Gamma\vdash U\,\mbox{\tt<\!:}\,T

The changes to the formulation of subtyping that are described above are substantial and do not satisfy the criterion of obvious equivalence required by the challenge. In particular, they considerably simplify the informal proof of transitivity of subtyping. Unlike with the original presentation of subtyping, the narrowing lemma can be proved independently of the transitivity property. Further, Prop 1, the transitivity property, can be proved by an induction only on QQ; the additional induction on the derivation of ΓS<​:Q\Gamma\vdash S\,\mbox{\tt<\!:}\,Q is not needed.333The development in [3] continues to use an induction on both but the second induction can be avoided.

In the above discussion, we have assumed that the narrowing lemma still needs to be proved. The claim in [3] is that this lemma is obtained for free. A closer look at the development shows that this “simplification” is actually the result of assuming the validity of a further rule related to subtyping that has the following form:

Γ1,X<​:U,Γ2X<​:VΓ1,X<​:V,Γ2M<​:NΓ1,X<​:U,Γ2M<​:N\Gamma_{1},X\,\mbox{\tt<\!:}\,U,\Gamma_{2}\vdash M\,\mbox{\tt<\!:}\,N\Gamma_{1},X\,\mbox{\tt<\!:}\,U,\Gamma_{2}\vdash X\,\mbox{\tt<\!:}\,V\qquad\Gamma_{1},X\,\mbox{\tt<\!:}\,V,\Gamma_{2}\vdash M\,\mbox{\tt<\!:}\,N

Observe that the sensibility of this rule depends on the conflation of subtyping in bindings in type environments with the subtyping judgement. While the narrowing lemma is an easy consequence of this rule in a situation where the SA-Tr-TVar rule is also present, the assumption of such a rule represents a significant deviation from the formulation of subtyping in Challenge 1A. We note that this rule can be shown to be admissible in the typing calculus of interest, but doing this requires an argument that is as complicated as the direct proof of narrowing. This observation raises in a different way the question of what it means to get something for free.

A comment on ordering of bindings in type environments

Typing calculi often attribute a significance to the order of bindings in contexts or type environments. In the discussions here, this sensitivity is manifest specifically in the statement of Prop 2. While order is important for well-scoping of type environments, the proofs of transitivity of subtyping and narrowing are not dependent on order. Moreover, ignoring order has the effect of simplifying the formalization of the arguments. It is possible to realize such an effect in a system like Abella [2], which treats contexts by default as multisets, by not imposing a well-scoping constraint initially on type environments. Once the desired properties have been proved when type environments have this less constrained form, it is easy to show that they must also hold when type environments are required to be well-scoped.

Acknowledgements

The observations in this note were made in the course of conducting research supported by the National Science Foundation under Grant No. CCF-1617771. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

References

  • [1] B. E. Aydemir, A. Bohannon, M. Fairbairn, J. N. Foster, B. C. Pierce, P. Sewell, D. Vytiniotis, G. Washburn, S. Weirich, and S. Zdancewic. Mechanized metatheory for the masses: The PoplMark challenge. In Theorem Proving in Higher Order Logics: 18th International Conference, number 3603 in LNCS, pages 50–65. Springer, 2005.
  • [2] D. Baelde, K. Chaudhuri, A. Gacek, D. Miller, G. Nadathur, A. Tiu, and Y. Wang. Abella: A system for reasoning about relational specifications. Journal of Formalized Reasoning, 7(2), 2014.
  • [3] B. Pientka. Proof pearl: The power of higher-order encodings in the logical framework LF. In K. Schneider and J. Brandt, editors, Proceedings of the 20th International Conference on Theorem Proving in Higher Order Logics (TPHOLs), volume 4732 of LNCS, pages 246–261. Springer, 2007.