This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

11institutetext: 11email: [email protected]

Partial Quantifier Elimination And Property Generation

Eugene Goldberg
Abstract

We study partial quantifier elimination (PQE) for propositional CNF formulas with existential quantifiers. PQE is a generalization of quantifier elimination where one can limit the set of clauses taken out of the scope of quantifiers to a small subset of clauses. The appeal of PQE is that many verification problems (e.g., equivalence checking and model checking) can be solved in terms of PQE and the latter can be dramatically simpler than full quantifier elimination. We show that PQE can be used for property generation that can be viewed as a generalization of testing. The objective here is to produce an unwanted property of a design implementation thus exposing a bug. We introduce two PQE solvers called ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}. ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is a very simple SAT-based algorithm. ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} is more sophisticated and robust than ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}. We use these PQE solvers to find an unwanted property (namely, an unwanted invariant) of a buggy FIFO buffer. We also apply them to invariant generation for sequential circuits from a HWMCC benchmark set. Finally, we use these solvers to generate properties of a combinational circuit that mimic symbolic simulation.

1 Introduction

In this paper, we consider the following problem. Let Fโ€‹(X,Y)F(X,Y) be a propositional formula in conjunctive normal form (CNF)111Every formula is a propositional CNF formula unless otherwise stated. Given a CNF formula FF represented as the conjunction of clauses C1โˆงโ‹ฏโˆงCkC_{1}\wedge\dots\wedge C_{k}, we will also consider FF as the set of clauses {C1,โ€ฆ,Ck}\{C_{1},\dots,C_{k}\}. where X,YX,Y are sets of variables. Let GG be a subset of clauses of FF. Given a formula โˆƒXโ€‹[F]\exists{X}[F], find a quantifier-free formula Hโ€‹(Y)H(Y) such that โˆƒXโ€‹[F]โ‰กHโˆงโˆƒXโ€‹[Fโˆ–G]\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus G]$}. In contrast to full quantifier elimination (QE), only the clauses of GG are taken out of the scope of quantifiers here. So, we call this problem partial QE (PQE)ย [1]. (In this paper, we consider PQE only for formulas with existential quantifiers.) We will refer to HH as a solution to PQE. Like SAT, PQE is a way to cope with the complexity of QE. But in contrast to SAT that is a special case of QE (where all variables are quantified), PQE generalizes QE. The latter is just a special case of PQE where G=FG=F and the entire formula is unquantified. Interpolationย [2, 3] is a special case of PQE as wellย [4] (see also Appendixย 0.A).

The appeal of PQE is threefold. First, it can be much more efficient than QE if GG is a small subset of FF. Second, many verification problems like SAT, equivalence checking, model checking can be solved in terms of PQEย [1, 5, 6, 7]. So, PQE can be used to design new efficient methods for solving known problems. Third, one can apply PQE to solving new problems like property generation considered in this paper. In practice, to perform PQE, it suffices to have an algorithm that takes a single clause out of the scope of quantifiers. Namely, given a formula โˆƒXโ€‹[Fโ€‹(X,Y)]\exists{X}[F(X,Y)] and a clause CโˆˆFC\in F, this algorithm finds a formula Hโ€‹(Y)H(Y) such that โˆƒXโ€‹[F]โ‰กHโˆงโˆƒXโ€‹[Fโˆ–{C}]\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus\mbox{$\{C\}$}]$}. To take out kk clauses, one can apply this algorithm kk times. Since HโˆงโˆƒXโ€‹[F]โ‰กHโˆงโˆƒXโ€‹[Fโˆ–{C}]H\wedge\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus\mbox{$\{C\}$}]$}, solving the PQE above reduces to finding Hโ€‹(Y)H(Y) that makes CC redundant in HโˆงโˆƒXโ€‹[F]H\wedge\mbox{$\exists{X}[F]$}. So, the PQE algorithms we present here employ redundancy based reasoning. Namely, we describe two PQE algorithms called ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} where โ€œEGโ€ stands for โ€œEnumerate and Generalizeโ€. ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is a very simple SAT-based algorithm that can sometimes solve very large problems. ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} is a modification of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} that makes the algorithm more powerful and robust.

Inย [6], we showed the viability of an equivalence checker based on PQE. In particular, we presented instances for which this equivalence checker outperformed ABCย [8], a high quality tool. In this paper, we describe and check experimentally one more important application of PQE called property generation. Our motivation here is as follows. Suppose a design implementation Imp meets the set of specification properties P1,โ€ฆ,PmP_{1},\dots,P_{m}. Typically, this set is incomplete. So, Imp can still be buggy even if every Pi,i=1,โ€ฆ,mP_{i},i=1,\dots,m holds. Let Pm+1โˆ—,โ€ฆ,Pnโˆ—P^{*}_{m+1},\dots,P^{*}_{n} be desired properties adding which makes the specification complete. If Imp meets the properties P1,โ€ฆ,PmP_{1},\dots,P_{m} but is still buggy, a missed property Piโˆ—P^{*}_{i} above fails. That is, Imp has the unwanted property Piโˆ—ยฏ\overline{P^{*}_{i}}. So, one can detect bugs by generating unspecified properties of Imp and checking if there is an unwanted one.

Currently, identification of unwanted properties is mostly done by massive testing. (As we show later, the input/output behavior specified by a single test can be cast as a simple property of Imp.) Another technique employed in practice is guessing a list of unwanted properties that may hold and formally checking if this is the case. The problem with these techniques is that they can miss an unwanted property. In this paper, we describe property generation by PQE. The benefit of PQE is that it can produce much more complex properties than those corresponding to single tests. So, using PQE one can detect bugs that testing overlooks or cannot find in principle. Importantly, PQE generates properties covering different parts of Imp. This makes the search for unwanted properties more systematic and facilitates discovering bugs that can be missed if one simply guesses unwanted properties that may hold.

In this paper, we experimentally study generation of invariants of a sequential circuit NN. An invariant of NN is unwanted if a state that is supposed to be reachable in NN falsifies this invariant and hence is unreachable. Note that finding a formal proof that NN has no unwanted invariants is impractical. (It is hard to efficiently prove a large set of states reachable because different states are reached by different execution traces.) So developing practical methods for finding unwanted invariants if very important. We also study generation of properties mimicking symbolic simulation for a combinational circuit obtained by unrolling a sequential circuit. An unwanted property here exposes a wrong execution trace.

The main body of this paper is structured as follows. (Some additional information can be found in appendices.) In Sectionย 2, we give basic definitions. Sectionย 3 presents property generation for a combinational circuit. In Sectionย 4, we describe invariant generation for a sequential circuit. Sectionsย 5 andย 6 present ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} respectively. In Sectionย 7, invariant generation is used to find a bug in a FIFO buffer. Experiments with invariant generation for HWMCC benchmarks are described in Sectionย 8. Sectionย 9 presents an experiment with property generation for combinational circuits. In Sectionย 10 we give some background. Finally, in Sectionย 11, we make conclusions and discuss directions for future research.

2 Basic Definitions

In this section, when we say โ€œformulaโ€ without mentioning quantifiers, we mean โ€œa quantifier-free formulaโ€.

Definition 1

We assume that formulas have only Boolean variables. A literal of a variable vv is either vv or its negation. A clause is a disjunction of literals. A formula FF is in conjunctive normal form (CNF) if F=C1โˆงโ‹ฏโˆงCkF=C_{1}\wedge\dots\wedge C_{k} where C1,โ€ฆ,CkC_{1},\dots,C_{k} are clauses. We will also view FF as the set of clauses {C1,โ€ฆ,Ck}\{C_{1},\dots,C_{k}\}. We assume that every formula is in CNF.

Definition 2

Let FF be a formula. Then ๐‘ฝ๐’‚๐’“๐’”โ€‹(๐‘ญ)\mathit{Vars}(F) denotes the set of variables of FF and ๐‘ฝ๐’‚๐’“๐’”โ€‹(โˆƒ๐‘ฟโ€‹[๐‘ญ])\mathit{Vars}(\mbox{$\exists{X}[F]$}) denotes ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(F)โˆ–X\mbox{$\mathit{Vars}(F)$}\!\setminus\!X.

Definition 3

Let VV be a set of variables. An assignment \vvโ€‹q\vv{q} to VV is a mapping Vโ€ฒโ†’{0,1}V^{\prime}~{}\rightarrow\mbox{$\{0,1\}$} where Vโ€ฒโІVV^{\prime}\subseteq V. We will denote the set of variables assigned in \vvโ€‹q\vv{q}ย ย as ๐‘ฝ๐’‚๐’“๐’”โ€‹(๐’’โ†’)\mathit{Vars}(\vec{q}). We will refer to \vvโ€‹q\vv{q} as a full assignment to VV if ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(qโ†’)=V\mbox{$\mathit{Vars}(\vec{q})$}=V. We will denote as \vvโ€‹๐’’โІ\vvโ€‹๐’“\mbox{$\vv{q}$}\subseteq\mbox{$\vv{r}$} the fact that a) ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(qโ†’)โІ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(rโ†’)\mbox{$\mathit{Vars}(\vec{q})$}\subseteq\mbox{$\mathit{Vars}(\vec{r})$} and b) every variable of ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(qโ†’)\mathit{Vars}(\vec{q}) has the same value in \vvโ€‹q\vv{q} and \vvโ€‹r\vv{r}.

Definition 4

A literal, a clause, and a formula are said to be satisfied (respectively falsified) by an assignment \vvโ€‹q\vv{q} if they evaluate to 1 (respectively 0) under \vvโ€‹q\vv{q}.

Definition 5

Let CC be a clause. Let HH be a formula that may have quantifiers, and \vvโ€‹q\vv{q} be an assignment to ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(H)\mathit{Vars}(H). If CC is satisfied by \vvโ€‹q\vv{q}, then ๐‘ช๐’’โ†’โ‰ก๐Ÿ\mbox{$C_{\vec{q}}$}\equiv 1. Otherwise, ๐‘ช๐’’โ†’C_{\vec{q}} is the clause obtained from CC by removing all literals falsified by \vvโ€‹q\vv{q}. Denote by ๐‘ฏ๐’’โ†’H_{\vec{q}} the formula obtained from HH by removing the clauses satisfied by \vvโ€‹q\vv{q} and replacing every clause CC unsatisfied by \vvโ€‹q\vv{q} with Cqโ†’C_{\vec{q}}.

Definition 6

Given a formula โˆƒXโ€‹[Fโ€‹(X,Y)]\exists{X}[F(X,Y)], a clause CC of FF is called a quantified clause if ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(C)\mathit{Vars}(C) โˆฉXโ‰ โˆ…\cap~{}X~{}\neq~{}\emptyset. If ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(C)โˆฉX=โˆ…\mbox{$\mathit{Vars}(C)$}\cap X=\emptyset, the clause CC depends only on free, i.e., unquantified variables ofโ€‰ FF and is called a free clause.

Definition 7

Let G,HG,H be formulas that may have existential quantifiers. We say that G,HG,H are equivalent, written ๐‘ฎโ‰ก๐‘ฏG\equiv H, if Gqโ†’=Hqโ†’\mbox{$G_{\vec{q}}$}=\mbox{$H_{\vec{q}}$} for all full assignments \vvโ€‹q\vv{q} to ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(G)โˆช๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(H)\mbox{$\mathit{Vars}(G)$}\cup\mbox{$\mathit{Vars}(H)$}.

Definition 8

Let Fโ€‹(X,Y)F(X,Y) be a formula and GโІFG\subseteq F and Gโ‰ โˆ…G\neq\emptyset. The clauses of GG are said to be redundant in โˆƒ๐‘ฟโ€‹[๐‘ญ]\exists{X}[F] if โˆƒXโ€‹[F]โ‰กโˆƒXโ€‹[Fโˆ–G]\mbox{$\exists{X}[F]$}\equiv\mbox{$\exists{X}[F\setminus G]$}. Note that if Fโˆ–GF\setminus G implies GG, the clauses of GG are redundant in โˆƒXโ€‹[F]\exists{X}[F].

Definition 9

Given a formula โˆƒX[F(X,Y))]\exists{X}[F(X,Y))] and GG where GโІFG\subseteq F, the Partial Quantifier Elimination (PQE) problem is to find Hโ€‹(Y)H(Y) such thatโˆƒ๐‘ฟโ€‹[๐‘ญ]โ‰ก๐‘ฏโˆงโˆƒ๐‘ฟโ€‹[๐‘ญโˆ–๐‘ฎ]\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus G]$}. (So, PQE takes GG out of the scope of quantifiers.) The formula HH is called a solution to PQE. The case of PQE where G=FG=F is called Quantifier Elimination (QE).

Example 1

Consider the formula F=C1โˆงC2โˆงC3โˆงC4F=C_{1}\wedge C_{2}\wedge C_{3}\wedge C_{4} where C1=xยฏ3โˆจx4C_{1}=\overline{x}_{3}\vee x_{4}, C2=y1โˆจx3C_{2}\!=\!y_{1}\!\vee\!x_{3}, C3=y1โˆจxยฏ4C_{3}=y_{1}\vee\overline{x}_{4}, C4=y2โˆจx4C_{4}\!=\!y_{2}\!\vee\!x_{4}. Let YY denote {y1,y2}\{y_{1},y_{2}\} and XX denote {x3,x4}\{x_{3},x_{4}\}. Consider the PQE problem of taking C1C_{1} out of โˆƒXโ€‹[F]\exists{X}[F], i.e., finding Hโ€‹(Y)H(Y) such that โˆƒXโ€‹[F]โ‰กHโˆงโˆƒXโ€‹[Fโˆ–{C1}]\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus\mbox{$\{C_{1}\}$}]$}. As we show later, โˆƒXโ€‹[F]โ‰กy1โˆงโˆƒXโ€‹[Fโˆ–{C1}]\mbox{$\exists{X}[F]$}\equiv y_{1}\wedge\mbox{$\exists{X}[F\setminus\mbox{$\{C_{1}\}$}]$}. That is, H=y1H\!=\!y_{1} is a solution to the PQE problem above.

Remark 1

Let DD be a clause of a solution HH to the PQE problem of Definitionย 9. If Fโˆ–GF\setminus G implies DD, then Hโˆ–{D}H\setminus\mbox{$\{D\}$} is a solution to this PQE problem too.

Proposition 1

Let HH be a solution to the PQE problem of Definitionย 9. That is, โˆƒXโ€‹[F]โ‰กHโˆงโˆƒXโ€‹[Fโˆ–G]\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus G]$}. Then Fโ‡’HF\Rightarrow H (i.e., FF implies HH).

The proofs of propositions are given in Appendixย 0.B.

Definition 10

Let clauses Cโ€ฒC^{\prime},Cโ€ฒโ€ฒC^{\prime\prime} have opposite literals of exactly one variable wโˆˆ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(Cโ€ฒ)โˆฉ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(Cโ€ฒโ€ฒ)w\!\in\!\mbox{$\mathit{Vars}(C^{\prime})$}\!\cap\!\mbox{$\mathit{Vars}(C^{\prime\prime})$}. Then Cโ€ฒC^{\prime},Cโ€ฒโ€ฒC^{\prime\prime} are called resolvable onย ww. The clause CC having all literals of Cโ€ฒ,Cโ€ฒโ€ฒC^{\prime},C^{\prime\prime} but those of ww is called the resolvent of Cโ€ฒC^{\prime},Cโ€ฒโ€ฒC^{\prime\prime}. The clause CC is said to be obtained by resolution on ww.

Definition 11

Let CC be a clause of a formula GG and wโˆˆ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(C)w\in\mbox{$\mathit{Vars}(C)$}. The clause CC is said to be blockedย [9] in GG with respect to the variable ww if no clause of GG is resolvable with CC on ww.

Proposition 2

Let a clause CC be blocked in a formula Fโ€‹(X,Y)F(X,Y) with respect to a variable xโˆˆXx\in X. Then CC is redundant in โˆƒXโ€‹[F]\exists{X}[F], i.e., โˆƒXโ€‹[Fโˆ–{C}]\exists{X}[F\setminus\mbox{$\{C\}$}] โ‰ก\equiv โˆƒXโ€‹[F]\exists{X}[F].

3 Property Generation By PQE

Many known problems can be formulated in terms of PQE, thus facilitating the design of new efficient algorithms. In Appendixย 0.C, we recall some results on solving SAT, equivalence checking and model checking by PQE presented in ย [1, 5, 6, 7]. In this section, we describe application of PQE to property generation for a combinational circuit. The objective of property generation is to expose a bug via producing an unwanted property.

Let Mโ€‹(X,V,W)M(X,V,W) be a combinational circuit where X,V,WX,V,W specify the sets of the internal, input, and output variables of MM respectively. Let Fโ€‹(X,V,W)F(X,V,W) denote a formula specifying MM. As usual, this formula is obtained by Tseitinโ€™s transformationsย [10]. Namely, FF equals FG1โˆงโ‹ฏโˆงFGkF_{G_{1}}\wedge\dots\wedge F_{G_{k}} where G1,โ€ฆ,GkG_{1},\dots,G_{k} are the gates of MM and FGiF_{G_{i}} specifies the functionality of gate GiG_{i}.

Example 2

Let GG be a 2-input AND gate defined as x3=x1โˆงx2x_{3}=x_{1}\wedge x_{2} where x3x_{3} denotes the output value and x1,x2x_{1},x_{2} denote the input values of GG. Then GG is specified by the formula FG=(xยฏ1โˆจxยฏ2โˆจx3)โˆง(x1โˆจxยฏ3)โˆง(x2โˆจxยฏ3)F_{G}\!=\!(\overline{x}_{1}\vee\overline{x}_{2}\vee x_{3})\wedge(x_{1}\vee\overline{x}_{3})\wedge(x_{2}\vee\overline{x}_{3}). Every clause of FGF_{G} is falsified by an inconsistent assignment (where the output value of GG is not implied by its input values). For instance, x1โˆจxยฏ3x_{1}\!\vee\overline{x}_{3} is falsified by the inconsistent assignment x1=0,x3=1x_{1}\!=\!0,x_{3}\!=\!1. So, every assignment satisfying FGF_{G} corresponds to a consistent assignment to GG and vice versa. Similarly, every assignment satisfying the formula FF above is a consistent assignment to the gates of MM and vice versa.

3.1 High-level view of property generation by PQE

One generates properties by PQE until an unwanted property exposing a bug is produced. (Like in testing one runs tests until a bug-exposing test is encountered.) The benefit of property generation by PQE is fourfold. First, by property generation one can identify bugs that are hard or simply impossible to find by testing. Second, using PQE makes property generation efficient. Third, by taking out different clauses one can generate properties covering different parts of the design. This increases the probability of discovering a bug. Fourth, every property generated by PQE specifies a large set of high-quality tests.

In this paper (Sectionsย 7,ย 9), we consider cases where identifying an unwanted property is easy. However, in general, such identification is not trivial. A detailed discussion of this topic is beyond the scope of this paper. (Nevertheless, in Appendixย 0.D, we give the main idea of the procedure for deciding if a property is unwanted.)

3.2 Property generation as generalization of testing

The behavior of MM corresponding to a single test can be cast as a property. Let wiโˆˆWw_{i}\in W be an output variable of MM and \vvโ€‹v\vv{v} be a test, i.e., a full assignment to the input variables VV of MM. Let Bvโ†’B^{\vec{v}} denote the longest clause falsified by \vvโ€‹v\vv{v}, i.e., ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(Bvโ†’)=V\mbox{$\mathit{Vars}(\mbox{$B^{\vec{v}}$})$}=V. Let lโ€‹(wi)l(w_{i}) be the literal satisfied by the value of wiw_{i} produced by MM under input \vvโ€‹v\vv{v}. Then the clause Bvโ†’โˆจlโ€‹(wi)\mbox{$B^{\vec{v}}$}\vee l(w_{i}) is satisfied by every assignment satisfying FF, i.e., Bvโ†’โˆจlโ€‹(wi)\mbox{$B^{\vec{v}}$}\vee l(w_{i}) is a property of MM. We will refer to it as a single-test property (since it describes the behavior of MM for a single test). If the input \vvโ€‹v\vv{v} is supposed to produce the opposite value of wiw_{i} (i.e., the one falsifying lโ€‹(wi)l(w_{i})), then \vvโ€‹v\vv{v} exposes a bug in MM. In this case, the single-test property above is an unwanted property of MM exposing the same bug as the test \vvโ€‹v\vv{v}.

A single-test property can be viewed as a weakest property of MM as opposed to the strongest property specified by โˆƒXโ€‹[F]\exists{X}[F]. The latter is the truth table of MM that can be computed explicitly by performing QE on โˆƒXโ€‹[F]\exists{X}[F]. One can use PQE to generate properties of MM that, in terms of strength, range from the weakest ones to the strongest property inclusively. (By combining clause splitting with PQE one can generate single-test properties, see the next subsection.) Consider the PQE problem of taking a clause CC out of โˆƒXโ€‹[F]\exists{X}[F]. Let Hโ€‹(V,W)H(V,W) be a solution to this problem, i.e., โˆƒXโ€‹[F]โ‰กHโˆงโˆƒXโ€‹[Fโˆ–{C}]\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus\mbox{$\{C\}$}]$}. Since HH is implied by FF, it can be viewed as a property of MM. If HH is an unwanted property, MM has a bug. (Here we consider the case where a property of MM is obtained by taking a clause out of formula โˆƒXโ€‹[F]\exists{X}[F] where only the internal variables of MM are quantified. Later we consider cases where some external variables of MM are quantified too.)

We will assume that the property HH generated by PQE has no redundant clauses (see Remarkย 1). That is, if DโˆˆHD\in H, then Fโˆ–{C}โ‡’ฬธDF\setminus\mbox{$\{C\}$}\not\Rightarrow D. Then one can view HH as a property that holds due to the presence of the clause CC in FF.

3.3 Computing properties efficiently

If a property HH is obtained by taking only one clause out of โˆƒXโ€‹[F]\exists{X}[F], its computation is much easier than performing QE on โˆƒXโ€‹[F]\exists{X}[F]. If computing HH still remains too time-consuming, one can use the two methods below that achieve better performance at the expense of generating weaker properties. The first method applies when a PQE solver forms a solution incrementally, clause by clause (like the algorithms described in Sectionsย 5 andย 6). Then one can simply stop computing HH as soon as the number of clauses in HH exceeds a threshold. Such a formula HH is still implied by FF and hence specifies a property of MM.

The second method employs clause splitting. Here we consider clause splitting on input variables v1,โ€ฆ,vpv_{1},\dots,v_{p}, i.e., those of VV (but one can split a clause on any subset of variables from ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(F)\mathit{Vars}(F)). Let Fโ€ฒF^{\prime} denote the formula FF where a clause CC is replaced with p+1p+1 clauses: C1=Cโˆจlโ€‹(v1)ยฏC_{1}=C\vee\overline{l(v_{1})},โ€ฆ, Cp=Cโˆจlโ€‹(vp)ยฏC_{p}=C\vee\overline{l(v_{p})}, Cp+1=Cโˆจlโ€‹(v1)โˆจโ‹ฏโˆจlโ€‹(vp)C_{p+1}=C\vee l(v_{1})\vee\dots\vee l(v_{p}), where lโ€‹(vi)l(v_{i}) is a literal of viv_{i}. The idea is to obtain a property HH by taking the clause Cp+1C_{p+1} out of โˆƒXโ€‹[Fโ€ฒ]\exists{X}[F^{\prime}] rather than CC out of โˆƒXโ€‹[F]\exists{X}[F]. The former PQE problem is simpler than the latter since it produces a weaker property HH. One can show that if {v1,โ€ฆ,vp}=V\mbox{$\{v_{1},\dots,v_{p}\}$}\!=\!V, then a) the complexity of PQE reduces to linear; b) taking out Cp+1C_{p+1} actually produces a single-test property. The latter specifies the input/output behavior of MM for the test \vvโ€‹v\vv{v} falsifying the literals lโ€‹(v1),โ€ฆ,lโ€‹(vp)l(v_{1}),\dots,l(v_{p}). (See Appendixย 0.E for more details.)

3.4 Using design coverage for generation of unwanted properties

Arguably, testing is so effective in practice because one verifies a particular design. Namely, one probes different parts of this design using some coverage metric rather than sampling the truth table (which would mean verifying every possible design). The same idea works for property generation by PQE for the following two reasons. First, by taking out a clause, PQE generates a property inherent to the specific circuit MM. (If one replaces MM with an equivalent but structurally different circuit, PQE will generate different properties.) Second, by taking out different clauses of FF one generates properties corresponding to different parts of MM thus โ€œcoveringโ€ the design. This increases the chance to take out a clause corresponding to the buggy part of MM and generate an unwanted property.

3.5 High-quality tests specified by a property generated by PQE

In this subsection, we show that a property HH generated by PQE, in general, specifies a large set of high-quality tests. Let Hโ€‹(V,W)H(V,W) be obtained by taking CC out of โˆƒXโ€‹[Fโ€‹(X,V,W)]\exists{X}[F(X,V,W)]. Let Qโ€‹(V,W)Q(V,W) be a clause of HH. As mentioned above, we assume that Fโˆ–{C}โ‡’ฬธQF\setminus\mbox{$\{C\}$}\not\Rightarrow Q. Then there is an assignment (\vvโ€‹x\vv{x},\vvโ€‹v\vv{v},\vvโ€‹w\vv{w}) satisfying formula (Fโˆ–{C})โˆงQยฏ(F\setminus\mbox{$\{C\}$})\wedge\overline{Q} where \vvโ€‹x\vv{x},\vvโ€‹v\vv{v},\vvโ€‹w\vv{w} are assignments to X,V,WX,V,W respectively. (Note that by definition, (\vvโ€‹v\vv{v},\vvโ€‹w\vv{w}) falsifies QQ.) Let (\vvโ€‹xโˆ—,\vvโ€‹v,\vvโ€‹wโˆ—)(\mbox{$\vv{x}$}^{*},\mbox{$\vv{v}$},\mbox{$\vv{w}$}^{*}) be the execution trace of MM under the input \vvโ€‹v\vv{v}. So, (\vvโ€‹xโˆ—,\vvโ€‹v,\vvโ€‹wโˆ—)(\mbox{$\vv{x}$}^{*},\mbox{$\vv{v}$},\mbox{$\vv{w}$}^{*}) satisfies FF. Note that the output assignments \vvโ€‹w\vv{w} and \vvโ€‹wโˆ—\mbox{$\vv{w}$}^{*} must be different because (\vvโ€‹v,\vvโ€‹wโˆ—)(\mbox{$\vv{v}$},\mbox{$\vv{w}$}^{*}) has to satisfy QQ. (Otherwise, (\vvโ€‹xโˆ—,\vvโ€‹v,\vvโ€‹wโˆ—)(\mbox{$\vv{x}$}^{*},\mbox{$\vv{v}$},\mbox{$\vv{w}$}^{*}) satisfies FโˆงQยฏF\wedge\overline{Q} and so Fโ‡’ฬธQF\not\Rightarrow Q and hence Fโ‡’ฬธHF\not\Rightarrow H.) So, one can view \vvโ€‹v\vv{v} as a test โ€œdetectingโ€ disappearance of the clause CC from FF. Note that different assignments satisfying (Fโˆ–{C})โˆงQยฏ(F\setminus\mbox{$\{C\}$})\wedge\overline{Q} correspond to different tests \vvโ€‹v\vv{v}. So, the clause QQ of HH, in general, specifies a very large number of tests. One can show that these tests are similar to those detecting stuck-at faults and so have very high quality (see Appendixย 0.F for more details).

4 Invariant Generation By PQE

In this section, we extend property generation for combinational circuits to sequential ones. Namely, we generate invariants. Note that generation of desired auxiliary invariants is routinely used in practice to facilitate verification of a predefined property. The problem we consider here is different in that our goal is to produce an unwanted invariant. We picked generation of invariants (over that of weaker properties just claiming that a state cannot be reached in kk transitions or less) because identification of an unwanted invariant is, arguably, easier. This simplifies bug detection by property generation.

4.1 Bugs making states unreachable

Let NN be a sequential circuit and SS denote the state variables of NN. Let Iโ€‹(S)I(S) specify the initial state \vvโ€‹s๐‘–๐‘›๐‘–\vv{s}\!_{\mathit{ini}} (i.e., Iโ€‹(\vvโ€‹s๐‘–๐‘›๐‘–)=1I(\mbox{$\vv{s}\!_{\mathit{ini}}$})\!=\!1). Let Tโ€‹(Sโ€ฒ,V,Sโ€ฒโ€ฒ)T(S^{\prime},V,S^{\prime\prime}) denote the transition relation of NN where Sโ€ฒ,Sโ€ฒโ€ฒS^{\prime},S^{\prime\prime} are the present and next state variables and VV specifies the (combinational) input variables. We will say that a state \vvโ€‹s\vv{s} of NN is reachable if there is an execution trace leading to \vvโ€‹s\vv{s}. That is, there is a sequence of states \vvโ€‹s0,โ€ฆ,\vvโ€‹sk\mbox{$\vv{s_{0}}$},\dots,\mbox{$\vv{s_{k}}$} where \vvโ€‹s0=\vvโ€‹s๐‘–๐‘›๐‘–\mbox{$\vv{s_{0}}$}=\mbox{$\vv{s}\!_{\mathit{ini}}$}, \vvโ€‹sk=\vvโ€‹s\mbox{$\vv{s}\!_{k}$}\!=\!\mbox{$\vv{s}$} and there exist \vvโ€‹vi\vv{v}\!_{i} i=0,โ€ฆ,kโˆ’1i=0,\dots,k\!-\!1 for which Tโ€‹(\vvโ€‹si,\vvโ€‹vi,\vvโ€‹si+1)=1T(\mbox{$\vv{s}\!_{i}$},\mbox{$\vv{v}\!_{i}$},\mbox{$\vv{s}\!_{i+1}$})=1. Let NN have to satisfy a set of invariants P0โ€‹(S),โ€ฆ,Pmโ€‹(S)P_{0}(S),\dots,P_{m}(S). That is, PiP_{i} holds iff Piโ€‹(\vvโ€‹s)=1P_{i}(\mbox{$\vv{s}$})=1 for every reachable state \vvโ€‹s\vv{s} of NN. We will denote the aggregate invariant P0โˆงโ‹ฏโˆงPmP_{0}\wedge\dots\wedge P_{m} as ๐‘ท๐’‚๐’ˆ๐’ˆ\mathit{P}_{\mathit{agg}}. We will call \vvโ€‹s\vv{s} a bad state of NN if P๐‘Ž๐‘”๐‘”โ€‹(\vvโ€‹s)=0\mbox{$\mathit{P}_{\mathit{agg}}$}(\mbox{$\vv{s}$})=0. If P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}} holds, no bad state is reachable. We will call \vvโ€‹s\vv{s} a good state of NN if P๐‘Ž๐‘”๐‘”โ€‹(\vvโ€‹s)=1\mbox{$\mathit{P}_{\mathit{agg}}$}(\mbox{$\vv{s}$})=1.

Typically, the set of invariants P0,โ€ฆ,PmP_{0},\dots,P_{m} is incomplete in the sense that it does not specify all states that must be unreachable. So, a good state can well be unreachable. We will call a good state operative (or op-state for short) if it is supposed to be used by NN and so should be reachable. We introduce the term an operative state just to factor out โ€œuselessโ€ good states. We will say that NN has an op-state reachability bug if an op-state is unreachable in NN. In Sectionย 7, we consider such a bug in a FIFO buffer. The fact that P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}} holds says nothing about reachability of op-states. Consider, for instance, a trivial circuit N๐‘ก๐‘Ÿ๐‘–๐‘ฃ\mathit{N}_{\mathit{triv}} that simply stays in the initial state \vvโ€‹s๐‘–๐‘›๐‘–\vv{s}\!_{\mathit{ini}} and P๐‘Ž๐‘”๐‘”โ€‹(\vvโ€‹s๐‘–๐‘›๐‘–)=1\mbox{$\mathit{P}_{\mathit{agg}}$}(\mbox{$\vv{s}\!_{\mathit{ini}}$})=1. Then P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}} holds for N๐‘ก๐‘Ÿ๐‘–๐‘ฃ\mathit{N}_{\mathit{triv}} but the latter has op-state reachability bugs (assuming that the correct circuit must reach states other than \vvโ€‹s๐‘–๐‘›๐‘–\vv{s}\!_{\mathit{ini}}).

Let R\vvโ€‹sโ€‹(S)R_{\vv{s}}(S) be the predicate satisfied only by a state \vvโ€‹s\vv{s}. In terms of CTL, identifying an op-state reachability bug means finding \vvโ€‹s\vv{s} for which the property Eโ€‹F.R\vvโ€‹sEF.R_{\vv{s}} must hold but it does not. The reason for assuming \vvโ€‹s\vv{s} to be unknown is that the set of op-states is typically too large to explicitly specify every property Eโ€‹T.R\vvโ€‹sET.R_{\vv{s}} to hold. This makes finding op-state reachability bugs very hard. The problem is exacerbated by the fact that reachability of different states is established by different traces. So, in general, one cannot efficiently prove many properties Eโ€‹F.R\vvโ€‹sEF.R_{\vv{s}} (for different states) at once.

4.2 Proving op-state unreachability by invariant generation

In practice, there are two methods to check reachability of operative states for large circuits. The first method is testing. Of course, testing cannot prove a state unreachable, however, the examination of execution traces may point to a potential problem. (For instance, after examining execution traces of the circuit N๐‘ก๐‘Ÿ๐‘–๐‘ฃ\mathit{N}_{\mathit{triv}} above one realizes that many operative states look unreachable.) The other method is to check unwanted invariants, i.e., those that are supposed to fail. If an unwanted invariant holds for a circuit, the latter has an op-state reachability bug. For instance, one can check if a state variable siโˆˆSs_{i}\in S of a circuit never changes its initial value. To break this unwanted invariant, one needs to find an operative state where the initial value of sis_{i} is flipped. (For the circuit N๐‘ก๐‘Ÿ๐‘–๐‘ฃ\mathit{N}_{\mathit{triv}} above this unwanted invariant holds for every state variable.) The potential unwanted invariants are formed manually, i.e., simply guessed.

The two methods above can easily overlook an op-state reachability bug. Testing cannot prove that an op-state is unreachable. To correctly guess an unwanted invariant that holds, one essentially has to know the underlying bug. Below, we describe a method for invariant generation by PQE that is based on property generation for combinational circuits. The appeal of this method is twofold. First, PQE generates invariants โ€œinherentโ€ to the implementation at hand, which drastically reduces the set of invariants to explore. Second, PQE is able to generate invariants related to different parts of the circuit (including the buggy one). This increases the probability of generating an unwanted invariant. We substantiate this intuition in Sectionย 7.

Let formula ๐‘ญ๐’ŒF_{k} specify the combinational circuit obtained by unfolding a sequential circuit NN for kk time frames and adding the initial state constraint Iโ€‹(S0)I(S_{0}). That is, Fk=Iโ€‹(S0)โˆงTโ€‹(S0,V0,S1)โˆงโ‹ฏโˆงTโ€‹(Skโˆ’1,Vkโˆ’1,Sk)F_{k}=I(S_{0})\wedge T(S_{0},V_{0},S_{1})\wedge\dots\wedge T(S_{k-1},V_{k-1},S_{k}) where Sj,VjS_{j},V_{j} denote the state and input variables of jj-th time frame respectively. Let Hโ€‹(Sk)H(S_{k}) be a solution to the PQE problem of taking a clause CC out of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] where Xk=S0โˆชV0โˆชโ‹ฏโˆชSkโˆ’1โˆชVkโˆ’1\mbox{$X_{k}$}=S_{0}\cup V_{0}\cup\dots\cup S_{k-1}\cup V_{k-1}. That is, โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] โ‰กHโˆง\equiv H\wedge โˆƒXkโ€‹[Fkโˆ–{C}]\exists{\mbox{$X_{k}$}}[F_{k}\setminus\mbox{$\{C\}$}]. Note that in contrast to Sectionย 3, here some external variables of the combinational circuit (namely, the input variables V0,โ€ฆ,Vkโˆ’1V_{0},\dots,V_{k-1}) are quantified too. So, HH depends only on state variables of the last time frame. HH can be viewed as a local invariant asserting that no state falsifying HH can be reached in kk transitions.

One can use HH to find global invariants (holding for every time frame) as follows. Even if HH is only a local invariant, a clause QQ of HH can be a global invariant. The experiments of Sectionย 8 show that, in general, this is true for many clauses of HH. (To find out if QQ is a global invariant, one can simply run a model checker to see if the property QQ holds.) Note that by taking out different clauses of FkF_{k} one can produce global single-clause invariants QQ relating to different parts of NN. From now on, when we say โ€œan invariantโ€ without a qualifier we mean a global invariant.

5 Introducing ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}

In this section, we describe a simple SAT-based algorithm for performing PQE called ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}. Here โ€™EGโ€™ stands for โ€™Enumerate and Generalizeโ€™. ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} accepts a formula โˆƒXโ€‹[Fโ€‹(X,Y)]\exists{X}[F(X,Y)] and a clause CโˆˆFC\in F. It outputs a formula Hโ€‹(Y)H(Y) such that โˆƒXโ€‹[F๐‘–๐‘›๐‘–]โ‰กHโˆงโˆƒXโ€‹[F๐‘–๐‘›๐‘–โˆ–{C}]\mbox{$\exists{X}[\mbox{$\mathit{F}_{\mathit{ini}}$}]$}\equiv H\wedge\mbox{$\exists{X}[\mbox{$\mathit{F}_{\mathit{ini}}$}\setminus\mbox{$\{C\}$}]$} where F๐‘–๐‘›๐‘–\mathit{F}_{\mathit{ini}} is the initial formula FF. (This point needs clarification because ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} changes FF by adding clauses.)

5.1 An example

Before describing the pseudocode of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, we explain how it solves the PQE problem of Exampleย 1. That is, we consider taking clause C1C_{1} out of โˆƒXโ€‹[Fโ€‹(X,Y)]\exists{X}[F(X,Y)] where F=C1โˆงโ‹ฏโˆงC4F=C_{1}\wedge\dots\wedge C_{4}, C1=xยฏ3โˆจx4C_{1}=\overline{x}_{3}\vee x_{4}, C2=y1โˆจx3C_{2}\!=\!y_{1}\!\vee\!x_{3}, C3=y1โˆจxยฏ4C_{3}=y_{1}\vee\overline{x}_{4}, C4=y2โˆจx4C_{4}\!=\!y_{2}\!\vee\!x_{4} and Y={y1,y2}Y=\mbox{$\{y_{1},y_{2}\}$} and X={x3,x4}X=\mbox{$\{x_{3},x_{4}\}$}.

๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} iteratively generates a full assignment \vvโ€‹y\vv{y} to YY and checks if (C1)yโ†’(C_{1})_{\vec{y}} is redundant in โˆƒXโ€‹[Fyโ†’]\exists{X}[\mbox{$F_{\vec{y}}$}] (i.e., if C1C_{1} is redundant in โˆƒXโ€‹[F]\exists{X}[F] in subspace \vvโ€‹y\vv{y}). Note that if (Fโˆ–{C1})yโ†’(F\setminus\mbox{$\{C_{1}\}$})_{\vec{y}} implies (C1)yโ†’(C_{1})_{\vec{y}}, then (C1)yโ†’(C_{1})_{\vec{y}} is trivially redundant in โˆƒXโ€‹[Fyโ†’]\exists{X}[\mbox{$F_{\vec{y}}$}]. To avoid such subspaces, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} generates \vvโ€‹y\vv{y} by searching for an assignment (\vvโ€‹y\vv{y},\vvโ€‹x\vv{x}) satisfying the formula (Fโˆ–{C1})โˆงCยฏ1(F\setminus\mbox{$\{C_{1}\}$})\wedge\overline{C}_{1}. (Here \vvโ€‹y\vv{y} and \vvโ€‹x\vv{x} are full assignments to YY and XX respectively.) If such (\vvโ€‹y\vv{y},\vvโ€‹x\vv{x}) exists, it satisfies Fโˆ–{C1}F\setminus\mbox{$\{C_{1}\}$} and falsifies C1C_{1} thus proving that (Fโˆ–{C1})yโ†’(F\setminus\mbox{$\{C_{1}\}$})_{\vec{y}} does not imply (C1)yโ†’(C_{1})_{\vec{y}}.

Assume that ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} found an assignment(y1=0,y2=1,x3=1,x4=0)(y_{1}\!=\!0,y_{2}\!=\!1,x_{3}\!=\!1,x_{4}\!=\!0) satisfying (Fโˆ–{C1})โˆงCยฏ1(F\setminus\mbox{$\{C_{1}\}$})\wedge\overline{C}_{1}. So \vvโ€‹y\vv{y} = (y1=0,y2=1)(y_{1}\!=\!0,y_{2}\!=\!1). Then ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} checks if Fyโ†’F_{\vec{y}} is satisfiable. Fyโ†’F_{\vec{y}} = (xยฏ3โˆจx4)โˆงx3โˆงxยฏ4(\overline{x}_{3}\vee x_{4})\wedge x_{3}\wedge\overline{x}_{4} and so it is unsatisfiable. This means that (C1)yโ†’(C_{1})_{\vec{y}} is not redundant in โˆƒXโ€‹[Fyโ†’]\exists{X}[\mbox{$F_{\vec{y}}$}]. (Indeed, (Fโˆ–{C1})yโ†’(F\setminus\mbox{$\{C_{1}\}$})_{\vec{y}} is satisfiable. So, removing C1C_{1} makes FF satisfiable in subspace \vvโ€‹y\vv{y}.) ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} makes (C1)yโ†’(C_{1})_{\vec{y}} redundant in โˆƒXโ€‹[Fyโ†’]\exists{X}[\mbox{$F_{\vec{y}}$}] by adding to FF a clause BB falsified by \vvโ€‹y\vv{y}. The clause BB equals y1y_{1} and is obtained by identifying the assignments to individual variables of YY that made Fyโ†’F_{\vec{y}} unsatisfiable. (In our case, this is the assignment y1=0y_{1}=0.) Note that derivation of clause y1y_{1} generalizes the proof of unsatisfiability of FF in subspace (y1=0,y2=1)(y_{1}\!=\!0,y_{2}\!=\!1) so that this proof holds for subspace (y1=0,y2=0)(y_{1}\!=\!0,y_{2}\!=\!0) too.

Now ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} looks for a new assignment satisfying (Fโˆ–{C1})โˆงCยฏ1(F\setminus\mbox{$\{C_{1}\}$})\wedge\overline{C}_{1}. Let the assignment (y1=1,y2=1,x3=1,x4=0)(y_{1}=1,y_{2}=1,x_{3}=1,x_{4}=0) be found. So, \vvโ€‹y\vv{y} = (y1=1,y2=1)(y_{1}\!=\!1,y_{2}\!=\!1). Since (y1=1,y2=1,x3=0)(y_{1}\!=\!1,y_{2}\!=\!1,x_{3}=0) satisfies FF, the formula Fyโ†’F_{\vec{y}} is satisfiable. So, (C1)yโ†’(C_{1})_{\vec{y}} is already redundant in โˆƒXโ€‹[Fyโ†’]\exists{X}[\mbox{$F_{\vec{y}}$}]. To avoid re-visiting the subspace \vvโ€‹y\vv{y}, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} generates the plugging clause D=yยฏ1โˆจyยฏ2D=\overline{y}_{1}\vee\overline{y}_{2} falsified by \vvโ€‹y\vv{y}.

๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} fails to generate a new assignment \vvโ€‹y\vv{y} because the formulaDโˆง(Fโˆ–{C1})โˆงCยฏ1D\wedge(F\setminus\mbox{$\{C_{1}\}$})\wedge\overline{C}_{1} is unsatisfiable. Indeed, every full assignment \vvโ€‹y\vv{y} we have examined so far falsifies either the clause y1y_{1} added to FF or the plugging clause DD. The only assignment ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} has not explored yet is \vvโ€‹y=(y1=1,y2=0)\mbox{$\vv{y}$}\!=\!(y_{1}\!=\!1,y_{2}\!=\!0). Since (Fโˆ–{C1})yโ†’=x4\mbox{$(F\setminus\mbox{$\{C_{1}\}$})_{\vec{y}}$}=x_{4} and (C1)yโ†’(C_{1})_{\vec{y}} = xยฏ3โˆจx4\overline{x}_{3}\vee x_{4}, the formula (Fโˆ–{C1})โˆงCยฏ1(F\setminus\mbox{$\{C_{1}\}$})\wedge\overline{C}_{1} is unsatisfiable in subspace \vvโ€‹y\vv{y}. In other words,(C1)yโ†’(C_{1})_{\vec{y}} is implied by (Fโˆ–{C1})yโ†’(F\setminus\mbox{$\{C_{1}\}$})_{\vec{y}} and hence is redundant. Thus, C1C_{1} is redundant in โˆƒXโ€‹[F๐‘–๐‘›๐‘–โˆงy1]\exists{X}[\mbox{$\mathit{F}_{\mathit{ini}}$}\wedge y_{1}] for every assignment to YY where F๐‘–๐‘›๐‘–\mathit{F}_{\mathit{ini}} is the initial formula FF. That is, โˆƒXโ€‹[F๐‘–๐‘›๐‘–]\exists{X}[\mbox{$\mathit{F}_{\mathit{ini}}$}] โ‰กy1โˆง\equiv y_{1}\wedge โˆƒXโ€‹[F๐‘–๐‘›๐‘–โˆ–{C1}]\exists{X}[\mbox{$\mathit{F}_{\mathit{ini}}$}\setminus\mbox{$\{C_{1}\}$}] and so the clause y1y_{1} is a solution HH to our PQE problem.

5.2 Description of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}

๐ธ๐บ-๐‘ƒ๐‘„๐ธโ€‹(F,X,Y,C)\mbox{$\mathit{EG}$-$\mathit{PQE}$}(F,X,Y,C) {
1 ๐‘ƒ๐‘™๐‘”:=โˆ…\mbox{$\mathit{Plg}$}:=\emptyset; F๐‘–๐‘›๐‘–:=F\mbox{$\mathit{F}_{\mathit{ini}}$}:=F
2 while (true) {
3 G:=Fโˆ–{C}G\!:=F\setminus\mbox{$\{C\}$}
4 \vvโ€‹y:=๐‘†๐‘Ž๐‘ก1โ€‹(๐‘ƒ๐‘™๐‘”โˆงGโˆงCยฏ)\mbox{$\vv{y}$}\!:=\!\mathit{Sat}_{1}(\mbox{$\mathit{Plg}$}\!\wedge G\!\wedge\!\overline{C})
5 if (\vvโ€‹y=๐‘›๐‘–๐‘™\mbox{$\vv{y}$}=\mathit{nil})
6 return(Fโˆ–F๐‘–๐‘›๐‘–F\setminus\mbox{$\mathit{F}_{\mathit{ini}}$})
7 (\vvโ€‹xโˆ—,B):=๐‘†๐‘Ž๐‘ก2โ€‹(F,\vvโ€‹y)(\mbox{$\vv{x}$}^{*},B):=\mathit{Sat}_{2}(F,\mbox{$\vv{y}$})
8 if (Bโ‰ ๐‘›๐‘–๐‘™B\neq\mathit{nil}) {
9 F:=Fโˆช{B}F:=F\cup\mbox{$\{B\}$}
10 continue }
11 D:=Pโ€‹lโ€‹uโ€‹gโ€‹Cโ€‹lโ€‹sโ€‹(\vvโ€‹y,\vvโ€‹xโˆ—,F)D\!:=\!PlugCls(\mbox{$\vv{y}$},\!\mbox{$\vv{x}$}^{*},\!F)
12 ๐‘ƒ๐‘™๐‘”:=๐‘ƒ๐‘™๐‘”โˆช{D}\mbox{$\mathit{Plg}$}:=\mbox{$\mathit{Plg}$}\cup\mbox{$\{D\}$}}}
Figure 1: Pseudocode of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}

The pseudo-code of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is shown in Fig.ย 1. ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} starts with storing the initial formula FF and initializing formula ๐‘ƒ๐‘™๐‘”\mathit{Plg} that accumulates the plugging clauses generated by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} (line 1). As we mentioned in the previous subsection, plugging clauses are used to avoid re-visiting the subspaces where the formula FF is proved satisfiable.

All the work is carried out in a while loop. First, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} checks if there is a new subspace \vvโ€‹y\vv{y} where โˆƒXโ€‹[(Fโˆ–{C})yโ†’]\exists{X}[\mbox{$(F\setminus\mbox{$\{C\}$})_{\vec{y}}$}] does not imply Fyโ†’F_{\vec{y}}. This is done by searching for an assignment (\vvโ€‹y\vv{y},\vvโ€‹x\vv{x}) satisfying ๐‘ƒ๐‘™๐‘”โˆง(Fโˆ–{C})โˆงCยฏ\mbox{$\mathit{Plg}$}\wedge(F\setminus\mbox{$\{C\}$})\wedge\overline{C} (lines 3-4). If such an assignment does not exist, the clause CC is redundant in โˆƒXโ€‹[F]\exists{X}[F]. (Indeed, let \vvโ€‹y\vv{y} be a full assignment to YY. The formula ๐‘ƒ๐‘™๐‘”โˆง(Fโˆ–{C})โˆงCยฏ\mbox{$\mathit{Plg}$}\wedge(F\setminus\mbox{$\{C\}$})\wedge\overline{C} is unsatisfiable in subspace \vvโ€‹y\vv{y} for one of the two reasons. First, \vvโ€‹y\vv{y} falsifies ๐‘ƒ๐‘™๐‘”\mathit{Plg}. Then Cyโ†’C_{\vec{y}} is redundant because Fyโ†’F_{\vec{y}} is satisfiable. Second, (Fโˆ–{C})yโ†’โˆงCyโ†’ยฏ\mbox{$(F\setminus\mbox{$\{C\}$})_{\vec{y}}$}\wedge\overline{\mbox{$C_{\vec{y}}$}} is unsatisfiable. In this case, (Fโˆ–{C})yโ†’(F\setminus\mbox{$\{C\}$})_{\vec{y}} implies Cyโ†’C_{\vec{y}}.) Then ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} returns the set of clauses added to the initial formula FF as a solution HH to the PQE problem (lines 5-6).

If the satisfying assignment (\vvโ€‹y\vv{y},\vvโ€‹x\vv{x}) above exists, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} checks if the formula Fyโ†’F_{\vec{y}} is satisfiable (line 7). If not, then the clause Cyโ†’C_{\vec{y}} is not redundant in โˆƒXโ€‹[Fyโ†’]\exists{X}[\mbox{$F_{\vec{y}}$}] (because (Fโˆ–{C})yโ†’(F\setminus\mbox{$\{C\}$})_{\vec{y}} is satisfiable). So, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} makes Cyโ†’C_{\vec{y}} redundant by generating a clause Bโ€‹(Y)B(Y) falsified by \vvโ€‹y\vv{y} and adding it to FF (line 9). Note that adding BB also prevents ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} from re-visiting the subspace \vvโ€‹y\vv{y} again. The clause BB is built by finding an unsatisfiable subset of Fyโ†’F_{\vec{y}} and collecting the literals of YY removed from clauses of this subset when obtaining Fyโ†’F_{\vec{y}} from FF.

If Fyโ†’F_{\vec{y}} is satisfiable, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} generates an assignment \vvโ€‹xโˆ—\mbox{$\vv{x}$}^{*} to XX such that (\vvโ€‹y,\vvโ€‹xโˆ—)(\mbox{$\vv{y}$},\mbox{$\vv{x}$}^{*}) satisfies FF (line 7). The satisfiability of Fyโ†’F_{\vec{y}} means that every clause of Fyโ†’F_{\vec{y}} including Cyโ†’C_{\vec{y}} is redundant in โˆƒXโ€‹[Fyโ†’]\exists{X}[\mbox{$F_{\vec{y}}$}]. At this point, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} uses the longest clause Dโ€‹(Y)D(Y) falsified by \vvโ€‹y\vv{y} as a plugging clause (line 11). The clause DD is added to ๐‘ƒ๐‘™๐‘”\mathit{Plg} to avoid re-visiting subspace \vvโ€‹y\vv{y}. Sometimes it is possible to remove variables from \vvโ€‹y\vv{y} to produce a shorter assignment \vvโ€‹yโˆ—\mbox{$\vv{y}$}^{*} such that (\vvโ€‹yโˆ—,\vvโ€‹xโˆ—)(\mbox{$\vv{y}$}^{*},\mbox{$\vv{x}$}^{*}) still satisfies FF. Then one can use a shorter plugging clause DD involving only the variables assigned in \vvโ€‹yโˆ—\mbox{$\vv{y}$}^{*}.

5.3 Discussion

๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is similar to the QE algorithm presented at CAV-2002ย [11]. We will refer to it as CAV02\mathit{CAV02}-๐‘„๐ธ\mathit{QE}. Given a formula โˆƒXโ€‹[Fโ€‹(X,Y)]\exists{X}[F(X,Y)], CAV02\mathit{CAV02}-๐‘„๐ธ\mathit{QE} enumerates full assignments to YY. In subspace \vvโ€‹y\vv{y}, if Fyโ†’F_{\vec{y}} is unsatisfiable, CAV02\mathit{CAV02}-๐‘„๐ธ\mathit{QE} adds to FF a clause falsified by \vvโ€‹y\vv{y}. Otherwise, CAV02\mathit{CAV02}-๐‘„๐ธ\mathit{QE} generates a plugging clause DD. (Inย [11], DD is called โ€œa blocking clauseโ€. This term can be confused with the term โ€œblocked clauseโ€ applied to a completely different kind of a clause. So, we use the term โ€œa plugging clauseโ€ instead.) To apply the idea of CAV02\mathit{CAV02}-๐‘„๐ธ\mathit{QE} to PQE, we reformulated it in terms of redundancy based reasoning.

The main flaw of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} inherited from CAV02\mathit{CAV02}-๐‘„๐ธ\mathit{QE} is the necessity to use plugging clauses produced from a satisfying assignment. Consider the PQE problem of taking a clause CC out of โˆƒXโ€‹[Fโ€‹(X,Y)]\exists{X}[F(X,Y)]. If FF is proved unsatisfiable in subspace \vvโ€‹y\vv{y}, typically, only a small subset of clauses of Fyโ†’F_{\vec{y}} is involved in the proof. Then the clause generated by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is short and thus proves CC redundant in many subspaces different from \vvโ€‹y\vv{y}. On the contrary, to prove FF satisfiable in subspace \vvโ€‹y\vv{y}, every clause of FF must be satisfied. So, the plugging clause built off a satisfying assignment includes almost every variable of YY. Despite this flaw of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, we present it for two reasons. First, it is a very simple SAT-based algorithm that can be easily implemented. Second, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} has a powerful advantage over CAV02\mathit{CAV02}-๐‘„๐ธ\mathit{QE} since it solves PQE rather than QE. Namely, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} does not need to examine the subspaces \vvโ€‹y\vv{y} where CC is implied by Fโˆ–{C}F\setminus\mbox{$\{C\}$}. Surprisingly, for many formulas this allows ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} to completely avoid examining subspaces where FF is satisfiable. In this case, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is very efficient and can solve very large problems. Note that when CAV02\mathit{CAV02}-๐‘„๐ธ\mathit{QE} performs complete QE on โˆƒXโ€‹[F]\exists{X}[F], it cannot avoid subspaces \vvโ€‹y\vv{y} where Fyโ†’F_{\vec{y}} is satisfiable unless FF itself is unsatisfiable (which is very rare in practical applications).

6 Introducing ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}

In this section, we describe ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}, an improved version of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}.

6.1 Main idea

The pseudocode of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} is shown in Figย 2. It is different from that of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} only in line 11 marked with an asterisk. The motivation for this change is as follows. Line 11 describes proving redundancy of CC for the case where Cyโ†’C_{\vec{y}} is not implied by (Fโˆ–{C})yโ†’(F\setminus\mbox{$\{C\}$})_{\vec{y}} and Fyโ†’F_{\vec{y}} is satisfiable. Then ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} simply uses a satisfying assignment as a proof of redundancy of CC in subspace \vvโ€‹y\vv{y}. This proof is unnecessarily strong because it proves that every clause of FF (including CC) is redundant in โˆƒXโ€‹[F]\exists{X}[F] in subspace \vvโ€‹y\vv{y}. Such a strong proof is hard to generalize to other subspaces.

๐ธ๐บ-๐‘ƒ๐‘„๐ธ+โ€‹(F,X,Y,C)\mbox{$\mathit{EG}$-$\mathit{PQE}^{+}$}(F,X,Y,C) {
1 ๐‘ƒ๐‘™๐‘”:=โˆ…\mbox{$\mathit{Plg}$}:=\emptyset; F๐‘–๐‘›๐‘–:=F\mbox{$\mathit{F}_{\mathit{ini}}$}:=F
2 while (true) {
โ€ฆโ€‹โ€ฆ..........
11โˆ— D:=Pโ€‹rโ€‹vโ€‹Cโ€‹lโ€‹sโ€‹Rโ€‹eโ€‹dโ€‹(\vvโ€‹y,F,C)D\!:=\!PrvClsRed(\mbox{$\vv{y}$},\!F,\!C)
12 ๐‘ƒ๐‘™๐‘”:=๐‘ƒ๐‘™๐‘”โˆช{D}\mbox{$\mathit{Plg}$}:=\mbox{$\mathit{Plg}$}\cup\mbox{$\{D\}$}}}
Figure 2: Pseudocode of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}

The idea of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} is to generate a proof for a much weaker proposition namely a proof of redundancy of CC (and only CC). Intuitively, such a proof should be easier to generalize. So, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} calls a procedure PrvClsRed generating such a proof. ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} is a generic algorithm in the sense that any suitable procedure can be employed as PrvClsRed. In our current implementation, the procedure ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE}ย [1] is used as PrvClsRed. ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} generates a proof stating that CC is redundant in โˆƒXโ€‹[F]\exists{X}[F] in subspace \vvโ€‹yโˆ—โІ\vvโ€‹y\mbox{$\vv{y}$}^{*}\subseteq\mbox{$\vv{y}$}. Then the plugging clause DD falsified by \vvโ€‹yโˆ—\mbox{$\vv{y}$}^{*} is generated. Importantly, \vvโ€‹yโˆ—\mbox{$\vv{y}$}^{*} can be much shorter than \vvโ€‹y\vv{y}. Appendixย 0.G gives a brief description of ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE}.

Example 3

Consider the example solved in Subsectionย 5.1. That is, we consider taking clause C1C_{1} out of โˆƒXโ€‹[Fโ€‹(X,Y)]\exists{X}[F(X,Y)] where F=C1โˆงโ‹ฏโˆงC4F=C_{1}\wedge\dots\wedge C_{4}, C1=xยฏ3โˆจx4C_{1}=\overline{x}_{3}\vee x_{4}, C2=y1โˆจx3C_{2}\!=\!y_{1}\!\vee\!x_{3}, C3=y1โˆจxยฏ4C_{3}=y_{1}\vee\overline{x}_{4}, C4=y2โˆจx4C_{4}\!=\!y_{2}\!\vee\!x_{4} and Y={y1,y2}Y=\mbox{$\{y_{1},y_{2}\}$} and X={x3,x4}X=\mbox{$\{x_{3},x_{4}\}$}. Consider the step where ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} proves redundancy of C1C_{1} in subspace \vvโ€‹y=(y1=1,y2=1)\mbox{$\vv{y}$}=(y_{1}\!=\!1,y_{2}\!=\!1). ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} shows that (y1=1,y2=1,x3=0)(y_{1}\!=\!1,y_{2}\!=\!1,\!x_{3}=0) satisfies FF, thus proving every clause of FF (including C1C_{1}) redundant in โˆƒXโ€‹[F]\exists{X}[F] in subspace \vvโ€‹y\vv{y}. Then ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} generates the plugging clause D=yยฏ1โˆจyยฏ2D=\overline{y}_{1}\vee\overline{y}_{2} falsified by \vvโ€‹y\vv{y}.

In contrast to ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} calls PrvClsRed to produce a proof of redundancy for the clause C1C_{1} alone. Note that FF has no clauses resolvable with C1C_{1} on x3x_{3} in subspace \vvโ€‹yโˆ—=(y1=1)\mbox{$\vv{y}$}^{*}=(y_{1}=1). (The clause C2C_{2} containing x3x_{3} is satisfied by \vvโ€‹yโˆ—\mbox{$\vv{y}$}^{*}.) This means that C1C_{1} is blocked in subspace \vvโ€‹yโˆ—\mbox{$\vv{y}$}^{*} and hence redundant there (see Propositionย 2). Since \vvโ€‹yโˆ—โŠ‚\vvโ€‹y\mbox{$\vv{y}$}^{*}\subset\mbox{$\vv{y}$}, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} produces a more general proof of redundancy than ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}. To avoid re-examining the subspace \vvโ€‹yโˆ—\mbox{$\vv{y}$}^{*}, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} generates a shorter plugging clause D=yยฏ1D=\overline{y}_{1}.

6.2 Discussion

Consider the PQE problem of taking a clause CC out of โˆƒXโ€‹[Fโ€‹(X,Y)]\exists{X}[F(X,Y)]. There are two features of PQE that make it easier than QE. The first feature mentioned earlier is that one can ignore the subspaces \vvโ€‹y\vv{y} where Fโˆ–{C}F\setminus\mbox{$\{C\}$} implies CC. The second feature is that when Fyโ†’F_{\vec{y}} is satisfiable, one only needs to prove redundancy of the clause CC alone. Among the three algorithms we run in experiments, namely, ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} only the latter exploits both features. (In addition to using ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} inside ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} we also run it as a stand-alone PQE solver.) ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} does not use the first featureย [1] and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} does not exploit the second one. As we show in Sectionsย 7 andย 8, this affects the performance of ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}.

7 Experiment With FIFO Buffers

In this and the next two sections we describe some experiments with๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} (their sources are available atย [12],ย [13] and ย [14] respectively). We used Minisat2.0ย [15] as an internal SAT-solver. The experiments were run on a computer with Intel Core i5-8265U CPU of 1.6โ€‰GHz.

ย ย ย ย ย ย ย ย ย ย ย ย ย โ‹ฏ\cdot\cdot\cdot
if (๐‘ค๐‘Ÿ๐‘–๐‘ก๐‘’==1&&๐‘๐‘ข๐‘Ÿ๐‘Ÿ๐‘†๐‘–๐‘ง๐‘’<n\mathit{write}==1~{}\&\&~{}\mathit{currSize}<n)
* if (๐‘‘๐‘Ž๐‘ก๐‘Ž๐ผ๐‘›!=๐‘‰๐‘Ž๐‘™\mathit{dataIn}~{}!\!=\mathit{Val})
begin
๐ท๐‘Ž๐‘ก๐‘Žโ€‹[๐‘ค๐‘Ÿ๐‘ƒ๐‘›๐‘ก]=๐‘‘๐‘Ž๐‘ก๐‘Ž๐ผ๐‘›\mbox{$\mathit{Data}$}[\mathit{wrPnt}]=\mathit{dataIn};
๐‘ค๐‘Ÿ๐‘ƒ๐‘›๐‘ก=๐‘ค๐‘Ÿ๐‘ƒ๐‘›๐‘ก+1\mathit{wrPnt}=\mathit{wrPnt}+1;
end
ย ย ย ย ย ย ย ย ย ย ย ย ย โ‹ฏ\cdot\cdot\cdot
Figure 3: A buggy fragment of Verilog code describing ๐น๐‘–๐‘“๐‘œ\mathit{Fifo}

In this section, we give an example of bug detection by invariant generation for a FIFO buffer. Our objective here is threefold. First, we want to give an example of a bug that can be overlooked by testing and guessing the unwanted properties to check (see Subsectionย 7.3). Second, we want to substantiate the intuition of Subsectionย 3.4 that property generation by PQE (in our case, invariant generation by PQE) has the same reasons to be effective as testing. In particular, by taking out different clauses one generates invariants relating to different parts of the design. So, taking out a clause of the buggy part is likely to produce an unwanted invariant. Third, we want to give an example of an invariant that can be easily identified as unwanted222Let Pโ€‹(S^)P(\hat{S}) be an invariant for a circuit NN depending only on a subset S^\hat{S} of the state variables SS. Identifying PP as an unwanted invariant is much easier if S^\hat{S} is meaningful from the high-level view of the design. Suppose, for instance, that assignments to S^\hat{S} specify values of a high-level variable vv. Then PP is unwanted if it claims unreachability of a value of vv that is supposed to be reachable. Another simple example is that assignments to S^\hat{S} specify values of high-level variables vv and ww that are supposed to be independent. Then PP is unwanted if it claims that some combinations of values of vv and ww are unreachable. (This may mean, for instance, that an assignment operator setting the value of vv erroneously involves the variable ww.) .

7.1 Buffer description

Consider a FIFO buffer that we will refer to as ๐น๐‘–๐‘“๐‘œ\mathit{Fifo}. Let nn be the number of elements of ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} and ๐ท๐‘Ž๐‘ก๐‘Ž\mathit{Data} denote the data buffer of ๐น๐‘–๐‘“๐‘œ\mathit{Fifo}. Let each ๐ท๐‘Ž๐‘ก๐‘Žโ€‹[i],i=1,โ€ฆ,n\mbox{$\mathit{Data}$}[i],i=1,\dots,n have pp bits and be an integer where 0โ‰ค๐ท๐‘Ž๐‘ก๐‘Žโ€‹[i]<2p0\leq\mbox{$\mathit{Data}$}[i]<2^{p}. A fragment of the Verilog code describing ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} is shown in Figย 3. This fragment has a buggy line marked with an asterisk. In the correct version without the marked line, a new element ๐‘‘๐‘Ž๐‘ก๐‘Ž๐ผ๐‘›\mathit{dataIn} is added to ๐ท๐‘Ž๐‘ก๐‘Ž\mathit{Data} if the write flag is on and ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} has less than nn elements. Since ๐ท๐‘Ž๐‘ก๐‘Ž\mathit{Data} can have any combination of numbers, all ๐ท๐‘Ž๐‘ก๐‘Ž\mathit{Data} states are supposed to be reachable. However, due to the bug, the number ๐‘‰๐‘Ž๐‘™\mathit{Val} cannot appear in ๐ท๐‘Ž๐‘ก๐‘Ž\mathit{Data}. (Here ๐‘‰๐‘Ž๐‘™\mathit{Val} is some constant 0<๐‘‰๐‘Ž๐‘™<2p0\!<\!\mathit{Val}\!<\!2^{p}. We assume that the buffer elements are initialized to 0.) So, ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} has an op-state reachability bug since it cannot reach operative states where an element of ๐ท๐‘Ž๐‘ก๐‘Ž\mathit{Data} equals ๐‘‰๐‘Ž๐‘™\mathit{Val}.

7.2 Bug detection by invariant generation

Let NN be a circuit implementing ๐น๐‘–๐‘“๐‘œ\mathit{Fifo}. Let SS be the set of state variables of NN and ๐‘บ๐’…๐’‚๐’•๐’‚โŠ‚๐‘บ\mbox{$\mathit{S}_{\mathit{data}}$}\subset S be the subset corresponding to the data buffer ๐ท๐‘Ž๐‘ก๐‘Ž\mathit{Data}. We used ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} to generate invariants of NN as described in Sectionย 4. Note that an invariant QQ depending only on S๐‘‘๐‘Ž๐‘ก๐‘Ž\mathit{S}_{\mathit{data}} is an unwanted one. If QQ holds for NN, some states of ๐ท๐‘Ž๐‘ก๐‘Ž\mathit{Data} are unreachable. Then ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} has an op-state reachability bug since every state of ๐ท๐‘Ž๐‘ก๐‘Ž\mathit{Data} is supposed to be reachable. To generate invariants, we used the formula Fk=Iโ€‹(S0)โˆงTโ€‹(S0,V0,S1)โˆงโ‹ฏโˆงTโ€‹(Skโˆ’1,Vkโˆ’1,Sk)F_{k}=I(S_{0})\wedge T(S_{0},V_{0},S_{1})\wedge\dots\wedge T(S_{k-1},V_{k-1},S_{k}) introduced in Subsectionย 4.2. Here II and TT describe the initial state and the transition relation of NN respectively and SjS_{j} and VjV_{j} denote state variables and combinational input variables of jj-th time frame respectively. First, we used a PQE solver to generate a local invariant Hโ€‹(Sk)H(S_{k}) obtained by taking a clause CC out of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] where Xk=S0โˆชV0โˆชโ‹ฏโˆชSkโˆ’1โˆชVkโˆ’1\mbox{$X_{k}$}=S_{0}\cup V_{0}\cup\dots\cup S_{k-1}\cup V_{k-1}. So, โˆƒXkโ€‹[Fk]โ‰ก\mbox{$\exists{\mbox{$X_{k}$}}[F_{k}]$}\equiv HโˆงH\wedge โˆƒXkโ€‹[Fkโˆ–{C}]\exists{\mbox{$X_{k}$}}[F_{k}\setminus\mbox{$\{C\}$}]. (Since Fkโ‡’HF_{k}\Rightarrow H, no state falsifying HH can be reached in kk transitions.) In the experiment, we took out only clauses of FkF_{k} containing an unquantified variable, i.e., a state variable of the kk-th time frame. The time limit for solving the PQE problem of taking out a clause was set to 10 sec.

Table 1: FIFO buffer with nn elements of 32 bits. Time limit is 10 sec. per PQE problem
buff. lat- time total pqe probs finished pqe probs unwant. invar runtime (s.)
size ches fra- ds- eg- eg- ds- eg- eg- ds- eg- eg- ds- eg- eg-
ย ย nn mes pqe pqe pqe+ pqe pqe pqe+ pqe pqe pqe+ pqe pqe pqe+
ย ย 8 ย 300 ย  5 1,236 311 ย 8 ย 2% 36% 35% no yes yes 12,141 2,138 52
ย ย 8 ย 300 ย  10 ย 560 737 ย 39 ย 2% 1% 3% yes yes yes 5,551 7,681 380
ย ย 16 ย 560 ย  5 2,288 2,288 ย 16 ย 1% 65% 71% no no yes 22,612 9,506 50
ย ย 16 ย 560 ย  10 653 2,288 ย 24 ย 1% 36% 38% yes no yes 6,541 16,554 153

For each clause QQ of every local invariant HH generated by PQE, we checked if QQ was a global invariant. Namely, we used a public version of IC3ย [16, 17] to verify if the property QQ held (by showing that no reachable state of NN falsified QQ). If so, and QQ depended only on variables of S๐‘‘๐‘Ž๐‘ก๐‘Ž\mathit{S}_{\mathit{data}}, NN had an unwanted invariant. Then we stopped invariant generation. The results of the experiment for buffers with 32-bit elements are given in Tableย 1. When picking a clause to take out, i.e., a clause with a state variable of kk-th time frame, one could make a good choice by pure luck. To address this issue, we picked clauses to take out randomly and performed 10 different runs of invariant generation and then computed the average value. So, the columns four to twelve of Tableย 1 actually give the average value of 10 runs.

Let us use the first line of Tableย 1 to explain its structure. The first two columns show the number of elements in ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} implemented by NN and the number of latches in NN (8 and 300). The third column gives the number kk of time frames (i.e., 5). The next three columns show the total number of PQE problems solved by a PQE solver before an unwanted invariant was generated, e.g., 8 problems for ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}. On the other hand, ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} failed to find an unwanted invariant and had to solve all 1,236 PQE problems of taking out a clause of FkF_{k} with an unquantified variable. The following three columns show the share of PQE problems finished in the time limit of 10 sec. For instance, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} finished 36% of 311 problems. The next three columns show if an unwanted invariant was generated by a PQE solver. (๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} found one whereas ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} did not.) The last three columns give the total run time. Tableย 1 shows that only ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} managed to generate an unwanted invariant for all four instances of ๐น๐‘–๐‘“๐‘œ\mathit{Fifo}. This invariant asserted that ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} cannot reach a state where an element of ๐ท๐‘Ž๐‘ก๐‘Ž\mathit{Data} equals ๐‘‰๐‘Ž๐‘™\mathit{Val}.

7.3 Detection of the bug by conventional methods

The bug above (or its modified version) can be overlooked by conventional methods. Consider, for instance, testing. It is hard to detect this bug by random tests because it is exposed only if one tries to add Val to ๐น๐‘–๐‘“๐‘œ\mathit{Fifo}. The same applies to testing using the line coverage metricย [18]. On the other hand, a test set with 100% branch coverageย [18] will find this bug. (To invoke the else branch of the if statement marked with โ€™*โ€™ in Fig.ย 3, one must set ๐‘‘๐‘Ž๐‘ก๐‘Ž๐ผ๐‘›\mathit{dataIn} to ๐‘‰๐‘Ž๐‘™\mathit{Val}.) However, such a test set is not hard to beat. Consider a slightly modified bug. Namely, assume that the buggy line isย ย ifโ€‰((๐‘‘๐‘Ž๐‘ก๐‘Ž๐ผ๐‘›!=๐‘‰๐‘Ž๐‘™)&&(๐‘‘๐‘Ž๐‘ก๐‘Ž๐ผ๐‘›!=๐‘‰๐‘Ž๐‘™โ€ฒ)\mathit{dataIn}~{}!\!\!=\mathit{Val})~{}\&\&~{}(\mathit{dataIn}~{}!\!\!=\mathit{Val}^{\prime})) and in the else branch of this if statement, the element ๐‘‰๐‘Ž๐‘™โ€ฒ\mathit{Val}^{\prime} is pushed to ๐น๐‘–๐‘“๐‘œ\mathit{Fifo}. So, instead of just ignoring the element ๐‘‰๐‘Ž๐‘™\mathit{Val}, the modified bug replaces it with ๐‘‰๐‘Ž๐‘™โ€ฒ\mathit{Val}^{\prime} in ๐น๐‘–๐‘“๐‘œ\mathit{Fifo}. Similarly to the bug of Subsectionย 7.1, ๐‘‰๐‘Ž๐‘™\mathit{Val} does not appear in ๐น๐‘–๐‘“๐‘œ\mathit{Fifo}. So, the modified bug can also be detected by generating an unwanted invariant. On the other hand, it can be overlooked by tests with 100% branch coverage because either branch of the modified if statement can be activated without assigning ๐‘‘๐‘Ž๐‘ก๐‘Ž๐ผ๐‘›\mathit{dataIn} to ๐‘‰๐‘Ž๐‘™\mathit{Val}.

Now consider the โ€œmanualโ€ generation of unwanted properties. It is virtually impossible to guess an unwanted invariant of ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} exposing the bug of Subsectionย 7.1 unless one knows exactly what this bug is. However, one can detect this bug by checking a property asserting that the element ๐‘‘๐‘Ž๐‘ก๐‘Ž๐ผ๐‘›\mathit{dataIn} must appear in the buffer if ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} is ready to accept it, i.e., (๐‘ค๐‘Ÿ๐‘–๐‘ก๐‘’==1)&&(๐‘๐‘ข๐‘Ÿ๐‘Ÿ๐‘†๐‘–๐‘ง๐‘’<n)(\mathit{write}==1)\&\&(\mathit{currSize}<n) holds. Note that this is a non-invariant property involving states of different time frames. The more time frames are used in such a property, the more guesswork is required to pick it. Consider, for instance, the following bug. Suppose ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} does not reject the element ๐‘‰๐‘Ž๐‘™\mathit{Val}. So, the non-invariant property above holds. However, if ๐‘‘๐‘Ž๐‘ก๐‘Ž๐ผ๐‘›==๐‘‰๐‘Ž๐‘™\mathit{dataIn}==\mathit{Val}, then ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} changes the previous accepted element if that element was ๐‘‰๐‘Ž๐‘™\mathit{Val} too. So, ๐น๐‘–๐‘“๐‘œ\mathit{Fifo} cannot have two consecutive elements ๐‘‰๐‘Ž๐‘™\mathit{Val}. Our method will detect this bug via generating an unwanted invariant falsified by states with consecutive elements ๐‘‰๐‘Ž๐‘™\mathit{Val}. One can also identify this bug by checking a property involving two consecutive elements of ๐น๐‘–๐‘“๐‘œ\mathit{Fifo}. But picking it requires a lot of guesswork and so the modified bug can be easily overlooked.

8 Experiments With HWMCC Benchmarks

In this section, we describe three experiments with 98 multi-property benchmarks of the HWMCC-13 setย [19]. (We use this set because it has a multi-property track, see the explanation below.) The number of latches in those benchmarks range from 111 to 8,000. More details about the choice of benchmarks and the experiments can be found in Appendixย 0.H. Each benchmark consists of a sequential circuit NN and invariants P0,โ€ฆ,PmP_{0},\dots,P_{m} to prove. Like in Sectionย 4, we call P๐‘Ž๐‘”๐‘”=P0โˆงโ‹ฏโˆงPm\mbox{$\mathit{P}_{\mathit{agg}}$}=P_{0}\wedge\dots\wedge P_{m} the aggregate invariant. In experiments 2 and 3 we used PQE to generate new invariants of NN. Since every invariant PP implied by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}} is a desired one, the necessary condition for PP to be unwanted is P๐‘Ž๐‘”๐‘”โ‡’ฬธP\mbox{$\mathit{P}_{\mathit{agg}}$}\not\Rightarrow P. The conjunction of many invariants PiP_{i} produces a stronger invariant P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}}, which makes it harder to generate PP not implied by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}}. (This is the reason for using multi-property benchmarks in our experiments.) The circuits of the HWMCC-13 set are anonymous, so, we could not know if an unreachable state is supposed to be reachable. For that reason, we just generated invariants not implied by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}} without deciding if some of them were unwanted.

Similarly to the experiment of Sectionย 7, we used the formula Fk=Iโ€‹(S0)โˆงTโ€‹(S0,V0,S1)F_{k}=I(S_{0})\wedge T(S_{0},V_{0},S_{1}) โˆงโ‹ฏโˆงTโ€‹(Skโˆ’1,Vkโˆ’1,Sk)\wedge\dots\wedge T(S_{k-1},V_{k-1},S_{k}) to generate invariants. The number kk of time frames was in the range of 2โ‰คkโ‰ค102\!\leq\!k\!\leq\!10. As in the experiment of Sectionย 7, we took out only clauses containing a state variable of the kk-th time frame. In all experiments, the time limit for solving a PQE problem was set to 10 sec.

8.1 Experiment 1

In the first experiment, we generated a local invariant HH by taking out a clause CC of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] where Xk=S0โˆชV0โˆชโ‹ฏโˆชSkโˆ’1โˆชVkโˆ’1\mbox{$X_{k}$}=S_{0}\cup V_{0}\cup\dots\cup S_{k-\!1}\cup V_{k-\!1}. The formula HH asserts that no state falsifying HH can be reached in kk transitions. Our goal was to show that PQE can find HH for large formulas FkF_{k} that have hundreds of thousands of clauses. We used ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} to partition the PQE problems we tried into two groups. The first group consisted of 3,736 problems for which we ran ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} with the time limit of 10 sec. and it never encountered a subspace \vvโ€‹sk\vv{s_{k}} where FkF_{k} was satisfiable. Here \vvโ€‹sk\vv{s_{k}} is a full assignment to SkS_{k}. Recall that only the variables SkS_{k} are unquantified in โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}]. So, in every subspace \vvโ€‹sk\vv{s_{k}}, formula FkF_{k} was either unsatisfiable or (Fkโˆ–{C})โ‡’C(F_{k}\setminus\mbox{$\{C\}$})\Rightarrow C. (The fact that so many problems meet the condition of the first group came as a big surprise.) The second group consisted of 3,094 problems where ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} encountered subspaces where FkF_{k} was satisfiable.

For the first group, ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} finished only 30% of the problems within 10 sec. whereas ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} finished 88% and 89% respectively. The poor performance of ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is due to not checking if (Fkโˆ–{C})โ‡’C(F_{k}\setminus\mbox{$\{C\}$})\Rightarrow C in the current subspace. For the second group, ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} finished 15%, 2% and 27% of the problems respectively within 10 sec. ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} finished far fewer problems because it used a satisfying assignment as a proof of redundancy of CC (see Subsectionย 6.2).

To contrast PQE and QE, we employed a high-quality tool ๐ถ๐ด๐ท๐ธ๐‘‡\mathit{CADET}ย [20, 21] to perform QE on the 98 formulas โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] (one formula per benchmark). That is, instead of taking a clause out of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] by PQE, we applied ๐ถ๐ด๐ท๐ธ๐‘‡\mathit{CADET} to perform full QE on this formula. (Performing QE on โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] produces a formula Hโ€‹(Sk)H(S_{k}) specifying all states unreachable in kk transitions.) ๐ถ๐ด๐ท๐ธ๐‘‡\mathit{CADET} finished only 25% of the 98 QE problems with the time limit of 600 sec. On the other hand, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} finished 60% of the 6,830 problems of both groups (generated off โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}]) within 10 sec. So, PQE can be much easier than QE if only a small part of the formula gets unquantified.

8.2 Experiment 2

The second experiment was an extension of the first one. Its goal was to show that PQE can generate invariants for realistic designs. For each clause QQ of a local invariant HH generated by PQE we used IC3 to verify if QQ was a global invariant. If so, we checked if P๐‘Ž๐‘”๐‘”โ‡’ฬธQ\mbox{$\mathit{P}_{\mathit{agg}}$}\not\Rightarrow Q held. To make the experiment less time consuming, in addition to the time limit of 10 sec. per PQE problem we imposed a few more constraints. The PQE problem of taking a clause out of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] terminated as soon as HH accumulated 5 clauses or more. Besides, processing a benchmark aborted when the summary number of clauses of all formulas HH generated for this benchmark reached 100 or the total run time of all PQE problems generated off โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] exceeded 2,000 sec.

Table 2: Invariant generation
pqe #bench results
solver marks local glob. not imp.
invar. invar. by Paโ€‹gโ€‹gP_{agg}
ds-pqe ย 98 5,556 2,678 2,309
eg-pqe ย 98 9,498 4,839 4,009
eg-pqe+ ย 98 9,303 4,773 3,940

Tableย 2 shows the results of the experiment. The third column gives the number of local single-clause invariants (i.e., the total number of clauses in all HH over all benchmarks). The fourth column shows how many local single-clause invariants turned out to be global. (Since global invariants were extracted from HH and the summary size of all HH could not exceed 100, the number of global invariants per benchmark could not exceed 100.) The last column gives the number of global invariants not implied by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}}. So, these invariants are candidates for checking if they are unwanted. Tableย 2 shows that ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} performed much better than ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE}.

8.3 Experiment 3

To prove an invariant PP true, IC3 conjoins it with clauses Q1,โ€ฆ,QnQ_{1}\!,\dots,\!Q_{n} to make PโˆงQ1โˆงโ‹ฏโˆงQnP\!\wedge Q_{1}\!\wedge\dots\wedge Q_{n} inductive. If IC3 succeeds, every QiQ_{i} is an invariant. Moreover, QiQ_{i} may be an unwanted invariant. The goal of the third experiment was to demonstrate that PQE and IC3, in general, produce different invariant clauses. The intuition here is twofold. First, IC3 generates clauses QiQ_{i} to prove a predefined invariant rather than find an unwanted one. Second, the closer PP to being inductive, the fewer new invariant clauses are generated by IC3. Consider the circuit N๐‘ก๐‘Ÿ๐‘–๐‘ฃ\mathit{N}_{\mathit{triv}} that simply stays in the initial state \vvโ€‹s๐‘–๐‘›๐‘–\vv{s}\!_{\mathit{ini}} (Sectionย 4). Any invariant satisfied by \vvโ€‹s๐‘–๐‘›๐‘–\vv{s}\!_{\mathit{ini}} is already inductive for N๐‘ก๐‘Ÿ๐‘–๐‘ฃ\mathit{N}_{\mathit{triv}}. So, IC3 will not generate a single new invariant clause. On the other hand, if the correct circuit is supposed to leave the initial state, N๐‘ก๐‘Ÿ๐‘–๐‘ฃ\mathit{N}_{\mathit{triv}} has unwanted invariants that our method will find.

In this experiment, we used IC3 to generate P๐‘Ž๐‘”๐‘”โˆ—P^{*}_{\mathit{agg}}, an inductive version of P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}}. The experiment showed that in 88% cases, an invariant clause generated by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} and not implied by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}} was not implied by P๐‘Ž๐‘”๐‘”โˆ—P^{*}_{\mathit{agg}} either. (See Appendixย 0.H.3 for more detail.)

9 Properties Mimicking Symbolic Simulation

Let Mโ€‹(X,V,W)M(X,V,W) be a combinational circuit where X,V,WX,V,W are internal, input and output variables. In this section, we describe generation of properties of MM that mimic symbolic simulationย [22]. Every such a property Qโ€‹(V)Q(V) specifies a cube of tests that produce the same values for a given subset of variables of WW. We chose generation of such properties because deciding if QQ is an unwanted property is, in general, simple. The procedure for generation of these properties is slightly different from the one presented in Sectionย 3.

Let Fโ€‹(X,V,W)F(X,V,W) be a formula specifying MM. Let Bโ€‹(W)B(W) be a clause. Let Hโ€‹(V)H(V) be a solution to the PQE problem of taking a clause CโˆˆFC\in F out of โˆƒXโ€‹โˆƒWโ€‹[FโˆงB]\exists{X}\exists{W}[F\wedge B]. That is, โˆƒXโ€‹โˆƒWโ€‹[FโˆงB]โ‰กHโˆง\mbox{$\exists{X}\exists{W}[F\wedge B]$}\equiv H\wedge โˆƒXโ€‹โˆƒWโ€‹[(Fโˆ–{C})โˆงB]\exists{X}\exists{W}[(F\setminus\mbox{$\{C\}$})\wedge B]. Let Qโ€‹(V)Q(V) be a clause of HH. Then MM has the property that for every full assignment \vvโ€‹v\vv{v} to VV falsifying QQ, it produces an output \vvโ€‹w\vv{w} falsifying BB (see Propositionย 3 of Appendixย 0.B). Suppose, for instance, Q=v1โˆจvยฏ10โˆจv30Q\!=\!v_{1}\vee\,\overline{v}_{10}\vee v_{30} and B=w2โˆจwยฏ40B\!=\!w_{2}\vee\,\overline{w}_{40}. Then for every \vvโ€‹v\vv{v} where v1=0,v10=1,v30=0v_{1}\!=\!0,v_{10}\!=\!1,\!v_{30}\!=\!0, the circuit MM produces an output where w2=0,w40=1w_{2}=0,w_{40}=1. Note that QQ is implied by FโˆงBF\wedge B rather than FF. So, it is a property of MM under constraint BB rather than MM alone. The property QQ is unwanted if there is an input falsifying QQ that should not produce an output falsifying BB.

To generate combinational circuits, we unfolded sequential circuits of the set of 98 benchmarks used in Sectionย 8 for invariant generation. Let NN be a sequential circuit. (We reuse the notation of Sectionย 4). Let Mkโ€‹(S0,V0,โ€ฆ,Skโˆ’1,Vkโˆ’1,Sk)M_{k}(S_{0},V_{0},\dots,S_{k-1},V_{k-1},S_{k}) denote the combinational circuit obtained by unfolding NN for kk time frames. Here Sj,VjS_{j},V_{j} are state and input variables of jj-th time frame respectively. Let FkF_{k} denote the formula Iโ€‹(S0)โˆงTโ€‹(S0,V0,S1)โˆงโ‹ฏโˆงTโ€‹(Skโˆ’1,Vkโˆ’1,Sk)I(S_{0})\wedge T(S_{0},V_{0},S_{1})\wedge\dots\wedge T(S_{k-1},V_{k-1},S_{k}) describing the unfolding of NN for kk time frames. Note that FkF_{k} specifies the circuit MkM_{k} above under the input constraint Iโ€‹(S0)I(S_{0}). Let Bโ€‹(Sk)B(S_{k}) be a clause. Let Hโ€‹(S0,V0,โ€ฆ,Vkโˆ’1)H(S_{0},V_{0},\dots,V_{k-1}) be a solution to the PQE problem of taking a clause CโˆˆFkC\in F_{k} out of formula โˆƒS1,kโ€‹[FkโˆงB]\exists{S_{1,k}}[F_{k}\wedge B]. Here S1,k=S1โˆชโ‹ฏโˆชSkS_{1,k}=S_{1}\cup\dots\cup S_{k}. That is, โˆƒS1,kโ€‹[FkโˆงB]โ‰กHโˆง\mbox{$\exists{S_{1,k}}[F_{k}\wedge B]$}\equiv H\wedge โˆƒS1,kโ€‹[(Fkโˆ–{C})โˆงB]\exists{S_{1,k}}[(F_{k}\setminus\mbox{$\{C\}$})\wedge B]. Let QQ be a clause of HH. Then for every assignment (\vvโ€‹s๐‘–๐‘›๐‘–\vv{s}\!_{\mathit{ini}},\vvโ€‹v0\vv{v_{0}},โ€ฆ,\vvโ€‹vkโˆ’1\vv{v}_{k-1}) falsifying QQ, the circuit MkM_{k} outputs \vvโ€‹sk\vv{s}\!_{k} falsifying BB. (Here \vvโ€‹s๐‘–๐‘›๐‘–\vv{s}\!_{\mathit{ini}} is the initial state of NN and \vvโ€‹sk\vv{s}\!_{k} is a state of the last time frame.)

In the experiment, we used ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE},๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} to solve 1,586 PQE problems described above. In Tableย 3, we give a sample of results by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}. (More details about this experiment can be found in Appendixย 0.I.) Below, we use the first line of Tableย 3 to explain its structure. The first column gives the benchmark name (6s326). The next column shows that 6s326 has 3,342 latches. The third column gives the number of time frames used to produce a combinational circuit MkM_{k} (here k=20k=20). The next column shows that the clause BB introduced above consisted of 15 literals of variables from SkS_{k}. (Here and below we still use the index kk assuming that k=20k=20.) The literals of BB were generated randomly. When picking the length of BB we just tried to simulate the situation where one wants to set a particular subset of output variables of MkM_{k} to specified values. The next two columns give the size of the subcircuit Mkโ€ฒM^{\prime}_{k} of MkM_{k} that feeds the output variables present in BB. When computing a property HH we took a clause out of formula โˆƒS1,kโ€‹[Fkโ€ฒโˆงB]\exists{S_{1,k}}[F^{\prime}_{k}\wedge B] where Fkโ€ฒF^{\prime}_{k} specifies Mkโ€ฒM^{\prime}_{k} instead of formula โˆƒS1,kโ€‹[FkโˆงB]\exists{S_{1,k}}[F_{k}\wedge B] where FkF_{k} specifies MkM_{k}. (The logic of MkM_{k} not feeding a variable of BB is irrelevant for computing HH.) The first column of the pair gives the number of gates in Mkโ€ฒM^{\prime}_{k} (i.e., 348,479). The second column provides the number of input variables feeding Mkโ€ฒM^{\prime}_{k} (i.e., 1,774). Here we count only variables of V0โˆชโ‹ฏโˆชVkโˆ’1V_{0}\cup\dots\cup V_{k-1} and ignore those of S0S_{0} since the latter are already assigned values specifying the initial state \vvโ€‹s๐‘–๐‘›๐‘–\vv{s}\!_{\mathit{ini}} of NN.

Table 3: Property generation for combinational
circuits
name lat- time size subc. Mkโ€ฒM^{\prime}_{k} results
ches fra- of gates inp. min max time 3-val.
mes BB vars (s.) sim.
6s326 3,342 ย 20 ย 15 348,479 1,774 27 28 2.9 ย no
6s40m 5,608 ย 20 ย 15 406,474 3,450 27 29 1.1 ย no
6s250 6,185 ย 20 ย 15 556,562 2,456 50 54 0.8 ย no
6s395 463 ย 30 ย 15 36,088 569 24 26 0.7 ย yes
6s339 1,594 ย 30 ย 15 179,543 3,978 70 71 3.1 ย no
6s292 3,190 ย 30 ย 15 154,014 978 86 89 1.1 ย no
6s143 260 ย 40 ย 15 551,019 16,689 526 530 2.5 ย yes
6s372 1,124 ย 40 ย 15 295,626 2,766 513 518 1.7 ย no
6s335 1,658 ย 40 ย 15 207,787 2,863 120 124 6.7 ย no
6s391 2,686 ย 40 ย 15 240,825 7,579 340 341 8.9 ย no

The next four columns show the results of taking a clause out ofโˆƒS1,kโ€‹[Fkโ€ฒโˆงB]\exists{S_{1,k}}[F^{\prime}_{k}\!\wedge\!B]. For each PQE problem the time limit was set to 10 sec. Besides, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} terminated as soon as 5 clauses of propertyHโ€‹(S0,V0,โ€ฆ,Vkโˆ’1)H(S_{0},V_{0},\dots,V_{k-1}) were generated. The first three columns out of four describe the minimum and maximum sizes of clauses in HH and the run time of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}. So, it took for ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} 2.9 sec. to produce a formula HH containing clauses of sizes from 27 to 28 variables. A clause QQ of HH with 27 variables, for instance, specifies 217472^{1747} tests falsifying QQ that produce the same output of Mkโ€ฒM^{\prime}_{k} (falsifying the clause BB). Here 1747=1774โˆ’271747=1774-27 is the number of input variables of Mkโ€ฒM^{\prime}_{k} not present in QQ. The last column shows that at least one clause QQ of HH specifies a property that cannot be produced by 3-valued simulation (a version of symbolic simulationย [22]). To prove this, one just needs to set the input variables of Mkโ€ฒM^{\prime}_{k} present in QQ to the values falsifying QQ and run 3-valued simulation. (The remaining input variables of Mkโ€ฒM^{\prime}_{k} are assigned a donโ€™t-care value.) If after 3-valued simulation some output variable of Mkโ€ฒM^{\prime}_{k} is assigned a donโ€™t-care value, the property specified by QQ cannot be produced by 3-valued simulation.

Running ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} on the 1,586 PQE problems mentioned above showed that a) ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} performed poorly producing properties only for 28% of problems; b) ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} showed much better results by generating properties for 62% and 66% of problems respectively. When ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} succeeded in producing properties, the latter could not be obtained by 3-valued simulation in 74% and 78% of cases respectively.

10 Some Background

In this section, we discuss some research relevant to PQE and property generation. Information on BDD based QE can be found inย [23, 24]. SAT based QE is described in [11, 25, 26, 27, 28, 29, 30, 31, 20]. Our first PQE solver called ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} was introduced inย [1]. It was based on redundancy based reasoning presented inย [32] in terms of variables and inย [33] in terms of clauses. The main flaw of ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is as follows. Consider taking a clause CC out of โˆƒXโ€‹[F]\exists{X}[F]. Suppose ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} proved CC redundant in a subspace where FF is satisfiable and some quantified variables are assigned. The problem is that ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} cannot simply assume that CC is redundant every time it re-enters this subspaceย [34]. The root of the problem is that redundancy is a structural rather than semantic property. That is, redundancy of a clause in a formula ฮพ\xi (quantified or not) does not imply such redundancy in every formula logically equivalent to ฮพ\xi. Since our current implementation of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} uses ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} as a subroutine, it has the same learning problem. We showed inย [35] that this problem can be addressed by the machinery of certificate clauses. So, the performance of PQE can be drastically improved via enhanced learning in subspaces where FF is satisfiable.

We are unaware of research on property generation for combinational circuits. As for invariants, the existing procedures typically generate some auxiliary desired invariants to prove a predefined property (whereas our goal is to generate invariants that are unwanted). For instance, they generate loop invariantsย [36] or invariants relating internal points of circuits checked for equivalence ย [37]. Another example of auxiliary invariants are clauses generated by IC3 to make an invariant inductiveย [16]. As we showed in Subsectionย 8.3, the invariants produced by PQE are, in general, different from those built by IC3.

11 Conclusions And Directions For Future Research

We consider Partial Quantifier Elimination (PQE) on propositional CNF formulas with existential quantifiers. In contrast to complete quantifier elimination, PQE allows to unquantify a part of the formula. We show that PQE can be used to generate properties of combinational and sequential circuits. The goal of property generation is to check if there is an unwanted property identifying a bug. We used PQE to generate an unwanted invariant for a FIFO buffer exposing a non-trivial bug. We also applied PQE to invariant generation for HWMCC benchmarks. Finally, we used PQE to generate properties of combinational circuits mimicking symbolic simulation. Our experiments show that PQE can efficiently generate properties for realistic designs.

There are at least three directions for future research. The first direction is to improve the performance of PQE solving. As we mentioned in Sectionย 10, the most promising idea here is to enhance the power of learning in subspaces where the formula is satisfiable. The second direction is to use the improved PQE solvers to design new, more efficient algorithms for well-known problems like SAT, model checking and equivalence checking. The third direction is to look for new problems that can be solved by PQE.

References

  • [1] E.ย Goldberg and P.ย Manolios, โ€œPartial quantifier elimination,โ€ in Proc. of HVC-14.ย ย ย Springer-Verlag, 2014, pp. 148โ€“164.
  • [2] W.ย Craig, โ€œThree uses of the herbrand-gentzen theorem in relating model theory and proof theory,โ€ The Journal of Symbolic Logic, vol.ย 22, no.ย 3, pp. 269โ€“285, 1957.
  • [3] K.ย McMillan, โ€œInterpolation and sat-based model checking,โ€ in CAV-03.ย ย ย Springer, 2003, pp. 1โ€“13.
  • [4] E.ย Goldberg, โ€œProperty checking by logic relaxation,โ€ Tech. Rep. arXiv:1601.02742 [cs.LO], 2016.
  • [5] E.ย Goldberg and P.ย Manolios, โ€œSoftware for quantifier elimination in propositional logic,โ€ in ICMS-2014,Seoul, South Korea, August 5-9, 2014, pp. 291โ€“294.
  • [6] E.ย Goldberg, โ€œEquivalence checking by logic relaxation,โ€ in FMCAD-16, 2016, pp. 49โ€“56.
  • [7] โ€”โ€”, โ€œProperty checking without inductive invariant generation,โ€ Tech. Rep. arXiv:1602.05829 [cs.LO], 2016.
  • [8] B.ย L. Synthesis and V.ย Group, โ€œABC: A system for sequential synthesis and verification,โ€ 2017, http://www.eecs.berkeley.edu/โˆผ\simalanmi/abc.
  • [9] O.ย Kullmann, โ€œNew methods for 3-sat decision and worst-case analysis,โ€ Theor. Comput. Sci., vol. 223, no. 1-2, pp. 1โ€“72, 1999.
  • [10] G.ย Tseitin, โ€œOn the complexity of derivation in the propositional calculus,โ€ Zapiski nauchnykh seminarov LOMI, vol.ย 8, pp. 234โ€“259, 1968, english translation of this volume: Consultants Bureau, N.Y., 1970, pp. 115โ€“125.
  • [11] K.ย McMillan, โ€œApplying sat methods in unbounded symbolic model checking,โ€ in Proc. of CAV-02.ย ย ย Springer-Verlag, 2002, pp. 250โ€“264.
  • [12] The source of ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, http://eigold.tripod.com/software/ds-pqe.tar.gz.
  • [13] The source of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, http://eigold.tripod.com/software/eg-pqe.1.0.tar.gz.
  • [14] The source of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}, http://eigold.tripod.com/software/eg-pqe-pl.1.0.tar.gz.
  • [15] N.ย Eรฉn and N.ย Sรถrensson, โ€œAn extensible sat-solver,โ€ in SAT, Santa Margherita Ligure, Italy, 2003, pp. 502โ€“518.
  • [16] A.ย R. Bradley, โ€œSat-based model checking without unrolling,โ€ in VMCAI, 2011, pp. 70โ€“87.
  • [17] An implementation of IC3 by A. Bradley, https://github.com/arbrad/IC3ref.
  • [18] M.ย Aniche, Effective Software Testing: A developerโ€™s guide.ย ย ย Manning publications, 2022.
  • [19] HardWare Model Checking Competition 2013 (HWMCC-13), http://fmv.jku.at/hwmcc13/.
  • [20] M.ย Rabe, โ€œIncremental determinization for quantifier elimination and functional synthesis,โ€ in CAV, 2019.
  • [21] CADET, https://github.com/MarkusRabe/cadet.
  • [22] R.ย Bryant, โ€œSymbolic simulationโ€”techniques and applications,โ€ in DAC-90, 1990, pp. 517โ€“521.
  • [23] โ€”โ€”, โ€œGraph-based algorithms for Boolean function manipulation,โ€ IEEE Transactions on Computers, vol. C-35, no.ย 8, pp. 677โ€“691, August 1986.
  • [24] P.ย Chauhan, E.ย Clarke, S.ย Jha, J.ย Kukula, H.ย Veith, and D.ย Wang, โ€œUsing combinatorial optimization methods for quantification scheduling,โ€ ser. CHARME-01, 2001, pp. 293โ€“309.
  • [25] H.ย Jin and F.Somenzi, โ€œPrime clauses for fast enumeration of satisfying assignments to boolean circuits,โ€ ser. DAC-05, 2005, pp. 750โ€“753.
  • [26] M.ย Ganai, A.Gupta, and P.Ashar, โ€œEfficient sat-based unbounded symbolic model checking using circuit cofactoring,โ€ ser. ICCAD-04, 2004, pp. 510โ€“517.
  • [27] J.ย Jiang, โ€œQuantifier elimination via functional composition,โ€ in Proceedings of the 21st International Conference on Computer Aided Verification, ser. CAV-09, 2009, pp. 383โ€“397.
  • [28] J.ย Brauer, A.ย King, and J.ย Kriener, โ€œExistential quantification as incremental sat,โ€ ser. CAV-11, 2011, pp. 191โ€“207.
  • [29] W.ย Klieber, M.ย Janota, J.Marques-Silva, and E.ย Clarke, โ€œSolving qbf with free variables,โ€ in CP, 2013, pp. 415โ€“431.
  • [30] N.ย Bjorner, M.ย Janota, and W.ย Klieber, โ€œOn conflicts and strategies in qbf,โ€ in LPAR, 2015.
  • [31] N.ย Bjorner and M.ย Janota, โ€œPlaying with quantified satisfaction,โ€ in LPAR, 2015.
  • [32] E.ย Goldberg and P.ย Manolios, โ€œQuantifier elimination by dependency sequents,โ€ in FMCAD-12, 2012, pp. 34โ€“44.
  • [33] โ€”โ€”, โ€œQuantifier elimination via clause redundancy,โ€ in FMCAD-13, 2013, pp. 85โ€“92.
  • [34] E.ย Goldberg, โ€œQuantifier elimination with structural learning,โ€ Tech. Rep. arXiv: 1810.00160 [cs.LO], 2018.
  • [35] โ€”โ€”, โ€œPartial quantifier elimination by certificate clauses,โ€ Tech. Rep. arXiv:2003.09667 [cs.LO], 2020.
  • [36] I.ย Dillig, T.ย Dillig, B.ย Li, and K.ย McMillan, โ€œInductive invariant generation via abductive inference,โ€ vol.ย 48, 10 2013, pp. 443โ€“456.
  • [37] J.Baumgartner, H.ย Mony, M.ย Case, J.ย Sawada, and K.ย Yorav, โ€œScalable conditional equivalence checking: An automated invariant-generation based approach,โ€ in 2009 Formal Methods in Computer-Aided Design, 2009, pp. 120โ€“127.
  • [38] A.ย Biere, A.ย Cimatti, E.ย Clarke, M.ย Fujita, and Y.ย Zhu, โ€œSymbolic model checking using sat procedures instead of bdds,โ€ in DAC, 1999, pp. 317โ€“320.

Appendix

Appendix 0.A PQE And Interpolation

In this appendix, we recall the observation ofย [4] that interpolation is a special case of PQE. Let Aโ€‹(X,Y)โˆงBโ€‹(Y,Z)A(X,Y)\wedge B(Y,Z) be an unsatisfiable formula. Let Iโ€‹(Y)I(Y) be a formula such that AโˆงBโ‰กIโˆงBA\wedge B\equiv I\wedge B and Aโ‡’IA\Rightarrow I. Then II is called an interpolantย [2]. Now, let us show that interpolation can be described in terms of PQE. Consider the formula โˆƒWโ€‹[AโˆงB]\exists{W}[A\wedge B] where AA and BB are the formulas above and W=XโˆชZW=X\cup Z. Let Aโˆ—โ€‹(Y)A^{*}(Y) be obtained by taking AA out of the scope of quantifiers i.e. โˆƒWโ€‹[AโˆงB]\exists{W}[A\wedge B] โ‰กAโˆ—โˆง\equiv A^{*}\wedgeโˆƒWโ€‹[B]\exists{W}[B]. Since AโˆงBA\wedge B is unsatisfiable, Aโˆ—โˆงBA^{*}\wedge B is unsatisfiable too. So, AโˆงBโ‰กAโˆ—โˆงBA\wedge B\equiv A^{*}\wedge B. If Aโ‡’Aโˆ—A\Rightarrow A^{*}, then Aโˆ—A^{*} is an interpolant.

The general case of PQE that takes AA out of โˆƒWโ€‹[AโˆงB]\exists{W}[A\wedge B] is different from the instance above in three aspects. First, one does not assume that AโˆงBA\wedge B is unsatisfiable. Second, one does not assume that ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(B)โŠ‚๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(AโˆงB)\mbox{$\mathit{Vars}(B)$}\subset\mbox{$\mathit{Vars}(A\wedge B)$}. In other words, in general, PQE does not remove any variables from the original formula. Third, a solution Aโˆ—A^{*} is implied by AโˆงBA\wedge B rather than by AA alone. Summarizing, one can say that interpolation is a special case of PQE.

Appendix 0.B Proofs Of Propositions

Proposition 1

Let HH be a solution to the PQE problem of Definitionย 9. That is โˆƒXโ€‹[F]โ‰กHโˆงโˆƒXโ€‹[Fโˆ–G]\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus G]$}. Then Fโ‡’HF\Rightarrow H (i.e. FF implies HH).

Proof

By conjoining both sides of the equality with HH one concludes thatHโˆงโˆƒXโ€‹[F]โ‰กHโˆงโˆƒXโ€‹[Fโˆ–G]H\wedge\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus G]$} and hence HโˆงโˆƒXโ€‹[F]โ‰กโˆƒXโ€‹[F]H\wedge\mbox{$\exists{X}[F]$}\equiv\mbox{$\exists{X}[F]$}. Then โˆƒXโ€‹[F]โ‡’H\mbox{$\exists{X}[F]$}\Rightarrow H and thus Fโ‡’HF\Rightarrow H.

Proposition 2

Let a clause CC be blocked in a formula Fโ€‹(X,Y)F(X,Y) with respect to a variable xโˆˆXx\in X. Then CC is redundant in โˆƒXโ€‹[F]\exists{X}[F] i.e. โˆƒXโ€‹[Fโˆ–{C}]\exists{X}[F\setminus\mbox{$\{C\}$}] โ‰ก\equiv โˆƒXโ€‹[F]\exists{X}[F].

Proof

It was shown inย [9] that adding a clause Bโ€‹(X)B(X) blocked in Gโ€‹(X)G(X) to the formula โˆƒXโ€‹[G]\exists{X}[G] does not change the value of this formula. This entails that removing a clause Bโ€‹(X)B(X) blocked in Gโ€‹(X)G(X) does not change the value of โˆƒXโ€‹[G]\exists{X}[G] either. So, BB is redundant in โˆƒXโ€‹[G]\exists{X}[G]. Let \vvโ€‹y\vv{y} be a full assignment to YY. Then the clause CC is either satisfied by \vvโ€‹y\vv{y} or Cyโ†’C_{\vec{y}} is blocked in Fyโ†’F_{\vec{y}} with respect to xx. (The latter follows from the definition of a blocked clause.) In either case Cyโ†’C_{\vec{y}} is redundant in โˆƒXโ€‹[Fyโ†’]\exists{X}[\mbox{$F_{\vec{y}}$}]. Since Cyโ†’C_{\vec{y}} is redundant in โˆƒXโ€‹[Fyโ†’]\exists{X}[\mbox{$F_{\vec{y}}$}] in every subspace \vvโ€‹y\vv{y}, CC is redundant in โˆƒXโ€‹[F]\exists{X}[F].

Proposition 3

Let Mโ€‹(X,V,W)M(X,V,W) be a combinational circuit where X,V,WX,V,W are the internal, input and output variables. Let Fโ€‹(X,V,W)F(X,V,W) be a formula specifying MM. Let Bโ€‹(W)B(W) be a clause. Let Hโ€‹(V)H(V) be a formula obtained by taking a clause CโˆˆFC\in F out of โˆƒXโ€‹โˆƒWโ€‹[FโˆงB]\exists{X}\exists{W}[F\wedge B]. That is โˆƒXโ€‹โˆƒWโ€‹[FโˆงB]\exists{X}\exists{W}[F\wedge B] โ‰กHโˆง\equiv H\wedge โˆƒXโ€‹โˆƒWโ€‹[(Fโˆ–{C})โˆงB]\exists{X}\exists{W}[(F\setminus\mbox{$\{C\}$})\wedge B]. Let Qโ€‹(V)Q(V) be a clause of HH. Then for every full assignment \vvโ€‹v\vv{v} to VV falsifying QQ, the circuit MM outputs an assignment \vvโ€‹w\vv{w} falsifying the clause BB.

Proof

From Propositionย 1 it follows that FโˆงBโ‡’HF\wedge B\Rightarrow H and hence FโˆงBโ‡’QF\wedge B\Rightarrow Q. This entails that Qยฏโ‡’BยฏโˆจFยฏ\overline{Q}\Rightarrow\overline{B}\vee\overline{F}. Let \vvโ€‹v\vv{v} be a full assignment to VV i.e. an input to MM. Let (\vvโ€‹x\vv{x},\vvโ€‹v\vv{v},\vvโ€‹w\vv{w}) be the execution trace produced by MM under the input \vvโ€‹v\vv{v}. Here \vvโ€‹x\vv{x},\vvโ€‹w\vv{w} are full assignments to XX and WW respectively. Suppose, \vvโ€‹v\vv{v} satisfies Qยฏ\overline{Q} (and so falsifies QQ). Then (\vvโ€‹x\vv{x},\vvโ€‹v\vv{v},\vvโ€‹w\vv{w}) satisfies BยฏโˆจFยฏ\overline{B}\vee\overline{F}. Since (\vvโ€‹x\vv{x},\vvโ€‹v\vv{v},\vvโ€‹w\vv{w}) is an execution trace, it satisfies FF and so falsifies Fยฏ\overline{F}. This entails that (\vvโ€‹x\vv{x},\vvโ€‹v\vv{v},\vvโ€‹w\vv{w}) (and specifically \vvโ€‹w\vv{w}) satisfies Bยฏ\overline{B} and hence falsifies BB.

Appendix 0.C Examples Of Problems That Reduce To PQE

SAT-solving by PQEย [1]. Consider the SAT problem of checking if formula โˆƒXโ€‹[Fโ€‹(X)]\exists{X}[F(X)] is true. One can view traditional SAT-solving as proving all clauses redundant in โˆƒXโ€‹[F]\exists{X}[F] e.g. by finding a satisfying assignment or by deriving an empty clause and adding it to FF. The reduction to PQE below facilitates developing an incremental SAT-algorithm that needs to prove redundancy only for a fraction of clauses.

Let \vvโ€‹x\vv{x} be a full assignment to XX and GG denote the clauses of FF falsified by \vvโ€‹x\vv{x}. Checking the satisfiability of FF reduces to taking GG out of the scope of quantifiers i.e. to finding HH such that โˆƒXโ€‹[F]โ‰กHโˆงโˆƒXโ€‹[Fโˆ–G]\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus G]$}. Since all variables of FF are quantified in โˆƒXโ€‹[F]\exists{X}[F], the formula HH is a Boolean constant 0 or 1. If H=0H=0, then FF is unsatisfiable. If H=1H\!=\!1, then FF is satisfiable because Fโˆ–GF\setminus G is satisfied by \vvโ€‹x\vv{x}.

Equivalence checking by PQEย [6]. Let Nโ€ฒโ€‹(Xโ€ฒ,Vโ€ฒ,wโ€ฒ)N^{\prime}(X^{\prime},V^{\prime},w^{\prime}) and Nโ€ฒโ€ฒโ€‹(Xโ€ฒโ€ฒ,Vโ€ฒโ€ฒ,wโ€ฒโ€ฒ)N^{\prime\prime}(X^{\prime\prime},V^{\prime\prime},w^{\prime\prime}) be single-output combinational circuits to check for equivalence. Here Xโ€ฒ,Vโ€ฒX^{\prime},V^{\prime} are the sets of internal and input variables and wโ€ฒw^{\prime} is the output variable of Nโ€ฒN^{\prime}. (Definition of Xโ€ฒโ€ฒ,Vโ€ฒโ€ฒ,wโ€ฒโ€ฒX^{\prime\prime},V^{\prime\prime},w^{\prime\prime} for Nโ€ฒโ€ฒN^{\prime\prime} is the same.) The reduction to PQE below facilitates the design of a complete algorithm able to exploit the similarity of Nโ€ฒN^{\prime} and Nโ€ฒโ€ฒN^{\prime\prime}. This is important because the current equivalence checkers exploiting such similarity are incomplete. If Nโ€ฒN^{\prime} and Nโ€ฒโ€ฒN^{\prime\prime} are not โ€œsimilar enoughโ€, e.g. they have no functionally equivalent internal points, the equivalence checker invokes a complete (but inefficient) procedure ignoring similarities between Nโ€ฒN^{\prime} and Nโ€ฒโ€ฒN^{\prime\prime}.

Let ๐‘’๐‘žโ€‹(Vโ€ฒ,Vโ€ฒโ€ฒ)\mathit{eq}(V^{\prime},V^{\prime\prime}) specify a formula such that ๐‘’๐‘žโ€‹(\vvโ€‹vโ€ฒ,\vvโ€‹vโ€ฒโ€ฒ)\mathit{eq}(\mbox{$\vv{v}$}\,^{\prime},\mbox{$\vv{v}$}\,^{\prime\prime}) = 1 iff \vvโ€‹vโ€ฒ=\vvโ€‹vโ€ฒโ€ฒ\mbox{$\vv{v}$}\,^{\prime}=\mbox{$\vv{v}$}\,^{\prime\prime} where \vvโ€‹vโ€ฒ,\vvโ€‹vโ€ฒโ€ฒ\mbox{$\vv{v}$}\,^{\prime},\mbox{$\vv{v}$}\,^{\prime\prime} are full assignments to Vโ€ฒV^{\prime} and Vโ€ฒโ€ฒV^{\prime\prime} respectively. Let formulas Gโ€ฒโ€‹(Xโ€ฒ,Vโ€ฒ,wโ€ฒ)G^{\prime}(X^{\prime},V^{\prime},w^{\prime}) and Gโ€ฒโ€ฒโ€‹(Xโ€ฒโ€ฒ,Vโ€ฒโ€ฒ,wโ€ฒโ€ฒ)G^{\prime\prime}(X^{\prime\prime},V^{\prime\prime},w^{\prime\prime}) specify Nโ€ฒN^{\prime} and Nโ€ฒโ€ฒN^{\prime\prime} respectively. (As usual, we assume that a formula GG specifying a circuit NN is obtained by Tseitin transformationsย [10], see Sectionย 3.) Let hโ€‹(wโ€ฒ,wโ€ฒโ€ฒ)h(w^{\prime},w^{\prime\prime}) be a formula obtained by taking eq out of โˆƒZโ€‹[๐‘’๐‘žโˆงGโ€ฒโˆงGโ€ฒโ€ฒ]\exists{Z}[\mathit{eq}\wedge G^{\prime}\wedge G^{\prime\prime}] where Z=Xโ€ฒโˆชVโ€ฒโˆชXโ€ฒโ€ฒโˆชVโ€ฒโ€ฒZ=X^{\prime}\cup V^{\prime}\cup X^{\prime\prime}\cup V^{\prime\prime}. That is โˆƒZโ€‹[๐‘’๐‘žโˆงGโ€ฒโˆงGโ€ฒโ€ฒ]\exists{Z}[\mathit{eq}\!\wedge\!G^{\prime}\!\wedge\!G^{\prime\prime}] โ‰ก\equiv hโˆงโˆƒZโ€‹[Gโ€ฒโˆงGโ€ฒโ€ฒ]h\wedge\mbox{$\exists{Z}[G^{\prime}\wedge G^{\prime\prime}]$}. If hโ‡’(wโ€ฒโ‰กwโ€ฒโ€ฒ)h\Rightarrow(w^{\prime}\equiv w^{\prime\prime}), then Nโ€ฒN^{\prime} and Nโ€ฒโ€ฒN^{\prime\prime} are equivalent. Otherwise, Nโ€ฒN^{\prime} and Nโ€ฒโ€ฒN^{\prime\prime} are inequivalent, unless they are identical constants i.e. wโ€ฒโ‰กwโ€ฒโ€ฒโ‰ก1w^{\prime}\!\equiv\!w^{\prime\prime}\!\equiv 1 or wโ€ฒโ‰กwโ€ฒโ€ฒโ‰ก0w^{\prime}\!\equiv w^{\prime\prime}\!\equiv\!0. It is formally proved inย [6] that the more similar Nโ€ฒ,Nโ€ฒโ€ฒN^{\prime},N^{\prime\prime} are (where similarity is defined in the most general sense), the easier taking ๐‘’๐‘ž\mathit{eq} out of โˆƒZโ€‹[๐‘’๐‘žโˆงGโ€ฒโˆงGโ€ฒโ€ฒ]\exists{Z}[\mathit{eq}\wedge G^{\prime}\wedge G^{\prime\prime}] becomes.

Model checking by PQEย [7]. One can use PQE to find the reachability diameter of a sequential circuit without computing the set of all reachable states. So, one can prove an invariant by PQE without generating a stronger invariant that is inductive. Let Tโ€‹(Sโ€ฒ,V,Sโ€ฒโ€ฒ)T(S^{\prime},V,S^{\prime\prime}) denote the transition relation of a sequential circuit NN where Sโ€ฒ,Sโ€ฒโ€ฒS^{\prime},S^{\prime\prime} are the present and next state variables and VV specifies the (combinational) input variables. Let Iโ€‹(S)I(S) specify the initial states of NN. For the sake of simplicity, we assume that NN can stutter i.e. for every state \vvโ€‹s\vv{s} there exists a full assignment \vvโ€‹v\vv{v} to VV such that Tโ€‹(\vvโ€‹s,\vvโ€‹v,\vvโ€‹s)=1T(\mbox{$\vv{s}$},\mbox{$\vv{v}$},\mbox{$\vv{s}$})=1, (Then the sets of states reachable in mm transitions and at most mm transitions are identical. If TT has no stuttering, it can be easily introduced by adding a variable to VV.)

Let ๐ท๐‘–๐‘Ž๐‘šโ€‹(I,T)\mathit{Diam}(I,T) denote the reachability diameter for initial states II and transition relation TT. That is every state of the circuit NN can be reached in at most ๐ท๐‘–๐‘Ž๐‘šโ€‹(I,T)\mathit{Diam}(I,T) transitions. Given a number mm, one can use PQE to decide if ๐ท๐‘–๐‘Ž๐‘šโ€‹(I,T)<m\mbox{$\mathit{Diam}(I,T)$}<m. This is done by checking if I1I_{1} is redundant in โˆƒXmโ€‹[I0โˆงI1โˆงTm]\exists{\mbox{$X_{m}$}}[I_{0}\wedge I_{1}\wedge T_{m}]. Here I0I_{0} and I1I_{1} specify the initial states of NN in terms of variables of S0S_{0} and S1S_{1} respectively, Xm=S0โˆชV0โˆชโ‹ฏโˆชSmโˆ’1โˆชVmโˆ’1\mbox{$X_{m}$}=S_{0}\cup V_{0}\cup\dots\cup S_{m-1}\cup V_{m-1} and Tm=Tโ€‹(S0,V0,S1)โˆงโ‹ฏโˆงTโ€‹(Smโˆ’1,Vmโˆ’1,Sm)T_{m}=T(S_{0},V_{0},S_{1})\wedge\dots\wedge T(S_{m-1},V_{m-1},S_{m}). If I1I_{1} is redundant, then ๐ท๐‘–๐‘Ž๐‘šโ€‹(I,T)<m\mbox{$\mathit{Diam}(I,T)$}<m.

The idea above can be used, for instance, to prove an invariant PP true in an IC3-like manner (i.e. by constraining PP) but without generating an inductive invariant. To prove PP true, it suffices to constrain PP to a formula HH such thata) Iโ‡’Hโ‡’PI\Rightarrow H\Rightarrow P, b) ๐ท๐‘–๐‘Ž๐‘šโ€‹(H,T)<m\mbox{$\mathit{Diam}(H,T)$}<m and c) no state falsifying PP can be reached from a state satisfying HH in mโˆ’1m\!-\!1 transitions. The conditions b) and c) can be verified by PQE and bounded model checkingย [38] respectively. In the special case where HH meets the three conditions above for m=1m=1, it is an inductive invariant.

Appendix 0.D Deciding If A Property Is Unwanted

In this appendix, we give the main idea of a procedure for deciding if a property is unwanted. We reuse the notation of Sectionย 3. That is, we consider property generation for a combinational circuit Mโ€‹(X,V,W)M(X,V,W) specified by formula Fโ€‹(X,V,W)F(X,V,W). Here X,V,WX,V,W denote the sets of the internal, input, and output variables of MM respectively. For the sake of simplicity, we consider generation of a property Hโ€‹(W)H(W) depending only on output variables of MM. Such a property can be obtained by taking a clause CC out of formula โˆƒXโ€‹โˆƒVโ€‹[F]\exists{X}\exists{V}[F] (where only variables of WW are not quantified). So, โˆƒXโ€‹โˆƒVโ€‹[F]โ‰กHโˆงโˆƒXโ€‹โˆƒVโ€‹[Fโˆ–{C}]\mbox{$\exists{X}\exists{V}[F]$}\equiv H\wedge\mbox{$\exists{X}\exists{V}[F\setminus\mbox{$\{C\}$}]$}.

Note that every clause of HH is a property too. HH is an unwanted property if and only if one of its clauses is an unwanted property. Let QโˆˆHQ\in H be a single-clause unwanted property (and so MM has a bug). This property says that MM never outputs an assignment falsifying QQ. To prove QQ unwanted one needs to find a test \vvโ€‹v๐‘๐‘’\vv{v}\!_{\mathit{be}} for which a correct version of MM would produce an output falsifying QQ. Here โ€™beโ€™ stands for โ€™bug-exposingโ€™. (If QQ is a desired property, such \vvโ€‹v๐‘๐‘’\vv{v}\!_{\mathit{be}} cannot be produced. So, the failure to find \vvโ€‹v๐‘๐‘’\vv{v}\!_{\mathit{be}} is an argument in favor of QQ being a desired property.)

A test \vvโ€‹v๐‘๐‘’\vv{v}\!_{\mathit{be}} can be found as follows. Since FF implies QQ, the formula FโˆงQยฏF\wedge\overline{Q} is unsatisfiable (and so MM cannot produce an output falsifying QQ). Let Fโ€ฒF^{\prime} be a formula obtained by removing some clauses from FF for which the formula Fโ€ฒโˆงQยฏF^{\prime}\wedge\overline{Q} is satisfiable. Then means that the circuit Mโ€ฒM^{\prime} specified by Fโ€ฒF^{\prime} can produce an output falsifying QQ. The intuition here is that if QQ is unwanted property, the clauses removed from FF to obtain Fโ€ฒF^{\prime} relate to the buggy part of MM. (Note that the circuit Mโ€ฒM^{\prime} specified by Fโ€ฒF^{\prime} is non-deterministic. The reason is that Fโ€ฒF^{\prime} can be satisfied by assignments (\vvโ€‹x1\vv{x}_{1},\vvโ€‹v\vv{v},\vvโ€‹w1\vv{w}_{1}) and (\vvโ€‹x2\vv{x}_{2},\vvโ€‹v\vv{v},\vvโ€‹w2\vv{w}_{2}) with the same input \vvโ€‹v\vv{v} and different outputs \vvโ€‹w1\vv{w}_{1} and \vvโ€‹w2\vv{w}_{2}).

Assume that all clauses making up the buggy part of MM has been removed from FF when obtaining Fโ€ฒF^{\prime}. Then there is an assignment (\vvโ€‹x\vv{x},\vvโ€‹v๐‘๐‘’\vv{v}\!_{\mathit{be}},\vvโ€‹w\vv{w}) satisfying Fโ€ฒโˆงQยฏF^{\prime}\wedge\overline{Q}. That is a correct version of MM would produce the assignment \vvโ€‹w\vv{w} falsifying QQ under the input \vvโ€‹v๐‘๐‘’\vv{v}\!_{\mathit{be}}. So, to obtain \vvโ€‹v๐‘๐‘’\vv{v}\!_{\mathit{be}} one needs to build the formula Fโ€ฒF^{\prime} above and examine the input part \vvโ€‹v\vv{v} of assignments satisfying Fโ€ฒโˆงQยฏF^{\prime}\wedge\overline{Q}.

The formula Fโ€ฒF^{\prime} can be obtained as follows. Recall that by our assumption Fโˆ–{C}โ‡’ฬธQF\setminus\mbox{$\{C\}$}\not\Rightarrow Q (see Subsectionย 3.2 and Remarkย 1). So the formula Fโ€ฒโˆงQยฏF^{\prime}\wedge\overline{Q} where Fโ€ฒ=Fโˆ–{C}F^{\prime}=F\setminus\mbox{$\{C\}$} is satisfiable. However, one may not extract a test \vvโ€‹v๐‘๐‘’\vv{v}\!_{\mathit{be}} from assignments satisfying Fโ€ฒโˆงQยฏF^{\prime}\wedge\overline{Q} because CC may not be the only clause making up the buggy part of MM. A good heuristic for forming Fโ€ฒF^{\prime} is to remove from FF every clause BB such that Fโˆ–{B}โ‡’ฬธQF\setminus\mbox{$\{B\}$}\not\Rightarrow Q. (The latter means that one could obtain the property QQ by taking out the clause BB from โˆƒXโ€‹โˆƒVโ€‹[F]\exists{X}\exists{V}[F]). However, the topic of obtaining formula Fโ€ฒF^{\prime} and producing a test \vvโ€‹v๐‘๐‘’\vv{v}\!_{\mathit{be}} needs further research.

Appendix 0.E Combining PQE With Clause Splitting

In this appendix, we consider combining PQE with clause splitting mentioned in Subsectionย 3.3. We show that the corresponding PQE problem of taking out a clause produced by splitting is solved by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} in linear time. We also show that if this clause is not redundant, the solution produced by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is a single-test property.

Here we reuse the notation of Sectionย 3 but, for the sake of simplicity, consider a single-output combinational circuit. Let Mโ€‹(X,V,w)M(X,V,w) be such a circuit where XX and VV specify the internal and input variables respectively and ww is the output variable of MM. Let Fโ€‹(X,V,w)F(X,V,w) be a formula specifying the circuit MM and CC be a clause of FF. Consider the case of splitting CC on all variables of VV. That is C=C1โˆงโ‹ฏโˆงCp+1C=C_{1}\wedge\dots\wedge C_{p+1} where C1=Cโˆจlโ€‹(v1)ยฏC_{1}=C\vee\overline{l(v_{1})},โ€ฆ,โ€‰Cp=Cโˆจlโ€‹(vp)ยฏC_{p}=C\vee\overline{l(v_{p})},โ€‰Cp+1=Cโˆจlโ€‹(v1)โˆจโ‹ฏโˆจlโ€‹(vp)C_{p+1}=C\vee l(v_{1})\vee\dots\vee l(v_{p}) and lโ€‹(vi)l(v_{i}) is a literal of viv_{i} and V={v1,โ€ฆ,vp}V=\mbox{$\{v_{1},\dots,v_{p}\}$}. Let Fโ€ฒF^{\prime} denote the formula obtained from FF by replacing the clause CC with C1โˆงโ‹ฏโˆงCp+1C_{1}\wedge\dots\wedge C_{p+1}. Denote by \vvโ€‹๐’—๐’”๐’‘๐’\vv{v}\!_{\mathit{spl}} the input assignment falsifying the literals lโ€‹(v1),โ€ฆ,lโ€‹(vp)l(v_{1}),\dots,l(v_{p}) where โ€™splโ€™ stands for โ€™splittingโ€™.

Consider applying ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} to solve the PQE problem of taking the clause Cp+1C_{p+1} out of โˆƒXโ€‹[Fโ€ฒ]\exists{X}[F^{\prime}]. ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} starts with looking for an assignment satisfying(Fโ€ฒโˆ–{Cp+1})โˆงCp+1ยฏ(F^{\prime}\setminus\mbox{$\{C_{p+1}\}$})\wedge\overline{C_{p+1}} (to find a subspace where Fโ€ฒโˆ–{Cp+1}F^{\prime}\setminus\mbox{$\{C_{p+1}\}$} does not imply Cp+1C_{p+1}). Consider the following three cases. The first case is that the formula above is unsatisfiable. Then Cp+1C_{p+1} is trivially redundant in Fโ€ฒF^{\prime} and hence in โˆƒXโ€‹[Fโ€ฒ]\exists{X}[F^{\prime}] and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} terminates.

The second case is that there is an assignment (\vvโ€‹x,\vvโ€‹v๐‘ ๐‘๐‘™,wโˆ—\mbox{$\vv{x}$},\mbox{$\vv{v}\!_{\mathit{spl}}$},w^{*}) satisfying Fโ€ฒF^{\prime} where \vvโ€‹x\vv{x} is a full assignment to XX and wโˆ—w^{*} is the output value taken by MM under the input \vvโ€‹v๐‘ ๐‘๐‘™\vv{v}\!_{\mathit{spl}}. (Note that any full assignment to VV that is different from \vvโ€‹v๐‘ ๐‘๐‘™\vv{v}\!_{\mathit{spl}} falsifies Cp+1ยฏ\overline{C_{p+1}}. So, any assignment satisfying (Fโ€ฒโˆ–{Cp+1})โˆงCp+1ยฏ(F^{\prime}\setminus\mbox{$\{C_{p+1}\}$})\wedge\overline{C_{p+1}} has to contain \vvโ€‹v๐‘ ๐‘๐‘™\vv{v}\!_{\mathit{spl}}.) Then formula Fโ€ฒF^{\prime} is satisfiable in subspace (\vvโ€‹v๐‘ ๐‘๐‘™,wโˆ—)(\mbox{$\vv{v}\!_{\mathit{spl}}$},w^{*}) and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} adds the plugging clause Dโ€‹(V,w)D(V,w) that is the longest clause falsified by (\vvโ€‹v๐‘ ๐‘๐‘™,wโˆ—)(\mbox{$\vv{v}\!_{\mathit{spl}}$},w^{*}). If (Fโ€ฒโˆ–{Cp+1})โˆงCp+1ยฏโˆงD(F^{\prime}\setminus\mbox{$\{C_{p+1}\}$})\wedge\overline{C_{p+1}}\wedge D is unsatisfiable, then Cp+1C_{p+1} is redundant in โˆƒXโ€‹[Fโ€ฒ]\exists{X}[F^{\prime}] and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} terminates.

The third case occurs when there is an assignment (\vvโ€‹x,\vvโ€‹v๐‘ ๐‘๐‘™,wโˆ—\mbox{$\vv{x}$},\mbox{$\vv{v}\!_{\mathit{spl}}$},w^{*}) satisfying(Fโ€ฒโˆ–{Cp+1})โˆงCp+1ยฏ(F^{\prime}\setminus\mbox{$\{C_{p+1}\}$})\wedge\overline{C_{p+1}} where wโˆ—w^{*} is the negation of the output value taken by MM under input \vvโ€‹v๐‘ ๐‘๐‘™\vv{v}\!_{\mathit{spl}}. In this case, formula Fโ€ฒF^{\prime} is unsatisfiable in subspace (\vvโ€‹v๐‘ ๐‘๐‘™,wโˆ—)(\mbox{$\vv{v}\!_{\mathit{spl}}$},w^{*}). Since, Fโ€ฒโˆ–{Cp+1}F^{\prime}\setminus\mbox{$\{C_{p+1}\}$} is satisfiable in this subspace, Cp+1C_{p+1} is not redundant in โˆƒXโ€‹[Fโ€ฒ]\exists{X}[F^{\prime}]. To make Cp+1C_{p+1} redundant in subspace (\vvโ€‹v๐‘ ๐‘๐‘™,wโˆ—)(\mbox{$\vv{v}\!_{\mathit{spl}}$},w^{*}), ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} has to add the clause Bโ€‹(V,w)B(V,w) that is the longest clause falsified by (\vvโ€‹v๐‘ ๐‘๐‘™,wโˆ—)(\mbox{$\vv{v}\!_{\mathit{spl}}$},w^{*}). The clause BB is a solution to the PQE problem at hand i.e. โˆƒXโ€‹[Fโ€ฒ]โ‰กBโˆงโˆƒXโ€‹[Fโ€ฒโˆ–{Cp+1}]\mbox{$\exists{X}[F^{\prime}]$}\equiv B\wedge\mbox{$\exists{X}[F^{\prime}\setminus\mbox{$\{C_{p+1}\}$}]$}.

The clause BB above is implied by Fโ€ฒF^{\prime} (and hence FF) and so, is a property of MM. This property specifies the input/output behavior of MM under the input \vvโ€‹v๐‘ ๐‘๐‘™\vv{v}\!_{\mathit{spl}}. Namely, to satisfy BB when the variables of VV are assigned as in \vvโ€‹v๐‘ ๐‘๐‘™\vv{v}\!_{\mathit{spl}}, one has to set the variable ww to wโˆ—ยฏ\overline{w^{*}}. The latter is the output produced by MM under the input \vvโ€‹v๐‘ ๐‘๐‘™\vv{v}\!_{\mathit{spl}}. So, the property BB specifies the behavior of MM under a single test. In all three cases above, the SAT problem considered by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is solved just by initial BCP. (The reason is that the formula at hand contains the unit clauses produced by negating Cp+1C_{p+1} or those specifying the subspace (\vvโ€‹v๐‘ ๐‘๐‘™,wโˆ—)(\mbox{$\vv{v}\!_{\mathit{spl}}$},w^{*}).) So, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} solves the PQE problem above in linear time.

Appendix 0.F Tests Specified By A Property Generated By PQE

In this appendix, we show the relation between tests specified by a property obtained via PQE (see Subsectionย 3.5) and those detecting stuck-at faults. Here, we reuse the notation of Sectionย 3. Let Mโ€‹(X,V,W)M(X,V,W) be a combinational circuit where X,V,WX,V,W are the internal, input and output variables respectively. Let Fโ€‹(X,V,W)F(X,V,W) be a formula specifying MM. Let GG be an AND gate of MM whose functionality is x3=x1โˆงx2x_{3}=x_{1}\wedge x_{2}. That is x1,x2x_{1},x_{2} are the input variables of GG and x3x_{3} is its output variable. The functionality of GG is specified by the formula C1โˆงC2โˆงC3C_{1}\wedge C_{2}\wedge C_{3} where C1=xยฏ1โˆจxยฏ2โˆจx3C_{1}=\overline{x}_{1}\vee\overline{x}_{2}\vee x_{3}, C2=x1โˆจxยฏ3C_{2}=x_{1}\vee\overline{x}_{3}, C3=x2โˆจxยฏ3C_{3}=x_{2}\vee\overline{x}_{3} (see Exampleย 2). The clauses C1,C2,C3C_{1},C_{2},C_{3} are present in formula FF. Consider taking C1C_{1} out of โˆƒXโ€‹[F]\exists{X}[F]. This clause makes GG produce the output value 1 when its input values are 1. (If x1x_{1} and x2x_{2} are set to 1, the clause C1C_{1} can be satisfied only by setting x3x_{3} to 1.)

Let Hโ€‹(V,W)H(V,W) be the property obtained by taking out C1C_{1}. That isโˆƒXโ€‹[F]โ‰กHโˆงโˆƒXโ€‹[Fโˆ–{C1}]\mbox{$\exists{X}[F]$}\equiv H\wedge\mbox{$\exists{X}[F\setminus\mbox{$\{C_{1}\}$}]$}. Let Qโ€‹(V,W)Q(V,W) be a clause of HH. As we mentioned earlier, we assume that HH does not have redundant clauses i.e. those implied by Fโˆ–{C1}F\setminus\mbox{$\{C_{1}\}$}. Then the formula (Fโˆ–{C1})โˆงQยฏ(F\setminus\mbox{$\{C_{1}\}$})\wedge\overline{Q} is satisfiable. Let (\vvโ€‹x\vv{x},\vvโ€‹v\vv{v},\vvโ€‹w\vv{w}) be an assignment satisfying this formula. Note that this assignment falsifies C1C_{1}. (Indeed, assume the contrary. Then (\vvโ€‹x\vv{x},\vvโ€‹v\vv{v},\vvโ€‹w\vv{w}) satisfies FF because it already satisfies Fโˆ–{C1}F\setminus\mbox{$\{C_{1}\}$}. Since this assignment falsifies QQ, we have to conclude that Fโ‡’ฬธQF\not\Rightarrow Q and hence Fโ‡’ฬธHF\not\Rightarrow H. So we have a contradiction.)

The fact that (\vvโ€‹x\vv{x},\vvโ€‹v\vv{v},\vvโ€‹w\vv{w}) falsifies C1C_{1} and satisfies Fโˆ–{C1}F\setminus\mbox{$\{C_{1}\}$} means that one can view this assignment as an execution trace of a faulty version M๐‘“๐‘™๐‘ก\mathit{M}_{\mathit{flt}} of MM. Namely, the output x3x_{3} of gate GG is stuck at 0 in M๐‘“๐‘™๐‘ก\mathit{M}_{\mathit{flt}}. (The clause C1C_{1} is falsified when x1=1,x2=1,x3=0x_{1}=1,x_{2}=1,x_{3}=0 i.e. if the gate GG outputs 0 when its input variables are assigned 1.) Let (\vvโ€‹xโˆ—,\vvโ€‹v,\vvโ€‹wโˆ—)(\mbox{$\vv{x}$}^{*},\mbox{$\vv{v}$},\mbox{$\vv{w}$}^{*}) be the execution trace of MM under the input \vvโ€‹v\vv{v}. As we showed in Subsectionย 3.5, \vvโ€‹wโˆ—\mbox{$\vv{w}$}^{*} is different from \vvโ€‹w\vv{w}. So the input \vvโ€‹v\vv{v} exposes a stuck-at fault by making M๐‘“๐‘™๐‘ก\mathit{M}_{\mathit{flt}} and MM produce different outputs.

Appendix 0.G Brief Description Of ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE}

In this appendix, we give a high-level view of ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and explain how it works in ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} in more detail. A full description of ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} can be found inย [1]. ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is based on the machinery of D-sequentsย [33] (โ€™DSโ€™ in the name ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} stands for โ€™D-sequentโ€™). Given a formula โˆƒXโ€‹[Fโ€‹(X,Y)]\exists{X}[F(X,Y)] and an assignment \vvโ€‹p\vv{p} to XโˆชYX\cup Y, a D-sequent is a record (โˆƒXโ€‹[F],\vvโ€‹p)โ†’C(\mbox{$\exists{X}[F]$},\mbox{$\vv{p}$}\,)\rightarrow C stating that clause CC is redundant in โˆƒXโ€‹[F]\exists{X}[F] in subspace \vvโ€‹p\vv{p}. In ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}, ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} is called in subspaces \vvโ€‹y\vv{y} where FF is satisfiable. (Here \vvโ€‹y\vv{y} is a full assignment to YY.) ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} terminates upon deriving a D-sequent (โˆƒXโ€‹[F],\vvโ€‹yโˆ—)โ†’C(\mbox{$\exists{X}[F]$},\mbox{$\vv{y}$}^{*})\rightarrow C where \vvโ€‹yโˆ—โІ\vvโ€‹y\mbox{$\vv{y}$}^{*}\subseteq\mbox{$\vv{y}$}. Such derivation means that CC is proved redundant in โˆƒXโ€‹[F]\exists{X}[F] in subspace \vvโ€‹y\vv{y}. Then the plugging clause DD falsified by \vvโ€‹yโˆ—\mbox{$\vv{y}$}^{*} is generated where ๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(D)=๐‘‰๐‘Ž๐‘Ÿ๐‘ โ€‹(\vvโ€‹yโˆ—)\mbox{$\mathit{Vars}(D)$}=\mathit{Vars}(\mbox{$\vv{y}$}^{*}).

๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} derives the D-sequent (โˆƒXโ€‹[F],\vvโ€‹yโˆ—)โ†’C(\mbox{$\exists{X}[F]$},\mbox{$\vv{y}$}^{*})\rightarrow C above by branching on variables of XX. A variable is assigned a value either by a decision or during Boolean Constraint Propagation (BCP). A branch of the search tree goes on until an atomic D-sequent is derived for CC. This occurs when proving CC in the current subspace becomes trivial. When backtracking, ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} merges D-sequents derived in different branches using a resolution like operation called join. For instance, the join operation applied to D-sequents (โˆƒXโ€‹[F],\vvโ€‹pโ€ฒ)โ†’C(\mbox{$\exists{X}[F]$},\mbox{$\vv{p}$}^{\prime})\rightarrow C where \vvโ€‹pโ€ฒ=(y1=0,x1=0)\mbox{$\vv{p}$}^{\prime}=(y_{1}=0,x_{1}=0) and (โˆƒXโ€‹[F],\vvโ€‹pโ€ฒโ€ฒ)โ†’C(\mbox{$\exists{X}[F]$},\mbox{$\vv{p}$}^{\prime\prime})\rightarrow C where \vvโ€‹pโ€ฒโ€ฒ=(y2=1,x1=1)\mbox{$\vv{p}$}^{\prime\prime}=(y_{2}=1,x_{1}=1) produces the D-sequent (โˆƒXโ€‹[F],\vvโ€‹p)โ†’C(\mbox{$\exists{X}[F]$},\mbox{$\vv{p}$}\,)\rightarrow C where \vvโ€‹p\vv{p} =(y1=0,y2=1)(y_{1}=0,y_{2}=1).

๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} has three situations where an atomic D-sequent is generated. First, when CC is blocked in the current subspace and hence is redundant there. Then an atomic D-sequent (โˆƒXโ€‹[F],\vvโ€‹p)โ†’C(\mbox{$\exists{X}[F]$},\mbox{$\vv{p}$})\rightarrow C is generated where \vvโ€‹p\vv{p} consists of assignments that made CC blocked in the current subspace. For instance, in Exampleย 3, ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} would generate an atomic D-sequent (โˆƒXโ€‹[F],(y1=1))โ†’C(\mbox{$\exists{X}[F]$},(y_{1}\!=\!1))\rightarrow C. Second, an atomic D-sequent is generated when CC is satisfied by an assignment w=bw=b where wโˆˆXโˆชYw\in X\cup Y and bโˆˆ{0,1}b\in\mbox{$\{0,1\}$}. (This can be a decision assignment or an assignment derived from a clause during BCP.) Then an atomic D-sequent(โˆƒXโ€‹[F],(w=b))โ†’C(\mbox{$\exists{X}[F]$},(w=b))\rightarrow C is built. Third, an atomic D-sequent is generated when a conflict occurs and a conflict clause C๐‘๐‘›๐‘“๐‘™\mathit{C}_{\mathit{cnfl}} falsified in the current subspace is derived. Adding C๐‘๐‘›๐‘“๐‘™\mathit{C}_{\mathit{cnfl}} to FF makes CC redundant in the current subspace. So, an atomic D-sequent (โˆƒXโ€‹[F],\vvโ€‹p)โ†’C(\mbox{$\exists{X}[F]$},\mbox{$\vv{p}$})\rightarrow C is generated where \vvโ€‹p\vv{p} is the shortest assignment falsifying C๐‘๐‘›๐‘“๐‘™\mathit{C}_{\mathit{cnfl}}.

Appendix 0.H Experiments With HWMCC-13 Benchmarks

In Sectionย 8, we described experiments with multi-property benchmarks of the HWMCC-13 setย [19]. In this appendix, we provide some additional information. Each benchmark consists of a sequential circuit NN and invariants P0,โ€ฆ,PmP_{0},\dots,P_{m} that are supposed to hold for NN. We will refer to the invariant P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}} equal to P0โˆงโ‹ฏโˆงPmP_{0}\wedge\dots\wedge P_{m} as the aggregate invariant. We applied PQE to the generation of invariants of NN that may be unwanted. Since every invariant PP implied by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{agg}} must hold, the necessary condition for PP to be unwanted is P๐‘Ž๐‘”๐‘”โ‡’ฬธP\mbox{$\mathit{P}_{\mathit{agg}}$}\not\Rightarrow P.

Similarly to the experiment of Sectionย 7, we used the formula Fk=Iโ€‹(S0)โˆงTโ€‹(S0,V0,S1)F_{k}=I(S_{0})\wedge T(S_{0},V_{0},S_{1}) โˆงโ‹ฏโˆงTโ€‹(Skโˆ’1,Vkโˆ’1,Sk)\wedge\dots\wedge T(S_{k-1},V_{k-1},S_{k}) to generate invariants. The number kk of time frames was in the range of 2โ‰คkโ‰ค102\!\leq\!k\!\leq\!10. Specifically, we set kk to the largest value in this range where |Fk||F_{k}| did not exceed 500,000 clauses. We discarded the benchmarks with |F2|>500,000|F_{2}|\!>\!500,000. We also dropped the smallest benchmarks. So, in the experiments, we used 98 out of the 178 benchmarks of the set.

We describe three experiments. In every experiment, we generated properties Hโ€‹(Sk)H(S_{k}) by taking out a clause of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] where Xk=S0โˆชV0โˆชโ‹ฏโˆชSkโˆ’1โˆชVkโˆ’1\mbox{$X_{k}$}\!=\!S_{0}\cup V_{0}\cup\dots\cup S_{k-1}\cup V_{k-\!1}. Propertyย HH is a local invariant claiming that no state falsifying HH can be reached in kk transitions. As in the experiment of Sectionย 7, we took out only clauses containing an unquantified variable (i.e a variable of SkS_{k}). In all experiments, the time limit for solving a PQE problem was set to 10 sec.

0.H.1 Experiment 1

The objective of the first experiment was to demonstrate that ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} could compute HH for realistic designs. We also showed in this experiment that PQE could be much easier than QE (see Sectionย 8) and that ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} outperforms ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}. In this experiment, for each benchmark out of 98 mentioned above we generated PQE problems of taking a clause out of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}]. Some of them were trivially solved by preprocessing. The latter eliminated the blocked clauses of FkF_{k} that were easy to identify and ran BCP launched due to the unit clauses specifying the initial state. In all experiments, we discarded problems solved by preprocessing i.e. we considered only non-trivial PQE problems.

Table 4: PQE problems of the first group
pqe total finished
solver probl. num. perc.
ds-pqe 3,736 1,132 ย 30%
eg-pqe 3,736 3,296 ย 88%
eg-pqe+ 3,736 3,329 ย 89%

Let CC be a clause taken out of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}]. We used ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} to partition the PQE problems we tried into two groups. The first group consisted of problems for which we ran ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} with the time limit of 10 sec. and it never encountered a subspace \vvโ€‹sk\vv{s_{k}} where FkF_{k} was satisfiable. Here \vvโ€‹sk\vv{s_{k}} is a full assignment to SkS_{k}. (Recall that the variables of SkS_{k} are the only unquantified variables of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}].) So, in every subspace \vvโ€‹sk\vv{s_{k}} tried by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}, formula FkF_{k} was either unsatisfiable or (Fkโˆ–{C})โ‡’C(F_{k}\!\setminus\!\mbox{$\{C\}$})\!\Rightarrow\!C. The second group consisted of problems where ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} encountered subspaces where FkF_{k} was satisfiable. For either group we generated up to 40 problems per benchmark. For some benchmarks, the total number of non-trivial problems generated for the first or second group was under 40. Many PQE problems of either group had hundreds of thousands of variables.

Table 5: PQE problems of the second group
pqe total finished
solver probl. num. perc.
ds-pqe 3,094 464 15%
eg-pqe 3,094 74 2%
eg-pqe+ 3,094 830 27%

The results for the first group are shown in Tableย 4. The first column gives the name of a PQE solver. The second column shows the number of PQE problems in the first group. The last two columns give the number and percentage of problems finished in the time limit of 10 sec. Tableย 4 shows that ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} performed quite well finishing a very high percentage of problems. The results of ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} are much poorer because it does not check if (Fkโˆ–{C})โ‡’C(F_{k}\setminus\mbox{$\{C\}$})\Rightarrow C in the current subspace i.e. if CC is trivially redundant.

The results for the second group are shown in Tableย 5 that has the same structure as Tableย 4. In particular, the second column gives the number of PQE problems in the second group. Tableย 5 shows that ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} finished 2% and 27% of the problems respectively. So, ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} significantly outperformed ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE}. The reason is that ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} uses a satisfying assignment as a proof of redundancy of the clause CC in subspace \vvโ€‹sk\vv{s}_{k}.

0.H.2 Experiment 2

The second experiment was an extension of the first one. Its goal was to show that PQE can generate invariants for realistic designs. For each clause QQ of a local invariant HH generated by PQE we used IC3 to verify if QQ was a global invariant. If so, we checked if P๐‘Ž๐‘”๐‘”โ‡’ฬธQ\mbox{$\mathit{P}_{\mathit{agg}}$}\not\Rightarrow Q held. To make the experiment less time consuming, in addition to the time limit of 10 sec. per PQE problem, we imposed the following constraints. First, we stopped a PQE-solver even before the time limit if it generated more than 5 free clauses. Second, the time limit for IC3 was set to 30 sec. Third, instead of constraining the number of PQE problems per benchmark (i.e. the number of single clauses taken out of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}]) like in the first experiment, we imposed the following two constraints. First, we stopped processing a benchmark as soon as the total of 100 free clauses was generated (for all the PQE problems generated for this benchmark). Second, we stopped processing a benchmark even earlier if the summary run time for all PQE problems generated for this benchmark exceeded 2,000 sec.

Table 6: A sample of HWMCC-13 benchmarks from
the experiment with ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}
name lat- in- time clau- single-clause properties
ches var. fra- ses gen. invariant? not
of mes taken props un- no yes impl.
P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} out dec. by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}}
6s306 7,986 ย 25 ย ย 2 ย 182 ย 100 ย 0 ย 94 ย 6 ย 6
6s176 1,566 ย 952 ย ย 3 ย 31 ย 100 ย 23 ย 11 ย 66 ย 11
6s428 3,790 ย 340 ย ย 4 ย 28 ย 100 ย 9 ย 5 ย 86 ย 83
6s292 3,190 ย 247 ย ย 5 ย 24 ย 100 ย 41 ย 0 ย 59 ย 59
6s156 513 ย 32 ย ย 6 ย 113 ย 100 ย 0 ย 0 ย 100 ย 100
6s275 3,196 ย 673 ย ย 7 ย 20 ย 100 ย 0 ย 50 ย 50 ย 50
6s325 1,756 ย 301 ย ย 8 ย 22 ย 100 ย 0 ย 1 ย 99 ย 97
6s391 2,686 ย 387 ย ย 9 ย 35 ย 100 ย 1 ย 29 ย 70 ย 70
6s282 1,933 ย 20 ย 10 ย 111 ย 100 ย 0 ย 64 ย 36 ย 35

A sample of 9 benchmarks out of the 98 we used in the experiment with ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} is shown in Tableย 6. Let us explain the structure of this table by the benchmark 6s306 (the first line of the table). The name of this benchmark is shown in the first column. The second column gives the number of latches (7,986). The number of invariants that should hold for 6s306 is provided in the third column (25). So, the aggregate invariant P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} of 6s306 is the conjunction of those 25 invariants. The fourth column shows that the number kk of time frames for 6s306 was set to 2 (since |F3|>500,000|F_{3}|\!>\!500,000). The value 182 shown in the fifth column is the total number of single clauses taken out of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] i.e. the number of PQE problems (where k=2k=2 for 6s306).

Every free clause QQ generated by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} when taking a clause out of โˆƒXkโ€‹[Fk]\exists{\mbox{$X_{k}$}}[F_{k}] was stored as a local single-clause invariant. The sixth column shows that taking clauses out of the scope of quantifiers was terminated when 100 free clauses (specifying 100 local single-clause invariants) were generated. Each of these 100 local invariants held in kk-th time frame. The following three columns show how many of those 100 local invariants were true globally. IC3 finished every problem out of 100 in the time limit of 30 sec. So, the number of undecided invariants was 0. The number of invariants IC3 proved false or true globally was 94 and 6 respectively. The last column gives the number of global invariants not implied by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} i.e. invariants that may be unwanted. For 6s306, this number is 6.

0.H.3 Experiment 3

To prove an invariant PP true, IC3 conjoins it with clauses Q1,โ€ฆ,QnQ_{1},\dots,Q_{n} to make PโˆงQ1โˆงโ‹ฏโˆงQnP\wedge Q_{1}\!\wedge\dots\wedge Q_{n} inductive. If IC3 succeeds, every QiQ_{i} is an invariant. Moreover, QiQ_{i} may be an unwanted invariant. Arguably, the cause of efficiency of IC3 is that PP is often close to an inductive invariant. So, IC3 needs to generate a relatively small number of clauses QiQ_{i} to make the constrained version of PP inductive. However, this nice feature of IC3 drastically limits the set of invariant clauses it generates. In this subsection, we substantiate this claim by an experiment. In this experiment, we picked the HWMCC-13 benchmarks for which one could prove all predefined invariants P1,โ€ฆ,PmP_{1},\dots,P_{m} within a time limit. Namely, for every benchmark we formed the aggregate invariant P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} = P1โˆงโ‹ฏโˆงPmP_{1}\wedge\dots\wedge P_{m} and ran IC3 to prove P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} true.

We selected the benchmarks that IC3 solved in less than 1,000 sec. (In addition to dropping the benchmarks not solved in 1,000 sec., we discarded those where P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} failed because some invariants PiP_{i} were false). Let ๐‘ท๐’‚๐’ˆ๐’ˆโˆ—P^{*}_{\mathit{agg}} denote the inductive version of P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} produced by IC3 when proving P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} true. That is, P๐‘Ž๐‘”๐‘”โˆ—P^{*}_{\mathit{agg}} is P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} conjoined with the invariant clauses produced by IC3.

Table 7: Invariants of ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} and
IC3
name lat- inva- glob. single cls. invars
ches riants glob. not not
of inva- impl. impl.
P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} riants by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} by P๐‘Ž๐‘”๐‘”โˆ—P^{*}_{\mathit{agg}}
6s135 2,307 ย 340 ย 53 ย 53 ย 27
6s325 1,756 ย 301 ย 99 ย 99 ย 96
ex1 130 ย 33 ย 25 ย 16 ย 16
ex2 212 ย 32 ย 64 ย 64 ย 47
6s106 135 ย 17 ย 100 ย 96 ย 96
6s256 3,141 ย 5 ย 6 ย 6 ย 6
ex3 61 ย 3 ย 4 ย 4 ย 4
ex4 63 ย 3 ย 1 ย 1 ย 1
6s209 5,759 ย 2 ย 95 ย 95 ย 89
6s113 994 ย 1 ย 19 ย 16 ย 16
6s143 260 ย 1 ย 97 ย 86 ย 77
6s170 3,141 ย 1 ย 13 ย 13 ย 13
6s252 170 ย 1 ย 55 ย 41 ย 34
Total 590 522

For each of the selected benchmarks we generated invariants by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} exactly as in Experiment 2. That is, we stopped generation of local single clause invariants when their number exceeded 100. Then we ran IC3 to identify local invariants that were global as well. After that we checked which of the global invariants generated by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} were not implied by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}}. The difference from Experiment 2 was that we also checked which global invariants generated by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} were not implied by P๐‘Ž๐‘”๐‘”โˆ—P^{*}_{\mathit{agg}}.

The results of the experiment are shown in Tableย 7. The first three columns of this table are the same as in Tableย 6. They give the name of a benchmark, the number of latches and the number of invariants P1P_{1},โ€ฆ\dots,PmP_{m} to prove. (The actual names of examples ex1,..,ex4 in the HWMCC-13 set are pdtvsarmultip, bobtuintmulti, nusmvdme1d3multi, nusmvdme2d3multi respectively.) The next column of Tableย 7 shows the number of local invariants generated by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} that turn out to be global. The last two columns give the number of global invariants that were not implied by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} and P๐‘Ž๐‘”๐‘”โˆ—P^{*}_{\mathit{agg}} respectively. The last row of the table shows that in 522 cases out of 590 the invariants not implied by P๐‘Ž๐‘”๐‘”\mathit{P}_{\mathit{\!agg}} were not implied by P๐‘Ž๐‘”๐‘”โˆ—P^{*}_{\mathit{agg}} either. So, in 88% of cases, the invariant clauses generated by ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} were different from those generated by IC3 to form P๐‘Ž๐‘”๐‘”โˆ—P^{*}_{\mathit{agg}}.

Appendix 0.I Experiment With Combinational Circuits

In this appendix, we give more information about the experiment with property generation for combinational circuits described in Sectionย 9. We formed PQE problems as follows. For each benchmark NN we picked the number kk of time frames to unroll. The value of kk ranged from 10 to 40. (For larger circuits we picked a smaller value of kk.) Then we unrolled NN for kk time frames to form a combinational circuit MkM_{k} and randomly generated a clause Bโ€‹(Sk)B(S_{k}) of 15 literals. So, BB depended on output variables of MkM_{k}. After that, we constructed the subcircuit Mkโ€ฒM^{\prime}_{k} of MkM_{k} as described in Sectionย 9. That is, Mkโ€ฒM^{\prime}_{k} was obtained by removing the logic of MkM_{k} that did not feed any output variable present in BB.

Let formula Fkโ€ฒF^{\prime}_{k} specify the subcircuit Mkโ€ฒM^{\prime}_{k}. (Here we reuse the notation of Sectionย 9.) For every benchmark, we generated PQE problems of taking different clauses CC out of โˆƒS1,kโ€‹[Fkโ€ฒ]\exists{S_{1,k}}[F^{\prime}_{k}]. That is each PQE problem was to find HH such that โˆƒS1,kโ€‹[Fkโ€ฒ]โ‰กHโˆงโˆƒS1,kโ€‹[Fkโ€ฒโˆ–{C}]\mbox{$\exists{S_{1,k}}[F^{\prime}_{k}]$}\equiv H\wedge\mbox{$\exists{S_{1,k}}[F^{\prime}_{k}\setminus\mbox{$\{C\}$}]$}. Each clause CC to take out was chosen among the clauses of Fkโ€ฒF^{\prime}_{k} that contained a variable of SkS_{k} (i.e. an output variable of Mkโ€ฒM^{\prime}_{k}). In this way, we formed a set of 3,254 PQE problems. 1,668 of these problems were solved by simple formula preprocessing i.e. turned out to be trivial. So, in the experiment we used the remaining 1,586 non-trivial PQE problems.

Table 8: Property generation
pqe num. properties were generated
solver of pqe num- stronger than
prob- ber ย ย % 3-val. sim.
lems num. ย ย %
ds-pqe 1,586 983 ย ย 62 ย 728 ย ย 74
eg-pqe 1,586 450 ย ย 28 ย 361 ย ย 80
eg-pqe+ 1,586 1,046 ย ย 66 ย 817 ย ย 78

The time limit for solving a PQE problem was set to 10 sec. Besides, solving a PQE problem terminated as soon as the size of HH reached 5 clauses. The results of the experiment are summarized in Tableย 8. The second column gives the total number of PQE problems. The next two columns show the number and percentage of problems where HH was non-empty i.e. had at least one clause. (Recall, that each clause of HH represents a property.) The last two columns give the number and percentage of cases where a clause of HH represented a property that was stronger than ones produced by 3-valued simulation, a version of symbolic simulationย [22]. Consider, for instance, the last line of the table corresponding to ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+}. For 817 out of 1,046 PQE problems where HH was not empty, at least one clause of HH constituted a property that could not be produced by 3-valued simulation. Tableย 8 shows that ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ\mathit{PQE} had the weakest results generating properties only for 28% of problems whereas ๐ท๐‘†\mathit{DS}-๐‘ƒ๐‘„๐ธ\mathit{PQE} and ๐ธ๐บ\mathit{EG}-๐‘ƒ๐‘„๐ธ+\mathit{PQE}^{+} performed much better producing properties for 62% and 66% problems respectively.