Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models

Given a character set, in a formal language, the generation rules uniquely determine the properties of the language. We assume that there also exists a set of generation rules $\mathcal{R}$ implicitly in natural language, and the language objects derived from these rules exhibit a series of features. Among these features, a subset $Y$ is selected as the probed feature for which the properties represent the logical constraints of the generation rule set. Assuming there is another model $\mathcal{M}$ that can assign a suitable representation vector to the generated language objects, the properties of $Y$ are then represented by the intrinsic geometric constraints of the vector set. By studying the geometric constraints that are implicit in the vector set and that correspond to $Y$, especially when $Y$ is expanded to all features of the language object, we can determine the correspondence between $\mathcal{M}$ and $\mathcal{R}$. The probe is a model that investigates the relationship between the geometric constraints of the vector set and $Y$. It is composed of a function set $F$ and a metric $E_{Y}$ defined on $Y$. The input of a function in $F$ is the representation vector of a language object, and the output is the predicted $Y$ feature of the input language object. The distance between the predicted feature and the true feature is calculated by using the metric $E_{Y}$, and a function $f$ in $F$ that minimizes the distance is determined. Here, $F$ limits the range of geometric constraints, and $E_{Y}$ limits the selection of a "good" geometry. Notably, this definition seems very similar to that of learning. Therefore, the larger the scope of $F$ is, the harder it is to discern the form of the geometric constraints, especially when $F$ is a neural network (Pimentel et al., 2020b; White et al., 2021). However, the purpose of the probe is different from that of learning. The goal of learning is to construct a model $\mathcal{M}$ (usually a black box), which may have multiple construction methods, while the purpose of the probe is to analyze the relationship between $\mathcal{M}$ and $\mathcal{R}$.

One of the goals of topology is to find homeomorphic or homotopic invariants (including invariant quantities, algebraic structures, functors, etc.) and then to characterize the intrinsic structure of a topological space with these invariants. Analogously, we can view $R$ as a geometric object and $Y$ as its topology. Can we then define a concept similar to topological invariants with respect to $Y$?

We define a feature invariant for $Y$ as a set of conditions $C_{Y}$ such that any element in $Y$ satisfies $C_{Y}$. $C_{Y}$ reflects the internal constraints of the probed feature, as well as a part of the logical constraints of $R$. Furthermore, if $C_{Y}$ is well defined, it induces a set $X_{C_{Y}}$ consisting of all objects satisfying $C_{Y}$, which naturally extends the metric defined on $Y$ to $X_{C_{Y}}$.

Furthermore, just as the distance measure between two points can induce a distance measure between a point and a plane, the distance measure between the predicted feature $px$ and $X_{C_{Y}}$ can also be induced by $E_{Y}$ (denoted as $E_{C_{Y}}$):

It can be easily verified that if $E_{Y}$ is a well-defined distance metric on $Y$, then $E_{C_{Y}}$ should also be a well-defined distance metric on $px$. Once we have $E_{C_{Y}}$, the supervised probe $(F,E_{Y},Y)$ can naturally induce a self-supervised probe $(F,E_{C_{Y}},C_{Y})$. We refer to $(F,E_{C_{Y}},C_{Y})$ as the self-supervised version of $(F,E_{Y},Y)$, also known as the topological probe.

Notably, the prerequisite for obtaining $(F,E_{C_{Y}},C_{Y})$ is that $C_{Y}$ must be well-defined, so $C_{Y}$ should not be a black box. Figure 1 shows an intuitive illustration.

Next, we present a specific topological probe that is based on the previously outlined design scheme and serves as a self-supervised variant of the structural probe.

Given a sentence $W$, it is represented by a model $M$ as a set (or sequence) of vectors, denoted as $H=M(W)$. The number of vectors in $H$ is denoted as $L_{H}$, and we assign an index $(1,2,...L_{H})$ to each vector in $H$ so that the order of the indices matches the order of the corresponding tokens in the sentence. Additionally, we denote the dimension of the vectors as $n$. For each $W$, there exists a syntax tree $T_{W}$, where each complete word in $W$ corresponds to a node in $T_{W}$.

The probed feature $Y$ that the structural probe defines is the depth of the nodes corresponding to complete words. Following the work in (Hewitt and Manning, 2019), we set the parameter space of $F$ for the structural probe to be all real matrices of size $m*n$, where $m<n$. The specific form for predicting the depth is as follows:      Given $p\in R,\,\forall 1\leq i\leq L_{H}$

where $\bm{p}dep(h_{i})$ is the prediction tree depth of $w_{i}$ in $T_{W}$ and $f$ is a real matrix of size $m*n$. Because $\forall p<2$, there is a tree that cannot be embedded as above (Reif et al., 2019), so $p$ is usually taken as $2$. $\bm{p}dep(h_{1})$, $\bm{p}dep(h_{2})$ $\cdots$, $\bm{p}dep(h_{L_{H}})$ form a sequence denoted as $\bm{p}dep_{H}$.

Moreover, we denote the true depth of $w_{i}$ as $dep(w_{i})$. Hence, $dep(w_{1})$, $dep(w_{2})$ $\cdots$, $dep(w_{L_{H}})$ also form a sequence denoted as $dep_{W}$. The metric $E$ in the structural probe is defined as follows:

Therefore, the structural probe is defined as $(|f*|^{2}$, $E$ , $dep)$.

Now we provide the constraints $C_{dep}$ for $dep$. An important limitation of $dep_{W}$ is that it is an integer sequence. Based on the characteristics of the tree structure, it is naturally determined that $dep_{W}$ must satisfy the following two conditions:

(Boundary condition). If $L_{H}\geq 1$, there is exactly one minimum element in $dep_{W}$, and it is equal to $1$; if $L_{H}\geq 2$, at least one element in $dep_{W}$ is equal to $2$.

(Recursion condition). If we sort $dep_{W}$ in ascending order to obtain the sequence $asdep_{W}$, then

or

We denote the set of all sequences that conform to $C_{dep}$ as $X_{C_{dep}}$. From equation 1, we can induce a metric $E_{C_{dep}}$:

Assuming we can construct an explicit sequence $mins_{W}$ such that:

We can obtain an analytical expression for $E_{C_{dep}}$ as follows:

Consider the following two examples:

1. 1.

   When $\bm{p}dep_{H}=0.8,1.5,1.8,2.4,4.5$, then $mins_{W}=1,2,2,3,4$.

2. 2.

   When $\bm{p}dep_{H}=0.8,1.5,1.8,2.4,7.5$, then $mins_{W}=1,2,3,4,5$.

It can be observed that the predicted depths for nodes further down the hierarchy can also influence the corresponding values of $mins_{W}$ for nodes higher up in the hierarchy. In the examples provided, due to the change from 4.5 to 7.5, 1.8 changes from 2 to 3 at the corresponding $mins_{W}$. Therefore, using a straightforward local greedy approach may not yield an accurate calculation of $mins_{W}$, and if a simple enumeration method is employed, the computational complexity will become exponential.

However, while a local greedy approach may not always provide an exact computation of $mins_{W}$, it can still maintain a certain degree of accuracy for reasonable results of $\bm{p}dep_{H}$. This is because cases like the jump from 2.4 to 7.5 should be infrequent in a well-trained probe’s computed sequence of predicted depths, unless the probed representation does not encode the tree structure well and exhibits a disruption in the middle.

Before delving into that, we first introduce some notations:

• •

  $\bm{ap}dep_{H}$ denote the sequence obtained by sorting $\bm{p}dep_{H}$ in ascending order.

• •

  $\bm{ap}dep_{i}$ represents the $i$-th element of $\bm{ap}dep_{H}$.

• •

  $pre_{W}$ be a sequence in $X_{C_{dep}}$.

Here, we introduce a simple method for constructing $mins_{W}$ from a local greedy perspective.

(Initialization). If $L_{H}\geq 1$, let $pre(w_{1})=1$; if $L_{H}\geq 2$, let $pre(w_{2})=2$.

(Recurrence). If $L_{H}\geq 3$ and $3\leq i\leq L_{H}$, let

where the values of $bias_{i-1}$ and $\bm{ap}dep_{H}$ are related if

$bias_{i-1}=1$; otherwise, $bias_{i-1}=0$.

(Alignment). Let $a_{i}$($1\leq i\leq L_{H}$) denote the index of $\bm{ap}dep_{i}$ in $\bm{p}dep_{H}$. Then, let

It can be shown that $pesu_{W}$ constructed in the above manner satisfies the following theorem:

Therefore, $pesu_{W}$ can be considered an approximation to $mins_{W}$. Appendix A contains the proof of this theorem. In the subsequent sections of this paper, we replace $E_{C_{dep}}(\bm{p}dep_{H},X_{C_{dep}})$ with $E(\bm{p}dep_{H},pesu_{W})$.

Additionally, an important consideration is determining the appropriate value of the minimum element for $dep_{W}$ in the boundary condition. In the preceding contents, we assumed a root depth of 1 for the syntactic tree. However, in traditional structural probe (Hewitt and Manning, 2019; Maudslay et al., 2020; Limisiewicz and Marecek, 2021; Chen et al., 2021; White et al., 2021), the root depth is typically assigned as 0 due to the annotation conventions of syntactic tree datasets. From a logical perspective, these two choices may appear indistinguishable.

However, in Appendix B, we demonstrate that the choice of whether the root depth is 0 has a significant impact on the geometry defined by the tree topological probe. Furthermore, we can prove that as long as the assigned root depth is greater than 0, the optimal geometry defined by the tree topological probe remains the same to a certain extent. Therefore, in the subsequent sections of this paper, we adopt the setting where the value of the minimum element of $dep_{W}$ is $1$.

Let the set of all language objects generated by rule $R$ be denoted as $\mathcal{X}_{R}$, and the cardinality of $\mathcal{X}_{R}$ be denoted as $|\mathcal{X}_{R}|$. The structural probe induces a metric that describes the relationship between model $M$ and $dep$:

The tree topological probe can also induce a similar metric:

$\mathcal{X}_{ssp}(M)=$

On the other hand, we let

similar to $mins_{W}$, and $maxs_{W}$, inducing the following metrics:

$\mathcal{X}_{essp}(M)$

Since $dep_{W}\in X_{C_{dep}}$, when $f$ is given, we have:

Furthermore, as $\mathcal{X}{sp}(M)$ and $\mathcal{X}{essp}(M)$ share the same set of probing functions $F$, we have:

Therefore, $\mathcal{X}_{essp}(M)$ provides us with an upper bound for the structural probe metric. Similarly, for $\mathcal{X}{ssp}(M)$, we also have:

Therefore, $\mathcal{X}_{ssp}(M)$ provides us with a lower bound for the structural probe metric. In summary, we have the following:

If $\mathcal{X}{ssp}(M)=\mathcal{X}{essp}(M)$, then there is no difference between the tree topological probe and the structural probe. On the other hand, if it is believed that a smaller $\mathcal{X}{sp}(M)$ is desirable, then estimating $\mathcal{X}{sp}(M)$ within the range $[\mathcal{X}{ssp}(M),\mathcal{X}{essp}(M)]$ becomes an interesting problem. We consider the following:

$\theta_{W}=$

This leads to an intriguing linguistic distribution, the distribution of $\theta_{W}\in[0,1]$ when uniformly sampling $W$ from $\mathcal{X}_{R}$. We suppose the density function of this distribution is denoted as $P_{\theta}$, and the expectation with respect to $\theta$ is denoted as $E_{P_{\theta}}$. Then we can approximate $\mathcal{X}{sp}(M)$ as follows:

While the analysis of $P_{\theta}$ is not the primary focus of this paper, in the absence of any other constraints or biases on model $M$, we conjecture that the distribution curve of $\theta$ may resemble a uniform bell curve. Hence, we consider the following distribution approximation:

At this point:

Therefore, utilizing a self-supervised metric can approximate the unbiased optimal geometry defined by the structural probe:

Moreover, $M_{G}$ is an analytically tractable object, implying that the metrics induced by the tree topological probe preserve to a certain extent the two metric properties discussed in the introduction. However, there is a crucial issue that remains unresolved. Can we explicitly construct $maxs_{W}$? Currently, we have not found a straightforward method similar to constructing $pesu_{W}$ for approximating $maxs_{W}$. However, based on the sorting inequality, we can construct a sequence that approximates $maxs_{W}$ based on $pre_{W}$. Let $d_{i}$($1\leq i\leq L_{H}$) denote $L_{H}-i+1$. Then, let

In our subsequent experiments, we approximate $E(\bm{p}dep_{H},maxs_{W})$ with $E(\bm{p}dep_{H},xpesu_{W})$.

We denote the model consisting of the input layer and the first $i$ transformer blocks of BERT-large as $M_{i}(0\leq i\leq 24)$. Since the input of $M_{i}$ consists of tokenized units, including special tokens [CLS], [SEP], [PAD], and [MASK], we can conduct at least four types of measurement experiments:

1. e1.

   Measurement of the vector set formed by token embedding and special token embedding.

2. e2.

   Measurement of the vector set formed solely by token embedding.

3. e3.

   Measurement of the vector set formed by estimated embedding of complete words using token embedding and special token embedding.

4. e4.

   Measurement of the vector set formed solely by estimated embedding of complete words using token embedding.

Similarly, due to space constraints, we focus on discussing e1 in this paper. The measurement results are shown in Tables 1 and 2. The precise measurement values can be found in Appendix E. Furthermore, as shown in Figure 2, we present the negative logarithm curves of three measurement values as a function of $M_{i}$ variation.

By examining the experimental results presented above, we can ascertain the following findings:

1. f1.

   $\mathcal{X}{ssp}$ and $\mathcal{X}{essp}$ indeed bound the actual $\mathcal{X}{sp}$, and for $M_{14}$ to $M_{18}$, their true $\mathcal{X}{sp}$ are very close to their $\mathcal{X}{ssp}$.

2. f2.

   $M_{0}$ serves as a good baseline model. Furthermore, using $\mathcal{X}{essp}$ and unbiased $\mathcal{X}{sp}$ allows for effective differentiation between embeddings generated by models consisting solely of the regular input layer and those generated by models incorporating transformer blocks.

3. f3.

   For $M_{1}$ to $M_{6}$, their true $\mathcal{X}{sp}$ are very close to their unbiased $\mathcal{X}{sp}$.

4. f4.

   Both the curve of $-log(\mathcal{X}{essp})$ and the curve of the true $-log(\mathcal{X}{sp})$ follow an ascending-then-descending pattern. However, the models corresponding to their highest points are different, namely, $M_{8}$ and $M_{16}$, respectively.

5. f5.

   For the curve of $-log(\mathcal{X}{ssp})$, its overall trend also shows an ascending-then-descending pattern but with some fluctuations in the range of $M_{3}$ to $M_{6}$. However, the model corresponding to its highest point is consistent with $-log(\mathcal{X}{essp})$, which is $M_{8}$.

6. f6.

   The true $\mathcal{X}{sp}$ does not effectively distinguish between $M_{0}$ and $M_{1}$.

Based on the above findings, we can confidently draw the following rigorous conclusions:

1. c1.

   Based on f1, we can almost infer that $dep_{W}\in\mathop{\arg\min}\limits_{x\in X_{C_{dep}}}\sum_{i=1}^{L_{H}}(\bm{p}dep(h_{i})-x(w_{i}))^{2}$ for $M_{14}$ to $M_{18}$. This implies that they memorize the preferences of the real data and minimize as much as possible to approach the theoretical boundary. Building upon f5, we can further conclude that the cost of memorizing $dep_{W}$ is an increase in $\mathcal{X}{ssp}$, which leads to a decrease in the accuracy of the embedding’s linear encoding for tree structures.

2. c2.

   Based on f1, we can conclude that there exists a model $M$ where the true $\mathcal{X}{sp}(M)$ aligns with the $\mathcal{X}{ssp}(M)$ determined by $C_{dep}$. This indicates that $C_{dep}$ serves as a sufficiently tight condition.

3. c3.

   Based on f3, we can infer that $M_{1}$ to $M_{6}$ may not capture the distributional information of the actual syntactic trees, resulting in their generated embeddings considering only the most general case for linear encoding of tree structures. This implies that the distribution curve of their $\theta_{W}$ parameters is uniformly bell-shaped.

4. c4.

   Based on f2 and f6, we can conclude that the tree topological probe provides a more fine-grained evaluation of the ability to linearly encode tree structures in embedding vectors compared to the structural probe.

5. c5.

   Based on f3, f4 and f5, we can conclude that in BERT-large, embedding generated by $M_{8}$ and its neighboring models exhibit the strongest ability to linearly encode tree structures. Moreover, they gradually start to consider the distribution of real dependency trees, resulting in the true $\mathcal{X}{sp}(M)$ approaching $\mathcal{X}{ssp}(M)$ until reaching $M_{16}$.

6. c6.

   Based on f4 and f5, we can conclude that starting from $M_{16}$, the embeddings generated by $M_{i}$ gradually lose their ability to linearly encode tree structures. The values of $\mathcal{X}{ssp}$ and $\mathcal{X}{essp}$ for these models are generally larger compared to models before $M_{16}$. However, they still retain some distributional information about the depth of dependency trees. This means that despite having a higher unbiased $\mathcal{X}{sp}$, their true $\mathcal{X}{sp}$ is still smaller than that of $M_{i}$ before $M_{8}$.

From the above conclusions, we can further speculate about the workings of pretrained language models such as BERT, and we identify some related open problems.

Based on c5 and c6, we can speculate that the final layer of a pretrained language model needs to consider language information at various levels, but its memory capacity is limited. Therefore, it relies on preceding submodules to filter the information. The earlier submodules in the model encode the most generic (unbiased) structures present in the language features. As the model advances, the intermediate submodules start incorporating preferences for general structures based on actual data. Once a certain stage is reached, the later submodules in the model start to loosen their encoding of generic structures. However, due to the preference information passed from the intermediate submodules, the later submodules can still outperform the earlier submodules in encoding real structures, rather than generic ones.

Based on c3 and c6, it appears that true $\mathcal{X}{sp}\leq$ unbiased $\mathcal{X}{sp}<$ $\mathcal{X}{essp}$. This suggests that for BERT, unbiased $\mathcal{X}{sp}$ serves as a tighter upper bound for $\mathcal{X}{sp}$, and there exists a submodule that achieves this upper bound. Now, the question arises: Is this also the case for general pretrained models? If so, what are the underlying reasons?

Let us denote the downstream task loss as $T(M_{24})$. Taking $\mathcal{X}{ssp}$ as an example, using $\mathcal{X}{ssp}$ as a regularizing loss during fine-tuning refers to replacing the task loss with:

where $\lambda$ is a regularization parameter. The purpose of this approach is to explore the potential for enhancing the fine-tuning performance by improving the submodules of BERT in their ability to linearly encode tree structures. If there exists a submodule that achieves both enhancement in linear encoding capabilities and improved fine-tuning performance, it implies that the parameter space of this submodule, which has better linear encoding abilities, overlaps with the optimization space of fine-tuning. This intersection is smaller than the optimization space of direct fine-tuning, reducing susceptibility to local optima and leading to improved fine-tuning results.

Conversely, if enhancing certain submodules hinders fine-tuning or even leads to its failure, it suggests that the submodule’s parameter space, which has better linear encoding abilities, does not overlap with the optimization space of fine-tuning. This indicates that the submodule has already attained the smallest $\mathcal{X}{ssp}$ value that greatly benefits the BERT’s performance.

Based on f1, we can infer that $M_{14}$ to $M_{18}$ are not suitable as enhanced submodules. According to c5, the submodules most likely to improve fine-tuning performance after enhancement should be near $M_{8}$. We conducted experiments on a single-sentence task called the Corpus of Linguistic Acceptability (CoLA) (Warstadt et al., 2019), which is part of The General Language Understanding Evaluation (GLUE) benchmark (Wang et al., 2019).

The test results are shown in Table 3. As predicted earlier, enhancing the submodules around $M_{14}$ to $M_{18}$ (now expanded to $M_{12}$ to $M_{19}$) proves to be detrimental to fine-tuning, resulting in failed performance. However, we did observe an improvement in fine-tuning performance for the sub-module $M_{10}$ near $M_{8}$ after enhancement. This gives us an intuition that if we have additional topological probes and similar metrics to $\mathcal{X}{ssp}$ and $\mathcal{X}{sp}$, we can explore enhancing submodules that are in the rising phase of true $\mathcal{X}{sp}$, away from the boundary of unbiased $\mathcal{X}{sp}$ and $\mathcal{X}{ssp}$, in an attempt to improve fine-tuning outcomes.