Layer Collaboration in the Forward-Forward Algorithm

Deep neural networks (DNNs) have been the backbone of many state-of-the-art machine learning systems in recent years. Their ability to learn complex representations from large amounts of data has led to breakthroughs in many fields such as image and speech recognition, natural language processing, and game playing (Lavin and Gray, 2016; Bojarski et al., 2016; Luss et al., 2019; Gat et al., 2021; Morais et al., 2022; Rosenberg et al., 2021; Ophir et al., 2020; Zhou et al., 2021; Clauset and Post, 2019; Fedorov et al., 2020). One of the key reasons for the success of deep neural networks is the backpropagation algorithm, which is currently the most widely used training paradigm.

Backpropagation (Rumelhart et al., 1986) utilizes the chain-rule to compute gradients for minimization of a loss function, which measures the discrepancy between the predicted output and the desired result. The algorithm starts with a forward pass through the network to compute the predicted output and then calculates the error between the prediction and the desired output. The error is then propagated back through the network using the chain rule of calculus, computing the gradient of the loss function with respect to the network’s parameters, i.e., the weights and biases. These gradients are then used to update the parameters via gradient descent variants, reducing the loss function and subsequently the error. This process is repeated until the network reaches a satisfactory error on the training data. The chain rule presented challenges for a variety of model architectures. In very deep networks, gradients of the loss function with respect to the weights can become small, resulting in vanishing gradients. Moreover, the gradient can also become too large. Both vanishing and exploding gradients lead to poor performance.

Recently, Hinton (2022) introduced the forward-forward algorithm, which offers an alternative to the backpropagation training process by replacing the forward and backward passes with two forward passes. The data that is forwarded in the network is the sample itself (e.g., image, text) concatenated with a one-hot encoded vector that represents a label. The first forward pass is done with positive data (i.e., data concatenated with the true label) and the second forward pass is done with negative data (i.e., data concatenated with a false label). The network then classifies whether a sample is concatenated with its true label. An important aspect of the forward-forward algorithm: the aforementioned optimization process is performed separately for each layer.

In this work, we examine the dynamics of individual layers during the forward-forward algorithm based training process. Our findings indicate that, during training, the layers exhibit a lack of effective intercommunication, which results in a suboptimal level of collaboration. Based on this observation, we propose a simple modification to the forward-forward algorithm to enhance inter-layer communication during training. Our empirical results demonstrate that incorporating this modification leads to a significant improvement in performance.

To further investigate this phenomenon, we analyze forward-forward-based models through the lens of functional entropy. We demonstrate that the forward-forward method implicitly maximizes the entropy within the network. Throughout this work, we analyze the dynamics of the entropy when training a model using the forward-forward algorithm.

Discriminative learning seeks to construct a mapping between data-instance $x\in\mathcal{X}$ and corresponding label $y\in\mathcal{Y}$, given training data $\{(x_{1},y_{1}),...,(x_{m},y_{m})\}$. Classical discriminative deep fully-connected neural networks have a recursive structure. Each layer processes its predecessor’s output. Thus, a k-layer fully connected network $f$ can be viewed as a chain of its sub-functions,

where each layer $i$ is denoted by $f_{i}$. In the following, we cover backpropagation and forward-forward algorithms. In this work, we also point out some of their differences, as well as pros and cons.

Deep nets consist of multiple layers. Each layer processes the input data and passes it to the next layer, allowing the network to learn increasingly complex representations of the data. Thus, layer collaboration is crucial in deep nets because it allows the network to learn and extract features from the data in a hierarchical and efficient manner. For example, lower layers of the network might learn to identify ‘simple’ features such as edges and textures, while later layers might learn complex features such as object shapes and patterns. This hierarchical structure allows the network to build a robust representation of the data and make accurate predictions.

The forward-forward algorithm permits communication between layers only through the forward pass because each layer only takes into account the output of its predecessor. In this work, we find that the forward collaboration solely is not sufficient for developing a complex representation of the data. One can observe from Equation 5 that the forward-forward process detailed above does not enable the flow of information to earlier layers (i.e., layers closer to the data) while training (see Figure 1). For example, the first layer is not aware of the existence of later layers and hence can not consider information from them in its optimization process. This is contrary to the backpropagation algorithm. From Equation 3 we observe that information can flow from a layer to its predecessors through the gradients of the optimization step.

In the following, we examine the collaboration between layers when training models using the forward-forward algorithm. We then propose a modification to the forward-forward algorithm that enables better collaboration between layers.

The functional entropy can measure the amount of information that is shared across layers in the forward-forward algorithm, and consequently allows us to quantify and compare layer collaboration. In the following section we formulate the connection between the forward-forward algorithm and functional entropy. Using this connection, we show that during training, the functional entropy of the network implicitly improves. In Section 5, we propose to directly use the functional entropy during training and show that it is beneficial to the performance.

The functional entropy is defined over a non-negative random variable, which we denote by $h(z)$ that is defined over the space ${\cal Z}$ with a probability measure $\mu(z)$. Here and throughout, we use $z$ to refer to a stochastic variable, which we will integrate over. The functional entropy of the non-negative function $h(z)\geq 0$ is

The functional entropy is non-negative, namely, $\operatorname{Ent}_{\mu}(h)\geq 0$ and equals zero only if $h(z)$ is a constant.

Our settings are defined over the input space $(x,y)$ where $x\in\mathbb{R}^{d}$ and $y\in{\cal Y}$. In the following, we denote by $z$ the data-label tuple $(x,y)$ combined with the layer index $i\in\{1,...,k\}$. We use ${\cal Z}=\mathbb{R}^{d}\times{\cal Y}\times\{1,...,k\}$ to refer to its domain. The functional entropy can be intuitively considered as a scaled KL-divergence between the distribution of the input of a layer and the output of a layer $h(z)\triangleq\left\lVert f_{1:i}(x,y)\right\rVert^{2}$. Formally, consider the prior and posterior distributions, $p(z),q(z)$, i.e.,

One can verify that $q(z)$ is a probability density function since $q(z)\geq 0$ and $\mathbb{E}_{z\sim\mu}[q(z)]=1$. The functional entropy can now be described as the scaled KL-divergence between the prior and posterior distributions above:

Thus, intuitively, the functional entropy is the amount of learning that is done by a function $h(z)$, measured by the KL-divergence from the prior distribution of the input $p(z)$ to the posterior distribution of the output $q(z)\propto p(z)h(z)$.

In Figure 4 we validate that our method improves the overall functional entropy of the learned network. We attribute its improvement to our learning algorithm, which encourages functional entropy to flow between the layers of the network. To formulate this notion, we decompose the functional entropy by layers, and measure a notion of conditional functional entropy with respect to the layers themselves. The probability measure $\mu(z)$ for $z=(x,y,i)$ accounts for the input $(x,y)$ of the $i$-th layer when computing its output $\left\lVert f_{1:i}(x,y)\right\rVert^{2}$. The probability measure is $\mu(z)=\mu_{1}(i)\times\mu_{2}(x,y)$ which does not enforce statistical dependencies between the data generating distribution $\mu_{1}(x,y)$ and the layer distribution $\mu_{2}(i)$, for which we take the uniform distribution. In this setting, the functional entropy decomposes into two components¹¹1This decomposition can be verified by adding and subtracting $\operatorname{\mathbb{E}}_{i}\left[\left(\operatorname{\mathbb{E}}_{(x,y)|i}[h(x,y,i)]\right)\log\left(\operatorname{\mathbb{E}}_{(x,y)|i}[h(x,y,i)]\right)\right]$ to $\operatorname{Ent}_{\mu}[h]$.:

The first component is the functional entropy of the layers’ distribution of the marginal function $\operatorname{\mathbb{E}}_{(x,y)}[h(x,y,\cdot)]$. This entropy accounts for the information across layers. The seconds component is an average of entropies that are conditioned on the i-th layer.

In this section, we study various modifications to the forward-forward algorithm. Specifically, we begin by studying the collaboration of layers when training using classical and collaborative forward-forward. We study the collaboration from both a performance and an entropy point of view. Then, we present the benefits of our method when using activations other than ReLU. In addition, we find that the introduced functional entropy concept is highly effective in replacing the sigmoid in forward-forward optimization.

Our work focuses on the collaboration between layers in forward-forward-based models. In this section, we review research directions that are related to the forward-forward method, layer collaboration in deep neural networks, and functional entropy.

This work studies layer collaboration of models that are trained using the forward-forward algorithm. We find that, in its current form, the forward-forward algorithm is limited in its collaborative capabilities, meaning information can not propagate backward throughout the network. We study this phenomenon and propose a method to overcome this challenge. We further propose an entropic-based view of the forward-forward algorithm, and, through extensive experiments, show the benefits of following the proposed method.

Nevertheless, the forward-forward algorithm is far from being comparable to the standard backpropagation. Many questions remain open as to how it can be applied to complex applications, e.g., image classification, text analysis, generative learning, and modern neural architectures, e.g., convnets, transformers, etc. We hope the community will keep pursing such research directions and challenge popular methods.