Instead of having $\boldsymbol{\mathrm{y}}$ be the target, we now want to replace it with a noisy target $\{\boldsymbol{\mathrm{y}}(t)\}_{t=0}^{1}$, so that $\boldsymbol{\mathrm{y}}(0)$ is the true target and $\boldsymbol{\mathrm{y}}(1)$ is pure noise. Let $(\boldsymbol{\mathrm{x}},\boldsymbol{\mathrm{y}}(0))$ be a pair of data/target sampled from the data distribution. Any type of time-dependent noise could be used. Following the recent success of score-based diffusion models (Ho et al., 2020; Song et al., 2020), we chose to gradually corrupt the target over time using a diffusion process:

where $f(\boldsymbol{\mathrm{y}},t):\mathbb{R}^{d}\times\mathbb{R}\to\mathbb{R}^{d}$ is the drift, $g(t):\mathbb{R}\to\mathbb{R}$ is the diffusion coefficient and $\boldsymbol{\mathrm{w}}(t)$ is the Wiener process indexed by $t\in[0,1]$.

The functions $f$ and $g$ are chosen so that $\boldsymbol{\mathrm{y}}(0)$ is the true label ($f(\boldsymbol{\mathrm{y}},0))=g(\boldsymbol{\mathrm{y}},0)=0$) and $\boldsymbol{\mathrm{y}}(1)$ is independent from $\boldsymbol{\mathrm{y}}(0)$ so that $\boldsymbol{\mathrm{y}}(1)$ can be sampled from even when $\boldsymbol{\mathrm{y}}(0)$ is unknown (at inference time). Given the good empirical results from the Variance Preserving (VP) process in score-based diffusion models (Song et al., 2020), we use it as our diffusion process; it is defined as follows:

We can create noisy targets from the VP process by sampling from the following distribution:

where $\beta(t)=\beta_{min}+t\left(\beta_{max}-\beta_{min}\right)$, $\beta_{min}=0.2$ and $\beta_{max}=20$.

Thus, as desired, $\boldsymbol{\mathrm{y}}(0)$ is approximately the target and $\boldsymbol{\mathrm{y}}(1)$ is approximately distributed as $\mathcal{N}(\boldsymbol{0},\boldsymbol{\mathrm{I}})$ and does not depend on $\boldsymbol{\mathrm{y}}(0)$.

Instead of using a diffusion process, it is also possible to use a different type of noise. For example, one could use Laplace distributed noise instead of Gaussian distributed noise. The Laplace distribution has heavier tails while having a higher concentration at its mean. The high concentration at the mean may make it easier to know how close the hint is to the truth. To add Laplace noise, we follow equation 5 but replace the Gaussian distribution by the Laplace distribution:

Our paper focuses on the Gaussian VP Process, but we also study its Laplace equivalent in a few settings.

Rather than solving the supervised learning problem using only $\boldsymbol{\mathrm{x}}$, we now want to further condition on the noisy target:

where $f_{\theta}$ is now a neural network taking $\boldsymbol{\mathrm{x}}$, $\boldsymbol{\mathrm{y}}(t)$, and $t$ as inputs.

At the optimum, the neural network is now the conditional expectation of $\boldsymbol{\mathrm{y}}$ given $\boldsymbol{\mathrm{x}}$, $\boldsymbol{\mathrm{y}}(t)$, and $t$:

At $t=0$, the problem is already solved and the the optimal network returns back the target:

At $t=1$,the optimal network ignores the noisy label since it independent of the label:

At inference, since we do not know the true label, we use $t=1$ to recover our original expected value of $\boldsymbol{\mathrm{y}}$ given $\boldsymbol{\mathrm{x}}$.

Since we sample from a random noisy $\boldsymbol{\mathrm{y}}(1)$, it might appear sensible to average over multiple random draws of $\boldsymbol{\mathrm{y}}(1)$ at inference; however, as shown above, the noise is completely independent of the true label at $t=1$, and the model should ignore the noise by converging to the expected value of $\boldsymbol{\mathrm{y}}$ given $\boldsymbol{\mathrm{x}}$. We found no benefit from using multiple random draws; it only made inference slower. We illustrate CNT with the VP process in Figure 1. During training, we condition on the $\boldsymbol{\mathrm{y}}(t)$ with a random noise-level $t$ as shown in Equation 7. At inference time, we do not know the true label, so we only provide pure noise $\boldsymbol{\mathrm{y}}(1)\sim\mathcal{N}(0,I)$ and the output should return the expected value of $\boldsymbol{\mathrm{y}}$ given $\boldsymbol{\mathrm{x}}$.

There are many possible ways of conditioning on the noisy target $\boldsymbol{\mathrm{y}}(t)$. A very popular way of input conditioning is through conditional normalization layers (Huang and Belongie, 2017; De Vries et al., 2017; Karras et al., 2019; Dhariwal and Nichol, 2021); this is the approach we use.

Our approached for conditioning on the noisy target is explained in Figure 3 and below. We start by processing $t$ through a Fourier positional embedding (Tancik et al., 2020) in order to increase the high-frequency signal. Then, we separately process both $\boldsymbol{\mathrm{y}}(t)$ and the Fourier processed $t$ through small fully-connected networks. Then, we concatenate both outputs to produce an overall embedding for $(t,\boldsymbol{\mathrm{y}}(t))$. After each normalization layer, instead of using learned fixed weights and bias, we learn noise-conditional weights and bias, which are linear projections of the embedding.

The model conditional on the noisy-target has more parameters than a unconditional model due to the embedding parameters and the weights of the linear projections. However, given a fixed $(t,\boldsymbol{\mathrm{y}}(t))$, the network is equivalent to an unconditional neural network. Furthermore, the model should learn to ignore $\boldsymbol{\mathrm{y}}(t)$ when $t=1$ given that the noisy target contains zero information about the target at that level of noise. This relatively simple process can be adapted to any network, even those without normalization layers, by simply applying the conditional weights and bias at different locations through the network.

We test CNT for classification with the following datasets: CIFAR-10 (Krizhevsky et al., 2009), CIFAR100 (Krizhevsky et al., 2009), TinyImageNet (Chrabaszcz et al., 2017), and ImageNet (Deng et al., 2009). We use a basic ResNet architecture (He et al., 2016) for all out models.

We optimize the cross-entropy loss function. As regularizations, we use dropout (Srivastava et al., 2014) with $p=0.1$ (Srivastava et al., 2014) and the mixup data augmentation (Zhang et al., 2017) with $\alpha=1$. The models are trained with Stochastic Gradient Descent (SGD) with learning rate $0.1$, momentum $0.9$, and weight decay $0.0001$. The learning rate is scheduled to decrease by a factor of 10 at $50\%$ and $80\%$ of the training time. TinyImagenet models are trained for 1000 epochs, while other models are trained for 2000 epochs. To reduce variance, we train on all settings three times and report the mean and standard deviation. Results are shown in Table 1.

From Table 1, we see that CIFAR-10 shows better results for only-noise, CIFAR-100 for CNT, and TinyImageNet for either CNT or baseline. Thus, only-noise and CNT almost always perform better than baseline, albeit by a small margin. The fact that only-noise is sometimes better (in CIFAR-10) suggests that adding noise may have its own benefits.

In the next set of experiments, we show that the benefits of CNT become larger and more consistent when the classification models have low capacity. To test the effect of CNT in a low-capacity model, we train the previous models with only 8 channels or/and with half the number of layers. These experiments use mixup $\alpha=0.5$. Results are shown in Table 2. From 2, we see that CNT almost always leads to higher accuracy than the other methods except for ResNet9 with TinyImageNet, which obtains slightly better accuracy with only-noise. This provides evidence that conditioning on the noisy target can be very beneficial, especially when the model struggles due to low capacity.

We test CNT on the task of detecting geometrical shapes. We build on the equilateral triangle detection task from (Ahmad and Omohundro, 2009). In addition to triangles, we also add squares and rectangles to our task. Each image in the dataset is of size $64\times 64$. Each vertex of a geometrical shape is represented by a closely formed cluster of points centered at a randomly chosen coordinate in the image. For triangles, we add three such clusters to the image and for quadrangles, we add four such clusters to the image. We ensure that the positioning of these clusters satify the geometrical properties of the shape they represent. For example, we ensure that the clusters are equidistant for the equilateral triangle and the square. Also, we ensure that the sides for the quadrangles are perpendicular to each other.

This task is framed as a classification task where we have two classification heads - One denotes whether the given shape has 4 sides or 3 sides and the second denotes whether the given shape has equal length sides or not. We use a basic ResNet architecture (He et al., 2016) and CaffeNet (Jia et al., 2014). We use dropout (Srivastava et al., 2014) with $p=0.1$ (Srivastava et al., 2014) in the ResNet architecture. We incorporate noise by feeding the noisy label through the normalization layers in each architecture similar to Figure 3. We minimize a binary cross entropy loss function for both classification heads. The models are trained for 200 epochs with Stochastic Gradient Descent (SGD) with learning rate $0.1$, momentum $0.9$, and weight decay $0.0001$. The learning rate is scheduled to decrease by a factor of 10 at $50\%$ and $80\%$ of the training time. To reduce the variance, we train each model three times and report the mean and standard deviation. For the result, we report the results for the second classification head i.e. whether the given shape has equal length sides or not.

We report the results in Table 3, we see that CNT consistently leads to higher accuracy in all settings. The effect of CNT is more significant with CaffeNet, probably because this architecture has a lower capacity than ResNet.

Generalization on relational reasoning tasks has often been difficult for deep supervised networks, which has been partially successfully addressed through the use of curriculum learning to encourage learning simpler skills which can be re-used and re-purposed to solve more complex tasks which require multiple skills. For example, if a model needs to solve a relational question such as: “What color is the shape to the left of the triangle?”, the model needs to be able to recognize colors, detect shapes, and understand relative positioning of shapes. If a model masters all of these simpler individual skills it will generalize better when solving novel composite tasks which use many different skills.

We evaluated CNT on the Sort-of-CLEVR relational reasoning benchmark (Santoro et al., 2017). Each image in the sort-of-clevr benchmark is of size $75\times 75$. The input consists of a question and an image. The task contains 3 types of questions - Unary, Binary, and Ternary. Unary questions consider properties of single objects. For example, what is the color of the square?. Binary questions consider relations between two objects. For example, what is the shape of the object closest to the red object?. Similarly, ternary questions consider relations between 3 obejcts. The Sort-of-CLEVR images consist of 6 randomly placed geometrical shapes of 6 possible colors and 2 possible shapes.

To demonstrate the generality of CNT, we evaluate both convolutional resnets (Preactresnet18 with 64 initial channels) and transformers on the Sort-of-CLEVR benchmark. Unlike with equilateral shape classification and image classification, we used relu everywhere in the convolutional network for the Sort-of-CLEVR tasks. We also injected a question-embedding along with the label and noise-level embeddings as a way of conditioning on the question. We report results on Sort-of-CLEVR (Table 4). Additionally, we found that accuracy converged fastest at the lowest noise levels (Figure 4) and more gradually at higher noise levels.

We test CNT on the atari benchmark (Bellemare et al., 2012) following the same setup as decision transformer (Chen et al., 2021). The key in decision transformer is the choice if trajectory representation. The trajectory is represented such that it allows conditional generation of actions based on future expected rewards. This is done by conditioning the model on return-to-go $\hat{R}_{k}=\sum_{k^{\prime}=k}^{K}r_{k}$, where $k$ denotes the timesteps. This results in the following trajectory representation: $\tau=\big{(}\hat{R}_{1},s_{1},a_{1},\hat{R}_{2},s_{2},a_{2},\hat{R}_{3},s_{3},a_{3},\ldots\big{)}$. At test time, we can condition the desired return $\hat{R}_{1}$ and the start state $s_{1}$ to generate actions.

Similar to decision transformer, we use a fixed context of length of $K$ to train our models i.e. we only feed in the last $K$ timesteps. This results in a sequence length of $3K$ (considering 3 modalities - returns, states, and actions). Each modality is processed into an embedding - The states are processed using a convolutional encoder into an embedding, the returns and actions are processed using linear layers. The processed tokens are fed into a GPT (Radford and Narasimhan, 2018) model. The outputs corresponding to $s_{k}$ are fed into a linear layer to predict the action $a_{k}$ to be taken at $k$. To incorporate CNT, we replace the layer normalization layers in GPT with conditional normalization layers. During training, we feed the noisy actions $a_{k}(t)$ and $t_{k}$ through the conditional normalization layers and the model is tasked with predicting $a_{k}(0)$ (the true action).

We train on 500000 examples from DQN replay dataset similar to (Agarwal et al., 2019). We run on 4 atari games - Qbert, Seaquest, Pong, and Breakout. We use a context length ($K$) of 30. We report the mean and variance across 10 seeds. We report the results in Table 5.

To demonstrate the generality of CNT, we also trained with noisy targets where the noise follows a Laplace distribution instead of a Gaussian distribution (Figure 5). Aside from this change, we kept all hyperparameters the same and used the Mish activation. These results are shown in Table 6.