MTL2L: A Context Aware Neural Optimiser

Deep learning has firmly established itself in image recognition (He et al., 2016), speech recognition (Oord et al., 2016), and text processing (Vaswani et al., 2017) through training models with high-capacity on a large amount of labelled data. However, it remains difficult to learn from a low resource environment. This challenging task is carefully studied in meta-learning, the branch of machine learning that addresses rapid knowledge acquisition from only a few examples. In this paper, we address extensions to the neural optimiser approach for the meta-learning task of Learning to learn (L2L) (Andrychowicz et al., 2016).

L2L formulates the concept of learning quickly as the task to rapidly converge a network with the least amount of iterations. This task involves training a meta-learner to assist the learning of a base learner. The learner performs a specific task, such as image classification; whereas the meta-learner captures how task structures vary within the domain (Rendell et al., 1987). This paper extends on meta-learners which directly update learner parameters.

Andrychowicz et al. (2016) cast optimisation as the learning target for a recurrent neural network (RNN). The RNN was meta-trained as a neural optimiser meta-learner; then meta-tested to update a base learner. They showed that the RNN neural optimiser converged

learners more rapidly than hand-designed optimisations such as SGD (Robbins and Monro, 1951) and ADAM (Kingma and Ba, 2014). Learning to optimise has since become a common approach in meta-learning, and has been extended by the LSTM-metalearner (Ravi and Larochelle, 2017) and Meta-SGD (Li et al., 2017).

However, neural optimisers have the two limitations¹¹1See more descriptions in Appendix A. illustrated in Figure 1. First, neural optimisers are specific to architectures. If they were meta-trained to update a multiple layer perceptron (MLP), it cannot be guaranteed to work on ResNet (He et al., 2016). Second, neural optimisers are specific to input-domain (dataset). When naively applied, they cannot update the same learner on two distinct datasets as the underlying task structures may vary significantly in the two input-domains. Neural optimisers are thus not as generally applicable as SGD.

We prepare Figure 2 to demonstrate and to convince our readers of the consequences of naively applying neural optimisers on an unseen dataset. The solid cyan lines represent the RNN-optimiser of Andrychowicz et al. (2016). Figure 2(a) illustrates meta-learning under the common assumption: to meta-test on the single identical input-domain used in meta-training; and Figure 2(b) depicts deteriorated performance in meta-testing on an unseen dataset. In this paper, we introduce Multi-Task Learning to Learn (MTL2L) – a novel context aware neural optimiser which supports multi-task learning (Caruana, 1997). MTL2L can capture shared task structures across multiple input-domains to update learners on a dataset unseen during meta-training. The experiments will be revisited in Section 4.

The classic gradient descent updates parameters $\theta_{t}$ of learner $F$ at time $t$ via

Andrychowicz et al. (2016) employed the long short-term memories (LSTMs) (Hochreiter and Schmidhuber, 1997; Gers et al., 1999) presented in Equations (5) to (8) in Table 1 as RNNs for their neural optimiser. Symbols $\sigma$, $\tanh$, and $\odot$ are the sigmoid function, hyperbolic tangent function, and element-wise dot product, respectively; with synapses $\mathbf{W}$ and $\mathbf{U}$, and bias $\mathbf{b}$. The forget gate $\mathbf{f}_{t}$ and the input gate $\mathbf{i}_{t}$ modulate cell $\mathbf{c}_{t}$ with new

memory $\mathbf{a}_{t}$. The LSTM output is hidden state $\mathbf{h}_{t}$, $\mathbf{c}_{t}$ transformed with controlled exposure from the output gate $\mathbf{o}_{t}$. Our novel Multi-task Learning to Learn (MTL2L) neural optimiser is introduced in Equations (9) to (16), and replace LSTMs as $\mathcal{V}$ in Equation (3).

We took a HyperNetwork (Ha et al., 2016) approach towards dataset-agnostic meta-learning. HyperNetworks are small auxiliary networks that re-implement weights for a larger network based on input data. Equations (13) and (14) describe MTL2L synapses as variables via singular value decomposition (SVD). SVD splits each synapse as fixed purple terms and orange components that require learning. The former are constant orthonormal matrices, and the latter are diagonal matrices with eigenvalues as non-zero entries.

The eigenvalues of the orange matrices are inferred with hypernetworks $\mathscr{N}=\mathscr{N}(\mathbf{x}_{t})$. Since $\mathscr{N}$ yield different combinations of eigenvalues for different datasets, MTL2L becomes context aware and self-modifies its optimisation rules based on input data. It is natural for MTL2L to support multi-task learning. During meta-training, we expose MTL2L to different datasets as shown in Figure 3(b) – the aim is to represent each task structure as different combinations of eigenvalues. During meta-testing, the eigenvalues enable MTL2L to compare the task structure of the input data to those learnt during meta-training. This enables MTL2L to update base learners on an unseen dataset at the meta-testing phase.

We followed Andrychowicz et al. (2016) and employed neural optimisers to update MLPs. The neural optimisers were meta-trained to update the MLPs for 100 steps, but during meta-testing, they updated MLPs for the much longer 1000 steps. Base on this setting, we present two experimental scenarios as below.

• Scenario 1

  : To test under the common meta-training assumption
  First meta-train neural optimisers to update MLP learners to classify images on one single dataset; and then meta-test by employing the learnt neural optimisers to update MLP learners to classify the identical dataset used in meta-training.

• Scenario 2

  : To test for input-domain heterogeneity
  Unlike Scenario 1, meta-test to update learners to classify an unseen dataset.

We tested MTL2Ls against LSTM-optimisers, and against the hand-designed optimisation algorithms of vanilla SGD, SGD with momentum 0.9, and ADAM; all with learning rates 0.01. We presented our results as the learning curves of the learners. The results were colour-coded. We use black solid lines for MTL2Ls, cyan solid lines for LSTM-optimisers, red ‘$\times$’ for vanilla SGD, yellow ‘$+$’ for SGD with momentum, and blue ‘$\star$’ for ADAM.

We upated the MLPs on MNIST (LeCun et al., 1998), Fashion-MNIST (Xiao et al., 2017), KMNIST (Clanuwat et al., 2018), and Cifar10 (Krizhevsky, 2009). Due to space constraint, we refer the readers to Appendix B for descriptions on the base learner, descriptions on the datasets, hyper-parameters for the neural optimisers, and choices of hypernetworks $\mathscr{N}$s (see Section 3) for MTL2Ls.

The results for Scenario 1 are shown in Figure 4. For all datasets, the neural optimisers updated learners more rapidly than their handcrafted counterparts. MTL2Ls also converged the learners more efficiently and yielded lower losses than LSTM-optimisers.

The results for Scenario 2 are shown in Figure 5. Since LSTM-optimisers only supported single-task learning, they were meta-trained to update MLPs for KMNIST. In contrast and as MTL2Ls naturally supports multi-task learning, we meta-trained them to update MLPs on both Fashion-MNIST and KMNIST. During meta-testing, all optimisers updated MLPs on a modified re-sized greyscale Cifar10 dataset (see the synthesis procedure in Appendix B.2). As shown in the figure, LSTM-optimisers exhibited instability with deteriorating performances whereas MTL2Ls displayed stability.

It was also interesting to observe the timing with which the LSTM-optimisers exhibited instability. As shown in Figure 5, the LSTM-optimiser instability was less severe in the first 400 iterations – the loss curve upraised but with no volatile oscillations. The volatility was only amplified after the first 400 iterations. This showed that there existed some underlying similarities between the task structures for updating learners on KMNIST and on the modified Cifar10. Hence, we conjectured that LSTM-optimisers were actually able to capture the “overall trends” in updating learners for an unseen dataset, but with imprecision. Furthermore, such imprecision introduced errors which reiterated in the LSTM and resulted in the eventual deteriorating performances.

In this present work, we extended on the seminal paper of Andrychowicz et al. (2016) which explored meta-learning via learning to optimise. As stated in Equation (2), Andrychowicz et al. (2016) proposed to learn alternative rules to replace the traditional gradient descent methods. Their work has also been extended by the LSTM-metalearner (Ravi and Larochelle, 2017) and Meta-SGD Li et al. (2017) to learn auxiliary components compatible with the gradient descent regime²²2An extended Related Work section can be found in Appendix A..

Learning to optimise is one approach to meta-learning. Alternatively, there are metric-based approaches (Vinyals et al., 2016) which learn a kernel function to estimate the similarity between two sampled data. There are also model-based approaches (Munkhdalai and Yu, 2017) that combine weights updated at different timescales to synchronise different levels of abstracted features.

Besides the task of L2L, there are different ways to formulate the concept of learning quickly. For instance, few-shot learning (Vinyals et al., 2016) trains a network to map unlabelled data to their labels based on a small labelled support set.

Model-agnostic meta-learning (MAML) (Finn et al., 2017) is a meta-learning technique that addresses both input-domain heterogeneity and learner architecture heterogeneity (see Figure 1). Instead of training a neural optimiser from the scratch, MAML adopts a double-loop optimisation setup with hand-designed optimisation algorithms. The inner loop finds new candidate weights for the learner, and the outer loop aggregates all candidates to find an optimal update. Hence MAML optimises the parametric initialisation for the least amount of updates to converge a learner.

The hierarchically structured meta-learning (HSML) (Yao et al., 2019) extended on the idea of MAML. HSML jointly trains a task clustering network to learn features specific to datasets as parametric initialisation for the base learner.

In this paper, we showed that neural optimisers are able to perform beyond the common meta-learning assumption: to meta-test on the single identical input-domain used during meta-training. We introduced our novel context aware neural optimiser Multi-task Learning to Learn (MTL2L) with synaptic connections described with SVD and hypernetworks. MTL2Ls encoded input features as eigenvalues for their synapses to self-modify its optimisation rules. In our experiments, we meta-trained our MTL2L on Fashion-MNIST and KMNIST, then meta-tested it on an unseen modified Cifar10 dataset. We demonstrated that MTL2Ls successfully converged learners on an unseen input-domain, thereby making neural optimisers more generally applicable.

Our focus in Section 5 was to highlight different approaches in meta-learning. Here and as requested by our reviewers, we present an additional literature review on studies that extended the original work of Andrychowicz et al. (2016).

This appendix documents the experimental details for Section 4. Information will be provided, in the order of

• (1)

  : The MLP Learner,

• (2)

  : Dataset description,

• (3)

  : Gradient pre-processing,

• (4)

  : The meta-training hyper-parametric setups,

• (5)

  : Choices of orthonormal SVD matrices, hypernetwork $\mathscr{N}$s, and
          the MTL2L multi-task setup.