Learning Vision-and-Language Navigation from YouTube Videos

Vision-and-Language Navigation (VLN) [2] is a challenging task and has received continuous and intense attention from the academic community in recent years [5, 13, 29, 35, 53, 31, 7, 52, 10]. Early methods attempt to learn the agent from sequence-to-sequence models [2, 11, 54]. However, these methods can not model the cross-modal relation between language and visual observation well. To address this issue, transformer [55] architecture is adopted to the agents followed by vision-and-language pre-training [40, 57, 65, 43, 14, 49, 6, 44, 27]. PREVALENT [16] pre-trains transformer-based agent via masked language modeling and action prediction tasks. Inspired by BERT [9], several works propose to use different variants of BERT for VLN pre-training. VLN-BERT [43] utilizes image-text data [39] to perform path-instruction matching pretext task. Airbert [14] proposes the shuffle loss to improve the ability of the model to learn the order of image-caption pairs. Recently, HOP [49] introduces the history and order-aware pretext tasks. However, these existing pretext tasks do not consider the learning of environment layout reasoning ability and bring limited performance for VLN tasks. In this work, we propose a trajectory judgment task to teach the agent to distinguish reasonable navigation trajectories. By proficiently accomplishing this task, the agent can acquire the ability to reason about environment layout, which enhances its generalization capability to unseen environments.

The major difficulty of generalizing a VLN agent to unseen environments is the scarcity of VLN training data. Well-labeled VLN datasets built from the simulators [3, 51, 59] allow the agent to obtain a promising performance, such as R2R [2], R4R [22], RxR [26] and SOON [64]. While the data built from simulators are laborious, Wang et al. [56] and ProbES [34] enrich the navigation instructions via self-exploration in the simulation environments. Some other endeavors [54, 32, 11, 12, 24, 30] seek to augment the data from existing datasets. AutoVLN [8] and Kamath et al. [23] enlarges the VLN data from simulations with a larger number of environments. However, these datasets are limited by the number of scenes in simulators. To ease this problem, VLN-BERT [43] leverages the abundant web image-captions pairs as VLN pre-training data. Airbert [14] further exploits indoor house images and captions from the web to construct path-instruction pairs for VLN pre-training. However, the trajectories constructed by simply splicing pictures may be confusing and ambiguous. In our work, we address these problems by proposing a large-scale in-domain VLN-like pre-training dataset, providing the agent with diverse visual environments and reasonable layouts.

We seek to construct discrete navigation trajectories from YouTube videos. Similar to discrete navigation datasets (e.g., R2R [2]), each trajectory comprises $K$ navigation nodes, representing different locations of a navigation path. This entails addressing two major challenges: 1) how to determine the locations composing a trajectory to make the trajectory more diverse and 2) how to represent the visual content at each node location. To tackle these challenges, we first collect large-scale consecutive indoor frames from YouTube videos. Then, we group the adjacent frames according to their room types and consider each group as a node. Last, we present an entropy-based technique to select the most informative frames in a group for representing the visual content in a node.

Collecting Navigation Data from YouTube. Real estate agents typically tour a house in each video. To satisfy the visual diversity and dataset scale, we create the YouTube-VLN dataset from 4078 videos collected from various uploaders, with a total duration of 433 hours. In contrast to prior work [4], which relied on a limited set of videos from a single uploader, our dataset features greater diversity and volume. We also employ sparse sampling and off-the-shelf image classifiers [17, 50] to pre-process the videos, filtering out redundant or noisy frames (those featuring people or outdoor scenes), resulting in a final set of 587k indoor frames suitable for constructing trajectories.

Determining the Locations of Trajectory Nodes. A real robot often needs to go through different locations for navigating to a goal. To mimic the real navigation process, we expect that our constructed trajectories also contain diverse visual content within the limited navigation nodes. To achieve this, we first utilize the powerful large model CLIP [50] to recognize the room type of each indoor image. Then, we gather temporally adjacent frames with the same room type as a group and consider this group as one of the navigation nodes. In this sense, the navigation nodes are diversely spread in different rooms and the constructed trajectories are able to mimic the real navigation process. We also call this kind of node a room node. In practice, to increase the visual diversity of trajectories, we also randomly insert transition nodes that are composed of video frames captured during the transition from one room to another one, between two adjacent room nodes.

Representing Visual Content in a Node. A node consists of a group of images and sometimes the number of images may exceed 100 as the photographer could stay in the same room for a long time. Hence, we have to select the most informative images for representing node features. Inspired by EATA [45], an image with lower classification entropy is more reliable, containing more information relevant to a specific class (room type in our case). We thus propose to select an image with the lowest classification entropy to represent the current view of a node. In order to mimic the panoramic visual context, we then merge $M$ adjacent consecutive images of the current view. It is worth noting that our node features better represent a panoramic view compared with Airbert [14] because we merge adjacent frames that belong to the same place as the current view.

Ultimately, we randomly chose $K$ continuous nodes to construct a trajectory. An example of the constructed trajectory is shown in Figure 2.

In addition to constructing navigation trajectories, one more important step for building a VLN dataset is to create the corresponding instructions without manual annotation. The main challenge for this step is how to correctly describe visual content and actions along navigation paths. To conquer this challenge, we first generate instruction templates with verb and noun phrase blanks. Then, we describe each node in trajectories using the CLIP [50] model and infer the native actions using an action inverse model [4]. To generate the final instructions, we fill the instruction templates with these visual descriptions and actions.

Specifically, we first generate templates with verb and noun blanks from instructions in the R2R dataset following Airbert [14]. For noun blanks, we fill them with visual content descriptions about each node. We select the frame with the lowest classification entropy (as described in Sec. 2) and use the CLIP model to infer the objects it contains, together with the room type to populate a noun blank. For verb blanks, the existing instruction generation method [14] is unable to fill them with the correct action words because it cannot figure out the actions taken for navigating from one image to another. This makes the agent confused when it observes similar viewpoints transition but is given different action descriptions. To tackle this problem, we propose an action-aware strategy to fill instruction templates with native actions instead of random inconsistent actions. To be specific, we follow [4] to train an action inverse model, which has 96% prediction accuracy for predicting native actions, to pseudo-label the trajectory with action labels from one location node to another. The predicted actions are then converted into actionable verbs, i.e., “go forward”, “turn left” and “turn right”. For each noun blank that has been filled with the description of one node, we find its closest verb phrase blank and fill it with the pseudo-labeled action which is executed to reach the next node. This eventually enables us to create action-aware instructions.

We follow Airbert [14] to leverage a ViLBERT-like [39] architecture as the model backbone. The model encodes the sequential visual region features and the text token via two separate transformers respectively. More formally, the path-instruction pair consists of $K$ nodes $\left\{V_{k}\right\}_{k=1}^{K}$ and $L$ text tokens $\left\{w_{l}\right\}_{l=1}^{L}$. Each node $V_{k}$ is composed of $R_{k}$ visual region features $\left\{v_{i}^{k}\right\}_{i=1}^{R_{k}}$. In this way, we represent the visual and text inputs respectively as follows:

Given the aforementioned model architecture, we propose to train a VLN agent with a trajectory judgment (TJ) task, enabling it to reason about layouts. Herein, we elaborate on the proposed trajectory judgment task.

Formulation. The trajectory judgment task aims to judge the reasonableness of trajectories. We consider the trajectories generated in the way described in Section 3.1 as positive (reasonable) samples and the shuffled trajectories as negative (unreasonable) ones. To finish this task, the agent is required to reason about the visual information and identify the room types, then infer whether the trajectory matches the real environment layout distribution. Specifically, we first calculate the dot product of the output features of [IMG] and [CLS] tokens. Then, we feed this vector feature to a linear layer to predict the probability that indicates whether the trajectory is reasonable. The model aims to minimize the binary cross-entropy loss:

Sample Generation. We propose to shuffle the positive sample to generate the negative samples: 1) shuffle only the transition nodes; 2) shuffle all the nodes; 3) keep the order of the room nodes, and randomly insert nodes from other videos. In this way, we create rich and hard negative samples, which increases the task difficulty, helping the agent understand the real layout in a more complex manner.

Combining with Existing Pre-Training Tasks. As depicted in Figure 3, we follow Airbert [14] to pre-train the model backbone using our proposed trajectory judgment task, additionally combining with three other existing pretext tasks, namely masked language modeling (MLM), masked vision modeling (MVM) and path ranking (PR) on YouTube-VLN dataset. For MLM, we randomly mask out the words in instruction and the goal is to recover the masked words. Similar to MLM, MVM is designed to predict masked image regions. PR is a ranking task, which aims to decide the most matching path-instruction pair among a few pairs.

We adapt the pre-trained model to both goal-oriented navigation task and object-oriented navigation task. All the tasks are based on the Matterport3D simulator [3]. We utilize R2R [2] as the benchmark for the goal-oriented navigation task, which is divided into discriminative setting and generative setting. As for the object-oriented task, we evaluate our model on REVERIE [48] in generative setting.

The discriminative setting formulates VLN as a path-selection problem, requiring the agent to choose the path that best matches the instruction from multiple candidate paths. Under the discriminative setting, we utilize the classifier used in the path ranking pretext task for decision-making and fine-tune the Lily agent on the R2R dataset.

In the generative setting, the agent needs to predict actions sequentially to reach the goal (R2R) or simultaneously find the object (REVERIE). We adopt DUET [7] as the architecture for fine-tuning, which feeds the cross-modal feature into a feed-forward network for decision-making. We initialize the text transformer encoder and cross-modal transformer encoder of the generative model with our Lily agent. Note that our Lily agent can apply to any generative model. More details are available in the supplementary.

Dataset and Evaluation Metrics. We conduct our experiments on two VLN benchmarks, i.e., R2R [2] and REVERIE [48]. These two datasets consist of 21,567 path-instruction pairs from 90 scenes in Matterport3D [3]. REVERIE follows the same train/val/test splits as the R2R, while requiring an agent to select the bounding box of the target object bounding box additionally. Following standard settings [14], we adopt five metrics for evaluating R2R, namely success rate (SR), oracle success rate (OSR), success rate weighted by the ratio between the length of the shortest path and the predicted path (SPL), trajectory length (TL) and navigation error (NE). As for REVERIE, we leverage four metrics for evaluating navigation performance, namely SR, OSR, SPL and TL, and two for object grounding performance, namely remote grounding success (RGS) and RGS weighted by path length (RGSPL).

Implementation Details. We implement our method based on Pytorch framework [47] and Matterport3D simulator [3]. Specifically, we divide our training process into two stages, i.e., pre-training and fine-tuning. For the pre-training stage, we distribute training over 4 NVIDIA 3090 GPUs for 500 epochs to convergence. The pre-trained model with the highest accuracy for the path ranking pretext task is selected for fine-tuning. During the fine-tuning stage, we distribute training over 8 NVIDIA 3090 GPUs for 30 epochs to convergence. Following Airbert [14], we use augmented data from EnvDrop [54] for fine-tuning by default. More details are provided in the supplementary.

We ablate our approach under discriminative setting on R2R benchmark. Considering the time efficiency, we do not use augmented data for fine-tuning on these experiments.

Data Source: Airbnb Images vs. YouTube Videos. One of the main differences between YouTube-VLN dataset and Airbnb dataset [14] is the data source. Airbnb consists of 713k image-caption pairs while YouTube-VLN consists of 587k images extracted from 433 hours of house tour videos. YouTube-VLN has fewer images but provides more information about a room from different camera angles by merging, which better simulates a panorama for downstream VLN tasks. To evaluate the effect of data source, we follow Airbert [14] to randomly select images in the same house to build a trajectory and its corresponding instruction. We keep the number of instructions the same in pre-training for a fair comparison. In Table 1 (# 1 vs. # 2), YouTube data performs slightly better than Airbnb data, surpassing the SR by 0.89% on the val unseen split. We speculate this is because data quality is more important than quantity.

Effectiveness of Reasonable Navigation Trajectory. YouTube-VLN dataset is collected from real house tour videos and is thus able to extract frames in chronological order to build reasonable navigation trajectories instead of combining multiple randomly chosen images. We use the generated reasonable navigation trajectories and shuffled trajectories to train two agents, respectively. In Table 1, the agent trained with reasonable navigation trajectories (# 3) achieves significantly better performance than the shuffled navigation trajectories variant (# 2), with 1.27% gains on SR under the val unseen split. This suggests that the agent can not well understand and ground the instruction to the trajectory without reasonable navigation paths for learning.

Effectiveness of Pseudo-Labeled Action. The core of the proposed action-aware instruction generator is the pseudo-labeled actions for the instructions. To evaluate the effectiveness of the pseudo-labeled actions, we construct a variant that replaces random action words with the pseudo-labeled actions which are filled in the instructions. The results are shown in Table 1, # 4. Compared to the variant (# 3) that fills the instructions with random action words, this variant boosts the SR metric on both the val unseen (+1.10$\%$) and seen (+0.30$\%$) splits, showing the effectiveness of pseudo-labeled actions. It indicates that with pseudo-labeled actions, the agent can effectively ground the action to the visual observation and recognizes the correct transition from one location to another.

Effectiveness of Trajectory Judgment Pretext Task. To explore the layout knowledge of the YouTube-VLN dataset and equip the agent with layout reasoning ability, we propose a self-supervised trajectory judgment pretext task. In Table 1, the variant with the proposed pretext task improves the performance (# 5 vs. # 4, +2.05% on val seen and +0.60% on val unseen w.r.t. SR). Moreover, given a room type and visual information of the current node as inputs, this variant predicts the relative orientation of nodes for that room type with a 30% increase in accuracy (see supplementary for more details). These substantiate the claim of the importance of the proposed trajectory judgment task, which helps the agent learn the layout reasoning ability.

Effectiveness of Entropy-Based Trajectory Generation Strategy. In order to acquire an informative frame to represent a node, we propose an entropy-based technique as mentioned in Section 3.1. To demonstrate the effectiveness of our strategy, we construct two variants, i.e., one randomly chooses a frame, and one decides the frame by temporal difference [62]. The second variant computes the temporal pixel difference between every two consecutive frames and picks the frames where the peaks are located as the frames to represent the nodes. In Table 2, our entropy-based method significantly outperforms these two variants, increasing the SR on the val unseen split from 64.41% and 65.30% to 66.70%, respectively. We speculate this is because 1) randomly sampling frames can generate consecutive redundant frames or meaningless frames (e.g., a wall takes up most of the frame); 2) the temporal difference method selects a frame that represents the junction of two different rooms, which is ambiguous and thus confuses the agent. In comparison, our entropy-based method is able to find a reliable frame to represent a node and ensure two adjacent nodes belong to different room types.

Results on R2R Dataset. We first compare our Lily agent with current methods under the discriminative setting. In Table 3, compared with VLN-BERT that uses image-caption pairs from the web for pre-training, our Lily agent significantly increases the SR by 10.74% on the val unseen split. This highlights the importance of providing the in-domain indoor data for pre-training. Moreover, compared with Airbert which uses in-domain image-caption pairs from online rental marketplaces, Lily still increases the SR from 73.85% to 79.31% on val seen and from 68.67% to 70.00% on val unseen. We attribute the improvement to our proposed VLN-like YouTube-VLN dataset and trajectory judgment pretext task, which have been thoughtfully evaluated in Section 5.2. When ensembled with the speaker-follower [11], all three methods increase the performance and our Lily agent performs the best.

In Table 3, we evaluate on R2R test split and our Lily ranks first on the VLN challenge leaderboard compared with the results whose manuscripts are publicly available, achieving the highest SR of 79%. As we follow Airbert to use 30 candidate trajectories from EnvDrop [54] and the leaderboard considers that our agent has walked through all these paths, the SPL metric is low for both Lily and Airbert.

Besides, our Lily agent also helps to increase the performance on R2R under the generative setting. We enhance the state-of-the-art DUET [7] method by pre-training the agent using our method as mentioned in Section 4. In Table 6, the agent incorporating Lily achieves 2% and 3% improvements w.r.t. SR on the val unseen split and test unseen split, respectively, compared to DUET. Notably, our method increases the SPL by 2% on the val unseen split, indicating the agent is able to reach the goal more efficiently. We attribute this to the agent’s acquisition of layout prior knowledge via the proposed trajectory judgment task on our YouTube-VLN dataset, enabling it to plan more efficient routes to the goals in new environments.

Results on REVERIE Dataset. Compared with R2R, REVERIE is more challenging as its instructions only describe the destinations without detailed path descriptions. This requires the agent to be equipped with common knowledge about the room layouts and to reason the possible paths that lead to the destinations. In Table 5, we outperform the SOTA agent (i.e., DUET) by 1.13% on SR and increase the SPL from 46.98% to 48.11% on the val unseen split. It is worth noting that our method achieves a higher OSR with a shorter TL, indicating that our agent finds the destination more quickly. We attribute this to the better layout reason ability learned from the large-scale diverse reasonable trajectories in the proposed YouTube-VLN dataset. A similar performance is obtained on the test unseen split, where the Lily agent improves the SR by 1.81% and the SPL by 1.28%. Although our objective is solely navigation, we still achieve comparable performance on grounding metrics and even slightly outstrip DUET on most metrics.

Our intuition is that pre-training on the YouTube-VLN dataset can mitigate the domain gap of training from scratch and is able to achieve excellent performance with only a few training environments. To verify this, we conduct a one-shot learning study, where we fine-tune our model on only one environment of the original training environments. Note that the candidate paths are generated from all of the possible paths from the start viewpoints to all navigable points, instead of from the expert model in EnvDrop [54]. All the candidate paths are the shortest paths in the navigation graphs. To reduce the bias, we randomly select 5 sets from the entire environments and report the average results.

In Table 7, our agent outperforms all the existing pre-training methods. On the val seen split using one-shot fine-tuning, compared to VLN-BERT and AirBERT, our Lily agent achieves 3.60% and 1.43% improvements, respectively. In the val unseen split using one-shot fine-tuning, we achieve 28.43% improvement compared to VLN-BERT. All these results suggest the effectiveness of our method.