Scene-Intuitive Agent for Remote Embodied Visual Grounding

Stage 1(a): Scene Grounding Task. We formulate the task as finding a viewpoint that best matches to a high-level instruction $L$ in a set of candidate viewpoints $\nu_{s}$. $\nu_{s}=\left\{V_{k}|V_{k}\in\nu\right\}\subseteq\nu$. Concretely, we define a mapping function $g_{sg}(,)$ that maps $(L,V_{k})$ to a matching score. The formula is defined as follows,

$\displaystyle V_{k}^{\star}$

$\displaystyle=\mathop{\arg\max}_{V_{k}\in\nu_{s}}g_{sg}(L,ResNet(V_{k}))$

(1)

Stage 1(b): Object Grounding Task. The goal of this task is to identify the best matching object among a set of candidate objects located at a target viewpoint. We denote $V_{T}$ as a target viewpoint and its corresponding annotated bounding boxes set is $B_{T}$. We define another compatibility matching function $g_{og}(,)$ that produce matching scores for all objects with a high-level instruction $L$. Thus, the problem is defined as follows,

$\displaystyle b_{T,i}^{\star}$

$\displaystyle=\mathop{\arg\max}_{b_{T,i}\in B_{T}}g_{og}(L,MRCNN(b_{T,i}))$

(2)

Stage 2: Memory-augmented action decoder. To mitigate the memory problem presented in previous section, a scene memory structure $\boldsymbol{M}_{t}$ is implemented to store the embedded observation and previous action at each time step $t$. The memory is updated by,

	$\displaystyle\tilde{\boldsymbol{v}}_{t}$	$\displaystyle=softmax(\boldsymbol{V}_{t}(\boldsymbol{W}_{1}\boldsymbol{h}_{t-1}))^{T}\boldsymbol{V}_{t}$		(3)
	$\displaystyle\boldsymbol{s}_{t}$	$\displaystyle=FC([\boldsymbol{a}_{t-1},\tilde{\boldsymbol{v}}_{t}]),$
	$\displaystyle\boldsymbol{M}_{t}$	$\displaystyle=Update(\boldsymbol{M}_{t-1},\boldsymbol{s}_{t})$

where $\boldsymbol{s}_{t}$ is current state representation; $\tilde{\boldsymbol{v}}_{t}$ is attentive visual feature;$\boldsymbol{h}_{t-1}$ and $\boldsymbol{a}_{t-1}$ are last time step hidden state and action embedding respectively;$\boldsymbol{W}_{1}\in\mathbb{R}^{2048\times D_{h}}$ is a trainable parameter. $FC$ stands for fully connected layer. The $Update$ operation appends $\boldsymbol{s}_{t}$ to $\boldsymbol{M}_{t}$. $\boldsymbol{V}_{t}\in\mathbb{R}^{36\times 2048},\tilde{\boldsymbol{v}}_{t}\in\mathbb{R}^{1\times 2048},\boldsymbol{a}_{t-1}\in\mathbb{R}^{1\times 3200},\boldsymbol{s}_{t}\in\mathbb{R}^{1\times D_{h}},\boldsymbol{h}_{t}\in\mathbb{R}^{D_{h}\times 1},\boldsymbol{M}_{t}\in\mathbb{R}^{t\times D_{h}}$.

Based on the observation, we believe that a model that could evaluate the alignment between an instruction and a viewpoint is able to localize the target viewpoint. Therefore, to implement this idea, we create a dataset from the REVERIE training set and fine-tune a ViLBERT model on the dataset. Specifically, we adopt a $5$-way multiple choice setting. We eliminate subscript for simplicity concern. Given an instruction $L$, we sample $5$ viewpoints $\left\{V_{1}^{+},V_{2}^{-},V_{3}^{-},V_{4}^{-},V_{5}^{-}\right\}$, out of which only one is aligned to the instruction (or in other words, positive). In detail, we choose the ending viewpoint in the ground-truth training path as $V_{1}^{+}$, the second last viewpoint along the ground-truth path as $V_{2}^{-}$ which is a hard negative sample and random sample $V_{3}^{-},V_{4}^{-}$ from the rest of the viewpoints along the path, and $V_{5}^{-}$ from other path . Then, we run the ViLBERT model on each of the $(L,V_{k})$ pair. As is shown in Fig. 3, the output tokens $CLS$ and $IMG$ encode instruction representation $\boldsymbol{h}_{CLS}$ as well as viewpoint representation $\boldsymbol{h}_{IMG}$ respectively. We define the matching scores as $\boldsymbol{Sc}$ and train the model with cross entropy loss $\mathcal{L}_{sr}$.

	$\displaystyle\boldsymbol{Sc}$	$\displaystyle=\left\{sc_{1},sc_{2},sc_{3},sc_{4},sc_{5}\right\}$		(4)
	$\displaystyle sc_{k}$	$\displaystyle=g_{sg}(L,ResNet(V_{k}))=\boldsymbol{W}_{2}(\boldsymbol{h}_{CLS}^{k}\odot\boldsymbol{h}_{IMG}^{k})$
	$\displaystyle\mathcal{L}_{sr}$	$\displaystyle=CELoss(softmax(\boldsymbol{Sc}),\mathbb{I}(V_{1}^{+}))$

where $\boldsymbol{W}_{2}\in\mathbb{R}^{1\times 1024}$ is a trainable parameter and $\mathbb{I}(.)$ is indicator function. $\boldsymbol{h}_{CLS}^{k}\in\mathbb{R}^{1024\times 1},\boldsymbol{h}_{IMG}^{k}\in\mathbb{R}^{1024\times 1}$ are the encoded language and visual representations of the language and vision encoding streams from our pre-trained ViLBERT model for $k$th $(L,V_{k})$ pair respectively.

First. At each time step $t$, to perceive current scene and instruction, we obtain $\tilde{\boldsymbol{x}}_{t}$ by grounding $L$ with $\boldsymbol{V}_{t}$ through $ViLEncoder$ and then selecting the fused language sequence as output. The formula is defined as follows,

	$\displaystyle\boldsymbol{X}_{t}$	$\displaystyle=ViLEncoder(L,\boldsymbol{V}_{t})$		(5)
		$\displaystyle=BiLSTM(ViLBERT(L,\boldsymbol{V}_{t}))$
	$\displaystyle\tilde{\boldsymbol{x}}_{t}$	$\displaystyle=softmax(\boldsymbol{X}_{t}(\boldsymbol{W}_{3}\boldsymbol{h}_{t-1}))^{T}\boldsymbol{X}_{t}$

where $\boldsymbol{W}_{3}\in\mathbb{R}^{1024\times D_{h}}$ is a trainable parameter and $\boldsymbol{X}_{t}$ is encoded language feature taking current scene $\boldsymbol{V}_{t}$ into consideration. $\boldsymbol{X}_{t}\in\mathbb{R}^{N_{l}\times 1024},\tilde{\boldsymbol{x}}_{t}\in\mathbb{R}^{1\times 1024},\boldsymbol{h}_{t-1}\in\mathbb{R}^{D_{h}\times 1}$.

Second. To decide which navigable direction to go next, we perform object level referring expression comprehension. The object level referring comprehension helps the agent infer whether a navigable view $v_{t,i}$ contains possible target object. In particular, the set of bounding boxes in view $v_{t,i}$ is denoted by $\hat{B}_{t,i}=\left\{b_{t,k}|b_{t,k}\in B_{t},Inside(b_{t,k},v_{t,i})=1\right\}$ where $Inside(,)$ function decides whether $b_{t,k}$ is inside view $v_{t,i}$. $ViLPointer$ is $ViLBERT$ pre-trained on the Object Grounding task and we select the fused bounding boxes features as the output. Then,

	$\displaystyle\boldsymbol{F}_{t,i}$	$\displaystyle=ViLPointer(L,MRCNN(\hat{B}_{t,i}))$		(6)
	$\displaystyle\tilde{\boldsymbol{v}}_{t,i}$	$\displaystyle=g_{top-k}(\boldsymbol{F}_{t,i})$

where $\boldsymbol{F}_{t,i}$ is the set of aligned bounding boxes features at view $v_{t,i}$ and $g_{top-k}(,)$ selects top-$k$ aligned bounding boxes and averages the corresponding aligned bounding boxes features from $\boldsymbol{F}_{t,i}$ to produce view comprehension $\tilde{\boldsymbol{v}}_{t,i}\in\mathbb{R}^{1\times 1024}$.

Third. We define the representation of each navigable view as $\boldsymbol{v}_{t,i}^{\prime}$:

$\displaystyle\boldsymbol{v}_{t,i}^{\prime}$

$\displaystyle=[\boldsymbol{v}_{t,i},(\cos\theta_{t,i},\sin\theta_{t,i},\cos\phi_{t,i},\sin\phi_{t,i}),\tilde{\boldsymbol{v}}_{t,i}]$

(7)

where the agent’s current orientation $(\theta_{t,i},\phi_{t,i})$ represents the angles of heading and elevation and is tiled $32$ times according to  [12]. $(\cos\theta_{t,i},\sin\theta_{t,i},\cos\phi_{t,i},\sin\phi_{t,i})\in\mathbb{R}^{1\times 128}$ and $\boldsymbol{v}^{\prime}_{t,i}\in\mathbb{R}^{1\times 3200}$. The set of navigable view representation is denoted as $\boldsymbol{O}^{\prime}_{t}=\left\{\boldsymbol{v}^{\prime}_{t,i}\right\}_{i=1}^{N_{o}}$. The grounded navigable visual representation $\tilde{\boldsymbol{o}}^{\prime}_{t}$ is represented as follows:

$\displaystyle\tilde{\boldsymbol{o}}^{\prime}_{t}=softmax(g(\boldsymbol{O}^{\prime}_{t})(\boldsymbol{W}_{4}\boldsymbol{h}_{t-1}))^{T}g(\boldsymbol{O}^{\prime}_{t})$

(8)

where $\boldsymbol{W}_{4}\in\mathbb{R}^{1024\times D_{h}}$ is a trainable parameter and $g(,)$ is a number of Fully Connected layers accompanied by ReLU nonlinearities. $\tilde{\boldsymbol{o}}^{\prime}_{t}\in\mathbb{R}^{1\times 1024},\boldsymbol{O}^{\prime}_{t}\in\mathbb{R}^{N_{o}\times 3200}$.

Fourth. The new context hidden state $\boldsymbol{h}_{t}$ is updated by a LSTM layer taking as input the grounded text $\tilde{\boldsymbol{x}}_{t}$ and navigable view features $\tilde{\boldsymbol{o}}^{\prime}_{t}$ as well as the current state representation feature $\boldsymbol{s}_{t}^{a}$.

$\displaystyle(\boldsymbol{h}_{t},\boldsymbol{c}_{t})=LSTM([\tilde{\boldsymbol{x}}_{t},\tilde{\boldsymbol{o}}^{\prime}_{t},\boldsymbol{s}_{t}^{a}],(\boldsymbol{h}_{t-1},\boldsymbol{c}_{t-1}))$

(9)

where $\boldsymbol{s}_{t}^{a}$ is memory augmented current state representation and is defined as,

	$\displaystyle\boldsymbol{M}_{t}^{a}$	$\displaystyle=[Transformer(\boldsymbol{M}_{t},\boldsymbol{M}_{t})]_{\times N_{mem}}$		(10)
	$\displaystyle\boldsymbol{s}_{t}^{a}$	$\displaystyle=[Transformer(\boldsymbol{s}_{t},\boldsymbol{M}_{t}^{a})]_{\times N_{state}}$

where $N_{mem}$ and $N_{state}$ are number of memory transformer blocks used and number of state transformer blocks used respectively. $\boldsymbol{s}_{t}^{a}\in\mathbb{R}^{1\times D_{h}},\boldsymbol{M}_{t}^{a}\in\mathbb{R}^{t\times D_{h}}$. $Transformer$ is the standard version Transformer block from  [42].

Finally. The action logit $\boldsymbol{l}_{t}$ is computed in an attentive manner.

$\displaystyle l_{t,i}=g(\boldsymbol{O}^{\prime}_{t,i})(\boldsymbol{W}_{5}[\boldsymbol{h}_{t},\tilde{\boldsymbol{x}}_{t}])$

(11)

where $\boldsymbol{W}_{5}\in\mathbb{R}^{1024\times(1024+D_{h})}$ is a trainable parameter and $\boldsymbol{l}_{t}\in\mathbb{R}^{N_{o}\times 1}$. In training stage, $a_{t}=Categorical(\boldsymbol{l}_{t})$ is selected based on categorical policy and in inference stage, it is selected by $a_{t}=\arg max(\boldsymbol{l}_{t})$. Action embedding is selected based on $\boldsymbol{a}_{t}=\boldsymbol{O}^{\prime}_{t}[a_{t}]$.

Performance Gain. To answer question $(a)$, we perform experiments $1$, $2$, $3$ and $6$ as is shown in Table 1. All agents are trained under $\mathcal{L}_{ce}$ and $\mathcal{L}_{pm}$ with $\alpha$ and $\beta$ both set to $0.5$. It is clear that the agent’s overall performance is incrementally improved by changing the language encoder from the simple $L_{enc}$ to our proposed $ViL_{enc}$, which proves our analysis that previous language encoder does not well capture the semantics of high-level instructions. The experimental results of $3$ and $6$ clearly suggests that the BERT-based structure is not the root cause of our performance gain and our proposed Scene Grounding task significantly increase the RG SPL metric to $33.9\%$ on Val Seen and $7.31\%$ on Val Unseen, even higher than the strong baseline in experiment $2$.

Component Effectiveness. To answer question $(b)$, based on the statistics from Table 1, we train six models in experiments from $3$ to $8$ and ablate the proposed component one by one to demonstrate the effectiveness. For fair comparison, we follow the settings of  [36]. All agents are trained under $\mathcal{L}_{ce}$ and $\mathcal{L}_{pm}$ with $\alpha$ and $\beta$ both set to $0.5$. We start from the baseline experiment $3$ and replace each component by our proposed ones. Specifically, in experiments $3$ and $6$, the proposed $ViLEncoder$ improves the RG SPL (and SPL) by a large margin, $11.98\%$ (and $21.52\%$) higher in Val Seen and $2.47\%$ (and $3.77\%$) higher in Val Unseen than the baseline respectively, which proves that the Scene Grounding task is effective; in experiments $3$ and $4$, our pointer $ViLPointer$ outperforms the MattNet counterpart by shortening the length of the search trajectory while maintaining a high RG SPL (and SPL), which demonstrates the effectiveness of the Object Grounding task; in experiments $3$ and $5$, the results show that the overall search trajectory of our action policy is longer than that of the baseline while our action policy achieves higher RGS and Navigation Success Rate, which demonstrates that the memory structure in our policy guides the agent to the correct target location at the cost of long trajectory; in experiments $7$ and $8$, we demonstrate that by integrating all our proposed methods, our agent improves previous SOTA in terms of RG SPL by $10.76\%$ on Val Seen and $5.6\%$ on Val Unseen.

Memory Blocks. To answer question $(c)$, we train five models with different $N_{mem}$ and $N_{state}$ values. In these experiments, we train the agents with $\mathcal{L}_{ce}$, $\mathcal{L}_{pm}$ and $\mathcal{L}_{RL}$ and $\alpha$, $\beta$ and $\gamma$ set to $1.0$. In general, according to the experiments from $9$ to $13$ in Table 1, all pairs of $(N_{mem},N_{state})$ exhibit superior performance compared to previous SOTA method in experiment $1$ and the strong BERT baseline model in experiment $2$. Moreover, the best performance model is achieved by setting $(N_{mem},N_{state})$ to $(3,3)$ in these five models, which suggests that using small values of $(N_{mem},N_{state})$ limits the agent’s memorization ability and using large values of $(N_{mem},N_{state})$ enables the agent to achieve good performance on Val Unseen while maintains good performance on Val Seen.

Logit Fusion. To answer question $(d)$, we report two accuracies to verify the effectiveness of $g_{og}$ and $g_{sg}$. In the Encoder Task of Table 3, given ground-truth path, our proposed $ViLBERT$ model achieves competitive performance on both Val Seen and Val Unseen, demonstrating the strong ability of $g_{sg}$ to identify a target viewpoint. In the Pointer Task of Table 3, the performance of $ViLPointer$-vp-based is significantly higher than previous image-based pointers because it is able to capture cross-image objects relationships, suggesting that $g_{og}$ has the ability to find the target location if the target object exists. According to experiments from $14$ to $17$, where the agents are trained with $\mathcal{L}_{ce}$, $\mathcal{L}_{pm}$ and $\mathcal{L}_{RL}$ and $\alpha$, $\beta$ and $\gamma$ set to $1.0$, summing $\boldsymbol{l}_{\tau}$, $g_{og}^{\tau}$, and $g_{sg}^{\tau}$ shortens the search trajectory and maintains a high RGS(Navigation Success Rate) and RG SPL(SPL). The motivation behind the summing strategy is to use model ensemble to reduce bias when searching for target locations considering the fact that the agent has no prior knowledge of the surrounding environments and the guidance of the high-level instructions is weak.

Then, we compare our final model with previous SOTA models in Table 2. As is clearly shown in Table 2, our model outperforms all previous models by a large margin. Specifically, in terms of SPL, our agent increases previous SOTA by $11.58\%$ on Val Seen, $9.09\%$ on Val Unseen and $3.24\%$ on Test respectively; for RG SPL, our agent increase previous SOTA by $12.99\%$ on Val Seen, $6.89\%$ on Val Unseen and $3.12\%$ on Test. The overall improvements indicate that our proposed scene-intuitive agent not only navigates better but also localizes target objects more accurately.