ADVERT: An Adaptive and Data-Driven Attention Enhancement Mechanism for Phishing Prevention

Throughout the paper, we use subscripts to index time and stages. Calligraphic letter $\mathcal{S}$ defines a set and $|\mathcal{S}|$ represents its cardinality. The indicator function $\mathbf{1}_{\{A\}}$ takes value $1$ if condition $A$ is true and value $0$ if $A$ is false. The rest of the paper is organized as follows. The related works are presented in Section II. We elaborate on the two feedback loops of Fig. 1 in Section III and IV, respectively. Section V presents a case study of ADVERT for email vetting and phishing recognition. Section VI discusses the limitations, and Section VII concludes the paper.

Phishing is the act of masquerading as a legitimate entity to serve malware or steal credentials. The authors in [6] have identified three human vulnerabilities that make humans the unwitting victims of phishing.

• •

  Lack of knowledge for computer system security; e.g., www.ebay-members-security.com does not belong to www.ebay.com.

• •

  Inadequacy to identify visual deception; e.g., the phishing email can contain an image of a legitimate hyperlink, but the image itself serves as a hyperlink to a malicious site. A human cannot identify the deception by merely looking at it.

• •

  Lack of attention (e.g., careless users fail to notice the phishing indicators, including spelling errors and grammar mistakes) and inattentional blindness (e.g., users focusing on the main content fail to perceive unloaded logos in a phishing email [7]).

Many works have attempted to mitigate the above three human vulnerabilities to prevent phishing attacks. First, security education and anti-phishing training, e.g., role-playing phishing simulation games [8] and fake phishing attacks [9], have been used to compensate for the user’s lack of security knowledge and increase users’ security awareness. Second, detection techniques based on visual similarities [10] and machine learning [11] have been applied to help users identify visual deception. Modern web browsers and email clients also provide security indicators (e.g., the protocol used, the domain name, and the SSL/TLS certificate) to assist users in decision-making [12]. Third, passive warnings (i.e., do not block the content-area) and active warnings (i.e., prohibits the user from viewing the content-data) have been developed empirically to draw users’ attention and prevent them from falling victim to phishing [13, 11]. Our work lays out a foundation to compensate for the third human vulnerability of inattention systematically and quantitatively.

Adversarial cyber deception has been a long-standing problem. It is easy for an attacker to deceive yet much more difficult for regular users to identify the deception given the universal human vulnerabilities. Previous works have mainly focused on human solutions (e.g., security training [1]) or technical solutions (e.g., defensive deception technologies [14, 15, 16]), to deter, detect, and respond to deceptive attacks. This work focuses on designing a human-technical solution through eye-tracking data, visual aids, and learning techniques to counteract adversarial cyber deception.

Biosensors, including eye trackers and electroencephalogram (EEG) devices, provide a window into an analytical understanding of human perception and cognition to enhance security and privacy [17]. In particular, researches have investigated the users’ gaze behaviors and attention when reading Uniform Resource Locators (URLs) [18], phishing webs [19], and phishing emails [5, 20, 21]. These works illustrate the users’ visual processing of phishing contents [18, 19, 22, 20] and the effects of visual aids [21]. The authors in [19] further establish correlations between eye movements and phishing identification to estimate the likelihood that users may fall victim to phishing attacks. Compared to these works that analyze human perception, we use eye-tracking data to design visual aids and modify the human perception process for better security decisions. Moreover, we use biometric data at different granularities. Compared to previous works that exploit the statistics of the biometric data (e.g., the number of fixations and gaze duration distributions), we use the dynamic transitions of the eye-tracking data to extract attention metrics for corrective measures in real-time.

Human plays significant roles in cybersecurity. It is challenging to model, quantify, and affect human behaviors and their mental processes such as reasoning, perception, and cognition. Therefore, various modeling and learning approaches are developed to mitigate human vulnerabilities in cyberspace, as shown in the following two paragraphs, respectively.

The authors in [23, 24] use Signaling Detection Theory (SDT) to quantify phishing susceptibility and prioritize behavioral interventions for reducing phishing risk, respectively. Adopting SDT, they treat the phishing risk management as a vigilance task, where individuals monitor their environment to distinguish signals (i.e., phishing emails) from noises (i.e., legitimate emails). Their approaches investigate phishing on a detailed level based on varying factors, including task, individual, and environmental ones. We adopt a system-level characterization, where system-scientific tools such as feedback, Reinforcement Learning (RL), and BO are used to adapt to these varying factors.

Due to the modeling challenges and the unpredictability, RL [25] has been used to characterize and mitigate human vulnerabilities, including bounded rationality [26], prospect theory [27], incompliance [3], and bounded attention [28, 29]. Using RL to detect, evaluate, and compensate for risks induced by human vulnerabilities is still in its infancy, but it is a promising direction as RL provides a quantitative and adaptive solution.

As illustrated by Step $2$ in Fig. 1, we establish the following transition model based on the AoI to which the user’s gaze location belongs at different times. We define $\mathcal{S}:=\{s^{i}\}_{i\in\mathcal{I}}\cup\{s^{ua},s^{da}\}$ as the set of $I+2$ Visual States (VSs), where $s^{i}$ represents the $i$-th AoI; $s^{ua}$ represents the uninformative area; and $s^{da}$ represents the distraction area. We provide an example transition map of these VSs in Fig. 3.

The links represent the potential shifts of the gaze locations during the email reading process; e.g., a user can shift his focus from the title to the main content or the distraction area. We omit most links for illustration purposes; e.g., it is also possible for a user to regain attention to the AoIs from distraction or inadvertence.

We denote $s_{t}\in\allowbreak\mathcal{S}$ as the VS of user $m\in\mathcal{M}$ vetting email $n\in\mathcal{N}$ at time $t\in\allowbreak[0,T_{m}^{n}]$. In this work, we do not distinguish among human users concerning their attention processes while they read different emails. Then, each user’s gaze path during the interval $[0,T_{m}^{n}]$ can be characterized as the same stochastic process $[s_{t}]_{t\in[0,T_{m}^{n}]}$. The stochastic transition of the VSs divides the entire time interval $[0,T_{m}^{n}]$ into different transition stages. We visualize an exemplary VS transition trajectory $[s_{t}]_{t\in[0,T_{m}^{n}]}$ in Fig. 4 under $I=4$ AoIs and $T_{m}^{n}=50$ seconds. As denoted by the colored squares, $40$ VSs arrive in sequence, which results in $40$ discrete transition stages.

Propel visual aids can help guide and sustain the users’ attention. Previous works have proposed different classes of visual aids to enhance phishing recognition, including highlights of contents [21, 30], warnings of suspicious hyperlinks and attachments [13, 31], and anti-phishing educational messages [32]. These potential classes of visual aids construct the visual-aid library denoted as a finite set $\mathcal{A}$.

As illustrated by Step $6$ in Fig. 1, different visual aids can affect the users’ visual behaviors. The influence, however, can be beneficial (e.g., timely highlights prevent users from mind-wandering) or detrimental (e.g., extensive highlights make humans weary and less attentive to the AoIs). The effectiveness of visual aids for preventing phishing may not be straightforward, especially under different environmental (e.g., security indicator designs) and human factors (e.g., users’ security knowledge and prior trust) [12]. In this paper, we focus on adapting visual aids to the human visual attention. We apply RL to learn the dynamic design of visual aids based on the real-time evaluation of the user’s attention status detailed in Section III-C.

The sequence of adaptive visual aids is generated with a period of length $T^{pl}$, and we refer to the time interval between every two visual aids as the generation stage indexed by $k\in\allowbreak\mathcal{K}_{m}^{n}:=\{1,2,\cdots,\allowbreak K_{m}^{n}\}$, where $K_{m}^{n}$ is the maximum generation stage during $[0,T_{m}^{n}]$; i.e., $K_{m}^{n}T^{pl}\leq T_{m}^{n}$ and $(K_{m}^{n}+1)T^{pl}\geq T_{m}^{n}$. Then, we denote $a_{k}\in\mathcal{A}$ as the visual aid at the $k$-th generation stage. Fig. 4 illustrates how visual aids affect the transition of VSs in $K_{m}^{n}=3$ generation stages divided by the two vertical dashed lines. During the second generation stage, an improper visual aid leads to more frequent transitions to the distraction area and also a longer sojourn time at the VS $s^{da}$. On the contrary, the proper visual aids during the first and the third generation stages engage the users and extend their attention spans, i.e., the amount of time spent on AoIs before a transition to $s^{da}$ or $s^{ua}$.

From the VS transition trajectory (e.g., Fig. 4), we aim to construct the Attention State (AS) used as the feedback value for the adaptive visual-aid design. We define $\mathcal{X}$ as the set of all possible attention states. Previous works (e.g., [22, 20]) have defined attention metrics based on the AoIs, including the proportion of time spent on each AOI, gaze duration means, fixation count, and average duration. Compared to these detailed-level metrics extracted directly from raw eye-gaze data, we propose the following system-level metric of attention level based on the VS transition history as will be shown in Section III-C2. Such system-level metric serves as sufficient statistics to effectively characterize the attention status. Moreover, it preserves the users’ privacy because the raw data of gaze locations can reveal sensitive information about their biometric identities, including gender, age, and ethnicity [33, 34].

To this end, we assign scores to each VS in Section III-C1 to evaluate the user’s attention (e.g., gaze at AoIs) and inattention (e.g., gaze at uninformative and distraction areas). The scores can be determined manually based on the expert recommendation and empirical studies (e.g., [22]), or based on other biometric data (e.g., the pupil sizes in Fig. 8). Moreover, we can apply BO for further fine-tuning of these scores as shown in Section IV-B.

In Section III-D, We elaborate on the adaptive learning block (i.e., Step $5$ in Fig. 1). Since the inspection time of a user reading one email is not sufficiently long, we consolidate a group of email inspection data to learn the optimal visual-aid generation policy over a population.

The QAAL $\bar{v}^{qu}_{k}\in\mathcal{X}$ represents the attention state at the generation stage $k\in\mathcal{K}_{m}^{n}$. Since the goal is to enhance the user’s attention represented by the CAL, the reward function $R:\mathcal{X}\times\mathcal{A}\mapsto\mathbb{R}$ should be monotone concerning the value of $\bar{v}^{qu}_{k}$, e.g., $R(\bar{v}^{qu}_{k},a_{k}):=\bar{v}^{qu}_{k},\forall a_{k}\in\mathcal{A}$. In this work, we assume that each visual aid $a_{k}\in\mathcal{A}$ exerts the same statistical effect on the attention process regardless of different users and emails. Thus, we can consolidate the data set of $\bar{M}\in\{1,\cdots,M\}$ users and $\bar{N}\in\{1,\cdots,N\}$ emails²²2When sufficiently large data sets are available, we can carefully choose these $\bar{M}$ users to share similar attributes (e.g., ages, sexes, races, etc.) and these $\bar{N}$ emails to belongs to the same categories (e.g., business or personal emails). to learn the optimal visual-aid generation policy $\sigma\in\Sigma:\mathcal{X}\mapsto\mathcal{A}$ in a total of $\bar{K}:=\sum_{m=1}^{\bar{M}}\sum_{n=1}^{\bar{N}}K_{m}^{n}$ stages. With a given discounted factor $\beta\in(0,1)$, the expected long-term objective can be represented as $\max_{\sigma\in\Sigma}\mathbb{E}[\sum_{k=1}^{\bar{K}}(\beta)^{k}\cdot R(\bar{v}^{qu}_{k},\sigma(\bar{v}^{qu}_{k}))]$.

The $Q$-table $[Q_{k}(\bar{v}^{qu}_{k},a_{k})]_{\bar{v}^{qu}_{k}\in\mathcal{X},a_{k}\in\mathcal{A}}$ represents the user’s attention pattern at generation stage $k\in\mathcal{\bar{K}}:=\{1,\cdots,\bar{K}\}$, i.e., the estimated payoff of applying visual aid $a_{k}\in\mathcal{A}$ when the attention state is $\bar{v}^{qu}_{k}\in\mathcal{X}$. Let the sequence of learning rate $\gamma_{k}(\bar{v}^{qu}_{k},a_{k})$ satisfy $\sum_{k=0}^{\infty}\gamma_{k}(\bar{v}^{qu}_{k},a_{k})=\infty$ and $\sum_{k=0}^{\infty}(\gamma_{k}(\bar{v}^{qu}_{k},a_{k}))^{2}<\infty$ for all $\bar{v}^{qu}_{k}\in\mathcal{X},a_{k}\in\mathcal{A}$. Then, we can update the attention pattern at each generation stage $k\in\mathcal{\bar{K}}$ as follows, i.e.,

where the visual-aid generation policy $\sigma_{k}(\bar{v}^{qu}_{k})$ at generation stage $k\in\mathcal{\bar{K}}$ is an $\epsilon_{k}$-greedy policy; i.e., with probability $\epsilon_{k}\in[0,1]$, the visual aid $a_{k}$ is selected randomly from $\mathcal{A}$ and with probability $1-\epsilon_{k}$, the optimal visual aid $a^{*}_{k}\in\textrm{arg}\max_{a\in\mathcal{A}}Q_{k}(\bar{v}^{qu}_{k},a)$ is implemented. To obtain a convergent visual-aid generation policy, the value of $\epsilon_{k}$ gradually decreases from $1$ to $0$.

As illustrated by Step $7$ in Fig. 1, we provide a metric to evaluate the outcome of the users’ phishing identification under a given hyperparameter $\theta\in\Theta^{d}$. After vetting email $n\in\{1,\cdots,\bar{N}\}$, the user $m\in\{1,\cdots,\bar{M}\}$ judges the email to be phishing or legitimate. The binary variable $z_{m}^{n}(\theta)\in\{z^{co},z^{wr}\}$ represents whether the judgment is correct (denoted by $z^{co}$) or not (denoted by $z^{wr}$). We can reshape the two-dimension index $(m,n)$ as a one-dimension index $\hat{n}$ and rewrite $z_{m}^{n}(\theta)$ as $z_{\hat{n}}(\theta)$. Once these users have judged in total of $N^{bo}$ emails, we define the following metric $c^{ac}\in\mathcal{C}:\Theta^{d}\mapsto[0,1]$ to evaluate the accuracy of phishing recognition, i.e.,

The goal is to find the optimal hyperparameter $\theta^{*}\in\Theta^{d}$ to maximize the accuracy of phishing identification, i.e., $\theta^{*}\in\textrm{arg}\max_{\theta\in\Theta^{d}}c^{ac}(\theta)$. However, we cannot know the value of $c^{ac}(\theta)$ for a $\theta\in\Theta^{d}$ a priori until we implement this hyperparameter $\theta$ in the attention enhancement mechanism. The implemented hyperparameter affects the adaptive visual-aid generation that changes the user’s attention and the anti-phishing performance metric $c^{ac}(\theta)$. Since the experimental evaluation at a given $\theta\in\Theta^{d}$ is time-consuming, we present an algorithm in Section IV-B to determine how to choose and update the hyperparameter to maximize the detection accuracy.

We illustrate the meta-adaptation (i.e., Step $8$ in Fig. 1) in Section IV-B. As illustrated in Fig. 6, we refer to the duration of every $N^{bo}$ security decisions as a tuning stage. Consider a time and budget limit that restricts us to conduct $L$ tuning stages in total. We denote $\theta_{l}$ as the hyperparameter at the $l$-th tuning stage where $l\in\mathcal{L}:=\{1,2,\cdots,L\}$. Since each user’s email inspection time is different, each tuning stage can contain different numbers of generation stages.

To find the optimal hyperparameter $\theta^{*}\in\Theta^{d}$ within $L$ tuning stages is challenging. The empirical methods (e.g., a naive grid search or a random search over $\Theta^{d}\subset\mathbb{R}^{d}$) become inefficient when $d>1$. BO [38] provides a systematic way to update the hyperparameter and balance between exploration and exploitation. BO consists of a Bayesian statistical model of the objective function $c^{ac}\in\mathcal{C}$ and an acquisition function for deciding the hyperparameter to implement at the next tuning stage. The statistical model of $c^{ac}\in\mathcal{C}$ is a Gaussian process $\mathcal{N}(\mu^{0},\Sigma^{0})$ with a mean function $\mu^{0}(\theta)=\bar{\mu}^{0}$ and covariance function or kernel $\Sigma^{0}(\theta,\bar{\theta})=\lambda^{0}\cdot\exp(\sum_{i=1}^{d}\lambda^{i}(\theta^{i}-\bar{\theta}^{i})^{2})$ for all $\theta,\bar{\theta}\in\Theta^{d}$, where $\bar{\mu}^{0}$, $\lambda^{0}$ and $\lambda^{i},i\in\{1,2,\cdots,d\}$, are parameters of the kernel. The kernel $\Sigma^{0}$ is required to be positive semi-definite and has the property that the points closer in the input space are more strongly correlated. For any $l\in\mathcal{L}$, we define three shorthand notations $\mu^{0}(\theta_{1:l}):=[\mu^{0}(\theta_{1}),\cdots,\mu^{0}(\theta_{l})]$, $c^{ac}(\theta_{1:l}):=[c^{ac}(\theta_{1}),\cdots,c^{ac}(\theta_{l})]$, and

Then, the evaluation vector of $l\in\mathcal{L}$ elements is assumed to be multivariate Gaussian distributed, i.e., $c^{ac}(\theta_{1:l})\sim\mathcal{N}(\mu^{0}(\theta_{1:l}),\Sigma^{0}(\theta_{1:l},\theta_{1:l}))$. Conditioned on the values of $\theta_{1:l}$, we can infer the value of $c^{ac}(\theta)$ at any other $\theta\in\Theta\setminus\{\theta_{{l}^{\prime}}\}_{{l}^{\prime}\in\{1,\cdots,l\}}$ by Bayesian rule, i.e.,

where $\mu^{n}(\theta)=\Sigma^{0}(\theta,\theta_{1:l})\cdot\Sigma^{0}(\theta_{1:l},\theta_{1:l})^{-1}\cdot(c^{ac}(\theta_{1:l})-\mu^{0}(\theta_{1:l}))+\mu^{0}(\theta)$ and $(\Sigma^{n}(\theta))^{2}=\Sigma^{0}(\theta,\theta)-\Sigma^{0}(\theta,\theta_{1:l})\cdot\Sigma^{0}(\theta,\theta_{1:l})^{-1}\cdot\Sigma^{0}(\theta_{1:l},\theta)$.

We adopt expected improvement as the acquisition function. Define $c^{*}_{l}:=\max_{l^{\prime}\in\{1,\cdots,l\}}c^{ac}(\theta_{l^{\prime}})$ as the optimal evaluation among the first $l$ evaluations and a shorthand notation $(c^{ac}(\theta)-c^{*}_{l})^{+}:=\max\{c^{ac}(\theta)-c^{*}_{l},0\}$. For any $l\in\mathcal{L}$, we define $\mathbb{E}_{l}[\cdot]:=\mathbb{E}[\cdot|c^{ac}(\theta_{1:l})]$ as the expectation taken under the posterior distribution of $c^{ac}(\theta)$ conditioned on the values of $l$ evaluations $c^{ac}(\theta_{1:l})$. Then, the expected improvement is $\text{EI}_{l}(\theta):=\mathbb{E}_{l}[(c^{ac}(\theta)-c^{*}_{l})^{+}]$. The hyperparameter at the next tuning stage is chosen to maximize the expected improvement at the current stage, i.e,

The expected improvement can be evaluated in a closed form, and (5) can be computed inexpensively by gradient methods [38].

At the first $L^{0}\in\{1,2,\cdots,L\}$ tuning stages, we choose the hyperparameter $\theta_{l},l\in\{1,2,\cdots,L^{0}\}$, uniformly from $\Theta^{d}$. We can use the evaluation results $c^{ac}(\theta_{l}),l\in\{1,2,\cdots,L^{0}\}$, to determine the parameters $\bar{\mu}^{0},\lambda^{0}$, and $\lambda^{i},i\in\{1,2,\cdots,d\}$, by Maximum Likelihood Estimation (MLE); i.e., we determine the values of these parameters so that they maximize the likelihood of observing the vector $[c^{ac}(\theta_{1:L^{0}})]$. For the remaining $L-L^{0}$ tuning stages, we choose $\theta_{l},l\in\{L^{0},L^{0}+1,\cdots,L\}$, in sequence as summarized in Algorithm 1.

The data set involves $M=160$ undergraduate students ($n_{\text{White}}=27$, $n_{\text{Black}}=19$, $n_{\text{Asian}}=64$, $n_{\text{Hispanic/Latinx}}=17$, $n_{\text{other}}=33$) who are asked to vet $N=12$ different emails (e.g., the email of NYU friends network in Fig. 2) separately and then give a rating of how likely they would take actions solicited in the emails (e.g., maintain membership in Fig. 2). When presented to different participants, each email is described as either posing a cyber threat or risk-free legitimate opportunities to investigate how the above description affects the participants’ phishing recognition.

While the participants vet the emails, the Tobii Pro T60XL eye-tracking monitor records their eye locations on a $1920\times 1200$ resolution screen and the current pupil diameters of both eyes with a sampling rate of $60\text{Hz}$. Fig. 7 illustrates the time-expanded eye-gaze trajectory of a participant vetting the sample email in Fig. 2. The $z$-coordinate of a 3D point $(x,y,z)$ represents the time when the participant gazes at the pixel $(x,y)$ in the email area. The participant’s eye gaze locations move progressively from the points in warmer color to the ones in cooler color. Fig. 7 illustrates the zigzag pattern of the participant’s eye-gaze trajectory; i.e., the participant reads emails from left to right and top to bottom. The participant starts with the title, spends the majority of time on the main content, and glances at other AoIs (e.g., the links and the signatures). There is also a small chance of revisiting the email content and looking outside the email area.

Fig. 8 illustrates the participant’s pupil sizes of left and right eyes in red and blue, respectively, concerning the same trial of the data set to generate Fig. 7. At different times, the average of the pupil diameters (resp. gaze locations) of the right and left eyes represent the pupil size (resp. gaze location). Following Section III-A, we obtain the $15$ VSs illustrated by the grey squares in Fig. 8 based on the gaze locations of the email pixels in Fig. 7. Since the covert eye-tracking system does not require head-mounted equipment or chinrests, the tracking can occur without the participants’ awareness. We refer the reader to the supplement materials of [5] for the survey data and the details of the experimental procedure³³3The processed data used in this manuscript, including the temporal transitions of AoIs and the pupil sizes, is available at https://osf.io/4y32d/. The raw eye-tracking data in the format of videos are available upon request..

Based on the benchmark attention score in Section V-A1, Fig. 11 illustrates the CAL of the VS transition trajectory shown in Fig. 10. We consider $X=2$ attention states $\mathcal{X}=\{x^{H},x^{L}\}$ with the attentive state $x^{H}$ and the inattentive state $x^{L}$. Define $X^{at}\in\mathbb{R}$ as the attention threshold. If the AAL at generation stage $k\in\mathcal{K}_{m}^{n}$ is higher (resp. lower) than the attention threshold, i.e., $\bar{v}_{k}\geq X^{at}$ (resp. $\bar{v}_{k}\leq X^{at}$), then the attention state $x_{k}\in\mathcal{X}$ at generation $k$ is the attentive state $x^{H}$ (resp. inattentive state $x^{L}$).

Fig. 12 further illustrates the impact of visual aids $a^{N}$ and $a^{Y}$ on the AAL in red and blue, respectively. The figure demonstrates that $a^{Y}$ can increase the mean of AAL yet increase its variance.

In Algorithm 2, we present the Q-learning process for participant $m\in\mathcal{M}$ who reads email $n\in\mathcal{N}$ for $T_{m}^{n}$ seconds. Define $\eta_{k}(x,a)$ as the total number of visits to attention state $x\in\mathcal{X}$ and visual aid $a\in\mathcal{A}$ up to generation stage $k$. Then, we choose the learning rate $\gamma_{k}(x_{k},a_{k})=\frac{\eta^{0}}{\eta_{k}(x,a)-1+\eta^{0}}$ for all $x_{k}\in\mathcal{X},a_{k}\in\mathcal{A}$ to guarantee the asymptotic convergence, where $\eta^{0}\in(0,\infty)$ is a constant parameter.

Based on the benchmark data set of $M=160$ participants who inspect $N=12$ emails in Section V-A, the inspection time $T_{m}^{n},\forall m\in\mathbf{M},n\in\mathcal{N}$, follows a Burr distribution; i.e., its cumulative distribution function is described by $F^{Burr}(t\mid\rho_{1},\rho_{2},\rho_{3})=1-\frac{1}{\left(1+\left({t}/{\rho_{1}}\right)^{\rho_{2}}\right)^{\rho_{3}}}$ with the scale parameter $\rho_{1}=11.7$, and the shape parameters $\rho_{2}=62.5,\rho_{3}=0.04$. The average inspection time of $M\times N$ samples is $18.7$ seconds. During $T_{m}^{n}$ seconds of the email vetting process, the eye-tracking device records the participant’s gaze locations, which leads to the VS transition trajectory. In Algorithm 2, we simulate the human email-reading process through the synthetic VS transition trajectory generated by the sufficient statistics $P(a_{t})$ and $\phi(a_{t})$. Every $T^{pl}$ seconds, ADVERT updates the Q-matrix and the visual aid based on (2).

Following Section III-D, we develop Algorithm 3 to illustrate the entire attention enhancement loop that involves the consolidation of the data set from $\bar{M}\in\{1,\cdots,M\}$ participants and $\bar{N}\in\{1,\cdots,N\}$ emails. After the participant $m\in\{1,\cdots,\bar{M}\}$ finishes reading the email $n\in\{1,\cdots,\bar{N}\}$, Algorithm 2 returns the Q-matrix and the attention state at the final generation stage $K_{m}^{n}$. These results then serve as the inputs for the next email inspection until $N^{bo}$ emails have been inspected.

Based on Algorithm 3, we plot the entire Q-learning updates with $N^{bo}=100$ emails in Fig. 13 that contains a total of $609$ generations stages. The learning results show that the visual aid $a^{Y}$ outweighs $a^{N}$ for both attention states and should be persistently applied under the current setting.

After we obtain a participant’s synthetic response (characterized by his VS transition trajectory) under the adaptive visual aids, we apply a pre-trained neural network to estimate whether the participant has made a correct judgment as shown in line $24$ of Algorithm 2. In Section V-C1, we elaborate on the training process of the neural network based on the data set used in Section V-A. We apply BO in Algorithm 1 to evaluate the accuracy metric $c^{ac}\in\mathcal{C}$, as illustrated by Step $8$ in Fig. 1. In Section V-C2, we show the results.