Expanding Targets in Virtual Reality Environments: A Fitts’ Law Study

Rongkai Shi e-mail: [email protected] 0000-0001-8845-6034 Yushi Wei e-mail: [email protected] 0000-0002-6003-0557 Yue Li e-mail: [email protected] 0000-0003-3728-218X Lingyun Yu e-mail: [email protected] 0000-0002-3152-2587 Hai-Ning Liang Corresponding author; e-mail: [email protected] 0000-0003-3600-8955 Department of Computing, School of Advanced Technology
Xi’an Jiaotong-Liverpool University

Abstract

Target pointing selection is a fundamental task. According to Fitts’ law, users need more time to select targets with smaller sizes. Expanding the target to a larger size is a practical approach that can facilitate pointing selection. It has been well-examined and -deployed in 2D user interfaces. However, limited research has investigated target expansion methods using an immersive virtual reality (VR) head-mounted display (HMD). In this work, we aimed to fill this gap by conducting a user study using ISO 9241-411 multi-directional pointing task to examine the effect of target expansion on target selection performance in VR HMD. Based on our results, we found that compared to not expanding the target, expanding the target width by 1.5 and 2.5 times during the movement can significantly reduce the selection time. We hope that the design and results derived from the study can help frame future work.

\CCScatlist\CCScatTwelve

Human-centered computingHuman computer interaction (HCI)Interaction paradigmsVirtual reality; \CCScatTwelveHuman-centered computingHuman computer interaction (HCI)Empirical studies in HCI

1 Introduction

Target pointing selection is a fundamental interaction task in all user interfaces (UIs). Expanding the target to a larger size is a practical approach that can facilitate pointing selection. In desktops, tablets, and other 2D UIs, users commonly need to point to the interface widgets, such as buttons and icons. Expanding the size of the target can improve the target selection performance. McGuffin and Balakrishnan [11, 12] showed empirical evidence that target expansion during users reaching the cursor to the target could reduce the pointing time. Except for the linear expansion, researchers also developed non-linear magnification effects, such as fisheye views [8, 9] or Voronoi-based expansions [6, 7], to support the target selection in 2D UIs.

While in 3D UIs, users are no longer pointing to 2D interface elements but to 3D objects. However, limited research has investigated target expansion methods in 3D UIs. Early in 2007, Vanacken [15] presented 3D bubble cursor to support target selection in dense 3D environments. Once the 3D bubble cursor captured a spherical target, a second, expanding sphere around the captured target would appear to be contained by the cursor. Argelaguet and Andujar [1] proposed two target expansion strategies for 3D selection tasks in CAVE. Both research studies focused on expanding targets to support selection in dense and occluded virtual environments.

With recent advancements, virtual reality (VR) head-mounted displays (HMDs) have become powerful and popular. VR HMDs offer users an immersive experience that is largely greater than other devices. The interaction in VR HMDs also differs from other devices. Users normally control the ray emitted from the 6-DOF controller (i.e., ray-casting [3]) to point to the target objects in virtual environments provided by VR HMDs. Yu et al. [18] tested Expanding Target technique in VR HMDs and found that the selection performance was improved by expanding the size of the pointing object. However, to the best of our knowledge, no research has been conducted to investigate the effect of target expansion in Fitts’ law tasks in VR HMDs. This can help understand and predict user behavior to aid designers and developers of VR environments in providing target expansion techniques to enable efficient and accurate target selection in different VR application scenarios.

Fitts’ law is one of the most famous HCI predictive models for human movements [5] and has recently been studied in immersive VR environments (e.g., [2]). Fitts’ law states that the movement time to acquire a target is affected by the distance to and the size of the target. One of the most widely used mathematical variations is defined in ISO 9241-411:2012 [10]:

MovementTime=a+b*log_{2}(\frac{A}{W}+1)=a+b*ID

(1)

where $A$ is the effective target distance, $W$ is the effective target size, and the $log$ term represents the effective index of difficulty ( $ID$ ).¹¹1Effective target distance, size, and index of difficulty are usually expressed as $A_{e}$ , $W_{e}$ , and $ID_{e}$ . To improve readability, we directly use $A$ , $W$ , and $ID$ to represent effective measures in this paper. The coefficients $a$ and $b$ are derived from linear regression using the empirical results. According to ISO standard [10], $ID$ and movement time can be further used to calculate the throughput:

Throughput=(\frac{ID}{MovementTime})

(2)

In this work, we adapted the multi-directional pointing task described in the ISO standard [10] to investigate the effect of target expansion on target selection performance in VR environments. We used movement time, error rate, and throughput as measurements to examine the selection performance. Besides, we also recorded the selection endpoints and visualized their distributions. We conducted a user study ( $N=17$ ) to compare three expansion factors (expanding the target width by 1.5, 2, or 2.5 times during the movement) against a baseline condition (no expansion) in the Fitts’ law task. Our results showed that expanding expansion factors of 1.5 and 2.5 can significantly reduce the movement time. This work mainly made an empirical contribution based on a user study with seventeen participants. We hope future work on target expansion and Fitts’ law in VR environments can be framed based on this work’s exploration.

Refer to caption — Figure 1: Experimental setup. (a) $W$ is the target width and $A$ is the movement amplitude. The blue arrows indicate the trial order. (b) $W_{expand}$ represents the expanded target width. $W$ , $A$ , and $W_{expand}$ are all effective distances using angular forms. The green button in the center represents the start button.

2 Preliminary Study

We aimed to investigate the effect of target expansion on target selection. To do so, we conducted a user study with seventeen participants recruited from a local university and used a Meta Quest 2 VR HMD to provide the testing environment. Our primary research question was how expanding targets would affect the speed, accuracy, and throughput of target selection.

2.1 Task and Stimuli

We used an ISO 9241-411:2012 [10] multi-directional tapping task with 11 grey spherical targets placed as a circle (Fitts’ ring design). Participants were asked to use the dominant-hand controller to point at the targets and press the trigger on the non-dominant-hand controller to perform the selection. This setting was made to avoid the Heisenberg effect [4]. They were informed to select the target as fast and as precisely as possible. The task started with the target at the top of the ring after participants triggered the start button located at the center of the ring. Once participants selected a target, the next target was on the diagonal, and continued clockwise on the ring. The goal target would be in blue as an indication. Participants were sitting in the middle of an empty virtual room where visual depth cues were provided via this setting.

The original target width was $W$ , and the movement amplitude was $A$ . When participants controlled the ray-casting to move past a specified fraction $P$ , that is, the expansion point, of the total distance $A$ , the target size $W$ would instantaneously expand to $W_{expand}$ . In this study, we set $P=0.9*A$ [12]. Figure 1 shows the experimental setup. Following the approach described by Yu et al. [19], we used a spherical coordinate system and angular representations whenever feasible. Thus, $W$ , $A$ , and $W_{expand}$ were all effective distances.

2.2 Experimental Design

We used three target widths $W$ (1.5°, 2.5°, and 3.5°) and two movement amplitudes $A$ (15° and 20°), which led to six indices of difficulty (IDs). This study followed a 6 $\times$ 4 within-subject design with two factors: index of difficulty $ID$ (2.402, 2.747, 2.807, 3.170, 3.459, 3.841) and expansion factor $E$ (1, 1.5, 2, and 2.5). An expansion factor of 1 represented a baseline condition where the target size would not change during the trial. While for the remaining three conditions, the target size would expand—the target width would increase to 1.5, 2, and 2.5 times the initial width. The orders of $W$ and $E$ were counterbalanced using a Latin Square approach separately. For a given $W$ , the order of $A$ was randomized. The data from the first trial for each $ID$ condition (i.e., each $W\times A$ combination or each ring) was discarded. Thus, in total, we recorded 6 indices of difficulty $\times$ 4 expansion factors $\times$ 10 repetitions $\times$ 17 participants = 4080 trials of data.

2.3 Measurements

For each trial, we recorded the movement time (second, s) and endpoint $(x,y)$ . The endpoints were also represented using angular forms [19]. We further calculated error rates (%) and throughput (°/s) for each condition per participant. Throughput was calculated using Equation 2.

2.4 Procedure

Before the experiment, participants were briefly introduced to the experimental setup, including the device, task, and procedure. After that, they started a 3-minute training session to familiarize themselves with the VR device and ISO pointing task. Then they began to complete the formal trials. The whole experiment lasted about 35 minutes per participant.

2.5 Results and Discussion

2.5.1 Movement Time, Error Rate, and Throughput

We used repeated measure (RM-) ANOVA to analyze the movement time, error rate, and throughput results. We applied Greenhouse-Geisser correction for degrees of freedom whenever needed ( $\epsilon>.75$ for our cases) and Bonferroni correction for post hoc analyses. Results are summarized in Figure 2(a-c) and Table 1.

We only found a significant main effect of ID ( $F(2.466,39.460)=3.776$ , $p=.024$ , $\eta_{p}^{2}=.191$ ) and a significant main effect of E on movement time ( $F(3,48)=6.734$ , $p<.001$ , $\eta_{p}^{2}=.296$ ). Post hoc analyses revealed that an expansion factor of 1.5 ( $M=.797s$ , $p=.015$ ) and 2.5 ( $M=.753s$ , $p=.024$ ) led to significantly longer movement time than an expansion factor of 1 ( $M=.872s$ ); that is, no expansion effect applied.

Table 1: The results of RM-ANOVAs in the study.

	Factor	$DF_{n}$	$DF_{e}$	$F$	$p$	$\eta_{p}^{2}$
Movement	$ID$	2.466	39.460	3.776	.024	.191
Time	$E$	3	48	6.734	$<$ .001	.296
	$ID$ $\times$ $E$	3.200	51.203	.709	.560	.042
Error	$ID$	2.712	43.396	1.238	.306	.072
Rate	$E$	3	48	2.613	.062	.140
	$ID$ $\times$ $E$	15	240	1.045	.409	.061
Through-	$ID$	1.151	18.414	2.358	.139	.128
put	$E$	1.186	18.979	2.105	.162	.116
	$ID$ $\times$ $E$	1.261	20.169	1.206	.299	.070

2.5.2 Fitts’ Law Analysis and Endpoints

Using Fitts’ Law in Equation 1, we found the MT (movement time) can be modeled as $MT=.186+.223*ID$ ( $R^{2}=.972$ ) for $E=1$ condition, $MT=.206+.193*ID$ ( $R^{2}=.963$ ) for $E=1.5$ condition, $MT=.112+.207*ID$ ( $R^{2}=.984$ ) for $E=2$ condition, and $MT=.221+.173*ID$ ( $R^{2}=.977$ ) for $E=2.5$ condition. Figure 2(d) shows the Fitts’ law results.

We plotted all the endpoints in Figure 3 to show their distributions in each condition.

2.6 Answers to the Research Question

According to the study results, expanding the target width by 1.5 and 2.5 times during the movement can significantly shorten the time for reaching the target. As expected, expanding the target size made the selection faster. However, no significant difference among expanding conditions was found (i.e., $E$ = 1.5, 2, and 2.5). One possible reason is that the expansion factors in our study did not vary largely from each other. Yet, for a large expansion factor, its application scenario would also be limited. Thus, we suggest VR developers consider expanding the target (e.g., buttons or 3D objects) to a suitable size to make the selection more efficient.

We did not identify significant improvements in the accuracy and throughput for target selection when expanding targets. Notably, the main effect of $E$ on the error rate was close to statistically significant ( $p=.062$ , $\eta_{p}^{2}=.140$ ). From Figure 3, we can observe that expanding the target ( $E$ = 1.5, 2, and 2.5) tolerated the selection pointed outside the initial target. While some endpoints were around the target in $E=1$ condition, which can be potentially selected if the target can expand. Future work with more participants, IDs, and testing trials are needed to substantiate when the difference would appear.

3 Limitations and Future Work

We only applied linear magnification effects to the target. Prior work has proposed non-linear magnification strategies [8, 9]. Future work can also test them using Fitts’ law approach to investigate the user motor performance. In addition, as a nature of the Fitts’ law task, the target selection tasks were given repeatedly. It is inevitable that participants could predict the expansion of the target, which could affect their performance. We plan to conduct a follow-up study involving conditions in which the target could unpredictably expand, remain unchanged, or even shrink [20] and test with more $IDs$ varying from a wider range [14]. Besides, prior work has been conducted to understand the endpoint distribution of pointing selection tasks [19, 16], which can help understand users’ behavior from another perspective and can be integrated into selection techniques to improve the selection performance further. We intend to model endpoint distributions based on our collected data. We envision that with the support of predictive models, expanding the target can improve the selection performance in different VR scenarios (e.g., in dense scenarios where objects overlap [13] or outside of users’ view [17]).

4 Conclusion

In this paper, we conducted a Fitts’ law study (N = 17) to investigate how expanding the target affects the selection performance in virtual reality (VR) head-mounted displays (HMDs). Specifically, we tested three expansion factors (expand the target width to 1.5, 2, and 2.5 times its initial size) and compared them with a baseline non-expansion condition. Our results showed that expanding the target can improve the speed of target selection compared to the baseline condition. We then discussed the limitations of the present study, which could represent avenues for future work. This work serves as an initial exploration of expanding targets for pointing selection tasks in VR HMDs from a Fitts’ law perspective and can help frame future work with relevant topics.

Acknowledgements.

The authors wish to thank the participants for their time and the reviewers whose comments helped improve our paper. This work was partly supported by the National Natural Science Foundation of China (#62207022; #62272396) and the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (#22KJB520038).

References

[1] F. Argelaguet and C. Andujar. Improving 3d selection in ves through expanding targets and forced disocclusion. In A. Butz, B. Fisher, A. Krüger, P. Olivier, and M. Christie, eds., Smart Graphics, pp. 45–57. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008.
[2] A. U. Batmaz, K. Yu, H.-N. Liang, and W. Stuerzlinger. Improving effective throughput performance using auditory feedback in virtual reality. In Proceedings of the 2022 ACM Symposium on Spatial User Interaction, SUI ’22. Association for Computing Machinery, New York, NY, USA, 2022. doi: 10 . 1145/3565970 . 3567702
[3] D. A. Bowman and L. F. Hodges. An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In Proceedings of the 1997 Symposium on Interactive 3D Graphics, I3D ’97, p. 35–ff. Association for Computing Machinery, New York, NY, USA, 1997. doi: 10 . 1145/253284 . 253301
[4] D. A. Bowman, C. Wingrave, J. Campbell, and V. Ly. Using Pinch Gloves^TM for both Natural and Abstract Interaction Techniques in Virtual Environments. 2001.
[5] P. M. Fitts. The information capacity of the human motor system in controlling the amplitude of movement. Journal of experimental psychology, 47(6):381, 1954.
[6] M. Guillon, F. Leitner, and L. Nigay. Static voronoi-based target expansion technique for distant pointing. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, AVI ’14, p. 41–48. Association for Computing Machinery, New York, NY, USA, 2014. doi: 10 . 1145/2598153 . 2598178
[7] M. Guillon, F. Leitner, and L. Nigay. Investigating visual feedforward for target expansion techniques. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15, p. 2777–2786. Association for Computing Machinery, New York, NY, USA, 2015. doi: 10 . 1145/2702123 . 2702375
[8] C. Gutwin. Improving focus targeting in interactive fisheye views. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’02, p. 267–274. Association for Computing Machinery, New York, NY, USA, 2002. doi: 10 . 1145/503376 . 503424
[9] K. Hornbæk and M. Hertzum. Untangling the usability of fisheye menus. ACM Trans. Comput.-Hum. Interact., 14(2):6–es, aug 2007. doi: 10 . 1145/1275511 . 1275512
[10] ISO. Ergonomics of human–system interaction—part 411: Evaluation methods for the design of physical input devices. Standard, International Organization for Standardization, Geneva, CH, May 2012.
[11] M. McGuffin and R. Balakrishnan. Acquisition of expanding targets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’02, p. 57–64. Association for Computing Machinery, New York, NY, USA, 2002. doi: 10 . 1145/503376 . 503388
[12] M. J. McGuffin and R. Balakrishnan. Fitts’ law and expanding targets: Experimental studies and designs for user interfaces. ACM Trans. Comput.-Hum. Interact., 12(4):388–422, dec 2005. doi: 10 . 1145/1121112 . 1121115
[13] R. Shi, J. Zhang, Y. Yue, L. Yu, and H.-N. Liang. Exploration of bare-hand mid-air pointing selection techniques for dense virtual reality environments. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA ’23. Association for Computing Machinery, New York, NY, USA, 2023. doi: 10 . 1145/3544549 . 3585615
[14] R. W. Soukoreff and I. S. MacKenzie. Towards a standard for pointing device evaluation, perspectives on 27 years of fitts’ law research in hci. International Journal of Human-Computer Studies, 61(6):751–789, 2004. Fitts’ law 50 years later: applications and contributions from human-computer interaction. doi: 10 . 1016/j . ijhcs . 2004 . 09 . 001
[15] L. Vanacken, T. Grossman, and K. Coninx. Exploring the effects of environment density and target visibility on object selection in 3d virtual environments. In 2007 IEEE Symposium on 3D User Interfaces, 2007. doi: 10 . 1109/3DUI . 2007 . 340783
[16] Y. Wei, R. Shi, D. Yu, Y. Wang, Y. Li, L. Yu, and H.-N. Liang. Predicting gaze-based target selection in augmented reality headsets based on eye and head endpoint distributions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23. Association for Computing Machinery, New York, NY, USA, 2023. doi: 10 . 1145/3544548 . 3581042
[17] D. Yu, H.-N. Liang, K. Fan, H. Zhang, C. Fleming, and K. Papangelis. Design and evaluation of visualization techniques of off-screen and occluded targets in virtual reality environments. IEEE Transactions on Visualization and Computer Graphics, 26(9):2762–2774, 2020. doi: 10 . 1109/TVCG . 2019 . 2905580
[18] D. Yu, H.-N. Liang, F. Lu, V. Nanjappan, K. Papangelis, and W. Wang. Target selection in head-mounted display virtual reality environments. Journal of Universal Computer Science, 24(9):1217–1243, 2018.
[19] D. Yu, H.-N. Liang, X. Lu, K. Fan, and B. Ens. Modeling endpoint distribution of pointing selection tasks in virtual reality environments. ACM Trans. Graph., 38(6), nov 2019. doi: 10 . 1145/3355089 . 3356544
[20] S. Zhai, S. Conversy, M. Beaudouin-Lafon, and Y. Guiard. Human on-line response to target expansion. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’03, p. 177–184. Association for Computing Machinery, New York, NY, USA, 2003. doi: 10 . 1145/642611 . 642644