Refreshable Tactile Displays for Accessible Data Visualisation

Leona Holloway
Monash University Peter Cracknell
Quantum RLV Kate Stephens
Blind Consumer Melissa Fanshawe
University of Southern Queensland Samuel Reinders
Monash University Kim Marriott
Monash University Matthew Butler
Monash University

(June 2023)

Abstract

Refreshable tactile displays (RTDs) are predicted to soon become a viable option for the provision of accessible graphics for people who are blind or have low vision (BLV). This new technology for the tactile display of braille and graphics, usually using raised pins, makes it easier to generate and access a large number of graphics. However, it differs from existing tactile graphics in terms of scale, height and fidelity. Here, we share the perspectives of four key stakeholders – blind touch readers, vision specialist teachers, accessible format producers and assistive technology providers – to explore the potential uses, advantages and needs relating to the introduction of RTDs. We also provide advice on what role the data visualisation community can take to help ensure that people who are BLV are best able to benefit from the introduction of affordable RTDs.

\CCScatlist\CCScatTwelve

Human-centered computingAccessibilityAccessibility technologies \CCScatTwelveHuman-centered computingVisualisation

1 Introduction

Tactile graphics, also known as raised line drawings, are considered best practice to convey spatial and graphical information for people who are blind or have low vision (BLV). Refreshable tactile displays (RTDs) are hardware devices with a moveable surface for the dynamic display of tactile graphics. They offer the opportunity for electronic storage, quick access to a large number of graphics, along with dynamic interaction and manipulation of content.

While RTDs are not new, they have remained a rarity due to their high price point. This will soon change, with several devices set to enter the market at significantly lower prices within the next two years. Those devices that look most promising for commercialisation, such as the DotPad [6] and the Monarch [2], use refreshable pin displays with between 2,400 (60 $\times$ 40) and 3,840 (96 $\times$ 40) pins. Additional features such as touch sensitivity and multiple pin heights are included in other devices such as the Graphiti [16] and Metec [14].

Here, we examine the stakeholder perspectives of how RTDs could be used to improve access to graphics, including those of data, by way of meaningful tactile data visualisations.

Refer to caption — Figure 1: Pixelated heat map for display on an RTD (left) and the heat map displayed on a Graphiti RTD with variable pin heights (right)

2 Related Work

Prior work into RTDs has mainly focused on haptics and perception [8], zooming and panning [7, 13], – with more work needed in this area – and applications for using RTDs to display sports matches [10, 15] and maps [5, 12]. There has also been a lot of interest in the potential use of RTDs for people who are BLV to create their own graphics as a tool for drawing [3, 11] or for collaborative tactile graphics production [4].

The display of data visualisations on RTDs has received much less attention. Visualisation of algebraic graphs has been the first application to be explored [1, 17] but there are many more applications for extension within the data visualisation field.

3 Stakeholder Perspectives

The voices of four primary stakeholders presented here are based on our own experiences and opinions, combined with the perspectives of colleagues and collaborators within the print accessibility community. We are: a blind touch reader who has experience reading traditional tactile graphics and using an RTD; a vision specialist teacher who is also a parent of a student with a vision impairment; an accessible formats producer with experience in producing tactile graphics and contributing to guidelines for tactile graphics design; and an assistive technology provider with expertise in the range of assistive technology devices for touch access and their use.

We have retained the first person voice to emphasise the personal nature of our ideas but also in the hope that we can bring forward the voice of stakeholders as co-designers at the beginning of the design process for a new technology.

3.1 Blind Touch Readers

I am excited by the many possibilities that RTDs offer.

Dynamic content: RTDs introduce the possibility of tactile animations [9], especially if the refresh rate can be improved.

Timeliness: The ability to have images updated in real-time and/or materials to be produced instantaneously.

User creation: Most importantly, RTDs open the possibility for people who are BLV to be creators rather than just passive receivers of data visualisations. We want to make our own data visualisations from equations or data; or to import existing images and make sense of them. I have also enjoyed the experience of using a touch-sensitive RTD to draw by dragging my finger across the surface.

3.2 Vision Specialist Teachers

Timeliness: As a teacher and a parent of a student with a vision impairment, I am most interested in RTDs enabling students who are BLV to access information at the same time as their peers. Currently, the delays in producing hard copy tactile graphics mean that students cannot access the curriculum at the same time as the rest of their classmates, limiting their participation in learning. Ideally, there would be a quick and easy way for classroom teachers to transfer data and graphs into a meaningful representation on an RTD without needing expertise in braille or accessible formats.

3.3 Accessible Formats Producers

Immediate feedback and corrections: The successful creation of accessible graphics relies on translation and interpretation in order to simplify a diagram for touch while retaining the most important information. Since the advent of computerised accessible graphics production, there has been the added difficulty of designing for touch using a flat visual display. The introduction of RTDs could potentially make our task much easier, firstly because we are able to check our work tactually throughout the editing process and secondly because any corrections or refinements suggested by the user can be implemented immediately.

User creation: Ultimately, we hope that RTDs could reduce our role dramatically by transferring power to the user, who should be able to generate and adjust their own tactile graphics, particularly in the realm of data visualisation.

Design guidelines: I was at first sceptical about the utility of RTDs given the small number of pins available. However, the multiple pin heights available on a Graphiti RTD allowed a surprising amount of information to be conveyed. This required a new way of thinking about the design of tactile graphics, with new design guidelines required specifically for RTDs.

3.4 Assistive Technology Providers

User creation: What most excites me about RTDs is the possibility that a blind person reviewing a tactile graphic may have an opportunity to edit it and create their own tactiles, perhaps with their finger position being tracked by sensors, or tracing shapes that pop up as the finger moves – just as sighted people draw on iPad apps.

Dynamic content: Being dynamic, the data on an RTD could change in real-time. For example, an RTD could be connected to laboratory equipment to display measurement values as they change. On a boat, sailing speed, wind direction, and compass bearing could be displayed in real-time.

Most likely use cases: RTDs are heavy and expensive, so they may not be suitable for accessing maps or other graphics on the go. They are more suitable for high-use cases at a fixed location, for example at school or in the workplace. US funding models mean that educational use will likely be the first big market for these devices.

4 Implications for the Visualisation Community

Stakeholders are clear that RTDs offer a fundamentally new kind of tactile graphic experience, one that is the haptic equivalent of interactive data visualisation. The effective introduction of RTDs must be informed by learnings from data visualisation, design guidelines for static tactile graphics, and co-design with people with lived experience of blindness. Key areas of inquiry include:

1.

Designing tactile graphics with visualisation best practice in mind, trialling on RTDs and undertaking user testing. This will determine the optimal visualisations and use cases. With a limited pins available, judicious design choices are essential and will be aided by new design guidelines based on research into whether traditional visualisation design principles hold and where they need to be adapted.
2.

The display of tactile graphics on RTDs is the first opportunity for BLV people to access dynamic content and interact with tactile diagrams. The visualisation community can help to investigate effective interaction methods for features such as zooming and panning, as well as the combination of RTDs with speech and sonification for labelling.
3.

Key understandings of visualisation creation should also be considered in the context of RTDs, in particular empowering BLV people as user creators. For example, the first DotPad software does not allow for nonvisual graphics creation but DotInc have made their API available for developers to make their own apps for the creation, conversion and display of graphics on the DotPad. Other manufacturers have indicated that they would like to collaborate on software design and would benefit from the expertise of the visualisation development community. Such authoring systems will also be vital in exploiting the opportunities for timeliness.

The data visualisation community and their expertise will be crucial in supporting the accessibility community to explore the potential of RTDs and help ensure that people who are BLV can best benefit from the introduction of affordable RTDs in the near future.

References

[1] P. Albert. Math class: An application for dynamic tactile graphics. In ICCHP: International Conference on Helping People with Special Needs, pp. 1118–1121. Springer, 2006.
[2] American Printing House for the Blind. Meet Monarch, 2023.
[3] J. Bornschein, D. Bornschein, and G. Weber. Comparing computer-based drawing methods for blind people with real-time tactile feedback. In CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 2018.
[4] J. Bornschein, D. Prescher, and G. Weber. Collaborative creation of digital tactile graphics. In ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’15), pp. 117–126. ACM, New York, NY, USA, 2015. doi: 10 . 1145/2700648 . 2809869
[5] L. Brayda, F. Leo, C. Baccelliere, E. Ferrari, and C. Vigini. Updated tactile feedback with a pin array matrix helps blind people to reduce self-location errors. Micromachines (Basel), 14(9):351, 2018. doi: 10 . 3390/mi9070351
[6] Dot Inc. Dot Pad, 2022.
[7] J. Ducasse, M. J.-M. Macé, B. Oriola, and C. Jouffrais. Botmap: Non-visual panning and zooming with an actuated tabletop tangible interface. ACM Transactions on Accessible Computing (TACCESS), 25(4):1–42, 2018. doi: 10 . 1145/3204460
[8] N. Garcia-Hernandez, N. G. Tsagarakis, and D. G. Caldwell. Feeling through tactile displays: A study on the effect of the array density and size on the discrimination of tactile patterns. IEEE Transactions on Haptics, 4(2), 2011.
[9] L. Holloway, S. Ananthanarayan, M. Butler, M. De Silva, K. Ellis, C. Goncu, K. Stephens, and K. Marriott. Animations at your fingertips: Using a refreshable tactile display to convey motion graphics for people who are blind or have low vision. In ASSETS International ACM SIGACCESS Conference on Computers and Accessibility. ACM, New York, NY, USA, 2022. doi: 10 . 1145/3517428 . 3544797
[10] M. Kobayashi, Y. Fukunaga, and S. Shimada. Basic study of blind football play-by-play system for visually impaired spectators using quasi-zenith satellites system. In ICCHP: International Conference on Computers Helping People with Special Needs, pp. 23–27. Springer International, 2018.
[11] M. Kobayashi and T. Watanabe. Communication system for the blind using tactile displays and ultrasonic pens – mimizu. In ICCHP: International Conference on Computers Helping People with Print Disabilities, pp. 731–738. Springer International, 2004.
[12] F. Leo, E. Cocchi, E. Ferrari, and L. Brayda. Maps as Ability Amplifiers: Using Graphical Tactile Displays to Enhance Spatial Skills in People Who Are Visually Impaired, p. 65. Springer, 2020.
[13] V. L’evesque, G. Petit, A. Dufresne, and V. Hayward. Adaptive level of detail in dynamic, refreshable tactile graphics. In IEEE Haptics Symposium (HAPTICS), pp. 1–5. IEEE, 2014. doi: 10 . 1109/HAPTIC . 2012 . 6183752
[14] Metec. HyperBraille, 2012.
[15] H. Ohshima, M. Kobayashi, and S. Shimada. Development of blind football play-by-play system for visually impaired spectators: Tangible sports. In 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–6. ACM, 2021. doi: 10 . 1145/3411763 . 3451737
[16] Orbit Research. Graphiti, 2016.
[17] T. Schwarz. Interface for automatic tactile display of data plots. In Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE), vol. Part I, pp. 73–81. Springer, Switzerland, 2022. doi: 10 . 1007/978-3-031-08648-9_10