Many people learn best by being shown how to do a task. Augmented Reality – which overlays virtual diagrams and instructions over the real world viewed through a camera or glasses – can reproduce this experience and enable specialist knowledge to become more widely available. It is anticipated that Extended Reality (XR) technologies could be revolutionary within industry but adoption has been slow.
RADMA scholar Dr. Leon Pietschmann investigated the potential of XR Visual Guidance on user performance and human factors in his PhD thesis ‘Human-Computer Interaction in Extended Reality’.
New insights into the effectiveness of visual guidance in XR
Based on several studies featuring a total of 258 participants, Leon’s thesis contributes the most comprehensive empirical investigation on Visual Guidance in XR to date. It provides a new framework for a more systematic study of XRVG and insights into the effectiveness of Visual Guidance in XR for procedural industrial tasks.
As part of the RADMA doctoral studies programme, the generous support by RADMA enabled Leon to conduct this applied research and establish collaborations with organisations such as the German Federal Agency for Technical Relief and Harvard University.
Leon’s thesis is open access and available via https://doi.org/10.17863/CAM.108303.
What constitutes appropriate visual guidance?
Extended Reality (XR) technologies, which include Virtual Reality (VR) and Augmented Reality (AR), are transforming the way we perceive and interact with the environment and have been described as a key enabling technology for the fourth industrial revolution.
Visualisations offer the potential to consolidate and convert data into digestible, actionable insights which can be absorbed faster than pure data or text instructions. When deployed through a head-mounted display, XR becomes a powerful tool for hands-free operations in an industrial environment, increasing productivity and performance.
A good example of widely used XR visual guidance is navigation, where arrows are used to direct people instead of written instructions.
Applications in industry include: assembly, maintenance and operator training.
A study by The Boston Consulting Group estimated that the use of AR technologies would facilitate faster operations, reduce training costs, improve first-time fix rates by 50% and decrease labour costs.
In his paper Leon provides the ideal scenario: “Ideally, XR systems would need to be designed in such a way that once users put on smart glasses, for example, all tasks and individual steps would become clear immediately and users would out their tasks in an intuitive and effortless manner, without the need for written instructions.”
He identified a number of ways that XRVG would contribute to increases in performance:
- reduction in cognitive load – freeing up working memory for other tasks
- increases in worker speed (i.e. reduced time to complete a task)
- fewer mistakes/errors (i.e. improved safety and product quality)
- enhanced learning and/or retention
The study
Leon identified that the term ‘XR Visual Guidance’ is poorly defined, and further that there has been little research on how to leverage visual guidance.
So through extensive literature review he looked at what constitutes appropriate visual guidance and created a framework as part of a systematic approach to XRVG for worker augmentation in industrial environments. The latter he tested with a series of practical exercises with volunteers, this included construction of complex Lego model using XRVG.
He has progressed the field by proposing the following definition
As a subcategory of instructional design, Visual Guidance serves to leverage visual cues with the overall goal of improving user performance. This may be achieved by positively impacting the underlying human factors such as reducing cognitive load and making instructions more intuitive.
Leon gained a number of learning points from the practical study.
- XRVG can significantly increase user performance if designed and applied systematically – he proposes that rather than transcribe standard instructions, that VG should exploit the potential of the technology to provide visual guidance.
- Action guidance reduced mistakes and decreased time to completion – for practitioners this means that it is vital to identify the types of challenges the user is experiencing. Eg in the Lego study the placement process was a significant source of mistakes
- Visual guidance should not obstruct users view – the most common complaint was that arrows obscured the field of view. Highlighting using a glow or colour change was more useful and intuitive.
The insights gained, especially as part of the free-text feedback received from participants, supported the overall model of the XRVG framework and its three dimensions (i.e. Gaze Guidance, Object Identification, and Action Guidance).
However, the work also revealed some contradictions, as Leon explains: “contrary to what was initially hypothesised, the underlying human factors, namely cognitive load, usability, and motivation, did not coincide with considerable changes in the observed performance when using XRVG. This suggests a more complex relationship between Visual Guidance and user performance, for example there may be improved retention of information. This is an interesting area for further study.”
What support did you get from RADMA and how did it help you?
“As a RADMA scholar, the generous support and funding enabled me to conduct my applied research, connect with leading experts in the R&D Management field, and establish fruitful partnerships with organisations such as the German Federal Agency for Technical Relief and Harvard University. Without RADMA’s generous support, the collaborative and often international projects I pursued as part of my PhD would not have been possible.
“Towards the end of my PhD, four publications have been published, with an equally large number in the publication pipeline, and several invited presentations have been held. Without RADMA, a similarly large research output would not have been possible.
“RADMA provides a platform for the R&D Management community to come together. This invaluable platform is a catalyst for the cross-pollination of ideas and the sharing of knowledge, which enables us to come together and learn from each other.”
Sci-fi predicts science
“For a light read, I recommend popular books such as “Experience on Demand” by Stanford Professor Jeremy Bailenson and the sci-fi novel “Ready Player One” by Ernest Cline.
“For a more in-depth exploration, my thesis is open access and available via https://doi.org/10.17863/CAM.108303.”
What are you doing now and does this build on your knowledge of R&D Management?
“Currently, I am a Management Consultant at The Boston Consulting Group (BCG) where I combine my expertise in XR and strategic management to foster technology adoption together with senior executives of some of the world’s biggest companies. My position allows me to play an active role in ensuring a smooth transfer of know-how and new technologies between academia and industry, ultimately bringing about meaningful and innovative change.”
Find Leon on LinkedIn.
