Multimodal Augmented Reality and Subtle Quidance for Industrial Assembly: A Survey and Ideation Method
2022 (English)In: Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349, Vol. 13318 LNCS, p. 329-349Article in journal (Refereed) Published
Abstract [en]
Industrial manual assembly is a relatively established use case for emerging head-mounted Augmented Reality (AR) platforms: operators get visual support in placing pieces depending on where they are in the assembly process. However, is vision the only suitable sensory modality for such guidance? We present a systematic review of previous work done on multimodal guidance and subtle guidance approaches, confirming that explicit visual cues dominate. We then outline a three-step method for generating multisensory guidance ideas intended for real-world task support based on task observation that led to identification of 18 steps in truss assembly, brainstorming AR guidance approaches related to assembly and maintenance, and mapping of brainstorming results to the observed task. We illustrated the use of the method by deploying it on our current mission in producing AR guidance approaches for an industrial partner involved in designing and assembling wooden trusses. In this work, we went beyond the standard visual AR guidance in two ways, 1) by opening for guidance through auditory, tactile, and olfactory sensory channels, 2) by considering subtle guidance as alternative or complement to explicit information presentation. We presented a resulting set of multisensory guidance ideas, each tied to one of the 18 steps in the observed truss assembly task. To mention a few which we intend to investigate further: smell for gradual warning about non-imminent potential hazardous situations; 3D sound to guide operators to location of different tools; thermos-haptics for subtle notifications about contextual events (e.g., happening at other assembly stations). The method presented helped us to explore all modalities and to identify new possibilities. More work is needed to understand how different modalities can be combined and the impact of different modality distractions on task performance. © 2022, Springer Nature Switzerland AG.
Place, publisher, year, edition, pages
Springer Science+Business Media B.V., 2022. Vol. 13318 LNCS, p. 329-349
Keywords [en]
Trusses; Assembly process; Guidance; Ideation methods; Industrial assemblies; Manual assembly; Multi-modal; Multimodal Interaction; Multisensory; Subtle cue; Survey methods; Augmented reality
National Category
Robotics Computer Systems
Research subject
Production Technology; Work Integrated Learning
Identifiers
URN: urn:nbn:se:hv:diva-19151DOI: 10.1007/978-3-031-06015-1_23ISI: 000870269100023Scopus ID: 2-s2.0-85131960208OAI: oai:DiVA.org:hv-19151DiVA, id: diva2:1707364
Conference
14th International Conference, VAMR 2022Held as Part of the 24th HCI International Conference, HCII 2022Virtual Event, June 26 – July 1, 2022Proceedings, Part II
2022-10-312022-10-312024-04-08Bibliographically approved