A NOVEL AUGMENTED REALITY BASED SYSTEM FOR PROVIDING MAINTENANCE ASSISTANCE

Abstract

Maintenance plays an important role in the equipment life cycle as it restores and improves the equipment performance, reliability, and safety. However, the increasing complexity and the technology advancement of equipment pose constant challenges to the maintenance technicians nowadays. Augmented reality (AR) technology, which is an interface between the virtual and real world, can be used to enhance maintenance activities. AR is an enabling technology for man-machine-information interaction that can assist maintenance activities. Using AR, maintenance information in multi-media forms can be augmented and aligned virtually in the maintenance area, and collaborative work between the maintenance technicians and the remote experts and engineers can be enabled and enhanced through visual interactions. Thus, AR is a better approach for providing maintenance information as compared to paper prints and computers and can potentially improve the workflow of remote maintenance. In this research, the reported AR-assisted maintenance systems have been analyzed and the opportunities of applying AR technology to routine and ad hoc equipment maintenance activities have been explored. Specifically, a novel AR-assisted maintenance system (ARAMS) has been proposed to assist the technicians in their daily maintenance activities. ARAMS has two features, i.e., providing a remote collaboration mechanism that allows the experts to create and provide AR-based visual instructions to the technicians efficiently and effectively, and exploring the authoring and visualization of context-aware AR contents. Using the ARAMS system, context-relevant information can be provided to the technicians, i.e., the information will be provided according to the status of the maintenance activity and the technician¿s expertise level. In addition, the AR contents can be interacted with instead of being ¿read-only¿, so that the technicians can rectify any incorrect AR contents that have been created by the AR developers and author AR contents to record and share the accumulated knowledge and experience on equipment maintenance with other technicians. In remote viii maintenance, where the technician may encounter difficult situations and seek assistance from a remote expert, the ARAMS system can assist the remote expert to instruct the technician through authoring AR-based instructions online conveniently. To achieve these functions of the proposed system, authoring and the generation of AR contents are addressed in this research. A bi-directional authoring tool, which enables the AR developers to create context-relevant information via a desktop 2D user interface and the maintenance technicians to author AR contents on-site, is proposed. In addition, an online authoring tool is proposed to enable an expert to create intuitive AR-based instructions in an environment with no a priori knowledge conveniently and efficiently. In this prototype, a Dual Window User Interface (DWUI) is proposed to broaden the expert¿s view and improve the robustness of the authoring process. Based on these proposed methodologies, a prototype of the ARAMS system has been developed, and three user studies have been conducted to validate the proposed concepts and techniques and test the usability of the ARAMS system. The user studies indicate that the proposed system can assist the AR developers to author context-aware AR contents effectively, the authored contents can be adapted to the contexts correctly, and the provided AR information can improve the maintenance work flow as compared to paper manuals. In remote maintenance, ARAMS enables more efficient and less error prone remote collaboration. Finally, user feedback has been collected to provide suggestions on further improvement of the prototype system in future.