State of the Art
Simulation fidelity is being radically redefined within AR and VR environments, including holographic overlays (such as that developed for LucinaAR), AI mediated interactivity, natural language processing, facial and gesture recognition. These technologies offer the promise for simulation to approach in situ experiences capable of providing effective training, even to advanced practitioners who require fully realistic immersive experiences. Achieving sufficient fidelity to allow natural expression of higher levels of expertise remains a challenge. Virtual or augmented artifacts that do not accurately emulate the real world create cognitive cue mismatch and prevent true representation for advanced learners, and may train to incorrect performance in novices.
Aspirations
Raise the level of fidelity to approach in situ experience visually, interactively and emotionally. Leverage AI developments to enable the incorporation of avatars representing team members, enabling the development of expertise in the critical skills of communication, leadership and teamwork.
Challenges / barriers
AR/VR technology continues to develop; however, its ease of use, comfort and resolution are limited and prevent prolonged experiences. Technology is rapidly advancing, requiring the development platform be flexible in its ability to accept advances instead of becoming obsolete.
Immediate Research Projects:
- Develop a platform that can, in concert with our data capture needs, leverage current AR capability to elevate fidelity. Platform will require controller functions, authoring, and capacity for AI integration.
- Optimize capability to overlay AR on existing manikins.
- Evaluate the impact of selected fidelity parameters on learning performance and outcomes.
- Identify characteristics of avatars that promote or inhibit the suspension of disbelief.
Longer Term Research Objectives:
- Develop an AI human avatar response model for baseline interpersonal interactions.
- Explore enhancements to tactile fidelity using haptic technology and manikins developed for AR overlay.
Educational Validation
Expertise is characterized by enhanced perceptual and cognitive efficiency, situational awareness, affect self-regulation, performance monitoring and execution, strategic decision making, outcome prediction and intrinsic motivation. We believe we can capture, analyze, characterize and adaptively respond to differing levels of expertise within high fidelity environments, and in so doing, enhance learning outcomes and operational performance.