State of the Art
Performance analysis within simulated environments is currently generated through subjective human observation. Cost-effective, mobile and high resolution devices have recently emerged for capturing cognitive, affective, communication and motor performance, providing new opportunities for detecting, classifying, assessing and responding to learner state and levels of expertise.
Aspirations
Create a robust framework to support granular data capture and analysis across every aspect of individual and group performance, including visual events, physiology, communication, behaviour and movement. The data can then be utilized to enable reliable and automated detection, assessment and classification of learner state and level of expertise against standardized competencies and entrustable professional activities (EPAs).
Challenges / barriers
Gaze tracking and physiological metrics are readily captured, but other dimensions, such as fine grained hand and finger movement, may be more challenging. And, although communication content, sentiment and speech prosody can be captured, the analysis and accurate interpretation of communication patterns related to expertise will require a large data set and sophisticated algorithms.
Immediate Research Projects:
- Develop an infrastructure which can capture gaze, movement and communication measures in real time, with sufficient resolution and minimal impact on natural behaviour.
- Identify the most reliable and practical indicators of performance.
- Conduct a first study to capture expression of expertise in situ and simulated environments, and monitor resulting data closely to determine robustness of infrastructure.
Longer Term Research Objectives: Develop effective data capture methods to support:
- Automated determination of target of attentional focus.
- Gross and fine grained movement capture for performance analysis across the learner expertise spectrum.
- Fine grained speech content and prosody analysis, including speech target, tone, sentiment, confidence, authority, and empathy.
- Fine grained behavioural analysis, including constructs such as situational awareness and strategic reasoning and response selection.
Educational Validation
Data capture and analysis underpins learner assessment and the potential for adaptive system response. Complexity and invasiveness of capture must be weighed against value of the collected data for learning insight and response generation, individually and in combination with other data sources. That value cannot be determined a priori, and consequently, data value determination will form a significant part of our research effort within a number of different projects.