Autonomy Research Center
NASA human-machine teaming research for Mars and Europa mission scenarios.
- Category
- Systems · Research
- Year
- 2026
The design decision in human–AI collaboration is not the interface. It is the confidence threshold at which the system is allowed to interrupt human attention. The rest of this chapter is the evidence: the environment, the instrument, the finding, and the interaction grammar that came out of it.
My work focuses on the interaction layer surrounding these decision systems: the interfaces, workflows, diagnostic instrumentation, and supporting tools that allow operators to maintain situational awareness while collaborating with intelligent systems.
C-01 · Context
ARCS is a multi-year engineering platform investigating how domain specialists collaborate with AI-generated guidance in dynamic, physically demanding operational environments. The platform combines spatial computing, multimodal interaction, and explainable AI to evaluate how people interpret, trust, and act upon machine-generated recommendations under real-world constraints.
P-02 · Problem
Most AI systems assume users can continuously interpret recommendations while managing their primary task. Operational environments rarely allow that. Users lose situational awareness when guidance becomes opaque, interfaces demand unnecessary attention, or feedback loops operate more slowly than the pace of the task itself.
Early platform iterations emphasized visual fidelity rather than operational usability. The critical design question became: what information belongs in an operator’s field of view, when should it appear, and how much confidence must exist before the system interrupts human attention?
E-03 · Evaluation framework
Participants completed structured evaluation sessions across three simulated operational environments designed to reproduce collaborative planning, navigation, and decision-making under changing conditions. Each session combined spatial interaction, verbal reasoning, and AI-assisted task execution.
Researchers recorded participant dialogue, environmental context, behavioral observations, and interaction telemetry throughout every evaluation.
R-04 · Research methodology
Collected data was coded into recurring behavioral patterns describing how teams resolved uncertainty, negotiated shared understanding, coordinated decisions, allocated attention, maintained situational awareness, and calibrated confidence in AI guidance.
The most important finding was not interface related. Participants consistently refused to context-switch under high cognitive load. This behavioral constraint became the foundation for every subsequent interaction and workflow decision.
- 45
- Participants
- 3
- Evaluation cycles
- ~15
- Per session
- 4
- Operational scenarios
H-06 · Hardware & behavioral findings
Stress-testing Microsoft HoloLens 2 and Apple Vision Pro within physically demanding workflows exposed several recurring interaction constraints.
- 01
Visual degradation
Dense spatial overlays reduced peripheral awareness and degraded depth perception when multiple interface layers competed for attention.
- 02
Vestibular load
Visual latency, headset weight, and calibration drift produced measurable balance compensation, causing participants to move more cautiously than task conditions required.
- 03
Situational awareness cost
Frequent vertical gaze shifts between digital overlays and the physical environment introduced additional cognitive workload during time-sensitive decision making.
- 04
Interaction latency
Even relatively small delays encouraged users to repeat gestures prematurely, producing unnecessary interaction loops as participants attempted to force system responsiveness.
O-07 · System outcome
The resulting interaction framework improved comprehension of AI-generated guidance while reducing decision latency across the most time-sensitive evaluation scenarios.
Rather than optimizing interface aesthetics, the project established a workflow architecture that prioritized glanceability, confidence calibration, and operational continuity. The resulting interaction grammar and diagnostic instrumentation became part of the broader platform while identifying ongoing research questions around sustained cognitive load, multimodal interaction, and long-duration human–AI collaboration.
Task decomposition resolves into decision points, which anchor a constrained interaction grammar, which in turn produces the instrumented signals that feed the research loop. Select a node to inspect, then open the Analytical Workspace below to teach the decision, contextualize the grammar, and substantiate the evidence.
Decision topology
The design decision in human–AI collaboration is not which interface. It is the confidence threshold at which the system is allowed to interrupt. Drag τ: or focus the handle and use ↑ / ↓ to rehearse the trade-off. Higher τ interrupts less; lower τ misses recovery moments.
- Layer
- Decision runtime
- Signal
- Confidence · operator intent
- Gate
- Threshold τ = 0.62
- Fallback
- 3 of 12 samples yield
A-08 · Scalable applications
Although developed within this research platform, the findings generalize well beyond spatial computing. The interaction constraints identified throughout ARCS apply wherever specialists collaborate with autonomous or AI-assisted systems under operational pressure.
- D-01
Autonomous & mission-critical operations
Decision-support systems for aerospace, defense, robotics, remote operations, and emergency response. Environments where maintaining situational awareness directly influences mission success.
- D-02
Healthcare & rehabilitation
Clinical simulation, rehabilitation technologies, exposure therapy, procedural guidance, and immersive training. Where interface timing, cognitive workload, and confidence calibration influence learning and patient outcomes.