--- title: "AI and XR: how intelligent systems enhance extended reality" description: "XR builds the place and AI makes it respond. How the two combine in responsive virtual environments, simulations and conversational practice for training." canonical: https://peopletechrevolution.com/learn/ai/ai-and-xr --- [Learn](https://peopletechrevolution.com/learn) # AI and XR [Extended reality](https://peopletechrevolution.com/learn/xr/what-is-xr) builds the place. Virtual, augmented and mixed reality put digital content around a person, from a fully simulated room to an object anchored on a real desk. Without conversational [AI](https://peopletechrevolution.com/ai), what that content says back is set in advance. A learner practises with a floating expert presence. ## AI is what lets the place answer back Put the two together and a character can listen, work out what was actually said, and answer differently as the conversation goes, instead of running a pre-recorded script. ## How AI and XR combine in practice Adding AI changes what an XR environment can do. Conversational virtual characters. Instead of a branching tree of pre-written choices, a character understands natural speech and responds in character on the fly. Real-time voice processing. Speech-to-text converts the words, a language model decides the reply, and text-to-speech vocalises it with synchronised lip movement. Adaptive scenario progression. The system reads performance and hesitation, then adjusts difficulty or offers a coaching cue. Computer vision and spatial awareness. In AR and MR, machine learning reads surfaces, walls, hands and tools, so digital objects interact convincingly with the real room. The engineering sense of "digital twin". A live, to-scale model of a real facility, separate from the AI companion kind PTR builds. A knowledge panel, floating beside the room. Approved information surfaced where the person is standing, not on a separate screen. ## The conversation loop, end to end The steps repeat in real time as the conversation continues. PTR builds the language-model step in that diagram: AI companions and conversational agents that answer from approved knowledge within defined boundaries. The same twin can be reached through chat, voice or an on-screen avatar. See [PTR's digital twins](https://peopletechrevolution.com/ai/digital-twins) for how one spans all three. ## Why the combination matters for practice-based learning The pairing is useful because it combines a spatial setting with a character that responds to what the learner says and does. A presenter explains VR workshop delivery in Townsville. - Spatial immersion. XR places the learner in a convincing context, a clinic, a workshop, a negotiation room, with a sense of presence a flat screen cannot match. - Dynamic responsiveness. AI provides an unpredictable, conversational counterpart that reacts to tone, phrasing and decision-making. Together they let someone rehearse a conversation, a procedure or a decision under pressure, as many times as it takes, with no patient, colleague or piece of equipment at risk. ## Current limitations and considerations Combining the two adds problems neither has on its own: response delay, model boundaries, and what the hardware can carry. Response latency. Fluid conversation needs a near-instant reply. A 2025 study found users preferred a character to show a small natural filler, a glance or gesture, while it composed its reply, rather than a loading indicator. [Gonzales et al, 2025](https://arxiv.org/abs/2508.11781) Hardware and connectivity. Detailed 3D graphics alongside real-time voice needs reliable compute, a low-latency network, or a capable standalone headset. Guardrails and consistency. A character must stay inside its scenario and its professional boundaries; that needs clear instructional design and technical limits. Privacy of spoken data. Voice recordings and transcripts from an immersive practice session need to be handled under the same data-protection standards as any other record. ## More than a chatbot inside VR Dropping a text window into a VR scene does not, on its own, create the effect people mean by "AI and XR." A scripted scene with written replies to pick. Participants practising interpersonal skills rated a VR-embodied conversational AI agent more effective than the same AI reached through text alone in [Cornell/Stanford, CSCW 2025](https://dl.acm.org/doi/10.1145/3715070.3749224) research. A separate comparison at [ACM SIGGRAPH MIG, 2025](https://dl.acm.org/doi/10.1145/3769047.3769051) found an AI NPC tutor reached moderate to high co-presence but rated well below a human tutor on overall social presence. Scripted character AI-driven character Dialogue options A fixed set of pre-written choices Open-ended natural speech or text Adapts to unexpected phrasing No, falls back to a default line Usually, within its guardrails Needs re-authoring for a new topic Yes, every branch is hand-written Often just an update to the source material ## A character that answers back, inside the scene A learner in a headset, a character who listens, and a reply nobody scripted. A character gestures, then waits for the reply. A tool guide aligned to the real workbench. A character pauses before answering the learner. A reviewer watches, approval controls in view. ## The short version A PTR guest tries a headset in Townsville. AI and XR combine when a language model decides what a virtual character says and does, and XR gives that character a spatial, embodied place to say and do it. Either piece alone falls short: XR without AI is a fixed script; AI without embodiment is a chat window. Still have a question about AI and XR? Ask our CEO Simon Lowe’s Digital Twin a question [Open the contact form](https://peopletechrevolution.com/learn/ai/ai-and-xr#contact) Related PTR work and further reading [PTR's digital twins. How the same twin is reached by chat, voice, on-screen avatar and booking.](https://peopletechrevolution.com/ai/digital-twins) [What is AI? How the language model behind the character actually works.](https://peopletechrevolution.com/learn/ai/what-is-ai) [AI for training and roleplay. The same pattern, without the headset.](https://peopletechrevolution.com/learn/ai/ai-training-roleplay) [XR overview. Every format PTR builds in virtual, augmented and mixed reality.](https://peopletechrevolution.com/xr) ## Sources and further reading Research on conversational agents in VR, social presence, interpersonal skills training and response delays. ### Young Merlin: an embodied conversational agent in virtual reality ACM ICMI · 2016 An early implementation of a conversational character living inside a VR scene. [Read it at dl.acm.org](https://dl.acm.org/doi/10.1145/2993148.2998534) ### Social Presence in Virtual Reality: A Comparative Study of AI NPCs and Human Instructors ACM SIGGRAPH MIG · 2025 The co-presence comparison behind the "more than a chatbot" band above. [Read it at dl.acm.org](https://dl.acm.org/doi/10.1145/3769047.3769051) ### Comparing Text-Only and Virtual Reality-Embodied Conversational AI Agents for Interpersonal Skills Training ACM CSCW Companion · 2025 The embodiment study behind the "more effective than text alone" note above. [Read it at dl.acm.org](https://dl.acm.org/doi/10.1145/3715070.3749224) ### Behavioral and Symbolic Fillers as Delay Mitigation for Embodied Conversational Agents in Virtual Reality IEEE Transactions on Visualization and Computer Graphics · 2025 The filler-action study behind the response-latency note above. [Read it at arxiv.org](https://arxiv.org/abs/2508.11781)