For the person running the pilot: how a pilot runs, stage by stage

What each stage decides, who does what on each side, and what you can show at the end.

Your questions

How long, how many staff, what do we provide, what do we get?

What a champion asks before committing a roster and a room.

A standup in front of a wall of notes at the start of a project, the discovery and consultation step. — Planning a pilot at a wall of notes.

How long, and how many staff?

Agree the timing and the decision criteria together, at alignment. Size recruitment for the evaluation you intend to run, so the measures you chose can be compared before and after.

A participant discusses an XR rehearsal with a facilitator. — A participant talks it through after the session.

What do we provide, and what do we get?

You provide the room, the people and a facilitator who will run the debrief once trained (headsets and equipment covers IT). You get your own evaluation against the measures chosen up front, and a record of who rehearsed what where you set it up that way: business case and evaluation.

The stages

Each stage ends with a decision

Digital NSW: innovation buying is "multiple repeated Source and Manage stages, with decision-gates between each repetition." A pilot follows the same shape.

A bearded man in a navy jacket talks across a table to colleagues, a tablet of coloured shapes between them. — A consultation across a table, tablet between.

Alignment: name the result up front

Your sponsor and champion meet PTR to name the conversation, who has it and the result you want at work. Kirkpatrick Partners advise starting with results, "understanding the ‘Why’ and the purpose of the program".

Two planners review a flood response plan. A VR training scenario for government and public sector. — Planners review a response plan together.

Discovery: incidents become the brief

Your educators and frontline staff describe the moments that go wrong; PTR writes the scenario brief, as in designing a scenario and its debrief.

People compare VR Library scene cards at a table. — Comparing scene cards from the VR Library.

Build or select

A PTR VR Library title that fits is selected; where the setting matters, PTR builds the scenario in recreations of your own spaces (fast builds, innovation teams).

People stand in a plain room, a woman gesturing as she speaks, wooden stools beside them, comparing the choices they made in a scenario. — People talk it over in a plain room.

The session itself: prebrief, practice, debrief

INACSL: "all simulation-based educational (SBE) activities must include a planned debriefing process." A trained facilitator runs it after the headset comes off; Google allocated 25 minutes to it in every session.

Two reviewers compare blank response panels. An AI roleplay scenario for AI roleplay and practice. — Reviewers compare responses on screen.

Evaluation: the levels named at alignment

The Kirkpatrick Model: reaction, "favorable, engaging, relevant"; learning, "knowledge, skills, attitude, confidence, and commitment"; behaviour, "the critical behaviors" at work; results, "targeted organizational outcomes occur". Instruments: how to evaluate a pilot.

A colleague hands a working phone build across the table to the person who asked for it. A rapid prototyping scenario for fast builds. — A working build handed across the table.

Decision: continue, widen or stop

Your sponsor reads the evaluation against the result named at alignment. At Mater the decision was to continue: its own facilitators were trained to run the technology across its campuses.

Roles on each side

Who does what

Yours: a sponsor who owns the result, a champion who owns the calendar, a facilitator, the staff who rehearse, the keeper of the training record, an IT contact. PTR’s: scenarios, headsets or browser practice, facilitator preparation, the debrief structure, the evaluation instruments (how we deliver). A trainer you already work with can facilitate.

A participant rehearses in the headset while observers watch from another part of the room. — A participant rehearses while colleagues watch.

A permits officer reviews a screen with colleagues. A VR training scenario for mining and resources. — A worker holds a tablet between colleagues.

A facilitator with grey hair talks through a flow diagram and sticky notes on a whiteboard while participants, including a wheelchair user, watch from the table. — A facilitator works through a whiteboard flow.

PTR’s published work

What Mater, Google and Mayo Clinic measured

Evaluated by the organisation that ran each program, or by researchers.

Inside the Mater build: an older patient sits up in a hospital bed, a meal tray across the blanket, curtains drawn beside him. — Inside the Mater build, at a bedside.

Mater Education, Australia: the client’s own evaluation

In its own closure report, 90% of participants reported an increase in concentration and over 96% wanted more training in VR: Mater Immersive Learning Pilot.

An office desk scenario in the Google DEI training, an overheard conversation involving tokenism. — An office desk scenario from Google.

Google, United States: peer reviewed at CHI 2024

Led by Leonie Sanderson, built with Equal Reality, a subsidiary of People Tech Revolution. In the paper, the 7 trainees who returned the survey gave 82% average agreement it beat standard DEI training and 94% average satisfaction, findings the authors call preliminary: VR DEI training at Google.

A clinical scenario for practising conversation. — A clinical scenario seen through a headset.

Mayo Clinic, United States: published research

People Tech Revolution built a DEI module for nurse leaders; the Journal of Nursing Administration paper reports delivery over 4 months, with 593 of 1,131 learners returning the evaluation. The published record covers speaking up, inclusion and nurse leadership; a pilot on another conversation sets its own baseline.

Download the Google research paper PDF Download the Mater case study PDF

Where practice fits