Immersive A/B testing: a method for future fleet design

So, what is the answer to the research question: How can we validate and test revolutionary OMCs?The study shows that immersive A/B testing is not only feasible, but also highly effective. By letting operators experience future concepts in realistic, high-fidelity simulations, navies can explore design questions early on. This reduces risk when exploring revolutionary OMCs, while allowing navies to be more innovative. 

As the naval world moves towards hybrid fleets of collaborating crewed and uncrewed platforms, MARIN’s Seven Oceans Simulator centre offers a powerful testbed for shaping the OMCs of tomorrow, before they ever touch the water. Within MOMiT, we will develop more methods to design future OMCs. 

A future frigate, four USVs, and two ways of working

Over two experiment days, 12 naval professionals stepped into a high-fidelity, fully integrated simulation environment. Their mission: to operate a future frigate travelling across the Mediterranean, accompanied by four autonomous uncrewed surface vessels (USVs). On deck a fast craft and drones are available. Around them, unexpected events happen: first a suspicious surface vessel contact, then propulsion trouble on one of the USVs.

What changed between simulator runs was not the scenario, but how the crew was organised. Two OMCs were put to the test: a lean, distributed, specialised and remotely supported OMC A and an even leaner and co-located OMC B.

OMC A: Distributed, Specialised and Supported

For this Operational Management Concept six operators worked from four different locations within the simulator centre:

• one on the frigate bridge

• two supporting situational awareness from a dedicated room

• two on the deck, maintaining drones and operating the fast craft and drones

• one analyst ashore providing technical support

OMC B: lean and co-located

For this concept four operators worked from only two locations: two on the bridge and two on deck.

To recreate the complexity of modern naval operations, MARIN connected multiple simulators and VR systems into one seamless environment:

• a bridge simulator for ship handling (Large Motion Simulator)

• a VR CAVE for a situational awareness support room on the ship (Experience Room)

• a remote technical support station for managing USVs (Shore Control Centre Lab)

• a motion based 6DOF simulator for the fast craft (Fast Small Ship Simulator)

• a ship deck (Maritime eXperience Lab)

• a drone-flying environment (VR)

Operators could walk, talk, gesture, and make decisions as they would at sea. Being at different locations they were connected with headsets.

Measuring performance from different angles 

The study combined quantitative data (eye tracking, skin conductance, heart rate variability and communication patterns) with qualitative insights from observation and structured debriefings using SWOT analyses.

This mixed methods approach provided a detailed view into workload, situational awareness, decision making, teamwork and performance under realistic operational pressure.

Although both concepts were workable, the preliminary results point strongly towards OMC A. The distributed setup, supported by specialised workstations and active remote technical support contributed to more effective and resilient operations. Participants experienced a lower workload (physiological and self-reported), smoother coordination across roles and better mission performance. 

Development should be based on the SWOT analyses and observations. It should also test OMCs in various scenarios with different events.

This transition is an ambitious shift, one that demands not only new technology, but new ways for humans and technology to work together. By bringing tomorrow’s naval operations into today’s simulators, MARIN is exploring a key question: How can we develop revolutionary Operational Management Concepts?

An Operational Management Concept (OMC) consists of operational functions and tasks that are conducted by humans, machines or both, at specific physical locations to achieve the objectives of an operation.