Virtual reality: how it works and which types industry uses

Virtual reality

Virtual reality replaces what a person sees and hears with a computer-generated environment that responds to their movements and lets them act inside it (Milgram and Kishino, 1994). The principle comes from the head-mounted display Ivan Sutherland described in 1968: a perspective image that changes as the user moves.

Fyware's XR glossary, with the source behind each definition.

Where the definition comes from

Virtual reality, in Fyware's XR glossary
Tom Klukowski

Tom Klukowski

Fyware Partner · Valcre Marketing

How can we help?

Let's Talk

The definition

Virtual reality replaces what a person sees and hears with a computer-generated environment that responds to their movements and lets them act inside it (Milgram and Kishino, 1994). The principle comes from the head-mounted display Ivan Sutherland described in 1968: a perspective image that changes as the user moves.

Sutherland’s headset, built at Harvard with part of its funding from ARPA, put two miniature cathode ray tubes in front of the eyes and measured the position of the head with either an ultrasonic sensor or a mechanical arm hanging from the ceiling. The arm supplied the six motions needed to track translation and rotation, and the sensors followed the head within a volume about six feet across and three feet high. Custom hardware recomputed the perspective of every line, up to 3,000 lines at 30 frames per second, and the synthetic image covered a 40-degree field of view. Sutherland wrote that stereo matters, though less than the change in the image when the observer moves their head, and current headsets follow the same logic: they track the head, often the hands as well, and redraw the scene for each eye. His display also let the wearer see the room through half-silvered mirrors, so today it would be filed among the optical see-through displays, on the augmented reality side.

The border with its neighbors is drawn by what the person stops seeing. When Ronald Azuma defined augmented reality in 1997, he described the virtual kind by contrast: while immersed, the user cannot see the surroundings. If the headset shows the room, through clear optics or through the video of its cameras, and adds digital content on top, the system belongs to augmented or mixed reality.

Proposals often treat two words as synonyms, and a buyer gains by keeping them apart. Immersion is what the system delivers, and it can be written into a specification: frame rate, tracking latency, field of view, image quality and how much of the body is tracked, among the parameters Mel Slater lists. Presence is the person’s response. Slater calls it place illusion, the sense of being there while knowing one is not, and adds plausibility illusion, the sense that what happens in the scene is actually happening. Immersion sets the boundaries within which place illusion can occur, while plausibility depends on whether the events in the scene relate to the person and are credible against what they expect.

Slater also separates the actions a person takes to perceive, such as turning the head or bending down to look under something, from the actions that change the environment, such as grabbing an object with a tracked hand. The first set is what divides immersive systems from non-immersive ones. Immersive virtual reality on a standalone headset runs with no computer or cables, like the four Fyware Safe simulators on Meta Quest 3, which also need no internet connection. On a PC-tethered headset it can carry heavier scenes. Non-immersive virtual reality runs on a screen with a mouse and keyboard: the person acts on the scene, but turning the head does not change the image.

For a buyer, the type follows from what the body has to do. A procedure that depends on bending down or on judging the distance to a suspended load needs a headset that tracks the head in position and in rotation. When it depends on the order of the steps and on what gets touched, a screen can be enough and saves buying headsets.

How it looks in a project

Gerdau Corsa trains its own crane operators on virtual reality simulators built by Fyware: eleven modules that cover six cranes, rigging at floor level, the electric arc furnace, lockout/tagout, a 360 induction and the platform that hosts them, with a report per person. It is the PC-tethered immersive type, on Meta Quest 2 and Meta Quest 3 headsets and developed in Unity, and the operator works physical industrial controls wired into the simulation, the same levers as in the cab, so the hand learns the movement. The procedures and the assessment criteria belong to Gerdau Corsa, and Fyware built the simulators to that specification.

Where the definition comes from

«[...] to present the user with a perspective image which changes as he moves.»

Sutherland’s paper sets out the principle of the head-mounted display: a perspective image that changes as the person moves. His equipment could draw 3,000 lines at 30 frames per second, with a 40-degree field of view for the synthetic image.

Source: Ivan E. Sutherland, “A head-mounted three dimensional display,” Proceedings of the AFIPS Fall Joint Computer Conference, Part I, December 1968, pp. 757-764, DOI 10.1145/1476589.1476686; text read in the reprint of the Ars Electronica 1990 catalog, section Introduction; the figures are in Operation of the display system and Presenting images to the user

«totally immersed in, and able to interact with, a completely synthetic world»

Milgram and Kishino summarize the conventional view of a virtual reality environment: the participant is fully immersed in a synthetic world and can interact with it.

Source: Paul Milgram and Fumio Kishino, “A Taxonomy of Mixed Reality Visual Displays,” IEICE Transactions on Information and Systems, vol. E77-D, no. 12, December 1994, pp. 1321-1329, section 1, Introduction: Mixed Reality

«While immersed, the user cannot see the real world around him.»

Azuma defines augmented reality by contrast with the virtual kind, whose defining trait is that the immersed person no longer sees the surroundings.

Source: Ronald T. Azuma, “A Survey of Augmented Reality,” Presence: Teleoperators and Virtual Environments, vol. 6, no. 4, August 1997, pp. 355-385, DOI 10.1162/pres.1997.6.4.355, section 1.2, Definition

«Immersion provides the boundaries within which PI can occur.»

Slater separates immersion, a physical property of the system, from place illusion, the person’s response; the first sets the limits of the second.

Source: Mel Slater, “Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments,” Philosophical Transactions of the Royal Society B, vol. 364, no. 1535, 2009, pp. 3549-3557, DOI 10.1098/rstb.2009.0138, section 3, Presence and place illusion

Frequently asked questions

How does virtual reality work?

A headset measures the position and orientation of the head, and a computer redraws the scene for each eye with every movement, the principle Ivan Sutherland described in 1968. Many headsets also track the hands, so the person can pick up objects without a controller. The faster and more precisely the image follows the head, the more room there is for what Mel Slater calls place illusion, the sense of being there.

What is non-immersive virtual reality?

It runs on a screen with a mouse and keyboard and no headset: the person acts inside a 3D environment, but moving the head does not change the image. One example is the sterile setup simulation Fyware built for medverse GmbH, which runs on Windows and macOS and lets people repeat the procedure without spending supplies. Slater notes that on a desktop system the sense of being there, if it comes at all, takes much more exposure and deliberate attention.

What is virtual reality used for in manufacturing?

To train people on tasks where practicing on the equipment costs production or puts someone at risk, such as running a crane or putting out a fire. Fyware does it with custom simulators, like the eleven modules built for Gerdau Corsa, and with the off-the-shelf Fyware Safe simulators. Another use is reviewing a plant change before it is installed: Mabe’s automation team walked through a modified paint booth and a new press area in virtual reality before a single machine moved.

Let's talk about your plant

Tell us what you want to solve in your plant. Fyware designs extended reality around the equipment and constraints of your operation, and builds it to grow from a pilot to the whole site.

Tom KlukowskiTom Klukowski, Fyware Partner · Valcre Marketing We reply in under 24 business hours, with scope, timeline and investment range.

In the meeting you see the simulator running and what it would take to bring it to your plant, with scenarios adapted to your layout. And if you need to sell it internally, this page is your presentation: share it with your team.