How virtual reality could change the journalism industry

The water is rising. Wind roars through your ears and you can feel the rooftop underneath your feet shake. In every direction you look, the flood waters are slowly creeping up. Through the driving rain you can see your neighbors, screaming in distress, standing atop their homes, as you are, desperately looking for help. A body floats by, a poor soul not lucky enough to have a rooftop. You hear, and then see, a helicopter fly overhead and you wave and wave but it passes by. When will the water stop? What happens if nobody comes to help you?

This harrowing experience was designed in collaboration with NPR journalist Barbara Allen, who approached me in 2012 to see how VR could be used to heighten the storytelling powers of journalism. We at Stanford University’s Virtual Human Interaction Lab had been thinking for years about the potential of immersive journalism, but until Barbara knocked on my door, we didn’t have the time, motivation, or journalistic expertise to create a simulation. It took us a while to find the right scenario for the project and we discussed several possibilities.

Eventually, Barbara came up with the brilliant idea to simulate the aftermath of Hurricane Katrina, so that users could get a better sense of the human terror and suffering caused by the storm, experiences that traditional media could only hint at through distant camera shots or in written reports. Barbara had covered the storm, and done a lot of documentary work, so she was well acquainted with the kind of details that could bring this terrible scenario to life. Not to mention that the physical tracking space in the lab at Stanford was about the size of an actual rooftop, so the experience would really leverage the space.

What followed was an iterative process: we’d pore over Barbara’s notes and videos, program the visual scene and interactivity to match, and then repeat the process, often bringing in new journalists to give us feedback and then going back to the drawing board. The project culminated in a large “opening” where a number of prominent visitors were able to experience Katrina.

At the time we were doing our Katrina experiment, we couldn’t have imagined that actual examples of VR journalism would be created and widely disseminated only a few years later. But almost immediately after consumer VR became possible, reporters, news organizations, and independent producers seized the moment to begin producing original journalistic content. No organization has been more bullish than the New York Times, which distributed over a million cardboard viewers to its print subscribers and created a high-end, VR-specific smartphone application to distribute Times-created VR experiences. Other news organizations, like VICE, the Wall Street Journal, PBS Frontline, and the Guardian in the UK have also experimented with the medium.

Bringing audiences closer to the reality of a story has always been the preoccupation of journalists, and VR, it seems, offers an ideal multimedia experience. There is a hint of urgent and perhaps forced optimism, too, as traditional journalism outlets try to figure out how they can recapture audiences they have been losing to the proliferating sources of news in our increasingly fragmented media world—why pay for it when you can get the news for free on your browser or Facebook feed? In this light, adding a new exclusive VR feature to the suite of services offered by the likes of the New York Times may be one way to address the financial woes that have beset the ailing news business.

After all, new media technologies have always gone hand-in-hand with journalism; the definition of journalism has constantly evolved along with those technologies. Beginning in the seventeenth century, early newspapers consisted of just the written word, but as soon as improved printing technologies allowed it, engraved illustrations and diagrams were added to written accounts. By the latter part of the nineteenth century, photographs, or detailed engravings based on them, began to appear alongside news stories.

Of course, how “truthful” these photographs actually were is questionable. Beyond the fact that photographs only capture reality from a particular point of view, we know that early photojournalists frequently staged their pictures. The Civil War photographer Matthew Brady, for instance, is known to have rearranged the bodies of fallen soldiers on battlefields to improve the composition of his pictures. No doubt this sleight of hand felt justified to Brady because by increasing the visual impact of his photographs of the carnage, he felt he could better convey the horrors of these bloody battles. But such manipulation would clearly be a serious violation of the professional ethics that guide photojournalists today, when even adding a digital filter to manipulate the brightness in an image is condemned.

In the 20th century, radio, newsreels, television, and later the Internet all continued to shape the news business with new multimedia and interactive features, each innovation raising questions about how to uphold journalistic ideals of objectivity, independence, and truth-telling that began to develop in the mid-twentieth century. Today, consumers of news are increasingly drawn to news sources that confirm their own beliefs. As the number of news organizations that attempt to pursue the established ideals of the profession decline, their deeply reported pieces can get drowned out by stories presented from partisan perspectives or even fake news designed to grab attention and generate outrage. In this fast-changing news landscape we have two big reasons to be wary of a medium like virtual reality.

Only the rooftops of houses are seen with the skyline in the background in flooded New Orleans August 30, 2005. Journalists have used VR to help viewers better understand events like Hurricane Katrina. File photo by REUTERS/Rick Wilking RTW/PN

One, it plays powerfully upon our emotions, an affordance that in many situations doesn’t exactly encourage rational decision-making. A viewer who exposes himself to a VR depiction of an atrocity, for instance, will feel like a personal witness to that event and experience the resultant level of outrage. Where does that outrage go? Stoking these kinds of emotions and playing upon people’s instinctive desire to lash out at perceived threats is a time-worn strategy for tyrants, terrorists, and politicians. I have little doubt that virtual reality will be an excellent tool for spreading propaganda.

A second, related worry about VR is that, due to its digital nature, it can be easily altered and manipulated. This doesn’t make it different from other media, of course. We know photos and video are subject to manipulation, and even written accounts can be biased to present an ideological point of view. The fact that it is common for other media to be strategically altered for deceptive ends shouldn’t give us comfort. In VR, which actually “feels real,” the potential dangers for misinformation and emotional manipulation are exponentially greater. When false events are put forth, it will be hard to argue with viewers who believe them. After all, they will have seen them with their own eyes.

Ethical journalists will respect the standards that evolve to ensure accuracy and objectivity in VR journalism. For example, the pioneer of using computer-generated VR for journalism is Nonny de la Peña, a formally trained journalist who has created a series of fully immersive experiences that allow viewers to witness real- world events like the controversial Trayvon Martin shooting and a tragic domestic homicide case that occurred in South Carolina.

But it’s safe to say some purveyors of nonfiction narratives will abuse the mutable nature of VR for ideological or sensational ends. Initially, these abuses of VR technology will occur with immersive computer-generated VR experiences, which have to be built from the ground up and are entirely within the control of the producer. Because they are computer-generated they still lack the detail and photo realism that can easily fool people into believing they are real. However, this may all soon change with advances in “light field” technology, which allows digital cameras coupled with powerful computers to extrapolate enough information from the light they receive to build photorealistic volumetric avatars and place them in three-dimensional spaces.

We can imagine a future when photorealistic VR video can be quickly recorded and edited after the fact with the same ease as photo editors shave pounds off models on magazine cover images. When this happens, it is not difficult to imagine unscrupulous creators of VR “news” tweaking footage to their ends, just as Soviet leaders were able to disappear discredited apparatchiks from historical photos, or as political campaigns in the United States have doctored images of campaign rallies to give the appearance of larger crowds.

In 2016, the Department of Communication at Stanford University sought to study these types of issues in a class dedicated to immersive journalism. Why use virtual reality in your storytelling? In what situations does it add to a journalistic narrative? The answer, they determined, was not as often as you might think. The conclusion of the class was that at the moment VR added value to the story in only specific cases, and then only as a supplement to a traditional reporting style.

Most VR news stories are done through immersive video, and capturing this video can be complicated in a journalistic setting. For one, because the camera is recording images in 360 degrees, if the camera operator doesn’t want to be confused with a character in the scene she is filming, she must run out of the spherical shot after setting it up. This means that which events are captured by the camera will be happening outside of the photographer’s control, and perhaps without her knowledge. This does not lead to the most riveting footage. It can be difficult to tell a story—an act of direction and selection—when the camera eye sits in one place passively.

In a way, our early VR journalists are in the same situation the 19th century photojournalists were in when they took their cameras into the field. Hauling around heavy, delicate equipment that takes a long time to set up and to execute a shot, the best way to ensure a good product is to set the scene. This is why some of the earliest experiments in VR journalism are documentaries with staged shots, dramatizing the lives of the people depicted, or static scenes of organized activities, like demonstrations, vigils, and political rallies. It may be some time before VR photojournalists are able to venture out into a breaking news event and capture effective footage in the middle of conflict.

Situations in which the surroundings or environment are a crucial part of the story make for the best VR, as the best, early practitioners are starting to realize. A good example of this type of storytelling is 6×9, a VR experience produced by the Guardian, which takes users inside a solitary confinement jail cell. Claustrophobia is one of the tried and true parlor tricks in VR. What makes this a good use of VR is that the entire environment surrounding the viewer is important to the story, and requires the viewer to explore it by turning her head or body. If all the action is front and center—say at a political debate—you don’t need spherical video. There is little point in filming in VR if the engagement with the story only requires you to look in one direction.

All these early experiments in virtual journalism raise the urgent question: If media organizations build VR, will the audience come? Will VR stick or will it go the way of 3D TV? They share some of the same issues—expensive technology and goofy, uncomfortable, wrap-around glasses. But there is too much investment and development going on in the space—it is already likely too big to fail. How is VR different from 3D TV? There was never a clear content reason to invest in 3D TV. No “killer” experience emerged, so content producers never reached the critical mass of the market. With the exciting content that has been produced so far, the trajectory for quality content in the VR space already has a foundation. Once you experience a VR “aha” moment, you can’t wait to find the next one

This essay has been excerpted from “Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do” by Jeremy Bailenson. © 2018 by Jeremy Bailenson. Used with permission of the publisher, W.W. Norton & Company, Inc. All rights reserved.