Climbing a skyscraper for the 25th time in a day quickly erodes the novelty of virtual reality.
For over three years, Attila Kardos was one of the quality assurance testers at PlayStation Liverpool, one of the testing centers of Sony Interactive Entertainment Europe. During his tenure, he worked on first-party behemoths like God of War, Horizon: Zero Dawn, and Detroit: Become Human. But today, heʼs talking about a different type of testing involving a headset and physically mimicking movements for multiple hours at your office desk.
In games like Blood & Truth, where you take a starring role in action movie-esque setpieces, the fun factor of virtual reality quickly makes itself apparent. However, experiences like this can be taxing. A player can stop at any time. A tester doesnʼt have that luxury. It doesnʼt take long for the monotonous physical effort to take a toll. And then they have to show up at work the next day and do it all again.
Job Simulator
“Eventually, you start feeling your shoulders,” Kardos laughs. “You feel the extra muscle pain.”
On average, he spent five to six hours a day in VR. He says it was “pretty intense,” to the extent that some colleagues asked management to swap to non-VR projects. Eventually, Kardos got acclimatized to working in virtual reality and recalls the experience fondly. However, during the first two weeks, heʼd often feel “detached from reality” whenever he moved his head around after clocking out from work.
As companies continue to iterate on past foundations or release new platforms altogether, from PlayStation to Apple, VR is constantly innovating. But the work conditions for QA folks who learn the inside out of these technologies rarely are. Game Informer spoke to testers, artists, and engineers who worked on VR games like Beat Saber, Job Simulator, Cosmonious High, Resident Evil 4 VR, and Blood & Truth about the differences with regular testing, how elements like physicality and hygiene come into play, unorthodox workarounds, and the unsung tradition of ginger candy.
Rift
Into Another Dimension
VR leaping into the mainstream meant new considerations for game development. And, as a consequence, testing. In order to understand the differences with traditional QA practices, itʼs important to look back at the technologyʼs origins to understand how impactful of a precedent it would set.
Few studios have the institutional knowledge of Owlchemy Labs. Based in Austin, Texas, the team began experimenting with the technology when the HTC Vive and Oculus Rift were only prototypes. There was no documentation or beginnerʼs manual. As promising as this empty virtual canvas was, it was a long time until the studio saw that potential materialized. “We had a very early headset that would shock you if you touched it on the wrong part of the circuit board,” says Graeme Borland, gameplay director at Owlchemy since 2013. “[It was] that level of true prototype.”
Multiple new considerations quickly came into play. Comfort, both inside and outside the headset, was paramount. As a principle, it intertwined with everything: locomotion methods, figuring out how elements like framerate and draw distances could make or break the experience, and the fact that visual bugs could lead to immediate discomfort.
Cosmonious High
During the first few years of the main headset companies, trial and error and experimentation led to different ideas. Unlike Owlchemyʼs Job Simulator, which had sold over a million copies by 2020, not every early project managed to successfully combine its ideas with this emerging technology. Such was the case of Rigs: Mechanized Combat League, developed by Guerrilla Cambridge and published by Sony.
“It was the first version of PlayStation VR, so we had resolution problems, sweaty eyes, everything was very fresh,” Kardos says. Rigs was his first project. In it, youʼre sitting in the cockpit of a mech – the movement of the robotʼs head, and as a result, your view direction, was controlled with the VR headset, while the body relied on the controller. Kardos thinks thatʼs one of the things that compromised the end result, as it was “absolutely dreadful” to run around and fight in arenas. As a fast-paced shooter with multiplayer modes, there were many things happening on screen at all times. It didnʼt take long to feel dizzy.
According to Kardos, there was a market research report dictating that “there are not many people interested in these kinds of games.” Initial feedback was fairly rough, too, pinpointing the complicated control scheme and proneness to motion sickness as weak points. Sony closed Guerilla Cambridge in 2017, a year after Rigs’ release.
Go Move
In terms of the testing itself, most of the regular QA processes are compatible. Tasks such as bug reports or running test cases – checklists of specific elements to test in different environments – largely apply to VR. The problem is that even mundane actions such as switching to a second monitor to take notes or sending a Slack message turn cumbersome.
Both Borland and Ben Hopkins, principal graphics engineer at Owlchemy, reinforce that the human experience of being inside a game is a significant factor of developing for VR. To try and mitigate constant usage of the headset, the team implemented a debug system, which essentially allows them to play the game with mouse and keyboard. “Itʼs good for a fast iteration, but there are so many things you canʼt actually understand whether they work or not unless youʼre actually in the headset,” he says.
When asking the QA testers at Owlchemy if they ever use the debug editor, they reply with laughter. “The approach is very different,” says Jessica Fly, senior QA specialist. Previously, she worked at Cloud Imperium Games testing Star Citizen. All she needed was to boot up the game on a flat screen, and she was set. The routine now involves putting on the headset and ensuring the room is set up properly to avoid tripping over anything. And thatʼs before the actual physicality of the job.
Blood & Truth
A clear example arises when doing full regression testing. During this stage, testers go through the game after an update or new version to ensure nothing is broken. In the case of Cosmonious High, one of the tasks involves testing all 18 recipes, each requiring a mixing-in-a-bowl type of movement. “Itʼs really murder on the arms,” Fly says. “Thereʼs this physical part of VR testing that is honestly one of the things that interest me.”
“Before Owlchemy, I was at Blizzard working on Hearthstone, and I canʼt think of a stronger differential between testing a fully immersive VR video game and testing a digital card game,” says Paul Henderson, QA tester 1 at Owlchemy. “You go from a game that is the most basic inputs; you drag a card to the slot, you drag it to your opponent to how can you move? Thatʼs your inputs! Go move.”
Through the Motions
Mayank Lal joined Keywords Studios in Gurgaon, India, back in early 2017. He started as a QA games tester. Almost seven years later, heʼs a quality assurance test lead. During this time, Lal witnessed VR gaining more relevance in the studioʼs portfolio. This, thankfully, led to better conditions for the 40 dedicated VR testers out of around 200 employees.
In the beginning, things were messy. It started with a partnership with Oculus back when Meta (formerly Facebook) wasnʼt involved. Testers quickly realized some issues needed to be addressed. “One of the most common things that we observed as testers in that particular time was feeling nausea in longer periods of use,” he says. “Headsets also heat up and that actually affects the skin on the face. Sometimes headsets are shared between testers, which increases the probability of getting infections.”