Designing for VR
Note: This post is an outcome from reseach I did around VR, as Design Director and resident of Tigerspike’s R&D. In only a few days, my team has managed to achieve: design, develop and test a VR prototype for Google Cardboard, which then was also moved to Oculus. The below video is the presentation for the prototype. This quick project allowed us to understand more about the technology, how to design an experience for it and how to estimate times for a project for a client.
Context
What we need to discuss is not “what we can do with this technology” but “what should we do with it”. After lots of reading, experimenting, playing and discussing Virtual Reality, here are some of my thoughts on: entertainment, education, health and law considerations and, of course, designing for VR.
You can’t just talk about Virtual Reality if you haven’t experienced it. You can assume things—especially if you had “some previous experience with an early demo” or “you’ve been reading about it since the 80’s”—but you have to witness with your own eyes what the latest technology has to offer. The potential, the needs, and considerations have changed, and it’s now safe to say that Virtual Reality is here, soon to be followed by Augmented Reality and Mixed Reality.
Virtual reality is a concept dating back to the 1950s, made possible today by great advances in technology and innovation from companies such as Facebook, Google, Samsung, and others. By definition, virtual reality is user experience, immersing a user into a computing environment they can interact with. Yes, it still feels like “someone stuck your face on a 9-inch screen with bad resolution and switched off the lights“ but, it’s more mature than ever. It’s going to be broadly used soon and it’s time (especially for designers) to talk seriously on the approach this new medium needs.
If you’re new in VR, I suggest you start by reading the following:
The History and Future of VR. If you don’t know where to start, start here.
Should I care about Virtual Reality? Let’s park the VR hype-machine for a moment.
Games
The Holy Grail. What drives and pushes every technology to its limits (even AI used a game to prove it’s got better). There are already some great titles out there that can keep you busy if you decide to spend the £2K required for a complete and future-proof VR set. The cost of the good titles is high and some say they are not worth it yet. From what I‘ve tried, I see great potential but limited actual use. The technology has raced ahead and now awaits game designers and developers to push it to it’s limits.
Anyhow, there will be more amazing games made in VR but, although gaming will be the primary VR narrative in the near future, we’ll soon stop seeing games as essential to this medium. Remember the first years of TV compared to now that we think of TV screens as content-agnostic input-output devices. A ‘big screen in the living room’ will be replaced by a VR set but they’ll work as complementary media and access points.
I can’t agree more with Chris Dixon on ‘A new creative medium where the default state is belief.‘ With presence, your brain goes from feeling like you have a headset on to feeling like you’re immersed in a different world.
• • •
Sure, VR games are a big thing, but I can’t stop imagining applications other than entertainment like education (any type), healthcare (from and for VR), and personal assistance on anything. It will get crazy!
“The future is already here — it’s just not very evenly distributed.”
But, before this, allow me first to share a few words on…
Mobile VR
Google Cardboard and Samsung Gear VR have made 2016 the year of smartphone-based VR. Although the tech is far more limited than that of a monster desktop PC, mobile VR holds a lot of potential; think ease of use, low cost and portability.
I firmly believe that the best VR experiences will be limited for desktop use. When AR and MR will be commercially available we’ll see all the three different types of Artificial Reality split their paths regarding the people’s needs. But, until then, mobile VR is going to be the dominant form of VR in terms of reach and engagement. It’s easy to predict that this will be the first VR experience for most people and the first thing that comes to mind when people think of VR (maybe it already is).
“The problem the ecosystem faces is that the consumers who are most interested in virtual reality do not have the spending power necessary to support high-end devices. Llamas says PC and console virtual reality’s [high barrier to entry] will cause three out of four early adopters to opt for more affordable mobile devices. ”
There are some restrictions, like the availability of hardware, and mainly the battery duration of smartphones, but we’re not far from morning trains full of commuters with their mobile VR goggles on and complete lack of reality awareness. You can imagine this, right? Now, can you imagine the first massive use case for mobile VR? Correct, airplanes. Forget tablets and seat back displays, they’ll be giving you a nice mobile VR set, that is if you don’t have your own, and why wouldn’t you? It’s possibly something that we’ll carry on with us everyday. Think of when positional, and hand tracking will be available (now you get that Apple patent for a ring-type wearable).
“A few years from now, I think there’s a strong chance that the mobile VR headset market will look a lot like today’s headphone market. Most people will use low-cost devices that they’ve gotten for either cheap or free, while high-end offerings find traction among hardcore users.”
If you want some great insights, like the above, read Chris Fralic’s ‘Why Mobile Will Win First in VR‘. For VR, I think a stronger distinction should be made between mobile solutions and dedicated, high end solutions.
Education
You simply cannot not see the potential here. From educational stories for younger kids to immersive experiences, into VR ancient worlds, as history lessons for teens. From traveling (geography) or art (visit museums of the world) to body anatomy lessons in universities. From driving schools to space mechanics. It’s endless. VR can take education to the next level by making it interesting, fun, engaging and thus, maybe, more accessible. And don’t worry if kids will ‘get’ or like it. The contrary.
Kids are closer to VR than you think. Here are some interesting links:
These Two School Districts Are Teaching Through Virtual Reality. Now virtual reality is allowing kids to experience planets and historical places like never before.
Kids are aware of VR. Kids and teens between the ages of 10–17 know what VR is, and when they see it in action they think it’s “off the charts cool”.
• • •
Actually, if there’s something to worry about with VR it’s not if it’s boring or difficult, but if it’s too addictive.
Considerations
We know digital worlds and especially games are addictive. It comes down to dopamine, one of the brain’s basic signaling molecules. Emotionally, we feel dopamine as pleasure, engagement, excitement, creativity, and a desire to investigate and make meaning out of the world (it’s the exact same effect you have with running). It’s released whenever we take risks, or encounter novelty. From an evolutionary standpoint, it reinforces exploratory behavior.
Video games are full of novelty, risk-taking, reward-anticipation, and exploratory behavior. They’re dopamine-production machines dressed up with joysticks and better graphics. And this is why video games are so addictive.
The past years we’ve heard and read many stories about people being completely lost in a video game in their apartment or an internet cafe, some with very sad results.
Today, “serious gaming” using VR is how we train astronauts, military pilots and surgeons. Why? Because our brains respond to a virtual world in ways that it can be tricked/trained into deepening those responses (treating phantom limb pain with a simple mirror technique is a great example).
So, we have to be seriously cautious with with what we offer with this new tech, e.g. this kid’s VR bedtime stories idea by Samsung is a big no for me. You see, there are many cons to this technology. It’s easy to foresee bad things happening: VR addictions, people starve inside their houses or losing their jobs, getting confused losing the boundaries between the two worlds or preferring the VR from their family and friends, you get the point.
Here are a few more related reads on this:
Keeping It Too Real. Half Of Kids Can’t Tell The Difference Between Virtual Reality And Regular Reality.
Legal Heroin: Is Virtual Reality Our Next Hard Drug. To understand what’s coming with video games, it’s actually helpful to know a bit more about flow.
As Virtual Reality rolls out, will addiction follow? VR, American’s new addiction.
• • •
But, let me be clear on something.
I’m not at all negative regarding the use of this technology, quite the opposite actually. But, I’m always worried about the type of use of this new technology by the majority of humans. You can use a paint brush to create a masterpiece or take someone’s eye out. So, we need to find out not what we can but what we should do with this technology, and it looks like there are some great things we can achieve.
Healthcare
Healthcare industries flying through digital transformation efforts have already begun to experiment with VR as an innovative solution to improve clinical care and even dynamically change the experience of patients during their hospital stay.
Donald Clark wrote on his blog, some months ago, “The thing that excited me the most in these encounters, was not the adrenalin rush of the rollercoaster rides, bungee jumps and horror games but the application of VR in three areas in healthcare:
Therapeutic VRT (Virtual Reality Therapy)
Education VRE (Virtual Reality Education)
Job aids VRJA (Virtual Reality Job Aids)”
Think of the amazing results that can be achieved for depression, with VR affecting changes in consciousness and mood to go from zero to sixty in seconds, PTSD (Post Traumatic Stress Disorder), curing phobias, dealing with them by creating safe exposure environments, stroke therapies, amputations, real-world experiences for physically disabled or memory loss, autism, people returning to society, etc. Or think of education in health with Virtual patients or hearing/visual impairment, field training, etc. Some of these are already happening but now it will get serious.
Virtual Reality is being used to treat heroin addiction. Sometimes the only way to beat your demons is to face them, even if it’s in a simulated reality.
Can Virtual Reality treat addiction? Researchers are plugging in smokers, alcoholics, and even crack addicts to expose them to a relapse environment — and teach them how to deal with it. Will it work?
Virtual reality can help people conquer their phobias. Exposure therapy has proved a highly successful treatment for phobias, but it’s impractical for things such as fear of public speaking or flying. The answer may be virtual reality.
• • •
So, we are in front of a game-changing invention that has finally found its way to the masses after many failed attempts over the last 30 years. The advancements of the past few years have given VR the legs to stand on—and to our surprise, wings to fly. Literally.
Virtual Reality gives us the means to design — not just experiences, but also influence the way we think, change our habits, affect our perception in the real world and so much more.
Design
Designing for Virtual Reality is going to be the next big thing, but it will be really hard and will require very responsible approaches. Building VR worlds and experiences from ground up needs never-stop-learning designers that care about humanity and its future.
It’s not long ago when most designers were crazy about creating really detailed UI elements, with lights, shadows and patterns, that resembled real life items, such as folders, drawers, notepapers, camera lenses etc.
Then, we jumped to flat design which luckily, combined with new tools and various methodologies, let us think more. Now, we are care about butter-smooth User Experiences. Now we talk about Human Centric Design. Now we talk about Empathy.
Speaking of Empathy, let me introduce you to Chris Milk, who uses cutting edge technology to produce astonishing films that delight and enchant.
Milk gave an awesome TED talk on the discovery of the link between empathy and virtual reality.
Fan fact: VR is older than UI. The first virtual reality (VR) and augmented reality (AR) head-mounted display (HMD) system was The Sword of Damocles and was created in 1968. On the other hand, the Xerox Alto was the first device to use a graphical user interface, not before 1973.
VR is a new medium and designing for it, as with all other media, has the same basic Principles, but the technology awareness needed is different; designers may not need to learn code, but we’ll need to try and learn and understand more things, maybe even shift from experience designers to world creators or something. It’s not going to be easy.
“VR technology has taken a big step forward, but the thinking behind VR experiences has not.”
Designing for a flat screen and designing for an immersive environment are two fundamentally different challenges. In this new context we need to think more like a humans, investigate more on using perspective to our advantage, explore the look around model, stop trying to force 2D solutions into a 3D space, think and build to scale.
Affordance is a common term in user experience design that means “a situation where an object’s sensory characteristics intuitively imply its functionality and use“. A simple version of this is can be found on the web. Rollover a text link and the arrow icon should change to a hand icon signifying that something will happen if you click. Rolling over a link with a mouse, trackpad, or stylus does not change the affordance; it stays as a hand. We are conditioned to expect the same behaviour irrespective of input. Virtual Reality will need affordances to indicate what can be interacted with, and when that interaction takes place. The sensory display of those affordances should scale with technology just like screen affordances. A highlight that now occurs on gaze, should still work for hand tracking, micro-gestures or a mind event. All the above but most of all, focus on the experience.
Designing for VR needs a lot of discussion and I’ll come back on the topic, but until then, if you plan to experiment with it (thumbs up!) start with this very interesting ‘VR Manifesto’ by Mike Alger.
““The best VR design will integrate UI into the core experience seamlessly, and take advantage of the new ergonomic capabilities afforded by the platform. Natural VR interfaces will be a part of the environment they create for users, not overlaid on top of it. They’ll make use of flexible space, 360-degree immersion, full-body tracking, depth, audio cues, haptic feedback, and all the other capabilities we’re just figuring out how to make use of.””
Check below for some great links for more great reads on the topic:
3 Tips On Designing For VR, From Google. A Google designer shared what he learned working on Cardboard so that you don’t have to suffer so much.
The Fundamentals of User Experience in Virtual Reality. “Building for VR is not the same as building for desktop or console gaming though…”
VR Interface Design and the Future of Hybrid Reality. Moving from button interaction design to a proof-of-concept VR operating system, he carefully navigates the divide between reality and digital fiction.
What VR Can Teach Us About UX. Virtual reality is the ultimate user experience–after all, you’re literally being immersed in a new world.
Practical VR: A Design Cheat Sheet. Talking about VR is like dancing about architecture. You need the experiential framework to hang that learning on.
• • •
A few days ago after playing with the Oculus I found myself walking on the street talking with a friend about the things we haven’t seen yet in VR and what’s coming next, a guy approached me, leaned a bit and whispered “Jesus loves you”. Then it hit me. We haven’t seen advertising (yet).
Retail (and advertising)
Did you think we’ll avoid this? The first thing that came in my mind was the Holomax Jaws scene from Back to Future. Gaming maybe the first commercialisation of VR, but you’ll know it’s big when you see your first VR ad.
If you don’t believe me, here are some words from people smarter than me:
Your VR experience will begin after this pre-roll. Are conventional digital ads coming to virtual reality? Why the ad industry should resist defiling yet another technology.
Is Monetisation the Next Step for Virtual Reality? How can marketers leverage virtual reality to further their business?
• • •
I’ll leave you with this quote from a Vanessa King’s great piece, “What I wish journalists knew about VR”. VR is a cool new tech that has the potential to change how we socialise, relax and learn.
“The future isn’t scary, but miscommunicating VR is.”