After years of false starts and incremental improvements, 2016 is the year where virtual reality (VR) has finally arrived. Thanks to a combination of advancements in hardware and software, an outpouring of investment, and a massive spike in business and consumer interest, we believe that virtual reality has hit an inflection point that will propel us from the mobile era to the virtual reality/ augmented reality era. In this post we discuss which existing processes continue to work with VR as well as the areas where change and innovation is required.
Investment and interest in virtual reality
Investment in VR (and augmented reality) is pouring in. According to Digi-Capital, $1.1 billion in investment occurred in Q1 2016 alone. More recently, it forecasted revenues of $120 billion for AR/VR by 2020. At the same time, companies such as Facebook (through Oculus), Google, HTC, and Samsung have already produced and sold hundreds of thousands of VR headsets. A few months down the road, Google Daydream (which we expect to be part of every high-end Android phone in a year), Sony Playstation VR, and the mysterious Magic Leap will continue to propel VR into the mainstream.
While gaming and entertainment are the first obvious applications of VR, the more interesting applications – and the ones we focus on here – are in B2B and B2C products and services. Brands such as McDonald’s, Volvo, Coca Cola, Marriott Hotels, and even President Obama, have successfully released VR content to consumers, with many more scrambling to join them. Airbus, Ford, and IA Interior Architects have all publicly invested in VR for business products.
These business applications include everything from education, e-commerce, travel, communications, healthcare, job training, and data visualization, to almost anything else you can imagine. For example, the Stanford Human Interaction Lab is exploring the ways VR creates empathy for various social groups; a project that will no doubt replace the HR training of today. NASA is using VR to train astronauts and technicians as well as for planning missions and simulating life on Mars. These types of applications may ultimately account for most VR investment.
Building non-gaming VR experiences
As we look at what it takes to develop non-gaming VR products, their separation from gaming becomes very important. Many video game studios have well established processes for working with 3D and immersive audio experiences – the cornerstones of VR experiences. They are able to start with their existing approach, and make adjustments for VR-specific challenges like locomotion and frame rate consistency within the experience. They are not changing their approach so much as working through a new presentation of their existing products.
Developing business products in VR, however, is like creating a new sport by combining American Football and what the rest of the world calls Football. They both take place on a field and there are two teams trying to score points with a ball, but outside these similarities it isn’t entirely clear where to begin.
While VR feels like a natural transition for the gaming world, the fundamental question of why create business products with VR is still being debated. In some cases it provides direct and clear value. VR is an ideal technology for training and simulations. Just consider the history of flight simulators, a multi-billion dollar industry in its own right, applied to thousands of industries at a fraction of the cost. On the other hand, what will it mean to use a VR environment as a fundamental communication mechanism? Will we have our email, Slack, and text messages simulacra or find that businesses begin to run on messaging systems native to a virtual environment?
For now we are content to acknowledge that in some cases, there is clear business value in applying VR, while in other cases, we have no idea. What is clear, however, is that we are blending two very different worlds. What we consider to be traditional application developers have become exceptionally good at defining products with static wireframes and producing topnotch web and mobile-based products. Video game developers have likewise perfected the art of creating massive 3D worlds and telling compelling stories to keep people engaged in those worlds.
It’s clear we will need help from both sides of the table.
The song remains the same
Some things will not change. VR applications grounded in delivering business value operate like web, mobile, and connected device products. They deliver value to a user who is willing to pay either money or attention. In Clayton Christensen speak, the user has hired the product for a job. Novelty might be a job they hire a VR product for, but can we base an industry shift on novelty? Product fundamentals remain if you are building a business around your product, so most of the tools we use to uncover product value are still critical.
The underlying approach to managing a project, such as agile, works fine. Communication tools like Slack, still fantastic. But when you move just beyond that, when you look at the content of a user story, when you look at the conversations in Slack, you realise that VR products introduce complexity that is completely foreign to traditional digital application developers.
As ever the devil is in the detail. On web and mobile, we can use 2D wireframes to explain most of what a product is going to do and operate in a generally linear and predictable fashion. In VR, partial representations of a dynamic environment that at best provide one perspective of the experience at a moment in time just don’t cut it. And don’t forget, this product needs to leverage and extend an existing design language and technical architecture. One more thing, audio is really important for immersion.
As soon as we move to detailed definition and development of the product things change quickly. We can borrow from both our experiences in traditional products and 3D gaming products, but neither is definitive in terms of navigating a VR product that is focused on delivering value to an end user.
So what does all of this mean for product development?
The times they are a-changin’
While the fundamental skills of product development still apply, we’re finding that a number of issues which can be ignored in web and mobile development become major considerations in VR. Some of the issues you’ll need to consider are:
- Immersion – The power of VR lies in its ability to fully immerse a user in an environment. A good experience aims to convey emotion and uses timing, space, lighting, sound, and interactions along with more traditional content to create a sense of presence and transport them into the world you create. While directors and game designers may be used to considering all these dimensions, for the rest of us this takes some getting used to.
As an example, imagine a submit button. On the web, you only need to define how it looks and works by default, on press, on hover, and when disabled. That same button in VR has the same elements but now you also need to think of how it sounds, how it’s lit, how it should interact with different other elements in a scene, and maybe even how it feels if you’re dealing with haptics. Additionally, you’ll want to support different interaction models so the button may need to be triggered by gazing at it, pointing a laser pointer at it, or simply by walking up and touching it.
Adding these dimensions also takes more in the way of expert resources. A single scene might be approached from many different angles. The sound needs to be spatial. The lighting must create an appropriate emotional response. These don’t come for free and aren’t the responsibility of a traditional user experience expert.
At Presence we’ve borrowed from the role of a producer in the video game industry to deliver this level of immersion. We have one person who is responsible for the overall experience. They understand the expertise required to deliver the entire product and are more focused on creating cohesion across disciplines than delivering assets. Outside of creating a dedicated role, we have found UX designers and product managers both tend to naturally fill these roles.
- Performance – VR works best at 90 frames per second (FPS) with 60 FPS on mobile being acceptable for now. It turns out that these frame rates are really hard to achieve in an immersive environment. Messing this up is not an option. It not only leads to a bad experience but it can also mean dizzy, motion sick users. Those users, as a rule, are not going to stick around.
You need a technical expert. Not the technical expert that ramped up over the last six months. You need the technical expert that has earned their expertise over years. Similarly on the design side, you will need your design team to have experience in crafting 3D models that not only look good but perform well. Polygon count, effective use of textures, appropriate lighting, and managing particle use is key here.
We’ve been pairing design and engineering from the traditional web and mobile product development world with experts in the 3D space, most often from the gaming and entertainment industries. This is all about making sure we deliver product value while also taking into account the hard-won understanding of what it takes to create powerful VR experiences.
- Platforms – If you thought the browser wars were bad, wait until you see the VR wars. Not only does each headset have its own APIs and performance differences, but they also come with varying interaction models, capabilities such as room scale and motion tracking, as well as distribution methods. Plus, WebVR will likely mean you’ll still have web compatibility issues to deal with.
Currently, we see VR broken up into three “classes” of platform. The first class consists of high-end rigs like the Oculus Rift, HTC Vive and the soon to be released PlayStation VR. These are expensive, require beefy PCs (or consoles) with cutting-edge graphics cards, and are capable of delivering the best overall VR experiences. While they are in the same class, only the Vive currently targets full room-scale experiences where the Rift and the PSVR are currently seated only. Additionally, each has its own unique controller and set of inputs.
Next we see the mobile platforms such as the GearVR and Google Daydream. While the Gear has enabled people to break into VR for only $100, we expect Android phones that support Daydream to be the first platform to bring VR to the masses. Any Android phone with the right spec will be able to support the inexpensive yet powerful Daydream headset and controller. While users won’t get the same experience here as on the PC-connected devices, the huge number of devices and Google’s backing makes this an exciting space to watch.
Finally, we have WebVR. While this will work through any of the headsets mentioned above, the browser will be responsible for most of the heavy lifting. Google, Firefox, and Microsoft Edge have introduced experimental support for VR but the WebVR spec is still some way from being ready for prime time. However, with the open and pervasive nature of the web, there’s no doubt that the web will eventually be a major part of virtual reality.
Our current approach to platform support is to start with the project goals. For more passive experiences, something like Google Cardboard or GearVR with its limited controls can be ideal due to its larger user base. Meanwhile, meatier interactive experiences or ones that rely on complex interactions are better suited to the Vive or the Oculus Rift.
Next, we’ll typically select a single platform as our initial target. This allows us to iterate quickly, decrease QA overhead, and focus on making a really polished MVP. Once we’ve validated the product, and have the development cycles to spare, we then move into porting the experience to other headsets. This isn’t as difficult as it sounds, assuming good programming practices, thanks to most game engines’ ability to make builds for any of the VR headsets. At that point, it’s mostly a matter of optimizing performance for different device specifications, handling different interactions, and lots of testing.
- Team Composition – Creating a polished VR experience takes a small army of talents. You need developers skilled in a 3D engine like Unity or Unreal, designers with skill sets ranging from UX and UI, technical artists skilled in 3D modelling, animations, particles, and lighting, and also QA experts who understand how best to test the huge possibility space. You even need sound engineers to create a fully immersive experience.
To get the performance and level of detail necessary to produce high-quality experiences, all these people need to have spent years understanding the intricacies of 3D development. The trick here is to start with a core team of technical experts and designers and rely on their expertise to bring in other skillsets at the correct time. For instance, you may not need an audio person from day one but you need to have a good enough understanding of your timeline to be able to pull someone in when you need them.
Of course, you’ll also still need competent back-end engineers if you’re going to be storing user data or doing anything social or server side. All told, expect team sizes to be at least twice as large as your typical web/mobile product.
- Experimentation – The patterns and best practices for VR are several years from crystallizing and what works well on paper may be a total failure in VR. Many times your developers won’t even know what’s possible without actually trying it. For now, the only way to be successful is rapid experimentation of concepts and closer interactions between development, design, and customers. That’s good news for lean/agile adherents but not so good if you’re worried about tight deadlines or basing success on a waterfall process.
For us the most successful rhythm of experimentation has been the one week prototype sprint. Start with a prioritized list of experiments and then have developers and designers spend one week per experiment. At the end of the week, demo the idea to the larger team, run user feedback studies, revise your experiments, and repeat.
The key with these experiments is to explore the key mechanics of the experience you want to build. Don’t worry about code quality, ignore good graphics, and attempt to distill the idea into its core essence. Always ask yourself, “What’s the smallest, simplest version of this idea we can build to test our hypothesis?”.
Once you’ve run a number of these experiments, your team will get an idea of what types of experiences actually meet your goals and which fall flat. You’re likely to need to do a number of these sprints to truly understand your problem space, but you also need to make sure you don’t get carried away.
- Costs – Producing high-quality content can be an expensive endeavour (from team composition alone). Fortunately, there are some techniques you can use to decrease those costs at the expense of some immersion.
360º video is the entry point of choice for many companies. It’s generally the cheapest to produce and can be immersive if done right. Additionally, with the introduction of stereoscopic cameras, you can add a sense of depth to an experience lacking in regular 360. Unfortunately, this type of content creates a passive user experience and will never be able to create the sense of presence and immersion that a full VR experience can.
At the next level, we’re seeing interactive 360º videos. They’re almost identical to regular 360º content but a layer of interactivity is added on top of the video to create a “choose your own adventure” type of navigational experience or to provide ways to get more information about something happening in the video.
Full VR experiences tend to be more expensive to create but also provide the most immersive and interactive experiences. These are the current gold-standard of VR and are the reason for its recent success. The cost for this type of development should begin to go down as new technologies and techniques are pioneered. Particularly, photogrammetry and LiDAR based technologies promise a future where photorealistic 3D models can be attained affordably.
Creating 360 video experiences, while still intensive, is not as risky as developing a fully immersive experience. As a result, we’ve tailored our approach depending on the type of VR experience we have set out to create. As the complexity of the project increases, we increase the amount of experimentation. On paper it looks like this may bloat the cost of a project but in reality it eliminates a huge amount of risk and cuts down on aimless spend. There is nothing as costly as an abandoned project.
The possibilities for VR are nearly infinite but so are the challenges. For many organizations, limited budgets, lack of experienced people, and the need to build such a large and diverse team in order to deliver high quality experiences is a major limiting factor. However, the same was said about early web technologies and look where we are now.
As product people, it’s our job to be looking over the horizon for what comes next and preparing our offerings and our teams for that new reality. At Presence, we feel strongly that VR is that next big thing and it’s up to us to help create ways for our clients to be successful in this new medium.
The good news is that building a good product still comes back to the fundamentals that we’re all experienced with. The exciting news is that it’s going to take a lot of smart people creating a lot of amazing (and mediocre) experiences to pioneer brand new processes and techniques to fit these emerging challenges. We hope you’ll join us in creating the future of VR and we’d love to hear what you think.