It’s been a while. Hope you had a good time last time we spoke. Two years ago, we spoke about virtual reality.We talked about the concept of what it meant to experience virtual reality through technology, focusing on a particular project called the Oculus Rift. At the time, it was in a developmental stage.Fast foward two more years and virtual reality is now a real reality.
If you remember from the last post about Pokémon GO, one of the lasting impressions was promoting the idea of augmented reality to a wider audience through people playing the game. The augmented reality aspects of the game was that it ‘placed’ pokémon into real-world locations when using a smartphone with GPS. If you happened to be looking at a building through your smartphone, the game would sitatued a pokemon at that location. Think of it this way: computer-generated images (CGI) are clearly not real but made to look or feel as authentic as possible. Think of Star Wars: The Force Awakens and how much CGI and other special effects went into that to make characters and scenes as believable as possible.
One issue I have to raise is that in the last post I’ve used both terms of augmented reality and virtual reality, incorrectly thinking they meant the same thing. They are similar but different, so I apologise if I’ve misled anyone. From now on, this post will refer to the technology as VR. So what is the difference?
In AR, users are able to interact with virtual contents in the real world and tell the difference between the two. In VR everything is virtual (in the sense that the user has no connect with the location they are in virtually). Pokémon GO is a classic example of the former, where virtual pokémon are placed in real-life locations. This is not the case with VR where everything is an illusion. However, let’s not forget that both terms are changing the way we see things from a technological perspective.
The main point of the 2014 post was discussing the Oculus Rift as the case study. This project was born out of a desire to see an affordable and better experience of AR. The Rift was essentially a headset (or head-mounted display) that useshighly-sensitive LEDs and external sensors to create a virtual experience. Since 2013, it was still in development. But now in 2016, we are receiving the first lot VR kits to be available on the market. But first, let’s have a revisit of what the Rift was.
Revisiting the Oculus Rift
Californian Palmer Luckey had a keen interest in virtual reality, stemming from his hobby fiddling around with several electronic projects such as lasers and coil guns in his garage. His interest allowed him to steadily build a collection of many head-mounted displays. The first concept of the Oculus Rift came about due to Luckey being become frustrated with the then-current head-mounted displays in his personal collection. He felt they were generally of poor quality and he could do better. ‘Virtual-world sci-fi like The Matrix and the anime show Yu-Gi-Oh! intensified the desire. Why, he asked himself, can’t we do that yet? His modding and iPhone repair work had left him with a lot of money, so he bought a $400 Vuzix iWear VR920, then the most cutting-edge consumer VR headset – enthusiasts call them HMDs, for head-mounted displays – on the market. Then he moved on to the more expensive eMagin Z800 3DVisor. And he kept looking’ (Rubin, 2014).
In order to realise his project, Luckey started a Kickstarter campaign, although he also supported himself by doing various jobs such as iPhone repair work. He regularly posted his reports on MTBS3D (MTBS standing for ‘meant to be seen’), a forum website that was used by fellow VR enthusiasts. One of those regulars happened to be John Carmack of id Software, creators of the popular Doom games. Carmack read with interest about Luckey’s project, ordered one of the prototypes, and showcased it at the Electronic Entertainment Expo (E3) 2012 using a modified version of Doom 3 BFG Edition.
The demonstration to a larger, enthusiastic audience was a pivotal moment in Luckey’s project. The Oculus Rift suddenly found a large audience that were as keen as he was, and he dropped out of university. The Kickstarter campaign eventually raised $2.5 million, an outstanding raise from the initial $250,000. This allowed him to start a new company called Oculus VR. Joining him was Brendan Iribe (as CEO), fellow VR enthusiast and former executive of Gaikai and Scaleform, Michael Antonov as Chief Software Archiect, as well as Michael Abrash (Chief Scientist). John Carmack also joined in some capacity.
The Rift itself has gone through numerous development stages through research and development. Initially, the first stages revolved around DIY kits for interested developers from the Kickstarter campaign. The Development Kit 1 (DK1) was given to backers as a ‘thank you’ to those who invested $300+ into the project in its early days. The idea was to give developers a chance to integrate their content in time for the Rift’s release. The current ‘stage’ is called the Crescent Bay.
How did the Rift work?
As the term implies, virtual reality was to create a truly virtual, but immersive, experience that was designed to trick the brain into being immersed, forgetting about the technology that went into creating it. This approach used several key factors to achieve this. As Sophie Charara (2016) explains, these typically included ‘a PC, console or smartphone to run the app or game, a headset which secures a display in front of your eyes (which could be the phone’s display) and some kind of input – head tracking, controllers, hand tracking, voice, on-device buttons or trackpads.’
1. The Headset
The Rift resembes an oversized goggle, referred to as a head-mounted display (HMD). Charara (2016) explains, ‘VR headsets use either two feeds sent to one display or two LCD displays, one per eye. There are also lenses which are placed between your eyes and the pixels. These lenses focus and reshape the picture for each eye and create a stereoscopic 3D image by angling the two 2D images to mimic how each of our two eyes views the world ever-so-slightly differently. One important way VR headsets can increase immersion is to increase the field of view, i.e. how wide the picture is. A 360 degree display would be too expensive and unnecessary. Most high-end headsets make do with 100 or 110 degree field of view which is wide enough to do the trick. And for the resulting picture to be at all convincing, a minimum frame rate of around 60 frames per second is needed to avoid stuttering or users feeling sick. The current crop of VR headsets go way beyond this – Oculus is capable of 90fps, for instance, Sony’s PlayStation VR manages 120fps.’
Let us take a moment to understand what she means by frame rate because it is quite important. On imagining devices like TVs or cameras, the frame rate is the frequencyin which they display consecutive frames (images). This is measured in seconds (frames-per-second or fps).When you watch a TV, how ‘smooth’ the images appear are a direct consequence of the frame rate employed. To put it into perspective, a low frame rate would give you choppy images.
Tarantola (2014) explains, ‘the human eye is capable of differentiating between 10 and 12 still images per second before it starts just seeing it as motion. That is, at an fps of 12 or less, your brain can tell that its just a bunch of still images in rapid succession, not a seamless animation. Once the frame rate gets up to around 18 to 26fps, the motion effect actually takes effect and your brain is fooled into thinking that these individual images are actually a moving scene.’ He further states, ‘so if a frame rate is too slow, motion looks jagged, but if it’s too fast you can have problems too. Live-action movies filmed at 48fps tend to have that certain soap-opera effect people hated in The Hobbit.‘ The Hobbit was criticised for being overly ‘hyper real’ to the points ome critics pointed that they could see the background sets such as make-up on actors and painted sets.
In the human visual system, it is thought that we are able to process 1000 seperate images per second but that would be an extreme to implment. The trick is to find a balance that makes the visuals smooth and not look like they will make people uncomfortable. In current videogames, 60fps seems to be the standard. I’ve seen two comparable videos of games at both 30fps and 60fps. Some people say that can’t see any difference, but I noticed that the 60fps version transistions much smoother and therefore the visual experience is more enjoyable. However, it seems the Rift goes beyond 60fps so the visuals so be that much smoother.
2. Motion Tracking
So you wear the headset but how to you input the data received into something quantifiable? Input is important as much as output is. For example, if you use a computer, you use a keyboard as an input device, which then translates into quantifiable data/responses (i.e. keystrokes). For the Rift, there is a compatible piece of technology called Oculus Touch, which is basically a wireless controller that resembles a gamepad attached to a strap. The idea is that becaase you physically hold something, allowing you to make hand gestures that are translated into the virtual expereince, thus giving you an input response.
Importantly, the Rift would need additional input devices in order for it to be able to track a user’s postion (as in motion tracking). Sensors would be the best solution as they needn’t be large.On CGI-heavy films like Avatar, actors would wear special suits that were attached with multiple sensors. Sensitive cameras placed around the set would capture their postions (including facial expressions) through the sensors, which would then be mapped into software. The film-makers would then do their special effects to the scene using the data.
The Rift worked in a similar way. On the actual headset, there are a series of infrared LEDs built in. These sensors then communicate with another input device, a wireless sensor. It looks like a small microphone-shaped pole but is designed to speak with the headset. Oculus calls this the Constellation Tracking System. The Rift has LEDs on most of its side, allowing for a full 360 degree rotation.
The communication between sensor and LEDs is accurate, pin-pointing precisely the position of the user. This is done by knowing the configuration of the LEDs; the information is then trtansferred with sub0millimeter accuracy and with almost zero lag (or latency). The Constellation can either use a single sensor or multiple ones placed around the area. By employing mutliple sensors at different angles, the system is able to track the entire room.
The VR Future?
In the last post, I talked briefly about what the future holds for this technology and I said only time would tell. Well, time certainly did tell in which the industry is taking off. We not only have Oculus Rift, but other brands on the market with technology heavy-hitters such as Samsung (Gear VR), Sony (Playstation VR/Morpheus), and HTC (Vive) getting in on the act. There was already an enthusiastic response (at least from a techological view) when the Rift was demonstarted at E3 so that should be an indication of its potential.
Even though we are seeing a lot of industry activity, that does not mean the market is ready. In order for VR technology to be truly successful, it would need transition from niche to mainstream markets. It has certainly grown in prominance; as I said earlier there are VR kits now to buy. But I wouldn’t say it has peaked yet. There are several issues at the moment that I think are holding it back.
One of the immediate issues is the high cost for such devices. The Rift is several hundred pounds so already many people will be priced out.This doens’t mean they still can’t experience VR. For people who want to experience VR on a more affordable level, there exists several cheaper versions ranging from £10-20. Most of these use a smartphone in order to create the experience. Many of the cheaper versions consists of nothing more than a cardboard base and some lenses. You place your smartphone into some holding slots, which you then look through with the lens in the comfrot of your own home. I’ve happened to try one of these cheaper models and I was surprised at how immersive it was. Looking through nothing more than lenses, I was actually turning my head/body around when playing a game. You forget somewhat that you’re in a room and lose yourself in the VR experience, and that’s what this is all about.
Another reason is the high cost in processing power. Not only do you need to have the necessary input devices, you also needed a beefy comptuer system to run the whole thing. ‘Chipset maker Nvidia predicted last December that in 2016 only 13 million PCs will be powerful enough to run VR, meaning that less than 1% of the 1.43 billion PCs in use globally this year are up to scratch. This explains the slow start’ (Rossi, 2016). Recntly, Oculus VR has issued statements about their ‘Asynchronous Spacewarp’. The ASW is a technique that allows VR titles to run at aorund half the processing power by ‘extrapolating frames’; in other words, it is a frame rate smoothing technique halving the CPU/GPU time required to produce nearly the same output from the same content. The idea is that the minimum system specifics of the Rift are lowered, allowing more people with less powerful computer systems to use it.
There’s no doubt that some of these issues will be corrected in the future. But these will most likely take several years at the very least. Remember that it was only three years ago that the project really started yet the potential for rapid growth is projected. ‘When Oculus Rift launched its $2.4 million Kickstarter crowdfunding campaign in 2013, it was billed as the first truly immersive virtual reality headset for video games. Expectations such as this have allowed potential virtual reality to grow quickly, with revenues from both VR hardware and software products projected to be a formidable $5.2 billion USD in 2018. At the same time, the number of users adopting the new tech, predominantly gamers, is expected to reach 171 million. At present, 43 million people worldwide own a VR headset, so ownership could rise more than three fold in just two years, even before the predicted peak in 2021’ (Rossi, 2016).
It is certainly a clearer picture than from 2014. There is a market, there is potential and there is money to be made. I wrote this post because we are seeing VR start to become more available. 2016 was supposed to be when VR really took off; while we’re seeing several companies developing and releasing their own models, I wouldn’t say it has peaked yet. Certainly the technology needs more refining but I think it’s safe to say it’s a question of when, not if, it will become a major thing.
Sophie Charara’s article, ‘Explained: How does VR actually work?’
Ben Rossi’s article, ‘The Future of Virtual Reality’
Peter Rubin’s article ‘The Inside Story of Oculus Rift and How Virtual Reality Became Reality’
Andrew Tarantola’s article, ‘Why Frame Rate matters’