We have an ongoing debate… or maybe it is just a voicing of opinions… in Second Life™ about VR and the Oculus Rift. Will it be a game changer or a minor fad? Perhaps like the Space Navigator. A few people use it, but the majority have ignored it. And when will it arrive? We have no guaranteed answers, but we have some good predictions coming out of the game industry.
In January 2014 at Steam Developer Days Michael Abrash gave a talk about the state of VR and what he expects to happen over the next two years. The video runs about 28 minutes. I cover some of the points from it below as well as some other information.[youtube G-2dQoeqVVo]
We are starting to hear of industries that will support VR headsets ramping up. Companies have leaks and there are some dandies. See: Massive leak regarding Sony VR headset? The competition is heating up.
Of course the thing with leaks is they usually cannot be verified and are often from anonymous sources. If there is anything politicians have taught us, it is anyone quoting an anonymous source is not to be trusted. So, we do need to salt such information pretty heavily. But, we are starting to get more rumors and hear more little leaks. This we call smoke and smoke usually leads or at least alerts us to the fire, the reality.
Maria Korolov has an article up: The race is on. She collects a few bits and pieces together and makes her predictions. Essentially: yes, there will be a boom in VR and… VR work. She recommends people like OpenSim and Second Life residents position to take advantage of the boom now. That is probably a good idea. How long do we have? And what about mobile?
Desktop or Mobile?
We recently saw OnLive demo SL Go. They brought Second Life to mobile devices. It works pretty well. I do suggest you try it. I bought 3 hours of play time (US$8) so I could get a good experience. Try a combat game. I found the lag frustrating for combat. But, for building, shopping, and exploring in a slower paced SL, I think it is pretty great, even if my S4 screen is really too small for it.
If you watched the video of Michael Abrash, above, you heard him talk about mobile devices not having the power to deliver VR. But, what about OnLive? I suspect he is right because OnLive has more lag in the feedback loop than desktops do. Since the Oculus requires very low lag in the feedback loop, OnLive may never be able to deliver Oculus level VR. I’ll explain.
We can see where the lag is in our desktop viewers when the region servers lag. Walking and moving becomes difficult. But, often we can turn in place, spin, without any lag. Walking, to a large extent, requires our viewer talk with the servers. The time to get a command to them, have it processed, and get it back can become unbearable. However, the viewer can allow us to move the camera through the scene whether the servers are lagging or not.
Another place where you can see ‘server free’ movement available in the viewer is in Firestorm’s scene freeze: Freeze Frame. You’l find it int he camera snapshot panel when taking a screen capture/snapshot. The feature takes a 3D frame and freezes it. It accepts no updates from the server and nothing in the frame moves. But, you can move your camera and look at the scene from any camera position. You have no lag in the camera movement, no matter how slowly your connection is performing. It is great for composing still shots.
This concept of holding a 3D frame of the world shows what the viewer needs to render a scene for local head movement.
Since OnLive renders scenes server side and sends a flat 2D image, they cannot provide a freeze frame that will behave like this.
The SL Viewer has zero network lag in this case. OnLive’s viewer will still have to make the trip to the server to get images as you move the camera. You only have 2D images in the local device. You can’t look behind something. The 3D image is in OnLive’s server. While they could freeze your server image and send you 2D images as you move your camera, there will be some lag.
The difference here is in whether your local computer is rendering the 3D scene in real time or you are waiting on the remote server to render the image and get it to you.
The question to whether Oculus/SL can go mobile is dependent on whether OnLive can continue to reduce the time needed for a signal and image loop. I am doubting it. But, they have proven the industry wrong before.
Typical packet travel time for SL/me routes is 60 to 150ms and I have much worse days. My viewer doesn’t have to go to the network during freeze frame. So, that wait is gone. OnLive will have to deal with it. Michael is telling us we have to get the lag down to something like 3ms to maintain a sense of presence and avoid nausea.
At 13:20 in the video Michael starts to talk about Presence and what creates it. Presence in this usage means a sense of being there, having a real sense of being in the room or space you are seeing.
At this stage of the technology the 1080p screens just suffice. The belief is Presence will be increased dramatically when 2k and 4k devices become available. We are just seeing 4k TV’s coming on the market. (Reference UHD = Ultra High Definition)
In the television world the resolutions are; standard definition (480p/540p) , high definition (720p), full high definition (1080i/p) and ultra high definition (2160p – 4Kx2K = 4,096×2,160 pixels). Both my S4 and my desktop run at 1920×1080 – 2mb pixel. The UDH is 9mb pixel resolution. This is about 4 times more graphics processing than current GPU loads. This is a lot of data to move around. That puts mobile devices at a huge disadvantage.
Plus, mobile devices use slow CPU speeds to save battery. The S4 runs full out at 1.9ghz and normally runs much slower. Mine spends most of its time at 384Mhz. The S5’s are up to 2.5Ghz. The top mobile CPU speeds have been about half to 2/3 desktop speed. This year they are just catching up. So, expect VR to be a desktop application for some time.
Mobile CPU speed is likely to catch up to desktop CPU’s late this year or early next year. The Galaxy S5 is a quad core running at 2.5Ghz, which is just a bit slower than my desktop machine. The Adreno 330 GPU supports up to a 2560×2048 display. Still, this is not 64-bit tech. New desktop GPU’s are 256 bit systems. The S5 still runs a 1080×1920 resolution display. So, there are some serious tech obstacles to getting high performing games and VR on mobile devices.
Did you know that Samsung designed the larger screens so they would have room for the larger battery the Galaxy line needs? I can attest that my S4 is a battery hog. I have charger connections scattered all over, office, car, home, lover’s home…
Our display systems place an image on the display by causing chemicals or diodes to glow various colors. To get to the next image the current image needs to clear from the screen. If the previous image takes too long to fade we have two or more images that we are seeing. The effect is blur.
Michael is pointing out that for VR they have found they need 3 milli-seconds (ms) refresh rates or less. The speed with which we can get rid of an image and place another is called refresh rate. Gaming monitors are generally considered to need 8ms or less with less being better. So, we are again touching the edges of technology.
For desktops and TV’s the refresh rate is not that important. A 60hz refresh rate is about 17ms. The 120hz rates are about 8ms. A 3ms refresh rate is 333hz.
If you watch action movies on TV, you often see fast movement blurring the screen image. We have grown used to it. Blurring in a headset is going to contribute to nausea and generally be unacceptable.
At 24:30 Michael talks about what the software will have to do. He points out that initially slow moving games are best for creating Presence. Does this mean Second Life will be a good candidate? Yes.
Initially games will be designed with the limitations of VR in mind. As the VR tech improves so will the games. We have always seen this happen.
Lots of companies are interested in VR and working on VR products. But, realistically we know there is a technology curve. Something looks hot and everyone runs to it. Then the hype dies down and interest wanes in a somewhat negative bounce. Then things stabilize and we can see a steady level of interest that tells us how well the tech is being accepted.
Will VR go through this typical curve? We don’t know, but the probability is high. I carry a Linux computer around with me now. I would never have believed anyone that told me I would 5 years ago. So, what is going to happen with VR and/or augmented reality (AR)?
It really depends on the applications developed for it. But, the true unknown in our discussions is the Presence-factor. People like presence. Game immersion is a goal for any game designer. If presence can improve person to person connection via a digital media, marketing companies will be all over it. I think Facebook’s move is the first major quake in the market. I expect more.
The most disappointing part of Michael’s speech is his expecting to only see VR in retail in 2015. I suspect we will have the 1.0 headsets in 2014. The rumor mill is saying that is Sony’s target date. Oculus was targeting early 2014. So, we may see something this year. But, some player may hold back, wait for additional problems to be found, fix those and try to capture the market with a better device. I suspect that device will be a 2015 release.