Various Realities

Most of us know virtual reality (VR) is a completely fake, computer generated space in which we can interact. Augmented reality (AR) is the term used for computer stuff on top of real space. It typically moves around with the user. Think of Google Glass, where you have an interface that's always at the same distance, and in front of you, no matter where you go. Mixed reality (MR) is a new term to replace AR that classically used markers to anchor items in space around you.

Got our acronyms down? Good. Now that we know what these things are, I'm going to ignore VR and AR, and share the potential of mixed reality, because exciting!

Productivity

Yes, mixed reality has great potential to be awesome, but we have yet to see anything that makes Minority Report seem like it's going to happen soon. The interactions for everyday and office use are so varied that until we come up with some kind of interaction system as killer as the mouse, there's a long way to go (I may or may not have some ideas on that though ;) This is Magic Leap's concept of productivity usage so far, which just makes me go "meh." I know they've got to be thinking bigger and better than this.

Gaming

This is the easiest use case to see. Games will happen in your environment around you. One Microsoft crime mystery game makes you the detective, and hides clues for you around your home. You could find a critical thumbprint on the kitchen counter, or a hidden clue under the coffee table. Or you're sitting on your couch next to a game character you're talking to.

More sweet videos at the bottom of this post

Education

Am I more excited about the educational, or the gaming aspect of MR? What I'm most excited about is the convergence of both. One of my longtime dreams has been to make learning so engaging and interactive that it's nearly indistinguishable from gaming, and something that makes you want to learn.

• Students could read books with QR codes on pages that bring up 3D movable models of items related to the subject matter! Maybe it's a 3D or 2D video that illustrates the concept being learned.
• Interactive grammar lessons where you choose the correct suffix or spellings.
• Immediate feedback for right and wrong answers!
• Practice safely taking apart an engine, before working on the real thing
• Better yet, overlay parts of the engine and easily see what you should stay away from (in red), and parts (in green) that you should work with.
• Content for more complex and memorable learning

Everyone learns better by immersion. MR creates an immersive learning experience that is both visual and auditory. Now if only we could find some tactile immersion! Oh wait, I think I have some ideas for that, too!

HoloLens

Running at 240 frames per second (good games run at 60, video at 30), HoloLens uses Kinect-like infra-red lasers to map the room around you. From there, it can project images into the lenses in front of your eyes based upon its gathered spatial data.

It runs on Windows 10, so it's already compatible with Windows 10 applications. I know, you're just dying to use Excel in real space. HoloLens has HUGE potential though. Coupled with Project Spark, this could be an unbeatable platform for content creation, communication, and game development.

Limitations   HoloLens currently has a very limited interactive field of view that does not immerse the viewer's eyesight entirely.

Magic Leap

This is a very secretive company building a competitor product to the HoloLens. In a recent interview with Wired they only described their version as less bulky than the HoloLens, and "as comfortable as slipping off sunglasses." We've only seen a couple of concept videos, so there's not much to go on, but Wired seemed to indicate that it surpasses HoloLens in its believability. Magic Leap's product also does not seem to have the limited interactive area that HoloLens has.

Differences from HoloLens   Magic Leap won't run Windows 10, they have to develop their own operating system. That said, it will lose lots of productivity apps, and any applications that HoloLens will have out of the box. However, with Google as a backer, will they simply ride off of the free Google productivity apps? Will it run some flavor of Android to increase its market capability?

Hurdles

With VR, it's very easy to lose spatial awareness. That's much less so in MR because it's not an entirely fake area surrounding your vision, preventing you from running into furniture, or tricking you into thinking you can lean on a table that's not really there.

• MR is not like Kinect, you couldn't play Dance Central with it and have the system know what your legs are doing.
• It will take time to build usable content for MR.
• Gesture improvement
• Body/limb tracking
• Bulky, no fast moves
• When will it connect to my mobile device?
• When will it become my mobile device?
• How will we socially abuse this tech?

VR is really just a stepping stone into MR. I'm excited about VR because it's coming first, but it's a mere stepping stone on the way to greater things. What are your thoughts and ideas?

---

---

Check out this HoloLens Minecraft Demo to see more potential (skip to 1m48s).

More MR video links
The Verge tries HoloLens
C|Net tries HoloLens
Ars Technica tries HoloLens