In this post, I’ll be trying to answer several questions around HoloLens. Since the initial HoloLens announcement, the HoloLens project has been buttoned up tight, but //build/ 2015 brought some HUGE announcements and closer looks at the HoloLens. I unfortunately did not get a hands on demo of the HoloLens, but I had the opportunity to interview several //build/ attendees that spent 90 minutes with the HoloLens in a very controlled environment. UPDATE: My co-worker just finished a 30-minute hands-on demo, and he was able to confirm everything I’ve written here.
Security around HoloLens has been incredibly tight. In order to get a closer look, a limited number of spots were available to //build/ attendees. Attendees could register for an opportunity to develop applications or get a closer hands-off demo. If you were lucky enough to get into either of the three different HoloLens closer look sessions, you were asked to go to a different building. Attendees I spoke with said they were quickly and quietly escorted through hotels, up stairs, down hallways, into separate buildings, and through multiple checkpoints. If you’re thinking NSA-style security, you’re not too far off.
I had the opportunity to speak with a fellow attendee that was able to develop and use the HoloLens. In that session, he was paired with another developer and a HoloLens expert that guided them through the development process. In their experience, they hardly wrote any code, but instead were quickly pushed through some point and click processes within Unity. Yes. Unity. Within Unity, they pasted code samples into the UI and learned Unity commands. After each change, they uploaded the code to their HoloLens and tried out the changes.
What is the HoloLens Experience
HoloLens straps to a user’s head via a plastic strap. You do have the ability to tighten and loosen the strap, but in order to align the unit properly to your head and field of vision, you must have it tightened reasonably well.
If anyone wears prescription glasses, you’re familiar with the importance of measuring the location of your pupils relative to your glasses frames and lenses. If you don’t get hits just right, your glasses won’t perform optimally, and in the worst case, you can get a sick feeling. The same concept applies to HoloLens – it’s fully customizable; however, the session participants did not have the opportunity to adjust the HoloLens units. Instead, their eyes were pre-measured and they were paired with a unit that was already configured and adjusted for their specific “magic number”.
Image quality shown in the videos and on the stage at //build/ seem almost unbelievable. How can you get an image that rich and detailed and in 3D right there. Is this real? Yes. The attendees that got to use the HoloLens said the resolution was reasonably high and the colors and images were extremely vivid. In fact, they were so strong and vivid that they could not see their hand waving in front of them when the hologram was displayed on the lens.
Users of the HoloLens also noted that when they placed a virtual hologram on the floor, it stayed in place amazingly well. In order to do this, head tracking technology is being used to identify how your head turns, rotates, and inclines.
Although the view through the HoloLens is reasonably high quality, but the field of vision on HoloLens is a bit misleading. The viewable area covers approximately 60% of the front visor – it does not stretch over the entire visor. Although this is disappointing, the technology is young.
Voice recognition and 3D audio is a major part of the HoloLens. Per users, voice recognition is done very well, and the 3D audio feedback (see the red audio bars on the HoloLens) really helps to add to the presence that you experience. The 3D audio is used for several purposes, one of which is to recognize and recall where holograms exist in a room. If you place a hologram and stick it in place, when you turn away, you can receive audio feedback as to where the hologram exists in a 3D space. This enables to move around freely and quickly turn to get focus back on a hologram. You can also use voice commands to quickly recall a hologram into you field of vision.
Questions and Answers from the HoloLens Panel
//build/ had a HoloLens panel discussion, severla questions were answered, and I’d like to share some of the questions and answers.
What are the Challenges with Developing for HoloLens?
When the JPL was starting off, it WAS challenging. The JPL lab had been working with VR and AR technologies; however, they were trying to apply the HoloLens technology was different. their first instincts with how to interact with the HoloLens were completely wrong. When they implemented their ideas, they found their intuition was wrong. It didn’t work as well as they had thought on paper. The first ideas and days using the HoloLens was discouraging.
Designing and developing for the UI is different. This is very iterative. It requires people to completely re-imagine how to interact with the HoloLens. It is a very creative process. There is also a need to “fail fast”.
Describe the Development Experience
The first time the developers started working with HoloLens, they had a cube sitting in front of them on the HoloLens within 5 minutes. The hard math of finding surfaces, slving for planes and angles wasn’t there. Most of the time working with the HoloLens was taking the data and interpreting it for the right fashion. Per the developers, the SDK is fantastic. All of the hard stuff is done.
What Surprised You the Most About HoloLens?
The JPL team noted that they were most surprised by how amazing it was to witness someone be brought to tears when putting the HoloLens on the first time. For JPL, it is an overwhelming moment for someone to be standing on Mars. The second surprise was around UI and interaction. It was unexpected that when using the HoloLens and a mouse and keyboard at once, they kept trying to move their mouse off of the screen.
From a graphic capability, it was amazing how it’s difficult to tell what is real what is holographs – in fact moving our house over both objects in your field of vision is amazing.
The architecture company indicated there was a disadvantage to using HoloLens – you can’t touch your virtual entities (today).
As a Developer, what Tools do I Need?
Unity and C#. But you can also leverage other APIs you’ve already built to reuse existing code.
How do you Interact with Others via HoloLens?
One of the panel members from Microsoft indicated that Skype calls on HoloLens are fully supported, and there will be some announcements over the next few months showcases specific Skype interactions that enable people with and without HoloLens to collaborate and share an experience.
What can we do to get Ready?
The best way to stay on top of this is to be part of the Windows Insider Program. News is coming, so join the program. To be prepared for HoloLens, you should get ready by working on mobile 3D application development.
Summary
If Microsoft nails the HoloLens experience, and it truly is what they’ve been advertising, there is an enormous potential for HoloLens to change the world and change the way in which we interact with each other. We have previously seen our society move from an analog to a digital society, but HoloLens, AR, and VR are a different way to visualize and interact with the digital world.
