Hands-on review: Updated: Microsoft HoloLens

Microsoft HoloLens at Build 2015

Note: Microsoft didn’t allow any technology, including cameras, inside the Build 2015 HoloLens demo room. The images below were taken of a HoloLens inside a glass case located outside the demo room.

HoloLens, Microsoft’s AR viewer, feels like the future of computing.

The headgear I tried at Build 2015 is still described as “early development hardware,” and it definitely felt like it. But the potential and how close HoloLens is to achieving it is remarkable.

The moment I tried on HoloLens, I thought, “This is like having a PC on my face.” It’s not that fully functional yet, but that’s how the headgear and what you see and can do with it make you feel.

There were no Mars missions or Minecraft-inspired games in the HoloLens session I attended. Instead, I became a developer for 90 minutes, crafting an application in Unity and adding layers of HoloLens functionality as I went. Every time a new function was added, like gesture controls and spatial sound, I got to see how it translated into the HoloLens experience. The session was intended to show how easy it is to develop for HoloLens, but it also demonstrated what users will experience once it’s available.

How HoloLens fits

HoloLens is essentially made of two rings, a thicker one along the outside that contains all the important innards and a slimmer one on the inside that wraps around your head. The inner ring has a roller in the back to tighten and loosen it, and it slides forward and backward so you can adjust how close HoloLens is to your head. HoloLens isn’t supposed to sit on your nose, but I found the rubber nose guard that comes with it inevitably fell on my nose no matter how many times I pried HoloLens forward. Thankfully, it’s optional and comes off easily. HoloLens felt a lot better for me with it off.

I also had a hard time getting HoloLens to fit just right every time I put it on. I had to regularly re-tighten, re-situate, and realign the headgear. When everything was synced up and it fit nicely, the holograms were in full view and it felt right. But if it was too tight, too high up or too far forward, it hindered the experience. Standing in one spot and not moving too much created the best overall viewing experience. The adjustment issues cropped up especially when I was moving around, which is part of the fun when wearing HoloLens.

If you have short hair or it’s pulled back, you might not have a much of a problem as someone with long, loose hair, like myself. It may have been my unskilled adjusting abilities, but I had a slight headache whenever I took HoloLens off, like I had been wearing a baseball cap that was two sizes too small.

The headgear I used was untethered and I didn’t need to use my hands for anything other than selecting my hologram to move it. It wasn’t wired up for battery life like the first early prototypes shown to press.

Fitting issues aside, when HoloLens fits right, it’s comfortable. But like all VR and AR headgear, it’s weight is front loaded. You can’t help but feel like there’s a noticeable amount of weight hanging off your forehead. It wasn’t uncomfortably heavy, but it was significant enough. If Microsoft can balance out the design and somehow put more weight on the sides or back, it would likely help alleviate the front-heavy sensation.

There’s a soft cushion lining the inner ring that touches that I appreciated. It’s a small touch, but one that makes it easier to forget you’re wearing the viewer and focus on the holograms in front of you.

I wear glasses, and I used HoloLens with them on. My glasses didn’t get pressed into my face or feel tight around my head, which has happened when I’ve worn VR headsets. I also didn’t get nauseous, a frequent occurance when I wear Oculus Rift. It helps that I could still see the world around me while wearing HoloLens, so I didn’t get disoriented or feel claustrophobic. It would be nice if the HoloLens see-through screen wasn’t so dark, but the room was also dimly lit so it may be the right amount of shading in a brighter room or outdoors. I felt most uneasy walking backward as I couldn’t quickly swivel my head to see if something was behind me. The headgear also obstruction your upper peripheral view, so some of your vision is obscured.

HoloLens looks and feels like a premium peripheral. Nothing about it screams “cheap” in materials or design, and that will likely be reflected in the price. HoloLens looks like it belongs in the office and would blend well in any living room. As is, it felt too flimsy to stay clean and unscathed in someplace like a construction zone. I found myself handling it gently, so unless Microsoft does some strengthening, you’ll probably want to keep HoloLens out of the reach of children.

Holographic viewing

The hologram that was part of my HoloLens experience consisted of two floating spheres, two yellow slides and some stacked blocks situated on a pad of paper. It was called Project Origami, so the materials were meant to look and sound like folded paper.

As I went along adding functions to the hologram (which turned into a holographic game), I added controls (gaze, gesture and voice), spatial sound, spatial mapping, and the ability to pick up, move and place the hologram around the room. Finally, an underworld was added to the hologram so that when the spheres rolled off, there was an explosion and they fell into a gaping hole that appeared in the floor. Inside the hole was an underworld, complete with rolling hills and cranes soaring underneath my feet.

The resolution of the holograms projected onto the real world around me was excellent. They were vibrant, sharp and realistic. When I moved around them, the holographic shapes behaved like real world objects would, so I could see their backsides or not see them at all if they became obstructed by other holograms. When the paper spheres rolled onto the floor, they rolled around just like real balls would, bouncing around objects and looking real enough to pick up. When I peered into the underworld that opened up on the floor, it was like I was looking into, as my Microsoft HoloLens “mentor” put it, a world I didn’t even know was there the whole time.

But looking at holograms overlain onto the real world is only part of the HoloLens equation. Controlling the holograms is the other.

The HoloLens gaze controls were responsive and should be easy for any user to get the hang of almost immediately. It’s the other kinds of input where HoloLens has slightly more trouble.

While voice controls worked, there was a lag between giving them and the hologram executing them. I had to say, “Let it roll!” to roll my spheres down the slides, and there was a one second or so pause before they took a tumble. It wasn’t major, but was enough to make me feel like I should repeat the command.

Gesture control was the hardest to get right, even though it was only a one-finger downward swipe. It took some time to figure out the best distance to hold my hand away from HoloLens and where I needed to place my finger in order for my swipe to register. Once I did though, the control worked better than expected. Instead of reaching out like I was going to touch the hologram, the swipe worked best when I held my hand comfortably in front of me.

With the gesture control, I was able to select the Project Origami diagram, use my gaze to move it around the room, and then use another pinch to get the hologram to stay in a new spot (I moved it from in front of me to a couch to a coffee tablet to a desk.). This wasn’t Minority Report-level selecting and swiping, but it was impressive nonetheless.

The HoloLens has spatial sound, which takes the experience to a whole new level. Ambient music played while the hologram was running, and when a sphere rolled, HoloLens added the sound of crumbling paper to go along with it. The sound got louder as I got closer to the hologram and faded as I moved away. It added another dimension to the HoloLens experience that made it that much more immersive.

Of course, as I was viewing the holograms, I was still able to see the real people and objects around me. The holograms weren’t disrupted if someone happened to walk through my projection. My favorite part of this HoloLens demo was a wire frame mesh that appeared on real-world objects to show how HoloLens perceives them. The mesh changed as HoloLens registered the objects and I moved my head to look at a new spot. It also went over people around me, though I could still make out their features from behind the digital framework. It was an awesome AR moment: the people around me became part of the hologram themselves.

As cool as it all was, it would have been less distracting if the holographic images disappeared when I started talking to someone. From speaking with Microsoft personnel, it sounded like this would be possible with other applications, but wasn’t a feature of this particular Project Origami demonstration. It also felt weird to talk to someone with HoloLens on; I felt a little rude, like I was wearing sunglasses inside.

The biggest issue with HoloLens’ holographic viewing is the holograph is limited to what amounts to the size of a large monitor in front of you. You can see the edges of the virtual space where the hologram lives (which are basically the edges of the inner HoloLens frames), and looking outside that space, or moving too close into it, cuts the the hologram off or makes it disappear completely. The wire mesh effect created a more immersive experience and the edges weren’t visible then, but viewing the origami hologram on its own felt like I was looking at something that existed on a limited image plane.

The hologram looked best when I was standing about four or five feet away from it and when there was more going on, like the wire frame mesh.

There were glitches in the software, and it didn’t work perfectly every time. That being said, the problems weren’t horrendous nor did they take away from my enjoyment of HoloLens.

Early verdict

HoloLens isn’t perfect, either in software or hardware. But it’s getting there, and if Microsoft can fix its issues, HoloLens is going to be an immersive entertainment device that could easily find a place in the work world, too.

Despite its issues, using HoloLens was, in a word, delightful. With more robust applications, more reliable functionality and an improved viewing experience, HoloLens may could easily become the AR viewer to own.

HoloLens is the most fascinating hardware I’ve tried in a long time. While we still don’t know how long its battery lasts nor how much it costs, HoloLens seems like it has unlimited potential. Not only that, it feels like it can live up to its potential. And that’s exciting.

GDC and first hands on review

Update March 2015: Looks like gaming is definitely a go for the HoloLens. During GDC 2015, Head of Xbox Phil Spencer led a panel discussing the importance of games on the augmented reality device:

“We see this as a full Windows 10 device with holographic capability,”

Spencer also mentioned that the HoloLens APIs will be made available with Windows 10 gaming SDK.

Original review by Mary Branscombe continued below …

The HoloLens that Microsoft will ship will be a Star Trek-style visor that wraps around your head but doesn’t isolate you from the world, with the Intel SoC and custom Holographic Processing Unit built in. What I tried out in the labs hidden (rather cheekily) under the Microsoft visitor centre didn’t look like that, but the engineers and developers who built it say the experience I had with the developer rig is essentially the same.

The HoloLens developer rig is built into a frame that slips onto your head and gets strapped into the right position; there’s a sensor array over your eyes to track where you’re looking and sensors on the frame to track how your head moves and the Holographic Processing Unit hangs around your neck on a strap (it’s about the size and weight of a car radio). Someone needs to help you fit it and before you do that you have to get the distance between your pupils measured. And when you have it set, there’s a power cord coming down from the ceiling rigged to move with you as you walk.

All of that will vanish into the final product, so what I tried is very obviously early hardware – like the custom developer kits Microsoft makes for early Xbox developers. Microsoft didn’t let us photograph the developer rig, and you need a special camera to capture the HoloLens view so our images were supplied by Microsoft; they’re a rather idealised version of the slightly grainier view I saw.

The HPU, as Microsoft calls it, isn’t that much of a misnomer (less so than suggesting that a retina screen has the same resolution as the human retina, say). What you see isn’t a holograph or a hologram; it’s a projection – but it’s being projected onto holographically printed lenses, which lets Microsoft produce very cheaply the extremely complex lenses that turn the projection into the 3D image you see.

The HPU turns the graphics into the right signal to project onto those lenses as well as processing information from the sensors that tell it where you’re looking and how you’re moving your head. It will speed up voice recognition and spatial sound processing, too.

That doesn’t just let you see the digital world projected around you; it lets you see it on top of the real world. You can see the person standing next to you and talk to them, avoid walking into walls and chairs and even look at a computer screen, because HoloLens detects the edge and doesn’t project over it so you don’t need to keep taking it on and off as you work. You can take notes or answer email on a computer with a keyboard or a pen instead of trying to force that kind of close up work into the world of gestures and gaze.

If you wear glasses, this close-fitting headset isn’t ideal. It’s hard to make it fit comfortably – I found it either pushed my glasses down onto my nose or pressed them hard into my face, and I much preferred using it without my glasses on. As I’m short sighted, that made it harder to see detail. Talking to people outside Microsoft who’ve tried the actual HoloLens headset, it’s light and comfortable to wear but the first version will probably still press on your glasses more than you’d like. And if you wear varifocals, you move your eyes automatically to look through the right part of your glasses for what you’re focusing on; that can mean you look down at things that aren’t in view for HoloLens (or for someone on a Skype call to your HoloLens) or look up and lose the HoloLens image.

The good news is that even if you’re very sensitive to motion and prone to get VR sickness, or if you get headaches wearing 3D glasses, HoloLens is comfortable to use. I’m very prone to both of those and have problems with many other systems; after a brief moment the first two times I put it on when I could tell I was adjusting to what I was seeing, I had no problems at all with nausea, headaches or the other discomfort that can come when you trick your brain into thinking it’s seeing something real.

The HoloLens projected screen moves as you move your head and you control apps either with voice commands or by using the equivalent of a mouse click – the air tap. You just hold your fist out in front of you where you can see it then raise and lower your finger. I didn’t have to worry about getting it in the right place or moving it at the right speed; as long as I made sure my other fingers and thumb were out of the way, HoloLens got the gesture every time.

Digital reality

I tried three different applications with the HoloLens. I also got to watch several people using the Holo Studio 3D building tool, which has the most sophisticated controls, using a combination of gaze, gesture and voice commands to let you design objects you can see in the real world, so you know they’re the size you want before you spend time and money 3D-printing them.

The most engaging was playing HoloBuilder; inspired by Minecraft and built with the help of the Minecraft team, this is a game that lets you build a digital landscape that exists in your physical space. Think the giant LEGO setup in the basement of Will Ferrell’s house in The LEGO Movie, only invisible until you put HoloLens on, and built both on top of and underneath your furniture – and even extending under the floor and into the walls.

Using voice commands and the air tap gesture and my own real feet, I walked around a village, tickled a sheep to stop it falling off the table, dug through a (real) bench to make a hole, blew up some TNT to drop zombies into the lava pool that was in the chamber I’d dug through to, then blew a hole in the wall and lit a lamp to see the bats flying through the caverns in the wall.

Like Minecraft, half the fun is that your creations are obviously digital, but seeing them perched on real tables and benches was even more fun. This combination of virtual and physical worlds was delightful and immersive and shows the obvious gaming and entertainment potential here.

But the other two apps I tried were actually more impressive and certainly more useful. Making a Skype call from HoloLens is a good way to try out voice and gesture commands; you can look at the person you want to call in the address book – which is a grid of faces – then air tap to call them. The video call doesn’t jump around the room if you move; it sits in one place unless you look too far away and then it moves back into view, or you can pin it in place.

The person you’re calling doesn’t need a HoloLens; they see in Skype what you’re looking at and they can draw diagrams on the video that appear in your view. So if you’re helping someone change a tyre or fix their dishwasher or fit a new light switch, you don’t have to explain what they need to look for or pull out or unscrew – you can take a pen and show them.

This would fantastic for teaching and training, for remote support or for getting an expert opinion; imagine a remote handyman who could give customers advice (and charge them for it) or a repair service that always turns up with the right part because you’ve shown them what they’re fixing in advance. It would also be a really interesting way to have a meeting where you’re collaborating remotely on physical objects, not just the usual documents and presentations.

Remote working is something NASA has to do, but the OnSight system the Jet Propulsion Lab is building with Microsoft made me feel like I was walking on Mars. Some years ago I visited the Supervisualisation Lab at the university of San Diego, where they have a wall of screens that can show a life-size image of Mars using the photos sent back by the Mars Rover and I could look out at Mars like looking out of a window. HoloLens meant I could step out onto the surface of Mars and walk around, bending over to look at rocks, turning round to see the view and looking up to see the sky. This is where I most wanted to have peripheral vision because with HoloLens you only see what you’re looking at; as you turn your head, the Rover suddenly comes into view and it’s a rather large surprise.

OnSight isn’t for digital tourism, delightful as that is; there are tools to tell the Rover where to go next to take photographs and samples – and the terrain that looks flat and easy to drive over on a PC screen is revealed as a treacherous series of slopes and ditches the Rover can easily fall down when you see it in 3D, so JPL scientists can look for an easy route before they air tap to give the Rover a target to photograph or burn with its laser.

They can also collaborate; other people in the landscape show up as stylised avatars, and a handy dotted line shows you where they’re looking (HoloLens knows that, so it can show you), which avoids all the creepy ‘uncanny valley’ problems of realistic avatars. This is something we’re going to have to work out social conventions for; when you say goodbye to someone miles away on the telephone it’s easy to hang up, but when you can still see their avatar just turning away and ignoring them feels slightly rude. (Equally, watching someone use HoloLens is disturbing because they’re turning and crouching and reaching out for things you can’t see, and it’s hard not to think they look strange.)

Walking around Mars with another scientist, or walking around an unfinished building and seeing where the walls will be, or being able to pull a virtual engine out of a physical car to see how it fits together so you can work out which screws to undo – there’s a huge range of possibilities here, because this is far more than just a gaming system. And Microsoft is sensitive enough to privacy issues to have avoided the creepy feeling of using augmented reality to spy on the public world that Google Glass is so prone to. HoloLens doesn’t just tag the physical world with information you can get other ways; it adds a realistic 3D digital world on top of (and underneath) it in a way that feels like magic and is delightful to use.

Early verdict

It’s far too early to tell whether the HoloLens will be a success; only a select few have used the final design – instead of the early developer rig we tried out – so I can’t comment first hand on how comfortable it is (with or without glasses).

I saw only four apps, one of which is more of a proof of concept (sorry Minecraft fans, HoloBuilder may not even ship). Price and battery life are both things I can only guess at. It’s entirely possible that HoloLens may be more of a curiosity than a mainstream success. But if it takes off, and later models get smaller and lighter and less obtrusive, then Microsoft has just changed the world of computing again the way it did with Windows. HoloLens quickly feels natural and it’s easy to see how useful it will be – and how much fun too.