Hololens
Mixed reality is an emerging space that requires an entirely new approach to design and interaction. Alongside Microsoft, we explored the role of physical design in the digital 3D space.
We spent 10 weeks working with the Microsoft HoloLens team on the concept of “Enchanted Objects”. The objective was to create a physical object that would come to life and create interactions within the holographic space.
“HoloLens is magic made real.”
The Team:
Trevan Strean, Ed Hanko, Peter Jacobson, Jill Kuyt
My primary role was to lead UX design and research. This included guiding research methods and providing primary insights. I also planned our final presentation and our overall storytelling strategy.
THE PROBLEM
Through discussing and testing the HoloLens, our team identified four key problems with the mixed reality experience.
Entering the holographic space can be very isolating. Although you still see your environment, the presence of the holographic content mentally separates you from the people around you.
How can HoloLens become a shared experience?
We are comfortable with 2D interfaces, they surround us every day. In working with the Hololens, our natural tendency toward 2D interfaces can limit our creativity and prevent us from imagining the full possibilities of mixed reality.
What is 3D control? How does space redefine inputs and interactions?
HoloLens controls lack the finesse of interaction that makes it useful for performing precise tasks.
How can HoloLens change creative processes? What new possibilities exist for productivity and visualization in mixed reality?
Control within the HoloLens can be fatiguing. The gaze control and hand gesture “clicks” require acute use of your physical body and can limit the immersive feeling. These interactions don’t reflect the dimensionality of the mixed reality experience.
What makes an interface feel natural and human? How can we make the Mixed reality more magical and attractive?
RESEARCH
To answer these questions and rediscover natural interaction, the team investigated a variety of sources. We wanted to capture imagination and intuition, while also focusing on ways to enable the productivity and overall magical experience of the HoloLens.
To get back in touch with our imaginative sides, we spent some time around toddlers to see how they interact with the world and discover new things. Imagination is a very powerful tool, so seeing their simple sense of fun and boundless joy in the undiscovered was very inspiring.
The kids are also too young to have been exposed to many 2D interfaces, so they have a purely intuitive way of interacting with their environment, something that most of us lose as we grow up.
We also investigated a variety of precision activities to see what type of actions were natural. We looked for patterns between a skills in Athletics, arts, occupations, and everyday tasks.
As a team, we had several discussions over the usefulness of the HoloLens, including its current limitations and future potential. We discussed what was needed particularly for creatives- better precision and control, with the ability to quickly change between tools. We also discussed the difficulties of sharing files and being able to keep team members updated.
INSIGHTS
The HoloLens provides a new kind of experience that integrates the physical and digital worlds into one. Our key insights led our concept development so we could create a simple and intuitive experience that went beyond traditional two dimensional inputs.
One of the most important features the HoloLens lacks is a way of sharing content and maintaining connection to the physical environment. This is the largest area of potential for the HoloLens.
Inputs and experiences within the HoloLens need be free, intuitive, and natural. The more it echos our real life experience, the more real the holographic world will feel.
Most activities require two hands. One hand is dominate, performing tasks that require more precision. The other hand acts as support, either stabilizing or completing secondary tasks.
Some of the capabilities of the HoloLens are truly magical, but the controls quickly cause fatigue and frustration.
The concept of delightful control was a synthesis of our research in human physicality, childlike imagination, and physical and digital interaction. Maintaining the magic and wonder of the holographic world through a fluid experience can make using the HoloLens more human and delightful.
Based on our research, we created a series of goals to gauge the success of our concepts moving forward. We included physical and technical specifications as well as the psychological experience of using the product. True success in designing for mixed reality goes beyond control and usability, it needs to bring beauty and human empathy to the entire experience.
DEVELOPMENT
We explored a variety of tactile controls to investigate what felt natural and expressive for working within HoloLens. The spherical shape was appealing as a physical device because it is the simplest representation of three dimensional space. Manipulating a ball connects directly and intuitively to the environment around us and allows for motion in any direction.
As we worked, we consistently revisited our inputs and interactions to assure that they maintained our goal of intuitive control. We used our research insights to define a sharing scheme, precise and quick controls, and added special features that would improve the productivity and magicality of HoloLens.
SOLUTION
Our final solution was a dual mode controller with a simple, versatile form, finished to fit within the HoloLens Suite of products. We prioritized a simple sharing experience, along with two modes of control that would provide for fatigue relief as well as high precision and intuitive gesture control.
We developed a dual mode controller that leverages the motion tracking and gyroscopic technology of the HoloLens, in combination with a hemispherical touch sensor for dynamic control.
The two modes allow for different types of input. The device physically changes to reflect the nature of the functionality, either rising to meet your thumb or retreating to allow mixed reality to fill the space.
Pencil Grip
This control mode allows for a high level of precision while working within HoloLens. It is ideal for creative and productive tasks, and allows you to use the controller as you would use a paintbrush or sculpture tools. Accompanying gesture controls can create shortcuts and allow different holographic tools to be used.
In precision mode, the tip of the controller can be populated with a mixed reality tip to act as a different tool. In addition, a haptic engine in the controller provides physical feedback when the digital tools make contact with the surface of a hologram.
Joystick Grip
This grip provides a simpler less precise control over the holographic world. It can be used with your hands at your side, preventing fatigue. The touch sensitive hemisphere can manipulate objects in any direction.
Using the joystick grip also allows a break from gaze control. The user has the option to scroll between holographic objects using the sensor ball without looking directly at the object.
SHARING
We felt that creating a shared experience within HoloLens was essential to improving usability, and we wanted to make it simple and seamless.
Physical sharing allows co-creation in a shared physical space. It is activated by tapping two controllers together. The experience requires no menus and allows tasks to begin immediately without setup. If required, the enviroment can be customized or saved information retrieved.
Digital sharing allows users to view the digital content within a different HoloLens, send messages, or enter a shared co-creation space. Digital sharing mimics the dynamics of creating in the same room by using the HoloLens’ capability of visually reproducing the avatar of your teammates.
A very special thank you to our friends at Microsoft for their help and guidance on this project. Your enthusiasm and passion for this emerging experience is challenging to explore as designers and as humans.
