2020-23 | Design Director | Ultraleap (Leap Motion)Ultraleap made hand-tracking technology for use in gesture-controlled XR interfaces. For those building XR applications, my team and I spent three years defining UX best practice, building a design system and developer tools.
Background
Ultraleap (previously Leap Motion)’s hand-tracking technology was the best in the business, but it wasn’t selling. The scale-up’s plan was to identify clearer use cases for it, persuade hardware manufacturers to include it in their VR headsets (along with other devices like cars and musical instruments) and persuade software developers to build gesture-only applications and games.
Ultraleap had found that developers expected design guidance, UI components and tools from them in order to build intuitive, ergonomic and accessible experiences quickly, and I was hired to deliver that.
I hired and established a user research and design team. Then, over three years of user research, iterative development, user testing and client demos, we established a solid understanding of what made gestural interfaces feel right and work for everyone. For a product /UX designer, this was fascinating and cutting-edge work that had little precedent at the time, and my skills in interaction design were thoroughly challenged as I worked to understand 3D interfaces without screens, mice or controllers.
Work & Outcomes
Over three years and many projects, the team explored UX best practice for VR and AR with clients in a range of industries - for example gaming, TV and movies, pilot training simulations and virtual car showrooms.
Each project involved fast, creative and iterative work between researchers, designers and developers, usually to build several Unity prototypes, test with users, iterate and document what we learned for inclusion in the Ultraleap developer toolkit. By year three, this had led to us delivering documentation, demos and tools covering:
How to build a 3D, spatial interface that’s ergonomically sound and accessible
A design system of interactive UI components designed from the ground up for gestural VR
A set of pre-packaged hand gestures with guidance on what to use, when, in different use cases
Tools for quickly adjusting hand detection setup for different users and conditions
A series of inspirational, open-source Unity and Unreal demos for developers to pull apart
This work culminated in me delivering a talk at the IXDA Interaction conference in Zurich, and lectures and workshops atFH Joanneum Design School, Graz (Switzerland) - both on our work at Ultraleap.
