The year is 2050. You—and your 3D avatar—are spending lots of time in the metaverse. It’s an interesting place. Let’s explore.
Boundaries between the digital and physical realms are porous in the metaverse. Here, users work, play and make friends at digital concerts and conferences or take virtual trips to far-flung lands. And here, virtual content is fully comprised of 3D objects, people and creatures that live in and move seamlessly through the physical and digital domains. Your eyes alone can’t decipher what’s real and what’s virtual. 2D videos on a flat-screen are so 2022.
When you enter a room in 2050, you can assume everyone has a form of augmented vision, allowing them to see digital, 3D content that exists in their physical space. You can train in open-heart surgery in the backseat of an autonomous taxi or test drive a car without leaving your home. Or you can simply disappear into a virtual world that’s completely disconnected from reality, says Tom Carter, an entrepreneur and technologist who’s helping architect that future through his company, Ultraleap.
“You are going to be there as a human or a humanoid-like avatar and you’re going to want to interact with the stuff around you,” says Carter, CEO and founder of Ultraleap. Based in Bristol, U.K., the company develops hand-tracking and mid-air haptics technologies.
“The two primary ways that humans will interact will be with their voice and their hands. You will reach out in the metaverse to grab things. You will use your hands to manipulate digital objects as if they were physical. You will recognize what your hands are doing—in other words, hand-tracking. And you will feel what you touch—in other words, haptics,” Carter says. “That’s what we do.”
Look, no hands
Indeed, while Carter can envision 2050, Ultraleap’s solutions are already in play with customers that span the automotive, gaming, retail, XR (extended reality), healthcare and training sectors—and in products that don’t require touchscreens, controllers or keypads to be activated or operated. The company employs 150 people across Europe, North America and Asia.
Ultraleap’s hand-tracking solutions, which use cameras and software, are “the fastest and most robust in the world,” Carter says. Its mid-air haptics solutions use ultrasound waves, enabling users to feel with their hands a range of tactile sensations in thin air—the click of a button on a finger, the curve of a 3D object in a palm, even the crackle of lightning bolts radiating from fingertips.
“We are the only company that can project the feeling of touch through the air and onto your hands,” says Carter, who was named as one of the top “Innovators Under 35” by MIT Technology Review.
For Carter, everything began with playful inspiration during his days as an undergrad student at the University of Bristol.
One evening, he watched his flatmates try out an Xbox with a Kinect sensor. They struggled to access the game’s digital menu screen, which was activated by hand movements. When his flatmates resorted to awkward, overexaggerated gestures to press the virtual buttons, Carter quickly understood what they needed was haptic feedback.
Taking the leap
The notion of using ultrasound to replicate touch has been discussed since the 1970s but programmers long struggled to crack the code. Carter, pursuing a computer science degree, began researching how to bridge that technology gap and create mid-air haptics for real-world use. Later, as a Ph.D. student at Bristol, he worked to develop that technology with hopes of launching a company.
In 2013, based on his Ph.D. work, he co-founded Ultrahaptics and served as its chief technology officer. Carter and his team wrote a set of algorithms and used ultrasound emitters to simulate the sensation of touching 3D shapes and textures in mid-air. (In 2019, Ultrahaptics acquired hand-tracking startup Leap Motion and became Ultraleap.) In 2021, Carter was named the CEO.
Currently, more than 350,000 developers at companies around the world are using Ultraleap’s technologies.
PepsiCo, for example, used the hand-tracking solution to launch a pilot program at a KFC chain in Poland, turning their food menus touchless and enabling safe ordering.
Lufthansa has used Ultraleap’s hand-tracking technology integrated into XR headsets to help train flight attendants on how to conduct pre-flight cabin checks.
“Rather than having to take multiple people into the aisles of mock airplanes for training, flight attendants can wear XR headsets and use their hands as they would normally to close virtual overhead bins or raise virtual table trays,” says Faye Lockier, global communications director at Ultraleap. “Right now, we’re seeing a large number of companies in the training and simulation space utilize our hand-tracking with XR headsets.”
Elsewhere, Warner Bros. Entertainment used Ultraleap’s haptics solution to create immersive movie posters that promoted its 2019 superhero film “Shazam!” Fans could use their hands to knock big, virtual letters off the posters.
And Qualcomm is pre-integrating Ultraleap’s latest hand-tracking platform, Ultraleap Gemini, into its Snapdragon XR2 5G Platform, which is being used to build the next generation of XR headsets. Ultraleap’s Gemini hand-tracking matches the positions of your virtual hands to your real hands more closely than ever.
“It’s an important milestone for us as a company because, before that, people could only use our hand-tracking if they bought our hardware,” Lockier says. “The relationship with Qualcomm means customers can use our software and not necessarily have to use our hardware. They can use their own camera systems.”
Lessons learned in the real world
To meet that fast-rising customer demand, Ultraleap relies on Dell EMC PowerStore. The PowerStore family of all-flash data storage appliances improved Ultraleap’s development cycles, ensuring zero downtime for developers and reducing latency, says Richard Goodwin, Ultraleap’s director of IT.
“We’re growing rapidly as an organization and we need to be able to move quickly in terms of any hardware procurement and software upgrades,” Goodwin says. “The resilience of PowerStore was key because we’ve gone through those upgrades and had no interruption to any of our service, our developer life cycles or our product development.”
All that growth today feeds into Carter’s vision of tomorrow—and how Ultraleap’s hand-tracking and haptics will mesh well with the metaverse in decades to come.
But even an uber-forward thinker like Carter has learned how to better focus on the now. Early in 2022, he tweeted: “Don’t place happiness at the end of the list.” It was a message of recently earned wisdom, he says. It was drawn from the foolhardy idea that only after you check all the boxes on your to-do list will you find contentment.
“Over recent years, I’ve very much learned the to-do list never ends. There is no endpoint. There is no destination,” Carter says.
“Like when you’re building a company. You start off thinking, ‘When I get to this stage, then I’ll feel we really made it, and then it’ll feel great, and happiness will be there.’ Well, that point never actually really comes. Life is more of a journey.”
Lead photo: Embodied Labs’ immersive VR training platform, featuring Ultraleap hand-tracking, is used in healthcare facilities across America. All photos courtesy of Ultraleap.