Episode 20: Augmented Reality… Enhancing Human Effort

ANNOUNCER: Luminaries, talking to the brightest minds in tech.
MICHAEL DELL: And my hope is that we come together to share more than technology and expertise and products, but that we share a vision of a future that is better than today, a vision of technology as the driver of progress.
ANNOUNCER: Your hosts are Mark Schaefer and Douglas Karr.
MARK SCHAEFER: Hey, everyone this is Mark Schaefer with my co-host Doug Karr and we’re about to bring you another episode of Luminaries, where we talk to the brightest minds in tech. You ready for this one, Doug?
DOUGLAS KARR: No and yes.
[LAUGHTER]
MARK SCHAEFER: You’re confusing me.
DOUGLAS KARR: I’m a little excited that’s all.
MARK SCHAEFER: You’re a little excited?
DOUGLAS KARR: Yes.
MARK SCHAEFER: Well, we’re going all virtual on you today. So gosh, one of the transformational technologies that I’ve been following for a long time, I’m involved in it for a long time, is virtual reality and augmented reality. We are going to talk to one of the industry leaders today is Roy Ashok. He Is the founder of a company called DAQRI. Roy, thank you for being with us and welcome to our show.
ROY ASHOK: Well, thank you for having me on the show. It’s exciting.
MARK SCHAEFER: So, DAQRI, how did you come up with that one? It’s D-A-Q-R-I. But that’s not what the listeners are going to hear and that’s not what they’re thinking about right now.
ROY ASHOK: It is pronounced that way though. It’s like a drink. One of our co-founders actually came up with a name. I won’t take any credit for this. And the way he thought about it back in the day was we wanted QR codes and he was trying to do something with QR codes. And so we wanted the letters QR in the name but couldn’t find anything that sounded easy to say and by multiple people with different accents. He really thought it through quite a bit.
But he was quite the Geek. So he wrote a script to go and look for domain names with the word QR in it. And then came back with a whole list of them and thought this sounded OK. So that’s the story. We don’t do anything with QR codes today. But that’s how we got DAQRI.
MARK SCHAEFER: It’s a cool name. Well, tell me tell me a little bit about yourself and how you got into this fascinating industry.
ROY ASHOK: Thank you. So I started at Qualcomm actually prior to coming to the DAQRI about 12 years, been immersed in AR and VR. I co-founded a project called Vuphoria first, which was AR platform for mobile devices and tablets long before we had AR Kit an AR Core that you see pretty widely adopted today. And at that time DAQRI was a really small company, kind of working on applications for AR, not really sure what to do. So I started advising them, trying to help them get their strategy together.
And then when I sold Vuphoria because Qualcomm had no idea what to do with it at some point. And so DAQRI was a very natural place for me to go and start working on their product. And then eventually I took over as CEO.
MARK SCHAEFER: Well, awesome. And it’s a fascinating product. We’re going to put relevant links to the product and to some videos in the show notes. And you’ve got to check it out. Because it really is fascinating. And Roy, you brought a set here that I’m seeing. But our listeners aren’t saying because it’s a podcast.
And Roy, I’ve been actually working in this area for about 2 and 1/2 years. I started on a project for the US Air Force. They are looking at implementing augmented reality into the workplace, into some of their depos. And 2 and 1/2 years ago, not that long ago, still wasn’t ready for prime time. So what are some of the features of DAQRI? Is it time for AR.
ROY ASHOK: It is time for AR is the short answer. And I want everybody to take that away. Even last year, from what I saw last year at Dellworld. And I was on a panel it was presenting a lot of customer data and seeing how people were people were adopting AR to where it is today, the market has moved tremendously.
The technology is real. We’ve tested it out in environments where they have to be used, so crazy environments, factory floors, by workforces and workers on the shop floor. So the deck is ready and the challenge is really about seeing how widely this can be deployed in the workforce. 2 and 1/2 years ago, none of this stuff was possible.
MARK SCHAEFER: What’s been the big leap? What’s been the big leap in the last year?
ROY ASHOK: When we think about augmented reality, there is two pieces of technologies that have to come together. One is the displays and the optics because you have to see virtual content through those displays. And they have to be light enough. They have to be bright. There’s got to be good color contrast. They have to have a wide field of view, because otherwise if it’s too narrow and you can’t paint enough of the real world with virtual content.
So that’s one aspect of it. The other part is you need computer vision. And this is very critical because computer vision is the fundamental building block for AR. It gives the device the intelligence, really the sense of sight actually, if you want to think about it like that.
Like if you and I as human beings, we walk into a room, we know it’s roughly rectangular, there’s tables, there’s chairs, there’s surfaces, there’s people. There’s a level of geometric and semantic understanding that these devices don’t have. And so you have to give them that sense of sight. And that’s what CV is.
So the fundamental building block for AR, is if you distill it to the simplest possible atomic entity if you will, is just knowing where you are in 3D space. This device needs to know precisely where it is. And if you know that, along with the display technologies, you combine that in a very low latency system that renders all this content in the world. And that did not exist when I first started.
DOUGLAS KARR: I suppose, I’m a movie fan. I hope you are too. You know we look at The Matrix and he plunged the thing in into your head in and get the lessons for driving a helicopter, flying a helicopter to you. DAQRI is a realization of this without the brain surgery. But it really is that now you can actually teach someone in real time to do something complex.
ROY ASHOK: Yeah. That is a good analogy, although I will say, we’re not plunging anything into anybody’s neck. We’re very safe to use. A lot of people are using this.
MARK SCHAEFER: I’m glad you cleared that up. I was beginning to tremble here sitting next to you.
ROY ASHOK: But the broader theme there is actually what we call skill elevation. Right, I’m elevating someone’s skill from learning how to drive a car to flying a helicopter. But of course for us, it’s not driving a car, it’s servicing a car and servicing a helicopter.
And how do you do that? You wear these devices and there’s a system and an application in there that recognizes the car that gives you step by step instructions on, OK, open up the hood, turn this valve here, you know, unscrew this component. And it walks you through it.
So as long as you have some basic skills on how to use tools, you’re good to go. Then tomorrow you have to serve as a Bell Helicopter for example. And then you wear the same glasses. You open up the Bell Helicopter app and it walks you through the same sort of thing. You don’t necessarily have to have prior knowledge or domain knowledge. Of course it’s helpful.
Now of course today we are not there, that you can switch in such a binary fashion. But if you take it in a more realistic way, if you’re on the shop floor, let’s say you’re in a production environment in a car factory. There are three or four different things, a piece of equipment there that are very related that you can service as opposed to having specialized labor there.
So that’s why I like to call it as, you know, in this world of automation and where we’re saying, you know, OK, we’re trying to take humans out of the picture, these things actually make us in a sense super human effectively.
MARK SCHAEFER: So, so interesting, because I just saw the incredible potential of this when I was working on these projects. But one of the issues, one of course was the technology wasn’t quite ready. And there was also some issues of adoption.
And so when I was looking through your website, I saw that you have a really impressive client list. So how is this, are you still getting into problems with adoption, people just don’t want to wear these things or they feel threatened by these things somehow? How are you overcoming that? What progress is being made on the human side of the equation?
ROY ASHOK: I think it’s a really good point. When we first started off roughly two years ago with our development platforms, there was a lot of skepticism about this. You could even call it people felt threatened by it in a sense. But you really have to engage stakeholders from day one.
So the way we did it in the early days with some of our marquee customers was, we went in there, not just with management, but we actually work with the folks on the floor, the line workers, the field service technicians, the inspectors. We asked them what they do in their jobs and what their problems were. And then we designed the solutions around them. Right?
I mean it wasn’t just another device that we say, here, go wear this. It’s like OK, what problem are you really solving? So we got them going from day one. And as we took them through that journey, they realized that, this is not a threat. This makes them do their jobs faster, better, safer as well.
And once they saw the value of that thing, they quickly became advocates for the technology. So that’s a very interesting thing. It went top down, but it’s become a very bottoms of advocacy thing for us.
MARK SCHAEFER: Great, cool. I’m thinking there also more digital natives are entering the workforce and they probably just think this is cool.
ROY ASHOK: There’s some element of that actually. There’s a construction project that we did with Mortenson Construction in Minnesota. This was about a year and a half ago. And it’s really the first public demonstration of the technology. So we just– nobody knew we were coming that day except the Mortenson project manager.
We landed on site out there, and we showed them the deck. And we put it on, on everybody’s head. There was one person who was three weeks away from retirement. There was another person who was like in mid-career and there was a lady six months out of vocational school and sort of getting into the workforce. And the reactions across the board were positive.
But the person who was closer to retirement is like, I’m never going to trust that device. I’m set in my ways. I have my ways of doing things. And I’m just going to do that. But the others could see that there’s sort of this range of OK, this is useful for certain things versus sort of, this is going to be the future kind of thing.
And clearly, the younger ones who are entering the workforce more comfortable with technology, very comfortable with digital data, wearing things, Karrying multiple things. They were definitely more receptive. There’s no doubt about it. But what surprised me was that everyone saw the value and exceeded my expectations in terms of saying, OK, you know, we will give this a shot.
DOUGLAS KARR: You know, in a prior life, I was in the United States Navy. So you were talking about the Air Force. In the Navy obviously you’ve got mission critical equipment there as to be ready at a moment’s notice. And it seems that a lot of the failure comes from human failure, not necessarily from– we have preventive maintenance and corrective maintenance systems set up.
This seems like it’s a technology that could vastly increase the efficiency of machines like ships and airplanes and everything else. Are you already seeing that with some of the companies you’re working at or is that the end goal?
ROY ASHOK: Well, let’s use the US Navy as an example because they are a customer of ours.
DOUGLAS KARR: Fantastic.
ROY ASHOK: And they’re actually down where I live in San Diego. The use cases that you just described, the sort of making the ship itself much more productive or more efficient, that’s still a little more of an advanced use case. Where we see the right mix, right position for AR is where you have these complex tasks that you perform on very high value assets, and a ship is a high value asset.
There’s lots of complex, almost chronic tasks that you do on them, right? Again and again and again, and you always make mistakes on those tasks because they’re too complex for any one person to do it. So the Navy recognizes that and they’ve been trying, getting the ship ready for a mission. How long does it take to turn it around when it comes back?
Simple things like people move things around. And how do you get it back to an operational readiness? So in our device, you’ll notice there’s a whole bunch of cameras in there. So one of the advantages of those cameras is actually you can capture the world in 3D, very detailed photorealistic models.
And so one of the tests that the Navy is doing right now actually, just started this week, is they want to capture the interiors of the ship in 3D at least for certain rooms before it goes out on a mission and then when it comes back from the mission and then you see the differences. And you see where things are moved. And so you can quickly find these things.
This is such a simple use case, but very, very powerful in its value. And so they see early tests are pretty solid and so they’re looking to advance this kind of technology. The other thing is there’s a Bima Lab down there has done this project where they show how you can avoid friendly fire incidents, so situational awareness on a ship.
And simple things like somebody wearing the glasses, the gunner basically, and in their field of view, showing where to shoot and where not to shoot. So you know, I can’t put a price tag on something like that.
DOUGLAS KARR: Where before it was just up a button, friend or foe.
ROY ASHOK: That’s right. Yeah.
MARK SCHAEFER: Interesting. One of the things that is always so fascinating to me talking about emerging technologies like this that there’s always some really nice surprises. And so what we’ve talked about so far, these use cases where, OK, you know you use this wrench and you turn it that many times. You know, put that hammer back where it belonged. I mean, those are some of the historical easy, low hanging fruit use cases.
As you get into this, I’m sure you’ve had to see ideas that people have to apply this that go, oh, wow. We never even thought of that. Maybe even we’re starting to think about some consumer applications. What are some things that have like surprised and delighted you about how people are starting to use these devices?
ROY ASHOK: Surprised, definitely, delighted is more, I’m not sure. I think what I’ve seen, being in this space, you know–
MARK SCHAEFER: We’re going to hear some good stories here, folks.
ROY ASHOK: I’ve been here too long I guess in some senses. But what I’ve realized is that the most value is actually from the mundane tasks. And it’s quite incredible. It’s like, how do I do my job and help me do my job better as I’m doing it to use this.
But the one that actually surprised me was the value of this technology in compliance. I always thought of, I need something to repair, I need I need information to do repair something. Or I need to learn how to do something. And that makes sense.
But in compliance, what happens is you have field service technicians or inspectors going out and doing something chronic. They go every day, they inspect the wind turbine. Or every day they’ll go out on a theme park ride for one of our customers who is a theme park operator. They go and they make sure that it’s safe and they go and inspect everything.
As humans, we miss spots. And we get very set in our ways and we say, oh, that’s probably not going to be a problem there. Let’s skip that. And so, it is a fact in pretty much every customer, the biggest problem we have is in compliance is that they don’t actually inspect everything.
So a device like this, they started using it to force the inspectors to go and look at everything. Simple, surprising. I was very surprised but hugely valuable.
MARK SCHAEFER: Awesome, Great.
DOUGLAS KARR: That’s incredible. You said the term skill elevation before and that and that piqued my interest. Because when I was doing research on this, one of the things that I read all the time about how you know, robots are going to take our job. And we’re going to have a sedentary workforce that’s not doing anything. DAQRI really opens up a window where you can provide more work to people, right?
ROY ASHOK: Yeah. I think it’s more the era, I think, the era of very specialized worker in any one particular thing, especially on the line is probably coming to an end. And I’m not saying that to be provocative. I’m just saying that because I see a device like this basically empowering more workers to do very diverse tasks and workflows.
AI is very interesting. But you know to me, it’s you know, we talked about in the panel as well, it’s very one dimensional. It is only in the digital domain. It works in digital information and it provides insights from that information. But with AR, what you’re doing is, you’re putting data in the real world in context of a machinery or a vehicle, or a facility.
So suddenly the data is beginning to be related to real world equipment, but physically in context, right? Now if you run AI systems on top of that, you now have the ability to control those machines in a very automated way, not in the traditional rules space automation, but in a much more intelligent automation.
However, at the end of the day, these are all designed to be very human centric. I don’t believe in a dystopian world where the AI everything. What I believe is that there is human emotions and intelligence can never truly be mimicked in that way. Yes, there’s probably some tasks that will go away through technology development.
But that’s par for the course. That’s happened for thousands of years actually. But I think AI will simply augment the human and server as another way. There is something about us combining our emotional intelligence with just raw intelligence to make decisions that I don’t think any machine can ever mimic.
MARK SCHAEFER: So that really leads me to the next question, because I saw in an interview that you did watch the video that you said that you’re very excited about AI in terms of how is going to help the progress of your technology, the AR technology. So how is AI and AR going to merge and what’s that going to create for the world?
ROY ASHOK: We’re used to thinking of this device as a consumer of information, right? I read it, I see data. But what it is, is it’s actually a sensor. And the reason I say that is, if you look at it, it’s got a whole bunch of cameras on it, right?
And so it’s beginning to capture real world data in a way that discrete sensors could never capture. Like if I put sensors on a machine somewhere, it’s capturing very specific information about a very specific point on that machine. So for example, if there’s a thermal sensor, it’s saying, here’s a temperature of this valve at this particular point in time.
But a device like this is beginning to capture three dimensional data. And so almost like every point on that machine is now a virtual sensor. And so that’s the first part. Now, what we’re doing is effectively collecting data about the world around you, about the state of the world, and it’s changing. That’s the raw material for AI.
You need that to build a real AI system that actually gives you some decent insight about the state of your factory, about production, about safety situations, maybe all your workers are wearing these glasses and all that 3D data is constantly going back to a system that’s analyzing it. And suddenly realizes that a pattern there that says that is going to be a safety incident in Hall 3A. Let me start evacuating them right away and guide them to safety before it happens. That’s kind of really what you’re going for.
MARK SCHAEFER: That is so cool. That’s a really neat application. I love that. So let me ask you a question about that. Because so, it’s not just projecting to the world, but you’re collecting data from all around you, right? Collecting and then interpreting. So how do you manage security in an environment like that? Cause you’ve got data flying all around like a plant location or a military installation. So it’s almost like you need like an umbrella of security over all this data flying around.
ROY ASHOK: Our view is we’re not a consumer grade company. So we don’t know we don’t process data for consumers. And so we don’t have anything of our own. What we do is, customers already process large amounts of data. They already have systems to handle this kind of situation.
So we work within those. So there are large automotive companies using our products today where the data never leaves their network or never leaves their data centers. We have no idea what it is. But we give them the tools to deploy these things on premise. So usually we work within those environments, within those systems rather than going off and trying to invent something completely new. In an enterprise you know you can’t do it all by yourself. You always partner and work within existing systems to make this actually deployable.
DOUGLAS KARR: We all know AR from picking our phone up and looking through a camera. With DAQRI’s system, and I wish people could see it. So definitely go out there and watch the videos.
MARK SCHAEFER: Check out the show notes.
DOUGLAS KARR: Yeah. But you’re doing the interpretation and tagging. Can you talk through the complexity of me just leaving a tag over a piece of equipment, and then someone else walking in and seeing that tag, that sequence of events.
ROY ASHOK: That actually one of the most common use cases in AR but actually one of the most complex things to do. So let’s walk back from the user experience. So let’s say a user puts these glasses on, right? The first thing the person has to know is his absolutely location, which is, I know exactly where I am in the room and orientations.
I need to know what I am looking at and when I say, it’s not general orientation. It’s got to be very, very precise. Otherwise the tag appears in the wrong place. So the way you get– now you don’t have GPS indoors, right? So you don’t actually have absolute location.
And so one of the ways we do absolute location is, we’ll put a barcode at the entrance in a known place. So as soon as the worker comes in, they look at the barcode, it immediately orients and saying, this is where I’m looking at. And from that point onwards, I know precisely where you are in the room in absolute terms. Because I know where you are relative to the barcode.
So that’s the first thing. And then so the user wears this. He or she goes to wherever the machine is and says, OK, I want to place a tag here. So then there’s all the user experience that comes with it, the UI that allows you to annotate in a hands free way to create the tag, place it, all with head motion only.
We actually don’t use– we want your hands to be free so you use actual tools. So there’s a lot of UX there that we’ve developed over time that’s very intuitive actually, very simple and intuitive, to place the tag there. Once you place the tag there, that has to be updated in a server somewhere. So that’s somewhat well-understood technology. So we use off the shelf systems for that.
Once you do that, the next person who’s supposed to service that piece of equipment has to get a notification saying, OK, here’s where the tag is in this particular room. Go grab it. Now that person can be in a hall somewhere very far away and wearing the glasses. And you can now, because you know where that person is as well, you can start giving them directions in the field of view with the arrows painted on the ground and then direct them to the tag.
Underlying all of this is what I talked about earlier, this like very, very optimized computer vision technologies around positioning, where you are, so visual inertia odometry is a technology. Mapping, so we do three dimensional mapping of the environment. We combine all of that so that tag appears precisely where you intended for it to appear.
DOUGLAS KARR: Wow.
ROY ASHOK: All happening in real time.
MARK SCHAEFER: It’s just that simple.
[LAUGHTER]
MARK SCHAEFER: It’s just like baking a cake.
ROY ASHOK: I put it simply but there are lots of much smarter people who are working on it.
MARK SCHAEFER: Just like making a daiquiri. You see how I just tied the ribbon on that, Roy? I just brought it back to yeah. Roy, this has been amazing. Thank you so much. And I’m sure our listeners are going to be so energized and inspired to learn more about what you’re doing. So where on the web can they find more about you and the amazing things you’re doing at DAQRI?
ROY ASHOK: Start with our website www.DAQRI.com.
MARK SCHAEFER: That’s D-A-Q-R-I.
ROY ASHOK: That’s right.
MARK SCHAEFER: D-A-Q-R-I. Well, thank you so much. And this has just been a joy. And thank you for listening everyone. We always love bringing these stories to you. And we hope you’ll continue to join us on our adventures here on Luminaries as we talk to the brightest minds in tech. This is Mark Schaefer and Doug Karr saying goodbye for now. And we’ll see you next time on Luminaries.
ANNOUNCER: Luminaries, talking to the brightest minds in tech, a podcast series from Dell Technologies.