Episode 22: Artificial Intelligence…Driving In the Streets

Most cities were built for a reality that no longer exists. The streets were laid out for horse-drawn carriages. To make cities more livable, city planners need better insight into how people actually use the city, how they move through it and live in it. In this episode, Dr. Richard Kelley explains how artificial intelligence, machine learning and the internet of things work together to create “smart cities” – cities that reflect the needs of citizens now and in the future.
Listen More
All Luminaries Podcasts

What you’ll hear in this Episode

  • How machine learning builds a robot’s social intelligence
  • How AI and IoT could make cities more accessible, smarter
  • How data being used to develop autonomous vehicles can also be used for town planning
  • How technology can improve the quality of life for the inhabitants of “smart” cities”

Artificial Intelligence, Autonomous Vehicles, and the Smart City of the Future

The average speed of traffic through London has not changed since the days of horse-drawn carts. All of the transportation advantages of the past 200 years, from the internal combustion engine to the highway system, haven’t increased the rate of travel. The infrastructure simply can’t keep up with the demands of the populace. But why is this significant?

To truly serve people’s needs now and in the future, cities need to get smarter to cope with a growing population and its demands on city services. Our guest in this episode is helping to create the “smart cities” of the future. Dr. Richard Kelley, Chief Engineer of the Autonomous Systems Group at University of Nevada Reno, explores how artificial intelligence, machine learning, and the Internet of Things can ease traffic congestion, increase mobility, and make civil services more accessible.

Featured Luminary: Dr. Richard Kelley,  Chief Engineer of the Autonomous Systems Group at University of Nevada Reno

Dr. Richard Kelley is the Senior Engineer at the Nevada Advanced Autonomous Systems Innovation Center. Dr. Kelley’s primary research interest is in the application of machine learning to robots operating in social situations. In 2015, Dr. Kelley joined the Nevada Advanced Autonomous Systems Center, where he worked on a number of projects related to unmanned aircraft systems, including the NASA-led UAS Traffic Management (UTM) System. Since 2017, Dr. Kelley has worked on the Intelligent Mobility Initiative, where he has led a team of engineers developing algorithms for socially-intelligent autonomous driving. He received his B.S. in Mathematics from the University of Washington in 2006, his M.S. in Computer Science in 2009, and his Ph.D. in Computer Science and Engineering from the University of Nevada, Reno in 2013.

“The essence of a smart city is using technology and artificial intelligence to increase accessibility.”

— Dr. Richard Kelley, Chief Engineer, Autonomous Systems Group, University of Nevada, Reno

Luminaries Hosts

  • Mark Schaefer Author, Consultant, College Educator. Mark is a leading authority on marketing strategy, consultant, blogger, podcaster, and the author of six best-selling books, including "KNOWN." He has two advanced degrees and studied under Peter Drucker in graduate school. Some of his clients include Microsoft, GE, Johnson & Johnson and the US Air Force
  • Douglas Karr Technologist, Author, Speaker. Pre-Internet, Douglas started his career as a Naval electrician before going to work for the newspaper industry. His ability to translate business needs into technology during the advent of the Internet paved the way for his digital career. Douglas owns an Indianapolis agency, runs a MarTech publication, is a book author, and speaks internationally on digital marketing, technology, and media.

NARRATOR: Luminaries– Talking to the Brightest Minds in Tech.
MICHAEL DELL: And my hope is that we come together to share more than technology, and expertise, and products, but that we share a vision of a future that is better than today. A vision of technology as the driver of human progress.
NARRATOR: Your hosts are Mark Schaefer and Douglas Karr.
MARK SCHAEFER: Do you recognize that theme song, ladies and gentlemen? This is Mark Schaefer, and we have sort of a Jetsons-themed show today here on Luminaries, where we talk to the brightest minds in tech. And I’m here with my co-host, Douglas Karr. Doug, how are you doing?
DOUGLAS KARR: I’m doing great.
MARK SCHAEFER: Thanks for the assist on the jingle there, Doug.
MARK SCHAEFER: Today we are certainly holding true to our promise. And we have with us today Dr. Richard Kelley. He’s a senior engineer at the University of Nevada, Reno. Now, the thing that is so interesting to me and why we sort of started off with this Jetsons theme is Dr. Kelley, first of all, welcome to the show.
RICHARD KELLEY: Thanks for having me.
MARK SCHAEFER: We’re so glad that you’re here. Thank you for obliging us. And your primary research interest is in the application of machine learning to robots operating in social situations. So obviously, my mind just went to Rosie, the robot from The Jetsons. I’m thinking that had to be an inspiration for you.
RICHARD KELLEY: You know, a lot of the robots we see in fiction are socially intelligent. And so yeah, Rosie, the robots in actually almost every movie, have social intelligence. And what’s amazing about that is that we don’t even really notice it. We’re social without thinking about it most of the time. But honestly, Rosie is light years ahead of what we can build today. So if I could build a robot with the social skills of a four-year-old, I call that a win.
MARK SCHAEFER: So where’s Rosie on the emotional intelligence scale?
RICHARD KELLEY: She’s up there. Yeah, for sure. Way, way beyond any robot we can build today.
DOUGLAS KARR: She’s into sarcasm. [LAUGHS]
RICHARD KELLEY: Yeah, exactly.
MARK SCHAEFER: True. How did you get into such a fascinating field? I mean, this is the cutting edge of interesting technology.
RICHARD KELLEY: Yeah. I think I’m really lucky.
RICHARD KELLEY: I decided to go to grad school and happened to land in the robotics lab. And the first project I was handed was actually building a system that could watch people and predict what they were going to do next.
MARK SCHAEFER: Oh my gosh, that’s cool.
RICHARD KELLEY: Yeah, it’s very cool. It turns out it’s something humans do effortlessly. We’re doing it right now as we’re interacting with each other. But when you try to program that into a machine, it turns out it’s extremely difficult. And so we turned to machine learning and collecting data to basically say, if humans are in this situation, what are they going to do next? And when I say next, I mean just on the timescale of a few seconds, because it turns out that’s enough to give a robot basic, basic social intelligence.
MARK SCHAEFER: I mean, that’s what AI does best. It just recognizes patterns.
RICHARD KELLEY: Yes, exactly.
MARK SCHAEFER: And so facial expressions and responses, I mean, it’s all pattern-based, right?
RICHARD KELLEY: Exactly, yeah. You collect a lot of data of people making sandwiches, and you parse that into tiny pieces. And then from there, you can build a model that watches a person walk into a kitchen and say, they’re going to make coffee or they’re going to make a sandwich.
MARK SCHAEFER: Yeah, I saw a demonstration from Google, I think it was. And it peeled back the curtain on how they develop some of this machine learning. And when you really see how simple it is, I mean, you think it’s just this genius thing that they can predict these things. But it’s just like, well, if pattern’s like this, then do this.
RICHARD KELLEY: Yeah, that’s exactly right.
MARK SCHAEFER: Yeah, it’s really pretty simple.
RICHARD KELLEY: Yeah, it can be. The key is having all that data, and then having a model that can make use of the data, too.
DOUGLAS KARR: So let’s take it up on a grand scale. Today we’re going to be talking about smart cities. And you’re obviously a leader in this. We hear that term a lot, but I’m curious about your definition of the term. And then maybe give us a scale of, do we already have quote, “smart cities,” or are we still years away from them?
RICHARD KELLEY: So I think we’re still a number of years away from a smart city in the way that most people use the term today. I don’t know that there’s a really good concise definition of smart city, but I think that it has some features. So in particular, I think that right now, access to services in a city can be very, very challenging.
And one of the main things that a smart city is going to do is make it effortless to access civic services, whether that’s public transportation or access to governments. And it’s going to be just as easy as calling an Uber is today. And so I think that’s really the essence of smart city is using technology and artificial intelligence to increase accessibility.
MARK SCHAEFER: So really, IoT is one of the key enablers.
RICHARD KELLEY: Oh yeah, absolutely. I think devices that you can embed pretty much anywhere that use very little power and can talk to each other are going to be really essential to give, for example, city officials insight into how people actually use the city and live in it. And that’s going to allow them to make more livable spaces.
MARK SCHAEFER: So as you dream about these cities, and collecting this data, and leveraging this incredible power of the internet of things, one of the things that we’ve talked to a lot of our guests about are the security concerns. In fact, it’s almost like a theme with almost everything that’s going on, as cities transform, and businesses transform, as our lives transform. And you hear about, well, are we going to have to reboot our cities? Are we going to have to reboot our cars? It’s like, oh, our city isn’t working. Just turn it off, and it’ll work again.
MARK SCHAEFER: So tell me about how is security built into these visions?
RICHARD KELLEY: So I think when you say built in, that’s exactly the right phrase that you need to have in mind talking about security in this setting. Because this technology is so new and we’re starting out not quite from scratch, but in a lot of ways, really from the beginning, we can bake security in from the start, which is the right way to do it.
So we look at security from a couple of different perspectives. We look at it as a hardware problem. We build hardware that is more secure. And so when we’re looking at building out a communications network for our intelligent intersections, that’s one of our key considerations. From a software side, because I’m a software guy at heart, my perspective is that every security hole starts its life as a software bug.
And so by starting to think about how we can write software better and safer, we can also make it more secure. So as we develop new algorithms for the self-driving car project I’m on, and we start out prototyping, but as we move into a more production-oriented environment, we’re looking at using better programming languages that make certain types of bugs just impossible, or at least very hard. We’re looking at the practices of aerospace, because they’ve been dealing with safety critical systems for a long time, and we’re trying to take those best practices and really incorporate them into our developments to make sure that our software is safe, and then by extension, secure as well.
MARK SCHAEFER: I know Dell is one of your big strategic partners.
MARK SCHAEFER: So I mean, is there a collaborative relationship on something like that where they can help you see some of these problems that could emerge?
RICHARD KELLEY: Yeah. I think our collaboration with Dell is really in another direction, which is data, which is also at the heart of a lot of the intelligence that we’re building. But as we’re working with Dell and deploying technology out in the wild, I’m sure that we’ll have the chance to collaborate on the security question as well. It’s so new taking security and combining it with autonomous systems and artificial intelligence that we’re really just beginning to ask, where are the big holes, and how do we find them and prevent them from becoming a crisis?
MARK SCHAEFER: Yeah, I see. So really, I mean, it’s just so new, but you’ve got to have all that locked down before you really start executing it in a city-wide type of thing.
RICHARD KELLEY: Yes, absolutely. And you have to be ready to hit the brakes and reconsider as you are trying experiments, because we really are in an experimental stage when it comes to some of the smart city ideas.
DOUGLAS KARR: A focus of the sport the ideas that we’ve talked about thus far been largely talking about transportation and mobility within a smart city. I’m curious about– there was a term that I read– intelligent mobility. How does intelligent mobility improve transportation? Because it’s well beyond just autonomous vehicles, right?
RICHARD KELLEY: Yeah, absolutely. So we’re looking at intelligent mobility as partly transportation, but also just understanding better how people are using cities. So one of the projects that we have in our intelligent mobility initiative is instrumenting intersections with lidar. So a lidar sensor is essentially a spinning laser scanner that gives you a 3D picture of the world around the sensor.
So we can put one of these sensors at an intersection, and all of a sudden, we can see in 3D, in real time, all the pedestrians who are using the crosswalks. We can see the cyclists who are maybe running lights. We can see cars. We can see trucks.
And we are taking that data, and we’re using it not just to make our autonomous vehicles better, but we’re also going to be working with the city of Reno and other government entities in the region to give them access to that data to help them understand from an urban planning perspective, how are people actually using the city? And then how can they redesign elements of the user experience of the city using this new class of data?
MARK SCHAEFER: You mentioned Reno, your home city. Beautiful, beautiful town. Love Reno.
RICHARD KELLEY: Glad you think so.
MARK SCHAEFER: Are there other star cities? Are there cities that are really progressive that are just pulling you into this sort of work?
RICHARD KELLEY: I think most cities in the United States, or at least cities of a certain size, have at least one smart city initiative that they are exploring. I think Columbus, Ohio has a large grant from the Department of Transportation to explore a lot of these ideas. And then internationally, I know there are a number of cities, particularly in Europe that come to mind, that are really pushing the envelope on the smart city idea. One example that really stood out actually in Asia is, I think, Singapore has a database of every tree in the city.
RICHARD KELLEY: Yeah, I think that’s right.
RICHARD KELLEY: And so when you think about, could parks and rec use that kind of data? Collect it first, and then make use of it, just the potential there to have a new level of understanding, or even a new kind of understanding about the city is substantial. So we try to take inspiration from all of these projects around the world.
MARK SCHAEFER: Maybe they’ll finally find out if their bark is worse than their bite.
MARK SCHAEFER: I think that would be categorized as a dad joke.
So Dr. Kelley, a very sensitive issue is a big part of the smart city idea is autonomous vehicles. And we’ve had a number of incidents in the news where we’ve had some failures. In fact, we’ve had a fatality or two. And in the past, these innovations in developments of transportation were able to gestate without the glare of social media. And today, now everything gets just amplified.
So what’s your position as a researcher, and really as a force, and a spokesperson for these technologies? Do you just wait that out? Is there a way to neutralize this? Because a lot of it is blown out of proportion. And I’m not saying people shouldn’t be concerned, and we shouldn’t be responsible and accountable, but the truth of the matter is that these are breathtaking new innovations, and just like the space program, things are going to happen along the way.
RICHARD KELLEY: They will. I don’t think we want to neutralize the experiences that we’re seeing, because I think we have a lot to learn about safety and safety practices from events like what happened with Uber a couple of weeks ago. I think we need to take each tragedy and first off, recognize it for what it is, but then secondarily, learn from it, and see what can we do moving forward to make sure that it doesn’t happen again.
And if you look back at the history of aviation, the Federal Aviation Administration was created because of a midair collision over the Grand Canyon, where many people died. So hopefully, with autonomous systems driving on our streets, we don’t have to have an event of that scale to really learn what to do and how to do it.
And so I think really using these events and learning from them is the best way for the industry to move forward. I think it’s interesting looking at the history of this technology, though, or of all technology. Starting with cars, people were very, very skeptical of cars at the very beginning. People didn’t like them. They didn’t want them. Even something as innocuous as elevators. Unmanned elevators were a point of contention in some cities several decades ago.
And now we don’t even think about elevators as being unmanned. They just are. I think we’ll eventually get to the same point with autonomous mobility technology. I think it will take some time, and I do think that social media can make it challenging to have a sensible reaction to some of these events.
So for us, we really just emphasized the safety element of what we do, because safety is the first value that we have in all of the research that we’re doing. And then we also point out that a lot of research still needs to be done to make this technology viable in the same way that airplanes today are viable and safe. So as a researcher at a university, I’m very fortunate that I get to ask these questions, and look at the things that really aren’t solved, and just recognize that there are a lot of things that still need to be done.
DOUGLAS KARR: I recently rented a car and traveled across the country. I told Mark this. And it had LaneSense technology and it had adaptive cruise control. And I found it just seamless and incredible how much safer. And I didn’t feel iffy working with it, which kind of takes me to the next question is, the interaction of these systems is quite phenomenal. I learned some new acronyms as I was researching this podcast, and some of them I want to go down the list.
There’s V to V, vehicle to vehicle, V to D, vehicle to device, V to P, vehicle to pedestrian, V to H, vehicle to home, V to G, vehicle to grid, and V to I, vehicle to infrastructure like highways. I mean, for one automobile, that’s an incredible amount of interaction.
Are you guys developing from a regulatory standpoint a set of standards for that interaction, or is the industry doing that, or how does that work? Right now, I think that the best answer is it’s a little bit of the Wild West when it comes to standards being developed around these technologies for a couple of reasons.
First off, the technology itself is so new that it might be a little premature to try to develop regulations, especially around some of the vehicle to x concepts, because we haven’t had enough experience really with how the technology is going to be used. So regulating vehicle to infrastructure may be challenging.
There are some standards that are beginning to emerge. So there are some short-range radio standards that most of the industry is settled on. And we are working with the state of Nevada, because a lot of our funding comes from Nevada, to explore the best way to develop a regulatory framework around the technology. Really lucky that Nevada has been working on laws for autonomous vehicles since 2011 or 2013.
RICHARD KELLEY: Yeah. And so Nevada was actually the first state to pass laws regulating autonomous vehicles. So we’ve been doing it for a while, and that means there’s a lot of built-up institutional knowledge in Nevada government around the right way to do autonomous systems. So we do leverage that and we work with the state. But fortunately, Nevada has taken an attitude of, let’s see how the technology is evolving, rather than try to lock down exactly what should or should not be done. Subject of course, to making sure that it’s safely deployed.
MARK SCHAEFER: That’s a very interesting insight that really, the technological progress, it’s going to be a dance between technological innovation and regulatory environment.
RICHARD KELLEY: Yeah, absolutely. Yeah, to really make sure the technology is used in the best way possible, I don’t think you can over-regulate or regulate too soon. But at the same time, when you have zero regulation, as we saw in Arizona, things can go south rather quickly.
RICHARD KELLEY: So it’s a fine balance.
MARK SCHAEFER: So as I was reading about some of the smart city efforts, I saw this really inspirational case study, where they had this example where they had children using smartphones as a data collection device to look for pollution counts in New York City. And what they found is that there were pockets of the city where the pollution was a lot worse than they thought. So every person was like a walking data collection point.
I’m sure you get exposed to all these amazing ideas and benefits. So as you see this world of integrated data collection points, what’s making you go, wow? What are some of the amazing benefits that you see are going to be in our future?
RICHARD KELLEY: So I the big one for me– and I am biased by my perspective, which comes from autonomous vehicles and autonomous robots– is lidar technology. So I mentioned it a little bit earlier. I think the interesting thing about lidar is that it’s getting very, very cheap. So in 2004, a lidar system that could see the world reasonably well would run you about $80,000. Now a similar system may run you maybe $8,000. And with new developments in solid state technology, in the not too distant future, maybe within the next five to 10 years, similar systems are going to cost less than 100 bucks.
So when you have a sensor that can see the world in 3D out to maybe the distance of a football field, that is going to give you a completely new perspective on the world around you. It’s going to let city planners see the world in a completely new way. It’s going to make it so that every vehicle on the road could have 3D vision that works better than our eyes do and works perfectly in the dark.
And I think cheap lidar technology is going to be absolutely revolutionary when it comes to perceiving the world around us. You think about our smartphones. Every smartphone is also a camera. And we’ve seen with things like Pokémon Go that as camera technology gets really, really good, we can take the digital world, and project it out into the real world. Lidar technology is going to be able to take that idea and just amplify it in a way that we are only beginning to understand.
MARK SCHAEFER: Will it be miniaturized, as well? I mean, with the cost going down, is this something that we could almost see in a smart device someday?
RICHARD KELLEY: So the iPhone X has the face-scanning technology. That’s very, very similar to lidar. Take that concept, and project it out to a whole football field. And eventually, it could end up if not on a phone, on a tablet.
MARK SCHAEFER: I mean, there’s really profound implications even for personal safety.
RICHARD KELLEY: Oh, yeah, absolutely.
MARK SCHAEFER: That’s incredible.
RICHARD KELLEY: Yeah. It’s going to be remarkable to watch, and I think it’s going to drive a tremendous amount of innovation in several different spaces.
DOUGLAS KARR: Wow. As you look at a modern city, we’re talking about how people are traveling and making that more efficient, making it more safe. But I’m curious, too, how does this transform cities where we often see urban blight or we see poverty sections of town where if we can make a transportation system so efficient and intelligent, does that start breaking up some of the problems that happen in urban centers of growth?
RICHARD KELLEY: Yeah. I think it really does. So for an example, we work with the Regional Transportation Commission in Northern Nevada. They are responsible for the bus routes. And one of the problems they have in Reno is that there are a lot of rural areas outside of Reno’s core where there would be people who would definitely be served by public transportation, but it’s really hard to justify the tremendous expense of having a bus go, say, 30 miles out of town just to pick up maybe a few people.
And so I think autonomy changes that calculation, and makes it so that maybe not a bus, but an autonomous car could go out and pick someone up. And now all of a sudden, you’ve got the opportunity for people who are in rural areas to work in the city much more cost-effectively than is possible right now. So I think access to city services is going to be a huge benefit of autonomy.
MARK SCHAEFER: Dr. Kelley, thank you so much. Our time has just flown by. And you’ve been so generous with your time. And this has just been an eye-opening discussion for me. I’ve learned so much from you. We’ll put the link to your site in our show notes, but if people want to learn more about you, learn more about what your center is doing at the University of Nevada, Reno, where can they find you on the web?
RICHARD KELLEY: So on the web, we can be found at www.unr.edu/ncar, NCAR for the Nevada Center for Applied Research.
MARK SCHAEFER: Absolutely fascinating. Thank you so much for your time today. And thanks to all of you. We never take you for granted. All our listeners and fans out there, we appreciate you. You have helped us become one of the top 1% business podcasts out there, and we appreciate every one of you. We never take that for granted. This is Mark Schaefer. On behalf of my co-host Doug Karr, thank you so much too listening to Luminaries, and we will see you next time.
NARRATOR: Luminaries– Talking to the Brightest Minds in Tech. A podcast series from Dell Technologies.