A Conversation with Nicole Johnson, a leader in UX and HMI Design: In Between the Tech

Raymond Yin
Welcome to In Between the Tech. Today we wrap up Mouser's in-depth look at Human Machine Interface. We're sitting down with Nicole Johnson, a leader in UX and HMI design who's worked at companies like Icon Group and Rivian. She's sharing her thoughts on the evolution of this technology and providing perspective on how engineers can better their designs by keeping the user in mind.

Host
Thanks for joining us today, Nicole. To start, please tell us a little about yourself and how your career led you to HMI.

Nicole Johnson
It's been a winding, but fun road. I first worked as a mechanic for Mercedes-Benz while I was going to school for business administration. Then after working at Mercedes, I didn't exactly know what I wanted to do in business. Business is such a huge umbrella and I ended up having an idea for a startup. Talked to a friend of mine who was a colleague at a consulting firm that I was doing some freelance contract work for, and we ended up working on a startup called Music Box, which didn't end up panning out, but that was my foray into user experience and that led me to then go deeper into learning more about user experience going to events. Ended up finding two really awesome mentors, Jose Caballer and Chris Doe. They actually have a really cool YouTube channel called “The Futur”. That's where I cut my teeth on a lot of user experience projects.

Then I ended up next at a consulting firm called Icon Mobile. It's a German innovation company and they specialize in IoT or Internet of Things, products for companies like Proctor and Gamble, which has Oral B and Olay and all sorts of fun products and different types of devices we worked on. But they also had another vertical within their business called Icon In-Car, and that's automotive user experience. And I was there for about six years. I got to work on a lot of cool automotive projects, even working on the first 15-inch instrument panel display for PACCAR, which are semi-trucks, basically. So I was taking 30 gauges that are normally physical things in a vehicle and trying to figure out how to fit them into a user experience on a screen. And that, then dovetailed into an opportunity that came around with Rivian to work on their human machine interface team or their HMI team. And then that led me leading that led to leading the team. And so I've been doing that for the last three and a half years and that is how I got into HMI.

Host
Can you give us your perspective on how HMI has evolved since its early days?

Nicole Johnson 
Yeah, disclaimer, I'm not speaking about HMI across every last industry. I haven't worked in every last industry. I've worked in several, but automotive is one of my core areas of expertise. And so that's a really fun topic for me. So just as a disclaimer there. But it's evolved so much in the last 20 to 30 years. For starters, traditional human machine interface is traditionally a function of engineering and also another discipline of automotive, which is called human factors, which is ergonomics and cognitive engineering. That's kind of the origins of when, if you would've said HMI 20, 30 years ago, what somebody would've thought. But as time has gone on and cars have evolved and they become more software defined, there's been an emergence of what we call HMI designers, some of them could be fluent in engineering, some of them are engineers that have an acumen for design, and some of them are designers that are conversational in engineering, but not necessarily fluent, put it that way.

And I think that HMI designer is an interesting one, at least from my perspective because it's a bit of a new category. It's been around for maybe let's say five, 10 years. And it's, again, a bit of a hybrid and a bridge between engineering and design. And there's a term going around right now, a trendy term called systems designer. That's a bit a good descriptor of what an HMI designer is doing. They're looking at the user experience, not necessarily just from an interaction standpoint, but also how it plays into the physicality of the vehicle. That could be lighting, that could be door handles, that could be doors, that could be any number of things. And so it's a convergence of this physical and digital aspects of traditional user experience and traditionally HMI, which was more of an engineering function.

Host
You mentioned your experience in IoT. How does the integration of IoT and Industry 4.0 affect HMI? 

Nicole Johnson 
Well, vehicles have gone from being computer controlled since the late 1970s, early 1980s. Fun fact, I think the first computer chip for vehicles was 1969, but you didn't really see it emerge in vehicles until -more popularly - until the late ‘70s, early ‘80s, and then you saw this crazy emergence of computer-controlled vehicles. However, now vehicles have gone from computer-controlled to software defined. And what I mean by that is they're not just more intelligent and they can give trouble codes to mechanics and engineers, but through sensors and cameras and integrations of data and Cloud connectivity, the entire experience, and touchscreens, of course, the entire experience is a lot more textured, layered ,and highly visible to the customer and also really, really malleable, which is actually part of the fun of HMI design present day.

And those implications are actually really huge. From a software perspective, you could pretty much have a completely different vehicle and a completely different user experience. Probably, a better way of saying that, then what you originally purchased because software you can change it at, I want to say at any time, of course, anybody that works in automotive is going to say you're not going to be able to change it at any time. It's not magic, but you can essentially create new features that never existed before, months and months, years after you've owned the vehicle that could arrive onto your vehicle, which is what makes companies like Tesla and companies like Rivian. Because they've done what's called integrated software development, they own all of their software capabilities and so they can put out features a lot faster, adjust them, get feedback from customers, and make adjustments very quickly. And so that is really amazing when you think about data and connectivity and things that could be integrated after the fact, after purchase. So your vehicle becomes this new and exciting thing every couple of months when new features come out, which is really fun and exciting to me.

But one other aspect of this when it comes to industry forward auto and how that affects the automotive industry, there's complex topics that I'm not going to go too deep into, but when you think about data and privacy and connectivity - that's sort of in its infancy right now, I think there's a lot of things around connectivity and data that doesn't necessarily translate a one-to-one from your cell phone to your car. There's a lot of things that do, like GPS of course, and things along those lines, but there's actually really cool things that you can do with a vehicle that doesn't necessarily translate over to your phone. But we're in the very, very infant stages of that. So, there's a lot of cool, exciting things to come, I'm sure.

Host
It sounds like we’ll have many exciting design updates coming in automotive. When we look at HMI design in broader industry terms, how does the design of an HMI affect the ease of usability between human and machine?

Nicole Johnson 
It doesn't just affect the ease of use, or rather, I'd like to say the joy of use. A good HMI experience is the very foundation of usability, and it stems from understanding the needs of your particular customer segments for whatever your product is. And it's really important to try to put yourself in the shoes of your customer. And it doesn't mean that you have to take this as black and white statement where you're literally trying to pretend you are an astronaut or a soccer mom or what have you. But what I'm really talking about is empathy and trying to imagine what your customer might be doing on a day-to-day basis in their day-to-day lives. And it's an area of design that's often in the background. You don't often see… you see the outcome of it, but you don't often see the work that goes into thinking about that. And it's not a game of guessing, it's a bit of an art / science mix. So, you're doing research and you're inferring insights from that research and then translating that into use cases or user journeys to try to put yourselves in the shoes of that customer. And without that, you can create something that's really aesthetically beautiful, not necessarily functional, though. You might be missing a piece of what is that person's day like? Hey, I need to be able to take a car seat and put it somewhere that's not visible. How do we do that? How do we do that in a way that's easy? Or being able to clean up a surface easily? How do we do that? So, just to say good HMI is really human-centered. It's not necessarily about usability. Usability should be something that is a foregone conclusion. It should be really just thinking about the customer.

And then the usability part comes naturally. Just to go back to use cases or user journeys, it's a really helpful method. You could do it as use cases, user journeys. We've done it where I have an old colleague of mine who is brilliant at doing beautiful storyboards, and that really helped to communicate that to our executives as well, to help them put themselves in the shoes of the customer or using user research where we have videos of people trying to use something that's very obvious that it's not working or they unearth something about the experience that designers or leadership might not have thought about. And so it's just storytelling tool to help teams collaborate.

Host
That’s a great phrase, “the joy of use.” Can you talk a bit more about that and what it means for you when you’re working on a project and how that translates for the user or consumer?

Nicole Johnson 
What I meant by that was having an experience that is fun, and I don't mean fun like there's confetti coming out of the car or something like that. But being able to, as a designer, be excited about what you're working on translates to being able to design experiences that people like using. And that really incites something small for them. It could be it just made them smile or they were like, “wow, that was really easy.” And again, the word usability is thrown around all the time. So, ease of use is as well, of course it should be easy, but it’s often better when it’s joyful to the outcome of whatever you’re designing your product or your service or whatever feature you’re designing for.

Host
It sounds a bit like an “aha” moment. Can you share with us what one of your “aha” moments might have been – or a particularly memorable moment that translated into a product?

Nicole Johnson 
Sure. There’s a feature that’s out already so I can talk about it. So, drive modes for Rivian specifically. Spent a lot of time off-roading myself and off-roading with other experts. And we discovered that there was data and information that would be useful for a customer to have access to. Which again, with a software-defined vehicle, there's so much data available, but at the time - and it was because it was early days - we weren't surfacing that to the customer, and we ended up agreeing that we should start showing some of that. So, it was things along the lines of tire pressure or showing a compass that wasn't there before, temperatures, for example. Now it's interesting because that starts to merge some things where you're like, well, yeah, a car right now shows temperature. Sure, but now it's in context to the drive modes when you're off-roading or when you're driving the vehicle. And so now you bring that together. And, this is me reading Reddit after the fact, after we launched the feature. Mind you, it's not every last customer was ecstatic about it, but those that off-road and those that care about that type of information, were really excited to see it. It gave them a sense of confidence within the feature, and it also just looked really nice. And they were able to have more information, more context, and it was aesthetically pleasing. So, it was kind of Venn diagram, if you will.

Host
What factors do you suggest engineers and designers consider when collaborating on HMI projects?

Nicole Johnson 
This actually ladders into the previous question around being empathetic to the customer. But when you're in the middle of a project, when you're in the thick of it, when there's a deadline, it's not as though the customer gets forgotten, but you are in a bit more of a fight or flight. We need to get to the end light at the end of the tunnel, so to speak. But that's when thinking about the customer at that point becomes even more important when collaborating. And, this is a really important one, which isn't necessarily about the customer, it's agreeing on the problem that you're solving or that you're trying to solve, right? If we're both working towards feature X, but you think that the goal of the feature is to get it out fast, and mine is to make it fun to use, then we have opposing goals.

We have the same goal, which is to get the feature out. But how we do it is something that I think it doesn't get talked about enough to agree on the problem that you're solving for. And that's been something that's been really helpful in collaborating with engineers and developers along my design journey. And, I've seen too many times where everybody's really passionate about getting something out but, the objectives and how we go about it might be different. And so that's why I mentioned earlier, designers and engineers aren't always speaking the same language. So to this, I would say quit being afraid to ask questions, clarifying questions.

And so, that would be my factors that really affect the way designers and engineers collaborate together. And some of the components to that, during the pandemic especially, you're on Zoom, you are on Slack, get somebody's phone number, call them up. Sometimes a phone call changes the entire conversation in 30 minutes than trying to slack back and forth for 45 minutes. So that's what I would say to that.

Host
How do you view HMI in relation to user experience design?

Nicole Johnson 
They're interdependent, especially in automotive, and some could argue it's a fine line. I would say that when designing for something that's as multidimensional as a car, you need both the zoom in and the zoom out. And the HMI tends to be a bit more of the zoom in, focused on the details of how something's working in relation to something physical… could be suspension, could be, like I said, lighting or door handles and things like that. And the UX component of that is zooming out to make sure that the small details are aligning to the bigger picture, which is what problem are we solving? What are we trying to do with this feature for the customer? What are we trying to solve for them with this feature? And so that's where I think, again, they're not completely separate, especially in automotive with software-defined vehicles.

Host
As touch UIs become standard and voice activation technology matures, what do you see as the next leap forward for HMI?

Nicole Johnson
So, this is an unpopular opinion, but I think specifically in automotive, until there's some sort of standardization or governance across OEMs around voice, it will be extremely challenging to reach a level of adoption that I think the industry is trying to get towards. Right now we have Google and we have Alexa as our predominant voice assistants on our phones and our homes. However, the commands for voice in vehicles are murky and they feel a little bit mysterious for most customers. I mean, you know that you can ask Google or Alexa for the time, weather - those types of basic things, but when you come to the physical aspects of the vehicle, doors, trunk, lift gate, whatever, those become like, okay, is that all I can do? What else can it do? 

I learned not too long ago, a customer at Rivian, she was like, “oh, did you know that you can play games?” And everybody in the room was like, what? And it's like, yeah, you don't know what Alexa's capable of. Why? Because it's voice. It's not visual for you. You have to know what the prompts are. And that's where I think that something like gesture on smartphones is a really good comparison. We've now become ubiquitous with gestures on smartphones, swipe up, swipe down, left, press and hold. Those are standard. I wouldn’t say that they’re governed, but they are standard. And if you have a tablet or smartphone, how to interact with that. Voice is just not quite there yet. And given that in a vehicle, your attention is really limited until there's common and easily understood prompts that we all tend to agree on voice and vehicles, I say still in its early days. I think there's promise as far as adoption is concerned, but we're not quite there yet.

Host
Is there anything we haven’t talked about that you’d like our engineering audience to know? 

Nicole Johnson 
I think when it comes to HMI specifically and even user experience, I think a lot of times those two categories tend to get put in a box as to what those designers are capable of. But the really cool thing about UX designers and HMI as well is they come from a lot of different backgrounds, and so I think saying, oh, we just need some design, for example, and what does that even mean? That's where engaging your design counterparts in whatever it is that you're creating. Sometimes I've seen engineers go off and they make something and then they come back and then they're very annoyed when they get feedback versus let's collaborate together because that’s how designers tend to work, right? Designers don't often go into a hiding hole and make something and appear like “ta-dah.” It's usually a collaborative process. If it's done right, it's a collaborative process because you're doing trial and error, you're ideating, somebody else has a different idea, that's cool, let's try that. What about this? And so forth. And so, I think that's just, it's not necessarily easy. I'm not saying somebody's going to be able to do it by snapping their finger and suddenly they're ready to do a bunch of ideation sessions, and I'm not talking about those that just ideation sessions I've seen where you're just kind of pontificating and it's not necessarily getting anywhere to a solution, but staying somewhat open-minded to other ideas about how to solve a problem. That would be it. 

Raymond Yin
We hope you enjoyed this episode of In Between the Tech. This podcast is one part of Mouser's in-depth look at HMI and its increased impact on how easily humans interact with machines. Explore the entire Empowering Innovation Together offering on this subject, with articles, videos and more at mouser.com/empowering-innovation.