Enhanced Visual Cues for deaf drivers: Navigating without sound

Navigation apps are filled with options. We can change orientation, zoom level, use audible alerts and more, but what is there for deaf drivers? According to the WHO's projections, there will be close to 700 million individuals that will need hearing rehabilitation due to severe loss of hearing by 2050. Such numbers suggest we need deaf-friendly navigation.

Let’s paint a picture

Picture this, you’re driving on the highway and your TomTom tells you to take the next right. You hear it just in time to make the turn. Next, it notifies you of an upcoming speed camera. You slow down enough to avoid a speeding ticket.

Now imagine this scenario again, except you’re hard of hearing — you keep glancing at your TomTom, taking your eyes off the road as you do. Eventually, you notice you need to turn. As you focus on the road, you don’t hear the speed warning notifying you of a change in speed limit. You receive a fine in the mail a week later.

This is just one example of how difficult it is to drive with a hearing impairment. So, what can we do to improve the navigation experience for the hard of hearing? How do we solve issues like this?

It all starts with an idea

TomTom software engineer Timon Kanters is usually the one that drives when he goes on trips with his girlfriend. Being a TomTom’er he naturally uses the company’s GO Navigation app to find his way around. Sometimes his partner would take the wheel, but she’s deaf and when she drives, Kanters would notice that her experience of the navigation app wasn’t as great as he would have hoped.

Kanters reasoned that it’s “Mainly because it wasn’t really oriented towards people with hearing impairments like her, and we noticed that it's very easy for her to miss exits”.

This inspired Kanters to create a feature for GO Navigation that could be used by drivers who are hard of hearing. That feature only ever made it to testing and was never publicly released, however, a version of it is now available in AmiGO, TomTom’s free navigation app. Before we get to how it was made, let’s look at what it does.

How it all works

When this feature was being developed for GO Navigation it used Enhanced Visual Cues to catch the driver’s attention instead of the typical audio cues. These visual cues include an early turn warning, turn signals and arrival alerts.

When Enhanced Visual Cues mode was enabled, it flashed transparent arrows pointing left or right depending on the upcoming turn. It was designed to catch the driver’s eye without covering the map entirely. This way, the driver could see the instruction in their peripheral vision without having to take their eyes off the road longer than necessary.

How Enhanced Visual Cues work within AmiGO. The first image shows the arrival indicator, and the second image shows the turn indicator.

As for the other instructions — the screen would briefly flash white to signal an upcoming speed camera. If you need to turn left, right, or even pull off a U-turn then a transparent version of the appropriate arrow marker would flash across the screen. They were eye-catching enough to engage your peripheral vision but not so distracting that they took your attention off the road.

Note: At the time of writing, the feature, now live in AmiGO’s public release, includes only turn warnings, turn signals and arrival alerts. But its developers tell us more functionality is in the pipeline.

Pressure can make diamonds

Kanters first developed this feature at a TomTom Hackathon, a competitive event where software engineers and other specialists come together to design and create disruptive software or hardware. This is typically done within 24 hours and the products/features made in these hackathons aim to benefit the world of mobility. It was here, that Kanters first breathed life into his idea, he gave it the working name: ”Hearing Impaired Mode*.”

His idea resonated across the company. His colleagues saw his passion for the project and how it could make a difference. They wanted to help him succeed. “I felt that this was really important for Timon. And I really wanted to help him somehow and I didn't want to fail”, Ilse Vitse, team lead of Kanters’ Hackathon group, tells me. “That was my driving factor, I saw his passion and it just turned over to me as well.”

While the team was developing the feature, they originally aimed to integrate it with the GO Navigation app, but as time progressed, they saw a bigger opportunity.

Jump forward two years. The team began to integrate this feature into TomTom Digital Cockpit, TomTom’s open digital cockpit platform. More team members were joining the project, spurred onward by Kanters’ infectious vision. Kanters’ description of how everybody worked together reminded me of a well-oiled machine, every individual cog working in harmony, moving from strength to strength as the feature continued to evolve.

Before they knew it, they had progressed through two hackathons. Kanters’ team didn’t waste the opportunities, using both events as steppingstones to improve the feature. They made further adjustments to the visuals, made a few more tweaks and finally began integrating the feature into AmiGO, TomTom’s free navigation app.

TomTom’s free navigation app AmiGO, available at your favorite app store.

The feature is, at the time of writing, being rolled out to all users of AmiGO. It has been available in Beta for a few months already — the integration marks an important point in the “Hearing Impaired Mode” story. AmiGO is a free app, and it’s available to anyone with a smartphone, meaning there is no cost or barrier for hearing-impaired drivers to get a tailored navigation experience.

Promises for the future

Testing has played an important role in Hearing Impaired Mode’s development. The team conducted several studies that helped them prioritize design elements and decide what new features to introduce.

The first survey was sent to 43 drivers who are hard of hearing. The results showed the app worked well. However, users requested the visuals be changed to make the arrows more transparent to not fully cover the map.

Katie Snell, a TomTom software engineer in the AmiGO team at TomTom, explained, “We decided to display an early indicator to show the instruction is coming up some time ahead. It's a bit further down on the screen, and pulses around the next instruction icon. The idea here was to make the user prepared to execute the instruction, giving them time to prepare for it.”

While the feature was originally inspired by a deaf driver, in testing, it proved to be incredibly useful for motorcyclists who wear helmets and earplugs and are often subject to lots of distracting road noises. Surveying 12 bikers, the team found that 80% of them were happy with the feature, even though it wasn’t explicitly designed for them.

This evidence suggested the team was witnessing the ‘curb-cut effect’ — a phenomenon where initiatives designed to benefit vulnerable groups often end up benefiting a much larger, broader group.

Take ramps over curbs for example. Designed to provide wheelchair access to the pavement they are used for far more than just that. Skateboarders and cyclists use them and even delivery drivers who are pushing a big trolley of parcels use them for easy access.

What other options are there?

Sadly, the market has very few navigation features tailored to deaf individuals. With Hearing Impaired Mode, we finally see a navigation feature that is not only tailored to helping deaf drivers, but any driver who doesn’t like having to — or in the case of bikers, can’t — turn down the volume to hear their navigation instructions better.

Despite there not being many other alternatives, the best example of another navigation system would be Hyundai’s audio-visual conversion (AVC) and audio-tactile conversion (ATC).

The AVC enables communication with whatever happens outside the car by showing visual representations of sound patterns outside the vehicle on the Heads-up display. The steering wheel is also equipped with colored LEDs to relay information about the outside world to the driver.

The ATC system translates sound outside the vehicle into vibrations through the steering wheel, letting the driver know about obstacles and the distance from the car. Hyundai showed off how this tech works by installing them into a taxi in South Korea dubbed “The Quiet Taxi”. The driver of the taxi is deaf and navigated entirely using these tools.

Can you imagine how much more difficult it is to be a professional driver with hearing loss? Follow the link above to see what that experience is like.

While these technologies are useful, the major downside is their cost. It would be very expensive to outfit a whole line of cars with AVC & ATC systems. They tend only to be available to the minority.

What’s next for Hearing Impaired Mode?

TomTom’s Enhanced Visual Cues mode is now in TomTom AmiGO, the company’s free navigation app for drivers. This means anyone with a smartphone will be able to use it, making it probably one of the most attainable accessibility features in any navigation app. The app’s developers tell me Enhanced Visual Cues is being released for everyone using the app today.

Speaking with Kanters, it’s clear that he’s in pursuit of perfection. Along with his cohort of dedicated and inspired colleagues, it seems that we can look forward to a more inclusive and accessible future of navigation.

* The team has called it Hearing Impaired Mode as it’s designed to offer a functionality that supports navigating while driving for anyone that may be hard of hearing, deaf, or have their hearing restricted, such as motorbikers. It does not refer to the individuals that may use it. We recognize the University of Washington’s description and use of terminology.