How we make our HD maps

HD maps are critical to making autonomous driving safe. To make them at scale, TomTom uses artificial intelligence to create HD maps that are an accurate representation of reality.

It’s Easter morning, 1900, and the weather is perfect. An intrepid pioneer of early photography sets up his tripod and gets a killer shot of 5th Avenue, New York City. There’s a sea of horses pulling carts as people rush to get to work. There’s one solitary car. A pioneer of automobiles.

Fast forward to Easter morning, 1913, 5th Avenue, New York City. The camera angle is different, this time taken from the top deck of a tram. But the buildings are undeniably the same. This time, there is a sea of cars. There is one solitary horse and cart. The pioneers are now the mainstream adopters of a new form of transport.

That’s how fast technology can change an entire cityscape and the way people use it to change the way they live.

In the next few decades, we’ll be facing just such a change in the way we use the automobile. The cars will look similar, but not the same. They’ll undoubtedly be electric, or hydrogen-powered, but that won’t change the way they look. But peer closer and you’ll see a radical redesign.

The drivers’ seat has gone. There’s no steering wheel. The internal seating is different, and passengers now sit facing one another. Cars are multi-use so they are always full of people. Look at the street and you’ll see the traffic lights and road signage have gone.

Welcome to a new age – one of the autonomous vehicle. The change is likely to be as rapid as the New York rush hour over a century ago. But the good news, especially for us at TomTom, is they will still always need a map.

Why do HD maps matter?

So why do maps matter to an autonomous vehicle? Well, over a million people die in traffic accidents every year on a global scale. It takes only one small incident featuring an autonomous vehicle to make headlines about the reliability of autonomous driving solutions.

For society to accept robot drivers, it’s clear the technology will have to be much safer than human drivers. High Definition (HD) maps are essential to providing enhanced safety by processing data from millions and millions of sources, including vehicles, in real-time to create instant maps.

By portraying maps in centimeter-scale and combining this with up-to-the-second traffic information, autonomous vehicles will be able to make instant decisions, far faster than a human would. In fact, once the technology does prove itself, it could lead to a ban on human drivers altogether.

HD maps are an essential component in autonomous vehicles. Maps are vital for filling in those blanks caused by high-rise buildings or around corners. Maps can complement sensors’ capabilities. They provide human drivers and autonomous cars with the gift of foresight to understand what is happening around them and on the road ahead.

HD maps are not limited to autonomous driving but can also be leveraged to fulfill an extensive portfolio of advanced driver assistance (ADAS) applications. These range from predictive powertrain control and highway pilot to adaptive cruise control.

At TomTom, our goal is to map the entire world for autonomous vehicles to drive safely. For this to happen, we must map tens of millions of kilometers of road every year. Making HD maps is not only a challenge because of the sheer amount of data that needs to be captured and stored, but also because of the intrinsic accuracy requirements.

To tackle this challenge, we rely on automation to scale the production pipeline. Using deep learning and machine learning algorithms, we are training AI models to perform tasks more accurately and at a much higher speed than humans.

Machine learning is all about learning from data and using the right kind of data in the right way in order to solve a certain problem. Its end goal is to gain the ability to make predictions based on previously unseen data.

Let’s take the example of traffic signs. To classify them, you would first need to show a machine learning algorithm many images with variations of traffic signs, from which it could learn a model for how each sign looks like and what makes it distinguishable. Once the algorithm would encounter a never-before seen image of a stop sign, it would be able to recognize it.

A similar approach is required to detect traffic signs. However, instead, you would need several images of road scenes with a rectangular box (or a mask) on top of each sign in order to teach the algorithm how a sign looks and where it usually is located in the environment.

This process extends to many other scenarios that are relevant to autonomous driving. One other example is labeling road geometry, such as signposts and road markings and integrating this information in our HD maps. Making accurate and consistent annotations is key to a map that knows every part of the road environment from every angle and condition.

Using AI is not only helping us achieve what would have otherwise been impossible in mapmaking but also accelerate the progress towards Level 5 autonomy. Let’s look at four important reasons why AI is key to creating HD maps.

Manual annotation simply does not scale. When the task at hand is to detect 30 types of lane dividers, some of them, such as solid and dashed lines, occurring frequently, and others only found in certain countries, it takes even experienced human map editors a considerable amount of time to annotate each divider correctly.

When done by a machine learning algorithm this process is sped up considerably, allowing for ten times more annotations to be made in the same time it would take someone to go through this manually.

For HD maps to be reliable, they must offer an objective and consistent representation of the road at all times. When humans are in charge of interpreting and labeling data, there is always a margin for error due to our inherent bias. This is because instructions can be interpreted in slightly different ways by different annotators when something is not 100% clear.

Let’s take the example of an image taken by a MoMa car, which needs to be processed for recognizing elements that will then be fed into the HD map. When an element in the picture is open for interpretation – be it a road sign or a lane divider – different annotators might label it differently, leading to clusters of different types of annotations due to several ways of interpreting the same element. When a machine learning model takes over interpretation is more consistent and stable, leading to a cleaner set of data.

Although high-quality manual annotations are often crucial for training robust machine learning algorithms, and human operators are an important stronghold for quality checking our HD maps, the key is to find the sweet spot where the minimal amount of human intervention is required and quality is kept within requirements.

For example, at TomTom, we use AI to aid manual annotations. Human operators are presented with annotations generated automatically by the machine learning model and their task is to check them and correct any mistake.

The corrections are then used to train an improved machine learning algorithm that will be less prone to mistakes, thus closing the virtuous circle that will lead to the desired sweet spot. With time, human operators can be reassigned to more meaningful and less tedious work.

For HD maps to play their role successfully, they need to reflect changes that happen on the road as close to real-time as possible. AI helps us reduce processing time as much as possible in order for updates to reach the map faster.

Using AI also leads to reduced costs. Once trained, the machine learning model can perform tasks efficiently. When the choice comes between training an algorithm or hiring several hundred people to process the same amount of data, it becomes clear how significant cost reductions can be if the technology is used instead of or to complement the work of human annotators.

The safest autonomous vehicles are those with HD mapping capabilities. It helps them see further than their sensors and with far greater context. It results in more accurate driving and faster, better decision making when the unexpected occurs.

For TomTom, using AI to create HD maps leads to peak accuracy at the lowest cost possible, enabling us to help the automotive industry reach higher levels of automation, faster.

So let’s take a photo of 5th Avenue in New York City today and compare the same shot in 10 years' time. I have a feeling we’ll be as surprised by the changes as we were almost 100 years ago…