All the Ways Google Gets Street View Images
In 15 years, Google Street View has circled the planet 400 times. WIRED walks through all those years of gadgets and gear to understand the tools Google Street View uses to map the world.
Video Transcript
- 102 countries, seven continents, 10 million miles of road-- in 15 years, Google Street View has circled the planet 400 times.
ETHAN RUSSELL: Here, you can see where we've mounted the Street View car onto a three-wheeled tricycle. Here, it's been mounted to a snowmobile. One of our earliest trekkers, a trolley through museums, places that expect there to be a lot of change-- we try to drive those areas more frequently. There are other areas of the world where we still have to go back and refresh that imagery again.
- Let's walk through 15 years of gadgets and gear to understand the tools Google Street View uses to map the world.
ETHAN RUSSELL: Maps are fundamentally about where things are. Think about a satellite map or even the map that you draw on a napkin for your friend. You're talking about where things are and where they are in relation to each other.
- And in that sense, Street View is different. It promised an immersive, on-the-ground experience of far-flung locales. Street View was the brainchild of Google co-founder Larry Page.
ETHAN RUSSELL: He had what was a pretty crazy idea at the time-- that you should enable people to explore the world right from their computers. He actually collected the first Street View imagery. He used a camcorder, and drove around the streets around the Google office, and brought this back for the engineers to play with.
And the engineers built a custom prototype camera system that had multiple camera units and multiple laser scanners. The whole thing weighed 500 pounds. The team used this to collect imagery around the Bay Area. The very first version, which had a small four-megapixel camera, it just took pictures.
And we had to figure out exactly where these pictures were taken using a GPS unit that followed along with the car. The idea was, how can we stitch this together into a seamless experience that can let people explore the world? The lights turned on for Street View in May of 2007. That was when we had the first consumer-facing release with five Street View cities worth of imagery collected.
- San Francisco, New York, Las Vegas, Miami, and Denver were the first. But the project expanded, covering more countries and thousands of miles of streets all using pretty much the same underlying process.
ETHAN RUSSELL: At the most basic level, it's all about collecting imagery of the world and processing this into one seamless model. A Street View car starts as a regular car that you could buy. We mount a custom roof rack onto the top, and then the camera gets mounted on a mast that can be raised and lowered.
The Street View camera has seven lenses and it takes full 360-degree panoramic shot a couple of times a second, so you'd be able to traverse the length of a road from picture to picture to picture. Using a technique called photogrammetry, which is a really old technique actually from the 1800s, you can take measurements from photos and you can basically figure out where objects are, how big they are, how far apart they are.
There's a cable that comes off the camera system, goes into the backseat, where it connects to the processing system and the hard drives for storing all of the imagery data. In the back, there were boxes of CPUs and hard drives in them that would collect all of the data. This typically took up a big portion of the backseat of the car.
And it's getting stored basically on the hard disk. This information is sent in batch, and then it's fed into our imagery processing pipelines. The cameras themselves have gotten much better. Not only have they gotten higher resolution, going from 4.8 megapixels to a 45-megapixel camera, 75, up to 140 megapixels-- their responsiveness in low light is much better.
- Apart from improvements in the cameras, the game-changing tech to the Street View system was the addition of LiDAR scanners-- first in 2008 and updated in 2013.
ETHAN RUSSELL: The LiDAR works with millimeter accuracy. They're laser radars. They have invisible laser beams that shoot out and map the world in 3D as the car moves down the street. So it helps us measure where the curbs are or even where there are lines of paint in the middle of the road.
All this information comes back and helps us build this very precise, highly detailed 3D reconstruction of the world. Every route is planned in advance. This is actually important from a privacy perspective.
We try to standardize our Google Street View hardware as much as possible. But sometimes there are local quirks that mean we have to introduce something a little bit different. This is our main car that we use in North America. And the mast on top of the car is pretty tall. This is so the camera can see you over parked cars that might be on the side of the street.
When we first took this car to Japan, we ran into a problem-- the streets are very narrow. And, oftentimes, people put up fences in front of their yard so that their yard won't be visible from the street. And we discovered that the Street View camera was actually seeing over these fences into people's yards. So we came up with a custom rig for the Japan market. And you can see that this car has a much lower mast.
- Another milestone expansion of the program was Trekker Gear, which took image-gathering off-road in 2012.
ETHAN RUSSELL: We realized that there were so many important places on Earth where you simply couldn't drive a car. Even in a city, there are narrow alleys, there are pedestrian walking paths. We'd like to take you inside transit stations or maybe even inside a museum-- or out into the wilderness-- to the top of Machu Picchu, and the top of Mont Blanc, down into the Amazon rainforest, and underwater by the Great Barrier Reef-- or up in the International Space Station.
The Street View Trekker is basically a Street View camera system that's attached to a backpack. Here, you can see the camera unit on top with the lenses. Here, you can see a GPS unit. And these are our laser scanners.
This is the processing unit. And what you see on the back here is a big heat sink, because it takes a lot of processing to handle all this imagery. And the unit can get hot. You actually put this on wearing the straps.
It's relatively comfortable to wear. But it's not the lightest thing, weighing in at about 35 pounds. You can just walk naturally and it will automatically snap photos as you're walking down the path. You can be tipping from side to side, and you don't want the pictures to come out blurry. So we use a lot of smart software algorithms.
- The typical Trekker outing takes a few hours, unless it's a hike down into the Grand Canyon. That'll take a Trekker 10 days. Google collects this imagery themselves through staff trekkers, through a network of third party contractors, or by tapping a community of contributors who use their own 360 cameras.
ETHAN RUSSELL: My favorite story about the Street View Trekker is about the Faroe Islands. Some enterprising folks who live there, they wanted the Faroe Islands to be on Street View as well. We actually sent them the Street View equipment, and they used sheep to carry the Street View camera around-- Sheep View.
One thing that we tried was putting high definition cameras that were facing to the sides so we could read small print that was on the storefronts of businesses. Think about opening hours that are on the door of a business, that it's really hard to see from the street.
- This side profile camera configuration was paired with machine learning and image recognition software, which was already in Google's wheelhouse.
ETHAN RUSSELL: The engineers who first worked on this project had actually been working on the Google Book Search project. They were experts in processing imagery. And the book search project was trying to digitize all of the world's books in libraries. So these people knew how to work with imagery.
And at first, we had humans looking at this imagery combined with thousands of other data sources in order to build the roads and the business listings that you see on Google Maps. But the world is a really big place. So it wasn't too long before we started introducing machine learning to look at the images and automatically extract information.
Blurring and privacy protection has been built in pretty much from the start. This is a process that's fully automatic. All of the people's faces, all the license plates, these are blurred through automatic algorithms. Because, otherwise, the scale of processing this which humans would just be too great.
- Recently, Google unveiled for us the latest camera set to roll out on Street View cars over the next year.
ETHAN RUSSELL: Our current generation cameras, they're designed for specific models of car that need to have a custom roof rack. With this, we can send it around the world. It has handles to make it easy to pick up.
The whole thing weighs less than 15 pounds. And one of the key things about this is it can be mounted on top of any vehicle at all. We can give it to a third party, and they can put it on any car that has a roof rack.
There are seven lenses arranged around the outside and one that's pointing upwards from the top. As the camera moves down the street or on top of a backpack, it takes a full 360-degree panorama. These are heat sinks. We've taken all of the processing power that used to take up the entire backseat of a car, and we've shrunk it down, and we've stuck it inside the camera unit so this is all you need in order to go collect Street View imagery.
- Google is innovating on the software side too, with Immersive View.
ETHAN RUSSELL: We launched Immersive View, which is all of the billions of Street View images and aerial photos. Let's say you're planning a trip to London-- we can take you to Westminster, and you can see where is Big Ben, and where's the London Eye, and where are they in relation to each other?
We can then overlay additional information on top of this model of the world. If you want to see what it looks like with the sun coming from one direction or the sun coming from another direction using all of this imagery that we've collected, we can actually put these experiences into what we've collected from Street View. And when you're ready to choose a restaurant, we can fly you down to street level and show you real-time busyness and these future rich, immersive experiences. This is what Street View will empower. And hopefully it's helped people learn more about places around the world, about other people's communities, and about their fellow citizens.
