What stops the auto-generated 3D worlds of Google Earth Pro from being MUCH more detailed and accurate?

Question

I'll admit it blew my mind when I first realized that you could actually enter a "first-person mode" in Google Earth Pro, and not just view the 3D maps from a floating camera in the air. But once the initial "wow factor" wears off, you quickly start to get annoyed and frustrated with how flawed it is.

Allow me to give you a couple of random examples where I compare the ground-level 3D representation to the Street View photo of the same locations and angles:

There are much worse examples, but I wanted to pick locations that actually have a Street View to show the comparison. For example, any kind of forest path location (for example) which I've been to in real life and which is not an actual street where Google cars can go are just bizarre, surreal nightmares of forest textures looking like mountains shooting into each other, like this:

Yes, you can argue that it's amazing that this can exist at all, and it's "not really meant to be inspected so closely and from a first-person perspective", but that's at least what I personally am interested in. My dream is to be able to walk around in something that looks like "Street View" but is actually a fully walkable 3D environment. Clearly, the current technology is lightyears from that dream.

But what exactly is limiting the realism? They seem to somewhat be able to make up "rough" representations of buildings and geometry. They certainly seem to have no lack of data to work with. So why does it look so "rough"? What is limiting their current program that analyses all their collected and bought data from producing much more clean and accurate representations of the surroundings?

I haven't been able to find any video where engineers at Google talk for hours about how exactly they make these 3D environments, but it seems odd to me that they are able to get them this "good", yet still fail at the end of the day.

Again, it looks very cool at first, but you soon realize that it's useless for anything but a novelty and "maaaaybe this one place looks pretty cool", but then you just get disappointed when it's just a big mess, like every other place. Unless you exit the mode and let the camera float around in the air, but again, that's not what I'm after. I want the "FPS freedom" controls with breath-taking detail. It would be insanely cool, but I have a terrible feeling that I'll be dead and gone when this ever becomes possible.

What are the main limiting factors?

Answer 1

These views are generated using photogrammetry. Usually by vehicles that are colleting GPS data about the streets at the same time. As the data collection vehicle drives by it take multiple photo's which are geolocated with the GPS data. The photo's are then used to generate rough 3D views.

Photogrammetry can be generated with just a handful of views but the results are limited to the quantity, quality (not the same quality as our eyes see though) and angles of the images, and to the ability of the software generating the views to correlate the images. It is much more difficult to correlate multiple images of a forest scene, then it is for a row of nice geometric objects like cars, hedges and homes.

This process is fundamentally limited because of the way the data is being collected. Typically a high quality photogram needs many angles and many photos to create a detailed image. A car driving down the road can only get so many images of its surroundings, with very limited angles. The results will probably improve as the software improves but until they find a way to gather much more data the 3D views will probably only improve incrementally. That said, generating 3D views of streets from a few pictures taken while driving by is pretty amazing stuff.