Recreate geometry from normal map

I have a group of low polygon meshes, with associated normal maps. The normal maps were created by baking from high polygon meshes, which I no longer have access to. Is it possible to recreate anything approximating the original high polygon meshes from my inputs?

I wouldn't expect the output to have the same kind of mesh structure, but want any kind of mesh that would have the same shape / volume as the original high polygon meshes. I would prefer the output to be a mesh, but it could also work if I could get a point cloud or set of voxels that recreated the volume of the original high polygon mesh.

Further clarification: the low poly mesh was created manually from the high poly mesh by retopologising and deleting polygons.

I've also attached a screenshot showing what I mean (as well as the original mesh file in .obj format). It shows a close-up of two meshes of the handle of a sword. The left one is a high poly one (you can see the ridges in the handle), while the one on the right is the low poly version (it is flat instead of having ridges, because the ridges are captured by the normal map). In my use case, I have the low poly mesh (like the one on the right) and a normal map, and want to get the high poly version (like the one on the left).

Sword handles image file

Swords mesh file

• I'll just a comment until I can find the source paper/notes, but a while ago I had a similar problem where I wanted to recreate a height map from the normal map. I tried to use some work published by (I think) Nvidia. It wasn't perfect, but it sounds like you would need this as a first step. I'll see if I can find the link – Simon F May 16 at 16:09
• I'm not sure if these were the ones we looked at, but perhaps they may be of use: Dave Eberly: geometrictools.com/Documentation/… Siggraph 2011: developer.download.nvidia.com/assets/gamedev/docs/… – Simon F May 16 at 16:35
• What do you mean with normal map associated to the mesh? How did you generate the normal map? Also how did you generate the low poly mesh? – user8469759 May 16 at 17:55
• Depending how much info you can share we can give an exact solution. It is also possible that your low poly mesh is already the best approximation of the original high poly one. If not you can try to minimize some cost function of the form $$\int_{\mathcal{M}} \lVert n(p) - n_{map}(p) \rVert^2 \mu(dp)$$, you can minimize such integral by using gradient descent, the process is analogous to what @SimonF proposed, however gradient descent on the integral above will take into account arbitrary topology. But you'll need at least boundary conditions. – user8469759 May 20 at 14:59
• Can you share two 3d models? – user8469759 Jun 4 at 17:54