Getting texture value of cartesian coordinate from a Obj file

Question

I have an obj file together with an associated mtl file and texture jpg file. I have managed to find some code that sets a specific camera view and raytraces the image to output a depth map plus a point cloud. The code is using assimp and CGAL to do this. I have access to the specific face the ray intersects but I don't know how to find the pixel color value at the intersection point. Can this be done? Or is there are better way to do this using some software or library?

My final goal is to create depth and color images of different viewpoints from a set of viewpoints for given camera parameters.

Answer 1

Your question is very broad because it is close again to "how to I render 3D objects". So some people might give you a more precise answer than mine but at least I will try to give you some pointers.

1. First look on the web. There are plenty of tutorials that explain to you exactly what you to do (which is to raytrace texture mapped objects). While it's great to ask questions, there is really no excuse for posting such a generic question here, considering the amount of information that is available on the topic on the Web.
2. Though I will try to give you some pointers nonetheless (because we are kind and not here to judge each other;0)

What you want to do is called texture mapping. The way it's done is simple. Each triangle of your geometry holds what we call vertex coordinates, basically it explains how a triangle is actually mapped onto an image (or texture). Imagine your triangle laying flat on the image. The part that it lays on is that part of the texture that should appear on the triangle once rendered. Note that texture coordinates are 2D coordinates and that the shape of the triangle in the texture space (when it lays flat on the texture) doesn't have to match the shape of the triangle in 3D space (in which case the texture will be stretched).

So your 3D object when you render it should have a set of 3D points (the vertex position) 3D normals (the normals of the object at the vertex position) and 2D coordinates (read from the OBJ file).

When the ray hits a triangle your compute the texture coordinates at the intersection point using the barycentric coordinates of the intersection point and the texture coordinates set at the 3 vertices of the hit triangle. This is very common technique you will find many tutorials about that.

Then once you have your interpolated texture coordinates you use it to do a lookup into the texture. The texture coordinates are in the range [0:1] (in general), where (0,0) might be the top-left corner if your texture and (1,1) the bottom right. You need to remap that texture coordinate to pixel space (multiply it by image width and height and cast to integers eventually).

Now just read the pixel value at that position (remapped to pixel coordinates), and that's the color of the mapped intersection point.

Easy, breezy!