# Access color of a point given the 3D position (on the surface of a polygon)

My problem consists in accessing the color of a given point (I know the 3D coordinates (x,y,z) that lies on one surface of a mesh.

I use raytracing to get the 3D position of the point casting a ray from the camera, and I need to select the point that are most far away from the camera (I managed to do that, just discard the closer intersections): now that I have the point, since I have no previous knowledge about how the mesh is colored (I need to develop a general method), I need a way to get the texture and be able to accurately select the color at a given position (I can get the 2D position on the surface, if needed).

The color I need to access is actually the color that would be obtained if the camera would be placed behind the object (but I guess once the information about the color is available, that could be calculated)

I checked other similar questions and found something interesting that I am trying to do right now here: Find the color at a particular surface point on a mesh.

However, I need to this from a python script, not using the GUI. If someone has some other ideas, I would be happy to hear them.

• Is the color to be found using a UV map? – lemon May 8 '17 at 11:22
• Yes, that would be an option. The final information that I need would be something like an RGB color that I can use to render an image. I can access the uv_layers of the meshes, but I don't know exactly how to get the color from that information. I have not so much experience with Blender, so I think I have all the data needed, but still need some time to figure out exaclty where every thing is and how to use it. – elgordorafiki May 8 '17 at 12:05
• Are you talking about the rendered color? Or a color corresponding to a texture point from a given 3D coordinates? – lemon May 8 '17 at 12:37
• The second! Color corresponding to a texture point. I cast a ray from the camera to the object, select the intersection that is the most distance to the camera, and I want to pick the color of that point (actually seen from the other side, but the color is the same, probably I should only take into account illumination) – elgordorafiki May 8 '17 at 12:48
• managed to get V' and the x,y coords from the barycentric_transform. Now a bit lost however as to how to convert that to the rgb values from the image... could anyone point me in the right direction? – Hvidsten Oct 23 '19 at 9:38

Some indications for a way to do that (maybe there are other possibilities... I'd be interested if some other answer could offer something more simple).

From the comments below the question, I assume the following entry parameters:

• A mesh
• A ray cast point 'V' on the mesh, so you also have the corresponding polygon index 'P' (considering you are using BVHTree from the mathutils module)
• A UV map
• A texture

First thing: at some point, you'll need to match V position on P to the V' position on the UV map part where P correspond to. That can be done using barycentric_transform (function which can be found in mathutils.geometry), but a triangle is needed (as we need all to be convex). This means that you need to work on a triangulated mesh (you could also triangulate on the fly, but I don' know how to do it).

So the overall process can be the following:

• Triangulate it
• Ray cast and obtain V and P (a triangle)
• Find the UV map part P' corresponding to P
• Calculate V' using barycentric_transform
• Remap V' on the texture

Some indications for all that using bpy and bmesh modules

Snippet

import bpy

• Triangulate it

Snippet

import bpy
import bmesh
def TriangulateMesh( obj ):
bm = bmesh.new()
bm.from_mesh( obj.data )
bmesh.ops.triangulate( bm, faces=bm.faces[:] )
bm.to_mesh( obj.data )
bm.free()


Eventually, work on a copy:

def CopyMesh( obj ):
copy = obj.copy()
copy.data = obj.data.copy()
return copy

• Ray cast

Snippet

from mathutils.bvhtree import BVHTree
... some code and at the end:
location, normal, index, distance = bvh.ray_cast( ... your parameters )


index here is P.

• Find the UV map part corresponding to P

Get 3D vertices indices

verticesIndices = obj.data.polygons[P].vertices


P vertices coordinates in 3D space

p1, p2, p3 = [obj.data.vertices[verticesIndices[i]].co for i in range(3)]


P UV map vertices indices are found in:

uvMapIndices = obj.data.polygons[P].loop_indices


The UV map

uvMap = obj.data.uv_layers['your uv map name']


The coordinates of P vertices in the UV map space:

uv1, uv2, uv3 = [uvMap.data[uvMapIndices[i]].uv for i in range(3)]

• Calculate V' using barycentric_transform

V' is given by (make uv1, 2 and 3 as 3D vectors before):

V' = barycentric_transform( V, p1, p2, p3, uv1, uv2, uv3 )

• Remap V' on the texture

Get the texture, the pixels and texture dimensions:

image = bpy.data.images['your texture name']
pixels = list( image.pixels ) #Faster than accessible image.pixels[x] each time
width = image.size[0]
height = image.size[1]


From V', get x and y. These values as give as if the image size was (1, 1). So you need to rescale and modulo x and y, then reach the corresponding image color using 'pixels' (which is a linear byte array of individual color components RGBA).

• Thank you very much lemon! I think this was exactly was I looking for. The first steps are already done, until the ray casting (I don't use a pre-made class because I need some particular results on the ray casting, so I wrote a snippet, but I can get V and P without problem. My issue was how to extract the UVMap and map it on the texture. The answer was very fast and exhaustive, thank you again – elgordorafiki May 8 '17 at 13:57
• Hello lemon! I have another small question on the topic. I had some problem the last weeks and couldn't follow my work. Now I came back and did it, everything works fine (at least, it looks like) apart from the texture image. When I retrieve the image from bpy.data.images I don't know which image is associated with each object on the scene, and if I try to extract this information from the object mesh, as described here (blender.org/forum/viewtopic.php?t=26224) and here (blender.stackexchange.com/questions/5121/…) – elgordorafiki May 22 '17 at 13:21
• (continuing on second comment) I fail, because the texture.data[n].image are all None element, so it looks like they are not connected. But if I go to the Material tab and check the texture image, almost every object has its own. Do you have any idea? – elgordorafiki May 22 '17 at 13:23
• @elgordorafiki, knowing/calculating the image corresponding to a particular face (in the case your object has several images) is not easy. You can either look to the texture layers linked to your object or look at the nodes setting. But as I commented below the question the idea is to use a known texture and not a rendered color, so that presumes you know the texture used for an object (if not, hard to know where the texture color is used on the mesh. Example: a Cycles node setting using several textures). – lemon May 23 '17 at 10:14
• Yes, but I do know which image correspond to which face. When I go to the material tab in the GUI, every object has its own material, and in the surface section the color is set to Image Texture and an image is linked (image_n.jpg). Those images are in bpy.data.images[n] and I can find them and access them without problem. My problem is when I access the object mesh, there is no reference to which image is linked to that object. When I look at ob.data.uv_textures.active.data[i].image this is empty, but I would expect this to have the name of the image (in the GUI it does have it) – elgordorafiki May 23 '17 at 10:22