I need to extract the RGB information from a a face on a mesh with a script. I know it sounds ridiculous, maybe it is the main reason I could not find any solutions. I know the index of the face but I don't know how to extract the color of the texture on it. The texture of the object is set from an image so I don't know if extracting color is even possible.
That's a great question! A given face can have any number of colors contained within the texture though – in order to solve this problem you'll also need a specific point to map to a specific pixel on your face's texture.
I recently solved this problem for the Blender addon Bricker, which applies the nearest texture colors of the original model to each of the generated LEGO bricks. See this link for more information about the problem and how I solved it.
Here's a simplified version of the code I wrote to solve the problem:
import bpy import math from mathutils import Vector from mathutils.interpolate import poly_3d_calc from bpy.types import Scene, Mesh, MeshPolygon, Image def getUVPixelColor(mesh:Mesh, face_idx:int, point:Vector, image:Image): """ get RGBA value for point in UV image at specified face index mesh -- target mesh (must be uv unwrapped) face_idx -- index of face in target mesh to grab texture color from point -- location (in 3D space on the specified face) to grab texture color from image -- UV image used as texture for 'mesh' object """ # ensure image contains at least one pixel assert image is not None and image.pixels is not None and len(image.pixels) > 0 # get closest material using UV map scn = bpy.context.scene face = mesh.polygons[face_idx] # get uv coordinate based on nearest face intersection uv_coord = getUVCoord(mesh, face, point, image) # retrieve rgba value at uv coordinate rgba = getPixel(image, uv_coord) return rgba def getUVCoord(mesh:Mesh, face:MeshPolygon, point:Vector, image:Image): """ returns UV coordinate of target point in source mesh image texture mesh -- mesh data from source object face -- face object from mesh point -- coordinate of target point on source mesh image -- image texture for source mesh """ # get active uv layer data uv_layer = mesh.uv_layers.active assert uv_layer is not None # ensures mesh has a uv map uv = uv_layer.data # get 3D coordinates of face's vertices lco = [mesh.vertices[i].co for i in face.vertices] # get uv coordinates of face's vertices luv = [uv[i].uv for i in face.loop_indices] # calculate barycentric weights for point lwts = poly_3d_calc(lco, point) # multiply barycentric weights by uv coordinates uv_loc = sum((p*w for p,w in zip(luv,lwts)), Vector((0,0))) # ensure uv_loc is in range(0,1) # TODO: possibly approach this differently? currently, uv verts that are outside the image are wrapped to the other side uv_loc = Vector((uv_loc % 1, uv_loc % 1)) # convert uv_loc in range(0,1) to uv coordinate image_size_x, image_size_y = image.size x_co = round(uv_loc.x * (image_size_x - 1)) y_co = round(uv_loc.y * (image_size_y - 1)) uv_coord = (x_co, y_co) # return resulting uv coordinate return Vector(uv_coord) # reference: https://svn.blender.org/svnroot/bf-extensions/trunk/py/scripts/addons/uv_bake_texture_to_vcols.py def getPixel(img, uv_coord): """ get RGBA value for specified coordinate in UV image pixels -- list of pixel data from UV texture image uv_coord -- UV coordinate of desired pixel value """ uv_pixels = img.pixels # Accessing pixels directly is quite slow. Copy to new array and pass as an argument for massive performance-gain if you plan to run this function many times on the same image (img.pixels[:]). pixelNumber = (img.size * int(uv_coord.y)) + int(uv_coord.x) r = uv_pixels[pixelNumber*4 + 0] g = uv_pixels[pixelNumber*4 + 1] b = uv_pixels[pixelNumber*4 + 2] a = uv_pixels[pixelNumber*4 + 3] return (r, g, b, a)
Assuming your mesh object is active, and the cursor is placed on the mesh in the location you want to extract the color from, you can call the
getUVPixelColor function like this (simply replace the
image_name argument with the name of your UV texture image):
obj = bpy.context.object mesh = obj.data point = bpy.context.scene.cursor.location face_idx = obj.closest_point_on_mesh(point)[-1] image_name = "test.jpg" # replace with the name of your texture image rgba = getUVPixelColor(mesh, face_idx, point, bpy.data.images.get(image_name))
Note that depending on which color space you're working with, you may have to apply a gamma correction to the color before using it as an RGBA value in Blender.
Hope this is helpful!
Unfortunately I don't have 50 reputation yet to comment Christophers answer, so I need to write another answer.
I also had the problem of finding the texture color at a given point on a mesh. This point was calculated using this function. Christophers approach worked fine as long as the mesh was not translated. After some trial and error, I figured out that it is necessary to transform the point coordinates into the object's cooridnate space by using
point = obj.matrix_world.inverted() @ point. Maybe this hint is useful for someone also dealing with this problem.