2
$\begingroup$

I need to extract the RGB information from a a face on a mesh with a script. I know it sounds ridiculous, maybe it is the main reason I could not find any solutions. I know the index of the face but I don't know how to extract the color of the texture on it. The texture of the object is set from an image so I don't know if extracting color is even possible.

$\endgroup$
  • $\begingroup$ Hi, Ján Hanes. What do you mean by 'the color'? If it has some texture on it, it can have a lot of colors. Which one do you need? I think it would be a good idea to define that. Could you provide some context? Why do you need to get 'the color'? What are you trying to achieve? Why does it sound ridiculous? $\endgroup$ – Martin Z Apr 29 at 19:50
  • $\begingroup$ I just need an RGB value. Maybe an average of all colors in the face. I'm trying to scan mesh into a matrix using my own version of marching cubes algorithm combined with BVH tree. It finds the nearest point on mesh (returns face index and distance) and detects if the point is in boundaries of the cube. It works pretty well but for each cube (position in the matrix), I need an approximate RGB value of that nearest face which it returned. $\endgroup$ – Ján Hanes Apr 29 at 22:29
2
$\begingroup$

That's a great question! A given face can have any number of colors contained within the texture though – in order to solve this problem you'll also need a specific point to map to a specific pixel on your face's texture.

I recently solved this problem for the Blender addon Bricker, which applies the nearest texture colors of the original model to each of the generated LEGO bricks. See this link for more information about the problem and how I solved it.

Here's a simplified version of the code I wrote to solve the problem:

import bpy
import math
from mathutils import Vector
from mathutils.interpolate import poly_3d_calc
from bpy.types import Scene, Mesh, MeshPolygon, Image


def getUVPixelColor(mesh:Mesh, face_idx:int, point:Vector, image:Image):
    """ get RGBA value for point in UV image at specified face index
    mesh     -- target mesh (must be uv unwrapped)
    face_idx -- index of face in target mesh to grab texture color from
    point    -- location (in 3D space on the specified face) to grab texture color from
    image    -- UV image used as texture for 'mesh' object
    """
    # ensure image contains at least one pixel
    assert image is not None and image.pixels is not None and len(image.pixels) > 0
    # get closest material using UV map
    scn = bpy.context.scene
    face = mesh.polygons[face_idx]
    # get uv coordinate based on nearest face intersection
    uv_coord = getUVCoord(mesh, face, point, image)
    # retrieve rgba value at uv coordinate
    rgba = getPixel(image, uv_coord)
    return rgba


def getUVCoord(mesh:Mesh, face:MeshPolygon, point:Vector, image:Image):
    """ returns UV coordinate of target point in source mesh image texture
    mesh  -- mesh data from source object
    face  -- face object from mesh
    point -- coordinate of target point on source mesh
    image -- image texture for source mesh
    """
    # get active uv layer data
    uv_layer = mesh.uv_layers.active
    assert uv_layer is not None # ensures mesh has a uv map
    uv = uv_layer.data
    # get 3D coordinates of face's vertices
    lco = [mesh.vertices[i].co for i in face.vertices]
    # get uv coordinates of face's vertices
    luv = [uv[i].uv for i in face.loop_indices]
    # calculate barycentric weights for point
    lwts = poly_3d_calc(lco, point)
    # multiply barycentric weights by uv coordinates
    uv_loc = sum((p*w for p,w in zip(luv,lwts)), Vector((0,0)))
    # ensure uv_loc is in range(0,1)
    # TODO: possibly approach this differently? currently, uv verts that are outside the image are wrapped to the other side
    uv_loc = Vector((uv_loc[0] % 1, uv_loc[1] % 1))
    # convert uv_loc in range(0,1) to uv coordinate
    image_size_x, image_size_y = image.size
    x_co = round(uv_loc.x * (image_size_x - 1))
    y_co = round(uv_loc.y * (image_size_y - 1))
    uv_coord = (x_co, y_co)

    # return resulting uv coordinate
    return Vector(uv_coord)


# reference: https://svn.blender.org/svnroot/bf-extensions/trunk/py/scripts/addons/uv_bake_texture_to_vcols.py
def getPixel(img, uv_coord):
    """ get RGBA value for specified coordinate in UV image
    pixels    -- list of pixel data from UV texture image
    uv_coord  -- UV coordinate of desired pixel value
    """
    uv_pixels = img.pixels # Accessing pixels directly is quite slow. Copy to new array and pass as an argument for massive performance-gain if you plan to run this function many times on the same image (img.pixels[:]).
    pixelNumber = (img.size[0] * int(uv_coord.y)) + int(uv_coord.x)
    r = uv_pixels[pixelNumber*4 + 0]
    g = uv_pixels[pixelNumber*4 + 1]
    b = uv_pixels[pixelNumber*4 + 2]
    a = uv_pixels[pixelNumber*4 + 3]
    return (r, g, b, a)

Assuming your mesh object is active, and the cursor is placed on the mesh in the location you want to extract the color from, you can call the getUVPixelColor function like this (simply replace the image_name argument with the name of your UV texture image):

obj = bpy.context.object
mesh = obj.data
point = bpy.context.scene.cursor.location
face_idx = obj.closest_point_on_mesh(point)[-1]
image_name = "test.jpg"  # replace with the name of your texture image

rgba = getUVPixelColor(mesh, face_idx, point, bpy.data.images.get(image_name))

Note that depending on which color space you're working with, you may have to apply a gamma correction to the color before using it as an RGBA value in Blender.

Hope this is helpful!

$\endgroup$
  • $\begingroup$ Thank you, it is working but function getPixel is somehow pretty slow. I will have to speed it up because I'm working with 532x532x256 array and mesh with multiple HD textures. $\endgroup$ – Ján Hanes Apr 30 at 12:04
  • $\begingroup$ And It returns floats like 0.47088, shouldn't it return integers? $\endgroup$ – Ján Hanes Apr 30 at 12:07
  • 1
    $\begingroup$ Ah, good catch. I made an edit to the first line of the 'getPixel' function to fix this performance issue. I had originally copied the pixels to a new array and passed it as an argument in the 'Bricker' source code for a massive performance gain, but when simplifying for this thread I forgot to remove the array duplication [:] as it's no longer necessary. See my comment after the first line of the 'getPixel' function for more info. $\endgroup$ – Christopher Gearhart May 1 at 7:50
  • 1
    $\begingroup$ As for the float vs. integer values, this returned value is on an RGB scale from 0 - 1, just like in the Blender color picker. To get an RGB value on the 0 - 255 scale, simply multiply the resulting values by 255 and convert to integer. $\endgroup$ – Christopher Gearhart May 1 at 7:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.