Generate 3d mesh from 2d sprite?

Question

I would like to know if it's possible to automatically generate a mesh based on a texture. Let's say I have a sprite like this:

And I would like to turn it into something like this:

Doesn't need to be exactly like this example, I'm fine if the mesh has flat colors and a flat surface with no additional details.

Is there a way to create a mesh like this from a sprite, preferably in a non-destructive way? The way I imagined it possibly working is by using a flat plane as the mesh, applying the sprite as a texture and using some sort of modifier to add depth to the plane based on the texture. Is there a modifier of some sort that would allow me to do that?

Bonus: if you know how to add detail to the generated mesh like in the example above, please post it too! I don't personally need it, but I imagine other people might find use for it.

Answer 1

You can use Python to do that. Execute this script :

How to run a script in Blender

import bpy

def threedify(image_name, mask_color):
    img = bpy.data.images.get(image_name)
    
    if not img:
        print(f"No image named {image_name} in memory")
        return
    
    # Get or create mesh
    mesh = bpy.data.meshes.get(image_name)
    
    if not mesh:
        mesh = bpy.data.meshes.new(image_name)
    
    mesh.clear_geometry()
        
    # Get or create object
    obj = bpy.data.objects.get(image_name)
    
    if not obj:
        obj = bpy.data.objects.new(image_name, object_data=mesh)
        bpy.context.scene.collection.objects.link(obj)
    
    # Fill verts, faces and colors
    verts = []
    faces = []
    colors = []
  
    img_width = img.size[0]
    for i, col in enumerate(zip(*[iter(img.pixels)]*4)):
        if col == mask_color:
            continue
        x, y = i % img_width, i // img_width 
        verts.extend(((x - 0.5, y - 0.5, 0), (x + 0.5, y - 0.5, 0), (x + 0.5, y + 0.5, 0), (x - 0.5, y + 0.5, 0)))
        verts_amount = len(verts)
        colors.extend((col,)*4)
        
        
        faces.append((verts_amount - 4, verts_amount - 3, verts_amount - 2, verts_amount - 1))

    mesh.from_pydata(verts, (), faces)
    
    
    # Add modifiers
    obj.modifiers.clear()
    solid = obj.modifiers.new(type='SOLIDIFY', name="solid")
    solid.thickness = 0.5
    bvl = obj.modifiers.new(type='BEVEL', name="bvl")    
    
    
    # Set vertex colors
    mode = bpy.context.active_object.mode
    
    if not obj.data.vertex_colors:
       obj.data.vertex_colors.new(name="vcols")
    vcols = obj.data.vertex_colors.get("vcols")
    if vcols is None:
        vcols = obj.data.vertex_colors.new(name="vcols")  
    for i, vcol in enumerate(vcols.data):
        vcol.color = colors[i]

    bpy.ops.object.mode_set(mode=mode)

if __name__ == "__main__":
    threedify("arrow.png", mask_color=(1, 0, 1, 1))

In a nutshell, this scripts iterates over all the pixels of an image which needs to be loaded in memory (case-sensitive), then recreates a mesh with 1 square face = 1 pixel from this image. The important thing is that the base image must have an exact mask color. I used this one made rapidly in paint with a mask key of (Red = 1, Green = 0, Blue = 1, Alpha = 1):

Result (Make sure to enable Vertex color mode) :

With your example - Make sure to replace the last line with

threedify("cbNd2.png", mask_color=(0, 0, 0, 0))

The result is not the same as in your question because your image is actually 72*96 pixels.

Warning : This will likely crash blender or hang your PC for non-pixel-art images.

You can then tweak the modifiers to fine tune the effect.

If you want to actually see it in material preview or rendered view, here is a minimal working example of a shader tree :

Add some lighting, and voilà :

Answer 2

You can use Hair particle system with Cube as Render > Object.
Cubes are emitted based on texture Density.

---

Sprite to Cubes

Like here your image 16 x 16 px ...

• add Plane, Subdivide by 4 to get 16x16 grid, delete one row to get 15x15 faces, because Cubes will be emitted from vertices
• add Particle System > Hair, Source > Vertices with disabled Random, Render > Object (assign Cube), Texture > add New and under Texture Properties assign your image, enable ColorRamp (that switch image to Black&White), under Influence enable Density.

Colors per cube should be easily taken from texture.
Like here, I just cant figure out what is wrong with setup in my file.

Edit: If this is still valid https://developer.blender.org/T42706 - emit from vertex is the issue.

---

Color per Cube

So you would have to go to Modifier Properties > Convert. Join Ctrl+J Cubes (if you want cubes as one object). In Edit mode with all selected Unwrap > From View. Switch to UV layout and in UV Editor switch to Individual Origins and Scale S the UV a bit.

Shader is smart enough ... UVMap is used automatically.

Notes:

• You can use plane and use Subdivision Surface Modifier and Particle System after, just se Use modifier stack if enabled under Particle System
• You can Separate P by Loose parts back into individual cubes if you need it.
• You can use just a Plane (instead of Cube) for whole proces to make it lighter and use Solidify modifier as final touch :)
• By default are images interpolated, to keep lowers images pixelated choose in Shader editor in Texture node > Closest. For Particle System Texture go to Properties Editor > Texture > Sampling > Interpolation disabled
• in some of my screens is missing one Cube at a hat because I used Mapping > Generated instead UV

Answer 3

Image to Volume

In some cases can be enough just volumetric shader.
This example rendered with 10 samples, Denoiser enabled.
It is just a few second to render.

Render Properties > Volume > Step Rate 0.01

Viewport screenshot is only Volume material - means it can have only a diffuse look. You can get glossiness for front / back by connecting Principled BSDF node to Surface socket, but sides (thickness) stay volumetric.

By adding PrincipledBSDF to surface socket you can benefit also in other areas

• sharp texture
• shorter render-time (switch from NLM denoiser to OpenImageDenoise cut render-time to half)

---

Originaly I wanted after this step use another object with Volume to Mesh modifier plus Remesh modifier, but it seems Volume to Mesh modifier supports only imported VDB files at the moment.