2
$\begingroup$

I would like to know if it's possible to automatically generate a mesh based on a texture. Let's say I have a sprite like this:

enter image description here

And I would like to turn it into something like this:

enter image description here

Doesn't need to be exactly like this example, I'm fine if the mesh has flat colors and a flat surface with no additional details.

Is there a way to create a mesh like this from a sprite, preferably in a non-destructive way? The way I imagined it possibly working is by using a flat plane as the mesh, applying the sprite as a texture and using some sort of modifier to add depth to the plane based on the texture. Is there a modifier of some sort that would allow me to do that?

Bonus: if you know how to add detail to the generated mesh like in the example above, please post it too! I don't personally need it, but I imagine other people might find use for it.

$\endgroup$
8
$\begingroup$

You can use Python to do that. Execute this script :

How to run a script in Blender

import bpy

def threedify(image_name, mask_color):
    img = bpy.data.images.get(image_name)
    
    if not img:
        print(f"No image named {image_name} in memory")
        return
    
    # Get or create mesh
    mesh = bpy.data.meshes.get(image_name)
    
    if not mesh:
        mesh = bpy.data.meshes.new(image_name)
    
    mesh.clear_geometry()
        
    # Get or create object
    obj = bpy.data.objects.get(image_name)
    
    if not obj:
        obj = bpy.data.objects.new(image_name, object_data=mesh)
        bpy.context.scene.collection.objects.link(obj)
    
    # Fill verts, faces and colors
    verts = []
    faces = []
    colors = []
  
    img_width = img.size[0]
    for i, col in enumerate(zip(*[iter(img.pixels)]*4)):
        if col == mask_color:
            continue
        x, y = i % img_width, i // img_width 
        verts.extend(((x - 0.5, y - 0.5, 0), (x + 0.5, y - 0.5, 0), (x + 0.5, y + 0.5, 0), (x - 0.5, y + 0.5, 0)))
        verts_amount = len(verts)
        colors.extend((col,)*4)
        
        
        faces.append((verts_amount - 4, verts_amount - 3, verts_amount - 2, verts_amount - 1))

    mesh.from_pydata(verts, (), faces)
    
    
    # Add modifiers
    obj.modifiers.clear()
    solid = obj.modifiers.new(type='SOLIDIFY', name="solid")
    solid.thickness = 0.5
    bvl = obj.modifiers.new(type='BEVEL', name="bvl")    
    
    
    # Set vertex colors
    mode = bpy.context.active_object.mode
    
    if not obj.data.vertex_colors:
       obj.data.vertex_colors.new(name="vcols")
    vcols = obj.data.vertex_colors.get("vcols")
    if vcols is None:
        vcols = obj.data.vertex_colors.new(name="vcols")  
    for i, vcol in enumerate(vcols.data):
        vcol.color = colors[i]

    bpy.ops.object.mode_set(mode=mode)

if __name__ == "__main__":
    threedify("arrow.png", mask_color=(1, 0, 1, 1))

In a nutshell, this scripts iterates over all the pixels of an image which needs to be loaded in memory (case-sensitive), then recreates a mesh with 1 square face = 1 pixel from this image. The important thing is that the base image must have an exact mask color. I used this one made rapidly in paint with a mask key of (Red = 1, Green = 0, Blue = 1, Alpha = 1):

enter image description here

Result (Make sure to enable Vertex color mode) :

enter image description here

enter image description here

With your example - Make sure to replace the last line with

threedify("cbNd2.png", mask_color=(0, 0, 0, 0))

The result is not the same as in your question because your image is actually 72*96 pixels.

enter image description here

Warning : This will likely crash blender or hang your PC for non-pixel-art images.

You can then tweak the modifiers to fine tune the effect.

enter image description here

If you want to actually see it in material preview or rendered view, here is a minimal working example of a shader tree :

enter image description here

Add some lighting, and voilà :

enter image description here

$\endgroup$
8
  • $\begingroup$ I can't get this to work. I never used scripts before, so I'm a little confused. I created a .py file with your code, and in the same folder, I have the image saved as "pixel.png", which I assume means I don't have to change your code, if the script starts from its own folder location to look for the file. I don't get any errors when I execute script, but I don't see the result, even when I switch to the vertex mode, and I don't see a new object in the scene. $\endgroup$ – Synn Ko Dec 18 '20 at 20:23
  • $\begingroup$ @SynnKo There is a link in the second line of the answer explaining how to run the script, it points to stackoverflow.com/questions/11604548/… $\endgroup$ – brockmann Dec 18 '20 at 20:51
  • $\begingroup$ @SynnKo you don't have to do anything outside of Blender, just open a text editor and paste the code into it. Then open an image editor or a UV texture editor, click on open in the top-middle of the editor and select your picture. Its name doesn't matter so long as you type the exact same name in the last line of the script instead of "arrow.png". Make sure to keep the " before and after the file name $\endgroup$ – Gorgious Dec 18 '20 at 21:22
  • $\begingroup$ @Gorgious Well... it sort of worked, but I don't really understand how it all works yet. Only thing that's missing is the colors, and I'm sure I switched the color mode to vertex. Is there another setting I need to change to make the texture show up? $\endgroup$ – Synn Ko Dec 19 '20 at 0:07
  • 1
    $\begingroup$ @Gorgious Here's the link to the GitHub repo I created based on your answer, just so there is a two-way connection between this Stackoverflow answer and the repository: github.com/SynnKo/blender-threedify $\endgroup$ – Synn Ko Feb 27 at 15:49
9
$\begingroup$

You can use Hair particle system with Cube as Render > Object.
Cubes are emitted based on texture Density.


Sprite to Cubes

Like here your image 16 x 16 px ...

  • add Plane, Subdivide by 4 to get 16x16 grid, delete one row to get 15x15 faces, because Cubes will be emitted from vertices
  • add Particle System > Hair, Source > Vertices with disabled Random, Render > Object (assign Cube), Texture > add New and under Texture Properties assign your image, enable ColorRamp (that switch image to Black&White), under Influence enable Density.

enter image description here

enter image description here

Colors per cube should be easily taken from texture.
Like here, I just cant figure out what is wrong with setup in my file.

Edit: If this is still valid https://developer.blender.org/T42706 - emit from vertex is the issue.


Color per Cube

So you would have to go to Modifier Properties > Convert. Join Ctrl+J Cubes (if you want cubes as one object). In Edit mode with all selected Unwrap > From View. Switch to UV layout and in UV Editor switch to Individual Origins and Scale S the UV a bit.

enter image description here

enter image description here

Shader is smart enough ... UVMap is used automatically.

Notes:

  • You can use plane and use Subdivision Surface Modifier and Particle System after, just se Use modifier stack if enabled under Particle System
  • You can Separate P by Loose parts back into individual cubes if you need it.
  • You can use just a Plane (instead of Cube) for whole proces to make it lighter and use Solidify modifier as final touch :)
  • By default are images interpolated, to keep lowers images pixelated choose in Shader editor in Texture node > Closest. For Particle System Texture go to Properties Editor > Texture > Sampling > Interpolation disabled
  • in some of my screens is missing one Cube at a hat because I used Mapping > Generated instead UV
$\endgroup$
5
  • $\begingroup$ I'm having issues with this approach. My plane stays flat and I'm not sure what I'm doing wrong. Do I need to assign the texture to the object? $\endgroup$ – Synn Ko Dec 18 '20 at 20:48
  • $\begingroup$ Would be easier for me if you can share your file via blend-exchange.giantcowfilms.com here. Thank you $\endgroup$ – vklidu Dec 18 '20 at 20:53
  • $\begingroup$ Sorry for the late response, I wanted to tinker with this method a bit more before I ask another stupid question, and the last few days left me with very little time. I now understand what I did wrong thanks to your Blender file. Technically you answered my question, although I don't know if it would be a good fit for me personally as I plan on working with many sprites and the method you suggested for coloring cubes seems like a huge time sink. $\endgroup$ – Synn Ko Dec 26 '20 at 15:00
  • $\begingroup$ Sure :) If you have a lot of these it is faster to use Gorgious script, that is great timesaver. I just wanted to mention other methods, that are more versatile and dont lay on a specific script, that has to be maintained in long term. Ad Coloring - almost three clicks Convert-Unwrap-Scale doesnt seem to be a big issue, but if I skipped something to make it clear let me know :) $\endgroup$ – vklidu Dec 26 '20 at 18:36
  • $\begingroup$ Three clicks doesn't seem like an issue, until you need to do it a few hundred times... thank you for your help anyway, I could probably still make use for it, but I don't really know to what extent yet. $\endgroup$ – Synn Ko Dec 27 '20 at 18:44
5
$\begingroup$

Image to Volume

In some cases can be enough just volumetric shader.
This example rendered with 10 samples, Denoiser enabled.
It is just a few second to render.

enter image description here

Render Properties > Volume > Step Rate 0.01

enter image description here

Viewport screenshot is only Volume material - means it can have only a diffuse look. You can get glossiness for front / back by connecting Principled BSDF node to Surface socket, but sides (thickness) stay volumetric.

By adding PrincipledBSDF to surface socket you can benefit also in other areas

  • sharp texture
  • shorter render-time (switch from NLM denoiser to OpenImageDenoise cut render-time to half)

enter image description here


Originaly I wanted after this step use another object with Volume to Mesh modifier plus Remesh modifier, but it seems Volume to Mesh modifier supports only imported VDB files at the moment.

$\endgroup$
4
  • $\begingroup$ Maybe this is just my beginner mind talking, but this seems like the least time consuming method out of all the answers. Can you provide a blender file like in your other answer? I'm not knowledgeable enough to recreate all steps (like I said, beginner) and I learn better when I have an example to work with. $\endgroup$ – Synn Ko Dec 26 '20 at 15:24
  • $\begingroup$ @SynnKo I can provide the file, but there is nothing more than you see here. This method I added just for couriousity. It is very limited, it isnt real geometry, so no bevel, no gaps controll between cubes ... so as I mentioned already Gorgious script is the quicker way to get result. $\endgroup$ – vklidu Dec 26 '20 at 19:42
  • $\begingroup$ That's fine, this method already achieves the look I want (the rest was optional anyway) and it might be easier to work with if I ever wanted to animate my 3D sprites, which would require switching around the texture on different frames instead of switching around different models. I'll still consider Gorgious' script, though. Btw. you forgot to add a camera to the scene. Also, how long did it take to render for you? For me, it took roughly a minute, which is not ideal for something this simple... $\endgroup$ – Synn Ko Dec 26 '20 at 20:44
  • $\begingroup$ Rendering FullHD format takes 22 sec on my Mac (CPU only, since GPU is not supported). When PrincipledBSDF node is used for surface socket of Output node and OpenImageDenoise type is used - time was 11 sec. Answer updated. $\endgroup$ – vklidu Dec 27 '20 at 19:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.