14
$\begingroup$

I am trying to generate a smooth curvature map from a normal map. I already know how to generate a sharp curvature (from this post: How to convert a normal map into a curvature map )

Here is the normal map source that i got from the free pack "Rock and boulders 2" of Unity:

High quality normal map for your test: https://drive.google.com/file/d/0B1wP1Y8dmh8aVmN6VUpYMU02Q28/view?usp=sharing

enter image description here

The result i am trying to get in the compositor:

enter image description here

(and the sharp curvature that i can already make in Blender, it uses 1 pixel width)

enter image description here

My goal is to import the curvature smooth and sharp in Linear mode and overlay them together to generate a greyscale, then a color ramp will be used to generate the albedo. I am trying to reproduce this hand painted effect texture: https://youtu.be/dE0CM1C_soc

$\endgroup$
  • 2
    $\begingroup$ Suggest adding images to question instead of the google drive links. $\endgroup$ – batFINGER Sep 13 '17 at 15:48
  • $\begingroup$ I don't think it would be easy to do that in compositor... maybe the best would be to use python and numpy for calculating all the necessary derivatives of the normal map. $\endgroup$ – Secrop Sep 14 '17 at 11:37
  • $\begingroup$ developer.download.nvidia.com/assets/gamedev/docs/… Is this you problem? Then you probably won't find any good solutions and certainly any solutions that work in the compositor alone. $\endgroup$ – piegames Sep 14 '17 at 18:57
  • 2
    $\begingroup$ @DanylBekhoucha - Are you looking for another answer other than the ones already given? If so please specify what those answers are lacking with regard to your expectations. $\endgroup$ – bertmoog Oct 14 '17 at 21:58
  • 1
    $\begingroup$ @DanylBekhoucha I’m happy to revisit my script if you let me know what the problem is. $\endgroup$ – Rich Sedman Oct 15 '17 at 21:56
11
+50
$\begingroup$

Python can be used to analyse your normal map and reconstruct the bump map and this can then be used to generate a curvature map.

Here's the code to run - simply paste this into a Text Editor window, load your image into the Image Editor, ensure it has '_nmp' or '.nmp' somewhere in the filename and click 'Run Script'.

import bpy
import os
import math

pixels = None
width = None
height = None

def get_pixel(x, y):

    global pixels, width, height

    base = (y*width+x)*4
    return (pixels[base], pixels[base+1], pixels[base+2], pixels[base+3])

def set_pixel(x, y, rgba):

    global pixels, width, height

    base = (y*width+x)*4
    pixels[base] = rgba[0]
    pixels[base+1] = rgba[1]
    pixels[base+2] = rgba[2]
    pixels[base+3] = rgba[3]


#Adapted from http://blender.stackexchange.com/a/80047/29586
def srgb_to_linear(c):
    a = 0.055
    if c <= 0.04045:
        return c / 12.92
    else:
        return ((c+a) / (1+a) ** 2.4)


# RGBA was previously held as 8-bit. Re-normalize to correct for 8-bit innacurracies
def normalize_normal(rgba):
    dist = math.sqrt(math.pow(rgba[0],2) + math.pow(rgba[1],2) + math.pow(rgba[2],2))
    if (dist > 0):
        return (rgba[0]/dist, rgba[1]/dist, rgba[2]/dist, rgba[3])
    else:
        return (0,0,0,0)


def calc_gradients(normal_x,normal_y):
    adj_x = (normal_x - 0.5)*2
    adj_y = (normal_y - 0.5)*2

    # Filter out the extremes to avoid infinities
    if adj_x < -0.99:
        adj_x = -0.99

    if adj_x > 0.99:
        adj_x = 0.99

    if adj_y < -0.99:
        adj_y = -0.99

    if adj_y > 0.99:
        adj_y = 0.99

    return (math.tan(math.asin(adj_x))/100, math.tan(math.asin(adj_y))/100)


def normalize(arr, width, height):        
    #Get range of depth
    max_depth = -99999999999999999999
    min_depth = 9999999999999999999
    for x in range(0,width):
        #print("Normalizing(1)... %i" % x)
        for y in range(0,height):
            depth = arr[y*width+x]
            if depth < min_depth:
                min_depth = depth

            if depth > max_depth:
                max_depth = depth

    print("max = %f, min = %f" % (max_depth, min_depth))
    if max_depth > 1000:
        print("Limited max to 1000")
        max_depth = 1000

    if min_depth < -1000:
        print("Limited min to -1000")
        min_depth = -1000

    if max_depth == min_depth:
        max_depth = min_depth + 1

    #Normalize depth to 0.0 to 1.0 range
    for x in range(0,width):
        #print("Normalizing(2)... %i" % x)
        for y in range(0,height):
            depth = (arr[y*width+x] - min_depth) / (max_depth - min_depth)
            #print(depth)
            arr[y*width+x] = depth

def generate_curviness(heights, width, height, radius):
    #determine curviness at each point and store in RED channel
    curviness = [0.0] * width * height
    print("Calculating Curviness (%i)..." % radius)
    for x in range(1,width-1):
        for y in range(1,height-1):
            for d in range(1,radius+1):

                if (x-d)<0 or (x+d)>=width or (y-d)<0 or (y+d)>=height:
                    pass
                else:

                    h = heights[y*width+x]
                    hv1 = heights[(y-d)*width+x]
                    hv2 = heights[(y+d)*width+x]
                    hh1 = heights[y*width+x-d]
                    hh2 = heights[y*width+x+d]

                    if hv1 < hv2:
                        vcurviness = (h - hv2) - (hv1 - h)
                    else:
                        vcurviness = (h - hv1) - (hv2 - h)

                    if hh1 < hh2:
                        hcurviness = (h - hh2) - (hh1 - h)
                    else:
                        hcurviness = (h - hh1) - (hh2 - h)

                    curviness[y*width+x] += (vcurviness + hcurviness)/(d*d)

    return curviness




def process_image(image_name, new_image_name):

    global pixels, width, height

    image = bpy.data.images[image_name]

    width = image.size[0]
    height = image.size[1]

    # The source image pixels - stream of R,G,B,A for each pixel, width x height
    pixels = list(image.pixels)

    # Gradient from each pixel in each direction - up, left, right, down
    gradient_u = [float] * width * height
    gradient_l = [float] * width * height
    gradient_r = [float] * width * height
    gradient_d = [float] * width * height

    # Determine gradient in each direction for each pixel
    for x in range(0,width):
        print("Capture gradients - Column %i of %i" % (x,width))
        for y in range(0,height):

            this_rgba = normalize_normal(get_pixel(x, y))
            this_rgba_gradient = calc_gradients(this_rgba[0], this_rgba[1])

            if x == 0:
                left_rgba = None
                left_rgba_gradient = (0,0)
            else:
                left_rgba = normalize_normal(get_pixel(x-1, y))
                left_rgba_gradient = calc_gradients(left_rgba[0], left_rgba[1])

            if y == 0:
                bottom_rgba = None
                bottom_rgba_gradient = (0,0)
            else:
                bottom_rgba = normalize_normal(get_pixel(x, y-1))
                bottom_rgba_gradient = calc_gradients(bottom_rgba[0], bottom_rgba[1])

            if x == (width-1):
                right_rgba = None
                right_rgba_gradient = (0,0)
            else:
                right_rgba = normalize_normal(get_pixel(x+1, y))
                right_rgba_gradient = calc_gradients(right_rgba[0], right_rgba[1])

            if y == (height-1):
                top_rgba = None
                top_rgba_gradient = (0,0)
            else:
                top_rgba = normalize_normal(get_pixel(x, y+1))
                top_rgba_gradient = calc_gradients(top_rgba[0], top_rgba[1])

            # Average each directional gradient with this one to give the gradient between the pixels in each direction
            gradient_u[y*width+x] = (this_rgba_gradient[1] + top_rgba_gradient[1])/2
            gradient_d[y*width+x] = -(this_rgba_gradient[1] + bottom_rgba_gradient[1])/2
            gradient_l[y*width+x] = -(this_rgba_gradient[0] + left_rgba_gradient[0])/2
            gradient_r[y*width+x] = (this_rgba_gradient[0] + right_rgba_gradient[0])/2


            # Store it         
            ent_u[y*width+x]))
            set_pixel(x, y, (this_rgba[0], this_rgba[1], this_rgba[2], gradient_u[y*width+x]))


    # Now start with 0's and process the whole image pixel by pixel averaging the resulting heights generated by the surrounding pixels
    heights = [0.0] * width * height
    numpasses = int(max(width, height)/2)
    for p in range(0,numpasses):
        print("Pass %i of %i" % (p, numpasses))
        for x in range(0,width):
            for y in range(0,height):
                if y < (height-1):
                    height_top = heights[(y+1)*width+x]
                else:
                    height_top = 0.0

                if y > 0:
                    height_bottom = heights[(y-1)*width+x]
                else:
                    height_bottom = 0.0

                if x < (width-1):
                    height_right = heights[y*width+x+1]
                else:
                    height_right = 0.0

                if x > 0:
                    height_left = heights[y*width+x-1]
                else:
                    height_left = 0.0

                baseindex = y*width+x
                height_this = heights[baseindex] 

                #Calculate and store the new height
                heights[y*width+x] = (height_this + (height_top + gradient_u[baseindex]) + (height_bottom + gradient_d[baseindex]) + (height_left + gradient_l[baseindex]) + (height_right + gradient_r[baseindex])) / 5

    normalize(heights, width, height)

    print("Storing Bump...")
    for x in range(0,width):
        for y in range(0,height):
            pixels[(y*width+x)*4+3] = heights[y*width+x]

    # Generate Red with radius 1
    curviness = generate_curviness(heights, width, height, 1)
    normalize(curviness, width, height)
    print("Storing Curviness...")
    for x in range(0,width):
        for y in range(0,height):
            pixels[(y*width+x)*4+0] = curviness[y*width+x]

    # Generate Green with radius 5
    curviness = generate_curviness(heights, width, height, 5)
    normalize(curviness, width, height)
    print("Storing Curviness...")
    for x in range(0,width):
        for y in range(0,height):
            pixels[(y*width+x)*4+1] = curviness[y*width+x]

    # Generate Blue with radius 25
    curviness = generate_curviness(heights, width, height, 25)
    normalize(curviness, width, height)
    print("Storing Curviness...")
    for x in range(0,width):
        for y in range(0,height):
            pixels[(y*width+x)*4+2] = curviness[y*width+x]




    print("Storing image...")
    new_image = bpy.data.images.new(new_image_name, width=width, height=height)
    new_image.pixels = pixels[:]
    new_image.update()
    new_image.pack(as_png=True)
    new_image.use_fake_user = True

    print("Done")



for img in bpy.data.images:
    if "_nmp" in img.name or ".nmp" in img.name:
        new_img_name = img.name
        new_img_name = new_img_name.replace('nmp', 'bump')
        print("%s -> %s" % (img.name, new_img_name))
        process_image(img.name, new_img_name)

The script will look for all images with '_nmp' or '.nmp' in the name and will generate an image of the same name, but with 'nmp' replaced with 'bump'. Each generated image will consist of 4 channels (RGBA). The Alpha channel will be a bump map generated from the normals, while the Red, Green and Blue channels are each a Curvature map - one with a radius of 1 pixels (Red channel), one with a radius of 5 pixels (Green channel) and one with a radius of 25 pixels. The curvature is measured only in the Horizontal and Vertical directions but the result seem reasonable to my eye (I've not got much experience of curvature maps - if there's a problem with these then let me know and I can update the calculations).

Here's the normal map (a section of the map from your question) :

normal map

Here's the result :

result

Bottom-right is the generated bump map, Bottom-left is the red channel (1 pixel radius curvature), top-right is the green channel (5 pixel radius curvature), top-left if the blue channel (25 pixel radius curvature).

The way this works is to first process the normals from the normal map to determine the horizontal and vertical gradients at each point. These are re-normalized to correct for any inaccuracies (such as 8-bit encoding) and extreme values are clipped to avoid problems before being averaged with its neighbours to get a gradient between those pixels. These gradients are then used to determine the difference in height between neighbouring pixels and those height differences built up into the bump map (this is the part that takes the time).

The resulting bump map is stored in the alpha channel and is then sampled to determine the curvature at each point, storing each in the R,G,B channels.

The script produces output of progress to the console - so run Blender from the command line to view the output. Beware large normal maps taking a considerable amount of time to process - on my system the full 2048x2048 map was estimates to take around 10 hours to complete so I used a cut down section for my testing (604x408 pixels) - which only took a few minutes to complete.

EDIT :

Just to clarify, the script takes the original Normal Map...

'rock5_nmp_clipped' normal map

... and extracts the height information into a bump map with the same name but with 'nmp' replaced with 'bump'...

'rock5_bump_clipped' bump map

...along with 3 separate channels, each with the curviness map calculated at different radii :

animated red, green, blue

To analyse curvature using a different radius simply amend the calls to the 'generate_curviness(...)' function and re-run the script to generate a new image :

#Use radius of 25
curviness = generate_curviness(heights, width, height, 25)

e.g., change to :

#Use radius of 40
curviness = generate_curviness(heights, width, height, 40)

Using the Normal Map using original normal map

The curviness stored in the R,G,B channels of the generated bump map can be used to add wear or additional effects :

normal with wear

Since we now have the actual underlying 'bump' information we can use this to drive other effects - such as the True Displacement - generating a much more realistic surface :

true displacement

(Above example rendered using a single plane, subdivided to 256x256 faces and displaced with True Displacement (see displacement properties panel at bottom-right of image). All depth, displacement, and wear derived from original normal map, extracted by script.)

$\endgroup$
  • 1
    $\begingroup$ Updated answer with enhanced script to process a normal map and generate the bump map along with curvature maps at different radii. Hope this is of some use. $\endgroup$ – Rich Sedman Sep 22 '17 at 21:03
  • $\begingroup$ Hi, could you share the node setup too so we can understand what the script does visually and see every steps? $\endgroup$ – Danyl Bekhoucha Oct 8 '17 at 11:35
  • $\begingroup$ Updated answer - hopefully this makes sense. Important to use Non-Color Data in the Image nodes, then just split into channels using Separate. Let me know if still not clear. $\endgroup$ – Rich Sedman Oct 10 '17 at 6:29
  • $\begingroup$ Hi. thank you for your explanation. How do i extract the height information? Can i do that using a node? $\endgroup$ – Danyl Bekhoucha Oct 11 '17 at 12:39
  • 3
    $\begingroup$ @Rich Sedman, the results from your script are quite good.. perhaps would be better to use numpy in it as it makes it a lot faster to calculate. $\endgroup$ – Secrop Oct 15 '17 at 12:21
6
$\begingroup$

I did a node setup that produces a nice result, but I don't consider it a perfect solution. It won't give the same results as other softwares (thought other softs also produce different results). The kernel function I used is very simple and basic, and while the best approach would be sampling a wide area, this kernel only samples two directions, and uses gaussian blurs to approximate the rest of the kernel range. enter image description here

$\endgroup$
  • $\begingroup$ Nice result! Also you should convert the image in Linear mode in the "n" menu and use Mix Overlay nodes instead of math multiply to keep the same grey level. Then at the end add a gamma 2.2 node. And you should use relative blur to make it work with every resolution. You can take the high quality 2k here: drive.google.com/file/d/0B1wP1Y8dmh8aVmN6VUpYMU02Q28/… And even your sharp effect is really good too, we can clearly see the reliefs. Thank you a lot for trying to help making this effect! $\endgroup$ – Danyl Bekhoucha Sep 15 '17 at 13:06
  • $\begingroup$ @DanylBekhoucha, as I said, the kernel is basic. I would like to do a good kernel for this but I'm afraid that will take much more than 20min in the composer. Perhaps in another time I will write an operator just for this. $\endgroup$ – Secrop Sep 15 '17 at 13:10
2
$\begingroup$

Generate a smooth curvature map from a normal map

To generate the curvature smooth you need first to create a node group of the curvature map, you can find the setup here: https://blender.stackexchange.com/a/72602/23134

Use the first node setup, you will need to replace the three Linear Light nodes to Mix and keep the same values at 0.5.

You can download all the node setups here: https://drive.google.com/file/d/1bixkxs6cSes-J9GVwDeIP7pW2OnGNhYd/view?usp=sharing

But you must not:

  • overlay the curvature on itself, the curvature must not be contrasted
  • multiply the blue channel
  • use the gamma node, every curvature maps must be bright (in sRGB) to overlay them

Note: I recommend to use a normal map in 32 bit Float to get smoother color variation.

To use it add several node group and increment the pixel width by a factor of 2 starting at 1 to 64: 1, 2, 4, 8, 16, 32, 64. For each of them add a blur node with a value that is double the pixel width: 2, 4, 8, 16, 32, 64, 128

Then overlay every nodes between them in the order you want, I prefer to start from the bottom so the last overlay is at the top like the image input:

enter image description here

How to avoid revealing the seams?

The curvature smooth compared to the curvature map can reveal your seams, it is more noticeable if you are using a Smart UVs projection.

To reduce the visibility of the seams you can:

  • add a noise bump map on your model that is baked on the normal map, the tiny details will make the seams less noticeable.
  • make a procedural tillable texture or procedural sculpt on a plane, your texture will be projected in Box mode (Box Mapping in the Unwrap menu or Input > Texture Coordinate, use Object and set the texture to Box instead of Flat and tweak the Blend value to remove the seams like 0.1).

What you can do with the curvature and curvature smooth map:

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.