2
$\begingroup$

LOOK FOR BOX TYPE IMAGES

I need to annotate each SINGLE plant separately from others, for accurate annotation, right now I've it scripted for ID MASK. It gives each object from a specifically named collection iterated ID pass, renders original scene in .png, and then renders and converts pixels (every pixel, which is not black) from every ID MASK image to .json. But as you can see it doesn't works as expected. Is there something similar I could do with cryptomatte? Again, For annotation I need to know in advance which objects which color will use or something like that to know exactly or define an object. maybe I can adjust emission shader or make my own so it can handle transparent textures correctly as well? or something along those lines enter image description here UPD cryptomatte works only when I render plants separately from scene Look here cryptomatte whole scene vs cryptomatte plants only enter image description here enter image description here

Blender doesn't render all pixels on distant objects, it gets better when I make resolution higher, like 4k, but it doesnt fixes my problem, but makes image pixels better. Also raising sample doesnt help. I tried to render with denoising and AA in compositor, that didnt help. How can I fix this, so it will render all object pixels on ID MASK using box pixel type? I need an alpha of an object as an ID MASK, for accurate object annotation.

Render Settings

Engine: Cycles

Render Resolution: 1280x720

Samples: 128

Pixel type: Blackman-Harris

Pixel width: 0.01

Full Image: enter image description here Problem:enter image description here enter image description here

Additional info: ID_MASK of the object in compositor enter image description here

Box type pixels looks so much better on the Image, but ID Mask dont recognize them, now how to fix this one?

Box type render: enter image description here enter image description here Box Type ID Mask: enter image description here Box type Render 4k: enter image description here enter image description here Box type 4k ID MASK: enter image description here

Annotations

Here's an addon, which do almost what I needed, but it uses emission shader, which doesnt work with custom textures.

By annotation I mean assigning each pixel of the original (rendered) image a corresponding value of instanceId

Where instanceId maps a specific blender object.

enter image description here

Here's more examples: Original Image without annotations: enter image description here Annotated Image: enter image description here enter image description here

For now I just need objectId and categories.

Right now I'm using ID masks which is the only reliable solution I've come up with.

As you can see I've tried a lot of other options such as: Cryptomatte Emission shader Denoising Playing with AlphaTreshold

Since ID masks don't work as reliable as I thought they would. I was thinking about writing my own shader that will use emission (to get the perfect color) and some magic to fix the transparent textures.

Here's why emission shader won't work in any case: enter image description here enter image description here enter image description here

I know I should start a new question, but... how to achive a solid color with nodes? If there's a way to get a fully opaque color w/o aa or any kind of transparency? That would solve my problem perfectly. Here's my node attempt: enter image description here enter image description here

Cryptomatte did the job, but it needs a manual nodes pre-setup enter image description here enter image description here

Improve this question
$\endgroup$
6
  • $\begingroup$ Increase the output resolution and you are done $\endgroup$
    – lost protonbender
    Mar 25, 2020 at 14:50
  • $\begingroup$ Is this the only solution? $\endgroup$
    – cxnt
    Mar 25, 2020 at 22:10
  • $\begingroup$ You might need to provide your scene setting about those plants. $\endgroup$
    – HikariTW
    Mar 26, 2020 at 6:29
  • $\begingroup$ I don't think there is any way to make ID mask to identify a pixel which is not the main sampled pass. Id mask only provide yes, no, transparent information, what you want is the alpha value of the solid color from that plant right? Did you try Cryptomatte? $\endgroup$
    – HikariTW
    Mar 26, 2020 at 7:29
  • $\begingroup$ I need to accurately annotate each plant on the scene. Yeh I tried cypromatte, but not with Box type, if that matters?@HikariTW updated answer with 4k box type $\endgroup$
    – cxnt
    Mar 26, 2020 at 7:32

2 Answers 2

Reset to default
5
+100
$\begingroup$

Don't use ID Masks

Just don't. They have little use, this is a really poorly designed feature. You simply cannot make a good mask with a 2bit color depth image.

Cryptomatte is the solution

Look into using Cryptomatte. You need to enable it's layers in the View Layer properties tab:

enter image description here

You can then get any masks easily using Cryptomatte node in the compositor and they will be a lot higher quality:

enter image description here

Improve this answer
$\endgroup$
19
  • $\begingroup$ I have updated my question, please take a look $\endgroup$
    – cxnt
    Apr 2, 2020 at 15:58
  • $\begingroup$ I am not sure I understand the issue. Cryptomatte should provide masks as good as they get as far as I know. You can output them to files with File Output node and run any scripts you need for further image processing outside Blender. They can be saved in Multilayer EXR format and you can then open them in other software like in Photoshop with this free plugin: exr-io.com. I have not tried it, but I think OpenEXR should also be quite a programming-friendly format if you need to process it programmatically. I might be mistaken, but it might be worth looking into it. $\endgroup$
    – Martynas Žiemys
    Apr 3, 2020 at 7:26
  • $\begingroup$ If you look in to the very first image of the question, you can see, that I'm already rendering masks in .exr. Please take a look at the Problem images, and at the Box type render. You can see that ID mask node doesnt cover those pixels rendered with box filter type, which I need to be also covered for accurate annotation $\endgroup$
    – cxnt
    Apr 3, 2020 at 10:51
  • 1
    $\begingroup$ Sure $\endgroup$
    – Martynas Žiemys
    Apr 3, 2020 at 13:56
  • 1
    $\begingroup$ Why didn't Cryptomatte work? I cannot imagine a situation where this complete solution for masking would not be better than ID masks in all ways possible. You are now asking for something that is impossible, because every pixel gets sampled many times and takes it's value from multiple locations, so an image cannot be masked with only black and white mask. Pixels around the edges of objects and materials just have values from a few different things and it is impossible to separate them once they are mixed, this is not the same as anti-aliasing in raster renders, you can't remove it. $\endgroup$
    – Martynas Žiemys
    Apr 4, 2020 at 13:41
2
$\begingroup$

Is there any reason that you need to use width : 0.01px Blackman-Harris Filter? You are not sampling all area from that pixel but rather choose a very tiny dot to represent that pixel. See this post for further description.

If you want all the pixel being equally sampled, use box or set the width to about 0.67 px width enter image description here

Or just render the plants itself and use a math node to retrieve all pixel that have been rendered with color: enter image description here

Improve this answer
$\endgroup$
8
  • $\begingroup$ So box should fix the problem? I've already tried box, that didnt make any difference tho. Should I try box + higher res? $\endgroup$
    – cxnt
    Mar 26, 2020 at 6:21
  • $\begingroup$ I'm not limited in using render settings, so any solution will be fine $\endgroup$
    – cxnt
    Mar 26, 2020 at 6:22
  • $\begingroup$ @cxnt1 Can you provide the box rendered image? ID-Mask do a threshold control which does not support transparent(make sense) $\endgroup$
    – HikariTW
    Mar 26, 2020 at 6:31
  • $\begingroup$ WHat is treshold control? do you mean alpha treshold? $\endgroup$
    – cxnt
    Mar 26, 2020 at 6:49
  • $\begingroup$ Question updated $\endgroup$
    – cxnt
    Mar 26, 2020 at 7:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .