The problem you are experiencing consists of two parts: first of all, as you already realized and tried to compensate with the Transparency BSDF, your leaves texture is not transparent between the leaves. If the original texture (which I cannot see in your question) has transparency, then you should plug the Alpha output of the leaves' Image Texture node into the Alpha input of their Principled BSDF.
Now you have transparency between your leaves. This reveals the second problem as you also have realized with the transparency you mixed in: The Mix Shader shows either only the roof texture where the mix factor is 0 or black, or it shows the leaves completely with transparency where you expect to see the underlying roof texture.
The Mix Shader unfortunately is no Alpha Over node like you have in the Compositor. It's not overlaying one shader above the other - it is either showing the first (0) or the second (1) with the grey values gradually changing between those two shaders.
So you have to integrate the Alpha of the leaves into the mix factor by combining it with the opacity map through a Math node:
- Where the opacity map is black = 0, you want to have roof texture.
- Where the opacity map is white = 1, you want the leaves to show.
- The leaves themselves have an alpha channel which is white = 1 where you want to see the leaves, and black = 0 where you want the roof texture to be seen.
If you now plug the opacity map in a Math node set to Multiply, you can use this output as a mix factor for the Mix Shader node:
- opacity 0 × alpha 0 or 1 results in 0, the roof texture is shown as expected.
- opacity 1 × alpha 1 gives 1, so in an area where the leaves are supposed to be and their texture is not transparent, leaves are showing.
- opacity 1 × alpha 0 results in 0, this is an area where leaves are, but their texture is transparent there and since the result is 0, the roof texture will show.
Example in images: let's say that purple color on the left is the roof, the image with transparency in the middle is the leaves texure and on the right is the opacity map, black where the roof should be and white for the leaves.
If you now plug the opacity map in the Fac input of the Mix Shader, with the roof in the first shader socket and the leaves in the second, you'll get this: The plane is either showing the roof where it's black or the leaves with a transparent background where it's white.
But what you actually want is to see the roof where the leaves are transparent like this:
Now if you think about the mixing, what does this mean? In the opacity map you need black where it is black and where the leaves are transparent, i.e. where their alpha map is black. And you can achieve this by simple math logic. 0 is false, 1 is true, logical AND is multiplying.
And this is the node setup for mixing it:
By the way, of course there are also possibilites to combine textures etc. before you plug them into a Principled BSDF so that you wouldn't even need to mix two of them together. But especially when you want more complex setups and have one material per shader output, the method I showed above is possibly better instead of thinking how you have to mix textures, bump maps, roughness etc. before the Principled BSDF.