Transparent object showing up in render layer mask?

Question

I have this transparent plane which is the texture face of my character. It's attached to my character which I render on Render Layer 2. On Render Layer 1 I render the background, and on Render Layer 3 I render the shadows (using the "Shadow Catcher"). I have Layer 2 set as a mask for Layer 3 so the shadows have cutouts in them.

However, when I render and composite these, the shadow has a weird cutout in it where the face plane is. The face plane is transparent, though.

How can I resolve this? This shouldn't be cutting out the shadow because it's invisible, yet it is. Is there a fix?

Thanks!