So i made the mout on a plane and used a mix shader and transparent shader to make it transparent, i only want the external freestyle line, and want the mouth highlighted as well but it only out lines the plane
With Freestyle, no, this is not possible; Freestyle acts on the geometry, not the rendered alpha.
However, it is possible to mimic Freestyle in the compositor by using edge detection to spot the transition between regions - in this case from opaque (low alpha) to transparent (high alpha).
See https://blender.stackexchange.com/a/186173/29586 for an example using the Normal and the Depth to detect edges - in your case you could use the same technique on the Alpha as is used on Depth. This will detect the edges of transparency.