You did not the check Rotation box in the particle settings, so all particles have the same orientation and rotation as their source: "Plane.004"
And you used the vertex group "Group" to emit particles form the entire lower body of the bird.
Together these two things cause a lot of the feather particles to stick through the lower body mesh.
You could make the vertex group smaller. For instance: limit the used vertex group to only the middle bottom edges of the body of the bird (highlighted and selected in the image below). Then there would only be feathers on the bottom of the body, and only a few would stick into the body a little:
Other things you can do:
(but might not fit your needs)
You have very little control over the location of the particles because your model is very low-poly (or low resolution in terms of vertices). If you apply your Subdivision modifier you can be more precise with the location of emission via vertex group.
You could check the Rotation box in the particle setting and for instance choose Normal for the Orientation axis.