This is not because of "non planar faces", they are planar, this is because of smoothed normals, which are interpolating with the holes, this can be fixed with a small inset, or an edge split modifier, or reducing the auto smooth angle or marking the hole edges sharp, you can check this is the case if you set the models shading to flat.
I would recommend the marking the edges sharp method.
I don't want to meddle too much but if you were to model for a subdivisions approach this would be a lot less messy, and probably easier.
A huge amount of N-gons will limit you, if you do know very well what you need go ahead; N-gons are a tool, very much like animation interpolation that should be left only to perform menial tasks.