At the beginning you could duplicate your cylinder and use the copy as the Target of a Shrinkwrap modifier so that you're sure that the circular hole you'll dig will stick to the cylinder profile. Then apply the Shrinkwrap and extrude to make the handle.
Also, you could use much less polygons:
- Create a hole in the cylinder.
- Extrude and make a circle with LoopTools > Circle.
- In side view check that the circle's vertices are aligned with the cylinder profile (or use the shrinkwrap method if you think it's easier).
- Extrude again to make the handle.