The vertices does not join to the edge so a gap is form. The thing is I do not want to create a new face to fill the hole because I want my mesh to be quad. How do I fill the hole while keeping things the same?
If you want only minimal changes, you can just snap the vertex to the edge. It is "messy" from a topology point of view but you will basically get what you want. (As long as you don't use any modifiers)
The quality way would be to add an edgeloop to the quads shown on the right (split each quad into two) and connect the vertex to the end of the edgeloop: