I've made the two open vertical lines using the knife tool, but then when I try joining them with the J hotkey, I can't. Does anyone know why this happens? Thank you in advance.
In edit mode J will make an edge between two vertices, and also cut the underlying faces and edges crossed over by the new connection. This does not work if there is no face associated with a selected vertex.
In your screenshot, the lower vertex looks like it is on the horizontal edge but the lack of selection highlighting shows that it is a floating vertex with only one edge attached to it.
Also looking at the upper selected vertex, you will see that there are two horizontal lines, one orange and one black, this would indicate that you have two horizontal edges, one of which has the selected vertex, there wouldn't appear to be a face attached to these edges.