Here is how it looks like when it is on the ground.
Here is how it looks like when I move the bone up along the z axis. As you can notice the yellow bone moves to the right a little. How do I fix this? I want it to NOT move left nor right, but just simply up.
(Note that I have not done any weight painting yet so for now I am not trying to control the body with the bones)
This is how I have setted it up
Or is everything working the way I wanted to and there is no mistake?