I don't want to add it as a plane because then my scene lights affect the image and it's going to be hard to make it follow camera movements.
I saw on Youtube that if you use some nodes you can get the background image for camera to render but it didn't work, perhaps something has changed since then?