How to set-up the perspective view based on a reference image, before starting to model what's on the image?
As an alternative to background images (which can't be displayed in perspective), you could use an Image empty (ShiftA> Empty > Image).
Image empties can be rotated, scaled, and translated just like any other object.
You can make empties transparent by adjusting the Transparency slider in Properties > Object data > Empty:
Note that when using transparent image empties, they will not be correctly drawn when they are not selected if Transparency is unchecked in Properties > Object > Display:
The documentation clearly states that this is currently not possible to setup a background image in perspective view:
They are only available for Top, Side and Front (and their complementary versions) orthographic views! The picture remains the same when you switch between these six views.
But there is one exception, when you switch camera mode NUMPAD-0 the background images will still be visible: