I stumbled upon the right answer and it doesn't really fit into my old one, so I think a separate try is more useful:
Create your two objects, rotate the cylinder 22.5 degrees if necessary, then switch on snapping, set it to Vertex and Active. Snapping to the active vertex is what makes it controllable.
Go into editmode of the cylinder. SHIFTRMB the upper left vertex to make it the active (white) one. That might be easier if you rotate the view so the verts don't overlap. Now move it to the cube until it snaps and confirm.
SHIFTS Cursor to Active. Set the pivot point to '3D Cursor'.
Keep all verts selected, but now make the bottom left the active one.
Scale around the cursor until it snaps (keep the mouse close to the target corner of the cube), then confirm.
And now they should be exactly the same size. Return to object mode, select the lesser of them, then shift select the one that's supposed to keep the object origin and join them with CtrlJ.
Remove doubles, then remove the trapped face and you have joind them. Maybe recalculate normals is needed, too.