This is more 'cutting the knot' than directly answering.. it is possible to capture the tangents of curves on the points they are converted to, before instancing your flowers on them, but it's quite a tangle.
Is it easier, in this case, to instance your flowers on the sides of a mesh box?
If you needed to get rid of the 45° at the corners, need stems, or somesuch, that would be more tweaking, but this seems an easier and more readable starting-point:
Re: your comment.. challenge accepted.
These nodes added after 'Align to Normal' to give the flowers a slight random twist away from the perfect normal:
And this cluster added as a branch from the topless and bottomless box:
It splits the faces, and then deletes horizontal edges, resulting in vertical edge-segments, flower-to-flower. Those are converted to curves, which are trimmed at both ends to avoid the holes at the centers of the flowers. Then they are converted back to a mesh to give some thickness:
My apologies.. this is Blender 3.1 (stable, at time of writing), because of the Edge Vertices node. There will be a getaround, if you really need it, but I think 3.1 is pretty safe to use...