I'll try to answer simply but even for me it's not. But what you try to do isn't possible. (see the edit 1 for the answer of your problem)
In our world light is linear, but we didn't see it linear, the eyes make it looks with more contrast (what we can call a transfer curve).
Your display (let's say a computer screen) have it's own transfer curve (for technical historical reason), so to see a picture close to what we see in real life (a linear light interpret non linear by the eyes) we need to bake a transfer curve "inverse" of the display one to bring the image a linear signal to the eyes.
At this stage it's a bit confusing, let's make it simple:
Real life: Linear light --> non linear interpretation by the eyes
sRGB image: non linear (transfer curve of sRGB) --> linear ("inverse" transfer curve by the screen) --> non linear interpretation by the eyes.
CG: Scene referred linear image (what Cycles calculate internally)--> non linear (ex: filmic+high contrast transfer curve) --> linear ("inverse" transfer curve by the screen) --> non linear interpretation by the eyes.
It's a bit simplified but what's important to understand here is that the eyes need to recieve a linear image, like in real life eyes recieve linear light but we don't see it linear.
One more important things is that each of the above have their own transfer curves (eyes, screen, filmic, sRGB, APN, smartphone... they are all different).
If you set up a color space to filmic for an image in 8bits sRGB .jpg that you don't know how it was manipulated, you try to see informations that were lost in the compression + you're mixing transfer curves.
You must look filmic in two parts.
1st, the filmic itself, that is a LUT which compress values to be in 0/1 range in a logarithmic way (transfer curve). So, the end result is a nonlinear image.
2nd, the contrast looks which modify the transfer curve shape.
There is one more caracteristic with filmic, the desaturation of the high value colors.
I'll see if I've made mistakes or forgot something in what I've said, and will correct them if it's the case.
Don't hesitate to tell me if I'm wrong or need more informations.
Edit 1: Sorry I've understand a bit more your problem and have more infos.
When you try to compose your image with your final render you need to remember that filmic in the color management is aim for linear scene referred images and is applied at the very end of the chain, so your image that is composed with the linear render will have filmic conversion too. But your image is an 8bits sRGB .jpg and must be threat as a sRGB EOTF.
What you need to do is save your render with filmic and all compositing but without the background, and after that compose your render with the background image both in sRGB for image colorspace and color management view in sRGB, no more color transformation at this stage since you are in 8bits.
This is the problem when you work with both linear scene referred data and display non linear data.
Hope I've all understood this time.