1
$\begingroup$

Is there a way in compositing (or directly in the blender VSE...) to create a persistent image effect like in

https://youtu.be/aLA7AlnDftE?t=9m33s

Basically I'm looking to blend a video together with an image based on the video's 'pixel displacement'. I imagine I could use the 'Displace' node but I'm not sure how to use the motion of the video as a vector...

Thank you!

$\endgroup$
0
$\begingroup$

That sample looks like a creative use of removing of an I frame in the stream.

Not sure how familiar you are with video encodes, but a stream is frequently comprised of I, P, and B frames. I frames are intra frames, or whole pictures, P frames are predictive, and B frames are bi-predictive.

Only I frames are whole images, with P and B frames being partial image chunks that move according to the pixel vectors in the encoded file. That is, the encoded file compression calculated what direction the pixels were travelling in, and uses those smaller chunks to save space by moving smaller portions of the image around instead of using a whole frame. Over time, smaller pieces might be updated because they have changed significantly from the original I frame.

When we strip out the I frames, we end up with the motion vectors telling the decoder to move the same area of the image. However, without the I frame, it is moving chunks of the previous shot, as with your example. Slowly, over time, some of those pixel vectors are updated with newer image chunks, hence why the dog gradually appears.

How to replicate this in Blender is no trivial task. If you were to track the entire scene, using many hundreds of points, you could possibly achieve a similar effect by gradually transitioning portions of your image from the first shot to the second. That is, assuming you have shot A and shot B, you could use a tracked version of shot B to gradually transition in portions of shot B over time. Coupled with shot A being a simple subdivided plane, you could possibly shape deform it based on shot B's tracked points. If the tracked points used smaller, geometric shapes, slightly blurred, you have the basis for a general transition by fading in portions over time, until the entire image is replaced.

Shooting from the hip, but that is a loose approximation as to what the datamoshing trick does in your sample.

$\endgroup$
  • $\begingroup$ Hey! Thank you for this answer! This got me on the right path and I accomplished this using avidemux and a few ffmpeg options. I haven't tried the scene tracking solution you posted since the avidemux approach was much quicker for the desired effect. I wish blender had the ability to take out I frames from video in the VSE... maybe one day! I'll have to post that on rightclickselect. $\endgroup$ – rot_god Jul 25 '16 at 13:46
  • $\begingroup$ it wouldbe great to have access to the motion estimation vectors for motion blur of small shutter angle media as well. Would be great to add blur back to chromakey shots that were captured with sharp edges. $\endgroup$ – 3pointedit Nov 15 '16 at 22:58
  • $\begingroup$ Also morphing with trackers is very challenging in Blender, not recommended as I've had little success. Also remapping UV distorted plane in compositing leaves you with much aliasing. $\endgroup$ – 3pointedit Nov 15 '16 at 23:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.