17
$\begingroup$

First let me apologize to ask this question, as this has been basically already asked, for instance here, here and here. However these answers don't seem to provide an understanding what's really going on, so I will pose my question anyway.

Every once in a while the video in a rendered video file will playback much faster than the audio, despite audio and video being perfectly in sync in the VSE. This problem seems always to pop up when then input source video is "mostly static", i.e. when the change from one frame to the next is miniscule (e.g. a screen recording). I should also add that I record the videos in question using a lossless codec (FRAPS or lagarith).

From the other questions linked above, I get the impression that this is due to Blender (or rather its version libavcodec) incorrectly decoding the compressed source video. It is also mentioned in one answer to the above questions that this is related to the use of b-frames in the encoding algorithm. However I had this problem also using the lagarith codec, which apparently uses only keyframes/i-frames.

Now in order to turn this cry for help into a question, let me ask: What the hell is going on here?

More precisely:

  1. Why can Blender (or its version of libavcodec) apparently correctly decode and show the video in the VSE, but not render it correctly? Is the decoding in the VSE done differently than in rendering?
  2. What version of libavcodec is used in Blender 2.74? Is the implementation of e.g. lagarith faulty in this version? If not, then why does Blender mess up the rendering?
  3. It has been suggested that one should use "Blender-friendly" codecs for compression. Which lossless codecs are known to be so?

Edit: It seems there are several error sources here, which just makes everything much more confusing. The problem of the video playing back at twice the normal speed I had with this one particular lagarith encoded source seemed to stem from a corrupted (??) .blend file. Namely during editing Blender got stuck and I had to force it quit. When I re-opened this file Blender didn't give me any complains, but comparing its file size with the sizes of some other .blend files which were similar, I noticed that it seemed about 3 kB smaller than the rest. Opening this file and saving it to a new file ("Save as") got rid of this playback issue. However the video is still out of sync, even more subtly. Instead of the video globally playing back faster, it sometimes seems to "get stuck" and then "catching up again".

These issues are also already visible in the VSE, so it seems that my problem at least in this instance is solved. On the other hand this makes the behaviour of Blender seem even more mysterious, as I do not understand how a corrupted (was it even corrupted??) .blend file can lead to such an effect.

As I wrote, I experienced this issue with several videos, one of them being a video encoded in FRAPS. There the audio and video seemed in sync in the VSE, but wasn't in the rendered video. Strangely, after re-creating this .blend file from scratch, the audio and video now was out of sync even in the VSE. I tried transcoding it with ffmpeg, and this also gave an out of sync video. However transcoding it with Windows Movie Maker gave perfect result. So I suppose that for this particular video, the problem is that the fraps codec is implemented imperfectly in ffmpeg/libavcodec.

2nd Edit: The problem is back. Apparently it didn't have anything to do with the file being 'corrupted'. Running blender again on the very same file this time produced again an out-of-sync video. This issue seems to appear and disappear randomly.

3rd Edit: Still no clue why it only appears sometimes. But I noticed something else. The audio in the rendered video is as it should be, i.e. its start and end are as defined in the .blend file. However the last frame in the rendered video is actually way beyond the end frame. For some reason Blender consumes more video frames than it is supposed to be. I am going to file a bug report on this.

$\endgroup$
2
  • $\begingroup$ If you identify the codec, what frame rate is returned using a recent FFMPEG? $\endgroup$
    – troy_s
    Feb 24, 2016 at 3:54
  • $\begingroup$ Problem reoccured to day (despite using proxy/timecode). Running ffprobe on this file gave me the following output for video stream: "Stream #0:0: Video: fraps (FPS1 / 0x31535046), bgr24, 640x480, 21429 kb/s, 30 fps, 30 tbr, 30 tbn, 30 tbc)". Thus the reported frame rate coincides with the frame rate I set in fraps (of 30 fps). $\endgroup$ Apr 17, 2017 at 13:10

5 Answers 5

10
$\begingroup$

I was asked to take a look at this post. Codecs can be rather puzzling to diagnose from a distance, but it does seem that some information can be offered.

This problem seems always to pop up when then input source video is "mostly static", i.e. when the change from one frame to the next is miniscule (e.g. a screen recording). I should also add that I record the videos in question using a lossless codec (FRAPS or lagarith).

The “mostly static” comment here is interesting. This does indeed sound like an I versus P / B frame issue, as the “mostly static” frames would be subject to many more instances of P / B frames. While I am unfamiliar with Lagarith or FRAPS, I would be extremely surprised if a non-GOP codec had this issue, as your solution hints toward.

Why can Blender (or its version of libavcodec) apparently correctly decode and show the video in the VSE, but not render it correctly? Is the decoding in the VSE done differently than in rendering?

The VSE is using certain techniques to display the video that, when decoded on a shot by shot basis for rendering are different. This is why non-GOP codecs are more robust, or time code indices will yield more consistent results. This is also why most always-online NLEs will transcode upon ingestion; to avoid the codec problems as well as try to reduce the always-online image degradation issues.

What version of libavcodec is used in Blender 2.74? Is the implementation of e.g. lagarith faulty in this version? If not, then why does Blender mess up the rendering?

FFMPEG's version was a bit nasty on some platforms as it previously used a bundled version on some while on others it used a local version. If you check the ffcompat code, you can see many version checks to support generic versions of FFMPEG.

It has been suggested that one should use "Blender-friendly" codecs for compression. Which lossless codecs are known to be so?

The easiest way to avoid all of the codec mess is to rely on still frame intermediates. If you must for whatever reason rely on codecs, you are likely going to be constantly faced with these sorts of issues unless you manage to coax one of the codecs to behave under generated timecode indices. I would also speculate that FFv1 should be a reasonable choice to test and put under duress, given the nature of its design. If you try FFv1, it would be interesting if you reported your results back here for future reference.

Best of luck.

$\endgroup$
6
  • $\begingroup$ Thank you so much for this quite informative answer! Didn't expect anymore to see some light shed on this issue. I will try your suggestion and transcode on of the 'troublesome' videos to FFv1 to see whether this helps. Even though adopting to generate timecode seems to have banished this issue, it also make video editing a bit more awkward since generating proxy/timecode takes quite a long time. Perhaps transcoding to an intermediate codec would be a better option. $\endgroup$ Jan 3, 2016 at 20:53
  • $\begingroup$ Don't take my word as the word, though. Remain skeptical. The most telling part of your question was the "mostly static" comment, which really hints at encoding formats, and timecode solutions. I haven't looked at the FFMPEG situation across platforms in quite some time. To be honest though, if you are serious about imaging, I'd encourage you to adopt an offline pipeline and rely on a stills based intermediate format for online work. If you must, rely on a frame accurate codec that holds up (again, let us know about FFv1) during offline work. I'm glad this answer helped you out in some way. $\endgroup$
    – troy_s
    Jan 3, 2016 at 21:06
  • $\begingroup$ Okay I've tried the FFv1 codec now, and it doesn't help unfortunately. Video sync is completely messed up in VSE and in rendered video. $\endgroup$ Jan 27, 2016 at 10:07
  • $\begingroup$ @BlenderBender Interesting. Does that include using the timecode generated indexes? Did you generate the FFv1 using FFMPEG standalone? Does it seek well? If it seeks fine, then the audio is likely an encoding botch. As an aside, are you using WAV for the internal audio codec? Similar issues exist for audio of course... $\endgroup$
    – troy_s
    Jan 29, 2016 at 2:28
  • $\begingroup$ The problem re-occured today. I generated timecode for the FPS1-encoded video (with "Record Run"). It was totally in sync in the VSE, and it seeked well. But when I rendered it, I had the same issue as before. $\endgroup$ Apr 17, 2017 at 13:14
7
$\begingroup$

Unfortunately I can't really answer my questions 1, 2, 3, but at least I figured out how to get rid of said problem. It seems that Blender cannot be trusted to render a video correctly, if said video is not endowed with a timecode, even though most of the times Blender seems to do well without one.

TL;DR: Generate a "timecode" for each video strip before rendering, by selecting the strip, checking the "Proxy/Timecode" checkbox in the "Edit Strip" menu, choosing a timecode option e.g. "Record Run" and finally running "Rebuild Proxy and Timecode Indices".

Update 2017-04-17: Unfortunately, my solution isn't universal. I just had this problem re-appear today, despite generating timecode as proposed above. Not only that, but in fact the a/v-sync issue disappeared when I disabled the use of timecodes! There seems to be something seriously wrong with Blender.

Update 2017-04-21 This was with Blender 2.76 (the last XP-supporting version). I tried reproducing this issue today with Blender 2.78c, but there it was fine. So perhaps, this issue only affects older versions of Blender.

Update 2017-05-24 The problem reoccured today: I can confirm that this issue still exists in Blender 2.78c.

$\endgroup$
5
  • $\begingroup$ +1 for fixing my problem with video without audio. Read a number of answers and this was the only one which worked. $\endgroup$ Mar 24, 2017 at 13:29
  • $\begingroup$ It isn't Blender, it is codecs. Hence why the above solution I provided ultimately ends with "use still image frames", which is the only surefire solution. $\endgroup$
    – troy_s
    Apr 18, 2017 at 3:46
  • $\begingroup$ I don't see how it's codecs. It is not even a function of codecs: this happens to me even if I transcode the source video to ffv1. Also, other players are perfectly well capable of seeking these videos properly. And in the instance of this problem that occurred to me yesterday, Blender actually did properly render the video (and thus seeked the video properly), but only when I disabled the timecode Blender generated. How could this be? $\endgroup$ Apr 18, 2017 at 11:26
  • $\begingroup$ In the end, the issue is the nature of the internal timestamps with certain codecs. Sure, there may be a bug or some player that manages fine, but the end sum is that it will depend on architecture, and there is no single solution that will solve all cases when dealing with a multitude of codecs. This is also precisely why there is an "ingest" phase in the bulk of NLEs. There are just too many variables to offer a solution here, and why Blender should gut the codec support down to two or three. Just far too complex. $\endgroup$
    – troy_s
    Apr 21, 2017 at 5:35
  • $\begingroup$ As far as I can tell, this is not an issue of "it doesn't work with all codecs" but rather "it works most of the time, and when it does not, it fails for all codecs". $\endgroup$ Apr 21, 2017 at 12:29
1
$\begingroup$

Well, I once posted a question identical to this, which, funny enough wasn't in your list of duplicates. :D I found after the whole that they actually weren't out of sync, but rather the software I used (Windows Media Player) was being laggy and played it back funny, while other players (VLC media player & YouTube) would do it fine! This is a possibility of you see this occuring after your video is rendered.

If this happens to you before rendering and only in Blender's live playback, most likely you are dropping frames. To fix this, in the toolbar if the timeline window there is a tab called "playback". Open it up and click "AV-SYNCING". Only click it if the checkbox next to the label isn't already checked.

$\endgroup$
1
  • $\begingroup$ As I wrote, the video plays back fine in VSE. Only the rendered video suffers this problem, and this problem shows up in every player. $\endgroup$ May 26, 2015 at 21:37
1
$\begingroup$

**edit: The problem went away on its own for some reason. I converted alot of my project to 16 bit png as I added more edits. So also believe this is the solution for fixing the problem, that the video originally used (an h.264 mkv) was to blame. And others have replied stating the same here and elsewhere. Animation cache doesn't seem to to fix audio issues like it used to, potentially improves quality slightly though. Honestly the quality is bad to begin with using audio/video renders....just use mixdown and recombine with ffmpeg or some other program that won't re-encode (tossing it back in blender re-encodes remember so dont do that).

I don't see any solution here, so here is mine :) I had the same problem. My fix for 2.79b was 1. update the animation cache right before u render under scene->audio ;2. render my video without an audio codec, specifically try it as matroska h.264 ;3. mixdown ur audio as a .wav ;4. combine the audio and video in ffmpeg. Use command: ffmpeg -i videoname.mkv -i audioname.wav -c copy makeupanoutputname.mkv; you guys can go through on ur own and determine what step specifically solves the problem and if other codecs/containers work as well all the same. its probably just NOT rendering with an audio codec that solves the problem.

This is clearly a blender problem and not an ffmpeg problem. I'm very upset that the chosen solution doesn't give people a way to fix anything and is telling us something that the evidence points to being a lie.

VSE: rendered audio is offset (audio+video render)

$\endgroup$
3
  • $\begingroup$ If you're upset by this problem (as I am), I highly recommend that you open a ticket on their bug tracker. I already opened two of them, but they got ignored; apparently, they don't think this issue is a priority. But perhaps, given that enough people complain, they will change their mind. $\endgroup$ Sep 19, 2018 at 14:44
  • $\begingroup$ Regarding your comment about (lack of) solutions: The only completely reliable way to avoid this issue that I've found is to convert your video into a sequence of images, and import those as an image strip. Yes, this does make video editing way more awkward, but, it is doable. $\endgroup$ Sep 19, 2018 at 14:55
  • 1
    $\begingroup$ I've had very similar experiences when opening tickets and asking devs questions. Often ignored or told it's a nonissue. If they can't replicate your system specs exactly, they won't/can't help...that's my experience and understanding so far anyways. This is doubled down by the fact that vse related problems are intentionally sidelined. Although it might help change opinions, I Personally can't recommend this route to myself or anyone else until VSE development is officially resumed. $\endgroup$
    – kite
    Nov 15, 2018 at 7:41
1
$\begingroup$

So after a lot of online searching, I did find an answer to fix this problem, but not in Blender. If you are like me and would like to use Blender for video editing and still get around the issue, then I found a workaround, but you need Shotcut for this. Shotcut is another great free and open-source video editor

  1. Export the entire long video from Blender (the rendered video has desync issues as expected).
  2. Open the video in Shotcut and detach the audio from it.
  3. Use the audio properties to make very fine adjustments to the audio playback speed to suit your requirements (make fine adjustments until video and audio are in sync).

Follow the GIF attached. (I am using a shorter video in the GIF but you get the idea)

enter image description here

Explanation

  • Blender has issues while rendering long videos and I noticed that the video is exported at 1.0x speed but the audio is sometimes faster (1.00400x or something like that) and hence the rendered video has audio not in sync with the video.
  • Another bad thing is that Blender does not really allow very fine playback speed adjustment just to the audio.
  • One trick is to adjust the pitch of the audio in Blender which in turn changes the playback speed but this is only allowed up to 2 decimal places (not what we want for long videos) and it makes the audio sound funny (since it actually changes the pitch).
  • Shotcut is a great tool that allows fine playback adjustment, and it also has a pitch compensation feature so that your pitch is kind of unaffected (since we don't want the characters to be sounding funny in our edited video).
  • Shotcut allows playback speed adjustment up to 6 decimal places.

I had posted a similar question earlier, link to that post can be found here.

I have also created a short video about this, which can be found here.

$\endgroup$
1
  • $\begingroup$ Unfortunately, it degrades the quality of the audio. Is there a way to add the original audio to the video in Shotcut? $\endgroup$ Apr 12, 2022 at 20:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .