Hear timing reference audio from VSE while editing other scenes? (2.8 beta)

Question

I'm making an animation using grease pencil, based on a narration track. What I'd like to do:

1. Import and place the master audio file (narration) as a single sound strip in the VSE, in Scene0

2. Create visuals as grease pencil scenes sequenced in the VSE too, from which a final export can be made. Scene1 - Scene50.

3. I want to use the narration track in the VSE in Scene0 as a timing reference for all visuals in all scenes of the project. e.g. in Scene3 I want a specific action to coincide with a particular word in the narration. In general while working on the action in Scene N I want to hear the relevant part of the narration audio which has been placed in the VSE in Scene0, a different scene.

I have a good handle on 1 and 2, but I'm not sure how to realise 3. Is it possible?

Suggestions for workarounds or alternative approaches are welcome too. Though I'd like to avoid any workaround that involves having to re-render audio files if the duration of any of the grease pencil scenes has to change.

EDIT

In After effects there's a plugin called 'universal audio' that achieves something similar to what I'm looking for in Blender: It provides some buttons that allow you to designate a 'master audio' track, and then from child comps automatically copy in the appropriate section of audio from the master audio track, with in point correctly reflecting its position with regard to how the two comps are related. https://aescripts.com/universal-audio/

EDIT 2

I think that if I make sure I use the Scene name 'Master' for the one containing the VSE and name the narration sound clip 'Narration', then I'd need something like the following script (i'd run this somehow each time I opened one of the scenes to work on). But I'm not clear yet on how to have this accessible via a button or something convenient.

# Run this from a 'child' scene to add narration track with correct timing offset.
# TODO add routine for removing narration track from child scene if it already exists, before re-adding it.

    import bpy

    current_scene = bpy.context.scene
    current_se = current_scene.sequence_editor_create()

    master_se = bpy.data.scenes['Master'].sequence_editor
    narration_sequence = master_se.sequences['Narration']

    timing_ref_sequence = master_se.sequences[current_scene.name] 

    if timing_ref_sequence:
      offset = timing_ref_sequence.frame_final_start * -1
      path = narration_sequence.sound.filepath
      current_se.sequences.new_sound('new',path,0,offset)

Answer 1

To have the python script conveniently accessible, we have to embed it into an addon for 2.8.

To activate the addon for this file only, copy the code (at the end of the post) and save it in a file (e.g. adjustNarratorTrack.py). Open it in the script editor in Blender and check the Register checkbox in the header of the script editor. With register enabled, a saved file will be automatically executed once the blend file is loaded, even if you close the script editor.

To run the script, without reloading the blend file, click RunScript.

The script creates a python operator. The operator can be called by

• opening the search menu in the 3D View F3 and typing Adjust Narrator
• clicking on the menu item, which we created under 3D View ▶ Object ▶ Adjust Narrator Track

_{Note, that this "addon" is not quite complete yet. Further research should be done as to how scene/sound strips behave, when they are cut off in the sequencer.}

In the script

• a custom Operator AdjustNarratorTrackOperator is defined. This will be called by Blender when you click on the menu items. It will be also available to python through bpy.ops..
• two annotations of the operator are defined: audio_strip_name, cut_scene_name. These are adjustable after the operator has been executed. This is useful if you have changed your naming conventions.
  
  ^{The operator annotations appear in the 3D View after the operator has been executed.}
• there are two methods:
  1. execute: called by Blender
  2. adjust_audio_reference: a modified version of your script
• In adjust_audio_reference the current_scene, audio_strip_name and cut_scene_name are referenced from the operators context.
  Various checks are done to ensure, that
  1. the "Master" scene exists
  2. the "Master" scene contains the audio track
  3. the current scene is not the "Master" scene
  4. the current scene is contained as a strip in the "Master" scene
  5. the sound strip is replicated in the current scene if necessary
  6. the offset in timing is applied

In this case we want to use the frame_start property and not the frame_start_final property, since the frame offset to the frame_start_final doesn't reflect on the audio strips position at the start of a scene strip.

import bpy


class AdjustNarratorTrackOperator(bpy.types.Operator):
    """Adjusts the narrator track"""
    bl_idname = "object.adjust_narrator_track"
    bl_label = "Adjust Narrator Track"

    audio_strip_name: bpy.props.StringProperty(default='Narration')
    cut_scene_name: bpy.props.StringProperty(default='Master')

    @classmethod
    def poll(cls, context):
        return True

    def execute(self, context):
        try:
            self.adjust_audio_reference()
        except Exception as e:
            self.report({'ERROR'}, str(e))
            return {'CANCELLED'}

        return {'FINISHED'}

    def adjust_audio_reference(self):
        current_scene = bpy.context.scene
        audio_strip_name = self.audio_strip_name
        cut_scene_name = self.cut_scene_name

        # check if the master scene exists and contains the audio_strip
        master_scn = bpy.data.scenes.get(cut_scene_name)
        if master_scn == None or master_scn.sequence_editor == None:
            raise Exception('Master scene not defined.')
        master_se = master_scn.sequence_editor
        ref_audio_strip = master_se.sequences.get(audio_strip_name)
        print(ref_audio_strip)
        if ref_audio_strip == None or ref_audio_strip.type != 'SOUND':
            raise Exception('Audio strip not found.')


        # check if the current scene is embedded in the master scene
        timing_ref_seqs = [s for s in master_se.sequences if s.type == 'SCENE' and s.scene == current_scene]
        if len(timing_ref_seqs) == 0:
            raise Exception('Current scene is not contained in the Master scene.')

        ref_scn_strip = timing_ref_seqs[0]


        current_se = current_scene.sequence_editor
        if current_se == None:
            current_se = current_scene.sequence_editor_create()
        current_audio_strip = current_se.sequences.get(ref_audio_strip.name)
        if current_audio_strip != None:
            if current_audio_strip.type != 'SOUND':
                current_audio_strip.name = current_audio_strip.name + '_notSound'
                current_audio_strip = None
        if current_audio_strip == None:
            path = ref_audio_strip.sound.filepath
            current_audio_strip = current_se.sequences.new_sound(ref_audio_strip.name,path,0,0)

        timing_difference = ref_audio_strip.frame_start - ref_scn_strip.frame_start + current_scene.frame_start

        current_audio_strip.frame_start = timing_difference
        current_audio_strip.frame_final_start = timing_difference

def menu_func(self, context):
    self.layout.operator(AdjustNarratorTrackOperator.bl_idname)

def register():
    bpy.utils.register_class(AdjustNarratorTrackOperator)
    bpy.types.VIEW3D_MT_object.append(menu_func)


def unregister():
    bpy.utils.unregister_class(AdjustNarratorTrackOperator)
    bpy.types.VIEW3D_MT_object.remove(menu_func)

if __name__ == "__main__":
    register()

Answer 2

Further to my comment above this is what I had in mind

Set up a master audio scene. All it has is audio in the sequencer, no camera nothing else. For process of example I have called this scene "MasterAudio".

MasterAudio Scene

Placing this scene as a strip in any other VSE and we can hear the audio. All that is left is to align them.

The soft trim is animatable. I have added a couple of simple drivers to trim the master audio scene strip to match the context scene. This is a bit dodgy as it doesn't seem to update that well, and has the tendency to set the scene strips frame start.

Result of running on empty scene

import bpy
context = bpy.context
master_audio_scene_name = "MasterAudio"


def add_master_audio_scene(scene):
    sed = scene.sequence_editor or scene.sequence_editor_create()
    ma_scene = bpy.data.scenes.get(master_audio_scene_name)
    mas = sed.sequences.new_scene(
            master_audio_scene_name,
            ma_scene,
            1,
            ma_scene.frame_start)
    # only use sed of mas
    mas.frame_start = scene.frame_start
    mas.frame_final_duration = scene.frame_end - scene.frame_start
    mas.use_sequence = True

    # drivers         
    fcurve = mas.driver_add("frame_offset_start")
    driver = fcurve.driver
    var = driver.variables.new()
    var.name = "fs"
    #var.id_data = scene
    var.targets[0].id_type = 'SCENE'
    var.targets[0].id = scene
    var.targets[0].data_path = "frame_start"

    var = driver.variables.new()
    var.name = "mfs"
    var.targets[0].id_type = 'SCENE'
    var.targets[0].id = ma_scene
    var.targets[0].data_path = "frame_start"


    driver.expression = "max(0, fs - mfs)"
    fcurve = mas.driver_add("frame_offset_end")
    driver = fcurve.driver
    var = driver.variables.new()
    var.name = "fe"
    var.targets[0].id_type = 'SCENE'
    var.targets[0].id = scene
    var.targets[0].data_path = "frame_end"

    var = driver.variables.new()
    var.name = "mfe"
    var.targets[0].id_type = 'SCENE'
    var.targets[0].id = ma_scene
    var.targets[0].data_path = "frame_end"

    driver.expression = "max(0, mfe -fe)"


    # drivers

scene = context.scene
add_master_audio_scene(scene)

Answer 3

Cut your audio e.g with Audacity and , in Blender 3D view, put a Speaker object to hear your audio bit. When you go to the next frame, insert respective audio bit and so on.