Try using a meta strip. Select the layers you want to apply the speed to and press Ctrl G (or go tho the menu Strip->Make meta Strip) That will create a new strip that contains the selected strips.
Then you set a speed strip (or whatever other effect you want) on top of the meta strip.
Quote from the wiki:
A Meta Strip is a strip which contain several strips. It's like a Group of strips, working the similar way as Nodes groups. It allows you to reduce significantly the (vertical) space used in the sequencer, and treat multiple strips as if it was one strip. You can move, resize, and edit properties exactly the same way strips are. (see Editing and deleting strips page for more informations.)
You can edit the content of a Meta Strip by pressing ⇆ Tab; It will unwrap the content temporarily and hide any other strips present on the workspace, allowing you to select, move and resize the contained strips, or edit their properties. Press ⇆ Tab again to re-wrap the strips into one Meta Strip.