I've been struggling with the same issue and found how to solve it.
 
1. Select all the points of your Grease pencil object in *Edit Mode*
2. Select not only all the keyframes but the animation layer in the [`Dope Sheet Editor`][1]
3. At the top panel turn on multiframe 
4. Now you can apply a new material to all the keyframes in your animation.

  [1]: https://docs.blender.org/manual/en/latest/editors/dope_sheet/grease_pencil.html