This was a fairly hard problem to find. I also discovered that it is not an easy one fix. The issue is that the particle info node gets messed up when you use time remapping.
I'm not going to explain, instead read this post with no answer. And if you didn't understand that one, read this one, with no answer. And why not this one, with an answer that only works if particles are your only moving objects.
I have many moving objects, rigs, particles, and cameras. The crazy particle movement is solved after baking, but the particle node, nope. 2011, 2013, 2016, and now... 2019. A solution is well deserved.
cycles - 2.79