Something is wrong with my cloth. I made a wizard robe skirt but it goes out of shape when I bake it.
Here is my file.
There are few things that seem to be causing this.
The main thing was that the legs had a largish outer collision margin which was causing the robe to collide constantly.
Try changing this to something like .004 on both legs:
Also try making the skirt bigger initially. There has to be room for the legs to move ;)
I also pinned the entire top edge loop instead of just the corner vertices, reduced the collision quality to about 3 and disabled self collision. These massively decreased the bake time without a lot of visual impact once the collision issue was fixed.