I'm pretty new to blender and 3d modeling in general but I feel like I'm on the right track with this material, I'm just not sure how to tweak it to make the effect more photorealistic.
Getting that pulled apart cotton ball-ish effect on the cloth material is important for the purposes of the image. I tried playing around with hair particles for that but it wasn't really doing what I wanted and it didn't seem to be super cooperative when I was trying to overlay the silk screened text into the material.
Here's a link to the blend file.
Let me know if there's anything else I could provide to help or make things more clear.