For some reason whenever I press Ctrl+P and choose the option "With Automatic Weights" so I can weight my model's meshes, Blender in some areas only weighs the body (leg bone only moving the leg but not the dress), while in others it weights more than it should (arm bone moves the whole shirt). What is causing this?
From the images you've provided it shows, that the body and cloths are different meshes. Bones would have to be close to/touching/pertruding cloth meshes to provide correct-ish automatic weights. I don't recommend this approach though, cause it will never give totally satisfactory outcome. To get it done properly you have to join parts of the body (hends, neck, feet) with the cloths. In my opinion though, better solution is to separate cloths and shoes meshes from the model and parent them separately to the same armature as body, then add mask modifier to the body to hide parts of it, that are hidden by the cloth, but can protrude during animation.