- Your UV-map is sort of twisted. I guess you've seen it yourself in a certain way and tried to fix it, so that in the end the color map kind of fits. But it remained twisted what will cause the shader to flow the wrong way around your object for example.
One way to test your UV-maps is to create a new texture in your UV-editor and instead of leave the listbox at "blank" switch it to UV-grid or color-grid.
- Every texture that is not a color-map must be set to non-color data. You will find this setting in the texture node inside of the Shading tab. The point is named Color Space.
Without getting too deep into it, your specular basically looks that weird due to the fact that your object has less geometry. Basically the more geometry you give your object, the more detailed your images will be. But more isn't best for every situation, it depends!
But when you add some loops to your object you will see a much better result right away.
I was lazy and just added a 2 times subsurf modifier to show.
When you compare both you see the squeezed sides and no edge specularity for example