The way origin to geometry works is it moves the origin to the average location of all vertices. So since your mesh has more vertices on the top than the bottom, the z-axis position of the new origin point will be closer to the top than the bottom.
To fix this, simply go into edit mode, select the eight vertices at the corners of the bounding box (I am assuming your model has some x-depth thickness, if not just select the four corners), and hit Shift+S > cursor to selection. Then go into object mode and select origin to 3D cursor.