Yes, the sides aren't planar, and so their shape depends on how they get triangulated. All meshes get implicitly triangulated just before render, because our computers draw planes, and planes are defined by 3 points, not 4. We can see how this mesh is getting triangulated by using a wireframe material:
I'm also comparing to the triangulation on a copy of the mesh with a triangulate modifier, on "shortest diagonal" mode. You can see that this modifier creates symmetrical triangulation, so long as it's possible, because distances between symmetrical vertices are also symmetrical.
The method Blender uses for its implicit, automatic triangulation is "fixed", which uses vertex order to determine how faces get triangulated. The reason it does this is to make triangulation work in the same way for a given face regardless of deformation (like armature deformation.) Whatever you do, you don't want your triangulation popping when your character bends their arm! The triangulate modifier shown here will do that if you put an armature modifier before it.
Note that using a triangulate modifier can affect the normals of the object. Here, I've enabled "Keep normals", but that can be misleading: only custom normals are kept. To create custom normals, I've data transferred the normals from an untriangulated copy of the mesh.
You're right to be concerned about triangulation, but what you should be doing depends on what your end goal is. In general, you shouldn't be using faces that are so out-of-plane that you can notice their triangulation. Whenever making a mesh for export to an engine which does not use quads, you should be thinking about triangulating it (after generating custom normals, or before generating a normal map) before export. For rendering in Blender, there is no single right way to do things. For example, shortest diagonal triangulation with custom normals before armature is fine, but it will affect subdivision, most likely in a way you don't want.
