I understand flat shading is a big no-no for parts of the geometry that curve, so transition between face do not show. I also understand the difference in the viewport: flat shading gives a single color to a whole face whereas smooth shading interpolates across a single face or many faces depending on lighting. But when it comes to parts of the geometry that ARE actually flat (e.g. table top) and have a cycles shader on them (vs the single viewport color), I find it really difficult to tell that the best option is (or at least why). See the comparison in my current project (low sampling):

The area circled in pink must be smooth: it is curvy and otherwise faces will show. However, area in green and blue have a completely flat geometry. The green area seems more realistic when flat shaded (ignoring the noise due to low sampling), which makes sense, but the blue area looks better smooth, for a reason I can't put my finger on. I know that there are never any 100% true flat surfaces in reality, but surely everything cannot be smooth shaded, because of what can be seen in the green area (makes a rectangular prism look like a cylinder). And the blue area is just fine smooth shaded, and the angle is 90d too! Why is that?

• What seems more convenient to me is adding a bevel modifier and setting up the bevel weight for each edges you want sharp. You can also bevel each edge independently if you are 100% sure of what you are doing. I never use flat shading, never ... – Yvain Oct 15 '15 at 2:14
• This was my first thought as well, and if you look closly, all the geometry is bevelled, including the green area (look at the flat shaded version to see it). The bevel seems to disappear in the smooth shaded version however. My guess is that's because the bevel itself is smooth shaded, making it seem like the 90d angle is round. But why not the blue area? I don't get it. Maybe because the green mesh is so small compared to the other? AFAIK, the is no way to change the shading resulting from a modifier unless you apply it. – Eranekao Oct 15 '15 at 2:29

The answer given by @R.M is the way to go to achieve a photo-realistic result, but it requires to modify your mesh. If you can't afford to slightly change your geometry, another solution exists: auto-smoothing.

Set your whole mesh as smooth shaded and go to the properties panel. Under the category object data / normals you will find the option Auto Smooth. Turn it on and tweak the angle to your liking. This will automatically set the faces above this angle threshold as flat and the others as smooth.

Here is a simple example with a cylinder:

Quick render with smooth shading, notice the unwanted behaviour near the top and bottom faces.

Quick render with auto smooth activated. The top and bottom faces are flat shaded while the other are smoothed.

I'll use this example cylinder to explain my answer:

It has smooth shading, and looks pretty horrible when rendered:

This is because the normals are calculated based on the vertices of every face, making Blender think that there should be a smooth transition between the top face and the faces around it.

To tell Blender that we want sharp edges to be there, we need to add a Bevel modifier:

This will add extra vertices around the faces, better defining the shape. Note that the width is very small and there should be at least two segments. Now it's better:

But there are still a few issues. The sides are now sharp, but I wanted them smooth! It's a cylinder, after all, and not some n-gonal prism.

To fix this, we need to set a limit on the angles that we want to bevel. Yes, we want sharpness in those 90° angles, but the edges forming 30° degrees should not be beveled. So enable the angle limit, and set it to a value that differentiates these two angles:

Now, the cylinder is much smoother where it needs to be:

But wait, what are those annoying black lines messing up my images?! I need to get rid of those. Let's add a subdivision surface modifier:

Now, the cylinder finally looks nice:

Plus, you get that shine at the edge that you wouldn't get with flat shading. This makes the cylinder look more realistic.

Use edge split modifier on your glossy object, you can set an angle as when to show a hard line, and what not to smooth out.