Certain things like image textures are cached after a render, so they don't have to loaded from disk if you render again. That can be disabled in the Render Properties panel(But i wouldn't recommend it unless you are really low on RAM):

As previously mentioned, after a rendering, the system memory usage is definitely higher, but there is no runaway memory leaks, so you can just keep rendering again and again without having to worry about Blender running out of memory.
For the GPU renderer, data on the GPU RAM are freed as soon as the render is done.
"Free RAM is wasted RAM". As long as your system has enough RAM that it's not swapping to disk, I wouldn't worry too much about it.