I would love to replicate this effect in Blender, but have no idea how to approach the problem. Does it need a custom shader? and if yes, can you link me to references/guides on how to achieve a similar effect?
Not really sure what you are asking about.
For the modelling part it could be done manually, although, depending on the size and complexity of the molecule it could be quite a lot of work. From the screenshot the user seems to be either using some sort of plugin to create it, or it was imported into Blender from some other third party application.
The lighting part seems to be very simplistic too, either some ambient light or even no lighting at all, just use a basic Cycles Ambient Occlusion shader.
As for the materials it also seems to be a very basic Diffuse Shader, or alternatively as mentioned previously an Ambient Occlusion shader with some basic plain flat colors, no texturing applied.
Apart from the modeling which was most likely not done by hand, nothing here seems to be using any sort of addon or non standard Blender feature.
Its a open source fork of Blender called Bioblender, which can generate 3D models from protein database files. As far as I can tell, the program uses the Blender Internal renderer to generate images.