It's impossible to tell what's going wrong in your particular case as there just isn't enough information in your question - you've only partially shown the 'internal' node tree (many of the nodes are hidden, meaning it's impossible to verify that they aren't marked as 'Clamped' and left assuming that, say, 'Root' is raising to the power of 0.5) and you don't show any detail of how the group is actually being used (it's hinted at behind your 'group' nodes but nothing explicit) - which is quite critical.
However, it is possible to produce such a result as shown below (assuming I've got my maths correct). The example image you provided must have been produced with a large number of iterations of the calculation - which would take a considerable amount of time to render. For my example I have limited it to a relatively small number of iterations to demonstrate the concept.
Firstly, rather than manually generate the node structures for the calculation I have created an add-on to generate the nodes directly from an entered expression (it's something I've been meaning to do for quite some time - building on the answer Cycles Materials Math node with more than two inputs). This add-on parses the entered expression and automatically generates the required nodes in a node group - this allows different expressions to be more easily tested since the expression can simply be edited. The add-on can be downloaded from https://github.com/baldingwizard/Blender-Addons - simply download the ZIP file and use Blender's User Preferences/Add-ons/Install Add-on from File to install, enable it, and Save User Settings. You should now have a new Custom Nodes menu in the Node Editor 'Add' menu that lets you add a Dynamic Maths Expression node. Entering an expression into the Expression field will automatically build the required maths nodes in a corresponding node Group. Since the expression is built from standard nodes it is fully GPU compatible (unlike an OSL solution). Note that the add-on is currently a work in progress and may contain rough edges (and I have some ideas for future improvements) - I'll tidy it up and file off any rough edges in due course but for now it shoul dbe reasonably functional.
With the new Dynamic Maths Expression node you can create the node trees for each of the functions by simply entering the relevant expression. For example,
'r' :
(x**2+y**2+z**2)**0.5
This automatically creates the following nodes behind the scenes :
Similarly, 'theta' :atan(((x**2+y**2)**0.5)/z)
And 'phi':atan(y/x)
We can then use a similar technique to create expressions for 'newx', 'newy', and 'newz' :
newx = (r**n)*sin(n*theta)*cos(n*phi)
newy = (r**n)*sin(n*theta)*sin(n*phi)
newz = (r**n)*cos(n*theta)
These can then be linked together to form the following node group for the calculation of one 'iteration' :
Note that some of the inputs are fed directly out to the outputs - this is to assist with linking mulitple iterations together. Also the 'Valid' input/output is modified based on the condition that the result of this node is detected as 'outside range'. This is to detect the termination of the series at the specific iteration - if any iteration is out of range then we need to detect it rather than taking acount of any subsequent iteration. Only those 'points' in the volume that successfully pass through all stages of the iteration are rendered as 'solid' (ie, part of the 'bulb'). The more iterations applied the more complicated the final surface (but the more complicated the calculation - so considerably longer render times).
The individual stages are linked together as follows :
Note that one group node is required for each iteration of the calculation (the more iterations, the more complicated the resultant surface). The 'n' setting dictates the 'order' of the mandelbulb (in this case '3').
In order to render as a well defined volume you will need to multiply-up the result to generate a reasonable Density (eg, 100) and use this in a Volume material similar to how you would for smoke. You will also need to significantly decrease the Volume Sampling Step Size parameter in the Render properties from it's default of 0.1 to, say, 0.005 or lower (this defines the resolution of the volumetric) - this will produce a more detailed result but will significantly affect render times.
This can produce the following results :
Blend file included (you'll need to install the add-on to activate the Dynamic Maths Nodes) 
Here is a further example - this time an 8th order mandelbulb (by setting 'n' to 8) with additional Iteration nodes in the chain to add detail (I used 16 iterations for this - ideally you'd use many more). This took approximately 3 hours to render on my system - it would fare a lot better on a system with GPU support (which I don't have).
