To do this, you need to use your custom property to drive a value node.
EDIT: Added compositor example in addition to the original cycles material example.
This is possible in either the material or compositor node trees. Below is an example demonstrating the process in the compositor, and at the end I'll also include an example from a cycles material node setup.
- Add a value node ( ShiftA --> Input --> Value ) to your node tree.
- Right click on the number in the input box and add driver.
- Open the graph editor window, change the view mode from F-Curve to Drivers, then open the properties panel (N).
- Change the driver Type to Scripted Expression.
- In the variable's Path box type the name of your custom property delimited by square brackets and double quotes. Example: ["myProp"]
- In the Expr (expression) input box, make sure the name of the driver variable (default "var") is present.
That's it. You're value node is now connected to the value of your custom property, and any changes you make to the custom prop will be reflected there.
In the compositor, however, changing the custom property does not immediately trigger a scene update, which means the result will not be shown until you perform some other action that updates the view. In the example below, I simply Ctrl + LMB clicked on the Color node to reconnect the Viewer node, which updated the view.
Cycles material example:
