There are three places blender looks for addons and config files, they are referred to as local, user and system - If config files are found in the local location then the user and system locations are ignored.
So for your blender preferences you need to place them inside blender.app. Which would be blender.app/Contents/MacOS/2.76/config/startup.blend You can also add any extra addons here, you should be able to use blender.app/Contents/MacOS/2.76/scripts/addons which will match with the normal settings directories and keep all your additions in one place. If the addons don't work there then put them in blender.app/Contents/Resources/2.76/scripts/addons_contrib
You will also find the bundled python included with blender while you are adding your config files, unless deleted, this copy of python will be used by blender before any system installed python and you shouldn't have any trouble adding any third party modules here (as long as any dependant libraries are available).
I haven't used a mac for a few years so I can't comment on how this will interact with gatekeeper, but remember the gatekeeper settings on the machine will be the biggest factor.