How to setup your system python for use with Blender

Question

I found lots of material on how to make Blender use the system version of python. Essentially: delete the python folder of the install, and Blender looks for a system version, where on some systems you need to supply the path yourself as:

PYTHONPATH=/usr/lib/python3.7 ./blender --python-use-system-env

Official information from Blender docs found here.

What I cannot find is how to set up the system python to correspond perfectly with that of a given Blender version. Prefferably, I would like to do this with virtualenv.

The workflow I am interested in woud look something like this:

1. make a virtualenv that corresponds exactly to that of the Blender bundled python + dependencies/modules.
2. make Blender use this system python, of this virtualenv (if I have to issue a command like workon Blender2_91_2 before I start Blender, thats ok)
3. modify my virtualenv as need arise. Any change I make, I know i am likely to break things, which hopefully would be only other addons, which will be ok (its all only a virtualenv that I can play around with, since i know the proper bundled one)

Answer 1

Edit.. If having encodings hassles try virtualenv

After installing python 3.77 to /usr/local/ from a tarball, thought I would create a new python venv for blender. Using /usr/local/bin/python3.7 -m venv foo --copies I ran into the no module named encodings error, and found no way to alleviate the situation.

Thought I'd give the somewhat deprecated virtualenv a try, and bingo, blender ran straight up.

/usr/bin/virtualenv -p /usr/local/bin/python3.7 --always-copy foo

creating a copy of the python to venv and from it blender runs without any extra steps required.

---

A way for Ubuntu

Having horrible issues building blender currently, so have gone thru the motions of setting up virtual environments for freshly downloaded 2.91.2 and 2.92.0

Have both in vers folder

Note commands shown below are <prompt> $ <command>

shitbox:~/blender/vers$ ls
blender-2.91.2-linux64  blender-2.92.0rc-linux64

renamed the python folder in both from python to _python

shitbox:~/blender/vers/blender-2.92.0rc-linux64/2.92$ mv python _python

Now set up a virtual environment named "bar" and changed into that directory

shitbox:~/blender$ ~/blender/vers/blender-2.92.0rc-linux64/2.92/_python/bin/python3.7m -m venv bar --copies 
shitbox:~/blender$ cd bar

next have made link copies of blender's distribution python lib to venv lib

shitbox:~/blender/bar$ cp -as /home/batfinger/blender/vers/blender-2.92.0rc-linux64/2.92/_python/lib .

Created symlink to blender into the venv's bin.

shitbox:~/blender/bar$ ln -s  /home/batfinger/blender/vers/blender-2.92.0rc-linux64/blender bin/blender

activate the venv

shitbox:~/blender/bar$ source bin/activate

and start blender telling it to use this venv

shitbox:~/blender/bar$ BLENDER_SYSTEM_PYTHON="$VIRTUAL_ENV" blender

The environment variable can be set in the activate script in your venv's bin folder, however on Ubuntu I am using direnv which automagically activates the environment when entering the folder using an .envrc file. So I have stuck it in there

export EDITOR=nvim
source $(pwd)/bin/activate
export BLENDER_SYSTEM_PYTHON="$VIRTUAL_ENV"

echo "Blender 2.92"

and similarly for 2.91 in "foo"

(foo) shitbox:~/blender/foo$ blender
Read prefs: /home/batfinger/.config/blender/2.91/config/userpref.blend
found bundled python: /home/batfinger/blender/foo

Can install in the venv with pip

pip install matplotlib

and bingo in blender

>>> import sys

>>> sys.executable
'/home/batfinger/blender/foo/bin/python3.7m'

>>> import matplotlib
>>> matplotlib
<module 'matplotlib' from '/home/batfinger/blender/foo/lib/python3.7/site-packages/matplotlib/__init__.py'>

Notes.

Above is demonstrating how to use the python that ships with blender to make a venv. Would be very inclined to download and build the matching version Python3.7.7 and use this as the basis for a venv. (It's my experience that python builds pretty well hassle free)

It's been a while since I downloaded an official build of blender and was a little surprised to see both versions above using python 3.7.7. (Have been building against 3.8.2 since blender 2.7, last run of install_deps.sh to build was using 3.9.1 .. miss not building with the "latest" .. don't miss when it doesn't work)

Since both venvs use same python version it feels a little overkill to install scipy, matplotlib, pynvim, plumbum and other 3rd party modules I use alot for each venv.

In the past any module I wish to have available for all instances I have used

pip install pynvim --user

which is unavailable in venvs as setup above, unless python --system-site-packages was used creating venv, or in hindsight edit pyvenv.cfg to have include-system-site-packages = true or add a path to python path in .envrc with

export PYTHONPATH="$HOME/.local/lib/python3.7/site-packages" 

will make that path available to python in your venv, but not in blender..

A quick way around this is make a script to add paths to blender

addpaths.py

import sys
sys.path.append("/home/batfinger/.local/lib/python3.7/site-packages")

and start blender with

blender -P addpaths.py

TODO write a script to automate the above.

Answer 2

There are few methods but I would prefer just inserting the path where our libraries are located or the path to Python interpreter to the sys.path list in python. For example, we could create a virtual environment and initialize it:

python3 -m venv venv
source venv/bin/activate
pip install matplotlib

Then in our python script we add the following lines before importing the needed libraries:

import sys
# Insert the path at the beginning of the list so it look for the needed
# libs in our venv first.
sys.path.insert(0, "<path_to_venv>/lib/<pythonx.xx>/site-packages/")
# Import installed libs.
import matplotlib

That way we won't break anything by deleting files and doing weird stuff.