Building `bpy` for Windows: where should `datafiles` and `scripts` be placed?

Question

This is the documentation explaining how bpy should be built.

I think it is outdated, for the following reasons:

1. It does not mention that make provides a convenience target called bpy, which sets the build variables automatically as recommended by the documentation (blender\build_files\cmake\config\bpy_module.cmake).

2. It recommends that once building is complete, the following instructions be run:

copy bin\bpy.pyd C:\Python37\Lib\site-packages\
copy bin\*.dll C:\Python37\Lib\site-packages\
del C:\Python37\Lib\site-packages\python37.dll
xcopy /E bin\2.81 C:\Python37\

A minor issue with the above instructions is that it seems to have the file structure of the build output wrong. In my case, I have bin\Releases\bpy.pyd etc., not bin\bpy.d.

A possibly minor issue is that it while it recommends deteting the copied over python37.dll (or alternatively, not copying it over in the first place), it does not recommend deleting python3.dll.

A major issue is that placing the folders under \bin\Releases\2.xy directly in the Python37 seems to not work, because running import bpy produces the following errors:

Color management: using fallback mode for management
Color management: Error could not find role data role.
Color management: scene view "Filmic" not found, setting default "Standard".
blf_load_font_default: 'fonts' data path not found for 'droidsans.ttf', will not be able to display text
blf_load_font_default: 'fonts' data path not found for 'bmonofont-i18n.ttf', will not be able to display text
blf_load_font_default: 'fonts' data path not found for 'bmonofont-i18n.ttf', will not be able to display text
bpy: couldn't find 'scripts/modules', blender probably wont start.
Freestyle: couldn't find 'scripts/freestyle/modules', Freestyle won't work properly.
ModuleNotFoundError: No module named 'bpy_types'
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): C:\Users\bhmer\blender-git\blender\source\blender\python\intern\bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): C:\Users\bhmer\blender-git\blender\source\blender\python\intern\bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): C:\Users\bhmer\blender-git\blender\source\blender\python\intern\bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): C:\Users\bhmer\blender-git\blender\source\blender\python\intern\bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): C:\Users\bhmer\blender-git\blender\source\blender\python\intern\bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): C:\Users\bhmer\blender-git\blender\source\blender\python\intern\bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
F0718 15:21:13.967409  3600 utilities.cc:346] Check failed: !IsGoogleLoggingInitialized() You called InitGoogleLogging() twice!
*** Check failure stack trace: ***
    @   00007FFA676DF6DB  google::LogMessage::Flush
    @   00007FFA676DED29  google::LogMessageFatal::~LogMessageFatal
    @   00007FFA676E3683  google::glog_internal_namespace_::InitGoogleLoggingUtilities
    @   00007FFA6737079B  libmv_initLogging
    @   00007FFA669D6ED7  main_python_enter
    @   00007FFA66E98134  dealloc_obj_dealloc
    @   00007FFA88723B4A  PyDict_SetItem
    @   00007FFA88717A9B  PyModule_AddObject
    @   00007FFA88733D30  _PyType_Name
    @   00007FFA88733B97  _PyType_Name
    @   00007FFA88733AFD  _PyType_Name
    @   00007FFA8870B7E0  _PyMethodDef_RawFastCallDict
    @   00007FFA88708E12  PyObject_SetAttr
    @   00007FFA887266C5  _PyEval_EvalFrameDefault
    @   00007FFA8870CDB6  _PyEval_EvalCodeWithName
    @   00007FFA887253EC  _PyMethodDef_RawFastCallKeywords
    @   00007FFA88725E02  _PyEval_EvalFrameDefault
    @   00007FFA887252D3  _PyMethodDef_RawFastCallKeywords
    @   00007FFA88725953  _PyEval_EvalFrameDefault
    @   00007FFA887252D3  _PyMethodDef_RawFastCallKeywords
    @   00007FFA887259FF  _PyEval_EvalFrameDefault
    @   00007FFA887252D3  _PyMethodDef_RawFastCallKeywords
    @   00007FFA887259FF  _PyEval_EvalFrameDefault
    @   00007FFA887252D3  _PyMethodDef_RawFastCallKeywords
    @   00007FFA887259FF  _PyEval_EvalFrameDefault
    @   00007FFA8870C95D  _PyFunction_FastCallDict
    @   00007FFA8871569F  _PyObject_CallMethodIdObjArgs
    @   00007FFA88715565  _PyObject_CallMethodIdObjArgs
    @   00007FFA88731B82  _PyBuiltin_Init
    @   00007FFA887153A7  PyImport_ImportModuleLevelObject
    @   00007FFA886F12AB  PyImport_ImportModuleLevel
    @   00007FFA66E9A170  BPy_init_modules

The initial complaints refer to not being able to find things that are in the datafiles and scripts sub-folders of bin\Release\2.xy\. So, I think that means that these folders are not placed correctly in the Python37 directory?

Am I interpreting the error messages correctly, and if so, where should I place the datafiles and scripts directories output by building Blender for a standalone bpy module?

Answer 1

I'll try to shed some light on your questions, building blender is not an easy task, because the Blender team provide many options that make Blender available for everybody's needs. So, to answer your questions through the 'stackexchange flavor':

1: Am I interpreting the error messages correctly ?

1.1: About your first statement ('1. It does not mention' ..):

The wiki on how to build Blender is not clear, but it is right. In order to be Ok together, you agree that :

• Blender use cmake from which you can set everything in order to build what you want (binary, python module, headless binary, ...)
• so you must know what you want to build, and where you want to install it.
• ccmake or cmake-gui (or any other cmake editor tool) provide a way to set build options
• you can toggle the 'advanced options mode' through your 'cmake editor' (ccmake, cmake-gui ..), from which you will find a lot of stuff, like target, install prefix. (the cmake options are really well done with Blender, the build mechanism is highly versatile)
• once configured (ccmake: 'c + g', aka 'configure' and 'generate'), cmake generates a Makefile according to your options, where you'll find the 'bpy' target you're talking about (issuing a 'make help' command from your build directory after the Makefile generation will show you all other available targets)

R1.1: so, it doesn't mention the 'bpy' target because it depends on the cmake option 'WITH_PYTHON_MODULE' which is 'OFF' by default. The cmake options file is really powerful, you can set everything you want before issuing a 'make all', and 'make install' commands. Of course, you still can use the 'make some_target' mechanism, but I would recommend to study and set cmake options, then issue a simple 'make' + 'make install'

1.2 'A minor issue with the above instructions ..'

You don't copy the wiki correctly: your last line:

'xcopy /E bin\2.81 C:\Python37'

should be:

xcopy /E bin\2.81 C:\Python37\2.81

which means: copy the content of bin\2.81 (so, everything after 2.81) to the directory C:\Python37\2.81 (so, inside the 2.81 directory)

R1.2: this is where you should find the missing 'datafiles' you need. The directory bin\2.81 above depends on the cmake option: CMAKE_INSTALL_PREFIX = some_path which will be created after issuing a 'make install' as the name 'CMAKE_INSTALL_...' indicates. So, it is by default set to (IIRC..)

CMAKE_INSTALL_PREFIX = 'path_to_blender_source/cmake_build_dir/bin'

1.3 'A major issue is that placing the folders under \bin\Releases\2.xy directly in the Python37 seems to not work, because running import bpy produces the following errors'

As said above, the wiki specify to issue the following command:

xcopy /E bin\2.83 C:\Python37\2.83

Note: because the target directory C:\Python37\2.83 doesn't exists yet, I guess Windows will ask you to create it.

R1.3: the way you copy things will not work, because you don't use the right directory. This is not an issue from the Blender build process point of view.

2. and if so, where should I place the datafiles and scripts directories output by building Blender for a standalone bpy module?

R2: Blender 'PORTABLE' or 'SYSTEM_WIDE' install, w/ or w/o python module.

Blender seeks some path at startup, whether a module or an app, but we have 2 cases:

R2.1 WITH_INSTALL_PORTABLE=OFF (default)

Blender provides a way to use the application as a standalone directory tree, where everything (including python binary) goes exactly where you want to use them and at the same location. If WITH_INSTALL_PORTABLE=OFF, installation goes to the 'SYSTEM_WIDE' default install, eg for Windows architecture something like 'Program Files\something' and so on for the datafiles (user/AppData or something like/..)

R2.2 WITH_INSTALL_PORTABLE=ON Blender will seeks its data into the 'x.y' version directory (like 2.83), which will reside where the binary (or module) is. In your case, you MUST choose WITH_INSTALL_PORTABLE=ON, and it means you must set the CMAKE_INSTALL_PREFIX accordingly. REF: see dev comment in build/CMakeCache.txt @993 (blender283):

//Enable building as a python module which runs without a user
// interface, like running regular blender in background mode (experimental,
// only enable for development), installs to PYTHON_SITE_PACKAGES
// (or CMAKE_INSTALL_PREFIX if WITH_INSTALL_PORTABLE is enabled).
WITH_PYTHON_MODULE:BOOL=OFF

R2.2 Where to put 'datafiles' ... and others mandatory data

• Blender seeks its modules from:
  • PORTABLE=OFF : SYSTEM_WIDE directory, according to the architecture (WIN, Linux, BSD, ...)
  • PORTABLE=ON it seeks from the local 'x.y' directory (x.y like 2.83) ; so, you must copy the 3 items:
    • datafiles\
    • python\ (for blender app)
    • scripts\ (addons, modules, startup, templates, ...)

into you 'x.y' directory, which in your case reside where the python module is. If you choose WITH_INSTALL_PORTABLE=ON, make will install everything into CMAKE_INSTALL_PREFIX The wiki tells you where to put everything, ie: C:\python37\x.y

To be clear, you must have:

• C:\python37\2.83\datafiles
• C:\python37\2.83\scripts

because your python module is into C:\python37\

When you build the blender app as 'PORTABLE', you can put you binary wherever you want, then simply copy the x.y directory at the same place your binary (or module) is, and I guess everything goes well.

To be honest, I never use blender as a python module (neither build it), I cannot help you on how to use it, but I'm pretty sure where the things goes, tell me if I'm wrong, I really could. Remember building blender as a Python module is for dev only and is experimental (283), like the team said in 'CMakeCache.txt' Hope it helps, take care, My 2 cents,

Answer 2

For future seekers, as it may be useful for someone. I also had problem with making use of blender compiled as a shared library, and I finally made it work. In my case it was caused by another blender installation which I have had added to PATH to make it possible to run blender via cmd. Moreover I had also added bpy.py module from that previous installation. Im not 100% sure which of those caused the problem but removing them both definitely fixed it. In the end I would like to leave here some links to sources which helped me with building process:

https://wiki.blender.org/wiki/Building_Blender/Other/BlenderAsPyModule

https://skwrites.in/building-blender-as-python-module-on-windows-10/

https://stackoverflow.com/questions/37187025/blender-python-bpy-init-py-apparently-importing-from-a-non-existent-mod

https://devtalk.blender.org/t/no-module-named-bpy-when-importing-bpy-in-python-script/14797

Edit, this one is also helpfull:

How to build Blender as a Python module?

Hope you will also manage to make your blender module work.