1
$\begingroup$

Setup: Blender 2.82

I want to display my webcam in real-time on a plane.

I managed to render images from my computer on a plane here.Can a image be render on planes through scripting?

However i did not manage to find something useful to try to capture in real-time my webcam continuously via a python script. So to open the camera and to display the image on the plane

Any help will be much appreciated.

$\endgroup$
3
$\begingroup$

Blender is more geared towards content creation and is not the right software for your project.

You can write a python operator which continuously grabs frame from the webcam and writes them onto an image texture.

  1. Add an image texture and name it image.
  2. Add a 3D object and texture it with the image. You will have to go to material or rendered view to see the image.
  3. Create a new text block. Write a modal timer operator. You can get the structure for this script in the text editor menu: Templates > Python > Operator Modal Timer.
    The modal timer operator is a Blender operator, which can be called from the interface and starts a timer. The timer calls a function at a fixed interval. In this function, we'll grab the camera frame and write it on to the image.
  4. Install opencv-python, which can access the webcam. Identify the folder which contains the python executable Blender is using. You can do this by going into the python console and typing import sys and in a new line sys.exec_prefix. Upon pressing enter you will see the path to the python executable folder. Go to that path (...\2.82\python\bin) and open a new command line interface. Install opencv-python with the command python -m pip install opencv-python, which will also install other relevant modules like numpy. Restart Blender.

Now that you're setup, open the modal operator and remove all content from it's classes methods. When entering the modal operator, execute is called. In this method, we have to initialize the modal timer, the opencv VideoCapture and a reference to the image in Blender.

def execute(self, context):
        wm = context.window_manager
        self._timer = wm.event_timer_add(0.5, window=context.window)
        self._cap = cv2.VideoCapture(0)
        self._img = bpy.data.images['image']
        self._size = tuple(self._img.size)

        wm.modal_handler_add(self)
        return {'RUNNING_MODAL'}

The modal method is called every timer tick. As in the template, let's leave the check for an exit condition (like a pressed Esc key) when we exit the modal operator. On a timer tick, we start by grabbing the frame with the opencv VideoCapture object.

ret, frame = self._cap.read()

Then we resize the captured frame to fit our blender images dimensions.

resized = cv2.resize(frame, self._size, interpolation = cv2.INTER_AREA)

To make the numpy array (which is the image opencv produced form the webcam) compatible with Blender's image datablock, we'll have to flip the R and B channels as well as the Y-dimension, convert the image pixels to floats and flatten it. (Blender expects image data to be a long list of rgba pixel values.)
This resulting "list" can be assigned to the pixels property of a Blender image.

rgb = np.flip(resized, axis=[0, 2])
rgba = np.ones((self._size[1], self._size[0], 4), dtype=np.float32)
rgba[:,:,:-1] = np.float32(rgb) / 255
self._img.pixels = rgba.flatten()

The full script at the end of this answer. Run it by pressing Run Script in the text editor. This will add the modal operator to Blenders runtime memory. In the 3D view, press F3 and type the name of your modal operator (the bl_label of your class). I named the classes bl_label Modal Timer Operator as in the template. Then click on the result.

Run this script at your own risk. Blender might crash, remember to backup.

While the framerate is insanely low, I can see my camera capture writing to the image. However the viewport updates sampling method doesn't get enough samples to udpate the image enough.

webcam capture

While my answer gives a method to inject your webcam image into Blender in "realtime", it clearly lacks speed and almost removes the Interactivity from the user interface. To make this remotely usable, you would have to implement multithreading storing the images in some sort of concurrent bag.

import bpy
import cv2
import numpy as np

class ModalTimerOperator(bpy.types.Operator):
    bl_idname = "wm.modal_timer_operator"
    bl_label = "Modal Timer Operator"

    _timer = None
    _cap = None
    _img = None
    _size = None

    def modal(self, context, event):
        if event.type in {'ESC'}:
            self.cancel(context)
            return {'CANCELLED'}

        if event.type == 'TIMER':
            ret, frame = self._cap.read()
            resized = cv2.resize(frame, self._size, interpolation = cv2.INTER_AREA)
            rgb = np.flip(resized, axis=[0, 2])

            rgba = np.ones((self._size[1], self._size[0], 4), dtype=np.float32)
            rgba[:,:,:-1] = np.float32(rgb) / 255
            self._img.pixels = rgba.flatten()

        return {'PASS_THROUGH'}

    def execute(self, context):
        wm = context.window_manager
        self._timer = wm.event_timer_add(0.5, window=context.window)
        self._cap = cv2.VideoCapture(0)
        self._img = bpy.data.images['image']
        self._size = tuple(self._img.size)

        wm.modal_handler_add(self)
        return {'RUNNING_MODAL'}

    def cancel(self, context):
        wm = context.window_manager
        wm.event_timer_remove(self._timer)
        self._cap.release()


def register():
    bpy.utils.register_class(ModalTimerOperator)


def unregister():
    bpy.utils.unregister_class(ModalTimerOperator)


if __name__ == "__main__":
    register()
$\endgroup$
3
  • $\begingroup$ Thank you very much! It worked perfectly. $\endgroup$ – cUser Apr 25 '20 at 19:12
  • $\begingroup$ Hi Leande sorry to reach you thru here, I couldn't find any email related in your profile here. Can we talk? andrepazleal@gmail.com $\endgroup$ – andrepazleal Aug 11 '20 at 23:09
  • $\begingroup$ @andre Use @ Leander to ping me in this room. $\endgroup$ – Leander Aug 12 '20 at 4:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.