# Detecting Object in FOV camera in BGE

I am trying to detect if an object is visible from a camera (i.e. not only if it is in its FOV but also it should not be behind another object).

I was looking at using the cam.boxInsideFrustum and own.rayCastTo

Now, the issue I am encountering is that the cam.boxInsideFrustum does not seem to be providing the right information.

I created a function that generates a bounding box of a mesh (I could not find a ready to use function)

The bounding_box_str (structure) is a dictionary that has, as a key the name of the object (own['P']) (I have many moving objects running around the environment where the property P is a unique identifier) and as the value a list of vertices.

check_max and check_min are obvious functions.

def bounding_box(own):

global bounding_box_str

temp_box_str = []

X = [math.inf, -math.inf]
Y = [math.inf, -math.inf]
Z = [math.inf, -math.inf]

transform = own.worldTransform

for i in range(own.meshes[0].getVertexArrayLength(0)):
vertex = transform * own.meshes[0].getVertex(0,i).getXYZ()
X[1] = check_max(vertex.x, X[1])
Y[1] = check_max(vertex.y, Y[1])
Z[1] = check_max(vertex.z, Z[1])

X[0] = check_min(vertex.x, X[0])
Y[0] = check_min(vertex.y, Y[0])
Z[0] = check_min(vertex.z, Z[0])

for i in range(8):
str = "{0:03b}".format(i)
temp_box_str.append([X[int(str[0])],Y[int(str[0])],Z[int(str[0])]])

bounding_box_str[own['P']] = temp_box_str


Then, later in another function I have

if cam.boxInsideFrustum(bounding_box_str[own['P']]) == cam.INSIDE:


As you can clearly see, I just adapted the example code from:

https://docs.blender.org/api/blender_python_api_2_78c_release/bge.types.KX_Camera.html?highlight=cam#bge.types.KX_Camera.boxInsideFrustum

Unfortunately, this does not seem to work at all.

I was not able to check if the bounding box computation above is really correct. I tried to create a bounding box around the object with the vertices (adding faces) in the bounding_box_str above, but without success. I understand it might not be possible to create a new mesh on the fly in BGE.

I was trying to get a feeling of what the frustum of the camera is, but I understand this cannot be visualized (yet).

I then abandoned this idea and decided to check if any point of the mesh (or the centroid of the mesh) representing the object is in the camera image plane.

I tried the function in bpy_extras.object_utils called world_to_camera_view, but as I am passing object from the BGE it complaints that the object does not have a matrix_world transform.

I liked the world_to_camera_view' as it provides a normalized output, thus the same code works for multiple cameras.

I abandoned that idea and decided to use the methods in the camera class. There is a getWorldtoCamera() transform or the cam.world_to_camera (they seem to be equivalent), but it does not seem to provide what I need. The snippet of code I used is the following:

transform = own.worldTransform

for i in range(own.meshes[0].getVertexArrayLength(0)):
vertex = cam.world_to_camera * transform * own.meshes[0].getVertex(0,i).getXYZ()


vertex however is an X,Y,Z vector (I would have expected a 2D vector). Furthermore, I am not sure how to check from there if the point is indeed in the image plane of the camera.

The rayCastTo works well to detect things in between. So that part seems to be OK

If anyone has any suggestion/pointer to possible alternative methods, I would really appreciate it a lot!!!

• BTW: I also tried to use if cam.pointInsideFrustum(own.position) != cam.OUTSIDE: (I thought maybe the box i created is not valid or something) ... nothing. It does not work either. Frustrating!!! – Alby Apr 6 '17 at 15:52
• OK made some progress: 1) I now understand why the vertex mapped back to the camera space has 3 coordinates as it give a point inside the camera 'pyramid' then one needs to check if it is between the two clipping planes and within the image plane. Not sure how to get those info from the camera class though. Examples seem to only show using bpy. 2) I read in some other blog that the reason the ____InsideFrustum might not be working is because the camera is not initialized (projective_matrix not initialized). Anyone has any experience with that? Thx! – Alby Apr 6 '17 at 15:57
• Furthermore the if cam.pointInsideFrustum(own.position) != cam.OUTSIDE above gives always True !! Even for objects that are not in the FOV ... – Alby Apr 6 '17 at 16:10
• OK, SOLVED THE PROBLEM: One needs to render from all the cameras using viewports. In this way the projection_matrix (not projective_matrix as above, sorry) is set correctly (otherwise is just an Id matrix). You can find how to do that here [youtube.com/watch?v=-NC7ZtKcG1U], 2) the cam.OUTSIDE has a value 2 whereas the pointInsideFrustum only returns a True (1) or False (0)`. I did not realize that (and it make sense as it would not be possible to detect a point to be at the boundary :-), which is instead possible for a box or sphere). – Alby Apr 6 '17 at 18:47