How to run scripts in the background of the game engine without freezing or opening new Blender instances on Windows?

Question

The Blender game engine does not move into the next frame if all the active scripts have not yet finished.

Say we have an HTTP script in the game. The script retrieves data from a server. I presume the retrieval process cannot be split into multiple small processes.

I know only of the subprocess module's ability to work outside the bge, but on Windows a new Blender window pops up every time the subprocess module is used to start any new Python script.

Answer 1

The threading module should take care of it. Just create a separate python thread and run your blocking code (e.g. networking code) in there.

import time
import threading

def main():
    thread = Worker(None)
    thread.start()

class Worker(threading.Thread):

    def __init__(self, args):
        self.args = args
        threading.Thread.__init__(self)

    def run(self):
        print("Starting worker")
        ...
        print("Finishing worker")

Note that Python 3's threading model does not map to the OS threads, all the threads will run under the Python interpreter anyway. Because of this, threading is only useful for executing blocking operations, not for performance gain.

Answer 2

Or simpler

import threading

def to_run(): # should not take arguments.
    ... # code to execute in the thread (can be a loop, ...) 
        # it uses the same variables as in the main thread.

new_thread = threading.Thread() # create a new thread object.
new_thread.run = to_run
new_thread.start() # the new thread is created and then is running.