Skip to content

Handling forked processes with universalasync decorators #25

Description

@cheshirex

I just debugged a case where a user of a module I maintain which uses universalasync had combined it with multiprocessing and caused a hang. I figure the case is atypical, but worth bringing up.

Combining multiprocessing with async doesn't normally make sense, but I'm using universalasync to maintain a library that provides service to both sync and async code, and in this particular case, had sync code running with a ProcessPool, and the called function was in turn calling another function using async_to_sync_wraps. The result was a hang, because the fork still picks up the event loop from the parent process, but that event loop isn't valid in the subprocess.

I've got a local hack fixing this in my own use case now, but the AI-suggested fix within universalasync would be:

_LOOP_PID: int = os.getpid()

def get_event_loop() -> asyncio.AbstractEventLoop:
    global _LOOP_PID
    loop = _get_event_loop()
    current_pid = os.getpid()
    if loop.is_closed() or current_pid != _LOOP_PID:
        _LOOP_PID = current_pid
        return _create_new_event_loop()
    return loop

i.e. check the current process ID against whatever process first entered the code, and if they don't match, we're in a forked process, so get a new event loop and save the process ID in memory for the current process.

Thanks!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions