Skip to content

Add shared RedisLock support - #125

Merged
wolph merged 9 commits into
developfrom
issue-124-shared-redis-lock
Jul 25, 2026
Merged

Add shared RedisLock support#125
wolph merged 9 commits into
developfrom
issue-124-shared-redis-lock

Conversation

@wolph

@wolph wolph commented Jul 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • add opt-in shared Redis locks through LockFlags.SHARED
  • prevent writer starvation with deterministic pending-writer election while preserving mixed-version safety
  • identify heartbeat holders individually so stale Pub/Sub clients can be reaped without evicting responsive readers
  • document shared-lock usage and add an optional tox -e redis-live environment

Why

RedisLock previously treated every subscriber as an exclusive holder. That made it unsuitable as an NFS-safe replacement for workflows such as parallel compilation that require multiple concurrent readers.

The heartbeat protocol now communicates holder identity and mode. Legacy timestamp responses remain exclusive, so rolling upgrades fail safely rather than allowing incompatible overlap.

User impact

Existing callers retain exclusive behavior by default. Callers can request concurrent readers with:

portalocker.RedisLock(channel, flags=portalocker.LockFlags.SHARED)

Waiting writers gate later readers, avoiding writer starvation.

Validation

  • uv run pytest -q — passed with 100% coverage
  • REDIS_PORT=6399 uv run tox -e redis-live — 74 passed, 1 expected fakeredis-only skip; no live-server skips
  • live deterministic writer-election regression repeated 10 times
  • uv run mypy --show-error-codes --no-color-output
  • uv run basedpyright
  • uv run pyrefly check --output-format min-text --summary=none --color never
  • uv run ty check
  • uv run tox -e codespell
  • uv run tox -e docs

Closes #124

@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

wolph added 8 commits July 24, 2026 14:52
…ock interpreter exit

Failing shared-lock tests on CI left unreleased locks whose non-daemon
pubsub worker kept pytest alive until the 30 minute job timeout. Losing
the connection releases a redis lock by design, so process exit must not
be blocked by a held lock. Also normalizes formatting that
'ruff format --check' flagged on CI.
…r paths deterministically

The pending-writer and election tests raced heavily loaded CI runners:
Windows exhausted a 5s acquire timeout and macOS missed the 1s liveness
window during election, electing the wrong writer. Timeouts are now
sized for worst-case runners while the assertions still never wait for
them on the happy path. The fail-fast regression now proves independence
from the timeout with a 30s budget instead of racing a 0.5s wall clock.
New deterministic elected-writer tests cover the shared-holder wait,
unavailable-holder wait and subscription reuse paths that timing
previously left uncovered on macOS.
…nistically

Both paths race the subscribers==1 fast path in the integration tests,
so their coverage depended on scheduler timing and failed the 100% gate
on quiet runners.
RedisLock.pubsub is assigned before SUBSCRIBE reaches the server, so
waiting for it proved nothing about election membership. On stalled
runners the second writer could subscribe, observe a complete holder
set without the first writer and legitimately elect itself. Both
contention tests now wait for the actual subscriber count, making the
election outcome deterministic.
…ners

Windows CI needs multiple seconds per fakeredis pubsub round trip, so
the contention tests can legitimately exceed the global 20s pytest
timeout while behaving correctly. The generous acquire budgets from the
previous commit only work if the surrounding test is allowed to wait
for them.
An unelected pending writer backs off by dropping its subscription
whenever its holder sample is incomplete, so it is legitimately
invisible to new readers until its first complete sample elects it.
On slow runners that window outlived the previous synchronization,
letting a late shared reader acquire and, in the election test, the
wrong writer win. Both tests now wait for the election state that the
reader-gating and ordering guarantees are actually defined by.
@wolph
wolph marked this pull request as ready for review July 25, 2026 14:51
Copilot AI review requested due to automatic review settings July 25, 2026 14:51
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@wolph
wolph merged commit e3a52b3 into develop Jul 25, 2026
27 of 29 checks passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Shared locks when using RedisLock?

2 participants