fuse: fence cached reads against NOTIFY invalidate with a percpu gate#189
Merged
hbirth merged 1 commit intoJul 21, 2026
Conversation
hazhou-ddn
reviewed
Jul 20, 2026
hazhou-ddn
reviewed
Jul 20, 2026
A FUSE_NOTIFY_INVAL_INODE is a coherency event: once the server signals a remote modify, no local read may return a page it has superseded. The invalidate ran unserialized against cache-serving reads, so a buffered read could hand back a stale folio it still held a reference to. Convert the per-inode wb_inval_rwsem to a percpu_rw_semaphore and take its read side around the cache-serving read as well as the existing buffered write. The read side is per-CPU, so it scales on a shared file; the NOTIFY takes the write side blocking, giving the invalidate priority -- it parks new readers, drains in-flight ones, then drops the cache. Every gated invalidate now runs under the write side, not just the storm-latching one. The gate is allocated only for writeback+dlm regular files and is NULL elsewhere (best-effort invalidate, as before). The blocking write side may run on the notify-delivering server thread, so it is safe only under a server that services request replies on other threads; redfs' dlm server provides that contract. Signed-off-by: Horst Birthelmer <hbirthelmer@ddn.com>
hbirth
force-pushed
the
redfs-ubuntu-hwe-6.17.0-16.16-24.04.1
branch
from
July 21, 2026 08:38
3456a3c to
d6180a2
Compare
hbirth
merged commit Jul 21, 2026
d6180a2
into
DDNStorage:redfs-ubuntu-hwe-6.17.0-16.16-24.04.1
2 checks passed
hazhou-ddn
reviewed
Jul 22, 2026
| return fuse_direct_read_iter(iocb, to); | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
As my earlier comment, why still miss a check on DLM lock here? The invalidation could be just done before we acquire the wb_sem, then the DLM lock is revoked, then continue calling generic_file_read_iter is dangerous as it is running without DLM lock protection.
hazhou-ddn
reviewed
Jul 22, 2026
| wb_guard = !!wb_sem; | ||
| if (wb_guard) { | ||
| down_read(&fi->wb_inval_rwsem); | ||
| percpu_down_read(wb_sem); |
There was a problem hiding this comment.
Again, we're still missing check the DLM lock state after acquiring the wb_sem. This is dangerous as it could be revoked by an just done invalidation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A FUSE_NOTIFY_INVAL_INODE is a coherency event: once the server signals a remote modify, no local read may return a page it has superseded. The invalidate ran unserialized against cache-serving reads, so a buffered read could hand back a stale folio it still held a reference to.
Convert the per-inode wb_inval_rwsem to a percpu_rw_semaphore and take its read side around the cache-serving read as well as the existing buffered write. The read side is per-CPU, so it scales on a shared file; the NOTIFY takes the write side blocking, giving the invalidate priority -- it parks new readers, drains in-flight ones, then drops the cache. Every gated invalidate now runs under the write side, not just the storm-latching one.
The gate is allocated only for writeback+dlm regular files and is NULL elsewhere (best-effort invalidate, as before). The blocking write side may run on the notify-delivering server thread, so it is safe only under a server that services request replies on other threads; redfs' dlm server provides that contract.