-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
shard hardlink directory #15590
Copy link
Copy link
Open
Labels
featureFeature request or proposalFeature request or proposal
Description
Is your feature request related to a problem?
I notice a severe pause between deleting storepaths and deleting hardlinks.
(Also my own tooling for hardlinking stats suffers from this pause as well)
Proposed solution
I assume this pause is caused by retrieving the list of all files in /nix/store/.links, which currently contains ~2M files, though is theoretically unbound, before being able to stat them to check number of hardlinks.
Sharding the folder could significantly speed up the initial retrieval and would also enable us to build "async" mode where discovery can happen per shard and feeds a queue, that gets started to work down immediately, while the shards are still scanned for more files.
Alternative solutions
I do not really have alternative solutions.
Additional context
Checklist
- checked latest Nix manual (source)
- checked open feature issues and pull requests for possible duplicates
Add 👍 to issues you find important.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
featureFeature request or proposalFeature request or proposal