Skip to content

shard hardlink directory #15590

@NobbZ

Description

@NobbZ

Is your feature request related to a problem?

I notice a severe pause between deleting storepaths and deleting hardlinks.

(Also my own tooling for hardlinking stats suffers from this pause as well)

Proposed solution

I assume this pause is caused by retrieving the list of all files in /nix/store/.links, which currently contains ~2M files, though is theoretically unbound, before being able to stat them to check number of hardlinks.

Sharding the folder could significantly speed up the initial retrieval and would also enable us to build "async" mode where discovery can happen per shard and feeds a queue, that gets started to work down immediately, while the shards are still scanned for more files.

Alternative solutions

I do not really have alternative solutions.

Additional context

Checklist


Add 👍 to issues you find important.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureFeature request or proposal

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions