Fetch the user's profile from the whole swarm at once by mpretty-cyro · Pull Request #161 · session-foundation/libsession-util

mpretty-cyro · 2026-09-15T06:10:09Z

Why

An ordinary poll asks one swarm member, once every poll interval, and the storage server does not promise a config has reached every member of a swarm. Measured against mainnet through a client bridge: an account created minutes earlier, restored on a second device, took 66 seconds for its display name to come back — several members sampled one at a time before one of them had it.

Both mobile clients already work around exactly this and say so in their own comments. Session Desktop does too. So the ordinary poll was never any client's design for this moment; it was just what a bridge without the workaround falls back to.

The case it matters for is a restore: nothing local can answer "what is this account called", and there is a person watching a progress indicator with a fifteen-second budget.

What

Core::fetch_user_profile(done) asks every swarm member concurrently for the UserProfile namespace, and is deliberately asymmetric about what it concludes:

the first answer carrying a config ends the fetch;
an empty answer never does, however many members give it — having the config is not a majority property, and a member that has not caught up is the condition being routed around rather than evidence against the one that has;
only every member having answered concludes there is nothing to find.

No agreement is required of the answer that wins. A config is merged rather than assigned, so a stale one taken from a member behind its swarm is corrected by the next ordinary poll; requiring a second member to say the same thing buys protection against something that already repairs itself, and costs a round trip in front of somebody waiting.

Every member is asked the same question — no last_hash — which is both what makes the answers comparable and what stops two members agreeing on a subset while a third, further along, is still answering.

Measured

Same account, same method, against mainnet:

	time for the display name
ordinary poll	66s
fan-out, requiring a second member to agree	17s
fan-out, first member that has it	5s

Also confirmed end to end on an iOS simulator through a client bridge: the restore lands on the conversation list rather than falling through to "pick a new display name".

Refactor that came with it

Two pieces of the poll path are now shared rather than duplicated:

decode_retrieved takes no node and touches no database, deliberately. The retrieve cursor is kept per (namespace, node), and a decoder that knew about nodes is the shape in which one member's cursor gets written from another's answer — silently, because each cursor is individually plausible, and showing up much later as a namespace that re-fetches for ever or one that skips messages.
Core::_record_swarm_cursor takes the node as a parameter for the same reason.

Draft because

The consuming client change is not up yet, and this wants a review from someone who knows the swarm layer better than the caller does — in particular whether the fan-out should be bounded to a subset of the swarm rather than all of it on a metered connection.

An ordinary poll asks one swarm member, once every poll interval, and the storage server does not promise a config has reached every member of a swarm. Measured against mainnet through a client bridge: an account created minutes earlier, restored on a second device, took 66 seconds for its display name to come back -- several members sampled one at a time before one of them had it. Both mobile clients work around exactly this and say so in their own comments. `fetch_user_profile` asks every member concurrently. It is deliberately asymmetric about what it concludes from what they say: having the config is not a majority property, so one member holding it is the whole answer and the others not having it yet is the condition being routed around rather than evidence against it. An answer carrying the config seconded by a second member ends the fetch early; an empty answer never does, however many members give it; and when everyone has answered, any member that had it still wins. Concluding there is no config is the expensive direction and waits for the whole swarm. Every member is asked the same question -- no `last_hash` -- which is both what makes the answers comparable and what stops two members agreeing on a subset while a third, further along, is still answering. Two pieces of the poll path are now shared rather than duplicated. `decode_retrieved` takes no node and touches no database, deliberately: the retrieve cursor is kept per (namespace, node), and a decoder that knew about nodes is the shape in which one member's cursor gets written from another's answer -- silently, because each cursor is individually plausible. `_record_swarm_cursor` takes the node as a parameter for the same reason. Measured after: 17 seconds, against 66 before.

Waiting for a second member to return the same answer bought protection against something that repairs itself. A config is merged rather than assigned, so a stale one taken from a member behind its swarm is corrected by the next ordinary poll; the seconding only delayed the answer, and it delayed it in front of somebody watching a progress indicator. Measured on mainnet, same account, same method: 66s through the ordinary poll, 17s with seconding, 5s without it. The cost of the confirmation was most of the remaining wait. The asymmetry that matters stays: an empty answer still never settles the fetch, however many members give it, and only every member having answered concludes there is nothing to find.

`NamespaceIndentation: Inner` indents an anonymous namespace nested inside `session`, and the decoder moved into that namespace without being re-indented. Whitespace and comment rewrapping only: `./utils/format.sh`'s own output, which is what the lint stage runs.

The swarm lookup and every member's answer were marshalled onto `_loop`. Core now has its own `_jq`, and work queued there is cancelled when Core goes away, so an answer that lands during teardown is dropped rather than run against components that are already being destroyed. That matters more with the next change, where late answers do real work.

The fetch settled on the first member to answer with a config: it merged that one, called `done`, and dropped every answer after it. Two problems with that. A member can be behind its swarm, so the first config to arrive is not necessarily the newest -- and a restore that lands on a stale member reports a stale profile to the user watching it. And the answers dropped afterwards were real data from our own swarm, which the next ordinary poll then fetched again. So absorbing an answer and concluding the fetch are now separate. Every answer that carries a config is merged and has its member's cursor recorded, whenever it arrives, including after `done` has been called. The first such answer opens a 500ms window instead of ending the fetch, and the fetch concludes when every member has answered or the window closes, whichever is first -- `done` exactly once. Configs merge rather than replace, so each extra answer can only bring the result forward. An empty answer still never ends the fetch on its own, and only a config opens the window: an answer with nothing in it is the condition the fan-out exists to route around. Concluding no longer takes an answer, so the pointer passed to it -- and the struct that carried a decoded answer to it -- are gone. The decode moves out of the merge's `try`, so a failure while merging is not logged as a failure to parse. Four tests, where there were none: a stale first config overtaken inside the window, a late config merged after the fetch reported with its cursor written against the member that gave it, the window closing early once everyone has answered, and all-empty concluding with nothing found. Run against the previous behaviour, the first three fail in seven places; the fourth passes on both, since that behaviour is unchanged. `MockNetwork` can now hand back a whole swarm.

The body builder checked `retrieve_requires_auth` before signing. For the profile namespace that is always true -- it is owner-writable -- and signing is allowed on any namespace, so the branch had no case in which skipping the signature would have been right. It is now an assertion of the invariant, and the signing is unconditional. A test holds the request's shape: every member is asked for the profile namespace, signed, and without a cursor.

mpretty-cyro marked this pull request as ready for review September 15, 2026 06:30

mpretty-cyro self-assigned this Sep 15, 2026