Skip to content

Handle keystore and VACUUM failures on the database-open path instead of crash-looping (#2213) - #2219

Open
mpretty-cyro wants to merge 4 commits into
devfrom
fix/keystore-unseal-crash-loop
Open

mpretty-cyro wants to merge 4 commits into
devfrom
fix/keystore-unseal-crash-loop

Conversation

@mpretty-cyro

@mpretty-cyro mpretty-cyro commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

Two independent startup failures, both of which end with the app dying on every launch and neither of
which could reach the database error screen the app already ships. Fixes #2213.

No new strings.

1. A keystore failure could not reach the error screen

KeyStoreHelper.unseal rethrows every crypto failure as AssertionError. The database secret was
only dereferenced from the openHelper lazy, outside any handler, so once the sealed secret could no
longer be decrypted the process died on every launch with no message and no way out.

Two things had to change for the existing screen to be reachable:

  • Resolve the secret inside the migration flow, and catch AssertionError alongside Exception
    so the failure becomes MigrationState.Error. migrateCipherSettings returns early for anyone
    already past the KDF migration and so never touched the secret at all — guarding only openHelper
    would have caught nothing for the users actually affected. AssertionError is caught because
    KeyStoreHelper throws it on purpose; OutOfMemoryError, StackOverflowError and LinkageError
    are deliberately left to kill the process.
  • Hold database-backed startup work until the migration reports Completed. Getting the state
    right was not enough on its own: the first component handed a failure took the process down before
    the screen could appear. The pollers start their own work from their constructors, so the
    components are now resolved through a Provider asked for only once the migration completes, which
    defers construction as well as the callback. And not all database work goes through the startup
    sequence — BlindMappingRepository reaches it from a flow started on login state — so the callers
    are not enumerable from one place. For that reason openHelper waits rather than throwing:
    parking a caller that cannot proceed beats handing it a failure it does not expect, and a retry
    that succeeds releases everyone waiting.

2. "Clear Device and Restore" could not restore

It preserves the account by re-applying the in-memory login state, so when that state could not
be unsealed either it silently did what "Clear Device and Restart" does — after the user accepted a
warning saying their account would be restored. Now hidden in that case, which is the same
precondition the implementation itself checks.

3. The keystore error code is now logged

android.security.KeyStoreException (API 33+) carries getNumericErrorCode() and
isTransientFailure(). That is the only thing distinguishing a transient keystore fault — where the
data is intact and a retry may well succeed — from a key that can no longer decrypt what it sealed.
Nothing else in the crash carries the distinction, and it was being discarded.

4. Separate defect: the weekly VACUUM could prevent startup

SQLCipherOpenHelper's postKey hook runs on the database-open path, and the VACUUM there was
unwrapped — so any failure reached the caller as a database that would not open rather than as failed
maintenance. It also recorded only success, so a VACUUM that threw was retried on every open from
then on: one full disk became a launch that never worked again.

It now records the attempt before making it, skips when there is less than twice the database file's
size free (roughly what rebuilding it needs), and catches what is left. Note this path is not
covered by the rest of this PR — postKey fires on the first lazy open, inside whichever caller
touches the database first, not inside the guarded migration block.

Verification

On an emulator, by corrupting the sealed secret's base64 in place — IV and keystore key untouched, so
a genuine GCM tag failure.

Scenario Result
Healthy secret normal startup, stable
Database secret corrupted process survives, error screen stays up, 0 uncaught exceptions, 0 ANRs
Database secret and login state corrupted does not reach this screen at all — see below
Retry tapped on a permanent failure re-attempts the unseal, stays up, no crash
Both restored, relaunched normal startup, database keyed, deferred components start as expected

Keystore failure: code=10, transient=false, systemError=false logged in each failing case —
ERROR_KEYMINT_FAILURE, correctly classified non-transient for a tag failure.

Pre-existing routing fault, unmasked by this change — needs fixing before this ships

This is the case that matters most, because if the keystore key itself is gone then everything
sealed under it fails — the database secret and the login state together. Measured sequence:

12:20:00.562  LoginStateRepository: Unable to unseal login state    -> login state becomes null
12:20:00.634  routeApplicationState() - STATE_WELCOME_SCREEN        -> routes to onboarding
12:20:00.653  DatabaseMigrationManager: Keystore failure            -> migration reaches Error, 19ms late

ScreenLockActionBarActivity.getApplicationState() reads migrationState.value while it is still
Idle, falls through, finds no login state, and routes to the welcome screen. The migration reaches
Error 19ms later, too late to affect the decision.

So the user is shown a fresh-install welcome screen with their database still intact on disk
(verified: session.db present, 1,130,496 bytes). That invites starting over, which destroys data
that a retry might have recovered — worse in that one respect than the crash loop it replaces, which
at least did not invite anything.

The fall-through is not introduced here. ScreenLockActionBarActivity's shouldShowUI covers
only Migrating and Error (:246-247), and the initial state is Idle, so the gap is already on
dev — this branch does not touch that file. It does not bite today only because the
AssertionError kills the process before routing completes; catching the error unmasks it.

The second arm of the same race is also dev's. openHelper waits
first { it == MigrationState.Completed } with no timeout, and Error is terminal — so once the
state settles there, that wait can never be satisfied and runBlocking holds the calling thread for
the life of the process. Both the terminal Error state and the unterminating wait are already on
dev; what changes here is only which faults reach them. On dev the keystore fault leaves
migration Completed, because the secret is never touched inside the try, and the process dies at
the SQLCipherOpenHelper(...) line instead. An ordinary Exception out of migrateCipherSettings
reaches Error on dev today and lands in the same wait.

Calling that "parking" was too comfortable a description on my part. The comment justifies blocking
until the migration resolves; Error is resolved, and the code cannot express it. So it is a
permanent thread leak rather than a wait, and openHelper is the entry point to the whole database
layer — bounded only by the IO dispatcher's thread cap, after which unrelated coroutine work starves
too. The foreground case is the visible one; the background case is the worse one to diagnose. A leak with
a ceiling is not a smaller problem than an unbounded one — its symptom arrives all at once and
somewhere else, as unrelated coroutine work stalling, with nothing in it pointing back at the
database layer. An ANR at least names its own thread.

So the routing change is necessary but not sufficient:

Race outcome Condition Closed by the routing change?
routes to WELCOME login state also unsealable yes
routes to NORMAL login state still readable only the foreground path

Any background caller reaching openHelper after an Error still blocks forever, routing or no
routing. Fully closing it needs Error as a terminating condition on the wait — first { it is Completed || it is Error } — plus a decision about what callers then get. Worth noting for whoever
takes that on: throwing there was tried in this branch's history and it killed the process, but that
was measured before the startup gating existed. With the gating and the routing change in place far
fewer callers arrive, so throwing may now be survivable — that is a thing to test, not an assertion.

Neither change is in this PR. Both are wider than it and neither is mine to decide.

Gaps a reviewer should know about

  • The VACUUM change is only partly verified. The block is entered when due and the timestamp now
    advances on entry, and startup is unaffected across repeated runs — but the skip and catch branches
    were never observed executing. A full emulator disk reclaims cache at exactly the boundary that
    would trigger them. Those two branches are reasoned, not tested.
  • On a permanent failure, whatever reaches openHelper parks for the life of the process, and
    runBlocking holds a thread. The gating keeps that to a handful. This is what the original code
    intended, but the Error state was previously unreachable, so it never actually happened.
  • Only exercised on one API level, and only the permanent (tag-failure) case. A genuinely transient
    keystore fault — where the retry would succeed and release the parked callers — was not reproduced.
  • The cause of the underlying key failures is still unidentified and is not addressed here. This turns
    an unexplained crash loop into a handled failure that can produce a log.

…aled (#2213)

KeyStoreHelper.unseal rethrows every crypto failure as AssertionError, and the
database secret was only ever dereferenced from the openHelper lazy, outside any
handler. The process died on every launch instead of reaching the error screen that
already exists. Resolve the secret inside the migration flow and catch Throwable so
the failure becomes MigrationState.Error, and end openHelper's wait on Error as well
as Completed — waiting for Completed alone would have hung every database caller once
that state was reachable.

"Clear Device and Restore" is now hidden when there is no login state, because it
restores from the in-memory state and would otherwise silently behave as "clear and
restart" after the user accepted a warning promising an account recovery.

Keystore failures now log the KeyStoreException error code, which is the only thing
distinguishing a transient fault, where the data is intact and a retry may succeed,
from a key that can no longer decrypt what it sealed.
…cret (#2213)

Routing the keystore failure into MigrationState.Error was not enough on its own: the
database is reached during startup by components that construct themselves eagerly and
by flows that start on login state, and the first of them to be handed an exception
took the process down before the migration screen could appear.

Split the startup components so everything that reaches the database is resolved
through a Provider that is only asked for once the migration reports Completed, which
defers construction as well as the callback — the pollers start their own work from
their constructors, so deferring onPostAppStarted alone would have started them
anyway.

openHelper goes back to waiting rather than throwing. Parking a caller that cannot
proceed is better than handing it a failure it does not expect, and the callers that
reach the database outside the startup sequence are not enumerable from one place. A
retry that succeeds reaches Completed and releases everyone waiting.

Verified on an emulator by corrupting the sealed secret in place: the app now survives
and shows the database error screen with Retry, Export Logs and Clear Device and
Restart, logs "Keystore failure: code=10, transient=false, systemError=false", drops
Clear Device and Restore when the login state is unsealable too, and starts normally
with no keystore failures once the secret is sound.
catch (Throwable) also caught OutOfMemoryError, StackOverflowError and LinkageError,
turning a process that is already lost into a database error screen offering Retry and
Clear Data. Catch Exception and AssertionError instead: AssertionError is the one Error
on this path thrown deliberately, as KeyStoreHelper's way of reporting a crypto
failure, which is what makes singling it out defensible rather than arbitrary.
The VACUUM in postKey runs on the database-open path, unwrapped, so any failure
reached the caller as a database that would not open rather than as failed
maintenance. It also recorded only success, so a VACUUM that threw was retried on
every open from then on — one full disk became a launch that never worked again.

Record the attempt before making it, skip it when there is less than twice the
database file's size free since that is roughly what rebuilding it needs, and catch
what is left. Note the timestamp is persisted with apply(), so on a genuinely full
disk the write can be dropped and the attempt repeats next launch; the catch, not the
ordering, is what stops that being fatal.

Verified only in part: the block is entered when due and the timestamp now advances on
entry, and startup is unaffected across repeated runs. The skip and catch branches were
not observed executing — a full emulator disk reclaims cache at exactly the boundary
that would trigger them — so those two paths are reasoned, not tested.
@mpretty-cyro mpretty-cyro changed the title Handle a keystore failure on startup instead of crash-looping (#2213) Handle keystore and VACUUM failures on the database-open path instead of crash-looping (#2213) Sep 22, 2026
@mpretty-cyro
mpretty-cyro marked this pull request as ready for review September 22, 2026 03:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Session app every time that i open the app it crashes

1 participant