Handle keystore and VACUUM failures on the database-open path instead of crash-looping (#2213) - #2219
Open
mpretty-cyro wants to merge 4 commits into
Open
mpretty-cyro wants to merge 4 commits into
mpretty-cyro wants to merge 4 commits into
Conversation
…aled (#2213) KeyStoreHelper.unseal rethrows every crypto failure as AssertionError, and the database secret was only ever dereferenced from the openHelper lazy, outside any handler. The process died on every launch instead of reaching the error screen that already exists. Resolve the secret inside the migration flow and catch Throwable so the failure becomes MigrationState.Error, and end openHelper's wait on Error as well as Completed — waiting for Completed alone would have hung every database caller once that state was reachable. "Clear Device and Restore" is now hidden when there is no login state, because it restores from the in-memory state and would otherwise silently behave as "clear and restart" after the user accepted a warning promising an account recovery. Keystore failures now log the KeyStoreException error code, which is the only thing distinguishing a transient fault, where the data is intact and a retry may succeed, from a key that can no longer decrypt what it sealed.
…cret (#2213) Routing the keystore failure into MigrationState.Error was not enough on its own: the database is reached during startup by components that construct themselves eagerly and by flows that start on login state, and the first of them to be handed an exception took the process down before the migration screen could appear. Split the startup components so everything that reaches the database is resolved through a Provider that is only asked for once the migration reports Completed, which defers construction as well as the callback — the pollers start their own work from their constructors, so deferring onPostAppStarted alone would have started them anyway. openHelper goes back to waiting rather than throwing. Parking a caller that cannot proceed is better than handing it a failure it does not expect, and the callers that reach the database outside the startup sequence are not enumerable from one place. A retry that succeeds reaches Completed and releases everyone waiting. Verified on an emulator by corrupting the sealed secret in place: the app now survives and shows the database error screen with Retry, Export Logs and Clear Device and Restart, logs "Keystore failure: code=10, transient=false, systemError=false", drops Clear Device and Restore when the login state is unsealable too, and starts normally with no keystore failures once the secret is sound.
catch (Throwable) also caught OutOfMemoryError, StackOverflowError and LinkageError, turning a process that is already lost into a database error screen offering Retry and Clear Data. Catch Exception and AssertionError instead: AssertionError is the one Error on this path thrown deliberately, as KeyStoreHelper's way of reporting a crypto failure, which is what makes singling it out defensible rather than arbitrary.
The VACUUM in postKey runs on the database-open path, unwrapped, so any failure reached the caller as a database that would not open rather than as failed maintenance. It also recorded only success, so a VACUUM that threw was retried on every open from then on — one full disk became a launch that never worked again. Record the attempt before making it, skip it when there is less than twice the database file's size free since that is roughly what rebuilding it needs, and catch what is left. Note the timestamp is persisted with apply(), so on a genuinely full disk the write can be dropped and the attempt repeats next launch; the catch, not the ordering, is what stops that being fatal. Verified only in part: the block is entered when due and the timestamp now advances on entry, and startup is unaffected across repeated runs. The skip and catch branches were not observed executing — a full emulator disk reclaims cache at exactly the boundary that would trigger them — so those two paths are reasoned, not tested.
mpretty-cyro
marked this pull request as ready for review
September 22, 2026 03:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent startup failures, both of which end with the app dying on every launch and neither of
which could reach the database error screen the app already ships. Fixes #2213.
No new strings.
1. A keystore failure could not reach the error screen
KeyStoreHelper.unsealrethrows every crypto failure asAssertionError. The database secret wasonly dereferenced from the
openHelperlazy, outside any handler, so once the sealed secret could nolonger be decrypted the process died on every launch with no message and no way out.
Two things had to change for the existing screen to be reachable:
AssertionErroralongsideExceptionso the failure becomes
MigrationState.Error.migrateCipherSettingsreturns early for anyonealready past the KDF migration and so never touched the secret at all — guarding only
openHelperwould have caught nothing for the users actually affected.
AssertionErroris caught becauseKeyStoreHelperthrows it on purpose;OutOfMemoryError,StackOverflowErrorandLinkageErrorare deliberately left to kill the process.
Completed. Getting the stateright was not enough on its own: the first component handed a failure took the process down before
the screen could appear. The pollers start their own work from their constructors, so the
components are now resolved through a
Providerasked for only once the migration completes, whichdefers construction as well as the callback. And not all database work goes through the startup
sequence —
BlindMappingRepositoryreaches it from a flow started on login state — so the callersare not enumerable from one place. For that reason
openHelperwaits rather than throwing:parking a caller that cannot proceed beats handing it a failure it does not expect, and a retry
that succeeds releases everyone waiting.
2. "Clear Device and Restore" could not restore
It preserves the account by re-applying the in-memory login state, so when that state could not
be unsealed either it silently did what "Clear Device and Restart" does — after the user accepted a
warning saying their account would be restored. Now hidden in that case, which is the same
precondition the implementation itself checks.
3. The keystore error code is now logged
android.security.KeyStoreException(API 33+) carriesgetNumericErrorCode()andisTransientFailure(). That is the only thing distinguishing a transient keystore fault — where thedata is intact and a retry may well succeed — from a key that can no longer decrypt what it sealed.
Nothing else in the crash carries the distinction, and it was being discarded.
4. Separate defect: the weekly VACUUM could prevent startup
SQLCipherOpenHelper'spostKeyhook runs on the database-open path, and the VACUUM there wasunwrapped — so any failure reached the caller as a database that would not open rather than as failed
maintenance. It also recorded only success, so a VACUUM that threw was retried on every open from
then on: one full disk became a launch that never worked again.
It now records the attempt before making it, skips when there is less than twice the database file's
size free (roughly what rebuilding it needs), and catches what is left. Note this path is not
covered by the rest of this PR —
postKeyfires on the first lazy open, inside whichever callertouches the database first, not inside the guarded migration block.
Verification
On an emulator, by corrupting the sealed secret's base64 in place — IV and keystore key untouched, so
a genuine GCM tag failure.
Keystore failure: code=10, transient=false, systemError=falselogged in each failing case —ERROR_KEYMINT_FAILURE, correctly classified non-transient for a tag failure.Pre-existing routing fault, unmasked by this change — needs fixing before this ships
This is the case that matters most, because if the keystore key itself is gone then everything
sealed under it fails — the database secret and the login state together. Measured sequence:
ScreenLockActionBarActivity.getApplicationState()readsmigrationState.valuewhile it is stillIdle, falls through, finds no login state, and routes to the welcome screen. The migration reachesError19ms later, too late to affect the decision.So the user is shown a fresh-install welcome screen with their database still intact on disk
(verified:
session.dbpresent, 1,130,496 bytes). That invites starting over, which destroys datathat a retry might have recovered — worse in that one respect than the crash loop it replaces, which
at least did not invite anything.
The fall-through is not introduced here.
ScreenLockActionBarActivity'sshouldShowUIcoversonly
MigratingandError(:246-247), and the initial state isIdle, so the gap is already ondev— this branch does not touch that file. It does not bite today only because theAssertionErrorkills the process before routing completes; catching the error unmasks it.The second arm of the same race is also dev's.
openHelperwaitsfirst { it == MigrationState.Completed }with no timeout, andErroris terminal — so once thestate settles there, that wait can never be satisfied and
runBlockingholds the calling thread forthe life of the process. Both the terminal
Errorstate and the unterminating wait are already ondev; what changes here is only which faults reach them. Ondevthe keystore fault leavesmigration
Completed, because the secret is never touched inside the try, and the process dies atthe
SQLCipherOpenHelper(...)line instead. An ordinaryExceptionout ofmigrateCipherSettingsreaches
Errorondevtoday and lands in the same wait.Calling that "parking" was too comfortable a description on my part. The comment justifies blocking
until the migration resolves;
Erroris resolved, and the code cannot express it. So it is apermanent thread leak rather than a wait, and
openHelperis the entry point to the whole databaselayer — bounded only by the IO dispatcher's thread cap, after which unrelated coroutine work starves
too. The foreground case is the visible one; the background case is the worse one to diagnose. A leak with
a ceiling is not a smaller problem than an unbounded one — its symptom arrives all at once and
somewhere else, as unrelated coroutine work stalling, with nothing in it pointing back at the
database layer. An ANR at least names its own thread.
So the routing change is necessary but not sufficient:
WELCOMENORMALAny background caller reaching
openHelperafter anErrorstill blocks forever, routing or norouting. Fully closing it needs
Erroras a terminating condition on the wait —first { it is Completed || it is Error }— plus a decision about what callers then get. Worth noting for whoevertakes that on: throwing there was tried in this branch's history and it killed the process, but that
was measured before the startup gating existed. With the gating and the routing change in place far
fewer callers arrive, so throwing may now be survivable — that is a thing to test, not an assertion.
Neither change is in this PR. Both are wider than it and neither is mine to decide.
Gaps a reviewer should know about
advances on entry, and startup is unaffected across repeated runs — but the skip and catch branches
were never observed executing. A full emulator disk reclaims cache at exactly the boundary that
would trigger them. Those two branches are reasoned, not tested.
openHelperparks for the life of the process, andrunBlockingholds a thread. The gating keeps that to a handful. This is what the original codeintended, but the
Errorstate was previously unreachable, so it never actually happened.keystore fault — where the retry would succeed and release the parked callers — was not reproduced.
an unexplained crash loop into a handled failure that can produce a log.