Skip to content

Up to 3.4× faster statement preparation via in-place authorizer region accumulation#1873

Open
guidedways wants to merge 1 commit into
groue:developmentfrom
BeehiveInnovations:perf/authorizer-region-accumulation
Open

Up to 3.4× faster statement preparation via in-place authorizer region accumulation#1873
guidedways wants to merge 1 commit into
groue:developmentfrom
BeehiveInnovations:perf/authorizer-region-accumulation

Conversation

@guidedways

@guidedways guidedways commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

SQLite invokes the statement authorizer once for every column read while compiling a statement. StatementAuthorizer handled each SQLITE_READ event by building a one-column DatabaseRegion and merging it with DatabaseRegion.formUnion, which rebuilds the table dictionary and column sets on every call. Compiling a statement that reads C columns therefore performs O(C²) region work, with temporary allocations on every callback. In a production app querying an 80-column table, this accounted for 1.73 seconds of main-thread time in a three-minute profile, from only a couple hundred statement preparations.

The authorizer now records each read through an internal DatabaseRegion operation that mutates one dictionary slot in place, making each callback O(1) amortized. Whole-table reads still promote a table region to all columns, a full-database region still absorbs every read, and the general union/formUnion operations are untouched. No public API or observable behavior changes.

Performance

Median of five release-build samples, each compiling 1,000 fresh statements after 100 warm-ups (Database.makeStatement(sql:), in-memory database):

Statement Before After Time reduction Speedup
SELECT * from a 120-column table 173.878 µs/prepare 50.790 µs/prepare −70.8% 3.42×
Three-table, 120-column join 190.086 µs/prepare 68.897 µs/prepare −63.8% 2.76×
SELECT id narrow control 2.124 µs/prepare 1.129 µs/prepare −46.8% 1.88×

Small statements improve as well, since the per-callback region construction and merge are gone regardless of statement width.

MacBook Pro (Apple M3 Max), macOS 26.5.1, Swift 6.3.2, system SQLite 3.51.0, default SPM configuration. Reproduce from the repository root:

swift test -c release --filter StatementPreparationPerformanceTests

The benchmark skips itself in debug builds, so plain swift test run time is unaffected.

As a broader check, five alternated release runs of the identical pre-existing test suite completed with a 1.6% lower median on this branch (76.2 s → 75.0 s). The effect is small, as expected: the suite's statements are mostly narrow and statement preparation is a minor share of its total time, so it exercises the affected path far less than the benchmark above.

Correctness

The new operation produces regions equal (==) to the formUnion fold it replaces. DatabaseRegionTests covers:

Case Test
Single column into an empty region testRegionFormUnionTableColumn
Whole-table promotion, both orders, empty column name testRegionFormUnionTableColumnWholeTable
Repeated columns testRegionFormUnionTableColumnRepeatedColumn
Multiple tables testRegionFormUnionTableColumnMultipleTables
Case-insensitive table identity testRegionFormUnionTableColumnCaseInsensitiveTable
Row IDs and all-rows promotion testRegionFormUnionTableColumnAndRowIds
Full-database absorption testRegionFormUnionTableColumnFullDatabase
Event-sequence folds testRegionFormUnionTableColumnSequenceEquivalence
End-to-end statement regions, including rowid testSelectStatement, testSelectStatement_rowid

No existing expectation changed. On this branch:

  • Full SwiftPM suite: 2,886 tests, 31 skipped, 0 failures.
  • make smokeTest: 0 failures across system SQLite (max and min targets), SQLCipher 3, SQLCipher 4 encrypted, custom SQLite, and SPM.

Pull Request Checklist

  • CONTRIBUTING: You have read https://github.com/groue/GRDB.swift/blob/master/CONTRIBUTING.md
  • BRANCH: This pull request is submitted against the development branch.
  • DOCUMENTATION: Inline documentation has been updated.
  • DOCUMENTATION: README.md or another dedicated guide has been updated. (Not applicable: internal change, no public API.)
  • TESTS: Changes are tested.
  • TESTS: The make smokeTest terminal command runs without failure.

Accumulate individual SQLite read events in a single database-region slot. Cover equivalence with general region unions and add a reproducible statement-preparation benchmark.
@guidedways
guidedways marked this pull request as ready for review July 20, 2026 05:47
@groue

groue commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Thank you @guidedways! I'll review this pull request shortly!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants