Skip to content

Improve integer read performance - #19

Merged
landabaso merged 3 commits into
bitcoinjs:masterfrom
landabaso:perf/direct-uint8array-reads
Sep 7, 2026
Merged

Improve integer read performance#19
landabaso merged 3 commits into
bitcoinjs:masterfrom
landabaso:perf/direct-uint8array-reads

Conversation

@landabaso

Copy link
Copy Markdown
Member

Summary

  • Improve integer reads by decoding directly from Buffer and Uint8Array inputs.
  • Avoid full-input copies and temporary Buffer allocations.
  • Preserve endianness, signed values and offset validation.
  • Limit ambient types to Node.js and Jest.

Why

The Node.js entry point called Buffer.from(buffer) before every integer read. Buffer.from(Uint8Array) copies the complete input even when a reader needs only 1, 2, 4 or 8 bytes. The cost of reading a fixed-width integer therefore grew with the size of the input. Each read also created a temporary allocation.

Integer readers now access only the required bytes. Their work is constant for each integer width and they do not allocate a temporary Buffer.

Implementation

The 16-bit and 32-bit readers use the same positional arithmetic approach as Node.js Buffer. Multiplication preserves the full unsigned 32-bit range without relying on signed bitwise coercion.

The 64-bit readers first compose exact lo and hi 32-bit numbers. They then convert only those two values to BigInt. This avoids converting and shifting each byte as a separate BigInt operation.

readInt8 and readInt16 use an explicit two's complement conversion. This is clearer than the branchless sign-extension expression used by Node.js Buffer. The wider signed readers use a signed high part, following the same approach as Node.js Buffer.

Direct Uint8Array indexing does not validate offsets like Buffer.read* does. A shared check rejects invalid and out-of-range offsets before access. The browser entry point is unchanged.

Performance

The main improvement comes from removing a full-array copy from each read. A local microbenchmark illustrates how that cost changes with input size. It compares readUInt32(bytes, offset, "LE") with the previous Buffer.from(bytes).readUInt32LE(offset) path.

Measurements were collected on arm64 macOS with Node.js v20.19.4 and the built ESM entry point. Both paths were warmed up, iteration counts were calibrated per path and each result is the median of seven timed runs.

Input size Previous copy Direct read Speedup
32 B 70.0 ns 20.7 ns 3.4x
256 B 82.2 ns 21.4 ns 3.8x
1 KiB 115.1 ns 21.5 ns 5.4x
512 KiB 23,951.3 ns 21.0 ns 1,141.0x

The smaller inputs are closer to hashes, scripts and transaction-sized data seen in common Bitcoin operations. The 512 KiB input is included only as a stress case to show that the previous cost grows with the full input while the direct read stays nearly constant.

The decoding formulas were also compared with small local checks:

  • Arithmetic and bitwise forms were comparable for 16-bit and 32-bit reads.
  • Two 32-bit halves were 2.26x faster for readUInt64 and 2.57x faster for readInt64 than shifting eight BigInt byte values.
  • Explicit two's complement conversion for 8-bit and 16-bit signed values was at least as fast as the branchless Node.js Buffer form and was easier to read.

These local measurements are directional evidence, not a performance guarantee or a maintained benchmark. The new readers remove the O(n) full-input copy and access only a fixed number of bytes, so their work no longer scales with input size. The decoding logic closely follows Node.js Buffer, but operates directly on the original input. This is expected to improve performance because the previous path performed the same fixed-width decoding after copying the full input. The benefit grows and becomes more visible as the input size increases. A maintained benchmark suite is outside the scope of this PR.

TypeScript Build

When 0.0.9 was published, clean installs resolved type definitions that worked with TypeScript 4.4.4. Without a lockfile, Jest can now resolve newer definitions that require TypeScript 5.1, which the existing compiler cannot parse. The compiler now loads only the Node.js and Jest ambient types this project needs, excluding unrelated definitions from the build.

Verification

  • 65 unit tests pass.
  • Statement, branch, function and line coverage remain at 100%.
  • ESM and CommonJS builds match Node.js Buffer across 2,000 deterministic byte vectors each.
  • Lint and clean TypeScript builds pass.
  • npm pack --dry-run includes the expected eight package files.

Comment thread ts_src/index.ts

@junderw junderw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@landabaso
landabaso merged commit f6bc048 into bitcoinjs:master Sep 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants