Skip to content

Optimize binary logger metadata serialization - #14929

Open
AlesProkop wants to merge 1 commit into
dotnet:mainfrom
AlesProkop:alesprokop/binlog-metadata-reference-cache
Open

Optimize binary logger metadata serialization#14929
AlesProkop wants to merge 1 commit into
dotnet:mainfrom
AlesProkop:alesprokop/binlog-metadata-reference-cache

Conversation

@AlesProkop

Copy link
Copy Markdown
Member

Related to #10490

Context

Binary log profiling identified task-item metadata enumeration and hashing in BuildEventArgsWriter.WriteTaskItemList as the dominant serialization cost. Copy-on-write task items frequently share the same immutable metadata dictionary, but the binary logger previously enumerated, unescaped, and hashed that metadata for every item.

Changes Made

  • Added an identity cache from immutable backing metadata dictionaries to previously emitted NameValueList record IDs.
  • Reused cached record IDs for items sharing copy-on-write metadata.
  • Enumerated immutable backing metadata directly on cache misses while preserving existing key ordering and value unescaping.
  • Preserved the existing fallback for TaskItemData, custom ITaskItem implementations, and .NET Framework transparent proxies.
  • Short-circuited empty backing metadata to the existing zero-record encoding.
  • Documented the escaped-value and ordering contract of IMetadataContainer.BackingMetadata.

Testing

  • Built the full repository — 0 warnings, 0 errors.
  • Built and replayed a binary log for src/Samples/Dependency/Dependency.csproj with the bootstrap MSBuild.
  • Added and ran focused tests in BuildEventArgsSerializationTests:
    • TaskParameterSerializationReusesSharedBackingMetadata — verifies reuse for shared Utilities.TaskItem metadata.
    • TaskParameterSerializationCachesEmptyMetadataAndFallsBackForTaskItemData — verifies empty metadata and the TaskItemData fallback.
    • TaskParameterSerializationReusesItemDefinitionMetadata — verifies reuse for shared item-definition metadata.
    • TaskParameterBackingMetadataFastPathPreservesSerializedBytes — verifies byte-for-byte equivalence with the legacy fallback path.
    • Result: 16 passed, 0 failed on net11.0; 16 passed, 0 failed on net472.

Notes

  • The binary log format is unchanged and does not require a format-version increment.
  • Matched OrchardCore warm-build measurements with ProjectImports=None reduced median wall time from 27.20 seconds to 23.20 seconds under MT/server execution.
  • Default out-of-process /m measurements reduced median wall time from 41.44 seconds to 39.79 seconds.
  • A 20-round small-project control showed no fixed-cost regression.

Reuse serialized metadata records for items that share immutable copy-on-write metadata, while preserving fallback and AppDomain behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f5f373de-cc23-46fa-bd6f-9ba8c1645411

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The optimization is localized, preserves the existing on-wire encoding, and is covered by targeted tests including byte-for-byte equivalence checks.

Pull request overview

This PR optimizes binary logger serialization of task item metadata by memoizing previously-emitted NameValueList record IDs for shared copy-on-write immutable metadata dictionaries, reducing repeated enumeration, unescaping, and hashing work in BuildEventArgsWriter.WriteTaskItemList.

Changes:

  • Added a ConditionalWeakTable<ImmutableDictionary<string,string>, StrongBox<int>> cache to reuse NameValueList record IDs for items sharing the same immutable backing metadata dictionary.
  • Introduced a fast path that enumerates immutable backing metadata directly on cache misses while preserving the existing unescape behavior and metadata ordering contract.
  • Added focused unit tests covering shared-metadata reuse, empty-metadata short-circuiting, TaskItemData fallback behavior, and byte-for-byte equivalence with the legacy path.
File summaries
File Description
src/Framework/IMetadataContainer.cs Documents the escaped-value + ordering contract required for BackingMetadata to be safely consumable for serialization fast paths.
src/Build/Logging/BinaryLogger/BuildEventArgsWriter.cs Implements metadata record-id identity caching and updates WriteNameValueList to return the emitted record id for reuse.
src/Build.UnitTests/BuildEventArgsSerialization_Tests.cs Adds regression/compat tests validating reuse, fallbacks, and byte-for-byte serialization equivalence.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants