Skip to content

Write the video start bitrate hint as one connection-level value, once - #1218

Merged
xianshijing-lk merged 2 commits into
mainfrom
sxian/video-start-bitrate-connection-level
Sep 24, 2026
Merged

xianshijing-lk merged 2 commits into
mainfrom
sxian/video-start-bitrate-connection-level

Conversation

@xianshijing-lk

Copy link
Copy Markdown
Contributor

Ports client-sdk-js#2102 to Flutter. Same change already landed for Rust in rust-sdks#1430.

Why the JS PR is the reference, not the Rust one

lib/src/core/transport.dart is a near-literal Dart transliteration of the pre-fix JS code — same TrackBitrateInfo shape, same append-only tracker list, same per-section msid match inside negotiate(), same setMungedSDP fallback that restores the original SDP on rejection. The JS diff maps onto it almost line for line, so this PR mirrors its structure: the logic moves out of negotiate() into top-level findTrackCodecPayload / computeTrackStartBitrate / computeConnectionStartBitrate / applyVideoStartBitrate functions, which is also what makes it testable without a PeerConnection.

Rust could not be a template. It keeps a single max_send_bitrate_bps slot on the transport and munges the SDP as a string over video payload types, with no m-section or msid concept at all — there is nothing to port across.

x-google-start-bitrate is connection-scoped, not per-section

libwebrtc reads it per m-section but doesn't apply it per m-section:

bitrate_config_ = GetBitrateConfigForCodec(send_codec()->codec);   // webrtc_video_engine.cc
...
call_->GetTransportControllerSend()->SetSdpBitrateParameters(bitrate_config_);

That lands in RtpBitrateConfigurator, which holds one BitrateConstraints for the entire peer connection. Every video m-section writes the same slot, last writer wins, and the order is just m-section order.

Today each track writes its own value, so publishing a camera and a screen share together seeds the estimator from whichever section happens to be last. computeConnectionStartBitrate now takes the largest hint among the video sections that can send, and every video section gets that same number.

Written once per connection, not on every offer

libwebrtc retains start_bitrate_bps in RtpBitrateConfigurator and re-applies it on network route changes (RtpTransportControllerSend::OnNetworkRouteChanged), so a WiFi→cellular handover re-seeds the estimator from this hint with no renegotiation. Rewriting it on a later offer is at best a no-op and at worst restarts a converged estimator.

_hasAppliedVideoStartBitrate latches only after setMungedSDP has accepted the offer — it falls back to the unmunged SDP on rejection — so a rejected munge retries on the next offer. A full reconnect builds a new Transport and seeds the new estimator again.

Only sections that can send

The bitrate tracker list is append-only and an unpublished section keeps its a=msid, so matching a section to a track by msid alone would still pair a stale entry with the section it used to occupy. That lets an uncapped screen-share target seed a connection that now carries only a camera, or consumes the one-shot hint on a section that sends nothing — which would leave every later publish with no hint at all.

The direction is the discriminator, as an exclusion rather than a match: recvonly and inactive are the only directions that cannot carry local media, and they are exactly where a section lands once its sender is removed (removeTrack moves sendonly → inactive and sendrecv → recvonly). sendonly, sendrecv and a section with no direction attribute — which SDP defaults to sendrecv — all send.

Two calculation changes

Cap and floor. Camera is now capped at 1 Mbps so the estimator does not open too aggressively; screen share is exempt, because its content needs the bitrate immediately to stay legible. Targets below 300 kbps get no hint. Both match client-sdk-js and the Rust SDK; Flutter had neither.

Target derivation. computeStartTargetBitrate replaces encodings[0].maxBitrate. Simulcast rids are ordered q, h, f, so reading the first encoding took the lowest layer — for a 720p publish that is ~150 kbps, giving a ~135 kbps "hint" that did nothing. This is not optional cleanup: with the new 300 kbps floor, the old derivation would have suppressed the hint entirely for most camera publishes. A single SVC stream still uses its first encoding, since it declares the whole budget there; plain simulcast and SVC-published-as-simulcast (L1T*) sum.

Tests

test/core/start_bitrate_test.dart, 13 cases mirroring the JS suite: section/codec matching, the cap and its screen-share exemption, the floor, the connection-level max, stale-entry filtering, exclusion of a section that stopped sending but kept its msid (both inactive and recvonly), inclusion of sendrecv and direction-less sections, and the target derivation.

Verified locally on Flutter 3.44.9 / Dart 3.12.2: flutter analyze clean, flutter test 437 passed / 1 skipped, dart format --set-exit-if-changed clean, import_sorter clean.

devin-ai-integration[bot]

This comment was marked as resolved.

@xianshijing-lk
xianshijing-lk force-pushed the sxian/video-start-bitrate-connection-level branch from f6af116 to dbd1524 Compare September 22, 2026 22:40
Ported from client-sdk-js#2102. libwebrtc reads `x-google-start-bitrate`
per m-section but applies it to the shared `Call`
(`WebRtcVideoSendChannel::ApplyChangedParams` -> `SetSdpBitrateParameters`),
where `RtpBitrateConfigurator` holds one config for the whole peer
connection. Writing a different value per track therefore resolved to
last-writer-wins on m-section order: publishing a camera and a screen
share together seeded the estimator from whichever section came last.
Every video section now carries the same number, the largest hint among
the sections that can send.

The hint is also written only on the first offer that carries local
video, instead of on every offer. libwebrtc retains `start_bitrate_bps`
and re-applies it on network route changes, so rewriting it later is at
best a no-op and at worst restarts a converged bandwidth estimator. The
latch is consumed only once `setMungedSDP` has accepted the offer, so a
rejected munge retries on the next one.

Only sections that can send local media count. The bitrate tracker list
is append-only and an unpublished section keeps its `a=msid`, so
matching on msid alone would pair a stale entry with the section it used
to occupy. `recvonly` and `inactive` are the only directions that cannot
carry local media, and they are where a section lands once `removeTrack`
runs; `sendonly`, `sendrecv` and an omitted direction all send.

Two calculation changes come with it. Camera is capped at 1 Mbps so the
estimator does not open too aggressively, with screen share exempt
because its content needs the bitrate immediately to stay legible, and
targets below 300 kbps get no hint at all. The target is now the sum of
the encodings rather than `encodings[0]`, which on a simulcast publish
read the `q` layer and understated the target by close to an order of
magnitude -- without this the new floor would have suppressed the hint
for most camera publishes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@xianshijing-lk
xianshijing-lk force-pushed the sxian/video-start-bitrate-connection-level branch from dbd1524 to 67592ac Compare September 22, 2026 22:44
Comment thread lib/src/core/transport.dart Outdated
Comment thread lib/src/core/transport.dart
A LocalTrack keeps its cid across unpublish and republish
(`Track.getCid()` caches `mediaStreamTrack.id`), and the tracker list
only ever grew, so republishing left two entries sharing a cid.
`computeConnectionStartBitrate` stops at the first entry whose cid the
section carries, which made the older one authoritative: a first publish
below the 300 kbps floor leaves the one-shot latch unset, and the stale
sub-floor entry then shadowed the republished track's real target and
suppressed the hint entirely.

Replace by cid in `setTrackBitrateInfo` rather than picking the newest
at read time. That fixes the cause instead of the symptom, keeps the
list from growing for the lifetime of the connection, and makes the
`break` correct by construction. It is safe here: the two call sites are
the parallel and sequential branches of the same publish, and backup
codec publishes register no tracker, so no two entries legitimately
share a cid.

`applyVideoStartBitrate` returned `true` from both exits, so the value
told the caller nothing. It is `void` now and the caller sets its own
flag.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@xianshijing-lk
xianshijing-lk merged commit 0e121fc into main Sep 24, 2026
15 checks passed
@xianshijing-lk
xianshijing-lk deleted the sxian/video-start-bitrate-connection-level branch September 24, 2026 01:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants