Skip to content

MINOR: [c++] reject surrogate code points in json string encoder - #3973

Open
arib06 wants to merge 1 commit into
apache:mainfrom
arib06:cpp-json-encode-surrogate
Open

MINOR: [c++] reject surrogate code points in json string encoder#3973
arib06 wants to merge 1 commit into
apache:mainfrom
arib06:cpp-json-encode-surrogate

Conversation

@arib06

@arib06 arib06 commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

JsonGenerator::doEncodeString (the C++ JSON value encoder) decodes each multi-byte UTF-8 sequence of a string value and re-escapes it, but it never checks for code points in the UTF-16 surrogate range (U+D800..U+DFFF). Surrogates are ill-formed in UTF-8 and have no JSON representation. The binary decoder reads string bytes without validating UTF-8, so a string value carrying WTF-8/CESU-8 surrogate bytes (for example ED A0 80 for U+D800) is serialized as a lone \ud800, which is invalid JSON. The JSON decoder already rejects lone surrogates (#3841); this makes the encoder consistent by throwing on a decoded surrogate code point. Valid UTF-8 never contains surrogate encodings, so well-formed values are unaffected.

Verifying this change

This change added tests and can be verified as follows:

  • Added testEncodeSurrogate in JsonTests.cc, which parses a string holding raw surrogate bytes (U+D800 and U+DFFF) and asserts that serializing it throws instead of emitting a lone \u surrogate. Without the fix the encoder produces "\ud800" / "\udfff"; with it the full JsonTests suite passes.

Documentation

  • Does this pull request introduce a new feature? no

@github-actions github-actions Bot added the C++ Pull Requests for C++ binding label Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C++ Pull Requests for C++ binding

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant