FEAT: Add PinyinConverter for Chinese Pinyin-mix transformations - #2649
Merged
Roman Lutz (romanlutz) merged 8 commits intoSep 15, 2026
Merged
Conversation
Adds a deterministic (no-LLM) text converter that rewrites Chinese (Hanzi) characters as their Pinyin romanization, a Chinese-specific adversarial pattern described in recent Chinese LLM safety work (e.g. CSSBench). Pinyin mixing can bypass keyword/token-level safety filters that match on Hanzi rather than on romanized readings. Supports three rendering modes (full reading, first-letter initial, and a per-character mix), a configurable proportion of Hanzi to convert (values below 1.0 leave the rest as Hanzi, producing mixed Hanzi/Pinyin text), an optional syllable separator, and a seed for reproducible selection. Non-Hanzi characters always pass through unchanged. pypinyin is added as an optional dependency (pip install pyrit[pinyin]) and imported lazily, following the pattern used by other optional-dependency converters. Tests that exercise conversion are skipped when pypinyin is absent. Closes microsoft#2647
Nantha kumar (Nanduu24)
force-pushed
the
feat/pinyin-converter
branch
from
September 13, 2026 23:45
3578241 to
9974b05
Compare
Contributor
Author
|
@microsoft-github-policy-service agree |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve readings against the complete prompt before selecting replacements. Preserve original trailing characters when inserting separators and add regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Include Unicode 17 CJK extensions G-J in Hanzi selection and preserve characters without dictionary readings. Cover romanization, block boundaries, and partial-selection counts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds PinyinConverter to doc/code/converters/1_text_to_text_converters.py with a Chinese-prompt example (full readings and first-letter initials), mirroring how the script-specific Arabic converters are documented. Satisfies test_all_converters_are_documented, which requires every converter to appear in a converter notebook.
Offload dictionary lookup, require pypinyin 0.55.0, synchronize the notebook examples and outputs, and extend async and seeded-converter regression coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Roman Lutz (romanlutz)
approved these changes
Sep 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds
PinyinConverter, a non-LLM text-to-text converter that rewrites selected Chinese (Hanzi) characters as Pinyin or initials. It supports Chinese-specific Pinyin-mix transformations described in CSSBench without translating the prompt into another language.Behavior
mode:fullproduces toneless Pinyin,initialproduces the first letter of each reading, andmixedindependently chooses full Pinyin or an initial for each selected character.银行or重庆.〇(ideographic zero), CJK Unified Ideographs Extensions A-J, and compatibility ideographs. Hanzi without a reading in the installed dictionary remain unchanged.proportion([0.0, 1.0]): selectsround(proportion * number_of_hanzi)positions. Unselected Hanzi remain unchanged. Zero selected characters, including counts that round to zero, leave the prompt unchanged.separator: inserted after converted characters, except at the end of the prompt. Original trailing spaces, newlines, and punctuation are preserved rather than trimmed.seed: provides reproducible position selection and mixed-mode rendering through the converter's scoped RNG.Examples:
mode="full")银行yinhangmode="full")二〇二六erlingerliumode="initial"银行yhseparator=" "重庆chong qingproportion=0.0, separator="!"你好!你好!Dependency and async behavior
pypinyin>=0.55.0is a regular dependency, not a separate optional extra. It is MIT-licensed, pure Python, and adds no runtime transitive dependencies on PyRIT's supported Python versions. The minimum version supplies the extended-Hanzi readings covered by the regression tests.uv.lockresolves version0.55.0with artifact hashes.The import remains lazy: importing PyRIT or constructing
PinyinConverterdoes not load the Pinyin dictionaries. Dictionary loading and phrase lookup run in a worker thread throughasyncio.to_thread, keeping synchronous dictionary I/O off the event loop. Conversion tests do not skip based on optional extras.Documentation
Adds full-Pinyin and initial-letter examples to the text-to-text converter notebook and its paired Python source, including the actual output. Refreshes the converter modality reference table in the overview notebook. Converter-specific behavior is documented in the class docstring.
Testing
pypinyin.0.55.0.Closes #2647