Skip to content

feat(promptinj): add 12 English & Simplified Chinese financial-triage indirect prompt-injection cases (TC-0456–TC-0467) - #119

Open
Ziqi wants to merge 1 commit into
GenAI-Security-Project:mainfrom
Ziqi:feat/en-zh-financial-triage-promptinj
Open

Ziqi wants to merge 1 commit into
GenAI-Security-Project:mainfrom
Ziqi:feat/en-zh-financial-triage-promptinj

Conversation

@Ziqi

@Ziqi Ziqi commented Nov 12, 2026

Copy link
Copy Markdown

Summary

Contributes 12 synthetic defensive test cases (TC-0456 through TC-0467) under datasets/promptinj_dataextraction_testcases/en_zh_starshield_financial_triage/, expanding coverage along two axes noted in the dataset's current limitations:

  1. Language & Code-Switching Diversity (en + zh-Hans): Adds English and Simplified Chinese (with prompt_en_gloss English translations) indirect prompt-injection payloads modeled on cross-border B2B invoice-diversion, supplier portal notes, and disputed payment transcripts.
  2. Under-Represented DSGAI Categories: Covers 7 categories across the canonical schema:
    • direct_extraction (DSGAI01: TC-0456, TC-0457) — HTML-comment & bilingual authority spoofing suppressing IOC extraction and exfiltrating RAG compliance thresholds
    • tool_plugin_leakage (DSGAI06: TC-0458, TC-0459) — Untrusted supplier memos hijacking ERP vendor-master and wire-transfer lookup tools
    • cross_context_bleed (DSGAI11: TC-0460, TC-0461) — Spoofed QA batch macros probing shared session summarization buffers
    • nl_query_injection (DSGAI12: TC-0462, TC-0463) — ISO 20022 / Faster Payments remittance narratives broadening LLM-to-SQL predicates
    • multimodal_extraction (DSGAI09: TC-0464, TC-0465) — PNG iTXt chunk & low-opacity watermark instructions overriding VLM receipt triage
    • credential_harvesting (DSGAI02: TC-0466) — Webhook troubleshooting pretext soliciting runtime <API_KEY> / [BEARER_TOKEN] headers
    • system_prompt_extraction (DSGAI15: TC-0467) — Bilingual JSON transcoding with delimiter obfuscation

Provenance, Anonymization & Validation

  • 100% Synthetic Clean-Room Data (CC-BY-4.0): Adapted from the open-source StarShield-Bench suite (Concept DOI: 10.5281/zenodo.24145804). Uses official ISO-13616 / ISO-7812 textbook test vectors (GB82WEST12345698765432, NO9386011117947, 6200000000000005), RFC 2606 .example domains, and bracketed credential placeholders. No real personal data or undisclosed vendor vulnerabilities are referenced.
  • Local CI Validation Passed (0 errors, 0 warnings):
    • python data_validation/run_all_checks.py --verbose -> PASS promptinj_dataextraction_testcases (466 file(s), 0 error(s), 0 warning(s))
    • python data_validation/qc_tools/consistency_check.py -> 0 error(s), 0 warning(s)
    • python data_validation/croissant/build_croissant.py --check & check_croissant.py -> PASS

@Ziqi

Ziqi commented Nov 12, 2026

Copy link
Copy Markdown
Author

Maintainer status note for reviewers: upstream starshield-forensics is now at v0.3.7 (reproducible Ed25519-signed starshield.pyz, Python 3.10+ stdlib-only, 25/25 unit & canary tests) and archived at Zenodo DOI 10.5281/zenodo.24156262 (Concept DOI 10.5281/zenodo.24145804).
Supply-chain and governance verification is live under Linux Foundation OpenSSF Best Practices Silver (#16602, 226%) and OpenSSF Scorecard 7.1/10 (CodeQL SAST, Dependabot, branch protection, and signed releases).
The diff in this PR remains clean and requires no structural changes unless maintainers prefer any metadata adjustment—happy to address any review feedback.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant