Skip to content

🐇🕳️ Compile fee rules to closures instead of running CEL - #5326

Draft
jmg-duarte wants to merge 2 commits into
mainfrom
jmgd/fee-engine-compiler
Draft

jmg-duarte wants to merge 2 commits into
mainfrom
jmgd/fee-engine-compiler

Conversation

@jmg-duarte

Copy link
Copy Markdown
Contributor

Description

The interpreter spent most of its time on moving values in and out of CEL (the order as a CEL map for each order, name lookups, fees as maps), not on the rule logic. On mainnet the engine took about 11 µs per order, about 70 ms per auction (Prometheus, 2026-11-19).

On a real mainnet auction (14281257, 6,363 orders) with the prod rules:

  • The fees are the same for every order.
  • The evaluation goes from 4.4 to 0.08 µs per order (28 ms to 0.5 ms per auction).

The rule language is now a fixed subset of CEL: the operators, has, the fee functions and the lists. Anything else is an error when the rules load. The order fields are the same as before.

After adding the CEL-based fee engine:
image

After replacing it with the compiler-based fee engine:
image

Important

Infra must change before this is deployed. Every rendered [fee-engine] section has wrapped_native.exists(t, t in correlated_N), and macros no longer compile, so the services would not start. See Rollout.

Behavior changes:

  • Mistakes show when the rules load. An unknown field, list or function, a wrong type (order.sellAmount == "x") or a constant fee argument off the grid (volume(0.001)) in any branch is a load error. Before, it was a skipped rule on each order, or found only if a sample order reached it.
  • Numbers are exact. Amounts are 256-bit integers and fractions are decimals (rust_decimal). Before, amounts were doubles: a real order with a sell amount of 999999999999999984204664 was equal to 1e24. Literals are read from the rule text, so 1e24 is exactly 10^24. All arithmetic is checked; an overflow skips the rule for that order.
  • order.creationDate is a number, in Unix seconds like validTo and validFrom. Before, it was an RFC 3339 string.
  • No time in rules. A scheduled fee change is a config deploy at that time.
  • Lists. A member that is a 0x address matches in any letter case. Any other member is compared as written. in is the only operation on a list.
  • appCode matches one spelling only, because lowerAscii is gone. Put every spelling in a list.
  • A part of an expression that reads nothing from the order is computed once at load. Before, only a fee that named no variable was.

The sample-order check (RuleSet::validate_on_sample_order) stays for the failures that depend on the order: an optional field read without has, arithmetic, a computed fee argument.

Changes

  • fee-engine:
    • New compile module. compile::expression walks the parsed tree once and gives a Node: a type and a Val, which is a constant or a boxed closure over the OrderView. number holds the exact number types and their promotion rules.
    • order_view is a static table of fields with Rust accessors. The engine builds no CEL map per order.
    • RuleSet::evaluate(view) replaces RuleSet::evaluator(now).evaluate(view). Evaluator is removed.
    • RuleError is now Missing, Arithmetic, Range and Fee. FeeError is removed.
    • Address lists are alloy_primitives::map::AddressSet. A list of at most 4 addresses is scanned.
    • README: the language, the three points where a mistake shows, and a section on how a rule compiles. tests/readme.rs now compiles every when: and fee: example line.
    • New tests/numbers.rs for the exact-number cases.
    • Dependencies: const-hex and serde_json are removed. alloy-primitives gets the map feature.
  • autopilot, orderbook: call RuleSet::evaluate. The engine side no longer takes now; the legacy fees still do.

How to test

  1. cargo nextest run -p fee-engine -p autopilot -p orderbook -p shared -p configs
  2. Without [fee-engine], the existing e2e tests run the legacy path.

I've also ran this on BNB production as the pictures above showed

The `cel` crate now only parses. Each rule compiles once to typed
closures that read the order through a field table, so no CEL map is
built per order. Numbers stay exact (U256, I256, Decimal).

Type errors, unknown names and constant fee arguments off the grid are
now load errors. The sample-order check stays for run-time failures.

Macros (`exists`, `all`, `map`, ...) are not supported: test a list
with `in`. Infra must replace `wrapped_native.exists(t, t in L)`
before this ships.

On mainnet auction 14281257 with the prod rules, the fees are the same
for all 6,363 orders and evaluation goes from 4.4 to 0.12 µs/order.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant