Repository navigation
Skip the lowercased read-back of a feature's full_name - #7398
Conversation
UC lowercases a feature's name, and create and get return it that way. Listing full_name in provided_id_fields skips that read-back difference, as it does for schemas and feature specs, so a mixed-case full_name no longer recreates the feature on every plan. The fake lowercases the name like the real endpoint. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
|
Commit: Regression test reportSelected 2 changed tests (limit=20, 0 not selected) 4 tests: regression=4 unreleased=0 coverage=0 failing=0
TestAccept/bundle/resources/features/uppercase-name/DMS=: branch ✅ | main (daad5b4) ❌ | latest (v1.26.0) ❌main (daad5b4): latest (v1.26.0): same output as main (daad5b4) TestAccept/bundle/resources/features/uppercase-name/DMS=true: branch ✅ | main (daad5b4) ❌ | latest (v1.26.0) ❌main (daad5b4): latest (v1.26.0): same output as main (daad5b4) TestAccept/bundle/resources/materialization_sets/uppercase-name/DMS=: branch ✅ | main (daad5b4) ❌ | latest (v1.26.0) ❌main (daad5b4): latest (v1.26.0): same output as main (daad5b4) TestAccept/bundle/resources/materialization_sets/uppercase-name/DMS=true: branch ✅ | main (daad5b4) ❌ | latest (v1.26.0) ❌main (daad5b4): latest (v1.26.0): same output as main (daad5b4) |
✅ Integration test reportCommit: b32af2f
Top 3 slowest tests (at least 2 minutes):
|
UC stores a feature's name lowercase and the service requires a set member to name its feature that way, while a member's feature_name keeps the config spelling, whether it is written out, a reference to a feature, or built from a schema resource's name. So a mixed-case member was rejected at create, and a set holding one planned a recreate on every deploy. - MaterializedFeatureMemberKey lowercases the feature name, so a mixed-case member pairs with its lowercase read-back. A name holding an unresolved reference keeps its case, since resource keys are case-sensitive. - OverrideChangeDesc skips the feature_name of a paired member, which can then differ only in case. - DoCreate and DoUpdate send a copy of the set with each member's feature name lowercased. - The test server's set create refuses a member that does not name its feature in lowercase, with the service's message. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
andrewnester
left a comment
There was a problem hiding this comment.
Neither test is Cloud = true (the directory is local-only by design): was the create-time refusal of a mixed-case feature_name and the lowercased update body checked against a real workspace?
Co-authored-by: Isaac <no-reply@databricks.com>
Thanks for pointing this out! Both create and update are tested against a staging workspace with a CLI built from this branch: Create: a member that names its feature in mixed case is refused with [BAD_REQUEST] Materialized feature requests must include the resolved feature names, followed by the lowercase name. Update: an update naming a member in mixed case is refused with Materialized features cannot be added to an existing materialization set, and feature '...' does not match any member of this set. Recreate the set instead. The set is left unchanged, and the test server returns the same text. With this PR, a bundle that names both features in mixed case (one by reference, one written out) deploys, and a plan on it changes nothing. |
Co-authored-by: Isaac <no-reply@databricks.com>
main now prints a blank line after each title, numbers distinct UUIDs, and runs both DMS legs for features and materialization sets. Co-authored-by: Isaac <no-reply@databricks.com>
Changes
features: addfull_name(which is a three part UC name) toprovided_id_fields. This was overlooked when the resource was added.materialization_sets: compute the composite key of a materialized feature member of the set using a case-insensitivefeature_name(a feature resource's name) and send each materialized feature member'sfeature_namelowercased on create and update. In addition,OverrideChangeDescskips a materialized feature member'sfeature_namechange that only differs in case.Why
Unity Catalog lowercases a feature's name, while the bundle keeps the spelling it was given, whether written out, referenced, or built from a schema resource's name. So a feature with uppercase letters in
full_namewas recreated on every deploy, and a materialization set materialized feature member naming its feature viafeature_namethat way was rejected at create or recreated the set on every deploy.Tests
bundle/resources/features/uppercase-name: mixed-case name plans no change; a rename recreatesbundle/resources/materialization_sets/uppercase-name: lowercase on create and update; case-only edits skip; a rename recreates