fix(model): keep quoting when normalizing column_descriptions keys - #5959
fix(model): keep quoting when normalizing column_descriptions keys#5959ReguiguiMohamed wants to merge 2 commits into
Conversation
_column_descriptions_validator built each key with part.this, which is the bare identifier string, so the quoted flag was gone before normalize_identifiers ran. A quoted key was then normalized as if it were unquoted, and on dialects where quoting makes a column case-sensitive the resulting name matched no column, so the description was dropped along with the rest of the table's comments. Normalize each part while it is still an identifier. Unquoted keys normalize exactly as before. Fixes SQLMesh#5943 Signed-off-by: ReguiguiMohamed <mohamedreguigui2004@gmail.com>
|
@ReguiguiMohamed Thanks for this PR! Any thoughts on this? Adding a dotted-key case ( Not a blocker though |
The join-by-parts path had no test. A BigQuery nested path normalizes the same as before the fix, since joining the parts and re-parsing produced a quoted identifier that BigQuery lowercases anyway. On Snowflake an unquoted path now normalizes per part, which is what an unquoted name should do there, and a quoted one keeps its case. Signed-off-by: ReguiguiMohamed <mohamedreguigui2004@gmail.com>
|
@StuffbyYuki Thanks for the review, and I'm glad to be contributing here. Good catch. I ran both cases against the pre-fix code. BigQuery is unchanged. A nested path still normalizes to Snowflake does change. Test for both pushed in 6df27fe. |
Fixes #5943.
_column_descriptions_validatorbuilds each key with".".join(part.this for part in v.this.parts).part.thisis the bare identifier string, so thequotedflag is gone beforenormalize_identifiersruns, and the key is normalized as if it had never been quoted. On a dialect where quoting makes a column case-sensitive,"myColumn"becomesMYCOLUMN, matches no column, and gets dropped along with the rest of the table's comments.Normalizing each part while it is still an identifier fixes it. Unquoted keys normalize exactly as before, and the dict branch used by Python models is unchanged in behaviour.
Before, with
dialect snowflake:After:
{'myColumn': ...}.test_column_descriptions_quoted_identifiercovers both halves, a quoted key keeping its case and an unquoted one still normalizing, and fails on main withMYCOLUMN != myColumn.tests/core/test_model.pypasses in full (334).ruffandruff-formatpass.mypyon the changed module reports nothing.