Title
Snowflake destination has no way to emit TIMESTAMP_LTZ -- only TIMESTAMP_TZ/TIMESTAMP_NTZ
Environment
Description
SnowflakeTypeMapper.to_db_datetime_type (dlt/destinations/impl/snowflake/factory.py)
only ever produces two possible column types for a timestamp-typed column:
timestamp = "TIMESTAMP_TZ" if timezone else "TIMESTAMP_NTZ"
Snowflake has three timestamp variants, not two:
TIMESTAMP_NTZ -- no timezone.
TIMESTAMP_TZ -- stores an explicit offset captured at write time; querying it
later always returns that same stored offset, regardless of the querying
session's TIMEZONE setting.
TIMESTAMP_LTZ -- stores an absolute instant (UTC-normalized internally); querying
it re-localizes the displayed value to whatever the querying session's TIMEZONE
parameter is set to.
There's no timezone/hint combination that reaches TIMESTAMP_LTZ today. This
matters concretely for any source system whose native timezone-aware type has
TIMESTAMP_LTZ's semantics rather than TIMESTAMP_TZ's -- e.g. Oracle's
TIMESTAMP WITH LOCAL TIME ZONE, which behaves exactly like TIMESTAMP_LTZ
(absolute instant, session-timezone-based display) and not like TIMESTAMP_TZ
(fixed offset). Mapping such a source column to TIMESTAMP_TZ is not just a
different type name -- it's a different, weaker contract: every consumer gets the
same fixed offset forever, instead of the source's actual "each client sees it in
their own timezone" behavior.
We verified this concretely end-to-end: loading an Oracle TIMESTAMP(6) WITH LOCAL TIME ZONE column through dlt into Snowflake produces TIMESTAMP_TZ(6), and
querying that column under two different Snowflake session TIMEZONE settings
returns the same fixed offset both times -- confirming it does not re-localize,
unlike the source column's actual behavior.
Suggested fix
Add a third value to the timezone column hint's effective range, or a separate
destination-specific hint (e.g. a snowflake_adapter-style column adapter, similar
to how other destinations expose destination-specific type overrides) that lets a
caller request TIMESTAMP_LTZ explicitly.
Workaround we're using in the meantime
A subclassed SnowflakeTypeMapper/Destination factory that overrides
to_db_datetime_type (and the sct_to_unbound_dbt/sct_to_dbt/dbt_to_sct lookup
tables) to emit TIMESTAMP_LTZ wherever the stock mapper would emit TIMESTAMP_TZ.
type_mapper is already a pluggable field on DestinationCapabilitiesContext, so
this is possible without patching dlt itself -- filing this mainly so the gap is
tracked upstream and others hitting the same Oracle/Postgres-style
"local timezone" source type don't have to rediscover it.
Title
Snowflake destination has no way to emit
TIMESTAMP_LTZ-- onlyTIMESTAMP_TZ/TIMESTAMP_NTZEnvironment
Description
SnowflakeTypeMapper.to_db_datetime_type(dlt/destinations/impl/snowflake/factory.py)only ever produces two possible column types for a
timestamp-typed column:Snowflake has three timestamp variants, not two:
TIMESTAMP_NTZ-- no timezone.TIMESTAMP_TZ-- stores an explicit offset captured at write time; querying itlater always returns that same stored offset, regardless of the querying
session's
TIMEZONEsetting.TIMESTAMP_LTZ-- stores an absolute instant (UTC-normalized internally); queryingit re-localizes the displayed value to whatever the querying session's
TIMEZONEparameter is set to.
There's no
timezone/hint combination that reachesTIMESTAMP_LTZtoday. Thismatters concretely for any source system whose native timezone-aware type has
TIMESTAMP_LTZ's semantics rather thanTIMESTAMP_TZ's -- e.g. Oracle'sTIMESTAMP WITH LOCAL TIME ZONE, which behaves exactly likeTIMESTAMP_LTZ(absolute instant, session-timezone-based display) and not like
TIMESTAMP_TZ(fixed offset). Mapping such a source column to
TIMESTAMP_TZis not just adifferent type name -- it's a different, weaker contract: every consumer gets the
same fixed offset forever, instead of the source's actual "each client sees it in
their own timezone" behavior.
We verified this concretely end-to-end: loading an Oracle
TIMESTAMP(6) WITH LOCAL TIME ZONEcolumn through dlt into Snowflake producesTIMESTAMP_TZ(6), andquerying that column under two different Snowflake session
TIMEZONEsettingsreturns the same fixed offset both times -- confirming it does not re-localize,
unlike the source column's actual behavior.
Suggested fix
Add a third value to the
timezonecolumn hint's effective range, or a separatedestination-specific hint (e.g. a
snowflake_adapter-style column adapter, similarto how other destinations expose destination-specific type overrides) that lets a
caller request
TIMESTAMP_LTZexplicitly.Workaround we're using in the meantime
A subclassed
SnowflakeTypeMapper/Destinationfactory that overridesto_db_datetime_type(and thesct_to_unbound_dbt/sct_to_dbt/dbt_to_sctlookuptables) to emit
TIMESTAMP_LTZwherever the stock mapper would emitTIMESTAMP_TZ.type_mapperis already a pluggable field onDestinationCapabilitiesContext, sothis is possible without patching dlt itself -- filing this mainly so the gap is
tracked upstream and others hitting the same Oracle/Postgres-style
"local timezone" source type don't have to rediscover it.