From 2b9f68d79bc46dbf3f991a636077617f0397d547 Mon Sep 17 00:00:00 2001 From: Charlie Le Date: Wed, 19 Aug 2026 14:34:36 -0700 Subject: [PATCH 1/2] Graduate native histogram ingestion and close the v1-guarantees gaps -blocks-storage.tsdb.enable-native-histograms shipped as experimental in 1.18.0, so 1.22.0 is four minors and roughly two years of exposure later. docs/guides/native-histograms.md already carries no experimental language, which contradicted v1-guarantees.md. Graduate the ingestion flag; the flag name does not change, so no deprecation shim is needed. This does not touch -blocks-storage.tsdb.out-of-order-cap-max, which has its own bullet. Also record the experimental flags this cycle added without a v1-guarantees entry: - -distributor.num-query-workers (#7623), marked EXPERIMENTAL in its flag help. - The whole Thanos PromQL engine surface. -querier.thanos-engine says "Experimental." in its help but nothing from pkg/engine/config.go was listed. - The whole parquet flag surface; only -parquet-converter.max-num-columns was listed. Grouped by prefix rather than enumerated, since the cache flags are boilerplate. - -querier.query-protection.rejection (#7442) and -querier.query-protection.eviction.max-evictions-per-cycle (#7488). Parquet and the tenant-federation regex matcher are deliberately not graduated: the parquet proposal is still "Status: Proposed", docs/guides/parquet-mode.md still warns it is experimental, and #7610 changed the file format this cycle. The regex matcher has only two minors of exposure and #7562 fixed cross-tenant stale cache reads this cycle. Refs #5922 Signed-off-by: Charlie Le --- CHANGELOG.md | 1 + docs/configuration/v1-guarantees.md | 26 ++++++++++++++++++++------ 2 files changed, 21 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8c2517bef4..989b96c5cf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,7 @@ - `-blocks-storage.tsdb.wal-compression-enabled` (deprecated in 1.19.0; use `-blocks-storage.tsdb.wal-compression-type`) - `-ingester.max-series-per-query` (a chunks-storage limit, ignored since blocks storage; use `-querier.max-fetched-series-per-query`) * [CHANGE] Ingester: Formally deprecate `-blocks-storage.tsdb.max-exemplars`, scheduled for removal in v1.24.0. Use the per-tenant `max_exemplars` limit instead. The flag still works as the global fallback when `max_exemplars` is 0, but setting it now logs a warning and increments `deprecated_flags_inuse_total`. #7793 +* [CHANGE] Ingester: Graduate native histogram ingestion (`-blocks-storage.tsdb.enable-native-histograms`) from experimental. #7789 * [CHANGE] Querier: Make query time range configurations per-tenant: `query_ingesters_within`, `query_store_after`, and `shuffle_sharding_ingesters_lookback_period`. Uses `model.Duration` instead of `time.Duration` to support serialization but has minimum unit of 1ms (nanoseconds/microseconds not supported). #7160 * [CHANGE] Cache: Setting `-blocks-storage.bucket-store.metadata-cache.bucket-index-content-ttl` to 0 will disable the bucket-index cache. #7446 * [CHANGE] HA Tracker: Move `-distributor.ha-tracker.failover-timeout` from a global config to a per-tenant runtime config. The flag name and default value (30s) remain the same. #7481 diff --git a/docs/configuration/v1-guarantees.md b/docs/configuration/v1-guarantees.md index a957e03c8b..0582882eb5 100644 --- a/docs/configuration/v1-guarantees.md +++ b/docs/configuration/v1-guarantees.md @@ -52,6 +52,7 @@ Currently experimental features are: - Do not extend writes on unhealthy ingesters (`-distributor.extend-writes=false`) - Accept multiple HA pairs in the same request (enabled via `-experimental.distributor.ha-tracker.mixed-ha-samples=true`) - Accept Prometheus remote write 2.0 request (`-distributor.remote-writev2-enabled=true`) + - Use a goroutine worker pool for query fan-out calls to ingesters (`-distributor.num-query-workers`) - Tenant Deletion in Purger, for blocks storage. - Blocks storage user index - Store Gateway max concurrent fetched data bytes limit @@ -103,8 +104,6 @@ Currently experimental features are: - Ingest delta temporality OTLP metrics (`-distributor.otlp.allow-delta-temporality=true`) - Persistent tokens in the Ruler Ring: - `-ruler.ring.tokens-file-path` (path) CLI flag -- Native Histograms - - Ingestion can be enabled by setting `-blocks-storage.tsdb.enable-native-histograms=true` on Ingester. - String interning for metrics labels - Enable string interning for metrics labels by setting `-ingester.labels-string-interning-enabled` on Ingester. - Query-frontend: query rejection (`-frontend.query-rejection.enabled`) @@ -113,9 +112,10 @@ Currently experimental features are: - Query-frontend: dynamic query splits - `querier.max-shards-per-query` (int) CLI flag - `querier.max-fetched-data-duration-per-query` (duration) CLI flag -- Ingester/Store-Gateway: Query rejection +- Ingester/Store-Gateway/Querier: Query rejection - `-ingester.query-protection.rejection` - `-store-gateway.query-protection.rejection` + - `-querier.query-protection.rejection` - Distributor/Ingester: Stream push connection - Enable stream push connection between distributor and ingester by setting `-distributor.use-stream-push=true` on Distributor. - Enable stream push authentication on Distributor/Ingester. (`-distributor.sign-write-requests-keys`) @@ -141,6 +141,7 @@ Currently experimental features are: - `-querier.query-protection.eviction.cooldown-period` (int) - `-querier.query-protection.eviction.eviction-metric` (string) - `-querier.query-protection.eviction.min-query-age` (duration) + - `-querier.query-protection.eviction.max-evictions-per-cycle` (int) - Ingester: Active Series Tracker - Per-tenant `active_series_trackers` configuration in runtime config overrides - Counts active series matching PromQL label matchers and exposes `cortex_ingester_active_series_per_tracker` metric @@ -154,6 +155,19 @@ Currently experimental features are: - `-ingester.head-queried-series-metrics-windows` time windows to report (default: 2h) - `-ingester.head-queried-series-metrics-window-duration` HLL sub-window size - `-ingester.head-queried-series-metrics-sample-rate` query sampling rate -- Parquet Converter: Maximum number of columns per file - - `-parquet-converter.max-num-columns` (int) CLI flag - - Automatically shards parquet files when the number of columns exceeds the configured limit +- Parquet storage + - Parquet Converter: the `-parquet-converter.*` CLI flags, including `-parquet-converter.enabled`, + `-parquet-converter.max-num-columns` (automatically shards parquet files when the number of columns + exceeds the configured limit) and the `-parquet-converter.ring.*` ring configuration + - Querier: `-querier.parquet-queryable-default-block-store`, `-querier.parquet-queryable-fallback-disabled`, + the `-querier.parquet-queryable.max-fetched-*` limits and `-querier.parquet-shard-cache-*` + - Store Gateway: `-blocks-storage.bucket-store.parquet-query-concurrency`, + `-blocks-storage.bucket-store.parquet-shard-cache-*` and the + `-blocks-storage.bucket-store.parquet-labels-cache.*` / + `-blocks-storage.bucket-store.parquet-row-ranges-cache.*` cache configuration +- Querier/Ruler: Thanos PromQL engine + - `-querier.thanos-engine` / `-ruler.thanos-engine` (boolean) CLI flags + - `-querier.enable-x-functions` / `-ruler.enable-x-functions` (boolean) CLI flags + - `-querier.optimizers` / `-ruler.optimizers` (string) CLI flags + - `-querier.decoding-concurrency` / `-ruler.decoding-concurrency` (int) CLI flags + - `-querier.selector-batch-size` / `-ruler.selector-batch-size` (int) CLI flags From e68288b2f1d130d7ebc627cda86c79e204053ed3 Mon Sep 17 00:00:00 2001 From: Charlie Le Date: Tue, 1 Sep 2026 13:41:30 -0700 Subject: [PATCH 2/2] Drop [EXPERIMENTAL] from -blocks-storage.tsdb.enable-native-histograms help Addresses PR review feedback: the flag is graduated in this PR, so the CLI help text should no longer mark it experimental. Regenerated config docs and JSON schema. Signed-off-by: Charlie Le --- docs/configuration/config-file-reference.md | 2 +- pkg/util/validation/limits.go | 2 +- schemas/cortex-config-schema.json | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/configuration/config-file-reference.md b/docs/configuration/config-file-reference.md index c713af4f12..1321bb42b7 100644 --- a/docs/configuration/config-file-reference.md +++ b/docs/configuration/config-file-reference.md @@ -4623,7 +4623,7 @@ The `limits_config` configures default and per-tenant limits imposed by Cortex s # series matching its matchers and exposes the count as a metric. [active_series_trackers: | default = []] -# [EXPERIMENTAL] True to enable native histogram. +# True to enable native histogram. # CLI flag: -blocks-storage.tsdb.enable-native-histograms [enable_native_histograms: | default = false] diff --git a/pkg/util/validation/limits.go b/pkg/util/validation/limits.go index b7c134093b..5123a4a8ca 100644 --- a/pkg/util/validation/limits.go +++ b/pkg/util/validation/limits.go @@ -313,7 +313,7 @@ func (l *Limits) RegisterFlags(f *flag.FlagSet) { f.IntVar(&l.MaxGlobalSeriesPerMetric, "ingester.max-global-series-per-metric", 0, "The maximum number of active series per metric name, across the cluster before replication. 0 to disable.") f.IntVar(&l.MaxLocalNativeHistogramSeriesPerUser, "ingester.max-native-histogram-series-per-user", 0, "The maximum number of active native histogram series per user, per ingester. 0 to disable. Supported only if ingester.active-series-metrics-enabled is true.") f.IntVar(&l.MaxGlobalNativeHistogramSeriesPerUser, "ingester.max-global-native-histogram-series-per-user", 0, "The maximum number of active native histogram series per user, across the cluster before replication. 0 to disable. Supported only if -distributor.shard-by-all-labels and ingester.active-series-metrics-enabled is true.") - f.BoolVar(&l.EnableNativeHistograms, "blocks-storage.tsdb.enable-native-histograms", false, "[EXPERIMENTAL] True to enable native histogram.") + f.BoolVar(&l.EnableNativeHistograms, "blocks-storage.tsdb.enable-native-histograms", false, "True to enable native histogram.") f.IntVar(&l.MaxExemplars, "ingester.max-exemplars", 0, "Enables support for exemplars in TSDB and sets the maximum number that will be stored. less than zero means disabled. If the value is set to zero, cortex will fallback to the deprecated blocks-storage.tsdb.max-exemplars value; that fallback is removed in v1.24.0.") f.Var(&l.OutOfOrderTimeWindow, "ingester.out-of-order-time-window", "[Experimental] Configures the allowed time window for ingestion of out-of-order samples. Disabled (0s) by default.") diff --git a/schemas/cortex-config-schema.json b/schemas/cortex-config-schema.json index 462435e200..2fa4d4eaf9 100644 --- a/schemas/cortex-config-schema.json +++ b/schemas/cortex-config-schema.json @@ -5670,7 +5670,7 @@ }, "enable_native_histograms": { "default": false, - "description": "[EXPERIMENTAL] True to enable native histogram.", + "description": "True to enable native histogram.", "type": "boolean", "x-cli-flag": "blocks-storage.tsdb.enable-native-histograms" },