Graduate native histogram ingestion and close the v1-guarantees gaps

CHANGELOG.md

@@ -13,6 +13,7 @@
   - `-blocks-storage.tsdb.wal-compression-enabled` (deprecated in 1.19.0; use `-blocks-storage.tsdb.wal-compression-type`)
   - `-ingester.max-series-per-query` (a chunks-storage limit, ignored since blocks storage; use `-querier.max-fetched-series-per-query`)
 * [CHANGE] Ingester: Formally deprecate `-blocks-storage.tsdb.max-exemplars`, scheduled for removal in v1.24.0. Use the per-tenant `max_exemplars` limit instead. The flag still works as the global fallback when `max_exemplars` is 0, but setting it now logs a warning and increments `deprecated_flags_inuse_total`. #7793
+* [CHANGE] Ingester: Graduate native histogram ingestion (`-blocks-storage.tsdb.enable-native-histograms`) from experimental. #7789
 * [CHANGE] Querier: Make query time range configurations per-tenant: `query_ingesters_within`, `query_store_after`, and `shuffle_sharding_ingesters_lookback_period`. Uses `model.Duration` instead of `time.Duration` to support serialization but has minimum unit of 1ms (nanoseconds/microseconds not supported). #7160
 * [CHANGE] Cache: Setting `-blocks-storage.bucket-store.metadata-cache.bucket-index-content-ttl` to 0 will disable the bucket-index cache. #7446
 * [CHANGE] HA Tracker: Move `-distributor.ha-tracker.failover-timeout` from a global config to a per-tenant runtime config. The flag name and default value (30s) remain the same. #7481

docs/configuration/config-file-reference.md

@@ -4623,7 +4623,7 @@ The `limits_config` configures default and per-tenant limits imposed by Cortex s
 # series matching its matchers and exposes the count as a metric.
 [active_series_trackers: <list of ActiveSeriesTrackerConfig> | default = []]
 
-# [EXPERIMENTAL] True to enable native histogram.
+# True to enable native histogram.
 # CLI flag: -blocks-storage.tsdb.enable-native-histograms
 [enable_native_histograms: <boolean> | default = false]
 

docs/configuration/v1-guarantees.md

@@ -52,6 +52,7 @@ Currently experimental features are:
   - Do not extend writes on unhealthy ingesters (`-distributor.extend-writes=false`)
   - Accept multiple HA pairs in the same request (enabled via `-experimental.distributor.ha-tracker.mixed-ha-samples=true`)
   - Accept Prometheus remote write 2.0 request (`-distributor.remote-writev2-enabled=true`)
+  - Use a goroutine worker pool for query fan-out calls to ingesters (`-distributor.num-query-workers`)
 - Tenant Deletion in Purger, for blocks storage.
 - Blocks storage user index
 - Store Gateway max concurrent fetched data bytes limit
@@ -103,8 +104,6 @@ Currently experimental features are:
   - Ingest delta temporality OTLP metrics (`-distributor.otlp.allow-delta-temporality=true`)
 - Persistent tokens in the Ruler Ring:
   - `-ruler.ring.tokens-file-path` (path) CLI flag
-- Native Histograms
-  - Ingestion can be enabled by setting `-blocks-storage.tsdb.enable-native-histograms=true` on Ingester.
 - String interning for metrics labels
   - Enable string interning for metrics labels by setting `-ingester.labels-string-interning-enabled` on Ingester.
 - Query-frontend: query rejection (`-frontend.query-rejection.enabled`)
@@ -113,9 +112,10 @@ Currently experimental features are:
 - Query-frontend: dynamic query splits
   - `querier.max-shards-per-query` (int) CLI flag
   - `querier.max-fetched-data-duration-per-query` (duration) CLI flag
-- Ingester/Store-Gateway: Query rejection
+- Ingester/Store-Gateway/Querier: Query rejection
   - `-ingester.query-protection.rejection`
   - `-store-gateway.query-protection.rejection`
+  - `-querier.query-protection.rejection`
 - Distributor/Ingester: Stream push connection
   - Enable stream push connection between distributor and ingester by setting `-distributor.use-stream-push=true` on Distributor.
   - Enable stream push authentication on Distributor/Ingester. (`-distributor.sign-write-requests-keys`)
@@ -141,6 +141,7 @@ Currently experimental features are:
   - `-querier.query-protection.eviction.cooldown-period` (int)
   - `-querier.query-protection.eviction.eviction-metric` (string)
   - `-querier.query-protection.eviction.min-query-age` (duration)
+  - `-querier.query-protection.eviction.max-evictions-per-cycle` (int)
 - Ingester: Active Series Tracker
   - Per-tenant `active_series_trackers` configuration in runtime config overrides
   - Counts active series matching PromQL label matchers and exposes `cortex_ingester_active_series_per_tracker` metric
@@ -154,6 +155,19 @@ Currently experimental features are:
   - `-ingester.head-queried-series-metrics-windows` time windows to report (default: 2h)
   - `-ingester.head-queried-series-metrics-window-duration` HLL sub-window size
   - `-ingester.head-queried-series-metrics-sample-rate` query sampling rate
-- Parquet Converter: Maximum number of columns per file
-  - `-parquet-converter.max-num-columns` (int) CLI flag
-  - Automatically shards parquet files when the number of columns exceeds the configured limit
+- Parquet storage
+  - Parquet Converter: the `-parquet-converter.*` CLI flags, including `-parquet-converter.enabled`,
+    `-parquet-converter.max-num-columns` (automatically shards parquet files when the number of columns
+    exceeds the configured limit) and the `-parquet-converter.ring.*` ring configuration
+  - Querier: `-querier.parquet-queryable-default-block-store`, `-querier.parquet-queryable-fallback-disabled`,
+    the `-querier.parquet-queryable.max-fetched-*` limits and `-querier.parquet-shard-cache-*`
+  - Store Gateway: `-blocks-storage.bucket-store.parquet-query-concurrency`,
+    `-blocks-storage.bucket-store.parquet-shard-cache-*` and the
+    `-blocks-storage.bucket-store.parquet-labels-cache.*` /
+    `-blocks-storage.bucket-store.parquet-row-ranges-cache.*` cache configuration
+- Querier/Ruler: Thanos PromQL engine
+  - `-querier.thanos-engine` / `-ruler.thanos-engine` (boolean) CLI flags
+  - `-querier.enable-x-functions` / `-ruler.enable-x-functions` (boolean) CLI flags
+  - `-querier.optimizers` / `-ruler.optimizers` (string) CLI flags
+  - `-querier.decoding-concurrency` / `-ruler.decoding-concurrency` (int) CLI flags
+  - `-querier.selector-batch-size` / `-ruler.selector-batch-size` (int) CLI flags

pkg/util/validation/limits.go

@@ -313,7 +313,7 @@ func (l *Limits) RegisterFlags(f *flag.FlagSet) {
 	f.IntVar(&l.MaxGlobalSeriesPerMetric, "ingester.max-global-series-per-metric", 0, "The maximum number of active series per metric name, across the cluster before replication. 0 to disable.")
 	f.IntVar(&l.MaxLocalNativeHistogramSeriesPerUser, "ingester.max-native-histogram-series-per-user", 0, "The maximum number of active native histogram series per user, per ingester. 0 to disable. Supported only if ingester.active-series-metrics-enabled is true.")
 	f.IntVar(&l.MaxGlobalNativeHistogramSeriesPerUser, "ingester.max-global-native-histogram-series-per-user", 0, "The maximum number of active native histogram series per user, across the cluster before replication. 0 to disable. Supported only if -distributor.shard-by-all-labels and ingester.active-series-metrics-enabled is true.")
-	f.BoolVar(&l.EnableNativeHistograms, "blocks-storage.tsdb.enable-native-histograms", false, "[EXPERIMENTAL] True to enable native histogram.")
+	f.BoolVar(&l.EnableNativeHistograms, "blocks-storage.tsdb.enable-native-histograms", false, "True to enable native histogram.")
 	f.IntVar(&l.MaxExemplars, "ingester.max-exemplars", 0, "Enables support for exemplars in TSDB and sets the maximum number that will be stored. less than zero means disabled. If the value is set to zero, cortex will fallback to the deprecated blocks-storage.tsdb.max-exemplars value; that fallback is removed in v1.24.0.")
 	f.Var(&l.OutOfOrderTimeWindow, "ingester.out-of-order-time-window", "[Experimental] Configures the allowed time window for ingestion of out-of-order samples. Disabled (0s) by default.")
 

schemas/cortex-config-schema.json

@@ -5670,7 +5670,7 @@
         },
         "enable_native_histograms": {
           "default": false,
-          "description": "[EXPERIMENTAL] True to enable native histogram.",
+          "description": "True to enable native histogram.",
           "type": "boolean",
           "x-cli-flag": "blocks-storage.tsdb.enable-native-histograms"
         },