I am trying to understand the behavior of queue_size in Alloy/OTel Collector when the exporter is down

[ashish-sparrows]

Setup

Processor:

otelcol.storage.file "local_file_storage" {
  directory = "C:\\temp\\alloy"
  
  compaction {
    on_start   = false

    on_rebound = true
    
    rebound_needed_threshold_mib = 2
    
    rebound_trigger_threshold_mib = 1
  }
}

otelcol.exporter.otlphttp "central_otlp_service_exporter" {
  client {
    endpoint = env("GRAFANA_ALLOY_OTEL_EXPORTER_OTLP_ENDPOINT")
    auth     = otelcol.auth.basic.central_otlp_service_basic_auth.handler
    tls {
      insecure = false
    }
  }

  retry_on_failure {
    enabled          = true
    initial_interval = "5s"
    max_interval     = "30s"
    max_elapsed_time = "0s" 
  }

  sending_queue {
    enabled       = true
    queue_size    = 10 
    storage       = otelcol.storage.file.local_file_storage.handler
  }
}

all other config is set to default

Test Scenario I am trying:
Log emit rate: 1 log/second
Exporter target: CloudWatch Logs
OTEL endpoint is down until I see the queue full error message in alloy

Observations
Exporter fully down

Logs are emitted at 1 log/sec

Around 90 logs are being stored in local storage
My understanding is that since the queue_size is only 10, there should be only 10 records in the log file
And from 11th on, logs should be dropped because (max_elapsed_time = 0) and queue is also full

is my understanding is correct?
can you please provide if there is any document which explain this in details with some examples

Thanks in Advance
Ashish Agrawal

[s3onghyun]

Your understanding is off by one detail, and it explains the whole observation: queue_size does not count log records. By default it counts requests — i.e. batches.

From the exporterhelper docs:

• sizer (default = requests): "number of incoming batches of metrics, logs, traces"
• queue_size (default = 1000): "Maximum size the queue can accept. Measured in units defined by sizer"

So queue_size = 10 means ten batches may sit in the queue, and each batch can hold many log records. Roughly 90 records living in ≤10 queued batches is exactly what that configuration should produce — nothing is misbehaving. The same doc note applies to the file-storage case: "the maximum number of batches stored to disk can be controlled using sending_queue.queue_size … defaults to 1000 batches".

If you want the limit expressed in records rather than batches, set the sizer:

sending_queue {
  enabled    = true
  sizer      = "items"     // count log records / spans / data points
  queue_size = 10          // now really means 10 records
  storage    = otelcol.storage.file.local_file_storage.handler
}

Available options are requests (default, cheapest), items (individual records), and bytes (serialized size — the one to use if your real constraint is disk space).

One more thing about your drop expectation: with max_elapsed_time = "0s" retries never give up, so a request stays in flight and the queue keeps backing up until it is full. Once full, new data is dropped at the queue — that part of your reasoning is right, it just triggers at 10 batches, not 10 records.

Docs: exporterhelper README — the sizer / queue_size entries and the persistent-queue section. Alloy wraps this same component, so the semantics carry over.