Collector sizing and scaling
How much CPU and memory the OpenTelemetry Collector needs depends on your traffic volume and on which pipeline features you run. This guide covers the settings that keep the Collector's agent, gateway, and receiver from running out of memory under load, and how to scale them out as traffic grows.
The most common failure mode is a collector pod that is OOMKilled at peak traffic. Almost always the fix is to make sure the pod has a memory limit set with enough headroom, because the Collector's built-in memory safeguards are calculated from that limit.
Set resource requests and limits
The Collector's memory safeguards - the memory_limiter processor and GOMEMLIMIT - are both derived from the pod's memory limit. If no limit is set, neither can cap memory, and the pod can grow until the node OOM-kills it.
In the Coralogix otel-integration chart, the node agent (daemonset) and the cluster collector ship with a memory limit. The gateway and the standalone receiver may ship without one, so set requests and limits explicitly before you put them under load.
Coralogix's recommended starting point for a gateway, suggested in the chart's tail-sampling value files, is:
opentelemetry-gateway:
replicaCount: 3
resources:
requests:
cpu: 0.5
memory: 256Mi
limits:
cpu: 2
memory: 2G
For the standalone receiver, set the same keys under opentelemetry-receiver.
Adjust these values based on the signals below. For the current defaults of every component, see the otel-integration values.yaml.
Control memory use
Three settings govern how much memory the Collector holds, and all of them assume a memory limit is set:
memory_limiterprocessor: Runs in every pipeline. By default it engages at about 80% of the pod's memory limit, with a 25% spike allowance, checked every 5 seconds. At the soft limit it refuses new data (applying backpressure to the sources); at the hard limit it forces garbage collection.GOMEMLIMIT: Set to about 80% of the memory limit, so Go's garbage collector works harder before the pod reaches its limit.- Coralogix exporter sending queue: Holds a bounded queue of data in memory while it sends and retries (200 MiB per exporter by default). If you raise the queue size, the batch size, or the number of consumers, raise the pod's memory requests and limits to match, or the collector can OOM under a backlog.
Keep batches within endpoint limits
The default batch settings (send_batch_size: 1024, send_batch_max_size: 2048, timeout: 1s) balance throughput against memory. Larger batches use more memory and must stay within the Coralogix endpoint limits and quotas: a hard limit of 10 MB per request, 2 MB recommended. Raising batch or queue sizes to push more data through is a common cause of memory pressure.
Scale out
The gateway and receiver scale horizontally: add replicas and let the agents load-balance across them.
- Replicas: Increase the replica count for the gateway and receiver as traffic grows (the recommended starting point above uses 3).
- Autoscaling: The chart supports a Horizontal Pod Autoscaler for deployment-mode collectors, but it is off by default. Enable it if your load varies through the day.
- When to scale: Add capacity when the exporter queue is consistently 60-70% full, or when the refused or enqueue-failed metrics below start rising.
Signals to watch
Use the Collector's own metrics to tell whether a collector is under-provisioned:
| Signal | What it means |
|---|---|
otelcol_exporter_queue_size vs otelcol_exporter_queue_capacity | The queue is filling up. Scale out at roughly 60-70% of capacity. |
otelcol_exporter_enqueue_failed_*, otelcol_exporter_send_failed_* | The collector is dropping data or failing to send it. |
otelcol_processor_memory_limiter_refused_spans (and the log record and metric point equivalents) | memory_limiter is applying backpressure because memory is tight. |
k8s.container.status.last_terminated_reason = OOMKilled | A pod is being killed for exceeding its memory limit. Raise the limit or scale out. |
Capacity planning
Coralogix does not publish a fixed throughput-to-resources table, because actual CPU and memory depend on your signal mix (logs, metrics, spans) and on the pipeline features you enable, such as resource detection, span metrics, tail sampling, and Kubernetes metadata enrichment. Start from the defaults above, watch the signals, and adjust. For general guidance on scaling the Collector, see the upstream OpenTelemetry Scaling the Collector documentation.