--- title: Recommended configurations description: Use this guide to configure Span Metrics for production environments, optimize performance, and integrate features such as sampling, latency buckets, setting metric expiration, Serverless environment, and more. url: https://coralogix.com/docs/user-guides/apm-v2/getting-started/span-metrics-recommended-configuration/ product: apm feature: OTel Span Metrics feature_id: apm-otel-collector-spanmetrics feature_identifier: apm/apm-otel-collector-spanmetrics last_updated: '2026-11-12' --- # Recommended configurations Use this guide to configure Span Metrics for production environments, optimize performance, and integrate features such as sampling, latency buckets, setting metric expiration, Serverless environment, and more. ## Generated metrics When Span Metrics is enabled, the OpenTelemetry Collector generates the following metrics from your spans: | Metric name (OTLP) | Prometheus exported name | Metric type | Auto-generated labels (dimensions) | | ------------------ | ------------------------------------------------------------ | ----------- | -------------------------------------------------------------------------- | | `requests` | `calls_total` | Counter | `service.name`, `span.name`, `span.kind`, `status.code` | | `duration` | `duration_ms_bucket`, `duration_ms_sum`, `duration_ms_count` | Histogram | `service.name`, `span.name`, `span.kind`, `status.code`, `le` (for bucket) | | `errors` | `errors_total` | Counter | `service.name`, `span.name`, `span.kind`, `status.code` | ### Identify Span Metrics among your metrics Span Metrics use reserved metric names, so you can tell them apart from your application and infrastructure metrics by name and by their span-metric labels (`service_name`, `span_kind`, `status_code`): * Full Span Metrics: `calls_total`, `errors_total`, `duration_ms_bucket`, `duration_ms_sum`, `duration_ms_count`. * [Compact Span Metrics](https://coralogix.com/docs/user-guides/apm-v2/getting-started/using-compact-span-metric.md): `compact_calls_total` and `compact_duration_ms_*` (Services catalog); `db_compact_calls_total` and `db_compact_duration_ms_*` (Databases catalog). For example, request throughput for a service, taken from Span Metrics, is: ``` sum(rate(calls_total{service_name="checkout"}[5m])) ``` ## Recommended dimensions The Coralogix `values.yaml` adds these dimensions to enable APM features and richer analysis: ``` extraDimensions: - name: http.method - name: cgx.transaction - name: cgx.transaction.root - name: status_code - name: db.namespace - name: db.operation.name - name: db.collection.name - name: db.system ``` Enabling the Span Metrics preset also turns on: * **Error tracking** (`errorTracking`): powers the [Errors](https://coralogix.com/docs/user-guides/apm-v2/features/errors.md) tab. * **Service versioning** (`serviceVersion`): powers [grouping service metrics by version](https://coralogix.com/docs/user-guides/apm/features/group-by-service-version.md). Make sure the `service.version` attribute [is added](https://coralogix.com/docs/user-guides/apm/features/group-by-service-version.md#adding-the-serviceversion-attribute-using-otel_resource_attributes) to your service spans. * **Database metrics** (`dbMetrics`): generates RED metrics for database spans, which is what makes your [databases](https://coralogix.com/docs/user-guides/apm-v2/databases.md) appear as entities in the catalog. ## Dimensions in the catalog The **Dimensions** you can filter and group by in the Services and Databases catalog are the **labels on your span metrics**, not raw span attributes. The Traces view, by contrast, shows every span and resource attribute. A span attribute becomes a selectable Dimension only once it is configured as a span-metric dimension and is present on the metric the catalog reads. Two consequences follow: * An attribute that is on your spans but not configured as a span-metric dimension does not appear in **Dimensions**, even though it is still visible on individual traces. * By default, the catalog reads the [Compact Span Metrics](https://coralogix.com/docs/user-guides/apm-v2/getting-started/using-compact-span-metric.md), which carry only a small, fixed label set. An attribute outside that set - for example the `service.instance.id` resource attribute - is not available as a Dimension. ### Add a custom dimension To make an extra **span attribute** available as a Dimension, add it to the `spanMetrics` preset: ``` presets: spanMetrics: enabled: true extraDimensions: - name: ``` If you run your own collector, add it to the `spanmetrics` connector's `dimensions` list instead. Note `extraDimensions` promotes **span** attributes. A **resource** attribute such as `service.instance.id` is not promoted this way, and the compact metric the catalog reads keeps only its fixed label set, so adding a resource attribute to `extraDimensions` does not surface it. To use a resource attribute as a Dimension, first copy it onto the span as a span attribute with a collector transform, then add that span attribute to `extraDimensions`. ## Use span sampling Span Metrics performs best with minimal span sampling (head or tail) allowing to have enough trace examples to power trace-based features in the UI. ### Why sampling matters While Span Metrics does not require trace sampling to be configured in any specific way, we recommend using a minimal sampling rate, either head or tail sampling, to ensure that enough trace examples remain available for UI features that rely on actual traces. These include service maps, drilldowns, Coralogix traces explorer, Spans Highlights, and more. Span Metrics will still provide complete RED metrics even with sampling, but maintaining a small set of representative traces significantly improves the troubleshooting experience. By default, head sampling is enabled in our [latest Helm chart](https://github.com/coralogix/telemetry-shippers/blob/master/otel-integration/k8s-helm/values.yaml). This setting can be changed either directly in the `values.yaml` or from the UI. To learn more about head and tail sampling, refer to the following documents: * [Head sampling guide](https://github.com/coralogix/telemetry-shippers/tree/master/otel-integration/k8s-helm#configuring-head-sampling-for-tracing) * [Tail sampling guide](https://github.com/coralogix/telemetry-shippers/tree/master/otel-integration/k8s-helm#tail-sampling-with-opentelemetry-using-kubernetes) ## Manage cardinality Use the following methods to reduce or control cardinality: * Normalize span names and URLs. * Sanitize database statements. * Remove high-cardinality resource attributes. * Set `aggregationCardinalityLimit` and `metricsExpiration` to protect the pipeline. For more details, see [Span Metrics Cardinality Limiting](https://coralogix.com/docs/user-guides/apm-v2/getting-started/span-metrics-cardinality-limits.md). To track per-service ingestion and spot services approaching their limit before they trip it, use [Service Governance](https://coralogix.com/docs/user-guides/account-management/payment-and-billing/metrics-usage-analyzer/apm.md) in the Metrics Usage Analyzer. ## Manage histogram metric buckets Histogram buckets define how latency distributions are computed for percentiles, SLOs, and Apdex scoring. Span Metrics does not include default buckets - you must configure them manually. ### Configure collector buckets ``` connectors: spanmetrics: histogram: explicit: buckets: [100us, 1ms, 2ms, 4ms, 6ms, 10ms, 100ms, 250ms] ``` Select thresholds that reflect the latency behavior of your workloads. ### Apdex settings For Apdex scoring, your bucket list must include: * the Apdex threshold (`T`) * the tolerated threshold (`4T`) In the example above, the Apdex threshold can be set to 1ms because both `T` (1ms) and `4T` (4ms) are specified. If either bucket is missing, the **Apdex score** widget displays an **Apdex cannot be calculated** message instead of a score. See [When Apdex cannot be calculated](https://coralogix.com/docs/user-guides/apm-v2/features/apdex-score.md#when-apdex-cannot-be-calculated) for the full message and recovery steps. ### Modifying buckets used by old SLO or Apdex If you modify bucket configuration for metrics already used in SLOs or Apdex scoring: * The active calculation stops. * The UI shows a bucket mismatch error. * You must update the SLO/Apdex definition to match the new buckets. * The calculation restarts as a new SLO. ### Selecting buckets Best practices: * Include both `T` and `4T`. * Analyze your latency distribution (minimum, median, maximum). * Add intermediate buckets for meaningful granularity. * Exclude buckets for rare anomalies unless they occur frequently. * Keep bucket lists consistent across all collectors and clusters. ### Configure different buckets per application (Kubernetes only) Use the `spanMetricsMulti` preset to define bucket sets per service. ``` presets: spanMetricsMulti: enabled: false defaultHistogramBuckets: [1ms, 4ms, 10ms, 20ms, 50ms, 100ms, 200ms, 500ms, 1s, 2s, 5s] configs: - selector: route() where attributes["service.name"] == "one" histogramBuckets: [1s, 2s] - selector: route() where attributes["service.name"] == "two" histogramBuckets: [5s, 10s] ``` Each selector requires an [OTTL](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/ottl/README.md) statement. Find out more [here](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/connector/routingconnector). ## Enable spanMetricsMulti in Helm Use this mode if you want to define metrics for each service. It is recommended to have a broad bucket definition that covers the system. `spanMetricsMulti` allows for more detailed per-service metrics. ``` spanMetrics: enabled: false spanMetricsMulti: enabled: true ``` Note If you use multiple collectors, ensure that the bucket configuration is consistent across all of them. ## Set metric expiration `metricsExpiration` removes inactive metric series after a defined window to control memory consumption. ``` metricsExpiration: 5m ``` Retention behavior * Metric series are stored only in-memory. * All series reset on Collector or pod restart. * If a service stops sending spans for the duration of the expiration period, its series are removed automatically. ## Use resource key attributes Use `resource_metrics_key_attributes` to group metric series at the resource level. This stabilizes metric grouping across services, environments, clusters, and tenants. ``` resource_metrics_key_attributes: - service.name - service.namespace ``` ## Use exemplars Exemplars link metrics to the traces that generated them, enabling seamless navigation from metric panels to the corresponding span data. This improves issue investigation and helps correlate latency spikes or error rates with specific requests. Exemplars are enabled by default in most configurations, but you can disable them if your environment if needed: ``` opentelemetry-agent: extraConfig: connectors: spanmetrics: exemplars: enabled: true ``` ## Configure serverless environments The Services catalog functions seamlessly with both Span Metrics and Events2Metric, as long as all instructions in the documentation are followed correctly. To display services based on AWS Lambda in the Services catalog, your organization must send spans or Span Metrics. With Events2Metric (E2M), this is done automatically. For Span Metrics, ensure that Lambda-generated spans are routed through the collector. Note This ensures that Lambda-based services, along with their metrics, are displayed in the Services catalog. Serverless catalog is supported only using Events2Metrics. ## Next steps Explore the [Span Metrics Cardinality Limiting](https://coralogix.com/docs/user-guides/apm-v2/getting-started/span-metrics-cardinality-limits.md) guide to learn about the three-layered approach to manage and mitigate cardinality issues.