Label Naming Conventions and Low Cardinality Strategies

What Are Labels?

In any metrics system, a label is a key-value pair attached to a data point. Labels add dimensions to your metrics, letting you slice and filter data by things like HTTP method, environment, or region.

Here is the critical thing to understand: in Prometheus, every unique combination of metric name and label key-value pairs creates a distinct time series. In Grafana Loki, labels group log entries into streams that get indexed. Either way, labels are the mechanism that turns a single metric name into potentially thousands of individual data streams.

A simple example:

http_requests_total{method="GET", status="200", service="api"}
http_requests_total{method="POST", status="201", service="api"}
http_requests_total{method="GET", status="404", service="api"}

Those three lines represent three separate time series. They share a metric name but differ in their label values. That distinction is what makes labels powerful for querying and dangerous for cardinality.

If you’re new to instrumenting applications, the metrics quickstart guide walks through defining metrics and labels from scratch.

What Is Label Cardinality?

Cardinality is the number of distinct values an attribute can have. A boolean field has a cardinality of 2. An environment label with values prod, staging, and dev has a cardinality of 3. A user_id label on a system with 500,000 users has a cardinality of 500,000.

Low cardinality means a label has a small, bounded set of possible values. Metrics with low cardinality labels offer broad trends with less overhead. Think HTTP methods (GET, POST, PUT, DELETE, PATCH) or status code classes (2xx, 3xx, 4xx, 5xx).

High cardinality means a label can take on many distinct values. Fields like user IDs, session tokens, email addresses, and raw URL paths are classic examples. Each new value mints a new time series.

The Multiplication Formula

This is where things get dangerous. The total number of time series for a single metric equals the product of unique values across all its labels:

total_series = unique_vals(label1) × unique_vals(label2) × unique_vals(label3) × ...

Consider a metric with three labels: 5 HTTP methods, 10 status codes, and 10,000 user IDs. That produces 5 × 10 × 10,000 = 500,000 time series from one metric. This is called a cardinality explosion.

Why This Costs Real Money

Each time series in the Prometheus head block requires approximately 3 to 4 KB of RAM. An instance with 1 million active time series typically consumes 4 to 6 GB of RAM just for the head block. Beyond memory, high cardinality degrades query performance, increases storage costs, and can crash your monitoring infrastructure at the worst possible moment.

Label Naming Conventions

Consistent naming is the foundation of usable metrics. Without conventions, you end up with a codebase where req_count, requestsTotal, http_requests, and api_request_counter all measure the same thing, each added by a different developer at a different time.

Here are the rules that matter.

Rule 1: Use snake_case for Everything

Both metric names and label names should use snake_case with lowercase characters. Underscores separate whole words. This keeps everything consistent and readable: http_requests_total, request_duration_seconds, status_code.

Rule 2: Include Units and Type Suffixes in Metric Names

Prometheus conventions call for embedding the base unit in the metric name: _seconds, _bytes, _total. Counters get the _total suffix. Histograms automatically generate _bucket, _sum, and _count from the base name. Gauges have no special suffix.

Examples:

  • Counter: http_requests_total
  • Histogram: request_duration_seconds (produces request_duration_seconds_bucket, etc.)
  • Gauge: temperature_celsius

Rule 3: Use a Domain Prefix

A metric name should have a single-word application prefix relevant to the domain. This prevents collisions when multiple services emit metrics to the same backend: myapp_http_requests_total, payment_transaction_amount_dollars.

Rule 4: Don’t Put Label Names in the Metric Name

This is a common mistake. If you have a label called method, don’t create http_get_requests_total and http_post_requests_total. Instead, use one metric with a label: http_requests_total{method="GET"}. Putting labels in the metric name introduces redundancy and breaks aggregation.

When working with the JavaScript metrics client, these conventions apply directly to how you define your metric names and label keys in code.

Rule 5: Be Descriptive, Not Abbreviated

Avoid abbreviations that might mean different things to different teams. req_dur could mean request duration, required duration, or something else entirely. Write request_duration_seconds and remove all ambiguity.

Prometheus vs. OpenTelemetry: The Naming Split

This is a real source of friction for teams adopting OpenTelemetry. The two systems disagree on naming:

Aspect Prometheus OpenTelemetry
Delimiter Underscores (http_request_duration_seconds) Dots for namespaces, underscores within components (http.server.request.duration)
Units in name Yes (_seconds, _bytes) No (units go in metadata)
Type suffix Yes (_total for counters) No

Prior to Prometheus 3.0, metric names from OpenTelemetry had to be translated by replacing dots with underscores. A survey of OpenTelemetry users who use PromQL found that 60% preferred keeping the original dot-delimited names, while 40% wanted Prometheus-compatible names. Prometheus 3.0 introduced UTF-8 metric name support, which eases this tension, but the ecosystem is still transitioning.

If your team uses OpenTelemetry instrumentation, pick one convention and enforce it. Mixing both creates confusion that compounds over time. For more background on how telemetry standards like OpenTelemetry relate to practical instrumentation, see this overview of telemetry and MELT.

Good Labels vs. Bad Labels

The fastest way to internalize label naming conventions and low cardinality strategies is a concrete comparison. This table is worth bookmarking.

Good Labels (Low Cardinality) Bad Labels (High Cardinality)
environment (prod, staging, dev) user_id (unbounded, grows with users)
method (GET, POST, PUT, DELETE) request_id (unique per request)
status_class (2xx, 3xx, 4xx, 5xx) email (PII and unbounded)
region (us-east, eu-west) raw_url_path (/users/12345)
service_name (bounded set of services) pod_uid (ephemeral, high churn)
error_type (timeout, auth_failure, validation) session_token (unique per session)
queue_name (bounded set) commit_sha (new value every deploy)

The deciding question: “Could this label have 10,000 unique values in production?” If yes, it does not belong on a metric. It belongs in a trace span, a log line, or structured metadata.

Low Cardinality Strategies

Knowing what low cardinality means is only half the battle. Here are eight strategies to enforce it.

Strategy 1: Never Use Unbounded Values as Labels

This is the most important rule. User IDs, email addresses, session tokens, raw URL paths, container IDs, and commit SHAs are all unbounded. Each new value creates a new time series that consumes memory and storage indefinitely.

A production runbook from fosskit.com captures the common failure mode well: a developer adds a label that looks reasonable in staging (10 users, 5 endpoints) but explodes in production with thousands of users and hundreds of routes. Cardinality is a staging-to-production trap.

Strategy 2: Bucket High-Cardinality Values

When you need some granularity from a high-cardinality dimension, bucket it. Instead of individual HTTP status codes (200, 201, 204, 301, 400, 401, 403, 404, 500, 502, 503), use status classes: 2xx, 3xx, 4xx, 5xx. Instead of exact response sizes, use size ranges. Instead of raw URL paths, use route templates (/users/:id rather than /users/12345).

Strategy 3: Prefer Static, Infrastructure-Level Labels

Good labels describe the origin or context of a metric, not the specific request. Labels like application, namespace, environment, and region are static and long-lived. They change infrequently and have bounded value sets.

The Grafana Loki documentation is explicit about this: avoid assigning dynamic labels unless they are genuinely low cardinality and long-lived. The Loki team even reconsidered using level as a label because a simple filter expression proved equally fast for medium-volume applications.

Strategy 4: Move High-Cardinality Data to Logs, Traces, or Structured Metadata

Metrics and traces serve fundamentally different purposes. Metrics answer “how many” and “how fast” across aggregate dimensions. Traces answer “what happened to this specific request.” Mixing their data models is the source of most cardinality mistakes.

If you need to search by customer ID or transaction ID, store those in structured metadata, trace spans, or log entries. Loki’s structured metadata feature lets you store frequently searched, high-cardinality fields without impacting the label index.

Understanding the boundaries between counters, histograms, and instrumentation patterns helps clarify which data belongs in metrics versus other telemetry signals.

Strategy 5: Drop Unnecessary Labels at Ingestion

Prometheus relabeling rules and Loki pipeline stages let you strip labels before they hit storage. Dropping universally high-cardinality labels like pod_uid, container_id, and image_id typically reduces cardinality by 60 to 80 percent.

This is also where ingestion payload limits and batching become relevant. Understanding your ingestion pipeline’s constraints helps you design label sets that stay within guardrails.

Strategy 6: Pre-Aggregate with Recording Rules

Recording rules let you pre-compute expensive PromQL expressions and store results as new, lower-cardinality time series. You aggregate away high-cardinality dimensions while keeping the derived metrics you actually query.

The Prometheus community standard for naming recording rules is level:metric:operations. For example, job:http_requests_total:rate5m clearly communicates the aggregation level, the source metric, and the operation applied. Following this convention consistently makes recording rules self-documenting.

Strategy 7: Set Ingestion Guardrails

Prevention beats remediation. Prometheus provides several configuration knobs:

  • sample_limit: fails any scrape returning more than N samples
  • label_limit: caps the number of labels per series
  • label_name_length_limit and label_value_length_limit: cap label string sizes
  • target_limit: caps discovered targets per scrape job

These guardrails catch cardinality problems before they hit your TSDB.

For serverless metrics services, the Distlang Metrics API handles ingestion constraints at the API level, so you don’t need to manage scrape configuration.

Strategy 8: Use Native Histograms to Eliminate Bucket Explosion

Classic Prometheus histograms interact badly with labels. A latency histogram with 10 buckets multiplied by 5 label combinations produces 50 time series for a single metric, plus _sum and _count series on top. Practitioners on observability forums frequently cite this as one of the biggest hidden sources of cardinality growth.

Native histograms (experimental since Prometheus v2.40, with mature tooling in the 3.x line) store the entire distribution in one series with dynamic buckets. The per-bucket series explosion disappears. If histograms dominate your series count, native histograms are the structural fix.

Bonus: Tiered Retention and Roll-Up

Roll-up strategies pair with tiered retention policies. Keep raw high-cardinality data for 24 to 48 hours, drop the high-cardinality labels after that window, and retain low-cardinality aggregates for 13 or more months. This gives you debugging granularity when you need it and cost-efficient long-term trends when you don’t.

Cardinality in Serverless and Edge Runtimes

Most cardinality guidance assumes long-running, host-based infrastructure. Serverless and edge runtimes are different in ways that matter.

No pod or host labels needed. Serverless functions don’t have persistent host identifiers. The instance label that dominates Kubernetes cardinality is simply irrelevant. This naturally eliminates one major cardinality vector.

Route normalization is critical. Edge functions handling parameterized routes must normalize paths to route templates. /users/12345 becomes /users/:id. Without this normalization, every unique URL parameter mints a new time series.

Churn from cold starts. Short-lived handlers create ephemeral execution contexts. Labels tied to function invocation IDs would be unbounded. This is where the distinction between cardinality and churn matters. As one observability engineer noted, you can have low cardinality with high churn or the reverse, and both hurt performance. Ephemeral serverless functions and Kubernetes pods create churn even with bounded label sets.

Flush semantics constrain label decisions. In short-lived serverless handlers, labels must be decided before the flush window. On Cloudflare Workers, that means inside ctx.waitUntil(). On Vercel, it means within the after() callback. There is no time for complex label computation. For details on managing this, see flush behavior and buffering.

Simpler label sets win. For serverless functions, the recommended label set is typically just four labels, all bounded and all low cardinality: environment, route, method, status_class. That’s it.

If you’re instrumenting Cloudflare Workers specifically, the guide on instrumenting Workers with counters and histograms covers these patterns in detail.

Common Mistakes

The Staging-to-Production Trap

A label with 10 unique values in staging can have 100,000 in production. Always evaluate cardinality against production traffic patterns, not development environments.

Confusing Churn with Cardinality

Churn refers to time series being created and destroyed rapidly, even if the total number of active series at any moment stays moderate. Kubernetes pod restarts and serverless cold starts cause churn. The symptoms (high memory, slow compaction) overlap with cardinality problems, but the fixes differ. Cardinality problems require reducing label values. Churn problems require longer-lived label associations or accepting the cost of head block turnover.

Histogram Label Multiplication

Developers often add labels to histogram metrics without accounting for the multiplier. Ten default buckets times five label dimensions equals fifty time series per metric. Before adding a label to a histogram, multiply out the full cost.

Inconsistent Naming Across Teams

Without enforced naming conventions, every team invents its own style. You end up with http_req_total, httpRequestCount, and api.requests all representing the same thing. Pick conventions, document them, and enforce them in code review.

Quick Checklist for Label Naming and Cardinality

Print this. Share it with your team. Reference it during code review.

  • [ ] All label names use snake_case with lowercase characters
  • [ ] Metric names include a domain prefix, unit, and type suffix
  • [ ] No label name is repeated inside the metric name
  • [ ] Every label has a bounded set of values (ideally under 100)
  • [ ] No user IDs, request IDs, email addresses, or session tokens as labels
  • [ ] URL paths are normalized to route templates
  • [ ] Histogram metrics account for bucket × label multiplication
  • [ ] Recording rules follow the level:metric:operations naming pattern
  • [ ] Ingestion guardrails (sample_limit, label_limit) are configured
  • [ ] High-cardinality debugging data goes to logs or traces, not metric labels
  • [ ] Labels are evaluated against production traffic volumes, not staging

If you’re starting from scratch, the developer onboarding checklist provides a step-by-step walkthrough for setting up metrics with these conventions built in.

Ready to apply these label naming conventions and low cardinality strategies to your serverless application? Distlang Metrics is built for edge and serverless runtimes, with a free tier that handles 500k rows per month.

FAQ

How many label values is too many?

There is no universal threshold, but most practitioners treat anything over 100 unique values per label as a warning sign. The real danger is the multiplicative effect: two labels with 100 values each produce 10,000 combinations. Aim for labels with tens of values, not hundreds or thousands.

Should I use Prometheus naming conventions or OpenTelemetry conventions?

Pick one and stick with it across your organization. If you query with PromQL, Prometheus conventions (snake_case, units in name, _total suffix) are the safer choice today. If you’re standardizing on the OpenTelemetry SDK and have adopted Prometheus 3.0 or a compatible backend, dot-delimited names work. The worst outcome is mixing both.

What’s the difference between cardinality and churn?

Cardinality is about how many unique time series exist. Churn is about how fast time series are created and destroyed. A Kubernetes deployment with 50 pods restarting every hour has moderate cardinality but high churn. A single metric with a user_id label has high cardinality but possibly low churn. Both cause performance problems, but they require different fixes.

Can I use high-cardinality labels if I have a big enough backend?

Throwing hardware at a cardinality problem works until it doesn’t. Costs grow linearly (or worse) with series count, and query performance degrades. It’s almost always better to move high-cardinality data to traces or logs and keep metrics focused on aggregate patterns.

How do label naming conventions differ for serverless environments?

Serverless runtimes eliminate host and pod labels, which reduces one cardinality vector automatically. But they introduce flush timing constraints (labels must be finalized before the handler exits) and route normalization requirements (parameterized paths must be templatized). The recommended label set is smaller: environment, route, method, and status_class cover most use cases.

Do native histograms really fix the bucket cardinality problem?

Yes, for the specific problem of bucket multiplication. Classic histograms create a separate time series for each bucket boundary. Native histograms store the entire distribution in a single series with dynamically adjusted buckets. If histograms account for a large share of your total series count, native histograms can dramatically reduce it. They are available in Prometheus 2.40+ (experimental) and more mature in the 3.x line.

What’s the best way to enforce naming conventions across a team?

Document your conventions in an internal style guide. Enforce them through linting in CI (tools like pint for Prometheus rules, or custom checks on metric definitions). Review metric and label names in code review the same way you review API naming. Inconsistency compounds fast when multiple teams contribute metrics independently.

How do I detect a cardinality explosion that has already happened?

In Prometheus, query prometheus_tsdb_head_series to see your active series count over time. Use topk() and count by() queries to identify which metrics and labels contribute the most series. In Grafana Loki, the label cardinality API shows which labels have the most unique values. Sudden jumps in series count almost always trace back to a recently added unbounded label.