Buffering and Flush Strategies for Short Lived Handlers 2026

Serverless functions are fast, cheap, and ephemeral. That last quality is exactly what makes observability tricky. If you’ve ever deployed a metrics or analytics SDK to an edge function and watched your dashboard stay empty, you’ve already encountered the core problem that buffering and flush strategies for short-lived handlers exist to solve.

This guide defines the key concepts, walks through platform-specific patterns, and gives you a decision framework for picking the right approach.

If you’re looking for a serverless-native metrics service built around these patterns, Distlang Metrics handles flush semantics out of the box for Workers and Vercel.

What Buffering Means in Short-Lived Handlers

In any metrics, logging, or telemetry client, buffering means collecting data points in an in-memory array instead of sending each one over the network immediately. When your code records a counter increment or a trace span, the SDK doesn’t fire off an HTTP request right away. It drops the record into a buffer and waits until the batch is full or a timer fires.

This is efficient. A long-running Node.js server can accumulate hundreds of data points and send them in a single batch every few seconds. The network overhead per data point drops dramatically.

The problem is that short-lived handlers don’t behave like long-running servers. A Cloudflare Worker, a Vercel serverless function, or an AWS Lambda invocation might process a single request and then stop existing. The buffer never fills. The timer never fires. The data just vanishes.

As Cloudflare’s own documentation states, “outstanding asynchronous tasks are canceled as soon as a Worker finishes sending its main response body to the client.” That’s not a bug. It’s the defined behavior. And it means any SDK that relies on background flushing will silently lose data in these environments.

For a deeper look at how buffer sizes and batch payloads interact, see this guide on batching and size limits.

What Flushing Means (and Why It’s Not Optional Here)

A flush is an explicit command that tells the SDK: stop waiting, send everything in the buffer right now. It’s the counterpart to buffering, the mechanism that converts queued-up data into actual network requests.

Two variants show up across SDKs:

Asynchronous flush returns control to your code immediately while the export runs in the background. This is the common default. The SDK starts delivering events but doesn’t wait for confirmation.

Synchronous flush blocks execution until delivery completes. Your handler won’t return a response until the data has been sent. This adds latency to the response but guarantees the data gets out.

flush() vs forceFlush() vs shutdown()

These three methods appear across OpenTelemetry and various analytics SDKs, and confusing them is a real foot-gun:

  • flush() sends buffered data. Behavior varies by SDK (sometimes async, sometimes sync with a timeout).
  • forceFlush() is the OpenTelemetry term for “export everything in the queue right now.” It’s designed to be called repeatedly across invocations.
  • shutdown() tears down the entire telemetry pipeline. It flushes, then destroys the exporter. Calling it in AWS Lambda breaks tracing for all subsequent warm invocations.

Practitioners in the Langfuse community learned this the hard way. Their recommendation: “call only forceFlush on your LangfuseExporter instance at the end of each Lambda invocation, do not call shutdown on the OpenTelemetry SDK.” Calling shutdown works perfectly on the first invocation and then silently breaks every warm start after that.

For details on configuring flush behavior in a JavaScript client, including flushAt and flushInterval settings, see configuring buffering and flush.

The Root Problem: Handler Lifecycle vs Buffer Lifecycle

Here’s the sequence that causes silent data loss:

  1. Handler receives a request
  2. Your code records metrics, spans, or log events into the SDK’s buffer
  3. Handler sends a response to the client
  4. The runtime freezes or terminates the execution context
  5. The buffer, still full of unsent data, disappears

No error is thrown. No warning appears in your logs (because the logs themselves may have been buffered too). Your dashboard just shows gaps, or worse, shows partial data that misleads you into thinking everything is working.

This is what makes buffering and flush strategies for short-lived handlers a distinct problem from general observability. In a traditional server, you configure a batch interval and forget about it. In serverless, you have to actively manage the moment of flush relative to the handler’s lifecycle.

Platform-Specific Flush Strategies

Cloudflare Workers: ctx.waitUntil()

Cloudflare Workers provide ctx.waitUntil(), which extends the Worker’s lifetime beyond the response. You pass it a Promise, and the runtime keeps the Worker alive until that Promise settles.

export default {
  async fetch(request, env, ctx) {
    // Do your work, build a response
    const response = new Response("OK");

    // Flush telemetry without blocking the response
    ctx.waitUntil(metricsClient.flush());

    return response;
  }
};

The Statsig SDK documentation shows this pattern explicitly: ctx.waitUntil(Statsig.flush(1000)) ensures events are sent before the Worker terminates, without adding latency to the response.

Key constraints to know:

  • The Worker gets up to 30 seconds after the response is sent. If the flush Promise doesn’t settle in that window, it’s cancelled.
  • For mission-critical data, Cloudflare recommends sending messages to a Queue and processing them in a separate consumer Worker. This adds infrastructure but removes the 30-second ceiling.

PostHog takes a more aggressive approach for Workers: they recommend setting flushAt to 1 and flushInterval to 0, effectively disabling batching entirely. Every event gets sent immediately. This eliminates the risk of buffer loss at the cost of more network calls.

For a complete Workers instrumentation walkthrough, the Cloudflare Workers metrics quickstart covers the waitUntil pattern step by step.

Vercel and Next.js: after()

Vercel provides the after() utility, which schedules work to run after the response has been sent to the client. It’s conceptually similar to waitUntil() but designed for the Next.js ecosystem.

import { after } from 'next/server';

export async function POST(request) {
  const result = processRequest(request);

  after(async () => {
    await metricsClient.flush();
  });

  return Response.json(result);
}

Practitioners on dev.to have documented this pain point clearly: “When implementing custom metrics with @vercel/otel in Next.js Route Handlers, you might notice that metrics aren’t being properly reported. This happens because Route Handlers complete their execution before the metrics have a chance to be flushed.”

The LangSmith SDK docs show the same pattern, calling after(async () => { await client.awaitPendingTraceBatches(); }) to ensure traces survive handler termination.

Distlang’s Vercel metrics integration guide walks through this pattern with concrete examples.

AWS Lambda: Flush Before Return

Lambda is the most constrained environment for flush strategies. There’s no waitUntil() equivalent. The runtime can freeze immediately after your handler function returns, and any background threads or pending Promises may never execute.

The standard pattern is straightforward but adds latency:

export const handler = async (event) => {
  // Process the event
  const result = doWork(event);

  // Flush before returning, this blocks
  await metricsClient.flush();

  return {
    statusCode: 200,
    body: JSON.stringify(result),
  };
};

A critical detail from the Kafka producer documentation reinforces the discipline: “When using a Kafka producer from within AWS Lambda, it’s important that you execute KafkaProducer#flush as the last action Lambda takes before exiting.” But they also warn to “call flush only once at the end of the handler method and not after each call to send.”

Initialize your SDK and telemetry pipeline outside the handler function so they persist across warm invocations. Call forceFlush() at the end of each invocation. Never call shutdown().

General JS SDK Configuration: flushAt and flushInterval

Most JavaScript analytics and metrics SDKs expose two configuration knobs:

  • flushAt: The number of events that triggers an automatic flush (e.g., send when the buffer has 20 items)
  • flushInterval: The time interval between automatic flushes (e.g., every 10 seconds)

For short-lived handlers, both defaults are usually wrong. A flushAt of 20 means you need 20 events per invocation before an automatic flush happens. A flushInterval of 10 seconds means the handler is probably long dead before the timer fires.

The two options: either lower both values aggressively (PostHog’s flushAt: 1, flushInterval: 0 approach), or keep reasonable defaults and call flush explicitly at the end of each invocation. The explicit flush approach is generally better because it preserves batching efficiency while still guaranteeing delivery.

BatchSpanProcessor vs SimpleSpanProcessor

This is where buffering and flush strategies for short-lived handlers intersect directly with OpenTelemetry configuration. It’s a decision that trips up a lot of developers.

SimpleSpanProcessor sends each span to the exporter the moment it finishes. No buffering, no batching. Every span triggers a network call.

BatchSpanProcessor collects spans in a queue and exports them in batches. The default configuration uses a max queue size of 2,048, a schedule delay of 5,000 ms, and a max export batch size of 512.

The intuition for serverless is “just use SimpleSpanProcessor, it’s safer.” And for very simple handlers that produce 1 to 3 spans per invocation, that’s fine. One practitioner on a popular OpenTelemetry blog put it directly: “Edge functions have short lifetimes. A batch processor might not flush before the function terminates, causing you to lose telemetry. The simple processor sends each span immediately.”

But the official OpenTelemetry Javadoc actually pushes back: “Most backends will not perform well with a single span per request so unless you know what you’re doing, strongly consider using BatchSpanProcessor instead, including in special environments such as serverless runtimes.”

The practical resolution: use BatchSpanProcessor with an explicit forceFlush() call at the end of each handler invocation. You get batching efficiency on warm invocations that handle multiple requests, and you get guaranteed delivery because the flush happens before the handler exits. This is the middle ground that works for most teams.

One thing to watch: when the queue reaches maxQueueSize, new spans are dropped. This protects against memory exhaustion during traffic spikes, but it means you could lose data silently if your handler produces more spans than the queue can hold. Size your queue appropriately.

For more on how batching interacts with payload formats and ingestion limits, see batching payload and size limits.

Common Anti-Patterns

Calling shutdown() instead of forceFlush()

Already covered above, but worth repeating because it’s the most common mistake. shutdown() is for application exit, not for per-invocation cleanup. In Lambda, calling shutdown() destroys the telemetry pipeline, and every subsequent warm invocation runs without tracing.

Flushing inside the response path

If you await flush() before sending the response (rather than using waitUntil() or after()), you add network latency directly to your user-facing response time. For some use cases this is acceptable. For most, it’s not. Cloudflare’s best practices are explicit: “Use waitUntil for metrics. Do not block responses for telemetry.”

Never flushing at all

This is the default behavior if you drop a standard SDK into a serverless function without any configuration changes. The SDK buffers events, the handler terminates, and the buffer evaporates. No errors, no warnings, just missing data.

Flushing per-record instead of per-invocation

The opposite extreme. Calling flush after every single record() or send() call eliminates all batching benefits and generates a burst of network calls. Flush once at the end of the invocation, not after each data point.

Concurrent flush race conditions

A production user reported a Datadog race condition in a GitHub issue where the default periodic flush strategy conflicted with the workers handling samples. It happened “a few times per million invocations and brings the whole Lambda down with it.” Their fix was setting DD_SERVERLESS_FLUSH_STRATEGY to end, removing the concurrent non-synchronized flushing. The lesson: in serverless, flush at the end, not on a timer.

Queue-Based Offload: The Nuclear Option

When waitUntil() and after() aren’t reliable enough, the answer is to decouple the flush entirely. Instead of exporting telemetry directly from the handler, you write events to a durable queue (Cloudflare Queues, AWS SQS) and process them in a separate consumer.

This eliminates the lifecycle coupling. The handler’s only job is to write a message to the queue, which is fast and reliable. A separate worker or Lambda processes the queue and handles the actual export.

The trade-off is infrastructure complexity. You now have two services to manage, a queue to configure, and retry logic to think about.

The Unkey team experienced this complexity firsthand. As reported by InfoQ, they eventually rebuilt from Cloudflare Workers to Go specifically because of what they called “the complexity tax of working around statelessness.” They built a custom proxy called chproxy to buffer analytics events before sending them to ClickHouse, which performed poorly with thousands of small inserts coming from individual Worker invocations. This is an extreme case, but it illustrates how flush complexity can compound into architectural decisions.

Decision Guide: Choosing the Right Flush Strategy

Strategy Platform Blocks Response? Best For
ctx.waitUntil(flush()) Cloudflare Workers No Most Workers use cases, up to 30s of post-response work
after(async () => flush()) Vercel / Next.js No Route Handlers and Server Actions with telemetry
await flush() before return AWS Lambda Yes Lambda where no post-response execution exists
flushAt: 1, flushInterval: 0 Any platform Depends Ultra-simple handlers producing very few events
BatchSpanProcessor + forceFlush OTel on any platform At handler end Production OTel setups that need batching efficiency
Queue-based offload Workers + Queues, Lambda + SQS No Mission-critical telemetry where no data loss is acceptable

Start with the simplest strategy that matches your platform. For Cloudflare Workers, that’s waitUntil(). For Vercel, it’s after(). For Lambda, it’s an awaited flush before return. Move to queue-based offload only when you’ve confirmed that the simpler approach isn’t meeting your reliability requirements.

Prometheus’s pull model, where a scraper fetches metrics from your service, doesn’t work at all in serverless. As Prometheus community members have noted, “true FaaS means that the function calls are lightweight. The additional overhead of sending anything over the network defeats that purpose.” Push-based metrics with proper flush timing is the only model that fits.

If you want to skip the complexity of building this infrastructure yourself, Distlang Metrics is an API-first serverless metrics service designed for exactly these short-lived handler patterns, with built-in support for Workers and Vercel flush semantics.

Putting It All Together

Getting buffering and flush strategies for short-lived handlers right is the difference between having observability and thinking you have observability. The data loss is silent. The gaps in your dashboards look like low traffic, not broken instrumentation.

The core principles are simple:

  1. Understand that your handler’s lifecycle is shorter than your SDK’s default flush interval
  2. Pick the flush mechanism that matches your platform
  3. Flush once per invocation, at the end, not per-record and not on a timer
  4. Use forceFlush(), never shutdown(), in environments with warm starts
  5. Escalate to queue-based offload only when needed

For a hands-on walkthrough that takes you from zero to a working metrics dashboard, the end-to-end example covers instrumentation through visualization in minutes.

Ready to start collecting metrics from your serverless handlers? Get started with Distlang Metrics and skip the flush plumbing.

FAQ

Why do my metrics disappear in serverless functions even though my SDK works fine locally?

Locally, your development server is long-running, so the SDK’s default background flush timer has time to fire. In serverless, the function terminates after sending the response, and the runtime cancels any pending async work. The buffer, still full of your metrics, gets garbage collected. You need an explicit flush tied to the handler’s lifecycle.

What’s the difference between forceFlush() and shutdown() in OpenTelemetry?

forceFlush() exports all queued spans and metrics immediately but keeps the telemetry pipeline intact for future invocations. shutdown() flushes and then tears down the entire pipeline. In Lambda or any environment with warm starts, calling shutdown() means subsequent invocations run without telemetry. Always use forceFlush() in serverless.

Should I use SimpleSpanProcessor or BatchSpanProcessor in serverless?

For most production workloads, use BatchSpanProcessor with an explicit forceFlush() at the end of each invocation. SimpleSpanProcessor is acceptable for very simple handlers producing 1 to 3 spans, but OpenTelemetry’s own documentation warns that most backends don’t perform well with single-span requests. The batch-plus-flush approach gives you efficiency and reliability.

Does ctx.waitUntil() guarantee my data will be sent?

Not unconditionally. Cloudflare Workers gives you up to 30 seconds after the response for your waitUntil() Promises to settle. If the flush takes longer than that, the Promise is cancelled. For mission-critical data, consider routing events through Cloudflare Queues instead.

What does “flushAt: 1” mean and when should I use it?

Setting flushAt to 1 tells the SDK to send data after every single event, effectively disabling batching. PostHog recommends this for Cloudflare Workers paired with flushInterval: 0. It works but generates more network calls. It’s a reasonable choice for low-traffic handlers where simplicity matters more than efficiency.

How do I flush telemetry in AWS Lambda without slowing down responses?

You can’t avoid some latency in Lambda because there’s no post-response execution mechanism like waitUntil(). The standard approach is to await flush() as the last operation before returning. To minimize the impact, keep your flush timeout short (1 to 2 seconds) and initialize the SDK outside the handler so it persists across warm invocations.

Can I use Prometheus for serverless metrics?

Prometheus uses a pull model where it scrapes metrics endpoints at regular intervals. Serverless functions aren’t continuously running, so there’s nothing to scrape. You need a push-based metrics system where the handler sends data to an ingestion endpoint during execution. This is exactly the pattern that flush strategies for short-lived handlers address.

What’s the queue-based offload pattern and when is it worth the complexity?

Instead of exporting telemetry directly from the handler, you write events to a durable queue (Cloudflare Queues, AWS SQS) and process them in a separate consumer. This decouples the flush from the handler lifecycle entirely. It’s worth the added infrastructure when you absolutely cannot tolerate data loss, such as billing metrics or compliance-sensitive audit logs. For most observability use cases, waitUntil() or after() with a direct flush is sufficient.