Cloudflare Adds Configurable HTTP Vary Header Support to Prevent Cache Thrashing

Cloudflare Adds Configurable HTTP Vary Header Support to Prevent Cache Thrashing - InfoQ

InfoQ Homepage News Cloudflare Adds Configurable HTTP Vary Header Support to Prevent Cache Thrashing

Development

InfoQ Certified AI-Assisted Engineering Program (online, Oct 19): Build the harness that holds.

Cloudflare Adds Configurable HTTP Vary Header Support to Prevent Cache Thrashing

Sep 29, 2026 3 min read

by

• Olimpiu Pop

#### Follow us on

Youtube

232K Followers

Linkedin

26K Followers

Instagram

New

RSS

19K Readers

X

57.1k Followers

Facebook

21K Likes

Bluesky

Listen to this article -

0:00

Audio ready to play

Your browser does not support the audio element.

Normal

1.25x

1.5x

Like

new dropdown rading list

• Reading list

Cloudflare has released native support for the HTTP Vary response header across its global edge network, making the capability available within Cache Rules on all customer plans. The enhancement enables origin servers to negotiate representations such as localised text, modern image encodings, or alternate payloads while providing platform operators with fine-grained controls to prevent severe cache fragmentation.

Historically, intermediary content delivery networks have treated the HTTP Vary header with caution. Under standard HTTP semantics, the Vary response header indicates to downstream caches that an origin server evaluated specific request headers before generating a response. If a cache respects Vary without transformation, slight syntactic differences in client headers, such as variations in whitespace, casing, or client-preferred language priority orderings, can splinter a single resource into dozens of distinct variants. In an empirical audit of over 120 million HTTP responses across approximately 50,000 top domains, Cloudflare discovered that nearly 3,000 origins varied on four or more request headers, with extreme cases varying across dozens of distinct fields.

Uncontrolled header variance frequently causes cache thrashing, where hit ratios collapse and origin load escalates. To mitigate this tension between correctness and edge caching efficiency, Cloudflare decoupled the negotiation process into two operational stages. The origin server continues to specify which request headers influence response generation, while Cloudflare Cache Rules dictate how the edge processes, normalizes, or ignores the values of those headers before computing the variant key.

When configuring Vary handling within Cache Rules, operators can define behaviours per header or apply fallback policies across unlisted fields. The engine exposes three distinct actions: normalise, pass through, and bypass. Normalisation canonicalises complex negotiation headers like Accept and Accept-Language into standardised, equivalent classes, eliminating incidental client differences. Pass through stores exact, case-sensitive strings, which is necessary when origin applications depend on precise token matches. Bypass skips edge caching entirely for requests matching volatile or high-cardinality headers, ensuring non-deterministic variants never consume edge cache capacity.

An origin server serving multi-representation endpoints can advertise its dynamic dependencies using standard headers:

HTTP/1.1 200 OK

Content-Type: application/json

Cache-Control: public, max-age=3600

Vary: Accept, Accept-Language

Engineers configure Cloudflare to match these requests and manage header variability using the Cloudflare Ruleset Engine or the dashboard. The following JSON configuration illustrates a Cache Rule specifying normalization for content negotiation while routing unknown variants to pass through:

{
    "expression": "(http.request.uri.path eq \"/api/catalog\")",
    "action": "set_cache_settings",
    "action_parameters": {
        "cache": true,
        "vary": {
            "headers": [
                {
                    "name": "accept",
                    "action": "normalize"
                },
                {
                    "name": "accept-language",
                    "action": "normalize"
                }
            ],
            "default_action": "pass_through"
        }
    }
}

Under the hood, Cloudflare records the Vary fields announced by the origin alongside the base cache key, which consists of the scheme, host, and path. Subsequent incoming requests execute the base key lookup first. If variants are registered for that key, the edge extracts the corresponding request headers, evaluates them against the designated Cache Rule actions, and serves the matching variant. If no cached variant matches the normalised criteria, the request is forwarded to the origin, and the resulting payload is indexed into the variant pool.

Product manager Alex Krivit and systems engineer Zaidoon Abd Al Hadi highlighted that this two-step architecture resolves longstanding limitations of manual workarounds. Previously, teams had to choose between disabling caching, relying on specialised Cloudflare Workers, maintaining brittle custom cache keys that anticipated headers before seeing the origin response, or using specialised extensions like Vary for Images . Custom cache keys evaluate rules upfront on incoming requests, meaning they apply even when an origin returns static, invariant responses. By contrast, Vary rules in Cache Rules trigger dynamically only when the origin includes a Vary header, avoiding needless key space expansion for unvarying responses.

While the new mechanism reduces implementation friction for content negotiation, operators must evaluate cache key cardinality before enabling full pass-through modes on high-entropy headers. When headers containing arbitrary tokens or session identifiers are passed through, each distinct client string creates an isolated edge entry, reducing cache lifetime and increasing cache eviction churn. Vary support in Cache Rules is currently active for all Free, Pro, Business, and Enterprise zones.

main wrapper for authors section

About the Author

section title

main wrapper for each author

#### Olimpiu Pop

Show more

Show less

#### This content is in the Cloudflare topic

##### Related Topics:

• Development

• HTTP

• Cloudflare

• Related Editorial

• Related Sponsors

• Related Sponsor October 8, 2026, 12 PM EDT When AI Accelerates Development, Can Your CI Pipeline Keep Up? Presented by: Eric Metaj - Product Marketing Manager, Software Delivery at Datadog, and Rohin Chandra - Product Manager, CI/CD Optimization at Datadog

The InfoQ Newsletter

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

We protect your privacy.