Sucuri Docs > Website Firewall > Troubleshooting > Investigating Cache Headers

Investigating Cache Headers

March 27, 201910 min read Nate Chaddock

Your origin server sends HTTP headers that tell caches whether a page can be stored and for how long. The Sucuri Firewall follows some of these headers and ignores others, depending on the Cache Level you’ve selected in the dashboard.

  • Site Caching and Disabled: The firewall respects your server’s cache headers.
  • Enabled and Minimal: The firewall uses its own fixed cache times and ignores most of your server’s cache headers.

This article shows you how to check your headers, what each one does on the firewall, and how to make a page cache (or not cache) the way you want.

How to check the cache headers

You can use curl or your browser’s Developer Tools (Network tab). On Microsoft® Windows, install Git for Windows first and use its terminal.

Run this command three times in a row, replacing the URL with the page or file you want to check:

curl -s -o /dev/null -D - https://yourdomain.com/your-page/

Example response (third run):

HTTP/2 200
server: Sucuri/Cloudproxy
content-type: text/html; charset=UTF-8
cache-control: max-age=900
x-sucuri-id: 14002
x-sucuri-cache: HIT

Why three times? The firewall stores a page only after it has been requested twice, so a cacheable page shows MISS on the first and second request and HIT from the third onwards. A single MISS doesn’t mean the page can’t be cached.

The two Sucuri headers to look at first

  1. X-Sucuri-Cache tells you what the firewall cache did with that request:

Value

Meaning

HIT

Served from the firewall cache.

MISS

Fetched from your server. Normal for the first two requests of a cacheable page, or every time for a page that is never cached.

BYPASS

The firewall skipped the cache on purpose for this request (see “Requests that are never cached” below).

EXPIRED

The cached copy was too old, so a fresh copy was fetched from your server.

REVALIDATED

The cached copy was too old, but your server confirmed it hasn’t changed, so it was reused.

UPDATING / STALE

An older cached copy was served while a fresh one is fetched, or because your server is responding with errors or timing out. This keeps your site online during origin problems.

(no header)

The request can’t be cached (for example, a POST form submission), or the firewall blocked it.
2. X-Sucuri-ID is the firewall node that answered. Each node has its own cache, so two requests can show different results if they reach different nodes.

Cache levels and cache times

When your server doesn’t set a cache time, or when the Cache Level ignores it, the firewall keeps pages for:

Response

Enabled

Minimal

Site Caching

Disabled

Page (200)

180 minutes

8 minutes

180 minutes, or your max-age

1 minute, or your max-age

Not found (404)

4 minutes

2 minutes

4 minutes

1 minute

Permanent redirect (301)

180 minutes

15 minutes

180 minutes

10 minutes

Temporary redirect (302)

10 minutes

5 minutes

10 minutes

10 minutes
Note: Disabled still caches pages for 1 minute for visitors who have no cookies. Any request that carries a cookie is never cached on Disabled.

Static files (images, CSS, JavaScript, fonts, etc.) are handled the same way on every Cache Level. See “Static files” below.

How each header affects the firewall cache

Cache-Control

This is the most important cache header.

  • max-age=N (or s-maxage=N): on Site Caching, the firewall keeps the page for N seconds. Disabled honors max-age the same way. Enabled and Minimal ignore it and use their fixed times (180 and 8 minutes). The header is passed on to the visitor’s browser, so it also controls browser caching. s-maxage applies only to shared caches such as the firewall, which makes it useful when you want the firewall to keep a page for a different time than browsers do.

    You can use a short max-age as a micro-caching layer for frequently updated content such as news sites, wikis or busy blogs. See step 8 of the Troubleshoot Caching Issues article.

  • private, no-store, no-cache or max-age=0: on Site Caching and Disabled, the page is not cached by the firewall. Enabled and Minimal ignore these and still cache the page. Use private, no-store, no-cache, must-revalidate for pages that belong to one visitor, such as account or cart pages, so browsers don’t keep a copy either.

Expires

Expires sets a date after which the content is stale. On Site Caching and Disabled, a date in the past (for example Expires: Thu, 19 Nov 1981 08:52:00 GMT) prevents the page from being cached. Enabled and Minimal ignore Expires. If you send both, Cache-Control: max-age takes precedence over Expires.

Cookies usually identify a visitor’s session.

  • On Site Caching and Disabled, a page that sends Set-Cookie is not cached.

  • On Enabled and Minimal, Set-Cookie does not prevent caching. The page is cached together with its cookie, and that cached copy, cookie included, is served to other visitors.

Important: if your site sets a per-visitor cookie on a page (sessions, shopping carts, login or CSRF tokens) and uses Enabled or Minimal, add that page to Non-Cache URLs in the dashboard. Alternatively, switch to Site Caching and send Cache-Control: private from your server for those pages.

Vary

  • Vary: Accept-Encoding: always safe to send. You don’t need to compress responses for the firewall: it compresses content (gzip or Brotli) for each visitor’s browser automatically.

  • Vary: User-Agent: on Site Caching, the firewall keeps a separate copy for each different User-Agent. Enabled ignores it. Mobile and desktop visitors are already cached separately by the firewall, so you don’t need this header to serve a separate mobile version. Don’t use it on responsive sites: since almost every browser version has a different User-Agent, it greatly reduces your cache hit rate.

  • Vary: Cookie: this does not prevent the firewall from caching a page. To stop a page from being cached, use the options in “Preventing a page from being cached” below.

  • Vary: *: prevents caching on Minimal, Site Caching and Disabled. Enabled ignores it. Avoid it unless you really want no caching, because every request then goes to your server.

Pragma

Pragma: no-cache is an obsolete HTTP/1.0 header. The firewall ignores it at every Cache Level, so it won’t stop a page from being cached. Use Cache-Control instead.

ETag and Last-Modified

When a cached copy expires, the firewall uses ETag and Last-Modified to ask your server whether the content changed. If it hasn’t, the cached copy is reused (X-Sucuri-Cache: REVALIDATED) without downloading the page again. The firewall also answers browsers’ If-Modified-Since requests with 304 Not Modified when the content hasn’t changed. Sending these headers is recommended.

Requests that are never cached

Whatever your Cache Level or headers, the firewall never serves these from the cache (X-Sucuri-Cache: BYPASS):

  • Logged-in visitors, recognized by common login and session cookies such as WordPress (wordpress_logged_in_…, comment_author_…), WooCommerce (woocommerce_items_in_cart), phpBB, vBulletin, XenForo and ASP.NET (.ASPXAUTH).

  • Requests with an Authorization header, such as HTTP Basic authentication or API tokens.

  • Non-Cache URLs you configured in the dashboard. These apply to pages, not to static files.

  • Sensitive pages, by URL keyword: any URL whose path contains wp-admin, wp-login, admin, login, logout, register, cart, checkout, orders, customer, account, profile, private, search, forms or captcha, among others. Because this is a “contains” match, a URL such as /private-events/ is also never cached.

  • If your IP address is listed within the Developer Mode settings.

  • Form submissions and other non-GET requests (POST, PUT, DELETE…).

Static files

Images, CSS, JavaScript, fonts, audio and video, PDFs and office documents, archives, and .txt / .xml files are handled the same way on every Cache Level:

  • They’re cached on the firewall for up to 3 days. They’re cached even if you selected Disabled or added them to Non-Cache URLs, and your server’s Cache-Control, Expires, Vary and Set-Cookie headers are ignored for them.

  • The firewall tells browsers to cache them for 1 day (Cache-Control: max-age=86400). This replaces any Cache-Control your server sends for these files.

  • Sitemaps and feeds (for example sitemap.xml, sitemap_index.xml, feed.xml, rss.xml, atom.xml) are cached for only 1 hour in browsers, so search engines see updates sooner.

  • robots.txt, ads.txt, app-ads.txt and security.txt are cached for only 5 minutes, so changes show up quickly.

  • Any Set-Cookie your server sends with a cached static file is removed before the file reaches the visitor.

To keep a specific static file out of the cache, see step 4 of the Troubleshoot Caching Issues article.

Final notes

Preventing a page from being cached

  • Site Caching or Disabled: send any one of these headers from your server:

    • Cache-Control: private (recommended: Cache-Control: private, no-store, no-cache, must-revalidate)

    • Cache-Control: no-store, no-cache or max-age=0

    • Expires with a date in the past

    • Set-Cookie

  • Enabled or Minimal: your server’s headers can’t prevent caching (except Vary: * on Minimal). Add the page to Non-Cache URLs in the dashboard.

  • Pragma: no-cache and Vary: Cookie do not prevent caching on any Cache Level.

Caching a page for a specific time

To make the firewall cache a page for a set amount of time, use the Site Caching mode in the firewall settings. If the caching level is set to Enabled or Minimal, the firewall ignores max-age and uses the fixed cache times from the table above.

Have your server send a response header such as this:

Cache-Control: max-age=900

Don’t include any of the headers listed above that prevent caching. If you do, they override max-age and the page won’t be cached.

With this header, the firewall keeps the page for 900 seconds (15 minutes). You can change the number to set a different duration.

Note

  • The cache works on demand: a page or file that nobody requests for 72 hours is removed from the cache, even if its cache time is longer.

  • After changing your headers or settings, use Clear Cache in the dashboard so the new behavior applies right away. Then check with the curl command above (three runs).

  • Caching is one of the most important ways to speed up your website, save server resources and improve your site’s resilience against DDoS attacks.

Was this article helpful?

Popular Articles