PRICING SUPPORT LOGIN

Cache headers will only be respected when using "Site Caching" or "Disabled" as Cache Level.

In order to analyze the cache headers, a tool called curl will be used. You can also use the Developer Tools of your browser.

If you are using Microsoft® Windows, install Git for Windows before proceeding.

Open the Terminal/Git for Windows and run the following command:

curl -IL http://yourdomain.com/

Change "yourdomain.com" for your domain or the complete URL of the page/file. For example, if we want to check Sucuri Blog's front page http headers:

$ curl -IL https://blog.sucuri.net/
HTTP/1.1 200 OK
Server: Sucuri/Website Firewall
Date: Thu, 19 Jan 2017 15:18:31 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
Set-Cookie: PHPSESSID=aha8r2fs3m0njv8h3be14hbtt2; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Cookie
Link: <https://blog.sucuri.net/wp-json/>; rel="https://api.w.org/"
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-Sucuri-ID: 11005

Here's the lines we need to pay attention for the purpose of this article:

Vary: Accept-Encoding
Set-Cookie: PHPSESSID=aha8r2fs3m0njv8h3be14hbtt2; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Cookie

Sucuri's blog uses "Enabled" cache level, so these http headers are completely ignored. However, if it were using "Site Caching" cache level, those headers would be honored and as result Sucuri's blog would never be cached by the Firewall.

Do you know why? Let's understand a bit more about how cache headers works.

Cache Headers

- Vary

It's a powerful header that is frequently used incorrectly. Here's a list of the possible values:

  • Vary: Accept-Encoding

When the origin server (your server) doesn't send this header, two things can happen:

1) If the content isn't compressed, you will spend more bandwidth, but all browser will be able to render the page.
2) If the content is compressed, but there isn't a Vary: Accept-Encoding header, older browsers won't be able to render the content.

To avoid that kind of issue, the origin server must send the Vary: Accept-Encoding when the content is compressed. That way Sucuri Firewall will keep two separate version of your content: one without compression and other with compression. Depending on the browser, Sucuri Firewall will serve the right version, saving bandwidth, speeding your website and keeping compatibility to older browsers.

  • Vary: Cookie

This header usually prevent authenticated users from seeing cached version of pages with Set-Cookie as guide. However, due to security reasons, we won't cache any page that has "Vary: Cookie" and "Set-Cookie" as http header. It'll basically have the same effect as "Cache-Control: private, no-cache" on Sucuri Firewall, therefore your cache rate will not be great.

  • Vary: User-Agent

If your website has a mobile version, this header can help you with your SEO. It'll make obligatory that cache servers create different versions of the pages depending on the User Agent.

Attention: Mobile version is different from Responsive design. Responsive design is interpreted differently by the browser, but the page content is exactly the same for all user agents. You shouldn't use this header for responsive designed websites.

  • Vary: Referrer

This header will instruct the browser to re-check with the sever for each different referrer. It isn't recommend, unless you know exactly what you are doing.

  • Vary: *

Don't use it. It'll prevent any sort of caching and increase (a lot) the origin server load.

- Set-Cookie

Cookies are usually used to identify a user session. Therefore if Set-Cookie is present, the Firewall will not cache the page. In case your page is being cached even with Set-Cookie present, please also add "Vary: Cookie" to make sure it'll not be cached.

- Cache-Control

It is the most common and important cache header. It has 2 possible values: public and private.

  • public

Mainly used for static content and public pages (guests only), public always comes with max-age=X (X is seconds) to control the cache expiration. Actually, when you specify the max-age=X, it won't be necessary to include public, as it is implicit.

It's specially useful for event-driven content websites, like Wikis, news portal, highly updated blogs, etc. You can use Sucuri Firewall cache as a micro-caching layer as explained on 8º step of Troubleshoot Caching Issues article.

For websites that doesn't updated its public content frequently, it is even possible to use a longer max-age.

To have even a better control, you can also use "s-maxage". The Shared Max Age is a specific max-age only for reverse caching or other public proxy servers and is generally set lower than Max Age.

If your server doesn't provide a Cache-Control header specifically for static content, like images, js, css, swf, mp3, mp4, pdf or fonts, our system will automatically set Cache-Control:"max-age=315360000" for those files.

Side note: the Firewall will check and revalidated (if necessary) those files every 3 days. However, static files will always be cached, regardless of the cache level or the non-cached URLs. Check 4º step of Troubleshoot Caching Issues article in order to know how to avoid static content cache.

  • private

It tells to the cache server (Sucuri Firewall) not to cache at all the content. You must combine with no-store, no-cache, must-revalidate, post-check=0, pre-check=0 to also avoid browser caching. It is commonly used for authenticated users sections and to prevent dynamic content caching.

- Pragma

It's basically a "obsolete" cache-control, used for HTTP/1.0 requests. It has two variables of self-explained meaning: cache and no-cache. It's still in use because older browser may not support the newest protocols, like HTTP/1.1.

- Expires

After the date Expires header explicits, the browser have to request the newest copy of the content. It won't be considered valid after this date, but until there, the local copy should be used. It's expected that Expires matches the max-age value of Cache-Control, but it's not necessary.

Just like Cache-Control, if your server doesn't provide an Expires header, our system will automatically set the cache lifetime of the page to the Caching Level default value.

Final Notes

Caching is one of the most important techniques available to speed your website, save resources and improve your website's resilience against DDoS attacks.

If you don't want to cache a page, you can either use "Vary: *", "Cache-Control: private", "Pragma: no-cache" or "Expires: Thu, 19 Nov 1981 08:52:00 GMT" for example, such as in the example before:

Vary: Accept-Encoding
Set-Cookie: PHPSESSID=aha8r2fs3m0njv8h3be14hbtt2; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Vary: Cookie

Just one of them would make the Firewall not cache the page if you were using Site Caching or Disabled more.

On the opposite, if you do want to cache a page, make sure none of the headers mentioned here are set to stop caching. You could use just "Cache-Control" for example:

Cache-Control: max-age=900

This header alone, without the others to interfering, will instruct the Firewall to cache your page for 900 seconds.

Keep in mind: If the page is not accessed in hours, even if you set a really long cache lifetime, it'll expire from the caching. Sucuri Firewall caching happens on demand and retain only the frequently accessed assets/pages.

ETag and Last-Modified headers weren't mentioned because they don't affect the Sucuri Firewall cache behavior.