The story of 'stale-while-revalidate' HTTP Cache and missing headers

The Cache-Control HTTP header offers a wide range of possibilities for optimizing web applications. Two of its relatively interesting values are the stale-while-revalidate and stale-if-error attributes, described in the RFC 5861 specification. Support for the former attribute has already been implemented in the most popular browsers (except Safari) for 2 years now.

A quite interesting feature of the Cache-Control header and its stale-while-revalidate value is the ability to immediately use a response from the browser cache and initiate a refresh in the background. Especially, when dealing with many proxy servers, this option, at a low cost, can significantly improve UX (user experience).

The feature is great for standard resources such as style or script files. In the case of web services, we can additionally use the 'Vary' header if the answer may differ significantly depending on the request headers. Unfortunately, as it turns out, in the current version of Firefox, we may encounter some problems with the cache refresh query.

Revalidation request

In the case of Chromium-derived browsers: Chrome (97.0.4692.99), Edge (97.0.1072.76), and Chrome (83.0.4254.27), handling of the stale-while-revalidate attribute looks the same. The first query that meets the specification requirements returns the previously cached response. The revalidation query is a copy of the original one and refreshes the stale entry in the background for future use.

Firefox (96.0.3) works a bit differently here. While the revalidation query reaches the HTTP resource in the background, it is devoid of the original query headers, replaced by a default request.

If our web service returns a response depending on a header set by the application, the client may receive an unexpected response, e.g. in a different language, or even an error. But what if we apply the Vary response header appropriate to this situation? Well, Chromium-based browsers will behave as before.

Firefox, on the other hand, correctly detects the difference between the missing headers of the revalidation query and the original request. The cached response is returned. On the next query, however, the browser acts as if the memory for that query was empty and repeats the correct request. Finally, the most up-to-date response is returned.

In both cases, when using Firefox, you may encounter some unfortunate effects. You can receive a potentially incorrect response or lose the advantage of background revalidation on every second request. For now, it's worth keeping this in mind if you plan to use the HTTP cache. This behavior is particularly troublesome when combined with web services that depend on header values.

Here is the lambda function code imitating a web service followed by a simple test that you can run in your browser.

exports.handler = async function (event) {
    const lang = event.headers['accept-language'] || 'en'
    const vary = event['queryStringParameters'].vary
    const response = {
        statusCode: 200,
        headers: {
            "Cache-Control": "max-age=1, stale-while-revalidate=60",
            "Access-Control-Allow-Origin": "*",
            "X-Robots-Tag": "noindex",
        },
        body: (lang === "de" ? "Hallo Welt!" : lang === "pl" ? "Hallo Welt!!" : "Hello World!")
            + " " + new Date().getTime(),
    };
    if (vary) {
        response.headers["Vary"] = "Accept-Languag``e";
    }
    return response;
}

From my point of view, the revalidation query in the case of Firefox does not seem to comply with the general specification, especially with the fetch method specification, which I used in the test as an alternative to the XHR. If you use other clients, I encourage you to verify the implementation.

Out of curiosity, I've checked the HTTP Cache extension of the Apache HTTP Client 4.5.13 (Java) library. The behavior was consistent with what we've seen on the Chromium browsers. An especially noteworthy feature was the stale-if-error attribute, the support of which is currently not implemented in browsers at the time.