No caching headers sent on some (all?) HTML files

[ttencate]

Godot version

4.1.1

System information

Linux

Issue description

https://godotengine.org/download/linux/ only has 4.1, whereas the other platforms already have 4.1.1.

https://github.com/godotengine/godot/releases/tag/4.1.1-stable does have Linux builds, so maybe something went wrong updating the website?

Steps to reproduce

n/a

Minimal reproduction project

n/a

[AThousandShips]

The page has 4.1.1, have you updated the page?

[ttencate]

It has 4.1, not 4.1.1 which was released yesterday :)

[YuriSizov]

Try refreshing your page without cache (Ctrl-R, Ctrl-F5). The page is definitely up to date, it's just somewhere between yourself and the server the data got cached.

[ttencate]

That works! Pity it destroyed the evidence, but maybe the caching is overly aggressive?

[YuriSizov]

but maybe the caching is overly aggressive?

We use CloudFlare and it should start showing you an updated version of the page within 24 hours. So unless your browser was unlucky enough to cache the page close to it being updated, you should see it within reasonable time.

[ttencate]

Strange though, I'm not getting any cache headers back from CloudFlare:

$ curl -I https://godotengine.org/download/linux/
HTTP/2 200 
date: Tue, 18 Jul 2023 13:35:14 GMT
content-type: text/html
last-modified: Mon, 17 Jul 2023 18:17:14 GMT
vary: Accept-Encoding
x-frame-options: SAMEORIGIN
cf-cache-status: DYNAMIC
report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=CNjtQOj9N7uzSJ57ETYTcLduHp05C2CatB7IyeTBQGrStWihfoXTd0FEh9AtkfQJuWhTTq%2BZeDIpaqg8LNJiWLMIjWcg3Go2Zgq17CDEwJWlqzhhWyGR%2B59%2FkslJUPWGlcLFyCRKRIeQUZwnxmg%3D"}],"group":"cf-nel","max_age":604800}
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
server: cloudflare
cf-ray: 7e8b1b51cb4cfa48-AMS
alt-svc: h3=":443"; ma=86400

This means it's left up to the browser, which is usually pretty aggressive and can be unpredictable. This explains what I encountered. The only advantage is that no request is sent to the server at all, not even just headers — but even F5 may show you an outdated page.

On the meaning of the response header CF-Cache-Status: DYNAMIC:

Cloudflare does not consider the asset eligible to cache and your Cloudflare settings do not explicitly instruct Cloudflare to cache the asset. Instead, the asset was requested from the origin web server. Use Page Rules or Cache Rules to implement custom caching options.

Maybe consider sending an ETag header from the origin server? Then the browser can validate cheaply whether the document has changed, without having to download it again if it already has a valid copy in its cache.

Or just set Cache-Control: no-cache on HTML pages since they're usually pretty small anyway (20 kB in this case).

At the very least, you can take control of the caching duration using Cache-Control: max-age=3600 (for example), instead of leaving it up to the browser's whims.