Owl Developer
API

Caching

How clients should store and refresh the catalog and media, with the reasons for each rule.

The goal is simple: a client should download a file only when it has changed, and never download the same image twice.

Two lifetimes

ResourceCache-ControlWhy
Catalogpublic, max-age=300, stale-while-revalidate=3600New servers appear within minutes. A stale catalog keeps working during an outage.
Mediapublic, max-age=31536000, immutableThe URL changes whenever the content changes, so the file can never go stale.

Refreshing the catalog

  1. Keep the last catalog you downloaded, with its version.
  2. On the next refresh, send If-None-Match with the stored ETag, if you have one.
  3. If the response is 304 Not Modified, keep what you have.
  4. If the response has a new body, compare its version with the stored one. Replace the stored catalog only when version changed.

The version field is the reliable signal. The HTTP ETag from the CDN is an optimization on top of it. The catalog's generatedAt changes with every build, so it must not be used to decide whether content changed.

Media

Store media by its hash. A cached file with the same sha256 as the current catalog entry is valid forever. Do not request media again to check it. Fetch media when a server is shown, not at startup, and keep the cache bounded in size.

Be a good citizen

  • Send a descriptive User-Agent that identifies your tool.
  • Do not poll the catalog more often than its five-minute lifetime. Refresh on demand, or on a long interval.
  • Do not download every image on startup.
  • Retry failed requests with backoff, not in a tight loop.

On this page