Memcached VS Varnish

Memcached: A simple, open-source, distributed memory object caching system primarily used for caching strings. Best suited for lightweight, non-persistent caching needs.

Oh, thank you! I didn't realize that LRU Maintainer Thread was more than an expiration reaper. When it was first being introduced that was its first responsibility as lazy expiration removal by size eviction meant dead entries wasted capacity. It was all work in progress when I had read about it [1] and talked to dormando, so it got fuzzy. The compat code [2, 3] might have also thrown me off if I only looked at the setting and not the usage. Its a neat variant to all of these ideas.

[1] https://github.com/memcached/memcached/pull/97

stores session state in a session store like Memcached or Redis.

Django supports using Memcached as a cache backend. Memcached is a high-performance, distributed memory caching system that can be used to store cached data across multiple servers.

In server-side authentication, the session state is stored on the server-side, which can be scaled horizontally across multiple servers using tools like Redis or Memcached.

Caching - while it's not possible to cache everything, there's always a large percentage of your website / app that can be cached for an hour or ten minutes or 1 day etc... - all depends on the type of content but the longer you can cache for without negatively effecting content quality - the better. A good caching server example would be redis : https://redis.io/ or https://memcached.org/

If you really care about optimising this, you need, as other traders pointed out, a cache. Caches are a way of ensuring that the data you query stays in memory on a separate machine so you don't have the delay to disk & to commit. Things like memcached are created for this exact purpose. If you care about optimisation, look into it and other options. This is not a simple problem. Distributed systems like these are a whole area of work and research, so it won't be as simple as just swapping a DB, but if you care about performance, this is the path you have to go down eventually.

A couple of dedicated server-side resource caching solutions have emerged over the years: Memcached, Varnish, Squid, etc. Other solutions are less focused on web resource caching and more generic, e.g., Redis or Hazelcast.

MEMCACHED

In this case, caching mechanism is situated in the proxy server or reverse proxy server like Nginx, Apache, or Varnish, and most probably it is a part of ISP (Internet Service Provider).

A couple of dedicated server-side resource caching solutions have emerged over the years: Memcached, Varnish, Squid, etc. Other solutions are less focused on web resource caching and more generic, e.g., Redis or Hazelcast.

VARNISH

Edge Side Includes (ESI): a more modern alternative to SSI. ESI can handle variables, have conditionals, and supports better error handling. ESI is supported by caching HTTP servers such as Varnish.

For reads, caches are the primary tool, such as Varnish or memcached.

In this case, caching mechanism is situated in the proxy server or reverse proxy server like Nginx, Apache, or Varnish, and most probably it is a part of ISP (Internet Service Provider).

It sucks when this happens, but it's easily avoidable by using a caching frontend of some sort.

My favorite is Varnish,[0] which I have used with great success for _many_ web sites throughout the years. Even a web site that 10+ millions of requests per day ran from a single web server for a long time a decade-ish ago.

[0] https://varnish-cache.org/

> Is there some document somewhere that goes over the choices, overlaps, etc?

I believe Apache Traffic Server doesn't really compete with the likes of nginx or traefik. Apache Traffic Server is a HTTP caching server/web accelerator, thus it's specialized for caching HTTP requests and consequently it's simpler to deploy and configure. With that in mind, it competes with the likes of Squid[1] or Varnish[2].

[1] http://www.squid-cache.org/

[2] https://varnish-cache.org/

You need to be using Varnish Cache for the Varnish Cache plugin to do anything.

If you're worried about performance then put a cache in front of your server (such as https://varnish-cache.org/), and focus on providing validators in your representation metadata (etag, last-modified) to improve cache hits. This is going to give you much more improvement than tinkering with the ordering of these steps.