Redis launches managed semantic cache for LLM applications
Redis has introduced LangCache, a fully managed semantic caching service designed to reduce repeated calls to large language models. It compares incoming prompts with earlier requests by meaning rather than exact wording and can return a stored answer when the similarity is high enough. Redis claims savings of up to 90% on API costs and cache-hit responses up to 15 times faster than a new model request.
LangCache is available as a public preview on Redis Cloud through a REST API, with Python and JavaScript SDKs. An application searches the cache before contacting its chosen LLM; after a miss, it stores the new prompt and response for later use. The service handles embeddings and supports configurable similarity thresholds, TTLs, eviction policies and adaptive controls. It is designed to work with any LLM provider and language.
In a demonstration, direct inference took 2.232 seconds and used 514 input and 250 output tokens, while a cached response took 0.37 seconds with no LLM tokens. Redis says results depend on traffic patterns and require monitoring for incorrect matches, along with suitable expiration and tenant-isolation settings.
Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.