Sakhanda Wire
NVDA $153.73 -1.54% MSFT $506.30 -2.26% GOOGL $280.09 -1.28% META $619.28 -1.14% AMZN $256.62 -0.44%
← Back to the news

Redis launches managed semantic cache for LLM applications

Redis has introduced LangCache, a fully managed semantic caching service designed to reduce repeated calls to large language models. It compares incoming prompts with earlier requests by meaning rather than exact wording and can return a stored answer when the similarity is high enough. Redis claims savings of up to 90% on API costs and cache-hit responses up to 15 times faster than a new model request.

LangCache is available as a public preview on Redis Cloud through a REST API, with Python and JavaScript SDKs. An application searches the cache before contacting its chosen LLM; after a miss, it stores the new prompt and response for later use. The service handles embeddings and supports configurable similarity thresholds, TTLs, eviction policies and adaptive controls. It is designed to work with any LLM provider and language.

In a demonstration, direct inference took 2.232 seconds and used 514 input and 250 output tokens, while a cached response took 0.37 seconds with no LLM tokens. Redis says results depend on traffic patterns and require monitoring for incorrect matches, along with suitable expiration and tenant-isolation settings.

Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.

Originally published by MarkTechPost on

Read the original on MarkTechPost ↗

The reporting and the image belong to MarkTechPost and are credited and linked above; the text on this page is Sakhanda Wire's own summary.

← Back to the news

More stories

All the latest news