Sakhanda Wire
NVDA $153.77 -1.52% MSFT $506.06 -2.30% GOOGL $280.21 -1.24% META $622.96 -0.55% AMZN $257.51 -0.09%

MarkTechPost

396 stories · all sources

MarkTechPost

llm-d 1.0 adds cache-aware routing for distributed AI inference

Google, Red Hat and IBM have released llm-d 1.0, the first stable version of an open-source, Kubernetes-native stack for distributed large-language-model inference. It coordinates vLLM and SGLang servers into a shared pool for self-hosted agent and reinforcement-learning workloads. The project is a CNCF Sandbox initiative, uses the Apache 2.0 license and has more than 500 contributors from over 25 organizations.

MarkTechPost

Google Open-Sources TorchTPU for Native PyTorch on Cloud TPUs

Google has released TorchTPU, an open-source PyTorch backend for Google Cloud TPUs developed with Meta’s PyTorch team. The project was announced at PyTorch Conference 2026 on October 20, 2026. Existing PyTorch programs can select a TPU through torch.device("tpu"), placing TPUs alongside CPUs and GPUs as native devices. Google recommends TorchTPU for new workloads as a replacement for PyTorch/XLA.

MarkTechPost

DeepSeek open-sources the DeepInfer inference framework

DeepSeek has released DeepInfer, the inference framework used to serve DeepSeek-V4.1-Flash in production. The project is available under the Apache 2.0 license, while the company’s deepseek-recipe token encoding and decoding layer uses the MIT license. DeepInfer is designed for offline inference and supports one model with a 1M-token context window.

MarkTechPost

Exa 3.0 reports leading results across eight search benchmarks

Exa has introduced Exa 3.0, an upgrade to its hosted web search API for AI agents. The release includes a larger embedding model, although Exa has not disclosed its size, and expands the company’s independently crawled index, which covers more than 2 trillion URLs and 100 billion pages.

MarkTechPost

Cloudflare Previews Clairic Extraction Model on Workers AI

Cloudflare has previewed Clairic, a System One model for extracting structured information from text and images. Instead of generating prose, it returns typed values such as numbers, dates, durations, lists, estimates and forecasts. A request can include text and up to four images, along with named fields and natural-language instructions. Scalar results include a confidence score and p_null, which indicates the estimated probability that a value is absent. Text results can include exact spans and character offsets, while forecasts provide a median and an 80% interval.

MarkTechPost

Inception launches Mercury 3 diffusion language model

Inception has released Mercury 3, a diffusion large language model that generates responses through parallel refinement rather than producing tokens strictly one after another. The company reports an output rate of 1,068 tokens per second on NVIDIA GPUs and a context window of 1 million tokens. Parameter counts and the exact GPU configuration have not been disclosed. The model is available through an API, without open weights or quantized versions.

MarkTechPost

Nace AI releases Drex Max decision model for computer-use agents

Nace AI has released Drex Max, a multimodal decision model designed for agents operating computers and browsers. It processes text and images, then returns calibrated probability distributions rather than generated text. Users provide a state and typed questions with predefined answer options: yes or no, a label choice, or an ordered score. The model can respond to up to 16 questions in one API request.

MarkTechPost

MiniMax releases M3.1 Flash MoE model with one-million-token context

MiniMax has released M3.1 Flash, a multimodal mixture-of-experts model with 428B total parameters and about 23B active parameters. It accepts text, images and video, produces text, and supports a 1,000,000-token context window. The model is designed for coding, agentic office tasks and visual reasoning. An earlier M3.1 Flash build, called Space Bunny, processed more than 200T tokens.

MarkTechPost

NVIDIA Releases Run:ai as Apache 2.0 Open Source

NVIDIA has released the complete Run:ai GPU orchestration platform under the Apache 2.0 license. The code is available on GitHub and includes a control plane, cluster services, installer, command-line interface and MCP server. Run:ai operates on Kubernetes or OpenShift clusters with NVIDIA GPUs and manages development, training and inference workloads without replacing Kubernetes.

MarkTechPost

Mistral AI releases Timestral forecasting model family

Mistral AI has released Timestral, its first family of time-series foundation models. The collection includes four zero-shot forecasting models with 5M, 28M, 123M and 209M parameters. The largest model supports contexts of up to 32,768 time steps and can produce forecasts on standard server CPUs without a GPU. Its weights are available under a modified MIT licence, according to the release.

MarkTechPost

TinyFish removes retrieval fees while web agents still pay model costs

AI agents incur two separate expenses when reading the web: charges for search or page retrieval, and model processing for the returned content, reasoning and response generation. TinyFish has made Search and Fetch available at no retrieval charge within published limits, but the resulting pages still consume model tokens. Its limits are 30 Search requests per minute and 500 per hour, plus 150 Fetch URLs per minute and 1,000 per day. Agent and Browser usage are billed separately.

MarkTechPost

Google Research releases Harness² for recursive agent improvement

Google Research has open-sourced Harness², an Apache-2.0 framework that modifies an AI agent’s surrounding code and files while leaving the underlying model weights unchanged. Its editable harness includes eight components: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins. The project is version 0.1.0 and runs on Linux with Python 3.12 or 3.13 and uv; model calls use Vertex AI.