Multi-agent workflows have changed the shape of local inference. A lead agent decomposes a task and spawns subagents. What looked like one user request becomes dozens of independent model calls. Pointed at a single local engine, those calls compete for the same execution slots. The queue grows while a workstation, laptop, or DGX Spark on the same network sits idle.
NVIDIA Personal AI Router (PAIR) targets exactly that bottleneck. Announced this week, PAIR is a virtual inference router. It discovers compatible machines on a home network and schedules independent inference requests across them. It is not a new inference engine. Ollama or LM Studio still executes the model on whichever node PAIR selects.
Is it deployable? Yes. PAIR ships today as a public beta (v0.1.1) with signed installers for Windows, macOS, and Linux, and the full source is on GitHub under Apache 2.0. It runs entirely on the local network, with internet needed only to download models.
No new API
The design decision that matters most is that PAIR introduces no cluster API. It proxies the Ollama-compatible and LM Studio-compatible interfaces agents already speak, taking over the default port each engine uses. If a harness listens elsewhere, the proxy port is configurable in PAIR’s engine settings. The repository also exposes OpenAI-compatible proxy endpoints.
The consequence: existing agent harnesses need no changes. The agent decides what work to request. PAIR decides where it runs.
Discovery, pairing, and transport
PAIR uses mDNS to find nearby systems automatically. A node can be added by IP address when discovery fails. Trust is established by a six-digit PIN shown on the inviting machine and entered on the invited one. All node-to-node communication is blocked until that pairing completes. Traffic between paired nodes is then secured with mTLS using generated certificates.
Each node runs Ollama or LM Studio. PAIR can install an engine and start model downloads on paired systems, removing most cross-machine setup work.
How the scheduler picks a node
A node becomes eligible for a request only when the required engine is enabled and the exact requested model is present. Models do not need to be identical across the cluster, different systems can hold different models, and PAIR routes according to model location. Loading the same tag on more nodes simply widens the eligible pool.
For each request the scheduler weighs five signals. Is the node online and ready. Is a supported engine enabled. Is the exact model present. What is the current node and engine job load. What is existing GPU utilization.
This is workload-level concurrency, and the boundary is explicit. PAIR assigns each request to one eligible node, where it stays for its lifetime. It does not pool VRAM, merge GPUs into one larger accelerator, or shard a single request across machines.
Credit: Source link
