Osegi
Software·2 min read

Windows RDMA over USB4 lets mini PCs pool memory for big AI models

Microsoft is testing a Windows feature that links mini PCs with plain USB4 cables, so their memory adds up and a 235-billion-parameter AI model can run across them.

Three small mini PCs on a desk connected to each other with USB4 cables, running a large AI model together.
Image: Frandroid

Three mini PCs on a desk, three USB4 cables running between them, and a 235-billion-parameter AI model running as if it lived on one machine. That is the demo behind Windows RDMA over USB4, a feature Microsoft showed on 19 November in developer preview. None of the three PCs could load the model alone: even compressed, Qwen3-235B needs more than the 128 GB each of them has.

RDMA, remote direct memory access, lets one computer read another's memory without going through the usual network stack. Until now that meant datacenter network cards. According to Microsoft, Windows and a USB4 cable are now enough. A small message makes the round trip in 28 microseconds, about four times faster than ordinary Ethernet over the same cable.

AMD tested three 128 GB Ryzen AI Max+ 395 machines, the chip in the GMKtec EVO-X2. Qwen3-235B produced 18.5 to 21.98 tokens per second (a token is roughly a piece of a word). Intel ran GPT-OSS-120B on three, then four, 64 GB Core Ultra Series 3 PCs and got 49.4, then 54.2 tokens per second.

How does it compare with Nvidia?

Nvidia sells this as a feature. The DGX Spark has a 200 Gb/s ConnectX-7 network chip, and two of them run models up to 405 billion parameters. USB4 tops out at 40 or 80 Gb/s, five times less at 40, but costs nothing extra. Microsoft says latency, which drives text generation, is the same at both speeds.

Can you mix AMD, Intel and Arm?

Nobody knows yet. Every PC must be cabled directly to every other, so two USB4 ports allow a trio and four ports allow five machines. Microsoft only validated identical setups, with the same Windows versions and drivers, and never mentions Arm. The modified llama.cpp it published uses Vulkan, which AMD, Intel and Nvidia GPUs all understand, so mixing is possible in theory.

The feature is not in any public Windows build. Microsoft promises an Insider release soon, and only the NetworkDirect module and the modified llama.cpp are on GitHub. You will also have to turn off Secure Boot. Apple has offered RDMA over Thunderbolt 5 between Macs since macOS 26.2.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:windowsmicrosoftusb4rdmamini pclocal ai

More from this day

See all →

More in: Software

See all →