Pooled is an open-source JavaScript library that distributes large language model inference across multiple devices using WebGPU and WebRTC. By splitting model layers across a laptop, phone, and desktop on the same Wi-Fi, it allows you to run models like Qwen 27B that exceed single-device memory limits. There is no server or installation required, with devices communicating directly to pass hidden states between browser tabs.
It is worth cloning if you want to experiment with distributed browser compute or need a local multi-device setup without managing a Python backend. The catch is that WebGPU performance varies heavily by hardware, and syncing token states over local WebRTC adds latency that makes it impractical for production workloads. Skip this if you just need a standard local runner like Ollama.
Leave a comment