Agent skill for tuning local LLM inference speed

19 11 2026
Agent skill for tuning local LLM inference speed

This repo provides a structured workflow for optimizing local LLM servers on Intel Arc, RTX, and H200 hardware. It guides an agent to compute theoretical ceilings and systematically test levers like speculative decoding and KV cache sizing. Past runs using this method boosted decode speeds from 43 to 53 tok/s on a B390 iGPU and fixed a crashing H200 deployment.

It is useful if you are running local models and want a rigorous path to performance gains rather than guessing at flags. The catch is that it requires an agent capable of executing long, serialized benchmark loops and maintaining detailed logs. It is not a magic fix if your hardware is already at the physical bandwidth limit.

more: https://github.com/sybil-solutions/inference-optimisation-skill

my other side: ivo.blogspot.com


Actions

Information

Leave a Reply