Ollaya is a self-hosted inference daemon and CLI that runs open decision models locally without generating text. Using a single forward pass, it evaluates states like JSON messages against typed questions to return calibrated probabilities in milliseconds. On an RTX 5090, the recommended 4B model processes five questions in 113 ms, while CPU-friendly alternatives take around half a second.
It is worth installing if you are already using TypeSafe workflows and want to run models locally under an Apache-2.0 license. However, it is built exclusively for structured classification and routing rather than text generation, so it will not replace a standard LLM backend. If your application relies on unstructured output or long-form reasoning, skip it.
Leave a Reply