ivo's home

This is a portable C99 inference engine for the Kimi K3 2.78-trillion-parameter model. It runs on a standard CPU with 8 GB of RAM, using no BLAS, framework, or GPU. The entire engine is 176 KB, and it streams the 1.56 TB checkpoint from disk to keep peak memory usage low.

It is worth looking at if you want to understand how to run massive models without heavy dependencies, but do not expect interactive speed. On an 8 GB laptop, generation takes 26.5 seconds per token because the model streams from disk every step. It is a proof of concept for resource-constrained environments, not a replacement for GPU-accelerated inference.

more: https://github.com/FareedKhan-dev/kimi-k3-in-c