ivo's home

Reef is an open-source infrastructure for continual learning in self-improving agents. It connects inference, feedback, and versioned delivery to allow training via Slime and SGLang or optimizing agent harnesses like prompts and rules. It acts as middleware between inference engines and RL frameworks to manage live updates without downtime.

It is worth exploring if you need an agent that improves from interaction logs rather than static retraining. The catch is that model weight training requires a trainable model and a supported GPU stack, which is a significant hardware commitment. It is not a drop-in replacement for simple API wrappers but a full deployment layer for evolving systems.

more: https://github.com/Human-Agent-Society/reef

TurboFieldfare is a Swift and Metal runtime that runs Gemma 4 26B-A4B using roughly two gigabytes of RAM. It streams expert weights from the SSD to keep the model in memory, allowing it to run on an eight gigabyte M2 MacBook Air. The project includes a native Mac app, a CLI, and an OpenAI-compatible server, all built specifically for Apple Silicon.

It is worth trying if you need local inference on low-memory hardware and want to avoid the overhead of general-purpose wrappers. The catch is that decode speed on base M2 hardware is slow, ranging from five to six tokens per second. This tool is not suitable for high-throughput tasks or older Intel Macs.

more: https://github.com/drumih/turbo-fieldfare

This is a portable C99 inference engine for the Kimi K3 2.78-trillion-parameter model. It runs on a standard CPU with 8 GB of RAM, using no BLAS, framework, or GPU. The entire engine is 176 KB, and it streams the 1.56 TB checkpoint from disk to keep peak memory usage low.

It is worth looking at if you want to understand how to run massive models without heavy dependencies, but do not expect interactive speed. On an 8 GB laptop, generation takes 26.5 seconds per token because the model streams from disk every step. It is a proof of concept for resource-constrained environments, not a replacement for GPU-accelerated inference.

more: https://github.com/FareedKhan-dev/kimi-k3-in-c

Graft is a CLI that generates a folder of linked markdown files to bridge coding agents to your codebase. It claims to reduce tool calls by 46 percent and token usage by 42 percent compared to cold starts. The graph acts as a local, regenerable cache that updates against the working tree in milliseconds.

It is worth trying if you use Claude Code and want to cut latency without adding a database dependency. The catch is that it relies on LLMs for summaries, so you need a compatible API key. It is not for teams that refuse to run external tools in their dev environment.

more: https://github.com/trailhq/Graft

Hypit is a Node.js tool that extends CLI agents like Claude Code and Codex to create videos. It uses a script-based language where visual elements are anchored to spoken words, not timestamps. A drop-in video can be cloned into a reusable workflow, or agents can build new ones from templates and descriptions.

The tool itself is free, but you still pay for the AI models used for footage and images. It requires a capable coding agent and some configuration for API connections, making it less suitable for users who want a simple GUI without touching code.

more: https://github.com/hypit-ai/hypit

FreeToken is an edge-native inference engine designed to serve frontier-scale Mixture-of-Experts models on personal hardware. It unifies GPU, CPU, and host memory into a single elastic platform to achieve interactive speeds for 290B+ parameter models. A desktop app is available for Windows and Linux to handle the initial setup and tuning.

It is worth a look if you have high-end consumer hardware and need local access to large MoE models without a datacenter cluster. The project is young and relies on a complex hardware abstraction, so expect some configuration friction. It is not a drop-in replacement for production-grade serving stacks like vLLM in a cloud environment.

more: https://github.com/FlashML-org/FreeToken