ivo's home

Mattt and the team at Hugging Face have released version 1.0 of AnyLanguageModel, a Swift library that acts as a drop-in replacement for Apple's Foundation Models framework. By changing a single import statement, you can swap between the on-device system model, local runtimes like MLX and llama.cpp, and hosted APIs using identical session code. It also includes support for guided generation, streaming tool calls, and model downloads via companion packages.

If you are already writing Swift apps and want to experiment with local weights or abstract away multiple inference backends without rewriting your session logic, this saves you real boilerplate. The catch is that managing memory for open-weight models on user hardware remains messy, and relying on a third-party wrapper means you are coupling your inference layer directly to Hugging Face's ecosystem.

more: https://huggingface.co/blog/anylanguagemodel-1-0

my other side: ivo.blogspot.com

Hugging Face's work on the PyTorch 2.15 MPS backend brings FP8 support, native grouped matmuls, and dedicated Metal kernels to Apple Silicon Macs running macOS 15. Mixture-of-experts models see decoding speedups up to 6.5 times faster on hardware like the M3 Ultra, while memory peaks for video VAE decoding drop from gigabytes of overhead to manageable footprints. The release replaces various MPSGraph bottlenecks with direct Metal execution for single-row linear layers, softmax, and common linear algebra operations.

It is worth adopting if you are actively running local inference or fine-tuning models on a modern Mac and hitting previous memory or fallback walls. However, the catch is the strict hardware floor requiring macOS 15 and Python 3.11, meaning older setups get left behind without an upgrade path. Skip this if you are strictly bound to CUDA infrastructure, as parity with NVIDIA hardware is still incomplete.

more: https://huggingface.co/blog/mps-pytorch-2-15

my other side: ivo.blogspot.com

DigUp is an open-source desktop app for macOS, Windows, and Linux that indexes local files for semantic search using Google DeepMind's EmbeddingGemma 2 model. It runs entirely on your hardware, processing text, images, audio, and video into a 740M parameter embedding space so you can query files using natural language. The initial run downloads an 865 MB model and indexes your selected directories locally without sending data to external servers.

It is worth installing if you regularly hunt through unindexed mixed-media directories and want something faster than grep or Spotlight for conceptual queries. The catch is that it requires a modern machine to index smoothly and relies on external players to jump directly to specific timestamps in media files. Skip it if you only need exact-match text searches or refuse to run local ML models in the background.

more: https://github.com/ARahim3/DigUp

my other side: ivo.blogspot.com

CodeAgent is an open-source, terminal-based AI programming assistant written in Python 3.11 with extensive Chinese commenting. It handles natural language instructions by reading files, executing commands, searching codebases, and running multi-model loops via OpenAI-compatible or Anthropic endpoints. The tool features a built-in session store, hierarchical context compression, permission panels, and a self-learning skill system stored locally under ~/.codeAgent.

It is worth checking out if you want a local, hackable alternative to proprietary CLI tools with built-in DeepSeek support and clear module architecture. However, Windows is the primary development target with Linux support trailing, while macOS remains largely unverified. Skip this if you need out-of-the-box cross-platform stability without configuring sandbox environments or API keys manually.

more: https://github.com/Harvil1/codeAgent

my other side: ivo.blogspot.com

StartLux-Decision provides open models in 4B, 9B, and 27B parameter sizes that output typed probabilities for structured questions rather than open text. The repository includes an HTTP server and inference engine supporting GGUF and MLX formats, with variants that conditionally generate reasoning traces when confidence falls below 0.6. It handles text, JSON, and image inputs in a single forward pass, making it suitable for programmatic control tasks like game bots and video frame analysis.

If you need structured classification or control loops without parsing free-form LLM output, it is worth testing locally via the provided MLX or GGUF weights. However, the specialized API and decision-focused architecture mean it will not replace a general-purpose chat model or handle open-ended text generation. Skip it unless you are actively building deterministic pipelines that require explicit probability scores for discrete options.

more: https://github.com/StartLuxLabs/StartLux-Decision

my other side: ivo.blogspot.com

The @receptron/laya npm package runs Convai Innovations' Laya decision model directly in Node.js and TypeScript via ONNX Runtime without needing Python or PyTorch. It takes a JSON state and typed questions, returning calibrated probabilities for choices, scores, and yes-no statements in a single forward pass taking roughly 140 milliseconds per batch on Apple silicon. The 1.7 GB model weights download automatically on first use, and the TypeScript types match the Python implementation to four decimal places.

It is worth testing if you need structured classification or confidence scoring on text inputs without spinning up an LLM API. The catch is strict input limits: states are truncated at 512 tokens and choice questions top out around 20 options. Skip this if you need generative text or inputs longer than a short support ticket.

more: https://github.com/receptron/laya

my other side: ivo.blogspot.com