ivo's home

FairStress is a 300‑scenario benchmark spanning 29 domains, hosted on Hugging Face, designed to measure how LLMs respond to demographic changes and adversarial prefixes. It integrates directly into Olmo‑eval, so you can run it on any open‑weight model or GPT‑style API with a single command. The dataset includes explicit, implicit, and no‑identity versions of each scenario to test subtle bias.

If you’re evaluating model fairness beyond surface decisions, FairStress gives a more nuanced view than standard parity checks. It does not cover equity‑based fairness or long‑term impact analysis, and its adversarial prefixes are manually curated, so you’ll need to supplement it for deeper coverage. It’s best suited for research labs or teams with evaluation pipelines, not for quick production testing.

more: https://huggingface.co/blog/allenai/fairstress

my other side: ivo.blogspot.com

Project Maya is a local inference engine for massive Mixture-of-Experts models like GLM-5.3-Flash and DeepSeek-V4.1. It offloads inactive experts to system RAM and NVMe SSDs, allowing 748B parameter models to run on single GPUs or multi-GPU setups. The API is compatible with OpenAI and Anthropic standards, supporting tool calls and concurrent requests.

This is worth trying if you have high-end local hardware and need private, low-latency LLM access without cloud costs. The catch is the steep hardware floor: expect to use 100GB+ of combined VRAM and RAM for usable performance. It is not a solution for standard consumer laptops or low-memory systems.

more: https://github.com/mw00/project-maya

my other side: ivo.blogspot.com

Mattt and the team at Hugging Face have released version 1.0 of AnyLanguageModel, a Swift library that acts as a drop-in replacement for Apple's Foundation Models framework. By changing a single import statement, you can swap between the on-device system model, local runtimes like MLX and llama.cpp, and hosted APIs using identical session code. It also includes support for guided generation, streaming tool calls, and model downloads via companion packages.

If you are already writing Swift apps and want to experiment with local weights or abstract away multiple inference backends without rewriting your session logic, this saves you real boilerplate. The catch is that managing memory for open-weight models on user hardware remains messy, and relying on a third-party wrapper means you are coupling your inference layer directly to Hugging Face's ecosystem.

more: https://huggingface.co/blog/anylanguagemodel-1-0

my other side: ivo.blogspot.com

Hugging Face's work on the PyTorch 2.15 MPS backend brings FP8 support, native grouped matmuls, and dedicated Metal kernels to Apple Silicon Macs running macOS 15. Mixture-of-experts models see decoding speedups up to 6.5 times faster on hardware like the M3 Ultra, while memory peaks for video VAE decoding drop from gigabytes of overhead to manageable footprints. The release replaces various MPSGraph bottlenecks with direct Metal execution for single-row linear layers, softmax, and common linear algebra operations.

It is worth adopting if you are actively running local inference or fine-tuning models on a modern Mac and hitting previous memory or fallback walls. However, the catch is the strict hardware floor requiring macOS 15 and Python 3.11, meaning older setups get left behind without an upgrade path. Skip this if you are strictly bound to CUDA infrastructure, as parity with NVIDIA hardware is still incomplete.

more: https://huggingface.co/blog/mps-pytorch-2-15

my other side: ivo.blogspot.com

DigUp is an open-source desktop app for macOS, Windows, and Linux that indexes local files for semantic search using Google DeepMind's EmbeddingGemma 2 model. It runs entirely on your hardware, processing text, images, audio, and video into a 740M parameter embedding space so you can query files using natural language. The initial run downloads an 865 MB model and indexes your selected directories locally without sending data to external servers.

It is worth installing if you regularly hunt through unindexed mixed-media directories and want something faster than grep or Spotlight for conceptual queries. The catch is that it requires a modern machine to index smoothly and relies on external players to jump directly to specific timestamps in media files. Skip it if you only need exact-match text searches or refuse to run local ML models in the background.

more: https://github.com/ARahim3/DigUp

my other side: ivo.blogspot.com