ivo's home

Allen Institute for AI released Olmo-core 3, an open-source framework designed to train mixture-of-experts models up to a trillion parameters. It abandons fully sharded data parallelism for a distributed data parallelism setup that keeps experts resident on the GPUs, routing data directly to avoid repeated weight gathering. In benchmarks on NVIDIA B300 hardware, a 47-billion-parameter MoE hit 52,000 tokens per second per GPU, a 2.7x improvement over their older stack.

It is worth a look if you are building massive sparse models and need an alternative to Megatron-Core, especially with the added MXFP8 support yielding a 21 percent throughput bump over BF16. However, the complexity here is substantial, meaning it is entirely irrelevant if you are sticking to dense architectures or smaller academic setups.

more: https://huggingface.co/blog/allenai/olmocore3

my other side: ivo.blogspot.com

Google Cloud API Gateway can now transcode standard OpenAPI specifications into remote Model Context Protocol servers. By adding a few annotation lines to an existing YAML config, your REST endpoints are exposed directly as agent tools at an /mcp endpoint. This eliminates the need to build, host, and secure a separate translation server since auth, quotas, and logging are handled by the gateway.

It is worth setting up if you already run your services behind Cloud Run and Google's API Gateway and want to connect Gemini agents without adding infrastructure. The catch is that API keys cannot protect the tool discovery endpoint in production, requiring you to configure JWT authentication instead. It is also limited to API Gateway and Apigee, so teams using custom NGINX or Envoy setups get no direct help here.

more: https://developers.googleblog.com/turn-your-rest-apis-into-mcp-tools-with-google-cloud-api-gateway/

my other side: ivo.blogspot.com

Concat is a cross-platform video editor written in Rust with a GPU compositor, built as a free alternative to CapCut. It runs entirely local on macOS, Windows, Linux, and Android, handling auto-captions, keyframes, background removal, and 4K export without an account or cloud upload. It also exposes JSON-RPC, gRPC, and MCP APIs so scripts and AI agents can automate cutting.

It is worth testing if you want offline captioning or programmatic editing without subscription software. The project is currently in beta, meaning you will run into rough edges and unsigned binary warnings on first launch. It is not for anyone who expects a polished, premiere-level NLE out of the box.

more: https://github.com/jub0t/Concat

my other side: ivo.blogspot.com

AIHOT is a Docker-ready, self-hosted framework that runs on Node.js and PostgreSQL to aggregate, filter, and cluster industry news. It takes inputs from RSS, web pages, and APIs, runs them through an LLM to score relevance and deduplicate stories, and generates daily briefings and event timelines. The repository includes all core source code, the complete prompt library, and built-in support for multiple upstream providers.

It is worth setting up if you want to replace manual curation with an automated pipeline tailored to a specific domain, provided you have a compatible LLM API key and are comfortable editing prompts. However, the system requires significant tuning of its selection thresholds and source lists to produce clean output for non-AI fields, and the documentation is currently only in Chinese. It is not a turn-key consumer product, but rather a snapshot of a working production codebase meant for developers willing to maintain it.

more: https://github.com/KKKKhazix/AIHOT

my other side: ivo.blogspot.com

Pooled is an open-source JavaScript library that distributes large language model inference across multiple devices using WebGPU and WebRTC. By splitting model layers across a laptop, phone, and desktop on the same Wi-Fi, it allows you to run models like Qwen 27B that exceed single-device memory limits. There is no server or installation required, with devices communicating directly to pass hidden states between browser tabs.

It is worth cloning if you want to experiment with distributed browser compute or need a local multi-device setup without managing a Python backend. The catch is that WebGPU performance varies heavily by hardware, and syncing token states over local WebRTC adds latency that makes it impractical for production workloads. Skip this if you just need a standard local runner like Ollama.

more: https://github.com/Nehanth/pooled

my other side: ivo.blogspot.com

Jevgrep is a command-line tool that lets coding agents search codebases using natural language queries instead of exact string matches. Requiring Node.js 22 and an API key for Vercel AI Gateway, OpenRouter, or similar providers, it indexes local repositories and outputs relevant file paths and source excerpts. In a ten-task SWE-bench repeat, using it cut agent token costs from about $7.62 to $4.52 per run.

It is worth installing if you regularly watch agents burn tokens wandering through unfamiliar directory trees. The catch is that it trades a small drop in solve rate for those savings, and you have to manually install the agent skill configuration per project. Skip it if you already rely on ripgrep and know your codebase layout.

more: https://github.com/dzhng/jevgrep

my other side: ivo.blogspot.com

geo-score is a Python script that evaluates a web site's readiness for AI search engine citations, outputting a zero to one hundred score based on an open rubric. Running it requires only Python 3.8 and the standard library, executing a single check via a piped curl command without local installation. It inspects robots.txt, sitemaps, structured data, and content formatting to identify mechanical gaps preventing retrieval.

The script is worth running for a quick baseline on how LLMs and answer engines see your static markup, but the higher tiers require your own API keys to query actual citation behavior. It separates the static score from dynamic query checks because a high technical score does not guarantee you will actually get cited against competitors. Skip it if you are not actively trying to optimize for generative search engines like Perplexity or Gemini.

more: https://github.com/jianruntech/geo-score

my other side: ivo.blogspot.com

MemoraX Code is a Node.js 20+ package that provides a persistent memory layer for various LLM coding agents including Claude Code, Cursor, and DeepSeek Harness. It runs locally as a background process to store repository rules, architectural history, and past engineering decisions across different sessions. The tool requires libsecret on Linux and specifically manages plugin integration for supported CLI and editor-based coding tools.

This is worth installing if you are tired of re-explaining project constraints to your AI assistant every time you open a new terminal window. It is not for users who want zero external dependencies, as it relies on a cloud-based backend for synchronization and requires careful management of API keys. If you frequently context-switch between projects, the overhead of setup is minimal compared to the time saved on redundant prompting.

more: https://github.com/memorax-ai/memorax-code

my other side: ivo.blogspot.com

fauxnix-cli is a Node.js CLI and MCP server for Windows systems running Node 18 or later and PowerShell 5.1+. It deterministically translates bash commands into native PowerShell at runtime without LLM calls, WSL, or virtual machines. Covering 109 core utilities verified against GNU coreutils test suites, it allows AI coding agents to run standard bash pipelines directly on Windows hosts.

It is worth setting up if you run coding agents natively on Windows and want to eliminate broken tool calls caused by the model's poor PowerShell generation. The main catch is the restricted surface area: with only 109 translated commands, complex shell scripts, unmapped flags, or heavy interactive tools will fail and still require a full Linux environment.

more: https://github.com/20000419/fauxnix