
This pip package runs the 421M or 322M Laya checkpoints via the MLX runtime without PyTorch or cloud APIs. It returns calibrated probabilities for choice, score, and yes/no questions in 7 to 13 milliseconds on an M3 Max. The system requires macOS 14 and Python 3.11, and downloads weights on first use.
It is useful if you need low-latency local classification for structured data on Mac hardware. The catch is the strict hardware requirement and the 512 to 1024 token context limit, which excludes long-document tasks. You are not replacing a generative LLM, so do not expect open-ended text generation.
more: https://github.com/mizorewww/laya-mlx
my other side: ivo.blogspot.com
Leave a Reply