Run local LLMs as practical coding tools on your MacBook
Imagine clicking a single menu bar icon to start a server and watching your usual coding agent converse naturally with a local model. oMLX is a server that handles LLM inference on Apple Silicon Macs, managing cache using memory and SSD. This allows you to run multiple models simultaneously on a single MacBook or write code while maintaining long conversation contexts.
A cache strategy linking memory and SSD
The core of oMLX is managing the KV cache in two stages. Frequently used data stays in fast RAM, and if capacity runs low, it is offloaded to the SSD. Even if the context changes during a conversation, previous content remains in the cache, so it does not need to be recalculated. The cache stored on the SSD persists even after restarting the server, creating an environment suitable for using local LLMs in actual coding tasks.
Management starting from the menu bar
You do not need to memorize terminal commands. From a menu bar icon built as a native Swift app, you can start or stop the server and check model status. The app automatically restarts upon a crash and provides usage heatmaps and per-model statistics. If desired, you can also run it as a background service using CLI commands like omlx start.
Connecting with coding tools
It provides OpenAI and Anthropic API-compatible endpoints, allowing you to connect existing clients directly. When used with tools like Claude Code, it automatically compresses based on the model's actual context window and maintains the connection so it does not drop during long prefilling. You can configure various tools such as OpenClaw, Codex, and Copilot at once from the management dashboard, eliminating the need for manual editing.
Pre-installation checks
macOS 15.0 (Sequoia) or later, Apple Silicon (M1~M5), and Python 3.11~3.13 are required. A custom kernel build is essential to run specific models like GLM-5.2 or MiniMax M3 quickly, which requires a full Xcode installation. If exposing the server to the network, you must set an API key; the system prevents starting the server without a key if it is not a loopback address. It is distributed under the Apache 2.0 license, and the experimental multi-Mac distributed inference feature is only available in source builds.
