Run Qwen 3.8 locally on Mac with high speed
When running coding agents or processing long documents, you can receive immediate responses on your MacBook without waiting for cloud APIs. MTPLX is a tool for running local language models on Apple Silicon that uses MTP (Multi-Token Prediction) technology, where the model predicts multiple tokens in advance, to increase speed. It supports OpenAI and Anthropic APIs as-is, so you can connect and use your existing coding tools or chat apps directly.
Twice as fast inference with MTP
Unlike the standard method where a model predicts only the next single token, MTPLX uses a method that writes and verifies multiple tokens at once. During this process, it maintains the model's original temperature setting and ensures result accuracy through rejection sampling and residual correction. When running Qwen 3.8 Flash Next on an M5 Max Mac, it recorded a speed of 125.8 tokens per second, delivering performance about twice as fast as standard decoding. Since a separate draft model does not occupy memory, you can run larger models on the same hardware.
Automatic model recommendations for your hardware
Upon installing the app, it checks your hardware specifications and automatically recommends models suitable for your Mac. For Macs with 16GB of memory, it suggests Ternary Bonsai 2 27B; for 32GB or more, Qwen 3.8 27B; and for 256GB or more, Qwen 3.8 Flash Next. Each model is provided in three packages: speed-optimized, quality-optimized, and default speed. M1 and M2 chips automatically select FP16 precision versions. The app automatically handles model downloads, Python engine setup, and fan control installation, so no separate configuration is needed.
A server that connects directly to existing tools
MTPLX runs a local API server compatible with OpenAI and Anthropic. For tools that support APIs, such as Claude Code, Cursor, OpenCode, and Cline, you only need to specify the http://127.0.0.1:8000 address. It supports chat, tool calls, and streaming responses, and uses session caching to quickly restore previous conversation states after a restart. Embedding and reranking models can also be run on the same server, so you do not need a separate inference server for RAG systems.
Finding the optimal depth for your Mac
MTP performance varies depending on chip type, memory bandwidth, and thermal state. MTPLX uses the mtplx tune command to measure the actual speed of each depth on your Mac and saves the fastest value. In a case where a 9B model was tuned on a 16GB M4 Mac mini, the speed increased from the default 14.4 tok/s to 23.0 tok/s by setting the depth to 1. The app's dashboard allows you to check real-time token speed, acceptance rate, and cache status, so you can immediately see which settings are effective.
Things to check before use
Apple Silicon (M1 or later) and macOS 14 or later are required. Macs with 16GB of memory can use an 8,192 token context window, and 18GB or more of memory is recommended if running agent clients simultaneously. Qwen 3.8 Flash Next requires 96GB or more of memory and delivers optimal performance with 256GB or more. You can convert Hugging Face models to MTP models using the Forge feature, but arbitrarily attaching external MTP heads to existing MLX trunks is not supported. You should understand the trade-off between model precision and speed and select the package that fits your work.