Run language models directly on Mac and NVIDIA GPUs without Python
Enter two commands in the terminal to see your computer's chip process a language model and generate a response. TensorFold is an engine that handles reading model weights and generating tokens as a single binary. It provides the experience of running a local AI service without installing a separate runtime.
Installation ends with a single binary
This engine is written in Zig and runs without additional libraries like Python or MLX. On Apple Silicon Macs, it is available via Homebrew, while on Linux, you can use the executable directly from the release archive. For NVIDIA GPUs, glibc 2.28 or higher and drivers R580 or newer are required. Once the server starts, it provides an OpenAI-compatible API, making it easy to connect with existing clients.
Model selection and hardware scope
Supported models include various families such as Qwen, Nemotron, GLM, and Gemma. Each model specifies a verified range for specific hardware, allowing you to choose a checkpoint suitable for your device. For example, Qwen3.5-2B has been verified on M5 Max Macs or GB10, while larger models may require combining multiple devices. You can check the minimum memory requirements for a checkpoint in advance using the tensorfold info command.
Experimental feature: Living Weights
There is an experimental feature that allows you to teach new facts directly to a running model. Enabling the --slide option lets the model accept new information, and this process can be observed in real-time in a 3D view showing neuron changes. Learned facts are recorded in the checkpoint file and can be reverted to a previous state using a rollback command if needed. This feature is supported on Apple Silicon Macs and NVIDIA GB10, and the learned model folder can be used on other devices.
Accurate decoding and parallel processing
All responses are designed to guarantee identical results when run with the same settings. When handling parallel requests, it verifies that each stream produces the same output as a standalone run. You can limit the number of concurrent requests with the --parallel option, and it automatically adjusts up to a maximum of 8 streams within memory limits. This enables stable and reproducible inference even in local environments.
Pre-use checks
This tool provides models verified only on specific hardware combinations. Models or hardware combinations not specified in the README may not be supported, so you must verify that your device falls within the range listed in the table. Additionally, some models require a conversion process or only work with specific driver versions or higher. If you want an enterprise deployment, refer to Starlight in a separate repository, which is currently in its early stages.