Run backpropagation on Apple's Neural Engine using reverse-engineered APIs
You can observe a neural network learning directly inside a MacBook's chip. This project reverse-engineers Apple's private APIs for the Neural Engine (ANE), which officially supports only inference, to demonstrate how to run backpropagation operations on the chip. It serves as a laboratory for checking how much software limits the hardware's potential.
Adding learning to an inference-only chip
Apple Silicon's Neural Engine officially supports only inference. This project reverse-engineers private APIs to go beyond this limitation. The goal is to determine whether the hardware can perform learning and whether the barrier is software-based. Research that started over a weekend has now proven that training small models is possible. This is not a production framework but a proof of concept that tests the boundaries of possibility.
Opening private paths through reverse engineering
It calls _ANEClient and _ANECompiler directly to compile and run models, bypassing the official API. It analyzes the model intermediate language format to upload a custom operation graph to the chip. A dynamic pipeline is implemented so that recompilation is not required when weights change. This allows most of the forward and backward propagation to be processed on the chip. Only weight gradient calculations are performed on the CPU to reduce bottlenecks.
Measured performance and limitations
This is not at a level that replaces GPU training, but it serves as an indicator of the chip's own computational capability. Applying INT8 quantization increases throughput by 1.88 times. In a 128x convolution with 512 channels and 64x64 settings, it records 35.1 TOPS compared to FP16. It improves performance by halving memory bandwidth. The utilization rate relative to the chip's peak performance is between 5 and 9 percent. Many operations still fall back to the CPU, making it unsuitable for training large models. Process restarts are required due to compiler resource leaks.
Preparation for experimentation
macOS 15 or higher and an Apple Silicon chip are required. It runs without external dependencies, using only system frameworks and private APIs. You must download the pre-tokenized TinyStories dataset. Specify the model type during the build to compile it. This code should be used only for research and educational purposes, and because it uses Apple's private APIs, it may break upon updates.
Who should touch this code
It is suitable for researchers interested in hardware architecture or developers exploring private APIs. It is difficult to use directly in production environments. Maintenance is slow, and responses to feature requests are limited. It is under the MIT license, so you can fork and modify it freely. It is recommended only for those who want to directly verify the chip's limits.
