Run 16 model families from Whisper to Parakeet in a single C++ library
You can run speech recognition models with different architectures from a single codebase. This repository is a C/C++ inference library that runs on the ggml runtime and supports various GPU backends, including Metal, Vulkan, and CUDA. This allows you to build a pipeline that quickly converts audio to text by selecting only the necessary models, without being tied to specific hardware.
Model families and feature scope
The library supports more than 16 model families and over 60 variants. Each model is provided in GGUF format and supports both streaming and batch processing. Some models offer additional features such as speaker diarization, translation, and word-level timestamps. All public models are numerically verified against reference implementations and undergo WER (word error rate) testing. The library also includes models dedicated to speaker diarization or language identification, allowing you to handle other stages of a voice analysis pipeline in addition to transcription.
Hardware acceleration and build configuration
On Apple Silicon, Metal is enabled automatically, while Vulkan is available on Linux and Windows. If you use an NVIDIA GPU, you can build with the CUDA backend explicitly specified; for AMD GPUs, you can use the HIP/ROCm backend. In CPU-only environments, tinyBLAS is enabled, providing inference that is much faster than the default scalar path. Installing libopenblas-dev improves host-side decoder performance by approximately 10 to 15 times. Each backend is controlled via CMake options, so you can adjust the build command to match the required GPU architecture.
Model conversion and quantization tools
Scripts are provided to convert directly from NVIDIA NeMo checkpoints to GGUF models. These run in a uv environment, and for Parakeet models, they operate in an environment that includes NeMo dependencies. Converted models follow llama.cpp-style naming conventions. You can also use the transcribe-quantize tool to shrink models into various quantization presets, such as F16, Q8_0, and Q4_K_M. This helps reduce memory usage and increase inference speed.
Language bindings and usage
Official bindings support Python, TypeScript/JavaScript, Rust, and Swift/ObjC. By wrapping the C API, the library integrates naturally into each language's ecosystem. When using the CLI, input audio must be in 16kHz mono WAV format. Audio in other formats should be converted using ffmpeg or sox before being passed in. When upgrading to version 0.2, you should check the accurate device selection API and changes in CLI flag meanings.
Verification system and release readiness
Model metadata, accuracy, and performance information are managed in the source of truth within the catalog/ directory. Each release includes a queryable catalog.db and SHA-256 checksums. Public tables are not written manually but are generated automatically from catalog records. Hugging Face card specs contain only editable text, while repository information, licenses, and performance data are injected automatically along with the catalog. This system ensures that model accuracy and performance remain consistent, and fixes are made if regressions occur.
