TrendWhat rose on GitHub, every day at 07:30 KST · 한국어

GitHub · as of November 15, 2026 · View on GitHub →

ggml-org/llama.cpp

C++ LLM inference engine

trending one of the last 30 days

Run language models locally without external dependencies

If you want to run language models directly on your computer without connecting to cloud servers, this tool lowers the barrier to entry. It is structured so you can build the code or download a binary and run a model immediately without complex environment setup. Written in C and C++, it is designed to operate on various hardware without being tied to specific language runtimes or frameworks.

Hardware-specific optimization

This project supports a wide range of hardware without relying on specific chips. Apple Silicon is optimized through ARM NEON, Accelerate, and the Metal framework, while x86 architecture utilizes AVX, AVX2, AVX512, and AMX instructions. The RISC-V architecture also supports RVV, ZVFH, ZFH, ZICBOP, and ZIHINTPAUSE instructions. NVIDIA GPUs use custom CUDA kernels, AMD GPUs compute via HIP, and Moore Threads GPUs use MUSA. Additionally, Vulkan and SYCL backends are supported to utilize more graphics cards.

Reducing memory usage

Various quantization options are provided to run models faster and reduce memory usage. Integer quantization from 1.5-bit to 8-bit is supported, allowing you to adjust model size to fit hardware specifications. A hybrid inference feature that uses both CPU and GPU is also available, enabling partial acceleration of models larger than VRAM capacity. This opens up the possibility of running large models in environments with limited memory.

Installation and execution

Installation is possible through several paths. You can execute a curl or PowerShell script, use Docker, or download pre-built binaries from the release page. Building directly from source is also provided, along with an Android build guide. After installation, you can download and run models directly from Hugging Face using the llama cli command. You can also start an OpenAI-compatible API server using the llama serve command.

Pre-use considerations

Because this tool is written in C/C++, the build process may be unfamiliar to users not accustomed to a specific language ecosystem. The README does not explicitly mention the license or the scope of long-term support, so legal review is necessary before applying it to commercial projects. Although the hardware support list is extensive, not all combinations guarantee identical performance, so it is recommended to test in your own environment. In particular, if you want GPU acceleration, you must verify that the backend matching your hardware among CUDA, HIP, MUSA, Vulkan, and SYCL is functioning properly.

Written by AI from this repository's README on November 12, 2026. For the facts, the original on GitHub is the reference.

By the numbers

GitHub trending history
1 day between Jun 19, when Trend began recording, and Nov 15, 2026 · first on Nov 12 · best daily rank #13 (Nov 12)

Read from GitHub, as of November 15, 2026.

Language
C++
Topics
ggml
Stars
135,168
Contributors
2,251
Forks
25,641
Latest release
v1.0.0 · November 13, 2026
Commits
12,421
Last commit
November 15, 2026
Open issues
705
Open pull requests
1,912

View on GitHub → · Homepage

README badge

trend.ondo.sh · GitHub daily trending #13 · November 12, 2026

The badge above shows the highest this repository has ranked on GitHub's daily trending since June 19, 2026, and the day it did. Paste the Markdown below into your README. The badge opens this page, and it updates by itself when the repository climbs higher.

[![trend.ondo.sh](https://trend.ondo.sh/github/r/ggml-org--llama.cpp/badge.en.svg)](https://trend.ondo.sh/github/r/ggml-org--llama.cpp/en.html)