Llama.Cpp
Local LLM inference engine
ggml-orgThird-party software from public repositories. bestappfor.me does not host it, does not verify it and is not responsible for its use. Download only from the official source and check the license. Terms and disclaimer
About
Run large language models on your own hardware. llama.cpp offers a command-line chat tool and a server with a built-in web UI and a REST API, so you can talk to a model locally or connect other programs to it.
It is plain C/C++ with no dependencies, and uses quantization from 1.5 to 8 bits to cut memory use. It supports Apple silicon, x86 and RISC-V CPUs, and GPUs through CUDA, HIP, Metal, Vulkan and SYCL, with hybrid CPU and GPU inference.
Highlights
- CLI chat tool and server with web UI
- 1.5 to 8-bit quantization
- CUDA, HIP, Metal, Vulkan and SYCL backends
Who it's for
Good for tinkerers who want to run language models on their own computer without a cloud service.
This description was written by our editors with automated assistance, not by the developer, and has not been reviewed yet.
Why it is here
- Code changed today
- New release 8 days ago
Official channels
Official channels only. We never host or repackage downloads.
