llama.cpp

llama.cpp is an open-source engine for running large language models and vision models on a wide range of hardware, locally or in the cloud. It is written in plain C and C++ without dependencies, and powers many other local AI tools. Its team also makes Llama, a small Mac menu bar app built on it.

llama.cpp downloads and runs models straight from Hugging Face with one command, and serves them through an OpenAI-compatible API server. It is tuned for state-of-the-art performance on everything from laptops to servers, and is available as pre-built binaries, Docker images or source.

llama.cpp is the foundation of the local AI ecosystem: many popular apps for running models on your own computer are built on it. It is MIT-licensed and one of the most active open-source AI projects.

Stars

0

Forks

0

Commits

0

Repository

ggml-org/llama.cpp

• LM StudioDesktop app for running local models, with a graphical interface but not open source.
• OllamaNo. 079Open-source tool for running open models locally, built on llama.cpp.
• OpenAI APIOpenAI's hosted models, billed per token.

See every open-source alternative to LM Studio in the collection.