📦 llama.cpp/ ggml-org

Quick start

A few options to get llama.cpp installed on your machine:

• Visit https://llama.app and follow the instructions
• Run with Docker - see our Docker documentation
• Download pre-built binaries from the releases page
• Build from source by cloning this repository - check out our build guide

Once installed:

# Download and run a model directly from Hugging Face
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

# Launch OpenAI-compatible API server
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF

<table align="center"> <tr> <td align="center" width=50%>

        <i>VLM session with <b>llama cli</b></i>
    </td>
    <td align="center">
        
        <i>Built-in web UI against <b>llama serve</b></i>
    </td>
</tr>

<table>