llama.cpp supports device backends including CUDA for NVIDIA, HIP for AMD, Metal for Apple silicon, SYCL for Intel and CANN for Ascend NPUs. Its README also lists backends for other accelerators and CPU libraries. Support for a backend does not guarantee that every model or operator runs on it.

Vulkan’s row is the shortest and the most interesting. Its target devices are listed simply as “GPU”.

Why Vulkan matters on consumer cards

Khronos describes Vulkan as a cross-platform industry standard that lets developers target a wide range of devices with the same graphics API. For llama.cpp that is the useful property. The HIP backend needs ROCm, and AMD’s own compatibility tables restrict official support to a specific list of GPUs on a specific list of Linux releases. The Vulkan backend needs a Vulkan driver, which ships in ordinary graphics drivers.

On Linux that driver is usually RADV for AMD cards, which Mesa documents as a userspace Vulkan driver implementing the API on most modern AMD GPUs, included by default in many distributions as part of their Mesa packages and used as the Vulkan driver on Valve’s Steam Deck. Building llama.cpp for Vulkan is two commands once the SDK or system packages are in place:

cmake -B build -DGGML_VULKAN=1
cmake --build build --config Release

On Debian and Ubuntu the dependencies are libvulkan-dev, glslc and spirv-headers, and the build doc specifically warns that the SPIR-V headers are not always pulled in by the Vulkan loader development package alone. The doc suggests verifying with vulkaninfo before building, then using a GPU-layer setting appropriate to the model to offload layers. Its sample output line shows how the backend reports what it found, including whether the device has unified memory.

Which backends ship binaries

The release assets identify the backend and platform combinations distributed as binaries. They include CPU packages and builds for CUDA, ROCm, SYCL, Vulkan, OpenVINO and selected mobile accelerators. Availability changes by release, so use the asset list for the version you are installing.

A backend documented in the source tree may still require a local build. CANN and MUSA, for example, have dedicated build instructions. Binary availability reduces installation work; it is not a performance or stability ranking.

Mixing backends

The build guide notes that in most cases you can build and use several backends at once, giving -DGGML_CUDA=ON -DGGML_VULKAN=ON as the example, then selecting devices at runtime with --device and listing them with --list-devices. Backends can also be compiled as dynamic libraries loaded at runtime, through the GGML_BACKEND_DL option, so one binary can serve machines with different GPUs.

With -ngl 0 the GPU may still accelerate parts of the computation, and fully disabling it takes --device none. The HIP section documents GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 for sharing main memory with an integrated GPU on Linux, while warning of a performance cost on discrete GPUs.

Comparing SYCL and HIP

The SYCL backend primarily targets Intel GPUs, including Data Center Max and Flex, Arc discrete cards and integrated graphics. Its changelog reports a prompt-processing improvement after a oneAPI toolchain update on an Arc B570. That comparison was between SYCL builds, so it provides no evidence that SYCL outperforms Vulkan on the same model.

HIP has the same logic on supported AMD hardware. For a card AMD does not officially support, Vulkan avoids the HSA_OVERRIDE_GFX_VERSION guesswork entirely.

FAQ

Is Vulkan slower than CUDA or ROCm in llama.cpp?

Use llama-bench on your own hardware with the same model, precision and workload. The sources reviewed here do not establish a universal speed ranking between CUDA, HIP and Vulkan. Toolchain changes can affect the result.

Can I use Vulkan on a Mac?

Yes, through a translation layer. llama.cpp documents MoltenVK and KosmicKrisp, both mapping Vulkan onto Metal, selected with VK_ICD_FILENAMES, and that build turns Metal off. Metal is on by default and is the normal path.

Which backend should I pick for an Intel Arc card?

SYCL is designed for Intel GPUs and ships prebuilt in FP16 and FP32 variants, and OpenVINO builds also ship. Vulkan is another option where the installed driver and runtime support the card.

Do I need the full Vulkan SDK?

Not on Linux. System packages are enough, and the build doc lists them per distribution. On Windows the LunarG SDK supplies glslc and the headers.