ollama run gemma4 downloads an open model and opens a chat with it on your own machine. Ollama, a free tool, also serves a local HTTP API, so apps and scripts can use the same model.
What Ollama is
Ollama packages model download, a command line and a local server into one install. Its README lists llama.cpp as its supported backend, so it runs GGUF-format models (covered in the GGUF and llama.cpp hub on this site). On a Mac with an M-series chip it uses the GPU, and on Windows it supports NVIDIA and AMD Radeon GPUs.
Install
The README gives one command per platform.
On macOS, run curl -fsSL https://ollama.com/install.sh | sh or download Ollama.dmg. The macOS docs ask for macOS Sonoma (v14) or newer, with Apple M series for CPU and GPU, or x86 for CPU only.
On Linux, run curl -fsSL https://ollama.com/install.sh | sh. The Linux docs describe a manual install and an AMD GPU package.
On Windows, run irm https://ollama.com/install.ps1 | iex or download OllamaSetup.exe. The Windows docs ask for Windows 10 22H2 or newer, and NVIDIA driver 551.61 or newer if you have an NVIDIA card.
Run a model
ollama run gemma4
The first run downloads the model, then opens a chat. The CLI reference’s other everyday commands are ollama pull gemma4, ollama ls and ollama rm gemma4. The full model list is at ollama.com/library.
Tags and quantisation names
A model name can carry a tag after a colon, for example gemma4:31b. The library’s tag pages list more specific tags that name the quantisation, such as qwen3:14b-q4_K_M, qwen3:14b-q8_0 and qwen3:14b-fp16. The suffix is the weight format: q4 and q8 are 4-bit and 8-bit quantisation, and fp16 is unquantised 16-bit. The GGUF hub on this site explains the K, S and M letters.
Where models come from
Most models come from one of two places: the Ollama library, or Hugging Face, since Ollama can run any GGUF repository directly.
ollama run hf.co/{username}/{repository}
ollama run hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF:Q8_0
Hugging Face’s docs say Ollama uses the Q4_K_M quantisation by default when the repo has it, and otherwise picks one reasonable quant from the repo. The quant name after the colon is case-insensitive. You can also import a local GGUF file with a Modelfile containing FROM /path/to/file.gguf, then run ollama create my-model. Ollama does not quantise GGUF files during import.
Context length
Ollama sets the default context length by memory: 4k below 24 GiB of VRAM, 32k from 24 to 48 GiB, and 256k at 48 GiB or more. The docs suggest at least 64000 tokens for agents, coding tools and web search. A larger context needs more memory.
OLLAMA_CONTEXT_LENGTH=64000 ollama serve
Inside ollama run, use /set parameter num_ctx 4096. ollama ps shows the context size in use and whether the model is split between GPU and CPU.
OpenAI-compatible API
The server listens on http://localhost:11434. OpenAI clients work if you point base_url at http://localhost:11434/v1/ with any API key value, which Ollama ignores. The docs list /v1/chat/completions, /v1/completions, /v1/models and /v1/embeddings, plus a stateless /v1/responses. The OpenAI API cannot set context size, so create a model from a Modelfile with PARAMETER num_ctx instead.
FAQ
Does Ollama run on a Mac?
Yes. The macOS docs require macOS Sonoma (v14) or newer, and an Apple M series chip gets both CPU and GPU support. x86 Macs run on CPU only.
How do I run a Hugging Face GGUF model in Ollama?
Use ollama run hf.co/{username}/{repository}, and add : plus a quant name such as Q8_0 to pick a specific file.
How do I give a model a longer context?
Set OLLAMA_CONTEXT_LENGTH when starting ollama serve, use /set parameter num_ctx in a chat, or pass num_ctx through the API. More context uses more memory.
Is Ollama’s API a drop-in for OpenAI’s?
It supports a subset. Chat completions, embeddings and model listing are covered, but some features such as logprobs are not.