Run open models 3
How to run open-weight models on your own machine. Start with the runtime that matches your hardware, then copy the commands.
- GGUF and llama.cpp: run quantised open modelsWhat the GGUF format is, how to run it with llama-cli and llama-server, how to read quant names like Q4_K_M, and how LM Studio and Ollama relate.
- Run open models with MLX on a MacHow to run open models on Apple silicon with MLX and mlx-lm: install, mlx_lm.generate, the local server, mlx-community repos, unified memory and mlx-vlm.
- Run open models with OllamaHow to install Ollama on macOS, Linux and Windows, run open models, pick quantised tags, pull GGUF from Hugging Face and use the OpenAI-compatible API.
Keep exploring
- WatchNew labs and architectures, with vendor claims checked against sources.
- LicensesWhat each model license allows, in plain English.
- China DeskWho owns the Chinese labs and what their models are.
- GuidesWhich hardware to pick and what it can run.
- CUDA and alternativesCUDA, ROCm, Metal and the other stacks that run models.
- Model typesText, coding, image, video, audio, embedding and vision models.