Ollama

Ollama is an open-source tool that packages large language models such as Llama, Mistral, Gemma, Qwen, and DeepSeek into single-command downloads that run on your own machine. A model registry, automatic hardware detection, and a Modelfile format for customizing system prompts and parameters reduce what used to be a GPU configuration project to something like installing an app.

It exposes a local REST API plus an OpenAI-compatible endpoint, so code written for cloud chat services can be pointed at localhost instead. It runs on macOS, Linux, and Windows, accelerates well on Apple Silicon, and can serve multiple models on capable hardware. It suits developers prototyping without API bills, teams with privacy or offline constraints, and tinkerers quantizing models to fit consumer GPUs; production-scale serving still calls for dedicated infrastructure.