What is Ollama?
Ollama lets you run large language models locally. Download a model once, run it entirely on your machine — no API keys, no internet connection required, no per-token costs.
Supported Models
- Llama 3.1 (Meta) — 8B, 70B, 405B
- Mistral / Mixtral — 7B, 8x7B
- Gemma 2 (Google) — 2B, 9B, 27B
- Phi-3 (Microsoft) — 3.8B, 14B
- Qwen 2.5 (Alibaba) — 0.5B to 72B
- CodeLlama — optimized for code tasks
- And 100+ more on ollama.com/library
Quick Start
# Install (macOS/Linux)
curl -fsSL https://ollama.com/install.sh | sh
# Pull and run a model
ollama run llama3.1
# OpenAI-compatible API (for integrating with apps)
curl http://localhost:11434/v1/chat/completions -d '{"model": "llama3.1", "messages": [{"role": "user", "content": "Hello!"}]}'
Hardware Requirements
A 7B model fits comfortably in 8GB VRAM. A 13B model needs 16GB. GPU acceleration works on NVIDIA (CUDA), AMD (ROCm), and Apple Silicon (Metal). CPU-only mode works but is slower.
Integration
Ollama's OpenAI-compatible API means you can point any tool that supports OpenAI (LangChain, LlamaIndex, Open WebUI, LibreChat) at localhost:11434 and it just works.