Set up Ollama (Local Models)
Run AI models locally on your machine. No API key required. Completely free and private.
Step-by-step
1
Install Ollama
# macOS brew install ollama # Linux curl -fsSL https://ollama.com/install.sh | sh # Or download from: https://ollama.com/download
2
Start the Ollama server
ollama serve
This starts a local API server at http://localhost:11434. On macOS, the Ollama app starts this automatically.
3
Pull a model
# Pull Llama 3 (recommended) ollama pull llama3 # Pull other models ollama pull mistral ollama pull codellama
Models are downloaded once and cached locally. Llama 3 (8B) is about 4.7 GB.
4
Verify it works
ollama run llama3 "Hello, world!"
5
Configure your AgentVoy project
No API key is needed. Just set the model in your .env:
# No API key needed for Ollama! DEFAULT_MODEL=llama3
100% free and private. All inference runs locally. No data leaves your machine. No API costs.
Compatible frameworks
CrewAI
Popular models
| Model | Size | Best for |
|---|---|---|
| llama3 | 4.7 GB | General purpose |
| mistral | 4.1 GB | General purpose, fast |
| codellama | 3.8 GB | Code generation |
System requirements
RAM: At least 8 GB for 7B models, 16 GB for 13B models
Storage: 5-10 GB per model
GPU: Optional but recommended for faster inference