Run a model on your own machine with Ollama

Install Ollama, pull a model, and NeuralVault finds it at localhost:11434 with no API key and no account.

Last checked against the app on · View as Markdown

Steps

  1. Install Ollama from ollama.com and let it finish starting. It runs as a background service and listens on http://localhost:11434.
  2. Pull a model. In a terminal, "ollama pull llama3.1" is a reasonable first choice; smaller models answer faster on modest hardware.
  3. Open NeuralVault and go to Settings, then AI & extensions, then API keys. Ollama appears under "Providers that run locally" and asks for nothing, because there is no key to give a server on your own machine.
  4. Open the model picker in Chat and choose the model you pulled.

How to tell it worked

Send one message in Chat. If it answers, the whole path is working: nothing left your machine to do it.

If it does not work

Ollama is installed but no models are listed.
Installing Ollama does not download a model. Run "ollama list" in a terminal. If it is empty, pull one.
The app cannot reach Ollama.
Open http://localhost:11434 in a browser. If that does not respond, Ollama is not running, and no setting in NeuralVault can fix that. Restart it, then reopen the model picker.
Answers are extremely slow.
That is a hardware answer, not a configuration one. A model larger than your available memory will be swapped to disk. Try a smaller variant.