llama.cpp

Use llama.cpp's llama-server with NeoStack AI.

llama.cpp can serve GGUF models through llama-server, which provides OpenAI-compatible endpoints.

Start llama-server

Run llama-server with a GGUF model:

llama-server -m models/model.gguf -c 8192

By default, the server listens on:

http://127.0.0.1:8080

Add It To NeoStack AI

1. Open Settings > Chat & Agents > Chat Providers.
2. Click + Add Custom Provider.
3. Name it llama.cpp.
4. Set the base URL:

http://127.0.0.1:8080/v1

5. Turn off Require an API key unless you configured one.
6. Add the model ID manually if discovery does not show it.
7. Start a chat with Local & BYOK Chat and pick the model.

Notes

• llama-server supports OpenAI-compatible chat completions.
• Use a model with strong instruction-following and tool-use behavior for editor automation.
• Increase context size with -c if your workflow needs more conversation history.