Tools
What we use
when the model runs locally
Not a survey of the market — a short list of things that actually run here, each with the point where it stops being enough. One of them we wrote ourselves, because nothing answered the question that comes up with every local model: did it even receive what I sent?
Ours
LlamaScope
A dashboard light for your local model — macOS, free, open source.
One screen showing what your Ollama is really doing: which model is loaded, how much of it sits in GPU memory, GPU utilisation, memory pressure.
And the part you cannot see anywhere else: whether your prompt was silently truncated. Ollama does not report this through its API — the only trace is a line in the server log. The symptom misleads, because it looks like a hallucination: the model “ignored” the documentation you pasted and answered as if it had never seen it. It did not ignore it. It never got it.
What this program sees
The proxy — the part that catches truncated conversations — is off by default. Turned on, it sits between your client and Ollama and sees the contents of your prompts, because that is the only place the request is visible before the server cuts it. It records numbers from them, never the text, and only on your own disk. Nothing leaves your machine — we receive nothing either.
So that none of this has to be taken on trust, the proxy is a separate program inside the bundle. “Off” means the process is not there, and you can see that in Activity Monitor. The app itself contains none of its code, and one command proves it:
Must return zero:
nm /Applications/LlamaScope.app/Contents/MacOS/LlamaScope | grep -c ProxyCoremacOS 14 or later, Apple Silicon
There is no Intel build, on purpose: GPU readings would mean nothing there, and a zero that looks like a measurement is worse than no program at all. On Intel, the Python script in the repository still works.
Through Homebrew:
brew tap MarcinSNowak/tap
brew install --cask llamascopeOur own tap, not the official Homebrew repository — that one asks a new package for 90 forks, 90 watchers or 225 stars. It installs exactly the same file as the button above, adding a checksum verification on the way.
- Version
- 0.9.1 · 1.2 MB
- License
- MIT
- SHA-256
- f49474b2984b79b4117f86b1b9ad6a64ee483e120c98c381f5d974ed10394c2c
Third-party
This list is short and meant to stay that way. It holds tools we have actually run and that do real work in the series — not tools we have read about. Saying “we recommend” about something we never tested would be the same kind of untruth LlamaScope exists to catch.
Ollama
A server that runs large language models on your own machine — one command, no account, no cloud.
The whole AI Basics series runs on it, and so do our own experiments. It installs in a minute and speaks an OpenAI-compatible protocol, so most libraries work against it unchanged.
It silently truncates prompts that do not fit and says nothing about it through the API — for a truncated conversation it may even report that it cut nothing. That is precisely why LlamaScope exists.
License: MITUsed in episode 01Project siteChroma
A lightweight vector database: it stores document fragments so they can be searched by meaning rather than by keyword.
We use it in the RAG episode because it runs as a Python library — no server, no Docker, no configuration. For one project’s knowledge base that is entirely enough.
It scales to a single machine and a single process. Concurrent writers, or collections running into millions of fragments, need something heavier.
License: Apache 2.0Used in episode 02Project siteSwiftBar
Puts the output of any script into the macOS menu bar and refreshes it on a timer.
The first version of LlamaScope ran on it — an ordinary Python script printing one line. A very good way to find out whether a tool is needed at all before writing an app.
The script restarts on every refresh, so there is no memory between readings — no history, no chart. That is exactly where this route ran out for us.
License: MITProject site