A library of 69 collections on my own machine, and a model that must quote them
A quick look through the shelf, then one real question, time-lapsed with the true clock.
Built 2026-10-29 · Filmed 2026-11-03
I keep a copy of 69 reference collections on my own server, and a small model that answers questions from them. They include the full text of English and Portuguese Wikipedia, the English Wiktionary and 18 Stack Exchange sites, along with repair guides and chemistry textbooks. Every claim the model makes comes with the collection, the page and the exact words it relied on. Nothing leaves the machine.
- On disk69offline collections, 135.6 GB, served to my own devices only and searched on the machine
- Kept fresh2collections replaced in last week's run, 67 already current, 0 problems
- One answer260.5 sfor the question in the film, on a processor with no graphics card
What you're watching
Two beats, narrated in my voice in English or Português. First, a quick look through the library: its home page with the 69 collections, then a dozen pages from them, a few seconds each and never read. It runs from sustainable building and emergency guides to first aid and repair, and on to gardening and knots. The point is how much is on the shelf, all of it served from my own machine with no internet.
Second, the model. One question that crosses two of those areas: keeping a house cool with no power, and first aid for heat exhaustion. This is one real run, shown time-lapsed at 14 times speed with the true clock beside it. The model plans its search words and the library is searched. Then the model picks pages and sections, reads them and writes its claims. Each claim carries a quote, and the program checks that the quote appears word for word in the text the model was shown. The answer cites two of the collections from the first beat, Ready.gov and WikiMed. The real time was 260.5 s.
The weekly refresh is not in the film. Its last run on record looked at all 69 collections, replaced 2 of them and reported 0 problems. Only the newest snapshot of the disk is kept. Host, disk and folder names are relabelled on the screen, and no number is changed.
How the refresh decides
- Once a week it asks the catalogue of collections, by exact name and flavour, whether a newer edition exists.
- If one does, it reads the new edition's published size and checksum, and refuses when the download would pass the disk quota.
- It downloads into a staging folder and checks the exact size and the checksum.
- It takes a disk snapshot, then moves the new file into place.
- It rebuilds the library list and tells the running server to reload, without a restart.
- It checks that the server is the same process, that the new collection is listed, and that a search against it returns hits.
- Only then it deletes the old file. If a check in steps five or six fails, the old edition goes back.
How often the model gets it right
The 20 questions were fixed in advance, 10 of them written and frozen before any answer to them existed. One person judged the answers, and that person built the tool.
| Reading | Value |
|---|---|
| Correct against the page it cites | 9 of 20 |
| Partly right | 10 of 20 |
| Honestly unsupported (the sources did not hold the answer) | 1 of 20 |
| Questions held out, judged correct / partly / unsupported | 4 / 6 / 0 |
| Median time for one answer (shortest to longest) | 164 s (93 to 328) |
| Claims whose quote was found word for word | 94 of 101 |
| Questions where the needed page never reached the model | 7 of 20 |
The question in the film is not one of the 20. Its run took 260.5 s, against a median of 164 s over the test (93 to 328). One demonstration is not the typical result, which is why the page keeps the table next to it.
The container that serves the collections would only be a set-up. What makes this more is the refresh rule, which treats every swap as something to prove before the old file is deleted, and the way the model is held to the library: it may only quote it, and the program checks each quote against the text it was given. The film's own run is not special, and the numbers above are the ones to weigh.
What doesn't come across on video. A word-for-word quote proves the words are on the page, not that they support the claim. In the answer shown, claim 4 repeats the wording of claim 1 while its quote is about clothing, and the advice about window air conditioners does not fit a power cut. Both quotes are on the pages, so the check passes; whether a claim fits its quote or the question is not what the program checks. The rollback is implemented and was rehearsed on one path, a forced failure on one small collection, and I never filmed a failure. It is not proven on every path, and the swap of a very large collection has not been tried. The model is slow because it runs on a processor with no graphics card. And 7 of the 20 test questions went wrong because the right page never reached it, which is the next thing to fix.