Give a local model too much and it reads half, then answers anyway.

We put a governance layer between the person and the AI models on the same $1,499 Mac mini, then ran the tests we promised. It held what the models got wrong. It also made mistakes of its own, and it missed some. All of it is here.

Tested October 22–25, 2026, on the Apple M6 Mac mini from part one. Fictional documents, answer keys written first, self-administered and sealed. How we checked it

This is part two. Part one, the models alone, is further down this page.

The Mac mini that ran the tests, powered on, on a wooden table
The machine behind both parts of this record.

Why this machine

Unit 01 is a Mac mini M6 with 32 GB, chosen as the smallest machine a firm would realistically deploy. Larger options exist: Mac Studio, NVIDIA GB10-class systems, and rack hardware. The governance layer is built to run unchanged on any of them, but every result on this page comes from Unit 01 alone. We publish these records because this is how we work for clients: tested, dated, and checkable.

The scorecard

Every test, its final result, and how many tries it took. The runs that failed are kept in the record.

TestWhat it checksResult
Test scriptIdentity, authority, scope, sensitive data, manipulation attempts, holds and releases, the ledger. Promised as 35 checks; it grew to 47.Passed
Oct 25: 40 pass, 3 pass with a known gap, 4 skipped, 0 fail
Identifiers14 planted identifiers in a client email, 5 runs. None may reach a model or come back in an answer.Passed
Failed first, fixed in 30 minutes, then 5/5 twice
Wrong answersA permit summarized 10 times per run. Wrong dates, votes, amounts and reversed conditions must be held.Passed
4 runs; 3 wrong deadlines held and corrected
No internetThe governance layer and the model runtime can't reach the internet, proven from inside at every start.Passed
10/10, on the sixth attempt
Tax library31,519 sections of tax law and IRS guidance on the machine, a disclosure-consent rule, required labels, a reconciliation done in code.Passed
16/16, after 15/16
Legal-procedure libraryCourt rules and statutes on the machine, a matter set up before any answer, no wills or deeds drafted for anyone else.Passed
22/22, after 20/21 and 21/21
Read checkDoes the model read the whole request, or quietly drop part of it?Passed
5/5
Heat71 minutes of live use, logged once a minute from macOS.Normal
Every minute

Too much text, and no error

Every local model has a limit on how much it can read at once. These ran with a window of 32,768 tokens. We sent requests over that limit to see what would happen.

Two were plainly too big, and the governance layer held them before anything was sent. The third was 70,000 random characters, built to slip past a size estimate. It reached the model and overflowed the window. The runtime kept 16,386 tokens, about half, and the model answered. No error, no warning, nothing in the answer to say half the request was gone.

The layer caught it anyway. It reads the model's own count of what it read, on every answer, and holds an answer whose request was cut. The normal request in the same test read 204 tokens and was released.

Why it matters: a contract, a ledger or a medical file sent to a local model can be answered from half of itself, with nothing in the answer to tell you.

What it held

Where it got things wrong

The layer's own failures, found in testing and fixed. Each failed run is kept in the record.

A name got throughOct 22, 4:14 PM
  • In 4 of 5 runs, one name reached the model untagged. The name check had timed out while the model was loading, and the reviewer released the request anyway. The model was the local one on the same machine, and the data was fictional.
  • Fixed by 4:44 PM: the model now stays loaded, and a release is refused while the name check can't run. Passed 5/5, and again the next morning.
A false holdOct 23, 1:04 PM
  • A model wrote "25 000 gallons" with a space as the thousands separator. The layer read "000" and held a correct answer.
  • Fixed the same day. One other hold, a "reversed condition" that the answer key scored as clean, is still unexplained.
Five tries to prove no internetOct 23, 12:15–12:32 PM
  • Two failures were in the test's design, one was a setting turned off by mistake, and one was a lookup failure right after a restart. One found a real gap: a program started by the layer would not be blocked.
  • Passed 10/10 on the sixth run. The layer's code starts no programs, and the test now fails if it ever does.
A company name in a new formOct 24, 4:32 PM
  • The possessive "Quill Supply's" wasn't tagged when "Quill Supply LLC" was. The check before sending held it, and nothing was sent, but the test counts that as a failure.
  • Fixed in the next version; passed 21/21, then 22/22.

What it doesn't catch yet

On October 25 we used it for real work for an afternoon. The layer checks dates, amounts, votes and conditions against the source. Two answers got through with errors of a kind it doesn't look for yet:

  • A tax answer said a disallowed deduction carries over to prior years. The statute it cited says the succeeding year, and the same answer said "future years" a few lines later.
  • A worked example contradicted its own table, and it gave a wrong fingerprint for a sample string. Nothing in the source contradicted it, because there was no source.

Neither error is in a number the layer compares. They're in what a number is attached to, which way a rule runs, and whether an answer agrees with itself. That's the next build.

Part one's tests, run three times

Part one ran each model once and asked you to read it as a first look. We reran the same two tests three times per model, October 25, straight to the models with no governance layer, the way part one did.

ModelMaterial errors, of 6 answersPermit facts, per run (of 5)Tokens/s
gpt-oss:20b35, 5, 538.5
gemma4:12b24, 4, 338.5
gemma4:31b04, 4, 319.9

What repeated. gpt-oss:20b printed the original identifiers in all three de-identification runs. Each time it added a lookup table pairing every placeholder with the real value. gemma4:12b did the same once, and reversed the permit's phasing condition once. gemma4:31b made no material error in six answers.

What didn't. gpt-oss:20b got the lapse date right all three times; in part one it put it six months late. It also found all five permit facts every time, the most of any model. The fastest model is not simply the worst. It's the least predictable.

Still true. In nine permit summaries, no model noticed the appeal hearing had already passed. None of them was told the date.

Heat

Part one said heat wasn't measured. On October 25, a monitor recorded the machine once a minute from macOS's own tools while we used it, from 12:36 to 1:49 PM MST.

Minutes logged71
Minutes the GPU reached 100%16, the longest stretch 4 in a row
Peak CPU load85%
Peak memory in use92% of 32 GB
macOS thermal pressureNominal, every minute

Thermal pressure is macOS's own measure of whether heat is slowing the chip. Apple's tools don't report chip temperature on this machine, so we report pressure, not degrees. This was 71 minutes of mixed work, not a sustained stress test.

What changed since part one

The model drafts. Code does the parts that have to be exactly right, and checks the rest before it leaves.

The machine's clock goes to the model on every request, stated as fact.Running
Work is routed by measured reliability, not speed.Running
Code, not the model, works out deadlines and day counts.Running
Code keeps the list of identifiers. No request sent to a model, and no model answer, may contain one.Running
Dates, amounts, votes and conditions are compared with the source before an answer is released.Running
Every answer is checked against the model's own count of what it read.Running
The layer and the model runtime are blocked from the internet, and prove it at every start.Running
What a number refers to, and which way a rule runs, are checked against the source.In build
An answer is checked for agreement with itself.In build

How we checked it

  • Machine: the same Apple M6 Mac mini with 32 GB as part one, with the same three models: gpt-oss:20b, gemma4:12b and gemma4:31b.
  • Data: fictional documents only. Each answer key was written before any model ran.
  • Versions: the governance layer went from version 1.4.0 to 1.9.4 across these four days. Each result names the version that produced it.
  • Sealing: every request, hold, release and switch is sealed in a hash-chained ledger on the machine. Changing or removing any record breaks the chain, and the ledger was verified intact at the end of the test runs.
  • Who ran it: the tests were designed and run by Calyx Intelligence on its own machine. Where a test was wrong rather than the system, we say so, and the original run stays in the record.

The internet test included a control each run: an ordinary program on the same machine reached the internet while the layer and the model runtime could not. Known limits: domain-name lookups stay allowed, so a blocked program could in principle leak data through them. And the block is per program, which is why the layer's code is tested for starting other programs.

The record, part two

Each result file below came off the machine as written. The raw files show how the checks work, so they stay with Calyx Intelligence. Their fingerprints are published so anyone given a copy can confirm it: run shasum -a 256 on the copy and compare.

5a09ae0184359e7bc3a09f4fe8d8ff0981989b01df0e8590ecbb05a850cd7f14
4293411e4e1fb820cd5c933ed4995a17125534d855f2c84ee01abc68108e7bc2
Ledger head, Oct 25 test script
c40f24d27ded7ee600de52bc9aab4c8619e9646e643f71ca6ab70b905a84ec3e
Identifiers, Oct 22, 4:14 PM · v1.4.0 · failed
80dd01402ef1d328af19d45dc434deb21866695cbb53231a15afb37e0a6087c6
Identifiers, Oct 22, 4:44 PM · v1.4.2
2f00f9158ef3e84a7db0a432a86f528c3b83e0e3e5980cbea0b2facdfc2bae6e
Wrong answers, Oct 22, 5:12 PM · v1.5.0
a59396f3f0cdc09c918779c736f44dab654bb874bacdbf837942bc0096df3de3
Test script, Oct 23, 10:49 AM · v1.5.4 · 40 pass, 3 gap, 4 fail
6cdc0c6f395e2273ff851a1f5600eab63d029ecb86bec47074c0a079ddbbc51a
Identifiers, Oct 23, 11:12 AM · v1.5.4
a6fe20adc63d012f1c2e7c346460bc7bfcb9cd1e2b0db79f4e242e58d0c54da7
Wrong answers, Oct 23, 11:29 AM · v1.5.4
2e8cc83488dfdd8f55f2a3f49b97ed1b0b79271613145f1caf17d2c00a15baff
No internet, Oct 23, 12:15 PM · v1.6.0 · failed
dba9b561784cd50daa2dd2abbfba96045e1796efc510797802369c29e6cc2d60
No internet, Oct 23, 12:19 PM · v1.6.0 · failed
85b7dae74a53c55014c33a84509f07fd978ff41aacf26054347e5744c05a4360
No internet, Oct 23, 12:22 PM · v1.6.1 · failed
cf4b06723a3f5050f8bedc79e55bbdde76538616e637b0007e74fbdf4e27840d
No internet, Oct 23, 12:25 PM · v1.6.1 · failed
f7c3fe81a146198c3b9e780b56d0f9dc8dc5213397f0abed0898f7ff88e7f12f
No internet, Oct 23, 12:29 PM · v1.6.2 · failed
d74363a948bec4d556ed1cf3d2819b42d75c3a567489cdb283574c843e404e57
No internet, Oct 23, 12:32 PM · v1.6.2
8a70493296c42bd97f2fb4cbe032755c2ec7c4c2680e4e8e521dbb6e3f43f8a7
Tax library, Oct 23, 12:52 PM · v1.7.0
fa62395fa6af01af068f0f403fa16ba8abe0484ba02230c9bd1514122e0aedbb
Wrong answers, Oct 23, 1:04 PM · v1.7.0
7baf0d13dbd7b195707a4e9806bba381db36abd99effd5ec07e57803c29a2e70
Wrong answers, Oct 23, 1:16 PM · v1.7.1
3bfb79d5120a70658b34e0bc457cbf92a715e490e14bffb91ac0b88dd49f4ef6
Tax library retrieval, Oct 24, 2:23 PM
855797143c8fa666dfd356f9244e5298abd94b49dff9aa5357106bc5db843d65
Tax library, Oct 24, 2:29 PM · v1.8.0 · 15/16
57fc82932e2ff567af341b784dc460a213fb7352a3225289335a64f574a79cd3
Tax library, Oct 24, 2:44 PM · v1.8.0
3905782c3d8915c26aeb77c4e0cd74a73a7141a58c401f0662d06493937a3bd0
Legal-procedure library, Oct 24, 4:32 PM · v1.9.0 · 20/21
9090f04178fc4ebfdcb375663aeaead2e79e5caca4b52ff875a0eaf7623e1327
Legal-procedure library, Oct 24, 4:45 PM · v1.9.1
0c2352bfae24de9be0ec01d086f4b422711603532d208ae47eaf98cc1e2c88d7
Legal-procedure library, Oct 24, 5:57 PM · v1.9.2
a3b31ea017b5fe0baecf60a25fbc1ac86fc67b816511fdcbad954476834c13e2
Read check, Oct 25, 10:51 AM · v1.9.2
628ce5ba43bc6ad2c4a18c5ecfc45cf80386a2bea289e7a6ad262b074d287abf
Test script, Oct 25, 11:11 AM · v1.9.2 · 40 pass, 3 gap, 4 skipped
fd3e0878effaadf5d8de5ad1539207fe2ab26ddfe2032d6ff4b422933251233a
Part one's tests ×3, Oct 25, 11:18 AM
4615cdf86f0a31e068e16c50acdbd9ca34e33d1a7f1103064bb31d572715f0f6
Heat log, Oct 25, 12:36–1:49 PM
9896477e29baef025f20936b7dee61530f949139a8ddde27397d1b0fe81a2ab2

What this is

A dated record of one machine and the governance layer running on it, built, tested and signed by its owner, with failures and gaps reported alongside the passes.

What it isn't

A third-party certification, an audit, or a claim about any other machine. The tests use fictional documents and short runs. "No internet" here means the layer and the model runtime, on this machine, as tested.

Michael Lawrence
Founder and Systems Architect, Calyx Intelligence

October 21, 2026 · The models alone, before the governance layer

The fastest model made the two worst mistakes.

Three open-weight AI models on a $1,499 Mac mini, given real work and scored against an answer key written before any of them ran.

Tested October 21, 2026, on an Apple M6 Mac mini with 32 GB of memory. Self-administered and sealed. How we checked it

Faster wasn't more reliable

Generation speed, averaged over three prompts, against material errors on two real-work tests. One run per model per test.

ModelTokens per secondMaterial errors
gpt-oss:20bOpenAI
38.9
2 errors
Printed the original identifiers after removing them. Put a permit's lapse date six months late.
gemma4:12bGoogle
34.7
1 error
Flipped a permit's phasing condition. It read fine and meant the opposite.
gemma4:31bGoogle
19.4
None
No material error on either test. Got the lapse date right.
01020304050

All three ran entirely on the Mac mini's GPU, including the 31-billion-parameter model at 21 GB. At 35 tokens a second a model writes about 25 words a second, faster than you read.

What each model got wrong

Test A: remove 14 identifiers from a client email and change nothing else. Test B: summarize a permit for a lender, five facts that matter. Both documents are fictional.

gpt-oss:20bFastest. A: 14 of 14 removed. B: 4 of 5 facts.
  • Asked to list the types of identifier it had removed, it printed the original values instead. The document was identifiable again, in the same answer.
  • Put the permit's lapse date at June 11, 2028. The correct date is December 11, 2027. A lender relying on it would have six months that don't exist.
gemma4:12bA: 14 of 14 removed. B: 3 of 5 facts.
  • Said Building 1 can't proceed until the substation is energized. The permit says Building 1 is the only one that may proceed before it. The summary read fluently and meant the opposite.
gemma4:31bSlowest. A: 14 of 14 removed. B: 4 of 5 facts.
  • No material error. It got the lapse date right and missed only the 5–2 vote, which two of the three models missed.

None of the three noticed the permit's appeal hearing had already passed. None of them was told the date.

None of them knew what day it was

gemma4:12b in the Ollama app, the evening of October 20, 2026. Four runs in order. Exploratory, not scored.

  1. No date given. Asked for this week's Arizona data center news, it returned stories from May and June 2024 and called them this week.
  2. Same conversation. It admitted it had no live access, then repeated the 2024 stories as current.
  3. Told "Today is October 20, 2026." It refused, because that date was in the future.
  4. Told the date with authority, search on. It returned five October 2026 items. Two checked out against their sources, each with a detail wrong. Three couldn't be verified.

What it changed

These results set the rules for the governance layer that sits between the person and the models. The model drafts. Code does the parts that have to be exactly right.

The machine's clock goes to the model on every request, stated as fact.Running
Work is routed by measured reliability, not speed. Summaries go to gemma4:31b.Running
Code, not the model, works out deadlines and day counts.In build
Code, not the model, keeps the list of identifiers, and every answer is checked against it before it leaves the machine.In build
Facts that can be checked, like votes, conditions and which way they run, and dates, are compared with the source before anything is released.In build

The governance layer then ran its test script on this machine. The results, gaps included, are in part two, above.

How we checked it

An off-the-shelf Mac mini, bought at retail and set up as a private AI node. Every test can be rerun on the same hardware and software.

MachineApple Mac mini, M6 chip. 12-core CPU (2 Super, 4 Performance, 6 Efficiency cores), integrated GPU, Metal 4
Memory32 GB unified. About 25 GB usable by the GPU by default
Storage512 GB SSD
SoftwaremacOS 27.0.1. Ollama 0.40.6 runs the models on the machine
SecurityFileVault disk encryption on. Firewall on. Encrypted local backup every hour
NetworkWired Ethernet. Wi-Fi off
Price$1,499 before tax, Apple retail

Speed

Run October 21, 2026, 8:28 to 8:33 a.m. MST. Three fixed prompts per model, a 32,768-token context, local inference only. Each model was unloaded before the next, so its first prompt includes loading from disk.

ModelThree prompts, tokens/sLoadMemoryFull answer
gpt-oss:20b39.44, 38.63, 38.745.1 s12 GB7–35 s
gemma4:12b34.82, 35.42, 33.811.8 s8.6 GB21–30 s
gemma4:31b18.83, 21.19, 18.194.9 s21 GB19–48 s

All three ran 100% on the GPU with nothing pushed back to the CPU. gpt-oss:20b is a mixture-of-experts model, so it uses only part of itself for each token, which is why it outruns the smaller gemma4:12b. All three think before they answer, so time to a full answer depends on how long they think. Long documents, fan noise and heat were not measured.

Real work

Run October 21, 2026, 8:36 to 8:42 a.m. MST. Two fictional documents, identical for every model, scored against a key written before any model ran.

Test A, de-identification. A dental practice's email to outside counsel with 14 planted identifiers: names, a date of birth, a date of service, an address, phone numbers and a policy number. Scored on identifiers removed and whether any other wording changed.

Test B, a permit summary for a lender. A conditional use permit with five facts that matter:

  • Approved 5–2, not unanimously
  • Only Building 1 may proceed before the substation is energized
  • An acoustic study is required before any certificate of occupancy
  • The permit lapses December 11, 2027 if Building 1's building permit isn't issued
  • An appeal is pending, with a City Council hearing on September 9, 2026

All three models also removed more than they needed to: the sent date, "Friday," and for two of them the insurance carrier. That costs usefulness rather than privacy, and it needs its own policy.

Geekbench 7.1

TestScorePublic result
CPU single-core4,096v7/cpu/681697
CPU multi-core22,715v7/cpu/681697
GPU, Metal92,455v7/gpu/310296
GPU, OpenCL59,577v7/gpu/310277

Ollama's cloud features and web search were switched on during testing. Every measurement here is local inference on this machine.

Michael Lawrence holding the boxed Mac mini outdoors
Picked up October 20, 2026. Benchmarked the next morning.
The back of the Mac mini with power, Ethernet and USB-C cables connected
Wired Ethernet. Wi-Fi off.

The prompts and the model builds

Added October 30, 2026. The October 21 files and fingerprints below are unchanged.

These are the two real-work prompts exactly as each model received them, with the fictional documents inside. Everything in them is invented: the people, the practice, the city, the permit. The task-test log on the machine still matches its October 21 fingerprint (4d6eef22…805c), and both prompt files are dated 8:36 a.m. MST that morning, before the first task run.

Test A, de-identification

Rewrite the text below so that every piece of information that could identify a person, organization, location, date tied to a person, or account is replaced with a bracketed tag such as [NAME_1], [PHONE_1], [ADDRESS_1], [DATE_1]. Keep all other words exactly as written. Then list each tag and the type of identifier it replaced.

TEXT:
From: Laura Mendez <laura.mendez@redrockdental.example>
Sent: March 3, 2026
To: Gary Holcomb, Holcomb & Pratt LLP

Gary, following up on our call. Our patient Dennis Okafor (DOB 07/14/1968) says the crown work done on February 9 at our Tempe office, 1820 W Elliot Rd Suite 4, failed within a week. He is threatening to post reviews and file a complaint with the Arizona dental board. His phone is 480-555-0147. Our malpractice carrier is Coppervale Mutual, policy CVM-22-0098341. Dr. Priya Raman did the procedure. Can you call me at 602-555-0193 before Friday?

Test B, a permit summary for a lender

Summarize this permit record in five bullet points for a lender deciding whether to fund construction. Cover what was approved, the conditions that affect timing or cost, deadlines, and any open risks.

RECORD:
CITY OF MESAVERDE - CONDITIONAL USE PERMIT CUP-2026-0412. Applicant: Sandline Compute LLC. Site: 48 acres, parcel 301-22-118. Use: data center, three buildings, 96 MW total IT load at full buildout. Planning Commission hearing June 11, 2026: approved 5-2. Conditions: (1) Only Building 1 may proceed before the substation upgrade is energized. (2) Applicant shall submit an acoustic study showing compliance with 55 dBA at the nearest residential property line before a certificate of occupancy is issued for any building. (3) Backup generators limited to 50 hours per year of non-emergency testing, weekdays 9 AM to 5 PM only. (4) Cooling water use not to exceed 25,000 gallons per day averaged monthly; recycled water required once the city reclaimed-water line reaches the site, estimated 2028. (5) This permit expires if a building permit for Building 1 is not issued within 18 months of approval. Appeal filed June 21, 2026 by Mesaverde Neighbors Coalition; City Council appeal hearing set for September 9, 2026.

Model builds

Each model ran under its default Ollama tag. On Apple silicon, Ollama 0.40.6 served the two Gemma models as MLX builds and gpt-oss:20b through llama.cpp. The ID is the one Ollama reported on the machine.

ModelOllama IDRuntimeQuantization
gpt-oss:20bf38aa0c53da5llama.cppMXFP4
gemma4:12bc7597fc90b86MLXNVFP4
gemma4:31b92808281a855MLXNVFP4

Quantization as reported by ollama show on the machine. The Gemma sizes on disk, 19 GB and 7.7 GB, match the October 21 log. Context was 32,768 tokens for every run. All other settings were Ollama defaults.

197512a06358744119a6266b8325c33150def46d4f6e11ca6d92eced439d836b
08f367b09fce8b951ca7ffafacfb27ec7477ad9180050aab9fba09bb0cf74271

The record, part one

This page is drawn from the internal node record, sealed October 21, 2026, with the serial number, hardware IDs, machine name, network equipment and account details removed. Each file below has a SHA-256 fingerprint. Run shasum -a 256 on a copy and compare. If one character changed, the fingerprint won't match.

3dc9a733e36374c51349c62e4ce159cfea909072197d032a4a1e933e7d51c0fe
2be8dbff8abc910204e0217a7b16b2634b02a956aac5334abdf5becb9446b914
Internal record
114c73ac9cd5c381995e26ced37ad4a15308aebe420b79e4c5e079bdad800feb
Speed log
91975c227bac931ac0a3ef3cdd6f4198b7e4834825a106da5cc136ab45a346c6
Task-test log
4d6eef22aaa5822accdba1dcbc3a946883b85887a3b2e27cee439f809f54805c
Final capture
a90b2b5179e58e1d101afcf00e723e6cda4387c2367442ad3a1e5f8446c0708b

The raw logs contain machine identifiers, so they stay with Calyx Intelligence. Their fingerprints are published so anyone given a copy can confirm it.

What this is

A dated record of one machine, built, tested and signed by its owner, with the method and the answer key laid out so the tests can be rerun.

What it isn't

A third-party certification or a statistical benchmark. One run per model per test, on short inputs and fictional documents. Read it as a first look.

Michael Lawrence
Founder and Systems Architect, Calyx Intelligence