Bottom line up front: an empirical benchmark of three ways to connect AI to small-business financial data — semantic retrieval (RAG), agentic file search, and governed structured query — scored on accuracy, token cost, and auditability against the five questions a business owner actually asks. The result most AI vendors have no incentive to tell you: the most expensive connection method exists mainly to compensate for messy books, while the two cheapest depend on and reward bookkeeping discipline. Clean books are cheaper than AI infrastructure. This follows "AI Debt: The Tax on Small Business" — AI Debt described the disease; this paper is the lab result. Paper #6 in the Thought-OS™ reading order.

Don Lovett, Fractional CFO & Managing Principal · ProjectBits Consulting · June 2026

Abstract

We connected an AI agent to the same small-business financial dataset three different ways — semantic retrieval over indexed documents (RAG), agentic file search over clean exports, and governed structured query against the system of record — and asked it the same five questions a business owner asks. We scored every answer on accuracy, token cost, and auditability, including the question every business model hides a different version of: do the books actually reconcile to the operational, payment, and customer systems the business runs on — and would they even show a problem if a whole dimension of how the business makes or loses money were missing? We planted two real-world instances we routinely find in client files: a payment-processor sub-ledger never reconciled to the general ledger (deposits booked net of fees against gross invoices), and a structural absence — a cost dimension the books were never built to record, so the answer is unanswerable as recorded. The results support a conclusion most AI vendors have no incentive to tell you: the most expensive connection method exists primarily to compensate for messy books, while the two cheapest methods depend on — and reward — bookkeeping discipline. Clean books are cheaper than AI infrastructure. The paper closes with the control architecture that makes the strongest method safe (Exhibit A: the Agent Control Matrix), a one-page cost model, and the full benchmark methodology for replication.

Introduction: The Disease and the Lab Result

In AI Debt: The Tax on Small Business, we argued that deploying AI without first naming your decisions creates a deferred liability — undocumented judgment, ungoverned automation, and cleanup costs that compound. That paper was the diagnosis. It described what the disease does to a business over time.

This paper is the lab result. We stopped arguing and ran the experiment.

The experiment exists because of a pattern in nearly every first conversation we have with an owner doing their own books. They have seen the demo: type "how’s my cash flow?" into a chatbot, receive a beautiful paragraph. They want that. What the demo seldom shows is where the paragraph came from, whether it is right, and what happens when an accountant, a lender, or the IRS asks them to prove it.

So we built the test the demo skips.

Design in one sentence: same business, same five questions, three different ways of wiring AI to the financial data, scored on the only three columns that matter when money is involved — Was it right? What did it cost? Can it show receipts?

Part I — The Test

The Five Questions

These are not benchmark trivia. They are the questions owners actually ask, usually at 11pm:

1. Margin. What was gross margin last month, and is it better or worse than the trailing average?
2. Receivables. Who owes me money more than 60 days, and how much in total?
3. Vendor spend. What have I actually paid [key vendor] this year, versus plan?
4. Runway. At current burn, how many months of cash do I have?
5. The trap. Do my books actually reflect what the systems my business runs on — my operational system, my payment processor, my CRM — recorded as happening? And if a whole dimension of how I make or lose money were missing from the books entirely, would I even be able to tell?

Question 5 is the one that separates a real diagnostic from a demo. The first four questions have answers sitting in the books, waiting to be retrieved. Question 5 asks whether the books are connected to reality — whether they reconcile to the operational and customer systems where the business actually happens, and whether they were built to record the thing that determines whether this particular business makes money. Every business model has a different version of this trap. The contractor’s books may have nowhere to put a job, so job costing silently doesn’t exist. The MSP’s books may carry a churning client as healthy recurring revenue. The card-revenue business may book processor deposits at the net amount that hit the bank while invoices carry the gross — revenue understated, fees never expensed, a mismatch compounding every month no one looks. The trap is a class, not a single error: the place where this owner’s books quietly misrepresent the one thing their business runs on. For our benchmark we planted a concrete, common instance of it — the card-processor case — and a second, deeper one: a dimension of cost the books were never built to record at all. Any AI connection method worth paying for has to handle both. The interesting question is which ones can.

The Three Doors

Clean Books Beat AI Infrastructure: the three doors to your data — the Librarian (RAG), the Filing Cabinet (agentic search), and the System of Record (governed query) — scored on accuracy, receipts quality, and setup cost, plus the industry signature defects the benchmark surfaces.

An AI model only knows what is placed in front of it at the moment of the question. The craft — what practitioners call context engineering — is choosing the right door for each kind of information. We tested the three doors that dominate the market:

Door 1 — The Librarian (RAG, retrieval-augmented generation). Every document the business has is chopped into chunks and indexed in a vector database. At question time, the system retrieves the chunks that are semantically similar to the question and hands them to the model to reason from. This is what sits under the hood of most products sold as "AI for your business," because it is the only method that functions — after a fashion — on messy data.

Door 2 — The Filing Cabinet (agentic search). No index. The agent receives one thing: a folder of clean monthly exports — P&L, AR aging, vendor payments, deposit detail — named identically every month. It searches filenames, opens files, and reads, the way a sharp staff accountant would.

Door 3 — The System of Record (governed structured query). A controlled, logged, read-only connection to the actual ledger and its supporting database. The agent does not read about the books. It queries the books, and the database answers with exact rows. Every query passes through a policy gate and lands in an audit log. (The control architecture is the subject of Part III and Exhibit A.)

The Setup Asymmetry — Read This Paragraph Twice

Before a single question was asked, the doors had already separated on cost. Door 1 required an embedding pipeline, a vector database, hosting, and a standing maintenance obligation: indexes go stale, embeddings drift across model versions, and re-indexing is forever. Doors 2 and 3 required almost no infrastructure — and one expensive-sounding thing instead: the books had to be clean first. Files consistently named. Accounts consistently coded. Vendors deduplicated. And — the part no software supplies — the books reconciled to the systems the business actually runs on, and structured to record the one thing that determines whether this business makes money. "Clean" is not generic hygiene; it is model-specific. A chart of accounts that is pristine for a consultancy is structurally blind for a contractor, because it has nowhere to put a job.

That asymmetry is the thesis of this paper. Infrastructure is a product, so the industry sells it hard. Discipline is not a product, so nobody sells it at all. The benchmark measures which one actually buys correct, provable answers.

Part II — The Scoreboard

Results Summary

(Every value below is the mean of five runs against a known-answer dataset; the full methodology and per-run results are in Appendix B, and the scoring rubric was fixed before the runs.)

†The four objectively-gradable questions are the receivables total (Q2), the vendor-spend total (Q3), the reconciliation defect (Q5a), and the structural-absence probe (Q5b), each scored against a known-answer key over five runs. Gross margin (Q1) and cash runway (Q4) are method questions graded for analytical soundness rather than a single number, and are reported separately. Every cell is the mean of five runs; full per-run results and variance are in Appendix B.

‡The Filing Cabinet got the receivables total right in four of five runs (one run miscomputed), and its token cost swung widely across runs — the single highest variance in the study (σ ≈ 1,400 tokens on that question alone). It can reach the right answer, but not reliably and not cheaply: reconciling figures across flat files in prose is clumsy work a query does in one line. Door 3 returned the same total in all five runs at roughly a third of the variance.

The Question 5 Story

Question 5 had two halves, and they failed the three doors in two different ways — neither of which was the failure we expected going in.

5a — the reconciliation defect. This is the instance we made concrete: a card-revenue business whose payment system and books had drifted apart. Pick a different business and the instance changes — the contractor’s job costing, the MSP’s churn buried in recurring revenue — but the shape holds: a system the business runs on, telling a different story than the books. The processor sub-ledger and the general ledger had not been tied out for twelve months: deposits were booked net of fees, invoices gross, and $9,742.88 of fee expense had quietly never reached the books. The System of Record returned the deposit rows beside the invoice rows and the gap was simply present in the output — visible, quantified, attached — in all five runs. The Filing Cabinet, reading the same figures from clean exports, found it too. The Librarian could not. Across five runs it concluded the question was unanswerable from what it had retrieved — and it was right to, because the two numbers it needed to compare never came back in the same retrieved passage. Detecting an unreconciled sub-ledger requires holding the gross figure and the net figure side by side; semantic retrieval pulls the chunk that best matches the question, not the two chunks whose difference is the answer. The Librarian did not get it wrong. It could not get it at all — which, for a tie-out, is the same business outcome.

5b — the structural absence. Then we asked which clients were unprofitable once the labor actually spent on each was counted. No door could answer, because the books had never recorded labor by client — and here is the result that surprised us: all three doors refused honestly. The Librarian, the Filing Cabinet, and the System of Record each reported, across all five runs, that the question could not be answered as recorded. A capable model recognizes a total absence regardless of how it reaches the data. That is the quiet, structural point under the whole paper: retrieval, search, and query can only work with what was written down, and this was never written down. The person who notices the column is missing in the first place is someone who understands the business model — and building the books to capture it is the work that comes before any of these tools can help.

So the honest scoreboard is sharper than the cautionary tale we set out to tell. The cheap doors do not blunder into confident, fabricated answers on a well-built absence; they decline it, same as the governed one. Where they actually break is the structured question with a real answer sitting in the data — the receivables total, the tie-out — and there the gap is decisive: the Librarian answered the receivables question wrong in every single run.

Answers versus Findings

This is the distinction the whole benchmark exists to make. An answer sounds right. A finding arrives with the evidence attached. When the number is going onto a tax return, into a covenant report, or in front of a buyer’s diligence team, answers are worthless and findings are everything. Door 1 produces answers. Door 3 produces findings. Door 2 sits between: it can list the files it read, which is real evidence, but reconciling two reports line-by-line in prose is clumsy work better done by a query.

The Cost Inversion

Now stack the columns, and watch the cheap-versus-correct trade come apart from the way it is usually sold. The Librarian is the cheapest door to run per answer — and it is the door the market sells hardest, precisely because it is the only one that tolerates messy books. But cheap is not the same as right: it answered the receivables question wrong in every run and could not perform the reconciliation at all. The Filing Cabinet is far more accurate, but it carries the highest token bill and the highest run-to-run variance — the right answer, sometimes, at an unpredictable price. The System of Record was correct on every structured question, declined the unanswerable one honestly, and produced a receipt for each answer — and it did so at lower per-answer cost than the Filing Cabinet, with a fraction of its variance. Its one real cost is upfront: a controlled connection and a policy gate, set up once.

What every door except the Librarian depended on is the thing money cannot shortcut and software cannot retrofit: clean accounts, consistent naming, correct mappings, a sub-ledger tied out to the general ledger. The governed door turns that discipline into provable answers cheaply. The expensive infrastructure exists to approximate that discipline for books that never had it — and the approximation still can’t sign its name to the answer.

You can pay for bookkeeping discipline once, or rent infrastructure forever to approximate it.

Clean books are cheaper than AI infrastructure. That is not a slogan. It is the scoreboard.

Part III — The Glass Box

The AI Stack: six layers from the deny-by-default control plane up through systems of record, data substrate, governed execution, rented intelligence, and integration — with the owner seated above the loop, setting policy the control plane enforces. Solid layers you own and can export; dashed layers plug in and swap without migration; amber layers are rented per use.

The Right Instinct

The most common thing owners say about AI touching their finances is not excitement. It is a question: "How would I know what it did?" That instinct is correct, and it is old — it is the same instinct behind every internal control ever designed. Part III describes what it looks like when that instinct is engineered into the system rather than bolted on after an incident.

Three words carry the architecture, each defined the CFO way:

Transparency — you can watch the work happen. The execution layer in our deployments is a visual workflow engine (n8n). Every AI workflow is literally a diagram: this node pulls the AR aging, this node drafts the reminder, this node stops and waits for a human. When a run executes, the diagram lights up node by node. Not a black box with a logo on it — a glass box with a flowchart inside. An owner, a bookkeeper, or an auditor can look at the procedure itself, not a marketing description of it.

Explainability — every answer carries its work product. When the agent reports your margin, the run log contains the exact query it executed and the exact rows it received. The explanation is not a paragraph the AI composed about its own reasoning; it is the actual workpaper, reviewable the way a staff accountant’s workpaper is reviewed. This is what elevates Door 3’s output from answer to finding.

Auditability — the trail survives. Every step of every run writes a timestamped record to an append-only event log: what was requested, what data was touched, which policy was evaluated, who approved, what was produced. Append-only means no one — including the AI — can quietly edit history. Your auditor does not have to trust the agent. Your auditor has to read a log. Auditors are very good at reading logs.

The Policy Gate: Deny by Default

Between the agent’s intention and any action sits a policy gate (in our stack, Open Policy Agent with policies written as code). It runs deny-by-default: the agent can do nothing that has not been explicitly permitted. The permission set reads like a controls matrix because it is one:

• Run a read-only ledger query: allowed, logged.
• Draft a customer reminder: allowed, logged.
• Send the reminder: allowed below $[threshold] of exposure; above it, queued for owner approval.
• Post a journal entry: always queued for approval. No autonomous path exists.
• Change a vendor’s bank details: denied, categorically. No rule permits it, so no phrasing of any request reaches it.

When the gate blocks an action, the block itself is logged — including which rule fired. Credentials never appear in the workflow at all; the agent’s connections draw from a secrets vault, so there is no password for the agent to see, leak, or be talked out of.

This is the harness paradigm stated plainly: the model provides intelligence; the harness provides control. Every governance fear an owner has about AI is resolved in the harness — and the harness is built from mechanisms finance has trusted for a century. Exhibit A maps them one-to-one.

The Owner Transition

Here is what the architecture actually means for the owner still doing their own books at 11pm.

The reason you haven’t let go is almost never love of bookkeeping. It is that you have never trusted anyone else’s output enough to stop redoing it. That is a rational position — until the output changes character. What this paper has described, end to end, is output you can verify without redoing: findings with evidence attached, procedures you can watch run, an append-only history, and a veto seated above every decision that matters.

That is the trade. You stop being the bookkeeper. You become the person the system reports to — the one who sets the thresholds, reviews the exceptions, and reads a one-page narrative instead of reconciling a payment processor at midnight. The machine does the work. A professional governs the machine. You govern the professional.

This is the position AI Debt warned you not to lose: human above the loop. Not in the loop — approving each transaction as it passes, which is just the overworked owner with extra steps — but above it: setting the policies, thresholds, and budgets the loop runs under, and reviewing exceptions on a cadence. What makes "above" real rather than aspirational is that your judgment is encoded in the policy gate — it runs on every decision even when you are not in the room. The agent improves within the policy you set; you revise the policy itself. That is the difference between owning an AI program and being a checkpoint inside someone else’s.

Live Receipts — the demonstrations behind this paper

None of the above is a description we are asking you to take on faith. Each claim has a runnable demonstration, against live infrastructure, that an owner or auditor can sit and operate:

• The audit trail is real and tamper-evident. The governed-query event log holds every query the system ran — request, policy decision, rule fired, rows returned, timestamp — append-only. The demonstration recomputes the log’s cryptographic hash chain on the spot and shows that even an edit made with full database-administrator privileges is detected, at the exact altered record. Append-only by privilege, tamper-evident by construction.
• The policy gate decides live. Type any query and watch the deny-by-default gate pass or fail it in real time, naming the rule that fired — a clean read passes; an attempt to write, to reach data the books never recorded, or to smuggle a write inside a read is refused before it reaches the database.
• The approval policy runs the real pipeline. The same policy engine evaluates live QuickBooks bills through a production workflow: a $250 invoice auto-approves, a $3,000 invoice routes to a manager, a $50,000 invoice escalates to the CFO, and a bill from an unapproved vendor is held — each decision made by policy-as-code, outside the model, and logged.
• The numbers flow through from the system of record. Post an invoice in QuickBooks and the receivables answer moves by exactly that amount, live; delete it and the answer returns. The figure is not a cached snapshot — it is derived from the books at the moment you ask, and it can be walked back from the answer to the exact query to the source rows.

These run on infrastructure the practice operates — a policy engine, an append-only ledger, a governed query path — the same components named throughout this paper. The point of a receipt is that you do not have to believe the claim; you can watch the mechanism produce it.

That is not surrendering control of your finances. By any definition an auditor would recognize, it is the first time you’ve had it.

What This Costs (One Page)

A realistic governed pilot — one financial workflow, full control architecture — for a business in the $500K–$50M range:

Infrastructure: $5–25/month. A small virtual server, or hardware you already own. Two of the three leading open-source harnesses are free software; the workflow engine is open source.

Model usage: metered, typically tens of dollars/month for one well-scoped workflow. Note a 2026 market reality: the major open-source harnesses can no longer ride flat-rate consumer AI subscriptions for frontier models; usage is pay-per-token API billing. Budget it like a utility, cap it like a purchase card (the harness enforces the cap).

Setup and governance design: the real line item. Defining the workflow, choosing the data doors, writing the policy gate rules, setting approval thresholds, standing up the audit log. This is professional work, performed once per workflow, and it is the entire difference between an asset and a liability.

The prerequisite, priced honestly: clean books. Doors 2 and 3 — the cheap, accurate, auditable doors — do not open onto messy books. If the chart of accounts is inconsistent, vendors are duplicated, or processor deposits are mismapped, the remediation is a bookkeeping engagement, not a technology purchase. This is the cheapest item on the page to fix and the most expensive to ignore.

The anti-budget. The alternative line item is the one AI Debt described: zero design cost up front, followed by unbounded cost later — rework, undocumented decisions, misstated revenue discovered at tax time or diligence. That cost compounds. This one doesn’t.

Exhibit A — The Agent Control Matrix

Mapping a century of financial control objectives to their harness mechanisms. Each row pairs a control any accountant recognizes with its software equivalent and the evidence it produces.

The point of the exhibit is not novelty — it is the absence of novelty. Nothing in the right-hand columns required inventing a new theory of control. The harness is internal controls, implemented in software, applied to a new class of worker.

Appendix B — Benchmark Methodology

Published for replication. Anyone with the same dataset shape can run this test and check our scoreboard.

B.1 Dataset

A sanitized demonstration company file representative of our client base (services business, ~$1.0M annual revenue), comprising:

• System of record: a QuickBooks-Online-shaped company file, 12 full months of activity, anchored to the financial scale of a real QBO sandbox company; chart of accounts conforming to the Tax Ready Bookkeeping architecture; held in a PostgreSQL schema (fqt_demo) that carries the transaction-level ledger, the supporting sub-ledgers, and an append-only event log. The dataset is generated deterministically from a fixed random seed, so the entire company is reproducible byte-for-byte.
• Document corpus (Door 1 input): 15 documents — monthly operating statements, a year-end reconciliation memo, processor reports, a vendor account letter, and a prior client-roster workpaper — chunked and indexed per B.3.
• Clean exports (Door 2 input): monthly P&L summary, AR detail, deposit detail, plus annual vendor-payment and bank-balance reports, exported as CSV with the fixed naming convention YYYY-MM_ReportName and derived directly from the system of record so the two stores agree.
• Planted defects (known-answer keys):
  1. Sub-ledger-to-ledger reconciliation failure: the card-processor sub-ledger and the general ledger were never tied out — processor deposits recorded net of fees for all 12 months, invoices gross. Known misstatement: $9,742.88 of fee expense unrecorded, revenue understated by the same. (~30% of revenue ran through the processor; fee rate 2.9%.)
  2. Structural absence: labor was never allocated to individual clients in the books — there is no job-costing or per-client time record anywhere in the system of record, the exports, or the corpus. The true per-client margin (held out of every door’s inputs) shows two of six clients unprofitable once real labor is counted. This defect is unanswerable as recorded by design.
  3. Dual-named vendor: the largest vendor appears under two name variants in the corpus ("Pinnacle Labor LLC" / "Pinnacle Contract Labor"), standardized in the ledger and exports. Known total spend: $57,000.00 against a $48,000 plan.
• All data is fully synthetic — no real client information is present; the dataset is available for inspection.

B.2 Models and Harnesses

Identical model across doors is a design requirement: the benchmark isolates the context-injection method, not model quality.

B.3 Door Configurations

• Door 1: corpus chunked at ~800 characters with 120-character overlap; top-k retrieval, k=4; no re-ranking. Configuration chosen to represent a competent default deployment, not a pathological one — it was tuned until it answered the single-number questions reasonably, then frozen before the scored runs.
• Door 2: agent granted read-only filesystem tools (list, search/grep, open) over the exports folder only, path-jailed; no index.
• Door 3: agent granted a single read-only SQL tool; every query first evaluated by the OPA deny-by-default gate (allow SELECT on the demo schema only; any write, DDL, cross-schema reference, or reference to the held-out table is denied, and the firing rule is recorded); allowed queries executed under a SELECT-only role; every call — allowed or denied — logged to an append-only, hash-chained event table.

B.4 Run Protocol

Each question asked 5 times per door (fresh session each time, no memory carryover) to capture variance — 90 runs in total (3 doors × 6 question-parts × 5). Prompts identical across doors except for the tool instructions inherent to each method. All runs recorded: full transcripts, tool calls, retrieved chunks / files read / queries executed, and token counts. Variance is reported alongside means in the scoreboard.

B.5 Token and Cost Accounting

Tokens counted as billed by the provider (input + output, including tool-call overhead and retrieved/loaded context). Door 1’s accounting includes the amortized indexing cost (embedding tokens ÷ expected query volume of 100/month) and excludes hosting, which is reported separately. Costs computed at list API pricing in effect on the run date (June 2026): $3.00 per million input tokens, $15.00 per million output tokens. The full 90-run matrix cost under $4 to execute.

B.6 Scoring Rubric (fixed before the runs)

Accuracy (per question): 1.0 = correct value(s) within rounding, correct direction/comparison where asked; 0.5 = materially correct but incomplete (e.g., found the mismatch but not its amount); 0 = wrong, or confidently unverifiable. Question 5a (the reconciliation defect) scores 1.0 only if the deposits are affirmatively identified as recorded net of fees; a fluent "looks correct" answer scores 0. Question 5b (the structural absence) scores 1.0 for correctly declining to answer — stating that per-client labor is not recorded and the question is unanswerable as recorded — and 0 for asserting any named client is profitable or unprofitable, since the books contain no basis for either claim. Grading the numeric questions (5a, the receivables total, the vendor total) is deterministic against the known-answer keys; the structural-absence judgment and the two method questions (margin, runway) were graded by the practice’s reviewer against a rubric fixed before the runs.

Receipts (per question, 0–3): 0 = assertion only. 1 = sources named but not inspectable. 2 = inspectable evidence (passages/files) without selection rationale. 3 = full workpaper — the exact query or file content from which the answer follows deterministically.

Cost: mean billed cost per answer, per B.5.

Scoring authority: numeric answers graded automatically against the known-answer keys in B.1 (exact match within a $1 tolerance); the structural-absence and method questions graded by the practice’s reviewer (Don Lovett, ProjectBits Consulting) against the rubric above. The full run archive — every transcript, query, and grading decision — is retained, and the governed-query receipts are held in an append-only, hash-chained event log whose integrity can be re-verified independently.

B.7 Limitations

Single dataset, single business shape, single model family per run; Door 1 results depend on chunking and retrieval configuration, which we froze at a defensible default rather than an adversarial worst case; the planted defects, while drawn from real client patterns, are two among many possible. The claim this benchmark supports is therefore deliberately narrow: for structured financial questions against a small-business ledger, governed structured query dominates semantic retrieval on accuracy, cost, and auditability, and bookkeeping discipline is the binding constraint on the two superior methods. Broader claims await broader runs — which the methodology above is published to invite.

Appendix C — The Same Test on Hardware You Own

The headline benchmark used a commercial cloud model, billed per token. A fair question follows: does the result depend on renting a frontier model, or does it hold on a model a small practice can run on its own hardware, with no per-query bill and no client data leaving the building?

We re-ran the identical matrix — same dataset, same three doors, same questions, same scoring — against a local open-weights model (qwen2.5:32b) on a single Tesla P40 GPU, the kind of card that costs a few hundred dollars used. Nothing left the local network. There is no per-query API charge; the only cost is electricity and the one-time hardware.

C.1 Local scoreboard

Same dataset, same questions, same scoring as the cloud run. "Correct" on Q2 credits an answer that returns the right per-customer rows even if the model omits the final sum — substance over presentation. Cloud, for comparison: Door 1 2/4, Door 2 4/4, Door 3 4/4.

C.2 What carried over, and what didn’t

The shape of the result held — with one sharpening that matters more than anything in the cloud run.

Door 3 (governed query) reproduced exactly: 4 of 4, on a used GPU. The reconciliation defect was caught in all five runs; the structural absence was correctly identified in all five; the receivables and vendor totals came back right. The database did the aggregation, so a smaller model only had to ask the right question — and when it occasionally asked it in the wrong SQL dialect, it corrected itself across tool calls, every attempt passing through the same governed gate and into the same append-only log. The one rough edge was cosmetic: on the receivables question the local model reliably returned the two correct customer balances but sometimes did not print their sum. The rows were right; the arithmetic of adding them was left to the reader.

Door 2 (agentic search) is where the downgrade bit. On the cloud model it scored 4 of 4; on the local model it dropped to 2 of 4. The failure was the receivables question — the local model read the December file, found nothing overdue in that one month, and concluded there were no past-due accounts, never reasoning across the full year of files to compute days-outstanding. It also ran with wild instability: on two questions its token use swung by thousands of tokens run-to-run, and three of its heaviest runs timed out entirely and had to be retried. Agentic file-search asks the model to be the query engine, and a smaller model is a worse query engine.

Door 1 (RAG) was unchanged: 2 of 4, the same two it failed in the cloud — it cannot construct the reconciliation comparison, and it cannot aggregate the receivables. Retrieval’s ceiling is a property of retrieval, not of the model behind it.

So the gap between the doors did not shrink on cheaper hardware — it widened. The method that pushes the work into the database held its score; the method that pushes the work into the model lost a quarter of its accuracy and most of its stability. That is the opposite of what "just use a bigger model" would predict, and it is the whole argument: on clean books, correctness is a property of the method, and the method is what survives when the model gets smaller, cheaper, and private.

C.3 What this changes about the cost argument

The cloud cost table in Part II is real, but it understates the case. The strongest method in this study — governed query — does not require a frontier model at all. It runs on a used GPU, on-premises, at zero marginal cost per question, and still produces the exact-query-and-rows receipt. The expensive infrastructure the market sells is not buying correctness or auditability; on clean books, those come from the method, not the model. What clean books buy you is the freedom to choose the cheap, private, owned option and still get the right answer with a receipt attached.

Local-run methodology mirrors Appendix B; model: qwen2.5:32b served via Ollama on a Tesla P40; temperature 0; N per cell matches the cloud run. Full local run archive retained alongside the cloud archive.

Reading Order (updated)

1. The Three Perspectives — the readiness diagnostic: technology, finance, and people. ~16 minutes.
2. Tax Ready Bookkeeping + The AI Stack. ~29 minutes.
3. The CFO Operating System. ~15 minutes.
4. ProjectBits Thought-OS™ — the methodology umbrella. ~9 minutes.
5. AI Debt: The Tax on Small Business — the cost of deploying AI without naming the decisions first. ~22 minutes.
6. The Five Questions Test — the lab result: why clean books beat AI infrastructure. ~22 minutes. (this paper)

This paper is the sixth in the ProjectBits reading order. Read the series in order at projectbits.com/method. It is the lab result that tests AI Debt — why clean books beat AI infrastructure.

ProjectBits Consulting · projectbits.com/method · Reston, VA. Tax Ready Bookkeeping™, The CFO Operating System™, and Thought-OS™ are trademarks of ProjectBits Consulting, Inc. This paper is published for practitioner and client education.