The Beginning

A pipeline with named stages solves who gets to decide what's true — the deterministic half decides, Claude writes. It doesn't automatically solve a related but separate problem: after the fact, can I actually tell why a given decision came out the way it did, not just that it did? A score of 14 tells you an evidence item ranked highly. It doesn't tell a future version of me, six months later, debugging why a particular cover letter over-indexed on one job, what specifically drove that number.

The First Solution

Early on, I treated the score itself as sufficient. scoreCandidateEvidence() produces a number per item, sorted descending, and for a while that felt like enough — the ranking is deterministic, so surely it's already inspectable by definition. Auditable meant "not random," and I stopped there.

What Broke

A bare number doesn't survive contact with a real "why" question, and every domain in this platform eventually got asked one. "Why did this cover letter emphasize this job over that one" isn't answered by "14 > 9" — it's answered by knowing which keywords matched, in which field, and why that field carries the weight it does. "Why did the assistant say these two photos were taken on the same trip" isn't answered by a similarity float — it's answered by knowing which stored fact — a shared taken_at window, a matching location string — actually produced that claim. A score is a conclusion. It isn't an explanation, and I'd been treating the two as the same thing.

The Decision

The fix already existed in the codebase before I fully understood the pattern I was applying — nearly every scoring stage in this platform had, on its own, grown a field whose entire job is carrying the reason forward, not the result. EvidenceItem doesn't just carry a score; it carries a reason string, generated per item — "Matches 3 keyword(s) in tech stack and responsibilities" — and a matchedKeywords array naming exactly which terms drove the number. Photo search results carry matchedBy[], naming which retrieval path — tag, location, mood, semantic — actually contributed. matchPercentage is deliberately documented as not a probability; it's relative-to-top-result-in-this-set only, an honest caveat rather than a confident-sounding number that would oversell what it measures.

The photography assistant's Evidence Relationships pushed the same idea one level deeper. A relationship claim's type reuses EvidenceRelationshipKind verbatim instead of inventing a separate vocabulary for what Claude is "allowed to say" versus what the graph actually contains — so a claim can never assert a relationship kind that doesn't structurally exist. And when I needed to decide how to verify a claim, I considered the obvious shortcut: ask Claude to explain its own reasoning in a follow-up turn, or trust its stated justification for a citation. I rejected that specifically, because an LLM's account of its own reasoning is just more unverified prose — asking a model to explain itself doesn't add a check, it adds another thing that needs one. validateClaims() is pure graph comparison instead: no language analysis, no regex, no embeddings, no second Claude call. Every claim's photo IDs have to exist in the pack Claude was actually given, the claimed relationship has to hold both directions between every pair, checked against data the deterministic Evidence Relationship Builder already computed — not assumed, not re-derived from prose.

I also had to decide what to do when Claude repeats a true claim rather than inventing a false one — narrating the same relationship twice and emitting a structured claim for each mention. Rejecting the whole response for that would have punished a correct answer for a formatting quirk, so normalizeClaims() collapses only exact duplicates before validation runs, and touches nothing else — it doesn't merge similar-but-different claims, doesn't rewrite a claim's type, doesn't rescue an actually-invalid one. Auditability includes knowing precisely where deduplication ends and correctness-checking begins, not blurring the two into one lenient pass.

The New Architecture

Every scoring/inclusion decision ships its own explanation, not just its result: EvidenceItem.score + EvidenceItem.reason, matchedKeywords[] PhotoSearchResult.finalScore + matchedBy[] matchPercentage (documented: relative-to-top-result, not a probability) AssistantClaim.type (reuses EvidenceRelationshipKind — cannot reference a relationship the graph doesn't actually contain) validateClaims(): pure graph comparison, no LLM, no regex, no embeddings — the check on Claude's claims is never Claude's own account of itself.

Lessons Learned

The most important thing I can say about this pattern isn't a success story — it's an admission. The Citation Gate and Claim Validator close the gap for anything Claude asserts through the structured claims channel. They do not close it for a relationship asserted only in free-text prose, with no matching structured claim behind it. That residual gap is tracked, not hidden — it's a real, currently-open item in this platform's own security tracking, narrowed by the structured-claim work but explicitly not fully closed, sitting on a soft defense (a system-prompt instruction) rather than a hard, provable check. I could have written this article as if every gap were closed. The more useful version of "explainable" includes admitting which parts of the system still aren't — because a system that can explain its decisions has to be willing to explain the ones it can't fully verify yet, too.

Looking Forward

Season 1 started with one database serving a rendering page and ended with the same rows scored, ranked, capped, validated, and explained across three independent domains that never shared a line of prompt text. Looking back at the open threads this season left on purpose rather than by accident: behavior-profile generation still hasn't been migrated onto structuredGenerate(), because nothing has forced the question yet. The knowledge-builder proposal framework exists in full with zero registered generators. HUSKY=0 remains a documented, legitimate way to skip the hooks that are supposed to be the safety net. And the free-text claim gap above is still open. None of those are failures so much as they are the next season's actual starting material — each one is a place where the platform currently trusts something (my memory, a convention, a soft instruction) that a later decision might eventually replace with a structural guarantee, the same way every other trust-Claude default in this codebase already has been.

Takeaway

Nothing in this season made the model smarter. Every fix, from the first shared structuredGenerate() wrapper to the last claim validator, was the same move made again in a new place: stop trusting a number, a prompt, or a model's own account of itself, and replace it with something that can be pointed to, inspected, and — when it's wrong — caught before it reaches anyone. That's not a feature I added at the end. It's the thing this whole platform was, from the first table, actually for.