The Photograph

Ask the assistant, in early July, "what did I photograph that day" and it could answer about any single photo perfectly — correct caption, correct tags, a citation you could verify. Ask it whether two of those photos belonged to the same afternoon, and it had nothing. Not a wrong answer — no mechanism to answer at all. Each photo in the evidence pack was still, quietly, an island.

The Problem

Phase 1 of the assistant could cite multiple photos in one answer, but it couldn't reason about how they related — every multi-photo answer was, structurally, single-photo prose repeated per citation. That's not what "memory" means. A memory isn't a list of independently correct facts; it's facts connected by time, place, and repetition — this photo and that one were the same day, that other one shares the same location, these three keep recurring together. None of that existed anywhere in the pipeline. The data to compute it — taken_at, location strings, camera/lens fields, dominant colors — had been sitting in photo_exif since Phase 1, unused for exactly this purpose.

The Investigation

The scope question came first: should relationships be inferred with embeddings — a similarity score between photos, the same tool that had already caused two separate failures earlier in this series — or computed from fields the database already stores? I tested the idea of a similarity-based "these feel related" score and rejected it before building it, for a reason this whole series kept teaching me the same way each time: a fuzzy relationship is a relationship Claude could later narrate past its actual confidence, and I'd just spent an entire chapter building a Claim Validator specifically to stop that. Adding a new fuzzy signal one chapter after closing that gap would have been building the next version of the same mistake.

So the actual investigation was about exactness: which pairs of fields, compared exactly, produce a relationship worth surfacing. Same location string. Same mood. A shared tag. Same camera. Same lens. Same calendar day, or an adjacent one. A shared dominant color — exact hex intersection, not RGB distance, because no color-distance utility existed anywhere else in the codebase and building one would have been a new heuristic this phase deliberately excluded. Same orientation bucket. Eight signals, each one either true or not, nothing in between.

The Decision

buildEvidenceRelationships() compares every pair of photos already inside the evidence pack — never a photo outside it — across those eight exact signals, and produces a chronologicalOrder for whichever pack photos have a known taken_at. Both sides of every comparison come from pack.photos itself, which makes "a relationship referencing a photo outside the pack" not a case the code has to defend against — it's a case that can't occur, because there's no data path to construct it. I deliberately kept the function pure: (EvidencePack, PhotoRelationshipMetadata[]) -> EvidencePack, no persistence, no accumulation across requests. Relationships are recomputed fresh every time, which rules out a knowledge graph that grows — that was a real trade-off, not an oversight; a persisted graph would need its own invalidation story the moment a photo's metadata changed, and this phase wasn't the place to take that on.

The closing decision of this whole season, though, wasn't a retrieval decision at all. On 2026-07-08, a is_featured column and a PassionProject component put a single photo — chosen deliberately, not algorithmically — on the About page and the homepage, next to the same personal narrative the rest of this site tells. Everything in this series exists to make search and explanation trustworthy. This last piece is the one place the system stops answering questions and just shows something I care about, unranked and unexplained, because some things don't need a match percentage.

The Architecture

Evidence Pack (≤8 photos, already built) ↓ getPhotoRelationshipMetadata() bounded read, scoped to exactly the pack's photoIds ↓ buildEvidenceRelationships() 8 exact-match signals per pair: location, mood, tag, camera, lens, same/adjacent day, dominant color, orientation ↓ EvidencePack.photos[].relatedPhotos per-photo, capped at 8, sorted for determinism EvidencePack.chronologicalOrder pack photoIds with known taken_at, ascending ↓ Claude narrates the graph — never computes it, never extends it ↓ (verified by validateClaims(), previous chapter) Separately, unranked: photo.is_featured = 1 → PassionProject component → About page + homepage

Lessons Learned

I expected relationship-building to be the most "AI" part of this entire feature — the place where a model would finally get to reason freely about connections. It's the opposite: it's the most rigidly deterministic piece in the whole pipeline, on purpose, precisely because "these photos are connected" is exactly the kind of claim that sounds harmless and turns out to be the one users would trust most, unverified.

The featured-photo decision surprised me by not being an engineering decision at all — is_featured is one boolean, one admin toggle, one component. After six chapters of scoring, thresholds, and validators, the smallest piece of code in the whole series is the one that actually says something true about why any of this existed.

Looking Forward

What's explicitly not built yet, and I'd rather say so than imply otherwise: conversational memory across multiple questions in a session, narrative album summaries, and any relationship discovery that isn't an exact field match. The relationship graph this chapter built is recomputed from scratch on every question — a real memory, in the sense a person means it, accumulates across time. This one still starts over every time you ask. That gap is the honest answer to the question this whole season opened with: the system remembers what's in front of it, one evidence pack at a time. It doesn't yet remember you.