The first mistake is the easiest one to make and the hardest one to notice while you're making it: treating every fuzzy-looking matching problem as a job for an embedding.

The vocabulary-expansion layer of this retrieval system exists to solve a real problem — a query for "biking" has no lexical path to a photo tagged "bike", and rejecting that query outright would be worse than promoting it onto the nearest controlled-vocabulary term when the two are close enough in meaning. expandPhotoQuery() does that: it embeds the actual vocabulary currently in use — whatever tags, moods, and locations exist on published photos — and promotes a free-text token onto the nearest term if cosine similarity clears a category-specific threshold. That's a real, working mechanism. It is also, structurally, a probabilistic answer to every question a query token could raise, whether or not the question underneath it was actually probabilistic.

Two of those questions turned out not to be.

The first was pluralization and casing. "mountains" and "mountain" are the same tag; "café" and "cafe" are the same tag. There is nothing to measure here — it's a closed transformation with a known, deterministic answer, and routing it through an embedding call would mean spending a network round-trip and a threshold judgment on something a suffix rule resolves correctly every time. canonicalizeTerm() folds case, Unicode form, accents, and regular English plurals before any embedding path is ever consulted, and it's deliberately scoped to the tag category only — moods are adjectival and rarely pluralize, and locations are proper nouns where a naive suffix strip would turn "Athens" into "Athen". The rule is narrow on purpose: it claims only the ground that's actually deterministic, and leaves the rest — "biking" versus "bike", a real synonym relationship with no shared root — to the embedding layer, which is where a probabilistic answer is genuinely needed.

The second was orientation, and it was a real production bug before it was a design correction. Orientation queries — "portrait", "landscape", "vertical" — were routed through the same category-thresholded embedding-promotion machinery as tags, moods, and colors, gated at 0.75, the strictest of the five category thresholds in use. The production logs caught the failure directly: a query token for "pictures" promoted onto "portrait" at a cosine similarity of 0.5781 — comfortably under the 0.75 gate, which meant it should never have promoted, except the gate wasn't the number that mattered. "pictures" and "portrait" cluster near each other in embedding space because they're both photography-adjacent words, not because a person typing "pictures" meant to filter by orientation. No threshold, however strict, closes that gap, because the threshold is answering a similarity question and the actual question — does this word mean one of exactly three fixed things — was never a similarity question at all. Orientation is a closed, enumerable set computed directly from a photo's own pixel dimensions. There is no fuzzy "70%-portrait-ish" query intent for a similarity score to usefully express.

The fix wasn't a stricter threshold. It was removing orientation from the vocabulary-expansion system entirely. orientationKeywords.ts is a fixed keyword map — portrait/vertical → portrait, landscape/horizontal → landscape, square → square — resolved by a pure function, resolveOrientationKeyword(), with no embedding call, no KNN lookup, no promotion logic. Both the query-expansion path and the filter-extraction path call this one function, so there's exactly one place orientation gets recognized, not two that could quietly drift apart. Once the controlled vocabulary stopped constructing orientation embeddings at all, the failure mode didn't just get rarer — it became structurally impossible. There's no embedding for findNearest() to accidentally return, because none exists.

The same shape of correction showed up again later, smaller in scope but identical in kind, in how a hyphen gets treated. A production audit of the tag and mood vocabularies found that of 120 distinct tags, 26 contain a hyphen — "golden-hour", "depth-of-field", "black-and-white" — and every single one is a compound concept that must never be split; splitting "depth-of-field" into three independent words produces search terms with no relationship to the photo. Moods showed the opposite shape: of 21 distinct mood strings, exactly one, "whimsical-yet-stark", uses a hyphen to join two genuinely independent feelings that need to be split apart to be searchable. Neither of these is a judgment call requiring a model. Both are deterministic rules, evidenced by an actual inventory of the data rather than assumed in advance — a shared low-level primitive folds dash variants onto a plain ASCII hyphen, and two separate, narrowly-scoped policies sit on top of it: one that treats a hyphen as a join point for tags, one that treats it as a split point for moods. Neither policy touches an embedding.

What connects canonicalization, orientation, and hyphen handling is not that they're all small fixes. It's that each one is evidence of the same prior mistake: treating "this looks like it needs fuzzy matching" as equivalent to "this needs an embedding," when the actual test is narrower — does the space of possible answers have a closed, computable structure, or doesn't it? A plural has one canonical form. An orientation has three possible values, derived from pixel dimensions that already exist. A hyphenated tag's status as one concept or two is a fact about the data, discoverable by looking at the data, not a probability distribution over compound-word phrases. None of these needed to be estimated. They needed to be looked up, or computed, or counted.

The transferable version of this lesson has nothing to do with photographs: before reaching for an embedding, a classifier, or any other probabilistic mechanism, ask whether the question actually has a closed-form answer that a deterministic rule already computes correctly. If it does, spending a similarity threshold on it doesn't make the system more sophisticated — it adds a failure mode that didn't need to exist, gated by a number that was never measuring the right thing.

Not every question in this system has that shape, though. Color and mood are genuinely open — the full set of tones a person would call "blue," or feelings a person would call "serene," changes as new photos are added, and there's no fixed enumeration to look up the way there is for orientation. You can't remove the probabilistic gate there, because the gate is answering a question that really is probabilistic. So the next question isn't whether to keep an embedding-based match in the pipeline. It's what has to be true about what that match is comparing before any threshold placed on it can mean anything at all.