Matthew Carlson · October 28, 2026 · 6 min readShort answer: when many AI tasks read from one source document, validate the source and grade one sample output before you run the rest. On one engagement, skipping that step meant regenerating 22 of 43 outputs.
The failure that taught me this
The agency I work with runs a 43-skill agent suite: campaign briefs, landing pages, email sequences, FAQ sets, local-SEO audits and more, all reading from a single client profile. Early on, one engagement ran all 43 against a profile nobody had validated. The profile had inconsistencies and the brand voice was slightly off. Every one of the 43 outputs inherited those problems, and 22 had to be regenerated after the client reviewed them.
The lesson wasn't "be more careful." It was that the pipeline had no gate between the source and the fan-out, and a pipeline without a gate will eventually do this again.
Six gates, each a counter-measure to a named failure
I rebuilt the process as a five-pass system with a gate layer in front of it. Every gate ends in a decision, not a report:
- Source document validator. Checks the client profile for internal consistency and returns GO or NO-GO. A NO-GO stops everything.
- Brand voice sample validator. Runs one representative skill, grades the output A to F against the client's voice anchors (attributes, words to use, words to avoid, signature claims), and only passes on a grade the team would ship. If the voice fails, the account manager fixes the profile instead of scaling 43 wrong-voice outputs.
- Capacity model. Compares each team member's available hours with what the engagement will ask of them. This exists because an earlier synthesis pass discovered 20–25 hours of work landing on a team that had eight.
- Sprint-zero intake. A hard gate on twenty critical data points that must be present before any content is produced.
- Engagement goal generator. Ten ranked candidate goals, scored on market position, competitive angle and trend fit, so the fan-out has a target.
- Skill router. Routes each of the 43 skills to Critical, High, Medium or Skip for this client, with structural disqualifiers. Not every client needs every output.
Freeze shared state before parallel reads
The second rule follows from the first. Validators change the source document, so the parallel readers must not start until it is frozen. The onboarding pipeline builds the profile serially, runs the validator as a barrier, checksums the frozen document, and only then fans out four parallel readers. Late corrections don't edit the frozen document mid-run; they queue as amendments and are applied once at the join. If one reader fails, it flags the failure and the others finish; one bad branch doesn't kill the run. Auto-remediation is bounded to two passes, so a broken input can't loop forever.
Keep the library honest
Gates protect each run. Something also has to protect the library over time. A quarterly auditor fingerprints every skill, finds near-duplicates, applies retirement criteria to dead skills, and proposes composites, against a target of 200 active skills. And the schema itself gets audited from evidence: a scan of 42 client profiles across 31 data dimensions found social links missing for more than 97% of older profiles and business-listing data captured for one of 42. Twenty recommendations from that scan shipped in the next release.
Post-mortems should ship as architecture
Every gate above is a named counter-measure to a named failure. None of them live in a lessons-learned document; they live in the tooling, where they run whether or not anyone remembers the lesson. That's the real principle. If an AI system failed once in a way that could repeat, the fix is a gate in the pipeline, not a note in a wiki.
What to ask about any multi-agent system
- What validates the source before the agents read it?
- Is one output graded before the rest are generated?
- What happens to the shared state while agents run in parallel?
- What happens when one agent fails?
Quick answers
What does it mean to gate before fanning out?
Before many AI agents read one shared source, validate the source and grade a single sample output. If either fails, fix the source instead of generating dozens of flawed outputs.
Why freeze a document before parallel agents read it?
Validators change the document. If readers start before it's frozen, they read different versions. A checksummed freeze, with late corrections queued as amendments at the join, keeps every output consistent.
What happens when one agent in a parallel run fails?
It flags the failure and the others finish. Remediation is bounded to two passes so a broken input can't loop forever, and nothing is silently truncated.
Have a process AI should be running? Describe it and I'll tell you whether it's worth building. Book a call or ask the assistant.