Summary
Some failures mean the software is wrong. Others mean the software has reached the edge of what it can decide safely.
Operational workflows encounter missing mappings, ambiguous identities, incomplete evidence, and provider mismatches as part of normal work. Stopping an entire run treats those conditions like defects. Ignoring them preserves throughput by hiding a business gap. Neither response helps the person who must decide what happens next.
We treat recoverable ambiguity as durable work: an issue, signal, review task, or failed-run state with evidence and a path back into the workflow. A log can say that something went wrong. A work item can say what happened, what it affected, what evidence survived, who has authority to decide, and how the system resumes.
That is the difference between recording failure and coordinating repair.
Operational Tension
The familiar succeeded-or-failed model breaks down as soon as a batch contains independent items or an external system returns plausible but inconclusive evidence. Retrying an ambiguous identity does not produce better identity evidence. Throwing an exception preserves a stack trace but often discards the operator's context. Skipping the item makes the run look healthy while the underlying obligation disappears.
There is an equally dangerous failure in the other direction: catch everything and create a reassuring "needs review" card. An authorization failure is not a judgment call. Neither is a violated invariant or a programmer error. If those conditions are softened into review work, the workflow can appear to continue after the control boundary itself has failed.
What constrains the design is classification. Which outcomes need domain judgment? Which can change with a bounded retry? Which must stop because the program or its authority is wrong?
Decision: Make Expected Ambiguity Durable
The architecture gives each class a different path. Recoverable business ambiguity and per-item validation failures become typed results, signals, or deduplicated issues at an orchestration boundary. Provider and infrastructure failures that may change retain attempt context and receive bounded retry. Permanent operational failures become failed runs or issues with evidence and a repair path. Programmer errors, authorization failures, and violated invariants remain hard failures.
This needs more than an exception object. The durable record carries safe scope and correlation context: tenant, actor or run, attempt, outcome, duration, and safe counts where relevant. A human or authorized agent can inspect the event without reconstructing it from transient logs. Domain services and review surfaces own the repair interaction; pipelines do not wait invisibly for a person.
That placement matters. A job may coordinate a batch, but it should not become a private catalog of special cases. When issue and repair semantics live at the service boundary, every entrypoint reaches the same visible state and the same rules for resumption.
Failure And Repair Posture
When automation reaches a decision it cannot make safely, it records the current state and candidate evidence, then gives the appropriate operator a specific action. The repair changes durable domain state. Resumption becomes a defined transition, a contextual retry, or a newly queued step—not a request to rerun an opaque script and hope.
This preserves partial progress. Accepted items remain accepted while uncertain ones stay visible. The operator sees what is blocked instead of a batch-level red light with no way inside. Where external effects are involved, idempotency, leases, heartbeats, and intent or outbox records help recovery avoid duplicating work.
The human role changes too. Instead of reading logs, reconstructing state, and manually replaying steps, a person evaluates assembled evidence and resolves the exception. That is a better use of judgment, but it still requires careful observability. The record must contain enough context to act without copying sensitive material into generic logs. Business audit history remains distinct from diagnostics.
Tradeoff
This costs more than a broad exception handler. It needs typed outcomes, durable persistence, deduplication, review UI, authorization, and tests for retry and resume. It also forces us to name the difference between ambiguity and defect instead of placing everything unexpected in one bucket.
The return is continuity with accountability. A person can repair the right state without reading code or guessing which worker emitted a message. Future agents participate through the same evidence, actions, approvals, and audit boundaries. Each repair leaves the workflow more legible than it was before.
Limits
This proves that expected uncertainty can remain inside a durable workflow without being mistaken for success. It does not prove that every failure should become an issue, or that every issue has a good repair policy. A corrupted invariant or unauthorized operation remains an error with its cause visible to engineers. This case study makes no claim about a particular incident, queue volume, or response time.
Transferable Lesson
When judgment is the next step, model the interruption as work. Give it state, evidence, ownership, and a defined route back into motion. When the system is wrong, fail visibly instead. The goal is not to make failure disappear. It is to make the difference between repairable uncertainty and a broken control clear enough to act on safely.