TheOnes.io

Determinism Over Prompts: How to Build AI Systems That Don't Drift

2 min readWojciech Wiesner

  • ai-architecture
  • enterprise
  • determinism
  • systems
  • case-study

The biggest mistake in enterprise AI today is letting probabilistic models make business-critical decisions.

When a team decides to “add AI agents” to an operational workflow, they usually start with an open-ended loop: give the LLM an API tool, pipe in user messages, and hope it decides the right steps in the right order.

In a demo, that looks like magic. In production—when processing real VAT invoices, legal onboarding documents, or financial settlements—it is a recipe for silent, catastrophic drift.

Here is the architectural rule that powers our production systems: 90% Determinism, 10% LLM.

The Core Invariant: Algorithms Govern, Models Interpret

In systems like InvoiceFlow (~10,000 invoices/month) and Onboarding Flow (~1,500 candidates processed end-to-end), the machine learning model never controls the state machine.

Instead, the workflow is split into two non-negotiable tiers:

  1. The Deterministic Outer Shell (90%):

    • State transitions (DAG).
    • Validation rules (PESEL/NIP checksums, tax calculation matching, mathematical equality).
    • Security envelopes (Row-Level Security, tenant isolation, OAuth mutexes).
    • Routing decisions (is this invoice VAT-exempt? The tax engine decides, not the prompt).

  2. The Probabilistic Core (10%):

    • OCR text extraction from messy scans.
    • Natural language comprehension of edge-case user queries.
    • Dynamic phrasing of the next required question in the user’s native language.

The LLM is a sensory organ and a synthesizer. It is never the judge or the ledger.

Real-World Case: The High-Water Mark Invariant

Take KSeF (Polish National e-Invoicing System) synchronization in InvoiceFlow. When querying tax authorities across multiple corporate entities, API limits are brutal (e.g. 20 requests per hour).

If an agentic prompt hallucinated the date window or failed to advance the high-water mark (HWM) per tax ID, the entire batch pipeline would choke on historical data, starving fresh invoices of verification.

We didn’t solve this with a better system prompt. We solved it with deterministic state guarantees:

  • Separate HWM tracking persisted per company tax ID.
  • Strict overlap buffers calculated algebraically.
  • Zero LLM involvement in query window calculation.

The result? 0 failures across thousands of invoices, automatic recovery from vendor rate limits, and 100% auditable accounting pipelines.

Why This Matters for AI Product Leadership

Enterprise customers do not buy “clever agents.” They buy predictability, compliance, and reduced operational cost.

When you pitch an AI product to a CTO or Head of Operations, showing a prompt that works 95% of the time is a red flag. Showing a deterministic wrapper where the 5% probabilistic failure mode is caught, isolated, and gracefully routed to human review is what closes enterprise deals.

Stop trying to prompt your way to reliability. Build the deterministic harness first.