← All notes

October 1, 2026 · Arin Tahmasian

Why AI Recommendations Become Risky When They Can Change Your Systems

An AI model can propose a plausible action, but a plausible response is not an authorized, safe command. Once a workflow can message customers, modify a CRM, issue refunds, or update a…

AI CAN RECOMMEND, NOT EXECUTE. A person at a control console faces a circular device separating blue input streams from orange connections to email, ID, payment, and server systems.

What changes when an AI feature can take action

An AI model can propose a plausible action, but a plausible response is not an authorized, safe command. Once a workflow can message customers, modify a CRM, issue refunds, or update a system of record, deterministic application logic must validate, authorize, log, execute once, and provide recovery when execution fails.

Imagine an AI support feature that correctly drafts a refund recommendation in a demo. Connecting that same feature to billing changes the problem. Model quality still matters, but execution controls now determine whether the automation is safe to operate.

Why is a correct-looking AI recommendation still unsafe to execute directly

A correct-looking recommendation can still be unsafe because external actions depend on identity, current state, policy, and delivery guarantees, not just a reasonable interpretation of text. The model may connect a request to the wrong customer, infer context that is missing, or suggest an exception that policy does not allow.

Even a sound recommendation can cause harm after it leaves the model. A timeout might trigger a retry and create a duplicate refund. An outdated record might cause a CRM update to overwrite newer work. A useful suggestion becomes an operational change only after the application establishes that the target, action, and circumstances are valid.

The practical principle is simple: model output is an input to a controlled workflow, not the workflow itself.

The execution boundary separates interpretation from authority

A safe AI action workflow separates the model's ability to interpret a request from the software's authority to change a system. The model can classify a ticket, extract fields, summarize a case, or recommend a next step. Deterministic application logic should own the final decision to execute.

A useful sequence is:

  1. The model produces a structured recommendation.
  2. The application validates required fields against current records.
  3. Policy, permissions, and approval requirements are checked.
  4. The application records the decision and executes the allowed action.
  5. The workflow captures the result and exposes a recovery path if it fails.
A process diagram shows an AI model producing a refund recommendation, then passing it through a boundary where software validates the live record, checks policy and permissions, requests approval if needed, executes once, and records recovery information. Wrong customer, policy violation, and duplicate retry paths are blocked before execution.

This boundary does not make AI less useful. It gives the model a clear job, while keeping authority where rules can be inspected, tested, and enforced. The question becomes: what evidence must the application see before this particular action is allowed?

What controls should an AI action workflow include

AI action workflows need explicit controls because each one prevents a different operational failure. Start with an allowed-action scope: an AI may draft a customer reply or create a review task, while refunding an account or changing contract terms may be unavailable without additional checks.

Then check the identity and role of the requester, validate the proposed target against live records, and require human approval above defined impact thresholds. Use idempotency keys or equivalent duplicate prevention so a retry cannot send the same message twice or issue the same refund twice.

A side-by-side comparison contrasts a direct AI connection to CRM, email, and billing tools, which can cause wrong-record, duplicate, and unrecoverable changes, with a controlled workflow that adds permissions, live-data validation, idempotency, audit logs, reversal, and high-impact human review.

Keep an audit record of the model input, structured recommendation, policy decision, approver where relevant, and execution result. Error handling should distinguish between an action that definitely failed, definitely succeeded, and has an uncertain outcome. Finally, design a reversal or compensation path, such as correcting a stale CRM update or escalating a customer-facing mistake to an owner.

Where should human approval remain in the loop

Human approval should remain where an action has material impact, is difficult to reverse, reaches a customer, or involves ambiguous context. Low-impact, reversible, well-bounded tasks may run automatically after policy checks, such as tagging a clearly classified internal request.

Approval is not a substitute for validation. A person cannot reliably catch a workflow that selected the wrong record or retried an already completed action. Validation is also not a substitute for approval when the consequence is material. Check the owner, the exception, and the recovery path.

Production-safe AI is accountable software, not a more confident model

Production-safe AI is accountable software wrapped around model judgment. The aim is not to prove that the model chose a convincing next step. It is to prove who was allowed to act, what changed, whether it happened once, and how the business can recover.

Success is not an AI action completed. It is a controlled business task completed safely.

Next note

Why SOC 2 Controls Still Fail When the Auditor Asks for Evidence

Like how we think?

We build the way we write. Plainly, and to completion.

Start a conversation