The hard part was never the reasoning

There is a slightly uncomfortable lesson buried inside Salesforce Agentforce.

The hard part of building an enterprise AI agent is not getting a large language model to reason. That is the easy bit. The hard part is building a system around the model that executes reliably, quickly and repeatedly at enterprise scale, without asking the model to make every decision itself.

That is essentially the problem Salesforce has been working through with the Atlas Reasoning Engine and its successors. The architecture is considerably more interesting than the "AI employee" story that Salesforce understandably uses to sell Agentforce.

Atlas is not an LLM

The simplest mistake is to imagine Agentforce like this:

User
  |
  v
LLM
  |
  v
Answer

That is a chatbot. An enterprise agent needs something closer to this:

User / Event
     |
     v
   Agent
     |
     v
  Runtime
     |
     v
   State
     |
     v
 Reasoning
     |
     v
  Action
     |
     v
Observation
     |
     v
   State
     |
     v
 Reasoning
     |
     v
 Response

Salesforce describes Atlas as being built on asynchronous, event-driven, graph-based workflows with concurrent design principles, implementing what it calls "System 2" reasoning at inference time. It defines every agent in terms of three parts: state (short-term and long-term memory), flow (the logical framework that determines the next step) and side effects (the actions that change the world, such as updating a record).¹

That distinction matters.

The LLM is not the application. The LLM is a component inside the application.

The important architectural decision: hybrid reasoning

The most consequential change in this line of architecture is the move toward hybrid reasoning.

Some things are inherently probabilistic. For example:

"Does this customer sound like they are considering cancelling?"

That is a judgement problem. A language model is appropriate.

Other things are not. For example:

"If the order is over $10,000, require approval."

That is software. There is almost no reason to spend a model call deciding it.

A good agent therefore looks like this:

                 Agent Graph
                      |
          +-----------+-----------+
          |                       |
    Deterministic           Probabilistic
      execution               reasoning
          |                       |
    rules / code                 LLM
          |                       |
          +-----------+-----------+
                      |
                    State
                      |
                  Next node

Salesforce calls this hybrid reasoning, and it is worth being precise about where the term belongs. Atlas was the original agentic reasoning engine. Agent Graph is described by Salesforce Engineering as "a natural evolution of our Atlas Reasoning Engine with hybrid reasoning" - a graph runtime that "marries LLM intelligence with deterministic control", decomposing workflows into focused subagents managed by finite state machines.²

This is more than an implementation detail. It is an architectural principle.

Why the graph matters

A purely prompt-driven agent effectively asks the model to invent the workflow every time. That produces something like:

"What should I do next?"
          |
          v
         LLM
          |
          v
"What should I do next?"
          |
          v
         LLM
          |
          v
"What should I do next?"
          |
          v
         LLM

It can work. It is also expensive, slow and difficult to reason about.

A graph moves part of the workflow outside the model:

START
  |
  v
Understand request
  |
  v
Find customer
  |
  v
Is customer found?
 +-- NO --> Ask user
 +-- YES
       |
       v
Determine intent
       |
       v
Execute action
       |
       v
Validate result
       |
       v
Respond

The model still provides judgement where judgement is required. But the runtime owns the workflow.

Salesforce's engineering team is direct about this: Agent Graph "externalizes reasoning into design-time graphs, ensuring reliable behavior", and treats orchestration as "design-time configuration, not runtime improvisation". The state is passed and shared between steps so that context is never lost. They call the result guided determinism.²

That is the crucial architectural shift.

Performance: every LLM call is a tax

This is where the architecture becomes particularly interesting.

Suppose an agent performs four sequential model calls:

LLM 1: classify
   |
   v
LLM 2: determine action
   |
   v
LLM 3: evaluate result
   |
   v
LLM 4: generate response

If each call takes only two seconds, you have already created eight seconds of model latency. Then add database round trips, API calls, retrieval, network, serialisation, security checks and tool execution, and a supposedly intelligent assistant takes ten or twenty seconds to answer a simple question.

Salesforce has publicly described exactly this problem, twice, and the two accounts are worth separating because they fix different things.

In one production deployment, its engineering team identified three bottlenecks: upstream order-system API latency, Data 360 lookup latency across millions of records, and multi-call LLM reasoning loops where "early versions made several sequential model calls to refine relevance scoring and determine next-step logic". Data was also being retrieved "in multiple incremental passes rather than through a single optimized pull". They consolidated the reasoning into a single model call, restructured the prompt for one-pass deterministic reasoning, and requested all required fields in one consolidated retrieval. Together those changes reduced end-to-end latency by 75%.³

Separately, Salesforce built Unified Planner, a single AI runtime behind every Agentforce interaction across voice, chat and text. There the headline mechanism was the opposite move: not fewer calls, but concurrent ones. The team describes "aggressively embracing parallel execution", redesigning operations that had historically run sequentially - prompt injection detection, citation generation, grounding validation, knowledge retrieval and context gathering - to run concurrently wherever dependencies allowed. Reported result: response latency down from roughly 20 seconds to approximately 2.3 seconds.⁴

Consolidate what is redundant. Parallelise what is independent. Those are two different levers, and both of them are architecture, not model choice.

The performance equation

A useful mental model:

Total latency ~=

    LLM latency x number of SEQUENTIAL calls
  + retrieval latency x number of round trips
  + action latency
  + network / orchestration overhead

The dangerous word is sequential.

If you need three independent pieces of information, this:

Customer --+
Product  --+--> Reasoning
Order    --+

is substantially better than this:

Customer
   |
   v
 reason
   |
   v
Product
   |
   v
 reason
   |
   v
 Order
   |
   v
 reason

Atlas was designed around asynchronous, event-driven execution with concurrent tasks running in parallel across function nodes, using a publish-subscribe pattern to decouple components.¹ That is not an AI insight. It is ordinary distributed-systems engineering, applied to agents.

Rule one: do not call an LLM to do something your runtime can do

This sounds obvious. It is not.

Consider: "Is the customer ID present?" Do not ask an LLM.

if (customerId != null)

Consider: "Did the action succeed?" Do not ask an LLM.

action.Status == Success

Consider: "Does the evidence pack contain the required artefact?" Do not ask an LLM.

evidencePack.Contains(requiredArtifact)

Use the model for this instead:

"Does this evidence support the proposition?"

That is semantic judgement. The distinction reduces cost, latency and variability at the same time.

Salesforce's own guidance lands in the same place: use structured actions such as Flows and Apex methods for critical business logic rather than relying solely on natural-language instructions, because it increases deterministic behaviour and consistency of outcome.⁵

Rule two: do not give the model a toolbox the size of a warehouse

One of the less glamorous performance problems is action selection.

Imagine presenting the model with this:

SearchCustomer
FindCustomer
GetCustomer
LookupCustomer
RetrieveCustomer
SearchCustomerByEmail
SearchCustomerByName
SearchCustomerByPhone
FindCustomerByEmail
FindCustomerByName
...

The model now has to decide which tool applies. That decision consumes context and inference capacity, and it is a decision you created for it.

Salesforce is specific here. Its developer guidance is to "limit the number of topics (10 max) that an agent has access to, and keep the number of actions within each topic low (12-15 max)".⁵ Note the shape of that: the cap on topics is tighter than the cap on actions within a topic. Routing comes first; capability surface comes second.

The underlying principle is more important than the numbers:

Give the model the smallest useful action surface. Do not make every capability globally available. Route the request first, then expose only the relevant capabilities.

                 Request
                    |
                    v
                  Router
                    |
           +--------+--------+
           |                 |
         Sales            Service
           |                 |
      6 actions          5 actions

That is better than:

                 Request
                    |
                    v
                   LLM
                    |
                    v
               87 actions

Rule three: action descriptions are part of the interface

An agent does not understand an action because your developers understand it. The model sees the name, the description, the inputs and the surrounding context. That is the whole interface.

So this:

UpdateRecord()

is a terrible agent interface. Something like this:

Update Customer Shipping Address

Use when the customer has explicitly requested a change to
the shipping address of an existing order.

Requires:
  - order ID
  - complete shipping address

Does not modify billing address.

is far more useful.

Salesforce states plainly that "the Atlas Reasoning Engine relies on the Agent action configuration, which incorporates the labels and descriptions of your invocable methods", and recommends explicit descriptions for every method and every input and output parameter.⁶

This is effectively API design, with a language model as the consumer.

Rule four: do not put business logic in prompts

This is probably the biggest anti-pattern in early agent implementations. You end up with something like:

You are an excellent customer service agent.

If the customer is a VIP...
unless the order is...
except when...
provided that...
unless...

Eventually the prompt becomes a badly written programming language. The worst part is that it is not deterministic.

Critical business logic belongs in executable code:

LLM
 |
 v
proposed action
 |
 v
runtime
 |
 v
business rules
 |
 v
action

not:

LLM
 |
 v
"please remember all 47 business rules"
 |
 v
database mutation

Salesforce's guidance for building Agentforce actions makes the same point from the other direction: pass data between actions using variables rather than relying on natural-language topic instructions, which it describes as non-deterministic.⁶

Rule five: do not retrieve data one fact at a time

Another surprisingly expensive pattern:

Get customer
  |
  v
 LLM
  |
  v
Get order
  |
  v
 LLM
  |
  v
Get product
  |
  v
 LLM

If the agent knows at the start that it needs the customer, the order and the product, retrieve them in one pass.

This is precisely what Salesforce's performance work found: incremental Data 360 retrieval was a measurable latency contributor, and requesting all required fields in a single consolidated pull was part of the 75% reduction.³

AI does not magically make network latency disappear.

Rule six: do not confuse context with intelligence

More context is not automatically better. Salesforce's context-engineering guidance names three distinct failure modes:⁵

• Context clash - conflicting instructions that confuse the agent.
• Context confusion - overwhelming the agent with too much at once, including too many unnecessary tools.
• Context poisoning - bad or outdated information entering the context and then being referenced repeatedly.

All three can reduce accuracy while increasing latency. The correct objective is not:

Give the LLM everything.

It is:

Give the LLM exactly what it needs to make the next decision.

That is a very different architecture.

What not to build

User
 |
 v
Huge prompt
 |
 v
LLM
 |
 v
"figure everything out"
 |
 v
LLM --> LLM --> LLM
 |
 v
Tools
 |
 v
LLM
 |
 v
Answer

It is expensive, slow, opaque and difficult to test.

What to build

              Execution Graph
                     |
         +-----------+-----------+
         |                       |
   Deterministic             Reasoning
         |                       |
  rules / actions               LLM
         |                       |
         +-----------+-----------+
                     |
                   State
                     |
                 Next node

• The graph determines what can happen.
• The LLM determines what should happen when judgement is required.
• The action layer determines how it happens.
• The data layer determines what is known.
• The completion gate determines whether the job is actually finished.

That separation is the architecture.

Where this is going

The naming lineage is itself revealing. Atlas was an agentic reasoning engine built around state, flow and side effects. Agent Graph extended it into explicit graph-based hybrid reasoning, with Agent Script compiling down to a graph specification the engine consumes. Unified Planner then collapsed previously separate runtimes into one execution and reasoning layer serving voice, chat and text.¹²⁴

Every step in that progression moves work out of the model and into the runtime.

The industry is not discovering that LLMs are bad. It is discovering that LLMs are extraordinarily good components inside software architectures, and extraordinarily bad substitutes for software architectures.

The best agent is not the one that reasons the most

It is the one that knows when not to reason.

• If a condition can be evaluated in microseconds, evaluate it in code.
• If a calculation can be performed deterministically, calculate it.
• If three independent records can be retrieved in parallel, retrieve them in parallel.
• If an action has a well-defined API, call the API.
• If a workflow is known, encode the workflow.
• If the system needs semantic interpretation, call the model.

And if the system genuinely does not know what to do? Then let the model reason. That is where it earns its keep.

The emerging architecture therefore looks less like an autonomous AI employee and rather more like a very sophisticated distributed application:

          +--------------------------+
          |       USER / EVENT       |
          +-------------+------------+
                        |
                        v
                +---------------+
                |   ROUTING /   |
                |   EXECUTION   |
                |     GRAPH     |
                +-------+-------+
                        |
          +-------------+-------------+
          |                           |
          v                           v
   DETERMINISTIC                PROBABILISTIC
     EXECUTION                    REASONING
          |                           |
   rules / code                      LLM
   APIs / tools                       |
          |                           |
          +-------------+-------------+
                        |
                        v
                      STATE
                        |
                        v
                   OBSERVATION
                        |
                        v
                    NEXT NODE
                        |
                        v
                    COMPLETION

Why we care about this at AppGenie

We build and govern delivery systems for environments where somebody eventually has to answer for the outcome. Federal and state government, IRAP-aligned and FedRAMP-aligned controls, regulated enterprise delivery. In those environments the question is never "did the agent sound clever". It is "can you show what it did, why, and prove it was allowed to".

That is the same argument the graph makes. A deterministic runtime is not just faster, it is evidenceable. A node either executed or it did not. A rule either fired or it did not. A prompt that "usually" enforces a $10,000 approval threshold is not a control, and it will not survive an audit finding, a security escalation or a post-incident review.

So we push the same separation into everything we build: judgement in the model, rules in code, workflow in the runtime, and a completion gate that decides whether the job is actually done. Not because it is architecturally elegant, but because it is the only version that produces an audit trail.

Once you have thousands of requests running concurrently, architecture becomes the performance feature. And once an agent is allowed to change something important, determinism becomes the trust feature.

The language model is still important. It just should not be allowed to run the building.