Skip to content

AI · 10 min read

AI Agents for Business: What They Are and Where They Actually Work

A practical guide to AI agents for business leaders: what they are, where they work, where they struggle and how to launch a first project with the right controls.

Server rack with glowing network equipment in a dark room
On this page 12 sections
  1. What is an AI agent, in plain terms?
  2. Agents vs. workflows vs. chatbots
  3. Beware “agent washing”
  4. Where AI agents actually work today
  5. Where agents struggle
  6. How widely are businesses using agents?
  7. The risks you need to design for
  8. How agents connect to your systems
  9. How to run your first agent project
  10. Questions to ask before you commit budget
  11. The bottom line
  12. FAQs

An AI agent is software that uses a large language model to decide which steps to take and which tools to use to complete a task, rather than following a fixed script. For businesses, agents work best on well-bounded, multi-step tasks involving messy inputs and judgment calls, with clear permissions and human approval for anything costly or irreversible. They are not yet a reliable replacement for whole roles.

That is the short answer. The longer one matters, because “agent” has become one of the most stretched words in technology marketing. This guide explains what an AI agent actually is, how it differs from automation and chatbots you may already use, where agents are delivering value today, where they tend to fail and how to run a first agent project with sensible controls.

What is an AI agent, in plain terms?

Most definitions from the companies building the underlying models agree on the essentials. OpenAI’s practical guide to building agents describes agents as “systems that independently accomplish tasks on your behalf,” and breaks every agent into three components:

  • A model — the language model that reasons about the task and decides what to do next.
  • Tools — the functions or APIs the agent can call to read data or take action, such as searching a knowledge base, looking up an order or creating a ticket.
  • Instructions — explicit guidelines and guardrails that define how the agent should behave.

The defining feature is who controls the path. In a traditional automated workflow, a developer decides every step in advance. In an agent, the model decides the next step based on what it has seen so far, calls a tool, looks at the result and decides again, looping until the task is done or it hits a stopping condition.

Agents vs. workflows vs. chatbots

Anthropic’s engineering guidance, Building effective agents, draws a useful line between two kinds of “agentic” systems. Workflows are systems where language models and tools “are orchestrated through predefined code paths.” Agents are systems where models “dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.”

That distinction is the most practical one a business leader can learn, because it determines cost, predictability and risk. Here is how the common options compare:

ApproachWho decides the steps?Best forMain trade-off
Rules-based automation (scripts, RPA, integration tools)A developer, in advanceStable, structured, high-volume processesBreaks when inputs vary or rules multiply
Chatbot or assistantThe user, one prompt at a timeAnswering questions, drafting, summarizingDoes not act on its own; a person drives every step
LLM workflowA developer, with the model handling specific stepsPredictable tasks with unstructured inputs, such as classifying and routing emailLess flexible, but easier to test and control
AI agentThe model, within the tools and limits you give itOpen-ended, multi-step tasks where the path varies case by caseHigher cost, more variability and a larger error surface

Many products marketed as agents are actually workflows, and that is often a good thing. A well-designed workflow with a language model inside it is easier to test, cheaper to run and more predictable than a fully autonomous agent.

Beware “agent washing”

The term has been stretched far enough that analysts have named the problem. In a June 2025 press release, Gartner described “agent washing” as the rebranding of existing products, such as AI assistants, robotic process automation and chatbots, without substantial agentic capabilities. Gartner estimated that only about 130 of the thousands of vendors claiming agentic AI were real.

When a vendor pitches you an agent, ask three direct questions:

  1. Which decisions does the model make on its own, and which are hard-coded?
  2. Which systems can it write to, not just read from, and under whose permissions?
  3. What happens when it is unsure, fails or encounters something outside its instructions?

Clear answers to those questions tell you far more than a demo does.

Where AI agents actually work today

OpenAI’s guide suggests considering an agent when a workflow involves one or more of three characteristics: complex decision-making with “nuanced judgment, exceptions, or context-sensitive decisions”; rules that have become difficult to maintain; and heavy reliance on unstructured data such as natural language and documents. Anthropic similarly positions agents for “open-ended problems where it’s difficult or impossible to predict the required number of steps.”

In practice, that points to a recognizable set of business tasks.

Customer support resolution

An agent can read a customer’s message, look up the order or account, check the relevant policy and either resolve a routine request or prepare a complete case for a human. The value comes from handling the variety of how customers describe problems, which rigid decision trees struggle with. Refunds, credits and account changes above a threshold should still go to a person.

Internal IT and operations requests

Password resets, access requests, “how do I” questions and simple troubleshooting are frequent, repetitive and governed by documented procedures. McKinsey’s State of AI in 2026 report found that respondents most often report scaling AI agents within IT, knowledge management and software engineering.

Research and preparation work

Gathering information from several internal and external sources, comparing it and producing a structured brief is a good fit: the steps vary by case, the output is reviewed by a person before it matters and mistakes are recoverable. Examples include account research before a sales call, supplier comparison and preparing a first review of a contract against your standard terms.

Document-heavy back-office tasks

Processing invoices with exceptions, reconciling mismatched records or assembling a complete file from emails and attachments often involves judgment that pure rules cannot capture. An agent can do the gathering and first-pass reasoning, then hand exceptions to a person with the evidence already assembled.

Software engineering

Coding agents that read a codebase, make changes, run tests and iterate are one of the more mature categories, partly because tests give the agent fast, objective feedback. McKinsey’s 2026 survey found that about two in ten respondents are scaling software coding agents.

Where agents struggle

Agents are weakest where the cost of an error is high and hard to detect, or where there is no objective way to check the result. Be cautious with:

  • Irreversible, high-value actions such as payments, deleting records, sending contractual commitments or changing prices in production.
  • Tasks without a feedback signal. If neither the agent nor a reviewer can tell whether the output is right, errors will pass through unnoticed.
  • Long, unsupervised chains of steps. Anthropic notes that the autonomy of agents brings “higher costs, and the potential for compounding errors.” A small misunderstanding early in a task can carry through every later step.
  • Regulated decisions about people, such as hiring, lending, housing or medical matters, where laws and fairness obligations require explainability and human accountability.
  • Processes that are already simple and stable. If a rule or a standard integration can do the job reliably, an agent adds cost and unpredictability without adding value.

Gartner’s same June 2025 release predicted that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. None of those causes is about the model being unintelligent. All three are about choosing, scoping and governing the project.

How widely are businesses using agents?

Adoption is real but uneven. McKinsey’s 2026 survey, fielded in May and June 2026 with 1,719 participants, found that 40% of respondents from large organizations (annual revenues above $1 billion) report scaling AI agents, up from 27% the year before. Among smaller organizations, the share reporting scaling stayed flat at 22%.

For small and mid-sized businesses, that gap is less a warning than a reminder. Large companies have platform teams, data infrastructure and budgets for experimentation. A smaller organization gets better results by choosing one narrow, valuable task and doing it properly rather than trying to “deploy agents” broadly.

The risks you need to design for

Agents introduce security and reliability risks that ordinary chatbots do not, because they can take actions. The OWASP Top 10 for LLM Applications (2025 edition) names two that every business should understand.

Excessive agency

OWASP defines excessive agency as the vulnerability that “enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from an LLM.” Its root causes are excessive functionality, excessive permissions and excessive autonomy. In plain terms: the agent can do too many things, with too much access, without anyone checking.

Prompt injection

OWASP’s prompt injection entry describes how instructions hidden in content the model processes, such as a web page, email or document, can alter its behavior. This “indirect” injection matters most for agents, because agents routinely read external content and then act. An agent that reads customer emails and can issue refunds must be designed on the assumption that some emails will try to manipulate it.

Practical controls

The mitigations recommended by OWASP and by OpenAI’s guide overlap heavily:

  • Least privilege. Give the agent only the tools it needs, with narrow, specific functions rather than open-ended ones, and run them with the permissions of the user or a restricted service account.
  • Human approval for high-impact actions. OpenAI’s guide recommends that actions which are “sensitive, irreversible, or have high stakes should trigger human oversight.”
  • Clear failure limits. Stop and escalate to a person after repeated failures or when the agent exceeds a retry limit.
  • Input and output checks. Filter for off-topic requests, unsafe inputs and unnecessary exposure of personal information, and validate outputs against expected formats.
  • Logging and monitoring. Record what the agent did and why, so that problems can be investigated and quality can be measured.

These controls belong in your wider AI governance. If you do not yet have an AI use policy, our guide to writing an AI policy and governance framework covers the basics.

How agents connect to your systems

An agent is only as useful as the tools it can reach. Historically, each connection between an AI system and a business application had to be built one by one. The Model Context Protocol (MCP) is an open-source standard for connecting AI applications to external systems such as data sources, tools and workflows. Its documentation lists support across AI assistants and development tools, including Claude, ChatGPT, Visual Studio Code and Cursor.

For a business, standards like MCP reduce integration effort, but they do not remove the need for design decisions. Every connection is also a permission. Before exposing a system to an agent, decide what it may read, what it may change and what must be approved by a person.

How to run your first agent project

Both Anthropic and OpenAI recommend starting simple and adding complexity only when it clearly improves results. In our approach, a first agent project follows six steps.

  1. Pick a bounded task, not a job title. “Resolve order-status and delivery-date questions” is a task. “Replace the support team” is not. Choose something frequent enough to matter and contained enough to test.
  2. Try the simplest design first. Ask whether a fixed workflow with one or two model calls would do the job. If it would, build that. Move to an agent only where the path genuinely varies from case to case.
  3. Define the tools and permissions. List each system the agent needs, whether it reads or writes, and which actions require approval. Start read-only where possible.
  4. Build an evaluation set. Collect real examples, including awkward and adversarial ones, with the outcome an expert would expect. Measure the agent against them before and after every change.
  5. Launch with a human in the loop. Start with the agent proposing actions for a person to approve, then widen its autonomy step by step as measured quality justifies it.
  6. Monitor cost, quality and adoption. Track completion rate, escalation rate, errors caught in review, time saved and cost per task. Agents can use many model calls per task, so cost deserves as much attention as accuracy.

If you are earlier in the journey and still choosing your first AI use case of any kind, start with our guide on where to start with AI in your business.

Questions to ask before you commit budget

Before approving an agent project, a leadership team should be able to answer the following:

  • What specific task will the agent perform, and how is that task done and measured today?
  • What is the worst realistic outcome of an error, and how would we detect it?
  • Which actions will require human approval at launch?
  • Who owns the agent in production, including its quality, cost and incidents?
  • What data will it access, and does that match our privacy and contractual obligations?
  • What result would lead us to stop, and when will we review it?

If those answers are vague, the project is not ready. That is not a reason to abandon it, but it is a reason to spend another week on scoping before spending months on building.

The bottom line

AI agents are a real and useful capability, not just a rebranded chatbot, but they are the most complex way to apply AI in a business. They pay off on bounded, multi-step tasks with varied inputs, clear tools, measurable outcomes and human approval where it counts. They disappoint when they are used for novelty, given broad access or asked to replace judgment that nobody can check.

Start with one task, the simplest design that works and tight permissions, then earn autonomy with evidence. If you would like help identifying where an agent, or a simpler workflow, fits your operations, explore our AI & Automation services and business automation solutions, or get in touch to talk it through.

Sources

FAQ

Questions, answered.

What is an AI agent in simple terms?

An AI agent is software that uses a large language model to decide which steps to take and which tools to use to complete a task. Instead of following a fixed script, it looks at the situation, takes an action such as looking up a record, checks the result and decides what to do next until the task is finished or it needs a person.

What is the difference between an AI agent and a chatbot?

A chatbot responds to one prompt at a time, and a person drives every step of the conversation. An AI agent is given a goal and access to tools, then plans and carries out several steps on its own, such as searching records, updating a ticket or drafting a reply. That ability to take actions is what makes agents more useful and also riskier.

How is an AI agent different from RPA or workflow automation?

Robotic process automation and workflow tools follow steps a developer defines in advance, so they are predictable but break when inputs vary. An AI agent lets the model choose the steps at run time, which helps with messy inputs and exceptions. Many good business solutions combine both, using fixed automation for stable steps and a model only where judgment is needed.

What are good first use cases for AI agents in a small or mid-sized business?

Strong first candidates are bounded, frequent tasks with varied inputs and recoverable mistakes. Examples include answering routine customer questions using order data, handling internal IT and operations requests, preparing research briefs and assembling documents for back-office review. Start with the agent proposing actions for a person to approve, then widen its autonomy as measured quality improves.

Are AI agents safe to connect to business systems?

They can be, with the right design. Give the agent only the tools and permissions it needs, require human approval for sensitive or irreversible actions, set limits on retries and failures, filter inputs and outputs, and log what it does. Assume it will read content that tries to manipulate it, a risk known as prompt injection, and design accordingly.

Why do so many agentic AI projects fail?

Gartner has predicted that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. In practice, projects struggle when the task is chosen for novelty, success is never defined, the agent is given broad access or nobody owns its quality and cost in production.

Do AI agents replace employees?

Today, agents are best at taking on specific tasks within a role, not whole roles. They can handle routine cases, gather information and prepare work so that people spend more time on exceptions and judgment. Reliable results still depend on human review for high-impact decisions, clear ownership and ongoing measurement of quality.

What is the Model Context Protocol (MCP)?

The Model Context Protocol is an open-source standard for connecting AI applications to external systems such as data sources, tools and workflows. It reduces the effort of building one-off integrations between AI tools and business software. Each connection still needs deliberate decisions about what the AI may read, what it may change and what requires approval.

Have something ambitious in mind?

Tell us where you are and where you want to be. We'll come back with honest thinking on how to get there.