Skip to content
Chat with AI Agent
Business & governance

It Was Never the Model: What 51 Successful AI Deployments Changed in the Organisation

Orange ITS — AI engineering team 9 min read

A managing director in a 150-person logistics firm approves an AI pilot for invoice processing. The IT team picks a capable model, builds a working demo in six weeks, and everyone nods in the review meeting. Six months later the demo still runs, but nobody in accounting uses it, the finance lead says the numbers cannot be reconciled, and the compliance officer has never seen the data flow.

The model worked. The organisation around it was never changed. Our work as forward-deployed AI engineers starts from that observation, and a recent Stanford study of 51 working deployments gives it unusually concrete support.

Across the deployments Stanford examined, the differences that mattered sat in ownership, process, data access and measurement, with the model playing a smaller part than most buyers expect. This article walks through what the organisations changed, how far the evidence carries, and what an owner can test before committing budget.

Scope: this article reads one study on its own terms and does not repeat the broader debate about AI failure statistics, which we cover in six AI statistics your board quotes. Where a claim is the authors’ interpretation rather than a measured result, we say so.


What the Stanford Study Is, and How Much It Can Prove

“The Enterprise AI Playbook: Lessons from 51 Successful Deployments” was published in April 2026 by Elisa Pereira, Alvin Wang Graylin and Erik Brynjolfsson of the Stanford Digital Economy Lab. You can read the full report or the publication page.

It covers 51 deployments at 41 organisations in seven countries, nine industries and 13 named functions plus an “other” group. To qualify, a system had to be live and integrated, used by teams for at least three months, and able to show quantified value. The evidence behind each case is mostly at least one structured 60-minute interview, supplemented by internal documents where available.

Four limits the authors and the record make clear

  • The sample is success-selected. The authors write that the study carries “selection bias toward positive outcomes” and that they do not claim to provide representative data on how common success is across the broader economy.
  • The data is self-reported. The report names reliance on interview participants as its primary limitation, and the productivity and value figures come from the organisations themselves. Many percentages are printed without a stated base, so read them as rough proportions.
  • The cases are anonymised. You cannot look up a company and check what happened.
  • Company size is open. The report gives no size for most of its 41 organisations, only an aggregate workforce of over one million and a few case-page size tags. The case pages that carry a size tag are marked mid-market or enterprise, so transfer to a 20 to 200 person firm is a question the study leaves unanswered.

One further disclosure belongs here. According to the report’s contributor biographies, Brynjolfsson co-founded Workhelix, a company that works on identifying and measuring the benefits of AI. The report does not say whether any of its 51 cases used Workhelix, and the study places weight on measurement, so a careful reader notes the overlap.

None of this makes the study useless. It means the findings work best as a list of hypotheses grounded in real cases, rather than as proof that doing X raises your odds of success.


The Second Attempt Is Where the Lesson Sits

According to the report, 61% of the cases had a failed AI project before the one now in production. Another section of the report, on how the authors handled failure, says two thirds of the companies had significant failed attempts. The report gives both figures.

The pattern is more interesting than the number. The authors describe a professional services firm whose first attempt, led by its CTO, failed. A second attempt succeeded when the CEO and the Head of Talent drove it together with the CTO. In eight cases overall, the authors say business and technical leaders working together made the difference.

One executive quoted in the report puts it bluntly: when AI is tech-led and tech-first, it rarely works. That is one interviewee’s opinion. It still matches what we see in project reviews. When the technical team owns the outcome, nobody on the business side feels responsible for changing how the work is done.

For a longer treatment of why projects stall at this point, see why AI agent projects fail.


Sponsorship Means More Than Signing the Budget

The report breaks executive sponsorship into four behaviours across the 51 cases: allocating resources (59%), integrating AI into strategic goals such as OKRs (49%), communicating across the organisation (32%) and removing blockers (20%).

The firmest statement concerns a small group. According to the report, the seven cases that achieved organisation-wide transformation all reached what the authors call strategic integration. The authors also note that the external studies of sponsorship they cite establish correlation but do not address what sponsors actually do. The behaviours list is their own coding of interview material.

What does this mean in a 100-person company? The sponsor is probably the owner or a managing director, and the question is concrete. Who decides when the new workflow replaces the old one? Who answers when the finance lead objects? If the honest answer is “IT will sort it out”, you have found the gap the study describes.


The Workflow Changed Before the AI Did

The report’s cases show process work coming first. In an invoice-processing project at a US logistics company the report describes as $1B+, the team reduced thousands of invoice templates to hundreds before any AI was built, according to the report. A recruiting project at a translation services firm began by mapping how recruiters actually worked.

Two points of caution apply. The report does not count how many of the 51 cases redesigned workflows, so “most successful deployments did this” is not something the study supports. And the reported results (the logistics firm describes going from seven to two full-time equivalents, and more than one million dollars in value) are self-reported figures without a defined metric.

What the cases do support is a sequence. Map the work, remove what should not exist, and then decide where a model fits. Our article on AI process redesign covers the mechanics, and the implementation roadmap shows where redesign sits in a phased plan.

Among the cases where the authors could identify the development method, all used an iterative approach, according to the report. The report also notes that similar use cases took weeks at one company and years at another. The three examples behind that comparison are anecdotes.


Where the Hidden Costs Were

Asked what was hardest to fix, interviewees named change management most often (33% of answers), followed by data quality and architecture (17%), process redesign (10%), output quality (10%) and the ROI or business case (7%). The authors group the first three, plus the last two, as “invisible costs” and report them at 77%, against 23% for visible technical problems. It is unclear whether respondents could name more than one challenge. Placing the business case in the invisible category is the authors’ choice.

Even with those caveats, the direction is plausible. Budgets usually cover licences, integration and development. They rarely cover the weeks that people in operations, finance and legal spend answering questions, testing edge cases and changing habits. If a quote for an AI project contains no line for that time, the real cost is higher than the quote.

Data access comes before data perfection

The report says only 6% of implementations had data fully ready for AI. It also reports that 88% used language models to reach data that was previously hard to use, such as free-text documents. The authors acknowledge that a success-only sample likely underrepresents cases where data quality proved insurmountable.

The practical reading is narrow. Perfect data is not a prerequisite in the cases studied, but access to the right data is. Check who can export it and under whose approval.


The People Who Slow You Down Are Not the Ones You Expect

In the 51 cases, staff functions (legal, HR, risk and compliance) were the most frequent source of resistance, at 35%. End users followed at 23%, external parties at 23%, C-level at 15% and no resistance at 8%. The figures sum to slightly over 100%, and the report does not explain why.

The report adds that frontline fear of being replaced appeared in only two cases. That cuts against the popular picture of workers sabotaging automation, at least within this sample.

For a Swiss firm the finding has a practical consequence. Your data protection adviser, works council equivalent or compliance officer should see the design before the build starts, well ahead of launch. The study’s cases do not cover Swiss or EU regulation, so what these functions will ask depends on your sector and on the data involved.

The report also records what happened to headcount: 45% of cases saw a reduction, 19% no reduction, 17% redeployment and 19% avoided hiring. These are self-reported, and the authors observe that cost-focused ownership tends toward reduction. Any owner should decide in advance which of these outcomes they want, because staff will ask. For the roles that shift and the headcount rules that backfire, see team roles in an AI-first company.


What the Study Says About the Model, and What It Does Not

For 42% of implementations, the authors rate model choice as fully commodity. 39% rate it of moderate importance and 19% a critical differentiator. Routine tasks were far more often commodity than advanced ones.

The authors then conclude that the durable advantage lies in the orchestration layer rather than the foundation model. That is their interpretation, supported by case examples but without a comparison group. It is nevertheless consistent with the sequence above: sponsor, process, data, then technology.

Two claims in the report deserve to be left alone. It describes autonomy and productivity gains in a way that partly reflects which tasks were chosen, as the authors themselves note. And it asserts that clear KPIs before deployment make scaling more likely, but the report does not compare organisations with and without KPIs. It does offer a catalogue of metrics reported by at least one case, which is useful for choosing what to measure. See AI agent KPIs for a working set.


A Four-Week Test Before You Commit Budget

If the lessons are hypotheses, test them cheaply on one process. Pick a recurring task where delay or error has a visible cost.

  1. Name one business owner. This person has authority over the process rather than the technology, and agrees to attend every review.
  2. Map the current workflow. Write down the steps, the exceptions and the handoffs. Mark which steps exist only because of an old system.
  3. Check data access. Identify the documents or records the process needs, who can export them, and what approvals apply.
  4. Bring in staff functions early. Show legal, compliance or HR the intended data flow and ask what would stop it.
  5. Record a baseline. Cycle time, error rate and cost per case, measured for a few weeks before anything changes.

If you cannot complete these steps in about four weeks, the obstacle is organisational, and a better model will not remove it. That is a useful result in its own right, and a cheaper one than a failed build.


What the Study Cannot Tell You

The report cannot tell you the odds that your deployment succeeds. It cannot tell you whether your competitors who stopped after one failed attempt would have made it with a different sponsor. And it cannot tell you whether a 40-person firm needs the same governance as a mid-market company with a dedicated AI team.

What it can do is describe, in 51 documented cases, what organisations did after they stopped treating a deployment as a technology purchase. Some of that will transfer to your business. Some will not. The way to find out is to apply the lessons to one process and measure the result against your own baseline, then decide what to scale on the evidence of your own numbers. For how to run the live system afterwards, see managing AI agents in production.

Frequently asked questions

What does the Stanford Enterprise AI Playbook actually study?

It studies 51 AI deployments at 41 organisations that were already in production, used by teams for at least three months and showing quantified value. Authors Pereira, Graylin and Brynjolfsson interviewed the organisations and anonymised the cases. Because every case is a success, the report describes what working deployments did and cannot show how often deployments work.

Why do many AI deployments fail before one succeeds?

According to the Stanford report, 61% of the 51 cases had a failed AI attempt first, while another section of the report says two thirds of the companies had significant failed attempts. The authors link the turnaround to organisational changes such as a business sponsor working with the technical team and a redesigned workflow. That reading is an interpretation drawn from interviews and has not been tested as a cause.

Do these lessons apply to a small or medium-sized company?

Probably in part, but the evidence is open. The report gives no size for most of its 41 organisations beyond an aggregate workforce of over one million, and the case pages that carry a size tag are marked mid-market or enterprise. Sponsorship, process mapping and data access are not size-specific ideas. Treat them as hypotheses to test on one process before you scale.

Who resists AI deployments inside a company?

In the Stanford sample, staff functions such as legal, HR, risk and compliance were the most frequent source of resistance at 35% of the 51 cases, ahead of end users at 23%. The authors report frontline fear of replacement in only two cases. The authors say overcoming that resistance took mandates tied to corporate goals and a governance role for those functions, rather than persuasion. Showing them the design early is our own practical suggestion.

Does the model choice decide whether an AI deployment succeeds?

The report suggests it matters less than many buyers assume. For 42% of implementations the authors rated model choice as fully commodity, and routine tasks mattered less than advanced ones. The authors interpret the durable advantage as sitting in the workflow and orchestration layer around the model.

Insights

Put these ideas to work

A 30-minute call is enough to find out whether an AI agent fits your workflow — and what it would return.