How Enterprise Architecture Keeps AI Projects From Failing

AI Is a Loaded Term

Industry watchers have suggested that more than 50% of AI implementations fail. Many of those failures could have been prevented if someone had asked the right questions first.

For example, what do we mean by “AI”, exactly? Is it generative AI, which can hold human-like conversations with users and provide them with first-line customer support? Or the kind that can recognize patterns in medical imagery and alert physicians about potential abnormalities before they would notice them themselves? Or do we mean agentic AI, which can autonomously perform complex, multi-step tasks by invoking and interacting with external tools such as email clients or web APIs?

Understanding these distinctions marks an important first step towards success, because every type of AI comes with its own limitations that will impact the performance and viability of the solution you’re trying to build. From there, you can ask other questions to narrow down the case even further. 

This article will walk you through a list of such dilemmas, or trade-offs. By the end, you should have a clearer understanding of what’s at stake when implementing an AI solution in your business. We will also discuss how a process called Enterprise Architecture (EA) can guide you in navigating AI implementation pitfalls and arriving at a solution that is optimal for your business case – including one that isn’t AI at all.

The Right Approach to Success With AI

Being successful with AI implementations isn’t magic. It is about making good choices when it comes to what problem to solve, what architecture pattern to follow, and what AI tooling to use. Try to solve a problem with AI that doesn’t need AI and which AI isn’t good at solving and you will likely fail. Similarly, try to use the wrong tool to solve the problem and you will fail.  Being successful is all about asking the right questions and then making the right choices which may lead you away from using AI or might tell you to use a different tool or to implement a different architecture. If you come forward with a decision to use Agentic AI even though the problem isn’t a good one for AI and doesn’t need to be agentic, then you shouldn’t be surprised if it isn’t successful.

If success means you need to try it out and accomplish something (faster? better? cheaper?) while doing so, then choosing the right problem to solve, the right architecture to solve it, and the right AI tool(s) is key.

Why Businesses Fail With AI

Conversely, if the definition of success is merely to kick the tires on this new AI “thing”, then randomly choosing tools and architecture and what problems to tackle will likely allow you to do that. Bear in mind, though, that this approach will only guarantee one thing: that you will have tried out AI. Whether it will save you any money, improve your operations, or improve the quality of your product or service, is anyone’s guess.

The Trade-Offs to Consider Before Implementing AI in the Enterprise

The following is a complete list of tradeoffs/decisions that typically need to be considered when implementing AI in business:

Capability vs. Fitness

Why Is It Important

AI is capable of doing lots of things, but isn’t necessarily fit for every purpose.  Choosing to use AI but for a purpose that it doesn’t necessarily align well with, such as creating a new product, may be folly.  Using it to improve the quality of something, for example, might be more fit for purpose.

Failure Examples

One company experimented with having a gen-AI solution create new recipes only to find out that having never tasted anything, the solution often put together things in such a fashion that no one liked the resulting food.

Using AI for customer support may sound like a more conventional use case, but the reality begs to differ. Some large retailers have tried to use AI for customer service, but with anywhere from 20-50% of the customers getting frustrated and eventually asked for a live representative. Maybe this problem isn’t an AI problem to solve.

Accuracy vs. Good Enough

Why Is It Important

Some things need to be right, such as a medical diagnosis or perhaps the invoice you get when you order a product or service. Close isn’t good enough since it might result in someone dying or perhaps someone paying too little or too much.

Failure Examples

AI has been used to diagnose illnesses in people and there have been studies done that show it can be as accurate as some specialty doctors for diagnosing some conditions.  However, it was also shown in some studies that AI misdiagnosed conditions which were properly diagnosed by other doctors.  Being 90% right isn’t good enough if the other 10% dies.

Build vs. Buy

Why Is It Important

Lots of companies suffer from the incorrect thinking that they are unique.  No other company has the same problem or could need the same solution, thus they decide to build something that they could have just bought, knowing it already worked fine for some other companies.

Failure Examples

Using computer vision to “watch” the manufacturing of something can catch and/or correct defects.  Having the same AI make decisions without human intervention may result in many false positives or false negatives. A feedback loop where an expert can override the AI can provide a mechanism for the AI to improve over time.

Agentic vs. Generative AI

Why Is It Important

This can be a tradeoff between time to respond to a request and the cost of getting the answer.  Token costs will add up. Instead, consider gathering information without having a customer sit on hold, and respond later. It can cost less and result in a happier customer.

Failure Examples

Gartner says as much as 50% of implementations of agentic AI today don’t require an agentic implementation.

Autonomous vs. Expert Systems

Why Is It Important

Does the AI system need to make the decision itself or could it be used to catch mistakes or advise an expert, thus improving the quality of decision making.

Failure Examples

Using computer vision to “watch” the manufacturing of something can catch and/or correct defects.  Having the same AI make decisions without human intervention may result in many false positives or false negatives. A feedback loop where an expert can override the AI can provide a mechanism for the AI to improve over time.

Cost vs. Benefit

Why Is It Important

How much will it cost to develop and run an AI-based solution?  What will be the benefits of using that solution?  It is important not to try to leverage AI to solve problems where the solution is going to cost more than the “old” way of doing things.

Failure Examples

In a widely reported example a large restaurant chain was using AI to take drive-thru orders but 20% of the customers had mistakes in their orders and had to talk to a human representative to correct the mistakes.  The argument for AI was that they could reduce the number of people working in the restaurant.  Instead, they found they had to pay for AI and still needed the same number of workers, thus increasing the operating cost rather than reducing it.

Consistency vs. Risk

Why Is It Important

How much will it cost to develop and run an AI-based solution?  What will be the benefits of using that solution?  It is important not to try to leverage AI to solve problems where the solution is going to cost more than the “old” way of doing things.

Failure Examples

There have been numerous stories of lawyers trying to have generative AI used to develop briefs or arguments in court cases where the cited cases do not exist or were interpreted wrong. Checking for accuracy and consistency is important, especially when getting caught can create significant financial or reputational risk.

The Role of Enterprise Architecture (EA) in AI Success

Enterprise Architecture (EA) is an umbrella term for various frameworks and processes whose job it is to ensure that critical decisions and operations align with an organization’s strategy and goals. To that end, businesses use EA to monitor and manage challenges like obsolescence risk, post-merger IT integration, business capability mapping, and other sensitive areas with potentially pivotal business ramifications. As such, EA can also be used to navigate the trade-offs of AI implementation.

Navigating AI Trade-Offs With Enterprise Architecture

So, how does EA help ensure success with AI? The key is in the common EA processes and governance espoused by frameworks like The Open Group Architecture Framework (TOGAF).  TOGAF says you start with a problem and then determine how best to solve the problem. You don’t start with AI (the tool) by default. That way depending upon the problem, you may choose AI, if appropriate, but could be steered towards a non-AI solution, if appropriate. Similarly, you can look at solutions which exist already and potentially buy or re-use a solution instead of trying to build a new solution from scratch.

The Tangible Benefits of Applying EA to Your AI Implementation

Avoiding Misinvestment

A lot of basic EA practices, like figuring out the cost/benefit of a solution before investing in it can stop AI failures from happening because you will know what something is going to cost and what the benefits of a solution will be before actually doing the work, spending the money, and finding out later it will never make economic sense to do the project.

Improving Accuracy

And finally, EA can help ensure accuracy and consistency through several mechanisms. One is through proper validation of answers and then storing those answers and checking future answers against them. You get to detect the inaccuracy and/or inconsistency and can reuse answers. This lowers the cost of AI and can keep you from litigation due to inconsistency or inaccuracy. 

For an example of how these principles can be applied in practice, read our blog on how to ensure, maintain, and monitor correctness in a RAG-based chatbot by implementing a testing pipeline: How to Evaluate RAG Systems. Beyond Pass/Fail.

Improving Results Through Standardized Tool Use

Also, by architecturally using the same tools and LLMs, for example, can ensure you don’t have different tools and different LLMs coming to different conclusions/answers. This is one of the areas where AI has often failed as different groups inside a company go in independent directions using different tools and different LLMs and then acting surprised when the answers to the same or similar questions end up being different. The words (“prompts”) matter and if the knowledge the AI model has to draw upon is different, different answers are inevitable.

For BigFilter, a social impact startup, we have designed a fact-checking prototype with built-in transparency controls, which gave the company granular insights into how the system performed at discreet stages depending on the kind of LLM involved. Read more in our case study: From Idea to Prototype in 3 Months: A Case Study in AI Fact-Checking for BigFilter.

De-Risk AI Implementation with EA and an Experienced Service Provider

Working with IT service providers, like Janea Systems, that have experience with these tradeoffs and have Architects who will go through all of them before settling on a proper roadmap going forward takes a lot of the risk out of implementing leading/bleeding edge technologies, like AI. If you are going to trust AI to help run aspects of your business, trust experts like Janea Systems to help you implement those solutions.

Industry Examples

Click the links below to see industry-specific examples:

• Enterprise Architecture for AI in Healthcare and Oncology
• Enterprise Architecture for AI in Finance

About the author

Thomas Feichtinger is a Chief Enterprise Architect (EA) who has started, led, and matured EA at large companies like General Motors, Waste Management, Hewlett Packard, Inspire Brands, and Novelis over the past 25 years. He has been on the bleeding edge of applying new technologies to solve tough business problems throughout his career, including AI, machine vision, mobile technologies, and networking.

Editorial contributions by Hubert Brychczyński