Hello, and welcome back to this three-part series on Knowledge-Based Systems. In the second part, we introduced Enterprise Knowledge Management Systems that provide centralized access to information within an organization.
In this part, we will go further and not only accelerate the access to information, but describe how LLM powered AI Agents can be used to automatize pre-define business processes.
Tier III – AI Agent Platforms
AI Agents have been a very hot topic over the last few months. Many see the future frontier of AI in developing autonomous AI Agents with super human capabilities. With use cases that include personal assistants, AI Software Developers, AI Researchers or robots operating in the physical world.
Those use cases are not the focus of this work. Instead, I want to look at knowledge-based systems as AI Agent Platforms that are a lot closer to "traditional" Enterprise Robotic Process Automation (RPA) that came into flourish in the early 2000s with companies like UiPath Blue Prism and provide solutions to automate pre-defined business processes. Further, we look at AI Agent platforms that make use of the available data corpus and service of an organization to automate and support knowledge work. While containing the capabilities to serve use cases like the Q/A and EKS, we will focus on their added value to enable the creation of user defined workflows.
Figure 3 illustrates an AI Agent Platform and what additional capabilities it has compared to a Knowledge Management System.
In contrast to EKS systems, an AI Agent platform not only serves human users through a chat interface but is intended to be used by other services and AI Agents (1). This is possible with an Agent Portal (2) extending the Chatbot Portal of the EKS system to permit the development of Workflows that are executed by semi-autonomous AI Agents with access to software tools and services. A separate Agent takes the place of an orchestrator who is responsible for identifying the intent of a request and coordinating how requests are routed to different agents and what workflows are triggered. Platform users can define their own workflows and configure agents using visual development platforms through no- or low-code options. The different Agents are augmented with the capability to not only search the information ingested from different document platforms, as an EKS could but in addition can access external resources and services through versatile API calls (3). This includes services like web search engines and can include, in combination with tools, the capability to automatically query external databases. While Agents can still serve use cases in which they respond with human-readable answers in a chat interface, their capabilities are extended to be able to trigger other "downstream" services (4). Such downstream services could be existing business processes or internal and external 3rd Party services.
Where the EKS has the focus on providing quick and easy access to an organizations information, plus limited generative capabilities, an AI Agent platform extends the capabilities of Agents to incorporate external resources and leverage external services in order to achieve pre-defined workflows.
A EKS is a tightly managed system that provides operators of the system the capability to control the data import process ensuring data quality and to manage Chat Bots and Profiles that enable a specific set of use cases.
The AI Agent platform in contrast enables a more self-service like approach, where users can build their own workflows and agents that serve their particular use case. This introduces additional challenges, which will be discussed later, but provide a chance for a bigger audience to benefit from the platform and encourages a "organic" growth and exploration of new use cases.
Example: How an AI Agent platform transforms customer service
Imagine a customer service platform for an electronic consumer device manufacturer. The company has an AI Agent platform in place that supports human service operators by handling multiple steps in the customer service journey. The following is an illustration of such a customer journey.
The first touch point is a customer service voice bot that is having a conversation with a customer reporting an issue with her smartphone speaker not being able to connect to her smart home system. The voice bot follows a partially pre-defined script, similar to how a human operator would work while providing the customer the freedom to have a natural conversion. After failing to resolve the customer’s issue, the voice bot creates a support ticket, including all relevant information contained in the conversation so far, and inquires about additional relevant topics like the smart home setup.
Another support ticket Agent is analyzing the newly created ticket and starts to augment the customer-provided information with background information that can help to resolve the case. This includes, for example, publicly available information about the products used in the smart home setup acquired through a web search. Based on the new extended support ticket, the Agent creates a suggestion on how to resolve the issue, relying on product documentation and previous support cases that describe similar issues. Part of the suggestions is a technical description of the likely causes of the issue, tailored to explain the situation clearly to the human customer service operator, as well as creating a draft for an email to the customer explaining how to resolve the issue.
When the human service operator opens the support case, they find all the information they need to verify if the diagnosis of the AI Agent is correct and if they agree with the proposed solution. They focus on validation and evaluation of the AI Agents' work instead of using tools like an EKS, to find a solution themselves.
When the human operator approves the proposal, the Ticket AI Agent takes over again in order to update the ticket and sends an email to the customer.
Example: How an AI Agent platform enables employee-driven use case development
The following example shows, how User created workflows, covering new use cases, when growing in popularity can become part of the standard platform functionality to benefit a bigger audience.
André, as a team lead in a medium-sized company, has many online meetings for which he tries to make sure that every meeting provides valuable outcomes and clear todos.
Keeping track of all todos that come up in a meeting can be cumbersome, and adding them to an Issue tracking system is even more sore.
For that reason, André has created a short workflow on the company’s AI Agent platform. To use it, he adds a bot to an online meeting. Once the meeting has ended, the Agent workflow is triggered. As part of the workflow, the meeting transcripts are analyzed (if transcripts do not exist, they are first created with a Speech-to-Text model) and a list of Todos including who is responsible and optional deadlines are created. Based on the list, an Agent creates separate Todo tickets, including a short description, deadline, and summary of the context in which the task was defined.
Other employees within the company quickly see how the workflow helps André to save time and create their own copy. The Agent Platform team, which is continuously observing how people are using the platform, can see the opportunity to help people within the company and create a slightly modified and more resilient Todo Agent that Platform users can make use of without creating and maintaining their own.
Current Landscape of AI Agent Platform
AI Agents and Agentic Systems in general, have received a lot of attention in the last months, pushing forward the hype train led by LLMs. Many companies providing chatbot and robotic process automation solutions have pivoted towards AI Agent platforms (see Zapier Agents, [Botpress](bot press.com), Make.com, ChatBase). The market is moving so quickly at the moment that it is difficult to predict with precision where it is heading and to pick favorites.
Broadly speaking, there are three categories of AI Agent Platforms and Frameworks at the moment.
- Business Process Automation: That focus on enterprise integration with existing data sources and services, providing no- or low-code platforms. Example of this are Stack AI and Microsoft Copilot Studio.
- General Purpose AI Automation: Targeted towards a bit more technical audience then the Business automation platforms, with great flexibility to implement a vast number of automation use cases ranging from individual so small business. Examples of this are n8n or relevance AI.
- Developer Frameworks: Software Frameworks are heavily Python focused and provide building blocks to implement custom AI Agent systems. Examples of this are Microsoft – AutoGen, LangGraph, PydanticAI or OpenAI Agents.
Having so many options on the table, the question arises, which one to choose under what circumstances. When to buy and when to build.
First of all, because of the velocity with which the technology is evolving and the rate of new products and frameworks emerging on the market, it is essential to prevent any vendor and platform lock-in, as well as to rely heavily on a single technology or solution.
The big majority of companies will benefit from AI automation, and can do so, without the need to be on the bleeding edge.
Depending on your sector, the investment into AI automation will be more or less pressing, but I would highly advise gathering experience and developing use cases by performing small prototypes using either general-purpose AI automation platforms or Business Process Automation systems before thinking about any bigger investment. Like the underlying LLMs, the platforms develop rapidly and current limitations may very well disappear within the next 6 months, making it possible and useful to re-evaluate the feasibility of previously failing use cases on a regular bases.
Building your own AI Agent platform
Building your custom AI Agent platform is a major endeavor with unclear benefits over the quickly evolving AI Agent platform market. It requires a dedicated team made up of experts in cloud/on-prem infrastructure, system developers, AI experts and UX developers and at least 12 months to get a first working platform.
Building a platform will require solving all challenges that have been presented for Knowledge Management Systems, plus the following.
- Unstable Ecosystem: The previously mentioned Developer Frameworks are all very new and building on them will likely require re-writes in the future. I think that many of even the highly popular frameworks like Langchain are great to get experiments started, but provide a lot of weight that is not needed for each particular project. Getting inspired by the different frameworks, and implementing the required parts for your own project is a viable choice. As mentioned in this blog post, I see great value in limiting the use of Python to experimentation and essential AI components of the system that require the Python ecosystem and tooling. Choosing, therefore, a microservice architecture that provides the flexibility to implement different services in different technology stacks is essential.
- Observability / Agentops: The stochastic nature of LLMs and, therefore, AI Agents that build on them require continuous monitoring and logging in order to study unintended system behavior. Multiple frameworks have emerged that focus on AI Agent use cases, like Langfuse, OpenLIT or Agentops. Similar to the Developer Frameworks mentioned previously, AI agent-specific observability tools might have certain convenience, but when building a project of this size, I would recommend to rely on more established observability technologies like Open Telemetry, especially because they as well try to improve their support for AI agent use cases as discussed here.
- Interoperability between Systems: AI Agents are empowered by the tools and services they can use. Writing and maintaining custom adapters to external resources and services is very resource-consuming. Although Anthropic has gathered certain momentum with its Model Context Protocol that could serve as a standard that helps to connect Agents to external services, it is not yet widely used and its at the moment unclear if its adaption will continue. For the near future, it is, therefore, very likely that one needs to develop and maintain a high number of adapters.
- Agent to Agent Communication: AI Agent platforms encourage Agent-to-Agent communication in order to extend the capabilities of individual agents. This includes agents on the same platform, but as well external agents that are available as API services. At the moment, no standard exists, and the Model Context Protocol does not explicitly cover this use case. What happens in practice is that at the moment, one model is communicating with another one through human language, including all its inefficiencies and potential for misunderstanding. This increases the brittleness of the system as a whole, as the behavior of both agents is stochastic in nature on an individual level, and established best practices like prompt engineering to dictate the behavior of a model are often model-dependent. This means that optimizing the prompt of model A to communicate with model B in a certain way suddenly has the challenge of attending to the model peculiarities of two models instead of one. In the case of both models running on the same platform, under one organization, this might be feasible, but as soon as an Agent starts to communicate with Agents on other platforms that have the freedom to change the underlying behavior at any time, the stability of the system is at risk.
- Agent & Service discovery: In distributed systems, the challenge of service discovery is well-known. It addresses the problem of identifying what services and resources are available and where within the system. In case of AI Agent platforms that have grown to a significant size, where internal and external Agents, tools and services are coming together, a similar discovery service is needed. It helps human and agents alike to understand what resources are available to them in order to solve a problem. This type of catalog includes existing Agents and their capabilities as well as services and sources with a specification (for example MCP) how to access them. Like in traditional distributed systems, it is a challenge to keep this information up to date and find a good balance between an on-demand discovery of the current state of the system and relying on previously gathered information. For most use cases it is not feasible to for each new request query all agents and resources for the current capabilities because of the time it takes to do so and the costs generated by such a discovery, but on the other side the platform as a whole is to dynamic to make it possible to maintain a curated list of resources. The recommendation is, therefore, to have an automatic system in place that periodically performs a discovery of the platform or mechanisms that allow individual agents and services to make their existence and capabilities known.
How to get started with AI Agent Platforms
The best way to get hands on experience with AI Agent Platforms is to try out some of the available web based offerings. Most of them have some kind of free subscription or 14 day evaluation plan.
I would recommend trying out StackAI or or n8n. An quick alternative to get an idea and see such a system in action is to watch a demo online (like for example this StackAI Demo).
During the evaluation one can of course not test a systems capability to integrate with an organization, but the user experience of creating AI Agents on a platform and how to create simple workflows.
I would recommend to try out the following
- Create a workflow that starts out with a document, by receiving an email or a web search.
- Uses a LLM to analyze the input and extract some structured information.
- Use the extracted information to call a 3rd party service.
Example flows can be
- Extract a Todo and meeting schedule from an email and create a calendar entry, including an email response.
- Create a tweet or LinkedIn post based on an uploaded document.
- Create a writing Assistant that is triggered by an email or a document. Analysis of a text, correct spelling and writing, and provide feedback and recommendations for individual paragraphs.
Where AI Agent Platforms fail
As described in the introduction of this part, the AI Agent platforms discussed here, are not to be confused with the Autonomous Agents that solve multi-step reasoning tasks, but expecting exactly that from such a platform will set it up for failure.
The AI Agents are capable to detect the intent of a request and map them to existing workflows and tools. Based on modern LLMs, Agents are capable of understanding and generating human language that enables sophisticated text extraction and transformation use cases. Armed with access to tools and services, Agents can be used in complex workflows that solve many common knowledge worker tasks or support them so that human users can focus on validation and decision-making.
AI Agent Platforms can accelerate business processes, help employees to increase their efficiency and automatize narrow work flows, but are not capable of replacing an employee.
AI Agent Platforms are a natural progression of digitization and automation that will broaden the responsibilities and capabilities of modern knowledge workers.
Key Takeaways for AI Agent Systems
- Many different AI Agent platforms exist and are being developed rapidly.
- Avoid picking favorites or building your own; instead, invest in experimenting with specific use cases.
- When building your own solution, be aware of the many open problems related to Knowledge Management Systems and AI Agent platforms.
The present and future of AI Agent Systems
In this last part, I want to reflect and hypothesis on what the current limits and possible future frontiers for AI Agents are.
I distinguish therefore, between three type of AI Agents that share commonalities and might in the future even become one, but currently are still distinct.
- Assistants: are Systems that are used by humans as a tool or copilots. Either through text or voice, the human is "talking" to the assistant, asking it questions or giving it instructions. Most assistants we see so far are tightly integrated into a specific system and domain, to limit its scope and improve its results. This is currently the most deployed type of AI Agents for use cases like information retrieval, content creation, simple customer support and many more. They excel in use cases where the result of the assistants work is easy for humans to evaluate and failure causes little harm. Judging a picture or a piece of music, created with the help of an assistant is easy and if the assistant fails, nothing is lost. On the other hand, using an assistant for tasks like planning and booking a short trip is more complicated, because checking the correctness of all individual bookings and if they align with your preference takes a lot of time and in case of a problem, resolving it might be time consuming or costly. If the system is so reliable and capable of performing all requests correctly, we get into the field of autonomous agents.
- Autonomous Agents: are capable, without human supervision, of solving multi-step reasoning problems, like the development of complex software systems, managing a company, performing research, or even operating in the real world. Many of the limitations and challenges of the varying knowledge-based systems could be resolved with such autonomous agents. Many see them as equivalent to Artificial General Intelligence (AGI) and a cornerstone in modern AI Research in the academy and private industries. As it stands currently, the limiting factor of these systems is the high failure rate in multi-step reasoning chains. If an Agent has to perform 10 steps to solve a problem, even with a success rate of 95% in each step, will only reach its goal in 59% of the cases. As with many challenging tasks, specially in the real world, and this includes the real digital world, the verification of a result is a problem that AI Agents struggle with. Failing to judge if the result of an action is positive and to what extend it aligns with a goal is one of the major blockers of Autonomous Agents. For a discussion of current technologies have a look at my blog post reviewing the biggest AI trends of 2024.
- Delegates: are what I call AI Agent that serve as representatives or interfaces between organizations and individuals. An example of a delegate agent, is one in which for example the human resources department is maintaining an "HR Agent" on an AI Platform that can be used by employees to book holidays, ask contract related questions or in general handle support requests in place of a human work in HR. Instead of answering requests directly, the HR team is maintaining "their" own HR Agent that is handling the majority of requests for them. This combines the scalability of Agents with the flexibility of human knowledge workers and ensures that the agent provides qualitative results and stays relevant. In bigger organizations, this enables "local" control while enabling global synergies through agent-to-agent communications and workflows that combine multiple delegates from different teams.
At the moment of this writing, Assistants are already quite common and I expect specially voice assistants to become more prominent on mobile devices very soon. With the adoption of AI Agent platforms, we will see a rise in Delegate Agents and I expect certain roles like first level customer support to be replaced with Delegate Agents in the near future with a few exceptions for very high priced services.
With respect to Autonomous Agents, its very difficult to make predictions. Systems like Manus try to bridge the gap between Assistants and Autonomous Agents by doing their best to improve the reliability of the system to improve its success rate and reasoning capabilities, but more than that, by providing a user experience which enables the human operator to understand and be aware of what the system is doing and by eliciting human feedback and supervision while giving the human the impression that the Agent is doing most of the work.
I believe that the separation of knowledge-based systems into 3 tiers will remain relevant. An AI Agent platform can, of course, be used to provide the capabilities of a Question / Answering System and an Enterprise Knowledge System, but both of these systems are optimized for information retrieval, where the unique capability of an AI Agent platform is the easy creation of User-defined workflows. This being said, individual Q/A installation can, of course, be replaced with a corresponding Agent once a Platform.
Conclusion
We have seen three types of knowledge-based systems:
- Question / Answering Systems that help an individual or team to gather information faster.
- Knowledge Management Systems that provide organizations a single point of access to all their otherwise distributed knowledge.
- AI Agent Platforms as the means to automatize business workflows.
We discussed the biggest challenges in building and operating each tier.
Ending with a look at the current and future applications of AI agents as:
- Assistants that help humans with specific tasks.
- Autonomous Agents that, without human supervision, solve multi-step reasoning exercises.
- Delegates that teams maintain to perform work on their behalf.
That’s all Folks!