Azure Cosmos DB Conference 2025

At the Azure Cosmos DB Conference 2025, organised by Microsoft, I had the opportunity to share some key insights from our work with AI agents in a short lightning talk. In just five minutes I showed how we built a system that allows multiple specialised AI agents to work together efficiently, with the help of Azure Cosmos DB.

The Idea: Not One Large Agent but Many Small Ones

For over two years we have been developing at bbv Software Services AG in Zurich a system based on the collaboration of multiple specialised AI agents. The central idea: instead of delegating all tasks to a single generalist model, we distribute them across specialised agents, each responsible for a specific sub-aspect.

"An agent with specific knowledge is easier to steer, drifts less from context, and is therefore considerably more reliable."

This division makes the overall system more robust and also allows complex tasks to be handled more efficiently. However, coordinating multiple agents also brings new challenges.

Communication Between Agents: More Than Just Text

At first it seemed natural to handle communication between agents simply via text, since language models are trained on it. But it quickly became clear that much more structure is needed.

"An agent needs to know what the other agents can do, only then can it interact meaningfully."

To enable this interaction we drew inspiration, among other things, from the Model Context Protocol (MCP) proposed by Anthropic. This protocol distinguishes between clients (e.g. LLMs) and servers that provide certain information or execute actions. These servers expose so-called tools: clearly described functions with defined inputs and expected outputs.

Agents as Clients and Servers

What does this have to do with agents? Simply put: an agent can act in the role of both client and server. An example from our system: an agent is tasked with providing a weather report. Instead of possessing the knowledge itself, it can call on another agent that offers a "weather API".

"By publishing tool definitions, agents become mutually accessible: they learn to build on each other."

This modularisation makes it possible to build a network of agents that can be extended flexibly. Not every task can be solved with a single API call, though; more complex tasks require coordination and progress tracking.

Structuring Tasks: Runs, Threads, and States

To structure complex workflows we use concepts such as Runs and Threads, as described in LangChain. A Thread bundles all steps of a larger undertaking (e.g. booking a holiday), while a Run represents a single action within that context (e.g. booking a flight).

"Complex tasks require an internal state model: that was a central aspect of our agent orchestration."

This is where Azure Cosmos DB comes in: we use the database to reliably store the state of these workflows. Both Runs and Threads are stored in Cosmos DB, as are the tool definitions of our agents.

Why Cosmos DB?

Cosmos DB gives us exactly the flexibility and speed our system requires: a schema-less data model, consistent latency, and global availability.

"The flexible structure and real-time synchronisation of Cosmos DB allow us to keep agent states current worldwide."

Particularly in today's dynamic AI landscape it is essential to be able to respond quickly to new developments. Cosmos DB supports us in testing new protocols, changing tool definitions, or improving the internal logic of agents, without costly migrations.

Agents Beyond Company Boundaries

To close my talk I broadened the perspective. Most companies are currently building agents only for internal use. But I am convinced that the real strength lies in collaboration across company boundaries.

"I envision a world in which agents are rented out, integrated via APIs, negotiate contracts, and jointly implement projects."

This requires common protocols and systems ready to connect with the outside world. That is why we closely follow the development of protocols like the Model Context Protocol. With the recent announcement by OpenAI to support this protocol in future, we see a clear direction.

"We continuously adapt our system to new standards, with Cosmos DB as the central platform for agents, tools, and states."

This talk was an opportunity for me to share our experience and perhaps inspire other developers to think about their agent systems more in terms of modularity and collaboration.