Swiss AI Hub

Starting point

Swiss companies that want to use AI seriously keep building the same stack: a vector database, an ingestion pipeline, an LLM gateway, a chat interface. The prototype is quick to build. Running it is the hard part: login across all services, cost per department, protection of personal data, traceability of every answer and upgrading dozens of containers. On top of that, much of the data may not leave the company's own infrastructure, or at least Switzerland. That often rules out cloud platforms from US providers.

The platform

The Swiss AI Hub does not reinvent these building blocks. It wires proven open-source projects into a stack that installs with a single command: LiteLLM as the gateway to local and external models, Milvus and LlamaIndex for RAG, Dagster for the data pipelines, Keycloak for login and tenants, Langfuse and OpenTelemetry for tracing. Agents communicate through a shared event protocol on NATS JetStream and are reachable via API, MCP, Teams and Slack. If needed, everything runs fully on-premises, without data leaving the building.

The platform was developed by bbv's AI team and is open source today.

My contribution

I was dev lead and solution architect of the hub, responsible for deployment and DevOps, and as project lead I ran the client projects built on top of the platform. The now public repository holds more than 100 commits of mine. My focus areas were:

• Deployment and DevOps: I was responsible for how the hub is built, shipped and run: infrastructure as code for Azure, the Docker Compose installation, versioned GitHub releases as an installable bundle, and the CI/CD pipelines that build and tag every image.
• Client projects on the hub: as dev lead and project lead I took client projects from identifying use cases in workshops through proof of concept to the MVP, including architecture and security concepts. What those projects needed fed straight back into the platform.
• Agent runtime: agent types such as the few-shot agent, agent-agnostic management of conversation threads and hand-over to humans (human in the loop), plus guards that check whether the retrieved context is sufficient for an answer.
• RAG pipeline: passing files directly through the API, enriched metadata at ingest, retrievers with runtime metadata filters, and alerting when a Dagster pipeline fails.
• Login and tenants: integrating Keycloak into the platform, a user service based on the Keycloak data, tenant handling at login and integration tests for all of it.
• Evaluation and tracing: the infrastructure for tracing and assessing the agents' answers, first with Phoenix, later with Langfuse.

What I learned

Most of the effort in an AI platform is not in the agents but in everything around them: login, tenants, releases, upgrades and traceability. That work decides whether a company can run a platform itself. And if you combine proven open-source projects instead of rebuilding them, you get their further development for free.