All Azure Technologies @ one Place
Microsoft’s Citadel architecture is not a standalone product; rather, it comprises essential decisions made in advance for managing AI systems effectively across teams.
Governance is straightforward when only one AI assistant exists within a company; built and maintained by a single team, everything is neatly contained with one Azure OpenAI account and a single key.
The complexity arises with the introduction of a second team asking, “Can we use the same configuration for a finance assistant?” Often, no one can clearly address the subsequent queries:
- Is sharing the HR team’s Foundry resource possible, or does a new one need to be created?
- Who decides which model they can use?
- Who covers the cost for their tokens?
- Who gets notified if their assistant malfunctions at 2 a.m.?
- Can they access the HR team’s search index? (The answer is no, but what’s stopping them?)
These questions exceed technical boundaries. Azure possesses the necessary capabilities; however, what is lacking is a mutually agreed framework demonstrating how these features are implemented across various teams. This is precisely the role of Microsoft’s Citadel reference architecture.
This article expands upon the previous discussion regarding AI landing zones and the AI gateway. To summarise: an AI landing zone categorises applications, shared gateways, and model services into distinct zones, each overseen by one owner; the gateway acts as the API Management layer in front of the models.
Understanding Citadel: What It Is and Isn’t
Citadel serves as a prescriptive reference architecture found in the foundry-citadel-platform sample. It is essential to clarify three points that are often overlooked.
It is neither a product nor a runtime. ‘Citadel’ does not process requests; all the services executing tasks are those already in place: API Management, Microsoft Foundry, Entra ID, Defender, Purview, and Azure Monitor.
It does not replace a landing zone. It depends on one existing for support. The landing zone provides the foundational deployment, Foundry supplies the inherent AI capabilities, and Citadel establishes standardised methods for teams to utilise both.
Its adoption is incremental. Microsoft advocates starting with a single workload and addressing one operational gap, validating the approach, documenting the onboarding process, and then proceeding to the next; rebuilding everything simultaneously is not feasible.
Four Key Governance Areas
Citadel identifies four areas of governance, which function concurrently rather than sequentially.
| Concern | Native Services | Standardised Decision by Citadel |
|---|---|---|
| Governance Hub | API Management as the AI gateway, API Center as the catalogue | Where access is managed, which models and tools are approved, how team consumption is tracked |
| AI Control Plane | Foundry evaluations, tracing, red teaming, Azure Monitor | How diagnostics from each application aggregate with cross-team views on usage, quality, and costs |
| Agent Identity | Entra ID, managed identities, Agent 365 | Every agent has an identity, sponsorship, and a lifecycle, similar to a service account |
| Security Fabric | Defender, Purview, Content Safety, Entra | Where protections apply, who manages exceptions, who holds the evidence |
Typically, organisations lack a governance hub, which was the main focus of the previous article. The other three components generally exist in some format; Citadel’s value lies in integrating them with the hub and assigning responsibilities.
Five Components of Workloads with Standardised Decisions
For each component of an AI application, Citadel adopts a uniform approach: Microsoft delivers the capability, while the platform defines usage standards. This is encapsulated in the following table.
| Component | Native Capability | Standardised Usage | Example |
|---|---|---|---|
| Foundry Resources and Projects | A Foundry resource defines management scope (network, security, models); projects delineate team boundaries within it | Which teams share which resources and which require their own | HR and Finance operate in separate projects within a shared Foundry resource. A team needing different network regulations may have its own resource, as projects do not define network-isolation limits |
| Data, Retrieval, Grounding | AI Search, Storage, Cosmos DB, embedding models | Approved data sources, chunking, re-indexing, permission management | The HR assistant retrieves only policy excerpts visible to the requesting employee. Indexing a document does not automatically copy its permissions |
| Models and Runtime | Foundry catalogue, deployment methods, quotas | Approved models, processing locations, access permissions, upgrade procedures | Before transitioning to a new model version, assess the evaluation set and compare metrics like quality, expense, and latency |
| Agents and Orchestration | Foundry Agent Service, tools, MCP | Approved tools, user permissions, authorisation for significant actions | The HR agent can view a leave balance, but submitting leave requests has a distinct approval pathway |
| Security and Responsible AI | Entra, Key Vault, private networking, Content Safety, Monitor | Application of controls, management of exceptions and incidents | Content filters mitigate risk; they do not ensure accuracy. Deploying Defender alone does not imply compliance |
One insight from Microsoft’s guidance is crucial for conversations regarding data residency: the positional location of a Foundry resource does not dictate where inference processing occurs. This is determined by the deployment type itself. A global deployment hosted in your own region remains classified as a global deployment.
Ownership Clarity
Citadel provides a straightforward division of roles and responsibilities, and much of its benefit stems from documenting this separation.
To effectively evaluate this division, consider if onboarding the Finance team using identical steps to those used for the HR assistant is feasible without granting access to any HR data. If it is, you possess a platform. If onboarding requires duplicating the resource group from HR and merely changing names, it is considered a template, not a true platform.
A Day in the Life of a Request
When an employee approaches the HR assistant with the query, “Could you explain my leave options?” here’s how the request navigates through a Citadel environment, highlighting access checks at critical junctions.
Observe that the gateway does not query the search index or HR database directly; those interactions are handled by the application using its identity, refined to match the employee’s permissions. The gateway governs calls to the model, ensuring access reviews occur at every step, rather than a singular check at the entry point.
Steps for Adoption, Especially During Migration
Microsoft delineates a four-step process for implementing this architecture, designed to be manageable.
For organisations currently undergoing a regional migration, queries about the timing of Citadel adoption frequently arise.
Prioritise migration, then enhancements. Implementing a regional shift alongside migrating platforms complicates issue identification when problems emerge. Transition the estate as is, addressing only free improvements during the rebuild (like enabling private endpoints, relocating secrets to Key Vaults, converting to managed identities where applicable), and plan the shift in a way that will not hinder future platform migration.
Three strategic choices during the migration can make the eventual gateway integration nearly seamless:
- Maintain consistent naming for deployments across environments and across the OpenAI resources you create. This allows for backend changes without the applications realising different setups.
- Route model traffic through configuration settings, rather than hardcoded values. Use endpoints in application settings or Key Vault references.
- If API Management is already implemented in front of the applications, remember that relocating the OpenAI endpoints behind it is merely a configuration update for the application team: just a new URL and a new key, which are already requirements of the migration. This offers an economical initial implementation of a Citadel pattern.
Key Takeaways
- Citadel is a framework of decisions, not a standalone deployment. The necessary services are those already in use.
- The focus is on sharing resources and ownership: identifying shareable resources, specifying operational controls, monitoring usage attribution, and determining alert responsibilities.
- The governance hub (the AI gateway) is often the critical missing element; the other three governance areas typically have some existing structure that needs connecting and ownership clarified.
- Project boundaries do not equate to network boundaries, and location of accounts does not determine inference locations. These clarifications will simplify future compliance discussions.
- Implement it one workload and one issue at a time. If you are in the midst of migration, complete the move before introducing changes, ensuring it does not limit future enhancements.
Share this content:
Discover more from Qureshi
Subscribe to get the latest posts sent to your email.
