Banking Challenges with AI and Agents

When BNY integrated AI agents on a large scale, they faced a common dilemma faced by banks: how to empower AI systems with enough independence to be effective, while still meeting the strict regulatory controls. This article delves into the architectural choices that banks are implementing today—from runtime hosting options to network isolation techniques—alongside the Microsoft technologies that are driving these production deployments.

The AI and Agent services that banks are currently implementing primarily fall into two categories: 1) Copilot Studio with fresh Copilot enhancements, and 2) Agent Harness or comparable frameworks that utilise Microsoft Foundry model deployments, as well as components from the Foundry platform. Focusing on the latter category, this article showcases numerous success stories highlighting how Microsoft services are being effectively used as the core agent architecture, including the case for BNY. All customers can access key Microsoft services today for building agents, such as GitHub for development, Foundry for deployment and scaling, IQ for grounding, Agent 365 for governance, and M365 or Teams for distribution. The diagram below outlines Microsoft’s enterprise platform designed for agent-based AI.

To unpack the details, Microsoft’s Global Black Belt team likens the agent platform to a suite of tools rather than simply one application: it comprises layers and components that customers can modify and customise for tailored solutions. At its core lies a large-language model, something many organisations are now familiar with and implementing extensively as of 2026. The next crucial element is an agent harness. Microsoft’s definition of a harness is a unified framework that guides an agent’s interaction with prompts, tools, and other services. Ranging from Copilot Studio with an integrated harness to bespoke Foundry solutions based on the Microsoft Agent Framework (MAF) harness, businesses have diverse options for customising and personalising a harness for each agent or multi-agent system. The specific design of this harness will differ based on the use-case, with both built-in and available via the Microsoft Agent Framework for customisable options.  

An agent harness, also known as an AI harness, is the operational framework that transforms a language model into an agent capable of performing tasks.

Agent Harness | Microsoft Learn

For enterprise agents, contemporary banking practices typically include several layers beyond a model: 1) grounding, 2) runtime, 3) orchestration, and 4) security. The agent harness interlinks all these components, specifying the exact processes, connections, and actions the agent can undertake. At present, an agent harness is indispensable for any multi-step task or extended workload, as per industry standards. The specific structuring and review processes for each agent built will differ, shaped by information security requirements, yet the essential components are standard, currently deployable using user-friendly Azure samples and accelerators. 

Copilot-generated image of layers of agent deployment.

One of the first challenges organisations need to tackle is defining what an agent actually is. Any company developing AI systems should establish a clear definition of an agent and determine the degree of autonomy that classifies different types of agents. The primary motivation behind this is to encourage businesses to think critically about whether there is a genuine need for an AI system, as the answer may often be that improvements in existing machine learning systems can suffice. It’s essential to consider whether generative AI and agent autonomy are truly necessary and, if so, to establish success metrics for each use case. Skipping this pivotal step can often lead to costly reworks or failed implementations. Here are some recommendations from Microsoft for organisations starting their AI journey and transitioning to agents: Set up a responsible AI framework, establish a governance review board, and build other necessary organisational structures that may be lacking in many companies. For practical advice on initiating this process with Microsoft Foundry and agents, check out: Microsoft Foundry rollout across my organisation – Microsoft Foundry | Microsoft Learn. 

AI agents have transitioned from basic chatbots to LangChain systems, then to more complex reasoning loops, and recently to advanced systems that require less supervision, such as long-running agentic configurations with recursive learning. This progression is precisely the pathway that banks and regulated organisations should navigate to define their range of potential agents securely, expanding incrementally with Microsoft Foundry.

Copilot-generated image representing the evolution of agents.

The second challenge to consider is how a harness ensures reliability, monitoring, and security. In a rapidly evolving field like large language models (LLMs), there isn’t a universal operating manual or set of criteria comparable to traditional production applications regarding reliability and security. This reality brings unique challenges; as LLMs and agent systems continuously reshape traditional cybersecurity frameworks. Hence, the larger question remains: Can we truly “trust” agents? This question, although approached from a technical standpoint in this article, raises philosophical issues that are gaining attention in the AI community and among policymakers. Various controls exist that a harness can deploy to enhance trust, but we must also remember that the foundational elements of these systems involve stochastic processes, which can lead to random and unexpected outputs. 

Next, consider the third challenge: how does an agent operate within my production environment on a grand scale? Regulated entities like banks can successfully create a proof-of-concept (POC), but scaling for full production poses significant challenges. Instead of just a handful of internal users interacting with the agent, you’ll now have hundreds, which may test your application’s capacity and raise security inquiries regarding rogue agents or possible prompt injection attacks. Moreover, you may encounter various risks, each with its degree of severity. If you’re not familiar with cybersecurity or strategic planning, the importance of ongoing evaluation, monitoring, and updates might not be clear. In today’s AI landscape, responses to incidents may not happen weekly or even daily, but potentially hourly, indicating a constant battle for resolution. It’s more of a continual journey than a challenge – but it’s not insurmountable. 

While these challenges aren’t exhaustive, they represent the core obstacles enterprises encounter today when working with agents. Fortunately, there are proactive solutions, workarounds, and recommended architectures developed by Microsoft that you can leverage right now, with continual updates anticipated. To better understand these options, let’s assess the technical decisions identified by banks, along with lessons learned applicable to various contexts.

The first crucial decision that influences many subsequent choices is where your agent operates. This refers to the execution component discussed earlier, and it’s common to use “host” and “run” interchangeably to describe the complexity of infrastructure required for an agent system. In the AI industry, a host is defined as the computing resource managing and granting access to the agent’s workloads. Hosting options could include virtual machines from cloud service providers under Infrastructure as a Service (IaaS) or a more managed setup like Microsoft Foundry, which operates akin to platform as a service (PaaS).  

Moreover, with Microsoft, agents typically have several hosting options, including Azure Virtual Machines, Azure Container Apps, Azure Kubernetes Service, Azure Functions, and Foundry Hosted Agents. The primary factors influencing the choice between these options will revolve around the level of control you, as the customer, wish to maintain against the complexity associated with managing that control. At the most complex level, hosting directly on a virtual machine entails building everything apart from the machine itself, such as images and containers, which entails significant operational overhead and self-management. In contrast, the least complex solution with the lowest amount of control would be Foundry agents. The no-code version includes Copilot agents or already defined and deployable agents, though this article won’t detail those. More details can be found here: Copilot and AI Agents | Microsoft Copilot. If you’re unsure how to choose your hosting option, consult this main reference in Azure documentation to assist you in this decision: Choose an Azure Compute Service – Azure Architecture Center. For integrating Foundry capabilities and running a defined agent, the suggested course is Foundry Hosted Agents. 

Following the choice of hosting option, two important additional decisions arise: the environment and network isolation. These elements correspond to the environmental component illustrated in the earlier diagram. For banks and other regulated entities, establishing environment isolation involves understanding and identifying security risks associated with AI before implementing any degree of autonomy or agent capability. Typically, this encompasses network isolation, private networking, alongside inspection and logging as mandated by security protocols and information security assessments. Practically, this manifests as guardrails, safety protocols, and access restrictions. For instance, an agent should only be accessible by verified users, should only allow approved prompts for business inquiries, and any communicated data or uploads should be traceable for audits or reviews. These decisions are primarily shaped by existing security infrastructures in banks, such as firewall protocols or validation processes for models. 

Given that AI and agents present novel risks, extensive customisation and tailored support are frequently required to harmonise an agent environment with existing security protocols. These adjustments cannot be rapidly implemented today, as identified challenges often hinder scaling and realising value from agent systems across various sectors. However, options are available today with numerous features recently released or expected in future updates. Here’s a summary of key services provided by Microsoft Foundry for environment isolation and controls, often integrated with other Microsoft security solutions. 

Despite the current challenges facing AI and agents in banks, it’s unlikely that progress will halt. There are already tangible benefits from using agents for enterprise applications, ranging from enhancing employee productivity to automating software development, indicating plenty of potential for further growth and scaling. Microsoft consistently updates resources concerning AI and agents, with significant announcements typically occurring during the annual conferences like Microsoft Ignite in the winter and Microsoft Build in the summer. While this article doesn’t detail every anticipated feature and service to tackle the aforementioned challenges, it highlights essential areas to monitor as Microsoft relentlessly advances in addressing concerns relevant to agents in the banking sector.