The Economics of Agent Optimization: Context engineering for enterprise AI agents

This article is the third instalment in the four-part series titled The Economics of Agent Optimization. Here, we explore strategies, capabilities, and evidence to assist you in reducing agent costs and managing AI effectively within the Microsoft Foundry framework. The first post identified three crucial decisions that systems rely on. The second post discussed requests made at runtime. In this piece, we focus on the next challenge: reducing the cost of each agent over time as it learns what works best.

Each agent operates on a mechanism that dictates what it can perceive during each interaction. In many production environments, these settings are established during the initial design and are seldom reassessed, even though they often account for a significant portion of ongoing costs and can lead to poor responses.

This component is also where agents can learn and improve independently. While the model’s capabilities remain constant unless revised, the knowledge an agent can access, recall, and utilise expands as it operates. This makes it a vital area for enhancing performance while simultaneously reducing costs over time. The management of this process is referred to as context engineering.

How the context window influences agent costs

A model lacks its own memory. Each time it engages, the context window provides all the resources it can draw from: instructions, available tools, retrieved documents, and the history of the conversation. When the interaction concludes, this context is lost and must be sent again in the next round.

This expense is quite manageable for a chatbot fielding one question. However, for agents that operate across multiple interactions aiming for a single outcome, this can result in substantial costs. Because the context window incurs charges with every interaction, repeating unnecessary content leads to increased expenses.

The less obvious cost concern is quality. Simply having more context does not assure better responses; for example, a significant detail buried among 40 pages can be challenging to utilise. Additionally, a lengthy list of tools increases the likelihood of selecting the wrong option. Each error adds more interactions—and thus costs—to rectify.

Consequently, context management deserves careful consideration. Many cost-cutting measures involve compromises: a budget-friendly model might offer lower quality, and simplified instructions may undermine responses. Conversely, minimising unnecessary context can reduce costs without sacrificing quality, making it a more straightforward enhancement for teams to adopt.

Practical applications of context engineering

Context engineering focuses on determining what content enters the context window during each interaction, ensuring that the agent receives only what is pertinent to the current request rather than everything it may ever need. This is both a one-time design decision and an ongoing practice of enhancement, as every interaction reveals the elements that are genuinely utilised. Four key questions guide this process, and teams typically address them in this order.

What knowledge should the agent possess?

Many teams start with broad queries that load entire documents into the prompt. Although this method is simple to implement, it can be costly, forcing the model to sift through irrelevant data to find the key information.

Foundry IQ addresses this by providing a managed knowledge layer. This knowledge base connects to sources across Work IQ, Fabric IQ, Web IQ, Microsoft Azure Blob Storage, SharePoint, OneLake, and Azure SQL. Whenever an agent submits a query, Foundry IQ breaks it down into subqueries, searches through connected sources concurrently, semantically re-ranks the outcomes, and returns corroborated passages along with citations. This refinement ensures that the context window is populated with only the most relevant information while maintaining traceability to the source.

Two aspects make this knowledge layer applicable across various agents and manageable on a large scale. A singular knowledge base can serve multiple agents, with indexed sources updating incrementally based on a set indexing schedule, while remote sources are queried as required. When a query is initiated, Foundry IQ utilises the caller’s Microsoft Entra identity to synchronise access-control lists for supported sources and comply with Microsoft Purview sensitivity labels, ensuring that the agent retrieves only the content the caller can access.

Our internal evaluations revealed that Foundry IQ knowledge bases boosted evidence recall by up to 54% on the BrowseComp-Plus benchmark while cutting retrieval token costs by 34%. These improvements resulted from agent-level retrieval, semantic re-ranking, enhanced answer synthesis, and more efficient token utilisation.

What tools should the agent access?

The additional costs associated with tools are often overlooked; while integrating a new tool might require just a line of code, its full description inflates the prompt size. Every tool attached to an agent sends its description to the model at each interaction, whether it’s needed or not, and enterprise agents tend to accumulate tools quickly as they connect to various systems.

Toolboxes in Foundry provide an agent with one managed Model Context Protocol (MCP) endpoint that includes built-in tools like web search, code interpreter, and file search, alongside custom MCP servers, OpenAPI 3.0 and 3.1 APIs, and A2A agents. Foundry manages authentication, access policies, and tool versions in one centralised location, eliminating the need to configure each integration separately for every agent. Once a new toolbox version is tested and approved, connected agents can access it without requiring code modifications or redeployment.

Toolboxes organise your tools effectively. The tool search feature within the Toolbox is instrumental in preventing excess costs from all tools. Instead of presenting the entire list, the model only receives two things: a method to articulate what it needs in simple terms and a means to call upon what’s returned. The expense related to the tool list remains constant, regardless of how extensive the toolbox becomes. In internal tests with a public, open-source tool-retrieval dataset, Toolboxes in Foundry reduced average input-token consumption by approximately 97% for extensive tool libraries—significantly lowering inference costs for clients developing agents.¹

Foundry also monitors the tools utilised by each toolbox, allowing it to highlight the most frequently used ones, thus making common tasks quicker and more cost-effective as the agent continues to operate. Accuracy improves alongside cost, as a concise, well-defined tool list reduces incorrect calls and the number of turns spent rectifying mistakes.

How should the agent approach tasks?

While knowledge and tools determine what an agent can discover and execute, they don’t specify how your organisation wishes tasks to be performed—such as the escalation process a support agent adheres to or the checklist applied during a code review. This guidance is typically housed in the agent’s instructions. Consequently, identical procedures may be repeated across various agents, leading to unnecessary repetition in every request, even when irrelevant.

A skill transforms that guidance into a defined, reusable process. Skills are centrally stored within Foundry and made accessible to agents through a toolbox. Instead of embedding a version of the procedure within every agent, the toolbox references the centrally managed skill. When your organisation updates a procedure, a new version can be released and set as the default, allowing all agents utilising that skill to follow the new procedure without needing code alterations or redeployment. To optimize context usage, the agent initially views only each skill’s name and brief description, loading the complete instructions only when the skill is relevant. This system allows for a broad library of detailed procedures without burdening every interaction with superfluous information.

What should an agent retain?

Agents require some continuity, but burdening them with every detail from past interactions is unnecessary. Continuously sending an entire conversation back to the model adds to costs and depletes context, even when few details are genuinely useful.

Memory in Foundry Agent Service allows agents to retain significant context without replaying entire dialogues. It supports three distinct types of memory:

• Session memory for the duration of the current conversation.
• User memory for preferences and information that persist across different sessions.
• Procedural memory that captures learned workflows and task execution patterns.

This functionality enables a returning customer to resume their previous conversation and allows an agent to reliably follow established processes without the need for re-instruction every time.

Collectively, these features empower an agent to continue customer interactions, tailor future responses, and improve the consistency with which it completes routine tasks. Procedural memory supplements centrally managed skills: a skill outlines the organisation’s approved processes, while procedural memory assists an agent in learning from its individual task performance. In Microsoft’s evaluations, activating procedural memory resulted in approximately a 5% enhancement on STATE-Bench and Tau-Bench tests. Organisations can also oversee memory through user-level isolation, retention settings, and time-to-live policies that determine what is stored and when to remove it.

Why context engineering is essential for system efficiency

Any team can put together knowledge retrieval, tools, procedural guidance, and memory systems. The challenge lies in ensuring these elements work harmoniously, under a unified permission structure, and remain updated as the organisation evolves.

Foundry integrates these components into a cohesive system. Knowledge, tools, skills, and memory can be managed via a shared infrastructure instead of separate products, with permissions enforced where data is accessed so that agents inherit the existing access controls applied to enterprise content. Foundry IQ expands this model across enterprise knowledge, business data, and organisational context, while also being compatible with frameworks such as Microsoft Agent Framework, LangGraph, GitHub Copilot SDK, and Claude Agent SDK.

The outcome is that context continuously improves without necessitating a rebuild of agents. Knowledge bases refresh as source systems evolve. Skills adapt in accordance with changing policies. Memory grows to include what’s essential about users and successful workflows. Tool search adjusts to reflect capabilities that people genuinely utilise. The Agent optimizer in Foundry Agent Service completes the feedback loop by analysing agent behaviours and generating enhanced instructions, skills, tool descriptions, and model configurations.

The ultimate objective of context engineering is not just to minimise prompt sizes or costs related to retrieval, but to develop agents that grow more effective with repeated use. As the knowledge they leverage, the tools they uncover, the procedures they adhere to, and the memories they retain improve over time, agents can become increasingly proficient and efficient without requiring a complete overhaul.

How to start

If you’re in the process of developing agents, begin by assessing what enters the context window during each interaction. Examine the documents being retrieved, the tools being revealed, the instructions being repeated, and the conversation history being carried forward. Often, enhancing these inputs can have a more significant impact on costs and quality than merely changing the models used.

Have you caught up on the previous articles in the The Economics of Agent Optimization series?

¹ Command Line, Tool search: Finding the right tool at the right time, July 29, 2026.