Mastering Microsoft Foundry Agents: From Development to Deployment

Welcome to the third installment of our series on the Microsoft agent platform! In this post, we’re diving into the Run in Foundry aspect, where we discuss taking your agents from development right through to production. We’ll explore features like Hosted Agents, the Agent Optimizer, Routines, Memory, Toolboxes, and Tracing. Let’s get started!

What Are Hosted Agents?

Hosted Agents serve as the fundamental deployment component within the Microsoft Foundry Agent Service. Essentially, you’ll create a container for your agent, ensuring it meets the OpenAI /responses contract, defining a manifest, and then deploying it. Foundry takes care of scaling, networking, and managing the lifecycle of your agents.

For example, FibreOps encapsulates its entire agent pipeline as a single hosted agent, characterised by four key components:

• ArtefactPurpose: src/fibreops/agents/hosted_app.py – This wraps the agent within the ResponsesHostServer, handling both /responses and /readiness on port 8088.
• Dockerfile: src/fibreops/agents/Dockerfile.hosted – This defines a Linux container image that executes the entry point.
• Agent.yaml: A manifest that specifies vital information like kind, image, CPU/memory requirements, protocol versions, and environment variables.
• Deployment Script: src/fibreops/agents/deploy.py – This builds a HostedAgentDefinition and invokes agents.create_version, polling until the agent is active.

The manifest lets Foundry Agent Service know about the hosted agent:

# agent.yaml
kind: hosted
api_version: V1Preview
name: fibreops-outage-response
image: .azurecr.io/fibreops-outage-response:v1
protocol_versions:
  - "2024-12-01-preview"
sandbox:
  cpu: "1"
  memory: "2Gi"
environment_variables:
  MODEL_DEPLOYMENT_NAME: gpt-4.1-mini
  FIBREOPS_AGENT_BACKEND: hosted
        

Note that Foundry automatically injects FOUNDRY_PROJECT_ENDPOINT, so you’re never storing credentials directly in the manifest.

Deployment Workflow Simplified

We leverage Azure Container Registry for deployment, which means you don’t need local Docker for the build. Here’s a quick overview of the steps:

1. Set Up Permissions: First, grant the Foundry project managed identity the role for AcrPull using the provided script.
2. Build and Deploy: Run the deployment script to build, push, and deploy your hosted agent.

You can also run these commands programmatically for efficiency:

$env:FIBREOPS_HOSTED_IMAGE = ".azurecr.io/fibreops-outage-response:v1"
python -m fibreops.demo deploy-hosted
        

Before deploying, validate locally by executing the model request with:

python -m fibreops.demo serve-hosted  # Visit http://localhost:8088/responses

Understanding the Roles

Introducing the Agent Optimizer

The Agent Optimizer is a fantastic tool that evaluates each agent run against a detailed rubric, providing you with valuable improvement suggestions. Here’s how it works:

1. Trace Capturing: Every decision, tool call, and output is logged as an OpenTelemetry span.
2. Evaluation: Each run is assessed based on criteria such as classification accuracy, dispatch appropriateness, SLA compliance, and communication quality.
3. Suggestions Generation: The optimizer analyses patterns across runs to offer specific, actionable improvement tips.
4. Integration with Foundry: By setting FIBREOPS_FOUNDRY_EVALS=1, cloud evaluators can enhance scoring dimensions further.

To manually run the optimizer, use:

python -m fibreops.demo run  # Execute some signals first

The optimizer also runs automatically after each batch or via the NOC console UI by clicking “Run optimiser”.

Defining Routines for Consistency

Routines help ensure your agents execute actions in a predictable manner, especially when dealing with well-defined workflows. Here’s a straightforward example of how to set a routine:

# src/fibreops/agents/routines.py — simplified
NETOPS_ROUTINE_DEFINITION = {
    "steps": [
        {"action": "file_ticket", "tool": "create_incident"},
        {"action": "post_teams_notice", "tool": "post_outage_notice"},
        {"action": "remember_ticket", "tool": "store_memory"},
    ],
    "decision": {
        "expression": "severity in ('critical', 'major')",
        "true_branch": "HANDOFF:DISPATCH",
        "false_branch": "MONITOR",
    }
}
        

Activating Routine Path

To enable the routine path, you can set the corresponding environment variable:

$env:FIBREOPS_NETOPS_ROUTINE = "1"
python -m fibreops.demo   # Look for "routine" mode in the netops pill

Routines can now also respond to events automatically, ensuring a more reactive approach to automation.

Understanding Agent Memory Types

Foundry offers three types of memory that your agents can use:

• Procedural Memory: This type stores lessons from past incidents to enhance future decision-making.
• User Memory: It keeps track of individual user preferences and context across sessions.
• Session Memory: This memory pertains to the context of a single conversation or run.

FibreOps leverages procedural memory to refine dispatch choices over time:

def store_memory(incident_id: str, lesson: str) -> dict:
    """Store a lesson learned for future reference."""
    ...
        

To link to the Foundry Memory provider, set FOUNDRY_MEMORY_STORE_NAME; otherwise, it defaults to using a local SQLite database.

Utilising Toolboxes for Enhanced Functionality

Toolboxes are ready-to-use collections of tools managed by Foundry, enabling agents to perform tasks without custom coding. You can enable FibreOps to utilise Foundry Toolbox tools simply:

$env:FIBREOPS_FOUNDRY_TOOLBOX = "1"
python -m fibreops.demo   # Agents gain access to web_search and other Toolbox tools

Tracing for Insightful Monitoring

Every interaction with an agent results in OpenTelemetry spans being generated. FibreOps provides two methods of output:

• Local: JSON spans are saved to state/traces.jsonl for offline examination.
• Application Insights: Set APPLICATIONINSIGHTS_CONNECTION_STRING to send spans to Azure Monitor.

We’ve also included ready-to-use KQL queries for analysing typical operational questions.

Provision All Required Components with Ease

The repository comes pre-packed with an azd template that sets up everything you need:

• Azure App Service for Linux Containers (NOC console)
• Azure Container Registry (to manage agent images)
• Azure Event Hub (for telemetry ingestion)
• Azure Key Vault (for secrets management)
• Log Analytics + Application Insights (for observability)

To get started, follow these straightforward steps:

azd auth login
azd env new fibreops-demo
azd env set AZURE_AI_PROJECT_ENDPOINT "https://.services.ai.azure.com/api/projects/"
azd env set AZURE_AI_MODEL_DEPLOYMENT "gpt-4.1-mini"
azd env set AZURE_LOCATION "swedencentral"
azd up
        

If you wish to deploy the hosted agent during the azd up process, simply run:

azd env set FIBREOPS_DEPLOY_HOSTED true
azd up   # Sets up infrastructure, deploys NOC console, then registers hosted agent

Summary

So there you have it! Hosted Agents make it easy to deploy containers that fulfil the /responses contract, while Foundry manages all the associated complexities for you. The Agent Optimizer ensures your agents continuously improve through robust evaluation and insightful suggestions. With Routines, you can guarantee consistent execution for your workflows, enhanced further by event-trigger support. Memory systems help your agents learn and evolve without needing constant retraining. And lastly, Toolboxes allow you to extend capabilities without requiring extensive custom integration.

FAQs

What exactly is a Hosted Agent?

A Hosted Agent is a packaged container that deploys agents on the Microsoft Foundry platform, handling networking and scaling automatically.

How does the Agent Optimizer work?

The Agent Optimizer evaluates agent performance against a filled rubric and provides actionable suggestions for improvement based on the analysed data.

What types of memory can Foundry agents use?

Foundry agents can utilize procedural memory, user memory, and session memory to enhance functionality and learning.

How can I track agent performance?

By using tracing capabilities available in OpenTelemetry spans, which can be sent to Application Insights for monitoring and analysis.