Round Table: Building Browser-Capable Agents with the Browser Automation Tool

Many of the most important tasks we handle still occur within a web browser. This includes things like booking a class, extracting figures from dashboards, completing portal forms, and gathering information across multiple tabs. These are the exact tasks that people often wish an agent could simply manage for them. On 22 July 2026 at 2:30 PM BST (7:00 PM IST), the Microsoft Foundry community will host a 40-minute Discord roundtable discussion. We’ll be diving into the Browser Automation tool, exploring how it supports agents in executing real browser workflows, where you might encounter challenges, and what documentation and product enhancements would help you embrace this technology.

Remember, this is a conversation — not a presentation. Bring your own projects and workflows that you’d like to automate, as well as any safety measures you’d want in place. Join us in the Microsoft Foundry Discord community. Please ensure you arrive on time for a quick technical check.

Opening question to kick off our discussion: “What browser-based task would you love an AI agent to automate for you today?”

Many workflows eventually lead to a website lacking a straightforward API, like a supplier portal, an admin console, or even a legacy dashboard. While traditional scripting can tackle these, issues often arise when selectors break or pages change. Each new site may need another fragile script to maintain. What developers truly seek is an agent that can examine a page, determine what to do, and execute those actions: navigating, reading, clicking, typing, and returning structured results.

This is exactly what the Browser Automation tool in Microsoft Foundry aims to achieve — and doing this responsibly, with appropriate safeguards, is why your feedback is so valuable to us.

Browser Automation Tool (BAT) empowers Foundry agents to control an actual browser for completing web tasks. It’s available as an MCP tool and leverages Playwright Workspaces — a widely available, cloud-based solution that handles navigation, clicking at coordinates, typing, and applying filters on its headless browser infrastructure. Whenever an agent receives a request, Foundry initiates a secure, isolated browser session for that interaction, ensuring that each run remains confidential and separate.

BAT operates on a perception–action loop. The model observes the current page state (including screenshots), evaluates what action to take next, and carries it out in the sandbox using Playwright, keeping a real-time oversight. After each action, BAT captures the updated state and relays it back to the model, continuing this process until the task is completed or the user halts it. Because the model can interpret HTML into a DOM, it can understand the page context instead of sticking to a rigid script. It also accommodates multi-turn conversations, allowing you to tweak a request in real-time for scenarios like form-filling or data scraping.

• Live View: This offers a human oversight option for any unclear or delicate steps, allowing you to watch the automation in real-time for debugging.
• Take Control: Each interaction gets its own sandboxed browser session, enabling a human-in-the-loop for sensitive steps.
• Isolated sessions: Enhance reliability, optimisation, and auditing with unique sandboxed browsers for every interaction.
• Built-in observability: For internal systems (currently in private preview) — ensuring reliability, optimisation, and auditing.
• Private website browsing: Available in various programming languages such as Python, C#, JavaScript, Java, and the REST API, also for internal systems (currently in private preview).
• Wide SDK support: However, be mindful, as the agent can be misled by malicious content on a page.

A brief note on responsible usage. BAT is highly capable; it can access credentials you provide to reach email, financial, or social accounts. However, with such power comes responsibility. It’s crucial that you review your applications, define which credentials to share, and implement necessary precautions. You can find more details in the Foundry Agent Service transparency note. This is the kind of trade-off we want to discuss together.

Consider this scenario: a user requests, “Report the year-to-date percentage change of Microsoft’s stock price.” The agent navigates to a financial website, inputs ‘MSFT’ in the search bar, selects the stock page, clicks the YTD view on the chart, retrieves the value, and returns a clean, structured answer. No bespoke scrapers or hard-coded selectors are involved, along with a complete record of actions taken.

Discussion prompt: “How would you incorporate browser automation into your existing projects or workflows?”

Before you scale, it’s beneficial to grasp the underlying framework. Here are two key elements:

• Create a Playwright Workspace in the Azure portal, enable access token authentication, and obtain the wss:// browser endpoint. Assign a Contributor (or custom) role to your project identity in the workspace.
• Connect the tool in Foundry under Build > Tools: create a toolbox, add Browser Automation, link it to your Playwright workspace and authentication method, and publish. Don’t forget to copy the Project connection ID from the tool’s details — it defines how agents engage with the web, use cases, and responsible practices. This ID corresponds to BROWSER_CONNECTION_ID in your code.

• Welcome & opening question (0:00–0:03): The browser task you wish to automate the most.
• What is Browser Automation? (0:03–0:07): Discuss the workflows you’re developing, differentiating between public and internal targets, and identifying when to favour automation over scripting.
• Scenario walkthrough (0:07–0:12): Explore what agents can do independently, what actions require approval, and what oversight and enterprise protections are necessary. Address your significant adoption barriers, any missing documentation, and prioritise requested SDK samples and demos.
• Use cases & opportunities (0:12–0:22): Engage in live voting on the top use cases, challenges, and feature requests relating to your workflows and automation preferences.
• Trust, security & governance (0:22–0:31): Identify which areas would benefit most from effective agents while looking at required approvals and oversight measures.
• Developer experience feedback (0:31–0:36): Share your key adoption challenges and document needs, as well as which SDK samples and demos are essential for your projects.
• Prioritisation & next steps (0:36–0:40): Take part in live voting on top use cases and challenges while discussing how credentials might be handled.

• Discuss your current browser-based workflows: navigation, data gathering, form filling, and research through an MCP tool utilizing Playwright Workspaces.
• Indicate whether your projects target public websites, internal systems, or both.
• Help us understand why you would prefer browser automation to traditional scripting.
• Clarify which actions you would allow an agent to execute autonomously: assessing perceptions and actions with screenshots and DOM parsing, which can handle disruption in fragile scripts, and which actions should definitely require approval.
• Discuss the observability, audit, and enterprise safeguards you would need before implementing this in a live environment.
• Specify the examples, samples, and tutorials that would facilitate a quick start.

BAT enables agents to perform live actions on active websites through isolated sessions, Live View, Take Control, and built-in observability for reliability and auditing. Due to the tool’s power and its current preview status, be cautious in how you manage credentials and ensure your own safeguards are in place. Governance is a crucial part of our dialogue, not to be overlooked. The isolated per-session sandboxes and human oversight options, like Live View and Take Control, ensure you can monitor, pause, and audit what the agent accomplishes. Bring any trust concerns, necessary guardrails, and governance requirements, as these will directly influence our development roadmap.

Important: The Browser Automation tool is in preview; its APIs and features may evolve, and it is not recommended for use in production environments at this time.

• Browser Automation empowers Foundry agents to fulfil genuine web workflows. This roundtable will feed directly into our engineering and product teams, enhancing navigation, data collection, form filling, and research tasks. Do arrive on time for the tech check.
• Agents reason, rather than merely replaying actions. Our browser-capable agents bridge this gap using a perception–action loop with screenshots and DOM parsing to manage pages that disrupt brittle scripts.
• Oversight is integrated: isolated sessions, Live View, Take Control, alongside observability ensure reliability and compliance.
• Shared responsibility: Handle credentials with caution and deploy necessary protections; this tool is both powerful and still in preview.
• Your feedback will shape our product: This roundtable discussion is instrumental for our engineering and product teams.

1. Add to your calendar: 22 July 2026, 2:30 PM BST / 7:00 PM IST, and arrive punctually for the technical check.
2. Join the community: https://aka.ms/foundry/discord
3. Prepare with the sample: Check out the browser automation sample in the foundry-samples repository.
4. Read the documentation: For more information on automating browser tasks, explore how to automate browser tasks with Foundry agents as well as the hosted-agent quickstart.
5. Event registration link:

   https://discord.gg/Z8JZsrP5P5?event=1527676149264679013

As the final frontier of automation often lies within browsers, browser-capable agents are essential for bridging that gap. We invite you to share your automation ideas, discuss boundaries, and express what you need to feel confident using this technology in production. Looking forward to seeing you on 22 July.