Reproducing an AI Action Plan Analysis with the Microsoft Discovery App

Just over a year ago, I shared a three-part series on this blog about harnessing generative AI to make sense of around 10,000 public responses (10,067 to be precise) to the OSTP/NITRD Request for Information regarding the AI Action Plan:

1. Listening at Scale: Using Generative AI to Understand 10,000 Voices
2. From Individual Voices to Collective Insight
3. Creating an AI Policy Analysis Copilot

The aim of that series was not to create a formal report. Instead, it was to empower analysts to engage with the data — to inquire, identify trends, and draw insights from a vast pool of voices that would be impractical to read individually, let alone remember simultaneously. It took about a day’s worth of AI support: fine-tuning prompts, conducting a model bake-off, establishing cloud infrastructure in Azure, configuring research tools like knowledge-graph packages (GraphRAG), developing a metadata transformation layer, and launching an MCP server to enable communication with a custom Copilot. It was a complex task with numerous technologies involved, and — importantly — a lot of my efforts to see it through from start to finish.

Fast forward a year, and the Microsoft Discovery app is now in preview. I’ve been experimenting with it to grasp its capabilities, and I was curious to see how it would handle the same data. This wasn’t an analytical project — it was more of a capability test. I wanted to know how far I could stretch it, how much I could delegate, and what results it could generate independently.

I uploaded the data, provided a simple, open-ended intent, activated Autopilot, and went out to see a film. Three hours later, I returned to a detailed 43-page research report.

This blog post serves as a “one year later” reflection on what I still performed, what the platform can now handle for me, and what this evolution indicates about the future of AI-driven discovery.

Case study, pre-released product, and AI disclaimers:

This document is for informational purposes only. Some details discuss a product still in preview, which may change significantly before becoming widely available. These outcomes are based on a specific instance. Generative AI systems are probabilistic, meaning repeating the process with identical prompts might yield different results.MICROSOFT PROVIDES NO WARRANTIES, EXPRESS OR IMPLIED, REGARDING THE INFORMATION CONTAINED HERE.

Microsoft Discovery is an advanced platform from Microsoft that combines orchestration, sophisticated reasoning, a graph-based knowledge base, and high-performance computing for research and development tasks. It comes in two complementary versions: Microsoft Discovery, an enterprise cloud solution running on Azure, which includes full governance, multi-user collaboration, and scalable computing; and the Microsoft Discovery app, a free download, local-first Windows client in preview that runs on your machine with just a GitHub Copilot subscription. Both operate on the same foundational elements:

• Discovery Engine for orchestrating autonomous, multi-step, long-running sessions to achieve desired results.
• Bookshelf for platform-native, graph-based document reasoning capabilities.
• An intelligent AI platform equipped with a variety of tools and agents right out of the box.

Projects developed in the app can easily transfer to the enterprise service if you require larger scale, private data handling, team collaboration, or compliance.

This post focuses on the Discovery app.

Here’s a quick overview of my actions:

I created a new Discovery app workspace called “aip-for-fun.” Ingesting documents was straightforward — I simply selected a source — files, a folder, a URL, or an open workspace — from the app’s Ingest Documents dialog:

I uploaded the AI Action Plan RFI into one bookshelf and the 10,067 public responses into another. Keeping them separate seemed sensible; the RFI represents the question, while the responses are the answers. Indexing time? Just about 15 minutes.

For comparison, in 2025, even opening those PDFs and preparing them for GraphRAG indexing took hours of engineering effort — creating an Azure Function to convert PDFs to Markdown, another to handle new files, all connected through Blob Storage. The Discovery app streamlines this process to simply directing it to a folder and letting it do its job.

I opted for the built-in project-setup-orchestrator agent in Discovery Chat and provided a single command:

“I want a report analysing the datasets in the bookshelves.”

That’s all. A deliberately vague prompt — I wanted to examine what Discovery could deduce independently.

The orchestrator took off: it examined the workspace, identified the bookshelves, posed a couple of clarification questions, and drafted a Purpose statement for me to approve.

From that single sentence and by examining the bookshelves, it produced:

A Purpose statement:

“Systematically analyse the U.S. public’s response to the OSTP/NITRD Request for Information on the AI Action Plan (EO 14179, Jan 2025), using a corpus of ~10,067 submissions to define who responded, what they requested, and what those signals indicate for AI policy.”

Six suggested Outcomes with grading rubrics. I reviewed them, approved, and added four more to explore particular aspects in greater detail. Discovery crafted the rubrics for those additional outcomes too — including a comprehensive rubric for the international competitiveness outcome based on a 0–4 Likert scale with four weighted questions. Discovery handled the drafting, while I focused on reviewing and validating.

Based on that Purpose and those Outcomes, Discovery generated a detailed task graph — 27 tasks across 5 milestones, from source acquisition through respondent analysis, policy demand mapping, sentiment evaluation, specialized domains, and synthesis.

I switched the chat function to Autopilot and entered:

“Complete DX-1.”

(A quick note: Every task in the graph gets a unique DX-N ID. DX-1 always represents the top-level task — finishing it means all underlying tasks are complete. You can see this numbering in the task tree screenshot below.)

Once the project was set, this was practically a one-shot prompt. I stepped away to enjoy a film. Three hours later, I returned to find a task tree where every node was green:

The task tree now shows six milestones and 32 tasks, a growth from the initial five and twenty-seven tasked by Discovery. This happened because after Milestone 5 yielded the first draft of the report, Discovery evaluated it against the ten outcomes and identified areas for improvement — autonomously adding a sixth milestone with five additional tasks to enhance depth, recreate charts, embed local images, and restructure the PDF. The autopilot run didn’t just carry out the plan; it iteratively improved it based on the quality of the work.

Behind DX-1 was a comprehensive 43-page PDF report with a branded front page, table of contents, nine narrative sections, three appendices, a bibliography, 18 embedded charts, and rubric ratings for each of the ten outcomes.

For context, throughout the entire project — including setup, the autopilot run, and later refinements — there were 34 user prompts, 225 assistant actions, and 386 tool calls. However, “34 user prompts” doesn’t tell the full story: I personally entered maybe four or five of those. The majority were initiated by Discovery’s own agents collaborating internally as they navigated through the task graph autonomously. (For full transparency: I did not track token counts through this run — capturing that end-to-end is something I’m still working on.)

The metrics I’ve shared reveal how the report came to be. What the report actually is remains equally fascinating. Here it is.

The report begins with a table of contents, an executive summary, and actionable conclusions — even before delving deeper into the content. This was the finishing touch Discovery added in Milestone 6: not just filling the report with information, but providing structural elements that an analyst would expect from a formal research document.

Going beyond those introductory pages, the report provides a wealth of detail. Here are some examples of what the autopilot run produced:

Respondent Landscape. Section 1 identifies who submitted responses — distinguishing thousands of individual submissions (often brief) from several hundred organisational filings (which can range from 8–10 pages to 40+). This distinction is significant: the same dataset reflects both a broad public sentiment and a narrower set of complex policy arguments, which the report addresses separately rather than merging them. (This was a key observation from my manual analysis a year ago — Discovery made the same distinction independently.)

Quantified Public Sentiment. Section 2.2 goes beyond stating “people are concerned” — it categorises tones according to respondent groups, illustrating where worry surpasses enthusiasm and vice versa. It presents charts I would have had to create and plot manually last year.

The Policy Demand Map. Section 3 offers a ranked view of what respondents are requesting from the federal government — ranging from federal preemption of state AI regulations to R&D investment, workforce policies, energy infrastructure, and establishing international standards.

Sector-by-Sector Analysis. Section 4 breaks down the dataset by sectors — including the creative economy, education, healthcare, financial services, and more — with subsections detailing each community’s specific policy requests. For instance, education features a concise bulleted list of requests from educators, districts, and higher education institutions.

The Conclusion. Section 9.1 wraps up the report with a decision-ready recommendation table featuring twelve actionable policy initiatives, each with accompanying evidence and implementation strategies. Additionally, the report outlines a recommended sequence of implementation across various timelines — including immediate administrative action (0–6 months), near-term regulatory measures (6–18 months), and legislative action (18–36 months) — allowing readers to grasp not just what should be done, but also in what order.

This isn’t simply a chatbot answering one-off queries. It’s a robust piece of research a policy analyst could peruse on a flight.

I want to clarify the context of this experiment: I was evaluating the Discovery app’s capabilities on its own. Had I been conducting a thorough analysis for a client, I would have collaboratively worked with Discovery step-by-step — reviewing, refining, diving deeper into specific areas, and addressing any oversimplifications from the autonomous run. Autopilot isn’t recommended for critical projects. It’s an impressive showcase of the platform’s potential when given the freedom to operate efficiently.

Last year, I was a data scientist and software engineer. I was busy crafting summarisation prompts, fine-tuning model selections, establishing Azure Functions and Cosmos DB, troubleshooting failed functions, and connecting MCP servers. I was building both the analytical framework and conducting the analysis.

This year, I stated my intention, reviewed the platform’s proposals, added any missing outcomes, initiated the process — and returned to graded results.

Over the past twelve to eighteen months, the software engineering field has been witnessing a similar transformation: transitioning from writing code to guiding agents that generate code. What’s exciting about Microsoft Discovery is it introduces this very shift into the fields of science, research, and analysis — for anyone adhering to the scientific method, sifting through bodies of research and data, deriving new insights, and reasoning towards a substantiated conclusion.

The corpus I examined consisted of public comments regarding AI policy. However, this same framework applies whether you’re analysing unstructured public feedback, developing a new quantum chip, or exploring anaerobic genes and optimising bioreactors for advanced manufacturing.

If you’re keen to give it a whirl, download the Discovery app. Once logged in with a GitHub Copilot account, all you need is a question to explore, a hypothesis to validate, or a product to create — or all three. Discovery is ready to assist with everything else.

Since this project, I’ve undertaken several more with Discovery. Each time, I challenge it a bit further — pursue a different angle, ask more unexpected questions, and see how far my curiosity leads me. Every time, it delivers.

Where will your curiosity take you?

About the corpus: The OSTP/NITRD RFI on the AI Action Plan (FR Doc. 2025-02305, 90 FR 9088; EO 14179, January 2025) received 10,067 submissions during its comment period.