Schema Validity Is Not Enough

Guest contribution by Adhish Rao, MSc Systems Engineering for the Internet of Things, UCL

This article was developed as part of UCL’s Industry Exchange Network (IXN) in collaboration with Microsoft.

Hello! I’m Adhish Rao, an Electronic Engineering graduate and a postgraduate student in MSc Systems Engineering for the Internet of Things at University College London (UCL). I enjoy exploring how systems engineering, artificial intelligence (AI), the Internet of Things (IoT), and robotics intersect, particularly focusing on how probabilistic AI components can integrate into larger systems without taking away the authority that should belong to the wider application.

I embarked on this journey as part of my MSc dissertation project, benefiting from UCL’s Industry Exchange Network (IXN) and guidance from both UCL and Microsoft. This article showcases my project and the engineering insights I gained along the way. If you want to know more about my experiences and projects, feel free to check out my LinkedIn  and GitHub.

My project started with a fundamental question in systems engineering: If an operator gives a task to a robot using natural language, can a small language model running locally create a useful, structured task proposal? Moreover, what criteria must this proposal meet for downstream systems to trust it?

To explore this inquiry, I built and tested a local AI robot task-planning demonstrator using Microsoft Foundry Local as the local inference environment. In this setup, the model’s capabilities were intentionally limited; it could interpret operator commands and suggest structured tasks but couldn’t determine if the proposal should be accepted. While Foundry Local provided the inference runtime, the governance logic remained a distinct component controlled by the project.

Throughout five prototypes, the project evolved from merely generating structured outputs to a more extensive examination of separating proposal authority. By the final architecture, every model output was treated as an untrusted proposal. A deterministic governance pipeline scrutinised aspects like parsing, schema correctness, task semantics, ambiguity, and adherence to project-defined policies. Only three outcomes were possible: ACCEPT, CLARIFY, or REJECT. Even an ACCEPT outcome merely indicated eligibility for a subsequent qualification stage, without implying feasibility or safety in execution.

This distinction is crucial because a proposal might appear structurally sound while still failing to capture the operator’s intent or harbouring ambiguity. Thus, the project’s focus shifted: instead of questioning if a local SLM could generate valid JSON, we instead considered how AI-generated proposals could be paired with a deterministic authority containment mechanism.

This shift in focus became the project’s guiding principle: a model-generated plan can be valuable without embodying authority. While the model can propose, the project’s deterministic governance dictates whether that proposal can proceed. Control over physical execution remains external to the prototype. The illustration below highlights this crucial separation.

.

Valid JSON was just the start.

During one trial with 30 local commands using Qwen2.5-Coder-0.5B, 25 out of 30 responses met the schema requirements, but only 4 of those could move to the next qualification phase. We refer to this as execution eligibility.

This discrepancy is at the heart of my UCL IXN MSc project: how can a small language model (SLM) serve effectively as a proposal generator without holding authority over subsequent actions?

Microsoft Foundry Local provided the inference runtime, treating each model response as an untrusted task proposal, while a separate project-owned deterministic governance layer determined whether it could go forward.

The model could suggest a task, but it couldn’t grant itself permission.

This article draws upon the findings documented in my submitted dissertation, Zero-Trust Governance for Local SLM Robot Task Planning.

Microsoft Foundry Local offers model management, local caching, and hardware-aware execution among its capabilities. Once you have the necessary model and runtime components on your local system, inference can operate independent of cloud services.

These features are particularly useful when you need offline functionality or local processing of commands and outputs. However, local operation does not inherently guarantee quicker responses, accuracy, or safety. An application may still depend on various online resources; moving to a local inference route doesn’t make the entire system independent of online requirements.

In this project, Foundry Local was fundamental to the inference runtime, but the deterministic governance layer was constructed separately, and the product capabilities differ from the historical configurations we examined.

Foundry Local provides the runtime; the project-owned governance uniquely manages progression.

Think of a request to move a blue component from input tray A to assembly fixture B.

A conceptual example would look like this, serving as a simplified proposal format (not the production API contract):

{
  "operation": "MOVE",
  "object": "blue_component",
  "source": "tray_A",
  "destination": "fixture_B"
}

A parser determines if the response is structured as JSON. A schema then checks its format, but neither step ensures that the proposal matches the request or complies with the project’s standards.

Structural validity and task admissibility respond to different questions. Does the proposal adhere to the intended task? Is there missing context? Is there a policy that prohibits the requested action? Merely passing the structural check does not answer all those questions.

This system distinguishes between proposal generation and authority, a design philosophy developed across five prototypes. Each stage examined a different trust assumption: structure, deterministic validation, empirical measurement, containment, and the integrated local AI operation.

The ultimate system maintains distinct responsibilities. Commands typed by operators or reviewed transcripts follow a shared pathway. An untrusted proposal emerges from the selected inference route. Deterministic checks then assess whole-response parsing, schema validity, semantics, ambiguity, and adherence to project-defined policy.

The governance outcomes are either ACCEPT, CLARIFY, or REJECT. Only an ACCEPT designation establishes eligibility for execution. CLARIFY and REJECT outcomes uphold the authority boundary when a proposal cannot proceed.

An ACCEPT only grants permission for a bounded qualification step. It doesn’t guarantee geometric feasibility, physical safety, or authority over robot control.

The demonstrator can display documented qualification evidence that holds its own provenance; it is not derived from the immediate proposal next to it. Developers analysing the system must clearly identify the proposal record, governance decision, and qualification documentation.

Only ACCEPT grants execution eligibility; the qualification process and physical execution authority remain distinct.

The starting example originated from the model benchmark (P3), which assessed four Qwen configurations, each using 30 commands. Over these configurations, schema validity rates fluctuated between 46.7% and 93.3%, while execution eligibility ranged from 6.7% to 16.7%.

Focusing on the Qwen2.5-Coder-0.5B run, we noted that although 25 out of 30 commands were schema-valid, only 4 could be deemed execution-eligible. Additionally, in this run, 12 responses were classified as model-level false accepts according to the project’s benchmark. It’s important to note that this classification addresses proposal evaluation rather than additional stages in the progression from 25 to 4 or physical incident counts.

For me, the engineering lesson here was clear: reporting schema validity as task accuracy or permission to act could obscure significant failures. This outcome is specific to the commands evaluated, models, and runtime configurations, and it doesn’t offer a general reliability estimate for local SLMs.

In the selected 30-command run, 25 were schema-valid but only 4 were execution-eligible; reports of 12 model-level false accepts are documented separately.

Simply having low eligibility doesn’t clarify whether the model underperformed or if the governance framework was overly strict. We scrutinised this distinction across 40 clear-command observations.

From those observations, 35 met schema validity, but only 14 were execution-eligible. The 26 non-eligible observations included:

• 5 structural failures;
• 17 failures related to semantic proposals;
• 4 instances of repetitive ambiguity rejections.

Thus, 22 of the 26 non-eligible observations failed at the proposal quality stage. However, deterministic governance wasn’t flawless either.

For instance, a clear command, C10, was “Pick up the gauze pack and place it on the tray”. The governance logic incorrectly flagged the pronoun “it” as ambiguous.

The same command led to repeated rejections across all four model configurations. This indicates a singular rule failure rather than separate failures for distinct commands, so we shouldn’t interpret 4 out of 40 as a false rejection rate across the population.

In the previous model benchmark (P3), this ambiguity resulted in a rejection. Conversely, the final qualified resolver adapts the ambiguity-only state to CLARIFY, turning the situation into evidence about the ambiguity rule rather than proof of identical decision labels across the contract versions used.

The lesson for developers is clear: deterministic governance doesn’t imply correctness. Rules also need validation.

The admission-rule comparison (P4) applied progressively stricter rules to the same 120 preserved records:

Dependent retrospective reclassification of the same 120 records; the 14 retained successes are earlier classified records.

The results showed that containment evidence was more robust than utility evidence; here utility indicates that only 14 previously successful records stood firm against stricter admission criteria.

This isn’t a randomised causal analysis or a Pareto optimisation, and it lacks any assessments of physical safety or guarantees of minimal risk in future actions.

The exact-score admission stage required a semantic score of 1.0 (P4), a stricter condition than the earlier semantic-pass metric from the benchmark (P3).

Project-authored reference intents and deterministic scoring underwent version control before the retained Foundry runs, and all 120 stored semantic outcomes were reproduced. This data wasn’t independently annotated or validated.

A different experiment explored deployment behaviours rather than admission policies. A comparison of retained local vs. cloud configuration (Mode C) successfully completed 30 requests in both environments. The average latency amounted to 1.75 seconds for the cloud system compared to 7.03 seconds for local. Cloud responses were also consistently more valid in JSON throughout this comparison.

Notably, the models and deployment configurations diverged. These observations are configuration-level; they do not provide causal estimates of how shifting from cloud to local would alter an otherwise identical system.

A separate resource-profile run recorded 30 successful local requests (Mode D), revealing an estimated average CPU utilisation of 48.3% of total logical CPU capacity on the targeted host. This resource measurement operated independently of the latency comparison conducted previously (Mode C).

When considering local AI deployments, it’s essential to evaluate latency and resource usage based on the specified hardware.

With speech input came another layer of uncertainty; therefore, raw transcriptions remained untrusted until explicitly reviewed by an operator. Once reviewed, this text joined the same governance pathway as the typed input.

Reviewed voice and typed input converge on the same governance path; input modality grants no authority.

In a test comparing typed input against reviewed voice commands, 18 out of 18 tested pairs yielded identical governance outcomes (D4.3). This confirms the shared authority pathway for those tested pairs; however, it does not constitute a study on ASR accuracy or human interaction factors.

In another boundary test involving a real microphone, we conducted seven valid trials (D4.5). Six reviewed commands were submitted, while one transcription was discarded. For instance, “input tray A” was misheard as “input trait A”; the operator corrected it prior to submission.

Further failures occurred elsewhere. All six submitted planner responses failed the comprehensive parse/JSON checks due to wrapping JSON in Markdown code fences, thus violating the strict JSON requirements expected by the parser. Consequently, downstream predicates concerning semantics, ambiguity, policy, and authority could not be assessed. These were parse errors, not rejections based on safety.

Explicit reviews prevented raw ASR text from being automatically submitted as commands. However, the strict parsing exposed a separate planner-interface defect. Neither finding establishes that operators reliably catch transcription errors or that the speech system is safe.

The downstream qualifications consistently resulted in a FAIL (B3.2), revealing 118 support-material penetration instances and no physics validation steps. Additionally, we opted not to alter the route or scenes solely to achieve a passing result.

These qualifications addressed discrete geometric aspects and did not guarantee continuous collision freedom, dynamics, or safety in physical execution.

The failures highlighted additional boundaries: execution eligibility and downstream qualification represent separate elements. While a frozen replay can exhibit these distinctions, it cannot confer authority over physical execution.

Here are seven design lessons that may apply beyond robotics:

• Select local AI based on deployment needs, not presumed advantages.
• Regard generative outputs as untrusted proposal data.
• Differentiate between structural validity and authority.
• Maintain deterministic governance outside the realm of probabilistic generation.
• Also validate governance rules.
• Ensure new modalities do not create additional authority pathways.
• Retain records of negative evidence and keep qualification processes distinct.

This pattern of authority separation might also be vital in areas where generated actions could affect complex systems, such as infrastructure modifications or agent tool interactions. These ideas draw analogies and aren’t strictly evaluated in this project.

Microsoft’s Agent Governance Toolkit addresses similar architectural challenges by implementing deterministic policy controls over agent actions. It’s important to note that Prototype 5 did not implement or evaluate this toolkit, highlighting an architectural comparison rather than a dependency on implementation.

The findings from my MSc project centre on software governance related to the evaluated models, records, and project-defined guidelines. They do not serve as a physical safety case, a certified controller, or evidence of authority in production robots. Control over physical execution remains NOT IMPLEMENTED.

In this context, “zero trust” pertains to an authority principle: no prior success automatically confers authority at the next stage; it’s not a claim of compliance with NIST SP 800-207.

Future inquiries necessitate broader evaluations, independent policy analysis, and more extensive quantitative assessments of model behaviour. Physical deployment will require additional assurance before considering execution authority.

I began my exploration with a question about generation: can a local SLM create a valid robot task proposal? Ultimately, I concluded with a question of authority: what criteria must that proposal meet for downstream processes to establish trustworthiness?

In this initial prototype, the model was responsible for providing proposals, while deterministic governance orchestrated whether they could advance. The governance itself remains an area in need of further testing.

This project was a collaborative effort through UCL’s Industry Exchange Network, leading to my MSc Systems Engineering for the Internet of Things, with academic support from UCL and industry mentorship from Microsoft.

Documentation for Microsoft Foundry Local

Learn more about UCL’s Industry Exchange Network

Visit the Microsoft Agent Governance Toolkit