Augusto Peña ← Work

Structured data · Agent

Governed NL→SQL Agent

Ask a question in plain language. Where a governed metric already exists the model never writes SQL at all. Where it does, five deterministic gates and a policy check stand between the draft and the database, and every stage is logged.

Role  Architecture & build· Stack  Python, Pydantic, DuckDB, FastAPI, semantic layer, RAG, multi-backend orchestration· Code ↗· Live demo ↗

The problem

Natural-language querying is appealing, but in regulated finance you cannot ship a black box that writes and runs its own SQL against real data. Every answer has to be grounded in the actual schema, checked before it executes, constrained by who is allowed to see what, and reconstructable afterwards. Whether a model can write SQL was never the hard part. Letting it help without handing over the keys is.

The pipeline

The model occupies one stage, and for a good share of questions it is skipped entirely. What comes before decides whether the model is needed at all; what comes after constrains and records the result.

Semantic layerRAG (schema + examples)LLMDeterministic validationGoverned policy retrievalSQLAudit
  1. The semantic layer answers first, if it can. Questions that map to a certified metric are resolved against a governed definition rather than freshly authored SQL. The definition carries an owner, a version and an approved expression, so two people asking the same question get the same number and that number matches the one finance already published. Here the model selects a metric. It does not invent a calculation.
  2. Retrieval grounds the rest. The relevant schema and curated query examples are retrieved and packed into context, so the model drafts against the real data model instead of guessing at it.
  3. The model interprets. Its only output is a schema-checked plan object: an intent classification (query, out of scope, or refused), the candidate SQL, the tables it touches, and any assumptions it had to make. Nothing outside that contract is accepted. It proposes. It never executes.
  4. Deterministic validation. Five gates run in ordinary code: single statement only, read-only, a fail-closed table allowlist, a mandatory row limit, and an EXPLAIN dry run against the engine. A failure stops the request rather than being retried into compliance.
  5. Governed policy retrieval. The applicable access and usage policy is fetched and enforced against the validated query: read-only surfaces, permitted tables and columns, scope limits.
  6. Execution. Only a plan that cleared both layers runs, against a read-only DuckDB connection.
  7. Audit. Retrieved context, the model draft, every individual gate result, the policy applied, the final query and the result shape are appended to a JSONL trail you can replay after the fact.

Why a semantic layer changes the problem

Text-to-SQL gives an agent access to data. It does not give it understanding of the business. In a warehouse where the same idea appears as a dozen similarly named columns and the same acronym means different things to treasury and retail, a fluent model will write confident SQL against the wrong one. The failure is not a syntax error you can catch. It is a plausible number nobody can reconcile.

Putting a semantic layer in front of the model reframes the job. For anything covered by a certified metric, the task becomes selection rather than authorship, which is a far smaller surface for a model to get wrong, and every answer traces back to a definition with an owner. The uncovered long tail still goes through the governed text-to-SQL path, with all the gates below applying. Two tiers, different risk, different treatment.

This is also the part most demos skip, because it is unglamorous and it is where the actual work in a bank lives.

Why these gates and not others

The gates are not a generic hardening checklist. Each one answers a question I used to be on the other side of, and anyone who has sat in a model risk or controls review will recognise the shape of them.

The point I care about is the ordering. Governance decided the architecture rather than being applied to it afterwards, which is the difference between a system that passes review and one that gets rebuilt after it.

Tested against attack, not just happy paths

A governance layer that has only been tried on well-behaved questions has not been tested. The suite here attacks it: SQL smuggled through the plan object, attempts to read the local filesystem, writes and schema changes dressed up as questions, and queries aimed at tables outside the allowlist. Eleven of eleven gates hold.

Refusal is part of the contract rather than an error case. When a request cannot be answered within policy, the system declines deterministically instead of improvising something close enough. The demo runs on a synthetic set of 5,000 transactions with roughly 1,500 flagged, so the questions are the kind a financial-crime analyst would actually ask.

Three interchangeable backends

The backend is swappable, and the demo currently runs on Qwen2.5 3B hosted locally through Ollama. That is a deliberate hold rather than a default I inherited: I want more testing before I claim a frontier model is doing anything the pipeline is not, and the same question can be sent through GLM 5.2 or a hosted commercial model without touching a line of the governance code.

That swap is the point. Changing the model changes nothing about the governance layer: same semantic resolution, same grounding, same validation, same policy enforcement, same audit trail. Quality and latency move. The controls stay put. Running on a four-billion-parameter model on a laptop is the sharper version of the argument, because if the scaffolding is doing the real work then the answer should hold, and if it only holds on a frontier model then the scaffolding was never the thing carrying it. That question is worth settling properly, which is why the small model stays in place until the evaluation says otherwise.

Backend choice turned into a hard number early. On the same smoke test, a hosted frontier API answered in 1.6 seconds, a warm local model in 11.6, and one free tier at 127.5, which ruled it out of the demo entirely. Measuring it beat guessing at it.

Demo: backend selector, generated query, per-gate result chips, audit trail

The web demo shows each pipeline stage as it runs, with a green or red chip per gate, so the governance is the visible part rather than a claim in the README. Access sits behind signed expiring tokens, because a public inference endpoint with no gate is somebody else's free compute.

What it demonstrates

This is the unit the rest of my work is built on, and it is deliberately the first block of Ask AGUS rather than a standalone demo. Build the governed structured-data agent first, prove it holds under attack, then add agents around it. It also settles the trade a controls function needs met before natural-language tooling goes near real data: flexibility at the top, deterministic authority underneath, and a record that proves what happened.