Agent benchmark · Supply Chain E2E · Reproducible workload

Agent → MCP → Foundgine → PostgreSQL

A stateful end-to-end benchmark that exercises an AI-agent-like workload through the MCP capability boundary, Foundgine semantic execution, authorization and transactional PostgreSQL state.

This is the working code behind the "top supplier in Texas" walkthrough — same request, same ambiguity resolution, run against a real seeded PostgreSQL database instead of narrated step by step.

← Back to benchmark Read the walkthrough it implements →Read Markdown report Raw JSON report

What this benchmark covers

It tests the execution boundary, not just database speed.

The workload deliberately mixes valid, invalid and unauthorized operations. It checks that business capabilities remain application-controlled as they cross MCP, semantic planning, authorization, execution and PostgreSQL.

Agent workloadStochastic operations across up to five customer identities plus Bob, Carol, Dave and Admin.

Semantic graphCustomer, Order, OrderItem, Product, Supplier, Category, Inventory, Warehouse, Shipment and Carrier relationships.

AuthorizationRole and customer-ownership rules are exercised on reads and mutations.

High-assurance mutationPlaceOrder validates ownership, quantity, inventory, server-side pricing, atomic writes and replay protection.

State integritySuccessful mutations must produce expected state while denied operations must not mutate PostgreSQL.

Execution evidenceSemantic plans are lowered through Foundgine ExecutionIR and receipts include a plan fingerprint.

Execution path

One capability boundary, several controlled layers.

AI agent→MCP→Foundgine semantics→Authorization→Planner→ExecutionIR→Npgsql→PostgreSQL

Ambiguity resolution · "top supplier in <state>"

Calculated evidence, no candidates at all, a tie, or a closed loop — never a guess.

The Foundgine walkthrough traces one request — "show me overdue purchase orders from our top supplier in Texas" — through every layer, including the step where "top supplier" is not a database key and has to be resolved through ranked candidates and evidence before anything downstream may execute. This benchmark's find_top_supplier_overdue_orders(actor, state, supplierName?) capability brings that exact case into the Supply Chain E2E workload, seeded so all four outcomes get exercised on every run.

Case A · Calculated evidence → executionState TX: Acme Industrial (482,000) unambiguously outranks Globex Components (210,000). Resolution binds the graph to Acme, authorizes it, and executes the overdue-purchase-order query — the response carries the rows plus evidence: rank, margin over the runner-up, and a plan fingerprint.

Case B · Candidates, no assurance → ask, don't guessState CA: Northstar Supply and Southline Parts tie at 300,000. Resolution stops before authorization or execution and returns clarification_needed with the tied candidates and suggested refinements — name the supplier, give a tiebreak criterion, or narrow the region — instead of picking one.

Case C · No candidates at allState NY has no seeded suppliers. Retrieval returns zero candidates, so the response is not_found — there is nothing to resolve, authorize or execute.

Case D · Closing the loopState CA plus an explicit supplierName: the agent has already been told the candidates are tied and comes back with a specific name. The name is validated against the real candidate set — an unmatched name still returns not_found, never a guess — and once matched resolves and executes exactly like Case A, marked resolvedBy: "explicit-name".

Every resolved response also exercises field-level authorization from step 7 of the walkthrough: the commercially sensitive Supplier.NegotiatedCost field is stripped and listed under deniedFields for every actor except Admin, even though the capability call itself was allowed. The agent workload picks randomly among all four shapes on each occurrence, so a single run exercises every outcome. Bob (purchasing/customer service) and Admin are authorized for it; every other actor is expected to be denied at the MCP boundary.

Published report

Latest Supply Chain E2E result

Loading the published report…

Modeled comparison · no live conventional flow

Without Foundgine: how many round trips to fetch the same data?

This workload only executes the Foundgine path — there is no conventional REST/GraphQL flow running side by side to measure directly here. The panel below models what a conventional discover → authorize → execute → verify choreography would need to reach the same result, using the same 4-step count measured directly in Run 1. It is an estimate, not a measurement — see the caveats inline.

Loading the round-trip estimate…

High-assurance PlaceOrder

The first vertical slice

1 · AuthorizationVerify the actor is permitted to place the order.

2 · OwnershipCustomer actors can only place orders for an allowed customer identity.

3 · ValidationResolve products and reject invalid quantities or unavailable inventory.

4 · Server-side pricingPricing remains application/database controlled rather than agent supplied.

5 · Atomic mutationOrder, order items and inventory reservation are committed as one transaction.

6 · Replay protectionIdempotency prevents duplicate order placement from the same replay key.

How to read the result

The Supply Chain run is the final application layer

This page should be read after the repository verification gates: unit tests establish deterministic contracts; PostgreSQL integration tests exercise the real provider; authorization and adversarial-input penetration tests attack the security boundary; and the performance smoke test proves the benchmark stack can execute real traffic without errors. The Supply Chain workload then composes those concerns into one agent-facing business workflow.

CorrectnessValid operations succeed and invalid operations fail without silently changing protected state.

SecurityRole and ownership constraints remain application-controlled rather than agent-controlled.

ExecutionIntent crosses semantic planning and ExecutionIR before reaching the PostgreSQL provider.

The repository gate definitions and exact local commands are documented in samples/Foundgine.SupplyChain.Advanced/VERIFY-GATES.md.

Reproduction

Run it yourself

cd samples/Foundgine.SupplyChain.Advanced
$env:SUPPLY_CHAIN_CUSTOMERS="5"
$env:SUPPLY_CHAIN_STEPS="25"
$env:SUPPLY_CHAIN_SEED="20260823"
./run-supply-chain.ps1

./publish-supply-chain-report.ps1

The runner writes the local JSON and Markdown report under reports/. The publish script copies those artifacts into the website's docs-site/assets/agent-benchmark/supply-chain/ asset folder used by this page.