Five experiments. One question: what should the agent execution boundary do?
The problem is not giving an agent an API. It is giving the agent useful capabilities without making the API the execution model.
A conventional endpoint can solve one operation very well, but as agent-facing capabilities multiply, the application can accumulate bespoke tools, repeated orchestration, duplicated policy checks and increasingly large interaction traces. These experiments ask whether a semantic execution boundary can reduce that agent-side work while preserving correctness and application control.
Read the runs as a progression: establish the baseline, scale it, measure efficiency, compare execution architectures, then test a high-assurance business capability.
Foundgine is intended to let an agent express a capability while the application remains responsible for semantics, authorization and execution. These runs progressively test that idea — from a small customer workflow to a high-assurance financial mutation.
From isolated contracts to a stateful agent-facing system
The benchmark suite is only meaningful when the execution boundary is verified underneath it. Foundgine's story progresses from deterministic unit behavior, through real PostgreSQL execution and security attack paths, into performance, and finally into the Supply Chain application workflow.
Agent → MCP → Foundgine → PostgreSQL
The application-level culmination: a stateful agent workload that brings semantic execution, authorization, high-assurance mutations, replay protection and PostgreSQL state integrity together.
Inspect the exact workload and concurrency result
The matrix shows the completed benchmark result for each workload/concurrency cell. Select a cell to compare the conventional and Foundgine execution paths using the published measurements.
Published workload/concurrency matrix
Select one workload/concurrency cell to inspect the conventional and Foundgine measurements for that exact experiment.
What do these controls mean?
- Run
- Which of the six published runs (1, 2, 3, 4, 5, 5b) you're looking at. Runs 2, 4, 5 and 5b have workload/concurrency grids; Runs 1 and 3 are focused single-scenario experiments, so they show as "no matrix for this run" here — see that run's page for its measured result.
- Metric
- What each cell shows: an RPS ratio (throughput), average wall time, estimated context tokens, or tool calls — always Foundgine compared against the conventional path.
- Customer workload / Concurrency
- Filters the grid to one row or column. "All" keeps every published cell visible; the rest are dimmed rather than hidden.
- Reading a cell
- A blue left edge means Foundgine wins on the selected metric; amber means the conventional path wins; a dashed grey edge means that metric wasn't recorded for this run. Click any solid cell for the full comparison.
| Workload \ Concurrency |
|---|
No matrix cell selected yet.
What this cell means
Click a matrix cell above to inspect the benchmark result for one exact workload/concurrency combination.
The matrix tells you what the system did. The published benchmark measurements show how the two execution paths compare.
1 · Select the experiment
Start with a cell such as 10,000 customers · C64. That fixes the exact workload and concurrency being discussed. The result above is the published benchmark measurement for that experiment.
2 · Read the execution result
Compare tool calls, estimated context, latency, throughput and success rate. These describe the agent and application execution path. A lower context figure means less estimated replay context; it does not by itself prove lower electricity use.
3 · Inspect the exact measurements
Use the selected-cell comparison and detailed table to see RPS, wall time, estimated context load and tool calls for both execution paths. The values come from the published benchmark artifacts.
4 · Keep the claims separate
This is deliberate. We can say lower estimated context when the benchmark supports it. We should only claim lower energy or carbon when the corresponding energy/emissions measurements were actually captured.
How much does this approach save, in one number?
Run1-5 above measure a live conventional-vs-Foundgine comparison. Two other workloads in this repo — Supply Chain E2E's agent-authorization test and the SupplyChain.Semantic pipeline-weight benchmark — have no conventional counterpart running next to them, so they report a modeled estimate instead of a measured one. Both kinds are shown below, clearly labelled, so they're never confused for each other.
Measured and estimated values are labelled separately
Measured benchmark signals include tool calls, request timing, RPS and success/failure counts. The replay harness also computes an estimated context load using the benchmark's documented heuristic; that is not provider billing data.
Estimated context tokens are included because they describe the amount of agent-facing context carried through the workflow. They should not be presented as exact model-provider usage unless the benchmark is run against a live model with provider usage reporting.
Home · What is Foundgine? · How it works · ArchitecturePackages