Proof row
No public lift number exists
The benchmark case publishes no numeric lift result and labels the current result as null.
Supports the claim-discipline argument that no benchmark number ships before the run clears gates.
Benchmark · Null result
Controlled benchmark is not public yet
A technical evaluator wants a controlled, reproducible answer to whether structured project context improves agent work.
benchmark benchmark pending planned sanitized 3 evidence rows 7 evidence values
benchmark gate
Benchmark gate
1 Design
2 Run
3 Review
4 Publish
No number before gates pass
HERMES case-study frame
Read the case before the evidence room.
A reviewer should be able to answer seven things before opening the JSON export: what decision is in front of them, what changed against a cold start, which row carries the claim, where the value appears, what source trail exists, what purchase decision it informs, and what the case does not prove.
- 01 · Buyer question
What is being decided?
A technical evaluator wants a controlled, reproducible answer to whether structured project context improves agent work.
- 02 · Cold start
Premature claim
The tempting path is to publish a broad productivity number before the benchmark earns it.
- 03 · KBPD-backed
Gated benchmark
The benchmark separates structured files from generic docs and prompt-only baselines before any public number ships.
- 04 · Boundary
Null result
No benchmark result exists on the public site yet.
Best public row no-public-lift-number Publicly inspectable
- Evidence value
- Current public number: none The site does not claim a measured productivity lift yet.
- Source trail
- Benchmark design measurement plan · Public projection
- Buyer decision
- Read the technology page This case exists to show claim discipline. The current public result is deliberately a null result.
- Limit
- Null result No benchmark result exists on the public site yet.
Evidence room
Inspect the evidence, values, and limits.
This is the public review surface for the case. Each row names the observation, the evidence values it stands on, the source trail, the boundary, and the matching structured entry in the evidence export.
Review order Question -> evidence -> value -> source -> limit Start with the case question, then inspect what the claim is allowed to stand on.
Evidence export /proof/substrate-benchmark/evidence.json The same evidence rows are emitted as JSON for independent review.
Claim boundary Null result The page cannot claim more than this status supports.
Source availability
A source trail is not always a raw public artifact. Public projection means the source has been redrawn or summarized; retained source means it is named for audit but withheld until redaction clears.
- Public projection
- 1
- Retained source
- 0
- Pending run
- 1
- Evidence rows
- 3 Reviewed public observations
- Evidence values
- 7 Metric and run values exposed for review
- Source trail
- 2 Artifacts named for review
- Claim status
- Null result Claim boundary for publication
Claim under review
No numeric lift claim is public until the preregistered benchmark runs and clears its validity gates.
This case exists to show claim discipline. The current public result is deliberately a null result.
01 No public lift number exists Publicly inspectable evidence_facts[0]
- Observed
- The benchmark case publishes no numeric lift result and labels the current result as null.
- Evidence
- Metric row says current public number is none; claim status is null result.
- Supports
- Supports the claim-discipline argument that no benchmark number ships before the run clears gates.
- Current public number: none The site does not claim a measured productivity lift yet. Value handle: data-01-current-public-number
- Provenance: Benchmark design summary Where the public projection says the evidence came from. Value handle: data-05-provenance
- Reproducibility: The public harness becomes the source once the run clears gates. What a reviewer can repeat or inspect publicly. Value handle: data-06-reproducibility
- Claim status: null result The maximum strength of the public claim. Value handle: data-07-claim-status
No benchmark result exists on the public site yet.
02 Three benchmark arms are planned Public projection evidence_facts[1]
- Observed
- The planned benchmark compares project system, generic docs, and competent prompt-only baseline arms.
- Evidence
- Metric row lists three planned arms; after-artifact summary describes separating structured files from generic docs and prompt-only baselines.
- Supports
- Supports the benchmark design claim without presenting the result as already run.
- Planned arms: 3 Project system, generic docs, and competent prompt-only baseline. Value handle: data-02-planned-arms
- Provenance: Benchmark design summary Where the public projection says the evidence came from. Value handle: data-05-provenance
- Claim status: null result The maximum strength of the public claim. Value handle: data-07-claim-status
The final claim may be narrow or null.
03 Run artifacts are pending Pending evidence_facts[2]
- Observed
- The source artifact list marks run artifacts as pending, and the limitations state no public benchmark result exists yet.
- Evidence
- Source artifacts include pending run artifacts; limitations explicitly state the result may be narrow or null.
- Supports
- Supports the status boundary for the benchmark placeholder.
- Run date: Pending Date label attached to this public proof case. Value handle: data-03-run-date
- Provenance: Benchmark design summary Where the public projection says the evidence came from. Value handle: data-05-provenance
- Claim status: null result The maximum strength of the public claim. Value handle: data-07-claim-status
The result will be published only after the validity gates pass.
Evidence values
The values this proof is allowed to stand on
Field Value Meaning
Current public number Value handle: data-01-current-public-number none The site does not claim a measured productivity lift yet.
Planned arms Value handle: data-02-planned-arms 3 Project system, generic docs, and competent prompt-only baseline.
Run date Value handle: data-03-run-date Pending Date label attached to this public proof case.
Sample Value handle: data-04-sample Not run publicly Sample size or unit under review.
Provenance Value handle: data-05-provenance Benchmark design summary Where the public projection says the evidence came from.
Reproducibility Value handle: data-06-reproducibility The public harness becomes the source once the run clears gates. What a reviewer can repeat or inspect publicly.
Claim status Value handle: data-07-claim-status null result The maximum strength of the public claim.
Source trail
Where the public proof can point
-
source-01-benchmark-designBenchmark design measurement plan · public projection -
source-02-run-artifactsRun artifacts benchmark output · pending run
Evidence status mix
What kind of support is each row?
- Publicly inspectable
- 1
- Public projection
- 1
- Pending
- 1
Metrics
Signals carried by the case
- none Current public number The site does not claim a measured productivity lift yet.
- 3 Planned arms Project system, generic docs, and competent prompt-only baseline.
Source artifacts
Where the evidence comes from
- Benchmark design measurement plan · public projection
- Run artifacts benchmark output · pending run
Case quality review
This case has a cold-review quality score.
This is a cold-review rubric, not a sales claim. It checks whether the case can be understood, inspected, bounded, and acted on.
Overall 4.2 Inspectable with caveats
Cold-audience clarity
5.0A reviewer gets the situation, contrast, and buyer decision without reading the JSON.
Evidence visibility
4.0The case exposes proof rows, values, source handles, and excerpts before the JSON export.
Limitation clarity
5.0The page names what the case proves and what it does not prove.
Visual hierarchy
3.0The route uses visuals, storyboards, or surface frames to break the proof into inspectable layers.
Purchase motivation
4.0The proof sends the buyer to an evidence room, artifact link, pricing path, or local run.
Next depth gate
Visual hierarchy: Promote the proof row into a diagram, table, screenshot projection, or openable artifact.
Open case evidenceEvidence docket
Review the evidence rows behind the case.
- Evidence rows
- 3
- Publicly inspectable
- 1
- Public projection
- 1
- Pending
- 1
- 01 Publicly inspectable
No public lift number exists
- Observed
- The benchmark case publishes no numeric lift result and labels the current result as null.
- Evidence
- Metric row says current public number is none; claim status is null result.
- Supports
- Supports the claim-discipline argument that no benchmark number ships before the run clears gates.
- 02 Public projection
Three benchmark arms are planned
- Observed
- The planned benchmark compares project system, generic docs, and competent prompt-only baseline arms.
- Evidence
- Metric row lists three planned arms; after-artifact summary describes separating structured files from generic docs and prompt-only baselines.
- Supports
- Supports the benchmark design claim without presenting the result as already run.
- 03 Pending
Run artifacts are pending
- Observed
- The source artifact list marks run artifacts as pending, and the limitations state no public benchmark result exists yet.
- Evidence
- Source artifacts include pending run artifacts; limitations explicitly state the result may be narrow or null.
- Supports
- Supports the status boundary for the benchmark placeholder.
Evidence Flow
Read the proof as a chain, not a claim.
Each case has to survive the same path: starting condition, system-backed change, visible result, retained evidence, stated limitation, and a next action.
- Before Premature claim marketing number
- Change System context enters benchmark
- After Gated benchmark controlled measurement plan
- Evidence 2 source artifacts Pending
- Limit 3 named limits Null result
Claim
No numeric lift claim is public until the preregistered benchmark runs and clears its validity gates.
This case exists to show claim discipline. The current public result is deliberately a null result.
Before / after
Same use case. Different starting conditions.
Before · marketing number
Premature claim
The tempting path is to publish a broad productivity number before the benchmark earns it.
Agents are 40% better with KBPD. This is exactly the claim we are not making yet.
After · controlled measurement plan
Gated benchmark
The benchmark separates structured files from generic docs and prompt-only baselines before any public number ships.
The result may be positive, narrow, or null. The studio will publish the earned result, not the desired one.
Evidence
What this case actually carries.
Run metadata
Metrics
- none Current public number The site does not claim a measured productivity lift yet.
- 3 Planned arms Project system, generic docs, and competent prompt-only baseline.
Source artifacts
- Benchmark design measurement plan · public projection
- Run artifacts benchmark output · pending run
Limitations
What this case does not prove.
- No benchmark result exists on the public site yet.
- The final claim may be narrow or null.
- The result will be published only after the validity gates pass.
Next