Load Performance Evidence

This page records the repeatable methodology for issue #87. Run it against a UAT-like stack, not a developer hot-reload process.

Targets

Area	Target	Evidence command
Search without rerank	p95 < 100ms	benchmark_query_latency.sh or /search samples
Search with rerank	p95 < 300ms when enabled	/search with rerank=true samples
Indexing throughput	documents/hour recorded	test_load.sh and load_performance_report.py
Large-document handling	large-document result recorded	Upload near configured max size in UAT
Resource sizing	CPU, memory, disk recorded	just status docker --iterations 1

Procedure

just prod-up rebuild
just status docker --iterations 1
./.agents/skills/search-engine-test/scripts/test_load.sh "$API_KEY" test-documents/banking 5 3
python scripts/load_performance_report.py \
  --search-latencies-ms 20,30,40 \
  --rerank-latencies-ms 120,180,240 \
  --documents-processed 6 \
  --elapsed-seconds 120 \
  --large-document-result pass \
  --cpu-percent 42.5 \
  --memory-percent 61.0 \
  --disk-percent 30.0

Replace the sample latency values with measured values from the run. Link the JSON output from the pull request or release checklist for go-live sign-off.

The report is also a readiness gate. It exits 0 only when all required evidence is present and passing; otherwise it exits 1 and sets:

{
  "go_live_ready": false,
  "evidence_status": "fail",
  "evidence_failures": [
    "search latency samples missing",
    "rerank latency samples missing",
    "throughput evidence missing",
    "large document evidence missing or failed",
    "resource sizing evidence missing"
  ]
}

Both search and rerank latency sections fail when samples are missing, not just when p95 exceeds the target. Treat a missing-evidence failure the same as a measured target failure until the UAT packet includes real samples.

Deviation Handling

If p95 search exceeds target, record whether rerank was enabled, current Qdrant collection size, worker counts, CPU, memory, disk, queue depth, and resource sizing assumptions. If rerank p95 exceeds the target, keep rerank opt-in and document the decision.