Pipeline & Throughput Benchmarks
End-to-End Pipeline Latency
| Stage | Target | Actual | Status | Per-Page |
|---|---|---|---|---|
| Upload | <500ms | 0.20s mean | PASS | ~40ms |
| Processing | 5-30s | 7.71s mean | PASS | ~1.54s |
| Search hybrid (no rerank) | <50ms | 19.6ms mean of 5 | PASS | N/A |
| Search dense-only | <50ms | 19.2ms mean of 5 | PASS | N/A |
| Search sparse-only | <50ms | 8.8ms mean of 5 | PASS | N/A |
| Reranking (20 docs) | <2s | 1,469.67ms mean of 3 | PASS | N/A |
| Total E2E | 5-35s | 26.93s wall clock | PASS | 0.90s |
Why can processing still be slow?
Processing time is dominated by document parsing, embedding, Qdrant upsert work, and the post-embedding ModernBERT ONNX classification callback. Some documents report near-zero poll duration because they completed before the polling request observed them.
Classification Benchmarks (Post-Embedding)
Classification runs after all embedding batches complete. It reads chunk embeddings from Qdrant, selects representative chunks via centroid + MMR, and scores them with ModernBERT ONNX.
Banking documents (5 pages, 11-17 chunks)
| Document | Chunks | Categories from live document metadata |
|---|---|---|
| bank_of_america_2024.pdf | 12 | financial reporting, risk management, corporate governance, regulatory compliance, Ledger |
| citigroup_2024.pdf | 11 | financial reporting, risk management, corporate governance |
| goldman_sachs_2024.pdf | 11 | financial reporting, risk management, corporate governance, business operations, market analysis |
| jpmorgan_chase_2024.pdf | 12 | financial reporting, corporate governance, regulatory compliance, Ledger |
| morgan_stanley_2024.pdf | 13 | financial reporting, risk management, corporate governance, market analysis, sustainability |
| wells_fargo_2024.pdf | 17 | financial reporting, risk management, corporate governance, regulatory compliance |
Gold classification benchmark
| Metric | Value |
|---|---|
| Inline path accuracy, prefilter off | 5/6 (83.3%) |
| Post-embedding path accuracy, prefilter off | 5/6 (83.3%) |
| Median latency, prefilter off | 3.257s |
| Median latency, prefilter on | 0.521s |
<8s per-document claim | Confirmed |
Classification variance
The gold benchmark missed q1-earnings-trading-update, where expected categories were financial reporting and market analysis, but the classifier returned financial reporting and risk management.
Page Throughput Analysis
| Metric | Value |
|---|---|
| Total Pages | 30 |
| Wall Clock Time | 26.93s |
| Pages per Minute | 66.85 |
| Seconds per Page | 0.90s |
| Avg Doc Latency | 7.91s |
Per-Document Latency
| Document | Upload | Processing | Total | Chunks |
|---|---|---|---|---|
| bank_of_america_2024.pdf | 0.51s | 18.11s | 18.62s | 12 |
| citigroup_2024.pdf | 0.51s | 18.11s | 18.62s | 11 |
| goldman_sachs_2024.pdf | 0.05s | 0.00s | 0.05s | 11 |
| jpmorgan_chase_2024.pdf | 0.05s | 2.01s | 2.05s | 12 |
| morgan_stanley_2024.pdf | 0.04s | 0.00s | 0.05s | 13 |
| wells_fargo_2024.pdf | 0.03s | 8.04s | 8.07s | 17 |
Embedding Stage Breakdown
| Stage | Mean | P50 | P95 | Count |
|---|---|---|---|---|
| NER | 0.5063s | 0.4685s | 0.7137s | 6 |
| Dense embedding | 15.9442s | 14.8100s | 21.7145s | 6 |
| Sparse embedding | 0.0055s | 0.0040s | 0.0087s | 6 |
| Qdrant upsert | 0.0192s | 0.0105s | 0.0523s | 6 |
| Total task | 16.8855s | 15.6760s | 22.8473s | 6 |
Resource Utilization
| Resource | Usage |
|---|---|
| Total Documents | 7 |
| Total Pages | 30 banking pages + 1 OCR image |
| Total Chunks | 78 |
| Memory Usage Before Run | 81% |
| CPU Usage Before Run | 20% |
| Queue Status Before Run | IDLE |
Throughput Summary
| Metric | Value |
|---|---|
| Documents/minute | 13.37 |
| Pages/minute | 66.85 |
| Chunks/minute | 169.36 |
| Search queries/second, hybrid | ~51.0 |
| Search queries/second, dense-only | ~52.1 |
| Search queries/second, sparse-only | ~113.6 |
Full Latency Reference
| Operation | Mean | Median | Range | Notes |
|---|---|---|---|---|
| Upload | 0.20s | 0.05s | 0.03-0.51s | Banking corpus upload |
| Processing (5 pages) | 7.71s | 5.02s | 0.00-18.11s | Poll-observed duration |
| Search (hybrid, no rerank) | 19.6ms | 16ms | 16-27ms | Historical 5-run mean, alpha=0.7 |
| Search (dense-only) | 19.2ms | 17ms | 16-26ms | 5-run mean, alpha=1.0 |
| Search (sparse-only) | 8.8ms | 9ms | 7-11ms | 5-run mean, alpha=0.0 |
| Reranking (20 candidates) | 1,469.67ms | 1,468ms | 1,435-1,506ms | 3-run mean |
| OCR image processing | 8.08s | 8.08s | 8.08s | Single image rerun |
| Health check | 1,016.25ms | 1,016.25ms | 1,016.25ms | Single check |