Scaling for High Traffic

Scale the component that owns the bottleneck. InfoConnect separates HTTP serving, document preprocessing, embedding, Redis coordination, Qdrant storage, and the publication authority. Start with the generated production runtime plan and measurements from your own document mix; do not copy an unmeasured capacity number from another deployment.

Current scaling shape

The current production deployment is Docker Compose on one host. Components have different scaling rules:

Component	Role	Supported scaling shape
api	HTTP requests and read-time publication snapshots	Separate replicas behind a load balancer; each replica needs a unique stable PUBLICATION_WORKER_SLOT when publication is enabled.
worker-embedding-1, worker-embedding-2	Dense and sparse embedding, NER, and embedding-related tasks	Horizontal replicas consuming the shared embedding queue; keep process concurrency at 1.
worker-preprocessing-1	Parsing, OCR fallback, chunking, and classification/reconciliation tasks	One planner-selected replica with concurrency 1 or 2.
redis	Application state, Celery broker/results, rate-limit state, and publication coordination	One Compose service; size its memory and preserve its persistence/eviction contract.
qdrant	Vector and payload storage	One Compose service; increase its host/storage capacity as the collection grows.
models volume	Shared model artifacts	Persist and pre-warm the model cache; it is not a request-serving replica.

Do not add Uvicorn processes inside a publication slot. There is one Uvicorn process per publication slot. PROD_API_WORKERS=1 is the production planner value, and the publication protocol maps one stable slot to one search process. To add HTTP capacity, deploy separately configured API replicas behind a load balancer and give every replica a unique slot. Reusing a slot or scaling the API container with the same slot breaks publication fencing.

Publication-aware scaling

When PUBLICATION_ENABLED=true, read-time generation authority is part of the scaling design:

• Qdrant Manifests are the sole read-time authority. A candidate is admitted only when the active manifest validates its immutable generation and lane evidence.
• Publication uses four dedicated Qdrant collections: content, fuzzy, metadata, and manifest. They are isolated from the pre-cutover collections.
• Redis DB 3 journals operations, holds fences and worker-roster state, and caches authority state. Redis cannot admit a search candidate in place of a Qdrant Manifest.
• Soft delete, restore, and metadata updates publish a new immutable generation. Hard delete first removes the identity from publication authority, then purges every publication lane; its durable generation watermark makes an interrupted purge retryable and lets reads sweep covered orphans. The detailed contract is owned by ADR-0011.
• Obsolete-generation cleanup uses idempotent intent records so a crash can be recovered safely. Four-store restore runs under a leased single-writer maintenance fence; serving fails closed while that fence is active.

Publication-enabled replicas therefore need the same Redis DB 3 authority and must use unique stable slots. The Search API publication contract defines when pinned reads succeed and when callers must retry. See the Architecture Guide and Backup, Restore, and Rollback before changing a publication deployment.

Production runtime planning

just prod-up runs scripts/generate_prod_runtime.py and writes the host-aware plan to .runtime/prod.env before starting the production stack. To generate or inspect the plan without starting services:

just prod-runtime-plan
cat .runtime/prod.env

The planner:

• rejects hosts with less than 8192 MB of memory;
• reserves between 512 and 1024 MB for the host, then reserves the minimum API, Qdrant, console, Redis, preprocessing, and embedding service baseline;
• fixes API memory at 6144 MiB (6 GiB) by default and fails with a shortfall message if the host cannot fit it alongside the other services;
• grows Qdrant memory from 384 toward 2048 MB as capacity permits — a host that cannot spare the 2 GiB Qdrant target plans the partial budget the shared pool leaves above the 384 MB floor and logs prod_runtime_qdrant_budget_short_of_target;
• plans Redis between 256 and 2048 MB, then sets REDIS_MAXMEMORY with 25% headroom;
• derives embedding slots from CPU and remaining memory, clamped to 1 through 8 total slots;
• fixes PROD_EMBEDDING_CONCURRENCY=1 and splits the resulting embedding replicas across the two Compose embedding services; and
• keeps one preprocessing replica, selecting concurrency 1 until the 4096 MB preprocessing target is reached and 2 thereafter.

For small QA hosts, explicitly select a lower API limit with PROD_API_MEMORY_OVERRIDE_MB=4608 just prod-runtime-plan, or pass --api-memory 4608 to scripts/generate_prod_runtime.py. The generated file records PROD_API_MEMORY_OVERRIDE_USED=true; the override is never inferred from a previous generated plan. Supply the override again when running prod-up, which regenerates the plan. A 12288 MiB host fails without an override; a sufficiently large host emits 6144M. An existing higher API limit is never overwritten downward without an explicit override: regeneration fails and preserves the file. Overrides below 2048 MiB are rejected.

The generated file is the effective production plan. Review it before deployment, especially after changing host size, CPU limits, or memory limits. Each PROD_*_MEMORY value is the container's RAM limit; the Compose overlays set no memswap_limit, so Docker keeps MemorySwap at twice that limit and the cgroup total stays RAM plus swap (the Qdrant budget of 2048 MB is therefore a 4096 MB RAM+swap total). Do not replace the planner's embedding replica strategy by raising embedding process concurrency: each prefork child can load the dense and sparse models and spaCy, so extra children duplicate heavy model state.

Worker queues and bottlenecks

Celery routes parsing, splitting, OCR-related processing, classification, publication reconciliation, finalization, and metadata updates to preprocessing. Embedding, representative chunk selection for publication classification, and re-embedding use embedding. Both embedding services consume the same queue, so adding host-planned embedding replicas increases available embedding slots without duplicating model state inside a process.

Worker recycling is part of the memory plan. The application defaults are:

Setting	Default
CELERY_WORKER_MAX_MEMORY_PER_CHILD	1572864 KB
CELERY_WORKER_MAX_TASKS_PER_CHILD	100
CELERY_WORKER_MAX_TASKS_PER_CHILD_EMBEDDING	250
CELERY_WORKER_MAX_TASKS_PER_CHILD_PREPROCESSING	500

The production Compose services apply the generated PROD_*_MAX_MEMORY_PER_CHILD_KB values and worker-specific task-recycle values. Tune batch sizes, timeouts, OCR, and ONNX threads only after confirming the queue and memory symptoms in the Performance Tuning guide.

API replicas and request capacity

API replicas increase concurrent HTTP-serving capacity; they do not make one search intrinsically faster. In publication mode:

1. Keep one Uvicorn process per API container (PROD_API_WORKERS=1).
2. Put separately configured API replicas behind a load balancer.
3. Assign each replica a unique stable PUBLICATION_WORKER_SLOT.
4. Point every replica at the same publication Redis authority and Qdrant deployment.
5. Watch /metrics for request latency, errors, component health, and publication authority state.

The built-in limiter shares per-client counters through Redis across API workers. It is not a substitute for a network-level limiter when the API is exposed to the public internet; add an edge or reverse-proxy control for that boundary.

Qdrant capacity

The standard Qdrant collection uses named vectors: text-dense with dimension 384 and cosine distance, and text-sparse for BM25 retrieval. Publication mode provisions the separate content, fuzzy, metadata, and manifest collections; the metadata lane has its own dense and sparse vectors.

The Compose deployment runs one Qdrant service with persistent qdrant_data storage and no configured Qdrant replica set. Give that service enough host memory and disk for the actual corpus, and measure search latency as the corpus grows. Do not infer publication authority or active generations from Qdrant points alone; the Manifest and Redis authority protocol must remain intact.

Use the Search & Retrieval benchmarks, Pipeline & Throughput benchmarks, and Load Performance Evidence for measured results and methodology. Those results are environment- and corpus-specific, not capacity guarantees.

Redis capacity and persistence

The Compose Redis service is pinned to Redis 7.4.10 and uses append-only persistence with appendfsync everysec. Its configured maxmemory-policy is noeviction:

• keys are not evicted to make room;
• writes fail when the configured memory ceiling is reached; and
• operators must increase planned capacity or reduce retained state before the ceiling is hit.

The logical databases have separate responsibilities:

Database	Purpose
/0	Application state, rate-limit counters, and job state
/1	Celery broker queues
/2	Celery task results (application default expiry: 86400 seconds)
/3	Publication journal, fences, worker roster, and authority registry/cache using the publication ACL

The production planner preserves 25% Redis headroom when it derives REDIS_MAXMEMORY. Monitor both ordinary Redis memory and publication runtime gauges. Never change to an eviction policy in an attempt to hide memory pressure: publication and task state must remain durable and explicit.

Measure before changing capacity

Before scaling, capture the current bottleneck and the effective runtime settings:

just status
just status live
cat .runtime/prod.env

Then use /metrics and the Monitoring Metrics guide to inspect queue depth, worker task duration and failures, HTTP latency/errors, host resource usage, component health, and publication state. For a representative corpus, run the repository's benchmark/evaluation recipes and record the host, model settings, document mix, OCR usage, classification mode, and reranking choice alongside the result.

Use this decision order:

1. Embedding queue grows: add planner-sized embedding replicas; do not raise embedding concurrency casually.
2. Preprocessing queue grows: check OCR, parsing, classification, and the planner's preprocessing concurrency before changing embedding capacity.
3. HTTP concurrency is exhausted: add separately slotted API replicas behind a load balancer.
4. Search latency or Qdrant resource use grows: inspect Qdrant capacity, filters, retrieval settings, and reranking before adding API replicas.
5. Redis headroom falls or writes fail: stop adding retained state and restore the planned memory budget while preserving noeviction.
6. Publication authority is not ready: follow the publication recovery runbook; do not route traffic around the Manifest authority or serve partial generations.

Scaling checklist

• Generate and inspect .runtime/prod.env with just prod-runtime-plan.
• Confirm PROD_EMBEDDING_CONCURRENCY=1 and the generated embedding replica scales.
• Confirm every publication-enabled API replica has one Uvicorn process and a unique stable PUBLICATION_WORKER_SLOT.
• Confirm all replicas use the same publication Redis DB 3, ACL credentials, and Qdrant deployment.
• Check queue depth, task failures, HTTP latency, component health, and publication gauges in /metrics.
• Check Redis INFO memory and confirm the configured noeviction policy remains in force.
• Compare Qdrant storage/resource use with the corpus and document-version growth forecast.
• Run a representative benchmark and review the Benchmarks section before treating a capacity change as validated.
• Recheck search quality after scaling or changing retrieval/reranking settings.

What's next

• Tune individual controls in the Performance Tuning guide.
• Review request budgets in the Rate Limiting guide.
• Monitor health and publication readiness in Monitoring Metrics.
• Follow Backup, Restore, and Rollback for Redis or publication incidents.