Performance Tuning

These settings control worker behavior, model inference, batch sizes, timeouts, OCR, and rate limits. Start with the symptom you are seeing, then tune the smallest matching knob.

Quick Tuning Map

Symptom	First Adjustment	Why
High embedding memory	Lower EMBEDDING_BATCH_SIZE or MAX_NODES_PER_TASK	Reduces texts held per FastEmbed call and per Celery task
Slow document ingestion	Add embedding workers before increasing embedding concurrency	Each embedding worker keeps model memory isolated
Slow classification	Lower CLASSIFICATION_SELECT_TOP_K or enable MODERNBERT_PREFILTER_ENABLED	Scores fewer chunks/categories with ONNX
ONNX CPU oversubscription	Keep the Settings/.env.example thread defaults at 1, then inspect effective Compose overrides	Prevents worker processes from competing for the same cores
OCR-heavy uploads are slow	Increase preprocessing capacity or raise OCR_PAGE_TEXT_THRESHOLD carefully	OCR runs in preprocessing and is CPU-heavy
Slow reranking	Reduce the requested rerank prefix or disable reranking; see Reranking	Requests wait for a bounded inference slot and worker inference; increasing the timeout does not bound native work
Too many protected-route 429s	Increase RATE_LIMIT, reduce request volume, or add an external limiter/capacity plan	Rate limiting is per resolved client id for protected routes; /search and /web-search also have route-specific counters
Too many auth-failure 429s	Increase AUTH_FAILURE_RATE_LIMIT only after ruling out credential probing	Failed auth attempts are rate limited per source address before client resolution

Worker Concurrency

Production sizing is generated by just prod-up in .runtime/prod.env:

Setting	Production planner behavior
PROD_EMBEDDING_CONCURRENCY	Fixed at 1
PROD_PREPROCESSING_CONCURRENCY	1 below the 4GB preprocessing target; otherwise 2
PROD_API_WORKERS	Fixed at 1

Do not casually raise embedding worker concurrency. Each prefork child can load the dense and sparse embedding models plus spaCy; add embedding replicas instead so each process keeps an isolated model copy. Publication-enabled API containers also keep one Uvicorn process per PUBLICATION_WORKER_SLOT; scale HTTP serving with separately slotted replicas, not extra processes in one slot.

The base Compose preprocessing service uses MODERNBERT_ONNX_INTRA_OP_THREADS=4 when no value is provided, while the Settings and .env.example defaults are 1. Set the value explicitly when CPU limits make the Compose default too aggressive.

Embedding and Search Knobs

Setting	Default	Why It Matters
MAX_NODES_PER_TASK	50	Number of chunks grouped into one embedding Celery task
EMBEDDING_BATCH_SIZE	32	Number of texts processed at once by FastEmbed
RETRIEVAL_TOP_K	10	Default number of Document Version Evidence Results
RERANK_TIMEOUT_SEC	2.0 seconds	Caller wait deadline; see timeout semantics

top_k_rerank is a request parameter (default 20), not an environment variable.

Classification and NER Knobs

Setting	Default	Why It Matters
CLASSIFICATION_ENABLED	true	Disables ModernBERT classification when set to false
SPACY_NER_ENABLED	true	Disables entity extraction when set to false
CLASSIFICATION_SELECT_TOP_K	3	Number of representative chunks selected via centroid + MMR
MODERNBERT_PREFILTER_ENABLED	false	Enables lexical prefiltering before ONNX scoring
MODERNBERT_PREFILTER_TOP_K	3	Number of candidate categories kept by the prefilter
MODERNBERT_ONNX_NLI_MAX_TOKENS	2048	Maximum NLI sequence length before truncation
MODERNBERT_ONNX_INTRA_OP_THREADS	1	ONNX Runtime threads within a single operator
MODERNBERT_ONNX_INTER_OP_THREADS	1	ONNX Runtime threads across operators

Container defaults

The Settings and .env.example defaults are 1 for both ONNX thread settings. The base Compose preprocessing service falls back to 4 intra-op threads when no value is supplied; set the environment value explicitly when CPU limits make that fallback too aggressive.

OCR and Upload Knobs

Setting	Default	Why It Matters
MAX_FILE_SIZE_MB	200	Upload size limit enforced before processing starts
OCR_PAGE_TEXT_THRESHOLD	80	PDF pages with fewer than 80 native text characters use OCR fallback
ALLOWED_EXTENSIONS	See .env.example	Controls accepted file extensions
ALLOWED_MIME_TYPES	See .env.example	Validates uploaded file MIME types

Office File Settings

Setting	Default	Why It Matters
OFFICE_LEGACY_CONVERSION_ENABLED	false	Enables LibreOffice conversion for .ppt/.xls (legacy binary formats)
OFFICE_LEGACY_CONVERSION_TIMEOUT_SEC	120	Timeout per legacy Office conversion; large files may need more time
LIBREOFFICE_BINARY	soffice	LibreOffice executable path
OFFICE_MAX_UNCOMPRESSED_BYTES	1000000000	Max decompressed size for OOXML files (zip-bomb guard)
OFFICE_MAX_COMPRESSION_RATIO	200	Max uncompressed:compressed ratio for OOXML files

Celery Worker Knobs

Setting	Default	Why It Matters
CELERY_BROKER_VISIBILITY_TIMEOUT_SEC	3600 seconds	Visibility timeout for broker and result-backend task delivery
CELERY_WORKER_MAX_MEMORY_PER_CHILD	1572864 KB	Base memory recycle threshold for worker children
CELERY_WORKER_MAX_TASKS_PER_CHILD	100	Global task recycle threshold
CELERY_WORKER_MAX_TASKS_PER_CHILD_EMBEDDING	250	Embedding worker recycle threshold
CELERY_WORKER_MAX_TASKS_PER_CHILD_PREPROCESSING	500	Preprocessing worker recycle threshold
CELERY_TASK_MAX_RETRIES	3	Maximum retry attempts for recoverable task failures
CELERY_RETRY_BACKOFF_BASE	60 seconds	Base delay for exponential retry backoff
CELERY_RESULT_EXPIRES	86400 seconds	Time Celery results remain in Redis DB /2
CELERY_EMBED_SOFT_TIME_LIMIT_SEC	900	Soft timeout for embedding tasks
CELERY_EMBED_TIME_LIMIT_SEC	1200	Hard timeout for embedding tasks

Rate-Limit Knob

Setting	Default	Why It Matters
RATE_LIMIT	30	Requests per minute per resolved client id for protected endpoints
AUTH_FAILURE_RATE_LIMIT	10	Failed authentication attempts per minute per source address

The built-in limiter uses a 60-second sliding window. Set RATE_LIMIT=0 to disable it. A limited request returns HTTP 429 with X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset headers. Production traffic across API workers shares Redis-backed counters through REDIS_URL; use a reverse proxy or edge limiter only for additional network-level protection.

The application and Compose fallbacks use RATE_LIMIT=30; the repository .env.example sets RATE_LIMIT=300 as a local-testing override. Copying that example changes the effective limit until you set the value intended for the deployment.

Status Endpoint Details

GET /status/{job_id} can return:

• pending
• processing
• completed
• failed
• partial_failed
• retrying

partial_failed means some embedding tasks failed while others succeeded.

When to Leave Defaults Alone

Keep the defaults when you are still validating correctness or running a small deployment. Most tuning should happen only after just status, logs, or benchmark results show a specific bottleneck.