Configuration

The InfoConnect Hybrid Search Engine uses environment variables to control how it works. You can create a .env file in the project root. The system reads this file when it starts.

Getting Started with Configuration

Most users only need to change the API key mapping and the three Redis credentials to get started. The rest can use their default values.

Here is the minimum you need for your first setup:

# API key to client mapping (required for security)
API_KEY_CLIENTS={"my-secret-key-123":"my-client"}

# Distinct URL-safe Redis ACL credentials (required by Docker Compose)
REDIS_APP_PASSWORD=replace-with-app-password
REDIS_CELERY_PASSWORD=replace-with-celery-password
REDIS_PUBLICATION_PASSWORD=replace-with-publication-password

# How fast users can call the API (optional)
RATE_LIMIT=30

# How much detail to show in logs (optional)
LOG_LEVEL=INFO

# Where your services run
REDIS_URL=redis://app:replace-with-app-password@localhost:6379/0
QDRANT_URL=http://localhost:6333

Quick Start

Start from cp .env.example .env, then replace the API key and all three Redis password placeholders with strong, distinct values. Keep the password text synchronized with the corresponding Redis URLs in that file, then run just dev.

The sections below explain each setting in detail. You can skip to the Example .env File for a complete reference.

Settings Most People Change

These are the settings you will likely want to adjust first.

API Keys and Access Control

Variable	Default	Description
API_KEY_CLIENTS	{}	JSON object mapping valid API keys to client ids for authentication
SEARCH_TRACE_ENABLED	true	Global switch for the durable search debug trace. On, every /search writes one bounded search_trace record to the sink; off, nothing is built or written. See Search debug trace.
RATE_LIMIT	30	Requests per minute per resolved client id for protected routes. See Rate Limiting guide for details.
AUTH_FAILURE_RATE_LIMIT	10	Failed authentication attempts per minute per source address before a request can resolve to a client
DEBUG	false	Enable debug mode (exposes /docs and /redoc)
CORS_ORIGINS	["http://localhost:3000"]	Allowed CORS origins

The API_KEY_CLIENTS variable is required. Every request to the API must include one configured key in the X-API-Key header. This setting is parsed as a JSON object where each key maps to the client id that owns the request data. An empty object rejects every API key.

Set DEBUG=true only in development or trusted environments. It exposes /docs for interactive Swagger UI requests and /redoc for a cleaner read-only API reference. Keep DEBUG=false in production unless you intentionally want those documentation endpoints available.

Why Multiple Keys?

You might give different keys to different teams or applications. This lets you rotate keys later without breaking everything at once.

SEARCH_TRACE_ENABLED defaults to true. While it is on, every /search request, for every API key, writes one bounded search_trace record — the scope and requested limit the search ran with, the raw query, the corrected query, the top scored results with document and page, and per-stage timings, tagged with the request id and the resolved client id (never the API key) — to the durable sink. The record never reaches the ordinary application log, which keeps today's redacted line. Set it to false to roll back: no record is built or written anywhere. See Search debug trace for the record shape, its bounds and the 30-day retention sizing.

Client isolation

Each API key maps to a client_id that owns the data it ingests. Every request — embed, search, update, delete, list — is automatically scoped to that client. Documents from different clients never mix, even when two clients use the same document_id. You do not pass client_id in your requests; the server resolves it from your API key.

If API_KEY_CLIENTS is empty or missing, the server rejects every request with 401 and logs a loud startup warning. This is intentional: the system fails closed rather than serving unscoped data.

For details on how client scoping affects search filters — including why client_id cannot be used as a filter key — see the Search Filters guide.

Example API Key Clients

# Development - simple keys are fine
API_KEY_CLIENTS={"test-key-123":"demo-client","another-key":"demo-client","super-secret-key":"demo-client"}

# Production - use strong random keys and stable client ids
API_KEY_CLIENTS={"prod-key-REPLACE_WITH_RANDOM_HEX":"prod-client"}

# Production with the QA-only key (keep it last so the Console and the backfill
# defaults still resolve a customer key)
API_KEY_CLIENTS={"prod-key-REPLACE_WITH_RANDOM_HEX":"prod-client","<generated-qa-key>":"qa-client"}

The optional qa-client entry is a QA-only client used for release live checks and isolation runs; it is owned by the captain and must never serve real users. Its generate, rotate, and revoke procedure is in the QA-Only API Key section of the Security Hardening guide.

Logging Level

Variable	Default	Description
LOG_LEVEL	INFO	Logging level (DEBUG, INFO, WARNING, ERROR)

Set this to DEBUG when troubleshooting. Use INFO for normal operation. Use WARNING or ERROR to reduce log noise in production.

The transport/client loggers urllib3, fastembed, httpcore, and httpx remain at WARNING or above even when the application root logger is configured for DEBUG, keeping third-party request chatter out of normal output.

Optional tracing and retained logs

Variable	Default	Description
TRACING_ENABLED	false	Enable API, search, transport and Celery spans. Requires the optional tracing dependency extra.
TRACING_EXTRA	empty	Compose/Docker build argument: set to 1 to include that extra in API and worker images. Empty keeps default images unchanged.
OTEL_SERVICE_NAME	infoconnect	Trace service name; resources also identify the api or worker component.
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT	http://localhost:4318/v1/traces	Full OTLP HTTP/protobuf trace endpoint, reachable from every enabled API/worker process.
TRACING_SAMPLE_RATIO	1.0	Root trace sample ratio, from 0 to 1; descendants honor their parent's decision.
OTEL_EXPORTER_OTLP_TRACES_HEADERS	empty	Optional collector authentication headers, read by the OTLP exporter. Never log their values.
OTEL_EXPORTER_OTLP_TRACES_CERTIFICATE	empty	Optional CA certificate path, read by the exporter. Mount the certificate into enabled containers.
DOCKER_LOG_MAX_SIZE	100m	Production json-file rotation size per file.
DOCKER_LOG_MAX_FILE	20	Production retained file count per container, including Console and infrastructure.
SEARCH_TRACE_LOG_PATH	/var/log/infoconnect (Compose)	Container directory of the durable search trace sink. Traces append to search-trace.jsonl there, rotated daily and pruned after SEARCH_TRACE_RETENTION_DAYS. Empty (the default outside Compose) leaves the sink unconfigured, so no trace is written.
SEARCH_TRACE_RETENTION_DAYS	30	Daily trace files kept in the durable sink, so the trace horizon is 30 days.
SEARCH_TRACE_LOG_HOST_DIR	./.runtime/search-trace	Compose host directory mounted at SEARCH_TRACE_LOG_PATH on the API service, so the sink survives container recreation and redeploys.

See request lookup, retention and tracing operations for startup behavior, local setup and collector requirements.

Search and Document Settings

These settings control how documents are split and how search results are returned.

Document Chunking

Variable	Default	Description
CHUNK_SIZE	512	Approximate tokens per chunk
CHUNK_OVERLAP	50	Token overlap between chunks

When you upload a document, the InfoConnect Hybrid Search Engine splits it into smaller pieces called chunks. This helps the search work better. Larger chunks mean more context but slower processing. Smaller chunks mean more precise matches but less context.

Choosing Chunk Size

Think of chunks like pages in a book. If your documents are short emails, use small chunks (256). If they are long reports, use larger chunks (1024). The default 512 works well for most documents.

Chunk Size Guidelines

Use Case	Chunk Size	Overlap
Short documents (emails)	256	25
Standard documents	512	50
Long documents (reports)	1024	100

Search Results

Variable	Default	Description
RETRIEVAL_TOP_K	10	Default search results count
HYBRID_ALPHA	0.5	Default dense weight for hybrid search; supported range is 0.0 to 1.0
SEARCH_DENSE_SCORE_FLOOR	0.70	Minimum raw cosine score for a dense candidate
SEARCH_SPARSE_SCORE_FLOOR	1.0	Minimum raw BM25 dot-product score for a sparse candidate
SEARCH_TOTAL_COUNT_CAP	1000	Maximum score-gated count scan before reporting a lower bound; /search totals are independently bounded to the 200-result retrieval window
RERANK_TOP_N	5	Number of results to rerank
RERANK_TIMEOUT_SEC	2.0	Caller deadline for waiting for the inference slot and reranker scores; on timeout, retain retrieval order
MAX_NODES_PER_TASK	50	Max nodes per search task

Reranker text extraction and score generation run in a blocking worker. The deadline does not cancel native inference already running: the shared SearchService keeps its inference slot occupied until that work finishes, so later requests cannot accumulate native jobs.

RETRIEVAL_TOP_K controls how many results the search returns by default. HYBRID_ALPHA sets the deployment-wide dense/sparse balance; an explicit /search alpha value overrides it for that request. Start with HYBRID_ALPHA=0.5. Lower values favor sparse/lexical matches; higher values favor dense/semantic matches.

The score floors gate raw dense and sparse candidates before hybrid fusion. The recommended defaults are SEARCH_DENSE_SCORE_FLOOR=0.70 and SEARCH_SPARSE_SCORE_FLOOR=1.0; a candidate is retained when either active stream clears its floor. Raising a floor can remove low-confidence noise, but setting it too high over-filters genuine matches and can produce no results. Keep gibberish and other queries with no confident match in the genuine no-results state. Recalibrate the floors when the embedding model or corpus distribution changes. See the Embeddings and Retrieval guide for scope selection and the caller-facing Evidence Result contract.

Content is the default search scope; callers should send it explicitly. Every scope requires active publication authority. Metadata and Both additionally require per-Client Discovery Readiness. Metadata semantic eligibility uses the immutable 0.75 dense floor for BAAI/bge-small-en-v1.5; callers cannot override that floor.

Web Search (Chat)

Optional public web search is controlled by a deployment capability gate and the Chat turn's Answer Mode. With the gate on, Auto permits web only for live, time-varying questions, Docs-only omits the web tool, and Web requires a web-search pass. With the gate off, Auto runs as Docs-only and an explicit Web turn is rejected; a browser preference cannot override deployment policy.

Variable	Default	Description
WEB_SEARCH_ENABLED	false	Deployment capability gate. When false, Auto runs Docs-only and explicit Web turns are rejected
WEB_SEARCH_PROVIDER	ddgs	Web search provider; only ddgs is supported
WEB_SEARCH_DDGS_BACKEND	bing	DDGS backend; allowed values are bing, google, auto, and duckduckgo
WEB_SEARCH_DDGS_TIMEOUT_SEC	5.0	Provider request timeout
WEB_SEARCH_TIMEOUT_SEC	5.5	Hard wall-clock deadline for the whole web branch

Web retrieval is best-effort: a provider error, timeout, or disconnect drops the web results and never breaks the document-backed answer. Only the exact submitted query is sent to the provider, and result URLs are validated against SSRF host rules (including IPv6 reserved ranges) before they reach the model.

bing is the default because keyless DuckDuckGo requests can be bot-blocked. Allowing a small backend set makes availability less predictable than a single fixed egress target; operators that require one predictable destination should set WEB_SEARCH_DDGS_BACKEND explicitly.

Enabling WEB_SEARCH_ENABLED is not the same as selecting Chat Web Answer Mode. The gate is a server capability; Answer Mode is a per-turn Console choice that cannot override it. After you change the variable, recreate API containers (just prod-up) — docker compose restart will not pick it up. See Web search enablement vs Chat Answer Mode.

OCR and Classification Settings

These settings control how the InfoConnect Hybrid Search Engine reads images and categorizes documents.

Document Classification

Variable	Default	Description
CLASSIFICATION_ENABLED	true	Enable automatic document classification
CLASSIFIER_BACKEND	modernbert-onnx	Classification backend selector

When you upload a document, the InfoConnect Hybrid Search Engine can automatically figure out what category it belongs to. This helps organize your documents.

Model Roles

RAPIDOCR_MODEL_NAME is used for OCR during parsing. ModernBERT ONNX is the only supported classification backend.

These model values reflect the repository's current .env.example / Docker mapping.

Upgrading with customized model IDs

The model selectors and artifact path below are a commercial-use allowlist, not deployment tuning choices. Before upgrading an existing deployment, compare its .env with these seven approved values and restore any customized entries:

SPACY_MODEL_NAME=en_core_web_sm
DENSE_MODEL_NAME=BAAI/bge-small-en-v1.5
SPARSE_MODEL_NAME=Qdrant/bm25
RERANK_MODEL_NAME=Xenova/ms-marco-MiniLM-L-6-v2
RAPIDOCR_MODEL_NAME=rapidocr-onnxruntime
MODERNBERT_ONNX_MODEL_REPO=onnx-community/ModernBERT-base-nli-ONNX
MODERNBERT_ONNX_MODEL_FILE=onnx/model_quantized.onnx

An unapproved value makes settings validation fail during API and worker import, so the affected service does not start. This fail-closed behavior is intentional; adding a model requires code and compliance-policy review.

OCR for Images and PDFs

Variable	Default	Description
RAPIDOCR_OCR_ENABLED	true	Master switch to enable/disable OCR
RAPIDOCR_MODEL_NAME	rapidocr-onnxruntime	OCR model for images and low-text PDF pages
OCR_PAGE_TEXT_THRESHOLD	80	Min characters before OCR fallback

If you upload an image or a scanned PDF, the InfoConnect Hybrid Search Engine uses RapidOCR to read the text. The threshold setting controls when it decides a PDF page needs OCR help.

Custom Categories

Variable	Default	Description
ALLOWED_CATEGORIES	["financial reporting","risk management","corporate governance","regulatory compliance","business operations","market analysis","legal","human capital","technology","sustainability","KYC","Credit","Collateral","Ledger","other"]	JSON array of category labels scored by ModernBERT
CATEGORY_DESCRIPTIONS	KYC/Credit/Collateral/Ledger descriptions	Optional JSON object mapping category labels to richer scoring text. Missing descriptions fall back to the category label.

Default Categories

Current defaults come from app/config.py and use the ModernBERT financial / corporate taxonomy:

["financial reporting","risk management","corporate governance","regulatory compliance","business operations","market analysis","legal","human capital","technology","sustainability","KYC","Credit","Collateral","Ledger","other"]

These settings are also parsed as JSON arrays. If you customize them, keep the JSON syntax:

ALLOWED_CATEGORIES=["financial reporting","technology","other"]

You can customize these to match your business, but the classifier still expects JSON arrays rather than plain delimited strings.

CATEGORY_DESCRIPTIONS is optional. You do not need to provide a description for every item in ALLOWED_CATEGORIES. If a category has no matching description, the InfoConnect Hybrid Search Engine falls back to the label text itself. For example, Ledger without a description is scored as Ledger.

Descriptions are strongly recommended for short, ambiguous, acronym-based, or client-specific labels. Labels such as KYC, Credit, Collateral, and Ledger often need richer text because real documents may say Kad Pengenalan, bankruptcy search, pledged asset, or journal entry instead of the exact label.

ALLOWED_CATEGORIES=["KYC","Credit","Collateral","Ledger"]
CATEGORY_DESCRIPTIONS={"KYC":"know your customer, identity verification, customer due diligence, IC, passport, Kad Pengenalan, Warganegara, onboarding, or beneficial ownership documents","Credit":"credit reports, bankruptcy searches, insolvency checks, lending, repayment history, or credit risk documents"}

You can describe only the categories that need extra context. The API still returns the configured category label, such as KYC; descriptions are used only for scoring and strong lexical evidence matching.

ModernBERT ONNX Settings

Variable	Default	Description
MODERNBERT_ONNX_MODEL_REPO	onnx-community/ModernBERT-base-nli-ONNX	Hugging Face repo for the ONNX classifier
MODERNBERT_ONNX_MODEL_FILE	onnx/model_quantized.onnx	ONNX model file loaded from the repo
MODERNBERT_ONNX_CATEGORY_TEMPLATE	The topic of this document is {label}.	Hypothesis template for category scoring
MODERNBERT_ONNX_CATEGORY_THRESHOLD	0.45	Minimum score required to keep a category
MODERNBERT_ONNX_LONG_DOC_WINDOW	1800	Sliding window size (tokens) for documents exceeding model context
MODERNBERT_ONNX_LONG_DOC_OVERLAP	256	Token overlap between consecutive windows
MODERNBERT_ONNX_TOP_K_CHUNKS	2	Max additional chunks to score after the first (long-doc optimization)
MODERNBERT_ONNX_NLI_MAX_TOKENS	2048	Maximum token length for NLI input sequences (premise + hypothesis). Lower values reduce inference time at the cost of truncating long document text
CLASSIFICATION_SELECT_TOP_K	3	Number of representative chunks selected via centroid + MMR. Classification scores these chunks instead of the full document text.
MODERNBERT_ONNX_INTRA_OP_THREADS	1	ONNX Runtime intra-op thread count
MODERNBERT_ONNX_INTER_OP_THREADS	1	ONNX Runtime inter-op thread count
MODERNBERT_PREFILTER_ENABLED	false	Enable lexical prefilter to reduce hypothesis space before ONNX inference
MODERNBERT_PREFILTER_TOP_K	3	Number of top categories to keep after lexical prefilter

Model Repo Compatibility References (MODERNBERT_ONNX_MODEL_REPO)

The classifier requires a 3-class NLI model outputting entailment (index 0), neutral (index 1), and contradiction (index 2). Only the approved repo below is runnable; the other row is an upstream compatibility reference, not a selectable deployment value.

Repo	Size	NLI Classes	Status
onnx-community/ModernBERT-base-nli-ONNX	Base (768d, 22 layers)	3 (entailment / neutral / contradiction)	Approved runnable value
onnx-community/ModernBERT-large-zeroshot-v2.0-ONNX	Large (1024d, 28 layers)	2 (entailment / not_entailment)	Rejected by startup allowlist; also incompatible

Non-runnable reference

Do not set MODERNBERT_ONNX_MODEL_REPO to ModernBERT-large-zeroshot-v2.0-ONNX. Settings validation rejects it before service startup. Its 2-class output would also be incompatible with the pipeline's required three NLI logits.

Upstream Model File References (MODERNBERT_ONNX_MODEL_FILE)

These files exist in the upstream repository, but only onnx/model_quantized.onnx is a runnable value in the current commercial-use allowlist. Every other row below is a non-runnable reference: selecting it through an environment override fails settings validation during service import. Adding another artifact requires a code and policy change with license evidence and review.

File	Precision	Size	Notes
onnx/model.onnx	fp32	~599 MB	Rejected by startup allowlist; reference only
onnx/model_fp16.onnx	fp16	~300 MB	Rejected by startup allowlist; reference only
onnx/model_quantized.onnx	int8	~151 MB	Approved runnable value
onnx/model_int8.onnx	int8	~151 MB	Rejected by startup allowlist; reference only
onnx/model_uint8.onnx	uint8	~151 MB	Rejected by startup allowlist; reference only
onnx/model_q4.onnx	4-bit	~225 MB	Rejected by startup allowlist; reference only
onnx/model_q4f16.onnx	4-bit/fp16	~140 MB	Rejected by startup allowlist; reference only
onnx/model_bnb4.onnx	BNB 4-bit	~218 MB	Rejected by startup allowlist; reference only

Approved model file

Use onnx/model_quantized.onnx for current deployments. Other upstream artifacts are reference options only until explicitly reviewed and added to the commercial-use allowlist.

Classification Order

The historical publication-disabled path classified asynchronously after embedding and read chunk embeddings from Qdrant; it is no longer bootable because search requires publication. With publication enabled, representative-chunk selection and scoring complete before the immutable generation's embedding chord is dispatched. See Architecture: Post-Embedding Classification for the trigger, retry, and replay contract.

note

Thread counts default to 1 across all environments. Increase MODERNBERT_ONNX_INTRA_OP_THREADS for CPU-bound preprocessing workers if your deployment has spare cores.

Named Entity Recognition

Variable	Default	Description
SPACY_NER_ENABLED	true	Enable spaCy NER extraction
NER_ENTITY_KEYS	See below	Entity types to extract and store

The InfoConnect Hybrid Search Engine can find and extract important information from your documents. This includes names, dates, amounts of money, and more.

Default Entity Keys

The following entity types are extracted and stored:

• persons — People names
• organizations — Companies, agencies
• dates — Date and time references
• locations — Places, countries, cities
• monetary_amounts — Currency values (regex-enhanced)
• account_numbers — Account identifiers (regex)
• transaction_refs — Transaction references (regex)
• account_types — Account types (regex)

Worker and Performance Settings

These settings control background processing and resource usage.

Celery Worker Settings

Variable	Default	Description
CELERY_WORKER_PREFETCH_MULTIPLIER	1	Tasks prefetched per worker
CELERY_WORKER_REVISION	unknown	Deployment marker used by the startup worker-isolation guard
CELERY_TASK_ACKS_LATE	true	Acknowledge after task completion
CELERY_WORKER_DEDUPLICATE_SUCCESSFUL_TASKS	true	Suppress redelivery of successful late-ack tasks using the persistent result backend
CELERY_BROKER_VISIBILITY_TIMEOUT_SEC	3600	Redis broker visibility timeout; publication reconciliation retries are kept below this value
CELERY_WORKER_MAX_MEMORY_PER_CHILD	1572864 (1.5GB)	Memory limit per worker (KB)
CELERY_WORKER_MAX_TASKS_PER_CHILD	100	Tasks before worker restart
CELERY_WORKER_MAX_TASKS_PER_CHILD_EMBEDDING	250	Embedding-worker task recycle target from config.py
CELERY_WORKER_MAX_TASKS_PER_CHILD_PREPROCESSING	500	Preprocessing-worker task recycle target from config.py
CELERY_TASK_MAX_RETRIES	3	Max retry attempts for failed tasks
CELERY_RETRY_BACKOFF_BASE	60	Base seconds for exponential backoff
CELERY_RESULT_EXPIRES	86400 (24h)	Result retention time (seconds)
CELERY_EMBED_SOFT_TIME_LIMIT_SEC	900	Soft time limit for embedding tasks
CELERY_EMBED_TIME_LIMIT_SEC	1200	Hard time limit for embedding tasks
CELERY_TASK_REJECT_ON_WORKER_LOST	true	Requeue task if a worker exits unexpectedly
CELERY_QUEUE_SPIKE_THRESHOLD	500	Queue depth that triggers a sampled celery_queue_depth_spike audit event
MEMORY_PRESSURE_WARN_PERCENT	85	cgroup memory usage percentage that starts a sampled pressure warning
MEMORY_PRESSURE_RECOVER_PERCENT	75	cgroup memory usage percentage below which pressure clears
OPS_MONITOR_INTERVAL_SECONDS	60	Sampling interval for queue-depth and memory-pressure monitors

What Are Workers?

Workers are background processes that handle document processing. When you upload a file, the API accepts it immediately. Then workers do the heavy lifting: parsing, chunking, embedding, and storing. This happens in the background so the API stays fast.

Workers automatically restart after processing a certain number of tasks or using too much memory. This prevents memory leaks from slowing things down.

just dev and just prod-up set CELERY_WORKER_REVISION to the current 12-character Git revision and include that value in every worker node name. If you start the API or workers manually, use the same valid revision marker for all processes and name workers as preprocessing-<revision>@<host> or embedding-<number>-<revision>@<host>.

A Celery worker revision conflict detected before startup. line during just prod-up means a worker from another revision is still on the shared broker (stderr also includes Expected deployment revision: and Offending worker node(s):). Recover with just prod-down (data preserved), not just prod-down clean or just wipe. See Celery worker revision conflict recovery.

Code defaults vs Docker Compose defaults

app/config.py keeps the general CELERY_WORKER_MAX_TASKS_PER_CHILD default at 100, but docker-compose.yml overrides the live worker values to 250 for embedding workers and 500 for the preprocessing worker.

app/config.py defaults CELERY_WORKER_MAX_MEMORY_PER_CHILD to 1572864 KB (about 1.5 GB). The preprocessing worker in docker-compose.yml can override this via PROD_PREPROCESSING_MAX_MEMORY_PER_CHILD_KB (default 4194304 KB / 4 GB) to match its larger container memory limit.

Docker / Runtime-only Overrides

Variable	Default	Description
PROD_PREPROCESSING_MEMORY	6G	Docker Compose memory limit for worker-preprocessing-1

This variable is only used by docker-compose.yml at runtime. It does not exist in app/config.py, but it controls the container memory limit for the preprocessing worker in production-style Docker runs.

Redis Configuration

Variable	Default	Description
REDIS_URL	redis://localhost:6379/0	Application-state connection using the app ACL user in deployed Compose
CELERY_BROKER_URL	redis://localhost:6379/1	Celery task queue using the celery ACL user
CELERY_RESULT_BACKEND	redis://localhost:6379/2	Celery results store using the celery ACL user
PUBLICATION_ENABLED	true	Search serves exclusively through the fenced four-lane protocol with Qdrant Manifest read authority; the API fails fast at startup when this is disabled or the publication Redis contract is unusable
PUBLICATION_REDIS_URL	redis://publication:<password>@localhost:6379/3	Dedicated DB 3 connection using the prefix-limited publication ACL user; the default carries a placeholder credential that must be replaced
PUBLICATION_WORKER_SLOT	api-1	Stable deployment-provided search-process slot; enabled startup rejects missing or duplicate slots. Production Compose pins the API service to api-1.
PUBLICATION_WAITAOF_TIMEOUT_MS	5000	Bounded local AOF fsync barrier timeout
PUBLICATION_LEASE_SECONDS	15	Redis-server-time worker lease
PUBLICATION_RENEW_SECONDS	5	Worker lease renewal interval
PUBLICATION_SAFE_DEADLINE_SECONDS	10	Monotonic local deadline after which a disconnected worker stops serving
PUBLICATION_LIFECYCLE_FENCE_WAIT_SECONDS	5	Wait bound in seconds for each Soft Delete/Restore snapshot read blocked by a sibling fence or snapshot movement; accepts 0–30. See the Documents API lifecycle contract
PUBLICATION_FINALIZE_RETRY_BUDGET_SECONDS	300	No-progress budget for jittered finalize retries; each completed publication renews the deadline for a healthy serialized backlog
QA_PUBLICATION_ACTIVATION_DELAY_SECONDS	0	Test-only publication ACTIVATED hold (seconds) for isolated QA stacks: widens the public contract-503 window so the stranded-worker kill lands inside it. Keep 0 in production, dev, and normal QA runs
PUBLICATION_EXPECTED_REDIS_VERSION	7.4.10	Qualified Redis runtime version required for publication readiness
PUBLICATION_REDIS_MEMORY_MB	512	Numeric Redis container-memory budget; must be positive and must match the budget the Redis server derives its maxmemory from. just prod-up generates it from the host plan
REDIS_MEMORY_HEADROOM_PERCENT	25	Capacity reserved before deriving Redis maxmemory
REDIS_MAXMEMORY	empty	Redis memory ceiling; just prod-up emits it from the shared host-wide runtime plan. When unset, development Compose derives it from PUBLICATION_REDIS_MEMORY_MB
PROD_API_WORKERS	generated (1)	Uvicorn worker count emitted by the production runtime planner; publication-enabled API containers must keep this at one
PROD_REDIS_MEMORY	generated	Redis container limit emitted by the production runtime planner

The deployed Redis uses separate app, celery, and publication ACL credentials. Provide REDIS_APP_PASSWORD, REDIS_CELERY_PASSWORD, and REDIS_PUBLICATION_PASSWORD through the deployment secret mechanism. Do not reuse them. Use URL-safe password characters because Compose injects the same value into both the ACL file and connection URI. The publication user can read and write pub:v1:{publication}:*. It also has the narrow, read-only %R~docjobs:* access and SMEMBERS command needed by the one-time legacy publication bootstrap to inspect document-job history in DB 2; it cannot scan or mutate Celery keys. The default user is disabled for network authentication but retains command capability so Redis can replay the multipart AOF during startup; a derived, unpublished password prevents accidental use.

Changing REDIS_APP_PASSWORD, REDIS_CELERY_PASSWORD, or REDIS_PUBLICATION_PASSWORD requires recreating Redis and its dependent API and worker containers — the ACL file is generated at container start, and those services receive interpolated credentials only when they are recreated. Use just prod-down followed by just prod-up to rotate the stack. Force-recreating redis alone is insufficient because API and workers would retain their old passwords.

Redis Database Usage

DB	Purpose
0	Application state and bounded caches
1	Celery task broker
2	Celery results backend
3	Durable publication registry, journal, worker roster, and ready state

Logical databases isolate names, not memory, AOF, rewrite load, or process failure. Publication readiness also requires the digest-pinned Redis 7.4.10 build, healthy AOF, appendfsync everysec, no-appendfsync-on-rewrite no, aof-load-truncated no, and noeviction.

While PUBLICATION_ENABLED=true, metadata update and re-embed operations return retryable 503 errors instead of mutating Qdrant outside the generation protocol. Disabling the flag keeps those legacy mutation paths available but makes search unavailable because there is no active publication authority. Soft delete, hard delete, and restore run through the generation protocol; see Document endpoints for their behavior.

Publication mode also requires CELERY_RESULT_BACKEND to use persistent Redis storage, CELERY_TASK_ACKS_LATE=true, and CELERY_WORKER_DEDUPLICATE_SUCCESSFUL_TASKS=true. Startup fails closed when these delivery guarantees are unavailable. Keep the result retention longer than the publication recovery window so a successful parent task remains durable evidence against redelivery.

Qdrant Configuration

Variable	Default	Description
QDRANT_URL	http://localhost:6333	Qdrant server URL
QDRANT_TIMEOUT_SECONDS	60	Qdrant client request deadline in seconds (integer, 1–600); see Hard Delete timeout handling for verification and retry behavior
COLLECTION_NAME	documents	Vector collection name
VECTOR_SIZE	384	Embedding dimensions (must match model)

Qdrant is the vector database that stores your document embeddings. This is what makes semantic search possible.

Embedding Models

Variable	Default	Description
DENSE_MODEL_NAME	BAAI/bge-small-en-v1.5	Dense embedding model
SPARSE_MODEL_NAME	Qdrant/bm25	Sparse/BM25 model
RERANK_MODEL_NAME	Xenova/ms-marco-MiniLM-L-6-v2	FastEmbed cross-encoder reranker
EMBEDDING_BATCH_SIZE	32	Embeddings per batch
FASTEMBED_CACHE_DIR	./models/fastembed	Model cache location

Embedding models turn text into numbers that capture meaning. The InfoConnect Hybrid Search Engine uses two types: dense (for semantic meaning) and sparse (for keyword matching). Together they give better search results.

The listed model identifiers are the reviewed commercial-use defaults. Arbitrary environment overrides are rejected; changing a model requires updating the runtime and compliance allowlists.

Upstream Dense Model References

Model	Dimensions	Speed	Quality	Commercial allowlist
BAAI/bge-small-en-v1.5	384	Fast	Good	Approved default
BAAI/bge-base-en-v1.5	768	Medium	Rejected by startup allowlist; reference only
BAAI/bge-large-en-v1.5	1024	Slow	Rejected by startup allowlist; reference only
sentence-transformers/all-MiniLM-L6-v2	384	Fast	Rejected by startup allowlist; reference only

About Model Size

Larger models may improve quality but use more memory and run slower. They are reference options, not runtime configuration choices, until their licenses and artifacts are reviewed and allowlisted.

File Upload Security

Variable	Default	Description
MAX_FILE_SIZE_MB	200	Maximum upload file size in MB
ALLOWED_EXTENSIONS	.pdf,.docx,.txt,.md,.text,.pptx,.ppt,.xlsx,.xls,.png,.jpg,.jpeg,.tif,.tiff	Permitted file extensions
ALLOWED_MIME_TYPES	See example below	Permitted upload MIME types mapped to extensions
UPLOAD_DIR	./models/uploads (/tmp/ic-uploads in Docker examples)	Local staging directory for files before Celery processing
OFFICE_LEGACY_CONVERSION_ENABLED	false	Enable LibreOffice conversion of legacy .ppt/.xls to OOXML before parsing
LIBREOFFICE_BINARY	soffice	LibreOffice executable used for legacy Office conversion
OFFICE_LEGACY_CONVERSION_TIMEOUT_SEC	120	Timeout for a single legacy Office conversion
OFFICE_MAX_UNCOMPRESSED_BYTES	1000000000	Max decompressed size of an OOXML (.docx/.pptx/.xlsx) upload (zip-bomb guard)
OFFICE_MAX_COMPRESSION_RATIO	200	Max uncompressed:compressed ratio for an OOXML upload (zip-bomb guard)

These settings protect your server from oversized uploads and unexpected file types. A file must pass the extension check, declared MIME mapping check when a known MIME type is provided, and sniffed-content check before it is enqueued.

UPLOAD_DIR is especially important in Docker deployments. The API writes the uploaded file there first, and the workers read it from the same path. Publication-enabled processing also stores its replayable classification dispatch outbox beside the staged upload. If you change UPLOAD_DIR, mount the same shared volume at that location in the API and every preprocessing and embedding worker container.

Production Runtime Threading

Variable	Default	Description
PROD_EMBEDDING_OMP_THREADS	1	Auto-tuned OpenMP/BLAS thread count for embedding worker containers
OMP_NUM_THREADS	From PROD_EMBEDDING_OMP_THREADS in Docker Compose	OpenMP thread cap inside embedding workers
OPENBLAS_NUM_THREADS	From PROD_EMBEDDING_OMP_THREADS in Docker Compose	OpenBLAS thread cap inside embedding workers
MKL_NUM_THREADS	From PROD_EMBEDDING_OMP_THREADS in Docker Compose	MKL thread cap inside embedding workers

scripts/generate_prod_runtime.py writes PROD_EMBEDDING_OMP_THREADS to .runtime/prod.env; Docker Compose passes it through to the lower-level numerical libraries. Keep this at 1 unless benchmark evidence shows spare CPU headroom.

Model Cache Paths

Variable	Default	Description
FASTEMBED_CACHE_DIR	./models/fastembed	FastEmbed model cache
HF_HOME	./models/huggingface (/models/huggingface in Docker Compose)	Hugging Face cache
HF_TOKEN	(unset)	Optional Hugging Face token for gated or rate-limited model downloads
SPACY_MODEL_NAME	en_core_web_sm	spaCy NER model

These control where approved models are stored. You usually do not need to change them. Set HF_TOKEN only when downloads of the approved Hugging Face repository are rate-limited; switching repositories or spaCy models requires a reviewed code and compliance-policy change.

Example .env File

Copy the authoritative, runnable template instead of maintaining another settings list here:

cp .env.example .env

The tables above explain the supported settings; .env.example owns the complete example and tracks deployment-required values such as Redis credentials and Celery visibility timeouts.

Configuration Best Practices

Recommended Settings for Production

DEBUG=false
LOG_LEVEL=INFO
RATE_LIMIT=30
API_KEY_CLIENTS={"REPLACE_WITH_A_STRONG_RANDOM_KEY":"prod-client"}
CELERY_WORKER_MAX_TASKS_PER_CHILD=100
CELERY_WORKER_MAX_MEMORY_PER_CHILD=1572864  # 1.5GB

Docker Compose

Environment variables are automatically loaded from .env when using:

just prod-up

just prod-up also writes .runtime/prod.env with computed service memory ceilings, Redis headroom, and Celery worker scale targets so operators can inspect the chosen runtime plan.

Or set in docker-compose.yml:

services:
  api:
    environment:
      - API_KEY_CLIENTS=${API_KEY_CLIENTS}
      - LOG_LEVEL=INFO

Testing Your Config

For local just dev, apply .env changes by restarting the process loop:

just stop
just dev

For production Compose, docker compose restart does not re-read .env. Recreate with just prod-up and just prod-ui-up. See Why docker compose restart does not pick up .env.

What's Next

Now that you understand the configuration options, you can:

• Follow the Quick Start Guide to get your first document uploaded
• Read the Architecture Overview to understand how everything fits together
• Review the Justfile Commands for common development tasks
• Follow Production and Staging Deploy and Recovery for tagged deploys and .env recreate rules

If something is not working as expected, check the logs with LOG_LEVEL=DEBUG and review Troubleshooting and FAQ or the installation troubleshooting section.