Skip to main content

CLI Tools

Most command-line workflows should go through just. The recipes wrap the maintained scripts and preserve shell quoting for JSON arguments.

Authentication​

The search and embed scripts read INFOCONNECT_API_KEY when no explicit API key is passed.

export INFOCONNECT_API_KEY=super-secret-key

Use this for local terminals and CI jobs where the API key is already provided by the environment.

Search from the terminal​

just search sends an explicit scope on every request. The API's canonical values are content (the default), metadata, and both, and the API accepts any letter case. The current CLI option exposes the accepted Content, Metadata, and Both spellings shown below; metadata and both are opt-in and additionally require Discovery Readiness for the caller's Client. Every scope requires active publication authority.

# Concise Content summary
just search "quarterly risk report" -- --scope Content

# Full Evidence Result JSON
just search "quarterly risk report" -- --scope Content --json

# Metadata evidence (requires a ready Client)
just search "Apollo" -- --scope Metadata --json

# Both evidence lanes with exact filters
just search "compliance" -- --scope Both --filter document_id=doc-123 --rerank --json

The command uses the same caller controls as POST /search: explicit scope, exact filters, pagination, alpha, and optional reranking. It does not request field selection or a caller-supplied score threshold.

The concise view renders each Evidence Result as a Document Version with its rank, document and version identity, lane-local retrieval contributions, and nested Content/Metadata Evidence Set counts and previews. Values are rendered as untrusted text.

Scope: Content
Query: quarterly risk report
Results: 10
Total Document Versions: 20
Query Time: 16 ms
Top result: malaysia-wiki-001 (unversioned)
  content: 1/1
  metadata: 0/0

Use --json when piping or debugging. JSON output preserves the complete nested document_version, canonical_metadata, retrieval_contributions, content_evidence, and metadata_evidence objects, including each set's items, total, and truncated values. Pagination metadata remains intact.

Embed files and folders​

just embed ./documents
just embed ./documents/report.pdf
just embed ./documents --max-pages 5
just embed ./documents --metadata '{"department":"Finance"}'

just embed calls scripts/embed_folder.py, uploads supported files (PDF, DOCX, PPTX, XLSX, TXT, MD, and images) through the public /embed API, and polls /status/{job_id}. Legacy PPT/XLS conversion is also supported when LibreOffice conversion is explicitly enabled. The command uses runtime concurrency from .runtime/prod.env when available; pass --concurrency to override it for a one-off run.

Monitor services​

just status
just status live
just status docker
just status local

Use just status live while a batch ingest is running. It shows API health, Redis/Qdrant readiness, worker state, queue counts, resource pressure, and recent alerts in a Rich terminal dashboard.

Operational scripts​

ScriptPurpose
scripts/download_models.pyPre-cache FastEmbed, spaCy, reranker, and ModernBERT assets before first boot or for offline deployments
scripts/backfill_qdrant_fuzzy_vectors.pyBackfill the client-scoped spelling vocabulary for documents embedded before typo correction was enabled
scripts/bootstrap_legacy_publication.pyOne-time pre-cutover Legacy Publication: admit existing visible Document Versions into generation-gated search
scripts/backfill_metadata_discovery.pyPost-cutover Metadata Backfill: create Metadata Leaves for accepted Document Versions so a client reaches Discovery Readiness
scripts/backfill_embedding_version.pyBackfill filterable version stamps for legacy and v1-signature embedding points
scripts/reembed_documents.pySelect Document Versions whose vectors predate the running embedder and enqueue re-embedding from stored chunk text
scripts/evaluate_retrieval.pyCompare BM25, hybrid, and hybrid+rerank retrieval quality
scripts/generate_ndcg_toy_dataset.pyCreate a small graded retrieval dataset from existing chunks
scripts/status_dashboard.pyLive container, worker, queue, and readiness dashboard
scripts/failed_tasks.pyList individual failed background tasks (id, name, Client, document, error, time) from the worker failure log — just failed-tasks
scripts/benchmark_pr_classification.pyBenchmark ModernBERT classification behavior

Prefer the matching just recipe when one exists. Use scripts directly only for specialized diagnostics or automation.

Backfill embedding versions​

New /embed writes carry an indexed top-level embedding_version and detailed metadata.embedding provenance. Classify older points for one client with the read-only default:

uv run python scripts/backfill_embedding_version.py \
  --client-id client-a \
  --json

Apply mode creates and verifies the INTEGER payload index first. It stamps points without captured provenance as legacy version 0, stamps the known v1 signature as version 1, and leaves non-default captured signatures un-stamped for operator review:

uv run python scripts/backfill_embedding_version.py \
  --client-id client-a \
  --apply \
  --json

Re-embed stale documents​

Selection depends on the publication mode the deployment runs in.

With publication enabled (the default), the driver reads the Content lane and selects live Document Versions whose captured metadata.embedding model identity or embedding_version differs from the running embedder. That makes an embedding-model swap selectable: change DENSE_MODEL_NAME / SPARSE_MODEL_NAME, restart the embedding worker, then re-embed. With publication disabled, it selects pre-publication chunk points whose embedding_version is below the running pipeline version, including unstamped points. Dry-run is the default in both modes:

uv run python scripts/reembed_documents.py \
  --client-id client-a \
  --document-id doc-123 \
  --json

Use --apply to enqueue one embedding-queue task per matching document.

Under publication the task never writes vectors in place. For each targeted Document Version it reads the active generation, re-embeds its stored chunk_text, then stages, verifies, and activates a new immutable generation before scheduling cleanup of the source generation. Content payloads, Soft Delete visibility, Fuzzy terms, and Metadata Leaves are republished unchanged: only Content vectors and metadata.embedding provenance change. Fuzzy vectors are term-derived and Metadata Leaf vectors belong to the immutable Metadata vectorizer identity, so neither is re-embedded. A generation that already carries the running model identity is reported as unchanged and is not republished.

Without publication the task updates vectors in place, then writes provenance and the indexed root stamp. It does not rebuild payloads, rerun classification, or touch the fuzzy-terms collection. Automatic selection excludes documents with captured non-default model or chunker signatures and logs them as re-ingest-only; an explicit --document-id selection refuses them with an instruction to re-ingest from the original source.

In both modes, chunker and extraction lineage must stay compatible: historical chunks without captured lineage are stamped with explicit unknown values, and known lineage must match the running chunker and extraction/OCR pipeline.

uv run python scripts/reembed_documents.py \
  --client-id client-a \
  --document-id doc-123 \
  --apply \
  --json

One embedding model is active per collection, and the Content lane's dense vector size is fixed at collection creation. Swapping to a model with different dimensions needs new collections, not a re-embed. A same-dimension model swap is a full-corpus re-embed: run the driver without --document-id and let every Document Version republish.

Backfill typo-correction vocabulary​

Newly embedded chunks maintain their spelling vocabulary automatically. For existing Qdrant data, run the backfill once per client. Start with its read-only dry run:

uv run python scripts/backfill_qdrant_fuzzy_vectors.py \
  --client-id client-a \
  --json

Before applying, quiesce ingestion and document delete/restore operations for that client. The apply command reconciles the entire client vocabulary, so --limit is intentionally available only for dry runs:

uv run python scripts/backfill_qdrant_fuzzy_vectors.py \
  --client-id client-a \
  --apply \
  --quiesced-ingestion \
  --json

--quiesced-ingestion is an explicit maintenance-window acknowledgement; the script does not pause writers itself. Apply re-reads each source chunk immediately before upserting its terms so a concurrent soft-delete, restore, or hard-delete cannot resurrect hidden vocabulary or create orphan memberships. Use --qdrant-url and --collection-name when the deployment differs from the configured defaults. A nonzero exit reports one or more skipped write batches as errors.

Bootstrap Legacy Publication​

Run this once per environment before the publication-aware release starts, while the old release is still serving traffic. It groups a Client's visible legacy chunks by exact Document Version identity, stamps an explicit generation per version, creates the publication records, and verifies what it admitted. Runtime search never treats a missing generation as active, so a Document Version that is not admitted here stays unavailable after cutover.

Dry-run is the default and writes nothing:

uv run python scripts/bootstrap_legacy_publication.py \
  --client-id client-a \
  --json

The dry-run report gives per-Client counts plus a bounded list of violating Document Version identities (--max-violations, default 20). It reports document_id, version_id, a reason, and remediation only. It never contains Custom Metadata values, and a --client-id run never contains another Client. --all-clients produces an operator-wide report; do not hand that one to a Client. --all-clients also discovers the Clients that actually own legacy chunks in Qdrant, not just the ones in the operator shell's API_KEY_CLIENTS, so a stale or partial API-key configuration cannot silently skip a Client and leave its Document Versions unavailable after cutover.

Quiesce ingestion, then admit and verify:

uv run python scripts/bootstrap_legacy_publication.py \
  --client-id client-a \
  --apply \
  --quiesced-ingestion \
  --json

--quiesced-ingestion is an explicit maintenance-window acknowledgement; the script does not pause writers itself. Re-prove a finished run at any time:

uv run python scripts/bootstrap_legacy_publication.py \
  --client-id client-a \
  --verify-only \
  --json

--verify-only writes nothing at all. It never creates a publication collection or a payload index, and fails with a bounded prerequisite error if the schema is absent, because creating it would build indexes on the collection the old release is still serving from. Only --apply provisions the publication lanes, recovers the authority, and records a durable block.

Because only --apply writes, a partial_failed seen only during a dry run is not recorded anywhere. If the Celery result for that job then expires (celery_result_expires, 24h by default) before the first --apply, that run observes no history and admits the version from its visible chunk set as historical_completion_record: absent, exactly like a genuinely history-less legacy version. The process cannot distinguish the two without refusing every history-less version, which it is required to admit. Run --apply within the result-expiry window of the dry run that reported the block, or re-ingest any Document Version a dry run reported as partial_failed before cutover.

What the process decides per Document Version:

SituationOutcome
A surviving historical partial_failed jobBlocked. The stored chunk set is known incomplete. Re-ingest the Document Version. An --apply run also records the block durably, so it outlives the Celery result that proved it.
A partial_failed job followed by a later successful oneAdmitted. Jobs are judged by completion time, not by which statuses appear: a strictly later successful job is the re-ingestion that repaired the failure, so it supersedes it. The failed result stays readable for its whole TTL, and re-running --apply in that window must not re-block a Document Version the operator has already fixed.
A partial_failed job that no later successful job followsBlocked. A re-ingestion that failed again is the most recent evidence, so it still blocks. A later job that hard-failed is not a repair either: it wrote nothing, so the known-incomplete chunk set is exactly what it was.
A job still processing/retrying, whatever else the Document Version recordsBlocked. An in-flight job has not finished, so nothing can supersede it. Quiesce ingestion, let the job settle, then re-run.
A blocking job whose completion time is unreadable, or one that only ties with a later successBlocked. Without an order there is no proof the failure was superseded, and absent evidence never admits a known-incomplete Document Version. A partial_failed job whose remaining children are still running reports no completion time for exactly this reason: its work is still outstanding, so a sibling job that finished in between cannot supersede it.
A Document Version an earlier --apply recorded as partial_failed, whose job results have since expiredBlocked. With no readable job evidence left, the durable record is the only guard. It lifts only when a visible chunk is written later than every chunk the version had stored when the block was recorded - that is, when it has actually been re-ingested. For the oldest legacy records, whose chunks carry no readable created_at, the moment the block was recorded stands in for that ceiling. A delete, restore, or metadata update during the window cannot lift it, and neither can the Celery record aging out.
A surviving partial_failed on a Document Version whose chunks are all soft-deletedReported as skipped, and --apply still records the block. Restores stay live during the window, so a version proven incomplete while invisible must not become admissible just by being restored after the Celery record expired.
A surviving completed jobAdmitted; the manifest records historical_completion_record: completed.
No surviving completion recordAdmitted from the visible stored chunk set; the manifest records historical_completion_record: absent with publication_provenance: legacy_bootstrap. No completion is invented.
Every chunk soft-deletedSkipped. Soft Delete state and existing visibility are preserved exactly; restore the version first.
Some chunks soft-deletedOnly the visible chunks are admitted.
Deleted, restored, or metadata-updated after an earlier run admitted itVerification fails closed. A generation is an immutable snapshot of the visible chunk set taken at admission, so restored chunks would be invisible after cutover and deleted chunks would still be served. The check is symmetric and covers both directions. The error names the Client, document_id, and version_id. Re-ingest the Document Version through the normal ingestion pipeline before cutover - the same remedy class as a partial_failed version. Re-running --apply cannot re-admit it: the idempotency key is stable per identity, so a changed corpus is refused rather than re-stamped.

Restart safety: the run is idempotent and resumable. --apply first recovers any in-flight publication operation using the same protocol the runtime performs at startup, so a run killed between fencing and READY resumes instead of leaving a fenced authority no pre-cutover release could clear. Each Document Version stages under a deterministic idempotency key, so an interrupted run replays the same generation instead of stamping a new one, already-verified authority is never rewritten, and identical lane records are a no-op. If a version's visible chunk set changed between runs, that version fails closed and is reported rather than silently re-admitted under the same generation. --apply reports that case as visible_chunks_moved_after_admission, names the Client, document_id, and version_id, and directs you to re-ingest - not to re-run, which reproduces the same refusal. Every other admission failure keeps the ordinary re-run remediation.

Metadata Leaves are intentionally not created here. The metadata lane is admitted empty; per-Client Metadata Backfill runs after cutover.

The bootstrap is additive and its output is isolated. Every publication lane is a collection of its own, separate from the chunk and fuzzy-terms collections the pre-cutover release serves from, so the old release cannot read or write anything the bootstrap stages: it keeps returning each chunk exactly once, its counts stay correct, and a Client delete, restore, or metadata update cannot reach a staged generation. The legacy records are never mutated or deleted; retiring them is the later contraction step.

Isolation keeps a generation immutable, but it cannot keep it current. A generation is a snapshot of the visible chunk set taken when the version was admitted, and delete, restore, and metadata updates stay live during the window (--quiesced-ingestion acknowledges ingestion only). Verification therefore compares each admitted version's current visible chunk set against its generation in both directions and fails closed on any difference, so a document deleted during the window cannot come back at cutover and chunks restored during the window cannot go missing. Re-run --verify-only immediately before cutover to prove the corpus still matches.

A nonzero exit means at least one Document Version was blocked or one admission failed. A whole-run prerequisite failure - an absent publication lane, more Clients than the discovery bound, a malformed stored client_id, or an unreachable Redis authority or Qdrant store - aborts before any Client runs and still emits the report contract with prerequisite_error set and verified: false, never a traceback. Use --qdrant-url, --collection-name, and --publication-redis-url when the deployment differs from the configured defaults.

Report counts describe durable state, not just what this run observed. A re-run that replays an already-admitted Document Version reports the completion record its manifest recorded, and activated_versions counts only the activations that run actually performed.