Status API

Check the processing status of uploaded documents.

Client Isolation

Job status is scoped to the client that created the job. A job is only visible to the client whose API key was used to submit the /embed or PATCH /update request, or to the client a Metadata Backfill was run for. Querying a job ID that belongs to a different client returns 404 — the same as a non-existent job — to avoid leaking information across clients.

Endpoint

GET /status/{job_id}

Request

Headers

Header	Required	Description
`X-API-Key`	Yes	Valid API key from `API_KEY_CLIENTS` env var

Path Parameters

Parameter	Description
`job_id`	Job ID returned from `/embed` or `PATCH /update`

Response

Pending/Started

{
  "job_id": "abc123-def456",
  "document_id": "doc-001",
  "status": "pending"
}

Processing

{
  "job_id": "abc123-def456",
  "document_id": "doc-001",
  "status": "processing"
}

Completed

{
  "job_id": "abc123-def456",
  "document_id": "doc-001",
  "status": "completed",
  "search_ready": true,
  "snapshot_generation": 7,
  "chunks_created": 12,
  "categories": ["financial reporting"],
  "classification_status": "completed",
  "entities_extracted": {
    "persons": ["John Smith", "Jane Doe"],
    "organizations": ["Acme Corp", "Tech Inc"],
    "dates": ["January 15, 2024"],
    "locations": ["New York", "San Francisco"]
  },
  "created_at": "2026-03-26T12:19:26.452416+00:00",
  "completed_at": "2026-03-26T12:19:46.580322+00:00"
}

The indexing status and classification_status are independent. A publication-backed document can report status: "completed" before this API process has installed the generation used by /search. For searchable content, wait for search_ready: true; snapshot_generation identifies the generation currently served by this process and is null until a snapshot reports one. Keep polling this endpoint until classification_status becomes "completed", "failed", or "disabled" when you need terminal classification; classification and categories are refreshed from the stored document version even when entities_extracted is already present.

Once the job's publication is ready, each poll proves the generation's Manifest against the evidence committed when it was published: the Manifest record id and the Content, Fuzzy, and Metadata lane counts and digests it declares. A poll reads only the Manifest, so its cost does not grow with document size; it does not re-read or re-hash the Content, Fuzzy, or Metadata records. A Manifest that disagrees with the committed evidence fails closed with 503 and Retry-After: 1.

After post-ingest convergence or a later upload republishes the same Document Version, the completed job remains readable even though its exact historical generation has been retired. The status endpoint verifies the active successor's publication evidence before preserving status: "completed", independently of Client-wide Metadata readiness. If successor verification fails during concurrent publication, it refreshes authority and retries only when the same Client, document, and version have advanced to a strictly newer generation, with at most three successor verification attempts. Missing, incomplete, mismatched, unchanged, older, unready, or exhausted successor evidence fails closed with 503 and Retry-After: 1 when the historical generation cannot be verified.

Superseded

{
  "job_id": "abc123-def456",
  "document_id": "doc-001",
  "status": "superseded",
  "publication_status": "superseded",
  "superseded_by_job_id": "newer-job-id",
  "superseded_reason": "newer_upload",
  "error": null
}

Two accepted uploads for the same Document Version can complete out of order. When a newer accepted upload owns the identity, the older job ends explicitly as superseded with superseded_reason: "newer_upload" instead of leaking an internal NON_MONOTONIC_GENERATION error. This is a terminal, non-error outcome; poll superseded_by_job_id for the newer upload's indexing result. An upload can also end with superseded_reason: "replacement_overtaken" when a later publication overtakes its replacement before fencing. In that case superseded_by_job_id is null: there may be no newer upload job to poll.

Failed

{
  "job_id": "abc123-def456",
  "document_id": "doc-001",
  "status": "failed",
  "error": "Error message describing what went wrong"
}

The error field preserves an operator-legible cause for parse failures. OOXML/ZIP decompression-guard rejections, malformed inner XML, and XML node-size-limit failures are reported with distinct messages; other unreadable inputs retain the generic fallback.

Partial Failed

Some embedding tasks failed while others succeeded:

{
  "job_id": "abc123-def456",
  "document_id": "doc-001",
  "status": "partial_failed",
  "chunks_created": 45,
  "entities_extracted": {
    "persons": ["John Smith"],
    "organizations": ["Acme Corp"]
  },
  "error": "Task xyz failed: Connection timeout"
}

Partial Failure

When processing large documents, chunks are processed in parallel. If some chunks fail but others succeed, you get partial_failed status. The successful chunks are searchable; failed chunks are lost.

Retry

Task is scheduled for retry after a failure:

{
  "job_id": "abc123-def456",
  "document_id": "doc-001",
  "status": "retrying",
  "error": "Connection timeout to Qdrant",
  "retry_count": 1,
  "max_retries": 3,
  "next_retry_in_seconds": 60
}

Status Values

Status	Description
`pending`	Job queued, waiting for worker
`started`	Worker picked up the job
`processing`	Actively processing (parsing/chunking/embedding)
`completed`	Successfully indexed
`superseded`	Terminal non-error outcome; see Superseded for reasons and the optional newer job ID
`failed`	Processing failed, see `error` field
`partial_failed`	Some chunks succeeded, others failed
`retrying`	Will be retried (up to 3 attempts)

Search-readiness fields

Publication-backed completed responses include these additive fields:

Field	Description
`search_ready`	`true` only when this API process's serving snapshot contains this job's generation or a newer one for the same document version (for example the post-ingest readiness republish); `false` while it catches up
`metadata_ready`	`true` only after readiness convergence commits `ready` for this client (post-ingest chain or periodic sweep); `false` while it is `backfilling`, `blocked`, or `not_started`
`readiness_state`	The client's durable Metadata Discovery state: `not_started`, `backfilling`, `blocked`, or `ready`
`snapshot_generation`	Generation currently present in that snapshot for this document, or `null` when absent

status: "completed" semantics are unchanged. Because search_ready is process-local, send the subsequent search request to the same API process when deployments have multiple replicas. metadata_ready is durable, not process-local: it flips to true only when the chained converge task commits ready, so a Metadata or Both search is honest immediately after it turns — before that, those scopes answer 409 with Retry-After while the state is backfilling.

Operations

Every job carries an operation discriminator so one polling loop can read all three:

`operation`	Started by	Scope
`embed`	`POST /embed`	One document
`metadata_update`	`PATCH /update`	One document, one or all versions
`metadata_backfill`	The operator Metadata Backfill command	One client's whole corpus

Client binding and the cross-client 404 are identical for all three.

Metadata Update Jobs

Jobs queued by PATCH /update report operation: "metadata_update" and carry update-specific progress fields instead of the indexing fields above:

{
  "job_id": "6f1c8f0a2b7d4e6b8c1d2e3f4a5b6c7d",
  "document_id": "doc-123",
  "version_id": null,
  "status": "completed",
  "operation": "metadata_update",
  "updated_fields": ["title", "author"],
  "updated_fields_count": 2,
  "updated_fields_truncated": false,
  "versions_total": 2,
  "versions_completed": 2,
  "versions_unchanged": 0,
  "updated_chunks": 24,
  "publication_status": "ready",
  "cleanup_tasks_scheduled": 2
}

Field	Description
`operation`	`"metadata_update"` for update jobs; see Operations
`version_id`	Targeted version, or `null` when every version of the document is targeted
`updated_fields`	Top-level patch keys, capped at 64 entries
`updated_fields_count`	Number of top-level keys in the patch
`updated_fields_truncated`	`true` when `updated_fields` was capped
`versions_total` / `versions_completed` / `versions_unchanged`	Targeted versions, versions finished, and versions the patch left unchanged (nothing republished)
`updated_chunks`	Chunks republished across the completed versions
`publication_status`	`"ready"` once the new generation is active
`cleanup_tasks_scheduled`	Obsolete-generation cleanup tasks enqueued

Failures are bounded: error is "Metadata update failed" for every reason except a merged-metadata contract violation, which reports "Metadata update validation failed" plus a value-free error_details object (code, path, observed, limit, unit). Custom Metadata values never appear in the status response. See the Update API for the request contract and merge semantics.

Examples

Basic Status Check

curl http://localhost:8000/status/abc123-def456 \
  -H "X-API-Key: super-secret-key"

Poll Until Complete

#!/bin/bash

JOB_ID="abc123-def456"
API_KEY="super-secret-key"

while true; do
  response=$(curl -s "http://localhost:8000/status/$JOB_ID" \
    -H "X-API-Key: $API_KEY")

  status=$(echo "$response" | jq -r '.status')
  echo "Status: $status"

  ready=$(echo "$response" | jq -r '.search_ready // true')
  if [ "$status" = "completed" ] && [ "$ready" = "true" ]; then
    echo "Document indexed successfully!"
    echo "$response" | jq '.'
    break
  elif [ "$status" = "failed" ]; then
    echo "Processing failed!"
    echo "$response" | jq '.'
    break
  elif [ "$status" = "superseded" ]; then
    echo "Upload superseded; inspect the reason and optional newer job ID."
    echo "$response" | jq '.'
    break
  fi

  sleep 2
done

Python Polling

import time
import requests

def wait_for_completion(job_id: str, api_key: str, timeout: int = 300) -> dict:
    """Poll status until completed, superseded, or failed."""
    url = f"http://localhost:8000/status/{job_id}"
    headers = {"X-API-Key": api_key}

    start = time.time()
    while time.time() - start < timeout:
        response = requests.get(url, headers=headers)
        data = response.json()

        print(f"Status: {data['status']}")

        if data['status'] == 'completed' and data.get('search_ready', True):
            print(f"Created {data['chunks_created']} chunks")
            return data
        elif data['status'] == 'failed':
            raise Exception(f"Processing failed: {data.get('error')}")
        elif data['status'] == 'superseded':
            return data

        time.sleep(2)

    raise TimeoutError("Processing took too long")

# Usage
result = wait_for_completion("abc123-def456", "super-secret-key")

Response Codes

Code	Description
200	Success - Returns status
401	Unauthorized - Invalid API key
404	Not Found - Job ID doesn't exist, or the job belongs to a different client
429	Rate Limited - Too many requests
503	Service Unavailable - Publication authority unavailable while loading job state; retry after the `Retry-After` interval

Error Responses

{
  "detail": "Invalid API key",
  "error_code": "auth_failed",
  "request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e"
}

{
  "detail": "Job not found",
  "error_code": "not_found",
  "request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e"
}

Best Practices

Poll with backoff - Start with 1s intervals, increase to 5s for longer jobs
Set timeouts - Large documents can take several minutes
Handle failures - Check for failed status and log the error
Store job IDs - Keep job IDs for later status checks or debugging
Wait for search readiness - For publication-backed jobs, require search_ready: true before searching

Typical Processing Times

Document Type	Size	Approximate Time
Text file	< 100KB	2-5 seconds
PDF	1-5 pages	5-15 seconds
PDF	10-50 pages	15-45 seconds
PDF	100+ pages	1-3 minutes
DOCX	Any	Similar to PDF

Note: First document after startup takes longer due to model loading (~30s).

Metadata Backfill Jobs

An operator Metadata Backfill binds one job id per client and reports operation: "metadata_backfill". It covers the client's corpus rather than a single document, so document_id is empty.

{
  "job_id": "6f1f0d0c8b4b4bd2a0d8f0d1a3c7e5b9",
  "document_id": "",
  "status": "processing",
  "operation": "metadata_backfill",
  "readiness_state": "backfilling",
  "versions_total": 1284,
  "versions_completed": 400,
  "versions_backfilled": 400,
  "versions_pending": 884,
  "versions_blocked": 0,
  "violations": [],
  "violations_total": 0,
  "violations_truncated": false
}

Field	Description
`readiness_state`	`not_started`, `backfilling`, `blocked`, or `ready` after this run
`versions_total`	Accepted Document Versions in the client's corpus
`versions_completed`	Versions already represented in Metadata Discovery
`versions_backfilled`	Versions this run republished
`versions_pending`	Versions still outstanding, including deferred batches
`versions_blocked`	Versions blocked by a Custom Metadata contract violation
`violations`	Bounded remediation entries: document identity, path, cap. Never a metadata value

A job still planning omits the version totals rather than reporting a completed total of zero. Single-writer refusal is terminal failed with error_details.code set to backfill_already_running and next_retry_in_seconds bounded by the claim lease. publication_authority_inconsistent means the client's authority needs repair; it is not an empty corpus or a metadata violation.

A blocked backfill reports readiness_state: "blocked" and leaves Metadata and Both gated. See the Discovery Readiness API for the client-facing view.

Endpoint​

Request​

Headers​

Path Parameters​

Response​

Pending/Started​

Processing​

Completed​

Superseded​

Failed​

Partial Failed​

Retry​

Status Values​

Search-readiness fields​

Operations​

Metadata Update Jobs​

Examples​

Basic Status Check​

Poll Until Complete​

Python Polling​

Response Codes​

Error Responses​

Best Practices​

Typical Processing Times​

Metadata Backfill Jobs​