Document Management API

Manage uploaded documents through the documents endpoints.

Client Isolation

All document operations are scoped to the client associated with the API key used. Documents are owned per client — two clients may use the same document_id without collision. Operations on one client's documents never affect another client's data.

List Documents

Retrieve a paginated list of all indexed documents belonging to the caller's client.

Endpoint

GET /documents

Request

Headers

Header	Required	Description
`X-API-Key`	Yes	Valid API key from `API_KEY_CLIENTS` env var

Query Parameters

Parameter	Type	Default	Description
`limit`	int	100	Max documents to return (1-1000)
`offset`	int	0	Pagination offset
`metadata_filter`	string	none	JSON filter for metadata fields
`include_deleted`	boolean	`false`	Include soft-deleted documents and return deletion marker fields

Response

{
  "documents": [
    {
      "document_id": "doc-123",
      "version_id": "v2",
      "metadata": {
        "source": {
          "filename": "report.pdf"
        },
        "location": {
          "page_number": 1,
          "page_label": "1"
        },
        "document": {
          "page_count": 5
        },
        "classification": {
          "categories": ["other"],
          "status": "completed"
        },
        "quality": {
          "text_extraction": "native",
          "ocr_used": false,
          "warnings": []
        },
        "custom": {
          "department": "Engineering"
        }
      },
      "chunk_count": 5,
      "created_at": "2024-01-15T10:30:00"
    }
  ],
  "total": 42,
  "limit": 100,
  "offset": 0,
  "has_more": true
}

With publication enabled, each summary represents one (document_id, version_id) identity. Different versions of the same document therefore appear as separate rows and paginate independently. Legacy publication-disabled listings retain their existing document-level grouping, and may report version_id: null when one version cannot be identified safely.

Examples

List All Documents

curl http://localhost:8000/documents \
  -H "X-API-Key: super-secret-key"

Paginated Results

# First page
curl "http://localhost:8000/documents?limit=10&offset=0" \
  -H "X-API-Key: super-secret-key"

# Second page
curl "http://localhost:8000/documents?limit=10&offset=10" \
  -H "X-API-Key: super-secret-key"

Include Soft-Deleted Documents

By default, document listings hide documents that were soft-deleted. Set include_deleted=true to include them and expose the root deletion markers:

curl "http://localhost:8000/documents?include_deleted=true" \
  -H "X-API-Key: super-secret-key"

Soft-deleted summaries include these additional fields:

{
  "document_id": "doc-123",
  "version_id": "v1",
  "deleted": true,
  "deleted_at": "2026-06-29T12:34:56Z",
  "deleted_by": "demo-client"
}

Filter by Metadata

Filter Logic: AND Only

All filter conditions are combined with AND logic — every condition must match. There is no OR support. metadata_filter values must be scalar string, boolean, or integer values; a list-valued filter is rejected with HTTP 400. Use POST /search when you need list values with OR semantics within a field.

Filter using explicit system metadata paths when you want system-generated values:

curl "http://localhost:8000/documents?metadata_filter={\"metadata.classification.categories\":\"other\"}" \
  -H "X-API-Key: super-secret-key"

Filter by source filename:

curl "http://localhost:8000/documents?metadata_filter={\"metadata.source.filename\":\"report.pdf\"}" \
  -H "X-API-Key: super-secret-key"

Or with simplified syntax:

curl "http://localhost:8000/documents?metadata_filter=\"metadata.classification.categories\":\"other\"" \
  -H "X-API-Key: super-secret-key"

Custom metadata keys with spaces use the explicit metadata.custom. prefix, the same as POST /search:

{"metadata.custom.Field A": "test"}

curl --get "http://localhost:8000/documents" \
  -H "X-API-Key: super-secret-key" \
  --data-urlencode 'metadata_filter={"metadata.custom.Field A":"test"}'

Get Document Text

Retrieve the parsed text already stored in the search index. This endpoint does not retrieve or retain the original uploaded file. Raw dense vectors are withheld by default and returned only when include_vector=true; sparse vectors are never returned.

Chunks are returned in document order with their chunk index and page number when available. Each chunk includes its ID, character count, entities, vector dimension, and server-computed L2 norm. Classification categories and entities are also aggregated across the returned chunks; metadata is returned in the same structured v2 shape as the document APIs.

The top-level embedding object is a curated processing summary for the Console. For newly indexed documents, its dense model, vector dimensions, sparse model, and embedded timestamp come from provenance captured in the document payload and provenance is captured. For older documents without that record, those fields fall back to the current server configuration and provenance is current_config; these fallback values must not be interpreted as the models originally used for that document. The reranker is always the current query-time configuration, and the classifier is always the current classification-only configuration; neither produces the stored search vectors.

When multiple fully versioned copies exist and their created_at values are valid, the endpoint returns the latest version. The response's version_id identifies the selected version; it is null when the returned chunks cannot be attributed safely to one version.

Endpoint

GET /document/:document_id/text

Request

Headers

Header	Required	Description
`X-API-Key`	Yes	Valid API key from `API_KEY_CLIENTS` env var

Path Parameters

Parameter	Description
`document_id`	The document ID whose indexed text to return

Query Parameters

Parameter	Type	Default	Description
`include_deleted`	boolean	`false`	Include chunks from a soft-deleted document
`include_vector`	boolean	`false`	Include each chunk's raw dense vector; vectors are withheld by default

Response

{
  "document_id": "doc-123",
  "version_id": "version-2024-01-15",
  "chunk_count": 2,
  "created_at": "2024-01-15T10:30:00Z",
  "chunks": [
    {
      "id": "doc-123-chunk-0",
      "index": 0,
      "page": 1,
      "text": "First page of the report.",
      "char_count": 25,
      "entities": {
        "ORG": ["Example Bank"]
      },
      "vector_dim": 384,
      "norm": 1.0,
      "vector": null
    },
    {
      "id": "doc-123-chunk-1",
      "index": 1,
      "page": 2,
      "text": "Second page of the report.",
      "char_count": 26,
      "entities": {},
      "vector_dim": 384,
      "norm": 1.0,
      "vector": null
    }
  ],
  "metadata": {
    "source": {
      "filename": "report.pdf"
    },
    "custom": {
      "department": "Finance"
    }
  },
  "classification": {
    "categories": ["financial reporting"],
    "status": "completed"
  },
  "entities": {
    "ORG": ["Example Bank"]
  },
  "embedding": {
    "provenance": "captured",
    "dense_model": "BAAI/bge-small-en-v1.5",
    "dense_dimensions": 384,
    "sparse_model": "Qdrant/bm25",
    "embedded_at": "2024-01-15T10:30:00Z",
    "embedding_version": 1,
    "dense_runtime": "fastembed==0.8.0",
    "chunker": "sentence-splitter:512:50",
    "extraction": "native",
    "reranker_model": "Xenova/ms-marco-MiniLM-L-6-v2",
    "reranker_scope": "query_time",
    "classifier_model": "onnx-community/ModernBERT-base-nli-ONNX",
    "classifier_model_file": "onnx/model_quantized.onnx",
    "classifier_scope": "classification_only",
    "classification_enabled": true
  }
}

When the four historical model fields were captured, provenance remains captured even if the added embedding_version, dense_runtime, chunker, and extraction fields are null. When the historical fields are absent, provenance is current_config and the model fields come from the running server configuration. The API does not infer missing provenance detail.

The request is scoped to the client resolved from X-API-Key. An unknown document, a document owned by another client, or a soft-deleted document when include_deleted is false returns 404 Document not found.

Examples

curl http://localhost:8000/document/doc-123/text \
  -H "X-API-Key: super-secret-key"

curl "http://localhost:8000/document/doc-123/text?include_deleted=true" \
  -H "X-API-Key: super-secret-key"

curl "http://localhost:8000/document/doc-123/text?include_vector=true" \
  -H "X-API-Key: super-secret-key"

Delete Document

Delete a document and all its chunks for the authenticated client. Deletes are soft by default: chunks remain in Qdrant with root payload markers and are hidden from normal list/search reads. Use mode=hard only when you need a physical delete.

Endpoint

DELETE /document/:document_id

Request

Headers

Header	Required	Description
`X-API-Key`	Yes	Valid API key from `API_KEY_CLIENTS` env var

Path Parameters

Parameter	Description
`document_id`	The document ID to delete

Query Parameters

Parameter	Type	Required	Description
`version_id`	string	No	Delete only chunks whose `document_id` and `version_id` both match for the authenticated client. Version ids are scoped to `document_id`, not globally unique. Omit it to delete all versions of the document.
`mode`	string	No	`soft` or `hard`. Defaults to `soft`. Hard delete physically removes matching chunks, including chunks already soft-deleted; an interrupted purge is retryable.

Response

{
  "document_id": "doc-123",
  "deleted_chunks": 5,
  "status": "soft_deleted"
}

With publication enabled, an unscoped soft delete republishes every active document version one at a time. A delete that arrives while a sibling publish holds the per-Client fence waits for the fence up to publication_lifecycle_fence_wait_seconds (default 5 s), then proceeds. When publication contention stops the request after at least one version already changed, the API returns 503 with error_code: "publication_busy" and Retry-After: 1 instead of a generic outage. The response details name the versions that already changed and the versions still pending, so the caller can retry the same request to finish:

{
  "detail": "Publication busy: some versions changed, retry to finish",
  "error_code": "publication_busy",
  "request_id": "9f2c…",
  "details": {
    "code": "publication_busy",
    "message": "publication contention after 1 version(s) changed; retry to finish",
    "changed_versions": ["v3"],
    "pending_versions": ["v1", "v2", "v4"]
  }
}

If the fence is still held when the wait bound expires, the API returns 503 with error_code: "publication_busy" and Retry-After: 1 before changing any version. The details carry code: "publication_busy" and the retry message; retry the same request after the indicated interval. Other contention or transient failures return 503 with error_code: "publication_busy". Missing readiness or incomplete publication evidence returns 503 with error_code: "publication_unavailable". Both carry Retry-After: 1. Programming defects return 500 with error_code: "internal_error" and no Retry-After header.

Examples

Soft-Delete Single Document

curl -X DELETE http://localhost:8000/document/doc-123 \
  -H "X-API-Key: super-secret-key"

Hard-Delete Single Document

curl -X DELETE "http://localhost:8000/document/doc-123?mode=hard" \
  -H "X-API-Key: super-secret-key"

Hard delete returns status: "hard_deleted", physically removes matching points, and removes /status/{job_id} visibility for jobs indexed to the hard-deleted document or version. If a prior attempt was interrupted after the authority purge, retrying the same request resumes the journaled purge instead of treating the forgotten scope as a missing document. Read paths exclude and best-effort sweep any covered orphan records.

With publication enabled, a Qdrant delete timeout triggers a re-read of the affected lane. If no matching records remain, the purge continues and returns the normal 200 when all lanes finish. If records remain, or a count request times out before or after the delete, the API returns 503 with error_code: "publication_purge_timeout" and Retry-After: 1. The purge is then incomplete or unverified; retry the same request within your retry budget to resume the journaled purge, even if the document is already absent from list/text reads. The request deadline is configured through QDRANT_TIMEOUT_SECONDS.

Delete One Document Version

curl -X DELETE "http://localhost:8000/document/doc-123?version_id=v2" \
  -H "X-API-Key: super-secret-key"

Chunks store version_id as a top-level indexed payload field. When upload metadata includes version_id, that value is used for the indexed version; otherwise the service uses document_id as the default version id. Version ids are scoped to the authenticated client and document_id, so different documents may reuse the same version_id.

Delete Multiple Documents

for doc_id in doc-001 doc-002 doc-003; do
  curl -X DELETE "http://localhost:8000/document/$doc_id" \
    -H "X-API-Key: super-secret-key"
done

Delete from Python

import requests

url = "http://localhost:8000/document/doc-123"
headers = {"X-API-Key": "super-secret-key"}

response = requests.delete(url, headers=headers)
print(response.json())
# {'document_id': 'doc-123', 'deleted_chunks': 5, 'status': 'soft_deleted'}

Restore Document

Restore a soft-deleted document by clearing its root deletion markers.

Endpoint

POST /document/:document_id/restore

Request

Headers

Header	Required	Description
`X-API-Key`	Yes	Valid API key from `API_KEY_CLIENTS` env var

Path Parameters

Parameter	Description
`document_id`	The document ID to restore

Query Parameters

Parameter	Type	Required	Description
`version_id`	string	No	Restore only chunks whose `document_id` and `version_id` both match for the authenticated client. Omit it to restore all versions of the document.

Response

{
  "document_id": "doc-123",
  "restored_chunks": 5,
  "status": "restored"
}

With publication enabled, an unscoped restore republishes every soft-deleted version one at a time and reports contention the same way as soft delete: a restore that arrives while a sibling publish holds the per-Client fence waits for the fence up to publication_lifecycle_fence_wait_seconds (default 5 s). A stop after at least one version changed returns 503 with error_code: "publication_busy", details carrying changed_versions and pending_versions, and Retry-After: 1. Retry the same request to finish; if the fence is still held when the wait bound expires, the API returns 503 publication_busy with Retry-After: 1 before restoring any version. Other contention or transient failures return 503 with error_code: "publication_busy". Missing readiness or incomplete publication evidence returns 503 with error_code: "publication_unavailable". Both carry Retry-After: 1. Programming defects return 500 with error_code: "internal_error" and no Retry-After header.

Example

curl -X POST http://localhost:8000/document/doc-123/restore \
  -H "X-API-Key: super-secret-key"

Update Document Fields

PATCH /update asynchronously updates Custom Metadata or the source filename without re-uploading a document. The Update API is the authoritative reference for its request, response, atomic Metadata Discovery validation, and error contract.

Response Codes

Code	Endpoint	Description
200	GET /documents	Success - Returns document list
200	GET /document/:document_id/text	Success - Returns ordered extracted-text chunks and document facets
200	DELETE	Success - Document soft-deleted or hard-deleted
200	POST /document/:document_id/restore	Success - Document restored
401	All	Unauthorized - Invalid API key
404	GET text / DELETE / restore	Not Found - Document doesn't exist, is not visible, has no matching version, or belongs to a different client
400	GET	Bad Request - Invalid parameters
503	GET list/text	Service Unavailable - the publication authority is unreachable, or a read cannot validate one immutable snapshot (a coherently empty pre-bootstrap client is returned as an empty `200` list). If a read snapshot changes during a request, the response uses `error_code: "publication_snapshot_changed"` and `Retry-After: 1`. For Manifest-less list or text reads, see the shared publication authority inconsistent contract.
503	DELETE / POST restore	Partial multi-version lifecycle outcome - contention stopped the request after at least one version changed. The response uses `error_code: "publication_busy"`, `details` with `changed_versions` and `pending_versions`, and `Retry-After: 1`; retry the same request to finish. If the per-Client fence is still held when the `publication_lifecycle_fence_wait_seconds` bound (default 5 s) expires, the response is `503` `publication_busy` with `Retry-After: 1` before any version changed. Snapshot movement, source-generation races, recovery/maintenance contention, and transient backend failures use `publication_busy`. Missing readiness or incomplete publication evidence uses `publication_unavailable`. Both carry `Retry-After: 1`; programming defects use `500 internal_error` without `Retry-After`.

Delete and restore classify operational failures through the same lifecycle boundary. An initial or per-version snapshot that moves during both consistency reads is retried within the configured fence-wait bound. A moved pre-fence proof also returns publication_busy; the request never activates on a torn proof. Genuine unreadiness uses publication_unavailable, distinct from a programming defect. Existing publication_purge_timeout and publication_authority_inconsistent contracts remain distinct.

When a DELETE or restore arrives while a sibling publish holds the per-Client fence, the API waits for the fence up to publication_lifecycle_fence_wait_seconds (default 5 s), then proceeds. If the fence is still held at the bound, the API returns 503 with error_code: "publication_busy" and Retry-After: 1 before mutating the target. This is a retryable busy response, not proof that every later 503 is side-effect-free: callers that retry must first confirm the target is still present, wait no longer than the indicated bounded interval and within the caller's total retry budget, and repeat the same idempotent request only when that absence-of-application check succeeds. A 503 from that proof read is itself retryable busy state; continue the proof only within the same bounded budget. Genuine 429 rate-limit responses remain distinct and must not be retried by the QA scripts.

The presence check above applies to publication-busy responses, not publication_purge_timeout: for that error, follow the journaled-purge retry guidance under Hard-Delete Single Document.

When publication is enabled, list and text reads fail the whole request with retryable 503 instead of returning partial or mixed-generation data. Before publication bootstrap, a client whose authority is coherently empty receives the established empty 200 response rather than a generic outage; genuine authority outages remain 503.

Soft delete and restore publish a new generation of the same document version carrying the new visibility, so content chunks, fuzzy terms, metadata leaves, and the publication manifest change visibility together and include_deleted reads the same evidence in every lane. Hard delete forgets the document version in the publication authority and then physically removes every generation from all four stores plus the document's job bindings; it is irreversible. Both are scoped to the caller's client and to version_id when supplied, so sibling versions and another client reusing the same document_id are never affected.

Error Responses

{
  "detail": "Invalid API key"
}

{
  "detail": "Document not found"
}

{
  "detail": "Invalid metadata_filter JSON"
}

{
  "detail": "metadata_filter must be a JSON object"
}

Document Summary Fields

Field	Type	Description
`document_id`	string	Unique document identifier
`metadata`	object	Structured metadata; see Metadata and Entities Reference for its system and custom groups
`chunk_count`	int	Number of text chunks indexed
`created_at`	string	ISO 8601 timestamp
`deleted`	boolean	Present only when `include_deleted=true`; true when the document is soft-deleted
`deleted_at`	string	Present only when `include_deleted=true`; soft-delete timestamp
`deleted_by`	string	Present only when `include_deleted=true`; resolved client id that soft-deleted the document

Metadata Structure

The Metadata and Entities Reference owns the system groups, fields, and reserved-key contract for the metadata object.

Custom Metadata: Fields uploaded via the metadata parameter in /embed or /update are stored under metadata.custom.*, even when their names overlap with system fields such as author, title, or categories. These fields are filterable in both search and document listing using the explicit metadata.custom. prefix (e.g. metadata.custom.department). To rename the system source filename, use the top-level filename field on PATCH /update instead of putting the name inside metadata.

Note: The categories field is also available as a top-level payload field for efficient filtering.

Best Practices

Check before hard-deleting - Use GET /documents?include_deleted=true to verify document state before physical removal
Prefer soft delete - The default delete mode hides documents from normal reads while preserving restore capability
Use restore for mistakes - POST /document/:document_id/restore brings soft-deleted chunks back into normal search/list results
Handle 404 gracefully - The document may not exist, may not match the requested version, or may belong to another client
Use metadata for organization - Filter documents by metadata before bulk operations

List Documents​

Endpoint​

Request​

Headers​

Query Parameters​

Response​

Examples​

List All Documents​

Paginated Results​

Include Soft-Deleted Documents​

Filter by Metadata​

Get Document Text​

Endpoint​

Request​

Headers​

Path Parameters​

Query Parameters​

Response​

Examples​

Delete Document​

Endpoint​

Request​

Headers​

Path Parameters​

Query Parameters​

Response​

Examples​

Soft-Delete Single Document​

Hard-Delete Single Document​

Delete One Document Version​

Delete Multiple Documents​

Delete from Python​

Restore Document​

Endpoint​

Request​

Headers​

Path Parameters​

Query Parameters​

Response​

Example​

Update Document Fields​

Response Codes​

Error Responses​

Document Summary Fields​

Metadata Structure​

Categories​

Best Practices​

List Documents

Endpoint

Request

Headers

Query Parameters

Response

Examples

List All Documents

Paginated Results

Include Soft-Deleted Documents

Filter by Metadata

Get Document Text

Endpoint

Request

Headers

Path Parameters

Query Parameters

Response

Examples

Delete Document

Endpoint

Request

Headers

Path Parameters

Query Parameters

Response

Examples

Soft-Delete Single Document

Hard-Delete Single Document

Delete One Document Version

Delete Multiple Documents

Delete from Python

Restore Document

Endpoint

Request

Headers

Path Parameters

Query Parameters

Response

Example

Update Document Fields

Response Codes

Error Responses

Document Summary Fields

Metadata Structure

Categories

Best Practices