Metadata and Entities Reference

The InfoConnect Hybrid Search Engine stores each chunk with a metadata object and an entities object. System metadata is written by the pipeline. Custom metadata is supplied by users through /embed and /update.

Metadata v2 groups

Group	Fields	Owner	Notes
schema_version	"2"	System	Identifies the public metadata schema version
source	filename, mime_type, extension	System	File identity and type
document	page_count	System	Document-level parse facts
location	page_number, page_label, chunk_index	System	Chunk location inside the source document
classification	status, categories	System	ModernBERT classification output
quality	text_extraction, ocr_used, warnings	System	Extraction method and OCR warnings
extracted	title, author, created_at	System	Optional parser-derived document properties
embedding	dense_model, dense_dimensions, sparse_model, embedded_at, embedding_version, dense_runtime, chunker, extraction	System	Optional per-document search-vector provenance captured at embed time
custom	Any allowed user keys	User	Metadata from upload and update requests

System groups are read-only from the public API. User metadata also rejects the legacy parsing, processing, and ocr groups; keys starting with metadata. or entities.; and the server-owned payload roots metadata, entities, client_id, version_id, deleted, deleted_at, and deleted_by.

The optional embedding group records the model IDs, dense dimensions, FastEmbed runtime, chunker settings, extraction path, and pipeline version that produced the stored dense and sparse vectors. The indexed top-level embedding_version mirrors the detail group's version for maintenance selection. The query-time reranker and classification-only ModernBERT model are excluded because they do not produce the stored vectors. Documents indexed before pipeline versioning may expose only the historical dense_model, dense_dimensions, sparse_model, and embedded_at fields until they are re-embedded; missing detail is not inferred during reads.

Custom metadata

Custom metadata is nested under metadata.custom.* in Qdrant, but public filters use the short key. For example, uploading {"department":"Finance"} lets you search with "filters": {"department": "Finance"}.

curl -X POST http://localhost:8000/embed \
  -H "X-API-Key: super-secret-key" \
  -F "file=@test-documents/malaysia-wikipedia.txt" \
  -F "document_id=metadata-entities-001" \
  -F 'metadata={"department":"Engineering","team":"Backend","author":["wanjia","adlin"],"tenant_id":"tenant-a","project":"alpha"}'

Entity fields

Entity extraction runs during embedding. spaCy supplies people, organizations, dates, and locations; regex patterns add finance-oriented fields.

Entity key	Source	Example values
persons	spaCy PERSON	"Jane Doe"
organizations	spaCy ORG	"Acme Bank"
dates	spaCy DATE / TIME	"January 2026"
locations	spaCy GPE / LOC	"Kuala Lumpur"
monetary_amounts	Regex	"MYR 10,000", "$500.00"
account_numbers	Regex	IBANs, card-like numbers, account-number phrases
transaction_refs	Regex	"TXN-ABC123456", SWIFT message refs
account_types	Regex	"savings account", "mortgage", "line of credit"

Entity filters use the entity key directly and support a single string or a list of strings. Lists match any value within that field; multiple filter keys are combined with AND logic.

curl -X POST http://localhost:8000/search \
  -H "X-API-Key: super-secret-key" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "capital city",
    "filters": {
      "locations": ["Malaysia", "Kuala Lumpur"],
      "department": "Engineering"
    }
  }'

Output excerpt from the 2026-06-11 run:

{
  "meta": {
    "query": "capital city",
    "total": 10,
    "offset": 0,
    "limit": 5,
    "has_more": true,
    "query_time_ms": 30,
    "retrieval": {
      "alpha": 0.7,
      "reranked": false
    },
    "applied_filters": {
      "department": "Engineering",
      "project": "alpha",
      "locations": [
        "Malaysia",
        "Kuala Lumpur"
      ]
    }
  }
}

Related docs

• Search Filters explains filter logic and supported paths.
• Search API shows request and response shapes.
• Update API explains how to change custom metadata after upload.