Skip to main content

Search API

The /search endpoint performs hybrid semantic search across embedded documents, combining dense vector similarity with sparse BM25 keyword matching.

Endpoint

POST /search

Request

Headers

HeaderRequiredDescription
X-API-KeyYesValid API key from API_KEY_CLIENTS env var
Content-TypeYesapplication/json

Body Parameters

ParameterTypeDefaultDescription
querystringrequiredSearch query text
alphafloat0.5Semantic weight (1.0=dense-only, 0.0=sparse-only)
limitint10Max results to return (max: 100)
offsetint0Pagination offset. offset + limit must not exceed 200.
filtersobject{}Metadata and entity filters; scalar values match exactly, list values match any value. See Filterable Fields for accepted keys.
fieldslist[str]nullTop-level result fields to include (id, score, document, chunk, classification, entities, metadata, highlights). meta is always returned. Legacy payload aliases such as document_id and chunk_text map to their nested sections.
highlightboolfalseReturn lexical Matched Term spans for each result chunk.
rerankboolfalseEnable cross-encoder reranking
top_k_rerankint20Number of candidates to rerank (max: 200)
min_scorefloatnoneMinimum score threshold
include_deletedboolfalseInclude soft-deleted chunks in the search scope. Client isolation still applies.

Alpha blending

The alpha parameter controls the balance between semantic and keyword search:

  • alpha: 1.0 - Dense vector search only (semantic meaning)
  • alpha: 0.0 - Sparse BM25 search only (keyword matching)
  • alpha: 0.5 - Hybrid default (equal dense and sparse weights)

For mixed values, the API applies weighted reciprocal rank fusion (RRF) with dense weight alpha and sparse weight 1 - alpha. Different mixed values therefore change the fused ranking rather than only selecting hybrid mode.

Use 0.5 as the recommended starting point. Lower values give sparse/lexical matching more weight, which helps exact names, identifiers, and terminology; higher values give dense/semantic matching more weight, which helps conceptual queries whose wording differs from the indexed text. Operators can tune the deployment default with HYBRID_ALPHA, and an explicit request value takes precedence.

Low-confidence filtering

RRF scores encode rank position, not absolute relevance. The API gates each active retrieval stream on its raw score before fusion instead of applying a floor to the fused RRF output:

  • dense candidates require cosine similarity of at least 0.70 (the recalibrated default, down from 0.75);
  • sparse candidates require a BM25 dot-product score of at least 1.0 (unchanged);
  • a hybrid candidate is retained when either active stream clears its floor.

These floors remove low-confidence candidates before ranking. A floor that is too high can over-filter genuine matches and produce an empty result set; the calibrated values preserve genuine matches while gibberish and queries with no confident match still return no results. Configure the model-specific floors with SEARCH_DENSE_SCORE_FLOOR and SEARCH_SPARSE_SCORE_FLOOR, and recalibrate them when the corresponding embedding model or corpus distribution changes. The request-level min_score remains a separate post-retrieval threshold.

Query normalization and typo correction

Before embedding and retrieval, the service expands a small conservative abbreviation set and corrects alphabetic query tokens against a spelling vocabulary built from the requesting client's own chunks. A correction must be within two character edits. Deleted-document terms are excluded unless include_deleted is enabled, and correction failures fall back to the normalized query instead of failing the search.

The corrected text is used for dense and sparse retrieval and optional reranking. The response still returns the original request in meta.query, and Matched Term highlights continue to use the original query text.

Response

{
"results": [
{
"id": "chunk-uuid",
"score": {
"value": 0.92,
"rerank": 0.92,
"strategy": "hybrid"
},
"document": {
"id": "doc-123",
"created_at": "2026-04-09T10:30:00Z"
},
"chunk": {
"id": "doc-123:0",
"index": 0,
"text": "Document content...",
"page": 1
},
"classification": {
"categories": ["financial reporting"]
},
"entities": {...},
"metadata": {...},
"highlights": [
{"start": 0, "end": 8, "term": "document"}
]
}
],
"meta": {
"query": "search query",
"total": 42,
"total_is_lower_bound": false,
"offset": 0,
"limit": 10,
"has_more": true,
"query_time_ms": 15,
"retrieval": {
"alpha": 0.5,
"reranked": true
},
"applied_filters": {...}
}
}

Search pagination is bounded to the first 200 retrievable matches. Requests where offset + limit > 200 return 422, and meta.total never exceeds 200. When the score-gated match count is larger, meta.total: 200 describes the complete retrievable window rather than the number of matches outside that window. meta.has_more is therefore false on a page ending at offset 200.

Within that window, the service counts confidence-filtered matches through SEARCH_TOTAL_COUNT_CAP (default 1000). If the configured count cap is reached before the retrievable total is known, meta.total_is_lower_bound is true; clients should display the returned value with a plus sign. Counting stops as soon as that lower bound is proven. An empty result has total: 0 and total_is_lower_bound: false.

If counting is interrupted on any count page, including the first, retrieval continues. meta.total is then the greater of the partial score-gated count confirmed before the interruption and the number of unpaginated matches returned by retrieval. meta.total_is_lower_bound is true, so the value is not exact and clients continue to display it with +.

When min_score is set, meta.total instead counts matches retained in the current retrieval and scoring window. With reranking enabled, that window grows with the requested offset, and normalized rerank scores can change as the window changes. The resulting total can therefore vary between pages. In this mode total_is_lower_bound is false: the value describes the current scoring window and the score-gated cap signal does not apply.

Examples

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "machine learning applications",
"limit": 5
}'
curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "Python developer",
"alpha": 0.3,
"limit": 10
}'

With Metadata Filters

Filter using system metadata paths when you want system-generated values. Both full metadata.* paths and the documented shorthand paths work:

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "project management",
"filters": {
"classification.categories": "financial reporting"
}
}'

Filter by custom metadata fields (auto-resolved to metadata.custom.*):

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "developer",
"filters": {
"department": "Engineering",
"team": "Backend"
}
}'

Short keys are user metadata. For example, this filters metadata.custom.author, not any system-derived author:

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "project management",
"filters": {
"author": "Jane Doe"
}
}'

Include Soft-Deleted Chunks

Search hides soft-deleted chunks by default. Set include_deleted to true when you need to audit or recover deleted content:

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "project management",
"include_deleted": true
}'

This only removes the live-only exclusion. Results remain scoped to the client resolved from X-API-Key; one client cannot search another client's soft-deleted chunks.

With Entity Filters

Filter by named entities extracted during document processing. Use a single string for an exact match, or a list to match any value (OR semantics within the list).

Single value:

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "quarterly earnings",
"filters": {
"persons": "John Doe"
}
}'

List value (matches any in the list):

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "loan application",
"filters": {
"organizations": ["Bank A", "Bank B"]
}
}'

Combining Multiple Filters

You can combine any number of filters in a single request. All filters are combined with AND logic — a document must match every condition to be returned.

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "quarterly earnings",
"filters": {
"document_id": "bank_of_america_2024.pdf",
"classification.categories": "financial reporting",
"department": "Engineering"
}
}'

This returns only chunks that satisfy all three conditions simultaneously:

  • Belong to bank_of_america_2024.pdf
  • Are classified as "financial reporting"
  • Have custom metadata department: "Engineering"

Custom metadata and entity filters can also be combined:

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "quarterly earnings",
"filters": {
"department": "Engineering",
"project": "alpha",
"persons": "John Doe",
"organizations": ["Acme Corp", "Globex"]
}
}'

This matches chunks where:

  • Custom metadata department is "Engineering"
  • Custom metadata project is "alpha"
  • persons contains "John Doe"
  • organizations contains "Acme Corp" OR "Globex"
Filter Combinations

Any mix of top-level fields (document_id), custom metadata (department, team, author), system metadata paths (classification.categories, source.filename, or their full metadata.* forms), and entity filters (persons, organizations) can be combined. There is no limit on the number of filter keys.

Filter by Category

Search within specific document categories:

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "loan application process",
"filters": {
"metadata.classification.categories": "risk management"
}
}'

Multiple categories (matches any):

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "compliance requirements",
"filters": {
"metadata.classification.categories": ["compliance", "risk management"]
}
}'

With Reranking

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "deep learning neural networks",
"rerank": true,
"top_k_rerank": 20,
"limit": 5
}'

With Score Threshold

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "quarterly earnings",
"min_score": 0.5,
"limit": 20
}'

Pagination

# First page
curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "product roadmap",
"limit": 10,
"offset": 0
}'

# Second page
curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "product roadmap",
"limit": 10,
"offset": 10
}'

With Matched Term Highlights

Set highlight to true when the frontend needs character offsets for query words in the returned chunk.text. Each highlight uses [start, end) offsets into the exact sanitized string returned by the API. The term value is the normalized lowercase query token.

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "invoice 2024",
"highlight": true,
"fields": ["chunk", "highlights"]
}'

Highlights are a lexical UI affordance, not a ranking explanation. Semantic or dense-only results can legitimately return highlights: [] when the exact query terms do not occur in the chunk. In v1, only chunk.text is scanned; metadata, entities, stemming, and server-rendered HTML markup are out of scope.

With Field Projection

Use fields when you only need selected result sections. The response always includes meta; each result includes only the requested top-level fields, including highlights when requested.

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "quarterly earnings",
"fields": ["document", "chunk", "score"]
}'

Legacy aliases such as document_id and chunk_text are accepted and mapped to their nested sections.

Filter by Created Time

created_at is a top-level filter for exact timestamp matches. For date ranges, add a custom metadata field such as report_period or business_date during upload and filter on that field.

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "risk report",
"filters": {
"created_at": "2026-04-09T10:30:00Z"
}
}'

Just Command

As an alternative to curl, you can use the just search command from the terminal:

# Basic search
just search "machine learning"

# Full JSON response
just search "machine learning" -- --json

# Use INFOCONNECT_API_KEY instead of passing --api-key repeatedly
export INFOCONNECT_API_KEY=super-secret-key

# With all options
just search "project management" -- --limit 10 --alpha 0.8 --rerank --filter document_id=doc-123

# With JSON filters, including list-valued metadata filters
just search "budget report" -- --filters '{"tenant_id":"tenant-a","department":["finance","operations"]}' --json

# Return only selected result sections
just search "quarterly report" -- --field document --field chunk --field score

just search now defaults to a concise terminal summary with a compact results table. Use --json when you want the full API response shape for piping or debugging. This is the natural next step after embedding documents with just embed. See the Justfile Commands guide for full details.

Response Codes

CodeDescription
200Success - Returns search results
401Unauthorized - Invalid API key
429Rate Limited - Too many requests
422Validation Error - Invalid parameters

Rate Limiting

Search requests are rate-limited per resolved client id using a sliding-window algorithm. Default: 30 requests per minute per client for protected routes, with an additional search-specific counter for POST /search. Configure via RATE_LIMIT environment variable. Multiple API keys mapped to the same client share the same budget.

For a full explanation of how the limiter works, its scope, and production recommendations, see the Rate Limiting guide.

Filterable Fields

Use the filters parameter to narrow results.

Filter Logic

All filter conditions are combined with AND logic across keys — every condition must match.

For metadata, custom fields, and entity filters, list values use OR logic within that field:

{
"tenant_id": "tenant-a",
"department": ["finance", "operations"],
"organizations": ["Bank A", "Bank B"]
}

This means the document must belong to tenant-a AND have department equal to finance OR operations AND mention Bank A OR Bank B.

Metadata list values must be homogeneous strings or homogeneous integers. Use scalar values for booleans such as metadata.quality.ocr_used.

Top-Level Fields (Indexed)

FieldTypeExampleDescription
document_idstring"doc-123"Document identifier
created_atdatetime"2026-04-09T10:30:00Z"Chunk creation timestamp

Soft-delete marker fields (deleted, deleted_at, and deleted_by) are reserved system fields. They are not accepted as user filters; use the top-level include_deleted request flag to include soft-deleted chunks.

Custom Metadata Fields

Any custom metadata field uploaded via /embed or /update can be used as a filter. These are stored under metadata.custom.* and resolved automatically:

# Upload with custom metadata
curl -X POST http://localhost:8000/embed \
-F "file=@doc.pdf" \
-F "document_id=doc-123" \
-F 'metadata={"department":"Engineering","team":"Backend","priority":"high"}'

# Search with custom metadata filter
curl -X POST http://localhost:8000/search \
-H "Content-Type: application/json" \
-d '{
"query": "developer",
"filters": {
"department": "Engineering",
"team": "Backend"
}
}'

The filter key department is automatically resolved to metadata.custom.department in Qdrant.

Custom metadata also supports list values for cross-department or multi-team searches:

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{
"query": "budget report",
"filters": {
"tenant_id": "tenant-a",
"department": ["finance", "operations"]
}
}'

This searches within tenant-a across both the finance and operations departments.

System-Generated Metadata Paths

The pipeline automatically extracts and stores structured metadata during document processing. Use the shorthand keys below, or prefix any of them with metadata. if you prefer the full Qdrant payload path.

Filter KeyFull PathExample Value
source.filenamemetadata.source.filename"doc.pdf"
source.mime_typemetadata.source.mime_type"application/pdf"
source.extensionmetadata.source.extension"pdf"
location.page_numbermetadata.location.page_number1
location.page_labelmetadata.location.page_label"1"
location.chunk_indexmetadata.location.chunk_index0
document.page_countmetadata.document.page_count10
classification.categoriesmetadata.classification.categories"risk management"
classification.statusmetadata.classification.status"completed"
quality.text_extractionmetadata.quality.text_extraction"native" or "ocr"
quality.ocr_usedmetadata.quality.ocr_usedtrue or false
extracted.titlemetadata.extracted.title"Annual Report"
extracted.authormetadata.extracted.author"Jane Doe"
extracted.created_atmetadata.extracted.created_at"2026-04-09"
Categories

Categories are automatically assigned during document processing using the local ModernBERT ONNX classifier. See the Auto-Categorizing Your Documents guide to learn how to customize or disable this feature.

Entity Filter Keys

Entity filters target payload.entities.{key} in Qdrant. These are extracted by spaCy NER during document processing. You can filter by any of the following keys using a single string or a list of strings:

Filter KeyExample ValueDescription
persons"John Doe" or ["John Doe", "Jane Smith"]People mentioned in the document
organizations"Bank of America" or ["Bank A", "Bank B"]Companies, banks, institutions
dates"2024-01-15" or ["2024-01-15", "2024-03-20"]Dates and time expressions
locations"New York" or ["New York", "London"]Geographic locations
monetary_amounts"$1,000,000" or ["$1,000,000", "€500,000"]Currency and monetary values
account_numbers"1234567890" or ["1234567890", "0987654321"]Account numbers
transaction_refs"TXN-12345" or ["TXN-12345", "REF-67890"]Transaction reference numbers
account_types"savings" or ["savings", "checking"]Types of accounts

Entity filters support OR within a list and AND across keys. For example:

{
"persons": "John Doe",
"organizations": ["Bank A", "Bank B"]
}

This matches chunks where persons contains "John Doe" AND organizations contains either "Bank A" OR "Bank B".

score.strategy uses dense, sparse, or hybrid for retrieval mode. It does not change to reranked; when cross-encoder reranking runs, meta.retrieval.reranked is true and score.rerank contains the normalized reranker score.

Error Responses

CodeWhen It HappensHow to Fix
401API key is missing or not in the configured API_KEY_CLIENTS mappingInclude a valid X-API-Key header
422Request body fails Pydantic validation (bad types, out-of-range values, missing fields, extra fields)Check the details array for the exact field and constraint violated
429The resolved client has exceeded the protected-route or search-specific rate limit (default: 30 requests/minute)Wait for the sliding window to reset or increase RATE_LIMIT
500Unexpected server error (Qdrant unavailable, embedding model failure, etc.)Retry with exponential backoff; check service health at /health

401 — Unauthorized

Returned when the X-API-Key header is missing or does not match any key in the API_KEY_CLIENTS environment variable.

{
"detail": "Invalid API key",
"error_code": "auth_failed",
"request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e"
}

Fix: Ensure you are sending a valid key:

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{"query": "test"}'

422 — Validation Error

Returned when the request body violates the SearchRequest schema. The API returns a stable error envelope and a details array pinpointing every invalid field.

Missing required field

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{"limit": 5}'
{
"detail": "Request validation failed",
"error_code": "validation_error",
"request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e",
"details": [
{
"type": "missing",
"loc": ["body", "query"],
"msg": "Field required",
"input": {"limit": 5}
}
]
}

Empty query string

The query field must contain at least one character.

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{"query": ""}'
{
"detail": "Request validation failed",
"error_code": "validation_error",
"request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e",
"details": [
{
"type": "string_too_short",
"loc": ["body", "query"],
"msg": "String should have at least 1 character",
"input": ""
}
]
}

Alpha out of range

alpha must be between 0.0 and 1.0 inclusive.

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{"query": "test", "alpha": 1.5}'
{
"detail": "Request validation failed",
"error_code": "validation_error",
"request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e",
"details": [
{
"type": "less_than_equal",
"loc": ["body", "alpha"],
"msg": "Input should be less than or equal to 1",
"input": 1.5
}
]
}

Limit out of range

limit must be between 1 and 100.

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{"query": "test", "limit": 500}'
{
"detail": "Request validation failed",
"error_code": "validation_error",
"request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e",
"details": [
{
"type": "less_than_equal",
"loc": ["body", "limit"],
"msg": "Input should be less than or equal to 100",
"input": 500
}
]
}

Extra fields not allowed

The SearchRequest model uses extra="forbid", so unknown fields are rejected.

curl -X POST http://localhost:8000/search \
-H "X-API-Key: super-secret-key" \
-H "Content-Type: application/json" \
-d '{"query": "test", "unknown_field": true}'
{
"detail": "Request validation failed",
"error_code": "validation_error",
"request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e",
"details": [
{
"type": "extra_forbidden",
"loc": ["body", "unknown_field"],
"msg": "Extra inputs are not permitted",
"input": true
}
]
}
Reading Validation Errors

The loc array tells you exactly which field failed (["body", "query"], ["body", "alpha"], etc.). The msg field contains a human-readable description of the constraint violated.

429 — Rate Limited

Returned when the resolved client has made more requests than allowed in the current sliding time window (default: 30 requests per minute). POST /search can be blocked by either the protected-route counter or the search-specific counter.

{
"detail": "Rate limit exceeded",
"error_code": "rate_limited",
"request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e"
}

Fix: Wait a few seconds and retry, or increase the RATE_LIMIT environment variable.

500 — Internal Server Error

Returned for unexpected failures such as Qdrant being unreachable, embedding model errors, or vector dimension mismatches. These are not client-recoverable.

{
"detail": "Internal server error",
"error_code": "internal_error",
"request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e"
}

Fix: Check the /health endpoint to verify Qdrant, Redis, and embedding service status. Retry with exponential backoff.