Skip to main content

Rate Limiting

The InfoConnect Hybrid Search Engine uses sliding-window rate limiting to protect authenticated API routes from abuse and to ensure fair usage across clients.

How It Works​

Sliding-Window Algorithm​

Rate limiting tracks every request in a rolling 60-second window rather than a hard reset every minute. This prevents burst attacks at the top of each fixed window.

What happens on every request:

  1. Evict old timestamps — drop any request records older than 60 seconds.
  2. Check the count — if the remaining records in the window are greater than or equal to the limit, the request is blocked with HTTP 429.
  3. Record and allow — otherwise the current timestamp is stored and the request proceeds.

Example Behavior​

With a limit of 2 requests per minute:

TimeEventResultWindow State
t=0sRequest 1Allowed[0]
t=1sRequest 2Allowed[0, 1]
t=2sRequest 3Blocked (429)[0, 1] — count == limit
t=61sRequest 4Allowed[1, 61] — 0 has aged out

Because the window slides continuously, a burst at t=59s and t=61s does not allow 4 requests in 2 seconds. The oldest record is evicted only after it is more than 60 seconds old.

Configuration​

SettingDefaultControl
RATE_LIMIT30Requests per minute for authenticated protected routes
AUTH_FAILURE_RATE_LIMIT10Failed authentication attempts per minute per source address

Set RATE_LIMIT=0 to disable the built-in limiter. This does not remove any external reverse-proxy or edge limits.

When a limit is exceeded, the API returns HTTP 429 with rate-limit headers; see the response example below for the exact header names and payload shape.

Quick Setup​

.env
# 30 requests per minute per resolved client id (default)
RATE_LIMIT=30

# 10 failed auth attempts per minute per source (default)
AUTH_FAILURE_RATE_LIMIT=10

# Higher per-client limit for a trusted internal deployment
RATE_LIMIT=60

# Disable the built-in limiter (not recommended for production)
RATE_LIMIT=0
Per-Client Limits

Protected route limits are keyed by resolved client id. Failed-auth limits are keyed by source address and apply before a request can resolve to a client.

Scope​

Rate limiting is applied in two layers:

  • A protected-route middleware checks every authenticated route except /health, /metrics, /docs, /redoc, /openapi.json, and OPTIONS CORS preflight requests.
  • POST /search keeps an additional search-specific counter because search remains the most frequent expensive operation.
  • POST /web-search keeps an additional web_search:{client_id} counter.

Currently Protected Endpoints​

EndpointRate-Limit Key
Protected authenticated routesprotected:{client_id}
POST /search additional route checksearch:{client_id}
POST /web-search additional route checkweb_search:{client_id}
Failed authenticationauth-failure:{source_address}

Unprotected Endpoints​

The following endpoints do not check the rate limiter:

  • GET /health — health checks
  • GET /metrics — Prometheus metrics
  • /docs, /redoc, /openapi.json — development documentation surfaces
  • OPTIONS preflight requests — handled by CORS middleware
Search and Web Search Have Extra Counters

POST /search consumes both the global protected-route budget and its search-specific budget. POST /web-search likewise consumes the global budget and its web_search:{client_id} budget. These route checks preserve separate protection for the expensive operations while the global middleware also protects upload, status, document, and update routes.

Response Format​

When the limit is exceeded, the API returns:

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
X-Request-ID: 5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e
X-RateLimit-Limit: 30
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 42

{
  "detail": "Rate limit exceeded",
  "error_code": "rate_limited",
  "request_id": "5d7f4c1a2b9e4b3e8f6f0d2a4c9b8a1e"
}

Response Codes​

CodeDescription
200Success
401Unauthorized — invalid API key
429Rate Limited — too many requests
422Validation Error — invalid parameters

Runtime State​

Redis-Backed Sliding Window​

When the FastAPI lifespan starts, the built-in limiter stores request timestamps in Redis using an atomic sorted-set script. This means:

  • Shared across API workers — all workers enforce the same client counters through REDIS_URL.
  • Reset on Redis data loss — counters survive worker restarts, but clearing Redis data clears the 60-second windows.
  • No client-supplied tenancy — protected-route counters are keyed by the server-resolved client_id, and auth-failure counters are keyed by source address.

The in-memory limiter remains only as a test fallback when no Redis client is configured.

Production Recommendations​

For normal production Compose deployments, the built-in Redis-backed limiter is sufficient for per-client request caps. For public internet exposure or DDoS protection, layer an edge or reverse-proxy limiter in front of the API as a broader network control.

LayerToolWhen to Use
Built-inRedis-backed RATE_LIMIT env varPer-client API quota across API workers
Reverse proxyNginx limit_req, Traefik middlewareNetwork-level throttling before requests reach FastAPI
Edge / CDNCloudflare rate limiting, AWS WAFPublic-facing APIs, DDoS protection

Testing Rate Limits​

You can test the limiter with a quick loop:

for i in {1..35}; do
  curl -s -o /dev/null -w "%{http_code}\n" \
    -X POST http://localhost:8000/search \
    -H "X-API-Key: super-secret-key" \
    -H "Content-Type: application/json" \
  -d '{"query": "test", "scope": "content"}'
done

With the default limit of 30, the first 30 requests for one resolved client should return 200 and later requests within the same 60-second window should return 429, even when the API runs multiple workers.

The application and Compose fallbacks use RATE_LIMIT=30. The repository .env.example uses RATE_LIMIT=300 as a local-testing override; copying that file therefore changes the effective limit until you set the value you intend to operate.

What's Next​