Skip to main content

Setting Up

Before you can upload documents and search them, you need to tell the system how to run. This is done through a configuration file called .env. Think of it like setting up your preferences before using an app.

What You Need​

  • The repository downloaded to your computer
  • A text editor (like Notepad, VS Code, or any code editor)
  • About 5 minutes

Step 1: Find the Example File​

In the main folder of the project, you'll see a file called .env.example. This file contains all the settings with sensible defaults.

# Navigate to your project folder (if you're not already there)
cd infoconnect-search-engine

# List files and look for .env.example
ls -la

You should see .env.example in the list along with other files. This output was captured from a real run on 2026-06-11:

-rw-rw-r-- 1 dnth dnth  7032 Jun 10 19:32 .env
-rw-rw-r-- 1 dnth dnth  7076 Jun 10 15:54 .env.example
drwxrwxr-x 2 dnth dnth  4096 Jun 10 21:14 docker
-rw-rw-r-- 1 dnth dnth 21481 Jun 11 12:12 justfile
-rw-rw-r-- 1 dnth dnth 27746 Jun 11 12:00 openapi.json

Step 2: Create Your Own .env File​

The easiest way to get started is to copy the example file:

cp .env.example .env

This creates a new .env file with all the default settings:

ls -la .env

Output from the same run:

-rw-rw-r-- 1 dnth dnth 7032 Jun 10 19:32 .env

Or you can do it manually:

  1. Open .env.example in your text editor
  2. Save it as a new file called .env (just .env, no extra name before the dot)
tip

The .env file is special — it holds your configuration but is kept private. It won't be shared if you upload your code somewhere (it's listed in .gitignore).

Step 3: Understanding the Settings​

Open your new .env file. It is organized into seven sections. Here's what the important ones do:

1. Identity & Access​

API_KEY_CLIENTS={"test-key-123":"demo-client","another-key":"demo-client","super-secret-key":"demo-client","finance-demo-key":"finance-demo-client","ops-demo-key":"ops-demo-client"}
RATE_LIMIT=30

API_KEY_CLIENTS maps each API key to the client id that owns its data. Include one configured key in the X-API-Key header for every request. RATE_LIMIT controls how many protected-route requests each resolved client can make per minute; /search also has a search-specific per-client counter.

warning

If you change these keys, remember what you set them to. You'll need one of them for every request.

2. Core Behavior​

DEBUG=true
CORS_ORIGINS=["http://localhost:3000","http://localhost:8080"]
LOG_LEVEL=INFO
RETRIEVAL_TOP_K=10
RERANK_TOP_N=5
MAX_FILE_SIZE_MB=200
ALLOWED_EXTENSIONS=[".pdf",".docx",".txt",".md",".text",".pptx",".ppt",".xlsx",".xls",".png",".jpg",".jpeg",".tif",".tiff"]

These settings control the API surface, upload limits, CORS origins, logging, and default search result counts. ALLOWED_MIME_TYPES sits beside ALLOWED_EXTENSIONS and validates the uploaded file's content type.

3. Domain Taxonomy​

ALLOWED_CATEGORIES=["financial reporting","risk management","corporate governance","regulatory compliance","business operations","market analysis","legal","human capital","technology","sustainability","KYC","Credit","Collateral","Ledger","other"]
NER_ENTITY_KEYS=["persons","organizations","dates","locations","monetary_amounts","account_numbers","transaction_refs","account_types"]

ALLOWED_CATEGORIES controls the labels ModernBERT can assign. NER_ENTITY_KEYS controls which extracted entity groups appear in API responses and search filters. Both values must stay in JSON array format.

4. Pipeline Tuning​

CHUNK_SIZE=512
CHUNK_OVERLAP=50
EMBEDDING_BATCH_SIZE=32
MAX_NODES_PER_TASK=50
RAPIDOCR_OCR_ENABLED=true
OCR_PAGE_TEXT_THRESHOLD=80

These settings tune document chunking, embedding batch sizes, and OCR fallback behavior. Keep them unchanged for your first run.

5. Classification & Entity Extraction​

CLASSIFICATION_ENABLED=true
SPACY_NER_ENABLED=true
CLASSIFIER_BACKEND=modernbert-onnx
SPACY_MODEL_NAME=en_core_web_sm
CLASSIFICATION_SELECT_TOP_K=3
MODERNBERT_PREFILTER_ENABLED=false

These settings enable automatic categorization and named-entity extraction. The ModernBERT options below this section control the ONNX model file, threshold, token limits, and thread counts.

6. AI / ML Models​

DENSE_MODEL_NAME=BAAI/bge-small-en-v1.5
SPARSE_MODEL_NAME=Qdrant/bm25
RERANK_MODEL_NAME=Xenova/ms-marco-MiniLM-L-6-v2
FASTEMBED_CACHE_DIR=./models/fastembed
HF_HOME=./models/huggingface

These are the embedding, sparse retrieval, reranking, and model-cache settings. The defaults work well for most cases and models download automatically on first use.

7. Infrastructure​

REDIS_URL=redis://localhost:6379/0
CELERY_BROKER_URL=redis://localhost:6379/1
CELERY_RESULT_BACKEND=redis://localhost:6379/2
QDRANT_URL=http://localhost:6333
COLLECTION_NAME=documents

These tell the system where to find Redis, Celery, and Qdrant. The defaults work when running everything locally through Docker Compose.

Quick Start Configuration​

For your first time, you only need to check a few things:

  1. API_KEY_CLIENTS: Keep the defaults or map each key to your client id
  2. CLASSIFICATION_ENABLED: Leave enabled if you want automatic categories
  3. Everything else can stay as-is

Next Steps​

Once your .env file is ready:

  1. Install the system if you haven't already
  2. Upload your first document

Common Problems​

ProblemCauseSolution
"File not found" when copyingWrong directoryMake sure you're in the project root where .env.example lives
Changes not taking effectForgot to restartStop and restart the services after changing .env
Permission deniedFile ownershipOn Linux/Mac, try chmod 644 .env

What's Next​