Context
Customers held the answers they needed in two places: documents such as PDF, Word, Excel and PowerPoint files, and operational databases on SQL Server and PostgreSQL.
Getting an answer meant reading reports or asking an analyst to write SQL. The goal was to let people simply ask.
The challenge
Answers had to be grounded in the customer’s own data, respect who is allowed to see what, and stay accurate as prompts and models change. A demo that works once was not enough; it had to be dependable in production.
What we did
- 01
Retrieval over documents
Built RAG over PDF, Word, Excel and PowerPoint: chunking, embeddings and vector search with pgvector, so answers draw on the right passages.
- 02
Agents that query live data
Built multi-step agents with tool calling and MCP that turn a question into validated SQL over SQL Server and PostgreSQL, instead of trusting generated queries blindly.
- 03
A streaming "Ask AI" API
Served answers through a FastAPI service with server-sent events, so responses stream as they are generated.
- 04
Monitoring and evaluation
Traced every request with OpenTelemetry and Arize Phoenix, and added evaluations that run on every model or prompt change.
Results
- Plain-language questions answered from documents and live databases.
- Generated SQL validated before it runs.
- Streaming answers in the product experience.
- Quality checked on every model or prompt change.
