Legal Document Intelligence RAG

A versioned public-document pipeline turns court opinions and SEC filings into 3,055 searchable chunks and returns evidence-linked answers with an explicit refusal path.

Open project Source

Abstract document pages passing through a retrieval and citation system

STAKEHOLDER VIEW

What this project is for.

Problem: Legal teams need answers that stay attached to the source, especially when documents are long and high stakes. This project demonstrates an evidence-first research workflow over public M&A litigation and transaction documents: a reviewer asks a question, receives a grounded answer, and can inspect the cited document, section, and page rather than trusting an unsupported response.
Intended user: A legal-technology reviewer or AI engineering hiring manager evaluating trustworthy document retrieval and deployment practices.
Decision supported: Whether the public corpus contains retrievable, traceable evidence for the research question.
Outcome: An Azure-hosted, evidence-first RAG workspace that makes every supported answer traceable to a registered public legal source.
What to try: Ask a question about a document in the registered public corpus
Important limitation: The public endpoint is anonymous and does not have the controls required for confidential legal matters.

Ask a legal research question and inspect the evidence

01Ask a question about a document in the registered public corpus
02Review the grounded answer or explicit refusal
03Open the numbered citations and inspect case, section, page, and canonical source
04Compare release metrics with the committed evaluation artifact

Technical reviewArchitecture, evidence, controls, deployment, and trade-offs

ARCHITECTURE

One source, one rendered system view.

Versioned adapters ingest public PDFs through Azure Document Intelligence and public SEC HTML through a native parser, normalize them into stable records, and build structure-aware chunks. Azure AI Search retrieves the promoted public corpus; Azure OpenAI generates only from that evidence, while the Flask workspace exposes research, corpus, evaluation, and health views. Managed identity separates deployed resources from local credentials, and the corpus index is promoted independently from application deployment.

System architecture showing a legal researcher, registered public court and SEC sources, checksum validation, extraction, chunk storage, Azure AI Search, Azure OpenAI, managed identity, observability, deployment, cited output, and evaluation. — Pan to inspect the system; zoom controls are available when detail is needed.

End-to-end flow

01Public source registration and checksum validation
02Azure Document Intelligence and native SEC HTML extraction
03Structure-aware chunking and embeddings
04Azure AI Search retrieval
05Citation-required grounded generation and refusal
06Read-only Flask research workspace on Azure App Service

Technology stack

Microsoft Azure

Azure Document IntelligenceAzure OpenAIAzure AI SearchAzure Blob Storage

PythonFlask

Technology decisions

Decision	Why	Alternative	Trade-off
Azure AI Search with Azure OpenAI	Managed retrieval, identity, and model services support a deployable Azure reference architecture.	Local Chroma and API-key based model access	The managed path improves deployment boundaries but costs more and requires explicit corpus-index promotion.
Vendor-neutral extraction records	Azure SDK objects terminate at the adapter so parsing, chunking, evaluation, and storage remain testable.	Propagate Azure SDK types through the pipeline	The stable boundary adds mapping code but reduces provider coupling and keeps tests deterministic.

Evaluation and evidence

Metric	Plain-language meaning	Score / value	Dataset / scenario	Threshold / baseline	Interpretation	Evidence
retrieval hit rate@8	What retrieval hit rate@8 measures for the stated evaluation scope.	1.0	45-question gold-qa-v2-delaware-expansion benchmark over the promoted 3,055-chunk public index	No threshold or baseline recorded.	The benchmark measures retrieval hit rate and citation provenance, not legal-answer correctness.	evaluation.release-v2
public retrieval chunks	What public retrieval chunks measures for the stated evaluation scope.	3,055	Promoted r3 Azure AI Search corpus with r2 retained for rollback.	No threshold or baseline recorded.	The public endpoint is anonymous and does not have the controls required for confidential legal matters.	disclosure.public-corpus

These are versioned integration evaluation against the promoted public retrieval index plus a live reachability observation results, not a production service-level objective. Unknown values are shown as “Not recorded”; units and special characters retain their source meaning.

Technical terms and value conventions

Term	Plain-language use	How this project uses it
Evaluation score	A measured result from a defined test or benchmark, not a general promise.	Read it with the dataset, scenario, and limitation shown in the evaluation table.
Unknown / null	A value that was not observed or is intentionally not claimed.	Rendered as “Not recorded” rather than silently converted to zero or omitted.
Special characters	Symbols such as %, currency, Unicode, and escaped input retain their source meaning.	Units and encoding are explained where they affect the decision.

Data boundary

Classification	Source	Permitted use	Excluded data
public	Official public Delaware court opinions and public SEC transaction documents registered with source URLs and checksums	Read-only portfolio demonstration, retrieval evaluation, and engineering research with source attribution	Client, firm, user-uploaded, confidential, privileged, and personal matter data; Private credentials, local environment files, and raw authorization material; Legal opinions, attorney work product, and production discovery decisions

Uses public court opinions and public SEC transaction documents. The application provides legal information for engineering demonstration and is not legal advice.

Security and privacy controls

Control	Implementation	Evidence	Limitation
Registered public-source boundary	Every corpus document is registered with its canonical public URL and checksum before extraction.	disclosure.public-corpus	Public availability does not make every source term identical; source-specific reuse terms still apply.
Managed identity	The deployed application authenticates to Azure resources without committed application credentials.	deployment.azure-root	Managed identity does not turn the anonymous interface into a confidential legal-matter system.
Citation-required generation	Answers are generated only from retrieved public evidence and retain source, section, page, and checksum provenance.	evaluation.release-v2	Retrieval and provenance metrics do not establish attorney-reviewed answer correctness.

Deployment and cost boundary

Provider	Runtime	State	Exposure	Verified	Production claim
Azure	Flask and Gunicorn on Azure App Service with managed identity, Azure AI Search, and Blob Storage	live	anonymous	2026-07-23T20:57:44Z	No

Component	Boundary	Implication
Azure AI Search and Azure OpenAI	Owner-managed Azure services with a promoted 3,055-chunk public index.	Managed retrieval and model calls incur cloud cost and require explicit index-release coordination.
Azure App Service and Blob Storage	Read-only anonymous portfolio demonstration over public documents.	Confidential uploads, retention, malware scanning, and production monitoring are outside the approved scope.

Known limitations

The public endpoint is anonymous and does not have the controls required for confidential legal matters.
The evaluation does not establish legal-answer correctness or legal advice quality.
Some source-specific reuse terms may differ even though the documents are available from official public sources.
HTTP reachability does not establish an availability, confidentiality, security, or production SLO.

Scalability roadmap

Add attorney-reviewed answer and citation-correctness evaluation beyond retrieval and provenance checks
Replace the residual style-based legal-outline parser heuristics with a branch-aware outline state machine
Add authenticated matter isolation, encrypted private storage, malware scanning, retention, and audit controls before any user uploads
Move ingestion and index construction to durable background workflows with reviewed index promotion
Add quota, cost, abuse, availability, and security monitoring appropriate to an owner-approved production service

All projects