Agentic RAG System

The Agentic RAG System is a research assistant built over a corpus of 100 AI-engineer job postings and 250 arXiv AI/ML abstracts. Rather than a plain 'chat with your docs' pipeline, the project centers on two things most student RAG projects skip: an agent layer that makes retrieval decisions instead of blindly stuffing context into a prompt, and an evaluation harness that proves the system actually works, with numbers to back it up. It's exposed as a FastAPI service with a Streamlit front end, containerized with Docker, and backed by a Chroma vector store of 2,299 chunks. Built end-to-end over six weeks: corpus and ingestion, retrieval tuning, the LangGraph agent, the service layer, evaluation, and final polish.

Job-market analysis of 100 AI-engineer postings showed the modern AI-engineer role is less about training models from scratch and far more about building systems on top of LLMs — retrieval pipelines, agent orchestration, and getting them into production. 'Agentic AI' and 'RAG' each appear in roughly half of all postings and co-occur heavily; LLMs appear in 84%. My prior portfolio was strong on classical ML (XGBoost, K-Means, NLP with logistic regression and random forest, PySpark) but predated the LLM era entirely. This project closes that gap, and does it in a way that stands out from the flood of 'chat with your PDF' clones that can't answer the basic interview question: is it actually good?

Scoped and built in six weekly phases: (1) locked scope, ingested and chunked both corpora, generated embeddings, loaded them into Chroma, and shipped an ugly-but-working baseline; (2) tuned chunk size and k, and hand-built a gold set of 20 question/expected-source pairs to measure retrieval hit-rate objectively; (3) converted the linear pipeline into a LangGraph agent that decides whether retrieved context is relevant, reformulates the query and retries (capped at 2) when it isn't, and only generates once it has good context or exhausts retries; (4) wrapped the agent in a FastAPI backend with source citations in the response payload, added a Streamlit UI, and containerized with Docker; (5) measured retrieval hit-rate against the gold set and answer faithfulness via an LLM-as-judge, comparing the agent against a simple non-agentic pipeline to quantify what the agent layer actually buys; (6) polish, README, and write-up. The guiding principle throughout was to keep scope tight and ship — every week had one concrete deliverable.

Retrieval hit-rate at k=10 reached 90% (18/20) against the hand-curated gold set, versus 85% (17/20) at k=5. On answer faithfulness (LLM-as-judge, 1–3 scale), the agent with query reformulation scored 2.80/3 versus 2.75/3 for a simple non-agentic pipeline — direct evidence the agent layer earns its complexity rather than just adding latency. Honest limitation: that faithfulness delta is small; a larger corpus or harder, more ambiguous questions would likely widen the gap. The system runs Chroma locally rather than a managed cloud vector database, has no streaming responses, and caps reformulation at 2 retries as a reasonable default rather than a tuned value.

outcome: Query-reformulation agent beat a simple RAG pipeline on answer faithfulness (2.80 vs. 2.75/3) at a 90% retrieval hit-rate, proven with a custom eval harness.

• LangGraph
• RAG
• FastAPI
• Chroma
• Docker