+ GenAI & RAG
PDF AI Chatbot with Local LLM
Private document Q&A: a RAG chatbot running Llama-3.2-3B locally with LlamaCpp, so no API costs and no data leaves the machine.
Overview
A privacy-first RAG system for chatting with PDF documents. PDFs are parsed, chunked and embedded into ChromaDB, and answers are generated by a quantised Llama-3.2-3B-Instruct model running locally on CPU through LlamaCpp, with streaming responses and multi-turn chat.
Highlights
- Runs fully locally on CPU: zero API cost, full data privacy
- Quantised Llama-3.2-3B-Instruct via LlamaCpp
- RAG with ChromaDB semantic retrieval
- Streaming, multi-turn chat about documents
Tech stack
Gallery
