Turn Your Raspberry Pi into a Chat-with-PDF AI Assistant
The Raspberry Pi 5 is powerful enough to handle some surprisingly advanced AI tasks. Today, we’ll show you how to build a Chat-with-PDF Assistant on your Pi — a local tool that lets you upload PDF files and ask questions about them using natural language, just like ChatGPT.
No need for the cloud. Everything runs locally on your Raspberry Pi.
What You’ll Build
A local AI assistant that can:
Ingest PDF files
Convert text to vector embeddings
Accept questions in plain English
Answer based on PDF contents
All without sending your documents to the cloud.
What You’ll Need
Raspberry Pi 5 (or Pi 4 with swap enabled)
64-bit Raspberry Pi OS
Python 3.9+
pip and virtualenv
PDF document(s) to test with
Internet connection (initial setup only)
Step 1: Install Required Packages
First, update your Pi and install basic dependencies:
sudo apt update && sudo apt upgrade -y
sudo apt install python3-pip python3-venv
Create and activate a virtual environment:
python3 -m venv pdfbot-env
source pdfbot-env/bin/activate
Now install the core Python libraries:
pip install --upgrade pip
pip install langchain faiss-cpu pypdf openai tiktoken
Optionally:
Replace openai with llama-cpp-python or transformers if you want to use local models instead of OpenAI’s API.
Step 2: Load and Split Your PDF
from langchain.document_loaders import PyPDFLoader
from langchain.text_splitter import CharacterTextSplitter
loader = PyPDFLoader("your_file.pdf")
pages = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
documents = text_splitter.split_documents(pages)
Step 3: Embed and Store Text
Use FAISS to store searchable vectors:
from langchain.vectorstores import FAISS
from langchain.embeddings.openai import OpenAIEmbeddings
embedding = OpenAIEmbeddings()
db = FAISS.from_documents(documents, embedding)
Step 4: Set Up the Chat Function
from langchain.chains.question_answering import load_qa_chain
from langchain.llms import OpenAI
chain = load_qa_chain(OpenAI(), chain_type="stuff")
def ask(query):
docs = db.similarity_search(query)
answer = chain.run(input_documents=docs, question=query)
return answer
Example usage:
print(ask("What does the report say about revenue growth in 2023?"))
️ Optional: Use a Local Model Instead of OpenAI
Replace OpenAI() with a local LLM like llama.cpp, or use an API like Ollama running on your Pi. Just swap out the LLM in load_qa_chain.
Final Notes
Raspberry Pi 5 handles vector search + API queries easily.
RAM-intensive operations like LLM inference locally may require swap files or 8GB model variants.
For full local use, combine with llama.cpp and a local embedding model.
Summary
You’ve now built a Chat-with-PDF Assistant that runs on your Raspberry Pi and can answer questions about any document you upload. It's private, fast, and extendable.
