Most RAG systems start with text: documents, paragraphs, tickets, or code snippets. Real knowledge bases are messier. Important evidence often lives in tables, charts, diagrams, screenshots, scanned PDFs, audio, and video.

Multimodal RAG extends the retrieval pipeline beyond plain text. The system extracts, describes, embeds, retrieves, and cites information from multiple modalities, then passes the right evidence to a model that can use it.

This chapter covers practical strategies for PDFs, tables, images, audio, video-derived content, multimodal embeddings, and the cost trade-offs that decide how far to go.