Document Chunking

Document chunking is the technique of breaking long text documents, web pages, and PDFs into smaller, semantically coherent segments (e.g. 500–1000 characters) before generating vector embeddings for retrieval in a RAG pipeline.

Detailed Explanation

If text chunks are too large, vector embeddings become diluted and retrieval accuracy drops. If chunks are too small, critical context is lost. CustomerGPT implements an optimal sliding-window chunking strategy with an 800-character window and 150-character overlap to preserve semantic continuity and header hierarchy.

How It Works in CustomerGPT

CustomerGPT automatically parses PDFs, Word docs, and web pages, applying intelligent header-aware chunking before saving vectors to pgvector.

Related Glossary Terms

Retrieval-Augmented Generation (RAG) →Vector Embeddings →

Frequently Asked Questions

Why is chunk overlap necessary?

Chunk overlap ensures that sentences or concepts split across boundary edges retain sufficient context for accurate vector retrieval.

Deploy Grounded RAG AI for Your Support

Get started in under 2 minutes with our 14-day free trial.

Start Free Trial →