What is Chunking?
Turkish: Chunking (Metin Parçalama)
Chunking splits long text into meaningful, manageable passages that search and RAG systems can retrieve accurately.
What is Chunking?
Chunking is the process of splitting long documents into passages that a model and search system can handle. The goal is not just to cut text at equal lengths, but to preserve meaningful context.
If a user manual is embedded as one large block, search results may become too broad. If it is split into tiny fragments, the needed context may disappear. This balance directly affects the quality of RAG answers.
Chunking Approaches
- Fixed length: Splits by a set token or character count, simple to implement
- Heading-based: Uses sections and subheadings to preserve document structure
- Semantic: Detects topic shifts and creates more natural boundaries
- Overlap: Keeps small repeated spans between neighboring chunks to reduce context loss
Business Use
Chunking is used for support articles, contracts, product catalogs, technical documentation, and company procedures. The chunks are turned into embedding vectors and searched in a vector database.
Good chunk design needs measurement. The right chunk size depends on the data type, query length, and model context capacity. Evaluation sets should regularly test whether the system retrieved the correct source.
Related Terms
An embedding represents text, images, products, or other data as numeric vectors that can be compared for semantic similarity.
RAG (Retrieval-Augmented Generation)RAG is an AI architecture where a language model retrieves relevant passages from documents or databases before generating an answer.
Vector DatabaseA vector database stores embeddings and retrieves records by semantic similarity, making it a core layer in AI search systems.