Chunking Strategies
A visual guide: every parameter, every picture, every example
How to read this guide |
The two sample texts used everywhere in this guide |
SHORT TEXT (54 characters) |
PARAGRAPH TEXT (the symbol ¶ marks a newline) (182 characters) |
A language model cannot read a 40-page PDF for every question, and an embedding model cannot turn one whole PDF into one useful vector. So the document is cut into pieces called chunks. Each chunk becomes one vector, one row in the database, and one possible search result.
A chunk is the unit of retrieval: the smallest thing your search can return.
The one sentence that explains why all these strategies exist |
Before looking at the strategies one by one, it helps to know that they are mostly re-mixes of four ideas. If you understand these four, the parameter lists stop looking random.
Unit, size, overlap, separator. Almost every parameter you will meet is one of these.
Knob | Typical parameter name | Plain meaning | Turn it up and... | Turn it down and... |
Unit | (implied by the strategy) | What one 'thing' is: a character, a token, a sentence, a paragraph. | - | - |
Size | chunk_size, sentences_per_chunk, paragraphs_per_chunk | The maximum amount of that unit allowed in one chunk. | fewer, longer chunks with more context and more noise | more, shorter chunks that are precise but may cut an idea |
Overlap | chunk_overlap, stride | How much of the end of one chunk is repeated at the start of the next. | fewer answers land on a seam, but storage and duplicate hits grow | no duplication, but facts sitting on a boundary can be lost |
Separator | separator, separators, keep_separator | The text pattern the splitter is allowed to cut at. | - | - |
Size is a limit, not a target |
The question this part answers |
fixed_size works in three steps. Step 1 is the one beginners never expect.
So the three parameters are not competing, they do three different jobs in three different steps:
Parameter | Which step it controls | In plain words |
separator | Step 1: where cuts are legal | 'You may only cut here.' If the pattern is not in the text, no cut can happen. |
chunk_size | Step 2: how much to pack | 'Keep adding pieces while the total stays under this number.' |
chunk_overlap | Step 3: what to repeat at a seam | 'When you start a new chunk, first copy this many characters back from the previous one.' |
keep_separator | Step 1 output | 'When you cut, throw the separator away, or keep it attached to the next piece?' |
The separator is the only thing that decides where a cut is even possible. Below is the same short text and the same chunk_size = 20, with only the separator changed. Notice that the chunks are completely different shapes, and that one setting produces no split at all.
SHORT TEXT (54 characters) |
Same text, same chunk_size = 20, only the separator changed. Coloured boxes are the chunks that came out.
Picking a separator is really picking the smallest unit you refuse to break.
separator = "" -> 3 chunk(s)
Cuts between characters. Chunks hit exactly 20, but 'Sunday' gets sliced.
chunk 1 | 20 | Alice bakes bread on |
chunk 2 | 19 | Sunday. Bob cooks s |
chunk 3 | 14 | oup on Monday. |
separator = " " -> 3 chunk(s)
Cuts between words. Every chunk ends on a whole word, so chunks land under 20.
chunk 1 | 20 | Alice bakes bread on |
chunk 2 | 17 | Sunday. Bob cooks |
chunk 3 | 15 | soup on Monday. |
separator = "." -> 2 chunk(s)
Cuts at full stops only. Sentences stay whole even though they are longer than 20.
chunk 1 | 27 | Alice bakes bread on Sunday |
chunk 2 | 24 | Bob cooks soup on Monday |
separator = "\n" -> 1 chunk(s)
The short text has no newline, so no cut is possible: one chunk of 54.
chunk 1 | 54 | Alice bakes bread on Sunday. Bob cooks soup on Monday. |
Gotcha 1: separator = "\n" is the default, and many documents have no newlines |
chunk_size is the budget for step 2: pieces are packed together while the running total fits. With the paragraph text and separator "\n\n", the pieces are the three paragraphs (57, 61 and 60 characters).
PARAGRAPH TEXT (182 characters) |
Raising chunk_size does not re-cut the text, it only decides how many paragraphs get packed together.
chunk_size = 60 -> 3 chunk(s), sizes [57, 61, 60]
Nothing fits together, so each paragraph is its own chunk. Note chunk 2 is 61 characters: bigger than chunk_size.
chunk 1 | 57 | Alice loves baking bread. She bakes every Sunday morning. |
chunk 2 | 61 | Bob prefers cooking soup. He grows fresh herbs in his garden. |
chunk 3 | 60 | Together they opened a small cafe near the park last spring. |
chunk_size = 120 -> 2 chunk(s), sizes [120, 60]
Paragraphs 1 and 2 are packed into one chunk (57 + 2 + 61 = 120). Paragraph 3 starts a new chunk.
chunk 1 | 120 | Alice loves baking bread. She bakes every Sunday morning.¶¶Bob prefers cooking soup. He grows fresh herbs in his garden. |
chunk 2 | 60 | Together they opened a small cafe near the park last spring. |
chunk_size = 200 -> 1 chunk(s), sizes [182]
All three paragraphs now fit the budget, so the whole document becomes a single chunk.
chunk 1 | 182 | Alice loves baking bread. She bakes every Sunday morning.¶¶Bob prefers cooking soup. He grows fresh herbs in his garden.¶¶Together they opened a small cafe near the park last spring. |
Gotcha 2: a chunk can come out LARGER than chunk_size |
Overlap exists because a fact can sit exactly on a cut. If chunk 1 ends with "he was born in" and chunk 2 starts with "1987", then neither chunk can answer "when was he born?". Overlap repeats the tail of the previous chunk at the start of the next one, so the seam text exists in both.
Short text, separator = "", chunk_size = 20. The yellow hatched area is text that now exists in two chunks.
chunk_overlap = 0 -> 3 chunk(s), sizes [20, 19, 14]
Clean cuts, no duplication. 'Sunday' is split across chunk 1 and chunk 2 and is unfindable as a phrase.
chunk 1 | 20 | Alice bakes bread on |
chunk 2 | 19 | Sunday. Bob cooks s |
chunk 3 | 14 | oup on Monday. |
chunk_overlap = 5 -> 4 chunk(s), sizes [20, 20, 20, 9]
Each chunk restarts 5 characters earlier, so the seam text appears twice and the chunk count grows.
chunk 1 | 20 | Alice bakes bread on |
chunk 2 | 20 | ad on Sunday. Bob co |
chunk 3 | 20 | ob cooks soup on Mon |
chunk 4 | 9 | n Monday. |
chunk_overlap = 10 -> 5 chunk(s), sizes [20, 20, 19, 20, 14]
Half of every chunk is a copy of the previous one. Recall improves, storage and duplicate hits grow.
chunk 1 | 20 | Alice bakes bread on |
chunk 2 | 20 | s bread on Sunday. B |
chunk 3 | 19 | Sunday. Bob cooks s |
chunk 4 | 20 | ob cooks soup on Mon |
chunk 5 | 14 | oup on Monday. |
And now the part that confuses everybody:
Run on PARAGRAPH TEXT | Chunks | Sizes |
chunk_size 60, separator "\n\n", chunk_overlap = 0 | 3 | [57, 61, 60] |
chunk_size 60, separator "\n\n", chunk_overlap = 30 | 3 | [57, 61, 60] |
Identical output. Overlap changed nothing because no chunk was ever built from more than one piece.
Gotcha 3: overlap does nothing when nothing is being merged |
When the splitter cuts at a separator, the separator itself has to go somewhere: it is either thrown away (False) or glued to the front of the following piece (True). It matters when the separator carries meaning, such as a full stop, a bullet marker or a comma in a list.
TEXT FOR THIS EXAMPLE (54 characters) |
keep_separator = False, separator = "." -> 3 chunk(s)
The full stops are gone. Cleaner text, but the sentence boundary is no longer visible to the model.
chunk 1 | 17 | Alice bakes bread |
chunk 2 | 14 | Bob cooks soup |
chunk 3 | 18 | They opened a cafe |
keep_separator = True, separator = "." -> 4 chunk(s)
Each chunk after the first begins with the '.' that caused the cut, and a trailing '.' becomes its own chunk.
chunk 1 | 17 | Alice bakes bread |
chunk 2 | 16 | . Bob cooks soup |
chunk 3 | 20 | . They opened a cafe |
chunk 4 | 1 | . |
Practical advice on keep_separator |
Use it when | Avoid it when |
You know exactly one boundary type in your data and want it respected, for example log lines ("\n") or paragraph exports ("\n\n"). Also fine with separator "" when you deliberately want blind, perfectly even character windows. | Your documents are mixed prose where you cannot name one reliable boundary. You will either get one giant chunk (separator missing) or oversized chunks (separator too coarse). Use recursive_character instead. |
Analogy: cutting a cake for guests. First try to give everyone a whole slice. If a slice is still too big for the plate, halve it. If it still does not fit, cut smaller. You only ever cut finer when you have to.
recursive_character is fixed_size with a list of separators instead of one. It tries the biggest boundary first and drops to a finer one only for the pieces that are still too long. This is why it is the sensible default: it can never get stuck, because the last separator on the ladder is the empty string.
The ladder of separators. fixed_size gives you one rung; this gives you all of them.
Parameter | Default | Plain meaning | What changes when you change it |
chunk_size | 1000 | Maximum characters per chunk. | Smaller means more, tighter chunks and deeper rungs of the ladder get used. |
chunk_overlap | 200 | Characters repeated from the previous chunk at a seam. | Higher protects facts that sit on a boundary, at the cost of duplicated text. |
separators | ["\n\n", "\n", " ", ""] | The ladder, tried in order from coarse to fine. | Add your own structure markers first, e.g. ["\n## ", "\n\n", "\n", " ", ""] for markdown. |
keep_separator | True | Keep the separator attached to the piece after the cut. | False gives cleaner text; True preserves punctuation and markers. |
is_separator_regex | False | Treat the separators as regular expressions. | True lets you cut at patterns such as "\n\d+\. " for numbered lists. |
PARAGRAPH TEXT (182 characters) |
Lowering chunk_size makes the splitter descend the ladder: paragraphs, then words.
chunk_size = 60, chunk_overlap = 0 -> 5 chunk(s)
Paragraph 1 fits (57). Paragraphs 2 and 3 are too long, so the splitter drops to the space separator and cuts at word boundaries.
chunk 1 | 57 | Alice loves baking bread. She bakes every Sunday morning. |
chunk 2 | 53 | Bob prefers cooking soup. He grows fresh herbs in his |
chunk 3 | 7 | garden. |
chunk 4 | 52 | Together they opened a small cafe near the park last |
chunk 5 | 7 | spring. |
chunk_size = 60, chunk_overlap = 20 -> 5 chunk(s)
Same cuts, but each continuation chunk now starts 20 characters earlier, so no phrase is lost at a seam.
chunk 1 | 57 | Alice loves baking bread. She bakes every Sunday morning. |
chunk 2 | 53 | Bob prefers cooking soup. He grows fresh herbs in his |
chunk 3 | 26 | fresh herbs in his garden. |
chunk 4 | 52 | Together they opened a small cafe near the park last |
chunk 5 | 26 | near the park last spring. |
chunk_size = 25, chunk_overlap = 0 -> 9 chunk(s)
A much smaller budget forces word-level cuts everywhere: nine short chunks, none over 25 characters.
chunk 1 | 25 | Alice loves baking bread. |
chunk 2 | 22 | She bakes every Sunday |
chunk 3 | 8 | morning. |
chunk 4 | 19 | Bob prefers cooking |
chunk 5 | 20 | soup. He grows fresh |
chunk 6 | 20 | herbs in his garden. |
chunk 7 | 22 | Together they opened a |
chunk 8 | 24 | small cafe near the park |
chunk 9 | 12 | last spring. |
Compare with fixed_size |
Use it when | Avoid it when |
This is your default for ordinary documents: reports, web pages, markdown, mixed prose. Start at chunk_size 800 with chunk_overlap 100. | You need exact, evenly sized windows (use sliding_window) or hard token limits (use token_based). |
Analogy: a book of quotations. You never cut a quote in half; you decide how many quotes go on a page.
This strategy finds sentence endings and then groups a fixed number of sentences per chunk. Length in characters is not controlled at all, so chunk sizes vary with how wordy the sentences are.
Parameter | Default | Plain meaning | What changes when you change it |
sentences_per_chunk | 5 | How many sentences are grouped into one chunk. | 1 gives maximum precision but no context; 3-5 keeps a small argument together. |
separator | " " | The string used to join the grouped sentences back together. | Use "\n" if you want each sentence on its own line inside the chunk. |
PARAGRAPH TEXT (5 sentences) (182 characters) |
sentences_per_chunk is the only real knob: it trades precision for context.
sentences_per_chunk = 1 -> 5 chunk(s), sizes [25, 31, 25, 35, 60]
One sentence per chunk. A search for 'Sunday' returns exactly the sentence about Sunday and nothing else.
chunk 1 | 25 | Alice loves baking bread. |
chunk 2 | 31 | She bakes every Sunday morning. |
chunk 3 | 25 | Bob prefers cooking soup. |
chunk 4 | 35 | He grows fresh herbs in his garden. |
chunk 5 | 60 | Together they opened a small cafe near the park last spring. |
sentences_per_chunk = 2 -> 3 chunk(s), sizes [57, 61, 60]
Two sentences per chunk. Chunk sizes now vary between 57 and 61 characters, because sentences differ in length.
chunk 1 | 57 | Alice loves baking bread. She bakes every Sunday morning. |
chunk 2 | 61 | Bob prefers cooking soup. He grows fresh herbs in his garden. |
chunk 3 | 60 | Together they opened a small cafe near the park last spring. |
sentences_per_chunk = 3 -> 2 chunk(s), sizes [83, 96]
Three sentences per chunk. Groups now cross paragraph boundaries: Bob's soup sentence joins Alice's chunk.
chunk 1 | 83 | Alice loves baking bread. She bakes every Sunday morning. Bob prefers cooking soup. |
chunk 2 | 96 | He grows fresh herbs in his garden. Together they opened a small cafe near the park last spring. |
Watch out |
Use it when | Avoid it when |
Content is naturally sentence-shaped: FAQs, policies, transcripts, product descriptions, anything where one fact equals one sentence. | Sentences are extremely long or extremely short, or the text is tables, code or bullet fragments where punctuation is unreliable. |
Analogy: filing a report by section. Each section stays intact in its own folder.
The simplest strategy in the list: split on blank lines, then group N paragraphs together. It respects the author's own structure, which is often better than any size you could pick.
Parameter | Default | Plain meaning | What changes when you change it |
paragraphs_per_chunk | 1 | How many paragraphs are grouped into one chunk. | 2 or 3 keeps an argument that spans paragraphs together, at the cost of precision. |
separator | "\n\n" | The pattern that marks a paragraph boundary. | Set to "\n" for text where each line is already a paragraph. |
PARAGRAPH TEXT (3 paragraphs) (182 characters) |
Grouping paragraphs: chunk length is decided by the author, not by you.
paragraphs_per_chunk = 1 -> 3 chunk(s), sizes [57, 61, 60]
One paragraph per chunk. Notice the chunk boundaries match the blank lines exactly.
chunk 1 | 57 | Alice loves baking bread. She bakes every Sunday morning. |
chunk 2 | 61 | Bob prefers cooking soup. He grows fresh herbs in his garden. |
chunk 3 | 60 | Together they opened a small cafe near the park last spring. |
paragraphs_per_chunk = 2 -> 2 chunk(s), sizes [120, 60]
Two paragraphs per chunk, joined with the blank line preserved inside the chunk (120 characters).
chunk 1 | 120 | Alice loves baking bread. She bakes every Sunday morning.¶¶Bob prefers cooking soup. He grows fresh herbs in his garden. |
chunk 2 | 60 | Together they opened a small cafe near the park last spring. |
THE SAME CONTENT WITH THE BLANK LINES REMOVED (119 characters) |
paragraphs_per_chunk = 1 -> 1 chunk(s), sizes [119]
No blank lines means no boundaries, so the whole text stays as one chunk of any length.
chunk 1 | 119 | Alice loves baking bread. She bakes every Sunday morning. Bob prefers cooking soup. He grows fresh herbs in his garden. |
The danger of this strategy |
Analogy: airline baggage. You care about kilograms, not the size of the suitcase. Models charge and limit by tokens, so measure in tokens.
A token is roughly three quarters of a word: common words are one token, rare words split into several. Every embedding model and every LLM has a hard limit expressed in tokens, so this is the only strategy that can guarantee a chunk fits.
Characters, words and tokens are three different rulers over the same text.
Parameter | Default | Plain meaning | What changes when you change it |
chunk_size | 512 | Maximum tokens per chunk (not characters). | Set it to your model's limit minus room for the prompt and the answer. |
chunk_overlap | 50 | Tokens repeated from the previous chunk. | 10-15 percent of chunk_size is a good starting point. |
encoding_name | "cl100k_base" | Which tokenizer counts the tokens. | Match your model family so the count matches what the model actually sees. |
model_name | None | Derive the tokenizer from a model name instead. | Set this and the encoding is chosen for you. |
PARAGRAPH TEXT (182 characters) |
The same text at three token budgets. Character lengths come out uneven because tokens, not characters, are being counted.
chunk_size = 10 tokens, chunk_overlap = 0 -> 4 chunk(s), character sizes [48, 51, 51, 32]
A tiny 10-token budget. Character lengths land between 32 and 51: proof that tokens are being counted, not characters.
chunk 1 | 48 | Alice loves baking bread. She bakes every Sunday |
chunk 2 | 51 | morning.¶¶Bob prefers cooking soup. He grows fresh |
chunk 3 | 51 | herbs in his garden.¶¶Together they opened a small |
chunk 4 | 32 | cafe near the park last spring. |
chunk_size = 10 tokens, chunk_overlap = 5 -> 7 chunk(s), character sizes [48, 53, 51, 44, 51, 52, 32]
Half the tokens are repeated, so chunk count rises from 4 to 7 and consecutive chunks visibly share text.
chunk 1 | 48 | Alice loves baking bread. She bakes every Sunday |
chunk 2 | 53 | She bakes every Sunday morning.¶¶Bob prefers cooking |
chunk 3 | 51 | morning.¶¶Bob prefers cooking soup. He grows fresh |
chunk 4 | 44 | soup. He grows fresh herbs in his garden.¶¶ |
chunk 5 | 51 | herbs in his garden.¶¶Together they opened a small |
chunk 6 | 52 | Together they opened a small cafe near the park last |
chunk 7 | 32 | cafe near the park last spring. |
chunk_size = 25 tokens, chunk_overlap = 0 -> 2 chunk(s), character sizes [122, 60]
A 25-token budget fits about two paragraphs, so the text becomes two chunks.
chunk 1 | 122 | Alice loves baking bread. She bakes every Sunday morning.¶¶Bob prefers cooking soup. He grows fresh herbs in his garden.¶¶ |
chunk 2 | 60 | Together they opened a small cafe near the park last spring. |
Rough conversion for English |
Analogy: a magnifying glass moved across a page in small steps. Each position sees a full view, and neighbouring positions share most of what they see.
This is fixed_size with the guarantee that overlap is always on and evenly spaced. Instead of asking how much to repeat, you say how far the window moves each time.
The window is chunk_size wide and moves stride characters at a time, so overlap = chunk_size - stride.
Parameter | Default | Plain meaning | What changes when you change it |
chunk_size | 1000 | Width of the window in characters. | Wider windows carry more context per hit. |
stride | None | How far the window jumps. Overlap becomes chunk_size - stride. | Smaller stride means more windows, more duplication, better recall. |
chunk_overlap | None | The same idea stated as overlap instead of step size. | Give either stride or chunk_overlap, never both: the code raises an error. |
separator | "" | What the window is allowed to cut at. Empty means raw characters. | Use " " to keep words whole at the window edges. |
If you set neither stride nor chunk_overlap |
SHORT TEXT (54 characters) |
Same 20-character window at three strides. Yellow hatching is duplicated text.
chunk_size = 20, stride = 20 (overlap = 0) -> 3 chunk(s)
stride equals chunk_size, so overlap is zero and this behaves exactly like fixed_size with separator "".
chunk 1 | 20 | Alice bakes bread on |
chunk 2 | 19 | Sunday. Bob cooks s |
chunk 3 | 14 | oup on Monday. |
chunk_size = 20, stride = 10 (overlap = 10) -> 5 chunk(s)
The window moves half its width: 50 percent overlap, chunk count nearly doubles, every seam is covered twice.
chunk 1 | 20 | Alice bakes bread on |
chunk 2 | 20 | s bread on Sunday. B |
chunk 3 | 19 | Sunday. Bob cooks s |
chunk 4 | 20 | ob cooks soup on Mon |
chunk 5 | 14 | oup on Monday. |
chunk_size = 20, stride = 5 (overlap = 15) -> 8 chunk(s)
The window creeps forward 5 characters at a time: 75 percent overlap and a very large number of near-duplicate chunks.
chunk 1 | 20 | Alice bakes bread on |
chunk 2 | 19 | bakes bread on Sund |
chunk 3 | 20 | s bread on Sunday. B |
chunk 4 | 20 | ad on Sunday. Bob co |
chunk 5 | 19 | Sunday. Bob cooks s |
chunk 6 | 20 | ay. Bob cooks soup o |
chunk 7 | 20 | ob cooks soup on Mon |
chunk 8 | 19 | oks soup on Monday. |
Use it when | Avoid it when |
Recall matters more than storage: legal clauses, medical notes, contracts, or any case where a missed fact is expensive and answers keep landing on chunk seams. | You are cost or latency sensitive. Every extra window is another embedding to compute, store and search, and near-duplicate chunks crowd out variety in the top-k results. |
Analogy: a library index card. The card is short and precise so you can find it fast, but it points you to the whole chapter so you can actually read the answer.
The text is cut twice: into large parent chunks, then each parent into small children. The children are what you search, because small chunks match questions precisely. The parent is what you send to the LLM, because it has the surrounding context.
Search the children, answer from the parent. Each child keeps a reference to its parent.
Parameter | Default | Plain meaning | What changes when you change it |
parent_chunk_size | 2000 | Size of the big context chunk. | Bigger parents give the LLM more background and cost more prompt tokens. |
child_chunk_size | 400 | Size of the small searchable chunk. | Smaller children match questions more sharply but carry less meaning on their own. |
parent_chunk_overlap | 0 | Overlap between parents. | Usually left at 0; parents are already large. |
child_chunk_overlap | 50 | Overlap between children inside a parent. | Protects facts sitting on a child boundary. |
include_parents | False | Whether parent chunks are also returned and stored. | True lets you inspect or embed parents too, at the cost of extra rows. |
PARAGRAPH TEXT (182 characters) |
parent_chunk_size = 120, child_chunk_size = 40, include_parents = False -> 6 chunk(s)
Only the children come back. Each one is small enough to match a question precisely.
chunk 1 | 35 | Alice loves baking bread. She bakes |
chunk 2 | 21 | every Sunday morning. |
chunk 3 | 34 | Bob prefers cooking soup. He grows |
chunk 4 | 26 | fresh herbs in his garden. |
chunk 5 | 38 | Together they opened a small cafe near |
chunk 6 | 21 | the park last spring. |
the same run with include_parents = True -> 8 chunk(s)
Now each parent appears before its own children, so you can see the grouping: a 120-character parent followed by its 40-character children.
chunk 1 | 120 | Alice loves baking bread. She bakes every Sunday morning.¶¶Bob prefers cooking soup. He grows fresh herbs in his garden. |
chunk 2 | 35 | Alice loves baking bread. She bakes |
chunk 3 | 21 | every Sunday morning. |
chunk 4 | 34 | Bob prefers cooking soup. He grows |
chunk 5 | 26 | fresh herbs in his garden. |
chunk 6 | 60 | Together they opened a small cafe near the park last spring. |
chunk 7 | 38 | Together they opened a small cafe near |
chunk 8 | 21 | the park last spring. |
Changing the parent-to-child ratio changes how many children hang off each parent.
Use it when | Avoid it when |
Precision and context are both required: technical manuals, standards, long policy documents. A common production setting is parent 1500-2000 with child 300-400. | You want the simplest possible pipeline. It needs retrieval code that knows how to fetch a parent from a child hit, otherwise the extra structure buys you nothing. |
Analogy: a good editor reading the text. They do not count words; they notice where the subject changes and put the break there.
Every other strategy decides where to cut by counting. This one embeds each sentence, compares each sentence with the previous one, and cuts where the meaning jumps. That means it needs an embedding model at chunking time, so it is the slowest and most expensive option.
Cuts happen where similarity to the previous sentence collapses, which is where the topic changed.
Parameter | Typical default | Plain meaning | What changes when you change it |
embeddings | required | The embedding model used to compare sentences. | Must be the same family you use for retrieval, or the breakpoints will not reflect your search behaviour. |
breakpoint_threshold_type | "percentile" | How a 'big enough' meaning jump is judged. | percentile adapts to the document; standard_deviation and interquartile react differently to outliers. |
breakpoint_threshold_amount | 95 | How sensitive the cut is. | Lower means more cuts and smaller chunks; higher means fewer, longer chunks. |
Why it is not the default in this app |
Same 182-character paragraph text, comparable settings, seven strategies. This is the single most useful picture in the guide: it shows that the choice of strategy changes the number of chunks by a factor of three and the spread of chunk lengths even more.
PARAGRAPH TEXT (182 characters) |
Left: how many chunks each strategy produced. Right: every individual chunk length, so you can see how even or uneven the output is.
Strategy and settings | Chunks | Smallest | Largest | What the shape tells you |
fixed_size (size 60, sep "\n\n") | 3 | 57 | 61 | Respects paragraphs, but one chunk exceeded the 60 limit. |
recursive_character (size 60, overlap 0) | 5 | 7 | 57 | Never exceeded the limit; fell back to word cuts, so lengths vary. |
sentence (2 per chunk) | 3 | 57 | 61 | Length follows the author's sentences, not your budget. |
paragraph (1 per chunk) | 3 | 57 | 61 | Length follows the author's paragraphs entirely. |
token_based (10 tokens) | 4 | 32 | 51 | Counted in tokens, so character lengths look uneven. |
sliding_window (size 60, stride 30) | 6 | 31 | 60 | Perfectly even windows, plus deliberate duplication. |
hierarchical (parent 120 / child 40) | 6 | 21 | 38 | Smallest chunks of all: built for precise matching. |
fixed_size (size 60, sep "\n\n") -> 3 chunk(s)
chunk 1 | 57 | Alice loves baking bread. She bakes every Sunday morning. |
chunk 2 | 61 | Bob prefers cooking soup. He grows fresh herbs in his garden. |
chunk 3 | 60 | Together they opened a small cafe near the park last spring. |
recursive_character (size 60, overlap 0) -> 5 chunk(s)
chunk 1 | 57 | Alice loves baking bread. She bakes every Sunday morning. |
chunk 2 | 53 | Bob prefers cooking soup. He grows fresh herbs in his |
chunk 3 | 7 | garden. |
chunk 4 | 52 | Together they opened a small cafe near the park last |
chunk 5 | 7 | spring. |
sentence (2 per chunk) -> 3 chunk(s)
chunk 1 | 57 | Alice loves baking bread. She bakes every Sunday morning. |
chunk 2 | 61 | Bob prefers cooking soup. He grows fresh herbs in his garden. |
chunk 3 | 60 | Together they opened a small cafe near the park last spring. |
paragraph (1 per chunk) -> 3 chunk(s)
chunk 1 | 57 | Alice loves baking bread. She bakes every Sunday morning. |
chunk 2 | 61 | Bob prefers cooking soup. He grows fresh herbs in his garden. |
chunk 3 | 60 | Together they opened a small cafe near the park last spring. |
token_based (10 tokens) -> 4 chunk(s)
chunk 1 | 48 | Alice loves baking bread. She bakes every Sunday |
chunk 2 | 51 | morning.¶¶Bob prefers cooking soup. He grows fresh |
chunk 3 | 51 | herbs in his garden.¶¶Together they opened a small |
chunk 4 | 32 | cafe near the park last spring. |
sliding_window (size 60, stride 30) -> 6 chunk(s)
chunk 1 | 60 | Alice loves baking bread. She bakes every Sunday morning.¶¶B |
chunk 2 | 60 | bakes every Sunday morning.¶¶Bob prefers cooking soup. He gr |
chunk 3 | 60 | ob prefers cooking soup. He grows fresh herbs in his garden. |
chunk 4 | 60 | ows fresh herbs in his garden.¶¶Together they opened a small |
chunk 5 | 58 | Together they opened a small cafe near the park last sprin |
chunk 6 | 31 | cafe near the park last spring. |
hierarchical (parent 120 / child 40) -> 6 chunk(s)
chunk 1 | 35 | Alice loves baking bread. She bakes |
chunk 2 | 21 | every Sunday morning. |
chunk 3 | 34 | Bob prefers cooking soup. He grows |
chunk 4 | 26 | fresh herbs in his garden. |
chunk 5 | 38 | Together they opened a small cafe near |
chunk 6 | 21 | the park last spring. |
Start at the top, match the description of your data, use the suggested starting point.
Your data | Strategy | Settings to start with |
Reports, web pages, mixed prose | recursive_character | chunk_size 800, chunk_overlap 100 |
FAQ, policy Q&A, transcripts | sentence | sentences_per_chunk 3 |
Well-formatted articles | paragraph | paragraphs_per_chunk 1 |
Hard model or cost limits | token_based | chunk_size 512, chunk_overlap 50 |
Contracts, clinical notes | sliding_window | chunk_size 800, stride 400 |
Manuals, standards, handbooks | hierarchical | parent 1500, child 300, child_overlap 50 |
Log lines, CSV-ish exports | fixed_size | chunk_size 1000, separator "\n" |
Topic-jumping transcripts | semantic | percentile 95 |
1. Get the separator right first. If a strategy returns one enormous chunk, the separator is missing from your text.
2. Then set chunk_size. Look at your answers: if they read as half-sentences, it is too small; if the LLM rambles about unrelated things, it is too big.
3. Then set overlap, starting at 10-15 percent of chunk_size. Only raise it if facts keep getting cut at seams.
4. Only then consider a fancier strategy. Most retrieval problems are a size or separator problem, not a strategy problem.
Term | Meaning in one line |
Chunk | One piece of text that becomes one vector and one possible search result. |
Character | A single letter, digit, space or punctuation mark. |
Token | A model's unit of text, roughly three quarters of a word. Model limits are in tokens. |
Separator | A text pattern the splitter is allowed to cut at. |
Overlap | Text from the end of one chunk repeated at the start of the next. |
Stride | How far a sliding window moves each step. overlap = chunk_size - stride. |
Parent / child chunk | A big context chunk and the small searchable chunks cut from it. |
Breakpoint | In semantic chunking, a place where the meaning changes enough to justify a cut. |
Recall | The share of relevant chunks your search actually returns. Overlap mainly buys recall. |
The one thing to remember |