Chunking, splitting documents into smaller pieces before generating embeddings for retrieval, usually comes up purely as a retrieval quality decision: smaller chunks give more precise matches, larger chunks keep more context intact. It has security implications too, and those get a lot less attention, because the boundaries you choose decide what information, including access metadata and protective context, travels with each retrievable piece and what gets left behind entirely.
Metadata inheritance across chunk boundaries
When a document splits into chunks, every chunk needs to inherit the access control and classification metadata of its source document. If the chunking process doesn't reliably propagate that metadata to every resulting chunk, some chunks end up with missing or default metadata, and depending on how the retrieval filter handles that gap, those chunks are either wrongly excluded from legitimate retrieval or, more dangerously, wrongly included in retrieval they should never have been part of. Testing that metadata propagates correctly across every chunk a document produces, not just the first or most obviously structured one, catches this before it becomes a real problem.
Context that provides protective framing can get stripped
Some documents carry framing that changes how a specific passage should be read: a caveat, a condition, an exception noted somewhere else in the document. Split a passage away from the framing that qualifies it, and a chunk retrieved in isolation can convey a different, sometimes more sensitive or more misleading, meaning than the same content read inside the full document. A policy document's general rule, chunked away from its exception clause, can lead a model to state that rule as absolute when the source document never intended anything of the sort.
Smaller chunks increase the number of retrievable units
Finer-grained chunking increases the total number of independently retrievable pieces pulled from the same underlying documents, which increases the surface area for both legitimate retrieval and any access control or injection-related failure. A fine-grained chunk carrying a sensitive detail, extracted out of otherwise appropriately-scoped surrounding context, can get retrieved entirely on its own, decoupled from the context that might have made its sensitivity obvious to a human reviewer looking at that chunk in isolation.
Chunk boundaries can be exploited for injection placement
An attacker poisoning a document for retrieval benefits from understanding how the target system chunks content, since placing an injection payload precisely inside a chunk boundary likely to be retrieved on its own, rather than buried inside a larger chunk alongside less relevant content, raises the odds the payload reaches a model's context cleanly. It's a subtle point, but it means chunking strategy isn't some purely retrieval-quality decision insulated from adversarial thinking.
Overlap between chunks has tradeoffs too
Overlapping chunk boundaries, where adjacent chunks share some content, can preserve context that would otherwise vanish at a hard boundary. It also means a single piece of sensitive or injected content can show up in multiple retrievable chunks at once, multiplying its odds of getting retrieved for a related query instead of only ever appearing in one place. None of this is a reason to avoid overlap, its retrieval-quality benefits are real, but it is a reason to apply access control and content review at least as strictly to overlapping regions as to unique content.
Testing retrieval behavior at the chunk level, not just the document level
Security testing for RAG systems needs to specifically examine chunk-level behavior, not just document-level access control: confirm individual chunks carry the metadata access filtering actually needs, and confirm that sensitive content stripped of its original protective framing by chunking doesn't produce a misleading or overly permissive response once it's retrieved in isolation.
Where TELEON fits
TELEON's audit trail can log which specific retrieved content, at whatever granularity the pipeline chunks at, contributed to a given interaction, applied through the gateway or supported middleware, supporting investigation into whether a specific chunk's isolated retrieval contributed to an unusual or misleading response.
Chunking strategy itself, metadata propagation across chunk boundaries, and decisions about overlap are pipeline-level design choices that belong to the team building the retrieval architecture, balanced against retrieval quality goals alongside these security considerations.
A short checklist
- Confirm access and classification metadata propagates correctly to
every chunk a document produces.
- Review whether chunking strips protective framing, caveats, or
exceptions, from the content it modifies.
- Weigh finer-grained chunking's retrieval benefits against its increased
surface area for isolated, decontextualized retrieval.
- Apply content review and access control at least as strictly to
overlapping chunk regions as to unique content.
- Test retrieval behavior at the chunk level specifically, not only at
the document level.
Chunking decisions shape more than retrieval precision. They determine what access metadata and protective context travel with each retrievable piece of content, and treating chunking purely as a relevance-tuning parameter misses the security tradeoffs built into that same decision.