Evaluating Chunking Strategies for Retrieval-Augmented Generation on Academic Texts
This paper evaluates the effectiveness of cluster-based semantic chunking compared to fixed-size and recursive chunking in Retrieval-Augmented Generation systems using the Retrieval Augmented Generation Assessment framework on long, structured academic theses.
Evaluated the impact of different chunking methods on Retrieval-Augmented Generation systems using a large-scale assessment framework.
Before reading this…
Applications
- →Natural Language Processing, Information Retrieval, Question Answering
To understand this paper, make sure you know these concepts first:
- Understanding of Retrieval-Augmented Generation systems, Large Language Models, Semantic Chunking, Evaluation Frameworksfind papers →
Abstract
More Like ThisRetrieval-Augmented Generation (RAG) systems use the question-answering capabilities of Large Language Models (LLMs) to access information outside their parameters. We evaluate if cluster-based semantic chunking improves retrieval and answer quality compared to fixed-size and recursive chunking evaluating on long, structured academic theses using the Retrieval Augmented Generation Assessment (RAGAs) framework. RAGAs based faithfulness shows limited reliability in this setup. Performance on fixed versus document specific questions varied substantially, likely related to the formatting of documents and preprocessing. Under the tested configuration, cluster-based chunking did not outperform simpler strategies.