Introducing Self-RAG, a new easy-to-train, customizable, and powerful framework for making an LM learn to retrieve, generate, and critique its own outputs and retrieved passages, by using model-predicted reflection tokens. 📜: 🌐:

Augmenting the input space of LMs with retrieved passages (often called RAG) is powerful. Yet, it may still result in hallucinations as LMs can still ignore passages and irrelevant passages can distract LMs. Unnecessary retrieval can also hurt the quality or efficiency. [2/N]

Self-RAG enables an LM to learn to retrieve on demand and criticize its own generations & passages by decoding special tokens (reflection tokens) while standard RAG often retrieves fixed times e.g., once at the beginning, and never second revisits the passages/outputs. [3/N]

Self-RAG trains an LM on augmented diverse instruction-output data with intermitting passages and reflection tokens. As it is a simple LM training with expanded vocabularies, it’s much easier & memory-efficient than RLHF / end-to-end RAG training. [4/N]

During decoding, Self-RAG uses reflection tokens probabilities as fine-grained critique scores and chooses the best output, enabling the LM to tailor its behavior to diverse task requirements (e.g., creativity/fluency v.s. citation precision/factuality). [5/N]

So how good they are? Self-RAG outperforms many powerful LLMs (incl. ChatGPT) w/ and w/o standard RAG on six tasks and improves factuality and citation accuracy in long-form generations. Also, Self-RAG can tailor its behaviors for different preferences at inference time. [6/N]

Our model checkpoints and codes are available, and you can download and try out self-rag (7B, 13b) easily! [7/N] We also provide an inference script run Self-RAG w/ Contriever on 1 GPU

Work done with my fantastic collaborators @zeqiuwu1 @yizhongwyz @aviaviavi__ @HannaHajishirzi at @uwnlp!

this is really cool! Currently working on building a massive RAG database + pipeline - I would love to hook this up to it for some tests.

Nice! How does it compares to reranking models? I use them for this specific case, it’s not perfect but it’s an easy and fast way to improve context—without burning tokens on a multi LLM step solution

Reranking context is another way to measure the relevance of documents to improve RAG quality. Yet, feeding the highest ranked passages may still result in unsupported generation ( which Self-RAG’s fine-grained critique guided decoding can address.