Introduction

These two approaches are often pitted against each other as to which one is the best.

RAG is known for improving accuracy via in-context learning and is very affective where context is important.

RAG is easier to implement and often serves as a first foray into implementing LLMs due to RAG’s inspectability, observability and not being as opaque as fine-tuning.

With RAG the input token size increases with the prompt size, and the study found that the output token size tend to be more verbose and harder to steer.

The study found that fine-tuning offers a precise, succinct output that is attuned to brevity. It is highly effective and presents opportunities to learn new skills in a specific domain.

But the initial cost is high in terms of data preparation and running the fine-tuning process.

But when it comes to usage, fine-tuning necessitates minimal input token size, making it a more efficient option for handling large data sets.

It is evident that additional cost will have to be incurred somewhere; with RAG it is during model inference and not upfront. With fine-tuning the cost upfront is higher, with lower inference cost.

Using RAG & Fine-Tuning In Conjunction

This research propose a pipeline for using both fine-tuning and RAG.

This study made use of Q&A generation, which creates question and answer pairs based on the information available.

RAG uses it as a knowledge source.

The generated data is then refined and used to  fine-tune several models  while their quality is evaluated using a combination of proposed metrics.

In the image below, the methodology pipeline is seen…

1. Domain-specific datasets are collected, and the
2. Content and structure of the documents are extracted.
3. This information is then used for the Q&A generation step.
4. Synthesised question-answer pairs are used to fine-tune the LLMs.

Models are evaluated with and without RAG under different GPT-4-based metrics.