I asked a 14 billion parameter LLM to remember a short story by Nathaniel Hawthorne and it told me it was written by Edith Wharton. This was a good thing, because I wanted to see first hand how Retrieval Augmented Generation (RAG) really works. I am writing this post to share the process and my results, in case there are others who are new to RAG and would like to read a case study on it. RAG is a method to improve the accuracy of LLM output. It creates and uses a separate database that the LLM refers to for additional information that it either lacks or has insufficient knowledge in its pretraining memory. The relevant context is pulled from the database and silently added to the user's prompt. With this additional information, the LLM has more to work with and, hopefully produce a better result. There are several design decisions in developing this process, which we will explore in a hands-on experiment using entirely local tools. This study was intentionally old school in the set-up.…