This week, OpenAI launched support for built-in RAG, as part of their larger strategy to enable non-developers to create custom GPTs for specific use cases.

Let's take a quick look at the functionality:

Image credit: DALL-E 3 by OpenAI

Prompt: "lets make a cool image for RAG fusion, with some nuclear theme"

Last week we published our first significant RAG pipeline refactor since releasing the bot for limited testing.

Even if we are just getting started, the results so far look promising. These techniques are being explored by many teams around the world and learning from the best can save us time and effort.

Here is one to take a look at:

https://towardsdatascience.com/forget-rag-the-future-is-rag-fusion-1147298d8ad1

Image credit: DALL-E 3 by OpenAI

Prompt: "high-tech graphic showing multi-stage text processing"

With all the focus on chat bots, it can be easy to forget how relevant large language models are for general purpose NLP tasks. Natural Language Processing is a broad field with an extensive history of specialized tool development for specific tasks. By connecting different models together with various pre- and post-processors, it is possible to hardwire an effective, multi-stage pipeline. But if something were to change in your input data, your entire downstream processing might be affected.

With the advent of LLMs, we got a new way of constructing processing pipelines. Rather than setup discrete processing stages, we could simply pack a whole series of instructions into a single prompt. Social media quickly started filling up with examples of amazing super-prompts. So what's the catch?

Image credit: DALL-E 3 by OpenAI - prompt: "Digital render of a high-tech computer server room bathed in blue and green ambient lighting. Positioned prominently above the servers, digital data streams converge clearly and boldly to spell out the letters 'R', 'A', and 'G' in succession"

You know the feeling.

The seller said it was amazing, things would work perfectly out of the box.

But now that you have packed it out and powered it up, your new gadget just doesn't live up to the hype.

Image credit: DALL-E 3 by OpenAI - prompt: "Cool header image for an article about Retrieval Augmented Generation (RAG)"

About the time that ChatGPT hit the scene in December 2022, a new technique was discovered that could turn any kind of documentation into a conversational agent, capable of answering an endless stream of questions without ever tiring.

Probably due to it's simplicity and universal appeal, this use case quickly became a popular entry point for experimenting with large language models and private data. So, how does it work?