Understanding RAG in Knowledge Stacks

What is RAG?

RAG (Retrieval Augmented Generation) is the technology that powers Knowledge Stacks in Msty. It's important to understand that RAG doesn't "train" or "teach" the AI new information - instead, it's more like giving the AI a temporary reference book to consult while answering your questions.

How RAG Works

Think of it like this:

1. You: Ask a question about your documents
2. Msty:
   • Searches your documents for relevant information
   • Uses embeddings to find matches
   • Selects the best chunks (default: 15)
3. AI:
   • Receives your question and selected chunks
   • Uses this default prompt:
     The following text has been extracted from a data source due to its probable relevance to the question. 
     Please use the given information if it is relevant to come up with an answer and don't use anything else. 
     The answer should be as concise and succinct as possible to answer the question.
     

   • Generates a focused answer

Search settings showing RAG controls

Fine-tuning RAG

Control how RAG works in the chat interface:

1. Similarity Threshold:
   • Low: Broader context, more results
   • Medium: Balanced matching
   • High: Strict matching
   • Highest: Only exact matches
2. Number of Chunks:
   • Default: 15 chunks
   • More chunks = broader context
   • Fewer chunks = focused answers
3. Custom Prompt:
   • Modify the default prompt
   • Guide AI response style
   • Maintain answer focus

Why Use RAG?

1. Accuracy:
   • References specific facts
   • Reduces "hallucinations"
   • Provides sourced answers
2. Privacy:
   • Documents stay local
   • Only relevant snippets sent
   • Full control over data
3. Cost Effective:
   • Sends minimal context
   • No training needed
   • Works with any AI model