Databricks RAG Application Design Quiz

1. What is the purpose of a reranking step in advanced RAG architectures?

To sort documents alphabetically before retrieval

To re-order retrieved documents by relevance using a more sophisticated model

To delete duplicate embeddings from the vector store

To encrypt retrieved documents for security

2. What is a critical consideration when deploying RAG applications in production on Databricks?

Ensuring low latency for retrieval and generation while maintaining accuracy

Using only free open-source models

Avoiding any use of external APIs

Storing all vectors in plain text files

3. Which component is responsible for converting text into numerical vectors in a RAG pipeline?

Retriever

Language model

Embedding model

Query parser

4. What is the significance of similarity metrics (e.g., cosine similarity) in RAG systems?

They measure how similar two embeddings are to determine document relevance

They encrypt vector data for privacy

They reduce the dimensionality of embeddings

They count the number of documents in the knowledge base

5. In Databricks, what is a typical use case for Retrieval-Augmented Generation?

Training neural networks from scratch

Building question-answering systems over proprietary documents or databases

Compressing image files

Managing user authentication

6. How can you reduce hallucinations in RAG-based LLM responses?

Increase the embedding dimension size

Provide high-quality retrieved context and use grounding techniques

Disable the retrieval component

Only use closed-source models

7. What role does a language model play in the generation phase of RAG?

It retrieves documents from the vector database

It generates embeddings for new queries

It synthesizes a coherent response using retrieved context and the user query

It indexes documents into the knowledge base

8. Which of the following is a best practice for managing document freshness in RAG systems?

Never update the vector database after initial indexing

Implement incremental updates and versioning for the knowledge base

Rebuild the entire vector store daily regardless of changes

Remove all old documents before adding new ones

9. In Databricks, what does 'prompt injection' vulnerability mean in RAG systems?

Injecting new documents into the vector database

Malicious user input that manipulates the LLM's behavior through retrieved context

Adding embeddings to speed up retrieval

Compressing prompts to reduce token usage

10. How does semantic search in RAG differ from keyword-based search?

Semantic search uses exact word matching; keyword search uses embeddings

Semantic search uses embeddings to find meaning-similar documents; keyword search matches exact terms

Semantic search is slower but always more accurate

Keyword search is only used for structured data

11. What is the primary purpose of a vector database in a RAG system?

Store and retrieve semantically similar documents using embeddings

Execute SQL queries on structured data

Cache LLM responses for faster inference

Compress large language models for mobile deployment

12. Which metric is most important for evaluating retrieval quality in RAG systems?

Mean response latency

Precision and recall of retrieved documents

Model parameter count

Storage capacity utilization

13. When designing a RAG system, what does 'context window' refer to?

The time limit for document retrieval

The maximum number of tokens an LLM can process in a single request

The size of the vector embedding dimensions

The number of concurrent user queries

14. What is a key advantage of using Databricks Mosaic AI for RAG development?

It eliminates the need for vector databases

It provides pre-built RAG templates and managed endpoints for LLMs

It only supports open-source models

It requires manual vector database configuration

15. How does chunking improve RAG system performance?

It reduces the total number of documents in the knowledge base

It breaks large documents into smaller, semantically coherent pieces for better retrieval

It encrypts sensitive data in the vector database

It eliminates the need for embedding models

16. Which Databricks service is best suited for storing and managing vector embeddings at scale?

Databricks SQL Warehouse only

Delta Lake with vector search capabilities

Apache Spark RDD

External S3 buckets without indexing

17. What is the typical workflow sequence in a RAG pipeline?

LLM generation → retrieval → embedding

User query → embedding → retrieval → LLM augmentation → response

Vector storage → query parsing → response generation

Document indexing → model training → inference only

18. In a RAG system, what does the retrieval component do before passing context to the LLM?

Fine-tunes the language model on domain data

Searches for and returns relevant documents from a knowledge base

Translates user queries into multiple languages

Compresses retrieved documents to reduce token count

19. Which embedding model is commonly used in Databricks RAG pipelines?

Text2Vec

BGE (BAAI General Embedding)

PCA-Transformer

FastText-Only

Databricks GenAI Engineer RAG Application Design Quiz