Databricks GenAI Engineer LLM Evaluation Quiz

1. Which technique helps mitigate hallucinations in RAG systems?

Increasing batch size during inference

Ensuring high-quality and relevant document retrieval

Using only transformer models

Removing all system prompts

2. Which approach is most effective for evaluating LLM safety and bias?

Automated metrics alone

Human evaluation combined with adversarial testing and bias benchmarks

Ignoring edge cases

Testing only on clean, curated data

3. What is the primary benefit of using Databricks Feature Store in LLM pipelines?

It replaces the need for embeddings

It provides consistent, reusable features for training and serving

It automatically trains models

It eliminates data preprocessing

4. In LLM fine-tuning within Databricks, what is a critical consideration for evaluation?

Always use the same test set as training

Evaluate on domain-specific data separate from training to assess generalization

Avoid validation to save compute

Use only automated metrics without human review

5. Which evaluation framework is commonly used to assess instruction-following capabilities?

GLUE benchmark

MMLU or custom instruction-following benchmarks

ImageNet

SQuAD

6. What does Chain-of-Thought prompting improve in LLM reasoning?

Inference speed

Model transparency and accuracy on complex tasks

Token efficiency

Model size

7. In Databricks, what is the role of UC (Unity Catalog) in LLM workflows?

To train models faster

To manage data governance and access control across workspaces

To automatically evaluate models

To compress model parameters

8. Which metric measures semantic similarity between two text sequences?

Cosine similarity or embedding-based distance

Levenshtein distance

Hamming distance

Edit distance

9. What is the primary purpose of A/B testing in LLM deployment?

To reduce model file size

To compare model performance and user satisfaction in production

To eliminate the need for validation

To automatically select hyperparameters

10. In LLM evaluation, what is the main drawback of relying solely on human evaluation?

It is not objective

It is expensive, time-consuming, and difficult to scale

It cannot assess grammatical correctness

It requires deep learning expertise

11. Which metric measures the overlap between generated and reference text using unordered word matching?

BLEU

METEOR

ROUGE

CIDEr

12. What does perplexity measure in language models?

Semantic correctness of outputs

How well a model predicts a test sequence

User satisfaction with generated text

Token-level accuracy

13. In Databricks Model Registry, what is the primary purpose of stage transitions?

To track model lineage and version history

To manage model lifecycle from development through production

To compress model artifacts

To automatically retrain models

14. Which approach combines multiple evaluation metrics to reduce bias in LLM assessment?

Ensemble evaluation

Weighted metric aggregation

Single metric validation

Cross-fold evaluation

15. What is a key limitation of automatic metrics like ROUGE and BLEU?

They require human annotation

They cannot detect semantic equivalence or paraphrasing

They only work for English text

They require real-time computation

16. In prompt engineering, what is the primary benefit of few-shot prompting over zero-shot?

Reduces inference latency

Provides examples to guide model behavior and improve output quality

Eliminates the need for validation data

Decreases model hallucinations automatically

17. Which evaluation metric is most appropriate for assessing factuality in generated text?

Perplexity

Token accuracy

Semantic textual similarity or fact verification

Word error rate

18. What does BLEU score primarily evaluate in machine translation?

Semantic similarity

N-gram precision against reference translations

Human preference alignment

Fluency and readability

19. In Databricks, which tool is commonly used for LLM monitoring and evaluation in production?

MLflow

TensorBoard

Weights & Biases

Kubeflow

20. What is the primary advantage of using Retrieval-Augmented Generation (RAG) in LLM applications?

Reduces model training time

Grounds responses in external knowledge and reduces hallucinations

Eliminates the need for fine-tuning

Increases model parameter count