Google ML Engineer Model Monitoring Quiz

1. What is the benefit of setting up automated alerting thresholds for model metrics?

Reduces storage costs

Enables rapid response to performance degradation

Eliminates need for retraining

Increases model accuracy automatically

2. How should you approach retraining decisions based on monitoring alerts?

Retrain immediately on every alert

Investigate root cause first, then decide if retraining or adjustment is needed

Never retrain based on alerts

Retrain only monthly

3. What does model explainability monitoring help identify in production?

Training data corruption

Unexpected feature attribution changes indicating drift

Batch size issues

Storage capacity problems

4. Which metric combination best evaluates model performance for imbalanced datasets?

Accuracy only

Precision, recall, and F1-score

Confusion matrix only

Loss value only

5. In Vertex AI, what does the model monitoring service track regarding prediction distribution?

Only minimum values

Mean and variance changes over time

Only maximum values

Training loss only

6. What is the primary advantage of continuous model validation in production?

Reduces model file size

Identifies performance issues before they affect users

Eliminates hyperparameter tuning

Increases training speed

7. How does class imbalance affect monitoring metrics in classification models?

It has no effect

It can skew accuracy, requiring focus on precision/recall per class

It only affects training time

It eliminates drift detection

8. What is the relationship between baseline metrics and drift detection?

Baselines are unnecessary for monitoring

Baselines establish expected performance ranges to compare against

Baselines only apply to training data

Baselines prevent all drift

9. In monitoring, what does ROC-AUC threshold monitoring help detect?

Training data size changes

Changes in model discrimination ability between classes

Batch processing speed

Model parameter count

10. Which Google Cloud tool integrates with Vertex AI for real-time monitoring dashboards?

Cloud Monitoring

Cloud Logging

Cloud Trace

All of the above

11. What is data drift in machine learning models?

Change in input feature distributions over time

Random noise in training data

Intentional model parameter adjustments

Loss of training data files

12. How should you handle missing values when monitoring production predictions?

Ignore them entirely

Flag them as anomalies and track frequency

Always impute with zero

Delete the entire prediction

13. What statistical test helps detect significant changes in model performance distributions?

Kolmogorov-Smirnov test

Linear regression

K-means clustering

Principal component analysis

14. Which logging strategy captures both input features and model predictions for monitoring?

Request-response logging

Error-only logging

Batch prediction logging

Model weight logging

15. In production monitoring, what does a prediction latency spike typically indicate?

Model is underfitting

System infrastructure issue or model complexity problem

Training data quality improved

Batch size decreased

16. What is a confusion matrix primarily used for in model monitoring?

Storing model weights

Evaluating classification performance metrics like precision and recall

Compressing model predictions

Detecting data corruption

17. Which type of drift occurs when the relationship between features and target changes?

Data drift

Feature drift

Concept drift

Label drift

18. What is the primary purpose of feature importance tracking in production?

To reduce model file size

To identify which features drive predictions and detect unexpected changes

To increase model accuracy

To replace model retraining

19. In Google Cloud, which service provides real-time model monitoring?

Cloud Storage

Vertex AI Model Monitoring

BigQuery only

Compute Engine

20. Which metric best detects concept drift in classification models?

Training loss

Prediction accuracy on recent production data

Model file size

Training time in seconds