AWS ML Model Evaluation & Optimization

1. What is the main limitation of using accuracy as an evaluation metric?

It is computationally expensive

It performs poorly with imbalanced datasets

It cannot be used for multi-class problems

It requires manual calculation

2. What does the F1-score balance between?

Accuracy and error rate

Precision and recall

Sensitivity and specificity

Training and validation loss

3. Which technique helps prevent overfitting by randomly dropping neurons during training?

Batch normalization

Dropout

Early stopping

Weight decay

4. In SageMaker, what does a high data drift metric indicate?

The model is performing well

The input data distribution has changed significantly

The model needs more training data

Feature importance is increasing

5. What is the purpose of feature scaling before training most ML models?

To reduce overfitting

To normalize features to similar ranges for faster convergence

To eliminate outliers

To increase model accuracy

6. Which metric represents the proportion of positive predictions that were correct?

Recall

Precision

F1-score

Specificity

7. In gradient descent optimization, what does a smaller learning rate typically cause?

Faster convergence

Slower but potentially more stable convergence

Increased overfitting

Divergence

8. What does the confusion matrix diagonal represent?

All misclassifications

Correct predictions

False positives and false negatives

Class imbalance

9. Which AWS service provides automated machine learning capabilities similar to AutoML?

SageMaker Autopilot

SageMaker Processing

SageMaker Pipelines

SageMaker Clarify

10. In ensemble methods, what is bagging designed to reduce?

Bias

Variance

Recall

Precision

11. Which metric is most appropriate when evaluating a binary classification model with severe class imbalance?

Accuracy

F1-score

True Positive Rate

Precision only

12. Which confusion matrix element represents correctly predicted positive cases?

False Positive

True Positive

False Negative

True Negative

13. In SageMaker Automatic Model Tuning, what is a trial?

A complete training job with a specific hyperparameter combination

A single epoch of training

A cross-validation fold

A feature engineering step

14. What does early stopping prevent during model training?

Underfitting

Overfitting

Class imbalance

High bias

15. Which metric is most sensitive to false positives in a medical diagnosis model?

Recall

Precision

Specificity

Sensitivity

16. In regularization, what does L2 (Ridge) regularization add to the loss function?

Sum of absolute weights

Sum of squared weights

Maximum weight value

Count of non-zero weights

17. What is the primary purpose of SageMaker Model Monitor?

To train multiple models in parallel

To detect data drift and model performance degradation in production

To automatically retrain models

To manage model versioning

18. Which cross-validation technique is best for time-series data?

K-fold cross-validation

Stratified k-fold

Time-series split

Leave-one-out

19. What does the AUC-ROC metric measure?

The area under the precision-recall curve

The probability the model ranks a random positive higher than a random negative

The average error across all classes

The percentage of correct predictions

20. In SageMaker Hyperparameter Tuning, what is the primary advantage of using Bayesian optimization over grid search?

It searches all parameter combinations

It intelligently samples the parameter space to minimize cost

It requires no prior knowledge of parameter ranges

It guarantees finding the global optimum

AWS ML Specialty Model Evaluation and Optimization Quiz

1. What is the main limitation of using accuracy as an evaluation metric?

2.

What first name or nickname would you like us to use?

2. What does the F1-score balance between?

3. Which technique helps prevent overfitting by randomly dropping neurons during training?

4. In SageMaker, what does a high data drift metric indicate?

5. What is the purpose of feature scaling before training most ML models?

6. Which metric represents the proportion of positive predictions that were correct?

7. In gradient descent optimization, what does a smaller learning rate typically cause?

8. What does the confusion matrix diagonal represent?

9. Which AWS service provides automated machine learning capabilities similar to AutoML?

10. In ensemble methods, what is bagging designed to reduce?

11. Which metric is most appropriate when evaluating a binary classification model with severe class imbalance?

12. Which confusion matrix element represents correctly predicted positive cases?

13. In SageMaker Automatic Model Tuning, what is a trial?

14. What does early stopping prevent during model training?

15. Which metric is most sensitive to false positives in a medical diagnosis model?

16. In regularization, what does L2 (Ridge) regularization add to the loss function?

17. What is the primary purpose of SageMaker Model Monitor?

18. Which cross-validation technique is best for time-series data?

19. What does the AUC-ROC metric measure?

20. In SageMaker Hyperparameter Tuning, what is the primary advantage of using Bayesian optimization over grid search?