Model Evaluation Metrics and Techniques Quiz

1. What is the primary purpose of a validation set in model development?

To train the model parameters

To tune hyperparameters and assess model performance during development

To measure final model performance on unseen data

To increase the size of the training dataset

2. What is the main drawback of using a single train-test split instead of cross-validation?

It requires more computational resources

Results depend heavily on how the data is split, reducing reliability

It always produces higher error rates

It cannot detect overfitting

Submit

4. Root Mean Squared Error (RMSE) penalizes larger errors more heavily than MAE.

True

False

5. Which evaluation approach uses stratified sampling to maintain class distribution?

Random train-test split

Stratified k-fold cross-validation

Leave-one-out cross-validation

Time series split

6. Grid search exhaustively evaluates all combinations of specified hyperparameters.

True

False

7. What does the learning curve show?

The relationship between feature importance and model accuracy

Training and validation error as a function of training set size

The hyperparameter values that minimize loss

The distribution of predictions across classes

Submit

9. In a highly imbalanced dataset, which metric is misleading?

Accuracy

Precision

Recall

F1-score

10. Mean Absolute Error (MAE) is appropriate for regression problems, while accuracy is for classification.

True

False

11. What does a confusion matrix display in binary classification?

True positives, true negatives, false positives, and false negatives

Only the total number of correct predictions

Feature importance rankings

Training and validation loss curves

12. Hyperparameter tuning involves adjusting parameters that are learned during model training.

True

False

13. Which of the following is a sign of underfitting?

High training accuracy and low validation accuracy

Low training accuracy and low validation accuracy

High training accuracy and high validation accuracy

Training accuracy increases indefinitely

Submit

15. What does AUC-ROC measure?

The area under the curve of true positive rate vs. false positive rate

The average of all prediction errors

The number of correctly classified samples

The variance in model predictions

16. In k-fold cross-validation, the model is trained and tested k times on different subsets.

True

False

17. The F1-score is the harmonic mean of which two metrics?

Accuracy and specificity

Precision and recall

Sensitivity and specificity

True positive rate and false positive rate

18. Cross-validation helps prevent overfitting by training and evaluating the model on different data subsets.

True

False

19. Which metric is most appropriate when you want to minimize false negatives in a medical diagnosis model?

Accuracy

Recall

Specificity

Precision

Submit

General Data Science Intermediate Model Evaluation Quiz

1. What is the primary purpose of a validation set in model development?

2.

What first name or nickname would you like us to use?

2. What is the main drawback of using a single train-test split instead of cross-validation?

3. A model with high bias typically exhibits ______ on both training and validation sets.

4. Root Mean Squared Error (RMSE) penalizes larger errors more heavily than MAE.

5. Which evaluation approach uses stratified sampling to maintain class distribution?

6. Grid search exhaustively evaluates all combinations of specified hyperparameters.

7. What does the learning curve show?

8. Specificity measures the proportion of negative cases correctly identified as ______ .

9. In a highly imbalanced dataset, which metric is misleading?

10. Mean Absolute Error (MAE) is appropriate for regression problems, while accuracy is for classification.

11. What does a confusion matrix display in binary classification?

12. Hyperparameter tuning involves adjusting parameters that are learned during model training.

13. Which of the following is a sign of underfitting?

14. Overfitting occurs when a model performs well on training data but poorly on ______ data.

15. What does AUC-ROC measure?

16. In k-fold cross-validation, the model is trained and tested k times on different subsets.

17. The F1-score is the harmonic mean of which two metrics?

18. Cross-validation helps prevent overfitting by training and evaluating the model on different data subsets.

19. Which metric is most appropriate when you want to minimize false negatives in a medical diagnosis model?

20. Precision measures the proportion of ______ predictions that are correct.