Regression and Decision Tree Algorithms Quiz

1. In decision trees, what does pruning accomplish?

Removes unnecessary branches to reduce overfitting

Adds more depth to improve accuracy

Scales features to improve splits

Increases the number of training samples

2. What metric would you use to compare a regression model with a baseline model?

Mean Squared Error (MSE) or R-squared to compare performance

Confusion matrix for regression predictions

Precision and recall scores

Feature importance rankings

3. When should you choose Ridge regression over Lasso regression?

When you want to keep all features and reduce coefficient magnitude

When you need automatic feature selection via zeroed coefficients

Ridge is always superior to Lasso

When target variable is categorical

4. For a decision tree predicting house prices, what determines leaf node predictions?

The average (or majority) value of target samples in that leaf

The feature with highest importance in the dataset

A random value between min and max in training data

The prediction from the root node

5. What is the relationship between bias and variance in regression models?

High bias (underfitting) and high variance (overfitting) represent opposite problems

Bias and variance are the same concept

High variance always indicates a good model

Bias can only be reduced by adding more data

6. How does cross-validation improve model evaluation?

Uses multiple train-test splits to get robust performance estimates

Eliminates the need for training data

Automatically prevents overfitting

Increases model interpretability

7. In logistic regression, what does the decision boundary represent?

The threshold where predicted probability equals 0.5

The maximum error tolerance for predictions

The optimal split point in a decision tree

The standard deviation of residuals

8. What is the main advantage of decision trees over linear regression?

Trees capture non-linear relationships without feature engineering

Trees always produce more accurate predictions

Trees require less computational power

Trees work only with continuous variables

9. How does feature scaling impact regression models?

Changes model predictions and coefficient interpretability

Has no effect on linear regression results

Only affects tree-based models, not regression

Always increases model accuracy

10. What assumption must hold for ordinary least squares (OLS) regression?

Residuals are normally distributed with constant variance

All features must be categorical

Target variable must be binary

Features must be completely uncorrelated

11. What is the primary objective of linear regression?

Minimize the sum of squared residuals between predicted and actual values

Maximize the classification accuracy of categorical outcomes

Identify clusters of similar data points

Reduce dimensionality of feature space

12. What is the purpose of train-test splitting in model evaluation?

Reduce computational cost of training

Estimate model performance on unseen data

Increase model accuracy automatically

Balance class distribution in data

13. How does multicollinearity affect linear regression models?

Increases prediction accuracy on test data

Makes coefficients unstable and harder to interpret

Reduces training time significantly

Automatically improves model generalization

14. What is the difference between Gini impurity and entropy as splitting criteria?

Gini is faster computationally; both measure node purity similarly

Entropy only works for binary classification; Gini works for all cases

Gini is always superior; entropy is outdated

They measure different aspects; entropy is information-based

15. Which regularization technique adds a penalty term to prevent large coefficients in regression?

L1 (Lasso) and L2 (Ridge) regularization

Cross-validation

Feature scaling

Data augmentation

16. In regression analysis, what does R-squared measure?

The proportion of variance in the target explained by predictors

The average absolute error of predictions

The probability of correct classification

The correlation coefficient between two variables

17. What is overfitting in decision trees, and how can it be prevented?

Model memorizes training data; prevent via pruning or depth limits

Model performs poorly on training data; add more features

Model uses too many features; use feature selection

Model has high bias; increase tree complexity

18. How does a decision tree make splits at each node?

Randomly selects features and thresholds

Selects feature and threshold that minimize impurity (Gini or entropy)

Uses the feature with the highest correlation to target

Splits all features equally at median values

19. Which metric is most appropriate for evaluating a binary classification model?

Mean Absolute Error

Area Under the ROC Curve (AUC)

Mean Squared Error

R-squared

20. In logistic regression, what function is used to map predictions to probabilities between 0 and 1?

Sigmoid function

Polynomial function

ReLU function

Tanh function

DataX Regression and Decision Tree Algorithms Quiz